Home MediaCodec Android: Add silent audio track
Reply: 0

MediaCodec Android: Add silent audio track

Jon G
1#
Jon G Published in 2017-12-07 13:17:40Z

I am trying to generate a new video, with both a video and audio track, using MediaCodec, but without any input videos to decode. I want to specify the length of the video and have control of the content.

I have been following the EncodeAndMuxTest that generates an mp4 with a video track showing some simple visuals. I am trying to modify it to also include a (silent) audio track. The format of the output audio is:

MediaFormat format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, 44100, 2);
format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);

I create my encoder like:

mAudioEncoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
mAudioEncoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
mAudioEncoder.start();
mAudioEncoderBuffers = new MediaCodecBufferCompatWrapper(mAudioEncoder);

I believe I just need to send a buffer of zeroes to the encoder. I am not sure how many bytes in total to send though. I guess that I need to send the audio bit rate times the video duration of bytes:

private static final long AUDIO_TOTAL_BYTES = AUDIO_BIT_RATE * VIDEO_DURATION_S;

Is that wrong? If I am sending this total number of bytes in buffers, I presumably need to calculate the presentation time of each buffer and send that too:

double progress = 1f * numBytesSubmitted / AUDIO_TOTAL_BYTES;
long presentationTimeUs = Math.round(progress * DURATION_S * ONE_MILLION);

?

Therefore, I create my audio using:

private void submitAudioInput() {
    final long ONE_MILLION = 1000000;

    int numBytesSubmitted = 0;
    boolean doneSubmittingInput = false;

    while (true) {
        int index;

        index = mAudioEncoder.dequeueInputBuffer(0);

        if (index != MediaCodec.INFO_TRY_AGAIN_LATER) {
            if (numBytesSubmitted >= AUDIO_TOTAL_BYTES) {
                mAudioEncoder.queueInputBuffer(index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);

                Log.d(TAG, "queued audio input EOS.");
                doneSubmittingInput = true;

            } else {
                ByteBuffer buffer = mAudioEncoderBuffers.getInputBuffer(index);
                buffer.clear();
                int size = buffer.limit();
                byte[] zeroes = new byte[size];
                buffer.put(zeroes);

                double progress = 1f * numBytesSubmitted / AUDIO_TOTAL_BYTES;
                long presentationTimeUs = Math.round(progress * DURATION_S * ONE_MILLION);

                mAudioEncoder.queueInputBuffer(index, 0, size, presentationTimeUs, 0);
                numBytesSubmitted += size;
            }
        }

        while (drainAudioEncoder(0) != DRAIN_STATE_NONE);
        if (doneSubmittingInput) break;
    }
}

where drainAudioEncoder is relatively similar to drainEncoder in the EncodeAndMuxTest.

The video I create plays OK in VLC on linux. I have also inspected it with ffprobe and it appears to have the correct format parameters and video frames. However, on the Android gallery it doesn't play correctly. The scrubber bar skips around, and the duration of the video is too short.

Is my idea of how to create a silent audio track fundamentally wrong?

Adding silent audio to a video on Android can be done with FFmpeg, but I am trying to use MediaCodec here.

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.306068 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO