Parsing and encapsulating mp4 files using MediaExtractor and MediaMuxer
brief introduction
MP4 or MPEG-4 Part 14 is a standard digital multimedia container format.
The audio format in MP4 is usually AAC (audio / mp4a LATM)
MediaExtractor
MediaExtractor can be used to separate video tracks and audio tracks in multimedia containers
- setDataSource() sets the data source. The data source can be a local file address or a network address
- getTrackFormat(int index) to obtain the MediaFormat of each track, and the MediaFormat to obtain the detailed information of the track, such as MimeType, resolution, sampling frequency, frame rate, etc
- selectTrack(int index) selects the specified channel by subscript
- readSampleData(ByteBuffer buffer, int offset) obtains the currently encoded data and stores it in the buffer with a specified offset
MediaMuxer
MediaMuxer can be used to mix basic code streams. Combine the information of all channels into one video. At present, the output format supports MP4, Webm and 3GP. Starting from Android Nougat, it supports mixing B-frames into MP4.
Extract and output the video part in the MP4 file
Extract video from an MP4 file to get an MP4 file without audio.
To implement the process, first use MediaExtractor to extract, and then use MediaMuxer to output MP4 files.
- MediaExtractor sets the data source, finds and selects the format and subscript of the video track
- MediaMuxer sets the output format to MUXER_OUTPUT_MPEG_4. Add the previously selected format and call start() to start
- The MediaExtractor reads the frame data and constantly transmits the frame data and related information to the MediaMuxer
- Finally, stop and release MediaMuxer and MediaExtractor
It is best to operate in a child thread.
/**
* Extract video
*
* @param sourceVideoPath Original video file
* @throws Exception error
*/
public static void extractVideo(String sourceVideoPath, String outVideoPath) throws Exception {
MediaExtractor sourceMediaExtractor = new MediaExtractor();
sourceMediaExtractor.setDataSource(sourceVideoPath);
int numTracks = sourceMediaExtractor.getTrackCount();
int sourceVideoTrackIndex = -1; // Original video file video track parameters
for (int i = 0; i < numTracks; ++i) {
MediaFormat format = sourceMediaExtractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
Log.d(TAG, "MediaFormat: " + mime);
if (mime.startsWith("video/")) {
sourceMediaExtractor.selectTrack(i);
sourceVideoTrackIndex = i;
Log.d(TAG, "selectTrack index=" + i + "; format: " + mime);
break;
}
}
MediaMuxer outputMediaMuxer = new MediaMuxer(outVideoPath,
MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
outputMediaMuxer.addTrack(sourceMediaExtractor.getTrackFormat(sourceVideoTrackIndex));
outputMediaMuxer.start();
ByteBuffer inputBuffer = ByteBuffer.allocate(1024 * 1024 * 2); // Allocate as much memory as possible
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
int sampleSize;
while ((sampleSize = sourceMediaExtractor.readSampleData(inputBuffer, 0)) >= 0) {
long presentationTimeUs = sourceMediaExtractor.getSampleTime();
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = presentationTimeUs;
outputMediaMuxer.writeSampleData(sourceVideoTrackIndex, inputBuffer, info);
sourceMediaExtractor.advance();
}
outputMediaMuxer.stop();
outputMediaMuxer.release(); // Stop and release MediaMuxer
sourceMediaExtractor.release();
sourceMediaExtractor = null; // Release MediaExtractor
}If the space allocated by the ByteBuffer above is too small, an IllegalArgumentException may occur in readSampleData(inputBuffer, 0).
Extract the audio part of the MP4 file and obtain the audio file
Method of extracting AAC file based on Java MP4 Parser
Java MP4 Parser - https://github.com/sannies/mp4parser Java implements reading, writing, and creating MP4 containers. But it is different from encoding and decoding audio and video. Here is mainly extraction and re synthesis.
Download isoparser-1.1 22. Jar and add it to the project; Tried to import gradle directly, but failed
Find all the audio tracks in the video file, extract them and write them to a new file
public void extractAudioFromMP4(String outAudioPath, String sourceMP4Path) throws IOException {
Movie movie = MovieCreator.build(sourceMP4Path);
List<Track> audioTracks = new ArrayList<>();
for (Track t : movie.getTracks()) {
if (t.getHandler().equals("soun")) {
audioTracks.add(t);
}
}
Movie result = new Movie();
if (audioTracks.size() > 0) {
result.addTrack(new AppendTrack(audioTracks.toArray(new Track[audioTracks.size()])));
}
Container out = new DefaultMp4Builder().build(result);
FileChannel fc = new RandomAccessFile(outAudioPath, "rw").getChannel();
out.writeContainer(fc);
fc.close();
}Successfully tested on Hongmi mobile phone. AAC files extracted from MP4 files (about 2 minutes and 20 seconds) can be played directly on the mobile phone.
Switch the AAC track to another MP4 file
MediaExtractor can directly extract ACC tracks from AAC files or MP4 files, and MediaMuxer can write new MP4 files.
The file providing audio can be MP4 file or AAC file; The other provides video and mixed output of new MP4 files.
The length of the generated video is determined by the file providing the video.
/**
* @param outputVideoFilePath Output video file path
* @param videoProviderPath The duration of MP4 file providing video shall be subject to this
* @param audioProviderPath Files that provide audio
* @throws Exception Operation exceptions, such as reading and writing file exceptions
*/
public static void replaceAudioForMP4File(String outputVideoFilePath, String videoProviderPath,
String audioProviderPath)
throws Exception {
MediaMuxer mediaMuxer = new MediaMuxer(outputVideoFilePath,
MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
// Video MediaExtractor
MediaExtractor mVideoExtractor = new MediaExtractor();
mVideoExtractor.setDataSource(videoProviderPath);
int videoTrackIndex = -1;
for (int i = 0; i < mVideoExtractor.getTrackCount(); i++) {
MediaFormat format = mVideoExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) {
mVideoExtractor.selectTrack(i);
videoTrackIndex = mediaMuxer.addTrack(format);
Log.d(TAG, "Video: format:" + format);
break;
}
}
// Audio MediaExtractor
MediaExtractor audioExtractor = new MediaExtractor();
audioExtractor.setDataSource(audioProviderPath);
int audioTrackIndex = -1;
for (int i = 0; i < audioExtractor.getTrackCount(); i++) {
MediaFormat format = audioExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) {
audioExtractor.selectTrack(i);
audioTrackIndex = mediaMuxer.addTrack(format);
Log.d(TAG, "Audio: format:" + format);
break;
}
}
mediaMuxer.start(); // start after adding all tracks
long videoEndPreTimeUs = 0;
// Encapsulated video track
if (-1 != videoTrackIndex) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(1024 * 1024);
int sampleSize;
while ((sampleSize = mVideoExtractor.readSampleData(buffer, 0)) >= 0) {
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = mVideoExtractor.getSampleTime();
videoEndPreTimeUs = info.presentationTimeUs;
mediaMuxer.writeSampleData(videoTrackIndex, buffer, info);
mVideoExtractor.advance();
}
}
Log.d(TAG, "video videoEndPreTimeUs " + videoEndPreTimeUs);
// Encapsulated audio track
if (-1 != audioTrackIndex) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(1024 * 1024);
int sampleSize;
while ((sampleSize = audioExtractor.readSampleData(buffer, 0)) >= 0 &&
audioExtractor.getSampleTime() <= videoEndPreTimeUs) {
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = audioExtractor.getSampleTime();
mediaMuxer.writeSampleData(audioTrackIndex, buffer, info);
audioExtractor.advance();
}
}
mVideoExtractor.release(); // Release MediaExtractor
audioExtractor.release();
mediaMuxer.stop();
mediaMuxer.release(); // Release MediaMuxer
}Video: format:{csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, frame-rate=30, height=1080, width=1920, max-input-size=1572864, isDMCMMExtractor=1, durationUs=12425577, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}
Audio: format:{max-input-size=5532, aac-profile=2, mime=audio/mp4a-latm, durationUs=340101875, csd-0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], channel-count=2, sample-rate=44100}Convert MP3 to AAC
Using Android audioconverter
AndroidAudioConverter - https://github.com/adrielcafe/AndroidAudioConverter
Third party Library Based on FFmpeg. Supported formats include AAC, MP3, M4A, WMA, WAV and FLAC
usage method:
app/build.gradle
repositories {
maven {
url "https://jitpack.io"
}
}
dependencies {
compile 'com.github.adrielcafe:AndroidAudioConverter:0.0.8'
}Request permission to read and write external storage
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
Load the library in the Application class
public class MuxerApp extends Application {
@Override
public void onCreate() {
super.onCreate();
AndroidAudioConverter.load(this, new ILoadCallback() {
@Override
public void onSuccess() {
// Great!
}
@Override
public void onFailure(Exception error) {
// FFmpeg is not supported by device
}
});
}
}Use conversion function
final String sourceMP3Path = SOURCE_PATH + File.separator + "music1.mp3";
Log.d(TAG, "Conversion start " + sourceMP3Path);
File srcFile = new File(sourceMP3Path);
IConvertCallback callback = new IConvertCallback() {
@Override
public void onSuccess(File convertedFile) {
Log.d(TAG, "onSuccess: " + convertedFile);
}
@Override
public void onFailure(Exception error) {
Log.e(TAG, "onFailure: ", error);
}
};
AndroidAudioConverter.with(getApplicationContext())
// Your current audio file
.setFile(srcFile)
// Your desired audio format
.setFormat(AudioFormat.AAC)
// An callback to know when conversion is finished
.setCallback(callback)
// Start conversion
.convert();Tested on Samsung Note4, it took about 3 minutes and 18 seconds to convert 13MB MP3 files.