preface
The first important requirement to undertake after half a year of employment is to make a video that can switch the background arbitrarily and generate its own lyrics and music, which can be exported and saved to the album. The following records several meaningful problems and innovations encountered in the development process.
- Innovation 1: according to the time length of the line of lyrics parsed in real time, the lyrics have the effect of gradual in and out (only 5 lines of Java code without Animation)
- Problem 1: after encountering the Activity onPause() callback, the onStop() callback is 10s slow. The reason and solution
- Question 2: surfaceView cannot use view Gone and view The reason why visible reports null pointer exception but does not crash
- Question 3: RecyclerView ViewHolder reuse problem: the bottom template has its own progress bar. After sliding horizontally, the return progress bar will disappear, and double clicking the progress bar continuously will flash
difficulty
- Two players (one background music + one video) are decoupled
- Video playback involves rendering and video export involves encoding and decoding
- Real time display of lyrics, gradual in and gradual out effect
- Lyrics parsing and song caching, considering network interruption
- Timing and synchronization of background template download and lyrics cache Download
- Continuously click the background template, progress bar display and video playback logic
UI effect and main function introduction
1. Slide the bottom template to switch the video background video. The white line is a rectangular progress bar, indicating the template download progress. If multiple backgrounds are clicked continuously, it will be downloaded synchronously and the progress bar will be displayed at the same time. However, after loading, only the progress bar of the playback template is retained (100% around). As shown in the above two figures.
2. Lyrics interface, sliding play and displayed lyrics, and automatically play 15s video according to the user's sliding stop position. As shown above.
3. After the production is completed, click export video coding to save it locally. After saving, call up the sharing pop-up box and click to jump to the corresponding third-party app. Users need to manually select videos from the album for publishing and sharing
Innovation 1: only 5 lines of code are used to realize the progressive effect of real-time lyrics
1. Introduction:
After the lyrics are parsed, a list is returned, in which list The information contained in get (I) includes (start: the time int type at which the lyrics of this sentence begin to play, end: the time int type at which the lyrics of this sentence end to play, and the string type of the lyrics of this line), and the total length of time of each lyrics varies, so I don't use Animation to perform the effect of alpha Animation here, which is a little troublesome. Instead, the alpha value is calculated in real time directly according to the parsed information.
2. Principle:
The schematic diagram is shown below. The known parameters startTime, endTime, factor (self setting, here I set 0.5) and toaltime. The progressive is from a - > b, the alpha value is from 0 - > 1, the gradual out is from C - > D, the alpha value is from 1 - > 0, the abscissa is the lyrics playing time, and the ordinate is the transparency alpha value. Then, in the principle of simplicity, here I set the gradient of 0 - > 1 as a first-order function y=kx+b, and the proportion of gradual gradual exit time is half of the total time, and the gradual exit time is equal.
1) Since the A coordinate is known (startTime, 0) and the B coordinate is known (start+total*factor/2, 1), the values of k and B can be obtained by getting A straight line from two points, so as to obtain the function y=k+b of point AB;
2) Since the function is an isosceles trapezoid, the slope k values of AB and CD are opposite to each other, and the gradual exit function of CD is y=-kx+b, so you only need to bring in any point of C and D to calculate the value of b.
It should be noted that the influencing parameters include factor and player refresh frequency. The size of factor affects the speed of gradual change (the size of k value), and the refresh frequency affects the player performance and effect. It should not be too low or too high. Too low will reduce the performance, and too high will make the gradient ineffective.
The code is as follows:
/**
* @param factor Specify the gradient time ratio of alpha in each lyrics
* @param startTime The time when the line begins
* @param endTime The time when the line ends
* @param curTime current time
* @implNote The calculation process takes curTime as abscissa, alpha as ordinate, and the function is an isosceles trapezoid piecewise function
* @return: Transparency alpha value: related to refresh rate, setVideoUpdateProgressTime is set to 10ms
*/
public static float getAlpha(float factor, long curTime, long startTime, long endTime) {
long totalTime = endTime - startTime;
if (curTime <= startTime) {//Just entered the lyrics of this sentence
return 0f;
} else if (endTime - startTime <= 20) {//If the lyrics time of this sentence is less than the refresh time, no gradient is required
return 1f;
} else {
if (curTime <= totalTime * factor / 2 + startTime) {//Fade in 0 - < 1 >
return 2 * (curTime - startTime) / (factor * totalTime);
} else if (curTime >= endTime - (totalTime * factor / 2) && curTime <= endTime) {//Fade out 1 - < 0 >
return -2 * (curTime - endTime) / (factor * totalTime);
} else {
return 1f;
}
}
}
Question 2: after exiting acitivity, onPause() calls back immediately, and onStop() calls back 10 s slower?
1. Problem phenomenon: to return to the full screen player after exiting the activity, you need to return to the playing state of the full screen player immediately, but the phenomenon is that the playing state of the previous page can be restored only after the song of the activity is played for 10s. Write a log for each life cycle, check the call time, find that onPause() calls back immediately, and onStop calls back every 10s.
2. Cause of the problem: it is preliminarily speculated that the previous page has animation or the acitvity has animation, and the main thread continues to post invalidate (), resulting in thread blocking and onStop unable to callback. The search found that the animation of the previous page has been sending messages to the main thread. But why only onStop is blocked, not onPause? And 10s each time? Search the blog with questions and find the following reasons:
1) The life cycle of returning from an activity a to activity B: onPause (b) - > onrestart (a) - > OnStart (a) - > onResume(A) - > onstop (b) - > ondestroy (b). Therefore, the onPause of the activity will be called back immediately, and the animation in acitivity A has been blocking the main thread during onResume(A), so the subsequent life cycle cannot be called
2) Thread blocking mechanism:
To sum up, just one sentence: LifeCircleManager enforces thread operations. The animation of the last acitivity has been calling and sending messages, resulting in queue blocking. If the handlerIdle does not receive messages within 10s, it enforces onStop(). The principle is shown in the above figure.
3. Solution: pause the animation during activity A onPause(), start playing again when onRestart(), and try not to use view Instead of postinvalidate (), change it to view invalidate().
Question 2: surfaceView uses view Gone and view Visible will report an error but not crash
1. Problem background: surfaceView is not a regular View. It is a subclass of GlsurfaceView and cannot be used GONE. If used, a null pointer exception will be reported. However, the reason why there is no crash has not been clarified, and it will be supplemented after the subsequent clarification.
2. Solution: set the sufaceView size according to the size of the cover image, and place it behind the default loaded image, otherwise the background will be black for a moment. After the video download is completed, make the cover image disappear and play the video.
Question 3: reuse of ViewHolder of Adapter in RecyclerView: the bottom template has its own progress bar. After sliding horizontally, the return progress bar will disappear, and double clicking the progress bar continuously will flash
1. Problem background 1: select template A to play the progress bar in A circle of white. After sliding, the return progress bar disappears. Because the bottom template is encapsulated and displayed by the RecyclerView adapter, the holder problem is involved. The reason is that there is no deep understanding of holder reuse.
2. Problem reason 1: Generally speaking, if RecyclerView When the getItemViewType method of the adapter returns the same value, the RecyclerView will reuse the ViewHolder that has slipped out of the screen and become invisible. If the View held by the reused ViewHolder has not been re assigned or restored to the original state, the View state before reuse will be displayed. Because the background template in this requirement is complex. Continuous clicking on the template and the progress bar of the template that has not been downloaded also need to display the progress. After loading, the last click shall prevail and the video of the corresponding background template shall be played. Therefore, a moduleBean object is encapsulated here, and the corresponding click event processing is performed according to the template download status state (0 not downloaded, 1 downloading, 2 downloaded).
3. Solution 1:
1) Directly in recyclerview Set the holder in the onBindViewHolder method of the adapter setIsRecyclable(false);
However, one problem with this method is that too many picture item s in the adapter will cause OOM, so try not to use this method
2) In the onBindViewHolder method, all views held by the viewholder are re assigned or restored to the initial state as needed. First, judge whether the position is within the visible range according to the LinearLayoutManager. If it is within the visible range, obtain the viewholder of the corresponding view according to its location.
if (curPostionIsVisiable(position)) { //If it is not visible, it will not be updated. This view may have been reused
LinearLayoutManager manager = (LinearLayoutManager) rcv.getLayoutManager();
if (manager == null) return;
int firstItemPosition = manager.findFirstVisibleItemPosition();
if (position - firstItemPosition >= 0) {
//Prevent progressBar reuse errors
View view = rcv.getChildAt(position - firstItemPosition);
if (null != rcv.getChildViewHolder(view)) {
ModuleAdapter.ViewHolder viewHolder = (ModuleAdapter.ViewHolder)rcv.getChildViewHolder(view);
viewHolder.pb.setProgress(progress);
}
}
}
public boolean curPostionIsVisiable(int position) {
//Is the current item visible
LinearLayoutManager manager = (LinearLayoutManager) rcv.getLayoutManager();
if (manager == null) return false;
int first = manager.findFirstVisibleItemPosition();
int last = manager.findLastVisibleItemPosition();
return position >= first && position <= last;
}
4. Problem background 2: because the status of each template is different, the progress bar will flash after fast clicking, and the progress bar will also flash after repeatedly clicking the same template being downloaded.
5. Problem reason 2: it is also because the ViewHolder is not reused correctly and the judgment of the module state lags behind, so the statement to judge the state has not arrived, and the click event triggers the download again, resulting in progress rewriting.
6. Solution 2: set the status of the module in advance and directly enter the code
Code in adapter:
//Set template click event
holder.image.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
if (!presenter.isDownload(position) && !NetworkUtils.isNetworkAvailable(mContext)) {//When there is no network, click the template being downloaded or not downloaded (except the downloaded status) to prompt the network error
MiguToast.showWarningNotice(mContext, R.string.lrc_video_module_load_net_error);
return;
}
if (curPostionIsVisiable(position)) { //If it is not visible, it will not be updated. This view may have been reused
holder.moduleSize.setVisibility(GONE);
holder.load.setVisibility(GONE);
}
if (presenter.isDownloading(position)) { //Click the item currently being downloaded, no other operation will be done, and only the last clicked position will be updated. In order to ensure that the last clicked by the user is played after the download is completed.
selectPostion = position;
} else if (presenter.isDownload(position)) { //Download completed
selectPostion = position;
curPostion = position;
delegate.playVideo(curPostion);
notifyDataSetChanged();//Notifies the adapter to update the current selection information
} else {//Need to download
selectPostion = position;
holder.pb.setVisibility(View.VISIBLE);
String videoUrl = module.getUrl();
String videoName = module.getName();
module.setState(2);//Set the module status here in advance!! Avoid that the module's status is still 0 when clicking quickly, so it can be loaded again
presenter.loadVideo(position, videoUrl, videoName, new LrcVideoPresenter.ProgressCallback() {
@Override
public void progressLoaded(int progress) {
if (curPostionIsVisiable(position)) { //If it is not visible, it will not be updated. This view may have been reused
LinearLayoutManager manager = (LinearLayoutManager) rcv.getLayoutManager();
if (manager == null) return;
int firstItemPosition = manager.findFirstVisibleItemPosition();
if (position - firstItemPosition >= 0) {
//Prevent progressBar reuse errors
View view = rcv.getChildAt(position - firstItemPosition);
if (null != rcv.getChildViewHolder(view)) {
ModuleAdapter.ViewHolder viewHolder = (ModuleAdapter.ViewHolder) rcv.getChildViewHolder(view);
viewHolder.pb.setProgress(progress);
}
}
}
}
@SuppressLint("NotifyDataSetChanged")
@Override
public void finish(String url) {
if (curPostionIsVisiable(position)) { //If it is not visible, it will not be updated. This view may have been reused
LinearLayoutManager manager = (LinearLayoutManager) rcv.getLayoutManager();
if (manager== null) return;
int firstItemPosition = manager.findFirstVisibleItemPosition();
View view = rcv.getChildAt(position - firstItemPosition );
if (null != rcv.getChildViewHolder(view)) {
ModuleAdapter.ViewHolder viewHolder = (ModuleAdapter.ViewHolder) rcv.getChildViewHolder(view);
if (selectPostion == position) { //If the user does not select another during the download
//Change to selection state
viewHolder.pb.setVisibility(View.VISIBLE);
viewHolder.pb.setProgress(100);
}else{
viewHolder.pb.setVisibility(GONE);
}
}
}
if (selectPostion == position) { //If the user does not select another one during the download, the downloaded one is ready to play, and else does not need to do any processing
curPostion = position;
delegate.playVideo(curPostion);
notifyDataSetChanged();//Notifies the adapter to update the current selection information
}
}
@Override
public void error() {
module.setState(0);
if (selectPostion == position) { //Reset the selectposition. If the user does not select another one during the download, it will be reset to the playing position
selectPostion = curPostion;
}
holder.pb.setProgress(0);
holder.pb.setVisibility(GONE);
}
@Override
public void start() {
//Here you should set Progress to 0 at the beginning
holder.pb.setProgress(0);
}
});
}
XLog.i("click item: position->" + position + "curPostion->" + curPostion + "selectPostion->" + selectPostion);
}
});
Download some codes:
//Loading video background resources, asynchronous
public void loadVideo(int pos, String videoUrl, String name,
final ProgressCallback progressCallback) {
VideoInfoBean bean = beanMap.get(pos);
if (bean.state == 1) {
progressCallback.finish(bean.getLoadUrl());
progressCallback.progressLoaded(100);
return;
}
//If the memory does not exist, start the thread to download the background video
File downLoadFolder = new File(SdPath);//Temporary documents
File tmpFile = new File(SdPath, name + "-tmp.mp4");
File file = new File(SdPath, name);
NetLoader.downLoad(videoUrl)
.savePath(downLoadFolder.getPath())
.saveName(tmpFile.getName()).execute(new DownloadProgressCallBack<String>() {
@Override
public void onStart() {
progressCallback.start();
}
@Override
public void onError(ApiException e) {
//Cache failed
bean.state = 0; //If the download fails, it is not downloaded
progressCallback.error();
MiguToast.showWarningNotice(delegate.getActivity(), "Download failed, please try again later!");
}
@Override
public void update(long bytesRead, long contentLength, boolean done) {
//Download progress
int progress = (int) (((double) bytesRead / (double) contentLength) * 100);
if (progress >= 99) {
progress = 100;
}
//Add background template cache progress bar
progressCallback.progressLoaded(progress);
bean.state = 2;
bean.progress = progress;
}
@Override
public void onComplete(String path) {
if (path == null) {
MiguToast.showWarningNotice(delegate.getActivity(), "Download failed, please try again later!");
return;
}
if (tmpFile.exists()) {
if (tmpFile.renameTo(file)) {
//Cache complete
bean.state = 1;
bean.progress = 100;
bean.setLoadUrl(file.getAbsolutePath());
progressCallback.finish(file.getAbsolutePath());
}
}
}
});
}
Subsequent optimization
1. Modify the layout: change the LinearLayout to ViewPager to facilitate the switching of newly added lyrics and pictures. The bottom lyrics module and background module are changed to ViewPager
2. Fixed a problem with the custom View: the rectangular progress bar flickered occasionally after one turn
3. Optimize sufaceView
Reference articles
Deeply analyze the reason why the Activity onStop() life cycle delays 10s callback
The Activity destroys onStop or onDestroy, delaying about 10s before callback