1. Introduction
The picture in picture mode was introduced in Android 8.0, which allows the Activity to be zoomed out and displayed above other activities. At the beginning, the project I maintained realized this function by itself. After Android joined picture in picture, the two functions were parallel and had a lot of problems when interacting with each other. Now almost all video software has added this function. The use method is very simple, but the problem of AudioFocus needs to be handled well.
2. Parameter introduction
In Android 8.0, you only need to call the Activity
enterPictureInPictureMode();
Or enterPictureInPictureModeIfPossible() that will do
public boolean enterPictureInPictureMode(@NonNull PictureInPictureParams params) {
try {
if (params == null) {
throw new IllegalArgumentException("Expected non-null picture-in-picture params");
}
return ActivityManagerNative.getDefault().enterPictureInPictureMode(mToken, params);
} catch (RemoteException e) {
return false;
}
}
The PictureInPictureParams type parameter needs to be passed in when entering.
At present, the higher version of Android has two methods, @ Deprecated enterPictureInPictureMode() and enterPictureInPictureMode(@NonNull PictureInPictureArgs args). To use picture in picture, you must pass in parameters.
Let's take a look at the source code of PictureInPictureParams
/*
* Copyright (C) 2017 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package android.app;
import android.annotation.Nullable;
import android.graphics.Rect;
import android.os.Parcel;
import android.os.Parcelable;
import android.util.Rational;
import java.util.ArrayList;
import java.util.List;
/**
* Represents a set of parameters used to initialize and update an Activity in picture-in-picture
* mode.
*/
public final class PictureInPictureParams implements Parcelable {
/**
* Builder class for {@link PictureInPictureParams} objects.
*/
public static class Builder {
@Nullable
private Rational mAspectRatio;
@Nullable
private List<RemoteAction> mUserActions;
@Nullable
private Rect mSourceRectHint;
/**
* Sets the aspect ratio. This aspect ratio is defined as the desired width / height, and
* does not change upon device rotation.
*
* @param aspectRatio the new aspect ratio for the activity in picture-in-picture, must be
* between 2.39:1 and 1:2.39 (inclusive).
*
* @return this builder instance.
*/
public Builder setAspectRatio(Rational aspectRatio) {
mAspectRatio = aspectRatio;
return this;
}
/**
* Sets the user actions. If there are more than
* {@link Activity#getMaxNumPictureInPictureActions()} actions, then the input list
* will be truncated to that number.
*
* @param actions the new actions to show in the picture-in-picture menu.
*
* @return this builder instance.
*
* @see RemoteAction
*/
public Builder setActions(List<RemoteAction> actions) {
if (mUserActions != null) {
mUserActions = null;
}
if (actions != null) {
mUserActions = new ArrayList<>(actions);
}
return this;
}
/**
* Sets the source bounds hint. These bounds are only used when an activity first enters
* picture-in-picture, and describe the bounds in window coordinates of activity entering
* picture-in-picture that will be visible following the transition. For the best effect,
* these bounds should also match the aspect ratio in the arguments.
*
* @param launchBounds window-coordinate bounds indicating the area of the activity that
* will still be visible following the transition into picture-in-picture (eg. the video
* view bounds in a video player)
*
* @return this builder instance.
*/
public Builder setSourceRectHint(Rect launchBounds) {
if (launchBounds == null) {
mSourceRectHint = null;
} else {
mSourceRectHint = new Rect(launchBounds);
}
return this;
}
/**
* @return an immutable {@link PictureInPictureParams} to be used when entering or updating
* the activity in picture-in-picture.
*
* @see Activity#enterPictureInPictureMode(PictureInPictureParams)
* @see Activity#setPictureInPictureParams(PictureInPictureParams)
*/
public PictureInPictureParams build() {
PictureInPictureParams params = new PictureInPictureParams(mAspectRatio, mUserActions,
mSourceRectHint);
return params;
}
}
/**
* The expected aspect ratio of the picture-in-picture.
*/
@Nullable
private Rational mAspectRatio;
/**
* The set of actions that are associated with this activity when in picture-in-picture.
*/
@Nullable
private List<RemoteAction> mUserActions;
/**
* The source bounds hint used when entering picture-in-picture, relative to the window bounds.
* We can use this internally for the transition into picture-in-picture to ensure that a
* particular source rect is visible throughout the whole transition.
*/
@Nullable
private Rect mSourceRectHint;
/** {@hide} */
PictureInPictureParams() {
}
/** {@hide} */
PictureInPictureParams(Parcel in) {
if (in.readInt() != 0) {
mAspectRatio = new Rational(in.readInt(), in.readInt());
}
if (in.readInt() != 0) {
mUserActions = new ArrayList<>();
in.readParcelableList(mUserActions, RemoteAction.class.getClassLoader());
}
if (in.readInt() != 0) {
mSourceRectHint = Rect.CREATOR.createFromParcel(in);
}
}
/** {@hide} */
PictureInPictureParams(Rational aspectRatio, List<RemoteAction> actions,
Rect sourceRectHint) {
mAspectRatio = aspectRatio;
mUserActions = actions;
mSourceRectHint = sourceRectHint;
}
/**
* Copies the set parameters from the other picture-in-picture args.
* @hide
*/
public void copyOnlySet(PictureInPictureParams otherArgs) {
if (otherArgs.hasSetAspectRatio()) {
mAspectRatio = otherArgs.mAspectRatio;
}
if (otherArgs.hasSetActions()) {
mUserActions = otherArgs.mUserActions;
}
if (otherArgs.hasSourceBoundsHint()) {
mSourceRectHint = new Rect(otherArgs.getSourceRectHint());
}
}
/**
* @return the aspect ratio. If none is set, return 0.
* @hide
*/
public float getAspectRatio() {
if (mAspectRatio != null) {
return mAspectRatio.floatValue();
}
return 0f;
}
/** @hide */
public Rational getAspectRatioRational() {
return mAspectRatio;
}
/**
* @return whether the aspect ratio is set.
* @hide
*/
public boolean hasSetAspectRatio() {
return mAspectRatio != null;
}
/**
* @return the set of user actions.
* @hide
*/
public List<RemoteAction> getActions() {
return mUserActions;
}
/**
* @return whether the user actions are set.
* @hide
*/
public boolean hasSetActions() {
return mUserActions != null;
}
/**
* Truncates the set of actions to the given {@param size}.
* @hide
*/
public void truncateActions(int size) {
if (hasSetActions()) {
mUserActions = mUserActions.subList(0, Math.min(mUserActions.size(), size));
}
}
/**
* @return the source rect hint
* @hide
*/
public Rect getSourceRectHint() {
return mSourceRectHint;
}
/**
* @return whether there are launch bounds set
* @hide
*/
public boolean hasSourceBoundsHint() {
return mSourceRectHint != null && !mSourceRectHint.isEmpty();
}
@Override
public int describeContents() {
return 0;
}
@Override
public void writeToParcel(Parcel out, int flags) {
if (mAspectRatio != null) {
out.writeInt(1);
out.writeInt(mAspectRatio.getNumerator());
out.writeInt(mAspectRatio.getDenominator());
} else {
out.writeInt(0);
}
if (mUserActions != null) {
out.writeInt(1);
out.writeParcelableList(mUserActions, 0);
} else {
out.writeInt(0);
}
if (mSourceRectHint != null) {
out.writeInt(1);
mSourceRectHint.writeToParcel(out, 0);
} else {
out.writeInt(0);
}
}
public static final Creator<PictureInPictureParams> CREATOR =
new Creator<PictureInPictureParams>() {
public PictureInPictureParams createFromParcel(Parcel in) {
return new PictureInPictureParams(in);
}
public PictureInPictureParams[] newArray(int size) {
return new PictureInPictureParams[size];
}
};
}
You can see that PictureInPictureParams is designed using the builder pattern of chain call. Its main members are
mAspectRatio :
/**
* Sets the aspect ratio. This aspect ratio is defined as the desired width / height, and
* does not change upon device rotation.
*
* @param aspectRatio the new aspect ratio for the activity in picture-in-picture, must be
* between 2.39:1 and 1:2.39 (inclusive).
*
* @return this builder instance.
*/
The aspect ratio is limited to 2.39:1 to 1:2.39.
mUserActions :
/**
* Sets the user actions. If there are more than
* {@link Activity#getMaxNumPictureInPictureActions()} actions, then the input list
* will be truncated to that number.
*
* @param actions the new actions to show in the picture-in-picture menu.
*
* @return this builder instance.
*
* @see RemoteAction
*/
Action is a control. The item in the earlier version of Android GMS Setting is composed of action.
The userAction here is a RemoteAction type. This type has nothing to do with Action, but it is similar. It is a serializable object with specific members
private final Icon mIcon;
private final CharSequence mTitle;
private final CharSequence mContentDescription;
private final PendingIntent mActionIntent;
private boolean mEnabled;
See RemoteAction.java for the specific code
To put it simply, after the Activity enters picture in picture, an Android navigation bar will be displayed under the reduced Activity, which is composed of several keys, and each key is a RemoteAction. Put the RemoteAction list into the PictureInPictureParams parameter, and these buttons will be displayed when you enter picture in picture. However, it is generally not required on the mobile terminal. The function of these keys is to return to the full screen display, close the picture in picture small window, and switch the size and position of the small window. Generally, the APP on the mobile terminal only needs to double-click the small window to return to the full screen. However, the Android TV terminal is needed, because the TV generally does not have a touch screen and depends entirely on the remote control. It takes the onKey event. You need to use the key to switch the control to obtain the focus.
The more important thing here is mcactionintent, which is an object of type PendingIntent. This PendingIntent defines the behavior after the RemoteAction is triggered. PendingIntent can be seen as an encapsulation of Intent, but it does not execute a behavior immediately. You can use getActivity, getActivities, getBroadcast, and getService methods to get the PendingIntent object that can start an Activity or send a broadcast to start a service.
PendingIntent official document
mSourceRectHint :
/**
* Sets the source bounds hint. These bounds are only used when an activity first enters
* picture-in-picture, and describe the bounds in window coordinates of activity entering
* picture-in-picture that will be visible following the transition. For the best effect,
* these bounds should also match the aspect ratio in the arguments.
*
* @param launchBounds window-coordinate bounds indicating the area of the activity that
* will still be visible following the transition into picture-in-picture (eg. the video
* view bounds in a video player)
*
* @return this builder instance.
*/
Boundary constraint
The introduction on the official document is
I know what each word means, but I don't think it speaks human words.
It probably means that after the Activity enters the picture in picture mode, because it becomes smaller and the aspect ratio may be different from the original screen, some pictures cannot be displayed, so set an area. This area is still visible after entering the picture in picture and shrinking. This area should correspond to the height width ratio set previously, so that it can be displayed better. (I guess)
Then I set various Rect values in the code and found that they had no impact on PIP. I couldn't guess what this thing was doing. This document doesn't talk to people, and the code doesn't work. I really don't know what this thing is for. If you know, please give me some advice.
3. Use
It was very simple to use in the earliest days.
To call an Activity directly
@Override
public void enterPictureInPictureModeIfPossible() {
if (mActivityInfo.supportsPictureInPicture()) {
enterPictureInPictureMode();
}
}
That's it. Now this API can't be called. You can use it
@Deprecated
public void enterPictureInPictureMode() {
enterPictureInPictureMode(new PictureInPictureParams.Builder().build());
}
Enter pip, but compared with the API above, if your Activity does not support PIP mode, an exception will be reported.
So you need to add this sentence to the list file
android:supportsPictureInPicture="true" android:configChanges="screenSize|smallestScreenSize|screenLayout|orientation"
I create a Button in the Activity and click it to call enterPictureInPictureMode()
Effect after clicking
Click the small window again
However, this method is Deprecated after all. It can still be used on API28 at present. Maybe this API will be cancelled in the next version.
Therefore, we still need to learn to use the parameter with PictureInPictureParams
Enterpictureinpicturemode (@ nonnull pictureinpictureparams) method
Icon icon = Icon.createWithResource(mContext, R.mipmap.ic_launcher);
Icon icon2 = Icon.createWithResource(mContext,R.mipmap.ic_launcher_round);
Intent intent = new Intent(this, MainActivity.class);
PendingIntent pendingIntent = PendingIntent.getActivity(mContext, 998, intent, PendingIntent.FLAG_ONE_SHOT);
RemoteAction remoteAction = new RemoteAction(icon, "title", "introduce", pendingIntent);
RemoteAction remoteAction2 = new RemoteAction(icon2, "Title 2", "Introduction 2", pendingIntent);
ArrayList<RemoteAction> arrayList = new ArrayList<RemoteAction>();
arrayList.add(remoteAction);
arrayList.add(remoteAction2);
Rational rational = new Rational(3, 7);//If the value set here is too large or too small, or an exception is reported
Rect rect = new Rect(0, 0, 0, 0);
final PictureInPictureParams params = new PictureInPictureParams.Builder()
.setActions(arrayList).setAspectRatio(rational)
.setSourceRectHint(rect).build();
button.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
// enterPictureInPictureMode();
enterPictureInPictureMode(params);
}
});
Rational's ratio needs to be between 2.39:1 and 1:2.39, otherwise
java.lang.IllegalArgumentException: enterPictureInPictureMode: Aspect ratio is too extreme (must be between 0.418410 and 2.390000).
effect:
After clicking the small window
You can see that there are two buttons. This is the RemoteAction we set. Clicking it will trigger PendingIntent.
The middle button is the built-in maximize button, the top right corner is also the built-in close button, and the top left is the built-in setting button. This may be different on different versions of Android and Android TV.
Hey, I was just about to change the size of the small window. I remember that when Android 8.0 got the beta version, it can still adjust the size of the small window. There are three sizes. As a result, it can't be adjusted with the latest Android Q. I don't know what google has changed. Then I can't find the PIP entry icon in station b and Betta with EMUI10. Youku and iqiyi can enter pip, but they can't adjust the size of the small window.
To get back to business, you can see that the proportion displayed after entering the small window is different from that of the original screen, so some contents cannot be displayed. At this time, you need to adjust the UI.
For example, if you play a Video, you can first let the Video play in full screen, or simply provide the PIP entry when playing in full screen, and then set the Rational corresponding to the Video to the PIP, that is, some page adjustments need to be made before entering and after exiting the PIP. For example, you have the logic to calculate the UI coordinates or dynamically modify the control position, which needs to be handled before entering the PIP, Return to the normal mode and restore, otherwise it will be inaccurate to obtain these after entering the PIP, and the whole UI will be disordered.
When the PIP function was first launched, QA gave me a lot of such bugs.
The onPictureInPictureModeChanged() callback is now also provided
@Override
public void onPictureInPictureModeChanged(boolean isInPictureInPictureMode, Configuration newConfig) {
super.onPictureInPictureModeChanged(isInPictureInPictureMode, newConfig);
if (isInPictureInPictureMode){
}else {
}
}
The logic of PIP entry and exit can be handled here. Disable some UI or simply load a new layout for PIP mode.
4. Attention
Some points needing attention
First, when an Activity enters the PIP, it is zoomed out and displayed above other applications. At this time, the life cycle goes to onPause(), but does not go to OnStop(), which is a bit like the Activity life cycle in the previous VisibleBehind mode. So if you pause the playback while onPause, it won't play after entering picture in picture.
AudioFocus problem. When I saw many small app windows in 8.0, I would rob AudioFocus. I remember I opened three small windows to play videos and sound at the same time. But the current version doesn't work. At the same time, only one app can get AudioFocus, and the video playing window on the main page will automatically pause, and only one PIP process can survive at the same time.
Article reprint: