Li Tao

He joined qunar.com in March 2015 as a client development engineer and once worked as a hotel business and self-service check-in and check-out project development. Now I am responsible for the development and maintenance of International Hotel capture system.

Android automatic interaction can replace manual completion of repetitive work, including black box test through automatic operation of App and automatic operation of third-party App. Common automated interactions include clicking, dragging and text input of App and view. With the improvement of App security capability, it becomes more and more difficult to realize the automatic interaction of the complete process. This paper mainly discusses the common automatic interaction schemes, the advantages and disadvantages of different schemes and application scenarios.

1. Traditional execution script scheme

ADB is a command line tool provided by Google that can interact with Android devices. We can write scripts to execute each event one by one according to the designed order. ADB needs to obtain the coordinates of interface elements in advance (the method of obtaining coordinates can use uiautomator or dump xml, which is not the focus of discussion here), and then pass the coordinates as command-line parameters.

adb shell input tap 100 500

The above command simulates clicking the control at the screen coordinates (100, 500).

adb shell input swipe 100 500 200 600

The above command is an operation that simulates a finger sliding down and to the right on the screen.

adb shell input keyevent "KEYCODE_BACK"

The above command simulates the click of the return button.
A complete automatic interactive process can be executed in sequence by the above series of commands.

Advantages of ADB scripting

The implementation is simple. You only need to obtain simple information such as the coordinates of the target element to complete the relevant operations

It can realize the automatic interaction of webview

Disadvantages of ADB scripting

The flexibility is not enough. It depends on the dead coordinates. The view position transformation caused by the change of App interface will make the relevant commands in the script unable to be executed. It is necessary to re analyze the page coordinates

An ADB link or socket link needs to be established. Changes in network conditions during interaction will affect the automatic interaction effect

ADB script application scenario

App s with simple interaction, low iteration frequency and low security level

webview page, App developed by fluent

2. Android native method realizes automatic interaction

We can use various plug-in frameworks to control the interface elements of the App page. One idea is to use ActivityLifecycleCallbacks in the plug-in to monitor the life cycle of each activity.

public class MyApplication extends Application {
    private static final String TAG = "MyApplication";
    //Declare an interface that listens to the life cycle of activities
    private ActivityLifecycleCallbacks activityLifecycleCallbacks = new ActivityLifecycleCallbacks() {
        /**
         * application When the declaration cycle of each Activity under changes, the following functions will be triggered.
         */
        @Override
        public void onActivityCreated(Activity activity, Bundle savedInstanceState) {
            //How to distinguish which activity in the parameter represents which activity you write.
            if (activity.getClass() == MainActivity.class)
                Log.d(TAG, "MainActivityCreated.");
            else if(activity.getClass()== SecondActivity.class)
                Log.d(TAG, "SecondActivityCreated.");
        }

        @Override
        public void onActivityStarted(Activity activity) {
            Log.d(TAG, "onActivityStarted.");
        }

        @Override
        public void onActivityResumed(Activity activity) {
            Log.d(TAG, "onActivityResumed.");
        }

        @Override
        public void onActivityPaused(Activity activity) {
            Log.d(TAG, "onActivityPaused.");
        }

        @Override
        public void onActivityStopped(Activity activity) {
            Log.d(TAG, "onActivityStopped.");
        }

        @Override
        public void onActivitySaveInstanceState(Activity activity, Bundle outState) {
        }

        @Override
        public void onActivityDestroyed(Activity activity) {
            Log.d(TAG, "onActivityDestroyed.");
        }
    };

    @Override
    public void onCreate() {
        super.onCreate();
        //Register the lifecycle callback interface of your own Activity.! [Alt text](./WechatIMG59.png)

        registerActivityLifecycleCallbacks(activityLifecycleCallbacks);
    }

    @Override
    public void onTerminate() {
        //Log off this interface.
        unregisterActivityLifecycleCallbacks(activityLifecycleCallbacks);
        super.onTerminate();
    }
}

After listening to the activity, you can analyze the viewId and attribute of the activity interface element with the help of uiautomator. Different automation methods can be adopted for different interface views.

2.1 processing method of simple view

As shown below:

For view s like this, you can directly obtain the resource id and confirm that the clickable attribute is true. The operation method is relatively simple. You can perform the following operations in the monitored activity life cycle:

int fl_btn = activity.getResources().getIdentifier("dashboard_title",         "id", "com.android.settings");
View v =  activity.findViewById(fl_btn);

v.performClick();

2.2 processing method of view with hidden attribute

In some pages that hide the properties of the view, especially those that are mixed developed with React Native, the above method is no longer effective, as shown in the following figure:

As shown in the figure, the clickable attribute of the selected viewgroup and its child views is false, and the resource id of the view cannot be obtained. At this time, you can use the layout information dump in the figure and use the Xpath element positioning tool to obtain the view of the interface. Because the click attribute of these views is false, Therefore, the method of clicking by calling performClick is invalid. At this time, consider the more important class related to touch event transmission at the lower layer of click: motionevent. Motionevent can simulate almost all interactive events, including clicking, sliding, two finger operation, etc. Take click as an example:

    private void simulateClick(View view, float x, float y) {
        long time = SystemClock.uptimeMillis();//Must be systemclock uptimeMillis(). 

        MotionEvent downEvent = MotionEvent.obtain(time, time, MotionEvent.ACTION_DOWN, x, y, 0);

        time += 500;

        MotionEvent upEvent = MotionEvent.obtain(time, time, MotionEvent.ACTION_UP, x, y, 0);

        view.onTouchEvent(downEvent);
        view.onTouchEvent(upEvent);
    }

In case of sliding operation, several actions can be added in the middle of the starting position_ MotionEvent of type move To sum up, the advantages and disadvantages of time interaction automation with the help of the system's native method are roughly as follows:

Advantages of automated interaction with plug-in framework

• Strong maintainability, because the view object of the interface can be obtained directly. Therefore, even if the page layout changes, as long as the view still exists, there is no need to modify the code

• The simulation effect is better and closer to human operation than script mode

Lack of automatic interaction with plug-in framework

• Poor support for webview and fluent framework App

Application scenario

• App s with frequent version iterations

• App s developed by non fluent framework

The above analyzes two common methods of simulating real users for automatic operation, script method and calling native method. These two methods can basically complete simple interaction process. On this basis, we can also go deep into some deeper interaction implementation, such as automating various verification, which can also be completed based on these two methods.