Android | InputManagerService and Input Event Collection

Posted by neiltaylormade on Sat, 27 Jul 2019 20:16:05 +0200

Preface

  • Event distribution mechanisms are fundamental and important knowledge in Android, and Activity#dispatchKeyEvent() or Activity#dispatchTouchEvent() is generally considered as the starting point for distribution.So the question arises, who called Activity's method?How do input events occur?
  • The Android system has a complete set of event handling mechanisms from the Linux kernel to the application framework layer to the application layer
  • This article will take InputManagerService as a clue to analyze the generation-collection-distribution process of input events, and hope to help
Mind Mapping
Input Event Diagram

1. Start the service

After the Android system starts, the system process SystemServer.java We will start each system service in turn, and we will search InputManagerService, which won't match too many things, to comb out the code:

More information about system services: [Android | System Startup Process]

// /frameworks/base/services/java/com/android/server/SystemServer.java
private void startBootstrapServices() {
    InputManagerService inputManager = null;
    WindowManagerService wm = null;
        
    // ...
        
    // Instantiate InputManagerService
    inputManager = new InputManagerService(context);
    // Instantiate WindowManagerService
    wm = WindowManagerService.main(context,inputManager,...);
    // Add to ServiceManager Unified Management
    ServiceManager.addService(Context.WINDOW_SERVICE, wm, ...);
    ServiceManager.addService(Context.INPUT_SERVICE, inputManager,...);
        
    // ...
        
    inputManager.setWindowManagerCallbacks(wm.getInputMonitor());
    // Start the InputManager service
    inputManager.start();
}
IMS and WMS UML Class Diagrams

You can see that the system processes instantiate InputManagerService and WindowManagerService, respectively. The instance of the former is passed directly to the latter, while the latter passes an InputMonitor object to the former. Why?Skip for a moment and continue looking down InputManagerService.java:

// /frameworks/base/services/core/java/com/android/server/input/InputManagerService.java

private static native long nativeInit(InputManagerService service,Context context, MessageQueue messageQueue);

private static native void nativeStart(long ptr);

public InputManagerService(Context context) {
    // ...
    // native Layer Called
    mPtr = nativeInit(this, mContext, mHandler.getLooper().getQueue());
    // ...
}

public void start() {
    // native Layer Called
    nativeStart(mPtr);
    // ...
}

Here are just two static methods that are called to the native layer and continue looking down com_android_server_input_InputManagerService.cpp:

// /frameworks/base/services/core/jni/com_android_server_input_InputManagerService.cpp

static jlong nativeInit(JNIEnv* env, jclass /* clazz */, jobject serviceObj, jobject contextObj, jobject messageQueueObj) {
    // ...
    // Instantiate NativeInputManager object
    NativeInputManager* im = new NativeInputManager(contextObj, serviceObj,messageQueue->getLooper());
    im->incStrong(0);
    return reinterpret_cast<jlong>(im);
}

static void nativeStart(JNIEnv* env, jclass /* clazz */, jlong ptr) {
    NativeInputManager* im = reinterpret_cast<NativeInputManager*>(ptr);
    // Call to InputManager#start()
    status_t result = im->getInputManager()->start();
}

NativeInputManager::NativeInputManager(jobject contextObj, 
        jobject serviceObj, const sp<Looper>& looper) : 
    mLooper(looper), mInteractive(true) {
    // ...
    // Instantiate EventHub Object
    sp<EventHub> eventHub = new EventHub();
    // Instantiate InputManager object
    mInputManager = new InputManager(eventHub, this, this);
}

public: 
inline sp<InputManager> getInputManager() const { return mInputManager; }

You can see that NativeInputManager is instantiated at the native level, and both InputManager and EventHub objects are instantiated in their construction methods. What is the latter?Skip for a moment and continue looking down InputManager.cpp:

// /frameworks/native/services/inputflinger/InputManager.cpp

InputManager::InputManager(
        const sp<EventHubInterface>& eventHub,
        const sp<InputReaderPolicyInterface>& readerPolicy,
        const sp<InputDispatcherPolicyInterface>& dispatcherPolicy) {
    // Instantiate InputDispatcher and InputReader
    mDispatcher = new InputDispatcher(dispatcherPolicy);
    // InputDispatcher instance and EventHub instance passed to InputReader
    mReader = new InputReader(eventHub, readerPolicy, mDispatcher);
    initialize();
}

void InputManager::initialize() {
    // Instantiate two threads
    mReaderThread = new InputReaderThread(mReader);
    mDispatcherThread = new InputDispatcherThread(mDispatcher);
}

status_t InputManager::start() {
    // Run two threads
    status_t result = mDispatcherThread->run("InputDispatcher", PRIORITY_URGENT_DISPLAY);
    result = mReaderThread->run("InputReader", PRIORITY_URGENT_DISPLAY);
    // ...
}

You can see that InputDispatcher and InputReader are instantiated in the construction method of InputManager, and then InputDispatcher Thread and InputReaderThread threads are started in start() (inherited from Thread.cpp).

Timethreads Diagram - 01

What do the two threads do separately?Keep looking down InputReder.cpp and InputDispatcher.cpp:

// /frameworks/native/services/inputflinger/InputReader.cpp

InputReaderThread::InputReaderThread(const sp<InputReaderInterface>& reader) : 
// Inherited from Thread
Thread(/*canCallJava*/ true), mReader(reader) {
}

/**
 * true: The loop executes threadLoop() until requireExit() is called to exit the loop
 **/
bool InputReaderThread::threadLoop() {
    // Read/Collect an Event
    mReader->loopOnce();
    return true;
}
// /frameworks/native/services/inputflinger/InputDispatcher.cpp

InputDispatcherThread::InputDispatcherThread(const sp<InputDispatcherInterface>& dispatcher) : Thread(/*canCallJava*/ true), mDispatcher(dispatcher) {
}

bool InputDispatcherThread::threadLoop() {
    // Distribute an event
    mDispatcher->dispatchOnce();
    return true;
}

You can see here that the InputManagerService startup is complete, extracting the key points:

  • InputManagerService and WindowManagerService are running in system process SystemServer
  • InputManagerService starts the InputReaderThread thread and the InputDispatcher Thread thread, calling InputReader#loopOnce() and InputDispatcher#dispatchOnce() in a dead loop, respectively.
Timethreads Diagram - 02

2. Collect Events

As mentioned in the previous section, the InputReaderThread thread executes InputReader#loopOnce() in a dead loop to collect events, simplifying the code as follows:

// /frameworks/native/services/inputflinger/InputReader.cpp

void InputReader::loopOnce() {
    // ...
    // Reads events from EventHub and stores them in pointer mEventBuffer
    size_t count = mEventHub->getEvents(timeoutMillis, mEventBuffer, EVENT_BUFFER_SIZE);
    if (count) {
        // Handle read events
        processEventsLocked(mEventBuffer, count);
    }
    // ...
    // What does this line do?Later on
    mQueuedListener->flush();
}

void InputReader::processEventsLocked(const RawEvent* rawEvents, size_t count) {
    // Traverse through each event
    for (const RawEvent* rawEvent = rawEvents; count;) {
        // ...
        int32_t type = rawEvent->type;
        if(type <  EventHubInterface::FIRST_SYNTHETIC_EVENT){
            // Branch 1: Input events
            // ...
                        
            // Handle each input event
            int32_t deviceId = rawEvent->deviceId;
            processEventsForDeviceLocked(deviceId, rawEvent, batchSize);
        }else{
            // Branch 2: Device events
            switch(rawEvent->type){
                    case EventHubInterface::DEVICE_ADDED:
                        addDeviceLocked(rawEvent->when, rawEvent->deviceId);
                        break;
                       // ...
            }
        }
    }
    // ...
}

You can see that loopOnce() takes the original events from the EventHub instance and processes each event in turn, in two ways:

  • Input Events (Branch 1)
  • Device Events (Branch 2)

Let's first look at processEventsForDeviceLocked() for Branch 1, simplifying the code as follows:

// /frameworks/native/services/inputflinger/InputReader.cpp

void InputReader::processEventsForDeviceLocked(int32_t deviceId, const RawEvent* rawEvents, size_t count) {
    ssize_t deviceIndex = mDevices.indexOfKey(deviceId);
    // ...
    InputDevice* device = mDevices.valueAt(deviceIndex);
    // ...
    device->process(rawEvents, count);
}

void InputDevice::process(const RawEvent* rawEvents, size_t count) {
    size_t numMappers = mMappers.size();
    // Traverse through each event
    for (const RawEvent* rawEvent = rawEvents; count != 0; rawEvent++) {
            // Handle each InputMapper in turn
            for (size_t i = 0; i < numMappers; i++) {
                    InputMapper* mapper = mMappers[i];
                    mapper->process(rawEvent);
            }
    }
}

You can see that the input events are handed over to InputDevice, which actually executes InputMapper#process() several times internally. What are InputDevice and InputMapper here? Remember the addDeviceLocked() of Branch 2?The simplified code is as follows:

// /frameworks/native/services/inputflinger/InputReader.cpp

void InputReader::addDeviceLocked(nsecs_t when, int32_t deviceId) {
    // ...
    // Create an InputDevice instance
    InputDevice* device = createDeviceLocked(deviceId, controllerNumber, identifier, classes);
    // ...
    // Add to mDevices list 
    // InputReader.h: KeyedVector<int32_t, InputDevice*> mDevices;
    mDevices.add(deviceId, device);
    // ...
}

InputDevice* InputReader::createDeviceLocked(int32_t deviceId, int32_t controllerNumber, const InputDeviceIdentifier& identifier, uint32_t classes) {
    // Create an InputDevice instance
    InputDevice* device = new InputDevice(&mContext, deviceId, bumpGenerationLocked(), controllerNumber, identifier, classes);

    // ...
        
    // Scroll wheel-like devices. -Wheel
    if (classes & INPUT_DEVICE_CLASS_ROTARY_ENCODER) {
        device->addMapper(new RotaryEncoderInputMapper(device));
    }
        
    // ...
        
    // Touchscreens and touchpad devices. - Touchscreen
    if (classes & INPUT_DEVICE_CLASS_TOUCH_MT) {
        // Multi-touch device
        device->addMapper(new MultiTouchInputMapper(device));
    } else if (classes & INPUT_DEVICE_CLASS_TOUCH) {
        // Single touch device
        device->addMapper(new SingleTouchInputMapper(device));
    }
        
    // Joystick-like devices. -joystick
    if (classes & INPUT_DEVICE_CLASS_JOYSTICK) {
        device->addMapper(new JoystickInputMapper(device));
    }
        // ...
}

In Branch 2, you can see that the InputDevice instance is created first, and then different InputMapper s are added depending on the type of device event, such as wheel, touch, and joystick. This also shows that multiple input devices are supported on an Android device. Of course, touch devices are the only ones in this article.Focus of analysis.

Let's go back to Branch 1, InputMapper#process(), because we focus on touch devices, so we only care about MultiTouchInputMapper and SingleTouchInputMapper, coded as follows:

// /frameworks/native/services/inputflinger/InputReader.cpp

// Multi-touch device
void MultiTouchInputMapper::process(const RawEvent* rawEvent) {
    TouchInputMapper::process(rawEvent);
    // ...
}

// Single touch device
void SingleTouchInputMapper::process(const RawEvent* rawEvent) {
    TouchInputMapper::process(rawEvent);
    // ...
}

void TouchInputMapper::process(const RawEvent* rawEvent) {
    // ...
    if (rawEvent->type == EV_SYN && rawEvent->code == SYN_REPORT) {
        sync(rawEvent->when);
    }
}

void TouchInputMapper::sync(nsecs_t when) {
    // ...
    processRawTouches(false /*timeout*/);
}

void TouchInputMapper::processRawTouches(bool timeout) {
    // ...
    cookAndDispatch(mCurrentRawState.when);
}

void TouchInputMapper::cookAndDispatch(nsecs_t when) {
    // ...
        
    // Key events
    if (consumeRawTouches(when, policyFlags)) {
        mCurrentRawState.rawPointerData.clear();
    }
    // ...
    // Contact Events
    dispatchPointerUsage(when, policyFlags, pointerUsage);
    // ...
}

From the simplified code, you can see that whether it's MultiTouchInputMapper or SingleTouchInputMapper, you end up with the cookAndDispatch() method, which handles two types of events:

  • Key events
  • Contact Events
// /frameworks/native/services/inputflinger/InputReader.cpp

bool TouchInputMapper::consumeRawTouches(nsecs_t when, uint32_t policyFlags) {
    // ...
    // Handle virtual key events
    dispatchVirtualKey(...)
    // ...
}

void TouchInputMapper::dispatchVirtualKey(...) {
    // ...
    // Instantiate a NotifyKeyArgs
    NotifyKeyArgs args(...);
    // Callback
    getListener()->notifyKey(&args);
}
// /frameworks/native/services/inputflinger/InputReader.cpp

void TouchInputMapper::dispatchPointerUsage(nsecs_t when, uint32_t policyFlags, PointerUsage pointerUsage) {
    // ...
    // Handle contact events
    dispatchPointerGestures(when, policyFlags, false /*isTimeout*/);
    // ...
}

void TouchInputMapper::dispatchPointerGestures(nsecs_t when, uint32_t policyFlags, bool isTimeout) {
    // ...
    // Handle contact events
    dispatchMotion(...)
    // ...
}

void TouchInputMapper::dispatchMotion(...){
    // ...
    // Instantiate a NotifyMotionArgs
    NotifyMotionArgs args(...);
    // Callback
    getListener()->notifyMotion(&args);
}

From the simplified code, NotifyKeyArgs and NotifyMotionArgs are instantiated for key and contact events, respectively, and getListener ()->notifyKey () and getListener ()->notifyMotion () are then called, respectively.

Event Collection UML Time Series Diagram - 01

GetListener()->notifyKey() looks like it tells the listener that an event has been collected, doesn't it?Find the code for getListener():

// /frameworks/native/services/inputflinger/InputReader.cpp

InputListenerInterface* InputReader::ContextImpl::getListener() {
    // Weak pointer usage, simply understood as returning mQueuedListener
    return mReader->mQueuedListener.get();
}

// --- InputReader ---

InputReader::InputReader(const sp<EventHubInterface>& eventHub, const sp<InputReaderPolicyInterface>& policy, const sp<InputListenerInterface>& listener) {
    // Instantiate QueuedInputListener
    mQueuedListener = new QueuedInputListener(listener);
}

You can see that the return value of getListener() is mQueuedListener, and the return value type is InputListenerInterface, which is created in the construction method of InputReader and wraps the third parameter listener of the InputReader construction method. What is this listener?Let's go back to where we instantiated InputReader:

// /frameworks/native/services/inputflinger/InputManager.cpp

InputManager::InputManager(...) {
    mDispatcher = new InputDispatcher(dispatcherPolicy);
    // InputDispatcher instance and EventHub instance passed to InputReader
    mReader = new InputReader(eventHub, readerPolicy, mDispatcher);
    initialize();
}

As it turns out, the InputDispatcher instance is the third parameter listener, so let's look at the definitions of InputDispatcher and InputListenerInterface: InputDispatcher.h and InputListener.h

// /frameworks/native/services/inputflinger/InputDispatcher.h

class InputDispatcher : public InputDispatcherInterface {
    // ...
}

class InputDispatcherInterface : public virtual RefBase, public InputListenerInterface {
    //...
}
// /frameworks/native/services/inputflinger/InputListener.h

/**
 * base class
 **/
struct NotifyArgs {
    virtual ~NotifyArgs() { }
    virtual void notify(const sp<InputListenerInterface>& listener) const = 0;
};

/**
 * Contact Events
 **/
struct NotifyMotionArgs : public NotifyArgs {
    // ...
}

/**
 * Key events
 **/
struct NotifyKeyArgs : public NotifyArgs {
    // ...
}

class InputListenerInterface : public virtual RefBase {/**Weak Pointer**/
    // ...
public:
    virtual void notifyKey(const NotifyKeyArgs* args) = 0;
    virtual void notifyMotion(const NotifyMotionArgs* args) = 0;
        
    // ...  
};

class QueuedInputListener : public InputListenerInterface {
    // ...
public:
    explicit QueuedInputListener(const sp<InputListenerInterface>& innerListener);
        
    virtual void notifyKey(const NotifyKeyArgs* args);
    virtual void notifyMotion(const NotifyMotionArgs* args);
        
    void flush();
        
    // ...

private:
    sp<InputListenerInterface> mInnerListener;
    Vector<NotifyArgs*> mArgsQueue;
};

You can see that both QueuedInputListener and InputDispatcher implement InputListenerInterface, which is actually the wrapper class of the latter.

InputDispatcher UML Class Diagram

Let's see what the wrapper class QueuedInputListener does, and simplify the code as follows InputListener.cpp:

// /frameworks/native/services/inputflinger/InputListener.cpp

void QueuedInputListener::notifyKey(const NotifyKeyArgs* args) {
    // push to queue
    mArgsQueue.push(new NotifyKeyArgs(*args));
}

void QueuedInputListener::notifyMotion(const NotifyMotionArgs* args) {
    // push to queue
    mArgsQueue.push(new NotifyMotionArgs(*args));
}

void NotifyMotionArgs::notify(const sp<InputListenerInterface>& listener) const {
    listener->notifyMotion(this);
}

void NotifyKeyArgs::notify(const sp<InputListenerInterface>& listener) const {
    listener->notifyKey(this);
}

void QueuedInputListener::flush() {
    size_t count = mArgsQueue.size();
    // Traverse each NotifyArgs
    for (size_t i = 0; i < count; i++) {
        NotifyArgs* args = mArgsQueue[i];
        // mInnerListener is InputDispatcher
        args->notify(mInnerListener);
        delete args;
    }
    mArgsQueue.clear();
}

You can see that the previous getListener ()->notifyMotion () or getListener ()->notifyKey () push ed the event into the mArgsQueue queue.So when exactly are these events in the queue handled?Remember the last step of InputReader#loopOnce()?

// /frameworks/native/services/inputflinger/InputReader.cpp

void InputReader::loopOnce() {
    // ...
    // Reads events from EventHub and stores them in pointer mEventBuffer
    size_t count = mEventHub->getEvents(timeoutMillis, mEventBuffer, EVENT_BUFFER_SIZE);
    if (count) {
        // Handle read events
        processEventsLocked(mEventBuffer, count);
    }
    // ...
    mQueuedListener->flush();
}

As it turns out, after all events have been push ed into the mArgsQueue queue, mQueuedListener->flush() is called, and eventually InputDispatcher#notifyKey() and InputDispatcher#notifyMotion() are called, summarized as follows:

Event Collection UML Time Series Diagram - 02

Continue to look at the simplified code for InputDispatcher#notifyKey() and InputDispatcher#notifyMotion():

// /frameworks/native/services/inputflinger/InputDispatcher.cpp

void InputDispatcher::notifyKey(const NotifyKeyArgs* args) {
    // ...
        
    // Instantiate KeyEntry
    KeyEntry* newEntry = new KeyEntry(...);
    // Join Event Queue
    enqueueInboundEventLocked(newEntry);
    // Wake up a thread waiting on mLooper
    mLooper->wake();
}

void InputDispatcher::notifyMotion(const NotifyMotionArgs* args) {
    // ...
        
    // Instantiate MotionEntry
    MotionEntry* newEntry = new MotionEntry(...);
    // Join Event Queue
    enqueueInboundEventLocked(newEntry);
    // Wake up a thread waiting on mLooper
    mLooper->wake();
}

bool InputDispatcher::enqueueInboundEventLocked(EventEntry* entry) {
    // Join the tail of the queue
    mInboundQueue.enqueueAtTail(entry);
    // ...
}

You can see that InputDispatcher#notifyKey() and InputDispatcher#notifyMotion() are similar in that they both join the event KeyEntry or MotionEntry to the end of the queue mInboundQueue and wake up in mLooper ( Looper.h ) the thread waiting on.

Timethreads Diagram - 03

Remember our well-known producer-consumer model?

Here we have executed loopOnce() once in a complete InputReaderThread thread to summarize the key points:

  • InputReader reads events from EventHub, some of which are input events, and we are concerned with key events and contact events
  • Events are eventually queued to mInboundQueue and the thread waiting on mLooper wakes up

So which thread is waking up?Is it the InputDispatcher Thread thread?The code is as follows:

// /frameworks/native/services/inputflinger/InputDispatcher.cpp

// --- InputDispatcherThread ---

InputDispatcherThread::InputDispatcherThread(...) {
    bool InputDispatcherThread::threadLoop() {
        mDispatcher->dispatchOnce();
        return true;
    }
}
// --- InputDispatcher ---

InputDispatcher::InputDispatcher(...){
    mLooper = new Looper(false);
}

void InputDispatcher::dispatchOnce() {
    nsecs_t nextWakeupTime = LONG_LONG_MAX;
    // ...
    // Distribute an event
    dispatchOnceInnerLocked(&nextWakeupTime);
    // ...
    nsecs_t currentTime = now();
    int timeoutMillis = toMillisecondTimeoutDelay(currentTime, nextWakeupTime);
    // Wait for a while on mLooper
    mLooper->pollOnce(timeoutMillis);
}

void InputDispatcher::dispatchOnceInnerLocked(nsecs_t* nextWakeupTime) {
    // ...
    // Get the event at the head of the mInboundQueue queue
    mPendingEvent = mInboundQueue.dequeueAtHead();
        
    //...
        
    switch (mPendingEvent->type) {
        // ...
        case EventEntry::TYPE_KEY: {
                // ...
                // Distribute key events
                done = dispatchKeyLocked(...);
                break;
        }
        case EventEntry::TYPE_MOTION: {
                // ...
                // Distribute Contact Events
                done = dispatchMotionLocked(...);
                break;
        }
    }
    // ...
}

You can see that dispatchOnce() runs in a dead loop in the InputDispatcher Thread thread, fetching the header event from the mInboundQueue queue each time, and performing an event distribution.After each event is distributed, mLooper->pollOnce() is called to wait for a while.During the wait, mLooper.wake() executed in the InputReaderThread thread may wake up early, triggering the next event distribution.

Here, we have a complete analysis of the event collection process for InputManagerService, and we will continue our analysis in subsequent articles on the event distribution process.

Extended reading

  • [Android | System Startup Process]
  • [Android | InputManagerService and Input Event Distribution]

Recommended reading

Thanks for liking it!Your compliment is my greatest encouragement!You can leave a message in the comments area below if you have any questions.

Topics: Android Java JoyStick Linux