1 Preface
As the core unit of Android graphics display system processing logic, it is necessary for us to understand how it starts, initializes and processes messages. In this article, we will briefly analyze some basic processing logic of the Binder system service, SurfaceFlinger. Next, it will be explained in two parts:
>>Surfaceflinger startup and initialization
>>Surface flinger message queue processing mechanism
Tips:
Code locations covered in this article:
/frameworks/native/services/surfaceflinger/
2 startup and initialization of surfaceflinger
SurfaceFlinger is a Binder system service. When the Android device starts up, it will bring the SurfaceFlinger service process and complete some initialization actions.
Starting with Android S, surfaceflinger is compiled into an executable binary file: surfacelinker (placed under device / system/bin /).
makefile of executable surfacelinker
In the following code, you can see the executable file and surface linker rc is associated with the init rc file, so that when starting up, the init process can parse the rc file and bring up the SurfaceFlinger service process
cc_binary { name: "surfaceflinger", defaults: ["libsurfaceflinger_binary"], init_rc: ["surfaceflinger.rc"], srcs: [ ":surfaceflinger_binary_sources", // Note: SurfaceFlingerFactory is not in the filegroup so that it // can be easily replaced. "SurfaceFlingerFactory.cpp", ], shared_libs: [ "libSurfaceFlingerProp", ], logtags: ["EventLog/EventLogTags.logtags"], }
Let's look at the surface linker RC the contents of this file are mainly used to set some SurfaceFlinger service process startup properties
service surfaceflinger /system/bin/surfaceflinger class core animation user system group graphics drmrpc readproc capabilities SYS_NICE onrestart restart zygote task_profiles HighPerformance socket pdx/system/vr/display/client stream 0666 system graphics u:object_r:pdx_display_client_endpoint_socket:s0 socket pdx/system/vr/display/manager stream 0666 system graphics u:object_r:pdx_display_manager_endpoint_socket:s0 socket pdx/system/vr/display/vsync stream 0666 system graphics u:object_r:pdx_display_vsync_endpoint_socket:s0
Here, we simply understand it as: when the device is started, the init process parses the surface linker RC, and then execute / system / bin / surfacelinker to start the SurfaceFlinger service process.
If you execute ps under the device console, you can see the process PID
console:/ $ ps -A | grep surfaceflinger system 210 1 133412 38160 0 0 S surfaceflinger
main function entry of executable surfacelinker
Here we only extract the main code and comment as follows:
* /frameworks/native/services/surfaceflinger/main_surfaceflinger.cpp int main(int, char**) { ... // When SF is launched in its own process, limit the number of // binder threads to 4. ProcessState::self()->setThreadPoolMaxThreadCount(4); ... // start the thread pool sp<ProcessState> ps(ProcessState::self()); ps->startThreadPool(); ... // Create a SurfaceFlinger object, pointed by a strong pointer. // SurfaceFlinger inherits the RefBase class, so once the new object is assigned to the sp pointer, the call of onFirstRef method of SurfaceFlinger class will be triggered immediately. // instantiate surfaceflinger sp<SurfaceFlinger> flinger = surfaceflinger::createSurfaceFlinger(); ... // Formal initialization of SurfaceFlinger class // initialize before clients can connect flinger->init(); // SurfaceFlinger registers Binder service with ServiceManager, // In this way, you can use getservice + service in other processes_ Name to get the SurfaceFlinger service, and then you can communicate with the SurfaceFlinger class Binder. // publish surface flinger sp<IServiceManager> sm(defaultServiceManager()); sm->addService(String16(SurfaceFlinger::getServiceName()), flinger, false, IServiceManager::DUMP_FLAG_PRIORITY_CRITICAL | IServiceManager::DUMP_FLAG_PROTO); ... // SurfaceFlinger class enters the main loop (note here that SurfaceFlinger class does not inherit Threads class and does not follow the interface execution order of Threads class) // run surface flinger in this thread flinger->run(); return 0; }
For the main function, simply grasp the following points:
- Create a SurfaceFlinger object and trigger the execution of SurfaceFlinger::onFirstRef()
- Call SurfaceFlinger::init() to initialize
- Register the service with servicemanager (name is "SurfaceFlinger")
- Call SurfaceFlinger::run()
Tips:
Execute the service list command on the device console to see the registered service: the registered name is SurfaceFlinger, and the interface implemented by this service is Android ui. ISurfaceComposer
console:/ $ service list | grep Surface 1 SurfaceFlinger: [android.ui.ISurfaceComposer]
SurfaceFlinger class definition
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.h class SurfaceFlinger : public BnSurfaceComposer, public PriorityDumper, private IBinder::DeathRecipient, private HWC2::ComposerCallback, private ISchedulerCallback {
Create a SurfaceFinger instance object
Call surfacelinker:: createsurfaceflinger() to create a SurfaceFlinger instance and pass a factory object as a parameter
* /frameworks/native/services/surfaceflinger/SurfaceFlingerFactory.cpp sp<SurfaceFlinger> createSurfaceFlinger() { static DefaultFactory factory; return new SurfaceFlinger(factory); }
Let's take a brief look at the definition of Factory:
* /frameworks/native/services/surfaceflinger/SurfaceFlingerFactory.h // The interface that SurfaceFlinger uses to create all of the implementations // of each interface. class Factory { public: virtual std::unique_ptr<HWComposer> createHWComposer(const std::string& serviceName) = 0; virtual std::unique_ptr<MessageQueue> createMessageQueue() = 0; virtual std::unique_ptr<scheduler::VsyncConfiguration> createVsyncConfiguration( Fps currentRefreshRate) = 0; virtual std::unique_ptr<Scheduler> createScheduler(const scheduler::RefreshRateConfigs&, ISchedulerCallback&) = 0; virtual sp<SurfaceInterceptor> createSurfaceInterceptor() = 0; virtual sp<StartPropertySetThread> createStartPropertySetThread( bool timestampPropertyValue) = 0; virtual sp<DisplayDevice> createDisplayDevice(DisplayDeviceCreationArgs&) = 0; virtual sp<GraphicBuffer> createGraphicBuffer(uint32_t width, uint32_t height, PixelFormat format, uint32_t layerCount, uint64_t usage, std::string requestorName) = 0; virtual void createBufferQueue(sp<IGraphicBufferProducer>* outProducer, sp<IGraphicBufferConsumer>* outConsumer, bool consumerIsSurfaceFlinger) = 0; virtual sp<IGraphicBufferProducer> createMonitoredProducer(const sp<IGraphicBufferProducer>&, const sp<SurfaceFlinger>&, const wp<Layer>&) = 0; virtual sp<BufferLayerConsumer> createBufferLayerConsumer(const sp<IGraphicBufferConsumer>&, renderengine::RenderEngine&, uint32_t tex, Layer*) = 0; virtual std::unique_ptr<surfaceflinger::NativeWindowSurface> createNativeWindowSurface( const sp<IGraphicBufferProducer>&) = 0; virtual std::unique_ptr<compositionengine::CompositionEngine> createCompositionEngine() = 0; virtual sp<BufferQueueLayer> createBufferQueueLayer(const LayerCreationArgs& args) = 0; virtual sp<BufferStateLayer> createBufferStateLayer(const LayerCreationArgs& args) = 0; virtual sp<EffectLayer> createEffectLayer(const LayerCreationArgs& args) = 0; virtual sp<ContainerLayer> createContainerLayer(const LayerCreationArgs& args) = 0; virtual std::unique_ptr<FrameTracer> createFrameTracer() = 0; virtual std::unique_ptr<frametimeline::FrameTimeline> createFrameTimeline( std::shared_ptr<TimeStats> timeStats, pid_t surfaceFlingerPid) = 0; protected: ~Factory() = default; };
The interface that SurfaceFlinger uses to create all of the implementations of each interface
SurfaceFlinger uses Factory to create all objects that implement the corresponding interface. DefaultFactory is used by default in SurfaceFlinger, which defines the implementation of various createXXX() methods. For details, see: / frameworks / native / services / surfacelinker / surfaceflingerdefaultfactory cpp
SurfaceFlinger constructor
The code of the constructor is only intercepted and posted here, mainly reading some attribute values to initialize some member variables. Some of these attribute values are debug parameters. In addition, use the createXXX method of mFactory to instantiate various objects.
For example: create message queue mEventQueue(mFactory.createMessageQueue())
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp SurfaceFlinger::SurfaceFlinger(Factory& factory, SkipInitializationTag) : mFactory(factory), mInterceptor(mFactory.createSurfaceInterceptor()), mTimeStats(std::make_shared<impl::TimeStats>()), mFrameTracer(mFactory.createFrameTracer()), mFrameTimeline(mFactory.createFrameTimeline(mTimeStats, getpid())), mEventQueue(mFactory.createMessageQueue()), mCompositionEngine(mFactory.createCompositionEngine()), mHwcServiceName(base::GetProperty("debug.sf.hwc_service_name"s, "default"s)), mTunnelModeEnabledReporter(new TunnelModeEnabledReporter()), mInternalDisplayDensity(getDensityFromProperty("ro.sf.lcd_density", true)), mEmulatedDisplayDensity(getDensityFromProperty("qemu.sf.lcd_density", false)), mPowerAdvisor(*this) { ALOGI("Using HWComposer service: %s", mHwcServiceName.c_str()); mSetInputWindowsListener = new SetInputWindowsListener([&]() { setInputWindowsFinished(); }); } SurfaceFlinger::SurfaceFlinger(Factory& factory) : SurfaceFlinger(factory, SkipInitialization) { ALOGI("SurfaceFlinger is starting"); hasSyncFramework = running_without_sync_framework(true); dispSyncPresentTimeOffset = present_time_offset_from_vsync_ns(0); useHwcForRgbToYuv = force_hwc_copy_for_virtual_displays(false); maxFrameBufferAcquiredBuffers = max_frame_buffer_acquired_buffers(2); maxGraphicsWidth = std::max(max_graphics_width(0), 0); maxGraphicsHeight = std::max(max_graphics_height(0), 0); hasWideColorDisplay = has_wide_color_display(false); ... }
Surfacelinker will read the system properties in surfaceflinger properties. For details, see the source code: / frameworks / native / services / surfacelinker / sysprop/
Tips:
For suggestions on system attributes, see:
https://source.android.google.cn/devices/architecture/sysprops-apis?hl=zh-cn
https://blog.csdn.net/askfgx2010/article/details/112308665
SurfaceFlinger::onFirstRef
After creating the SurfaceFlinger object above, it will be executed to SurfaceFlinger::onFirstRef immediately. In this method, one thing is done to initialize the message queue, as follows:
void SurfaceFlinger::onFirstRef() { mEventQueue->init(this); }
We will talk about the detailed analysis of message queue operation in the next section, which will not be expanded here.
SurfaceFlinger::init
Return to the main function of the executable surfacelinker. After creating the surfaceflinger object, the SurfaceFlinger::init() method is called:
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp void SurfaceFlinger::init() { ALOGI( "SurfaceFlinger's main thread ready to run. " "Initializing graphics H/W..."); Mutex::Autolock _l(mStateLock); // Set the CompositionEngine property to create a RenderEngine object // Get a RenderEngine for the given display / config (can't fail) // TODO(b/77156734): We need to stop casting and use HAL types when possible. // Sending maxFrameBufferAcquiredBuffers as the cache size is tightly tuned to single-display. mCompositionEngine->setRenderEngine(renderengine::RenderEngine::create( renderengine::RenderEngineCreationArgs::Builder() .setPixelFormat(static_cast<int32_t>(defaultCompositionPixelFormat)) .setImageCacheSize(maxFrameBufferAcquiredBuffers) .setUseColorManagerment(useColorManagement) .setEnableProtectedContext(enable_protected_contents(false)) .setPrecacheToneMapperShaderOnly(false) .setSupportsBackgroundBlur(mSupportsBlur) .setContextPriority( useContextPriority ? renderengine::RenderEngine::ContextPriority::REALTIME : renderengine::RenderEngine::ContextPriority::MEDIUM) .build())); // Set SF main policy after initializing RenderEngine which has its own policy. if (!SetTaskProfiles(0, {"SFMainPolicy"})) { ALOGW("Failed to set main task profile"); } // Create an HWComposer object and pass in a name attribute, and then set the object attribute through mcompositionengine - > sethwcomposer. mCompositionEngine->setTimeStats(mTimeStats); mCompositionEngine->setHwComposer(getFactory().createHWComposer(mHwcServiceName)); mCompositionEngine->getHwComposer().setCallback(this); ClientCache::getInstance().setRenderEngine(&getRenderEngine()); if (base::GetBoolProperty("debug.sf.enable_hwc_vds"s, false)) { enableHalVirtualDisplays(true); } // processDisplayHotplugEventsLocked(); Process any initial hot plug and display the results of the changes // This method mainly calls initScheduler(displayId); // Process any initial hotplug and resulting display changes. processDisplayHotplugEventsLocked(); const auto display = getDefaultDisplayDeviceLocked(); LOG_ALWAYS_FATAL_IF(!display, "Missing internal display after registering composer callback."); const auto displayId = display->getPhysicalId(); LOG_ALWAYS_FATAL_IF(!getHwComposer().isConnected(displayId), "Internal display is disconnected."); // initialize our drawing state mDrawingState = mCurrentState; // Initialize Display information // set initial conditions (e.g. unblank default device) initializeDisplays(); mPowerAdvisor.init(); char primeShaderCache[PROPERTY_VALUE_MAX]; property_get("service.sf.prime_shader_cache", primeShaderCache, "1"); if (atoi(primeShaderCache)) { if (setSchedFifo(false) != NO_ERROR) { ALOGW("Can't set SCHED_OTHER for primeCache"); } mRenderEnginePrimeCacheFuture = getRenderEngine().primeCache(); if (setSchedFifo(true) != NO_ERROR) { ALOGW("Can't set SCHED_OTHER for primeCache"); } } getRenderEngine().onPrimaryDisplaySizeChanged(display->getSize()); // Inform native graphics APIs whether the present timestamp is supported: const bool presentFenceReliable = !getHwComposer().hasCapability(hal::Capability::PRESENT_FENCE_IS_NOT_RELIABLE); mStartPropertySetThread = getFactory().createStartPropertySetThread(presentFenceReliable); // Start a thread for setting properties, and use property in this thread_ Set to set some attribute values if (mStartPropertySetThread->Start() != NO_ERROR) { ALOGE("Run StartPropertySetThread failed!"); } ALOGV("Done initializing"); }
The work completed by the SurfaceFlinger::init method is about:
♦ Configuration of CompositionEngine, create RenderEngine object for Client compositing mode (GPU) compositing;
♦ Initialize HWComposer and register the callback interface mcompositionengine - > gethwcomposer() Setcallback (this), HAL will call back some methods;
♦ Handle the hot plug and change events of the Display screen processdisplayhotplugevents locked;
♦ Initialize display deviceinitializedisplays;
♦ Start a thread for setting properties, and use property in this thread_ Set to set some property values mstartpropertysetthread - > start();
SurfaceFlinger::run
In the main function of the executable surfacelinker, after registering the service SurfaceFinger into ServiceManger, the method SurfaceFlinger::run is subsequently executed. The code is as follows:
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp void SurfaceFlinger::run() { while (true) { mEventQueue->waitMessage(); } }
This method is also very simple. It is an infinite loop of while(true), and the message queue waits for the arrival of the message. The surface linker waits for message processing in the main thread.
At this point, the process started by SurfaceFlinger is probably completed. Of course, we haven't analyzed many details in depth.
3. Message queue processing mechanism of surfaceflinger
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.h //Declaration of member variable mEventQueue std::unique_ptr<MessageQueue> mEventQueue; ======== * /frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp // Create a message queue in the SurfaceFlinger constructor mEventQueue(mFactory.createMessageQueue()),
* /frameworks/native/services/surfaceflinger/SurfaceFlingerDefaultFactory.cpp std::unique_ptr<MessageQueue> DefaultFactory::createMessageQueue() { return std::make_unique<android::impl::MessageQueue>(); }
Android Native -- Message/Handler/Looper mechanism (principle)
Android Native -- Message/Handler/Looper mechanism (application)
Definition of android::impl::MessageQueue
* /frameworks/native/services/surfaceflinger/Scheduler/MessageQueue.h namespace impl { class MessageQueue : public android::MessageQueue { sp<SurfaceFlinger> mFlinger; sp<Looper> mLooper; sp<Handler> mHandler; ... }
* /frameworks/native/services/surfaceflinger/Scheduler/MessageQueue.h void init(const sp<SurfaceFlinger>& flinger) override; void initVsync(scheduler::VSyncDispatch&, frametimeline::TokenManager&, std::chrono::nanoseconds workDuration) override; void setDuration(std::chrono::nanoseconds workDuration) override; void setInjector(sp<EventThreadConnection>) override; void waitMessage() override; void postMessage(sp<MessageHandler>&&) override; // sends INVALIDATE message at next VSYNC void invalidate() override; // sends REFRESH message at next VSYNC void refresh() override;
MessageQueue::init method
* /frameworks/native/services/surfaceflinger/Scheduler/MessageQueue.cpp void MessageQueue::init(const sp<SurfaceFlinger>& flinger) { mFlinger = flinger; mLooper = new Looper(true); mHandler = new Handler(*this); }
MessageQueue::postMessage method
void MessageQueue::postMessage(sp<MessageHandler>&& handler) { mLooper->sendMessage(handler, Message()); }
MessageQueue::waitMessage method
Keep calling mloop - > pollonce (- 1) in the waitMessage method to check whether there is a message to be processed in the loop. If so, call the MessageHandler corresponding to the message for processing.
void MessageQueue::waitMessage() { do { IPCThreadState::self()->flushCommands(); int32_t ret = mLooper->pollOnce(-1); ... } while (true); }
Definition of Handler
Handler inherits from MessageHandler and implements the handleMessage method, in which messages are processed
There is also a member mQueue in the Handler that points to the MessageQueue object reference
class Handler : public MessageHandler { enum : uint32_t { eventMaskInvalidate = 0x1, eventMaskRefresh = 0x2, eventMaskTransaction = 0x4 }; MessageQueue& mQueue; std::atomic<uint32_t> mEventMask; std::atomic<int64_t> mVsyncId; std::atomic<nsecs_t> mExpectedVSyncTime; public: explicit Handler(MessageQueue& queue) : mQueue(queue), mEventMask(0) {} void handleMessage(const Message& message) override; virtual void dispatchRefresh(); virtual void dispatchInvalidate(int64_t vsyncId, nsecs_t expectedVSyncTimestamp); virtual bool invalidatePending(); };
What I mentioned earlier are some definitions and methods. The connection between them and how to operate seem to be still in the clouds. I haven't mentioned it.
In fact, the SurfaceFlinger message queue mainly handles two important events: Refresh and Invalidate
Let's analyze the processing flow of Refresh
When a refresh action is required under some operations, SurfaceFlinger::signalRefresh() will be called
void SurfaceFlinger::signalRefresh() { mRefreshPending = true; mEventQueue->refresh(); }
The SurfaceFlinger::signalRefresh() method will continue to call meventqueue - > refresh()
void MessageQueue::refresh() { mHandler->dispatchRefresh(); }
Then call mhandler - > dispatchrefresh()
void MessageQueue::Handler::dispatchRefresh() { if ((mEventMask.fetch_or(eventMaskRefresh) & eventMaskRefresh) == 0) { mQueue.mLooper->sendMessage(this, Message(MessageQueue::REFRESH)); } }
In the Handler::dispatchRefresh() method, you need to prepare the message and specify the Handler to process the message, and then seedMessage to the Looper's message queue for subsequent processing.
For the refresh message, the what field is MessageQueue::REFRESH. It is processed in the MessageQueue::Handler::handleMessage method, as follows:
void MessageQueue::Handler::handleMessage(const Message& message) { switch (message.what) { case INVALIDATE: mEventMask.fetch_and(~eventMaskInvalidate); mQueue.mFlinger->onMessageReceived(message.what, mVsyncId, mExpectedVSyncTime); break; case REFRESH: mEventMask.fetch_and(~eventMaskRefresh); mQueue.mFlinger->onMessageReceived(message.what, mVsyncId, mExpectedVSyncTime); break; } }
See if it's a little clearer here. When WaitMessage -- > looper:: pollonce detects that this refresh message needs to be processed, it will call back to the MessageQueue::Handler::handleMessage method, and then execute to SurfaceFlinger::onMessageReceived
void SurfaceFlinger::onMessageReceived(int32_t what, int64_t vsyncId, nsecs_t expectedVSyncTime) { switch (what) { case MessageQueue::INVALIDATE: { onMessageInvalidate(vsyncId, expectedVSyncTime); break; } case MessageQueue::REFRESH: { onMessageRefresh(); break; } } }
Then there is the specific work to be completed by refresh. Ten thousand words are omitted here
I don't know if you understand. It's really not systematic without graphical annotations, but it should be easy to understand as long as you understand the Android Native Looper/Handler/Message mechanism.
Simple understanding:
- The main thread of SurfaceFlinger keeps calling WaitMessage -- > looper:: pollonce to detect whether there are messages to be processed;
- If there is a message to be processed, Looper will take out the message and call back the corresponding MessageHandler::handleMessage method to process the message event;
- External users can send messages to be processed to the Message queue through MessageQueue::postMessage, Looper::sendMessage and other methods;
Practical skills
There are two unique skills in the Wulin, one Yang finger and the lion roar skill. It took me 30 years to combine the two unique skills into one. Come here!
One Yang finger and lion roar before Android 11:
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.h // post an asynchronous message to the main thread status_t postMessageAsync(const sp<MessageBase>& msg, nsecs_t reltime = 0, uint32_t flags = 0); // post a synchronous message to the main thread status_t postMessageSync(const sp<MessageBase>& msg, nsecs_t reltime = 0, uint32_t flags = 0);
After Android 11, two unique moves have become a whole move
* /frameworks/native/services/surfaceflinger/SurfaceFlinger.h // Schedule an asynchronous or synchronous task on the main thread. template <typename F, typename T = std::invoke_result_t<F>> [[nodiscard]] std::future<T> schedule(F&&);
The function of these methods is to put the specified task into the main thread for execution.
Take a look at the definition of schedule:
template <typename F, typename T> inline std::future<T> SurfaceFlinger::schedule(F&& f) { auto [task, future] = makeTask(std::move(f)); mEventQueue->postMessage(std::move(task)); return std::move(future); }
Let's look at the definitions of makeTask and Task:
* /frameworks/native/services/surfaceflinger/Scheduler/MessageQueue.h template <typename F> class Task : public MessageHandler { template <typename G> friend auto makeTask(G&&); explicit Task(F&& f) : mTask(std::move(f)) {} void handleMessage(const Message&) override { mTask(); } using T = std::invoke_result_t<F>; std::packaged_task<T()> mTask; }; template <typename F> inline auto makeTask(F&& f) { sp<Task<F>> task = new Task<F>(std::move(f)); return std::make_pair(task, task->mTask.get_future()); }
MessageQueue::postMessage
* /frameworks/native/services/surfaceflinger/Scheduler/MessageQueue.cpp void MessageQueue::postMessage(sp<MessageHandler>&& handler) { mLooper->sendMessage(handler, Message()); }
It is basically clear that the familiar words: postMessage, MessageHandler and handleMessage are essentially message queue processing on main thread.
In fact, changing the soup without changing the medicine is only abstracted and encapsulated by using the new feature of C + +. In essence, it still sends a Message to the main thread and specifies its MessageHandler. When the main thread receives the Message, it calls MessageHandler::handleMessage, and then calls the function entity code specified by us.
We have an example to illustrate:
static_cast<void>(schedule([=]() { ALOGD("I was printed on the main thread"); }));
The function entity we pass to schedule will be executed in the main thread.
This may be a good technique when we modify some logic and want to ensure that the main thread executes.
4 Summary
Here, the SurfaceFlinger startup and initialization process and the SurfaceFlinger message queue processing mechanism are over, and I haven't fully understood many contents. The purpose of this article is to focus on explaining the process and ignore the specific details, understand the process, and then analyze the details according to work needs, which will make me feel more pragmatic.
Required reading:
Android 12(S) graphics display system - Opening
Finally, share a good mood