Distributed scheduling middleware XXL job: Executor executor -- task execution

Posted by neonorange79 on Tue, 08 Feb 2022 12:51:18 +0100

1, Foreword

  in the previous study, we can see that the task execution of the actuator is triggered by the trigger. We will learn how to route triggers and specific scheduling strategies later. The focus of this chapter is to see how actuators are called and perform tasks.

2, Task execution

1. Overview of execution process

   the following figure is a simple sequence diagram of trigger calling actuator to execute tasks:

2. Task receiving

   as can be seen from the above timing diagram, the trigger initiates a task execution request to the actuator through Netty. The specific logic for processing the request is in the process method of the EmbedHttpServerHandler class. Here we mainly look at the logic of the request processing part:

if ("/beat".equals(uri)) {
    return executorBiz.beat();
} else if ("/idleBeat".equals(uri)) {
    IdleBeatParam idleBeatParam = GsonTool.fromJson(requestData, IdleBeatParam.class);
    return executorBiz.idleBeat(idleBeatParam);
} else if ("/run".equals(uri)) {
    TriggerParam triggerParam = GsonTool.fromJson(requestData, TriggerParam.class);
    return executorBiz.run(triggerParam);
} else if ("/kill".equals(uri)) {
    KillParam killParam = GsonTool.fromJson(requestData, KillParam.class);
    return executorBiz.kill(killParam);
} else if ("/log".equals(uri)) {
    LogParam logParam = GsonTool.fromJson(requestData, LogParam.class);
    return executorBiz.log(logParam);
} else {
    return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping("+ uri +") not found.");

   perform heartbeat detection, idle detection, running task, ending task, log and other request processing in the process method. Here, we mainly look at the processing of running task (i.e. "/ run").

   by the way, the trigger initiates a task execution request. The request path is: {actuator embedded service and address} / run. The request method is POST request, and the specific data format is:

    "jobId":1,                                  // Task ID
    "executorHandler":"demoJobHandler",         // Task identification
    "executorParams":"demoJobHandler",          // Task parameters
    "executorBlockStrategy":"COVER_EARLY",      // Task blocking strategy. For optional values, refer to com xxl. job. core. enums. ExecutorBlockStrategyEnum
    "executorTimeout":0,                        // Task timeout, in seconds, takes effect when it is greater than zero
    "logId":1,                                  // Current scheduling log ID
    "logDateTime":1586629003729,                // Current scheduling log time
    "glueType":"BEAN",                          // Task mode, optional values refer to com xxl. job. core. glue. GlueTypeEnum
    "glueSource":"xxx",                         // GLUE script code
    "glueUpdatetime":1586629003727,             // The update time of GLUE script is used to determine whether the script is changed and whether it needs to be refreshed
    "broadcastIndex":0,                         // Slice parameters: current slice
    "broadcastTotal":0                          // Slice parameters: total slice

3. Get jobThread

   you can see that executorbiz. Is executed in the logical branch of "run" run(triggerParam) . The run method can be divided into the following four steps:

  1. Obtain jobThread and jobHandler;
    1. Load the old jobThread and jobHandler according to the jobId in TriggerParam;
    2. Judge whether the jobHandler in the TriggerParam request is the same as the original jobHandler according to the GLUE mode. If it is different, load a new jobHandler and set the jobThread to null;
  2. When the jobThread is not null, obtain the blocking policy from the trigger request TriggerParam and execute it.
  3. After executing the blocking policy, if the jobThread is null, register the new jobHandler in the task thread library to replace the original task thread, that is, call xxljobexecutor Registjobthread method;
  4. Finally, put the TriggerParam request into the TriggerQueue. In this process, check whether the task is repeated through the logId. If the task is repeated, it is better to check and return an error, and then return the queue result.

GLUE is an "executable logic unit" created by big man Xueli. It extends the dynamic language support of the JVM and is essentially an executable code. GLUE can be easily embedded in business code. The logic code in GLUE supports online development, dynamic push update and real-time compilation. In short, it's very cow leather.

4. Task thread registration and startup

  after obtaining the new task, execute xxljobexecutor Register jobthread registers the task thread, starts a new task thread and replaces the original task thread in the task thread library.    here we mainly focus on the JobThread class, which performs specific task execution in the run method of this class. The logic here is relatively easy to understand. You can read the source code for the specific code logic. Let's look at the following two interesting treatments:

  • When judging whether the thread is terminated, a Boolean toStop variable is used separately. This is due to thread Interrupt only supports the blocking state (wait, sleep, join) of terminating threads. InterruptedException will be thrown at the blocking position, but the running thread will not be terminated. Therefore, toStop variable is used here to identify whether the thread is terminated;
  • Because the above uses a loop to determine whether the jobThread terminates, it needs to use poll instead of take when obtaining the task request parameters in the TriggerQueue. This is because the take method will wait until it obtains the available elements in the queue, which will cause thread blocking. The use of poll can effectively limit the number of idle threads and prevent them from crashing;

3, Summary

  this chapter briefly introduces the process of the actuator receiving the request of trigger task execution and executing the task. Since some of the processes are similar to rpc calls, I won't repeat them here. Here I mainly share the basic call processes and some interesting processing ideas of big man Xueli.

The above is purely personal opinions. If there are any mistakes, please correct them.