Thread pool ThreadPoolExecutor

introduce

Thread pool mainly solves two problems: one is that thread pool can provide better performance when executing a large number of asynchronous tasks. When the thread pool is not used, a new thread is required to execute whenever a task needs to be executed. Frequent creation and destruction consume performance. The threads in the thread pool can be reused and do not need to be recreated and destroyed every time a task needs to be executed. Second, the thread pool provides a means of resource restriction and management, such as limiting the number of threads, dynamically increasing threads, etc.

In addition, the thread pool also provides many adjustable parameters and extensible interfaces to meet the needs of different situations. We can use the more convenient factory method of Executors to create different types of thread pools, or customize the thread pool ourselves.

Working mechanism of thread pool

1. When the thread pool is first created, there is no thread. When a new request comes, a core thread will be created to process the corresponding request
2. When processing is complete, the core thread does not recycle
3. Before the core thread reaches the specified number, each request will create a new core thread in the thread pool
4. When all the core threads are occupied, the new request will be put into the work queue. A work queue is essentially a blocking queue
5. When the work queue is full, new requests will be handed over to the temporary thread for processing
6. The temporary thread will continue to survive for a period of time after use, and will not be destroyed until there is no request processing

Class diagram introduction

As shown in the above class diagram, Executors is a tool class that provides a variety of static methods and provides different thread pool instances according to our choices.

ThreadPoolExecutor inherits the AbstractExecutorService abstract class. In ThreadPoolExecutor, the member variable ctl is an atomic variable of Integer, which is used to record the status of the thread pool and the number of threads in the thread. It is similar to ReentrantReadWriteLock using a variable to save the two kinds of information.

Assuming that the Integer type is a 32-bit binary representation, the upper 3 bits represent the status of the thread pool, and the last 29 represents the number of threads in the thread pool.

//The default RUNNING state is 0
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));

Get the upper 3 bits and the running status

private static int runStateOf(int c)     { return c & ~CAPACITY; }

Get the lower 29 bits and the number of threads

private static int workerCountOf(int c)  { return c & CAPACITY; }

The meaning of thread pool status is as follows:

• RUNNING: accept new tasks and process tasks in the blocking queue
• SHUTDOWN: reject new tasks but process tasks in the blocking queue
• STOP: reject the new task and discard the task in the blocking queue, while interrupting the task being processed
• TIDYING: after all tasks are executed (including tasks in the blocking queue), when the number of active threads in the frontline process pool is 0, the terminated method will be called
• terminated: terminated status. terminated the state after the method is called

The state transition of thread pool is as follows:

• Running - > shutdown: explicitly call the shutdown method, or implicitly call the shutdown method in the finalize method
• RUNNING or shutdown - > stop: when the shutdown now method is explicitly called
• Shutdown - > tidying: when the thread pool and task queue are empty
• Stop - > tidying: when the thread pool is empty
• When executing the method named - > named

Thread pool parameters are as follows:

• corePoolSize: refers to the number of core threads. After the initialization of the thread pool is completed, by default, there are no threads in the thread pool. The thread pool will wait for the arrival of the task, and then create a new thread to execute the task.

• maxPoolSize: the thread pool may add some additional threads to the number of core threads, but these newly added threads have an upper limit, which cannot exceed maxPoolSize.

  • If the number of threads is less than corePoolSize, a new thread will be created to run the task even if other worker threads are idle.
  • If the number of threads is greater than or equal to corePoolSize but less than maxPoolSize, the task is put into the work queue.
  • If the queue is full and the number of threads is less than maxPoolSize, a new thread is created to run the task.
  • If the queue is full and the number of threads is greater than or equal to maxPoolSize, the reject policy is used to reject the task.
• keepAliveTime: if a thread is idle and the current number of threads is greater than corePoolSize, the idle thread will be destroyed after the specified time. The specified time here is set by keepAliveTime.

• workQueue: after a new task is submitted, it will enter this work queue first, and then take out the task from the queue when scheduling the task. There are four types of work queues available in the jdk:

  • ArrayBlockingQueue: array based bounded blocking queue, sorted by FIFO. When a new task comes in, it will be placed at the end of the queue. A bounded array can prevent resource depletion. When the number of threads in the thread pool reaches the corePoolSize and a new task comes in, the task will be placed at the end of the queue and waiting to be scheduled. If the queue is full, a new thread will be created. If the number of threads has reached maxPoolSize, the reject policy will be executed.
  • LinkedBlockingQueue: unbounded blocking queue based on linked list (in fact, the maximum capacity is Interger.MAX), sorted according to FIFO. Due to the approximate boundlessness of the queue, when the number of threads in the thread pool reaches corePoolSize, new tasks will come in and will be stored in the queue instead of creating new threads until maxPoolSize. Therefore, when using the work queue, the parameter maxPoolSize actually does not work.
  • Synchronous queue: a blocked queue that does not cache tasks. The producer puts in a task and must wait until the consumer takes out the task. That is, when a new task comes in, it will not be cached, but will be directly scheduled to execute the task. If there are no available threads, a new thread will be created. If the number of threads reaches maxPoolSize, the rejection policy will be executed.
  • PriorityBlockingQueue: an unbounded blocking queue with priority. The priority is realized through the parameter Comparator.
  • delayQueue: delay unbounded blocking queue with priority.
  • LinkedTransferQueue: unbounded blocking queue based on linked list.
  • LinkedBlockingDeque: double ended blocking queue based on linked list.
• threadFactory: the factory used when creating a new thread. It can be used to set the thread name, whether it is a daemon thread, and so on
• handler: when the task in the work queue has reached the maximum limit and the number of threads in the thread pool has reached the maximum limit, if a new task is submitted, the rejection policy will be executed.

As shown in the ThreadPoolExecutor class diagram above, mainLock is an exclusive lock used to control the atomicity of new Worker thread operations. termination is the condition queue corresponding to the lock. It is used to store blocked threads when a thread calls awaitTermination.

private final ReentrantLock mainLock = new ReentrantLock();
private final HashSet<Worker> workers = new HashSet<Worker>();
private final Condition termination = mainLock.newCondition();

The worker class inherits the AQS and Runnable interfaces and is an object that specifically carries tasks. Worker inherits AQS and implements a simple non reentrant exclusive lock. state = 0 indicates that the lock has not been acquired, state = 1 indicates that the lock has been acquired, state = -1 is the default state when creating worker, and the state is set to - 1 when creating to avoid the thread being interrupted before running runWorker method. The variable firstTask records the first task executed by the worker thread, and thread is the thread that specifically executes the task.

private final class Worker extends AbstractQueuedSynchronizer implements Runnable{
    final Thread thread;
    Runnable firstTask;
    //...

DefaultThreadFactory is a thread factory, and the newThread method is a modification of the thread. poolNumber is a static atomic variable, which is used to count the number of thread factories. threadNumber is used to record how many threads are created in each thread factory. These two values are also part of the thread pool and thread name.

Source code analysis

execute method

The main function of the execute method is to submit the task command to the thread pool for execution.

As can be seen from the figure, the implementation of ThreadPoolExecutor is actually a producer consumer model. When a user adds a task to the thread pool, it is equivalent to the producer production element, and the thread in the workers thread directly executes the task or obtains the task from the task queue, it is equivalent to the consumer consumption element.

The specific codes are as follows:

public void execute(Runnable command) {
        //1. Check whether the task is null
        if (command == null)
            throw new NullPointerException();
        //2. Get the combined value of current thread pool status + number of threads
        int c = ctl.get();
        //3. Judge whether the number of threads in the thread pool is less than the corePoolSize. If it is small, start a new thread (core thread) to run
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        //4. If the thread pool is in the RUNNING state, add the task to the blocking queue
        if (isRunning(c) && workQueue.offer(command)) {
            //4.1 secondary inspection
            int recheck = ctl.get();
            //4.2 if the current thread pool state is not RUNNING, delete the task from the queue and execute the reject policy
            if (! isRunning(recheck) && remove(command))
                reject(command);
            //4.3 if the current thread pool is empty, add a thread
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        //5. If the queue is full, a new thread will be added. If the addition fails, the reject policy will be executed
        else if (!addWorker(command, false))
            reject(command);
    }

If the number of threads in the current thread pool is greater than or equal to corePoolSize, execute code (4). If the current thread is RUNNING, it will be added to the task queue.

It should be noted that the thread pool status is judged here because it is possible that the thread pool is already in a non RUNNING state, and new tasks will be abandoned in a non RUNNING state.

If the task is added successfully, execute the code (4.2) for secondary verification, because the status of the thread pool may be changed before executing 4.2. If the status of the thread pool is not RUNNING, remove the task from the task queue, and then execute the rejection policy; If the secondary verification passes, it will be judged again whether there are threads in the thread pool. If not, a new thread will be added.

If code (4) fails to add a task, it indicates that the queue is full. Then execute code (5) to try to add new threads, that is, thread3 and thread4 threads in the above figure to execute the task. If the current number of thread pools > maximumpoolsize, execute the rejection policy.

Let's next look at the addWorker method:

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
   // Each for loop needs to get the latest ctl value
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
                //1. Check whether the queue is empty only when necessary
        if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN &&
               firstTask == null &&
               ! workQueue.isEmpty()))
            return false;
              //2. Increase the number of threads in cyclic CAS
        for (;;) {
            int wc = workerCountOf(c);
            //2.1 if the number of threads exceeds the limit, false will be returned
            if (wc >= CAPACITY ||
                wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            //2.2 increase the number of threads through CAS
            if (compareAndIncrementWorkerCount(c))
                break retry;
            //2.3 after CAS fails, check whether the thread state has changed. If so, jump to the outer loop and try to obtain the thread pool state again, otherwise the inner loop will restart CAS
            c = ctl.get();
            if (runStateOf(c) != rs)
                continue retry;
        }
    }

      //3. This step indicates the success of CAS
    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        //3.1 create Worker
        w = new Worker(firstTask);
        final Thread t = w.thread;
        if (t != null) {
            //3.2 add exclusive lock to realize synchronization, because multiple threads may call the execute method of thread pool at the same time
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                //3.3 recheck the thread pool state and avoid calling shutdown before getting the lock.
                int rs = runStateOf(ctl.get());
                if (rs < SHUTDOWN ||
                    (rs == SHUTDOWN && firstTask == null)) {
                    if (t.isAlive()) // precheck that t is startable
                        throw new IllegalThreadStateException();
                    //3.4 adding tasks
                    workers.add(w);
                  // Update the current maximum number of threads. maximumPoolSize and corePoolSize can be dynamically modified after the thread pool is created
                    int s = workers.size();
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                    workerAdded = true;
                }
            } finally {
                mainLock.unlock();
            }
            //3.5 start task after adding successfully
            if (workerAdded) {
                t.start();
                workerStarted = true;
            }
        }
    } finally {
       // If t.start() has not been executed, the woker should be deleted from the workers and the number of workers in ctl should be reduced by 1
        if (! workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

Firstly, it is clear that the purpose of the first part of double loop is to add the number of threads through CAS operation. In the second part, the task is added to workers safely through ReentrantLock, and then the task is started.

First look at the first part of the code (1).

if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN && //(1)
               firstTask == null && //(2)
               ! workQueue.isEmpty())) //(3)

Code (1) returns false in the following three cases:

• (1) The current thread pool status is STOP, TIDYING, or TERMINATED
• (2) The current thread pool status is SHUTDOWN and the first task has been created
• (3) The current thread pool state is SHUTDOWN and the task queue is empty

The function of inner loop in code (2) is to use CAS operation to increase the number of threads.

When code (8) is executed, it indicates that the number of threads has been successfully increased through CAS. Now the task has not started to execute, so this part of code adds workers to the work set workers through global lock control.

Execution of Worker thread

After the user thread is submitted to the thread pool, it is executed by the Worker. The following is the constructor of the Worker.

Worker(Runnable firstTask) {
    setState(-1); // inhibit interrupts until runWorker
    this.firstTask = firstTask;
    this.thread = getThreadFactory().newThread(this);//Create a thread
}

First set the Worker status to - 1 in the constructor to avoid the current Worker being interrupted before calling the runWorker method (when other threads call the shutdown now of the thread pool, if the Worker status > = 0, the thread will be interrupted). Here, the status of the thread is set to - 1, so the thread will not be interrupted.

In runWorker code, the unlock method will be called when running code (1), which sets the status to 0, so calling shutdown now will interrupt the Worker thread.

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); //1. Set state to 0 to allow interruption
    boolean completedAbruptly = true;
    try {
        //2.
        while (task != null || (task = getTask()) != null) {
            //2.1
            w.lock();
            ////If the status value is greater than or equal to STOP and the current thread has not been interrupted, the thread will be actively interrupted
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() &&
                  runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                //2.2 perform pre task processing. It is an empty implementation by default; In subclasses, the processing behavior before task execution can be changed by rewriting
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    //2.3 execution of tasks
                    task.run();
                } catch (RuntimeException x) {
                    thrown = x; throw x;
                } catch (Error x) {
                    thrown = x; throw x;
                } catch (Throwable x) {
                    thrown = x; throw new Error(x);
                } finally {
                    //2.4 post task processing is the same as beforeExecute
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                //2.5 count the number of tasks completed by the current worker
                w.completedTasks++;
                w.unlock();
            }
        }
        //Set to false, indicating that the task is completed normally
        completedAbruptly = false;
    } finally {
        //3. Clean up
        processWorkerExit(w, completedAbruptly);
    }
}

The lock is added during the execution of specific tasks to avoid interrupting the tasks being executed after other threads call shutdown (shutdown will only interrupt the currently blocked and suspended threads)

The cleaning work code is as follows:

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    // If completedAbruptly is true, an unhandled exception is thrown during task execution
    // Therefore, the worker count has not been reduced correctly. Here, you need to reduce the worker count once
    if (completedAbruptly)
        decrementWorkerCount();

      //1. Count the number of tasks completed in the thread pool and delete the current Worker from the work set
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }
        //When the thread pool is closed, wait until all threads are in the state of at1.2, and then try to close the pool
    tryTerminate();
        //1.3 if the thread pool status is < stop, that is, RUNNING or SHUTDOWN, you need to consider creating a new thread to replace the destroyed thread
    int c = ctl.get();
    if (runStateLessThan(c, STOP)) {
        // If the worker is executed normally, it is necessary to judge whether the minimum number of threads has been met
        // Otherwise, create a substitute thread directly
        if (!completedAbruptly) {
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && ! workQueue.isEmpty())
                min = 1;
            if (workerCountOf(c) >= min)
                return; // replacement not needed
        }
        // Recreate a worker to replace the destroyed thread
        addWorker(null, false);
    }
}

In the above code, the code (1.1) counts the number of tasks completed by the thread pool, and adds a global lock before the statistics. Add the tasks completed in the current working thread to the global counter, and then delete the current Worker from the working set.

The code (1.2) judges that if the current thread pool state is SHUTDOWN and the work queue is empty, or the current thread pool state is STOP and there are no active threads in the current thread pool, set the thread pool state to TERMINATED. If it is set to the TERMINATED state, you also need to call the signalAll method of the condition variable termination to activate all threads blocked by calling the awaitTermination method of the thread pool.

Code (1.3) determines whether the number of threads in the current thread pool is less than the number of core threads. If so, add a new thread.

shutdown method

After the shutdown method is called, the thread pool will not accept new tasks, but the tasks in the work queue still need to be executed. This method will return immediately without waiting for the queue task to complete.

public void shutdown() {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        //1. Permission check
        checkShutdownAccess();
        //2. Set the current thread pool status to SHUTDOWN. If it is already SHUTDOWN, it will be returned directly
        advanceRunState(SHUTDOWN);
        //3. Set interrupt flag
        interruptIdleWorkers();
        onShutdown(); // hook for ScheduledThreadPoolExecutor
    } finally {
        mainLock.unlock();
    }
    //4. Try to change the status to TERMINATED
    tryTerminate();
}

First, see whether the thread currently called has permission to close the thread.

Then the code of code (2) is as follows. If the current thread pool state > = SHUTDOWN, it will be returned directly; otherwise, it will be set to SHUTDOWN state.

private void advanceRunState(int targetState) {
    for (;;) {
        int c = ctl.get();
        if (runStateAtLeast(c, targetState) ||
            ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
            break;
    }
}

The source code of code (3) is as follows, which sets the interrupt flag of all idle threads. First, a global lock is added. At the same time, only one thread can call the shutdown method to set the interrupt flag. Then try to obtain the Worker's own lock. If the lock is obtained successfully, set the interrupt flag. Because the executing task has acquired the lock, the executing task is not interrupted. What is interrupted here is the thread that blocks the getTask method and attempts to get the task from the queue, that is, the idle thread.

private void interruptIdleWorkers(boolean onlyOne) {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        for (Worker w : workers) {
            Thread t = w.thread;
            if (!t.isInterrupted() && w.tryLock()) {
                try {
                    t.interrupt();
                } catch (SecurityException ignore) {
                } finally {
                    w.unlock();
                }
            }
            if (onlyOne)
                break;
        }
    } finally {
        mainLock.unlock();
    }
}

Finally, try to change the status to TERMINATED. First, use CAS to set the current thread pool status to TIDYING. If the setting is successful, execute the extension interface TERMINATED. Do something before the thread pool status changes to TERMINATED, and then set the current thread pool status to TERMINATED. Finally, call termination.. Signalall activates all threads blocked by calling the await series methods of the conditional variable termination.

final void tryTerminate() {
    for (;;) {
        int c = ctl.get();
        if (isRunning(c) ||
            runStateAtLeast(c, TIDYING) ||
            (runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
            return;
        if (workerCountOf(c) != 0) { // Eligible to terminate
            interruptIdleWorkers(ONLY_ONE);
            return;
        }

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try {
                    terminated();
                } finally {
                    ctl.set(ctlOf(TERMINATED, 0));
                    termination.signalAll();
                }
                return;
            }
        } finally {
            mainLock.unlock();
        }
        // else retry on failed CAS
    }
}

Shutdown now method

After calling the shutdown now method, the thread pool will no longer accept new tasks, and will discard the tasks in the work queue. The executing tasks will be interrupted, and the method will return immediately without waiting for the execution of the activated tasks. The return value is the task list discarded in the queue at this time.

public List<Runnable> shutdownNow() {
    List<Runnable> tasks;
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        //1. Permission check
        checkShutdownAccess();
        //2. Set the thread pool status to STOP
        advanceRunState(STOP);
        //3. Interrupt all threads
        interruptWorkers();
        //4. Move the task queue to tasks
        tasks = drainQueue();
    } finally {
        mainLock.unlock();
    }
    tryTerminate();
    return tasks;
}

It should be noted that all threads interrupted include idle threads and threads executing tasks.

awaitTermination method

When a thread calls the awaitTermination method, the current thread will be blocked and will not return until the thread pool state changes to TERMINATED or the waiting time expires.

public boolean awaitTermination(long timeout, TimeUnit unit)
        throws InterruptedException {
        long nanos = unit.toNanos(timeout);
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            for (;;) {
                if (runStateAtLeast(ctl.get(), TERMINATED))
                    return true;
                if (nanos <= 0)
                    return false;
                nanos = termination.awaitNanos(nanos);
            }
        } finally {
            mainLock.unlock();
        }
}

First obtain the exclusive lock, and then judge whether the current thread pool state is at least TERMINATED within the infinite loop. If so, it will be returned directly. Otherwise, it means that there are still threads executing in the current thread pool. It depends on whether the set timeout nanos is less than 0. If it is less than 0, it means that there is no need to wait, so it will be returned directly, If the waiting time of the calling process variable is greater than the expected time of terminals, the process will change to the waiting time of terminals.

summary

The thread pool skillfully uses an Integer type atomic variable to record the thread pool status and the number of threads in the thread pool. The execution of tasks is controlled by the thread pool state, and each Worker thread can process multiple tasks. Thread pool reduces the overhead of thread creation and destruction through thread reuse.