The previous article has analyzed the Executor framework and members of its family in detail, paving the way for the introduction of this article, so analyzing the core implementation principles of ThreadPoolExecutor is a call to action. Go straight to the topic!

Start with the construction method of ThreadPoolExecutor.

Construction method of ThreadPoolExecutor

    public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue) {
        this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
             Executors.defaultThreadFactory(), defaultHandler);
    }
    public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue,
                              ThreadFactory threadFactory) {
        this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
             threadFactory, defaultHandler);
    }
    public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue,
                              RejectedExecutionHandler handler) {
        this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
             Executors.defaultThreadFactory(), handler);
    }
    public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue,
                              ThreadFactory threadFactory,
                              RejectedExecutionHandler handler) {
        if (corePoolSize < 0 ||
            maximumPoolSize <= 0 ||
            maximumPoolSize < corePoolSize ||
            keepAliveTime < 0)
            throw new IllegalArgumentException();
        if (workQueue == null || threadFactory == null || handler == null)
            throw new NullPointerException();
        this.corePoolSize = corePoolSize;
        this.maximumPoolSize = maximumPoolSize;
        this.workQueue = workQueue;
        this.keepAliveTime = unit.toNanos(keepAliveTime);
        this.threadFactory = threadFactory;
        this.handler = handler;
    }

The meaning of each parameter in the constructor:

CorePoolSize: The size of the core pool. After creating the thread pool, by default, there are no threads in the thread pool. Instead, they wait for tasks to arrive before creating a thread to execute the task. When the number of threads in the thread pool reaches corePoolSize, the arrived tasks are placed in the cache queue.Threads larger than this number will only be created when the work queue is full.If a thread is idle longer than its lifetime, it is marked as recyclable and terminated only if the current size of the thread pool exceeds corePoolSize.Users can call the prestartAllCoreThreads() or prestartCoreThread() methods to pre-create threads, that is, to create corePoolSize threads or a thread before a task arrives.

maximumPoolSize: The maximum number of threads in the thread pool, which is also an important parameter to indicate the maximum number of threads that can be created in the thread pool; greater than this value will cause Thread to be handled by a discard processing mechanism.

KeepAliveTime: Indicates how long a thread can last without a task being executed.By default, keepAliveTime only works if the number of threads in the thread pool is greater than corePoolSize, until the number of threads in the thread pool is not greater than corePoolSize, that is, when the number of threads in the thread pool is greater than corePoolSize, if a thread reaches keepAliveTime idle, it terminates until the thread poolThe number of threads in does not exceed corePoolSize.However, if the allowCoreThreadTimeOut(boolean) method is called, the keepAliveTime parameter also works when the number of threads in the thread pool is not greater than the number of corePoolSize, until the number of threads in the thread pool is 0;

Unit: The time unit of the keepAliveTime parameter has seven values and seven static properties in the TimeUnit class.

workQueue: A blocked queue used to store tasks waiting to be executed. When the number of threads in the thread pool reaches corePoolSize, the arrived tasks are placed in the cache queue.

threadFactory: Thread factory, used primarily to create threads;

handler: Indicates the policy for rejecting processing tasks, that is, the method for discarding processing when the parameter maximumPoolSize is reached.There are four values:

ThreadPoolExecutor.AbortPolicy: Discards the task and throws a RejectedExecutionException exception. 
ThreadPoolExecutor.DiscardPolicy: Tasks are also discarded, but no exceptions are thrown. 
ThreadPoolExecutor.DiscardOldestPolicy: Discard the top task in the queue and try to execute it again (repeat the process)
ThreadPoolExecutor.CallerRunsPolicy: This task is handled by the calling thread

Users can also implement the interface RejectedExecutionHandler to customize their own policies.

Here's an in-depth look at how thread pools work:

Thread pool state

In JDK1.7, the atomic variable ctl is used to control the state of the thread pool, where ctl wraps the following two field s:

workerCount: Indicates the number of valid threads
runState: Indicates the state of the thread pool, whether running, shutting down, etc.

Since workerCount and runState are stored in an int, workerCount is limited to (2 ^ 29) - 1 (approximately 500 million) threads.It uses the shift / mask constant to calculate the values of workerCount and runState.The source code is as follows:

    private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
    private static final int COUNT_BITS = Integer.SIZE - 3;
    private static final int CAPACITY   = (1 << COUNT_BITS) - 1;

    // runState is stored in the high-order bits
    private static final int RUNNING    = -1 << COUNT_BITS;
    private static final int SHUTDOWN   =  0 << COUNT_BITS;
    private static final int STOP       =  1 << COUNT_BITS;
    private static final int TIDYING    =  2 << COUNT_BITS;
    private static final int TERMINATED =  3 << COUNT_BITS;

    // Packing and unpacking ctl
    private static int runStateOf(int c)     { return c & ~CAPACITY; }
    private static int workerCountOf(int c)  { return c & CAPACITY; }
    private static int ctlOf(int rs, int wc) { return rs | wc; }

WorerCount is the number of worker threads that have been started but not stopped.

runState controls the life cycle state of a thread pool, mainly including the following values:

Transition relationships between states:

RUNNING -> SHUTDOWN:shutdown() called

(RUNNING or SHUTDOWN) -> STOP:shutdownNow() called

SHUTDOWN -> TIDYING: Both queue and pool are empty

STOP -> TIDYING: Pool empty

TIDYING -> TERMINATED: The hook method terminated() has been completed.

When the thread pool state is TERMINATED, the thread calling awaitTermination() returns from the wait.

Worker

Considering that adding Worker source analysis to this article would make the article too long to read, see the core implementation of Worker and source analysis of runWorker https://my.oschina.net/7001/blog/889770.

addWorker

AddiWorker is used to add and start worker threads, starting with its flowchart:

The source code is parsed as follows:

    private boolean addWorker(Runnable firstTask, boolean core) {
        retry:
        for (;;) {
            int c = ctl.get();
            int rs = runStateOf(c);

            /** It is possible to return false here:
              * 1 Thread pool state greater than SHUTDOWN
              * 2 Thread pool state is SHUTDOWN, but firstTask is not empty, that is, thread pool has SHUTDOWN and refuses to add new tasks
              * 3 Thread pool state is SHUTDOWN and firstTask is empty, but workQueue is empty, meaning no tasks need to be executed
              */
            if (rs >= SHUTDOWN &&
                ! (rs == SHUTDOWN &&
                   firstTask == null &&
                   ! workQueue.isEmpty()))
                return false;

            for (;;) {
                int wc = workerCountOf(c);
                /** Returning false has the following possibilities:
                  * 1 Number of worker threads exceeds maximum capacity
                  * 2 core To true, the number of worker threads exceeds the boundaries of corePoolSize
                  * 3 core For false, the number of worker threads exceeds the boundary maximumPoolSize
                  */
                if (wc >= CAPACITY ||
                    wc >= (core ? corePoolSize : maximumPoolSize))
                    return false;
                if (compareAndIncrementWorkerCount(c))
                    break retry;//Jump out of outermost loop directly
                c = ctl.get();  // Re-read ctl
                if (runStateOf(c) != rs)//Thread pool state changes to restart from outermost loop
                    continue retry;
                // else CAS failed due to workerCount change; retry inner loop
            }
        }

        Worker w = new Worker(firstTask);
        Thread t = w.thread;

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            // After holding a lock, the thread pool state needs to be re-checked to prevent ThreadFactory from returning a failure or the thread pool from being closed before being locked
            int c = ctl.get();
            int rs = runStateOf(c);
             /** Returning false has the following possibilities:
               * 1 t For null, ThreadFactory failed to create a thread, possibly OutOfMemoryError
               * 2 Thread pool state greater than SHUTDOWN
               * 3 Thread pool state is SHUTDOWN, but firstTask is not empty
               */
            if (t == null ||
                (rs >= SHUTDOWN &&
                 ! (rs == SHUTDOWN &&
                    firstTask == null))) {
                decrementWorkerCount();
                tryTerminate();
                return false;
            }

            workers.add(w);

            int s = workers.size();
            if (s > largestPoolSize)
                largestPoolSize = s;
        } finally {
            mainLock.unlock();
        }

        t.start();
        // Threads may have been added to workers but have not yet been started during the transition from thread pool to stop (this is unlikely, this may be
        // Causes a rare lost interrupt because Thread.interrupt does not guarantee validity for non-startup threads
        if (runStateOf(ctl.get()) == STOP && ! t.isInterrupted())
            t.interrupt();

        return true;
    }

The addWorker first checks the current thread pool state and the given boundaries to create a new worker, during which the number of workers will be adjusted appropriately; if the condition is met, a new worker will be created and started, with the first Task in the parameter as the first task of the worker.

Task Execution

In the ThreadPoolExecutor class, the core task submission method is the execute() method. Submitting tasks using submit() ultimately calls the execute() method. First, look at the flowchart:

The source code is as follows:

    public void execute(Runnable command) {
        if (command == null)
            throw new NullPointerException();
        /*
         * Processing in 3 steps:
         *
         * 1. Current number of worker threads < corePoolSize, create new worker threads directly to execute tasks (call addWorker)
         *
         * 2. The current number of worker threads >=corePoolSize, the thread pool status is RUNNING, and the task joined the work queue successfully.
         * Check again if the current state of the thread pool is RUNNING, if not, remove the task from the queue, or reject the task if the removal succeeds
         * If RUNNING, determine if the current number of worker threads is 0, if 0, add a worker thread
         *
         * 3. Thread pool state is not RUNNING or task queuing failure. An attempt to open a normal thread to perform a task rejects the task if it fails
         */
        int c = ctl.get();
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        else if (!addWorker(command, false))
            reject(command);
    }

From the above analysis, you can summarize the four stages in which a thread pool runs a task:

1. When poolSize < corePoolSize and the queue is empty, a new thread is created to process the submitted task
2. poolSize == corePoolSize, the submitted task enters the work queue, and the worker thread gets the task execution from the queue, where the queue is not empty and not full.
3. poolSize == corePoolSize, and when the work queue is full, a new thread is created to process the submitted task, but poolSize < maxPoolSize
4. poolSize == maxPoolSize, and the queue is full, this triggers the rejection policy.

When checking again that the current state of the thread pool is not RUNNING, not only will the task be removed from the task queue, but the thread pool will also be attempted to terminate.

    public boolean remove(Runnable task) {
        boolean removed = workQueue.remove(task);
        tryTerminate(); // In case SHUTDOWN and now empty
        return removed;
    }

Attempt to terminate thread pool

tryTerminate is called in many places, so what does this do?

Scenario analysis: When the shutDown() method of the thread pool is invoked, interruptIdleWorkers are invoked to attempt to interrupt the worker thread, which only has the opportunity to be interrupted during getTask ().Assuming the interrupt state of multiple threads successfully set by interruptIdleWorkers, if the task queue is not empty at this time, getTask() will get the task successfully from the task queue because the thread pool state is SHUTDOWN; when runWorker executes a task, the thread pool state is SHUTDOWN (less than STOP), then the current worker thread is interruptedThe status will be cleared.When the interrupt state is cleared, tasks taken from the work queue will not respond to the interrupt until the work queue is empty, at which point worker threads that have successfully set the interrupt state may be blocked in workQueue.take(), because the thread pool in SHUTDOWN state will not receive new tasks, worker threads will always be blocked and will never be blockedWill exit.What shall I do?tryTerminate will come in handy, and Doug Lea cleverly places tryTerminated() logic to try to terminate the thread pool in all places that could cause it to terminate. tryTerminated terminates the idle thread until there are no idle threads, and then terminates the thread pool.

Here's a look at the implementation of tryTerminated:

    final void tryTerminate() {
        for (;;) {
            int c = ctl.get();
            //From previous state transitions, RUNNING cannot jump directly to TERMINATED, so it returns
            //The state is already TERMINATED and terminated() no longer needs to be called to return
            //The status is SHUTDOWN and the queue is not empty, the tasks in the queue still need to be processed, terminated() cannot be called to return
            if (isRunning(c) ||
                runStateAtLeast(c, TIDYING) ||
                (runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
                return;
            if (workerCountOf(c) != 0) { // Meet Termination Conditions
                interruptIdleWorkers(ONLY_ONE);//Interrupt only one thread at a time
                return;
            }

            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                    try {
                        terminated();
                    } finally {
                        ctl.set(ctlOf(TERMINATED, 0));//Set state to TERMINATED and workerCount to 0
                        termination.signalAll();
                    }
                    return;
                }
            } finally {
                mainLock.unlock();
            }
            // else retry on failed CAS
        }
    }
    //Interrupt idle threads
    private void interruptIdleWorkers(boolean onlyOne) {
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            for (Worker w : workers) {
                Thread t = w.thread;
                if (!t.isInterrupted() && w.tryLock()) {
                    try {
                        t.interrupt();
                    } catch (SecurityException ignore) {
                    } finally {
                        w.unlock();
                    }
                }
                if (onlyOne)
                    break;
            }
        } finally {
            mainLock.unlock();
        }
    }

From the source code, the following changes the thread pool to TERMINATED termination state:

1 Thread pool status is SHUTDOWN, number of worker threads in thread pool is 0, worker queue is empty

2. The thread pool state is STOP and the number of worker threads in the thread pool is 0

Close Thread Pool

You can use shutdown() and shutdownNow() to close the thread pool, but the effect and implementation are different; you can also call awaitTermination(long timeout, TimeUnit unit) to wait for the thread pool to terminate.Understanding the logic of closing thread pools may require reference to the article https://my.oschina.net/7001/blog/889770 The runWorker and getTask() logic described in this section.

shutdown()

When shutdown closes the thread pool, previously submitted tasks are executed, but new tasks are rejected.Shutdown does not wait for a previously submitted task to finish executing, in which case awaitTermination() can be used.The source code is as follows:

    public void shutdown() {
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            checkShutdownAccess();//Permission Check
            advanceRunState(SHUTDOWN);
            interruptIdleWorkers();//Interrupt all idle threads
            onShutdown(); // hook for ScheduledThreadPoolExecutor
        } finally {
            mainLock.unlock();
        }
        tryTerminate();
    }
    //Update thread pool status to SHUTDOWN, complete with spin guarantee
    private void advanceRunState(int targetState) {
        for (;;) {
            int c = ctl.get();
            if (runStateAtLeast(c, targetState) ||
                ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
                break;
        }
    }

Idle threads have been mentioned more than once in this article, so what is an idle thread in the thread pool?

Idle worker: Getting worker for task from workQueue blocking queue;

Running worker: A worker that is using runWorker to perform tasks.

The worker blocking the getTask() acquisition task throws an InterruptedException after it is interrupted, and the acquisition task is no longer blocked.Continue into the spin operation, when the thread pool is shutdown and workQueue.isEmpty(), getTask() returns null and enters the worker exit logic.

shutdownNow()

shutdownNow means closing the thread pool immediately, it will try to stop all active executing tasks and stop processing tasks in the task queue, it will return the list of tasks waiting to be executed.shutdownNow tries to stop running tasks without task guarantees.Canceling a task is accomplished by issuing an interrupt signal from Thread.interrupt().As the runWorker source code knows, tasks that have entered the locked area do not respond to interrupts, so only when the worker thread finishes executing the current task, does it become aware that the thread pool state is STOP and begin processing the exit logic.

shutdownNow immediately signals all threads to interrupt in order to prevent taking tasks from the task queue and allow them to enter exit logic as soon as possible; threads executing code in the runWorker lock area will detect the state of the thread pool and enter exit logic as soon as the current task is completed.The source code is as follows:

    public List<Runnable> shutdownNow() {
        List<Runnable> tasks;
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            checkShutdownAccess();
            advanceRunState(STOP);//Modify thread pool state to STOP
            interruptWorkers();//Interrupt all worker threads
            tasks = drainQueue();
        } finally {
            mainLock.unlock();
        }
        tryTerminate();
        return tasks;
    }
    /**
      * Use drainTo method to add tasks from the work queue to the taskList at one time and remove them from the work queue;
      * If the queue is DelayQueue or any other type of queue, poll or drainTo may not be able to delete certain elements, they will be deleted one by one
      */
    private List<Runnable> drainQueue() {
        BlockingQueue<Runnable> q = workQueue;
        List<Runnable> taskList = new ArrayList<Runnable>();
        q.drainTo(taskList);
        if (!q.isEmpty()) {
            for (Runnable r : q.toArray(new Runnable[0])) {
                if (q.remove(r))
                    taskList.add(r);
            }
        }
        return taskList;
    }

awaitTermination

awaitTermination() loops whether the thread pool is terminated or has exceeded the timeout, and each time a condition is not satisfied, the specified time is blocked using the Condition object termination.termination.awaitNanos() is a blocking wait implemented through LockSupport.parkNanos(this, nanosTimeout).After shutdown is called, awaitTermination() is blocked until:

1 All tasks completed normally, thread pool changed to TERMINATED

2 Task still incomplete, arrival timeout

3. The current thread is interrupted

The following specific conditions occur during the blocking wait process to unblock the blocking:

1 The task completes normally, the thread pool changes to TERMINATED normally, and termination.signalAll() is called to wake up all blocked waiting threads

2. Arrival timeout, nanos <= 0 condition succeeds, returns false

3. The current thread is interrupted, termination.awaitNanos() will wake up from the blocking and throw an InterruptException exception up.

The source code is as follows:

   public boolean awaitTermination(long timeout, TimeUnit unit)
        throws InterruptedException {
        long nanos = unit.toNanos(timeout);//Slice timeout
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            for (;;) {
                if (runStateAtLeast(ctl.get(), TERMINATED))//Status >=TERMINATED
                    return true;
                if (nanos <= 0)//Timeout reached
                    return false;
                nanos = termination.awaitNanos(nanos);
            }
        } finally {
            mainLock.unlock();
        }
    }

Several common thread pools

Typically, we use Executors'static factory method to create thread pools. Here are a few common ways to create thread pools:

    /** newFixedThreadPool A fixed-length thread pool will be created, creating a thread whenever a task is submitted.
      * Until the maximum number of thread pools is reached, the size of the thread pool does not change
      * (If a thread terminates due to an unexpected Exception, the thread pool will be supplemented with a new thread)
      */
    public static ExecutorService newFixedThreadPool(int nThreads) {
        return new ThreadPoolExecutor(nThreads, nThreads,
                                      0L, TimeUnit.MILLISECONDS,
                                      new LinkedBlockingQueue<Runnable>());
    }
    /** newCachedThreadPool A cacheable thread pool will be created if the current size of the thread pool exceeds processing requirements.
      * Then idle threads will be recycled, and when demand increases, another thread can be created without any restrictions on the size of the thread pool
      */
    public static ExecutorService newCachedThreadPool() {
        return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                      60L, TimeUnit.SECONDS,
                                      new SynchronousQueue<Runnable>());
    }
    /** newSingleThreadExecutor Is a single-threaded Executor that creates only one thread to perform tasks.
      * If the thread ends abnormally, a new thread is created to replace it.
      * newSingleThreadExecutor Ensure that tasks are executed serially in the order in which they are queued (e.g. FIFO, LIFO, priority)
      */
    public static ExecutorService newSingleThreadExecutor() {
        return new FinalizableDelegatedExecutorService
            (new ThreadPoolExecutor(1, 1,
                                    0L, TimeUnit.MILLISECONDS,
                                    new LinkedBlockingQueue<Runnable>()));
    }
    /** 
      * newScheduledThreadPool Create a fixed-length thread pool and perform tasks in a delayed or timed manner
      */
    public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
        return new ScheduledThreadPoolExecutor(corePoolSize);
    }

Welcome to point out the errors in this article, please indicate the source of the original text for reprinting https://my.oschina.net/7001/blog/889931