State definition
Thread pool ThreadPoolExecutor has two properties: running state and number of threads.
The number of threads is easy to understand, that is, how many threads are in the pool; There are five operating states, namely
RUNNING: In operation, Is the initial state, that is, the thread pool just created is in this state. SHUTDOWN: In downtime status, no new tasks will be received, but the tasks in the queue will continue to be processed. STOP: Stop status, no new tasks will be received, no existing tasks will be processed, and the executing tasks will be interrupted. TIDYING: During cleanup, all tasks are stopped and the number of threads is reduced to 0. TERMINATED，Termination status, hook function terminated()The execution is complete and the thread pool is completely destroyed. Copy code
State flow
The flow chart of the five states is as follows
The source code comments are very good. Post them here
* RUNNING: Accept new tasks and process queued tasks * SHUTDOWN: Don't accept new tasks, but process queued tasks * STOP: Don't accept new tasks, don't process queued tasks, * and interrupt in-progress tasks * TIDYING: All tasks have terminated, workerCount is zero, * the thread transitioning to state TIDYING * will run the terminated() hook method * TERMINATED: terminated() has completed * * The numerical order among these values matters, to allow * ordered comparisons. The runState monotonically increases over * time, but need not hit each state. The transitions are: * * RUNNING -> SHUTDOWN * On invocation of shutdown(), perhaps implicitly in finalize() * (RUNNING or SHUTDOWN) -> STOP * On invocation of shutdownNow() * SHUTDOWN -> TIDYING * When both queue and pool are empty * STOP -> TIDYING * When pool is empty * TIDYING -> TERMINATED * When the terminated() hook method has completed Copy code
code implementation
The number of threads is a non negative integer, which can be represented by an int.
There are only five fixed operating states. In order to save space, three bits can be used to represent, because three bits can represent up to eight states.
These two properties of the thread pool need to be accessed at the same time, that is, atomic operations need to be guaranteed. If they are represented separately above, it is estimated that a lock must be added when modifying.
Is there a more elegant implementation? Let's take a look at the implementation of Doug Lea Daniel
/**
* The main pool control state, ctl, is an atomic integer packing
* two conceptual fields
* workerCount, indicating the effective number of threads
* runState, indicating whether running, shutting down etc
*/
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
// Number of bit s representing the number of threads
private static final int COUNT_BITS = Integer.SIZE - 3;
// The maximum number of threads supported by the thread pool (536870911, fully enough)
private static final int CAPACITY = (1 << COUNT_BITS) - 1;
// Runstate is stored in the high order bits
// It can be seen from the annotated figures that the number corresponding to the operation state increases in turn, which is very important to facilitate the subsequent operation state judgment
// 1110 0000 0000 0000 0000 0000 0000 0000(-536870912)
private static final int RUNNING = -1 << COUNT_BITS;
// 0000 0000 0000 0000 0000 0000 0000 0000(0)
private static final int SHUTDOWN = 0 << COUNT_BITS;
// 0010 0000 0000 0000 0000 0000 0000 0000(536870912)
private static final int STOP = 1 << COUNT_BITS;
// 0100 0000 0000 0000 0000 0000 0000 0000(1073741824)
private static final int TIDYING = 2 << COUNT_BITS;
// 0110 0000 0000 0000 0000 0000 0000 0000(1610612736)
private static final int TERMINATED = 3 << COUNT_BITS;
// Packing and unpacking ctl, all atomic operations
// Get running status
private static int runStateOf(int c) { return c & ~CAPACITY; }
// Get the number of threads
private static int workerCountOf(int c) { return c & CAPACITY; }
// The assembly running state and the number of threads become ctl
private static int ctlOf(int rs, int wc) { return rs | wc; }
/*
* Because there are three higher bits of int in the running state, and they increase in turn, the relationship between the two states can be known through size comparison
*/
private static boolean runStateLessThan(int c, int s) {
return c < s;
}
private static boolean runStateAtLeast(int c, int s) {
return c >= s;
}
// Determine whether the thread is running
// Why use < shutdown instead of direct = RUNNING here, because ctl has the number of threads represented by the lower 29 bits
private static boolean isRunning(int c) {
return c < SHUTDOWN;
}
Copy codeDoug Lea skillfully uses bit operation and solves the atomicity problem of operating two attributes with an AtomicInteger; At the same time, putting the status in the top three digits can also easily compare the value of the status of the thread pool.
Doug Lea also gave a very considerate explanation on why to use int instead of long
* In order to pack them into one int, we limit workerCount to * (2^29)-1 (about 500 million) threads rather than (2^31)-1 (2 * billion) otherwise representable. If this is ever an issue in * the future, the variable can be changed to be an AtomicLong, * and the shift/mask constants below adjusted. But until the need * arises, this code is a bit faster and simpler using an int. Copy code
Using 29 bits to represent the number of threads can support 500 million threads. Generally, we will set the thread stack size to 1M through - Xss1024K. If we reach 500 million threads, the ray stack space needs to occupy at least 500T of memory. It's really hard to imagine that we can reach this level one day.
However, Doug Lea carefully considered this possibility. If AtomicInteger is not enough, AtomicLong will be used instead. However, at present, AtomicInteger is faster and easier to use. It is good enough without over design~
State operation
State judgment
public boolean isShutdown() {
return ! isRunning(ctl.get());
}
public boolean isTerminating() {
int c = ctl.get();
return ! isRunning(c) && runStateLessThan(c, TERMINATED);
}
public boolean isTerminated() {
return runStateAtLeast(ctl.get(), TERMINATED);
}
Copy codeWith the above explanation of state definition, it is easy to understand the judgment method of thread pool state here.
isShutdown: as long as the thread pool is not RUNNING, it is closed.
isTerminating: in termination, it is neither RUNNING nor in termination status.
isTerminated: TERMINATED. The top three digits must be 011, so the value of ctl must be greater than or equal to TERMINATED.
Add worker thread addWorker
When a new task arrives, if the number of core threads is not full or the blocking queue is full, a worker thread will be added, that is, the addWorker method will be called. The addWorker method has two input parameters. The first is the Runnable task, that is, the latest task, but the parameter may be empty; The second parameter indicates whether the added is a core thread.
private boolean addWorker(Runnable firstTask, boolean core) {
retry:
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// step 1
if (rs >= SHUTDOWN &&
! (rs == SHUTDOWN &&
firstTask == null &&
! workQueue.isEmpty()))
return false;
for (;;) {
int wc = workerCountOf(c);
// step 2.1
if (wc >= CAPACITY ||
wc >= (core ? corePoolSize : maximumPoolSize))
return false;
// step 2.2
if (compareAndIncrementWorkerCount(c))
break retry;
c = ctl.get(); // Re-read ctl
// step 2.3
if (runStateOf(c) != rs)
continue retry;
// step 2.4
// else CAS failed due to workerCount change; retry inner loop
}
}
// step 3
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// Recheck while holding lock.
// Back out on ThreadFactory failure or if
// shut down before lock acquired.
int rs = runStateOf(ctl.get());
if (rs < SHUTDOWN ||
(rs == SHUTDOWN && firstTask == null)) {
if (t.isAlive()) // precheck that t is startable
throw new IllegalThreadStateException();
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
} finally {
mainLock.unlock();
}
if (workerAdded) {
t.start();
workerStarted = true;
}
}
} finally {
if (! workerStarted)
addWorkerFailed(w);
}
return workerStarted;
}
Copy codeNext, let's take a look at the specific processing process in turn.
The first is an endless loop with the retry label. In java, the label is generally used in multiple loops to facilitate the control of program flow. Here is a double dead loop.
Step 1: in the outer loop, the status of the thread pool is judged,
if (rs >= SHUTDOWN && ! (rs == SHUTDOWN && firstTask == null && ! workQueue.isEmpty())) Copy code
This judgment condition is very complex. Doug Lea's style likes to write a lot of processing logic into a large judgment logic, but it also makes it difficult to understand the code. Here, we convert this judgment condition according to de Morgan's law
// According to de Morgan's law if (rs >= SHUTDOWN && (rs != SHUTDOWN || firstTask != null || workQueue.isEmpty())) // Continue to simplify, remove the brackets, and become the or operation of three conditions, that is, if any of the three conditions is met, the addition will fail if((rs >= SHUTDOWN && rs != SHUTDOWN) || (rs >= SHUTDOWN && firstTask != null) || (rs >= SHUTDOWN && workQueue.isEmpty())) return false; Copy code
It seems easier to understand after simplification. Let's look at the three sub conditions separately
Condition 1, (RS > = shutdown & & RS! = shutdown) is equivalent to RS > shutdown, that is, the thread pool is in the three states of STOP, TIDYING and TERMINATED. According to the definition of the initial state, no tasks will be processed, so addWorker must fail.
Condition 2, (RS > = SHUTDOWN & & firstTask! = null), SHUTDOWN is in SHUTDOWN status and will not receive new tasks. If firstTask is not empty, it indicates that a new task is being added. Of course, it should be stopped, let alone RS > SHUTDOWN (see condition 1).
Condition 3, (RS > = shutdown & & workqueue. Isempty()), if the task queue is empty, it means that there are no tasks to do, and the thread pool has been immediately or closed. Why do you add a worker thread? Needless to say, you should naturally discourage it.
Step 2.1, after crossing the level of state judgment in the outer cycle, it enters the inner dead cycle, which has the smell of breaking through levels. In this level, first judge the number of threads wc. If the input parameter core is true, judge whether the number of core threads has reached the upper limit. If the core is false, judge whether the maximum number of threads has reached the upper limit. As long as the upper limit has been reached, you can no longer add threads.
Step 2.2, add one to ctl through CAS, that is, add one to the number of threads. As long as the CAS returns success, you can jump out of the retry double loop and enter the next level (step 3).
Step 2.3, CAS failed. There is only one possibility, that is, other threads have also modified the ctl value. We know that ctl contains two attributes: running status and number of threads, that is, these two attributes may change. Therefore, first judge whether the thread pool state has changed. If so, return to the first level (retry outer loop) and return to before liberation overnight~
Step 2.4, although there is no corresponding code in this step, it is also meaningful, that is, the number of threads in ctl has changed. At this time, there is no need to return to the first level, just start from the inner cycle.
Step 3, the number of ctl threads has increased by 1. You can't just yell without doing anything, so you start to add workers. This step is relatively simple. Create a worker object and add it to the workers collection. This step is locked because the ctl status needs to be checked again and elements need to be added to the HashSet of workers. We know that the HashSet is not thread safe.
/** * Set containing all worker threads in pool. Accessed only when * holding mainLock. */ private final HashSet<Worker> workers = new HashSet<Worker>(); Copy code
Worker thread worker
What does a successful worker look like after going through layers of hurdles? Now it's time to reveal the true face of Lushan~
private final class Worker extends AbstractQueuedSynchronizer implements Runnable
{
/**
* This class will never be serialized, but we provide a
* serialVersionUID to suppress a javac warning.
*/
private static final long serialVersionUID = 6138294804551838833L;
/** Thread this worker is running in. Null if factory fails. */
final Thread thread;
/** Initial task to run. Possibly null. */
Runnable firstTask;
/** Per-thread task counter */
volatile long completedTasks;
/**
* Creates with given first task and thread from ThreadFactory.
* @param firstTask the first task (null if none)
*/
Worker(Runnable firstTask) {
setState(-1); // inhibit interrupts until runWorker
this.firstTask = firstTask;
this.thread = getThreadFactory().newThread(this);
}
/** Delegates main run loop to outer runWorker */
public void run() {
runWorker(this);
}
// Lock methods
//
// The value 0 represents the unlocked state.
// The value 1 represents the locked state.
protected boolean isHeldExclusively() {
return getState() != 0;
}
protected boolean tryAcquire(int unused) {
if (compareAndSetState(0, 1)) {
setExclusiveOwnerThread(Thread.currentThread());
return true;
}
return false;
}
protected boolean tryRelease(int unused) {
setExclusiveOwnerThread(null);
setState(0);
return true;
}
public void lock() { acquire(1); }
public boolean tryLock() { return tryAcquire(1); }
public void unlock() { release(1); }
public boolean isLocked() { return isHeldExclusively(); }
void interruptIfStarted() {
Thread t;
if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
}
}
}
}
Copy codeThe Worker class implements the Runnable interface and inherits the AQS abstract class. Therefore, it also maintains a synchronization state state with three values:
-1: Initial state, in Worker The value assigned in the construction method is rigid new Come out, not yet run get up 0: It is in the unlocked state. As you can see later, it is in the unlocked state during non execution of tasks 1: Description of locked status worker Working (performing submitted tasks) Copy code
In the implementation of the tryAcquire method, success can be returned only when the state is 0, indicating that it is not reentrant.
The interruptIfStarted method, as its name implies, is to interrupt the worker when it starts. How do you know if the worker starts? You can know from the three states of state. If there is no start, state is the initial value - 1. Therefore, there is a judgment in the method that state > = 0. If it is a non negative number, it means that the worker has been started.
After reading the Worker class, I found that there is no variable about whether it is a core thread. How to distinguish between a core thread and a non core thread? In fact, in the implementation of Doug Lea thread pool, there is no difference between all working threads. The number of core threads and the maximum number of threads are only used to control some processing processes of the thread pool, such as addWorker above and getTask below.
The Worker class holds a thread variable. The thread variable is created as follows:
this.thread = getThreadFactory().newThread(this); Copy code
Therefore, when the thread start s, the run method of the Worker class will be executed, and the run method is directly delegated to the runWorker method of ThreadPoolExecutor.
runWorker
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
// step 1
w.unlock(); // allow interrupts
boolean completedAbruptly = true;
try {
// step 2
while (task != null || (task = getTask()) != null) {
// step 3
w.lock();
// If pool is stopping, ensure thread is interrupted;
// if not, ensure thread is not interrupted. This
// requires a recheck in second case to deal with
// shutdownNow race while clearing interrupt
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
try {
beforeExecute(wt, task);
Throwable thrown = null;
try {
// step 4
task.run();
} catch (RuntimeException x) {
thrown = x; throw x;
} catch (Error x) {
thrown = x; throw x;
} catch (Throwable x) {
thrown = x; throw new Error(x);
} finally {
afterExecute(task, thrown);
}
} finally {
task = null;
// step 5
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
// step 6
processWorkerExit(w, completedAbruptly);
}
}
Copy codeThe runWorker method is relatively long, and the key 6 steps are marked here.
Step 1. Since it has been run, first unlock it, that is, set the state state from the initial value of - 1 to 0, so as not to fail to lock later (as mentioned above, locking can succeed only when the state is 0).
Step 2, get the task. The task is either the first task passed in when creating the worker, or grab the task in the blocking queue.
Step 3, I'll start working immediately after I get the task. First lock it and tell others I'm busy. It's like going to the bathroom and locking the door.
Step 4, if you really work, you can't occupy the pit and don't shit~
Step 5, after finishing the work, successfully complete a task, add one to the chicken leg, and then unlock it, so that others can know that I'm free.
Step 6, the while loop exits, indicating that the task is not retrieved, either the thread pool is closed, or the non core thread does not retrieve the task within the timeout. In either case, the worker must be destroyed.
getTask
A core idea that thread pool can ensure thread reuse is that threads constantly obtain and execute tasks to ensure that threads always survive. Next, let's take a look at how worker threads fetch tasks.
private Runnable getTask() {
boolean timedOut = false; // Did the last poll() time out?
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// Check if queue empty only if necessary.
if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
decrementWorkerCount();
return null;
}
int wc = workerCountOf(c);
// Are workers subject to culling?
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
if ((wc > maximumPoolSize || (timed && timedOut))
&& (wc > 1 || workQueue.isEmpty())) {
if (compareAndDecrementWorkerCount(c))
return null;
continue;
}
try {
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
}
}
Copy codeHere is a brief talk about the idea of this method. Method is another dead loop. Each loop will first judge the thread pool status. If it is closed, it will directly reduce the number of threads and return null to let the working thread exit. Then judge whether the timeout is required. If the timeout is required and the timeout is not taken, the task will enter the next cycle. In the next cycle, a thread will exit because the maximum idle survival time has been exceeded. How to ensure that only one thread exits in the case of concurrency? The core lies in
if (compareAndDecrementWorkerCount(c)) return null; Copy code
That is, reduce the number of threads by one through CAS, only the successful threads of CAS will exit, and the failed threads will enter the next cycle. In the next loop, since the number of threads is reduced by one, the timed variable may no longer be true. In this way, the number of threads can be maintained dynamically without maintaining a core ID for each worker, which simplifies the implementation. It is really wonderful~
processWorkerExit
private void processWorkerExit(Worker w, boolean completedAbruptly) {
if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
decrementWorkerCount();
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
completedTaskCount += w.completedTasks;
workers.remove(w);
} finally {
mainLock.unlock();
}
tryTerminate();
int c = ctl.get();
if (runStateLessThan(c, STOP)) {
if (!completedAbruptly) {
int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
if (min == 0 && ! workQueue.isEmpty())
min = 1;
if (workerCountOf(c) >= min)
return; // replacement not needed
}
addWorker(null, false);
}
}
Copy codeWhen the worker exits, the following things should be done:
- If completedAbruptly is true, that is, the worker exits abnormally, the number of threads will be reduced by one; Because if you exit normally, the getTask method has been reduced by one.
- Add the number of completed tasks of the current worker to the number of completed tasks of the thread pool for statistics; Remove the worker from the workers collection;
- If a worker exits, it indicates that something may happen to the thread pool, so try to close a thread pool (more details will be explained below ~).
- As long as the thread pool is not in the STOP state, either the worker exits abnormally, or exits normally, but there are not enough workers to handle the existing tasks. In both cases, new workers need to be added to fill the position, so as not to affect the processing of tasks.
Close thread pool
There are two ways to close the thread pool:
- Shutdown: change the thread pool state to shutdown and interrupt all idle threads. You can see that the execution of existing tasks has not been stopped.
- Shutdown now: change the thread pool status to STOP, interrupt all threads, no longer process any tasks, and package the unprocessed tasks back.
shutdown
Let's look at the shutdown method first
public void shutdown() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
advanceRunState(SHUTDOWN);
interruptIdleWorkers();
onShutdown(); // hook for ScheduledThreadPoolExecutor
} finally {
mainLock.unlock();
}
tryTerminate();
}
private void advanceRunState(int targetState) {
for (;;) {
int c = ctl.get();
if (runStateAtLeast(c, targetState) ||
ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
break;
}
}
private void interruptIdleWorkers(boolean onlyOne) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers) {
Thread t = w.thread;
if (!t.isInterrupted() && w.tryLock()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
} finally {
w.unlock();
}
}
if (onlyOne)
break;
}
} finally {
mainLock.unlock();
}
}
final void tryTerminate() {
for (;;) {
int c = ctl.get();
if (isRunning(c) ||
runStateAtLeast(c, TIDYING) ||
(runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
return;
if (workerCountOf(c) != 0) { // Eligible to terminate
interruptIdleWorkers(ONLY_ONE);
return;
}
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
try {
terminated();
} finally {
ctl.set(ctlOf(TERMINATED, 0));
termination.signalAll();
}
return;
}
} finally {
mainLock.unlock();
}
// else retry on failed CAS
}
}
Copy codeshutdown this calls the following methods in turn
- advanceRunState, or modify the state through dead loop + CAS.
- interruptIdleWorkers know the lock status through tryLock, so as to judge whether the thread is idle. If it is idle, it will be interrupted.
- onShutdown, hook function, reserved for subclass extension.
- tryTerminate, this method is a little more complex, which is explained below.
In the tryTerminate method, there is another dead loop. First, judge the status of the thread pool. If it is RUNNING or SHUTDOWN, but there are tasks that have not been processed, it cannot be closed; If it is TIDYING or TERMINATED, it is unnecessary to terminate again and return directly.
If the number of threads is not 0, an idle worker is interrupted and returned. Why only one idle thread is interrupted here? There is mainly a scenario: the shutdown method only interrupts idle threads. If threads are executing tasks, they will not be interrupted. However, when these threads finish executing all tasks, they will eventually become idle threads and execute the processWorkerExit method. As you can see above, the processWorkerExit method will call the tryTerminate method to propagate and eventually interrupt all idle threads.
Next, it is very simple. Set the CAS status to TIDYING. After CAS succeeds, execute the hook function terminated. After executing the terminated method, you can set the state to terminated, which is also in line with the state transition diagram at the beginning. If CAS fails, enter the next cycle and continue to judge various states.
When the ctl status is set to TERMINATED, it passes termination Signalall() to wake up the thread waiting on the termination condition queue. What is the waiting thread? Of course, it is the thread calling the awaitTermination method of the thread pool~
public boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException {
long nanos = unit.toNanos(timeout);
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (;;) {
if (runStateAtLeast(ctl.get(), TERMINATED))
return true;
if (nanos <= 0)
return false;
nanos = termination.awaitNanos(nanos);
}
} finally {
mainLock.unlock();
}
}
Copy codeshutdownNow
public List<Runnable> shutdownNow() {
List<Runnable> tasks;
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
advanceRunState(STOP);
interruptWorkers();
tasks = drainQueue();
} finally {
mainLock.unlock();
}
tryTerminate();
return tasks;
}
private void interruptWorkers() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers)
w.interruptIfStarted();
} finally {
mainLock.unlock();
}
}
Copy codeAfter understanding the shutdown method, it is much easier to look at shutdown now. There are three main differences:
- Change the thread pool status to STOP.
- Interrupt all worker threads using the interruptIfStarted method, that is, whether idle or not.
- Remove all unexecuted tasks from the blocking queue and return.
summary
- Thread pools have five states, but not all of them.
- The Worker class inherits AQS and determines whether it is idle by locking.
- There is no distinction between core threads and non core threads. Only two numbers are used to control the different processing logic of the thread pool.
- After reading the thread pool source code, we will find that multithreading concurrent processing is very complex, and there are various situations to consider. The source code often uses dead loop + state judgment + CAS. Doug Lea provides us with a proven J.U.C concurrency toolkit, so we generally don't want to make our own wheels.