1. Thread pool related concepts

1.1. What is a thread pool

The principle of thread pool is similar to that of database connection pool. Creating threads to process business may take longer than processing business. If the system can create threads in advance and use them directly when necessary, it will not close them directly after use, but return them to the thread for other needs, This directly saves the time of creation and destruction and improves the performance of the system.

Simply put, after using thread pool, creating a thread becomes getting a free thread from the thread pool, and then using, closing the thread becomes returning the thread to the thread pool.

1.2. Implementation principle of thread pool

After submitting a task to the thread pool, the processing flow of the thread pool is as follows:

1. Judge whether the number of core threads is reached. If not, directly create a new thread to process the current incoming task, otherwise enter the next process

2. Whether the work queue in the thread pool is full. If not, the task will be put into the work queue and stored for processing first, otherwise it will enter the next process

3. Whether the maximum number of threads is reached. If not, a new thread will be created to process the current incoming task. Otherwise, it will be handed over to the saturation strategy in the thread pool for processing.

The flow chart is as follows:

        

2. Thread pool in Java

2.1. Main construction methods

jdk provides the specific implementation of thread pool. The implementation class is Java util. concurrent. ThreadPoolExecutor, main construction method:

1. corePoolSize: core thread size. When a task is submitted to the thread pool, the thread pool will create a thread to execute the task. Even if there are other idle threads that can handle the task, it will innovate the thread. When the number of working threads is greater than the number of core threads, it will not be created. If the prestartCoreThread method of the thread pool is called, the thread pool will create and start the core threads in advance

2. maximumPoolSize: the maximum number of threads allowed to be created in the thread pool. If the queue is full and the number of threads created is less than the maximum number of threads, the thread pool will create a new thread to execute the task. If we use unbounded queue, all tasks will be added to the queue, and this parameter has no effect

3. keepAliveTime: the time that the worker thread of the thread pool remains alive after it is idle. If there is no task processing, some threads will be idle. If the idle time exceeds this value, it will be recycled. If there are many tasks and the execution time of each task is relatively short, to avoid repeated creation and recycling of threads, you can increase this time and improve the utilization of threads

4. Unit: the time unit of keepAliveTIme. The selectable units are day, hour, minute, millisecond, subtle, millisecond and nanosecond. Enumeration Java. Java whose type is time unit util. concurrent. TimeUnit

5. workQueue: work queue, which is used to cache the blocking queue of pending tasks. There are four common types. See 2.4.1 for details Common work queues in thread pools

6. threadFactory: the factory that creates threads in the thread pool. You can set a more meaningful name for each created thread through the thread factory

7. handler: saturation strategy. When the thread pool is unable to handle new tasks, it needs to provide a strategy to handle new tasks submitted. By default, there are four strategies, which will be mentioned later in the article

public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue,
                              ThreadFactory threadFactory,
                              RejectedExecutionHandler handler) {
        if (corePoolSize < 0 ||
            maximumPoolSize <= 0 ||
            maximumPoolSize < corePoolSize ||
            keepAliveTime < 0)
            throw new IllegalArgumentException();
        if (workQueue == null || threadFactory == null || handler == null)
            throw new NullPointerException();
        this.acc = System.getSecurityManager() == null ?
                null :
                AccessController.getContext();
        this.corePoolSize = corePoolSize;
        this.maximumPoolSize = maximumPoolSize;
        this.workQueue = workQueue;
        this.keepAliveTime = unit.toNanos(keepAliveTime);
        this.threadFactory = threadFactory;
        this.handler = handler;
    }

2.2. execute method execution process

1. Judge whether the number of threads running in the thread pool is less than the core pool size. Yes: create a new thread to process the task. No: execute the next step

2. Attempt to add a task to the queue specified by workQueue. If it cannot be added to the queue, proceed to the next step

3. Judge whether the number of threads running in the thread pool is less than the maximumPoolSize. Yes: add a new thread to process the current incoming task. No: pass the task to the rejectedExecution method of the handler object for processing

2.3. Steps for using thread pool

1. Call constructor to create thread pool

2. Call the method of thread pool to process the task

3. Close thread pool

2.4. Common work queues in thread pools

When there are too many tasks, the work queue is used to temporarily cache the tasks to be processed. There are five common blocking queues in jdk:

ArrayBlockingQueue: it is a bounded blocking queue based on array structure. This queue sorts the elements according to the first in first out principle

LinkedBlockingQueue: it is an unbounded blocking queue based on linked list structure. This queue sorts elements according to first in first out, and its throughput is usually higher than that of ArrayBlockingQueue. Static factory method executors Newfixedthreadpool uses this queue.

SynchronousQueue: a blocking queue that does not store elements. Each insert operation must wait until another thread calls the remove operation. Otherwise, the insert operation always handles the blocking state. The throughput is usually higher than that of LinkedBlockingQueue and static factory method executors Newcachedthreadpool uses this queue

PriorityBlockingQueue: priority queue, unbounded blocking queue. The elements entering the queue will be sorted according to priority

The example of thread priority code is as follows: it can be seen from the output that except for the first task, the thread executes in the order of the set priority from high to low.

public class ThreadPriorityTest {
    static class Task implements Runnable, Comparable<Task> {
        private int priority;
        private String taskName;

        public Task(int priority, String taskName) {
            this.priority = priority;
            this.taskName = taskName;
        }

        @Override
        public void run() {
            try {
                TimeUnit.MILLISECONDS.sleep(100);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println(Thread.currentThread().getName() + "handle" + this.taskName);
        }

        @Override
        public int compareTo(Task o) {
            return Integer.compare(o.priority, this.priority);
        }
    }

    public static void main(String[] args) {
        ExecutorService executor = new ThreadPoolExecutor(1, 1,
                60L, TimeUnit.SECONDS,
                new PriorityBlockingQueue());
        System.out.println("Test thread priority");
        for (int i = 0; i < 10; i++) {
            String taskName = "task" + i;
            executor.execute(new Task(i, taskName));
        }
        for (int i = 100; i >= 90; i--) {
            String taskName = "task" + i;
            executor.execute(new Task(i, taskName));
        }
        executor.shutdown();
    }
}

2.5. Common methods of creating thread pools

The disadvantages of using the thread pool object returned by Executors are as follows:

1) FixedThreadPool and SingleThreadPool: the allowed request queue length is integer MAX_ Value, which may accumulate a large number of requests, resulting in OOM.

2) CachedThreadPool and ScheduledThreadPool: the number of threads allowed to create is integer MAX_ Value, a large number of threads may be created, resulting in OOM.

public static ExecutorService newFixedThreadPool(int nThreads) {
        return new ThreadPoolExecutor(nThreads, nThreads,
                                      0L, TimeUnit.MILLISECONDS,
                                      new LinkedBlockingQueue<Runnable>());
    }

  public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
        return new ScheduledThreadPoolExecutor(corePoolSize);
    }

    public static ExecutorService newSingleThreadExecutor() {
        return new FinalizableDelegatedExecutorService
            (new ThreadPoolExecutor(1, 1,
                                    0L, TimeUnit.MILLISECONDS,
                                    new LinkedBlockingQueue<Runnable>()));
    }

    public static ExecutorService newCachedThreadPool() {
        return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                      60L, TimeUnit.SECONDS,
                                      new SynchronousQueue<Runnable>());
    }

2.6. Common saturation strategies

When the queue in the thread pool is full and the thread pool has reached the maximum number of threads, the thread pool will pass the task to the saturation policy for processing. These policies implement the RejectedExecutionHandler interface. There are two methods in the interface:

void rejectedExecution(Runnable r, ThreadPoolExecutor executor);

2.6.1. Four common saturation strategies

• AbortPolicy: throw an exception directly

• CallerRunsPolicy: in the current caller's thread, run the task, that is, the task to be lost, which is processed by himself.

• DiscardOldestPolicy: discards the oldest task in the queue, that is, discards a task at the head of the queue, and then executes the current incoming task

• DiscardPolicy: discard it directly without processing. The method is empty

2.6.2. Custom saturation policy

When too many tasks are submitted to the thread pool and the thread pool cannot add a new task, the task is passed to the rejectedExecution method of the RejectedExecutionHandler object for processing.

RejectedExecutionHandler interface needs to be implemented. When the task cannot be processed, if you want to record the log, the example code is as follows:

Create a thread pool with a core thread and a maximum thread of 2 and a queue of 2. When five tasks are submitted, the fifth task will be rejected for execution.

public class ThreadTest {
    static class Task implements Runnable {
        String taskName;

        public Task(String taskName) {
            this.taskName = taskName;
        }

        @Override
        public void run() {
            System.out.println(Thread.currentThread().getName() + "handle" + this.taskName);
            try {
                TimeUnit.SECONDS.sleep(5);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        @Override
        public String toString(){
            return String.format("taskName:%s",taskName);
        }
    }

    public static void main(String[] args) {
        ThreadPoolExecutor executor = new ThreadPoolExecutor(2,
                2,
                60L,
                TimeUnit.SECONDS,
                new ArrayBlockingQueue<Runnable>(2),
                Executors.defaultThreadFactory(),
                (r, executors) -> {
                    System.out.println(String.format("Current tasks:%s,Tasks that cannot be processed:%s",executors.getActiveCount(),r));
                });
        for (int i = 0; i < 5; i++) {
            executor.execute(new Task("task-" + i));
        }
        executor.shutdown();
    }
}

The log output is as follows:

pool-1-thread-1 processing task - 0
pool-1-thread-2 processing TASK-1
Current number of tasks: 2, tasks that cannot be processed: taskName: Task - 4
pool-1-thread-2 processing TASK-2
pool-1-thread-1 processing TASK-3

2.7. Custom factory for creating threads

To create a custom thread factory, you need to implement Java util. concurrent. The parameter of Thread newThread(Runnable r) method in threadfactory interface is the incoming task, which needs to return a working thread.

The purpose of creating a custom thread factory: give the threads responsible for different business modules a meaningful name that can represent business functions. When there is a problem in the system, it is easier to find the problem in the system through the thread stack information.

The sample code is as follows: it is recommended to use the ThreadFactoryBuilder provided by guava to create a thread factory.

public static void main(String[] args) {
        ThreadFactory factory = new ThreadFactoryBuilder().setNameFormat("stat-%d").setDaemon(true).build();
        ThreadPoolExecutor executor = new ThreadPoolExecutor(5, 5,
                60L, TimeUnit.SECONDS,
                new ArrayBlockingQueue<Runnable>(10), factory);
        for (int i = 0; i < 5; i++) {
            String taskName = "task-" + i;
            executor.execute(() -> {
                System.out.println(Thread.currentThread().getName() + "handle" + taskName);
            });
        }
        executor.shutdown();
    }

2.8. Differences between thread pool closing methods

Shutdown only sets the state of the thread pool to the shutdown state. The executing tasks will continue to be executed, and those not executed will be interrupted.

shutdownNow sets the status of the thread pool to STOP, the executing task is stopped, and the non executing task is returned.

Take an example of Workers eating steamed stuffed buns: Workers in a factory are eating steamed stuffed buns (which can be understood as a task). If they receive the shutdown order, the Workers who get the steamed stuffed buns will eat the steamed stuffed buns on hand, and the Workers who don't get the steamed stuffed buns can't eat them (they can't take them from the cage). If you receive the order of shutdown now, all Workers will immediately stop eating steamed stuffed buns and directly put down the unfinished steamed stuffed buns on hand, let alone the steamed stuffed buns in the cage.

Shutdown: when this method is executed, the state of the thread pool immediately changes to shutdown state. At this time, no more tasks can be added to the thread pool, otherwise RejectedExecutionException will be thrown. However, at this time, the thread pool will not exit immediately until the tasks added to the thread pool have been processed.  

Shutdown now: when this method is executed, the state of the thread pool immediately changes to STOP state, and attempts to STOP all executing threads. It will no longer process the tasks waiting in the pool queue. Of course, it will return those unexecuted tasks. It attempts to terminate the thread by calling thread interrupt() method, but its function is limited. If there are no sleep, wait, Condition, timing lock and other applications in the thread, interrupt() method cannot interrupt the current thread. Therefore, shutdown now () does not mean that the thread pool will exit immediately. It may have to wait for all the tasks being executed to complete before exiting.  

2.9. How to extend thread pool

jdk provides the ThreadPoolExecutor, a high-performance thread pool, but what should I do if I want to make some extensions on this thread pool, such as monitoring the start time and end time of each task execution, or some other customized functions?

ThreadPoolExecutor provides several methods before execute, after execute and terminated, which can be used by developers themselves

• BeforeExecute: the method called before task execution has 2 parameters, the first parameter is the thread executing the task, and the second parameter is the task protected void beforeExecute(Thread t, Runnable r) {}.

• AfterExecute: the method that is called after the task is completed, the 2 parameter, the first parameter represents the task, and the second parameter indicates the abnormal information when the task is executed. If there is no exception, the second parameter is null protected void afterExecute(Runnable r, Throwable t) {}.

• Terminated: the method called after the thread pool is finally closed. When all working threads exit, the thread pool will eventually exit. When exiting, call the method {protected void terminated() {}

Code examples are as follows:

public class ThreadPoolExtTest {
    static class Task implements Runnable {
        String taskName;

        public Task(String taskName) {
            this.taskName = taskName;
        }

        @Override
        public void run() {
            System.out.println(Thread.currentThread().getName() + "handle" + this.taskName);
            try {
                TimeUnit.SECONDS.sleep(5);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

        @Override
        public String toString() {
            return String.format("taskName:%s", taskName);
        }
    }

    public static void main(String[] args) {
        ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 10, 60L,
                TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(1), Executors.defaultThreadFactory()) {
            @Override
            protected void beforeExecute(Thread t, Runnable r) {
                System.out.println(System.currentTimeMillis() + "," + t.getName() + ",Start task:" + r.toString());
            }

            @Override
            protected void afterExecute(Runnable r, Throwable t) {
                System.out.println(System.currentTimeMillis() + "," + Thread.currentThread().getName() + ",task:" + r.toString() + "，completion of enforcement!");
            }

            @Override
            protected void terminated() {
                System.out.println(System.currentTimeMillis() + "," + Thread.currentThread().getName() + "，Close thread pool!");
            }
        };
        for (int i = 0; i < 3; i++) {
            executor.execute(new Task("task-" + i));
        }
        executor.shutdown();
    }
}

3. Suggestions on the use of thread pool

3.1. Reasonably configure thread pool

You need to analyze the characteristics of the task from the following four perspectives. According to the characteristics of the task, it is all to configure the thread pool

• Nature of tasks: CPU intensive tasks, IO intensive tasks and hybrid tasks

• Task priority: high, medium and low

• Task execution time: long, medium and short

• Task dependency: whether to rely on other system resources, such as database connection.

Tasks with different properties can be processed separately with thread pools of different sizes.

• For CPU intensive tasks, the thread pool should be as small as possible, such as the thread pool with the number of CPUs + 1 thread.

• IO intensive tasks are not always executing tasks. If the cpu cannot be idle, configure as many threads as possible, such as the number of cup s * 2.

• If a hybrid task can be split into a CPU intensive task and an IO intensive task, as long as the execution time difference between the two tasks is not too large, the throughput after decomposition will be higher than that of serial execution.

You can use runtime getRuntime(). The availableprocessors () method gets the number of CPUs. Tasks with different priorities can be processed by priority queue on the thread pool, so that those with higher priority can execute first.

When using queues, it is recommended to use bounded queues. Bounded queues increase the stability of the system. If unbounded queues are used, too many tasks may lead to system OOM and direct system downtime.

3.2. Configuration of the number of threads in the thread pool

The total thread size in the thread pool has a certain impact on the performance of the system. The goal is to hope that the system can give full play to the best performance. Too many or too small threads can not effectively use the performance of the machine.

The Java Concurrency in Practice book gives the formula for estimating the size of thread pool:

Ncpu = number of cups, < Ucpu = utilization rate of target CPU, 0 < = Ucpu < = 1 "W/C = ratio of waiting time to calculation time

In order to save the processor to achieve the desired utilization, the size of the most thread pool is equal to: Nthreads = Ncpu × Ucpu × (1+W/C)

3.3. Specifications in Alibaba Java development manual

• When creating a thread or thread pool, please specify a meaningful thread name to facilitate backtracking in case of error

Java high concurrency series - day 18: java thread pool, this article is enough