This article has been included in github. It is continuously updated. Welcome to star! Java-Interview-Tutorial

In order to improve the processing capacity and concurrency, the Web container will generally put the task of processing requests into the thread pool, and the JDK's native thread pool is inherently suitable for CPU intensive tasks, so Tomcat transformed it.

Tomcat thread pool principle

In fact, the parameters of ThreadPoolExecutor mainly include the following key points:

• Limit the number of threads

• Limit queue length

Tomcat needs to limit these two resources. Otherwise, the CPU and memory under high load may be exhausted. Therefore, the thread pool parameters of Tomcat:

// Customized task queue
taskqueue = new TaskQueue(maxQueueSize);

// Custom thread factory
TaskThreadFactory tf = new TaskThreadFactory(namePrefix,
							                 daemon,
							                 getThreadPriority()
);

// Custom thread pool
executor = new ThreadPoolExecutor(getMinSpareThreads(),
								  getMaxThreads(),
				 			      maxIdleTime, 
				 			      TimeUnit.MILLISECONDS,
				 			      taskqueue,
				 			      tf);

Tomcat also limits the number of threads. Settings:

• Number of core threads (minSpareThreads)
• Maximum thread pools (maxThreads)

Tomcat thread pool also has its own characteristic task processing flow. It implements its own characteristic task processing logic by rewriting the execute method:

1. When the first corePoolSize task comes, a new thread is created for each task
2. If there is another task, put the task into the task queue and let all threads grab it. If queue is full, a temporary thread is created
3. If the number of bus processes reaches maximumPoolSize, continue to try to put the task into the task queue
4. If the buffer queue is also full and the insertion fails, execute the reject policy

The difference between Tomcat and JDK thread pool is in step3. When the total number of threads reaches the maximum, Tomcat does not immediately execute the reject policy, but tries to add tasks to the task queue again. After adding failed, Tomcat executes the reject policy.

How is it realized?

public void execute(Runnable command, long timeout, TimeUnit unit) {
    submittedCount.incrementAndGet();
    try {
        // Call execute of JDK native thread pool to execute the task
        super.execute(command);
    } catch (RejectedExecutionException rx) {
       // When the number of bus processes reaches maximumPoolSize, the JDK native thread pool will execute the default reject policy
        if (super.getQueue() instanceof TaskQueue) {
            final TaskQueue queue = (TaskQueue)super.getQueue();
            try {
                // Continue trying to put the task in the task queue
                if (!queue.force(command, timeout, unit)) {
                    submittedCount.decrementAndGet();
                    // If the buffer queue is still full and the insertion fails, execute the reject policy.
                    throw new RejectedExecutionException("...");
                }
            } 
        }
    }
}

Custom task queue

The first line of the execute method of the Tomcat thread pool:

submittedCount.incrementAndGet();

If the task execution fails and the exception is thrown, the counter will be decremented by one:

submittedCount.decrementAndGet();

The Tomcat thread pool uses the submittedCount variable to maintain the number of tasks that have been submitted to the thread pool but have not been completed.

Why maintain such a variable?

The task queue of Tomcat extends the LinkedBlockingQueue of JDK. Tomcat gives it a capacity and passes it to the constructor of the parent class LinkedBlockingQueue.

public class TaskQueue extends LinkedBlockingQueue<Runnable> {

  public TaskQueue(int capacity) {
      super(capacity);
  }
  ...
}

The capacity parameter is set by the maxQueueSize parameter of Tomcat, but the default value of maxQueueSize is Integer.MAX_VALUE: after the current number of threads reaches the number of core threads, the thread pool will add the task to the task queue, and it will always succeed, so there will never be a chance to create a new thread.

To solve this problem, TaskQueue rewrites LinkedBlockingQueue#offer, and returns false when appropriate, indicating that the task addition fails. At this time, the thread pool will create a new thread.

What is the right time?

public class TaskQueue extends LinkedBlockingQueue<Runnable> {

  ...
   @Override
  // When the thread pool calls the method of the task queue, the current number of threads > the number of core threads
  public boolean offer(Runnable o) {

      // If the number of threads has reached max, you cannot create a new thread and can only put it into the task queue
      if (parent.getPoolSize() == parent.getMaximumPoolSize()) 
          return super.offer(o);
          
      // So far, it indicates that max threads > current threads > core threads
      // Description to create a new thread:
      
      // 1. If the number of submitted tasks is less than the current number of threads
      //    Indicates that there are still idle threads and there is no need to create a new thread
      if (parent.getSubmittedCount()<=(parent.getPoolSize())) 
          return super.offer(o);
          
      // 2. If the number of submitted tasks > the current number of threads
      //    If the thread is not enough, return false to create a new thread
      if (parent.getPoolSize()<parent.getMaximumPoolSize()) 
          return false;
          
      // By default, tasks are always placed in the task queue
      return super.offer(o);
  }
  
}

Therefore, Tomcat maintains the number of submitted tasks so that the thread pool can have the opportunity to create new threads when the task queue length is unlimited.