Calculate the score of Posts every hour and clean up the temporary files of the server every half an hour. These needs need to be solved by the task scheduling component.
The task scheduling component is based on multithreading. If multithreading is used, the thread pool will be used, because creating threads has overhead and large overhead. Using thread pool to manage threads can reuse threads, improve processing capacity and save resources.
Thread pool is also the focus.
jdk thread pool comes with jdk and spring thread pool comes with spring. executor services and thread pool task executor are ordinary thread pools
Threads created by scheduled executor services and thread pool task scheduler can execute scheduled tasks. However, when distributed, these two problems exist. The scheduler is not appropriate, because the parameters relied on by the scheduler program are stored in the database, and the scheduler does not solve the problem of distributed.
For example, if you delete a temporary file every ten minutes, both of them will do so, which will lead to a certain conflict
The timing task components of jdk and spring are based on memory. How often the configuration runs and the configuration parameters are in memory, but the memory of servers 1 and 2 is not shared and they cannot know what they are doing, so there will be some conflicts
Therefore, Quartz is used more in distributed environment
The parameters that the Quartz program depends on are stored in the database (DB), so no matter how many Quartz are deployed, the same database will be accessed. (there can be multiple servers, but there is only one database)
The browser sends the request to Nginx. According to certain policies, Nginx selects the server to process the request. If it is an ordinary request, it is assigned to the controller for processing
If it is changed to Quartz, different requests can be queued.
Our ultimate goal is to use Quartz, but before that, we should learn to use jdk and spring thread pool.
Next, create test
ThreadPoolTests
@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = CommunityApplication.class)
public class ThreadPoolTests {
private static final Logger logger = LoggerFactory.getLogger(ThreadPoolTests.class);//When the logger outputs content, it will naturally bring the id and time of the thread
// JDK common thread pool
private ExecutorService executorService = Executors.newFixedThreadPool(5);//Instantiate through factory Executors. After instantiation, there are five threads, which are reused repeatedly
// JDK thread pool that can execute scheduled tasks
private ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(5);
// Spring common thread pool
@Autowired
private ThreadPoolTaskExecutor taskExecutor;
// Spring thread pool for executing scheduled tasks
@Autowired
private ThreadPoolTaskScheduler taskScheduler;
@Autowired
private AlphaService alphaService;
//ThreadPoolTests is a junit test method, which is different from the main method. If you start a thread in the main method and the thread does not hang up, main will block the execution of ThreadPoolTests, so that main will block the execution of ThreadPoolTests and will not end immediately.
//However, every time junit test method starts a sub thread, it is concurrent with the current thread. If the test method has no logic, it will end immediately, regardless of whether the started thread is completed or not.
//Therefore, after the test method starts a thread, close the thread after its execution. Then let the current main thread sleep for a while
private void sleep(long m) {//m is in milliseconds
try {
Thread.sleep(m);//Current thread blocking
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 1.JDK common thread pool
@Test
public void testExecutorService() {//You need to assign a task to the thread pool to assign a thread to execute. The task is the thread body.
Runnable task = new Runnable() {
@Override
public void run() {
logger.debug("Hello ExecutorService");
}
};
for (int i = 0; i < 10; i++) {//Execute 10 times
executorService.submit(task);//Each time a submit method is called, the thread pool will allocate a thread to execute the thread body.
}
sleep(10000);//Otherwise, the method ends before the thread has finished executing. 1w milliseconds is 10s
}
result:
1-5 threads back and forth
Output content:
// 2.JDK timed task thread pool (set the time interval for continuous execution, and provide a thread body)
@Test
public void testScheduledExecutorService() {
Runnable task = new Runnable() {
@Override
public void run() {
logger.debug("Hello ScheduledExecutorService");
}
};
scheduledExecutorService.scheduleAtFixedRate(task, 10000, 1000, TimeUnit.MILLISECONDS);//The first parameter is the task. The second parameter is how many ms the task is delayed. The third parameter is the time interval ms, and the third parameter is the declared time unit timeunit MILLISECONDS
sleep(30000);
}
Add in application properties
# TaskExecutionProperties spring common thread pool configuration # There are several core threads in the thread pool spring.task.execution.pool.core-size=5 # When the core thread is not enough, it can be expanded to 15 at most spring.task.execution.pool.max-size=15 #Queue capacity refers to the queue capacity. When 15 threads are not enough, they need to be put in the queue and wait. Setting a queue can buffer 100 tasks spring.task.execution.pool.queue-capacity=100 # TaskSchedulingProperties spring can start the configuration of thread pool for scheduled tasks spring.task.scheduling.pool.size=5
// Spring common thread pool
@Autowired
private ThreadPoolTaskExecutor taskExecutor;
// 3.Spring common thread pool
@Test
public void testThreadPoolTaskExecutor() {
Runnable task = new Runnable() {//Declare thread body
@Override
public void run() {
logger.debug("Hello ThreadPoolTaskExecutor");
}
};
for (int i = 0; i < 10; i++) {
taskExecutor.submit(task);
}
sleep(10000);
}
Errors will be reported during execution
In ThreadPoolConfig
@Configuration
@EnableScheduling
//If enablesscheduling is not added, it indicates that the scheduled task is not started
@EnableAsync//Make the @ Async annotation in AlphaService effective
public class ThreadPoolConfig {
}
Execution result:
Spring ordinary thread pool is more flexible than jdk thread pool, because you can set the number of core threads and expand the number of threads
// Spring thread pool for executing scheduled tasks
@Autowired
private ThreadPoolTaskScheduler taskScheduler;
// 4.Spring timed task thread pool
@Test
public void testThreadPoolTaskScheduler() {
Runnable task = new Runnable() {
@Override
public void run() {
logger.debug("Hello ThreadPoolTaskScheduler");
}
};
Date startTime = new Date(System.currentTimeMillis() + 10000);//The current time is delayed by 10000 milliseconds, which is the current time
taskScheduler.scheduleAtFixedRate(task, startTime, 1000);//Execute at a fixed frequency and at intervals
sleep(30000);
}
// 5.Spring common thread pool (Simplified)
@Test
public void testThreadPoolTaskExecutorSimple() {
for (int i = 0; i < 10; i++) {
alphaService.execute1();
}
sleep(10000);//Otherwise, the thread has not been executed and the method has ended
}
In AlphaService,
// Let the method be called asynchronously in a multithreaded environment That is, start a thread to execute this method, which is executed concurrently (asynchronously) with the main thread
@Async//Previously, @ EnableAsync in ThreadPoolConfig made this configuration effective.
public void execute1() {
logger.debug("execute1");
}
Why should a thread sleep
// 6.Spring timed task thread pool (Simplified)
@Test
public void testThreadPoolTaskSchedulerSimple() {
sleep(30000);
}
Next, we will demonstrate Quartz, but Quartz is database dependent.
DB has a set of tables that we need to create in advance. This table is what Quartz needs
This table is the table that Quartz depends on
We need to use the command line tool and the source command to import it into the community library.
After import:
47.12
Import Quartz package in maven
Then look at the source code:
Job, Scheduler, jobdetail (used to configure job), trigger trigger (set how often to run repeatedly)
Defining a task through the job interface and configuring a job through the jobdetail and trigger interfaces mainly do these three things.
After configuration and restart, quartz will restart the configuration information and save the read configuration information to the database (table). Quartz will read the information in the table to perform tasks in the future.
trigger is used when the service is started for the first time and will not be used in the future
The information configured through jobdetail is saved in this table.
Name of the task
Name of job
Grouping of job s
job description
Class corresponding to job
...
Create a new AlphaJob to implement the Job interface
public class AlphaJob implements Job {
@Override
public void execute(JobExecutionContext context) throws JobExecutionException {
System.out.println(Thread.currentThread().getName() + ": execute a quartz job.");//Print the name of the current thread +: execute a quartz job
}
}
QuartzConfig
// Configuration (only read for the first time) - > the information is initialized to the database - > quartz accesses the database to call the task, not accessing the configuration file
@Configuration
public class QuartzConfig {
//FactoryBean is essentially different from the BeanFactory learned by IOC at the beginning; BeanFactory is the top-level interface of the whole IOC container
// The main purpose of FactoryBean is to simplify the instantiation process of beans, because some Bean instantiation processes are complex:
// 1. Encapsulate the instantiation process of Bean through FactoryBean
// 2. Assemble FactoryBean into Spring container
// 3. Inject FactoryBean into other beans
// 4. The Bean gets the object instance managed by FactoryBean
// Configure JobDetail
// @Bean
public JobDetailFactoryBean alphaJobDetail() {//The name of the bean is alphaJobDetail. Initialize the bean to assemble it into a container
JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();//Instantiate object
factoryBean.setJobClass(AlphaJob.class);
factoryBean.setName("alphaJob");//Declare the name of the job task
factoryBean.setGroup("alphaJobGroup");//Group that declares the task
factoryBean.setDurability(true);//Declare whether the task is saved for a long time, even if the task is no longer running. Even if there is no trigger, it will be reported and stored all the time
factoryBean.setRequestsRecovery(true);//Declare whether the task is recoverable
return factoryBean;
}
// Configure Trigger(SimpleTriggerFactoryBean is relatively simple and needs to be done every ten days...; CronTriggerFactoryBean is complex and needs to be done two days before the end of each month...)
// @Bean
public SimpleTriggerFactoryBean alphaTrigger(JobDetail alphaJobDetail) {//Trigger depends on JobDetail, so it needs to be read
SimpleTriggerFactoryBean factoryBean = new SimpleTriggerFactoryBean();
factoryBean.setJobDetail(alphaJobDetail);
factoryBean.setName("alphaTrigger");
factoryBean.setGroup("alphaTriggerGroup");
factoryBean.setRepeatInterval(3000);//How often do you perform tasks
factoryBean.setJobDataMap(new JobDataMap());//The bottom layer of Trigger needs to store some states and create a JobDataMap object to store them
return factoryBean;
}
Test:
The bottom layer of Quartz also depends on the thread pool. The thread pool has a default configuration. If you want to reconfigure the bottom thread pool, you need to configure it in application properties.
```c # QuartzProperties spring.quartz.job-store-type=jdbc//Use jdbc to store tasks spring.quartz.scheduler-name=communityScheduler//Scheduler name spring.quartz.properties.org.quartz.scheduler.instanceId=AUTO//The id of the scheduler is automatically generated spring.quartz.properties.org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX spring.quartz.properties.org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate//Drive StdJDBCDelegate spring.quartz.properties.org.quartz.jobStore.isClustered=true//Whether cluster mode is adopted? Yes spring.quartz.properties.org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool//Use org quartz. simpl. SimpleThreadPool thread pool spring.quartz.properties.org.quartz.threadPool.threadCount=5//Number of threads
After the above configuration is made,
The following information appears in the table:
New QuartzTests
package com.nowcoder.community;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.quartz.JobKey;
import org.quartz.Scheduler;
import org.quartz.SchedulerException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;
@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = CommunityApplication.class)
public class QuartzTests {
@Autowired
private Scheduler scheduler;
@Test
public void testDeleteJob() {
try {
boolean result = scheduler.deleteJob(new JobKey("alphaJob", "alphaJobGroup"));
System.out.println(result);
} catch (SchedulerException e) {
e.printStackTrace();
}
}
}
Return true
Did you delete it
There is not a job but a scheduler in it, so things are still there
