catalogue
summary
In previous projects, the spring boot's built-in scheduled task function was basically used to manage scheduled tasks. The benefits of this are
- Easy to use, and its functions are integrated in spring boot out of the box
- The code is easy to write and clear. Basically, the requirements can be completed through annotations and cron expressions
However, this scheme also has some disadvantages. For example, when a scheduled task needs to change its running time or cycle, you need to modify the code and restart the service to take effect. That is, this scheme is not dynamic enough. I remember when I first contacted the scheduled task, I received a message that I need to retrieve some exchange rates regularly, but the requirements of this cycle are changed. For example, I crawl once an hour, and then I may stop the task and continue to crawl. At that time, I didn't know the existence of quartz, so I wrote a thread management method to control the start and stop of tasks at that time, which is complex and may have many loopholes. After touching quartz, I knew that there were ready-made wheels to use.
The first time quartz was used was in a distributed project. A timed task management module was extracted from the public service to uniformly manage the timed tasks of the whole system. By exposing some information management interfaces of timed tasks in the management system, it can dynamically manage the start, stop and cycle of tasks, If you only create a task in memory without persisting the task information, the restart and downtime of the service will lead to the loss of the task. Therefore, we generally need to persist the task information into the database.
Preparation dependence
The first time to learn and use spring to integrate quartz, many knowledge and understanding are not in place, so it can only be regarded as an introductory summary. First, we need to introduce our related dependencies into the spring boot project.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>This is the dependency that mainly uses quartz. Because we need to persist the scheduled tasks into the database, we need to design the scheduled task table for the database, and we can use the template given by quartz official website
Put the portal here
It mainly includes schedule,trigger and other configurations.
Configure
Start to configure the project. Since we need to persist the tasks, although the official website has database scripts, our project requirements are different. We should customize a table to store the information of these tasks. In addition, the most important thing in system operation is the log, so we need to design another table class to record the operation records of these scheduled tasks. The following is my personal design of these two items. I use spring data jpa, which is very good for demo.
@Data
@Entity
@Table(name = "self_job")
@org.hibernate.annotations.Table(appliesTo = "self_job",comment = "task model ")
@EntityListeners(AuditingEntityListener.class)
public class SelfJobPO implements Serializable {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "job_id",nullable = false,columnDefinition = "bigint(20) comment 'Primary key'")
private Long jobId;
@Column(name = "job_name",nullable = false,columnDefinition = "varchar(45) comment 'Task name'")
private String jobName;
@Column(name = "group_name",nullable = false,columnDefinition = "varchar(45) comment 'Task grouping'")
private String groupName;
@Column(name = "bean_name",nullable = false,columnDefinition = "varchar(45) comment 'java Container name'")
private String beanName;
@Column(name = "method_name",nullable = false,columnDefinition = "varchar(45) comment 'Method name'")
private String methodName;
@Column(name = "cron_expression",nullable = false,columnDefinition = "varchar(100) comment 'cron expression'")
private String cronExpression;
@Column(name = "params",nullable = false,columnDefinition = "varchar(100) default '' comment 'parameter'")
private String params;
@Column(name = "job_status",nullable = false,columnDefinition = "int(1) default 0 comment 'Task status'")
private Integer jobStatus;
@CreatedDate
@Column(name = "create_date",nullable = false,columnDefinition = "datetime default current_timestamp comment 'Creation time'")
private LocalDateTime createDate;
@LastModifiedDate
@Column(name = "update_date",nullable = false,columnDefinition = "datetime default current_timestamp on update current_timestamp comment 'Update time'")
private LocalDateTime updateDate;
}The first one we want to design is the task management class. Each record is a scheduled task. Each scheduled task needs to store the name of the task to identify our task. Since we will call the task for execution through reflection methods later, we need to record the class name, method name and required parameters of the task. At the same time, due to the different execution cycles of each task, we need to record the cron expression of the task to manage the task. Each task may be running or may have stopped running. We need to use a status value to record the status of the current task.
@Data
@Entity
@Table(name = "self_job_log")
@org.hibernate.annotations.Table(appliesTo = "self_job_log",comment = "Task log")
@EntityListeners(AuditingEntityListener.class)
public class SelfJobLogPO implements Serializable {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "log_id",nullable = false,columnDefinition = "bigint(20) comment 'journal id'")
private Long logId;
@Column(name = "job_id",nullable = false,columnDefinition = "bigint(20) comment 'task id'")
private Long jobId;
@Column(name = "user_time",nullable = false,columnDefinition = "bigint(20) default 0 comment 'It took milliseconds'")
private Long userTime;
@Column(name = "create_date",nullable = false,columnDefinition = "datetime default current_timestamp comment 'Start execution time'")
private LocalDateTime createDate;
@Column(name = "execute_status",nullable = false,columnDefinition = "int(11) default 1 comment 'Execution status 0 failed 1 succeeded'")
private Integer executeStatus;
@Column(name = "error",columnDefinition = "varchar(500) default '' comment 'error'")
private String error;
}
The second type of task log is to record the execution of each task. Therefore, first save the id of the task and associate it with the task. In addition, record whether the task execution is successful and how long it takes. If the task execution fails, what is the printed error stack, so that we can find and solve the problem of task execution failure through logs.
After creating the two classes, we need to create a new configuration class to load the configuration parameters of quartz.
@Configuration
public class QuartzConfiguration {
private static final String SCHEDULER_NAME = "Self_Scheduler";
private static final String SCHEDULER_CONTEXT_KEY = "applicationContextKey";
/**
* @description Scheduler factory class
* @author zhou
* @create 2021/3/24 12:27
* @param
* @return org.springframework.scheduling.quartz.SchedulerFactoryBean
**/
@Bean("schedulerFactoryBean")
public SchedulerFactoryBean schedulerFactoryBean(DataSource dataSource, Properties quartzProperties){
SchedulerFactoryBean schedulerFactoryBean = new SchedulerFactoryBean();
schedulerFactoryBean.setOverwriteExistingJobs(true);
schedulerFactoryBean.setDataSource(dataSource);
schedulerFactoryBean.setSchedulerName(SCHEDULER_NAME);
schedulerFactoryBean.setQuartzProperties(quartzProperties);
//Delay 30s
schedulerFactoryBean.setStartupDelay(30);
schedulerFactoryBean.setApplicationContextSchedulerContextKey(SCHEDULER_CONTEXT_KEY);
return schedulerFactoryBean;
}
@Bean
public Properties quartzProperties(){
Properties properties = new Properties();
//Timer instance name
properties.put("org.quartz.scheduler.instanceName","SelfQuartzScheduler");
properties.put("org.quartz.scheduler.instanceId","AUTO");
//Thread pool
properties.put("org.quartz.threadPool.class","org.quartz.simpl.SimpleThreadPool");
properties.put("org.quartz.threadPool.threadCount","20");
properties.put("org.quartz.threadPool.threadPriority","5");
//jobStore
properties.put("org.quartz.jobStore.class","org.quartz.impl.jdbcjobstore.JobStoreTX");
properties.put("org.quartz.jobStore.tablePrefix","QRTZ_");
properties.put("org.quartz.jobStore.driverDelegateClass","org.quartz.impl.jdbcjobstore.StdJDBCDelegate");
properties.put("org.quartz.jobStore.misfireThreshold", "12000");
return properties;
}
}The main configuration parameters can be found on the official website. First, we need to briefly understand the concept of components in quartz. There are three main components in quartz: scheduler, jobdetail and trigger.
- The scheduler is a scheduler, which is used to schedule the execution of tasks. The details and triggers of tasks will be registered in it, which can be regarded as a commander in chief
- jobdetail is the task information, which mainly records the parameter information that the task needs to carry. It can be regarded as the carrier of the task
- Trigger is a trigger. When a task is executed depends on the trigger.
First, we register a factory bean. Because we need to persist the task to the database, we need to inject data source parameters. setOverwriteExistingJobs is used to overwrite the original tasks in the container. setStartupDelay refers to the scheduling of tasks after a delay of 30 seconds after the program starts. In some projects, the configuration parameters of quartz will be read through a properties file. Here we write them directly in the code.
Design scheduled tasks
After the configuration is completed, we design the scheduled task. Because we need to dynamically manage the task, that is, we can modify the task cycle, start and stop the task, we first need to write some business codes for adding, deleting, modifying and querying, and expose these operations for our previously defined human task classes into interfaces. After this step is completed, To write and design an execution class of a task.
@Slf4j
@Component
public class QuartzJob extends QuartzJobBean {
@Override
protected void executeInternal(JobExecutionContext jobExecutionContext) throws JobExecutionException {
SelfJobLogPO logPO = new SelfJobLogPO();
long startTime = System.currentTimeMillis();
logPO.setCreateDate(LocalDateTime.ofEpochSecond(startTime/1000,0, ZoneOffset.ofHours(0)));
log.warn(logPO.getCreateDate().toString());
SelfJobPO selfJobPO = null;
LogService logService = SpringContextUtil.getBean(LogService.class);
try {
selfJobPO = (SelfJobPO) jobExecutionContext.getMergedJobDataMap().get(TackConstants.TASK_NAME);
log.debug("Scheduled task start execution,jobId:[{}]", selfJobPO.getJobId());
this.execute(selfJobPO);
logPO.setJobId(selfJobPO.getJobId());
logPO.setExecuteStatus(ExecuteEnum.SUCCESS.getCode());
logPO.setError(StringUtils.EMPTY);
} catch (Exception e) {
ErrorLogUtil.errorLog(e);
log.error("Scheduled task execution failed,jobId:[{}]", selfJobPO.getJobId());
logPO.setExecuteStatus(ExecuteEnum.FAIL.getCode());
logPO.setError(StringUtils.substring(e.toString(), 0, 500));
} finally {
long useTime = System.currentTimeMillis() - startTime;
log.debug("Scheduled task execution ends,jobId[{}],time consuming:[{}]millisecond", selfJobPO.getJobId(), useTime);
logPO.setUserTime(useTime);
logService.add(logPO);
}
}
/**
* @param selfJobPO Timed task model
* @return void
* @description Reflection execution timing task method
* @author zhou
* @create 2021/3/26 13:26
**/
private void execute(SelfJobPO selfJobPO) throws Exception {
Object bean = SpringContextUtil.getBean(selfJobPO.getBeanName());
Method method = bean.getClass().getDeclaredMethod(selfJobPO.getMethodName(), String.class);
method.invoke(bean, selfJobPO.getParams());
}
}First, declare a class and inherit QuartzJobBean, and inject this class into the spring container, which is mainly used for the specific implementation of quartz job. My understanding of it is that you can think of it as an embodiment of spring aop. For example, for the log of system operation, we often use the feature of spring aop to print the log before and after the method. So this is the aop understood as task execution. Each task execution will enter this method. Its input parameter can be understood as a context object. First, we need to convert our customized task object from the context. getMergedJobDataMap this method will get a data structure similar to map, and then get the scheduled tasks in our thread through the key.
When we get our own scheduled task, we need to call this scheduled task. Generally speaking, the specific implementation of timing task is that we write specific business code in our code in advance, and then call it through a method. So how do we dynamically execute a method when the program is running? This time we need to use our reflection. Previously, in our custom task class, we recorded the bean name, method name and parameters required to call the method.
We encapsulate a method that gets the object of the execution method through the reflection of the bean name, and then reflects it to the specific method through the method name and then calls the execution. Here is a simple java reflection execution.
In addition, we need to record the execution record of this task, so we need to generate the log record class defined before in executeInternal to save the log record.
Here, the task execution has been managed uniformly. We also need to conduct dynamic log management. We need to create another tool class
public class QuartzUtil {
private static final String KEY = "TASK_";
public static JobKey getJobKey(Long jobId,String group){
return JobKey.jobKey(KEY+jobId,group);
}
public static TriggerKey getTriggerKey(Long jobId,String group){
return TriggerKey.triggerKey(KEY+jobId,group);
}
public static Trigger getJobTrigger(Scheduler scheduler,Long jobId,String group) throws SchedulerException {
return scheduler.getTrigger(getTriggerKey(jobId, group));
}
/**
* @description Create scheduled task
* @author zhou
* @created 2021/4/18 21:50
* @param scheduler dispatch
* @param selfJobPO Custom task information
* @return void
**/
public static void createJob(Scheduler scheduler, SelfJobPO selfJobPO){
try{
//job information
JobDetail jobDetail = JobBuilder.newJob(QuartzJob.class).withIdentity(getJobKey(selfJobPO.getJobId(), selfJobPO.getGroupName()))
.build();
//cron
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(selfJobPO.getCronExpression())
.withMisfireHandlingInstructionDoNothing();
//trigger
CronTrigger cronTrigger = TriggerBuilder.newTrigger().withIdentity(getTriggerKey(selfJobPO.getJobId(), selfJobPO.getGroupName()))
.withSchedule(cronScheduleBuilder).build();
//Store information
jobDetail.getJobDataMap().put(TackConstants.TASK_NAME,selfJobPO);
//The scheduler stores task information and triggers
scheduler.scheduleJob(jobDetail,cronTrigger);
}catch (SchedulerException e){
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.ADD_JOB_ERROR);
}
}
/**
* @description Update scheduled tasks
* @author zhou
* @created 2021/4/18 23:19
* @param
* @return void
**/
public static void updateJob(Scheduler scheduler,SelfJobPO selfJobPO){
try {
//Get trigger key
TriggerKey triggerKey = getTriggerKey(selfJobPO.getJobId(),selfJobPO.getGroupName());
//Rebuild cron
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(selfJobPO.getCronExpression())
.withMisfireHandlingInstructionDoNothing();
//Get the original trigger
CronTrigger cronTrigger = (CronTrigger) scheduler.getTrigger(triggerKey);
if(cronTrigger.getCronExpression().equalsIgnoreCase(selfJobPO.getCronExpression())){
return;
}
//Update trigger
cronTrigger = cronTrigger.getTriggerBuilder().withIdentity(triggerKey).withSchedule(cronScheduleBuilder).build();
//Update scheduling information in trigger
cronTrigger.getJobDataMap().put(TackConstants.TASK_NAME,selfJobPO);
//Update task
scheduler.rescheduleJob(triggerKey,cronTrigger);
} catch (SchedulerException e) {
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.MODIFY_JOB_ERROR);
}
}
/**
* @description Pause scheduled tasks
* @author zhou
* @create 2021/4/19 14:50
* @param
* @return void
**/
public static void pauseJob(Scheduler scheduler,SelfJobPO selfJobPO){
try {
scheduler.pauseJob(getJobKey(selfJobPO.getJobId(),selfJobPO.getGroupName()));
} catch (SchedulerException e) {
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.PAUSE_JOB_ERROR);
}
}
/**
* @description Resume scheduled tasks
* @author zhou
* @create 2021/4/19 14:56
* @param
* @return void
**/
public static void resumeJob(Scheduler scheduler,SelfJobPO selfJobPO){
try {
scheduler.resumeJob(getJobKey(selfJobPO.getJobId(),selfJobPO.getGroupName()));
} catch (SchedulerException e) {
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.RESUME_JOB_ERROR);
}
}
/**
* @description Delete scheduled task
* @author zhou
* @create 2021/4/19 16:55
* @param
* @return void
**/
public static void deleteJob(Scheduler scheduler,SelfJobPO selfJobPO){
try{
scheduler.deleteJob(getJobKey(selfJobPO.getJobId(),selfJobPO.getGroupName()));
}catch (SchedulerException e){
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.DELETE_JOB_ERROR);
}
}
/**
* @description Execute scheduled tasks immediately
* @author zhou
* @create 2021/4/19 17:01
* @param
* @return void
**/
public static void execJob(Scheduler scheduler,SelfJobPO selfJobPO){
try {
JobDataMap jobDataMap = new JobDataMap();
jobDataMap.put(TackConstants.TASK_NAME,selfJobPO);
scheduler.triggerJob(getJobKey(selfJobPO.getJobId(),selfJobPO.getGroupName()),jobDataMap);
} catch (SchedulerException e) {
ErrorLogUtil.errorLog(e);
throw new ServiceErrorException(ServiceErrorEnum.EXEC_JOB_ERROR);
}
}
}
The common methods of dynamically managing scheduled tasks are designed here. In fact, the structure is relatively clear. The main thing is to first construct two components, one is the information of jobdetail task, the other is trigger trigger, and then register these two components in scheduler scheduler. Then it is called through the scheduler's own api.
After writing, we associate these tools and methods with the business code of our management tasks, that is, calling the tool class code in business code to manage business tasks interworking with quartz tasks.