Initial task execution and scheduling

We need to regularly calculate the score of each post and clear temporary files, which requires the component of task scheduling.

JDK thread pool

• ExecutorService: normal thread pool
• ScheduledExecutorService: can perform scheduled tasks (distributed environment may have problems)

Spring thread pool

• ThreadPoolTaskExecutor: a common thread pool
• ThreadPoolTaskScheduler: can perform scheduled tasks (distributed environment may have problems)

Distributed timed task

• Spring Quartz (store the data in the database and share the data when distributed)

1. Core scheduling interface Scheduler
2. Define the execute method of the Job interface of the task
3. Jobdetail interface to configure Job name, group, etc
4. The Trigger interface configures when and how often a Job runs
5. QuartzConfig: configuration - > Database - > call

• Factorybeans simplify the instantiation of beans

1. Encapsulate the instantiation process of beans through factorybeans
2. Assemble FactoryBean into Spring container
3. Inject FactoryBean into other beans
4. The Bean gets the object instance managed by the FactoryBean

If we use JDK or Spring Scheduler to execute scheduled tasks, there may be problems in the distributed environment. Because the execution parameters set by the JDK or Spring Scheduler are stored in memory, and memory is not shared between servers, problems will arise when multiple schedulers operate. It can be solved through Quartz. The configuration parameters of Quartz are saved in the database.
 

1. Configuration file

#TaskExecutionProperties
  task:
    execution:
      pool:
        core-size: 5
        max-size: 15
        queue-capacity: 100

#TaskSchedulingProperties
    scheduling:
      pool:
        size: 5

2. Configure the ThreadPoolConfig class

import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;

@Configuration
@EnableScheduling
@EnableAsync
public class ThreadPoolConfig {
}

3. Test

3.1 AlphaService

    //Let the method be called asynchronously in a multithreaded environment
    @Async
    public void execute1(){
        logger.debug("execute1");
    }

    @Scheduled(initialDelay = 10000,fixedRate = 1000)
    public void execute2(){
        logger.debug("execute2");
    }

3.2 ThreadPoolTests

import com.nowcoder.community.service.AlphaService;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.Date;
import java.util.concurrent.*;

@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = CommunityApplication.class)
public class ThreadPoolTests {

    private static final Logger logger= LoggerFactory.getLogger(ThreadPoolTests.class);

    // 1.JDK common thread pool
    private ExecutorService executorService= Executors.newFixedThreadPool(5);

    // 2.JDK thread pool that can execute scheduled tasks
    private ScheduledExecutorService scheduledExecutorService= Executors.newScheduledThreadPool(5);

    // 3.Spring common thread pool
    @Autowired
    private ThreadPoolTaskExecutor taskExecutor;

    // 4.Spring thread pool for executing scheduled tasks
    @Autowired
    private ThreadPoolTaskScheduler taskScheduler;

    @Autowired
    private AlphaService alphaService;

    private void sleep(long m){
        try {
            Thread.sleep(m);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    // 1.JDK common thread pool
    @Test
    public void testExecutorService(){

        for (int i = 0; i < 10; i++) {
            executorService.execute(()->{logger.debug("Hello ExecutorService");});
        }

        sleep(10000);
    }

    // 2.JDK timed task thread pool
    @Test
    public void testScheduledExecutorService(){
        scheduledExecutorService.scheduleAtFixedRate(()->{logger.debug("Hello ScheduledExecutorService");},
                10000,1000, TimeUnit.MILLISECONDS);

        sleep(30000);
    }

    // 3.Spring common thread pool
    @Test
    public void testThreadPoolTaskExecutor(){
        for (int i = 0; i < 10; i++) {
            taskExecutor.execute(()->{logger.debug("Hello ThreadPoolTaskExecutor");});
        }

        sleep(10000);
    }

    // 4.Spring timed task thread pool
    @Test
    public void testThreadPoolTaskScheduler(){
        Date startTime=new Date(System.currentTimeMillis()+10000);

        taskScheduler.scheduleAtFixedRate(()->{logger.debug("Hello ThreadPoolTaskScheduler");},startTime,1000);

        sleep(30000);
    }

    // 5.Spring common thread pool (Simplified)
    @Test
    public void testThreadPoolTaskExecutorSimple(){
        for (int i = 0; i < 10; i++) {
            alphaService.execute1();
        }

        sleep(10000);
    }

    // 6.Spring timed task thread pool (Simplified)
    @Test
    public void testThreadPoolTaskSchedulerSimple() {
        sleep(30000);
    }
}

4.Quartz

Because Quartz is database-based, first initialize tables_mysql_innodb.sql

• qrtz_job_detail: the table describing the job
• qrtz_simple_triggers: simple configuration related to triggers
• qrtz_triggers: trigger related configuration
• qrtz_scheduler_state: related contents of timer
• qrtz_locks: lock information

Basic components of Quartz:

• Scheduler: scheduler: core scheduling interface
• Task: time Job interface, which declares tasks. Configure the detailed parameters of a Job through JobDetail
• Trigger: trigger, including SimpleTrigger and CronTrigger, which configures the parameters of Job runtime

4.1 import dependency

        <!--quartz-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-quartz</artifactId>
        </dependency>

4.2 implementation of Job interface AlphaJob

import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

//example
public class AlphaJob implements Job {
    @Override
    public void execute(JobExecutionContext jobExecutionContext) throws JobExecutionException {
        System.out.println(Thread.currentThread().getName() + "quartzJob");
    }
}

4.3 configuration class QuartzConfig

import com.nowcoder.community.quartz.AlphaJob;
import org.quartz.JobDataMap;
import org.quartz.JobDetail;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.JobDetailFactoryBean;
import org.springframework.scheduling.quartz.SimpleTriggerFactoryBean;

@Configuration
public class QuartzConfig {

    //FactoryBean function: simplify the instantiation process of beans
    //1. Encapsulate the Bean instantiation process through FactoryBean
    //2. Assemble FactoryBean into spring container
    //3. Inject FactoryBean into other beans
    //4. Other bean s can obtain the object instances managed by FactoryBean

    //1. Configure jobDetail
    @Bean
    public JobDetailFactoryBean alphaJobDetail(){
        JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
        factoryBean.setJobClass(AlphaJob.class);    //Declare the job class to configure
        factoryBean.setName("alphaJob");           //Declare the name of the job
        factoryBean.setGroup("alphaJobGroup");    //Declare the grouping of job s
        factoryBean.setDurability(true);         //Does the job exist for a long time
        factoryBean.setRequestsRecovery(true);  //Can the job be restored

        return factoryBean;
    }

    //2. Configure trigger
    //  SimpleTriggerFactoryBean: simple timing
    //  Crontrigger factorybean: it can realize complex timing, such as clearing on the 1st of each year
    @Bean
    public SimpleTriggerFactoryBean alphaTrigger(JobDetail alphaJobDetail){
        SimpleTriggerFactoryBean factoryBean = new SimpleTriggerFactoryBean();
        factoryBean.setJobDetail(alphaJobDetail);         //Confirm JobDetail
        factoryBean.setName("alphaTrigger");             //Trigger's name
        factoryBean.setGroup("alphaTriggerGroup");      //Grouping of triggers
        factoryBean.setRepeatInterval(3000);           //Time interval 3 seconds
        factoryBean.setJobDataMap(new JobDataMap());  //Type of saved data

        return factoryBean;
    }
}

4.4 configuration file

# QuartzProperties
  quartz:
    job-store-type: jdbc                       #Storage mode
    scheduler-name: communityScheduler         #Scheduler name
    properties: {
                  org.quartz.scheduler.instanceId: AUTO,  #Automatically generate the id of the scheduler
                  org.quartz.jobStore.class: org.quartz.impl.jdbcjobstore.JobStoreTX, #Store required classes
                  org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate, #drive
                  org.quartz.jobStore.isClustered: true,   #Adopt cluster mode
                  org.quartz.threadPool.class: org.quartz.simpl.SimpleThreadPool,  #Thread pool used
                  org.quartz.threadPool.threadCount: 5    #Number of thread pools
    }

4.5 delete job} QuartzTests in Quartz database

import org.junit.Test;
import org.junit.runner.RunWith;
import org.quartz.JobKey;
import org.quartz.Scheduler;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(classes = CommunityApplication.class)
public class QuartzTest {

    //Delete the job in the Quartz database
    @Autowired
    private Scheduler scheduler;

    @Test
    public void deleteJob(){
        // Jobkey (name of job, group name of job)
        try {
            boolean b=scheduler.deleteJob(new JobKey("alphaJob","alphaJobGroup"));
            System.out.println(b);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}