Click "taro source code" above and select“ Set as star"

Whether she's ahead or behind?

A wave that can wave is a good wave!

Update the article at 10:33 every day and lose 100 million hair every day

Source code boutique column

• Original | Java 2021 super God Road, very liver~
• Open source project with detailed Chinese Notes
• RPC framework Dubbo source code analysis
• Analysis of Netty source code of network application framework
• Analysis of message middleware RocketMQ source code
• Source code analysis of database middleware sharding JDBC and MyCAT
• Analysis of job scheduling middleware elastic job source code
• Source code analysis of distributed transaction middleware TCC transaction
• Eureka and Hystrix source code analysis
• Java Concurrent source code

Source: Juejin cn/post/

6930912870058328071

• Single point timed task
  • JDK native
  • Spring Task
  • Redis based implementation
• Distributed timed task
  • Introducing distributed timed task component or middleware
• summary

Single point timed task

JDK native

Since jdk1 After 5, ScheduledExecutorService is provided to replace TimerTask to execute scheduled tasks, which provides good reliability.

public class SomeScheduledExecutorService {
    public static void main(String[] args) {
        //Create a task queue with a total of {10 threads
        ScheduledExecutorService scheduledExecutorService =
                Executors.newScheduledThreadPool(10);
        //Task execution: start after 1 second, and execute every 30 seconds
        scheduledExecutorService.scheduleAtFixedRate(() -> {
            System.out.println("Perform tasks:" + new Date());
        }, 10, 30, TimeUnit.SECONDS);
    }
}

Spring Task

The Spring Framework comes with timed tasks and provides cron expressions to realize rich timed task configuration. Recommended for beginners https://cron.qqe2.com/ This site to match your cron expression.

@Configuration
@EnableScheduling
public class SomeJob {
    private static final Logger LOGGER = LoggerFactory.getLogger(SomeJob.class);

    /**
     * Every minute (example: 18:01:00, 18:02:00)
     * Second, minute, hour, day, month, week, year
     */
    @Scheduled(cron = "0 0/1 * * * ? *")
    public void someTask() {
       //...
    }
}

Single point timing service in the current microservice environment, the application scenarios are becoming more and more limited, so try distributed timing tasks.

Redis based implementation

Compared with the previous two methods, this Redis based implementation can increase scheduled tasks and consumption through multiple points. However, we should be prepared to prevent repeated consumption.

By means of ZSet

Store the scheduled task in the ZSet set, and store the expiration time in the Score field of ZSet, and then judge whether there is a scheduled task to be executed within the current time through a cycle. If so, execute it.

The specific implementation code is as follows:

/**
 * Description: ZSet timing task based on Redis< br>
 *
 * @author mxy
 */
@Configuration
@EnableScheduling
public class RedisJob {
    public static final String JOB_KEY = "redis.job.task";
    private static final Logger LOGGER = LoggerFactory.getLogger(RedisJob.class);
    @Autowired private StringRedisTemplate stringRedisTemplate;

    /**
     * Add task
     *
     * @param task
     */
    public void addTask(String task, Instant instant) {
        stringRedisTemplate.opsForZSet().add(JOB_KEY, task, instant.getEpochSecond());
    }

    /**
     * Scheduled task queue consumption
     * Consume once per minute (the interval can be shortened to 1s)
     */
    @Scheduled(cron = "0 0/1 * * * ? *")
    public void doDelayQueue() {
        long nowSecond = Instant.now().getEpochSecond();
        //Query all tasks at the current time
        Set<String> strings = stringRedisTemplate.opsForZSet().range(JOB_KEY, 0, nowSecond);
        for (String task : strings) {
            //Start consuming {task
            LOGGER.info("Perform tasks:{}", task);
        }
        //Delete tasks already executed
        stringRedisTemplate.opsForZSet().remove(JOB_KEY, 0, nowSecond);
    }
}

The applicable scenarios are as follows:

• 15 minutes after the order is placed, if the user does not pay, the system needs to automatically cancel the order.
• The red envelope has not been checked for 24 hours, and the return business needs to be delayed;
• An activity is specified to be effective & invalid within a certain time;

The advantages are:

• The query operation of MySQL is omitted, and Redis with higher performance is used instead;
• The tasks to be performed will not be missed due to shutdown and other reasons;

Key space notification mode

We can implement scheduled tasks through Redis's key space notification. Its implementation idea is to set an expiration time for all scheduled tasks. After expiration, we can perceive that the scheduled tasks need to be executed by subscribing to the expiration message. At this time, we can execute the scheduled tasks.

By default, Redis does not enable keyspace notifications. We need to manually enable them through the command of config set notify keyspace events ex. The code of the scheduled task after opening is as follows:

Custom listener

 /**
  * Custom listener
  */
public class KeyExpiredListener extends KeyExpirationEventMessageListener {
    public KeyExpiredListener(RedisMessageListenerContainer listenerContainer) {
        super(listenerContainer);
    }

    @Override
    public void onMessage(Message message, byte[] pattern) {
        // channel
        String channel = new String(message.getChannel(), StandardCharsets.UTF_8);
        //Expired key
        String key = new String(message.getBody(), StandardCharsets.UTF_8);
        //todo your handling
    }
}

Set this listener

/**
 * Description: The scheduled task is realized by subscribing to the expiration notice of Redis< br>
 *
 * @author mxy
 */
@Configuration
public class RedisExJob {
    @Autowired private RedisConnectionFactory redisConnectionFactory;
    @Bean
    public RedisMessageListenerContainer redisMessageListenerContainer() {
        RedisMessageListenerContainer redisMessageListenerContainer = new RedisMessageListenerContainer();
        redisMessageListenerContainer.setConnectionFactory(redisConnectionFactory);
        return redisMessageListenerContainer;
    }

    @Bean
    public KeyExpiredListener keyExpiredListener() {
        return new KeyExpiredListener(this.redisMessageListenerContainer());
    }
}

Spring will listen for Redis messages in the following format

private static final Topic TOPIC_ALL_KEYEVENTS = new PatternTopic("__keyevent@*");

Redis based timing tasks can be applied to limited scenarios, but the implementation is relatively simple, but there are great requirements for functional idempotence. In terms of usage scenario, it should be called delayed task.

Scenario example:

• 15 minutes after the order is placed, if the user does not pay, the system needs to automatically cancel the order.
• The red envelope has not been checked for 24 hours, and the return business needs to be delayed;

The advantages and disadvantages are:

• Passive triggering, less resource consumption for services;
• Redis's Pub/Sub is unreliable and there is no ACK mechanism, but it can be tolerated in general;
• The keyspace notification function consumes some CPU

The background management system + user applet based on spring boot + mybatis Plus + Vue & element supports RBAC dynamic permission, multi tenant, Data permission, workflow, tripartite login, payment, SMS, mall and other functions. Project address: https://github.com/YunaiV/ruoyi-vue-pro

Distributed timed task

Introducing distributed timed task component or middleware

Taking scheduled tasks as a separate service curbs repeated consumption, and independent services are also conducive to expansion and maintenance.

quartz

Relying on MySQL, it is relatively simple to use and can be deployed on multiple nodes. Only one node can perform tasks by competing for database locks. There is no graphical management page, so it is relatively troublesome to use.

elastic-job-lite

Depending on zookeeper, servers can be added dynamically through the registration and discovery of zookeeper.

• Multiple operation modes
• Failure transfer
• Running status collection
• Multithreading data processing
• Idempotency
• fault tolerant
• Support spring namespace
• There is a graphical management page

LTS

Depending on Zookeeper and cluster deployment, servers can be added dynamically. You can manually add scheduled tasks, start and pause tasks.

• Business log recorder
• SPI extended support
• Failover
• Node monitoring
• Diversified task execution result support
• FailStore fault tolerance
• Dynamic capacity expansion
• Relatively friendly to spring
• Graphical interface for monitoring and management

xxl-job

Domestic, relying on MySQL and based on competitive database lock, it ensures that only one node performs tasks and supports horizontal capacity expansion. You can manually add scheduled tasks, start and pause tasks.

• Elastic expansion
• Split broadcast
• Failover
• Rolling real time log
• GLUE (support online code editing, no release)
• Task progress monitoring
• Task dependency
• data encryption
• Mail alarm
• Run Report
• Graceful shutdown
• Internationalization (Chinese friendly)

Based on the idea of micro service, the project practice under the B2C e-commerce scenario is constructed. The core technology stack is Spring Boot + Dubbo. In the future, Spring Cloud Alibaba will be reconstituted. Project address: https://github.com/YunaiV/onemall

summary

Under microservices, it is recommended to use component services such as XXL job to manage scheduled tasks reasonably and effectively. The single point timing task has its limitations, which is suitable for services with small scale and low requirements for future expansion.

Relatively speaking, the scheduled tasks based on spring task are the simplest and fastest, while the difficulty of XXL job is mainly reflected in integration and debugging. No matter what kind of scheduled tasks, you need to ensure that:

• Tasks will not be executed multiple times due to cluster deployment.
• Exceptions in the task are handled effectively
• The slow processing of tasks leads to a large backlog
• The task should be performed at the expected point in time

Middleware can decouple services, but it increases the complexity

Welcome to my knowledge planet to discuss architecture and exchange source code. Join method: long press the QR code below:

The source code has been updated on the knowledge planet. The analysis is as follows:

Recently updated the "Introduction to taro SpringBoot 2.X" series, which has more than 101 articles, covering MyBatis, Redis, MongoDB, ES, sub database and sub table, read-write separation, SpringMVC, Webflux, permissions, WebSocket, Dubbo, RabbitMQ, RocketMQ, Kafka, performance test, etc.

Provide SpringBoot examples with nearly 3W lines of code and e-commerce micro service projects with more than 4W lines of code.

The way of obtaining: click "see", pay attention to the official account and reply to 666, and more contents will be offered.

If the article is helpful, read it and forward it. Thank you for your support (*^__^*)