RabbitMQ deferred message implementation

Posted by GYK on Tue, 04 Jan 2022 08:41:25 +0100

What is a delay queue:

   delay queue, first of all, it is a queue. Queue means that the internal elements are orderly, the outgoing and incoming elements are directional, and the elements enter from one end and take out from the other end.

   secondly, the most important feature of delay queue is its delay attribute. Unlike ordinary queue, elements in ordinary queue always wait to be taken out and processed earlier, while elements in delay queue want to be taken out and processed at a specified time. Therefore, elements in delay queue have time attribute, Generally speaking, it is a message or task that needs to be processed.

  in short, a delay queue is a queue used to store elements that need to be processed at a specified time.

Delay queue usage scenario:

  so when do you need to use delay queues? Consider the following scenario:

    if the order is not paid within ten minutes, it will be automatically cancelled.

     if the newly created store has not uploaded goods within ten days, it will automatically send a message reminder.

     if the bill is not paid within one week, it will be settled automatically.

     after the user is registered successfully, if he does not log in within three days, he will be prompted by SMS.

     the user initiates a refund. If it is not handled within three days, notify the relevant operation personnel.

     after the scheduled meeting, all participants shall be notified to attend the meeting ten minutes before the scheduled time point.

Implementation of delay queue:

1. Using TTL + dead letter queue

   the producer produces a delayed message. According to the required delay time, different routingkeys are used to route the message to different delay queues. Each queue has different TTL attributes and is bound to the same dead letter switch. After the message expires, it will be routed to different dead letter queues according to different routingkeys, Consumers only need to listen to the corresponding dead letter queue for processing.

   the disadvantages of this method are that it can not be universal. Every time a new delayed task is carried out, it is troublesome to implement an implemented TTL + dead letter queue;

2. Using RabbitMQ plug-in

  just install a plug-in: https://www.rabbitmq.com/community-plugins.html , Download rabbitmq_delayed_message_exchange plug-in, and then unzip and place it in the plug-in directory of RabbitMQ.

  view the release history and download 3.8 0 version

3. RabbitMQ container configuration plug-in:

   3.1. Enter the container to view the MQ plug-in Directory: plugins/

[root@VM-0-12-centos ~]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@VM-0-12-centos ~]# docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS                      PORTS               NAMES
74f929cfb3c9        rabbitmq:3-management   "docker-entrypoint..."   4 weeks ago         Exited (0) 18 minutes ago                       rabbitMQ
[root@VM-0-12-centos ~]# docker start 74f929cfb3c9
[root@VM-0-12-centos ~]# docker exec -it 74f929cfb3c9 bash
root@74f929cfb3c9:/# ls
bin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  plugins  proc  root  run  sbin  srv  sys  tmp  usr  var
root@74f929cfb3c9:/# cd plugins
root@74f929cfb3c9:/plugins# ls
README                            cuttlefish-3.0.0.ez        ra-1.1.8.ez                             rabbitmq_consistent_hash_exchange-3.8.18.ez  rabbitmq_peer_discovery_consul-3.8.18.ez    rabbitmq_stomp-3.8.18.ez               recon-2.5.1.ez
accept-0.3.5.ez                   eetcd-0.3.3.ez             rabbit-3.8.18.ez                        rabbitmq_event_exchange-3.8.18.ez            rabbitmq_peer_discovery_etcd-3.8.18.ez      rabbitmq_top-3.8.18.ez                 stdout_formatter-0.2.4.ez
amqp10_client-3.8.18.ez           gen_batch_server-0.8.4.ez  rabbit_common-3.8.18.ez                 rabbitmq_federation-3.8.18.ez                rabbitmq_peer_discovery_k8s-3.8.18.ez       rabbitmq_tracing-3.8.18.ez             syslog-3.4.5.ez
amqp10_common-3.8.18.ez           goldrush-0.1.9.ez          rabbitmq_amqp1_0-3.8.18.ez              rabbitmq_federation_management-3.8.18.ez     rabbitmq_prelaunch-3.8.18.ez                rabbitmq_trust_store-3.8.18.ez         sysmon_handler-1.3.0.ez
amqp_client-3.8.18.ez             gun-1.3.3.ez               rabbitmq_auth_backend_cache-3.8.18.ez   rabbitmq_jms_topic_exchange-3.8.18.ez        rabbitmq_prometheus-3.8.18.ez               rabbitmq_web_dispatch-3.8.18.ez
aten-0.5.5.ez                     jose-1.11.1.ez             rabbitmq_auth_backend_http-3.8.18.ez    rabbitmq_management-3.8.18.ez                rabbitmq_random_exchange-3.8.18.ez          rabbitmq_web_mqtt-3.8.18.ez
base64url-1.0.1.ez                jsx-3.1.0.ez               rabbitmq_auth_backend_ldap-3.8.18.ez    rabbitmq_management_agent-3.8.18.ez          rabbitmq_recent_history_exchange-3.8.18.ez  rabbitmq_web_mqtt_examples-3.8.18.ez
cowboy-2.8.0.ez                   lager-3.9.2.ez             rabbitmq_auth_backend_oauth2-3.8.18.ez  rabbitmq_mqtt-3.8.18.ez                      rabbitmq_sharding-3.8.18.ez                 rabbitmq_web_stomp-3.8.18.ez
cowlib-2.9.1.ez                   observer_cli-1.6.2.ez      rabbitmq_auth_mechanism_ssl-3.8.18.ez   rabbitmq_peer_discovery_aws-3.8.18.ez        rabbitmq_shovel-3.8.18.ez                   rabbitmq_web_stomp_examples-3.8.18.ez
credentials_obfuscation-2.4.0.ez  prometheus-4.6.0.ez        rabbitmq_aws-3.8.18.ez                  rabbitmq_peer_discovery_common-3.8.18.ez     rabbitmq_shovel_management-3.8.18.ez        ranch-2.0.0.ez

  3.2 rabbitmq to be downloaded_ delayed_ message_ Put the exchange plug-in under the / home/docker/rabbitmq / path of the server

  3.3. Rabbitmq_ delayed_ message_ Copy the exchange plug-in to the plugins directory of the container and confirm

  command: docker cp local file path container ID or full name: container path

[root@VM-0-12-centos /]# docker cp home/docker/rabbitmq/rabbitmq_delayed_message_exchange-3.8.0.ez 74f929cfb3c9:plugins/
[root@VM-0-12-centos /]# docker exec -it 74f929cfb3c9 bash
root@74f929cfb3c9:/# cd plugins
root@74f929cfb3c9:/plugins# ls
README                            cuttlefish-3.0.0.ez        ra-1.1.8.ez                             rabbitmq_consistent_hash_exchange-3.8.18.ez  rabbitmq_peer_discovery_common-3.8.18.ez    rabbitmq_shovel_management-3.8.18.ez   ranch-2.0.0.ez
accept-0.3.5.ez                   eetcd-0.3.3.ez             rabbit-3.8.18.ez                        rabbitmq_delayed_message_exchange-3.8.0.ez   rabbitmq_peer_discovery_consul-3.8.18.ez    rabbitmq_stomp-3.8.18.ez               recon-2.5.1.ez
amqp10_client-3.8.18.ez           gen_batch_server-0.8.4.ez  rabbit_common-3.8.18.ez                 rabbitmq_event_exchange-3.8.18.ez            rabbitmq_peer_discovery_etcd-3.8.18.ez      rabbitmq_top-3.8.18.ez                 stdout_formatter-0.2.4.ez
amqp10_common-3.8.18.ez           goldrush-0.1.9.ez          rabbitmq_amqp1_0-3.8.18.ez              rabbitmq_federation-3.8.18.ez                rabbitmq_peer_discovery_k8s-3.8.18.ez       rabbitmq_tracing-3.8.18.ez             syslog-3.4.5.ez
amqp_client-3.8.18.ez             gun-1.3.3.ez               rabbitmq_auth_backend_cache-3.8.18.ez   rabbitmq_federation_management-3.8.18.ez     rabbitmq_prelaunch-3.8.18.ez                rabbitmq_trust_store-3.8.18.ez         sysmon_handler-1.3.0.ez
aten-0.5.5.ez                     jose-1.11.1.ez             rabbitmq_auth_backend_http-3.8.18.ez    rabbitmq_jms_topic_exchange-3.8.18.ez        rabbitmq_prometheus-3.8.18.ez               rabbitmq_web_dispatch-3.8.18.ez
base64url-1.0.1.ez                jsx-3.1.0.ez               rabbitmq_auth_backend_ldap-3.8.18.ez    rabbitmq_management-3.8.18.ez                rabbitmq_random_exchange-3.8.18.ez          rabbitmq_web_mqtt-3.8.18.ez
cowboy-2.8.0.ez                   lager-3.9.2.ez             rabbitmq_auth_backend_oauth2-3.8.18.ez  rabbitmq_management_agent-3.8.18.ez          rabbitmq_recent_history_exchange-3.8.18.ez  rabbitmq_web_mqtt_examples-3.8.18.ez
cowlib-2.9.1.ez                   observer_cli-1.6.2.ez      rabbitmq_auth_mechanism_ssl-3.8.18.ez   rabbitmq_mqtt-3.8.18.ez                      rabbitmq_sharding-3.8.18.ez                 rabbitmq_web_stomp-3.8.18.ez
credentials_obfuscation-2.4.0.ez  prometheus-4.6.0.ez        rabbitmq_aws-3.8.18.ez                  rabbitmq_peer_discovery_aws-3.8.18.ez        rabbitmq_shovel-3.8.18.ez                   rabbitmq_web_stomp_examples-3.8.18.ez

   3.4. Next, enter the sbin directory under the RabbitMQ installation directory, execute the following command to make the plug-in effective, and then restart RabbitMQ.

rabbitmq-plugins enable rabbitmq_delayed_message_exchange

root@74f929cfb3c9:/plugins# cd /sbin
root@74f929cfb3c9:/sbin# rabbitmq-plugins enable rabbitmq_delayed_message_exchange
Enabling plugins on node rabbit@74f929cfb3c9:
The following plugins have been configured:
Applying plugin configuration to rabbit@74f929cfb3c9...
The following plugins have been enabled:

started 1 plugins.

RabbitMqConfig configuration:

1. Configure delay queue

 * delayedDirect Switch name
public static final String DELAYED_DIRECT_EXCHANGE="delayedDirectExchange";

 * delayedDirect Queue name
public static final String DELAYED_DIRECT_QUEUE="delayedDirectQueue";

 * delayedDirec The key that the switch binds to the delayedDirec queue
public static final String DELAYED_DIRECT_ROUTINGKEY="delayedDirectRouingkey";

 * Define a delayedDirect switch
 * Custom switch
 * @return
public CustomExchange delayedDirectExchange(){
     * CustomExchange(String name, String type, boolean durable, boolean autoDelete, Map<String, Object> arguments)
     * Description of construction parameters:
     *  name: Switch name, type: user-defined type (the plug-in provides a type called x-delayed-message), durable persistence, autoDelete automatic deletion, and arguments are used to pass other attribute configurations
    Map<String, Object> args = new HashMap<>();

    args.put("x-delayed-type", "direct");

    return new CustomExchange(DELAYED_DIRECT_EXCHANGE,"x-delayed-message", true, false, args);

 * Define a delayedDirect queue
 * @return
public Queue delayedDirectQueue(){
    return new Queue(DELAYED_DIRECT_QUEUE);

 * Define a binding rule between the delayedDirect queue and the delayedDirect switch
 * @return
public Binding delayedDirectBinding(){
    return BindingBuilder.bind(delayedDirectQueue()).to(delayedDirectExchange()).with(DELAYED_DIRECT_ROUTINGKEY).noargs();

2. Producer sending message method

 * Send delayed message
 * @param message news
 * @param delayTime Delay time in milliseconds
public void sendDelayedMessage(String message, Integer delayTime) {
        return messagePostProcessor;

3. Method for consumers to receive messages

@RabbitListener(queues = {RabbitMqConfig.DELAYED_DIRECT_QUEUE}) //listen queue 
public void receiveMessage2(String message) {
    System.out.println("Receive delay message:"+message +" : "+ new Date().toString());

4. Send a message test and close the manual ACK

package com.java996.producer;

import com.java996.producer.Service.RabbitMqService;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;

public class ProducerApp {
    public static void main(String[] args) {
     ConfigurableApplicationContext context = SpringApplication.run(ProducerApp.class,args);
        RabbitMqService rabbitMqService = (RabbitMqService) context.getBean("RabbitMqService");
        rabbitMqService.sendDelayedMessage("Test delay message: 10 seconds",10000);
        rabbitMqService.sendDelayedMessage("Test delay message: 20 seconds",20000);

5. Test result: the consumption time interval between two messages is 10 seconds

Topics: RabbitMQ