Author: Hai Xiang
source: RabbitMQ delay queue, message delay push - Haixiang - blog Park#

Application scenario #

At present, the common application software has the shadow of delayed message push, which is also widely used, such as:

• Taobao automatically confirms receipt within seven days. After we sign for the goods, the logistics system will delay sending a message to the payment system seven days later to inform the payment system to call the merchant. This process lasts seven days, which uses the delayed push function of message middleware.
• 12306 ticket purchase payment confirmation page. We often have a countdown in the page where we select the ticket and click OK to jump, which means that if the order is not confirmed within 30 minutes, the order will be cancelled automatically. In fact, when the ticket purchase business starts at the moment of placing the order, the system will send a delay message to the order system, delaying 30 minutes to tell the order system that the order has not been completed. If we complete the order within 30 minutes, we can ignore the received information through logic code judgment.

In the above two scenarios, if we use the following two traditional solutions, it will undoubtedly greatly reduce the overall performance and throughput of the system:

• Use redis to set the expiration time for the order. Finally, determine whether the order has been completed by judging whether the order still exists in redis. Compared with the delayed push of messages, the performance of this solution is lower, because we know that redis is stored in memory. When we place an order or swipe an order maliciously, it will put great pressure on the memory.
• Traditional database polling is used to judge the status of orders in database tables, which undoubtedly increases the number of IO and has very low performance.
• Using the jvm's native DelayQueue also consumes a lot of memory, and there is no persistence strategy. Order information will be lost when the system goes down or restarts.

Implementation of message delay push #

Before RabbitMQ 3.6.x, we generally used dead letter queue + TTL expiration time to implement delay queue. We won't introduce it here. You can refer to the previous articles: TTL, dead letter queue

Starting with RabbitMQ 3.6.x, RabbitMQ officially provides a plug-in for delay queue, which can be downloaded and placed in plugins under the root directory of RabbitMQ. Delay queue plug-in download

First, we create the switch and message queue. The configuration in application.properties is the same as that in the previous article.

import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.HashMap;
import java.util.Map;

@Configuration
public class MQConfig {

    public static final String LAZY_EXCHANGE = "Ex.LazyExchange";
    public static final String LAZY_QUEUE = "MQ.LazyQueue";
    public static final String LAZY_KEY = "lazy.#";

    @Bean
    public TopicExchange lazyExchange(){
        //Map<String, Object> pros = new HashMap<>();
        //Set the switch to support delayed message push
        //pros.put("x-delayed-message", "topic");
        TopicExchange exchange = new TopicExchange(LAZY_EXCHANGE, true, false, pros);
        exchange.setDelayed(true);
        return exchange;
    }

    @Bean
    public Queue lazyQueue(){
        return new Queue(LAZY_QUEUE, true);
    }

    @Bean
    public Binding lazyBinding(){
        return BindingBuilder.bind(lazyQueue()).to(lazyExchange()).with(LAZY_KEY);
    }
}

We can set exchange.setDelayed(true) in the declaration of Exchange to open the delay queue, or the following contents can be passed into the method of the declaration of Exchange, because the underlying of the first method is implemented in this way.

         //Map<String, Object> pros = new HashMap<>();
        //Set the switch to support delayed message push
        //pros.put("x-delayed-message", "topic");
        TopicExchange exchange = new TopicExchange(LAZY_EXCHANGE, true, false, pros);

  When sending a message, we need to specify the delay push time. Here, we pass in the parameters in the method of sending the message   new MessagePostProcessor()   To get   Message object, because you need to use   The api of the message object to set the delay time.

import com.anqi.mq.config.MQConfig;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageDeliveryMode;
import org.springframework.amqp.core.MessagePostProcessor;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.util.Date;

@Component
public class MQSender {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    //Confirm callback returncallback code is omitted. Please refer to the previous article
  
    public void sendLazy(Object message){
        rabbitTemplate.setMandatory(true);
        rabbitTemplate.setConfirmCallback(confirmCallback);
        rabbitTemplate.setReturnCallback(returnCallback);
        //id + timestamp globally unique
        CorrelationData correlationData = new CorrelationData("12345678909"+new Date());

        //Specify the header delay time when sending a message
        rabbitTemplate.convertAndSend(MQConfig.LAZY_EXCHANGE, "lazy.boot", message,
                new MessagePostProcessor() {
            @Override
            public Message postProcessMessage(Message message) throws AmqpException {
                //Set message persistence
                message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
                //message.getMessageProperties().setHeader("x-delay", "6000");
                message.getMessageProperties().setDelay(6000);
                return message;
            }
        }, correlationData);
    }
}
We can observe setDelay(Integer i)The underlying code is also in header Medium setting x-delay. Equivalent to our manual setting header

message.getMessageProperties().setHeader("x-delay", "6000");

Copy
/**
 * Set the x-delay header.
 * @param delay the delay.
 * @since 1.6
 */
public void setDelay(Integer delay) {
	if (delay == null || delay < 0) {
		this.headers.remove(X_DELAY);
	}
	else {
		this.headers.put(X_DELAY, delay);
	}
}

Consumer side consumption

import com.rabbitmq.client.Channel;
import org.springframework.amqp.rabbit.annotation.*;
import org.springframework.amqp.support.AmqpHeaders;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.util.Map;

@Component
public class MQReceiver {

    @RabbitListener(queues = "MQ.LazyQueue")
    @RabbitHandler
    public void onLazyMessage(Message msg, Channel channel) throws IOException{
        long deliveryTag = msg.getMessageProperties().getDeliveryTag();
        channel.basicAck(deliveryTag, true);
        System.out.println("lazy receive " + new String(msg.getBody()));

    }

test result

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@SpringBootTest
@RunWith(SpringRunner.class)
public class MQSenderTest {

    @Autowired
    private MQSender mqSender;

    @Test
    public void sendLazy() throws  Exception {
        String msg = "hello spring boot";

        mqSender.sendLazy(msg + ":");
    }
}

Sure enough, I received the message six seconds later   lazy receive hello spring boot:

Reprint:

RabbitMQ delay queue, message delay push - Haixiang - blog Park