RabbitMQ's death queue

Posted by dmort on Fri, 07 Jan 2022 17:19:43 +0100

1 concept of dead letter

first clarify the definition from the conceptual explanation. Dead letter, as the name suggests, is information that cannot be consumed. The literal meaning can be understood as follows. Generally speaking, the producer delivers messages to the broker or directly to the queue, and the consumer takes messages from the queue for consumption, but sometimes some messages in the queue cannot be consumed due to specific reasons, If there is no subsequent processing for such a message, it will become a dead letter. If there is a dead letter, there will be a dead letter queue.
application scenario: in order to ensure that the message data of the order business is not lost, it is necessary to use the dead letter queue mechanism of RabbitMQ to put the message into the dead letter queue when the message consumption is abnormal Another example: after the user successfully places an order in the mall and clicks to pay, it will automatically become invalid if it is not paid within the specified time.

2 source of dead letter

Message TTL expired
The queue has reached the maximum length (the queue is full and can no longer add data to mq)
The message is rejected (basic.reject or basic.nack) and request = false

3 dead letter actual combat

3.1 code architecture diagram

3.2 message TTL expiration

3.2.1 producer code

public class Producer {
    /**
     * Name of common switch
     */
    private static final String NORMAL_EXCHANGE = "normal_exchange";

    public static void main(String[] args) throws IOException, TimeoutException {
        Channel channel = RabbitMQUtils.getChannel();
        //Setting TTL time for dead letter message
        AMQP.BasicProperties properties = new AMQP.BasicProperties().builder().expiration("10000").build();
        for (int i = 1; i <= 10; i++) {
            String message = "info-" + i;
            channel.basicPublish(NORMAL_EXCHANGE, "zhangsan", properties, message.getBytes(StandardCharsets.UTF_8));
        }
    }
}

3.2.2 consumer C1 code (after startup, close the consumer to simulate that it cannot receive a message)

public class Consumer01 {
    //Name of common switch
    private static final String NORMAL_EXCHANGE = "normal_exchange";
    //Name of dead letter switch
    private static final String DEAD_EXCHANGE = "dead_exchange";
    //Name of the normal queue
    private static final String NORMAL_QUEUE = "normal_queue";
    //Name of the dead letter queue
    private static final String DEAD_QUEUE = "dead_queue";

    @SneakyThrows
    public static void main(String[] args) {
        Channel channel = RabbitMQUtils.getChannel();
        //Declare common switch
        channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
        //Declare dead letter switch
        channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
        //Declare normal queue
        Map<String, Object> arguments = new HashMap<>();
        //Expiration time: 10s = 10000ms, which is generally set by the manufacturer, because it can be changed at will
        //arguments.put("x-message-ttl", 10000);
        //Normal queue setting dead letter switch
        arguments.put("x-dead-letter-exchange", DEAD_EXCHANGE);
        //Set the dead letter routingKey to let the dead letter switch distribute messages according to this keyword
        arguments.put("x-dead-letter-routing-key", "lisi");
        channel.queueDeclare(NORMAL_QUEUE, false, false, false, arguments);
        //Declare dead letter queue
        channel.queueDeclare(DEAD_QUEUE, false, false, false, null);
        //Bind common switch and common queue
        channel.queueBind(NORMAL_QUEUE, NORMAL_EXCHANGE, "zhangsan");
        //Bind dead letter switch and dead letter queue
        channel.queueBind(DEAD_QUEUE, DEAD_EXCHANGE, "lisi");
        System.out.println("Consumer01 Waiting to receive message......");
        DeliverCallback deliverCallback = (consumerTag, message) -> {
            String msg = new String(message.getBody(), StandardCharsets.UTF_8);
            System.out.println("Consumer01 Receive message:" + msg);
        };
        channel.basicConsume(NORMAL_QUEUE, true, deliverCallback, consumerTag -> {});
    }
}

The producer did not send a message

The producer sent 10 messages. At this time, there are 10 non consumed messages in the normal message queue

After 10 seconds, the messages in the normal queue enter the dead queue because they are not consumed

3.2.3 consumer C2 code (after the above steps are completed, start C2 consumer to consume the messages in the dead letter queue)

public class Consumer02 {
    //Name of the dead letter queue
    private static final String DEAD_QUEUE = "dead_queue";

    @SneakyThrows
    public static void main(String[] args) {
        Channel channel = RabbitMQUtils.getChannel();
        System.out.println("Consumer02 Waiting to receive message......");
        DeliverCallback deliverCallback = (consumerTag, message) -> System.out.println("Consumer02 Receive message:" + new String(message.getBody(), StandardCharsets.UTF_8));
        channel.basicConsume(DEAD_QUEUE, true, deliverCallback, consumerTag -> {});
    }
}

3.3 maximum queue length

3.3.1 remove the TTL code of the producer

public class Producer {
    /**
     * Name of common switch
     */
    private static final String NORMAL_EXCHANGE = "normal_exchange";

    public static void main(String[] args) throws IOException, TimeoutException {
        Channel channel = RabbitMQUtils.getChannel();
        for (int i = 1; i <= 10; i++) {
            String message = "info-" + i;
            channel.basicPublish(NORMAL_EXCHANGE, "zhangsan", null, message.getBytes(StandardCharsets.UTF_8));
        }
    }
}

3.3.2 C1 consumer modifies the following code (after startup, close the consumer to simulate that it cannot receive a message)

In order to check whether the length of the queue is limited to 6, we can only start C1 and close it, let the messages accumulate in the queue, and then see whether the redundant messages enter the dead queue

Note that the original queue needs to be deleted at this time because the parameters have changed

3.3.3 C2 consumer code remains unchanged (start C2 consumer)

3.4 message rejected

3.4.1 the message producer code is consistent with the producer whose queue reaches the maximum length

3.4.2 C1 consumer code

There is no need to close it after this startup. In order to demonstrate rejection, we do not accept the message with info-5 and start the manual response
I'll paste all the C1 code now. By the way, remember to delete normal_queue, because the configuration has changed

public class Consumer01 {
    //Name of common switch
    private static final String NORMAL_EXCHANGE = "normal_exchange";
    //Name of dead letter switch
    private static final String DEAD_EXCHANGE = "dead_exchange";
    //Name of the normal queue
    private static final String NORMAL_QUEUE = "normal_queue";
    //Name of the dead letter queue
    private static final String DEAD_QUEUE = "dead_queue";

    @SneakyThrows
    public static void main(String[] args) {
        Channel channel = RabbitMQUtils.getChannel();
        //Declare common switch
        channel.exchangeDeclare(NORMAL_EXCHANGE, BuiltinExchangeType.DIRECT);
        //Declare dead letter switch
        channel.exchangeDeclare(DEAD_EXCHANGE, BuiltinExchangeType.DIRECT);
        //Declare normal queue
        Map<String, Object> arguments = new HashMap<>();
        //Normal queue setting dead letter switch
        arguments.put("x-dead-letter-exchange", DEAD_EXCHANGE);
        //Set the dead letter routingKey to let the dead letter switch distribute messages according to this keyword
        arguments.put("x-dead-letter-routing-key", "lisi");
        channel.queueDeclare(NORMAL_QUEUE, false, false, false, arguments);
        //Declare dead letter queue
        channel.queueDeclare(DEAD_QUEUE, false, false, false, null);
        //Bind common switch and common queue
        channel.queueBind(NORMAL_QUEUE, NORMAL_EXCHANGE, "zhangsan");
        //Bind dead letter switch and dead letter queue
        channel.queueBind(DEAD_QUEUE, DEAD_EXCHANGE, "lisi");
        System.out.println("Consumer01 Waiting to receive message......");
        DeliverCallback deliverCallback = (consumerTag, message) -> {
            String msg = new String(message.getBody(), StandardCharsets.UTF_8);
            if ("info-5".equals(msg)) {
                System.out.println("Consumer01 Waiting for message:" + msg + ": This message was C1 Rejected");
                channel.basicReject(message.getEnvelope().getDeliveryTag(), false);
            } else {
                System.out.println("Consumer01 Receive message:" + msg);
                channel.basicAck(message.getEnvelope().getDeliveryTag(), false);
            }
        };
        //Turn on manual response when rejecting
        channel.basicConsume(NORMAL_QUEUE, false, deliverCallback, consumerTag -> {});
    }
}

After the producer sends a message

3.4.3 C2 consumer code unchanged

Start consumer 1 and see that info-5 is rejected and enters the dead queue

Then start consumer 2

OK, that's all for today. Next, there's the delay queue, but it's more convenient to use SpringBoot to demonstrate the delay queue

Topics: Java RabbitMQ

Programmer Think