A pit about the client (consumer) opening the automatic response and restarting the "lost unprocessed messages". (Mainly lack of understanding of RabbitMQ)

First of all, to state that the so-called "lost message" in this article does not refer to the loss of message in memory due to server downtime, restart and other reasons, that is to say, it is not about message persistence.

 

Use C # to write tests.

Question Representation: Consumers turn on automatic response, and at some point, consumers drop the line (shutdown/collapse, etc.), then restart consumers, and find that the consumer's unfinished message is lost.

Conditions: The server is not down and restarted, only one consumer and one producer.

Message Flow: Message - > Producer - > Exchange - > Queue - > Consumer

How to deal with the problem: The consumer opens the manual response, and if the situation occurs before, the message will not be lost.

Give me a code first.

The producer code is as follows:

static void Main(string[] args)
        {
            var factory = new ConnectionFactory() { HostName = "localhost" };
            using (var connection = factory.CreateConnection())
            using (var channel = connection.CreateModel())
            {
　　　　　　　　　 //Declare Broadcast Type Switch
                channel.ExchangeDeclare(exchange: "ex1", type: "fanout");
　　　　　　　　　 //Declare queue
                channel.QueueDeclare(queue: "test1",
                                     durable: false,
                                     exclusive: false,
                                     autoDelete: false,
                                     arguments: null);

                int count = 0;
                
                while (true)
                {
                    count++;
                    var body = Encoding.UTF8.GetBytes(count.ToString());
　　　　　　　　　　　　//Push data up to switch ex1 whose key is p
                    channel.BasicPublish(exchange: "ex1",
                                     routingKey: "p",
                                     basicProperties: null,
                                     body: body);

                    Console.WriteLine($"send msg {count}");
                    System.Threading.Thread.Sleep(1000);
                }
            
            }
        }

 

The consumer code is as follows (turn on automatic response):

static void Main(string[] args)
        {
            var factory = new ConnectionFactory() { HostName = "localhost" };
            using (var connection = factory.CreateConnection())
            using (var channel = connection.CreateModel())
            {
　　　　　　　　　 //Queue and switch binding
                channel.QueueBind(queue: "test1",
                              exchange: "ex1",
                              routingKey: "p");

                var consumer = new EventingBasicConsumer(channel);

                consumer.Received += (model, ea) =>
                {
                    var body = ea.Body;
                    var message = Encoding.UTF8.GetString(body);

                    Console.WriteLine($"Receive message -- {message}");

                    System.Threading.Thread.Sleep(2000);
                };

                channel.BasicConsume(queue: "test1",
                                     autoAck: true,
                                     consumer: consumer);

                Console.ReadLine();
            }

        }

* Switches and queues must be declared before binding.  

 

 

The producer pushes the message to the queue, and the data displayed in the queue as read is the message that has not been consumed.

Push 90 messages:

 

There are 90 unanswered messages in the queue

 

Open up consumers:

It was found that all messages were automatically answered and the queue was empty.

 

Restart consumers:

As expected, it's empty.

 

Summary: Open the consumer (open the automatic response), find that all the messages in the queue with the status of Ready are answered, and the messages in the queue with the status of Ready are empty. The consumer will not receive the message until the consumer has finished processing these messages, shut down the consumer, and then open the consumer. The consumer will not receive the message again, and the problem of losing the message that the consumer has not processed appears.

 

Ninety messages were sent in advance.

Then turn off the automatic response.

 

Open consumers:

 

Message status becomes unacked all at once. Because there is no logic to handle messages manually, there will be no fewer unacked status messages.

 

Then shut down consumers:

RabbitMQ did not delete the unanswered message, and the message reverted to Ready status, waiting for the connection consumer to process.

 

Re-open the consumer:

There was no loss of unprocessed messages.

 

Summary: Open the consumer (turn off the automatic response), find that the status of the message in the queue is changed to unacked, and the status of the message in the queue is empty. With the consumer's response, the status of unacked message in the queue decreases gradually. Close the consumer and find that the status of unacked message in the queue changes back to read state.

　　　　　　 

Conclusion:

Turning off automatic reply prevents such message "loss".

In addition, in the case of automatic response ack=true, it is necessary to ensure that there must be consumers online to ensure that messages are received and processed. Opening a manual response inevitably consumes more resources, because RabbitMQ needs to delete the corresponding message in the queue according to the response number.

 

The above is only personal understanding. If there are any mistakes, please correct them.~