RabbiteMq property
1. Why do you want to confirm the message?
2. What is the rabbitmq message confirmation mechanism?
3. How does the sender confirm that the message is sent successfully? What is a successful sending?
4. How does the consumer inform rabbitmq of the success or failure of consumption?
reflection
1. Why do you want to confirm the message?
The word "lost message" is often heard. For the previous demo, there is a hidden danger of losing message
The sender can't confirm whether the transmission is successful, and the consumer can't feed back if the processing fails
If there is no message confirmation mechanism, the message will disappear inexplicably and I don't know what the situation is
2. What is the rabbitmq message confirmation mechanism?
First, let's look at the introduction of message confirmation on the official website http://www.rabbitmq.com/confirms.html
Networks can fail in less-than-obvious ways and detecting some failures takes time. Therefore a client that's written a protocol frame or a set of frames (e.g. a published message) to its socket cannot assume that the message has reached the server and was successfully processed. It could have been lost along the way or its delivery can be significantly delayed.
The network may fail in a less obvious way, and it takes time to detect some failures. Therefore, if the client writes one or a set of protocol frames (for example, published messages) in its socket, it cannot be assumed that the message has arrived at the server and been processed successfully. It may be lost en route or severely delayed in transit.
Using standard AMQP 0-9-1, the only way to guarantee that a message isn't lost is by using transactions – make the channel transactional then for each message or set of messages publish, commit. In this case, transactions are unnecessarily heavyweight and decrease throughput by a factor of 250. To remedy this, a confirmation mechanism was introduced. It mimics the consumer acknowledgements mechanism already present in the protocol.
Using the standard AMQP 0-9-1, the only way to ensure that messages are not lost is to use transactions -- and then publish and commit each message or group of messages to make the channel transactional. In this case, the transaction throughput will be 250 times lower than necessary. In order to solve this problem, a confirmation mechanism is introduced. It imitates the existing consumer confirmation mechanism in the protocol
To enable confirms, a client sends the confirm.select method. Depending on whether no-wait was set or not, the broker may respond with a confirm.select-ok. Once the confirm.select method is used on a channel, it is said to be in confirm mode. A transactional channel cannot be put into confirm mode and once a channel is in confirm mode, it cannot be made transactional.
To enable confirm, the client sends confirm Select method. Depending on whether no wait is set, the agent may respond to confirm select-ok. Once confirmed, use confirm on the channel Select, which is called in confirmation mode. The transaction channel cannot be set to the confirmation mode, and once the channel is set to the confirmation mode, it cannot be set to the transaction channel. In short, the physical object and the confirmation mechanism cannot be used at the same time
Once a channel is in confirm mode, both the broker and the client count messages (counting starts at 1 on the first confirm.select). The broker then confirms messages as it handles them by sending a basic.ack on the same channel. The delivery-tag field contains the sequence number of the confirmed message. The broker may also set the multiple field in basic.ack to indicate that all messages up to and including the one with the sequence number have been handled.
Once the channel is in confirm mode, both broker and client will count the messages (counting from 1 on the first confirm.select). The broker then processes the message by sending basic ack . On the same channel. The delivery tag field contains the serial number of the confirmed message. The agent can also set multiple fields in basic.ack to indicate that all messages have been processed until and including messages with serial numbers
Click here to see the official example
3. What is the success or failure of the message sent? How to confirm?
Judging the success or failure of a message actually depends on the timing of message confirmation, because the result will be told to the sender after success or failure
When Will Published Messages Be Confirmed by the Broker?
For unroutable messages, the broker will issue a confirm once the exchange verifies a message won't route to any queue (returns an empty list of queues). If the message is also published as mandatory, the basic.return is sent to the client before basic.ack. The same is true for negative acknowledgements (basic.nack).
For non routable messages, once the switch verifies that the message will not be routed to any queue (returning an empty queue list), the broker will issue an acknowledgement. If the message is also forced to be published, basic Return in basic ACK is sent to the client before. The same is true for negative confirmation (basic.nack).
For routable messages, the basic.ack is sent when a message has been accepted by all the queues. For persistent messages routed to durable queues, this means persisting to disk. For mirrored queues, this means that all mirrors have accepted the message.
For routable messages, send basic when the message is accepted by all queues ack. For persistent messages routed to persistent queues, this means persistent to disk. For the mirror queue, this means that all mirrors have accepted the message
4. How does the consumer inform rabbitmq of the success or failure of consumption?
According to different confirmation modes of consumers, the confirmation time is also different
Automatic confirmation will be confirmed immediately after the message is sent to the consumer. If it is manual, it will be confirmed when the consumer calls ACK, NACK and reject
Manual mode is usually set, and some operations can be carried out after business failure
Sender confirmation
We learned from the official website that it is difficult to ensure that messages can be sent successfully when they are sent to the queue during message production: in this way, we need a certain mechanism for data accuracy or security. The official website also said that using transactions can help us solve such problems, and rabbitmq also supports transactions, but transactions will reduce the concurrency of rabbitmq by 250 times, That's why we never use business in our work. Since we don't use it, we won't study it. Let's look at other mechanisms:
Summary:
confirm mechanism:
- Confirmation of message
- Add a listener at the production end
- Ensure that the message is successfully sent to the mq server
return mechanism: - Used to process some non routable messages (unreachable messages)
- Confirm message confirmation mechanism
Message confirmation refers to that the producer sends a message to the broker, and then the broker will give a reply to the producer;
The producer receives the response to determine whether the message is normally sent to the broker. This method is also the core guarantee of reliable message delivery
2.Return Listener is used to process some non routable messages;
The message producer sends the message to a queue by specifying an Exchange and Routingkey, and then the consumer listens to the queue for consumption processing;
However, in some cases, if the current exchange does not exist or the specified routing key cannot be reached when we send a message, we need to Return Listener if we need to listen to such unreachable messages!
Code implementation confirm
Sender's confirmation on:
In fact, the code has been released when configuring the connection above, which is a line of code annotated in the connection factory:
connectionFactory.setPublisherConfirms(true);
In case of yml configuration:
spring:
rabbitmq:
publisher-confirms: true
Implement a callback class
public class MyConfirmCallback implements RabbitTemplate.ConfirmCallback {
Logger logger = LoggerFactory.getLogger(MyConfirmCallback.class);
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.out.println(ack);
logger.info("confirmcallback:correlationData={} ,ack={} , cause= {}",correlationData,ack,cause);
}
}
Set it in RabbitmqTemplate
template.setConfirmCallback(new MyConfirmCallback());
Moreover, we can attach a CorrelationData parameter when sending messages. This object can set an id, which can be your business id, so as to facilitate corresponding operations
public void testSend() {
//As for why this API is called, it will be explained later
//Parameter introduction: switch name, route establishment, message content
rabbitTemplate.convertAndSend("directExchange", "direct.key", "hello11",new CorrelationData(UUID.randomUUID().toString()));
}
Print out the effect:
true 2021-02-06 10:20:09.452 INFO 37444 --- [nectionFactory1] c.w.r.callback.MyConfirmCallback : confirmcallback:correlationData=CorrelationData [id=16e3e463-2fb2-4922-9148-41d964175d86] ,ack=true , cause= null
return implementation
Note that the sender confirmation mode also needs to be enabled when using the failure callback. The startup method is shown below
Change RabbitmqTemplate:
//Enable mandatory mode (failed callback)
template.setMandatory(true);
//Specifies the implementation class of the failed callback interface
template.setReturnCallback(new MyReturnCallback());
Implementation class of callback interface:
Implement rabbittemplate The returnedMessage method in returncallback can be used. It will pass relevant parameters to you
public class MyReturnCallback implements RabbitTemplate.ReturnCallback {
Logger logger = LoggerFactory.getLogger(MyReturnCallback.class);
@Override
public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
logger.info("message={} , repylCode={},replyText={}, exchange={},routingkey={}",message,replyCode,replyText,exchange,routingKey);
}
}
A failed sending is simulated here: when the specified switch cannot route the message to the queue (if there is no specified route, or the specified right button does not bind the corresponding queue, or there is no binding queue at all, the message will fail to send. The effect is as follows:
2021-02-06 10:34:44.456 INFO 60952 --- [nectionFactory1] c.w.r.callback.MyReturnCallback : message=(Body:'hello11' MessageProperties [headers={spring_returned_message_correlation=2887149f-b208-4d07-b512-75283fc201a9}, contentType=text/plain, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, deliveryTag=0]) , repylCode=312,replyText=NO_ROUTE, exchange=directExchange,routingkey=direct.key111
true
2021-02-06 10:34:44.459 INFO 60952 --- [nectionFactory2] c.w.r.callback.MyConfirmCallback : confirmcallback:correlationData=CorrelationData [id=2887149f-b208-4d07-b512-75283fc201a9] ,ack=true , cause= null
If confirm and return mechanisms are used, can you guarantee 100% successful delivery
Reliable delivery of messages, 100% guarantee that messages are not lost (large factory high-frequency interview)?
What is reliable delivery at the production end?
Ensure the successful sending of the message
Ensure the successful reception of mq node
The sender receives the confirmation response from the Mq node (confirm mechanism)
The above three points can not ensure 100% successful delivery of messages. We need to design our own compensation mechanism
How to set
Scheme I:
Scheme 1: the advantages are relatively simple. The scheduled task can run down the sent message status
The disadvantages are also obvious. There are a lot of database access, mq is not generally high, and io will become the bottleneck of the system
Scheme II:
Consumer confirmation
Why confirm consumption? By default, consumers will automatically confirm that the message has been consumed when they get the rabbitmq message. In the vernacular, the message will be deleted from the rabbitmq queue, but we will inevitably encounter this situation in actual development. For example, I can't handle it when I get the message. For example, the parameters are wrong, Another example is that there is a problem with my current system and I can't process this message for the time being, but this message has been consumed by you and deleted from the rabbitmq queue. If you can't handle it yourself, then this message will be abandoned. This situation is unreasonable in actual development. Rabbitmq provides a solution to this problem, that is, the confirm mode we mentioned above is just the sender's mode, and this time we're talking about the consumer's mode.
Set the following message confirmation to manual confirmation:
Of course, if we want to configure our consumer listener, we need to first instance the Container of a listener, that is, the Container. Then our listener (one consumer can instance multiple listeners) can specify this Container. Then we only need to configure this Container
First, declare a container and specify the message confirmation as manual confirmation in the container:
@Bean
public SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory(ConnectionFactory connectionFactory){
SimpleRabbitListenerContainerFactory s = new SimpleRabbitListenerContainerFactory();
//The connectionFactory, which is manually configured by ourselves, is injected
s.setConnectionFactory(connectionFactory);
//Set the confirmation mode none manual Auto
//No confirmation mode NONE
//Confirmation mode: MANUAL, AUTO
s.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return s;
}
Consumer modification
package com.hrp.mq.springboot;
import com.rabbitmq.client.Channel;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.Random;
/**
* ecs
*
* @Title: com.hrp.mq.springboot
* @Date: 2020/8/6 20:16
* @Author: wfg
* @Description:
* @Version:
*/
@Component
public class AckConsumer {
//containerFactory: specify the container we just configured
// @RabbitListener(queues = "testQueue1",containerFactory = "simpleRabbitListenerContainerFactory")
public void toMessage(Message message, Channel channel) throws IOException {
System.out.println("Consumer 1============="+new String(message.getBody()));
//Here, you can choose whether to confirm or not according to the success of your business
if (doSomthing()){
/**
* Parameter 1: Message ID, which we can get through Message. This is maintained by RabbitMq
* Parameter 2: batch processing
*/
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}else {
/**
* Of course, if the order processing fails, we also need to tell rabbitmq that the message can be returned if the processing fails
* It can also be abandoned. It should be noted that this message must be notified whether it is successful or not. Even if it fails, rabbitmq will display this message if it is not notified
* If the message is always in an unconfirmed state, the message will always accumulate on rabbitmq. Unless it is disconnected from rabbitmq, it will send this message
* The message is re sent to others, so be sure to remember to notify!
* The first two parameters have the same meaning as the above. The last parameter is whether the message is returned to the original queue or voided
* Just don't return it.
*/
channel.basicNack(message.getMessageProperties().getDeliveryTag(),false,true);
}
}
public boolean doSomthing(){
boolean flag= new Random().nextBoolean();
System.out.println(flag);
return flag;
}
//containerFactory: specify the container we just configured
//@RabbitListener(queues = "testQueue1",containerFactory = "simpleRabbitListenerContainerFactory")
public void toMessage1(Message message, Channel channel) throws IOException {
System.out.println("Consumer 2=============" + new String(message.getBody()));
//Confirm whether your business selection is successful or not
/**
* Parameter 1: Message ID, which we can get through Message. This is maintained by RabbitMq
* Parameter 2: batch processing
*/
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
@RabbitListener(queues = "testQueue",containerFactory = "simpleRabbitListenerContainerFactory")
public void toMessage2(Message message, Channel channel) throws IOException {
System.out.println("Consumer 1============="+new String(message.getBody()));
//Here, you can choose whether to confirm or not according to the success of your business
if (false){
/**
* Parameter 1: Message ID, which we can get through Message. This is maintained by RabbitMq
* Parameter 2: batch processing
*/
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}else {
/**
* Of course, if the order processing fails, we also need to tell rabbitmq that the message can be returned if the processing fails
* It can also be abandoned. It should be noted that this message must be notified whether it is successful or not. Even if it fails, rabbitmq will display this message if it is not notified
* If the message is always in an unconfirmed state, the message will always accumulate on rabbitmq. Unless it is disconnected from rabbitmq, it will send this message
* The message is re sent to others, so be sure to remember to notify!
* The first two parameters have the same meaning as the above. The last parameter is whether the message is returned to the original queue or voided
* Just don't return it.
*/
System.out.println("The message was returned==========");
channel.basicNack(message.getMessageProperties().getDeliveryTag(),false,true);
}
}
}
We can observe rabbitmq's management page and console printing to help us understand the confirmation mode and return messages
