How rabbimq ensures data security

Posted by kraadde on Wed, 02 Mar 2022 06:26:09 +0100

I Persistence

RabbitMQ supports message persistence, that is, data is written on disk.
(1) exchange persistence, specifying durable = > 1 when declaring
(2) queue persistence, specifying durable = > 1 when declaring
(3) Message persistence, specifying delivery when delivering_ Mode = > 2 (1 is non persistent)
Note: if the message is persistent and the queue is not persistent, the restart service message will still be lost, but if the exchange is not persistent, it will not have an impact. Therefore, the queue must be persistent while the message is persistent;

II ACK

The above persistence ensures that the data in the queue will not be lost when the server is down. However, when the consumer fails to remove the message in the queue, the server exception will lead to the failure, which will also lead to data loss. The ACK mechanism is to solve this kind of problem,
In RabbitMQ, there are two modes of message acknowledgement:

  1. In automatic mode, we do not need any operation. After the message is received by the consumer, it will be automatically confirmed and the message will be deleted from the queue.
  2. In manual mode, after the message is consumed, we need to call the API provided by RabbitMQ to realize message confirmation.

III Transaction:

Making messages persistent is not a complete guarantee that they will not be lost. Persistence only tells RabbitMq to save the message to the hard disk, but there is still a small interval between RabbitMq receiving the message and saving it. Because RabbitMq does not use synchronous IO for all messages - it may just be saved to the cache and not necessarily written to the hard disk. There is no guarantee of true persistence, but it is enough to cope with our simple work queue. If you must ensure persistence, we need to rewrite the code to support transaction s.

Producer mode transactions:
The transaction will confirm that the message is stored in the hard disk and then submitted;
Consumer mode usage transactions
Assuming that transactions are used in the consumer mode and transaction rollback is performed after message confirmation, what changes will happen to RabbitMQ?
The results are divided into two cases:
autoAck=false. Transactions are supported in manual response. That is to say, even if you have manually confirmed that the message has been received, after the confirmation message will wait for the transaction to be returned and resolved, you decide whether to confirm the message or put it back in the queue. If you roll back the transaction after manually confirming that it is now, the transaction rollback is the main, This message will be put back into the queue;
autoAck=true if the self confirmation is true, the transaction is not supported, that is to say, it is useless to roll back the transaction even after receiving the message. The queue has removed the message;

Confirm mode

summary
Above, we introduced a problem that RabbitMQ may encounter, that is, the generator does not know whether the message really arrives at the broker, and then provides us with a transaction mechanism through the AMQP protocol level to solve this problem. However, the implementation of the transaction mechanism will reduce the message throughput of RabbitMQ. Is there a more efficient solution? The answer is to use Confirm mode.
Implementation principle of confirm mode on producer side
The producer sets the channel to confirm mode. Once the channel enters the confirm mode, all messages published on the channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, the broker will send a confirmation to the producer (including the unique ID of the message), This makes the producer know that the message has arrived at the destination queue correctly. If the message and queue are persistent, the confirmation message will be sent after writing the message to the disk. The delivery tag field in the confirmation message returned by the broker to the producer contains the serial number of the confirmation message. In addition, the broker can also set basic The multiple field of ACK indicates that all messages before this serial number have been processed.

The biggest advantage of confirm mode is that it is asynchronous. Once a message is published, the producer application can continue to send the next message while waiting for the channel to return the confirmation. When the message is finally confirmed, the producer application can process the confirmation message through the callback method. If RabbitMQ loses the message due to its own internal error, A nack message will be sent, and the producer application can also process the nack message in the callback method.

After the channel is set to confirm mode, all subsequent messages publish ed will be confirmed (i.e. ack) or nack once. However, there is no guarantee for the speed of the message being confirmed, and the same message will not be confirmed and nack.

How to turn on confirm mode
The producer sets the channel to confirm mode by calling the confirmSelect method of the channel. If the no wait flag is not set, the broker will return confirm Select OK means that the sender agrees to set the current channel channel to confirm mode (judging from the latest version 3.6 of RabbitMQ, if the channel.confirmSelect method is called, no wait is directly set to false by default, that is, the broker must return confirm.select-ok by default).

confirm Code:

/**
 * producer End confirm mode
 *
 * @date 2019-06-18
 */
public function actionProducerConfirm()
{
    $channel = RabbitmqBase::createChannel();
    $channel->exchange_declare('exchange_confirm', 'direct', false, true,
        false);//This is the declaration exchange. If there is no declaration, it will be sent to the exchange of the default queue, and the default sending type is direct)
    $channel->queue_declare('CQ1', false, true, false, false, false);
    $channel->queue_bind('CQ1', 'exchange_confirm');
    try {
        //Set asynchronous callback message acknowledgement (producer prevents information loss)
        $channel->set_ack_handler(
            function (AMQPMessage $message) {
                echo "Message acked with content " . $message->body . PHP_EOL;
                sleep(1);
            }
        );
        //Post processing method of message failure
        $channel->set_nack_handler(
            function (AMQPMessage $message) {
                throw new UserException($message->body);
            }
        );
        //Select confirm mode (this mode cannot be compatible with transaction mode)
        $channel->confirm_select();
        // send message
        $newMsg = new AMQPMessage('hello test_confirm',
            [
                'delivery_mode' => 2,
                'message_id' => uniqid(),
            ]);
        $channel->basic_publish($newMsg, 'exchange_confirm'); //The queue here is the message name
 
        //Block waiting for message confirmation (producer prevents information loss)
        $channel->wait_for_pending_acks();
    } catch (Exception $e) {
        echo sprintf("fail%s. . . \n", $e->getMessage());
    } finally {
        $channel->close();
        RabbitmqBase::closeConnection();
    }
}
public function actionProducerConfirms()
{
    $channel = RabbitmqBase::createChannel();
    $channel->exchange_declare('exchange_confirm', 'direct', false, true,
        false);//This is the declaration exchange. If there is no declaration, it will be sent to the exchange of the default queue, and the default sending type is direct)
    $channel->queue_declare('CQ1', false, true, false, false, false);
    $channel->queue_bind('CQ1', 'exchange_confirm');
    try {
        //Set asynchronous callback message acknowledgement (producer prevents information loss)
        $channel->set_ack_handler(
            function (AMQPMessage $message) {
                echo "Message acked with content " . $message->body . PHP_EOL;
                sleep(1);
            }
        );
        //Post processing method of message failure
        $channel->set_nack_handler(
            function (AMQPMessage $message) {
                throw new UserException($message->body);
            }
        );
        //Select confirm mode (this mode cannot be compatible with transaction mode)
        $channel->confirm_select();
        // send message
        $newMsg = new AMQPMessage('hello test_confirm',
            [
                'delivery_mode' => 2,
                'message_id' => uniqid(),
            ]);
        $channel->basic_publish($newMsg, 'exchange_confirm'); //The queue here is the message name
 
        //Block waiting for message confirmation (producer prevents information loss)
        $channel->wait_for_pending_acks_returns();
    } catch (Exception $e) {
        echo sprintf("fail%s. . . \n", $e->getMessage());
    } finally {
        $channel->close();
        RabbitmqBase::closeConnection();
    }
}

https://blog.csdn.net/qq_38234594/article/details/93382476

Topics: Database Redis Cache message queue