1, Principle of RocketMQ transaction message:
RocketMQ implements complete transaction messages after version 4.3. The MQ based distributed transaction scheme essentially encapsulates the local message table. The overall process is consistent with the local message table. The only difference is that the local message table is stored in MQ instead of the business database, Transaction messages solve the atomicity problem between message sending at the production end and local transaction execution. The boundary here must be clear. It is to ensure that the MQ production end sends messages correctly without multiple or missing messages. As for whether there are normal consumption messages at the consumer end after sending, this abnormal scenario will be guaranteed by the MQ message consumption failure retry mechanism.
The two-way communication between broker and producer in RocketMQ design enables broker to act as a transaction coordinator naturally; The storage mechanism provided by RocketMQ itself provides persistence for transaction messages; RocketMQ's high availability mechanism and reliable message design are that transaction messages can still ensure the final consistency of transactions in case of system exceptions.
1. The principle of RocketMQ to achieve transaction consistency:
Note: the rollback of local transactions depends on the ACID feature of the local DB. The successful consumption of subscribers is guaranteed by the failure retry mechanism of MQ Server.
(1) Normal condition: when the service of the transaction initiator is normal and there is no failure, the message sending process is as follows:
Step ①: the MQ sender sends a half message to the MQ Server. The MQ Server marks the message as prepared. At this time, the MQ subscriber of the message cannot consume it
Step ②: after the MQ Server successfully persists the message, it confirms to the sender that the message has been successfully received by ACK
Step ③: the sender starts executing the local transaction logic
Step ④: the sender submits a second confirmation, commit or rollback, to the MQ Server according to the local transaction execution result
Final step: if the MQ Server receives a commit operation, mark the semi message as deliverable, and the MQ subscriber will eventually receive the message; If you receive a rollback operation, delete the half half message, and the subscriber will not accept the message; If the local transaction execution result does not respond or times out, MQ Server checks the transaction status. See the exception description in step (2) for details.
(2) Exception: in case of network disconnection or application restart, the second confirmation timeout submitted in step ④ in the figure does not reach the MQ Server. The processing logic is as follows:
Step ⑤: MQ Server checks the message back
Step ⑥: after receiving the message, the sender checks the local transaction execution result of the message
Step ⑦: the sender submits the secondary confirmation again according to the final status of the local transaction.
Final step: MQ Server delivers or deletes messages based on commit/rollback
2. Implementation process of RocketMQ transaction message:
Taking rocketmq version 4.5.2 as an example, transaction messages have a dedicated queue RMQ_SYS_TRANS_HALF_TOPIC, all prepare messages are put here first. When the message receives the Commit request, it will be transferred to the real topic queue for consumption by the Consumer and sent to RMQ_SYS_TRANS_OP_HALF_TOPIC inserts a message. The simple flow chart is as follows:
When the transaction of the application module cannot respond immediately due to interruption or other network reasons, RocketMQ will process it as unknown. This RocketMQ transaction message provides a remedy: regularly check the transaction execution status of the transaction message. The simple flow chart is as follows:
2, Springboot integrates RocketMQ to implement transaction messages:
This section will introduce how SpringBoot integrates RocketMQ and uses transaction messages to ensure final consistency from the case of "placing an order + deducting inventory". The core idea is that the order service (production side) sends an inventory deduction message to RocketMQ, and then executes the local order generation logic. Finally, RocketMQ notifies the inventory service to deduct inventory and ensure that the inventory deduction message is consumed normally.
The services used in the case are divided into two parts: order Service and inventory Service; There are three database tables involved: order table, storage table and local transaction status table. Because these tables are relatively simple, the corresponding table creation statements will not be pasted here, and the corresponding Pojo object, Dao layer and Service layer codes will not be pasted. Only the codes of the core logic are shown below.
1. Start RocketMQ server:
Please refer to this article for the installation and deployment of RocketMQ: https://blog.csdn.net/a745233700/article/details/122531859
2. Introduce dependency in parent pom file:
<!-- rocketmq Transaction message -->
<dependency>
<groupId>org.apache.rocketmq</groupId>
<artifactId>rocketmq-spring-boot-starter</artifactId>
<version>2.2.1</version>
</dependency>3. Production end code:
The core logic of the production side is to deliver transaction messages to RocketMQ, execute local transactions, and finally notify RocketMQ of the execution results of local transactions
(1) RocketMQ related configurations:
In application Add the following configuration to the properties configuration file:
rocketmq.name-server=172.28.190.101:9876 rocketmq.producer.group=order_shop
(2) Create a listening class:
Implement the TransactionListener interface, and simulate the results in the implemented database transaction submission method executelocetransaction() and check transaction status method checkLocalTransaction()
/**
* rocketmq Transaction message callback class
*/
@Slf4j
@Component
public class OrderTransactionListener implements TransactionListener
{
@Resource
private ShopOrderMapper shopOrderMapper;
/**
* half After the message is sent successfully, this method is called back to execute the local transaction
*
* @param message For the returned message, the unique Id of the message can be obtained by using transactionId
* @param arg Parameters passed when calling the send method. If additional parameters can be passed to the send method when sending, you can get them here
* @return Return transaction status, COMMIT: COMMIT ROLLBACK: ROLLBACK unknown: callback
*/
@Override
@Transactional
public LocalTransactionState executeLocalTransaction(Message message, Object arg)
{
log.info("Start local transaction: order information:" + new String(message.getBody()));
String msgKey = new String(message.getBody());
ShopOrderPojo shopOrder = JSONObject.parseObject(msgKey, ShopOrderPojo.class);
int saveResult;
LocalTransactionState state;
try
{
//When modified to true, simulate local transaction exceptions
boolean imitateException = true;
if(imitateException)
{
throw new RuntimeException("An exception was thrown while updating the local transaction");
}
// When generating orders, the rollback of local transactions depends on the ACID feature of DB, so the Transactional annotation needs to be added. When the local transaction fails to commit, rollback is returned_ Message, the half message in rocketMQ will be rolled back to ensure the consistency of distributed transactions.
saveResult = shopOrderMapper.insert(shopOrder);
state = saveResult == 1 ? LocalTransactionState.COMMIT_MESSAGE : LocalTransactionState.ROLLBACK_MESSAGE;
// Update the local transaction and persist the transaction number to prepare for subsequent idempotents
// TransactionDao.add(transactionId)
}
catch (Exception e)
{
log.error("Local transaction execution exception, exception information:", e);
state = LocalTransactionState.ROLLBACK_MESSAGE;
}
//When it is modified to true, the local transaction timeout is simulated. For the timeout message, rocketmq will call the checkLocalTransaction method to check the execution status of the local transaction
boolean imitateTimeout = false;
if(imitateTimeout)
{
state = LocalTransactionState.UNKNOW;
}
log.info("Local transaction execution result: msgKey=" + msgKey + ",execute state:" + state);
return state;
}
/**
* Back check local transaction interface
*
* @param messageExt Determine the local transaction execution status of this message by obtaining the transactionId
* @return Return transaction status, COMMIT: COMMIT ROLLBACK: ROLLBACK unknown: callback
*/
@Override
public LocalTransactionState checkLocalTransaction(MessageExt messageExt)
{
log.info("Call back to the local transaction interface: msgKey=" + new String(messageExt.getBody()));
String msgKey = new String(messageExt.getBody());
ShopOrderPojo shopOrder = JSONObject.parseObject(msgKey, ShopOrderPojo.class);
// Note: the unique ID should be used here to query whether the local transaction is executed successfully. The unique ID can use the transactionId of the transaction. However, for the convenience of verification, you can only query whether there are corresponding records in the order table of DB
// TransactionDao.isExistTx(transactionId)
List<ShopOrderPojo> list = shopOrderMapper.selectList(new QueryWrapper<ShopOrderPojo>()
.eq("shop_id", shopOrder.getShopId())
.eq("user_id", shopOrder.getUserId()));
LocalTransactionState state = list.size() > 0 ? LocalTransactionState.COMMIT_MESSAGE : LocalTransactionState.ROLLBACK_MESSAGE;
log.info("Call to query the execution result of the local transaction interface:" + state);
return state;
}
}To facilitate verification, the above Demo uses two boolean variables, imitateException and imitateTimeout, to simulate transaction execution exceptions and timeouts respectively. You only need to set the boolean value to true.
(3) Post transaction message:
import com.alibaba.fastjson.JSONObject;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.producer.SendStatus;
import org.apache.rocketmq.client.producer.TransactionMQProducer;
import org.apache.rocketmq.client.producer.TransactionSendResult;
import org.apache.rocketmq.spring.core.RocketMQTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;
@Slf4j
@Service
public class ShopOrderServiceImpl extends ServiceImpl<ShopOrderMapper, ShopOrderPojo> implements ShopOrderService
{
@Resource
private RocketMQTemplate rocketMQTemplate;
@Autowired
private OrderTransactionListener orderTransactionListener;
/**
* Send transaction message
*/
@Override
public boolean sendOrderRocketMqMsg(ShopOrderPojo shopOrderPojo)
{
String topic = "storage";
String tag = "reduce";
// Set the listener. If other versions of MQ are used here, strong conversion exceptions may be caused
((TransactionMQProducer) rocketMQTemplate.getProducer()).setTransactionListener(orderTransactionListener);
//Build message body
String msg = JSONObject.toJSONString(shopOrderPojo);
org.springframework.messaging.Message<String> message = MessageBuilder.withPayload(msg).build();
//Send a transaction message for the consumer to reduce inventory
TransactionSendResult sendResult = rocketMQTemplate.sendMessageInTransaction(topic + ":" + tag , message, null);
log.info("Send transaction msg result: " + sendResult);
return sendResult.getSendStatus() == SendStatus.SEND_OK;
}
}4. Consumer code:
The core logic of the consumer is to listen to MQ and receive messages; Deduct inventory after receiving message
(1) RocketMQ related configurations:
In application Add the following configuration to the properties configuration file:
rocketmq.name-server=172.28.190.101:9876 rocketmq.consumer.group=order_shop
(2) Consumer monitoring class:
import com.alibaba.fastjson.JSONObject;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Component;
import javax.annotation.Resource;
/**
* Inventory management consumer
**/
@Component
@RocketMQMessageListener (consumerGroup = "order_storage", topic = "storage")
public class StorageConsumerListener implements RocketMQListener<String>
{
@Resource
private TStorageService tStorageService;
/**
* rocketMQ consumer
*/
@Override
public void onMessage(String message)
{
System.out.println("Consumers begin to consume: from MQ The message obtained in is:" + message);
ShopOrderPojo shopOrder = JSONObject.parseObject(message, ShopOrderPojo.class);
// 1. Idempotent check to prevent repeated consumption of messages -- relevant code logic is omitted here:
// TransactionDao.isExistTx(transactionId)
// 2. Perform message consumption operation -- reduce commodity inventory:
TStoragePojo shop = tStorageService.getById(shopOrder.getShopId());
shop.setNum(shop.getNum() - 1);
boolean updateResult = tStorageService.updateById(shop);
// 3. Add transaction operation record -- Code omitted this time:
// TransactionDao.add(transactionId)
System.out.println("Consumer completed the operation result:" + updateResult);
}
}So far, the final consistency of a complete distributed transaction based on RocketMQ transaction message is completed.
Reference article: https://www.cnblogs.com/huangying2124/p/11702761.html