Preface
RocketMQ provides transaction messages to resolve issues where the program rolls back abnormally but the message has been sent, such as service B needs to modify user data after service A inserts a data, service A sends a message and program exception causes data insertion rollback, while service B listens for messages and modifies the data, causing data problems
Environmental Science
Spring Cloud Hoxton.SR9 + Spring Cloud Alibaba 2.2.6.RELEASE + RocketMQ 4.7.0
Distributed Transaction Message Flow
Flow chart
Process resolution
- Step 1: The producer sends a half message to the MQ Server (special messages are stored in the MQ Server and marked as temporarily undeliverable) which the consumer will not receive.
- Step 23: When the half-message is successfully sent, the producer goes to the local transaction
- Step 4: The producer sends a second confirmation request to the MQ Server based on the execution status of the local transaction. If the MQ Server receives a commit, it marks the half message as deliverable, which the consumer can consume, and if it receives a rollback, it deletes the half message
- Step 5: If the second confirmation in step 4 fails to be successfully sent to the MQ Server, over time, the MQ Server sends a lookup message to the producer to get the execution status of the local transaction
- Step 6: Producer checks local transaction execution status
- Step 7: The producer tells the MQ Server whether it should commit or rollback based on the results of the local transaction, if it is commit then delivers the message to the consumer and if it is rollback discards the message
Note:
Step 1234 is a secondary acknowledgment mechanism in which the producer sends the message to MQ, which is marked not to consume the message, the producer executes local transactions, and posts or discards the message based on the execution status.
Step 567 is the fault tolerant processing MQ did not receive a second confirmation
Transaction message three states
- Commit: Submit a transaction message that consumers can consume
- Rollback: Rollback transaction message, broker deletes it, consumer cannot consume
- UNKNOWN: broker needs to check back to confirm the status of the message
Specific implementation
Implementation Code
Problem scenario: User center needs to modify user data after content center inserts a data, and program exception after content center sends a message causes data insertion to roll back, while user center listens for message and modifies the data, which results in inconsistency of data. The following will verify how this scenario is handled with distributed transaction message from RocketMQ
Content Center
- Table structure
CREATE TABLE `test` ( `id` int(11) NOT NULL AUTO_INCREMENT, `title` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci COMMENT='Insert Data Test Table' CREATE TABLE `rocketmq_transaction_log` ( `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'id', `transaction_Id` varchar(45) COLLATE utf8_unicode_ci NOT NULL COMMENT 'affair id', `log` varchar(45) COLLATE utf8_unicode_ci NOT NULL COMMENT 'Journal', PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci COMMENT='RocketMQ Transaction Log Table'
- TestRocketController.java
@PostMapping("test1")
public Test test1() {
return testService.insertTest();
}
- TestService.java
import com.alibaba.csp.sentinel.annotation.SentinelResource;
import lombok.RequiredArgsConstructor;
import org.apache.rocketmq.spring.core.RocketMQTemplate;
import org.apache.rocketmq.spring.support.RocketMQHeaders;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.UUID;
@Service
@RequiredArgsConstructor(onConstructor = @__(@Autowired))
public class TestService {
private final TestMapper testMapper;
private final RocketMQTemplate rocketMQTemplate;
private final RocketmqTransactionLogMapper rocketmqTransactionLogMapper;
public Test insertTest() {
Test test = Test.builder()
.title("The world is as short as a dream in spring. A dream in spring is traceless. For example, a dream in spring, a dream in which the Yellow sorghum and uncooked banana deer go away")
.build();
/**
* Send Half Message Correspondence Step One
* Parameter 1:Topic
* Parameter 2: Message body
* header can be set and passed as a parameter
* Parameter 2:arg can be passed as a parameter
*/
rocketMQTemplate.sendMessageInTransaction(
"add-test",
MessageBuilder.withPayload(test)
.setHeader(RocketMQHeaders.TRANSACTION_ID, UUID.randomUUID().toString())
.build(),
test
);
return test;
}
/**
* Insert data and log transactions
*/
@Transactional(rollbackFor = Exception.class)
public void insertTestDataWithRocketMqLog(Test test, String transactionId) {
this.insertTestData(test);
rocketmqTransactionLogMapper.insertSelective(
RocketmqTransactionLog.builder()
.transactionId(transactionId)
.log("Inserted a strip Test data...")
.build()
);
}
/**
* Insert Test Data
* @param test
*/
@Transactional(rollbackFor = Exception.class)
public void insertTestData(Test test) {
testMapper.insertSelective(test);
}
}
- TestMapper.java
public interface TestMapper extends Mapper<Test> {
}
- RocketmqTransactionLogMapper.java
public interface RocketmqTransactionLogMapper extends Mapper<RocketmqTransactionLog> {
}
- Test.java
@AllArgsConstructor
@NoArgsConstructor
@Builder
@Data
@Table(name = "test")
public class Test {
/**
* id
*/
@Id
@GeneratedValue(generator = "JDBC")
private Integer id;
/**
* Title
*/
private String title;
}
- RocketmqTransactionLog.java
@AllArgsConstructor
@NoArgsConstructor
@Builder
@Data
@Table(name = "rocketmq_transaction_log")
public class RocketmqTransactionLog {
/**
* id
*/
@Id
@GeneratedValue(generator = "JDBC")
private Integer id;
/**
* Transaction id
*/
@Column(name = "transaction_Id")
private String transactionId;
/**
* Journal
*/
private String log;
}
- AddTestTransactionListener.java
import lombok.RequiredArgsConstructor;
import org.apache.rocketmq.spring.annotation.RocketMQTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionState;
import org.apache.rocketmq.spring.support.RocketMQHeaders;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageHeaders;
import java.util.Objects;
/**
* Transaction monitoring
*/
@RocketMQTransactionListener
@RequiredArgsConstructor(onConstructor = @__(@Autowired))
public class AddTestTransactionListener implements RocketMQLocalTransactionListener {
private final TestService testService;
private final RocketmqTransactionLogMapper rocketmqTransactionLogMapper;
/**
* Perform local transactions, corresponding to step 3
* @param message
* @param o
* @return
*/
@Override
public RocketMQLocalTransactionState executeLocalTransaction(Message message, Object o) {
MessageHeaders headers = message.getHeaders();
String transactionId = (String) headers.get(RocketMQHeaders.TRANSACTION_ID);
try {
testService.insertTestDataWithRocketMqLog((Test) o, transactionId);
return RocketMQLocalTransactionState.COMMIT;
} catch (Exception e) {
return RocketMQLocalTransactionState.ROLLBACK;
}
}
/**
* Local transaction review, step 6
* @param message
* @return
*/
@Override
public RocketMQLocalTransactionState checkLocalTransaction(Message message) {
MessageHeaders headers = message.getHeaders();
String transactionId = (String) headers.get(RocketMQHeaders.TRANSACTION_ID);
// Transaction review based on record
RocketmqTransactionLog transactionLog = rocketmqTransactionLogMapper.selectOne(
RocketmqTransactionLog.builder()
.transactionId(transactionId)
.build()
);
// Local transaction executed successfully
if (Objects.nonNull(transactionLog)) {
return RocketMQLocalTransactionState.COMMIT;
}
// Local transaction execution failed
return RocketMQLocalTransactionState.ROLLBACK;
}
}
User Center
- TestRocketConsumer.java
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Service;
@Service
@Slf4j
@RocketMQMessageListener(consumerGroup = "consumer-group", topic = "add-test")
public class TestRocketConsumer implements RocketMQListener<Test> {
@Override
public void onMessage(Test test) {
// TODO Business Processing
try {
log.info("The subject of the monitoring is'add-test'Message:" + new ObjectMapper().writeValueAsString(test));
log.info("You can start your business now");
} catch (JsonProcessingException e) {
e.printStackTrace();
}
}
}
test
- As shown in the diagram, when the execution state of a local transaction has not been sent to MQ Server after data insertion, the service is kill ed by simulating a service downtime
- Kill Content Center Process
- The execution status of local transactions is not sent to MQ Server at this time, messages in MQ Server are not delivered to the user center, and no messages are received by the user center for subsequent business processing, as shown below. Enter the local transaction review after restarting the application
- User Center normally listens for messages for business processing after local transaction review
- At this point, the implementation of the RocketMQ distributed transaction message has been completed
- End - ﹀ ﹀ ﹀ Lyrics at Risk Point Zanga Collection
