Rocketmq distributed transaction processing details

I Related concepts
Based on its message definition, RocketMQ extends two related concepts to transaction messages:

1. Half(Prepare) Message -- half message (preprocessed message)
Semi message is a special message type. Messages in this state cannot be consumed by consumers temporarily. When a transaction message is successfully delivered to the Broker, but the Broker does not receive the secondary confirmation sent by the Producer, the transaction message is in the "temporarily unavailable for consumption" state, and the transaction message in this state is called semi message.

2. Message Status Check - Message Status Check
The secondary confirmation message sent by the Producer to the Broker may not be delivered successfully due to network jitter, Producer restart and other reasons. If the Broker detects that a transaction message is in the semi message state for a long time, it will actively initiate a backcheck operation to the Producer to query the transaction state (Commit or Rollback) of the transaction message at the Producer. It can be seen that Message Status Check is mainly used to solve the timeout problem in distributed transactions.

Step 1: the Producer sends a Half Message to the Broker;
Step 2: Broker ACK, Half Message sent successfully;
Step 3: the Producer executes local transactions;
Step 4: after the local transaction is completed, the Producer sends a secondary confirmation message to the Broker according to the transaction status to confirm the Commit or Rollback status of the Half Message. After the Broker receives the secondary confirmation message, for the Commit status, it will directly send it to the Consumer to execute the consumption logic, while for the Rollback, it will be directly marked as failed, cleared after a period of time, and will not be sent to the Consumer. Under normal circumstances, this distributed transaction has been completed, and the rest to be handled is the timeout problem, that is, the Broker still does not receive the secondary confirmation message from the Producer after a period of time;
Step 5: for the timeout status, the Broker actively initiates a message callback to the Producer;
Step6: Producer processes the query back message and returns the execution result of the corresponding local transaction;
Step 7: the Broker performs a Commit or Rollback operation on the result of the query back message, the same as step 4.

Code example

package com.lazycece.sbac.rocketmq.transaction;

import com.lazycece.sbac.rocketmq.message.Message;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.core.RocketMQTemplate;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Component;

import javax.annotation.Resource;
import java.util.UUID;

/**
 * @author hjt
 * @date 2021/8/6
 */
@Component
@Slf4j
public class TransactionProducer {

    @Resource
    private RocketMQTemplate rocketMQTemplate;

    public void produce() {
        Message<String> message = new Message<>();
        message.setId(UUID.randomUUID().toString());
        message.setContent("transaction message");
        log.info("========sending message=========");
        rocketMQTemplate.sendMessageInTransaction("tx-group", "topic-tx", MessageBuilder.withPayload(message).build(), null);
        log.info("========finish send =========");
    }

}

monitor

package com.lazycece.sbac.rocketmq.transaction;

import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionState;
import org.springframework.messaging.Message;

/**
 * @author hjt
 * @date 2021/8/6
 */
@Slf4j
@RocketMQTransactionListener(txProducerGroup = "tx-group")
public class TransactionListenerImpl implements RocketMQLocalTransactionListener {

    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message msg, Object arg) {

        // Simulated local transaction failed
        log.info("============== Simulated local transaction failed executeLocalTransaction");

        return RocketMQLocalTransactionState.UNKNOWN;
    }

    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message msg) {

        // Simulate backtracking of local transactions
        log.info("============== Simulate backtracking of local transactions checkLocalTransaction");

        return RocketMQLocalTransactionState.COMMIT;
    }
}

among

If the message is in this UNKNOWN state, the simulated local transaction will be queried back. If the status is COMMIT when checking the local transaction, a consumption message will be sent to the consumer.

Consumer end

package com.lazycece.sbac.rocketmq.transaction;

import com.lazycece.sbac.rocketmq.message.Message;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Service;

/**
 * @author hjt
 * @date 2021/8/6
 */
@Slf4j
@Service
@RocketMQMessageListener(topic = "topic-tx", consumerGroup = "tx-consumer-group")
public class TransactionConsumer implements RocketMQListener<Message> {

    @Override
    public void onMessage(Message message) {
        log.info("==========  Consumer News");
        log.info("topic-tx received message: {}", message);
    }

}

results of enforcement

If you listen
Change the status to ROLLBACK

The consumer will not consume messages

If the listening status is COMMIT

The consumer directly consumes the message and does not go through local simulated transaction processing

The code can be sent to me by private letter