Distributed transaction Seata principle

Posted by forumnz on Mon, 10 Jan 2022 01:22:51 +0100

1, Introduction to Seata:

1. Introduction to Seata:

Seata is an open source distributed transaction solution, which is committed to providing high-performance and easy-to-use distributed transaction services. It provides users with several different transaction modes such as AT, TCC, SAGA and XA:

  • AT mode: a non intrusive distributed transaction solution, which is suitable for scenarios that do not want to transform the business. However, due to the need to add global transaction locks, it will affect the performance of high concurrency systems. This mode mainly focuses on the data consistency of multi DB access, and also includes the data access consistency of multi DB under multi services
  • TCC mode: a high-performance distributed transaction solution, which is suitable for scenarios with high performance requirements. This mode mainly focuses on business splitting and solves the consistency of calls between services when expanding resources horizontally according to business
  • Saga mode: a distributed transaction solution for long transactions, which is applicable to business systems with long business processes and the need to ensure the final consistency of transactions. Saga mode will submit local transactions in the first stage. It has no lock and can ensure performance in the case of long process. It is mostly used in channel layer and integration layer business systems. Transaction participants can be services of other companies or services of legacy systems. Saga mode can also be used for failure to transform and provide interfaces required by TCC

2. Seata's core components:

There are three main roles in Seata. TM and RM are integrated with the business system as Seata's client, and TC is deployed independently as Seata's server:

  • Transaction Coordinator (TC): maintains the running state of global transactions, coordinates and drives global commit or rollback
  • Transaction manager (TM): the transaction initiator controls the scope of global transactions, is responsible for starting a global transaction, and finally initiates the global commit or rollback resolution
  • Resource Manager (RM): the transaction participant manages the resources being processed by the local transaction, registers the local transaction with the TC, reports the status of the local transaction, and receives the command from the TC to drive the submission or rollback of the local transaction

3. Overall execution process of Seata:

The overall execution mechanism of Seata distributed transaction is shown in the figure above, which can be roughly divided into two stages:

  • ① The initiator TM applies to TC to start a global transaction. The global transaction is successfully created and generates a unique global transaction ID XID, which propagates in the context of the service call link of subsequent transactions (implemented through Aop))
  • ② RM registers Branch transactions with TC, reports resource readiness, and binds with XID (Branch transactions refer to each independent local transaction in distributed transactions)
  • ③ TM initiates a global commit or rollback request for all branch transactions under XID to TC (the end of the first phase of the transaction)
  • ④ TC summarizes transaction information and determines whether distributed transactions are committed or rolled back;
  • ⑤ TC notifies all RM to commit / rollback resources, and the second phase of the transaction ends;

2, AT mode principle of Seata:

Seata AT mode is evolved from XA transaction (XA is a distributed transaction protocol implemented based on database). It needs database support. If it is MySQL, it needs more than 5.6 versions to support XA protocol. The feature of AT mode is that it is non intrusive to the business. Users only need to pay attention to their own business SQL. Seata framework will intercept and parse the user's SQL in the first stage, save the data image before and after the change, form undo log, and automatically generate the commit and roll back operations in the second stage of the transaction.

1. Overall execution process of AT mode:

The behavior of RM driving branch transactions in AT mode is divided into the following two stages:

(1) Execution phase:

  • (1) Proxy JDBC data source, intercept and parse business SQL, generate image data before and after update, and form UNDO LOG.
  • (2) Register branch with TC.
  • (3) After the branch registration is successful, the update of business data and UNDO LOG are submitted in the same local transaction.

(2) Completion phase:

  • Global commit: after receiving the branch commit request from TC, delete the UNDO LOG of the corresponding branch asynchronously.
  • Global rollback: after receiving the branch rollback request from TC, query the UNDO LOG record corresponding to the branch, generate SQL statements to compensate for rollback, execute branch rollback and return the results to TC

2. AT mode two-stage detailed process:

2.1 detailed implementation process of phase I:

In the first stage, when RM writes a table, Seata intercepts the business SQL through the proxy data source (so as to achieve the effect of no intrusion into the business), parses the SQL semantics, finds the business data to be updated in the SQL, saves it as a before image, then executes the business SQL, and saves it as an after image after the business data is updated, Finally, a row lock is generated. By organizing the business data before and after the update into the undo log, and using the ACID characteristics of the local transaction, the update of the business data and the writing of the undo log are committed in the same local transaction, so as to ensure that the update of any submitted business data must have the corresponding rollback log (i.e. the atomicity of the operation).

Based on this mechanism, the local transaction of the branch can be committed in the first phase of the global transaction and immediately release the resources locked by the local transaction. This is also the difference between Seata's AT mode and XA transaction. The two-phase commit often locks the resources until the actual commit or rollback operation in the second phase. After the rollback log is available, The lock on resources can be released in the first stage, which reduces the lock range and improves efficiency. Even if an exception occurs in the second stage and needs to be rolled back, you only need to find the corresponding data image in the undo log and reverse parse it into SQL to achieve the purpose of rollback

2.2. Detailed implementation process of phase II submission:

If the global voting result of the second phase is submitted, it means that the business SQL of all branch transactions has taken effect in the first phase. At this time, the Seata framework only needs to asynchronously delete the image data, rollback logs and row locks saved in the first phase of all branches, and complete the data cleaning. This process is very fast

2.3. Detailed execution process of phase II rollback:

If the second phase is rollback, Seata needs to rollback the SQL executed in the first phase to restore business data. The TC informs all RMS to make reverse compensation according to the rollback log undo log (i.e. before image) in the first stage. After receiving the rollback request sent by the TC, the RM finds the corresponding rollback log record through XID and Branch ID, generates and executes the reverse update SQL through undo log to complete the business data restoration of the branch, and finally deletes undo log redo log and row lock. However, before restoring, you need to verify dirty writes. Compare the database "current business data" and "after image". If the two data are completely consistent, it means that there are no dirty writes. You can restore business data. If they are inconsistent, it means that there are dirty writes. If there are dirty writes, you need to transfer them to manual processing.

3, Principle of TCC, Saga and XA modes of Seata:

The second part of the article introduces the AT mode of Seata. Next, we will introduce several transaction modes of Seata:

1. TCC mode:

The behavior of RM driving branch transactions in TCC mode is divided into the following two stages:

(1) Execution phase:

  • ① Register branch with TC.
  • ② Execute the Try method of the business definition.
  • ③ Report the execution of Try method to TC: success or failure.

(2) Completion phase:

  • Global submission: after receiving the branch submission request from TC, execute the Confirm method defined by the business.
  • Global rollback: after receiving the branch rollback request from TC, execute the Cancel method defined by the business.

2. Saga mode:

The behavior of Saga mode RM driving branch transactions includes the following two stages:

(1) Execution phase:

  • ① Register branch with TC.
  • ② Execute business methods.
  • ③ Report the implementation of business methods to TC: success or failure.

(2) Completion phase:

  • Global commit, RM does not need to process.
  • Global rollback: after receiving the branch rollback request from TC, execute the compensation rollback method defined by the business.

3. XA mode:

The behavior of XA mode RM driving branch transactions includes the following two stages:

(1) Execution phase:

  • ① Register branch with TC
  • ② XA Start, execute business SQL, XA End
  • ③ XA prepare and report the execution status of XA branch to TC: success or failure

(2) Completion phase:

  • Receiving the branch submission request from TC, XA Commit
  • Received a branch rollback request from TC, XA Rollback

Reference article:

https://help.aliyun.com/document_detail/157850.html

https://blog.csdn.net/k6T9Q8XKs6iIkZPPIFq/article/details/107273472

4, Spring cloud Alibaba integrates the Seata AT model:

1. Set up Seata TC coordinator:

1.1. Download Seata TC coordinator:

The coordinator of Seata is actually an open source service of Alibaba. We only need to download it (download address: https://github.com/seata/seata/releases )And start it. After downloading, you can decompress it directly, but it can't run directly at this time. You still need to make some configuration

1.2. Tables required to create TC:

https://github.com/seata/seata/tree/1.4.2/script/server/db

The TC operation needs to save the transaction information in the database, so you need to create some tables. Find the script\server\db directory of the seata-1.4.2 source code, and you will see the following SQL files:

Take MySQL database as an example, create database seata and execute mysql sql statement in sql file:

CREATE TABLE IF NOT EXISTS `global_table`
(
    `xid`                       VARCHAR(128) NOT NULL,
    `transaction_id`            BIGINT,
    `status`                    TINYINT      NOT NULL,
    `application_id`            VARCHAR(32),
    `transaction_service_group` VARCHAR(32),
    `transaction_name`          VARCHAR(128),
    `timeout`                   INT,
    `begin_time`                BIGINT,
    `application_data`          VARCHAR(2000),
    `gmt_create`                DATETIME,
    `gmt_modified`              DATETIME,
    PRIMARY KEY (`xid`),
    KEY `idx_gmt_modified_status` (`gmt_modified`, `status`),
    KEY `idx_transaction_id` (`transaction_id`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8;

-- the table to store BranchSession data
CREATE TABLE IF NOT EXISTS `branch_table`
(
    `branch_id`         BIGINT       NOT NULL,
    `xid`               VARCHAR(128) NOT NULL,
    `transaction_id`    BIGINT,
    `resource_group_id` VARCHAR(32),
    `resource_id`       VARCHAR(256),
    `branch_type`       VARCHAR(8),
    `status`            TINYINT,
    `client_id`         VARCHAR(64),
    `application_data`  VARCHAR(2000),
    `gmt_create`        DATETIME(6),
    `gmt_modified`      DATETIME(6),
    PRIMARY KEY (`branch_id`),
    KEY `idx_xid` (`xid`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8;

-- the table to store lock data
CREATE TABLE IF NOT EXISTS `lock_table`
(
    `row_key`        VARCHAR(128) NOT NULL,
    `xid`            VARCHAR(128),
    `transaction_id` BIGINT,
    `branch_id`      BIGINT       NOT NULL,
    `resource_id`    VARCHAR(256),
    `table_name`     VARCHAR(32),
    `pk`             VARCHAR(36),
    `gmt_create`     DATETIME,
    `gmt_modified`   DATETIME,
    PRIMARY KEY (`row_key`),
    KEY `idx_branch_id` (`branch_id`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8;

The three tables created are as follows:

  • global_table: global transaction table. Whenever a global transaction is initiated, the ID of the global transaction will be recorded in the table
  • branch_table: Branch transaction table, which records the ID of each branch transaction, which database the branch transaction operates, and other information
  • lock_table: global lock

1.3. Modify TC's registration center and configuration center:

Locate the seata-server-1.4.2\seata\conf directory with a registry Conf file, which configures the TC's registry and configuration center. The default registry is in the form of file, which cannot be used in actual use. It needs to be changed to the form of nacos; Similarly, the configuration center of Seate TC also uses the file form by default, which needs to be modified to nacos as the configuration center:

registry {
  # file ,nacos ,eureka,redis,zk,consul,etcd3,sofa
  type = "nacos"
  nacos {
    application = "seata-server"
    serverAddr = "localhost:8848"
    namespace = "XXXXXXXXXX"
    cluster = "default"
    username = "nacos"
    password = "nacos"
  }
}

config {
  # file,nacos ,apollo,zk,consul,etcd3
  type = "nacos"
  nacos {
    serverAddr = "localhost:8848"
    namespace = "XXXXXXXXXX"
    group = "SEATA_GROUP"
    username = "nacos"
    password = "nacos"
  }
}

The changes are as follows:

  • type: change to nacos, which means that nacos is used as the registry or configuration center
  • application: name of the service
  • serverAddr: address of nacos
  • group: group
  • Namespace: namespace
  • username: user name
  • Password: password

After the above configuration is modified, the nacos configuration will be automatically read when the TC is started, and the last register The configuration of the conf file needs to be consistent with the configuration in each seata client project below

1.4. Import seata Server configuration:

It is recommended to download the seata project locally and read it https://github.com/seata/seata/tree/1.4.2/script/config-center Readme under path MD content

We have configured the configuration center of seata TC above. What are the configurations that TC needs to store in Nacos and how to push them? stay https://github.com/seata/seata/tree/1.4.2/script/config-center There is a config Txt file, which is all the configurations required by TC, and https://github.com/seata/seata/tree/1.4.2/script/config-center/nacos There is a nacos config The SH script can convert config Txt is automatically pushed to nacos.

We start the nacos service first, and then in nacos config Execute the following command under the SH directory to set config Txt to nacos:

#- h represents the IP of nacos service- p represents the port number of nacos- g) grouping information- t namespace ID- u user name, - p password

$ sh nacos-config.sh -h 127.0.0.1 -p 8080 -g SEATA_GROUP -t 7a7581ef-433d-46f3-93f9-5fdc18239c65 -u nacos -w nacos

After the command is executed successfully, you can see the following configuration in nacos:

1.5. Modify the TC transaction information storage method:

There are three transaction information storage modes (store.mode) on the TC side of seata: file, db and redis. The file mode does not need to be changed and can be started directly. The following steps are devoted to the startup of db and redis.

Note: the file mode is stand-alone mode. The global transaction session information is read and written in memory and the local file root is persisted Data, high performance;

(1) Store in DB mode:

The db mode is highly available. The global transaction session information is shared through db, and the corresponding performance is worse; In the previous section, all configuration information has been pushed to Nacos, and TC will be read from Nacos when it is started. Therefore, we need to modify it in Nacos. The configuration to be modified is as follows:

## db storage form is adopted
store.mode=db
## druid data source
store.db.datasource=druid
## mysql database
store.db.dbType=mysql
## mysql driver
store.db.driverClassName=com.mysql.jdbc.Driver
## Database url of TC
store.db.url=jdbc:mysql://127.0.0.1:3306/seata_server?useUnicode=true
## user name
store.db.user=root
## password
store.db.password=Nov2014

Search for the above configuration in nacos and directly modify the values, such as store Mode, as shown below:

(2) store in Redis mode:

The redis mode supported by Seata server version 1.3 and above has high performance, but there is a risk of loss of transaction information. Therefore, the redis persistence configuration suitable for the current scenario needs to be configured in advance. The following configuration needs to be changed for this mode:

store.mode=redis
store.redis.host=127.0.0.1
store.redis.port=6379
store.redis.password=123456

1.6 start TC:

After all the above steps are configured successfully, you can start TC and directly click Seata server in the seata-server-1.4.2\bin directory Bat (windows) runs. After successful startup, you can see that TC has registered in the service list of Nacos, as shown in the following figure:

So far, the TC of Seata has been started and completed

2. Set up Seata client (RM):

2.1. Add seata dependency to maven:

<dependency>
   <groupId>com.alibaba.cloud</groupId>
   <artifactId>spring-cloud-starter-alibaba-seata</artifactId>
   <!-- Exclude dependencies, and the specified version is consistent with the server -->
    <exclusions>
       <exclusion>
           <groupId>io.seata</groupId>
           <artifactId>seata-spring-boot-starter</artifactId>
       </exclusion>
       <exclusion>
           <groupId>io.seata</groupId>
           <artifactId>seata-all</artifactId>
       </exclusion>
   </exclusions>
</dependency>
<dependency>
    <groupId>io.seata</groupId>
    <artifactId>seata-spring-boot-starter</artifactId>
    <version>1.4.2</version>
</dependency>
<dependency>
   <groupId>io.seata</groupId>
   <artifactId>seata-all</artifactId>
   <version>1.4.2</version>
</dependency>

Note: the dependent version of the seata client must be consistent with that of the server.

2.2,application.yml add seat configuration:

seata:
  # Pay special attention to the consistency with the configuration in nacos. It is recommended to configure it as ${spring. Application. Name} - TX group
  tx-service-group: my_test_tx_group
  registry:
    type: nacos
    nacos:
      # The ID of the namespace where the configuration is located. If the default public space is not configured
      server-addr: 127.0.0.1:8848
      namespace: XXXXXXXXXX
      group: SEATA_GROUP
      application: seata-server
      userName: nacos
      password: nacos
  config:
    type: nacos
    nacos:
      server-addr: 127.0.0.1:8848
      namespace: XXXXXXXXXX
      group: SEATA_GROUP
      userName: nacos
      password: nacos

The value of TX service group configuration can be customized, but after definition, you need to add a service in the nacos configuration center vgroupMapping. For the configuration of XXX = default, this attribute must be consistent with the configuration of seata server, otherwise it will not take effect; For example, in the above configuration, you need to add a configuration item service in the nacos configuration center vgroupMapping. my_ Test TX group = default, and set the group to SEATA_GROUP, as shown below:

Note: my_ Test TX group is just a suffix. When adding a configuration in nacos, remember to add the prefix service vgroupMapping.  

2.3. Add rollback log table to the database:

https://github.com/seata/seata/tree/1.4.2/script/client/at/db

Rollback log table: undo_log, which is required by Seata. Each business library should create one. SQL is as follows:

CREATE TABLE IF NOT EXISTS `undo_log`
(
    `branch_id`     BIGINT       NOT NULL COMMENT 'branch transaction id',
    `xid`           VARCHAR(128) NOT NULL COMMENT 'global transaction id',
    `context`       VARCHAR(128) NOT NULL COMMENT 'undo_log context,such as serialization',
    `rollback_info` LONGBLOB     NOT NULL COMMENT 'rollback info',
    `log_status`    INT(11)      NOT NULL COMMENT '0:normal status,1:defense status',
    `log_created`   DATETIME(6)  NOT NULL COMMENT 'create datetime',
    `log_modified`  DATETIME(6)  NOT NULL COMMENT 'modify datetime',
    UNIQUE KEY `ux_undo_log` (`xid`, `branch_id`)
) ENGINE = InnoDB
  AUTO_INCREMENT = 1
  DEFAULT CHARSET = utf8 COMMENT ='AT transaction mode undo table';

2.4. Use seata as global transaction control:

Add the global transaction annotation @ GlobalTransactional at the distributed service portal, and other service interfaces do not need to be configured; Suppose service A calls service B, then add it to the method of service A, but not service B

    /**
     * seata The GlobalTransactional annotation of only needs to load the distributed business entry, and other service interfaces do not need to be configured, so as to realize distributed transactions
     */
    @GlobalTransactional
    @PostMapping (value = "addOrder",produces = {"application/json;charset=utf-8"})
    public String addOrder(@RequestParam String title, @RequestParam int price, @RequestParam int shopId, @RequestParam int userId)
    {
        shopOrderService.save(ShopOrderPojo.builder().price(price).title(title).shopId(shopId).userId(userId).build());
        storageFeignService.reduceStorage(shopId, 1);
        return "success";
    }

Note: if you catch exceptions, seata will think you have handled exceptions and will not roll back the data

  • (1) For example, if you configure @ ControllerAdvice, the data may not be rolled back
  • (2) If Feign is used to call the distributed service and fallback is configured, the following service will throw an exception and directly execute fallback, resulting in failure to rollback (rollbackFor = Exception.class); At this time, you can manually call seata global rollback in the fallback implementation method, as shown below:
    @Override
    public BizResponse insertAge(Integer age) {
        //After feign calls the fallback interface, it needs to manually call the global transaction rollback
        try {
            String xid = RootContext.getXID();
            GlobalTransactionContext.reload(xid).rollback();
        } catch (TransactionException e) {
            e.printStackTrace();
        }
        return BizResponse.fail("Client demotion processing insertAge," + Thread.currentThread().getName());
    }

2.5 introduction to undo log table:

As mentioned above, Seata is based on undo_log, but undo after abnormal rollback_ Log table is empty? What's going on? It's because of undo_log is deleted. You want to see undo_log table. We need to look at the break point. When the exception is not thrown, we need to look at the break point, and then look at the database undo_ Data in log table:

Introduction to undo log table: https://blog.csdn.net/hosaos/article/details/89136666

 

 

Reference article:

Official documents of Seata: https://seata.io/zh-cn/docs/overview/what-is-seata.html

https://blog.csdn.net/qq_34162294/article/details/120984951

Topics: Interview seata