Relevant versions of Spring used in this battle:
SpringBoot:2.1.7.RELEASE
Spring Cloud:Greenwich.SR2
Spring CLoud Alibaba:2.1.0.RELEASE
1. Overview
In the process of building micro-services, no matter what framework or components are used to build them, there is no question about how business operations across services can maintain data consistency.
2. What is a distributed transaction?
First, imagine a traditional monolithic application, no matter how many internal calls it makes, and ultimately, operate on the same database to complete a business operation all the way, as shown in the following figure:
With the development of business volume, business requirements and architecture have changed dramatically. The overall architecture has been gradually split from the original single application into micro services. The original three services have been separated from a single architecture into three separate services, using separate data sources, and not sharing the same number before.Source, the specific business will be completed by the invocation of three services, as shown in the figure:
At this point, the internal data consistency of each service is still guaranteed by local transactions.But how can we guarantee the transaction across the business process?This is the challenge under the microservice architecture to ensure data consistency in microservices.
3. Common distributed transaction solutions
3.1 Two-stage submission/XA
The so-called XA scenario, two-phase commit, has the concept of a transaction manager responsible for coordinating transactions among multiple databases (resource managers). The transaction manager first asks each database if you are ready?If each database replies ok, the transaction is formally committed and the operation is performed on each database; if one of the databases does not answer ok, the transaction is rolled back.
One of the difficulties of distributed systems is how to ensure the consistency of multiple nodes in transactional operations.To achieve this goal, the two-stage commit algorithm is based on the following assumptions:
- In this distributed system, one node acts as a coordinator and the other as a participant (Cohorts).Network communication is possible between nodes.
- All nodes use pre-written logs, and once the logs are written, they are kept on a reliable storage device, even if the node is damaged, the log data will not disappear.
- All nodes will not be permanently damaged, even if they are damaged.
3.2 TCC Scheme
The full name of TCC is Try, Confirm, Cancel.
- Try phase: This phase is about detecting and locking or reserving resources for each service.
- Confirm phase: This phase is about performing actual operations in each service.
- Cancel phase: If any of the service's business methods fail to execute, compensation is required here to perform a rollback operation that has successfully executed the business logic.(roll back those that performed successfully)
This scenario is used by few people to tell the truth, but there are also scenarios where it can be used.Because this transaction rollback actually relies heavily on you to write code to roll back and compensate for it, it can result in enormous compensation code.
TCC's theory is a bit abstract, so let's describe the process of TCC transactions with the help of a real business scenario of account splitting. We hope it will help to understand TCC.
Business process: Accounts A, B, C, A and B in three different repositories transfer a total of 80 yuan to C together:
Try: Try your business.
Complete all business checks (consistency): Check whether the account status of A, B, C is normal, whether the balance of account A is not less than 30 yuan, and whether the balance of account B is not less than 50 yuan.
Reserve mandatory business resources (quasi-isolation): Account A's freeze amount is increased by 30 yuan and Account B's freeze amount is increased by 50 yuan, which ensures that no other concurrent process deducts the balances of these two accounts, resulting in insufficient available balances for Account A and B during subsequent real transfer operations.
Confirm: Confirm the execution of the business.
True business execution: If account A, B, C is in normal state in stage Try and account A, B balance is sufficient, then perform transfer operation of account A to account C of 30 yuan and account B to account C of 50 yuan.
No business check is done: no business check is needed at this time, and the business check has been completed in the Try phase.
Use only business resources reserved for the Try phase: Simply use the amount frozen by Account A and Account B for the Try phase.
Cancel: Cancel the business.
Release business resources reserved for the Try phase: If the Try phase is partially successful, such as if the balance of Account A is sufficient and the freeze is successful, and the balance of Account B is insufficient and the freeze fails, the Cancel operation on Account A is required to thaw the frozen amount of Account A.
4. Spring Cloud Alibaba Seata
Seata's scheme is actually an improved version of an XA two-phase submission, with the following specific differences:
Architectural aspects:
The RM of the XA schema is actually at the database level, which is essentially the database itself (supplied for use by providing XA-enabled drivers).
Seata's RM is deployed on the side of the application as a two-party package, independent of the support for the protocol with the database itself, and of course, without the XA protocol being supported by the database.This is important for a microserviced architecture: the application tier does not need to adapt two different sets of database drivers for two different scenarios, local transaction and distributed transaction.
This design strips away the protocol support requirements of distributed transaction scenarios for databases.
Two-stage submission:
Whether Phase2's resolution is commit or rollback, the locks on transactional resources will not be released until Phase2 is complete.
Imagine a functioning business where more than 90% of transactions should eventually be successfully committed. Can we commit local transactions in Phase1?In this way, in more than 90% of cases, Phase2 lock-holding time can be omitted and overall efficiency can be improved.
- Local locks on data in branch transactions are managed by local transactions and released at the end of branch transaction Phase1.
- At the same time, the connection is released as the local transaction ends.
- Global locks on data in branch transactions are managed on the transaction coordinator side and can be released immediately when Phase2 global commit is decided.Global locks are held to branch Hase2 only if the resolution global rollback is used.
This design greatly reduces the lock-in time of branch transactions on resources (data and connections) and provides the basis for overall concurrency and throughput improvements.
5. Seata Actual Warfare Cases
5.1 Goal Introduction
In this section, we will describe how Seata is used through a real-world case. We will simulate a simple scenario where users purchase goods and order them. We will create three sub-projects: order-server, storage-server and pay-server. The flow charts are as follows:Fig.
5.2 Environmental Preparation
In this battle, we used Nacos as the service center and configuration center. Nacos deployment refers to Chapter 11 of this book, which is not repeated here.
Next we need to deploy the Server side of Seata at the following download address: https://github.com/seata/seat... It is recommended to choose the latest version to download. The latest version I see is v0.8.0. After downloading seata-server-0.8.0.tar.gz and unzipping it, we open the conf folder, and we need to make some modifications to some of the configuration.
The 5.2.1 registry.conf file is modified as follows:
registry { type = "nacos" nacos { serverAddr = "192.168.0.128" namespace = "public" cluster = "default" } } config { type = "nacos" nacos { serverAddr = "192.168.0.128" namespace = "public" cluster = "default" } }
Here we choose Nacos as the service center and configuration center. Here we make the corresponding configuration. At the same time, we can see Seata's registration service support: file, nacos, eureka, redis, zk, consul, etcd3, sofa, etc. Configuration support: file, nacos, apollo, zk, consul, etcd3, etc.
5.2.2 file.conf File Modification
Here we need to configure the database-related configuration as follows:
## database store db { ## the implement of javax.sql.DataSource, such as DruidDataSource(druid)/BasicDataSource(dbcp) etc. datasource = "dbcp" ## mysql/oracle/h2/oceanbase etc. db-type = "mysql" driver-class-name = "com.mysql.jdbc.Driver" url = "jdbc:mysql://192.168.0.128:3306/seata" user = "root" password = "123456" min-conn = 1 max-conn = 3 global.table = "global_table" branch.table = "branch_table" lock-table = "lock_table" query-limit = 100 }
The database uses mysql by default, and you need to configure the corresponding database connection, user name, password, and so on.
The 5.2.3 nacos-config.txt file is modified as follows:
service.vgroup_mapping.spring-cloud-pay-server=default service.vgroup_mapping.spring-cloud-order-server=default service.vgroup_mapping.spring-cloud-storage-server=default
The syntax here is: service.vgroup_mapping.${your-service-gruop}=default, ${your-service-gruop} in the middle defines the name of the service group for itself, which needs to be configured in the program's configuration file. The author uses the program's spring.application.name directly here.
5.2.4 Database Initialization
You need to execute the data initial script db_store.sql in the database you just configured, which is a global transaction-controlled table that needs to be initialized in advance.
Here we are just demonstrating that in theory the three business services above should belong to different databases. Here we are just creating three Schema s under the same database, db_account, db_order and db_storage, respectively, as shown in the following figure:
5.2.5 Service Startup
Since we are using Nacos as the configuration center, we need to execute a script to initialize the relevant configuration of Nacos with the following commands:
cd conf sh nacos-config.sh 192.168.0.128
After successful execution, you can open Nacos's console, and in the configuration list, you can see that many Groups are initialized for SEATA_GROUP configuration, as shown in the figure:
After successful initialization, you can start the Server side of Seata with the following command:
cd bin sh seata-server.sh -p 8091 -m file
After startup, you can see a service named serverAddr under the list of services in Nacos
At this point, we're done preparing our environment, and we're going to start Code Actual.
5.3 Code Actual
Due to the large amount of code in this example, only the core code and some code that needs attention are described here. The rest of the code can be accessed by readers from the code repository that accompanies this book.
The subproject common is used to place some public classes, mainly including the view VO class and the response class OperationResponse.java.
5.3.1 Parent project seata-nacos-jpa depends on pom.xml file
Code List: Alibaba/seata-nacos-jpa/pom.xml
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!-- Spring Cloud Nacos Service Discovery --> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <!-- Spring Cloud Nacos Config --> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId> </dependency> <!-- Spring Cloud Seata --> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-seata</artifactId> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-alibaba-dependencies</artifactId> <version>${spring-cloud-alibaba.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
Description: This example uses JPA as the database to access the ORM layer, Mysql as the database, and needs to introduce JPA and Mysql related dependencies. The version of spring-cloud-alibaba-dependencies is 2.1.0.RELEASE. The component version about Seata is v0.7.1. Although it does not match the server version, it has not been issued after simple testing.The current problem.
5.3.2 Data Source Configuration
Seata implements transaction branching through a proxy data source, so you need to configure the Bean for io.seata.rm.datasource.DataSourceProxy, which is the default data source for @Primary. Otherwise, transactions will not roll back and distributed transactions will not be possible. The data source configuration class DataSourceProxyConfig.java is as follows:
Code List: Alibaba/seata-nacos-jpa/order-server/src/main/java/com/springcloud/order server/config/DataSourceProxyConfig.java
@Configuration public class DataSourceProxyConfig { @Bean @ConfigurationProperties(prefix = "spring.datasource") public DruidDataSource druidDataSource() { return new DruidDataSource(); } @Primary @Bean public DataSourceProxy dataSource(DruidDataSource druidDataSource) { return new DataSourceProxy(druidDataSource); } }
5.3.3 Open Global Transactions
We start the entire business process in the order-server service and need to add a comment @GlobalTransactional for the global transaction to the method here, as follows:
Code List: Alibaba/seata-nacos-jpa/order-server/src/main/java/com/springcloud/order server/service/impl/OrderServiceImpl.java
@Service @Slf4j public class OrderServiceImpl implements OrderService { @Autowired private RestTemplate restTemplate; @Autowired private OrderDao orderDao; private final String STORAGE_SERVICE_HOST = "http://spring-cloud-storage-server/storage"; private final String PAY_SERVICE_HOST = "http://spring-cloud-pay-server/pay"; @Override @GlobalTransactional public OperationResponse placeOrder(PlaceOrderRequestVO placeOrderRequestVO) { Integer amount = 1; Integer price = placeOrderRequestVO.getPrice(); Order order = Order.builder() .userId(placeOrderRequestVO.getUserId()) .productId(placeOrderRequestVO.getProductId()) .status(OrderStatus.INIT) .payAmount(price) .build(); order = orderDao.save(order); log.info("Save Order{}", order.getId() != null ? "Success" : "fail"); log.info("current XID: {}", RootContext.getXID()); // Deduct inventory log.info("Start deducting inventory"); ReduceStockRequestVO reduceStockRequestVO = ReduceStockRequestVO.builder() .productId(placeOrderRequestVO.getProductId()) .amount(amount) .build(); String storageReduceUrl = String.format("%s/reduceStock", STORAGE_SERVICE_HOST); OperationResponse storageOperationResponse = restTemplate.postForObject(storageReduceUrl, reduceStockRequestVO, OperationResponse.class); log.info("Deduction of inventory results:{}", storageOperationResponse); // Deduction balance log.info("Start deducting balance"); ReduceBalanceRequestVO reduceBalanceRequestVO = ReduceBalanceRequestVO.builder() .userId(placeOrderRequestVO.getUserId()) .price(price) .build(); String reduceBalanceUrl = String.format("%s/reduceBalance", PAY_SERVICE_HOST); OperationResponse balanceOperationResponse = restTemplate.postForObject(reduceBalanceUrl, reduceBalanceRequestVO, OperationResponse.class); log.info("Result of deduction balance:{}", balanceOperationResponse); Integer updateOrderRecord = orderDao.updateOrder(order.getId(), OrderStatus.SUCCESS); log.info("Update Order:{} {}", order.getId(), updateOrderRecord > 0 ? "Success" : "fail"); return OperationResponse.builder() .success(balanceOperationResponse.isSuccess() && storageOperationResponse.isSuccess()) .build(); } }
Second, we need to add the annotation @Transactional to the approach of the other two services to indicate that the transaction is open.
The remote service call here is through RestTemplate, which needs to be injected into the Spring container to manage when the project starts.
5.3.4 Profile
In the project, add the configuration file registry.conf for Seata to the resources directory as follows:
List of codes: Alibaba/seata-nacos-jpa/order-server/src/main/resources/registry.conf
registry { type = "nacos" nacos { serverAddr = "192.168.0.128" namespace = "public" cluster = "default" } } config { type = "nacos" nacos { serverAddr = "192.168.0.128" namespace = "public" cluster = "default" } }
The configuration in bootstrap.yml is as follows:
List of codes: Alibaba/seata-nacos-jpa/order-server/src/main/resources/bootstrap.yml
spring: application: name: spring-cloud-order-server cloud: nacos: # nacos config config: server-addr: 192.168.0.128 namespace: public group: SEATA_GROUP # nacos discovery discovery: server-addr: 192.168.0.128 namespace: public enabled: true alibaba: seata: tx-service-group: ${spring.application.name}
- spring.cloud.nacos.config.group: Here Groupis SEATA_GROUP, which is the configuration we generated earlier when we used nacos-config.sh to generate the configuration of Nacos. Its Groupis SEATA_GROUP.
- spring.cloud.alibaba.seata.tx-service-group: Here is the service.vgroup_mapping.${your-service-gruop}=default intermediate ${your-service-gruop} that we previously configured when modifying the Seata Server side configuration file nacos-config.txt.Make sure the two configurations are the same, otherwise no available server to connect will always be reported after the project is started.
5.3.5 Business Database Initialization
The database initial script is in Alibaba/seata-nacos-jpa/sql, executed in three different Schema s.
5.3.6 Test
Test Tool We chose to use PostMan to start three services in an order independent of order-server, pay-server, and storage-server.
Send a test request using PostMan, as shown in the figure:
The database initialization balance is 10, which consumes 5 for each order. We can order normally twice, the third should fail, and roll back the data in db_order.The data in the database is shown in the following figure:
We make a third order, as shown in the figure:
Here you see Direct Error 500, looking at the data in the database db_order, as shown in the figure:
As you can see, there is no increase in the data here, so let's look at the console printing of subproject_rder-server:
Log has been simplified
Hibernate: insert into orders (pay_amount, product_id, status, user_id) values (?, ?, ?, ?) c.s.b.c.service.impl.OrderServiceImpl : Save Order Successfully c.s.b.c.service.impl.OrderServiceImpl : current XID: 192.168.0.102:8091:2021674307 c.s.b.c.service.impl.OrderServiceImpl : Start deducting inventory c.s.b.c.service.impl.OrderServiceImpl : Deduction of inventory results:OperationResponse(success=true, message=Operation Successful, data=null) c.s.b.c.service.impl.OrderServiceImpl : Start deducting balance i.s.core.rpc.netty.RmMessageListener : onMessage:xid=192.168.0.102:8091:2021674307,branchId=2021674308,branchType=AT,resourceId=jdbc:mysql://192.168.0.128:3306/db_order,applicationData=null io.seata.rm.AbstractRMHandler : Branch Rollbacking: 192.168.0.102:8091:2021674307 2021674308 jdbc:mysql://192.168.0.128:3306/db_order i.s.rm.datasource.undo.UndoLogManager : xid 192.168.0.102:8091:2021674307 branch 2021674308, undo_log deleted with GlobalFinished io.seata.rm.AbstractRMHandler : Branch Rollbacked result: PhaseTwo_Rollbacked i.seata.tm.api.DefaultGlobalTransaction : [192.168.0.102:8091:2021674307] rollback status:Rollbacked
It is not clear from the log that the service order-server performs the order writing operation first and invokes the interface for deducting inventory. By looking at the log of the storage-server, you can also find that the inventory modification operation is performed first, until the balance is deducted, the balance is insufficient, starting with 192 xid168.0.102:8091:2021674307 performs a rollback operation, which is a global rollback.
6. Attention
Currently in Seata v0.8.0, cluster deployment is not supported on the Server side, and is not recommended for production environments. Open source teams plan to use and produce environments when v1.0.0 is released, so you can keep an eye on this open source project.
7. Sample Code
Reference material: Seata Official Documents
