In the project, there is a problem that the two systems can adjust to each other but can not guarantee the consistency of transactions. It happens that this period of time follows dtm Brother Fu has learned some knowledge about distributed transactions, roughly understood some usage scenarios of distributed transactions, and read the official demo After that, I simply wrote a demo myself, and the code has been uploaded to Code cloud In the distributed transaction mode, we use tcc, i.e. (try, confirm, cancel). The calling method is http. You can go to here Learn more.
dtm is a distributed transaction manager developed by go language. Here we run dtm first.
## Pull code cd /home git clone https://github.com/dtm-labs/dtm && cd dtm
According to the official documentation, dtm relies on mysql and docker20.0 is installed After 04 +, you can pass
docker-compose -f helper/compose.mysql.yml
Start the mysql service in docker. You can also use the existing mysql service like me, just under the dtm project:
cp conf.sample.yml conf.yml # Modify conf.yml
Copy the configuration file and modify the relevant information:
Store: # specify which engine to store trans status Driver: 'mysql' Host: 'localhost' User: 'root' Password: '' Port: 3306 ExamplesDB: Driver: 'mysql' Host: 'localhost' User: 'root' Password: '' Port: 3306
Note here that the mysql account you configure must have permission to create dtm database. After the changes are completed, we can start the project (provided that you already have a go environment locally, otherwise you still need to use docker. You can check the dtm official document through docker):
// The entry file is app / main go go run app/main.go dev
As shown in the figure below, the startup is successful, and there will be a regular task printing log
You can also use the database tool to see if there is an additional dtm database. According to the performance test, at present, dtm can process more than 900 transactions per second with mysql. When I wrote this article, I saw the author's article that using redis as the storage engine can process 1W + transactions per second. If there are such high business requirements, redis can be used as the storage engine of dtm. For details, please refer to( mp.weixin.qq.com/s/lmIVQ2aVksZxiCx... )OK, you don't have to worry about DTM when you get here. Let's look at the specific business.
We simulate an order deduction inventory service, including order service and inventory service. php hyperf Framework, other frameworks can also be used. The project structure is as follows:
|- php-dtm-tcc/
|- api-hyperf/
|- order-server/
|- stock-server/
|- sql/index.sql ## Create table statementIn order to simplify the database, I use the same database. Different systems can use different databases. The database tables include commodity table, commodity inventory table, order table, order commodity table and inventory locking table.
The request goes to API hyperf first, and then calls the two sub services order server and stock server.
Order service generation writes orders, order goods and other records. Inventory service deducts inventory and locks inventory.
In the composer. Of the api-hyperf project Add the following two packages to JSON
"linxx/dtmcli-php": "*",//It was originally DTM / dtmcli PHP, but I don't think the data returned in his package is what I want, so I modified the returned data based on the original package, and will make a comparison below "mix/vega": "^3.0"
Then we start API hyperf service, order server service and stock server service respectively.
Before writing the specific code, let's take a look at the overall process. Here is an official demo code:
<?php
require __DIR__ . '/vendor/autoload.php';
function FireTcc () {
$dtm = 'http://localhost:36789/api/dtmsvr';
$svc = 'http://localhost:4005/api';
Dtmcli\tccGlobalTransaction($dtm, function ($tcc) use ($svc) {
/** @var Dtmcli\Tcc $tcc */
$req = ['amount' => 30];
echo 'calling trans out' . PHP_EOL;
$tcc->callBranch($req, $svc . '/TransOutTry', $svc . '/TransOutConfirm', $svc . '/TransOutCancel');
echo 'calling trans in' . PHP_EOL;
$tcc->callBranch($req, $svc . '/TransInTry', $svc . '/TransInConfirm', $svc . '/TransInCancel');
});
}
$vega = new Mix\Vega\Engine();
//Transfer out try
$vega->handleFunc('/api/TransOutTry', function (Mix\Vega\Context $ctx) {
var_dump('TransOutTry', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
//Transfer out confirm
$vega->handleFunc('/api/TransOutConfirm', function (Mix\Vega\Context $ctx) {
var_dump('TransOutConfirm', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
//Transfer out commit
$vega->handleFunc('/api/TransOutCancel', function (Mix\Vega\Context $ctx) {
var_dump('TransOutCancel', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
//Transfer to try
$vega->handleFunc('/api/TransInTry', function (Mix\Vega\Context $ctx) {
var_dump('TransInTry', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
//Go to confirm
$vega->handleFunc('/api/TransInConfirm', function (Mix\Vega\Context $ctx) {
var_dump('TransInConfirm', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
//Transfer to commit
$vega->handleFunc('/api/TransInCancel', function (Mix\Vega\Context $ctx) {
var_dump('TransInCancel', $ctx->request->getQueryParams(), $ctx->request->getParsedBody());
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
$vega->handleFunc('/api/FireTcc', function (Mix\Vega\Context $ctx) {
FireTcc();
$ctx->JSON(200, ['result' => 'SUCCESS']);
})->methods('POST');
$http_worker = new Workerman\Worker("http://0.0.0.0:4005");
$http_worker->onMessage = $vega->handler();
$http_worker->count = 4;
Workerman\Worker::runAll();From the above example, we can see that this is an operation modeled on the transfer business. First, we define the try, confirm and cancel operations of transfer out and in. After the workerman service is started, we request http: 127.0 The 0.1:4006 / API / firetcc address will call the FireTcc() function. In this function, there is a tccGlobalTransaction function, which is the related operations of dtm management transactions, The return value of the tccGlobalTransaction function of the official package dtm / dtmcli PHP is gid (distributed transaction id), which returns the distributed transaction id when the whole transaction operation is successful and null when the transaction fails.
function tccGlobalTransaction(string $dtmUrl, callable $cb): string
{
$tcc = new Tcc($dtmUrl, genGid($dtmUrl));
$tbody = [
'gid' => $tcc->gid,
'trans_type' => 'tcc',
];
$client = new \GuzzleHttp\Client();
try {
$response = $client->post($tcc->dtm . '/prepare', ['json' => $tbody]);
checkStatus($response->getStatusCode());
$cb($tcc);
$client->post($tcc->dtm . '/submit', ['json' => $tbody]);
} catch (\Throwable $e) {
$client->post($tcc->dtm . '/abort', ['json' => $tbody]);
return '';
}
return $tcc->gid;
}The package linxx / dtmcli PHP returns an array:
function tccGlobalTransaction(string $dtmUrl, callable $cb): array
{
$tcc = new Tcc($dtmUrl, genGid($dtmUrl));
$tbody = [
'gid' => $tcc->gid,
'trans_type' => 'tcc',
];
$client = new \GuzzleHttp\Client();
$message = 'Operation succeeded';
$gid = $tcc->gid;
try {
$response = $client->post($tcc->dtm . '/prepare', ['json' => $tbody]);
checkStatus($response->getStatusCode());
$cb($tcc);
$client->post($tcc->dtm . '/submit', ['json' => $tbody]);
} catch (\Throwable $e) {
$client->post($tcc->dtm . '/abort', ['json' => $tbody]);
$message = $e->getMessage();
$gid = '';
}
return [
'gid' => $gid,
'message' => $message
];
}In addition to gid, a message field is also returned, which can also return the cause of the sub service failure, so as to facilitate the API hyperf service to prompt error messages.
Now let's start writing specific business code. First, we go to order server and stock server and write the corresponding try, confirm and cancel operations.
Order service:
<?php
declare(strict_types=1);
/**
* This file is part of Hyperf.
*
* @link https://www.hyperf.io
* @document https://hyperf.wiki
* @contact group@hyperf.io
* @license https://github.com/hyperf/hyperf/blob/master/LICENSE
*/
namespace App\Controller;
use Hyperf\DbConnection\Db;
class Order extends AbstractController
{
public function addOrderTry()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
$totalAmount = 0;
$totalNumber = 0;
foreach ($postData['goods'] as &$value) {
$totalNumber = $value['number'] + $totalNumber;
$goodInfo = Db::table('good')->where('id', $value['good_id'])->first();
$amount = $value['number'] * $goodInfo->price;
$value['amount'] = $amount;
$value['price'] = $goodInfo->price;
$totalAmount = $totalAmount + $amount;
}
unset($value);
if (round($totalAmount, 2) != round($postData['pay_amount'], 2)) {
throw new \Exception('Calculation error of commodity amount', 10010);
}
$orderId = Db::table('order')->insertGetId([
'user_id' => $postData['user_id'],
'order_no' => $postData['order_no'],
'total_amount' => $totalAmount,
'total_number' => $totalNumber,
'create_time' => time(),
'update_time' => time(),
]);
foreach ($postData['goods'] as $value) {
Db::table('order_goods')->insert([
'order_id' => $orderId,
'good_id' => $value['good_id'],
'number' => $value['number'],
'amount' => $value['amount'],
'price' => $value['price'],
'create_time' => time(),
'update_time' => time(),
]);
}
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
'msg' => $e->getMessage(),
];
}
}
public function addOrderConfirm()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
$orderInfo = Db::table('order')->where('order_no', $postData['order_no'])->first();
Db::table('order')->where('id', $orderInfo->id)->update([
'is_ok' => 1,
'update_time' => time(),
]);
Db::table('order_goods')->where('order_id', $orderInfo->id)->update([
'is_ok' => 1,
'update_time' => time(),
]);
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
$e->getMessage(),
];
}
}
public function addOrderCancel()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
$orderInfo = Db::table('order')->where('order_no', $postData['order_no'])->first();
Db::table('order')->where('id', $orderInfo->id)->update([
'delete_time' => time(),
]);
Db::table('order_goods')->where('order_id', $orderInfo->id)->update([
'delete_time' => time(),
]);
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
'msg' => $e->getMessage(),
];
}
}
}Inventory service:
<?php
declare(strict_types=1);
/**
* This file is part of Hyperf.
*
* @link https://www.hyperf.io
* @document https://hyperf.wiki
* @contact group@hyperf.io
* @license https://github.com/hyperf/hyperf/blob/master/LICENSE
*/
namespace App\Controller;
use Hyperf\DbConnection\Db;
class Stock extends AbstractController
{
public function decGoodStockTry()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
foreach ($postData['goods'] as $value) {
$goodInfo = Db::table('good')->where('id', $value['good_id'])->first();
if (empty($goodInfo)) {
throw new \Exception('Item does not exist', 10010);
}
//Search inventory
$stockInfo = Db::table('good_stock')->where('good_id', $value['good_id'])->first();
if (empty($goodInfo)) {
throw new \Exception('Commodity inventory is empty', 10010);
}
if (round($value['number'], 2) > round($stockInfo->total_number, 2)) {
throw new \Exception('Insufficient inventory of goods', 10010);
}
Db::table('good_stock')->where('id', $stockInfo->id)->decrement('total_number', $value['number']);
//Lock inventory
Db::table('good_stock_lock')->insert([
'order_no' => $postData['order_no'],
'good_id' => $value['good_id'],
'number' => $value['number'],
'create_time' => time(),
'update_time' => time(),
]);
//@todo inventory record
}
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
'msg' => $e->getMessage(),
];
}
}
public function decGoodStockConfirm()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
foreach ($postData['goods'] as $value) {
$info = Db::table('good_stock_lock')->where('good_id', $value['good_id'])
->where('order_no', $postData['order_no'])
->first();
if (! empty($info) && $info->status == 0) {
Db::table('good_stock_lock')->where('id', $info->id)->update([
'status' => '1',
'update_time' => time(),
]);
}
}
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
'msg' => $e->getMessage(),
];
}
}
public function decGoodStockCancel()
{
Db::beginTransaction();
try {
$postData = $this->request->post();
foreach ($postData['goods'] as $value) {
$info = Db::table('good_stock_lock')->where('good_id', $value['good_id'])
->where('order_no', $postData['order_no'])
->first();
if (! empty($info) && $info->status == 0) {
Db::table('good_stock_lock')->where('id', $info->id)->update([
'status' => '2',
'update_time' => time(),
]);
Db::table('good_stock')->where('good_id', $value['good_id'])->increment('total_number', $value['number']);
}
}
Db::commit();
return [
'code' => 0,
'data' => 'SUCCESS',
'msg' => 'success',
];
} catch (\Exception $e) {
Db::rollBack();
return [
'code' => 10010,
'data' => 'FAILURE',
'msg' => $e->getMessage(),
];
}
}
}Then go to API hyperf definition and call related operations:
<?php
declare(strict_types=1);
/**
* This file is part of Hyperf.
*
* @link https://www.hyperf.io
* @document https://hyperf.wiki
* @contact group@hyperf.io
* @license https://github.com/hyperf/hyperf/blob/master/LICENSE
*/
namespace App\Controller;
use App\Constants\ErrorCode;
use App\Utils\DtmCli;
class Order extends AbstractController
{
public function addOrder()
{
$orderNo = get_order_no();
$req = [
'user_id' => 1,
'pay_amount' => 100,
'order_no' => $orderNo,
'goods' => [
[
'good_id' => 10000,
'number' => 10,
],
],
];
$serverList = [
[
'server' => 'http://localhost:9551 ', / / you can use etcd and other services to register the discovery middleware
'try' => 'addOrderTry',
'confirm' => 'addOrderConfirm',
'cancel' => 'addOrderCancel',
],
[
'server' => 'http://localhost:9552',
'try' => 'decGoodStockTry',
'confirm' => 'decGoodStockConfirm',
'cancel' => 'decGoodStockCancel',
],
];
$ret = DtmCli::handleDtmTransaction($serverList, $req);
if ($ret['is_ok']) {
return $this->success([]);
}
return $this->success([], ErrorCode::CODE_ERROR, $ret['message']);
}
}DtmCli::handleDtmTransaction is an operation encapsulated by me:
<?php
declare(strict_types=1);
/**
* This file is part of Hyperf.
*
* @link https://www.hyperf.io
* @document https://hyperf.wiki
* @contact group@hyperf.io
* @license https://github.com/hyperf/hyperf/blob/master/LICENSE
*/
namespace App\Utils;
use App\Constants\ErrorCode;
use Dtmcli\Tcc;
use function Dtmcli\tccGlobalTransaction;
class DtmCli
{
/**
* @param $serverList
* @param $req
*/
public static function handleDtmTransaction($serverList, $req): array
{
$isOk = false;
try {
$dtm = 'http://localhost:36789/api/dtmsvr';//dtm service address
if (empty($serverList)) {
throw new \Exception('Sub service cannot be empty', ErrorCode::CODE_ERROR);
}
$ret = tccGlobalTransaction($dtm, function ($tcc) use ($req, $serverList) {
/*
* @var Tcc $tcc
*/
foreach ($serverList as $value) {
if (empty($value['server']) || empty($value['try']) || empty($value['confirm']) || empty($value['cancel'])) {
throw new \Exception('Sub service error', ErrorCode::CODE_ERROR);
}
$tryUrl = $value['server'] . '/' . $value['try'];
$confirmUrl = $value['server'] . '/' . $value['confirm'];
$cancelUrl = $value['server'] . '/' . $value['cancel'];
$tcc->callBranch($req, $tryUrl, $confirmUrl, $cancelUrl);
}
});
if (! empty($ret['gid'])) {
$isOk = true;
}
$message = $ret['message'];
} catch (\Exception $e) {
$message = $e->getMessage();
}
return [
'is_ok' => $isOk,
'message' => $message,
];
}
}After you've written everything, let's request the addOrder interface under API hyperf:
We can see that the return operation is successful. Let's look at the data in the table.
The data in the dtm table also shows that the transaction is successful
Let's ask again:
We can see that the returned goods have insufficient inventory. Because we have only 10 items in stock, 10 items have been sold in the first request above, so the inventory service returns that the inventory is insufficient. We also prompt here.
The relevant data of the order table has been rolled back. The dtm data table failed to identify the transaction
common problem
- The hyperf framework will not take effect after modifying the code. Execute composer dump autoload - O in the root directory
- In the tcc mode, confirm really executes the business without any business check, and only uses the business resources reserved in the try phase, so the resources required by confirm should be locked in the previous try phase.