分布式事务之dtm
github: https://github.com/dtm-labs/dtm
本人使用场景, 目前微服务中存在的用户服务, 商品服务,订单服务, 支付服务, 在进行下单操作的时候,需要创建订单并扣减库存, 这个时候就需要保证事务的一致性, 但是对于目前的微服务架构来说就需要一套分布式的事务来实现,于是引入DTM
介绍:
他是一种跨语言的分布式事务管理器,无论你是php,python,golang还是java,nodejs等,都可以使用它来解决
使用场景:
- 缓存管理:彻底保证缓存最终一致及强一致
- 秒杀扣库存:极端情况下,也能保证Redis中精准的库存,和最终创建的订单完全一致,无需手动调整
- 非单体的订单系统: 大幅简化架构
- 事件发布/订阅:更好的发件箱模式
使用:
1)本地运行dtm
git clone https://github.com/dtm-labs/dtm && cd dtm go run main.go 注:他是无需配置也可以运行起来,但是如果有定制需求的话, 可以将根目录下的conf.sample.yml文件改为conf.yml 启动的时候指定配置文件即可 go run .\main.go -c ./conf.yml
注: 在启动服务之前, 需要先执行sqls目录下的对应的存储引擎的相关sql,否则启动的时候会一直报表不存在的错误, 例如使用的是mysql, 则需要执行
CREATE DATABASE if not exists dtm_busi /*!40100 DEFAULT CHARACTER SET utf8mb4 */ ; drop table if exists dtm_busi.user_account; create table if not exists dtm_busi.user_account( id int(11) PRIMARY KEY AUTO_INCREMENT, user_id int(11) UNIQUE, balance DECIMAL(10, 2) not null default '0', trading_balance DECIMAL(10, 2) not null default '0', create_time datetime DEFAULT now(), update_time datetime DEFAULT now(), key(create_time), key(update_time) ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4; insert into dtm_busi.user_account (user_id, balance) values (1, 10000), (2, 10000) on DUPLICATE KEY UPDATE balance = values (balance); --------------------------------- create database if not exists dtm_barrier /*!40100 DEFAULT CHARACTER SET utf8mb4 */ ; drop table if exists dtm_barrier.barrier; create table if not exists dtm_barrier.barrier( id bigint(22) PRIMARY KEY AUTO_INCREMENT, trans_type varchar(45) default '', gid varchar(128) default '', branch_id varchar(128) default '', op varchar(45) default '', barrier_id varchar(45) default '', reason varchar(45) default '' comment 'the branch type who insert this record', create_time datetime DEFAULT now(), update_time datetime DEFAULT now(), key(create_time), key(update_time), UNIQUE key(gid, branch_id, op, barrier_id) ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4; ------------------------------------- CREATE DATABASE IF NOT EXISTS dtm /*!40100 DEFAULT CHARACTER SET utf8mb4 */ ; drop table IF EXISTS dtm.trans_global; CREATE TABLE if not EXISTS dtm.trans_global ( `id` bigint(22) NOT NULL AUTO_INCREMENT, `gid` varchar(128) NOT NULL COMMENT 'global transaction id', `trans_type` varchar(45) not null COMMENT 'transaction type: saga | xa | tcc | msg', `status` varchar(12) NOT NULL COMMENT 'transaction status: prepared | submitted | aborting | succeed | failed', `query_prepared` varchar(1024) NOT NULL COMMENT 'url to check for msg|workflow', `protocol` varchar(45) not null comment 'protocol: http | grpc | json-rpc', `create_time` datetime DEFAULT NULL, `update_time` datetime DEFAULT NULL, `finish_time` datetime DEFAULT NULL, `rollback_time` datetime DEFAULT NULL, `options` varchar(1024) DEFAULT '' COMMENT 'options for transaction like: TimeoutToFail, RequestTimeout', `custom_data` varchar(1024) DEFAULT '' COMMENT 'custom data for transaction', `next_cron_interval` int(11) default null comment 'next cron interval. for use of cron job', `next_cron_time` datetime default null comment 'next time to process this trans. for use of cron job', `owner` varchar(128) not null default '' comment 'who is locking this trans', `ext_data` TEXT comment 'extra data for this trans. currently used in workflow pattern', `result` varchar(1024) DEFAULT '' COMMENT 'result for transaction', `rollback_reason` varchar(1024) DEFAULT '' COMMENT 'rollback reason for transaction', PRIMARY KEY (`id`), UNIQUE KEY `gid` (`gid`), key `owner`(`owner`), key `status_next_cron_time` (`status`, `next_cron_time`) comment 'cron job will use this index to query trans' ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4; drop table IF EXISTS dtm.trans_branch_op; CREATE TABLE IF NOT EXISTS dtm.trans_branch_op ( `id` bigint(22) NOT NULL AUTO_INCREMENT, `gid` varchar(128) NOT NULL COMMENT 'global transaction id', `url` varchar(1024) NOT NULL COMMENT 'the url of this op', `data` TEXT COMMENT 'request body, depreceated', `bin_data` BLOB COMMENT 'request body', `branch_id` VARCHAR(128) NOT NULL COMMENT 'transaction branch ID', `op` varchar(45) NOT NULL COMMENT 'transaction operation type like: action | compensate | try | confirm | cancel', `status` varchar(45) NOT NULL COMMENT 'transaction op status: prepared | succeed | failed', `finish_time` datetime DEFAULT NULL, `rollback_time` datetime DEFAULT NULL, `create_time` datetime DEFAULT NULL, `update_time` datetime DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `gid_uniq` (`gid`, `branch_id`, `op`) ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4; drop table IF EXISTS dtm.kv; CREATE TABLE IF NOT EXISTS dtm.kv ( `id` bigint(22) NOT NULL AUTO_INCREMENT, `cat` varchar(45) NOT NULL COMMENT 'the category of this data', `k` varchar(128) NOT NULL, `v` TEXT, `version` bigint(22) default 1 COMMENT 'version of the value', create_time datetime default NULL, update_time datetime DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE key `uniq_k`(`cat`, `k`) ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;
使用的配置参考:
##################################################################### ### dtm can be run without any config. ### all config in this file is optional. the default value is as specified in each line ### all configs can be specified from env. for example: ### MicroService.EndPoint => MICRO_SERVICE_END_POINT ##################################################################### Store: # specify which engine to store trans status Driver: 'mysql' Host: 'localhost' User: 'root' Password: '123456' Port: 3306 Db: 'dtm' # Driver: 'boltdb' # default store engine # Driver: 'redis' # Host: 'localhost' # User: '' # Password: '' # Port: 6379 # Driver: 'postgres' # Host: 'localhost' # User: 'postgres' # Password: 'mysecretpassword' # Port: '5432' # Db: 'postgres' # Schema: 'public' # default value is 'public' ### following config is for only Driver postgres/mysql MaxOpenConns: 500 MaxIdleConns: 500 ConnMaxLifeTime: 5 # default value is 5 (minutes) ### flollowing config is only for some Driver # DataExpire: 604800 # Trans data will expire in 7 days. only for redis/boltdb. # FinishedDataExpire: 86400 # finished Trans data will expire in 1 days. only for redis. # RedisPrefix: '{a}' # default value is '{a}'. Redis storage prefix. store data to only one slot in cluster MicroService: # gRPC/HTTP based microservice config Driver: 'dtm-driver-gozero' # name of the driver to handle register/discover Target: 'etcd://localhost:2379/dtmservice' # register dtm server to this url EndPoint: 'localhost:36790' ### the unit of following configurations is second TransCronInterval: 3 # the interval to poll unfinished global transaction for every dtm process TimeoutToFail: 35 # timeout for XA, TCC to fail. saga's timeout default to infinite, which can be overwritten in saga options RetryInterval: 10 # the subtrans branch will be retried after this interval RequestTimeout: 3 # the timeout of HTTP/gRPC request in dtm LogLevel: 'debug' # default: info. can be debug|info|warn|error Log: Outputs: 'stderr' # default: stderr, split by ",", you can append files to Outputs if need. example:'stderr,/tmp/test.log' RotationEnable: 0 # default: 0 RotationConfigJSON: '{}' # example: '{"maxsize": 100, "maxage": 0, "maxbackups": 0, "localtime": false, "compress": false}' HttpPort: 36789 GrpcPort: 36790 JsonRpcPort: 36791 ### advanced options UpdateBranchAsyncGoroutineNum: 1 # num of async goroutine to update branch status TimeZoneOffset: '+8' #default '' using system default. '+8': Asia/Shanghai; '0': GMT AdminBasePath: '' #default '' set admin access base path # ConfigUpdateInterval: 10 # the interval to update configuration in memory such as topics map... (seconds) # TimeZoneOffset: '' # default '' using system default. '+8': Asia/Shanghai; '0': GMT # AlertRetryLimit: 3 # default 3; if a transaction branch has been retried 3 times, the AlertHook will be called # AlertWebHook: '' # default ''; sample: 'http://localhost:8080/dtm-hook'. this hook will be called like this: ## curl -H "Content-Type: application/json" -d '{"gid":"xxxx","status":"submitted","retry_count":3}' http://localhost:8080/dtm-hook
1)指定使用了MySQL作为数据存储层
2)开启微服务的支持MicroService,并选择了etcd
这样服务已经启动了之后就可以参考官方文档进行一些demo测试和与项目的结合使用了
官方文档: https://dtm.pub/
文档中讲解的很细致,并且都有中文翻译, 还有一些接入的例子,
结合go-zero使用的实例: