分布式数据库TiDB的部署
转自:https://my.oschina.net/Kenyon/blog/908370
一、环境
CentOS Linux release 7.3.1611 (Core)
172.26.11.91 pd & tidb
172.26.11.92 tikv
172.26.11.93 tikv
172.26.11.94 tikv
二、安装
分别在4台服务器上上传安装包
wget http://download.pingcap.org/tidb-latest-linux-amd64.tar.gz
tar -xzf tidb-latest-linux-amd64.tar.gz
cd tidb-latest-linux-amd64
mkdir -p /data/tidb/log
ln -s /root/software/tidb/tidb-latest-linux-amd64/bin/pd-tso-bench /usr/bin
ln -s /root/software/tidb/tidb-latest-linux-amd64/bin/tikv-server /usr/bin/
ln -s /root/software/tidb/tidb-latest-linux-amd64/bin/tidb-server /usr/bin/
ln -s /root/software/tidb/tidb-latest-linux-amd64/bin/pd-server /usr/bin/
ln -s /root/software/tidb/tidb-latest-linux-amd64/bin/pd-ctl /usr/bin/
三、配置使用
1.按照顺序启动
在91上启动pd服务
pd-server --name=pd1 --多个pd以不同名字命名
--data-dir=/data/tidb/pd --pd路径
--client-urls="http://172.26.11.91:2379"
--peer-urls="http://172.26.11.91:2380"
--initial-cluster="pd1=http://172.26.11.91:2380" --多个pd以逗号分隔
--log-file=/data/tidb/log/pd.log &
在92,93,94上启动tikv
tikv-server --pd="172.26.11.91:2379" \
--addr="172.26.11.92:20160" \
--data-dir=/data/tidb/tikv \
--log-file=/data/tidb/log/tikv.log
tikv-server --pd="172.26.11.91:2379" \
--addr="172.26.11.93:20160" \
--data-dir=/data/tidb/tikv \
--log-file=/data/tidb/log/tikv.log
tikv-server --pd="172.26.11.91:2379" \
--addr="172.26.11.94:20160" \
--data-dir=/data/tidb/tikv \
--log-file=/data/tidb/log/tikv.log &
在91上启动tipd服务
tidb-server --store=tikv \ --tikv引擎允许分布式存储,其他如LevelDB等是本地存储
--path="172.26.11.91:2379" \
--log-file=/data/tidb/log/tidb.log &
2.登陆使用
[root@test05 ~]# mysql -h 172.26.11.91 -P 4000 -u root -D test
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.7.1-TiDB-1.0 MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database db_kenyon;
Query OK, 0 rows affected (2.02 sec)
mysql> use db_kenyon;
Database changed
mysql> create table tbl_kenyon(user_code varchar(64) primary key,user_name varchar(32),ctime timestamp);
Query OK, 0 rows affected (2.03 sec)
mysql> insert into tbl_kenyon values('01','qiaofeng',now()),('02','murong',now());
Query OK, 2 rows affected (0.01 sec)
mysql>
三、高可用
tidb的数据都是保存在tikv节点上面,比如上面配置了3套tikv,每套tikv都是独立的,数据保存的方式和传统关系型不一样的是,在tidb里面或者说tikv里面是映射成kv模式存储的
把92的tikv人为挂掉,此时数据库的使用会受影响,简单的一个查询就会被挂起,直到切换成功
--切换过程
mysql> select * from tbl_kenyon;
+----+-----------+---------------------+
| id | cname | ctime |
+----+-----------+---------------------+
| 1 | qiaofeng | 2017-05-23 10:50:43 |
| 2 | murong | 2017-05-23 10:50:43 |
| 3 | saodiseng | 2017-05-23 10:50:43 |
+----+-----------+---------------------+
3 rows in set (10.24 sec)
--切换以后
mysql> select * from tbl_kenyon;
+----+-----------+---------------------+
| id | cname | ctime |
+----+-----------+---------------------+
| 1 | qiaofeng | 2017-05-23 10:50:43 |
| 2 | murong | 2017-05-23 10:50:43 |
| 3 | saodiseng | 2017-05-23 10:50:43 |
+----+-----------+---------------------+
3 rows in set (0.00 sec)
可以发现经过投票,PD已经连到94上去了(93,92上都能看到),此时读取表数据很快
2017/05/23 17:59:12.151 server.rs:153: [INFO] TiKV is ready to serve
2017/05/23 17:59:12.517 raft.rs:846: [INFO] [region 2] 3 [term: 1487] received a MsgHeartbeat message with higher term from 7 [term: 1488]
2017/05/23 17:59:12.517 raft.rs:681: [INFO] [region 2] 3 became follower at term 1488
2017/05/23 17:59:12.525 server.rs:460: [INFO] resolve store 6 address ok, addr 172.26.11.94:20160
2017/05/23 17:59:13.517 apply.rs:621: [INFO] [region 2] 3 execute admin command cmd_type: CompactLog compact_log {compact_index: 6437 compact_term: 1488} at [term: 1488, index: 6439]
2017/05/23 17:59:13.644 raftlog_gc.rs:117: [INFO] [region 2] collected 225 log entries
2017/05/23 17:59:43.517 apply.rs:621: [INFO] [region 2] 3 execute admin command cmd_type: CompactLog compact_log {compact_index: 6498 compact_term: 1488} at [term: 1488, index: 6500]
2017/05/23 17:59:43.643 raftlog_gc.rs:117: [INFO] [region 2] collected 61 log entries
这是因为92是Region中的leader,假如不是leader的tikv服务器受影响如93,94,数据因为默认做了三个副本(也可以配置5个或者7个副本),服务并不会受影响,但是在日志中会不停地告警
四、水平扩展
其实主要是以上组件模块的扩展,对于tidb来说,本身是无状态的,比较容易扩展,pd也可以部署成集群的模式,通过Haproxy、F5或者其他第三方软件来实现,比较难的Tikv的水平扩展。在每个TiKV的节点里,逻辑上划分了一个或多个store,每个store里又划了一个或多个Region,数据就是存放在Region里面,每个Region的默认值是64M,扩容的过程类似以下细胞分裂的过程,比传统的RDBMS采用的Sharding方式以及中间件模式要透明很多,也许以后市面上的诸多中间件日子要难过了。
1.添加pd
动态添加pd
pd-server --name=pd2 \
--client-urls="http://172.26.11.95:2379" \
--peer-urls="http://172.26.11.95:2380" \
--join="http://172.26.11.91:2379" --要加入的原pd集群
动态删除pd
pd-ctl -u http://172.26.11.91:2379
>> member delete pd2
2.添加tikv
添加tikv,比较简单,直接注册一个新的tikv,剩下的数据迁移工作就交给pd,以下在91上新注册一个tikv
tikv-server --pd="172.26.11.91:2379" \
--addr="172.26.11.91:20160" \
--data-dir=/data/tidb/tikv \
--log-file=/data/tidb/log/tikv.log &
查看store,新增了一个1001的store,另外也能看出当前的leader在93上面
[root@test05 ~]# pd-ctl -u http://172.26.11.91:2379
» store
{
"count": 4,
"stores": [
{
"store": {
"id": 6,
"address": "172.26.11.94:20160",
"state": 0,
"state_name": "Up"
},
"status": {
"store_id": 6,
"capacity": "21 GB",
"available": "21 GB",
"leader_count": 0,
"region_count": 1,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-23T18:05:07+08:00",
"last_heartbeat_ts": "2017-05-24T17:52:03.842239159+08:00",
"uptime": "23h46m56.842239159s"
}
},
{
"store": {
"id": 1001,
"address": "172.26.11.91:20160",
"state": 0,
"state_name": "Up"
},
"status": {
"store_id": 1001,
"capacity": "21 GB",
"available": "15 GB",
"leader_count": 0,
"region_count": 0,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-24T17:50:12+08:00",
"last_heartbeat_ts": "2017-05-24T17:52:03.290658649+08:00",
"uptime": "1m51.290658649s"
}
},
{
"store": {
"id": 1,
"address": "172.26.11.92:20160",
"state": 0,
"state_name": "Up"
},
"status": {
"store_id": 1,
"capacity": "21 GB",
"available": "21 GB",
"leader_count": 0,
"region_count": 1,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-23T17:59:12+08:00",
"last_heartbeat_ts": "2017-05-24T17:52:06.843194072+08:00",
"uptime": "23h52m54.843194072s"
}
},
{
"store": {
"id": 4,
"address": "172.26.11.93:20160",
"state": 0,
"state_name": "Up"
},
"status": {
"store_id": 4,
"capacity": "21 GB",
"available": "21 GB",
"leader_count": 1,
"region_count": 1,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-23T17:22:48+08:00",
"last_heartbeat_ts": "2017-05-24T17:52:09.766282426+08:00",
"uptime": "24h29m21.766282426s"
}
}
]
}
3.删除tikv
查看1001的store状态是0,也就是up
» store 1001
{
"store": {
"id": 1001,
"address": "172.26.11.91:20160",
"state": 0,
"state_name": "Up"
},
"status": {
"store_id": 1001,
"capacity": "21 GB",
"available": "15 GB",
"leader_count": 0,
"region_count": 0,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-24T17:50:12+08:00",
"last_heartbeat_ts": "2017-05-24T17:58:57.490156968+08:00",
"uptime": "8m45.490156968s"
}
}
--删除过程,state=1表示正在下线
» store delete 1001
Success!
» store 1001
{
"store": {
"id": 1001,
"address": "172.26.11.91:20160",
"state": 1,
"state_name": "Offline"
},
"status": {
"store_id": 1001,
"capacity": "21 GB",
"available": "15 GB",
"leader_count": 0,
"region_count": 0,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "2017-05-24T17:50:12+08:00",
"last_heartbeat_ts": "2017-05-24T17:59:17.690136502+08:00",
"uptime": "9m5.690136502s"
}
}
--state=2表示数据已经清理,可以关闭
» store 1001
{
"store": {
"id": 1001,
"address": "172.26.11.91:20160",
"state": 2,
"state_name": "Tombstone"
},
"status": {
"store_id": 0,
"capacity": "0 B",
"available": "0 B",
"leader_count": 0,
"region_count": 0,
"sending_snap_count": 0,
"receiving_snap_count": 0,
"applying_snap_count": 0,
"is_busy": false,
"start_ts": "1970-01-01T08:00:00+08:00",
"last_heartbeat_ts": "0001-01-01T00:00:00Z",
"uptime": "0s"
}
}
»
五、总结:
1.维护相对很简单,官方文档是有中英文版本,资料更新相对及时,但是实际使用者提供的资料较少
2.Scale Out相比较传统的RDBMS方案上来看简化很多,特别是可以摒弃五花八门的中间件
3.从架构上来看,小数据量的性能应该一般,不建议使用,大数据量理论上会较好
4.期待GA版本