【MongoDB 高可用篇】MongoDB 副本集环境搭建
目录
副本集（replica set）是一组mongod进程维护的相同的数据集，提供了MongoDB的冗余和高可用，本篇将对MongoDB下副本集环境（PSA架构，即一主一从一仲裁）的搭建进行演示。
1 软件环境
使用的软件分别为：
- VirtualBox 6.0
- Oracle Linux 6.7
- MongoDB 4.2.0
2 副本集架构
采用一主一从一仲裁的架构，即PSA架构：
3 搭建过程
3.1 创建Mongod实例
参考：安装MongoDB单实例
3.2 创建KeyFile文件
1）创建KeyFile
[mongod@pri data]$ openssl rand -base64 741 > /u01/data/conf/keyfile
[mongod@pri conf]$ chmod 400 /u01/data/conf/keyfile
[mongod@pri conf]$ ll
total 8
-r--------. 1 mongod mongod 1004 Oct 2 17:50 keyfile
-rw-r--r--. 1 mongod mongod 484 Oct 2 17:52 mongod.cnf
2）复制KeyFile至其它主机
[mongod@pri conf]$ scp * root@192.168.56.103:/u01/data/conf
The authenticity of host '192.168.56.103 (192.168.56.103)' can't be established.
RSA key fingerprint is c1:97:8c:3d:6e:c5:3e:90:f4:52:5b:77:8b:1a:94:d0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.56.103' (RSA) to the list of known hosts.
root@192.168.56.103's password:
keyfile 100% 1004 1.0KB/s 00:00
mongod.cnf 100% 484 0.5KB/s 00:00
[mongod@pri conf]$ scp * root@192.168.56.104:/u01/data/conf
The authenticity of host '192.168.56.104 (192.168.56.104)' can't be established.
RSA key fingerprint is c1:97:8c:3d:6e:c5:3e:90:f4:52:5b:77:8b:1a:94:d0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.56.104' (RSA) to the list of known hosts.
root@192.168.56.104's password:
keyfile 100% 1004 1.0KB/s 00:00
mongod.cnf 100% 484 0.5KB/s 00:003.3 启动Mongod实例
1）启动每个成员的mongod实例
[mongod@pri conf]$ mongod -f ./mongod.cnf
about to fork child process, waiting until server is ready for connections.
forked process: 3328
child process started successfully, parent exiting3.4 初始化复本集
1）通过mongo连接副本集的一个成员，进行副本集的初始化
[mongod@pri data]$ mongo localhost:27017/admin
MongoDB shell version v4.2.0
connecting to: mongodb://localhost:27017/admin?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("7a223330-bc53-46c6-a194-31b28041fb67") }
MongoDB server version: 4.2.0
> cfg={
... _id:'rep1',
... members:[
... {_id:0,host:'192.168.56.102:27017'},
... {_id:1,host:'192.168.56.103:27017'},
... {_id:2,host:'192.168.56.104:27017',arbiterOnly:true}
... ]
... }
{
"_id" : "rep1",
"members" : [
{
"_id" : 0,
"host" : "192.168.56.102:27017"
},
{
"_id" : 1,
"host" : "192.168.56.103:27017"
},
{
"_id" : 2,
"host" : "192.168.56.104:27017",
"arbiterOnly" : true
}
]
}
> rs.initiate(cfg)
{ "ok" : 1 }
rep1:OTHER>2）查看副本集状态
rep1:PRIMARY> rs.status()
{
"set" : "rep1",
"date" : ISODate("2019-10-02T12:51:11.466Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2019-10-02T12:51:08.934Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2019-10-02T12:51:08.934Z"),
"appliedOpTime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2019-10-02T12:51:08.934Z"),
"lastDurableWallTime" : ISODate("2019-10-02T12:51:08.934Z")
},
"members" : [
{
"_id" : 0,
"name" : "192.168.56.102:27017",
"ip" : "192.168.56.102",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 330,
"optime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2019-10-02T12:51:08Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "could not find member to sync from",
"electionTime" : Timestamp(1570020627, 1),
"electionDate" : ISODate("2019-10-02T12:50:27Z"),
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "192.168.56.103:27017",
"ip" : "192.168.56.103",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 54,
"optime" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1570020668, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2019-10-02T12:51:08Z"),
"optimeDurableDate" : ISODate("2019-10-02T12:51:08Z"),
"lastHeartbeat" : ISODate("2019-10-02T12:51:09.661Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T12:51:09.870Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "192.168.56.102:27017",
"syncSourceHost" : "192.168.56.102:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.56.104:27017",
"ip" : "192.168.56.104",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 54,
"lastHeartbeat" : ISODate("2019-10-02T12:51:09.489Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T12:51:09.488Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1570020668, 1),
"signature" : {
"hash" : BinData(0,"dP422ax5e/TsXbGGCmXe6MiXHgI="),
"keyId" : NumberLong("6743187251305381890")
}
},
"operationTime" : Timestamp(1570020668, 1)
}
rep1:PRIMARY>3.5 创建root用户
rep1:PRIMARY> db.createUser({user:'root',pwd:'root',roles:[{role:'root',db:'admin'}]})
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}3.6 验证主从数据一致
1）主库增加数据
rep1:PRIMARY> use scott
switched to db scott
rep1:PRIMARY> db.emp.insert({id:7369,ename:'Alen',age:30,male:'M'})
WriteResult({ "nInserted" : 1 })2）从库查询
rep1:SECONDARY> rs.slaveOk()
rep1:SECONDARY> db.emp.find()
{ "_id" : ObjectId("5d949fe80f887da69c4ee0f9"), "id" : 7369, "ename" : "Alen", "age" : 30, "male" : "M" }
3.7 模拟主库宕机
1）主库宕机
rep1:PRIMARY> db.shutdownServer()
2019-10-02T21:06:18.921+0800 I NETWORK [js] DBClientConnection failed to receive message from localhost:27017 - HostUnreachable: Connection closed by peer
server should be down...
2019-10-02T21:06:18.925+0800 I NETWORK [js] trying reconnect to localhost:27017 failed
2019-10-02T21:06:18.925+0800 I NETWORK [js] reconnect localhost:27017 failed failed
2019-10-02T21:06:18.928+0800 I NETWORK [js] trying reconnect to localhost:27017 failed
2019-10-02T21:06:18.929+0800 I NETWORK [js] reconnect localhost:27017 failed failed
>2）查看备库信息
rep1:PRIMARY> rs.status()
{
"set" : "rep1",
"date" : ISODate("2019-10-02T13:07:10.732Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1570021569, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2019-10-02T13:06:09.030Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1570021569, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2019-10-02T13:06:09.030Z"),
"appliedOpTime" : {
"ts" : Timestamp(1570021630, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1570021630, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2019-10-02T13:07:10.041Z"),
"lastDurableWallTime" : ISODate("2019-10-02T13:07:10.041Z")
},
"members" : [
{
"_id" : 0,
"name" : "192.168.56.102:27017",
"ip" : "192.168.56.102",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2019-10-02T13:07:10.450Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T13:06:18.305Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Error connecting to 192.168.56.102:27017 :: caused by :: Connection refused",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : -1
},
{
"_id" : 1,
"name" : "192.168.56.103:27017",
"ip" : "192.168.56.103",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1279,
"optime" : {
"ts" : Timestamp(1570021630, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-10-02T13:07:10Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1570021578, 1),
"electionDate" : ISODate("2019-10-02T13:06:18Z"),
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "192.168.56.104:27017",
"ip" : "192.168.56.104",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 1013,
"lastHeartbeat" : ISODate("2019-10-02T13:07:10.405Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T13:07:10.096Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1570021630, 1),
"signature" : {
"hash" : BinData(0,"WRjcX9JfAE5cnUiGBSsvv+sZ1aY="),
"keyId" : NumberLong("6743187251305381890")
}
},
"operationTime" : Timestamp(1570021630, 1)
}可以看到，备库自动升级为主库。
3）启动原主库
rep1:SECONDARY> rs.status()
{
"set" : "rep1",
"date" : ISODate("2019-10-02T13:10:52.786Z"),
"myState" : 2,
"term" : NumberLong(2),
"syncingTo" : "192.168.56.103:27017",
"syncSourceHost" : "192.168.56.103:27017",
"syncSourceId" : 1,
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1570021852, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2019-10-02T13:10:52.244Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1570021852, 1),
"t" : NumberLong(2)
},
"readConcernMajorityWallTime" : ISODate("2019-10-02T13:10:52.244Z"),
"appliedOpTime" : {
"ts" : Timestamp(1570021852, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1570021852, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2019-10-02T13:10:52.244Z"),
"lastDurableWallTime" : ISODate("2019-10-02T13:10:52.244Z")
},
"members" : [
{
"_id" : 0,
"name" : "192.168.56.102:27017",
"ip" : "192.168.56.102",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 25,
"optime" : {
"ts" : Timestamp(1570021852, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-10-02T13:10:52Z"),
"syncingTo" : "192.168.56.103:27017",
"syncSourceHost" : "192.168.56.103:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "192.168.56.103:27017",
"ip" : "192.168.56.103",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 23,
"optime" : {
"ts" : Timestamp(1570021850, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1570021850, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2019-10-02T13:10:50Z"),
"optimeDurableDate" : ISODate("2019-10-02T13:10:50Z"),
"lastHeartbeat" : ISODate("2019-10-02T13:10:52.184Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T13:10:50.786Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1570021578, 1),
"electionDate" : ISODate("2019-10-02T13:06:18Z"),
"configVersion" : 1
},
{
"_id" : 2,
"name" : "192.168.56.104:27017",
"ip" : "192.168.56.104",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 23,
"lastHeartbeat" : ISODate("2019-10-02T13:10:52.091Z"),
"lastHeartbeatRecv" : ISODate("2019-10-02T13:10:52.381Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1570021852, 1),
"signature" : {
"hash" : BinData(0,"VmxpFmh1tLkre7ut5F5nRoneEUw="),
"keyId" : NumberLong("6743187251305381890")
}
},
"operationTime" : Timestamp(1570021852, 1)
}可以看到，原主库重启后，变为新的从库。
3.8 实例目录结构
[mongod@pri data]$ ll
total 12
drwxrwxr-x. 2 mongod mongod 4096 Oct 2 21:16 conf
drwxr-xr-x. 8 mongod mongod 4096 Oct 2 21:17 db
drwxrwxr-x. 2 mongod mongod 4096 Oct 2 18:59 log
[mongod@pri data]$ ll conf/
total 8
-r--------. 1 mongod mongod 1004 Oct 2 17:50 keyfile
-rw-r--r--. 1 mongod mongod 556 Oct 2 20:39 mongod.cnf
[mongod@pri data]$ ll db
total 224
drwx------. 4 mongod mongod 4096 Oct 2 20:50 admin
drwx------. 4 mongod mongod 4096 Oct 2 20:50 config
drwx------. 2 mongod mongod 4096 Oct 2 21:17 diagnostic.data
drwx------. 2 mongod mongod 4096 Oct 2 21:11 journal
drwx------. 4 mongod mongod 4096 Oct 2 18:59 local
-rw-------. 1 mongod mongod 36864 Oct 2 21:10 _mdb_catalog.wt
-rw-------. 1 mongod mongod 5 Oct 2 21:10 mongod.lock
-rw-r--r--. 1 mongod mongod 5 Oct 2 21:10 mongod.pid
drwx------. 4 mongod mongod 4096 Oct 2 21:02 scott
-rw-------. 1 mongod mongod 36864 Oct 2 21:17 sizeStorer.wt
-rw-------. 1 mongod mongod 114 Oct 2 18:59 storage.bson
-rw-------. 1 mongod mongod 44 Oct 2 18:59 WiredTiger
-rw-------. 1 mongod mongod 4096 Oct 2 21:10 WiredTigerLAS.wt
-rw-------. 1 mongod mongod 21 Oct 2 18:59 WiredTiger.lock
-rw-------. 1 mongod mongod 1188 Oct 2 21:17 WiredTiger.turtle
-rw-------. 1 mongod mongod 102400 Oct 2 21:17 WiredTiger.wt
[mongod@pri data]$
[mongod@pri data]$ ll log/
total 120
-rw-------. 1 mongod mongod 118592 Oct 2 21:16 mongod.log至此，一主一从一仲裁三节点副本集环境搭建成功。
