分布式文档存储数据库之MongoDB副本集
在mongodb中对数据做冗余的方式有两种，一种是主从架构（master/slave），这种架构和mysql中的主从架构没有什么不同，但是在mongodb中，主从架构的方式几乎没有人用，处于废弃的状态。。另外一种是副本集(replica set)，副本集本质上也是主从架构的一种，它和我们刚才说的主从架构，有一个显著的区别，副本及支持故障自动转移，不需要人工干预；所谓的故障自动转移是指当一个副本集中主节点因各种原因宕机或者从节点在一定时间内没有收到主节点发送到心跳信息，此时就会触发从节点重新选举主节点的操作；在多个从节点中选举出来一个主节点以后，其他从节点能够自动指向新选举的主节点同步数据，从而完成这次故障转移；
前文我们聊到了mongodb的索引的相关作用和介绍以及索引的管理，回顾请参考https://www.cnblogs.com/qiuhom-1874/p/13950287.html；今天我们来聊下mongodb的副本集；
1、什么是副本集？主要作用是什么？它是怎么工作的呢？
所谓副本集是指服务于同一数据集的多个mongodb实例；这意味着在mongodb中数据集副本的数量可以是多个，每个副本的数据都是一样的；副本的存在主要作用是对mongodb数据库中的数据做冗余备份和提高数据服务的可用性；在mongodb中对数据做冗余的方式有两种，一种是主从架构（master/slave），这种架构和mysql中的主从架构没有什么不同，但是在mongodb中，主从架构的方式几乎没有人用，处于废弃的状态。。另外一种是副本集(replica set)，副本集本质上也是主从架构的一种，它和我们刚才说的主从架构，有一个显著的区别，副本集支持故障自动转移，不需要人工手动干预；所谓的故障自动转移是指当一个副本集中主节点因各种原因宕机或者从节点在一定时间内没有收到主节点发送到心跳信息，此时就会触发从节点重新选举主节点的操作；在多个从节点中选举出来一个主节点以后，其他从节点能够自动指向新选举的主节点同步数据，从而完成这次故障转移；
2、mongodb副本集架构
提示：对于mongodb中的副本集来说，为了保证数据的一致性，一个副本集中只能有一个主节点，并且只有主节点才拥有读写权限，其他从节点都只有读权限；在mongodb的副本集中，主节点我们叫做primary，从节点叫做secondary；
3、副本集是通过什么来复制数据的呢？
我知道mysql的主从复制是通过主节点记录写操作到binlog中，从节点通过同步主节点的binlog，然后把同步的binlog在本地进行重放，从而实现数据同步；在mongodb中也是类似的操作，不同的是在mongodb中主节点将写操作记录到oplog(操作日志)，各从节点通过复制主节点的oplog，在本地重放实现数据同步；oplog的作用和mysql中的binlog的作用很类似，都是用来记录写操作事件，都是用来同步数据；mongodb中的oplog是一个定容集合，所谓定容是指它的大小是固定的，它不会像binlog随日志的增多而逐渐变大；通常oplog我们不人为指定其大小，默认就是占用所在文件系统空闲磁盘的5%；除了大小是不变的，它还有一个幂等的特点；所谓幂等就是不管在monogdb中执行多少次oplog中的操作，其最终的结果都是一样的；如果写操作日志把oplog写满了，它会从头覆盖之前的oplog，依次循环写oplog；
4、副本集是怎么完成故障转移的？
在前边我们聊zookeeper时，聊到过分布式场景中的选举过程；mongodb的主节点故障以后，也是通过这种机制来完成选举；在mongodb的副本集中，主节点每隔两秒向各从节点发送心跳信息，各从节点通过检测主节点的心跳信息来判定主节点是否存活；如果在一定的时间范围内，从节点没有收到主节点发送到心跳信息，此时从节点会认为主节点挂了，从而触发主节点选举的操作；这个选举和zookeeper里的选举很类似，通常情况都是大于集群总票的一半一方可以代表集群正常工作；所以通常情况mongodb副本集都是基数个节点组成；当然mongodb中也可以是偶数节点（正常提供服务的节点），如果是偶数节点，通常会借助一个仲裁节点来完成选举；仲裁节点拥有选票，但不被选举成为主节点，也不拥有副本数据，同时它必须要能够检测主节点心跳；简单总结就是mongodb副本集是通过从节点检测主节点心跳来判断主节点是否存活，如果在一定时间范围内，没有检测到主节点的心跳信息，此时就会触发主节点选举操作，如果集群节点数量为偶数个，通常会借助仲裁节点来完成选举；从而实现完成故障自动转移；
5、mongodb副本集中特殊类型节点分类
0优先级节点：这种节点的特点是优先级为0，可参与选举，拥有副本数据，但不被选举成为主节点，可读不可写；这种节点我们也叫冷备节点；通常用于异地容灾使用；
被隐藏的从节点：这种节点的特点是，可参与选举，拥有副本数据，但不被选举成为主节点，对客户端不可读写也不可见；通常用于同步同一副本集中的其他节点的不同工作流的场景中；
延迟复制的从节点：这种节点的特点是，副本数据落后主节点一个时间窗口，也就说这种节点上的副本数据总是过期的；除此它可参与选举，不可被选举为主节点；主要用于特殊功用；比如在主节点上执行了一个删除操作，我们可以在延迟复制的从节点上把数据找回；
arbiter节点：这种就是我们说的仲裁节点，它可参与选举，不拥有副本数据，不被选举成为主节点，不可读写；主要作用是辅助判定主节点是否存活，辅助从节点完成选举，实现故障转移；
6、创建mongodb的副本集合
环境准备
| 主机名 | ip地址 |
| node01 | 192.168.0.41 |
| node02 | 192.168.0.42 |
| node03 | 192.168.0.43 |
在三个节点上分别做好时间同步，关闭selinux,关闭防火墙，主机名解析，有必要还可以做ssh互信。准备好基础环境以后，在三台server上配置mongodb的yum源
[root@node01 ~]# cat /etc/yum.repos.d/mongodb.repo [mongodb-org] name = MongoDB Repository baseurl = https://mirrors.aliyun.com/mongodb/yum/redhat/7/mongodb-org/4.4/x86_64/ gpgcheck = 1 enabled = 1 gpgkey = https://www.mongodb.org/static/pgp/server-4.4.asc [root@node01 ~]# scp /etc/yum.repos.d/mongodb.repo node02:/etc/yum.repos.d/ mongodb.repo 100% 206 80.4KB/s 00:00 [root@node01 ~]# scp /etc/yum.repos.d/mongodb.repo node03:/etc/yum.repos.d/ mongodb.repo 100% 206 88.9KB/s 00:00 [root@node01 ~]#
安装mongodb-org
yum install -y mongodb-org
准备数据目录和存放日志的目录，并将其属主和属组更改为mongod用户
[root@node01 ~]# mkdir -pv /mongodb/{data,log}
mkdir: created directory ‘/mongodb’
mkdir: created directory ‘/mongodb/data’
mkdir: created directory ‘/mongodb/log’
[root@node01 ~]# chown -R mongod.mongod /mongodb/
[root@node01 ~]# ll -d /mongodb/
drwxr-xr-x 4 mongod mongod 29 Nov 10 19:36 /mongodb/
[root@node01 ~]#
提示：以上需在三个节点都要做一遍；
配置mongodb
提示：主要配置replication，其中oplogSizeMB用来指定oplog的大小，默认不指定，其大小就是所在文件系统空闲磁盘的%5；replSetName用来指定副本集的名称，这个名称非常重要，主要用来标识不同副本集；enableMajorityReadConcern用来指定是否开启mongodb周期性的做快照，并记录oplog的时间戳；
完整配置
[root@node01 ~]# cat /etc/mongod.conf systemLog: destination: file logAppend: true path: /mongodb/log/mongod.log storage: dbPath: /mongodb/data/ journal: enabled: true processManagement: fork: true pidFilePath: /var/run/mongodb/mongod.pid timeZoneInfo: /usr/share/zoneinfo net: port: 27017 bindIp: 0.0.0.0 replication: oplogSizeMB: 2048 replSetName: test_replset enableMajorityReadConcern: false #security: #operationProfiling: #sharding: ## Enterprise-Only Options #auditLog: #snmp: [root@node01 ~]#
复制配置文件到node02,node03
[root@node01 ~]# scp /etc/mongod.conf node02:/etc/ mongod.conf 100% 494 315.1KB/s 00:00 [root@node01 ~]# scp /etc/mongod.conf node03:/etc/ mongod.conf 100% 494 351.6KB/s 00:00 [root@node01 ~]#
启动服务
提示：请确保对应节点上的mongodb所监听的端口正常即可；
连接三个节点任意一个节点查看副本集状态
> rs.help()
rs.status() { replSetGetStatus : 1 } checks repl set status
rs.initiate() { replSetInitiate : null } initiates set with default settings
rs.initiate(cfg) { replSetInitiate : cfg } initiates set with configuration cfg
rs.conf() get the current configuration object from local.system.replset
rs.reconfig(cfg) updates the configuration of a running replica set with cfg (disconnects)
rs.add(hostportstr) add a new member to the set with default attributes (disconnects)
rs.add(membercfgobj) add a new member to the set with extra attributes (disconnects)
rs.addArb(hostportstr) add a new member which is arbiterOnly:true (disconnects)
rs.stepDown([stepdownSecs, catchUpSecs]) step down as primary (disconnects)
rs.syncFrom(hostportstr) make a secondary sync from the given member
rs.freeze(secs) make a node ineligible to become primary for the time specified
rs.remove(hostportstr) remove a host from the replica set (disconnects)
rs.secondaryOk() allow queries on secondary nodes
rs.printReplicationInfo() check oplog size and time range
rs.printSecondaryReplicationInfo() check replica set members and replication lag
db.isMaster() check who is primary
reconfiguration helpers disconnect from the database so the shell will display
an error, even if the command succeeds.
> rs.status()
{
"operationTime" : Timestamp(0, 0),
"ok" : 0,
"errmsg" : "no replset config has been received",
"code" : 94,
"codeName" : "NotYetInitialized",
"$clusterTime" : {
"clusterTime" : Timestamp(0, 0),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
>
提示：所有关于副本集的操作命令都可以使用rs.help()方法去查看帮助；其中rs.status()是用来查看副本集状态；这里显示没有副本集配置，其原因是没有初始化副本集；
初始化副本集
> rs.initiate()
{
"info2" : "no configuration specified. Using a default configuration for the set",
"me" : "node01.test.org:27017",
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605010821, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605010821, 1)
}
test_replset:SECONDARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:20:32.079Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 1,
"writeMajorityCount" : 1,
"votingMembersCount" : 1,
"writableVotingMembersCount" : 1,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:20:21.720Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:20:21.720Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2020-11-10T12:20:21.632Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605010821, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 1,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"newTermStartDate" : ISODate("2020-11-10T12:20:21.694Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:20:21.719Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 674,
"optime" : {
"ts" : Timestamp(1605010821, 8),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:20:21Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "Could not find member to sync from",
"electionTime" : Timestamp(1605010821, 2),
"electionDate" : ISODate("2020-11-10T12:20:21Z"),
"configVersion" : 1,
"configTerm" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605010821, 8),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605010821, 8)
}
test_replset:PRIMARY>
提示：初始化副本集以后，再次查看副本集状态，它告诉我们有一个成员，其主机名为node01.test.org:27017,其health状态为1，stateStr为PRIMARY等等一堆信息；
查看副本集配置
test_replset:PRIMARY> rs.conf()
{
"_id" : "test_replset",
"version" : 1,
"term" : 1,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 0,
"host" : "node01.test.org:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY>
提示：可以看到心跳时间间隔为2s，超时为10s,选举超时时长为10等等信息；
添加node02，node03节点到副本集成员
test_replset:PRIMARY> rs.add("node02")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011243, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011243, 1)
}
test_replset:PRIMARY> rs.add("node03")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011250, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011250, 1)
}
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:27:45.849Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 3,
"writableVotingMembersCount" : 3,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:27:41.873Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:27:41.873Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2020-11-10T12:20:21.632Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605010821, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 1,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"newTermStartDate" : ISODate("2020-11-10T12:20:21.694Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:20:21.719Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1107,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605010821, 2),
"electionDate" : ISODate("2020-11-10T12:20:21Z"),
"configVersion" : 3,
"configTerm" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 21,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:27:41Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:27:44.967Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:27:43.983Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node01.test.org:27017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 14,
"optime" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1605011261, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2020-11-10T12:27:41Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:27:41Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:27:44.967Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:27:44.571Z"),
"pingMs" : NumberLong(1),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605011261, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605011261, 1)
}
test_replset:PRIMARY>
提示：添加节点到副本成员使用rs.add（），括号里面写主机名，或者ip地址，如果对应节点监听端口不是27017，还需要写明端口；除此我们也可以直接用一个子文档的形式，手动指定节点的各属性信息来添加节点到副本集成员；
到此副本集的配置就完成了，三个节点都添加到副本集；
验证：在主库上插入数据，看看从库是否都可以正常同步其数据呢？
在主节点插入数据
test_replset:PRIMARY> use students
switched to db students
test_replset:PRIMARY> db.student_info.insert({name:"tom",age:18,gender:"M"})
WriteResult({ "nInserted" : 1 })
test_replset:PRIMARY> db.student_info.find()
{ "_id" : ObjectId("5faa89297077300f4fc31d1c"), "name" : "tom", "age" : 18, "gender" : "M" }
test_replset:PRIMARY>
在从节点查看数据
test_replset:SECONDARY> show dbs
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa83025f9218c5d19ea91b"),
"counter" : NumberLong(4)
},
"operationTime" : Timestamp(1605011801, 1),
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk",
"$clusterTime" : {
"clusterTime" : Timestamp(1605011801, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:SECONDARY>
提示：这里在从节点上没法查看数据，原因是默认情况副本集配置好以后，要在从节点运行rs.secondaryOk()来告诉从节点已经配置好了，否则它会不允许我们读，有点类似从节点为就绪的感觉；
在从节点上运行rs.secondaryOk(),然后再次查看数据
test_replset:SECONDARY> rs.secondaryOk()
test_replset:SECONDARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
students 0.000GB
test_replset:SECONDARY> use students
switched to db students
test_replset:SECONDARY> show collections
student_info
test_replset:SECONDARY> db.student_info.find().pretty()
{
"_id" : ObjectId("5faa89297077300f4fc31d1c"),
"name" : "tom",
"age" : 18,
"gender" : "M"
}
test_replset:SECONDARY>
提示：可以看到在secondary节点上能够查询到对应数据库中的对应collection中插入的数据；
验证：在从节点上插入数据，看看是否能插入成功？
test_replset:SECONDARY> db.student_info.insert({name:"jerry",age:19,gender:"M"})
WriteCommandError({
"topologyVersion" : {
"processId" : ObjectId("5faa831bde03c4ae108f0be4"),
"counter" : NumberLong(3)
},
"operationTime" : Timestamp(1605012904, 1),
"ok" : 0,
"errmsg" : "not master",
"code" : 10107,
"codeName" : "NotMaster",
"$clusterTime" : {
"clusterTime" : Timestamp(1605012904, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
})
test_replset:SECONDARY>
提示：我们在从节点上插入数据，它提示我们不是主节点，不允许插入数据；这说明副本集的从节点是不允许写；
验证：将主节点上的mongodb服务停掉，然后在其他两个节点上查看副本集状态，看看两个节点是否重新选举了master?
[root@node01 ~]# systemctl stop mongod.service
[root@node01 ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@node01 ~]# mongo --host node02
MongoDB shell version v4.4.1
connecting to: mongodb://node02:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("90d60f76-fc95-44a8-bbf7-d927d5947331") }
MongoDB server version: 4.4.1
---
The server generated these startup warnings when booting:
2020-11-10T20:09:39.555+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
2020-11-10T20:09:39.555+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
2020-11-10T20:09:39.555+08:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never'
---
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T12:59:25.775Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 3,
"writableVotingMembersCount" : 3,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2020-11-10T12:59:24.417Z"),
"lastDurableWallTime" : ISODate("2020-11-10T12:59:24.417Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "stepUpRequestSkipDryRun",
"lastElectionDate" : ISODate("2020-11-10T12:58:34.398Z"),
"electionTerm" : NumberLong(2),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"priorPrimaryMemberId" : 0,
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2020-11-10T12:58:34.410Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:58:34.506Z")
},
"members" : [
{
"_id" : 0,
"name" : "node01.test.org:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:59:24.558Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:58:35.045Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "Error connecting to node01.test.org:27017 (192.168.0.41:27017) :: caused by :: Connection refused",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 1
},
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2987,
"optime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T12:59:24Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605013114, 1),
"electionDate" : ISODate("2020-11-10T12:58:34Z"),
"configVersion" : 3,
"configTerm" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1914,
"optime" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1605013164, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T12:59:24Z"),
"optimeDurableDate" : ISODate("2020-11-10T12:59:24Z"),
"lastHeartbeat" : ISODate("2020-11-10T12:59:24.459Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T12:59:25.506Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 3,
"configTerm" : 2
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605013164, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605013164, 1)
}
test_replset:PRIMARY>
提示：可以看到当node01上的mongodb服务停掉以后，对应的主节点也就不能够正常的向从节点发送心跳，所以剩下的两个从节点都认为主节点挂了，此时他俩就开始重新选举主节点；我们再次连接到node02上的mongodb查看副本集状态时，此时node02上的mongodb就成为了主节点；同时我们也能看到node01的健康状态为0，表示不健康；
查看从节点的复制信息
test_replset:PRIMARY> rs.printSecondaryReplicationInfo()
source: node01.test.org:27017
syncedTo: Thu Jan 01 1970 08:00:00 GMT+0800 (CST)
1605013534 secs (445837.09 hrs) behind the primary
source: node03:27017
syncedTo: Tue Nov 10 2020 21:05:34 GMT+0800 (CST)
0 secs (0 hrs) behind the primary
test_replset:PRIMARY>
提示：这里可以看到node01落后主节点很长时间，node03不落后主节点；
查看同步信息时间窗口，以及oplog大小
test_replset:PRIMARY> rs.printReplicationInfo() configured oplog size: 2048MB log length start to end: 2291secs (0.64hrs) oplog first event time: Tue Nov 10 2020 20:27:23 GMT+0800 (CST) oplog last event time: Tue Nov 10 2020 21:05:34 GMT+0800 (CST) now: Tue Nov 10 2020 21:05:35 GMT+0800 (CST) test_replset:PRIMARY>
提示：可以看到oplog的小为2048MB，日志开始到结束的时间窗口为2291秒，意思是说当前启动副本集到查询复制信息是的时长为2291秒；第一个oplog时间时间和最后oplog事件时间以及现在的时间都可查看得到；
移除节点（从副本集中，将node01节点移除）
test_replset:PRIMARY> rs.remove("node01.test.org:27017")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605014063, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605014063, 1)
}
test_replset:PRIMARY> rs.status()
{
"set" : "test_replset",
"date" : ISODate("2020-11-10T13:14:35.362Z"),
"myState" : 1,
"term" : NumberLong(2),
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"votingMembersCount" : 2,
"writableVotingMembersCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"lastCommittedWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"readConcernMajorityWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"appliedOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"durableOpTime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"lastAppliedWallTime" : ISODate("2020-11-10T13:14:34.514Z"),
"lastDurableWallTime" : ISODate("2020-11-10T13:14:34.514Z")
},
"electionCandidateMetrics" : {
"lastElectionReason" : "stepUpRequestSkipDryRun",
"lastElectionDate" : ISODate("2020-11-10T12:58:34.398Z"),
"electionTerm" : NumberLong(2),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1605013112, 1),
"t" : NumberLong(1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"priorPrimaryMemberId" : 0,
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2020-11-10T12:58:34.410Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2020-11-10T12:58:34.506Z")
},
"members" : [
{
"_id" : 1,
"name" : "node02:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3897,
"optime" : {
"ts" : Timestamp(1605014074, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T13:14:34Z"),
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1605013114, 1),
"electionDate" : ISODate("2020-11-10T12:58:34Z"),
"configVersion" : 4,
"configTerm" : 2,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 2,
"name" : "node03:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 2824,
"optime" : {
"ts" : Timestamp(1605014063, 1),
"t" : NumberLong(2)
},
"optimeDurable" : {
"ts" : Timestamp(1605014063, 1),
"t" : NumberLong(2)
},
"optimeDate" : ISODate("2020-11-10T13:14:23Z"),
"optimeDurableDate" : ISODate("2020-11-10T13:14:23Z"),
"lastHeartbeat" : ISODate("2020-11-10T13:14:33.958Z"),
"lastHeartbeatRecv" : ISODate("2020-11-10T13:14:33.970Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "node02:27017",
"syncSourceId" : 1,
"infoMessage" : "",
"configVersion" : 4,
"configTerm" : 2
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605014074, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605014074, 1)
}
test_replset:PRIMARY>
提示：移除节点需要写明对应节点的主机名地址和端口，或者是ip地址端口；这个字串必须是同加入到副本集中的名字相同才可移除，否则它会提示我们给定主机字符串找不到；
设置node03的优先级为10
test_replset:PRIMARY> cfg = rs.conf()
{
"_id" : "test_replset",
"version" : 4,
"term" : 2,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY> cfg.members[1].priority = 10
10
test_replset:PRIMARY> rs.reconfig(cfg)
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605015945, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605015945, 1)
}
test_replset:PRIMARY> rs.config()
{
"_id" : "test_replset",
"version" : 5,
"term" : 3,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 10,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:SECONDARY>
提示：设置某个成员的优先级，首先要把当前配置用一个变量保存，然后通过修改保存的配置变量来修改优先级以后，再使用rs.reconfig()来读取修改后的配置即可生效；这里需要注意我在修改配置文件时指定成员，是指定对应成员在members列表的下标，而非id；所以修改前要先确定对应成员的下标是多少；我们修改了node03的优先级以后，心细的你一定会发现node02此时就变成了secondary;这意味着，只要在副本集中有更高优先级的成员加入，会触发一次选举，当然选举成为主节点，影响最大的就是各节点的优先级，优先级越高，当选主节点的几率就越大；默认情况每个成员的优先级都为1，优先级设定的取值范围是0-1000；
将node01上的mongodb服务启动起来，然后在主节点上将其添加为仲裁节点
启动node01上的mongodb服务
[root@node01 ~]# systemctl start mongod.service [root@node01 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 *:27017 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node01 ~]#
在主节点上将node01添加为仲裁节点
test_replset:PRIMARY> rs.addArb("node01")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605016848, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605016848, 1)
}
test_replset:PRIMARY>
查看配置信息，看看node01上的arbiterOnly是否变成了true?
test_replset:PRIMARY> rs.config()
{
"_id" : "test_replset",
"version" : 8,
"term" : 3,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 1,
"host" : "node02:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "node03:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 10,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 3,
"host" : "node01:27017",
"arbiterOnly" : true,
"buildIndexes" : true,
"hidden" : false,
"priority" : 0,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5faa85853d5a2d8fdf8af85e")
}
}
test_replset:PRIMARY>
提示：可以看到node01节点上的arbiterOnly属性已经启用，说明这就是一个仲裁节点；我们说过仲裁节点上没有保存副本，我们到node01上看看是否保存的有副本集数据呢？
test_replset:ARBITER> show databases
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa9d87e504fa222c4a584d"),
"counter" : NumberLong(1)
},
"ok" : 0,
"errmsg" : "not master and slaveOk=false",
"code" : 13435,
"codeName" : "NotMasterNoSlaveOk"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:ARBITER> rs.secondaryOk()
test_replset:ARBITER> show databases
uncaught exception: Error: listDatabases failed:{
"topologyVersion" : {
"processId" : ObjectId("5faa9d87e504fa222c4a584d"),
"counter" : NumberLong(1)
},
"ok" : 0,
"errmsg" : "node is not in primary or recovering state",
"code" : 13436,
"codeName" : "NotMasterOrSecondary"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs/<@src/mongo/shell/mongo.js:147:19
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:99:12
shellHelper.show@src/mongo/shell/utils.js:937:13
shellHelper@src/mongo/shell/utils.js:819:15
@(shellhelp2):1:1
test_replset:ARBITER>
提示：我们在仲裁节点上查看数据库列表，它提示我们不是主节点和从节点，不允许读；既然都不允许读，写肯定是没有办法进行；
ok,到此mongodb的副本集的配置、测试就到此为止；
作者：Linux-1874
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利.
