MongoDB的集群模式--Sharding(分片)
分片是数据跨多台机器存储，MongoDB使用分片来支持具有非常大的数据集和高吞吐量操作的部署。
具有大型数据集或高吞吐量应用程序的数据库系统可能会挑战单个服务器的容量。例如，高查询率会耗尽服务器的CPU容量。工作集大小大于系统的RAM会强调磁盘驱动器的I / O容量。
有两种解决系统增长的方法：垂直和水平缩放。
垂直扩展涉及增加单个服务器的容量，例如使用更强大的CPU，添加更多RAM或增加存储空间量。可用技术的局限性可能会限制单个机器对于给定工作负载而言足够强大。此外，基于云的提供商基于可用的硬件配置具有硬性上限。结果，垂直缩放有实际的最大值。
水平扩展涉及划分系统数据集并加载多个服务器，添加其他服务器以根据需要增加容量。虽然单个机器的总体速度或容量可能不高，但每台机器处理整个工作负载的子集，可能提供比单个高速大容量服务器更高的效率。扩展部署容量只需要根据需要添加额外的服务器，这可能比单个机器的高端硬件的总体成本更低。权衡是基础架构和部署维护的复杂性增加。
MongoDB支持通过分片进行水平扩展。
一、组件
- shard：每个分片包含分片数据的子集。每个分片都可以部署为副本集（replica set）。可以分片，不分片的数据存于主分片服务器上。部署为3成员副本集
- mongos：mongos充当查询路由器，提供客户端应用程序和分片集群之间的接口。可以部署多个mongos路由器。部署1个或者多个mongos
- config servers：配置服务器存储群集的元数据和配置设置。从MongoDB 3.4开始，必须将配置服务器部署为3成员副本集
注意：应用程序或者客户端必须要连接mongos才能与集群的数据进行交互，永远不应连接到单个分片以执行读取或写入操作。
shard的replica set的架构图：
config servers的replica set的架构图：
分片策略
1、散列分片
- 使用散列索引在共享群集中分区数据。散列索引计算单个字段的哈希值作为索引值; 此值用作分片键。
- 使用散列索引解析查询时，MongoDB会自动计算哈希值。应用程序也不会需要计算哈希值。
- 基于散列值的数据分布有助于更均匀的数据分布，尤其是在分片键单调变化的数据集中。
2、范围分片
- 基于分片键值将数据分成范围。然后根据分片键值为每个块分配一个范围。
- mongos可以将操作仅路由到包含所需数据的分片。
- 分片键的规划很重要，可能导致数据不能均匀分布。
二、部署
1、环境说明
| 服务器名称 | IP地址 | 操作系统版本 | MongoDB版本 | 配置服务器(Config Server)端口 | 分片服务器1(Shard Server 1 | 分片服务器2(Shard Server 2) | 分片服务器3(Shard Server 3) | 功能 |
| mongo1.example.net | 10.10.18.10 | Centos7.5 | 4.0 | 27027（Primary） | 27017（Primary） | 27018（Arbiter） | 27019（Secondary） | 配置服务器和分片服务器 |
| mongo2.example.net | 10.10.18.11 | Centos7.5 | 4.0 | 27027（Secondary） | 27017（Secondary） |
27018（Primary） | 27019（Arbiter） | 配置服务器和分片服务器 |
| mongo3.example.net | 10.10.18.12 | Centos7.5 | 4.0 | 27027（Secondary） | 27017（Arbiter） | 27018（Secondary） | 27019（Primary） | 配置服务器和分片服务器 |
| mongos.example.net | 192.168.11.10 | Centos7.5 | 4.0 | mongos的端口：27017 | mongos |
官方推荐配置中使用逻辑DNS，所以该文档中，将服务器名称和IP地址的DNS映射关系写入到各服务器的/etc/hosts文件中。
2、部署MongoDB
环境中4台服务器的MongoDB的安装部署，详见：MongoDB安装
创建环境需要的目录：
mkdir -p /data/mongodb/data/{configServer,shard1,shard2,shard3} mkdir -p /data/mongodb/{log,pid}
3、创建配置服务器（Config Server）的 Replica Set（副本集）
3台服务器上配置文件内容: /data/mongodb/configServer.conf
mongo1.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/configServer.log" logAppend: true storage: dbPath: "/data/mongodb/data/configServer" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/configServer.pid" net: bindIp: mongo1.example.net port: 27027 replication: replSetName: cs0 sharding: clusterRole: configsvr
mongo2.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/configServer.log" logAppend: true storage: dbPath: "/data/mongodb/data/configServer" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/configServer.pid" net: bindIp: mongo2.example.net port: 27027 replication: replSetName: cs0 sharding: clusterRole: configsvr
mongo3.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/configServer.log" logAppend: true storage: dbPath: "/data/mongodb/data/configServer" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/configServer.pid" net: bindIp: mongo3.example.net port: 27027 replication: replSetName: cs0 sharding: clusterRole: configsvr
启动三台服务器Config Server
mongod -f /data/mongodb/configServer.conf
连接到其中一个Config Server
mongo --host mongo1.example.net --port 27027
结果：
1 MongoDB shell version v4.0.10 2 connecting to: mongodb://mongo1.example.net:27027/?gssapiServiceName=mongodb 3 Implicit session: session { "id" : UUID("1a4d4252-11d0-40bb-90da-f144692be88d") } 4 MongoDB server version: 4.0.10 5 Server has startup warnings: 6 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] 7 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 8 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 9 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 10 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] 11 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] 12 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 13 2019-06-14T14:28:56.013+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 14 2019-06-14T14:28:56.014+0800 I CONTROL [initandlisten] 15 2019-06-14T14:28:56.014+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. 16 2019-06-14T14:28:56.014+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 17 2019-06-14T14:28:56.014+0800 I CONTROL [initandlisten] 18 >
配置Replica Set
rs.initiate( { _id: "cs0", configsvr: true, members: [ { _id : 0, host : "mongo1.example.net:27027" }, { _id : 1, host : "mongo2.example.net:27027" }, { _id : 2, host : "mongo3.example.net:27027" } ] } )
结果：
1 { 2 "ok" : 1, 3 "operationTime" : Timestamp(1560493908, 1), 4 "$gleStats" : { 5 "lastOpTime" : Timestamp(1560493908, 1), 6 "electionId" : ObjectId("000000000000000000000000") 7 }, 8 "lastCommittedOpTime" : Timestamp(0, 0), 9 "$clusterTime" : { 10 "clusterTime" : Timestamp(1560493908, 1), 11 "signature" : { 12 "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), 13 "keyId" : NumberLong(0) 14 } 15 } 16 }
查看Replica Set的状态
cs0:PRIMARY> rs.status()
结果: 可以看出三个服务器：1个Primary，2个Secondary
1 { 2 "set" : "cs0", 3 "date" : ISODate("2019-06-14T06:33:31.348Z"), 4 "myState" : 1, 5 "term" : NumberLong(1), 6 "syncingTo" : "", 7 "syncSourceHost" : "", 8 "syncSourceId" : -1, 9 "configsvr" : true, 10 "heartbeatIntervalMillis" : NumberLong(2000), 11 "optimes" : { 12 "lastCommittedOpTime" : { 13 "ts" : Timestamp(1560494006, 1), 14 "t" : NumberLong(1) 15 }, 16 "readConcernMajorityOpTime" : { 17 "ts" : Timestamp(1560494006, 1), 18 "t" : NumberLong(1) 19 }, 20 "appliedOpTime" : { 21 "ts" : Timestamp(1560494006, 1), 22 "t" : NumberLong(1) 23 }, 24 "durableOpTime" : { 25 "ts" : Timestamp(1560494006, 1), 26 "t" : NumberLong(1) 27 } 28 }, 29 "lastStableCheckpointTimestamp" : Timestamp(1560493976, 1), 30 "members" : [ 31 { 32 "_id" : 0, 33 "name" : "mongo1.example.net:27027", 34 "health" : 1, 35 "state" : 1, 36 "stateStr" : "PRIMARY", 37 "uptime" : 277, 38 "optime" : { 39 "ts" : Timestamp(1560494006, 1), 40 "t" : NumberLong(1) 41 }, 42 "optimeDate" : ISODate("2019-06-14T06:33:26Z"), 43 "syncingTo" : "", 44 "syncSourceHost" : "", 45 "syncSourceId" : -1, 46 "infoMessage" : "could not find member to sync from", 47 "electionTime" : Timestamp(1560493919, 1), 48 "electionDate" : ISODate("2019-06-14T06:31:59Z"), 49 "configVersion" : 1, 50 "self" : true, 51 "lastHeartbeatMessage" : "" 52 }, 53 { 54 "_id" : 1, 55 "name" : "mongo2.example.net:27027", 56 "health" : 1, 57 "state" : 2, 58 "stateStr" : "SECONDARY", 59 "uptime" : 102, 60 "optime" : { 61 "ts" : Timestamp(1560494006, 1), 62 "t" : NumberLong(1) 63 }, 64 "optimeDurable" : { 65 "ts" : Timestamp(1560494006, 1), 66 "t" : NumberLong(1) 67 }, 68 "optimeDate" : ISODate("2019-06-14T06:33:26Z"), 69 "optimeDurableDate" : ISODate("2019-06-14T06:33:26Z"), 70 "lastHeartbeat" : ISODate("2019-06-14T06:33:29.385Z"), 71 "lastHeartbeatRecv" : ISODate("2019-06-14T06:33:29.988Z"), 72 "pingMs" : NumberLong(0), 73 "lastHeartbeatMessage" : "", 74 "syncingTo" : "mongo1.example.net:27027", 75 "syncSourceHost" : "mongo1.example.net:27027", 76 "syncSourceId" : 0, 77 "infoMessage" : "", 78 "configVersion" : 1 79 }, 80 { 81 "_id" : 2, 82 "name" : "mongo3.example.net:27027", 83 "health" : 1, 84 "state" : 2, 85 "stateStr" : "SECONDARY", 86 "uptime" : 102, 87 "optime" : { 88 "ts" : Timestamp(1560494006, 1), 89 "t" : NumberLong(1) 90 }, 91 "optimeDurable" : { 92 "ts" : Timestamp(1560494006, 1), 93 "t" : NumberLong(1) 94 }, 95 "optimeDate" : ISODate("2019-06-14T06:33:26Z"), 96 "optimeDurableDate" : ISODate("2019-06-14T06:33:26Z"), 97 "lastHeartbeat" : ISODate("2019-06-14T06:33:29.384Z"), 98 "lastHeartbeatRecv" : ISODate("2019-06-14T06:33:29.868Z"), 99 "pingMs" : NumberLong(0), 100 "lastHeartbeatMessage" : "", 101 "syncingTo" : "mongo1.example.net:27027", 102 "syncSourceHost" : "mongo1.example.net:27027", 103 "syncSourceId" : 0, 104 "infoMessage" : "", 105 "configVersion" : 1 106 } 107 ], 108 "ok" : 1, 109 "operationTime" : Timestamp(1560494006, 1), 110 "$gleStats" : { 111 "lastOpTime" : Timestamp(1560493908, 1), 112 "electionId" : ObjectId("7fffffff0000000000000001") 113 }, 114 "lastCommittedOpTime" : Timestamp(1560494006, 1), 115 "$clusterTime" : { 116 "clusterTime" : Timestamp(1560494006, 1), 117 "signature" : { 118 "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), 119 "keyId" : NumberLong(0) 120 } 121 } 122 }
创建管理用户
use admin db.createUser( { user: "myUserAdmin", pwd: "abc123", roles: [{ role: "userAdminAnyDatabase", db: "admin" },"readWriteAnyDatabase"] } )
开启Config Server的登录验证和内部验证
使用Keyfiles进行内部认证，在其中一台服务器上创建Keyfiles
openssl rand -base64 756 > /data/mongodb/keyfile chmod 400 /data/mongodb/keyfile
将这个keyfile文件分发到其它的三台服务器上，并保证权限400
/data/mongodb/configServer.conf 配置文件中开启认证
security: keyFile: "/data/mongodb/keyfile" clusterAuthMode: "keyFile" authorization: "enabled"
然后依次关闭2个Secondary，在关闭 Primary
mongod -f /data/mongodb/configServer.conf --shutdown
依次开启Primary和两个Secondary
mongod -f /data/mongodb/configServer.conf
使用用户密码登录mongo
mongo --host mongo1.example.net --port 27027 -u myUserAdmin --authenticationDatabase "admin" -p 'abc123'
注意：由于刚创建用户的时候没有给该用户管理集群的权限，所有此时登录后，能查看所有数据库，但是不能查看集群的状态信息。
1 cs0:PRIMARY> rs.status() 2 { 3 "operationTime" : Timestamp(1560495861, 1), 4 "ok" : 0, 5 "errmsg" : "not authorized on admin to execute command { replSetGetStatus: 1.0, lsid: { id: UUID(\"59dd4dc0-b34f-43b9-a341-a2f43ec1dcfa\") }, $clusterTime: { clusterTime: Timestamp(1560495849, 1), signature: { hash: BinData(0, A51371EC5AA54BB1B05ED9342BFBF03CBD87F2D9), keyId: 6702270356301807629 } }, $db: \"admin\" }", 6 "code" : 13, 7 "codeName" : "Unauthorized", 8 "$gleStats" : { 9 "lastOpTime" : Timestamp(0, 0), 10 "electionId" : ObjectId("7fffffff0000000000000002") 11 }, 12 "lastCommittedOpTime" : Timestamp(1560495861, 1), 13 "$clusterTime" : { 14 "clusterTime" : Timestamp(1560495861, 1), 15 "signature" : { 16 "hash" : BinData(0,"3UkTpXxyU8WI1TyS+u5vgewueGA="), 17 "keyId" : NumberLong("6702270356301807629") 18 } 19 } 20 } 21 cs0:PRIMARY> show dbs 22 admin 0.000GB 23 config 0.000GB 24 local 0.000GB
赋值该用户具有集群的管理权限
use admin
db.system.users.find() #查看当前的用户信息 db.grantRolesToUser("myUserAdmin", ["clusterAdmin"])
查看集群信息
1 cs0:PRIMARY> rs.status() 2 { 3 "set" : "cs0", 4 "date" : ISODate("2019-06-14T07:18:20.223Z"), 5 "myState" : 1, 6 "term" : NumberLong(2), 7 "syncingTo" : "", 8 "syncSourceHost" : "", 9 "syncSourceId" : -1, 10 "configsvr" : true, 11 "heartbeatIntervalMillis" : NumberLong(2000), 12 "optimes" : { 13 "lastCommittedOpTime" : { 14 "ts" : Timestamp(1560496690, 1), 15 "t" : NumberLong(2) 16 }, 17 "readConcernMajorityOpTime" : { 18 "ts" : Timestamp(1560496690, 1), 19 "t" : NumberLong(2) 20 }, 21 "appliedOpTime" : { 22 "ts" : Timestamp(1560496690, 1), 23 "t" : NumberLong(2) 24 }, 25 "durableOpTime" : { 26 "ts" : Timestamp(1560496690, 1), 27 "t" : NumberLong(2) 28 } 29 }, 30 "lastStableCheckpointTimestamp" : Timestamp(1560496652, 1), 31 "members" : [ 32 { 33 "_id" : 0, 34 "name" : "mongo1.example.net:27027", 35 "health" : 1, 36 "state" : 1, 37 "stateStr" : "PRIMARY", 38 "uptime" : 1123, 39 "optime" : { 40 "ts" : Timestamp(1560496690, 1), 41 "t" : NumberLong(2) 42 }, 43 "optimeDate" : ISODate("2019-06-14T07:18:10Z"), 44 "syncingTo" : "", 45 "syncSourceHost" : "", 46 "syncSourceId" : -1, 47 "infoMessage" : "", 48 "electionTime" : Timestamp(1560495590, 1), 49 "electionDate" : ISODate("2019-06-14T06:59:50Z"), 50 "configVersion" : 1, 51 "self" : true, 52 "lastHeartbeatMessage" : "" 53 }, 54 { 55 "_id" : 1, 56 "name" : "mongo2.example.net:27027", 57 "health" : 1, 58 "state" : 2, 59 "stateStr" : "SECONDARY", 60 "uptime" : 1113, 61 "optime" : { 62 "ts" : Timestamp(1560496690, 1), 63 "t" : NumberLong(2) 64 }, 65 "optimeDurable" : { 66 "ts" : Timestamp(1560496690, 1), 67 "t" : NumberLong(2) 68 }, 69 "optimeDate" : ISODate("2019-06-14T07:18:10Z"), 70 "optimeDurableDate" : ISODate("2019-06-14T07:18:10Z"), 71 "lastHeartbeat" : ISODate("2019-06-14T07:18:18.974Z"), 72 "lastHeartbeatRecv" : ISODate("2019-06-14T07:18:19.142Z"), 73 "pingMs" : NumberLong(0), 74 "lastHeartbeatMessage" : "", 75 "syncingTo" : "mongo1.example.net:27027", 76 "syncSourceHost" : "mongo1.example.net:27027", 77 "syncSourceId" : 0, 78 "infoMessage" : "", 79 "configVersion" : 1 80 }, 81 { 82 "_id" : 2, 83 "name" : "mongo3.example.net:27027", 84 "health" : 1, 85 "state" : 2, 86 "stateStr" : "SECONDARY", 87 "uptime" : 1107, 88 "optime" : { 89 "ts" : Timestamp(1560496690, 1), 90 "t" : NumberLong(2) 91 }, 92 "optimeDurable" : { 93 "ts" : Timestamp(1560496690, 1), 94 "t" : NumberLong(2) 95 }, 96 "optimeDate" : ISODate("2019-06-14T07:18:10Z"), 97 "optimeDurableDate" : ISODate("2019-06-14T07:18:10Z"), 98 "lastHeartbeat" : ISODate("2019-06-14T07:18:18.999Z"), 99 "lastHeartbeatRecv" : ISODate("2019-06-14T07:18:18.998Z"), 100 "pingMs" : NumberLong(0), 101 "lastHeartbeatMessage" : "", 102 "syncingTo" : "mongo2.example.net:27027", 103 "syncSourceHost" : "mongo2.example.net:27027", 104 "syncSourceId" : 1, 105 "infoMessage" : "", 106 "configVersion" : 1 107 } 108 ], 109 "ok" : 1, 110 "operationTime" : Timestamp(1560496690, 1), 111 "$gleStats" : { 112 "lastOpTime" : { 113 "ts" : Timestamp(1560496631, 1), 114 "t" : NumberLong(2) 115 }, 116 "electionId" : ObjectId("7fffffff0000000000000002") 117 }, 118 "lastCommittedOpTime" : Timestamp(1560496690, 1), 119 "$clusterTime" : { 120 "clusterTime" : Timestamp(1560496690, 1), 121 "signature" : { 122 "hash" : BinData(0,"lHiVw7WeO81npTi2IMW16reAN84="), 123 "keyId" : NumberLong("6702270356301807629") 124 } 125 } 126 }
4、部署分片服务器1（Shard1）以及Replica Set（副本集）
3台服务器上配置文件内容: /data/mongodb/shard1.conf
mongo1.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard1.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard1" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard1.pid" net: bindIp: mongo1.example.net port: 27017 replication: replSetName: "shard1" sharding: clusterRole: shardsvr
mongo2.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard1.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard1" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard1.pid" net: bindIp: mongo2.example.net port: 27017 replication: replSetName: "shard1" sharding: clusterRole: shardsvr
mongo3.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard1.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard1" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard1.pid" net: bindIp: mongo3.example.net port: 27017 replication: replSetName: "shard1" sharding: clusterRole: shardsvr
开启三台服务器上Shard
mongod -f /data/mongodb/shard1.conf
连接Primary服务器的Shard的副本集
mongo --host mongo1.example.net --port 27017
结果
1 MongoDB shell version v4.0.10 2 connecting to: mongodb://mongo1.example.net:27017/?gssapiServiceName=mongodb 3 Implicit session: session { "id" : UUID("91e76384-cdae-411f-ab88-b7a8bd4555d1") } 4 MongoDB server version: 4.0.10 5 Server has startup warnings: 6 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] 7 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 8 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 9 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 10 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] 11 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] 12 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 13 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 14 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] 15 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. 16 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never' 17 2019-06-14T15:32:39.243+0800 I CONTROL [initandlisten] 18 >
配置Replica Set
rs.initiate( { _id : "shard1", members: [ { _id : 0, host : "mongo1.example.net:27017",priority:2 }, { _id : 1, host : "mongo2.example.net:27017",priority:1 }, { _id : 2, host : "mongo3.example.net:27017",arbiterOnly:true } ] } )
注意：优先级priority的值越大，越容易选举成为Primary
查看Replica Set的状态：
1 shard1:PRIMARY> rs.status() 2 { 3 "set" : "shard1", 4 "date" : ISODate("2019-06-20T01:33:21.809Z"), 5 "myState" : 1, 6 "term" : NumberLong(2), 7 "syncingTo" : "", 8 "syncSourceHost" : "", 9 "syncSourceId" : -1, 10 "heartbeatIntervalMillis" : NumberLong(2000), 11 "optimes" : { 12 "lastCommittedOpTime" : { 13 "ts" : Timestamp(1560994393, 1), 14 "t" : NumberLong(2) 15 }, 16 "readConcernMajorityOpTime" : { 17 "ts" : Timestamp(1560994393, 1), 18 "t" : NumberLong(2) 19 }, 20 "appliedOpTime" : { 21 "ts" : Timestamp(1560994393, 1), 22 "t" : NumberLong(2) 23 }, 24 "durableOpTime" : { 25 "ts" : Timestamp(1560994393, 1), 26 "t" : NumberLong(2) 27 } 28 }, 29 "lastStableCheckpointTimestamp" : Timestamp(1560994373, 1), 30 "members" : [ 31 { 32 "_id" : 0, 33 "name" : "mongo1.example.net:27017", 34 "health" : 1, 35 "state" : 1, 36 "stateStr" : "PRIMARY", 37 "uptime" : 43, 38 "optime" : { 39 "ts" : Timestamp(1560994393, 1), 40 "t" : NumberLong(2) 41 }, 42 "optimeDate" : ISODate("2019-06-20T01:33:13Z"), 43 "syncingTo" : "", 44 "syncSourceHost" : "", 45 "syncSourceId" : -1, 46 "infoMessage" : "could not find member to sync from", 47 "electionTime" : Timestamp(1560994371, 1), 48 "electionDate" : ISODate("2019-06-20T01:32:51Z"), 49 "configVersion" : 1, 50 "self" : true, 51 "lastHeartbeatMessage" : "" 52 }, 53 { 54 "_id" : 1, 55 "name" : "mongo2.example.net:27017", 56 "health" : 1, 57 "state" : 2, 58 "stateStr" : "SECONDARY", 59 "uptime" : 36, 60 "optime" : { 61 "ts" : Timestamp(1560994393, 1), 62 "t" : NumberLong(2) 63 }, 64 "optimeDurable" : { 65 "ts" : Timestamp(1560994393, 1), 66 "t" : NumberLong(2) 67 }, 68 "optimeDate" : ISODate("2019-06-20T01:33:13Z"), 69 "optimeDurableDate" : ISODate("2019-06-20T01:33:13Z"), 70 "lastHeartbeat" : ISODate("2019-06-20T01:33:19.841Z"), 71 "lastHeartbeatRecv" : ISODate("2019-06-20T01:33:21.164Z"), 72 "pingMs" : NumberLong(0), 73 "lastHeartbeatMessage" : "", 74 "syncingTo" : "mongo1.example.net:27017", 75 "syncSourceHost" : "mongo1.example.net:27017", 76 "syncSourceId" : 0, 77 "infoMessage" : "", 78 "configVersion" : 1 79 }, 80 { 81 "_id" : 2, 82 "name" : "mongo3.example.net:27017", 83 "health" : 1, 84 "state" : 7, 85 "stateStr" : "ARBITER", 86 "uptime" : 32, 87 "lastHeartbeat" : ISODate("2019-06-20T01:33:19.838Z"), 88 "lastHeartbeatRecv" : ISODate("2019-06-20T01:33:20.694Z"), 89 "pingMs" : NumberLong(0), 90 "lastHeartbeatMessage" : "", 91 "syncingTo" : "", 92 "syncSourceHost" : "", 93 "syncSourceId" : -1, 94 "infoMessage" : "", 95 "configVersion" : 1 96 } 97 ], 98 "ok" : 1 99 }
结果: 可以看出三个服务器：1个Primary，1个Secondary，1一个Arbiter
创建管理用户
use admin db.createUser( { user: "myUserAdmin", pwd: "abc123", roles: [{ role: "userAdminAnyDatabase", db: "admin" },"readWriteAnyDatabase","clusterAdmin"] } )
开启Shard1的登录验证和内部验证
security: keyFile: "/data/mongodb/keyfile" clusterAuthMode: "keyFile" authorization: "enabled"
然后依次关闭Arbiter、Secondary、Primary
mongod -f /data/mongodb/shard1.conf --shutdown
依次开启Primary和两个Secondary
mongod -f /data/mongodb/shard1.conf
使用用户密码登录mongo
mongo --host mongo1.example.net --port 27017 -u myUserAdmin --authenticationDatabase "admin" -p 'abc123'
5、部署分片服务器2（Shard2）以及Replica Set（副本集）
3台服务器上配置文件内容: /data/mongodb/shard2.conf
mongo1.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard2.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard2" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard2.pid" net: bindIp: mongo1.example.net port: 27018 replication: replSetName: "shard2" sharding: clusterRole: shardsvr
mongo2.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard2.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard2" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard2.pid" net: bindIp: mongo2.example.net port: 27018 replication: replSetName: "shard2" sharding: clusterRole: shardsvr
mongo3.example.net服务器上
systemLog: destination: file path: "/data/mongodb/log/shard2.log" logAppend: true storage: dbPath: "/data/mongodb/data/shard2" journal: enabled: true wiredTiger: engineConfig: cacheSizeGB: 2 processManagement: fork: true pidFilePath: "/data/mongodb/pid/shard2.pid" net: bindIp: mongo3.example.net port: 27018 replication: replSetName: "shard2" sharding: clusterRole: shardsvr
开启三台服务器上Shard
mongod -f /data/mongodb/shard2.conf
连接Primary服务器的Shard的副本集
mongo --host mongo2.example.net --port 27018
配置Replica Set（注意：三个服务器的角色发生了改变）
rs.initiate( { _id : "shard2", members: [ { _id : 0, host : "mongo1.example.net:27018",arbiterOnly:true }, { _id : 1, host : "mongo2.example.net:27018",priority:2 }, { _id : 2, host : "mongo3.example.net:27018",priority:1 } ] } )
查看Replica Set的状态：
1 shard2:PRIMARY> rs.status() 2 { 3 "set" : "shard2", 4 "date" : ISODate("2019-06-20T01:59:08.996Z"), 5 "myState" : 1, 6 "term" : NumberLong(1), 7 "syncingTo" : "", 8 "syncSourceHost" : "", 9 "syncSourceId" : -1, 10 "heartbeatIntervalMillis" : NumberLong(2000), 11 "optimes" : { 12 "lastCommittedOpTime" : { 13 "ts" : Timestamp(1560995943, 1), 14 "t" : NumberLong(1) 15 }, 16 "readConcernMajorityOpTime" : { 17 "ts" : Timestamp(1560995943, 1), 18 "t" : NumberLong(1) 19 }, 20 "appliedOpTime" : { 21 "ts" : Timestamp(1560995943, 1), 22 "t" : NumberLong(1) 23 }, 24 "durableOpTime" : { 25 "ts" : Timestamp(1560995943, 1), 26 "t" : NumberLong(1) 27 } 28 }, 29 "lastStableCheckpointTimestamp" : Timestamp(1560995913, 1), 30 "members" : [ 31 { 32 "_id" : 0, 33 "name" : "mongo1.example.net:27018", 34 "health" : 1, 35 "state" : 7, 36 "stateStr" : "ARBITER", 37 "uptime" : 107, 38 "lastHeartbeat" : ISODate("2019-06-20T01:59:08.221Z"), 39 "lastHeartbeatRecv" : ISODate("2019-06-20T01:59:07.496Z"), 40 "pingMs" : NumberLong(0), 41 "lastHeartbeatMessage" : "", 42 "syncingTo" : "", 43 "syncSourceHost" : "", 44 "syncSourceId" : -1, 45 "infoMessage" : "", 46 "configVersion" : 1 47 }, 48 { 49 "_id" : 1, 50 "name" : "mongo2.example.net:27018", 51 "health" : 1, 52 "state" : 1, 53 "stateStr" : "PRIMARY", 54 "uptime" : 412, 55 "optime" : { 56 "ts" : Timestamp(1560995943, 1), 57 "t" : NumberLong(1) 58 }, 59 "optimeDate" : ISODate("2019-06-20T01:59:03Z"), 60 "syncingTo" : "", 61 "syncSourceHost" : "", 62 "syncSourceId" : -1, 63 "infoMessage" : "could not find member to sync from", 64 "electionTime" : Timestamp(1560995852, 1), 65 "electionDate" : ISODate("2019-06-20T01:57:32Z"), 66 "configVersion" : 1, 67 "self" : true, 68 "lastHeartbeatMessage" : "" 69 }, 70 { 71 "_id" : 2, 72 "name" : "mongo3.example.net:27018", 73 "health" : 1, 74 "state" : 2, 75 "stateStr" : "SECONDARY", 76 "uptime" : 107, 77 "optime" : { 78 "ts" : Timestamp(1560995943, 1), 79 "t" : NumberLong(1) 80 }, 81 "optimeDurable" : { 82 "ts" : Timestamp(1560995943, 1), 83 "t" : NumberLong(1) 84 }, 85 "optimeDate" : ISODate("2019-06-20T01:59:03Z"), 86 "optimeDurableDate" : ISODate("2019-06-20T01:59:03Z"), 87 "lastHeartbeat" : ISODate("2019-06-20T01:59:08.220Z"), 88 "lastHeartbeatRecv" : ISODate("2019-06-20T01:59:08.716Z"), 89 "pingMs" : NumberLong(0), 90 "lastHeartbeatMessage" : "", 91 "syncingTo" : "mongo2.example.net:27018", 92 "syncSourceHost" : "mongo2.example.net:27018", 93 "syncSourceId" : 1, 94 "infoMessage" : "", 95 "configVersion" : 1 96 } 97 ], 98 "ok" : 1, 99 "operationTime" : Timestamp(1560995943, 1), 100 "$clusterTime" : { 101 "clusterTime" : Timestamp(1560995943, 1), 102 "signature" : { 103 "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), 104 "keyId" : NumberLong(0) 105 } 106 } 107 }
结果: 可以看出三个服务器：1个Primary，1个Secondary，1一个Arbiter
配置登录认证的用户请按照 Shard1 的步骤
6、部署分片服务器3（Shard3）以及Replica Set（副本集）
3台服务器上配置文件内容: /data/mongodb/shard3.conf
mongo1.example.net服务器上
systemLog:
destination: file
path: "/data/mongodb/log/shard3.log"
logAppend: true
storage:
dbPath: "/data/mongodb/data/shard3"
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true
pidFilePath: "/data/mongodb/pid/shard3.pid"
net:
bindIp: mongo1.example.net
port: 27019
replication:
replSetName: "shard3"
sharding:
clusterRole: shardsvr
mongo2.example.net服务器上
systemLog:
destination: file
path: "/data/mongodb/log/shard3.log"
logAppend: true
storage:
dbPath: "/data/mongodb/data/shard3"
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true
pidFilePath: "/data/mongodb/pid/shard3.pid"
net:
bindIp: mongo2.example.net
port: 27019
replication:
replSetName: "shard3"
sharding:
clusterRole: shardsvr
mongo3.example.net服务器上
systemLog:
destination: file
path: "/data/mongodb/log/shard3.log"
logAppend: true
storage:
dbPath: "/data/mongodb/data/shard3"
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
processManagement:
fork: true
pidFilePath: "/data/mongodb/pid/shard3.pid"
net:
bindIp: mongo3.example.net
port: 27019
replication:
replSetName: "shard3"
sharding:
clusterRole: shardsvr
开启三台服务器上Shard
mongod -f /data/mongodb/shard3.conf
连接Primary服务器的Shard的副本集
mongo --host mongo3.example.net --port 27019
配置Replica Set（注意：三个服务器的角色发生了改变）
rs.initiate( { _id : "shard3", members: [ { _id : 0, host : "mongo1.example.net:27019",priority:1 }, { _id : 1, host : "mongo2.example.net:27019",arbiterOnly:true }, { _id : 2, host : "mongo3.example.net:27019",priority:2 } ] } )
查看Replica Set的状态：
1 shard3:PRIMARY> rs.status() 2 { 3 "set" : "shard3", 4 "date" : ISODate("2019-06-20T02:21:56.990Z"), 5 "myState" : 1, 6 "term" : NumberLong(1), 7 "syncingTo" : "", 8 "syncSourceHost" : "", 9 "syncSourceId" : -1, 10 "heartbeatIntervalMillis" : NumberLong(2000), 11 "optimes" : { 12 "lastCommittedOpTime" : { 13 "ts" : Timestamp(1560997312, 2), 14 "t" : NumberLong(1) 15 }, 16 "readConcernMajorityOpTime" : { 17 "ts" : Timestamp(1560997312, 2), 18 "t" : NumberLong(1) 19 }, 20 "appliedOpTime" : { 21 "ts" : Timestamp(1560997312, 2), 22 "t" : NumberLong(1) 23 }, 24 "durableOpTime" : { 25 "ts" : Timestamp(1560997312, 2), 26 "t" : NumberLong(1) 27 } 28 }, 29 "lastStableCheckpointTimestamp" : Timestamp(1560997312, 1), 30 "members" : [ 31 { 32 "_id" : 0, 33 "name" : "mongo1.example.net:27019", 34 "health" : 1, 35 "state" : 2, 36 "stateStr" : "SECONDARY", 37 "uptime" : 17, 38 "optime" : { 39 "ts" : Timestamp(1560997312, 2), 40 "t" : NumberLong(1) 41 }, 42 "optimeDurable" : { 43 "ts" : Timestamp(1560997312, 2), 44 "t" : NumberLong(1) 45 }, 46 "optimeDate" : ISODate("2019-06-20T02:21:52Z"), 47 "optimeDurableDate" : ISODate("2019-06-20T02:21:52Z"), 48 "lastHeartbeat" : ISODate("2019-06-20T02:21:56.160Z"), 49 "lastHeartbeatRecv" : ISODate("2019-06-20T02:21:55.155Z"), 50 "pingMs" : NumberLong(0), 51 "lastHeartbeatMessage" : "", 52 "syncingTo" : "mongo3.example.net:27019", 53 "syncSourceHost" : "mongo3.example.net:27019", 54 "syncSourceId" : 2, 55 "infoMessage" : "", 56 "configVersion" : 1 57 }, 58 { 59 "_id" : 1, 60 "name" : "mongo2.example.net:27019", 61 "health" : 1, 62 "state" : 7, 63 "stateStr" : "ARBITER", 64 "uptime" : 17, 65 "lastHeartbeat" : ISODate("2019-06-20T02:21:56.159Z"), 66 "lastHeartbeatRecv" : ISODate("2019-06-20T02:21:55.021Z"), 67 "pingMs" : NumberLong(0), 68 "lastHeartbeatMessage" : "", 69 "syncingTo" : "", 70 "syncSourceHost" : "", 71 "syncSourceId" : -1, 72 "infoMessage" : "", 73 "configVersion" : 1 74 }, 75 { 76 "_id" : 2, 77 "name" : "mongo3.example.net:27019", 78 "health" : 1, 79 "state" : 1, 80 "stateStr" : "PRIMARY", 81 "uptime" : 45, 82 "optime" : { 83 "ts" : Timestamp(1560997312, 2), 84 "t" : NumberLong(1) 85 }, 86 "optimeDate" : ISODate("2019-06-20T02:21:52Z"), 87 "syncingTo" : "", 88 "syncSourceHost" : "", 89 "syncSourceId" : -1, 90 "infoMessage" : "could not find member to sync from", 91 "electionTime" : Timestamp(1560997310, 1), 92 "electionDate" : ISODate("2019-06-20T02:21:50Z"), 93 "configVersion" : 1, 94 "self" : true, 95 "lastHeartbeatMessage" : "" 96 } 97 ], 98 "ok" : 1, 99 "operationTime" : Timestamp(1560997312, 2), 100 "$clusterTime" : { 101 "clusterTime" : Timestamp(1560997312, 2), 102 "signature" : { 103 "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), 104 "keyId" : NumberLong(0) 105 } 106 } 107 }
结果: 可以看出三个服务器：1个Primary，1个Secondary，1一个Arbiter
配置登录认证的用户请按照 Shard1 的步骤
7、配置mongos服务器去连接分片集群
mongos.example.net 服务器上mongos的配置文件 /data/mongodb/mongos.conf
systemLog: destination: file path: "/data/mongodb/log/mongos.log" logAppend: true processManagement: fork: true net: port: 27017 bindIp: mongos.example.net sharding: configDB: "cs0/mongo1.example.net:27027,mongo2.example.net:27027,mongo3.example.net:27027" security: keyFile: "/data/mongodb/keyfile" clusterAuthMode: "keyFile"
启动mongos服务
mongos -f /data/mongodb/mongos.conf
连接mongos
mongo --host mongos.example.net --port 27017 -u myUserAdmin --authenticationDatabase "admin" -p 'abc123'
查看当前集群结果：
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") } shards: active mongoses: autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true }
在集群中先加入Shard1、Shard2，剩余Shard3我们在插入数据有在进行加入（模拟实现扩容）。
sh.addShard("shard1/mongo1.example.net:27017,mongo2.example.net:27017,mongo3.example.net:27017") sh.addShard("shard2/mongo1.example.net:27018,mongo2.example.net:27018,mongo3.example.net:27018")
结果：
1 mongos> sh.addShard("shard1/mongo1.example.net:27017,mongo2.example.net:27017,mongo3.example.net:27017") 2 { 3 "shardAdded" : "shard1", 4 "ok" : 1, 5 "operationTime" : Timestamp(1561009140, 7), 6 "$clusterTime" : { 7 "clusterTime" : Timestamp(1561009140, 7), 8 "signature" : { 9 "hash" : BinData(0,"2je9FsNfMfBMHp+X/6d98B5tLH8="), 10 "keyId" : NumberLong("6704442493062086684") 11 } 12 } 13 } 14 mongos> sh.addShard("shard2/mongo1.example.net:27018,mongo2.example.net:27018,mongo3.example.net:27018") 15 { 16 "shardAdded" : "shard2", 17 "ok" : 1, 18 "operationTime" : Timestamp(1561009148, 5), 19 "$clusterTime" : { 20 "clusterTime" : Timestamp(1561009148, 6), 21 "signature" : { 22 "hash" : BinData(0,"8FvJuCy8kCrMu5nB9PYILj0bzLk="), 23 "keyId" : NumberLong("6704442493062086684") 24 } 25 } 26 }
查看集群的状态
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.example.net:27017,mongo2.example.net:27017", "state" : 1 } { "_id" : "shard2", "host" : "shard2/mongo2.example.net:27018,mongo3.example.net:27018", "state" : 1 } active mongoses: "4.0.10" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true }
8、测试
为了便于测试，设置分片chunk的大小为1M
use config db.settings.save({"_id":"chunksize","value":1})
在连接mongos后，执行创建数据库，并启用分片存储
sh.enableSharding("user_center")
创建 "user_center"数据库，并启用分片，查看结果：
1 mongos> sh.status() 2 --- Sharding Status --- 3 sharding version: { 4 "_id" : 1, 5 "minCompatibleVersion" : 5, 6 "currentVersion" : 6, 7 "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") 8 } 9 shards: 10 { "_id" : "shard1", "host" : "shard1/mongo1.example.net:27017,mongo2.example.net:27017", "state" : 1 } 11 { "_id" : "shard2", "host" : "shard2/mongo2.example.net:27018,mongo3.example.net:27018", "state" : 1 } 12 active mongoses: 13 "4.0.10" : 1 14 autosplit: 15 Currently enabled: yes 16 balancer: 17 Currently enabled: yes 18 Currently running: no 19 Failed balancer rounds in last 5 attempts: 0 20 Migration Results for the last 24 hours: 21 No recent migrations 22 databases: 23 { "_id" : "config", "primary" : "config", "partitioned" : true } 24 config.system.sessions 25 shard key: { "_id" : 1 } 26 unique: false 27 balancing: true 28 chunks: 29 shard1 1 30 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 31 { "_id" : "user_center", "primary" : "shard1", "partitioned" : true, "version" : { "uuid" : UUID("3b05ccb5-796a-4e9e-a36e-99b860b6bee0"), "lastMod" : 1 } }
创建 "users" 集合
sh.shardCollection("user_center.users",{"name":1}) #数据库user_center中users集合使用了片键{"name":1}，这个片键通过字段name的值进行数据分配
现在查看集群状态
1 mongos> sh.status() 2 --- Sharding Status --- 3 sharding version: { 4 "_id" : 1, 5 "minCompatibleVersion" : 5, 6 "currentVersion" : 6, 7 "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") 8 } 9 shards: 10 { "_id" : "shard1", "host" : "shard1/mongo1.example.net:27017,mongo2.example.net:27017", "state" : 1 } 11 { "_id" : "shard2", "host" : "shard2/mongo2.example.net:27018,mongo3.example.net:27018", "state" : 1 } 12 active mongoses: 13 "4.0.10" : 1 14 autosplit: 15 Currently enabled: yes 16 balancer: 17 Currently enabled: yes 18 Currently running: no 19 Failed balancer rounds in last 5 attempts: 0 20 Migration Results for the last 24 hours: 21 No recent migrations 22 databases: 23 { "_id" : "config", "primary" : "config", "partitioned" : true } 24 config.system.sessions 25 shard key: { "_id" : 1 } 26 unique: false 27 balancing: true 28 chunks: 29 shard1 1 30 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 31 { "_id" : "user_center", "primary" : "shard2", "partitioned" : true, "version" : { "uuid" : UUID("33c79b3f-aa18-4755-a5e8-b8f7f3d05893"), "lastMod" : 1 } } 32 user_center.users 33 shard key: { "name" : 1 } 34 unique: false 35 balancing: true 36 chunks: 37 shard2 1 38 { "name" : { "$minKey" : 1 } } -->> { "name" : { "$maxKey" : 1 } } on : shard2 Timestamp(1, 0)
写pyhton脚本插入数据
#enconding:utf8 import pymongo,string,random def random_name(): str_args = string.ascii_letters name_list = random.sample(str_args,5) random.shuffle(name_list) return ''.join(name_list) def random_age(): age_args = string.digits age_list = random.sample(age_args,2) random.shuffle(age_list) return int(''.join(age_list)) def insert_data_to_mongo(url,dbname,collections_name): print(url) client = pymongo.MongoClient(url) db = client[dbname] collections = db[collections_name] for i in range(1,100000): name = random_name() collections.insert({"name" : name , "age" : random_age(), "status" : "pending"}) print("insert ",name) if __name__ == "__main__": mongo_url="mongodb://myUserAdmin:abc123@192.168.11.10:27017/?maxPoolSize=100&minPoolSize=10&maxIdleTimeMS=600000" mongo_db="user_center" mongo_collections="users" insert_data_to_mongo(mongo_url,mongo_db,mongo_collections)
插入数据后查看此时集群的状态：
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.example.net:27017,mongo2.example.net:27017", "state" : 1 } { "_id" : "shard2", "host" : "shard2/mongo2.example.net:27018,mongo3.example.net:27018", "state" : 1 } active mongoses: "4.0.10" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: 3 : Success databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "user_center", "primary" : "shard2", "partitioned" : true, "version" : { "uuid" : UUID("33c79b3f-aa18-4755-a5e8-b8f7f3d05893"), "lastMod" : 1 } } user_center.users shard key: { "name" : 1 } unique: false balancing: true chunks: shard1 9 shard2 8 { "name" : { "$minKey" : 1 } } -->> { "name" : "ABXEw" } on : shard1 Timestamp(2, 0) { "name" : "ABXEw" } -->> { "name" : "EKdCt" } on : shard1 Timestamp(3, 11) { "name" : "EKdCt" } -->> { "name" : "ITgcx" } on : shard1 Timestamp(3, 12) { "name" : "ITgcx" } -->> { "name" : "JKoOz" } on : shard1 Timestamp(3, 13) { "name" : "JKoOz" } -->> { "name" : "NSlcY" } on : shard1 Timestamp(4, 2) { "name" : "NSlcY" } -->> { "name" : "RbrAy" } on : shard1 Timestamp(4, 3) { "name" : "RbrAy" } -->> { "name" : "SQvZq" } on : shard1 Timestamp(4, 4) { "name" : "SQvZq" } -->> { "name" : "TxpPM" } on : shard1 Timestamp(3, 4) { "name" : "TxpPM" } -->> { "name" : "YEujn" } on : shard1 Timestamp(4, 0) { "name" : "YEujn" } -->> { "name" : "cOlra" } on : shard2 Timestamp(3, 9) { "name" : "cOlra" } -->> { "name" : "dFTNS" } on : shard2 Timestamp(3, 10) { "name" : "dFTNS" } -->> { "name" : "hLwFZ" } on : shard2 Timestamp(3, 14) { "name" : "hLwFZ" } -->> { "name" : "lVQzu" } on : shard2 Timestamp(3, 15) { "name" : "lVQzu" } -->> { "name" : "mNLGP" } on : shard2 Timestamp(3, 16) { "name" : "mNLGP" } -->> { "name" : "oILav" } on : shard2 Timestamp(3, 7) { "name" : "oILav" } -->> { "name" : "wJWQI" } on : shard2 Timestamp(4, 1) { "name" : "wJWQI" } -->> { "name" : { "$maxKey" : 1 } } on : shard2 Timestamp(3, 1)
可以看出，数据分别再Shard1、Shard2分片上。
将Shard3分片也加入到集群中来
mongos> sh.addShard("shard3/mongo1.example.net:27019,mongo2.example.net:27019,mongo3.example.net:27019")
在查看集群的状态：
mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5d0af6ed4fa51757cd032108") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.example.net:27017,mongo2.example.net:27017", "state" : 1 } { "_id" : "shard2", "host" : "shard2/mongo2.example.net:27018,mongo3.example.net:27018", "state" : 1 } { "_id" : "shard3", "host" : "shard3/mongo1.example.net:27019,mongo3.example.net:27019", "state" : 1 } active mongoses: "4.0.10" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: 8 : Success databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "user_center", "primary" : "shard2", "partitioned" : true, "version" : { "uuid" : UUID("33c79b3f-aa18-4755-a5e8-b8f7f3d05893"), "lastMod" : 1 } } user_center.users shard key: { "name" : 1 } unique: false balancing: true chunks: shard1 6 shard2 6 shard3 5 { "name" : { "$minKey" : 1 } } -->> { "name" : "ABXEw" } on : shard3 Timestamp(5, 0) { "name" : "ABXEw" } -->> { "name" : "EKdCt" } on : shard3 Timestamp(7, 0) { "name" : "EKdCt" } -->> { "name" : "ITgcx" } on : shard3 Timestamp(9, 0) { "name" : "ITgcx" } -->> { "name" : "JKoOz" } on : shard1 Timestamp(9, 1) { "name" : "JKoOz" } -->> { "name" : "NSlcY" } on : shard1 Timestamp(4, 2) { "name" : "NSlcY" } -->> { "name" : "RbrAy" } on : shard1 Timestamp(4, 3) { "name" : "RbrAy" } -->> { "name" : "SQvZq" } on : shard1 Timestamp(4, 4) { "name" : "SQvZq" } -->> { "name" : "TxpPM" } on : shard1 Timestamp(5, 1) { "name" : "TxpPM" } -->> { "name" : "YEujn" } on : shard1 Timestamp(4, 0) { "name" : "YEujn" } -->> { "name" : "cOlra" } on : shard3 Timestamp(6, 0) { "name" : "cOlra" } -->> { "name" : "dFTNS" } on : shard3 Timestamp(8, 0) { "name" : "dFTNS" } -->> { "name" : "hLwFZ" } on : shard2 Timestamp(3, 14) { "name" : "hLwFZ" } -->> { "name" : "lVQzu" } on : shard2 Timestamp(3, 15) { "name" : "lVQzu" } -->> { "name" : "mNLGP" } on : shard2 Timestamp(3, 16) { "name" : "mNLGP" } -->> { "name" : "oILav" } on : shard2 Timestamp(8, 1) { "name" : "oILav" } -->> { "name" : "wJWQI" } on : shard2 Timestamp(4, 1) { "name" : "wJWQI" } -->> { "name" : { "$maxKey" : 1 } } on : shard2 Timestamp(6, 1)
加入后，集群的分片数据重新平衡调整，有一部分数据分布到Shard3上。
9、备份和恢复
备份
备份的时候需要锁定配置服务器（ConfigServer）和分片服务器（Shard）
在备份前查看当前数据库中数据总条数
mongos> db.users.find().count() 99999
然后启动前面的python脚本，可以在脚本中添加time.sleep来控制插入的频率。
在mongos服务器上停止平衡器。
mongos> sh.stopBalancer()
锁定配置服务器和各分片服务器，登录配置服务器和各分片服务器的Secondary执行命令
db.fsyncLock()
开始备份数据库
mongodump -h mongo2.example.net --port 27027 --authenticationDatabase admin -u myUserAdmin -p abc123 -o /data/backup/config mongodump -h mongo2.example.net --port 27017 --authenticationDatabase admin -u myUserAdmin -p abc123 -o /data/backup/shard1 mongodump -h mongo3.example.net --port 27018 --authenticationDatabase admin -u myUserAdmin -p abc123 -o /data/backup/shard2 mongodump -h mongo1.example.net --port 27019 --authenticationDatabase admin -u myUserAdmin -p abc123 -o /data/backup/shard3
锁定配置服务器和各分片服务器
db.fsyncUnlock()
在mongos中开启平衡器
sh.setBalancerState(true);
在备份的过程中不会影响到数据的写入，备份后查看此时的数据
mongos> db.users.find().count()
107874
恢复
将Shard1分片服务器1中的数据库删除
shard1:PRIMARY> use user_center switched to db user_center shard1:PRIMARY> db.dropDatabase() { "dropped" : "user_center", "ok" : 1, "operationTime" : Timestamp(1561022404, 2), "$gleStats" : { "lastOpTime" : { "ts" : Timestamp(1561022404, 2), "t" : NumberLong(2) }, "electionId" : ObjectId("7fffffff0000000000000002") }, "lastCommittedOpTime" : Timestamp(1561022404, 1), "$configServerState" : { "opTime" : { "ts" : Timestamp(1561022395, 1), "t" : NumberLong(2) } }, "$clusterTime" : { "clusterTime" : Timestamp(1561022404, 2), "signature" : { "hash" : BinData(0,"GO1yQDvdZ6oJBXdvM94noPNnJTM="), "keyId" : NumberLong("6704442493062086684") } } }
然后使用刚备份的数据库进行恢复
mongorestore -h mongo1.example.net --port 27017 --authenticationDatabase admin -u myUserAdmin -p abc123 -d user_center /data/backup/shard1/user_center
2019-06-20T17:20:34.325+0800 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2019-06-20T17:20:34.326+0800 building a list of collections to restore from /data/backup/shard1/user_center dir
2019-06-20T17:20:34.356+0800 reading metadata for user_center.users from /data/backup/shard1/user_center/users.metadata.json
2019-06-20T17:20:34.410+0800 restoring user_center.users from /data/backup/shard1/user_center/users.bson
2019-06-20T17:20:36.836+0800 restoring indexes for collection user_center.users from metadata
2019-06-20T17:20:37.093+0800 finished restoring user_center.users (30273 documents)
2019-06-20T17:20:37.093+0800 done
根据上述步骤恢复Shard2、Shard3的数据
最后恢复的结果：
mongos> db.users.find().count() 100013
这个应该是我在锁的时候插入的数据。
