mongodb时间点恢复
环境:
Os:Centos 7
mongodb:5.0.19
单节点的副本集(注意:这里必须是副本集环境,否则没有oplog.rs这个表)
我这里新机器也部署单节点的副本集
1.写入数据
/usr/local/services/mongodb/bin/mongo 192.168.1.105:29001
> use db_pushmsg
> db.tb_test.insert({"id":"1"})
> db.tb_test.insert({"id":"2"})
> db.tb_test.insert({"id":"3"})
2.备份表
4.4版本后面的mongodump需要单独安装,可参考链接https://www.cnblogs.com/hxlasky/p/15984805.html
创建导出文件存放目录
[root@localhost bin]#mkdir -p /tmp/tb_test
导出
[root@localhost bin]# ./mongodump -h 192.168.1.105:29001 --db=db_pushmsg -c tb_test -o /tmp/tb_test
2023-07-31T01:57:31.484-0400 writing db_pushmsg.tb_test to /tmp/tb_test/db_pushmsg/tb_test.bson
2023-07-31T01:57:31.485-0400 done dumping db_pushmsg.tb_test (3 documents)
3.再插入几条数据
s1:PRIMARY> db.tb_test.insert({"id":"4"})
s1:PRIMARY> db.tb_test.insert({"id":"5"})
s1:PRIMARY> db.tb_test.insert({"id":"6"})
s1:PRIMARY> db.tb_test.insert({"id":"7"})
s1:PRIMARY> db.tb_test.insert({"id":"8"})
4.导出 oplog.rs
mkdir /tmp/oplog
[root@localhost bin]# ./mongodump --host=127.0.0.1 --port 29001 -d local -c oplog.rs -o /tmp/oplog
2023-07-31T02:01:10.252-0400 writing local.oplog.rs to /tmp/oplog/local/oplog.rs.bson
2023-07-31T02:01:13.246-0400 local.oplog.rs 626278
2023-07-31T02:01:15.395-0400 local.oplog.rs 1003194
2023-07-31T02:01:15.397-0400 done dumping local.oplog.rs (1003194 documents)
该导出会导出整个oplog.rs日志,不单是表tb_test的日志而已,恢复的时候也会把其他的表一起恢复了.
导出时间范围的oplog日志
./mongodump --host=127.0.0.1 --port 29001 -d local -c oplog.rs -q '{ts:{$lt:Timestamp(1575793969, 1),$gt: Timestamp(1575793846, 1)}}' -o /tmp/oplog
./mongodump --host=127.0.0.1 --port=29001 -d local -c "oplog.rs" -q '{"ts":{"$gt": {"$timestamp":{"t":1653924081, "i":1}}}}' --out /tmp/oplog/
./mongodump --host=127.0.0.1 --port=29001 -d local -c oplog.rs --query '{ts:{$gte:Timestamp('1653924081', 1)}}' --out /tmp/oplog/
5.将日志导出文件
./bsondump /tmp/oplog/local/oplog.rs.bson>/tmp/oplog.txt
查找日志点恢复到id=5的记录
cat /tmp/oplog.txt|grep db_pushmsg.tb_test
[root@localhost bin]# cat /tmp/oplog.txt|grep db_pushmsg.tb_test
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74d0655dc04566cda3870"},"id":"1"},"ts":{"$timestamp":{"t":1690782982,"i":3}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690782982779"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74d0c55dc04566cda3871"},"id":"2"},"ts":{"$timestamp":{"t":1690782988,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690782988401"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74d1355dc04566cda3872"},"id":"3"},"ts":{"$timestamp":{"t":1690782995,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690782995057"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74db455dc04566cda3873"},"id":"4"},"ts":{"$timestamp":{"t":1690783156,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690783156983"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74dba55dc04566cda3874"},"id":"5"},"ts":{"$timestamp":{"t":1690783162,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690783162330"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74dc055dc04566cda3875"},"id":"6"},"ts":{"$timestamp":{"t":1690783168,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690783168543"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74dc655dc04566cda3876"},"id":"7"},"ts":{"$timestamp":{"t":1690783174,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690783174104"}}}
{"op":"i","ns":"db_pushmsg.tb_test","ui":{"$binary":{"base64":"4nOQcBqeQZG7wYakp1rIrA==","subType":"04"}},"o":{"_id":{"$oid":"64c74dca55dc04566cda3877"},"id":"8"},"ts":{"$timestamp":{"t":1690783178,"i":1}},"t":{"$numberLong":"2"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1690783178719"}}}
那我们要恢复到 1690783168(不包含这个时间点) 这个时间点
6.把备份文件和oplog文件拷贝到新的服务器
scp /tmp/tb_test/db_pushmsg/tb_test.bson root@192.168.1.107:/tmp/tb_test/db_pushmsg/
scp /tmp/tb_test/db_pushmsg/tb_test.metadata.json root@192.168.1.107:/tmp/tb_test/db_pushmsg/
scp /tmp/oplog/local/oplog.rs.bson root@192.168.1.107:/tmp/oplog/local/
scp /tmp/oplog/local/oplog.rs.metadata.json root@192.168.1.107:/tmp/oplog/local/
7:先恢复表:
把上面步骤拷贝传输过来的问题拷贝到与备份时候的目录一致
./mongorestore --host=127.0.0.1 --port=29001 -d db_pushmsg /tmp/tb_test/db_pushmsg/
8.恢复oplog到时间戳1489135569
./mongorestore --host 192.168.1.107 --port 29001 --oplogReplay --oplogLimit "1690783168:1" /tmp/oplog/local/oplog.rs.bson
该恢复过程会不但是恢复表tb_test,其他的表也一起恢复到时间点1690783168,若是oplog.rs很大的话,恢复过程需要些时间.
9.查看恢复的数据
/usr/local/services/mongodb/bin/mongo 192.168.1.107:29001
s1:PRIMARY> db.tb_test.find()
{ "_id" : ObjectId("64c74d0655dc04566cda3870"), "id" : "1" }
{ "_id" : ObjectId("64c74d0c55dc04566cda3871"), "id" : "2" }
{ "_id" : ObjectId("64c74d1355dc04566cda3872"), "id" : "3" }
{ "_id" : ObjectId("64c74db455dc04566cda3873"), "id" : "4" }
{ "_id" : ObjectId("64c74dba55dc04566cda3874"), "id" : "5" }
