linux运维、架构之路-MySQL备份与恢复(四)
一、备份方式
①逻辑备份(文件表示:SQL语句)
②物理备份(数据文件的二进制副本)
③基于快照的备份
④基于复制的备份
二、备份工具
①mysqldump:原生自带的逻辑备份工具
②mysqlbinlog:实现binlog备份的原生态命令
③xtrabackup:precona公司开发的性能很高的物理备份工具
三、Mysqldump
1、常用参数
-A, --all-databases #备份所有库 -B 增加建库(create)及"use库"的语句,可以接多个库名,同时备份多个库 -B 库1 库2 -d #仅备份表结构 -t #仅备份数据 --compact #减少无用数据输出(调试) -R, --routines #备份存储过程和函数数据 --triggers #备份触发器数据 -F,--flush-logs #刷新binlog日志 --master-data=1|2 #告诉备份时刻的binlog位置 -x, --lock-all-tables #锁所有备份表 -l, --lock-tables #锁单表
2、参数使用详解
①全库备份
mysqldump -uroot -p123456 -A >/backup/full.sql
②备份多个库
mysqldump -uroot -p123456 -B oldboy oldgirl>/backup/oldboy_oldgirl.sql
③分库备份
for name in `mysql -uroot -p123456 -e "show databases;"|sed 1d` do mysqldump -uroot -p'123456' -B $name done
④单表备份
mysqldump -uroot -p123456 oldboy test>/backup/oldboy_test.sql
⑤多表备份
mysqldump -uroot -p123456 oldboy 表1 表2 表3 …… /backup/oldboy_test.sql
⑥分库分表备份
#!/bin/sh Myuser=root Mypass=123456 Mycmd="mysql -u$Myuser -p$Mypass" Mydump="mysqldump -u$Myuser -p$Mypass -x -F -R" Dblist=`$Mycmd -e "show databases;"|sed '1,2d'|egrep -v "_schema|mysql"` for database in $Dblist do Tablist=`$Mycmd -e "show tables from $database;"|sed 1d` for table in $Tablist do mkdir -p /tmp/${database} $Mydump $database $table|gzip >/tmp/${database}/${table}_$(date +%F).sql.gz done done
⑦压缩备份
mysqldump -uroot -p123456 -B --master-data=2 oldboy|gzip >/backup/oldboy.sql.gz
⑧innodb引擎备份命令
mysqldump -uroot -p123456 -A -B -R --triggers --master-data=2 --single-transaction|gzip >/backup/all.sql.gz
⑨适合多引擎混合备份
mysqldump -uroot -p123456 -A -B -R --triggers --master-data=2|gzip >/backup/alL_$(date +%F).sql.gz
3、mysqldump备份恢复实战案例
===================================================
环境:正在运行的网站,mysql数据库,数据量25G,日业务增量10-15M
[root@db ~]# cat /etc/redhat-release CentOS release 6.9 (Final) [root@db ~]# uname -r 2.6.32-696.el6.x86_64 [root@db ~]# getenforce Disabled [root@db ~]# hostname -I 172.19.5.56 172.16.1.56 [root@db ~]# mysql -V mysql Ver 14.14 Distrib 5.6.36, for linux-glibc2.5 (x86_64) using EditLine wrapper
备份方式:每天晚上00:00,计划任务调用mysqldump执行全备脚本
mysqldump -uroot -p123456 -A -B -R --triggers --master-data=2 |gzip >/backup/all_$(date +%F_%T).sql.gz
故障时间点:第二天上午10点,某开发人员误删除了一个表
如何恢复?
=====================================================
解题思路:
1、使用测试库,恢复全备 source
2、恢复从00:00到10点之间的binlog
a.截取00:00到10点这段 binlog在测试库恢复
b.导出删除表
3、将删除的表进行恢复到生产库
①模拟数据
create database oldboy;#创建oldboy库 use oldboy; create table test( id int(4) not null,name char(20) not null);#在oldboy库中创建test表 insert into test values(1,"oldboy");#test表中插入数据 insert into test values(2,"oldgirl"); insert into test values(3,"inca"); insert into test values(4,"zuma"); insert into test values(5,"kaka"); insert into test values(6,"老男孩");
mysql> select * from oldboy.test; +----+-----------+ | id | name | +----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | +----+-----------+
②模拟全备之后增量数据
mysql -e "use oldboy;insert into test values(7,'bingbing');" mysql -e "use oldboy;insert into test values(8,'xiaoting');" mysql -e "select * from oldboy.test;" +----+-----------+ | id | name | +----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | | 7 | bingbing | | 8 | xiaoting | +----+-----------+
③误删除数据
[root@db ~]# date -s "2018/01/23 10:00" mysql> use oldboy; Database changed mysql> drop table test; mysql> show tables; Empty set (0.00 sec)
④开始恢复准备
iptables -I INPUT -p tcp --dport 3306 ! -s 172.19.5.56 -j DROP #非172.19.5.56禁止访问数据库3306端口
收集恢复所需数据
cp -a /data/mysql/mysql-bin.* /backup/ #copy所有的binlog日志到/backup下面,binlog指定在哪里看配置文件即可 gzip -d all_2018-01-22_00\:00\:05.sql.gz #解压晚上00:00点的全备 [root@db backup]# sed -n '22p' all_2018-01-22_00\:00\:05.sql #提取出binlog位置点 -- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=658960;
mysqlbinlog -d oldboy --start-position=658960 mysql-bin.000002 -r bin.sql #截取binlog位置点 [root@db backup]# ll /backup/ #收集到最终全备及增量恢复所需的数据 total 1952 -rw-r--r-- 1 root root 656383 2018-01-22 00:00 all_2018-01-22_00:00:05.sql -rw-rw---- 1 root root 9864 2018-01-23 09:17 bin.sql -rw-rw---- 1 mysql mysql 143 2018-01-22 10:54 mysql-bin.000001 -rw-rw---- 1 mysql mysql 1317215 2018-01-23 09:02 mysql-bin.000002
⑤恢复全备
mysql <all_2018-01-22_00\:00\:05.sql [root@db backup]# mysql -e "select * from oldboy.test;" +----+-----------+ | id | name | +----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | +----+-----------+
⑥恢复增量数据
[root@db backup]# mysql<bin.sql [root@db backup]# mysql -e "select * from oldboy.test;" +----+-----------+ | id | name | +----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | | 7 | bingbing | | 8 | xiaoting | +----+-----------+
恢复数据完毕,调整iptables允许用户访问,截取多个binlog问题
mysqlbinlog -d oldboy --start-position=339 mysql-bin.000001 mysql-bin.0000002 -r bin.sql
四、XtraBackup
xtrabackup:是一款基于InnoDB的在线热备工具,具有开源,免费,支持在线热备,占用磁盘空间小,能够非常快速地备份与恢复mysql数据库,(备份时不影响数据读写)
1、工作原理
2、安装部署
①添加yum源安装依赖
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo yum -y install perl perl-devel libaio libaio-devel perl-Time-HiRes perl-DBD-MySQL
②下载安装xtrabackup
wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.4/binary/redhat/6/x86_64/percona-xtrabackup-24-2.4.4-1.el6.x86_64.rpm yum -y install percona-xtrabackup-24-2.4.4-1.el6.x86_64.rpm
3、XtraBackup备份恢复实战
①全备
innobackupex --no-timestamp /backup/full
由于备份命令依赖于/etc/my.cnf文件中basedir datadir socket,如配置文件中未配置,需在命令行指定
[client] user = root password = 123456 [mysqld] log-bin=/data/mysql/mysql-bin character-set-server = utf8 basedir = /application/mysql/ datadir = /application/mysql/data/ port = 3306 socket = /tmp/mysql.sock
②应用日志到全备中(预处理数据)
innobackupex --apply-log /backup/full/
③模拟数据损坏
rm -fr /application/mysql/data/*
pkill mysql
④恢复全备数据
innobackupex --copy-back /backup/full/ chown -R mysql.mysql /application/mysql-5.6.36/ #这里重新授权注意一下,有时候发现授权软链接不管用
⑤查看恢复结果
[root@db ~]# mysql -e "select * from oldboy.test;" +----+-----------+ | id | name | +----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | | 7 | bingbing | | 8 | xiaoting | +----+-----------+
innobackupex命令常用参数说明
--defaults-file #指明服务器的配置文件,此参数必须作为innobackupex的第一个参数,否则报错 --user #备份的用户名 --password #指明备份用户名的密码 /backup #备份的目录 --apply-log #指明为重做日志 --copy-back #指明为恢复 --slave-info #备份从库的show slave status信息,仅用于在备份从库时使用 --no-lock #不锁表,仅适用于存储引擎为innodb,并且不在乎备份位置点时使用 --no-timestamp#不按时间生成备份目录
4、XtraBackup增量备份恢复实战
=================================================
场景:xtrabackup全备+增量
备份策略:定时任务周日晚上00点全备,周一到周六都是基于上一天的增量备份
故障场景:周三的时候,下午两点,某开发小菜鸟误删除了一张test表,大于2G
解决思路:
①准备上周日全备,并--apply-log --redo-only
②合并增量,周一 、周二 --apply-log --redo-only 周三 --apply-log
③在测试库恢复以上数据,数据的目前状态应该周三凌晨1:00
④需要恢复的数据状态是,下午2点钟左右,删除test表之前的数据状态,从1点开始的binlog恢复到删除之前的那个events的position
⑤导出删除的表test,恢复到生产库,验证数据可用性、完整性。
⑥启动应用连接数据库
==================================================
①模拟案例
mkdir /backup/inc{1,2} -p #创建增量备份的目录 innobackupex --no-timestamp /backup/full/ #周日全备 #模拟插入数据# insert into test values(9,'outman'); insert into test values(10,'man');
②周一增量备份
innobackupex --incremental --no-timestamp --incremental-basedir=/backup/full/ /backup/inc1
③再次模拟插入数据
insert into test values(11,'110'); insert into test values(12,'120');
④周二增量备份
innobackupex --incremental --no-timestamp --incremental-basedir=/backup/inc1 /backup/inc2
⑤再次模拟数据变化
insert into test values(130,'newdata1'); insert into test values(140,'newdata2');
此时表中内容为
mysql> select * from test; +-----+-----------+ | id | name | +-----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | | 7 | bingbing | | 8 | xiaoting | | 9 | outman | | 10 | man | | 11 | 110 | | 12 | 120 | | 130 | newdata1 | | 140 | newdata2 | +-----+-----------+ 14 rows in set (0.00 sec)
⑥故障场景模拟:下午2点误删除test表
mysql> use oldboy; Database changed mysql> drop table test; Query OK, 0 rows affected (0.01 sec) mysql> select * from test; ERROR 1146 (42S02): Table 'oldboy.test' doesn't exist
故障恢复:
①全备进行apply-log,暂时不需要undo回滚,只做redo
innobackupex --apply-log --redo-only /backup/full
②将周一的inc1合并到全备当中去,暂时不需要undo回滚,只做redo
innobackupex --apply-log --redo-only --incremental-dir=/backup/inc1 /backup/full
③将周二的inc2合并到全备当中去,undo和redo都apply
innobackupex --apply-log --incremental-dir=/backup/inc2 /backup/full
④最终把所有合并后的备份集合做apply
innobackupex --apply-log /backup/full
⑤截取周二增量备份inc2之后产生的binlog,截取到delete之前,即可恢复故障
确认binlog起点
[root@db ~]# cat /backup/inc2/xtrabackup_binlog_info mysql-bin.000017 1022
截取drop操作之前的binlog
mysqlbinlog --start-position=1022 --stop-position=1484 /data/mysql/mysql-bin.000017 >/backup/incbinlog.sql
使用命令查看binlog的event
mysql> show binlog events in 'mysql-bin.000017'; +------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+ | mysql-bin.000017 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.36-log, Binlog ver: 4 | | mysql-bin.000017 | 120 | Query | 1 | 203 | BEGIN | | mysql-bin.000017 | 203 | Query | 1 | 316 | use `oldboy`; insert into test values(9,'outman') | | mysql-bin.000017 | 316 | Xid | 1 | 347 | COMMIT /* xid=37 */ | | mysql-bin.000017 | 347 | Query | 1 | 430 | BEGIN | | mysql-bin.000017 | 430 | Query | 1 | 541 | use `oldboy`; insert into test values(10,'man') | | mysql-bin.000017 | 541 | Xid | 1 | 572 | COMMIT /* xid=38 */ | | mysql-bin.000017 | 572 | Query | 1 | 655 | BEGIN | | mysql-bin.000017 | 655 | Query | 1 | 766 | use `oldboy`; insert into test values(11,'110') | | mysql-bin.000017 | 766 | Xid | 1 | 797 | COMMIT /* xid=57 */ | | mysql-bin.000017 | 797 | Query | 1 | 880 | BEGIN | | mysql-bin.000017 | 880 | Query | 1 | 991 | use `oldboy`; insert into test values(12,'120') | | mysql-bin.000017 | 991 | Xid | 1 | 1022 | COMMIT /* xid=58 */ | | mysql-bin.000017 | 1022 | Query | 1 | 1105 | BEGIN | | mysql-bin.000017 | 1105 | Query | 1 | 1222 | use `oldboy`; insert into test values(130,'newdata1') | | mysql-bin.000017 | 1222 | Xid | 1 | 1253 | COMMIT /* xid=77 */ | | mysql-bin.000017 | 1253 | Query | 1 | 1336 | BEGIN | | mysql-bin.000017 | 1336 | Query | 1 | 1453 | use `oldboy`; insert into test values(140,'newdata2') | | mysql-bin.000017 | 1453 | Xid | 1 | 1484 | COMMIT /* xid=78 */ | | mysql-bin.000017 | 1484 | Query | 1 | 1605 | use `oldboy`; DROP TABLE `test` /* generated by server */ | +------------------+------+-------------+-----------+-------------+-----------------------------------------------------------+ 20 rows in set (0.00 sec)
停库,备份binlog日志,本实例中binlog日志单独存放在/data/mysql目录下,无需备份
[root@db ~]# /etc/init.d/mysqld stop Shutting down MySQL.. SUCCESS! [root@db ~]# rm -fr /application/mysql/data/*
恢复xtrabackup数据
innobackupex --copy-back /backup/full/ chown -R mysql.mysql /application/mysql-5.6.36/ /etc/init.d/mysqld start
恢复binlog
mysql> set sql_log_bin=0; #临时不记录binlog Query OK, 0 rows affected (0.00 sec) mysql> source /backup/incbinlog.sql Query OK, 0 rows affected (0.00 sec)
⑥故障恢复完毕
mysql> select * from oldboy.test; +-----+-----------+ | id | name | +-----+-----------+ | 1 | oldboy | | 2 | oldgirl | | 3 | inca | | 4 | zuma | | 5 | kaka | | 6 | 老男孩 | | 7 | bingbing | | 8 | xiaoting | | 9 | outman | | 10 | man | | 11 | 110 | | 12 | 120 | | 130 | newdata1 | | 140 | newdata2 | +-----+-----------+ 14 rows in set (0.00 sec)