rsync
Rsync服务
【1】、rsync定义
作用:
1.本地的备份 类似cp命令 意义不大
2.远程的拷贝 类似scp命令 意义不大 (增量备份)
如果第一次远程执行文件拷贝动作使用scp和rsync一样
如果第二次远程执行,有新增加的文件可以使用rsync命令
3.守护进程模式 持续不断地在后台运行,接收客户端发送的重要 数据。类似于百度网盘。
scp 每次都是全量拷贝
rsync 每次都是增量拷贝
# 全量备份
[root@web01 ~]# scp -r test backup:~/
Authorized users only. All activities may be monitored and reported.
01.txt 100% 0 0.0KB/s 00:00
02.txt 100% 0 0.0KB/s 00:00
03.txt 100% 0 0.0KB/s 00:00
04.txt 100% 0 0.0KB/s 00:00
05.txt 100% 0 0.0KB/s 00:00
[root@web01 ~]# touch test/{06..09}.txt
[root@web01 ~]# scp -r test backup:~/
# 全量备份再有新增加的内容时，不仅会备份新增的内容，还会再备份一次就的数据
Authorized users only. All activities may be monitored and reported.
01.txt 100% 0 0.0KB/s 00:00
02.txt 100% 0 0.0KB/s 00:00
03.txt 100% 0 0.0KB/s 00:00
04.txt 100% 0 0.0KB/s 00:00
05.txt 100% 0 0.0KB/s 00:00
06.txt 100% 0 0.0KB/s 00:00
07.txt 100% 0 0.0KB/s 00:00
08.txt 100% 0 0.0KB/s 00:00
09.txt 100% 0 0.0KB/s 00:00
# 增量备份
[root@web01 ~]# rsync -avz test backup:~/
The authenticity of host 'backup (192.168.121.41)' can't be established.
ECDSA key fingerprint is SHA256:ojGuz7nAhVgYaZgcnxgbxZOsIwQOL7DkBGqc38t7hXw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'backup' (ECDSA) to the list of known hosts.
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
test/06.txt
test/07.txt
test/08.txt
test/09.txt
test/10.txt
sent 595 bytes received 210 bytes 230.00 bytes/sec
total size is 0 speedup is 0.00
[root@web01 ~]# touch test/new.txt
[root@web01 ~]# rsync -avz test backup:~/
# 增量备份之备份第一备份后新增加的内容
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/new.txt
sent 296 bytes received 39 byte
【2】、rsync三种工作模式
SYNOPSIS
Local: rsync [OPTION...] SRC... [DEST]
Access via remote shell:
Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
Push: rsync [OPTION...] SRC... [USER@]HOST:DEST
Access via rsync daemon:
Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST
1、本地模式
类似于cp命令，了解即可
2、远程模式
类似于scp命令(在有增量需求拷贝的时候或者有数据同步的要求才使用)
# 注意rsync的目录后面加/表示拷贝目录下的文件 不加/表示拷贝目录及下所有内容
[root@web01 ~]# rsync -avz test1/ backup:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
./
01.txt
02.txt
03.txt
04.txt
sent 258 bytes received 95 bytes 706.00 bytes/sec
total size is 0 speedup is 0.00
[root@web01 ~]# rsync -avz test backup:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
sent 329 bytes received 115 bytes 296.00 bytes/sec
total size is 0 speedup is 0.00
# rsync拉取模式，将远程上的东西拉取到本地
[root@backup ~]# rsync -avz web01:~/test ./
Authorized users only. All activities may be monitored and reported.
receiving incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
sent 123 bytes received 329 bytes 129.14 bytes/sec
total size is 0 speedup is 0.00
在使用rsync时，如果dest path没有加用户，默认使用本地登录的用户去登录远程
[oldboy@web01 ~]$ rsync -avz /etc/hosts backup:~/
# 我们在传递时，需要保证本地和远程都有相同的用户
The authenticity of host 'backup (192.168.121.41)' can't be established.
ECDSA key fingerprint is SHA256:ojGuz7nAhVgYaZgcnxgbxZOsIwQOL7DkBGqc38t7hXw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'backup,192.168.121.41' (ECDSA) to the list of known hosts.
Authorized users only. All activities may be monitored and reported.
oldboy@backup's password:
3、守护进程模式
主机角色 外网IP(WAN) 内网IP(LAN) 主机名称
Rsync服务端 192.168.121.41 172.16.1.41 backup
Rsync客户端 192.168.121.7 172.16.1.7 web01
1、服务端安装rsync服务
[root@backup ~]# yum install -y rsync
2、服务端配置rsync
[root@backup ~]# rpm -qc rsync
/etc/rsyncd.conf
/etc/sysconfig/rsyncd
[root@backup ~]# cat /etc/rsyncd.conf
uid = rsync # 运行进程的用户
gid = rsync # 运行进程的用户组
port = 873 # 监听端口
fake super = yes # 无需让rsync以root的身份运行，允许接受文件的完整属性
use chroot = no # 禁锢推送的数据到某一个目录，不允许跳出该目录
max connections = 200 # 最大连接数
timeout = 600 # 超时时间
ignore errors # 忽略错误信息
read only = false # 对备份数据不是只读的，可读可写
list = false #不许查看模块信息
auth users = rsync_backup # 定义虚拟用户，作为连接认证用户
secrets file = /etc/rsync.passwd # 定义rsync服务用户连接认证密码文件路径
log file = /var/log/rsyncd.log
#####################################
[backup] # 定义模块信息
path = /backup # 定义接收备份数据的目录
3.根据配置文件创建必要的数据信息
[root@backup ~]# grep rsync /etc/passwd
[root@backup ~]#
# 创建虚拟用户rsync
[root@backup ~]# useradd -M -s /sbin/nologin rsync
# 创建密码文件
[root@backup ~]# cat /etc/rsync.passwd
rsync_backup:123456
[root@backup ~]# chmod 600 /etc/rsync.passwd
# 创建接收目录
[root@backup ~]# mkdir /backup
[root@backup ~]# ll /backup -d
drwxr-xr-x 2 root root 6 Dec 3 16:17 /backup
# rsync是以启动进程的用户身份往/backup目录中写入
[root@backup ~]# chown rsync:rsync /backup/
4.服务端启动rsync服务
[root@backup ~]# systemctl enable rsyncd --now
Created symlink /etc/systemd/system/multi-user.target.wants/rsyncd.service → /usr/lib/systemd/system/rsyncd.service.
[root@backup ~]# ss -tunlp | grep rsync
tcp LISTEN 0 5 0.0.0.0:873 0.0.0.0:* users:(("rsync",pid=78018,fd=3))
tcp LISTEN 0 5 [::]:873 [::]:* users:(("rsync",pid=78018,fd=5))
需要使用守护进程模式的语法格式推送文件
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
推送: rsync -avz /etc/passwd rsync_backup@10.0.0.41::模块的名称
[root@web01 ~]# rsync -avz /etc/passwd rsync_backup@backup::backup
Password:
sending incremental file list
passwd
sent 847 bytes received 43 bytes 356.00 bytes/sec
total size is 1,848 speedup is 2.08
客户端参数：--password-file，指定密码文件在哪
[root@web01 ~]# echo 123456 > /etc/pa.txt
[root@web01 ~]# chmod 600 /etc/pa.txt
[root@web01 ~]# rsync -avz /etc/hosts rsync_backup@backup::backup --password-file=/etc/pa.txt
sending incremental file list
hosts
sent 176 bytes received 43 bytes 438.00 bytes/sec
total size is 200 speedup is 0.91
rsync客户端密码内置变量
[root@web01 ~]# echo $RSYNC_PASSWORD [root@web01 ~]# export RSYNC_PASSWORD=123456 [root@web01 ~]# echo $RSYNC_PASSWORD 123456 [root@web01 ~]# rsync -avz /etc/rc.local rsync_backup@backup::backup sending incremental file list rc.local -> rc.d/rc.local sent 62 bytes received 23 bytes 170.00 bytes/sec total size is 13 speedup is 0.15rsync在推送数据时，首先会查RSYNC_PASSWORD
# 案例
1.客户端提前准备存放的备份的目录，目录规则如 下:/backup/web01_172.16.1.7_2018-09-02 date +%F 2.客户端在本地打包备份(系统配置文件、应用配置等)拷贝 至/backup/web01_172.16.1.7_2018-09-02
3.客户端最后将备份的数据进行推送至备份服务器 守护进程
4.客户端每天凌晨1点定时执行该脚本
5.客户端服务器本地保留最近7天的数据, 避免浪费磁盘空间
#!/bin/bash
mkdir -p /backup
IP=`hostname -I | awk -F" " '{print $1}'`
path=/backup/web01_${IP}_`date +%F`
tar -zcvf $path /etc/
rsync -avz $path rsync_backup@backup::backup
find /backup -mtime +7 -exec rm -f {} \;
# 拉取数据
[root@web01 ~]# ls
backup.sh html
[root@web01 ~]# rsync -avz rsync_backup@backup::backup/hosts .
receiving incremental file list
hosts
sent 43 bytes received 180 bytes 446.00 bytes/sec
total size is 200 speedup is 0.90
[root@web01 ~]# ls
backup.sh hosts html
新增模块，可以存储到多个服务的目录下
[root@backup ~]# vim /etc/rsyncd.conf
# 新增一个模块即可
[data]
path = /data
[root@backup ~]# systemctl restart rsyncd
[root@backup ~]# mkdir /data
[root@backup ~]# chown rsync:rsync /data/
# 在客户端进行传递，指定新加的模块名
[root@web01 ~]# rsync -avz /etc/rc.local rsync_backup@backup::data
sending incremental file list
rc.local -> rc.d/rc.local
sent 62 bytes received 23 bytes 170.00 bytes/sec
total size is 13 speedup is 0.15
【3】、rsync参数
-a #归档模式传输, 等于-tropgDl
-v #详细模式输出, 打印速率, 文件数量等
-z #传输时进行压缩以提高效率
-r #递归传输目录及子目录，即目录下得所有目录 都同样传输。
-t #保持文件时间信息
-o #保持文件属主信息
-p #保持文件权限
-g #保持文件属组信息
-l #保留软连接
-P #显示同步的过程及传输时的进度等信息
-D #保持设备文件信息
-L #保留软连接指向的目标文件
-e #使用的信道协议,指定替代rsh的shell程序
--exclude=PATTERN #指定排除不需要传输的文件模式
--exclude-from=file #文件名所在的目录文件
--bwlimit=100 #限速传输
--partial #断点续传
--delete #让目标目录和源目录数据保持一致
--password-file=xxx #使用密码文件，守护进程模式下才可以使用
# 使用--exclude 排除文件，不参与传输
[root@backup ~]# rsync -avz test/ web01:~/ --exclude=43.txt
Authorized users only. All activities may be monitored and reported.
sending incremental file list
./
hosts
passwd
sent 992 bytes received 57 bytes 2,098.00 bytes/sec
total size is 2,005 speedup is 1.91
# 使用{} 排除多个文件
[root@backup ~]# rsync -avz test web01:~/ --exclude={43.txt,passwd}
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/hosts
sent 208 bytes received 39 bytes 164.67 bytes/sec
total size is 200 speedup is 0.81
--exclude-from=file 可以指定文件，安装指定的文件中的名字排除不需要进行传输的文件
[root@backup ~]# vim /opt/ex.txt
[root@backup ~]# cat /opt/ex.txt
passwd
hosts
[root@backup ~]# rsync -avc test web01:~/ --exclude-from=/opt/ex.txt
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/43.txt
sent 136 bytes received 39 bytes 350.00 bytes/sec
total size is 0 speedup is 0.00
--bwlimit 限速传递
[root@backup ~]# dd if=/dev/zero of=/1g.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 10.4023 s, 101 MB/s
[root@backup ~]# rsync -avcP 1g.txt web01:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
1g.txt
448,299,008 42% 26.47MB/s 0:00:22
[root@backup ~]# rsync -avcP 1g.txt web01:~/ --bwlimit=1m
Authorized users only. All activities may be monitored and reported.
sending incremental file list
1g.txt
26,509,312 2% 1.00MB/s 0:16:37
--password-file只能在守护进程模式下使用
--delete 可以让两端服务器的内容保持一致
# 谁在前就以谁为准
[root@backup ~]# rsync -avz web01:~/ . --delete
Authorized users only. All activities may be monitored and reported.
receiving incremental file list
deleting test/passwd
deleting test/hosts
deleting test/43.txt
deleting test/
deleting passwd.txt
deleting 1g.txt
./
.bash_history
.lesshst
.viminfo
.ssh/
.ssh/authorized_keys
.ssh/id_rsa
.ssh/id_rsa.pub
.ssh/known_hosts
sent 364 bytes received 4,451 bytes 9,630.00 bytes/sec
total size is 9,732 speedup is 2.02
# 可以用于快速同步信息，一般用在网站被篡改，可以通过一条命令进行同步
[root@backup ~]# rsync -avz html web01:~/
Authorized users only. All activities may be monitored and reported.
root@web01's password:
sending incremental file list
html/
html/1.html
html/2.html
html/3.html
sent 287 bytes received 77 bytes 104.00 bytes/sec
total size is 45 speedup is 0.12
# 模拟网站被篡改
[root@web01 ~]# find html/ -type f -name "*.html" -exec sed -i 's#haha#???#g' {} \;
[root@web01 ~]# cat html/1.html
www.???ha.com
[root@web01 ~]# cat html/2.html
www.???ha.com
[root@web01 ~]# cat html/3.html
www.???ha.com
# 使用rsync进行同步
[root@backup ~]# rsync -avz html/ web01:~/html --delete
Authorized users only. All activities may be monitored and reported.
root@web01's password:
sending incremental file list
./
1.html
2.html
3.html
sent 276 bytes received 94 bytes 105.71 bytes/sec
total size is 45 speedup is 0.12
[root@web01 ~]# cat html/1.html
www.hahaha.com
[root@web01 ~]# cat html/2.html
www.hahaha.com
[root@web01 ~]# cat html/3.html
www.hahaha.com
本文来自博客园，作者：Linux小菜鸟，转载请注明原文链接：https://www.cnblogs.com/xuruizhao/p/18585478
