Use Rsync Backup Service on openEuler

一、Use Rsync Backup Service on openEuler

1 地址

• https://rsync.samba.org
• https://github.com/WayneD/rsync

二、Rsync 概述

rsync英文称为 remote synchronizetion，rsync具有可使本地和远程两台主机之间的数据快速复制同步镜像、远程备份的功能，功能类似于ssh带的scp命令，优于scp命令的功能，scp每次都是全量拷贝，而rsync可以增量拷贝。

1 rsync 常用的传输工具，多台服务器之间传输数据，具备全量同步和增量同步

• 备份服务：rsync + 定时任务 实现定时备份
• 默认端口号：873
• rsync 运行模式：c/s [clent/server] rsync守护进程模式

假设客户端上有xyz1,xyz2,xyz3 文件，服务端上有xyz1文件，现在将客户端上的数据备份 之服务端

• 完全备份：将客户端所有的数据内容xyz1,xyz2,xyz3 全部备份之服务端（效率低下，占用空间）
• 增量备份：将客户端的xyz2,xyz3 增量备份之服务端（提高备份效率，节省空间，适合异地备份）

2 rsync 的传输方式

• push 推：客户端将数据从本地推送至服务端
• pull   拉：客户端将数据从服务端拉取到本地

3 rsync 传输模式

• 本地方式（类似于cp，不支持推送和拉取，只是单纯的复制）
• 远程方式（类似于scp，又不同于scp），scp只支持全量备份，rsync支持增量备份和差异备份
• 守护进程方式（客户端和服务端）

4 下载 & 安装

## 检查是否安装
rpm -qa |grep rsync
## 查找rsync包
dnf search rsync
## dnf安装
dnf install -y rsync

##
mkdir /opt/software;cd /opt/software
## 使用rpm安装
wget https://repo.openeuler.org/openEuler-22.09/source/Packages/rsync-3.2.3-4.oe2209.src.rpm
## 源码安装
wget https://download.samba.org/pub/rsync/src/rsync-3.2.7.tar.gz

5 本地模式

## /tmp/ 表示目录及目录下面的内容
rsync -a /tmp/ /opt/

## /tmp 目录下面的内容
rsync -a /tmp /opt/

6 远程模式（ssh隧道模式）

[root@backup01 ~]# ## push 推
[root@backup01 ~]# echo " iyuyi.xyz@aliyun.com " > /opt/xyz.txt
[root@backup01 ~]# rsync -a /opt/xyz.txt 10.0.1.55:/opt/
The authenticity of host '10.0.1.55 (10.0.1.55)' can't be established.
ED25519 key fingerprint is SHA256:PwdF+Oj+MFaPo6MbT2R/efN40YyaemXqlZnSluQuUA0.
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:1: 192.168.182.55
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.1.55' (ED25519) to the list of known hosts.

Authorized users only. All activities may be monitored and reported.
root@10.0.1.55's password: 

[root@backup01 ~]# ## pull 拉
[root@backup01 ~]# rsync -av 10.0.1.55:/opt/xyz.txt /root/

Authorized users only. All activities may be monitored and reported.
root@10.0.1.55's password: 
receiving incremental file list
xyz.txt

sent 43 bytes  received 126 bytes  30.73 bytes/sec
total size is 23  speedup is 0.14

[root@backup01 ~]# cat xyz.txt 
 iyuyi.xyz@aliyun.com
 
rm -rf /opt/xyz.txt 
rm -rf /root/xyz.txt

三、Rsync 配置文件

常用配置参数说明

[root@backup01 ~]# ll /etc/rsyncd.conf
-rw-r--r--. 1 root root 458 Aug 26 08:00 /etc/rsyncd.conf

## 启动服务的用户id和用户的组id
uid = rsync
gid = rsync
## 服务默认监听端口
port = 873
## 无需使用root用户启动，解决故障
fake super = yes
## 安全机制选项，关闭
use chroot = no
## 最大连接数
max connections = 200
## 客户端连接超时时间 [为秒 查看帮助文档 man rsyncd.conf]
timeout = 600
## 进程信息文件
pid file = /var/run/rsyncd.pid
## 忽略错误
ignore errors
## 只读权限
read only = false
## 查看模块列表
list = false
## 定义虚拟用户（rsync传输过程使用的用户）
auth users = rsync_backup
## 定义虚拟用户的密码
secrets file = /etc/rsync.passwd

## 获取进程号停止进程
## 判断服务是否启动 避免反复启动
## 锁文件
lock file = /var/run/rsync.lock
## 服务运行时日志文件
log file = /var/log/rsyncd.log
## 指定备份目录的权限为可读可写，false关闭[可读写]
read only = false
## 安全策略配置 设置一个白名单
hosts allow = 10.0.1.51/24
## 安全策略配置 设置一个黑名单
hosts deny = 10.0.1.51/32
## 定义认证用户
auth users = rsync_backup
## 密码文件 用户密码信息 格式为：user:passwd
secrets file = /etc/rsync.password
## 自定模块名称，可以写多个
[backup]
## 模块的备注
comment = welcome to backup!
## 模块对于的位置[路径]
path = /opt/backupxyz

[iyuyixyz]
comment = welcome to iyuyixyz!
path=/tmp/xyz

 

四、Rsync 参数选项

## -v 表示：显示详细信息
## -z 表示：传输是进行压缩提高传输效率
## -a 表示：递归传输数据，保持文件属性和权限

## -r 表示：表递归传输
## -p 表示：保持权限不变

## -t 表示：保持时间信息

## -P 表示：显示同步过程及传输进度信息

## -e 表示：信道协议
echo " huaxiayuyi " > /opt/demo.txt
rsync -avz -e "ssh -p22" /opt/com.txt root@10.0.1.51:/opt/backupxyz

## --delete 表示：目标和源目录数据保持一致 /home/xyz/ 后面得有斜线
rsync -avz --delete  /opt/demo.txt rsync_backup_xyz@::backup --password-file=/etc/rsync.passwd

## --exclude 参数指定排除的文件或目录信息
mkdir /opt/xyz;cd /opt/xyz
touch /opt/xyz/{1..10}.txt
rsync -avz /opt/xyz/ --exclude=10.txt --exclude=9.txt rsync_backup_xyz@10.0.1.51::backup --password-file=/etc/rsync.passwd

## --exclude-from 参数指定排除多个数据信息文件
rsync -avz /home/xyz/ --exclude-from=  rsync_backup_xyz@10.0.1.51::backup–password-file=/etc/rsync.passwd

## bwlimite = 1000 表示：限速传输
## partial    表示：断点续传

 

五、启动 & 测试

Rsync 服务端配置及检测

1 创建配置文件

cat > /etc/rsyncd.conf << EOF
uid = rsyncxyz
gid = rsyncxyz
port = 873
fake super = yes
use chroot = no
max connections = 200
timeout = 600
ignore errors
read only = false
list = false
auth users = rsync_backup_xyz
secrets file = /etc/rsync.passwd
log file = /var/log/rsyncd.log
[backup]
comment = welcome to backup!
path = /opt/backupxyz
EOF

2 用户、密码文件、权限600、创建目录设置

## 增加用户
useradd -s /sbin/nologin -M rsync
id rsyncxyz
## 配置用户名和密码
cat > /etc/rsync.passwd << EOF
rsync_backup_xyz:123
EOF

## -A 查看是否有空格
cat -A /etc/rsync.passwd

## 修改文件权限
chmod 600 /etc/rsync.passwd
ll /etc/rsync.passwd

## 创建共享目录
mkdir /opt/backupxyz
## 修改目录权限
chown -R rsyncxyz:rsyncxyz /opt/backupxyz

ll /opt
drwxr-xr-x. 2 rsyncxyz rsyncxyz 4096 Nov 23 16:52 backupxyz

3 开启服务检查端口进程

## 启动rsync服务
systemctl start rsyncd
## 加入rsync开机启动
systemctl enable rsyncd

## 检查是否已经成功启动
netstat -lntup |grep 873
ss -lntup |grep 873
## 查看进程
ps -ef |grep rsync

systemctl daemon-reload
systemctl start rsyncd
systemctl enable rsyncd
systemctl status rsyncd

systemctl restart rsyncd
systemctl status rsyncd

systemctl stop rsyncd

4 测试

echo " 娇小赤雅 " > /opt/xyz.txt
## 服务端
rsync -av /opt/xyz.txt rsync_backup_xyz@10.0.1.51::backup

## 客户端节点操作
echo " iyuyi.xyz@aliyun.com " > /opt/com.txt
echo "123" > /etc/rsync.passwd
chmod 600 /etc/rsync.passwd

rsync -av /opt/com.txt rsync_backup_xyz@10.0.1.51::backup
## 免输入密码
rsync -av /opt/com.txt rsync_backup_xyz@10.0.1.51::backup --password-file=/etc/rsync.passwd

六、防火墙 & Selinux

## 关闭防火墙
systemctl stop firewalld
systemctl disable --now firewalld

firewall-cmd --zone=public --add-port=873/tcp --permanent
## 配置立即生效
firewall-cmd --reload

## 关闭Selinux
## 永久关闭
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
## 临时关闭
setenforce 0

 

X、One Step Success

1 rsync 命令

查看代码

 [root@backup01 ~]# rsync -h
rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    no SIMD, asm, openssl-crypto
Checksum list:
    md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.

Usage: rsync [OPTION]... SRC [SRC]... DEST
  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
  or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
  or   rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
  or   rsync [OPTION]... [USER@]HOST:SRC [DEST]
  or   rsync [OPTION]... [USER@]HOST::SRC [DEST]
  or   rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.

Options
--verbose, -v            increase verbosity
--info=FLAGS             fine-grained informational verbosity
--debug=FLAGS            fine-grained debug verbosity
--stderr=e|a|c           change stderr output mode (default: errors)
--quiet, -q              suppress non-error messages
--no-motd                suppress daemon-mode MOTD
--checksum, -c           skip based on checksum, not mod-time & size
--archive, -a            archive mode; equals -rlptgoD (no -H,-A,-X)
--no-OPTION              turn off an implied OPTION (e.g. --no-D)
--recursive, -r          recurse into directories
--relative, -R           use relative path names
--no-implied-dirs        don't send implied dirs with --relative
--backup, -b             make backups (see --suffix & --backup-dir)
--backup-dir=DIR         make backups into hierarchy based in DIR
--suffix=SUFFIX          backup suffix (default ~ w/o --backup-dir)
--update, -u             skip files that are newer on the receiver
--inplace                update destination files in-place
--append                 append data onto shorter files
--append-verify          --append w/old data in file checksum
--dirs, -d               transfer directories without recursing
--mkpath                 create the destination's path component
--links, -l              copy symlinks as symlinks
--copy-links, -L         transform symlink into referent file/dir
--copy-unsafe-links      only "unsafe" symlinks are transformed
--safe-links             ignore symlinks that point outside the tree
--munge-links            munge symlinks to make them safe & unusable
--copy-dirlinks, -k      transform symlink to dir into referent dir
--keep-dirlinks, -K      treat symlinked dir on receiver as dir
--hard-links, -H         preserve hard links
--perms, -p              preserve permissions
--executability, -E      preserve executability
--chmod=CHMOD            affect file and/or directory permissions
--acls, -A               preserve ACLs (implies --perms)
--xattrs, -X             preserve extended attributes
--owner, -o              preserve owner (super-user only)
--group, -g              preserve group
--devices                preserve device files (super-user only)
--copy-devices           copy device contents as regular file
--specials               preserve special files
-D                       same as --devices --specials
--times, -t              preserve modification times
--atimes, -U             preserve access (use) times
--open-noatime           avoid changing the atime on opened files
--crtimes, -N            preserve create times (newness)
--omit-dir-times, -O     omit directories from --times
--omit-link-times, -J    omit symlinks from --times
--super                  receiver attempts super-user activities
--fake-super             store/recover privileged attrs using xattrs
--sparse, -S             turn sequences of nulls into sparse blocks
--preallocate            allocate dest files before writing them
--write-devices          write to devices as files (implies --inplace)
--dry-run, -n            perform a trial run with no changes made
--whole-file, -W         copy files whole (w/o delta-xfer algorithm)
--checksum-choice=STR    choose the checksum algorithm (aka --cc)
--one-file-system, -x    don't cross filesystem boundaries
--block-size=SIZE, -B    force a fixed checksum block-size
--rsh=COMMAND, -e        specify the remote shell to use
--rsync-path=PROGRAM     specify the rsync to run on remote machine
--existing               skip creating new files on receiver
--ignore-existing        skip updating files that exist on receiver
--remove-source-files    sender removes synchronized files (non-dir)
--del                    an alias for --delete-during
--delete                 delete extraneous files from dest dirs
--delete-before          receiver deletes before xfer, not during
--delete-during          receiver deletes during the transfer
--delete-delay           find deletions during, delete after
--delete-after           receiver deletes after transfer, not during
--delete-excluded        also delete excluded files from dest dirs
--ignore-missing-args    ignore missing source args without error
--delete-missing-args    delete missing source args from destination
--ignore-errors          delete even if there are I/O errors
--force                  force deletion of dirs even if not empty
--max-delete=NUM         don't delete more than NUM files
--max-size=SIZE          don't transfer any file larger than SIZE
--min-size=SIZE          don't transfer any file smaller than SIZE
--max-alloc=SIZE         change a limit relating to memory alloc
--partial                keep partially transferred files
--partial-dir=DIR        put a partially transferred file into DIR
--delay-updates          put all updated files into place at end
--prune-empty-dirs, -m   prune empty directory chains from file-list
--numeric-ids            don't map uid/gid values by user/group name
--usermap=STRING         custom username mapping
--groupmap=STRING        custom groupname mapping
--chown=USER:GROUP       simple username/groupname mapping
--timeout=SECONDS        set I/O timeout in seconds
--contimeout=SECONDS     set daemon connection timeout in seconds
--ignore-times, -I       don't skip files that match size and time
--size-only              skip files that match in size
--modify-window=NUM, -@  set the accuracy for mod-time comparisons
--temp-dir=DIR, -T       create temporary files in directory DIR
--fuzzy, -y              find similar file for basis if no dest file
--compare-dest=DIR       also compare destination files relative to DIR
--copy-dest=DIR          ... and include copies of unchanged files
--link-dest=DIR          hardlink to files in DIR when unchanged
--compress, -z           compress file data during the transfer
--compress-choice=STR    choose the compression algorithm (aka --zc)
--compress-level=NUM     explicitly set compression level (aka --zl)
--skip-compress=LIST     skip compressing files with suffix in LIST
--cvs-exclude, -C        auto-ignore files in the same way CVS does
--filter=RULE, -f        add a file-filtering RULE
-F                       same as --filter='dir-merge /.rsync-filter'
                         repeated: --filter='- .rsync-filter'
--exclude=PATTERN        exclude files matching PATTERN
--exclude-from=FILE      read exclude patterns from FILE
--include=PATTERN        don't exclude files matching PATTERN
--include-from=FILE      read include patterns from FILE
--files-from=FILE        read list of source-file names from FILE
--from0, -0              all *-from/filter files are delimited by 0s
--protect-args, -s       no space-splitting; wildcard chars only
--copy-as=USER[:GROUP]   specify user & optional group for the copy
--address=ADDRESS        bind address for outgoing socket to daemon
--port=PORT              specify double-colon alternate port number
--sockopts=OPTIONS       specify custom TCP options
--blocking-io            use blocking I/O for the remote shell
--outbuf=N|L|B           set out buffering to None, Line, or Block
--stats                  give some file-transfer stats
--8-bit-output, -8       leave high-bit chars unescaped in output
--human-readable, -h     output numbers in a human-readable format
--progress               show progress during transfer
-P                       same as --partial --progress
--itemize-changes, -i    output a change-summary for all updates
--remote-option=OPT, -M  send OPTION to the remote side only
--out-format=FORMAT      output updates using the specified FORMAT
--log-file=FILE          log what we're doing to the specified FILE
--log-file-format=FMT    log updates using the specified FMT
--password-file=FILE     read daemon-access password from FILE
--early-input=FILE       use FILE for daemon's early exec input
--list-only              list the files instead of copying them
--bwlimit=RATE           limit socket I/O bandwidth
--stop-after=MINS        Stop rsync after MINS minutes have elapsed
--stop-at=y-m-dTh:m      Stop rsync at the specified point in time
--write-batch=FILE       write a batched update to FILE
--only-write-batch=FILE  like --write-batch but w/o updating dest
--read-batch=FILE        read a batched update from FILE
--protocol=NUM           force an older protocol version to be used
--iconv=CONVERT_SPEC     request charset conversion of filenames
--checksum-seed=NUM      set block/file checksum seed (advanced)
--ipv4, -4               prefer IPv4
--ipv6, -6               prefer IPv6
--version, -V            print the version + other info and exit
--help, -h (*)           show this help (* -h is help only on its own)

Use "rsync --daemon --help" to see the daemon-mode command-line options.
Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation.
See https://rsync.samba.org/ for updates, bug reports, and answers

Y、 Error message

1 Permission denied

[root@backup01 ~]# rsync -av /opt/xyz.txt rsync_backup_xyz@10.0.1.51::backup
Password: 
sending incremental file list
xyz.txt
rsync: [receiver] mkstemp ".xyz.txt.L1r1D5" (in backup) failed: Permission denied (13)

sent 127 bytes  received 134 bytes  104.40 bytes/sec
total size is 15  speedup is 0.06
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1332) [sender=3.2.3]

解决

• 共享目录的属主和属组不正确，不是 rsyncxyz  chown -R rsyncxyz:rsyncxyz /opt/backupxyz
• 共享目录的权限不正确，不是755  chmod 755 /opt/backupxyz
• 关闭 selinux  setenforce 0

Z、Related Links

Use Rsync Backup Service on Redhat - 娇小赤雅 - 博客园 (cnblogs.com)

Liunx Sersync - 娇小赤雅 - 博客园 (cnblogs.com)