Linux Shell --- 数据同步
一、概要
1. 承上启下
(3) Rocky Linux 系列6 --- inotify-tools
2. 问题的提出
(1) 我们需要解决什么问题？
a. 网站发布之后多个节点的文件同步；
b. 数据库备份文件发生变更后传递到备份服务器；
(2) 如何解决？
利用上文讲到的SSH和rsync和inotify，我们可以实现当某一目录(如网站或数据备份目录)发生变化时，由inotify触发rsync将已经改变的文件同步至目标服务器。
a. 首先需要搭建通过公钥验证的SSH登录方式；
b. 安装rsync和inotify；
c. 创建配置文件，内容为需要同步(监控)的目录或文件，当文件或目录变更后需要同步的目标服务器以及目标目录也需要考虑；
d. 编写脚本读取d中的配置文件并对其内容使用inotifywait进行监控，当有监控时间产生时调用rsync进行后续处理；
e. 创建Linux Service将脚本以守护进程的方式进行维护；
二、解决方案
假设我有三台服务器，192.168.3.1(master)，192.168.3.2(备份1)，192.168.3.3(备份2)
1. 初始化
(1) 目录
sudo mkdir -p /opt/LSync
(2) 脚本
sudo touch /opt/LSync/main.sh
sudo chmod +x /opt/LSync/main.sh
(3) 配置文件config_watch_directories
sudo touch /opt/LSync/config_watch_directories
内容：
/tmp/backup/dir1
/tmp/backup/dir2
(4) 配置文件config_sync_servers
sudo touch /opt/LSync/config_sync_servers
内容:
bkuser@192.168.3.2:/home/bkuser/backup bkuser@192.168.3.3:/home/bkuser/backup
(5) 日志
sudo touch /opt/LSync/lsync.log
(6) 权限
sudo chmod -R 755 /opt/LSync
2. SSH
(1) 登录192.168.3.1
ssh-keygen -t rsa -b 4096 chmod 700 ~/.ssh ssh-copy-id bkuser@192.168.3.2 ssh-copy-id bkuser@192.168.3.3
(2) 登录192.168.3.2
sudo vi /etc/ssh/sshd_config PubkeyAuthentication yes --确保开启 sudo systemctl restart sshd
(3) 登录192.168.3.3
sudo vi /etc/ssh/sshd_config PubkeyAuthentication yes --确保开启 sudo systemctl restart sshd
3. 脚本
(1) 创建
sudo touch /opt/LSync/main.sh
sudo chmod +x /opt/LSync/main.sh
(2) 初始化
#!/bin/bash export LC_ALL=C DIR_BASE="/opt/LSync" FILE_NAME_DATE="$(date +%Y%m%d)" TIME="$(date +'%Y-%m-%d %H:%M:%S')" PATH_CONFIG_4_WATCH="${DIR_BASE}/config_watch_directories" PATH_CONFIG_4_SERVER="${DIR_BASE}/config_sync_servers" DIR_LOG="${DIR_BASE}/logs" PATH_LOG="${DIR_LOG}/${FILE_NAME_DATE}.log" error() { printf "[${TIME}]-[%s]: %s\n" "$(basename "${BASH_SOURCE}")" "${1}" >&2 exit 1 } info() { printf "[${TIME}]-[%s]: %s\n" "$(basename "${BASH_SOURCE}")" "${1}" >&1 } trap 'error "An unexpected error occurred."' ERR init_log() { if [ ! -d "${DIR_LOG}" ]; then mkdir -p ${DIR_LOG} if [ "$?" -ne "0" ]; then error "Create ${DIR_LOG} failed." fi fi if [ ! -r "${PATH_LOG}" ]; then touch $PATH_LOG fi } verify() { # Check whether the config file is available if [ ! -r "${PATH_CONFIG_4_WATCH}" ]; then error "Lack of config file: ${PATH_CONFIG_4_WATCH}" fi # Check whether the config file is empty if [ ! -s "${PATH_CONFIG_4_WATCH}" ]; then error "Empty config file: ${PATH_CONFIG_4_WATCH}" fi # Check whether the config file is available if [ ! -r "${PATH_CONFIG_4_SERVER}" ]; then error "Lack of config file: ${PATH_CONFIG_4_SERVER}" fi # Check whether the config file is empty if [ ! -s "${PATH_CONFIG_4_SERVER}" ]; then error "Empty config file: ${PATH_CONFIG_4_SERVER}" fi } do_sync() { info "Start sync ${1} to ${2}" rsync -ahrvz --delete -e "ssh -i /home/[User]/.ssh/id_rsa" $1 $2 } do_transfer() { source_path=$1 while IFS= read -r line; do do_sync $source_path $line done <$PATH_CONFIG_4_SERVER } init_transfer() { while IFS= read -r line; do do_transfer $line done <$PATH_CONFIG_4_WATCH } init() { init_log init_transfer } main() { verify inotifywait -m -r -e modify,create,delete --excludei "\.(swp|swx)" --fromfile "${PATH_CONFIG_4_WATCH}" | while read path events file; do info "Events: ${events}, path: ${path}, file: ${file}" init_log while IFS= read -r line; do if [ "$line" = "${path:0:${#line}}" ]; then do_transfer $line break fi done <$PATH_CONFIG_4_WATCH done } init && main | tee -a ${PATH_LOG}
4. Service
(1) 创建服务文件
sudo touch /etc/systemd/system/lsync.service sudo chmod 755 /etc/systemd/system/lsync.service
(2) 初始化
[Unit] Description=Linux file synchronization system After=network.target [Service]
User=root
Group=root ExecStart=/opt/LSync/main.sh [Install] WantedBy=multi-user.target
(3) 启动
sudo systemctl daemon-reload
sudo systemctl enable --now lsync.service systemctl status lsync.service
三、参考
https://linuxhint.com/inotofy-rsync-bash-live-backups/
https://bartsimons.me/sync-folders-and-files-on-linux-with-rsync-and-inotify/
