day-036实时同步

合集 - oldboy-sre学习笔记(1)

1.day-036实时同步02-08

收起

实时同步的配置文件

实时同步的具体流程

1、服务端配置

1.1安装服务

yum   -y install  rsync

1.2修改配置文件

#/etc/rsyncd.conf
####rsyncd.conf start##
fake super = yes
uid = rsync
gid = rsync
use chroot = no
max connections = 2000
timeout = 600
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
#hosts deny=0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
[data]
comment = www 
path = /data
[backup]
comment = gether anli
path = /backup
[nfsbackup]
comment = sersync
path = /nfsbackup

tip:我这边做了3个模块，这里只用nfsbackup就行

1.3根据配置文件执行配置步骤

1.3.1创建启服务的虚拟用户

useradd   -s  /sbin/nologin  -M  rsync

1.3.2创建密码配置文件并设置权限

vim /etc/rsync.password
rsync_backup:1
chmod  600  /etc/rsync.password

1.3.3创建同步的目录

mkdir -p  /nfsbackup
chown  -R   rsync.rsync /nfsbackup

1.3.4启动服务

systemctl start rsyncd
systemctl enable rsyncd

2、客户端的配置

2.1安装服务

yum   -y install  rsync  

2.2创建密码文件，并设置权限

echo  '1' >/etc/rsync.client
chmod  600   /etc/rsync.client

2.3创建目录

mkdir -p /server/tools /app/tools/   /nfsdata

2.4上传sersync2.5.4_64bit_binary_stable_final.tar.gz到/server/tools，解压到指定目录

tar  -xf sersync2.5.4_64bit_binary_stable_final.tar.gz -C  /app/tools/

2.5查看解压的文件，重新命令

cd /app/tools/
mv GNU-Linux-x86/ sersync
tree /app/tools/
/app/tools/
└── sersync
    ├── confxml.xml
    └── sersync2

2.6配置文件修改和注释

#备份原始配置文件
[root@nfs01 /app/tools/sersync]# cp confxml.xml confxml.xml.bak

#编辑配置
[root@nfs01 /app/tools/sersync]# vim confxml.xml


#修改后配置与原始配置对比，有改动的地方如下
[root@nfs01 /app/tools/sersync]# diff confxml.xml confxml.xml.bak
<       <localpath watch="/nfsdata">
<           <remote ip="10.0.0.41" name="nfsbackup"/>
---
>       <localpath watch="/opt/tongbu">
>           <remote ip="127.0.0.1" name="tongbu1"/>
30,31c30,31
<           <commonParams params="-az"/>
<           <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.pass"/>
---
>           <commonParams params="-artuz"/>
>           <auth start="false" users="root" passwordfile="/etc/rsync.client"/>
36c36
<       <failLog path="/var/log/sersync.log" timeToExecute="60"/><!--default every 60mins execute once-->
---
>       <failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->

1. 监控本地目录/nfsdata
2. 服务端ip和rsync模块名
3. 修改为密码验证，配置用户名和密码验证文件
4. 配置使用参数，修改日志文件路径

2.7给confxml.xml赋予权限

 chmod 644 sersync/confxml.xml 
 ll sersync/ 
 total 1772 
 -rw-r--r-- 1 root root    2214 Oct 26  2011 confxml.xml 
 -rwxr-xr-x 1 root root 1810128 Oct 26  2011 sersync2

2.8测试同步

#查看使用帮助
[root@nfs01 ~]# /app/tools/sersync/sersync2 -h
set the system param
execute：echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute：echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
_______________________________________________________
参数-d:启用守护进程模式
参数-r:在监控前，将监控目录与远程主机用rsync命令推送一遍
c参数-n: 指定开启守护线程的数量，默认为10个
参数-o:指定配置文件，默认使用confxml.xml文件
参数-m:单独启用其他模块，使用 -m refreshCDN 开启刷新CDN模块
参数-m:单独启用其他模块，使用 -m socket 开启socket模块
参数-m:单独启用其他模块，使用 -m http 开启http模块
不加-m参数，则默认执行同步程序

#启用守护进程模式进行监控
[root@nfs01 ~]# /app/tools/sersync/sersync2 -dro /app/tools/sersync/confxml.xml
set the system param
execute：echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute：echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -d      run as a daemon
option: -r      rsync all the local files to the remote servers before the sersync work
option: -o      config xml name：  /app/tools/sersync/confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost     host port: 8008
daemon start，sersync run behind the console
use rsync password-file :
user is rsync_backup
passwordfile is         /etc/rsync.pass
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12  = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
Max threads numbers is: 22 = 12(Thread pool nums) + 10(Sub threads)
please according your cpu ，use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /nfsdata && rsync -az -R --delete ./ rsync_backup@10.0.0.41::nfsbackup --password-file=/etc/rsync.pass >/dev/null 2>&1
run the sersync:
watch path is: /nfsdata

#查看进程
[root@nfs01 ~]# ps -ef | grep sersync
root      24881      1  0 17:52 ?        00:00:00 /app/tools/sersync/sersync2 -dro /app/tools/sersync/confxml.xml
root      25550 122922  0 17:55 pts/0    00:00:00 grep --color=auto sersync

#backup主机 /nfsbackup目录没有文件
[root@backup /nfsbackup]# ll
total 0

#nfs01主机上创建测试文件
[root@nfs01 ~]# cd /nfsdata/
[root@nfs01 /nfsdata]# touch  1.txt

#backup主机上查看，有了！
[root@backup /nfsbackup]# ls
1.txt