inotify

安装inotify

  1. [root@server ~]# mkdir -p /home/oldboy/tools

安装inotify-tools-3.14.tar.gz

  1. [root@server tools]# ls -l /proc/sys/fs/inotify/ #出现下面三个表示支持inotify
  2. total 0
  3. -rw-r--r-- 1 root root 0 Feb 6 15:36 max_queued_events
  4. -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_instances
  5. -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_watches

解压：

  1. [root@server tools]# tar zxf inotify-tools-3.14.tar.gz

编译安装：

  1. [root@server tools]# cd inotify-tools-3.14
  2. [root@server inotify-tools-3.14]# ./configure --prefix=/usr/local/inotify-tools-3.14
  3. [root@server inotify-tools-3.14]# echo $?
  4. 0
  5. [root@server inotify-tools-3.14]# make && make install
  6. [root@server inotify-tools-3.14]# echo $? #0表示安装正确
  7. 0
  8. [root@server tools]# ln -s /usr/local/inotify-tools-3.14/ /usr/local/inotify-tools #创建软连接
  9. [root@server tools]# ls -l /usr/local/inotify-tools/
  10. total 16
  11. drwxr-xr-x. 2 root root 4096 Feb 6 15:43 bin #inotify执行命令（二进制）
  12. drwxr-xr-x. 3 root root 4096 Feb 6 15:42 include #inotify程序所需要的头文件
  13. drwxr-xr-x. 2 root root 4096 Feb 6 15:43 lib #动态链接的库文件
  14. drwxr-xr-x. 4 root root 4096 Feb 6 15:42 share #帮助文档

 

  1. [root@server inotify-tools]# ll bin/
  2. total 88
  3. -rwxr-xr-x. 1 root root 44287 Feb 6 15:43 inotifywait
  4. -rwxr-xr-x. 1 root root 41409 Feb 6 15:43 inotifywatch

inotifywait：在被监控的文件或目录上等待特定文件系统事件（open、close、delete等）发生，执行后处于阻塞转态，适合在shell脚本中使用。

inotifywatch：收集被监控的文件系统使用度统计数据，指文件系统事件发生的次数统计。

inotifywait命令常用参数

  1. [root@server inotify-tools]# ./bin/inotifywait --help
  2. inotifywait 3.14
  3. Wait for a particular event on a file or set of files.
  4. Usage: inotifywait [ options ] file1 [ file2 ] [ file3 ] [ ... ]
  5. Options:
  6.         -h|--help Show this help text.
  7.         @<file> Exclude the specified file from being watched.
  8.         --exclude <pattern>
  9.                         Exclude all events on files matching the
  10.                         extended regular expression <pattern>.
  11.         --excludei <pattern>
  12.                         Like --exclude but case insensitive. #排除文件或目录时，不区分大小写
  13.         -m|--monitor Keep listening for events forever. Without
  14.                         this option, inotifywait will exit after one
  15.                         event is received. #始终保持事件监听状态
  16.         -d|--daemon Same as --monitor, except run in the background
  17.                         logging events to a file specified by --outfile.
  18.                         Implies --syslog.
  19.         -r|--recursive Watch directories recursively. #递归查询目录
  20.         --fromfile <file>
  21.                         Read files to watch from <file> or `-' for stdin.
  22.         -o|--outfile <file>
  23.                         Print events to <file> rather than stdout.
  24.         -s|--syslog Send errors to syslog rather than stderr.
  25.         -q|--quiet Print less (only print events). #打印很少的信息，仅仅打印监控事件的信息
  26.         -qq Print nothing (not even events).
  27.         --format <fmt> Print using a specified printf-like format
  28.                         string; read the man page for more details.
  29.         --timefmt <fmt> strftime-compatible format string for use with
  30.                         %T in --format string. #指定事件输出的格式
  31.         -c|--csv Print events in CSV format.
  32.         -t|--timeout <seconds>
  33.                         When listening for a single event, time out after
  34.                         waiting for an event for <seconds> seconds.
  35.                         If <seconds> is 0, inotifywait will never time out.
  36.         -e|--event <event1> [ -e|--event <event2> ... ]
  37.                 Listen for specific event(s). If omitted, all events are
  38.                 listened for. #监控事件
  39.  
  40. Exit status:
  41.         0 - An event you asked to watch for was received.
  42.         1 - An event you did not ask to watch for was received
  43.               (usually delete_self or unmount), or some error occurred.
  44.         2 - The --timeout option was given and no events occurred
  45.               in the specified interval of time.
  46.  
  47. Events:
  48.         access file or directory contents were read
  49.         modify file or directory contents were written
  50.         attrib file or directory attributes changed
  51.         close_write file or directory closed, after being opened in
  52.                         writeable mode
  53.         close_nowrite file or directory closed, after being opened in
  54.                         read-only mode
  55.         close file or directory closed, regardless of read/write mode
  56.         open file or directory opened
  57.         moved_to file or directory moved to watched directory
  58.         moved_from file or directory moved from watched directory
  59.         move file or directory moved to or from watched directory
  60.         create file or directory created within watched directory
  61.         delete file or directory deleted within watched directory
  62.         delete_self file or directory was deleted
  63.         unmount file system containing file or directory unmounted

测试一：

窗口一：

  1. [root@server inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e create /data #create创建（监控创建事件）
  2. 06/02/17 16:13 /data/test.txt
  3. 06/02/17 16:13 /data/test2.txt
  4. 06/02/17 16:14 /data/1.txt
  5. 06/02/17 16:14 /data/2.txt
  6. 06/02/17 16:14 /data/3.txt
  7. 06/02/17 16:14 /data/4.txt

窗口二：

  1. [root@server ~]# cd /data/
  2. [root@server data]# ls
  3. [root@server data]# touch test.txt
  4. [root@server data]# touch test2.txt
  5. [root@server data]# touch {1..4}.txt

测试二：

窗口一：

  1. [root@server inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e delete /data #delete删除（监控删除事件）
  2. 06/02/17 16:17 /data/test.txt
  3. 06/02/17 16:17 /data/test2.txt

窗口二：

  1. [root@server data]# rm -f test.txt
  2. [root@server data]# rm -f test2.txt

同时监控多个事件，事件之间用逗号分隔。

  1. /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e delete,create,close_write /data

测试：

窗口一：

  1. [root@server scripts]# cat inotify.sh
  2. #!/bin/bash
  3. inotify=/usr/local/inotify-tools/bin/inotifywait
  4. $inotify -mrq --format '%w%f' -e create,close_write,delete /data \
  5. |while read file
  6. do
  7.   cd / &&
  8.   rsync -az ./data --delete rsync_backup@192.168.31.132::backup/ \
  9.   --password-file=/etc/rsync.password
  10. done
  11. [root@server scripts]# sh -x inotify.sh
  12. + inotify=/usr/local/inotify-tools/bin/inotifywait
  13. + read file
  14. + /usr/local/inotify-tools/bin/inotifywait -mrq --format %w%f -e create,close_write,delete /data
  15. + cd /
  16. + rsync -az ./data --delete rsync_backup@192.168.31.132::backup/ --password-file=/etc/rsync.password
  17. + read file
  18. + cd /
  19. + rsync -az ./data --delete rsync_backup@192.168.31.132::backup/ --password-file=/etc/rsync.password
  20. + read file

窗口二：

  1. [root@server data]# touch oldboy.txt
  2. [root@server data]# touch test.log

备份服务器：

  1. [root@backup backup]# ls
  2. data
  3. [root@backup backup]# tree
  4. .
  5. └── data
  6.     ├── 2.txt
  7.     ├── 3.txt
  8.     ├── 4.txt
  9.     ├── oldboy.txt
  10.     ├── test.log
  11.     └── test.txt

 

  1. [root@server scripts]# /bin/sh /server/scripts/inotify.sh & #放入后台执行

写入rc.local

应用场景：10—300k小文件并发200—300，不会有延迟。

关键参数说明：

在/proc/sys/fs/inotify目录下有三个文件，对inotify机制有一定的限制。

max_user_watches：设置inotifywait或inotifywatch命令可以监视的文件数量（单进程）。

max_user_instances：设置每个用户可以运行的inotifywait或inotifywatch命令的进程数。

max_queued_events：设置inotify实例事件（event）队列可容纳的事件数量。

  1. [root@server scripts]# cat /proc/sys/fs/inotify/max_user_watches # 修改为50000000
  2. 8192
  3. [root@server scripts]# cat /proc/sys/fs/inotify/max_user_instances #
  4. 128
  5. [root@server scripts]# cat /proc/sys/fs/inotify/max_queued_events #修改为50000000
  6. 16384

每秒200个文件并发，数据同步几乎无延迟。

inotify优点：

实时数据同步。

inotify缺点：

1、并发如果大于200个文件（10—100K），同步会有延迟。

2、监控到事件后，调用rsync同步是单进程的（加&并发），sersync多进程同步。

sersync的功能：

1、配置文件。

2、真正的守护进程socker。

3、可以对失败文件定时重传（定时任务）。

4、第三方的http接口。

5、默认多线程同步。

高并发数据实时同步方案：

1、文件级别：inotify（sersync）+rsync。

2、文件系统级别：drbd。

3、第三方软件的同步功能：mysql同步、oracle、mongodb。

4、程序双写。

5、业务逻辑解决。

posted on 2017-02-06 20:40  yinshoucheng  阅读(616)  评论(0)  编辑  收藏  举报

导航