获取客户端ip访问网站的次数和请求页面脚本
统计日志内容如下:
[root@localhost pv]# head -10 /data/iplog/2018-08-23/pv/1.txt
2018-08-23 01:00:00 182.115.166.136 http://m.31xs.net/13/13178/9224032.html
2018-08-23 01:00:00 119.116.198.0 https://m.biqugexsw.com/wapbook/73979_24947398_2.html
2018-08-23 01:00:00 115.60.156.155 http://m.qushuba.com/shu18486/10060220.html
2018-08-23 01:00:00 120.82.128.244 https://wap.xxbiquge.com/78_78360/76520_2.html
2018-08-23 01:00:00 122.194.9.240 https://m.liewen.cc/b/0/691/551539.html
2018-08-23 01:00:00 125.37.161.171 http://m.biyuwu.cc/book/11736/14972260.html
2018-08-23 01:00:00 112.96.176.248 http://m.xianqihaotianmi.com/book_8963/8016345.html
2018-08-23 01:00:00 111.85.40.148 https://www.douliushu.com/0/42/416468.html
2018-08-23 01:00:00 112.248.63.202 https://wap.xxbiquge.com/60_60978/3326763.html
2018-08-23 01:00:00 112.97.63.83 http://m.163ks.cc/wapbook-318_11/
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
获取客户端ip访问网站的次数和请求页面
sh -x test.sh 2018-08-23 251 ##日志文件日期,251代表机器的ip地址
[root@localhost ~]# cat test.sh
#/bin/bash
cd /data/iplog/$1/pv
txt=$(ls -l|awk '{print $NF}'|sed '1d')
for n in $txt
do
ip=$(cat /root/iplist)
for n1 in $ip
do
[ ! -d /opt/txt01 ]&&mkdir /opt/txt01 -p
grep $n1 $n >>/opt/txt01/$n1-$1-cpv.txt
done
done
rep_txt=$(cd /opt/txt01/&&ls -l *cpv.txt|awk '{print $NF}')
cd /opt/txt01
for n2 in $rep_txt
do
[ ! -d /opt/txt02 ]&&mkdir /opt/txt02 -p
##cat $n2 |awk -F [/] '{print $3}'|sort|uniq -c|sort -nr >>$n2-test
cat $n2|awk -F '[ ]+' '{print $4}'|sort|uniq -c|sort -nr >>/opt/txt02/$n2-cpv
done
sleep 0.5
cd /opt/
tar zcf /opt/$1-$2-cpv-iplog.tar.gz ./txt02
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
[root@localhost opt]# ll /opt/txt01/
总用量 332
-rw-r--r-- 1 root root 261 8月 29 12:03 101.66.139.196-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 1799 8月 29 12:03 110.52.211.133-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 51490 8月 29 12:03 112.224.1.34-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 5102 8月 29 12:03 116.113.1.49-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 45998 8月 29 12:03 124.160.215.79-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 1018 8月 29 12:03 182.115.166.136-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 279 8月 29 12:03 219.155.99.220-2018-08-23-cpv.txt
-rw-r--r-- 1 root root 209465 8月 29 12:03 61.158.149.51-2018-08-23-cpv.txt
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
[root@localhost opt]# ll /opt/txt02/
总用量 124
-rw-r--r-- 1 root root 156 8月 29 12:03 101.66.139.196-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 708 8月 29 12:03 110.52.211.133-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 17310 8月 29 12:03 112.224.1.34-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 2094 8月 29 12:03 116.113.1.49-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 13783 8月 29 12:03 124.160.215.79-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 293 8月 29 12:03 182.115.166.136-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 116 8月 29 12:03 219.155.99.220-2018-08-23-cpv.txt-cpv
-rw-r--r-- 1 root root 68950 8月 29 12:03 61.158.149.51-2018-08-23-cpv.txt-cpv
[root@localhost opt]#
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
[root@git-server txt01]# head -10 /opt/txt02/117.136.79.37-2018-05-17-cpv.txt-cpv
6 http://m.31xs.net/13/13431/10319091.html
4 https://m.yubook.net/novel/61462/7938632.html
4 https://m.ybdu.com/xiaoshuo/18/18935/6895763.html
4 https://m.xiaoshuoli.com/i35445/18566473.html
4 https://m.xiaoshuoli.com/i18640/18476060.html
4 https://m.biquxu.com/24/24853/12085909.html
4 http://m.wutuxs.com/html/1/1071/842907.html
4 http://m.sanhao33.com/sort-8-5/
4 http://m.jw46.com/sort/
4 http://m.bxwx.io/book_77838/22048312.html
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
过滤出来的IP101.66.139.196 2018-08-23 的访问日志内容如下:
[root@localhost opt]# cat /opt/txt01/101.66.139.196-2018-08-23-cpv.txt
2018-08-23 00:37:55 101.66.139.196 https://m.2kxs.com/book/86335/16812393.html
2018-08-23 00:56:20 101.66.139.196 https://m.2kxs.com/book/86335/16841303.html
2018-08-23 01:00:00 101.66.139.196 https://m.2kxs.com/book/86335/16841304.html
- 1.
- 2.
- 3.
- 4.
统计出来的IP101.66.139.196请求的不同url次数的日志: