Shell脚本的编写

筛选后统计总数

cat logs | grep IconsendRedirect | wc -l >> bb.log

筛选后分类统计并且排序

cat logs | grep IconsendRedirect | awk '{{FS=" "}s[$11] ++}END{ for(i in s){ print i, s[i] } }' >> bb.log

赋予*.sh文件执行权限

chmod u+x *.sh

根据关键字筛选指定列

cat /data/tools/apache-tomcat-8085/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
cat /data/tools/apache-tomcat-8086/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
cat /data/tools/apache-tomcat-8580/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
cat /data/tools/apache-tomcat-8581/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
cat /data/tools/apache-tomcat-8582/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
cat /data/tools/apache-tomcat-8583/logs/catalina.`date --date='0 days ago' +%Y-%m-%d`.out | grep DateNewUser | awk '{FS=" "}{print $8,$9,$10,$11,$12}' >> /data/tools/`date --date='0 days ago' +%Y%m%d`.txt
tar -zcvf `date --date='0 days ago' +%Y%m%d`.tar.gz `date --date='0 days ago' +%Y%m%d`.txt
rm `date --date='0 days ago' +%Y%m%d`.txt

s3cmd压缩前一天文件并上传

s3cmd put `date --date='0 days ago' +%Y%m%d`.tar.gz s3://ngemobi/logs/gp_android_id/dt=`date --date='0 days ago' +%Y-%m-%d`/`date --date='0 days ago' +%Y%m%d`.tar.gz

 scp远程传输

命令：scp -r bb.log ngsteam@52.74.22.232:/data/litao/
密码：TKp*****@ac