Shell企业案例实战和企业面试题
shell企业面试题
1、批量创建带有随机小写字符文件程序
使用for循环在/pizza目录下创建10个html文件，其中每个文件包含10个随机小写字母加固定字母_pizza
1、思路分析：
核心是：创建10个随机小写字母
第一种：$RANDOM
[root@web-01 /server/scripts]# echo $RANDOM 9839 范围0-32767 ，第一个容易被破解，使用的时候最好再加个字符串
第二种：openssl rand -base64 10
[root@web-01 /server/scripts]# openssl rand -base64 10（最后面是长度） yz+FH2zUNMlVnw== [root@web-01 /server/scripts]# openssl rand -base64 100 wkfkVmliczOgoLl0z/m5S/7InZ8+4AzdHmR6t6hhE80oghRY46598L+no+HtDcHD HyvQYnBWi6nQ0GbsjafyWZps7y6JpMEA6JOwQ+HlIOICXT7YLCcI9mQa6FUE+vHR OcxHog==
第三种：date
[root@web-01 /server/scripts]# date +%N%s（） 3057895901553937109
第四种：head /dev/urandom |cksum
[root@web-01 /server/scripts]# head /dev/urandom |cksum 1831677657 1682
第五种：uuidgen
[root@web-01 /server/scripts]# uuidgen 218780c9-ee6f-41dc-9058-a9a3a717cde1
第六种：cat /proc/sys/kernel/random/uuid
[root@web-01 /server/scripts]# cat /proc/sys/kernel/random/uuid 542f71d9-b240-4891-b61d-632083ecf6be
第七种：expect的mkpasswd
[root@web-01 /server/scripts]# mkpasswd（yum install expect -y） k3WlhJ|e7 [root@web-01 /server/scripts]# mkpasswd -l 20 -d 1 -c 2 （-l长度、-d数字、-c小写字母、-C大写字母、-s特殊字符） nebiv;bnZi6vjluvczgP
本次使用$RANDOM，前面加字符串，md5加密 ，将数字替换为字母，截取第2-第11个，共10位
[root@web-01 ~]# echo "PIZZA$RANDOM" |md5sum|tr "0-9" "a-z"|cut -c 2-11 befdbcdaee
2、for循环创建
path=/pizza [ -d $path ] || mkdir $path for n in {1..10} do random=`echo "PIZZA$RANDOM" |md5sum|tr "0-9" "a-z"|cut -c 2-11` touch $path/${random}_pizza.html done
2、批量改名
将题1中的pizza都改成linux（最好用for循环实现，并且扩展名html全部改成大写）
思路分析：
1、先改一个
2、for循环
name=linux path=/pizza/ cd $path for file in `ls *.html` do mv $file `echo ${file/pizza.html/linux.HTML}` done
3、方法二--用一条命令
[root@web-01 /pizza]# ls *.HTML |awk -F 'linux.HTML' '{print "mv",$0,$1"pizza.html"}' |bash
4、方法三--专业的rename
[root@web-01 /pizza]# rename "pizza.html" "linux.HTML" *.html
3、批量创建特殊需求要求用户案例
批量创建10个系统账号 pizza01-pizza10 并设置密码（密码是随机数，要求字符和数字等混合）
思路分析：
1、创建10个账号
第一种：echo pizza{01..10}
第二种：seq -w 10
2、随机密码，题1已经讲了很多，这次用openssl rand -base64 100
3、创建用户的命令
第一种：
useradd pizza01
echo 密码 | passwd --stdin
第二种：chpasswd命令
格式要符合下面的形式
pizza01：passwd
pizza02：passwd
4、for循环
for n in {01..10} do passwd=`openssl rand -base64 10` useradd pizza$n echo $passwd |passwd --stdin pizza$n echo -e "pizza$n\t$passwb" >> /pizza/user.list done
或者
for n in {01..10} do passwd=`openssl rand -base64 10` useradd pizza$n echo "pizza$n:$passwd" >> /pizza/pass.log done chpasswd < /pizza/pass.log
5、优化，搞的专业一点
for n in {1..10} do pass=`openssl rand -base64 10` if `grep "pizza$n" /etc/passwd &>/dev/null` # 判断是不是存在 then useradd pizza$n &&\ # &&\设置强逻辑关系 echo $pass|passwd --stdin pizza$n &&\ echo -e "pizza$n\t$pass" >> /pizza/user.log else echo "pizza$n is exist." fi done
优化：
for n in {1..10} do pass=`openssl rand -base64 10` if [ `grep -w "pizza$n" /etc/passwd|wc -l` -eq 0 ] then useradd pizza$n &&\ echo $pass|passwd --stdin pizza$n &&\ echo -e "pizza$n\t$pass" >> /pizza/user.log echo "adduser successful" else echo "pizza$n is exist." fi done
优化：
. /etc/init.d/functions for n in {1..10} do pass=`openssl rand -base64 10` if [ `grep -w "pizza$n" /etc/passwd|wc -l` -eq 0 ] then useradd pizza$n &&\ echo $pass|passwd --stdin pizza$n &>/dev/null &&\ echo -e "pizza$n\t$pass" >> /pizza/user.log action "adduser successful" /bin/true else action "pizza$n is exist." /bin/false fi done
优化：
. /etc/init.d/functions if [ $UID -ne 0 ] then echo "必须用root执行本脚本" exit 1 fi for n in {1..10} do pass=`openssl rand -base64 10` if [ `grep -w "pizza$n" /etc/passwd|wc -l` -eq 0 ] then useradd pizza$n &&\ echo $pass|passwd --stdin pizza$n &>/dev/null &&\ echo -e "pizza$n\t$pass" >> /pizza/user.log action "adduser successful" /bin/true else action "pizza$n is exist." /bin/false fi done
6、不用for循环的实现
http://user.qzone.qq.com/49000448/blog/1422183723
4、扫描网络内存活主机
写一个Shell脚本，判断10.0.0.0/24网络里面，当前在线的IP有哪些
思路分析
1、判断主机存活
ping c 2 i 1 w 3 10.0.0.7 nmap -sP 10.0.0.0/24
2、搞起来
第一种：ping
for n in {1..254} do # 将整个执行用大括号括起来 加 &，进行批量ping，原理就是放到后台执行 { if `ping -c 1 -w 3 10.0.0.$n &>/dev/null` then echo "10.0.0.$n is up" else echo "10.0.0.$n is down" fi } & done
# 没有进行过滤，所以输出很多，可以优化一下
第二种：使用nmap命令行就可以
[root@web-01 /server/scripts]# nmap -sP 10.0.0.0/24 |awk '/Nmap scan report for/{print $NF}'
5、MySQL分库备份
实现对MySQL数据库进行分库备份，用脚本实现
为什么要进行分库备份呢，因为，如果在以后需要备份一个小库，就会很麻烦
常规方法：
mysqldump -B userinfo click test | gzip >bak.sql.gz
分库备份：
mysqldump -B userinfo | gzip >bak.sql.gz mysqldump -B click | gzip >bak.sql.gz mysqldump -B test | gzip >bak.sql.gz
执行命令获取库名
mysql -uroot -ppizza123 -e "show databases" | grep -v _schema | sed 1d
把密码放到配置文件中
cat /etc/my.cnf [client] user = root passwd = pizza123
做完这一步就不用在脚本中添加-u和-p参数
备份脚本
path=/backup mysql="mysql -uroot -ppizza123" mysqldump="mysqldump -uroot -ppizza123" [ -d $path ] || mkdir $path for dbname in `$mysql -e "show databases;" 2>/dev/null|grep -v _schema | sed 1d` do # 这个命令还有很多参数没有写 $mysqldump -B $dbname |gzip >/backup/${dbname}_$(date +%F).sql.gz done
6、MySQL分库、分表备份
常规备份
mysqldump pizza test test1 | gzip > bak.sql.gz pizza是库名、test和test1是表名
分库、分表备份
mysqldump -B pizza | gzip >bak.sql.gz mysqldump pizza test1 mysqldump pizza test2 mysqldump pizza test3
脚本编码
path=/backup mysql="mysql -uroot -ppizza123" mysqldump="mysqldump -uroot -ppizza123" [ -d $path ] || mkdir $path for tname in `$mysql -e "show tables from $dbname;" 2>/dev/null|sed 1d` do if [ "$dbname" = "mysql" ] then # 这个命令还有很多参数没有写 $mysqldump --skip-lock-tables $dbname $tname |gzip >$path/${dbname}-$tname_$(date +%F).sql.gz 2>/dev/null else $mysqldump $dbname $tname |gzip >$path/${dbname}-$tname_$(date +%F).sql.gz 2>/dev/null fi done done
7、SSH服务批量分发与管理服务
确保主机能用root登陆
vim /etc/ssh/sshd_config 字段 PermitRootLogin 为 yes
建立密钥对
ssh-keygen
发送公钥到其他服务器
ssh-copy-id -i id_rsa.pub 10.0.0.8
可能会很慢，调整其他机器的配置，让操作快一些
useDNS no
GSSAPIAuthentication no
重启服务，继续
ssh-copy-id -i id_rsa.pub 10.0.0.8 ssh-copy-id -i id_rsa.pub 10.0.0.9
写一个链接主机并可以执行命令的脚本
vim ssh_.sh
if [ $# -ne 1 ] then echo "usage:$0 cmd" exit 1 fi for n in 8 9 do echo "-----10.0.0.$n------" ssh 10.0.0.$n $1 done
执行脚本
bash ssh_.sh "free -m"
编写分发脚本
. /etc/init.d/functions if [ $# -ne 2] then echo "usage:$0 localdir remotedir" exit 1 fi for n in 8 9 do scp -rp $1 10.0.0.$n:$2 &>/dev/null if [ $? -eq 0 ] then action "10.0.0.$n sucessful" /bin/true else actin "10.0.0.$n fail" /bin/false fi done
8、破解RANDOM随机数案例
已知下面这些字符串是通过RANDOM随机变量 md5sum 后，再截取一部分连续字符串的结果，亲个破解这些字符串对用的使用md5sum 处理前的RANDOM对应的数字
21023299
00205d1c
a3da1677
1f6d12dd
890684b
解答：
1、分析
RANDOM的随机范围是0-32767 。
显现需要把范围内的数字都加密，输出到md5.log中
2、比较
grep “890684b” md5.log | wc -l
3、编码实现
array=( 21023299 00205d1c a3da1677 1f6d12dd 890684b ) md5(){ for n in {0..32767} do echo -e "$n\t`echo $n|md5sum`" >> /pizza/md5.log done } crack_num(){ for num in ${array[*]} do find=`grep $num /pizza/md5.log` if [ `echo $find|wc -l` -eq 1 ] then echo $find fi done } main(){ md5 crack_num } main
第二种方法：egrep实现
[root@web-01 /server/scripts]# array=( > 21023299 > 00205d1c > a3da1677 > 1f6d12dd > 890684b > ) [root@web-01 /server/scripts]# cmd=`echo ${array[*]}|tr " " "|"` [root@web-01 /server/scripts]# egrep "$cmd" /pizza/md5.log
修改第一版
array=( 21023299 00205d1c a3da1677 1f6d12dd 890684b ) md5(){ for n in {0..32767} do echo -e "$n\t`echo $n|md5sum`" > /pizza/md5.log & done } crack_num(){ cmd=`echo ${array[*]}|tr " " "|"` egrep "$cmd" /pizza/md5.log } main(){ md5 crack_num } main
利用time命令对比两个版本的时间
time sh 8_random_crack.sh
9、检查多个网站地址是否正常
要求
1、使用shell数组方法，检测策略精良模拟用户访问
2、每10秒种做一次所有的检测，无法访问的输出报警
3、待检测网址如下
https://www.cnblogs.com/yxiaodao/
https://www.baidu.com/
检测工具：
url
curl
wget
脚本编码
. /etc/init.d/functions url=( https://www.cnblogs.com/yxiaodao/ https://www.baidu.com/ ) check_url(){ wget -t 2 -T 5 -o /dev/null -q $1 if [ $? -eq 0 ] then action "$1 is ok" /bin/true else action "$1 is lost" /bin/false fi } DealUrl(){ for url in ${url[*]} do check_url $url done } main(){ while true do DealUrl sleep 10 done } main
修改题目，不用数组，将网址放在文件中，做如下修改
DealUrl(){ while read line do check_url $line done < ./pizza/url.log }
有一个问题，在我们操作完后，会产生大量的网页文件，因为我们的命令将网页下载了
需要在命令中添加参数 -- spider
10、利用Shell编程分析Web日志解决Dos攻击生产案例
DOS Deny of Service
DDOS 分布式dos攻击
请根据web日志或者网络连接数，监控当某个IP并发连接数或者短时间内PV达到100（根据实际情况设定），即调用防火墙命令封掉对应的IP。
防火墙命令：iptables -l INPUT -s IP地址 -j DROP
分析：
1、web日志或者网络连接数
日志文件，netstat -an | grep -i est，排序去重
2、判断PV 或者连接数大于100 ，取出IP ，封IP
IP 在日志的第一列，取到IP---->排序---->统计数量----> 按数量从大到小排序
[root@web-01 /server/scripts]# awk '{print $1}' access_2010-12-8.log |sort|uniq -c|sort -rn 35 59.33.26.105 23 123.122.65.226 8 124.115.4.18
也能通过awk的数组来完成
[root@web-01 /server/scripts]# awk '{S[$1]++}END{for(key in S) print S[key],key}' access_2010-12-8.log |sort -rn 35 59.33.26.105 23 123.122.65.226 8 124.115.4.18
编码脚本
9 awk '{S[$1]++}END{for(key in S) print S[key],key}' access_2010-12-8.log |sort -rn > /pizza/ip.log 10 while read line 11 do 12 ip=`echo $line|awk '{print $2}'` 13 count=`echo $line|awk '{print $1}'` 14 if [ $count -gt 30 -a `grep $ip /pizza/drop.log|wc -l` -lt 1 ] 15 then 16 iptables -I INPUT -s $ip -j DROP &&\ 17 echo "$ip" >>/pizza/drop.log 18 else 19 echo "$ip" >>/pizza/accept.log 20 fi 21 done</pizza/ip.log
本次采用了读取drop.log日志的方法，也可以采用查看 iptables -nL的方法
10、利用Shell编程分析Linux服务器网络链接数解决DOS攻击生产案例实践
还是上一个题，上面的题监控的是web日志，本次是监控网络链接数实现
命令：ESTABLISHED 正在建立的链接状态
[root@web-01 /server/scripts]# netstat -an |grep -i ESTABLISHED Active Internet connections (servers and established) tcp 0 0 172.17.214.84:47778 107.175.240.135:2222 ESTABLISHED tcp 0 0 172.17.214.84:34820 100.100.30.25:80 ESTABLISHED tcp 0 52 172.17.214.84:22 163.125.30.51:37793 ESTABLISHED Active UNIX domain sockets (servers and established)
获取外部地址，统计，排序（为方便，将命令的输出到了日志）
[root@web-01 ~]# awk '/ESTAB/{print $0}' netstat.log|awk -F "[ :]+" '{print $(NF-3)}'|sort|uniq -c|sort -rn
高级写法，通过awk数组
[root@web-01 ~]# awk -F "[ :]+" '/ESTAB/{S[$(NF-3)]++}END{for(k in S) print S[k],k}' netstat.log | sort -rn |head
不用日志，用netstat -an
[root@web-01 ~]# netstat -an|awk -F "[ :]+" '/ESTAB/{S[$(NF-2)]++}END{for(k in S) print S[k],k}' | head 1 163.125.30.51 1 100.100.30.25 1 107.175.240.135
因数据差异，具体命令中参数还要自己调
脚本编写，只需修改前一个脚本的第一行获取ip的命令即可
netstat -an|awk -F "[ :]+" '/ESTAB/{S[$(NF-2)]++}END{for(k in S) print S[k],k}'|head >/pizza/ip.log while read line do ip=`echo $line|awk '{print $2}'` count=`echo $line|awk '{print $1}'` if [ $count -gt 30 -a `grep $ip /pizza/drop.log|wc -l` -lt 1 ] then iptables -I INPUT -s $ip -j DROP &&\ echo "$ip" >>/pizza/drop.log else echo "$ip" >>/pizza/accept.log fi done</pizza/ip.log
在实际工作中，可以设置定时任务，每3分钟执行一次，每天晚上0点取消
11、开发MySQL服务启动停止脚本
要求：用函数，case语句，if语句等实现 /etc/init.d/mysqld {start | stop | restart} 命令
分析：
1、启动
mysqld_safe --user=mysql &
2、停止
mysqladmin -uroot -ppasswd shutdown
killall，pkill（参考之前写的rsync 第九章-case结构条件句）
3、脚本编码
看一下mysql的pid的文件位置
# 定义锁文件 lockfile=/var/lock/subsys/mysqld # 定义变量，指定mysqld的的pid,是需要自己mysql的conf中去创建 mysql_pid_file_path=/application/mysql/data/`uname -n.pid` . /etc/init.d/functions start(){ mysql_safe --user=mysql &>/dev/null & retval=$? if [ $retval -eq 0 ] then action "mysql startup ok" /bin/true touch $lockfile return $retval else action "mysql startup fail" /bin/false return $retval fi } stop(){ if test -s "$mysql_pid_file_path" then mysql_pid=`cat $mysql_pid_file_path` if (kill -0 $mysql_pid &>/dev/null) then # 为了在重复停止操作的时候，不提示，将其扔到黑洞 kill $mysql_pi retval=$? if [ $? -eq 0 ] then action "mysql stop ok" /bin/true rm -f $lockfile return $retval else action "mysql stop fail" /bin/false return $retval fi else echo "mysqld_process is not exit." return 2 fi else echo "$mysqld_pid_file_path is not exist,or mysqld does not startup" fi } restart(){ killall mysql && sleep 1 && mysql --deamon retval=$? if [ $? -eq 0 ] then action "mysql restart ok" /bin/true return $retval else action "mysql restart fail" /bin/false return $retval fi } case "$1" in start) start # 我了向外传值 retval=$? ;; stop) stop retval=$? ;; restart) stop sleep 2 start retval=$? ;; *) echo "usage:$0 {start|stop|restart}" exit 1 esac exit $retval
出现问题，启动了，但是没有pid文件
解决：
1、先使用系统的命令开启。
2、通过查看ps -ef |grep mysql 查看 启动参数
添加启动参数 --pid-file=$mysql_pid_file_path
问题：进不去mysql
添加参数--datedir=/application/mysql/data
问题：启动的过程很快，执行脚本后，启动成功，但是没有发现进程和pid，无法进入
1、查看mysql日志
cat /application/mysql/data/web01.err
2、发现使用脚本中的命令，手动也起不来
3、使用系统执行后的启动命令
/bin/sh /application/masql/bin/mysqld_safe --datedir=/application/mysql/data --pid-file=$mysql_pid_file_path
脚本一定要现在命令行测试成功，再写入脚本中
最后一个任务
拷贝到/etc/init.d中 ，变成chkconfig 可已使用的脚本
12、按单词去重排序
In the world of hackers, the kind of answers you get to your technical questions depends as much on the way you ask the questions as on the difficulty of developing the answer. This guide will teach you how to ask questions in a way more likely to get you a satisfactory answer. Now that use of open source has become widespread, you can often get as good answers from other, more experienced users as from hackers. This is a Good Thing; users tend to be just a little bit more tolerant of the kind of failures newbies often have. Still, treating experienced users like hackers in the ways we recommend here will generally be the most effective way to get useful answers out of them, too. The first thing to understand is that hackers actually like hard problems and good, thought-provoking questions about them. If we didn't, we wouldn't be here. If you give us an interesting question to chew on we'll be grateful to you; good questions are a stimulus and a gift. Good questions help us develop our understanding, and often reveal problems we might not have noticed or thought about otherwise. Among hackers, “Good question!” is a strong and sincere compliment. Despite this, hackers have a reputation for meeting simple questions with what looks like hostility or arrogance. It sometimes looks like we're reflexively rude to newbies and the ignorant. But this isn't really true. What we are, unapologetically, is hostile to people who seem to be unwilling to think or to do their own homework before asking questions. People like that are time sinks — they take without giving back, and they waste time we could have spent on another question more interesting and another person more worthy of an answer. We call people like this “losers” (and for historical reasons we sometimes spell it “lusers”). We realize that there are many people who just want to use the software we write, and who have no interest in learning technical details. For most people, a computer is merely a tool, a means to an end; they have more important things to do and lives to live. We acknowledge that, and don't expect everyone to take an interest in the technical matters that fascinate us. Nevertheless, our style of answering questions is tuned for people who do take such an interest and are willing to be active participants in problem-solving. That's not going to change. Nor should it; if it did, we would become less effective at the things we do best. We're (largely) volunteers. We take time out of busy lives to answer questions, and at times we're overwhelmed with them. So we filter ruthlessly. In particular, we throw away questions from people who appear to be losers in order to spend our question-answering time more efficiently, on winners. If you find this attitude obnoxious, condescending, or arrogant, check your assumptions. We're not asking you to genuflect to us — in fact, most of us would love nothing more than to deal with you as an equal and welcome you into our culture, if you put in the effort required to make that possible. But it's simply not efficient for us to try to help people who are not willing to help themselves. It's OK to be ignorant; it's not OK to play stupid. So, while it isn't necessary to already be technically competent to get attention from us, it is necessary to demonstrate the kind of attitude that leads to competence — alert, thoughtful, observant, willing to be an active partner in developing a solution. If you can't live with this sort of discrimination, we suggest you pay somebody for a commercial support contract instead of asking hackers to personally donate help to you. If you decide to come to us for help, you don't want to be one of the losers. You don't want to seem like one, either. The best way to get a rapid and responsive answer is to ask it like a person with smarts, confidence, and clues who just happens to need help on one particular problem.
按单词出现的频率降序排序
1、把空格和符号都转换成空格--排序--统计--排序
2、命令
[root@web-01 /server/scripts]# cat english.txt |tr "“”! ,.)( " "\n" |sort|uniq -c |sort -rn
方法二：
[root@web-01 /server/scripts]# cat english.txt |tr "“”! ,.)( " "\n" |awk '{S[$1]++}END{for(k in S) print S[k],k}'|sort -rn
方法三：
cat english.txt |xargs -n1
12、按字母去重排序
按字母出现的频率降序排序
1、使用 grep -o ‘.’ 匹配任意 之后，会挨个输出 或者 grep -o "[^ ]"
grep -o "[^ ,.()]" english.txt |awk '{S[$1]++}END{for(k in S) print S[k],k}'|sort -rn
2、awk可以用空做分隔符
sed 's#[ ,\.\]##g' english.txt|awk -F "" '{for(i=0;i<NF;i++)S[$i]++}END{for(k in S) print S[k],k}' |sort -rn
暂时没有设计出过滤换换行符
13、按单词去重排序高级方案
基于上面的awk统计单词
awk -F "[ ,.]" '{for(i=1;i<NF;i++)S[$i]++}END{for(k in S) print S[k],k}' english.txt |sort -rn
教学例题讲解
合格的运维人员必会的脚本列表
1）系统及各类服务的监控脚本，例如：文件、内存、磁盘、端口，URL监控报警等。
2）监控网站目录下文件是否被篡改，以及站点目录批量被篡改后如何批量恢复的脚本。
3）各类服务Rsync、Nginx、MySQL等的启动及停止专业脚本（使用chkconfig管理）。
4）MySQL主从复制监控报警以及自动处理不复制故障的脚本。
5）一键配置MySQL多实例、一键配置MySQL主从部署脚本。
6）监控HTTP/MySQL/Rsync/NFS/Memcached等服务是否异常的生产脚本。
7）一键软件安装及优化的脚本，比如LANMP、Linux一键优化，一键数据库安装、优化等。
8）MySQL多实例启动脚本，分库、分表自动备份脚本。
9）根据网络连接数以及根据Web日志PV封IP的脚本。
10）监控网站的PV以及流量，并且对流量信息进行统计的脚本。
11）检查Web服务器多个URL地址是否异常的脚本，要可以批量处理且通用。
12）系统的基础优化一键优化的脚本。
13）TCP连接状态及IP统计报警脚本。
14）批量创建用户并设置随机8位密码的脚本
