awk、sed、grep三大shell文本处理工具之sed的应用

sed

流编辑器 对文本中的行，逐行处理 非交互式的编辑器
是一个编辑器

1、工作流程

1）将文件的第一行读入到自己的缓存空间（模式空间--pattern space），删除掉换行符
2）匹配，看一下该行是不是要编辑的行，如果是-->3；不是-->再读入下一行到模式空间，删除换行符
3）执行编辑命令
4）加上换行符输出到屏幕
5）判断是否为文件最后一行，是-->sed退出；不是，再重复1~4步骤

注意：
1）默认情况下，sed缓存空间内的行都会输出到屏幕，除非使用-n拟制未编辑过的行的输出。
2）默认，sed将修改的行输出到屏幕，并没有修改源文件。-i可以修改源文件（强制备份一份源文件）

 

2、命令用法
2.1 语法
sed [option] [address1[,address2]] [command][flag] [filename]
option:
　　-i ：　　修改源文件，需要强制备份一份源文件(-i.bak 建议这样写-i选项，会将源文件生成一个.bak备份文件)
       　　       例：sed -i.bak '/baidu$/a\souhu' 1.txt
　　-r ：　　让sed支持扩展的正则表达式（不加选项即支持标准的正则表达式） 匹配你要修改的行
　　-f ：　　指定文件。将sed的命令写到文件中，然后执行sed -f 文件名 要操作的文件
　　-e ：　   允许一条命令执行多个sed子命令

　　-n ： 　  拟制输出，不输出未修改的行。强制输出用命令“p”

[address1[,address2] 匹配连续的范围 3,5 表示文本的第3~5行

[command]：增删改查
　　a： 　　append（追加）会在定位的行的下面追加一行；默认对所有的行下面都追加
　　i ：　　 insert(插入) ....上面插入一行；...所有行上面插入一行
　   d：　　 delete(删除)
例：删除第6行

sed -i.bak  '6d' 1.txt //
例：删除匹配行
sed -i.bak  '/^a.*./d' tmp.txt

　　s ： 　  substitute 替换字符串（改）

[flag]
　　g 　　全局（针对sed处理的每一行）
　　n 　　数字，处理替换掉该行中第几个出现的关键字
　　p 　　print输出，打印到屏幕
　　w 　　保存到文件
　　c 　　 change 更改某行

[filename]:指定要处理的文件，如果没有，结合|，处理的文本行来自其他命令的输出。

2.2 如何定位或者匹配要修改的行
2.2.1 使用行号
x 数字 10 定位第10行；101定位101行
[root@localhost html]# sed '5s/laowang/xiaowang/' sed.txt

定位一个范围：
3,5 表示文本的第3~5行
[root@localhost html]# sed '3,5s/laowang/xiaowang/' sed.txt


定位第一行和最后一行：
1 $
[root@localhost html]# sed '$s/laowang/xiaowang/' sed.txt        //匹配最后一行

奇数行或偶数行
0~2 偶数行
1~2 奇数行

[root@localhost html]# sed '1~2s/laowang/xiaowang/' sed.txt

定位某行之后的n行
x,+2 第x行及之后的2行
[root@localhost html]# sed '2,+2s/laowang/xiaowang/' sed.txt

2.2.2 正则表达式匹配
[root@localhost html]# cat /etc/httpd/conf/httpd.conf | sed '/^Listen/s/80/8080/' | grep Listen
cat /etc/httpd/conf/httpd.conf | sed -n '/^Listen/s/80/8080/p'
例：在apache中找出监听端口哪一行并添加或插入一行
添加 cat /etc/httpd/conf/httpd.conf | sed '/^Listen/a\Listen 8080' | grep Listen    下

插入 cat /etc/httpd/conf/httpd.conf | sed '/^Listen/i\Listen 8080' | grep Listen　 上

找到这一行并替换 cat /etc/httpd/conf/httpd.conf | sed '/^Listen/c\hahaha'

修改并备份 sed -i.bak '/^Listen/c\hahaha' /etc/httpd/conf/httpd.conf

[root@localhost conf]# ls
httpd.conf httpd.conf.bak magic

 

2.3 使用sed对文本增删改查

2.3.1 追加 命令：a

sed 'agebi' sed.txt
sed 'a\gebi' sed.txt
sed '2a\gebi' sed.txt
sed '2,4a\gebi' sed.txt
sed '2,+2a\gebi' sed.txt
sed '$a\gebi' sed.txt 
sed '1~2a\gebi' sed.txt (1~2:表示从1开始间隔2的数:1,3,5....) 

2.3.2 插入 命令：i
[root@localhost html]# sed 'i\xiaowang' sed.txt

2.3.3 删 命令：d
[root@localhost html]# sed '2,4d' sed.txt

2.3.4 改
替换字符串 s substitute
更改某行 c change

替换字符串：
s substitute 替换字符串（改）
[flag]
g 全局（针对sed处理的每一行）
n 数字，处理替换掉该行中第几个出现的关键字
p print输出，打印到屏幕
w 保存到文件

[root@localhost html]# sed '1s/nice/nonice/' sed.txt
1.laowang is a nonice man!laowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
[root@localhost html]# sed '1s/nice/nonice/2' sed.txt
1.laowang is a nice man!laowang is a nonice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
[root@localhost html]# sed '1s/nice/nonice/g' sed.txt
1.laowang is a nonice man!laowang is a nonice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
[root@localhost html]# sed '1s/nice/nonice/gp' sed.txt
1.laowang is a nonice man!laowang is a nonice man!
1.laowang is a nonice man!laowang is a nonice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
[root@localhost html]# sed '1s/nice/nonice/gw /tmp/sed1.txt' sed.txt
1.laowang is a nonice man!laowang is a nonice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
[root@localhost html]# cat /tmp/sed1.txt
1.laowang is a nonice man!laowang is a nonice man!

更改某行 c **********修改一个范围
[root@localhost html]# sed 'c\jj' sed.txt　　　　//修改全部
jj
jj
jj
jj
jj
[root@localhost html]# sed '2c\jj' sed.txt　　　　//修改第二行
1.laowang is a nice man!laowang is a nice man!
jj
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!

*******需要注意更改多行时，需要加上\n
sed '2,4c\abc\nabc\nabc' sed.txt

2.3.5 查 p

[root@localhost html]# sed 'p' sed.txt　　　　　　//打印输出
1.laowang is a nice man!laowang is a nice man!
1.laowang is a nice man!laowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!

[root@localhost html]# sed -n 'p' sed.txt 　　　　 //-n拟制输出
1.laowang is a nice man!laowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.laowang is a nice man!laowang is a nice man!

 

3、选项
option:
-n 拟制输出，不输出未修改的行。强制输出用命令“p”
-i 修改源文件，需要强制备份一份源文件
-r 让sed支持扩展的正则表达式（不加选项即支持标准的正则表达式） 匹配你要修改的行
-f 指定文件。将sed的命令写到文件中，然后执行sed -f 文件名 要操作的文件
-e 允许一条命令执行多个sed子命令

-i 直接修改源文件，不建议，建议这么用 -i.bak 在修改前备份一份源文件
[root@localhost html]# sed -i '/^Listen/a\Listen 8080' /etc/httpd/conf/httpd.conf //可以直接修改
[root@localhost html]# sed -i.bak '/^Listen/a\Listen 8080' /etc/httpd/conf/httpd.conf //修改源文件同时保留一份备份的文件，文件名*.bak

-r 让sed支持扩展的正则表达式
删除/etc/passwd中每行的第一个字符
[root@localhost conf]# sed -r 's/(^.)(.*)/\2/' /etc/passwd   　　//扩展正则中的向前引用

删除/etc/passwd中每行的第二个字符
[root@localhost conf]# sed -r 's/(^.)(.)(.*)/\1\3/' /etc/passwd　　//向前引用第一个(^.)和第三个（.*）不引用第二个，达到删除第二个字符的效果

删除/etc/passwd中每行的第三个字符

[root@localhost conf]# sed -r  's/(^..)(.)(.*)/\1\3/' /etc/passwd

[root@localhost ~]# cat a.txt
my me you are thetest
my me you are thetest
my me you are thetest
my me you are thetest
my me you are thetest
my me you are thetest
my me you are thetest

[root@localhost ~]# sed -r 's/(^.)(.)(.*$)/\1\3/' a.txt (删除第二个字符)---->向前引用实例，将a.txt文件分为三个子表达式，\1\3引用第一和第三个子表达式
m me you are thetest
m me you are thetest
m me you are thetest
m me you are thetest
m me you are thetest
m me you are thetest
m me you are thetest

[root@localhost ~]# sed -r 's/(^.)(.*)/\2/' a.txt （删除第一个字符）
y me you are thetest
y me you are thetest
y me you are thetest
y me you are thetest
y me you are thetest
y me you are thetest
y me you are thetest

 

-f 将sed的命令写到文件中，然后执行sed -f 文件名 要操作的文件

[root@danny test]# cat test.txt 
1.xiaowang is a nice man!xiaowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.xiaowang is a nice man!xiaowang is a nice man!
[root@danny test]# cat a.sed 
1s/xiaowang/laowang/g
[root@danny test]# sed -f a.sed test.txt 
1.laowang is a nice man!laowang is a nice man!
2.laowang is a nice man!laowang is a nice man!
3.laowang is a nice man!laowang is a nice man!
4.laowang is a nice man!laowang is a nice man!
5.xiaowang is a nice man!xiaowang is a nice man!

-e 允许一条命令执行多个sed子命令
[root@localhost html]# sed -e 's/a/A/g' -e 's/nice/nonice/g' sed.txt
1.lAowAng is A nonice mAn!lAowAng is A nonice mAn!
2.lAowAng is A nonice mAn!lAowAng is A nonice mAn!
3.lAowAng is A nonice mAn!lAowAng is A nonice mAn!
4.lAowAng is A nonice mAn!lAowAng is A nonice mAn!
5.lAowAng is A nonice mAn!lAowAng is A nonice mAn!

对一个地址做多个操作：
sed [option] [address] {
command1
command2
...
} filename

示例：

#!/bin/bash
result=`sed '2,4{
s/a/A/g
s/nice/nonice/g
2i abc
}' sed.txt`

echo -e "$result\n"