Linux命令篇 - sed 命令

sed

sed - stream editor for filtering and transforming text；

sed：利用脚本来处理、编辑文本文件；

格式：sed [OPTION]... {script-only-if-no-other-script} [input-file]...

常用参数：

OPTIONS	意义
-e或--expression=<script文件>	以选项中指定的script来处理输入的文本文件
-f<script文件>或--file=<script文件>	以选项中指定的script文件来处理输入的文本文件
-h或--help	显示帮助
-n或--quiet或--silent	仅显示script处理后的结果
-V或--version	显示版本信息
-n	用于禁止打印所有内容
G	在每行数据后插入换行
-i	将sed修改后内容写入文件

参考案例：

# 案例数据 使用ctrl+c结束输出
$ cat > content.out<<EOF
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.
EOF

• 输出首行|末尾行

# 输出首行, 打印第一行文本内容
$ sed -n '1p' content.out 

# 输出末尾行
$ sed -n '$p' content.out
learn more about dummy text.

• 单个字符串替换（词或符号）

# 替换匹配的第一个word or characters
# 默认只会替换第一个匹配的lorem,后面的lorem不再替换
# lorem是匹配的词,Lorem是替换的词
$ sed 's/lorem/Lorem/' content.out
Lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
Lorem ipsum is highly used by designers. lorem ipsum is great for developers.
Lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

• 多个字符串替换

# 替换匹配的全部word or characters
# 若需要替换匹配的全部则在''之间使用/g
$ sed 's/lorem/Lorem/g' content.out
Lorem ipsum is a dummy text. Lorem ipsum has been the industry's standard dummy text.Lorem ipsum has been the industry's standard dummy text.
Lorem ipsum is highly used by designers. Lorem ipsum is great for developers.
Lorem ipsum is used for dummy text. Lorem ipsum doesn't have meaning.
learn more about dummy text.

• 替换指定位置的单个字符串

# 将位置为第二的lorem替换为Lorem
# 在每一行中替换第n个出现的word or characters
# 使用 /1,/2 or n (any number) 指定替换位置
$ sed 's/lorem/Lorem/2' content.out
lorem ipsum is a dummy text. Lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. Lorem ipsum is great for developers.
lorem ipsum is used for dummy text. Lorem ipsum doesn't have meaning.
learn more about dummy text.

• 多个字符替换 (类似tr命令)

# 将set1中出现的所有字符替换为set2中对应的字符
# 在''之间使用y参数
$ echo 'a for apple' | sed -e 'y/af/AF/'
A For Apple

• 替换指定开始位置的字符串

# 从指定位置2开始往后的lerem都被替换成Lorem
$ sed 's/lorem/Lorem/2g' content.out 
lorem ipsum is a dummy text. Lorem ipsum has been the industry's standard dummy text.Lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. Lorem ipsum is great for developers.
lorem ipsum is used for dummy text. Lorem ipsum doesn't have meaning.
learn more about dummy text.

• 指定行替换字符串

# 在s前面使用数字 表示指定行替换
# 在下面的命令中，我们可以发现只有第二行被字符串'Lorem'取代
$ sed '2 s/lorem/LOREM/' content.out 
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
LOREM ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

• 显示文本的部分内容

# -n: 用于禁止显示全部内容
# p: 用于打印特定行
# 显示2到4行的内容
$ sed -n 2,4p content.out 
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

• 显示指定行外的文本内容

# 显示除去1到2行外的内容
$ sed 1,2d content.out 
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

• 显示匹配行外的文本内容

$ sed '/learn/d' content.out
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.

• 显示替换的行

# 显示被替换的行
# 最后一行learn more about dummy text.没有匹配lorem就不显示出来
$ sed -n 's/lorem/Lorem/p' content.out 
Lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
Lorem ipsum is highly used by designers. lorem ipsum is great for developers.
Lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.

• 'p'和'-n'

# 打印第一行文本内容
$ sed -n '1p' content.out 
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.

# 打印指定范围行文本内容
# 打印第一行到第四行文本内容
$ sed -n '1,4p' content.out 
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

# 打印多行文本内容
# 打印第一行和第四行文本内容
$ sed -n '1p;4p' content.out

# 打印一个文件内容
$ sed -n 'p' content.out 
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

# 显示匹配字符串的行
$ sed -n '/lorem/p' content.out

# 输出包含数字的行
$ sed -n '/[0-9]/p' content.out

# 显示匹配的行并在前面添加`Matched--`内容
$ sed -n 's/^l/Matched--&/p' content.out 
Matched--lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
Matched--lorem ipsum is highly used by designers. lorem ipsum is great for developers.
Matched--lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
Matched--learn more about dummy text.

• 使用sed替代grep

# 使用sed完成grep的功能
$ sed -n '/root/p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

• sed多参数

# 将所有出现的“lorem”替换为“lorem”，并删除与/learn/搜索模式匹配的行;
$ sed -e 's/lorem/Lorem/g' -e '/learn/d' content.out 
Lorem ipsum is a dummy text. Lorem ipsum has been the industry's standard dummy text.Lorem ipsum has been the industry's standard dummy text.
Lorem ipsum is highly used by designers. Lorem ipsum is great for developers.
Lorem ipsum is used for dummy text. Lorem ipsum doesn't have meaning.

• 插入换行符

# 插入单行换行符
$ sed G content.out

# 插入多行换行符
$ sed 'G;G' content.out

• 备份文本并修改文本内容

# use 'i.<file extension>' for backup file name and -e editing.
# 创建一个原始文件'content.txt'的备份为'content.txt.bak'
$ sed -i.bak -e 's/lorem/LOREM/g' content.out

# Results
# 修改的文件
$ cat content.out
LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
learn more about dummy text.
# 备份文件
$ cat content.out.bak
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.

• 删除匹配的行

# In the following output, the line with starting 'lorem' and ending with 'text.' is deleted
# 删除以lerem且test.结尾的行
$ sed -e 's/^lorem.*text.$//g' content.out
LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
learn more about dummy text.

• 添加数据到指定位置

# 可以看到“Here”被添加在每一行的前面
# 添加数据到行首
$ sed -e 's/.*/Here &/' content.out
Here LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
Here LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
Here LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
Here learn more about dummy text.

# 插入字符串在每行开始并换行
$ sed 'i \inserted line' content.out
inserted line
LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
inserted line
LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
inserted line
LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
inserted line
learn more about dummy text.

# 插入字符串在每行末尾并换行
$ sed 'a \Appended line' content.out
LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
Appended line
LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
Appended line
LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
Appended line
learn more about dummy text.
Appended line

• 提取数据

# 将提取所有可用的linux用户名
$ sed 's/\([^:]*\).*/\1/' /etc/passwd

• 打印不带注释#和空行的数据

# 打印不带注释和空行的数据
$ sed -e 's/#.*//;/^$/d' content.out
LOREM ipsum is a dummy text. LOREM ipsum has been the industry's standard dummy text.LOREM ipsum has been the industry's standard dummy text.
LOREM ipsum is highly used by designers. LOREM ipsum is great for developers.
LOREM ipsum is used for dummy text. LOREM ipsum doesn't have meaning.
learn more about dummy text.

• 移除带有注释#的行

$ sed 's/#.*//' /etc/yum.repos.d/CentOS-Base.repo

• 从文本种提取ip地址

# 添加测试数据
$ vi content.txt
lorem ipsum is a dummy text. lorem ipsum has been the industry's standard dummy text.
lorem ipsum is highly used by designers. lorem ipsum is great for developers.
lorem ipsum is used for dummy text. lorem ipsum doesn't have meaning.
learn more about dummy text.
122.65.49.2
221.11.165.233
219.158.9.97
219.158.19.137

# 复杂版 - 提取ip地址
$ sed '/\n/!s/[0-9.]\+/\n&\n/;/^\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}\n/P;D' content.txt

# 简化版 - 提取ip地址
$ sed -n '/[0-9]/p' content.txt

# 使用管道(|)将sed与其他命令组合使用
$ ip addr | sed -n '/inet/p' |sed -e 's/  */ /g'|cut -d ' ' -f3
127.0.0.1/8
::1/128
192.168.188.188/24
fe80::400e:cc35:e4ab:8c3f/64
172.17.0.1/16
fe80::42:65ff:fee9:cfb0/64
fe80::4cf2:b7ff:feb4:44fa/64
fe80::4478:34ff:fe5b:8d9a/64

• 数据提取并重定向并写到新的文件中

$ sed -n '/root/w rootpwd.txt' /etc/passwd
$ cat rootpwd.txt 
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

• 路径替换

# 创建测试文本
$ cat << 'EOF' > a.out
base_path = /aaa/bbb/ccc
base_path1 = /aaa/bbb/ccc
base_path2 = /aaa/bbb/ccc
EOF

# 备份文本并修改文本内容
$ sed -i.bak "s#base_path = /aaa/bbb/ccc#base_path = /ccc/ddd/eee#g" a.out

# 查看修改后的文件
$ cat a.out
base_path = /ccc/ddd/eee
base_path1 = /aaa/bbb/ccc
base_path2 = /aaa/bbb/ccc

# 查看修改前的文件
$ cat a.out.bak
base_path = /aaa/bbb/ccc
base_path1 = /aaa/bbb/ccc
base_path2 = /aaa/bbb/ccc

• 特殊字符替换成回车并统计行数

$ cat > 2.out <<EOF
01|0102|010200||2022-10-28 08:42:42|1|||2||||||||||||||||0|||
EOF

$ cat 2.out|sed 's/|/\n/g'|wc -l
28

• 回车替换成特殊字符

$ cat > 3.out <<EOF
01
0102
010200

2022-10-285 08:42:42
1


EOF
$ cat 3.out | tr "\n" "|" 
01|0102|010200||2022-10-28 08:42:42|1|||

• 删除多余的空格

$ cat > a.out <<EOF
a   b
c  d
e    f
EOF

$ sed 's/  //g' a.out 
a b
cd
ef