awk命令总结

打印所有行：（$0代表整条记录）

awk '{print $0}' employee.txt
awk '{print}' employee.txt

上面两条命令等价，都是打印employee.txt所有行。

输入字段分隔符：

• 使用FS代表输入字段分隔符
• 如果只有body部分，只用F即可，如果有不同的分隔符间隔，就需要使用FS了
• FS分隔符只能在BEGIN中使用

举例：

cat employee-multiple-fs.txt
101,John Doe:CEO%10000

awk 'BEGIN{FS="[,:%]"}{print $2,","$3}' employee-multiple-fs.txt

John Doe ,CEO

 

cat employee.txt
101,John Doe,CEO

awk -F "," '{print $2,","$3}' employee.txt

John Doe ,CEO

 输出字段分隔符：

• 使用OFS作为输出字段分隔符
• 如果不使用OFS，awk会默认以空格方式分割
• OFS也是在BEGIN中使用的

举例：

cat employee.txt
101,John Doe,CEO

awk -F "," '{print $2,":",$3}' employee.txt

John Doe : CEO

 

cat employee.txt
101,John Doe,CEO

awk -F "," 'BEGIN{OFS=":"}{print $2,$3}' employee.txt

John Doe:CEO

 

记录分隔符：

• 使用RS代表记录分隔符，即人为的将多条记录分成一行行的

举例：

cat employee-change-fs-ofs.txt
101
John Doe
CEO
-
102
Jason Smith
IT Manager
-
103
Raj Reddy
Sysadmin
-
104
Anand Ram
Developer
-
105
Jane Miller
Sales Manager

打印雇员名称和职位：

awk 'BEGIN{RS="-\n";FS="\n";OFS=":"}{print $2,$3}' employee-change-fs-ofs.txt

John Doe:CEO
Jason Smith:IT Manager
Raj Reddy:Sysadmin
Anand Ram:Developer
Jane Miller:Sales Manager

输出记录分隔符：

• 使用ORS代表输出记录分隔符，用来输出一行一行的分隔符

举例：

cat employee-change-fs-ofs.txt
101
John Doe
CEO
-
102
Jason Smith
IT Manager
-
103
Raj Reddy
Sysadmin
-
104
Anand Ram
Developer
-
105
Jane Miller
Sales Manager

awk 'BEGIN{RS="-\n";FS="\n";OFS=":";ORS="\n------\n"}{print $2,$3}' employee-change-fs-ofs.txt

John Doe:CEO
------
Jason Smith:IT Manager
------
Raj Reddy:Sysadmin
------
Anand Ram:Developer
------
Jane Miller:Sales Manager
------

 

记录序号：

• 用NR代表记录序号
• 用于统计当前记录在所有记录中的行号
• 末尾处可以表示总行数，即：在END区域时，代表输入文件的总记录数

举例：

cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

awk 'BEGIN{FS=","}{print "Emp id of record number",NR,"is",$1}END{print "Total number of records:",NR}' employee.txt

Emp id of record number 1 is 101
Emp id of record number 2 is 102
Emp id of record number 3 is 103
Emp id of record number 4 is 104
Emp id of record number 5 is 105
Total number of records: 5

• 当NR在awk中有多个输入文件时，NR不会被重置为1，而是在前一个文件的NR基础上继续增加

举例：

cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

cat employee-multiple-fs.txt
101,John Doe:CEO%10000
102,Jason Smith:IT Manager%5000
103,Raj Reddy:Sysadmin%4500
104,Anand Ram:Developer%4500
105,Jane Miller:Sales Manager%3000

awk 'BEGIN{FS=","}{print FILENAME": record number",NR,"is",$1}END{print "Total number of records:",NR}' employee.txt employee-multiple-fs.txt

employee.txt: record number 1 is 101
employee.txt: record number 2 is 102
employee.txt: record number 3 is 103
employee.txt: record number 4 is 104
employee.txt: record number 5 is 105
employee-multiple-fs.txt: record number 6 is 101
employee-multiple-fs.txt: record number 7 is 102
employee-multiple-fs.txt: record number 8 is 103
employee-multiple-fs.txt: record number 9 is 104
employee-multiple-fs.txt: record number 10 is 105
Total number of records: 10

• FNR与NR的区别：同时有多个输入文件时，NR不会重置，会一直累加；而FNR会重置

举例：

cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

cat employee-multiple-fs.txt
101,John Doe:CEO%10000
102,Jason Smith:IT Manager%5000
103,Raj Reddy:Sysadmin%4500
104,Anand Ram:Developer%4500
105,Jane Miller:Sales Manager%3000

awk 'BEGIN{FS=","}{print "FILENAME="FILENAME,"NR="NR,"FNR="FNR}END{print "END Block:NR="NR,"FNR="FNR}' employee.txt employee-multiple-fs.txt

FILENAME=employee.txt NR=1 FNR=1
FILENAME=employee.txt NR=2 FNR=2
FILENAME=employee.txt NR=3 FNR=3
FILENAME=employee.txt NR=4 FNR=4
FILENAME=employee.txt NR=5 FNR=5
FILENAME=employee-multiple-fs.txt NR=6 FNR=1
FILENAME=employee-multiple-fs.txt NR=7 FNR=2
FILENAME=employee-multiple-fs.txt NR=8 FNR=3
FILENAME=employee-multiple-fs.txt NR=9 FNR=4
FILENAME=employee-multiple-fs.txt NR=10 FNR=5
END Block:NR=10 FNR=5

 

打印偶数行：

cat items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Appliance,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5

awk 'NR%2 ==0' items.txt

102,Refrigerator,Appliance,850,2
104,Tennis Racket,Sports,190,20

 

 

 

在awk中调用文件执行命令：

cat fnr.awk
BEGIN{FS=","}{print "FILENAME="FILENAME,"NR="NR,"FNR="FNR}END{print "END Block:NR="NR,"FNR="FNR}

执行awk命令：

awk -f fnr.awk employee.txt employee-multiple-fs.txt

FILENAME=employee.txt NR=1 FNR=1
FILENAME=employee.txt NR=2 FNR=2
FILENAME=employee.txt NR=3 FNR=3
FILENAME=employee.txt NR=4 FNR=4
FILENAME=employee.txt NR=5 FNR=5
FILENAME=employee-multiple-fs.txt NR=6 FNR=1
FILENAME=employee-multiple-fs.txt NR=7 FNR=2
FILENAME=employee-multiple-fs.txt NR=8 FNR=3
FILENAME=employee-multiple-fs.txt NR=9 FNR=4
FILENAME=employee-multiple-fs.txt NR=10 FNR=5
END Block:NR=10 FNR=5

在sed中调用文件执行命令：

cat mycommands.sed
#!/bin/sed -f
#交换第一列与第二列
s/\([^,]*\),\([^,]*\),\(.*\).*/\2,\1,\3/g
#把整行内容放入<>中
s/^.*/<&>/
#把Developer替换为IT Manager
s/Developer/IT Manager/
#把Manager替换为Director
s/Manager/Director/

执行sed命令：

cat files.txt
/etc/passwd
/etc/group

./mycommands.sed files.txt

</etc/passwd>
</etc/group>

执行sed命令与awk命令的区别：

• sed文件需要写上#!/bin/sed -f  ，而awk文件只需要写awk ' '引号中的内容
• 注意：awk文件不能写引号
• 执行sed文件前要给.sed文件赋权限，而awk文件需要使用命令-f调用.awk文件

 

 getline命令：

awk在开始执行body区域时，执行任何命令之前，都会从文件中先读取一行数据，保存在变量$0中，然后再执行body中的内容。

比如：

awk -F"," '{getline tmp; print $0;}' items.txt

cat items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Appliance,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5

结果为：

101,HD Camcorder,Video,210,10
103,MP3 Player,Audio,270,15
105,Laser Printer,Office,475,5

只打印奇数行内容。getline tmp强制awk读取下一行，并保存在变量$0中，print $0时，$0仍然是第一行数据，因为getline tmp没有覆盖$0，因此会打印第一行数据，而不是第二行。body区域继续执行，只打印奇数行的数据。