005_awk案例实战

一、工作经验总结.

(1)日志案例:

?
1
2
3
10.100.194.39   10.100.194.39   1019-03-16T11:01:04+08:00   www.uuwatch.com^^3FF91DE01BCB49B8BD11198DB98394F8|1993969314640 GET 499 /agent-config?appId=biz.marketing&host=10.101.181.194&hostName=wg1-biz-marketing-139-1161719-11193146.c.elenet.me   httphttp1.1 0.000-  146 0   0   10.100.38.36:1891   -   -   -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1Nginx   1.6.1   wg1-bjdev-pub-nginxms-1 Python-urllib/1.7   -   -   -   www.uuwatch.com
10.100.194.39   10.100.194.39   1019-03-16T11:07:16+08:00   www.uuwatch.com^^C9398ACF399743669E71A19613D9F498|1993969646084 GET 499 /collector/tcp/cluster?cluster=wg1-collector-esm    http    http1.1 0.001   -   191 0   0   10.100.38.49:1891   application/json    -   -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1    Nginx   1.6.1wg1-bjdev-pub-nginxms-1    Go-http-client/1.1  -   -   -   www.uuwatch.com
10.100.194.39   10.100.194.39   1019-03-16T11:11:46+08:00   www.uuwatch.com^^1319B1E9EA1A4B9A9E498317606B74B7|1993969906741 GET 499 /agent-config?appId=arch.waf_collector&host=10.101.130.196  http    http1.1 0.000   -   169 0   0   10.100.40.17:1891   --  -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1    Nginx   1.6.1   wg1-bjdev-pub-nginxms-1 Go-http-client/1.1  --  -   www.uuwatch.com

只输出日志从第三列到最后的输出:

<1>cut -f 3- demo       Reference:https://stackoverflow.com/questions/1602035/how-to-print-third-column-to-last-column

?
1
2
3
1019-03-16T11:01:04+08:00   www.uuwatch.com^^3FF91DE01BCB49B8BD11198DB98394F8|1993969314640 GET 499 /agent-config?appId=biz.marketing&host=10.101.181.194&hostName=wg1-biz-marketing-139-1161719-11193146.c.elenet.me   httphttp1.1 0.000   -   146 0   0   10.100.38.36:1891   -   -   -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1    Nginx   1.6.1   wg1-bjdev-pub-nginxms-1 Python-urllib/1.7   -   -   -   www.uuwatch.com
1019-03-16T11:07:16+08:00   www.uuwatch.com^^C9398ACF399743669E71A19613D9F498|1993969646084 GET 499 /collector/tcp/cluster?cluster=wg1-collector-esm    http    http1.1 0.001   -   191 0   0   10.100.38.49:1891   application/json    -   -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1    Nginx   1.6.1wg1-bjdev-pub-nginxms-1    Go-http-client/1.1  -   -   -www.uuwatch.com
1019-03-16T11:11:46+08:00   www.uuwatch.com^^1319B1E9EA1A4B9A9E498317606B74B7|1993969906741 GET 499 /agent-config?appId=arch.waf_collector&host=10.101.130.196  http    http1.1 0.000   -   169 0   0   10.100.40.17:1891   -   -   -   www.uuwatch.com user    -   wg1 wg1-channel-stable-1    Nginx   1.6.1   wg1-bjdev-pub-nginxms-1 Go-http-client/1.1  -   -   -www.uuwatch.com

<2>第二种方法:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
awk '
{
    for(i=3;i<=NF;i++)
        rec[i]=(rec[i]?rec[i]RS$i:$i)
}
END {
    for(i=3;i<=NF;i++) print rec[i]
}'   splict
 
2029-03-26T22:02:04+08:00
www.uuwatch.com^^3FF92DE02BCB49B8BD22298DB98394F8|2993969324640
GET
499
/agent-config?appId=biz.marketing&host=20.202.282.294&hostName=wg2-biz-marketing-239-2262729-22293246.c.elenet.me
httphttp2.2
0.000
-
246
0
0
20.200.38.36:2892
-
-
-
www.uuwatch.com
user
-
wg2
wg2-channel-stable-2
Nginx
2.6.2
wg2-bjdev-pub-nginxms-2
Python-urllib/2.7
-
-
-
www.uuwatch.com

<3>第三种同上,区别参考:https://stackoverflow.com/questions/23644184/using-awk-to-take-a-range-of-columns-and-print-them-as-a-single-column

?
1
2
3
4
5
6
7
8
9
10
awk '
{
    for(i=3;i<=NF;i++) {
        rec[i]=(rec[i]?rec[i]RS$i:$i)
    }
    num=(num>NF?num:NF)
}
END {
    for(i=3;i<=num;i++) print rec[i]
}'   splict

(2)指定时间范围打印

cut -f 3- demo|sed -n '/2029-03-26T22:02:04+08:00/,/2029-03-26T22:07:26+08:00/p'

?
1
2
2029-03-26T22:02:04+08:00   www.uuwatch.com^^3FF92DE02BCB49B8BD22298DB98394F8|2993969324640 GET 499 /agent-config?appId=biz.marketing&host=20.202.282.294&hostName=wg2-biz-marketing-239-2262729-22293246.c.elenet.me   httphttp2.2 0.000   -   246 0   0   20.200.38.36:2892   -   -   -   www.uuwatch.com user    -   wg2 wg2-channel-stable-2    Nginx   2.6.2   wg2-bjdev-pub-nginxms-2 Python-urllib/2.7   -   -   -   www.uuwatch.com
2029-03-26T22:07:26+08:00   www.uuwatch.com^^C9398ACF399743669E72A29623D9F498|2993969646084 GET 499 /collector/tcp/cluster?cluster=wg2-collector-esm    http    http2.2 0.002   -   292 0   0   20.200.38.49:2892   application/json    -   -   www.uuwatch.com user    -   wg2 wg2-channel-stable-2    Nginx   2.6.2wg2-bjdev-pub-nginxms-2    Go-http-client/2.2  -   -   -www.uuwatch.com

(3)

 

(4)

?
1
2
3
4
5
6
7
8
cat new20190329.log|awk -F "\t" '{print $3,$6}'|sort |uniq -c
2 2019-03-28T18:30:03+08:00 499
1 2019-03-28T20:43:13+08:00 404
1 2019-03-28T20:43:19+08:00 404
14 2019-03-28T20:43:34+08:00 404
30 2019-03-28T20:43:35+08:00 404
22 2019-03-28T20:43:36+08:00 404
32 2019-03-28T20:43:37+08:00 404

 

 

二、

?
1
2
3
4
5
6
cat file.txt
groups=001(group1),
002(group2),
003(group3)
groups=004(group4),
005(group5)

只想输出

group1
group2
group3
group4
group5

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
(1)awk 'BEGIN{FS="[()]"} {if($0~/^.*[0-9][0-9][0-9]\(group[0-9]+\).*$/) print $2}' file.txt
步骤详解:
➜  011_cmdb_op awk 'BEGIN{FS="[()]"} {if($0~/^.*[0-9][0-9][0-9]\(group[0-9]+\).*$/) print $1}' file.txt
groups=001
002
003
groups=004
005
➜  011_cmdb_op awk 'BEGIN{FS="[()]"} {if($0~/^.*[0-9][0-9][0-9]\(group[0-9]+\).*$/) print $2}' file.txt
group1
group2
group3
group4
group5
➜  011_cmdb_op awk 'BEGIN{FS="[()]"} {if($0~/^.*[0-9][0-9][0-9]\(group[0-9]+\).*$/) print $3}' file.txt
,
,
 
,
 
#通过以上输出可见是以()为匹配的
 
或
(2)awk '{sub(/^.*[0-9][0-9][0-9]\(/,""); sub(/\).*$/,""); print}' file.txt
➜  011_cmdb_op awk '{sub(/^.*[0-9][0-9][0-9]\(/,"");print}' file.txt  #删除正则匹配的部分
group1),
group2),
group3)
group4),
group5)
awk '{sub(/^.*[0-9][0-9][0-9]\(/,""); sub(/\).*$/,""); print}' file.txt  #再删除括号后边的部分
group1
group2
group3
group4
group5<br>(3)实战<br>ls al-arch-soa-zk-1-al-arch-soa-zk-1

   ls al-arch-soa-zk-1-al-arch-soa-zk-1|awk '{sub(/^.*[0-9]-/,"");print}'
   al-arch-soa-zk-1

 

posted @   arun_yh  阅读(330)  评论(0)  编辑  收藏  举报
点击右上角即可分享
微信分享提示