linux 下jq的使用
安装：
yum install jq -y
文档：
https://stedolan.github.io/jq/manual/
更多:
https://blog.csdn.net/Cheat1173010256/article/details/118230562
数据源
333.json
[{
"name": "站长工具",
"url": "http://tool.chinaz.com",
"address": {
"city": "厦门",
"country": "中国"
},
"arrayBrowser": [{
"name": "Google",
"url": "http://www.google.com"
},
{
"name": "Baidu",
"url": "http://www.baidu.com"
}
]
},
{
"name": "站长之家",
"url": "http://tool.zzhome.com",
"address": {
"city": "大连",
"country": "中国"
},
"arrayBrowser": [{
"name": "360",
"url": "http://www.so.com"
},
{
"name": "bing",
"url": "http://www.bing.com"
}
],
"127.0.0.1": {
"error": 200,
"msg": "ok"
}
}
]
例子基础
haima@haima-PC:~/Desktop$ cat 333.json |jq '.'
[
{
"name": "站长工具",
"url": "http://tool.chinaz.com",
"address": {
"city": "厦门",
"country": "中国"
},
"arrayBrowser": [
{
"name": "Google",
"url": "http://www.google.com"
},
{
"name": "Baidu",
"url": "http://www.baidu.com"
}
]
},
{
"name": "站长之家",
"url": "http://tool.zzhome.com",
"address": {
"city": "大连",
"country": "中国"
},
"arrayBrowser": [
{
"name": "360",
"url": "http://www.so.com"
},
{
"name": "bing",
"url": "http://www.bing.com"
}
],
"127.0.0.1": {
"error": 200,
"msg": "ok"
}
}
]
管道线 |
jq支持管道线 |，它如同linux命令中的管道线——把前面命令的输出当作是后面命令的输入。如下命令把.[0]作为{…}的输入，进而访问嵌套的属性，如.name和.address.city。
观察如下几个命令，通过改变|前后的输入和输出来达到不同的效果：
cat json.txt | jq '.[0] | {name:.name,city:.address.city}'
{
"name": "站长工具",
"city": "厦门"
}
cat json.txt | jq '.[0] | {name:.arrayBrowser[1].name,city:.address.city}'
{
"name": "Baidu",
"city": "厦门"
}
cat json.txt | jq ".[] | {name:.arrayBrowser[1].name,city:.address.city}"
{
"name": "Baidu",
"city": "厦门"
}
{
"name": "bing",
"city": "大连"
}
[]
如果希望把jq的输出当作一个数组，可以在前后加上[]：
cat json.txt | jq "[.[] | {name:.arrayBrowser[1].name,city:.address.city}]"
[
{
"name": "Baidu",
"city": "厦门"
},
{
"name": "bing",
"city": "大连"
}
]
自定义key
在{}中，冒号前面的名字是映射的名称，你可以任意修改，如：
```sh
cat json.txt | jq "[.[] | {name_001:.arrayBrowser[1].name,city_002:.address.city}]"
[
{
"name_001": "Baidu",
"city_002": "厦门"
},
{
"name_001": "bing",
"city_002": "大连"
}
]
原文链接：https://blog.csdn.net/qq_26502245/article/details/100191694
取0号单元下的元素
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].name'
"站长工具"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address'
{
"city": "厦门",
"country": "中国"
}
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address.city'
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address["city"]'
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq ".[0].address[\"city\"]"
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[1]["127.0.0.1"]'
{
"error": 200,
"msg": "ok"
}
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'keys' #获取0号单元的所有key
[
"address",
"arrayBrowser",
"name",
"url"
]
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0].address" # 获取0号单元的address
{
"city": "厦门",
"country": "中国"
}
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0].address" | jq 'keys' #获取0号单元的address里的所有 key(嵌套提取)
[
"city",
"country"
]
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'has("name")' #判断是否有某个 key
true
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'has("test")' #判断是否有某个 key
false
压缩json
haima@haima-PC:~/Desktop$ cat 333.json | jq -c .
[{"name":"站长工具","url":"http://tool.chinaz.com","address":{"city":"厦门","country":"中国"},"arrayBrowser":[{"name":"Google","url":"http://www.google.com"},{"name":"Baidu","url":"http://www.baidu.com"}]},{"name":"站长之家","url":"http://tool.zzhome.com","address":{"city":"大连","country":"中国"},"arrayBrowser":[{"name":"360","url":"http://www.so.com"},{"name":"bing","url":"http://www.bing.com"}],"127.0.0.1":{"error":200,"msg":"ok"}}]
例子二:
数据源
22.json
{"ip":"34.120.75.75"}
{"ip":"35.193.30.131"}
{"ip":"35.227.228.135"}
{"ip":"130.211.21.179"}
{"ip":"35.192.215.111"}
{"ip":"35.214.27.221"}
{"ip":"34.120.136.4"}
{"ip":"35.209.185.188"}
{"ip":"34.121.213.115"}
{"ip":"34.98.116.47"}
取出ip
$ cat 22.json |jq '.ip'
"34.120.75.75"
"35.193.30.131"
"35.227.228.135"
"130.211.21.179"
"35.192.215.111"
"35.214.27.221"
"34.120.136.4"
"35.209.185.188"
"34.121.213.115"
"34.98.116.47"
取出ip并去除双引号 (需要安装jq)
$ cat 22.json |jq '.ip' | sed 's/\"//g'
34.120.75.75
35.193.30.131
35.227.228.135
130.211.21.179
35.192.215.111
35.214.27.221
34.120.136.4
35.209.185.188
34.121.213.115
34.98.116.47
排序去重后,取出ip并去除双引号 (需要安装jq)
$ cat 22.json |sort | uniq | sort -n | jq '.ip' | sed 's/\"//g'
34.120.241.13
34.125.140.101
34.126.67.33
34.65.22.11
34.77.137.149
34.85.82.186
34.86.160.16
34.90.193.87
34.92.44.105
34.93.161.137
34.94.248.178
解析：
s 为替换
以g结尾表示的是:全局性,意即”替代文本取代正则表达式中每一个匹配的”
直接改变的话用i：
sed i 's/\"//g' 11.txt
另外输出的话：
sed 's/\"//g' aa.txt > 11.clean.txt
例子:
取出port和value ,组装json格式
{"took":648,"timed_out":false,"_shards":{"total":50,"successful":50,"skipped":0,"failed":0},"hits":{"total":9893731,"max_score":1.0,"hits":[{"_index":"fofapro_service","_type":"order","_id":"1.169.157.54:80","_score":1.0,"_source":{"port":80}}]},"aggregations":{"distinct_ips":{"value":9883243}}}
{"took":648,"timed_out":false,"_shards":{"total":50,"successful":50,"skipped":0,"failed":0},"hits":{"total":9893731,"max_score":1.0,"hits":[{"_index":"fofapro_service","_type":"order","_id":"1.169.157.54:80","_score":1.0,"_source":{"port":81}}]},"aggregations":{"distinct_ips":{"value":9883243}}}
命令:
$ cat es_port.json | jq '{port:.hits.hits[0]._source.port , count:.aggregations.distinct_ips.value}' -c | awk -F'port":' '{print $2}' | awk -F',"count":' '{print "{\"" $1"\":\""$2}' | sed 's/}/"}/g'
{"80":"9883243"}
{"81":"9883243"}
-c ：一行显示
4 JSON parse array
cat json_raw.txt | jq '.employees[1].name'
"Laura"
5 内建函数
jq还有一些内建函数如 key，has
key是用来获取JSON中的key元素的：
cat json_raw.txt | jq 'keys'
[
"employees",
"location",
"name"
]
has是用来是判断是否存在某个key：
cat json_raw.txt | jq 'has("name")'
true
cat json_raw.txt | jq 'has("noexisted")'
false
[Haima的博客]
http://www.cnblogs.com/haima/
