json数据处理实用工具Jq

jq

jq可以对json数据进行分片、过滤、映射和转换。jq是用C编写，没有运行依赖。预编译的二进制文件直接可以在Linux、OS和windows系统上运行。

下载地址：https://stedolan.github.io/jq/download/

Linux安装包: jq,epel源

参考地址：

https://stedolan.github.io/jq/tutorial/

https://www.json.cn/wiki.html

#cat jq.txt
[{"name":"工具","url":"http://www.baidu.com","address":{"city":"厦门","country":"中国"},"arrayBrowser":[{"name":"Google","url":"http://www.google.com"},{"name":"Baidu","url":"http://www.baidu.com"}]},{"name":"大国小家","url":"http://www.baidu1.com","address":{"city":"大连","country":"中国"},"arrayBrowser":[{"name":"360","url":"http://www.baidu2.com"},{"name":"bing","url":"http://www.bing.com"}]}]

"."

最简单的jq程序是表达式“.”,不改变输入，将其优雅输出便于阅读和理解

​
#cat jq.txt |jq '.'
[
  {
    "name": "工具",
    "url": "http://www.baidu.com",
    "address": {
      "city": "厦门",
      "country": "中国"
    },
    "arrayBrowser": [
      {
        "name": "Google",
        "url": "http://www.google.com"
      },
      {
        "name": "Baidu",
        "url": "http://www.baidu.com"
      }
    ]
  },
  {
    "name": "大国小家",
    "url": "http://www.baidu1.com",
    "address": {
      "city": "大连",
      "country": "中国"
    },
    "arrayBrowser": [
      {
        "name": "360",
        "url": "http://www.baidu2.com"
      },
      {
        "name": "bing",
        "url": "http://www.bing.com"
      }
    ]
  }
]
​

[index]

输出列表中的第一个元素可以使用[index]

#cat jq.txt |jq '.[0]'
{
  "name": "工具",
  "url": "http://www.baidu.com",
  "address": {
    "city": "厦门",
    "country": "中国"
  },
  "arrayBrowser": [
    {
      "name": "Google",
      "url": "http://www.google.com"
    },
    {
      "name": "Baidu",
      "url": "http://www.baidu.com"
    }
  ]
}

| 管道

jq支持管道|，类似Linux命令的管道|——把前面命令的标准输出当做后面的命令的标准输入。如，命令把.[0]作为{...}的输入，进而访问嵌套的属性，如.name和.address.city

#cat jq.txt |jq '.[0]|{name:.name,city:.city}'
{
  "name": "工具",
  "city": null
}
​
#cat jq.txt |jq '.[0] | {name:.arrayBrowser[1].name,city:.address.city}'
{
  "name": "Baidu",
  "city": "厦门"
}
​
#cat jq.txt |jq '.[] | {name:.arrayBrowser[1].name,city:.address.city}'
{
  "name": "Baidu",
  "city": "厦门"
}
{
  "name": "bing",
  "city": "大连"
}
​

[]

将jq的输出当做一个数组，可以在前后加上[]

#cat jq.txt |jq '.[] | {name:.arrayBrowser[1].name,city:.address.city}'
{
  "name": "Baidu",
  "city": "厦门"
}
{
  "name": "bing",
  "city": "大连"
}
​

自定义key

在{}中，冒号前面的名字是映射的名字，可以任意修改

#cat jq.txt  | jq "[.[] | {name_001:.arrayBrowser[1].name,city_002:.address.city}]"
[
  {
    "name_001": "Baidu",
    "city_002": "厦门"
  },
  {
    "name_001": "bing",
    "city_002": "大连"
  }
]
​

JSON API

# URL 为我们获取 jq 存储库的最后 5 次提交
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5'
​
#返回格式良好的 JSON
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq'.'
​
#使用 jq 来提取第一次提交
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.[0]'
​
#取出父提交数组中的所有“html_url”字段，并制作一个简单的字符串列表，以配合我们已有的“消息”和“作者”字段。parents 字段被设置为[.parents[].html_url]，它收集了在parents 对象中定义的所有父提交URL
jq '[.[] | {消息：.commit.message，名称：.commit.committer.name，父母：[.parents[].html_url]}]'