Elasticsearch7.8.0教程（一）
Elasticsearch7.8.0教程（一）
一. 前序
Elasticsearch是一个基于
Apache Lucene(TM)的开源搜索引擎。无论在开源还是专有领域，Lucene 可以被认为是迄今为止最先进、性能最好的、功能最全的搜索引擎库。但是，Lucene只是一个库。想要 使用它，你必须使用Java来作为开发语言并将其直接集成到你的 应用中，更糟糕的是，Lucene非常复杂，你需要深入了解检索的相关知识来理解它是如何工作的。
Elasticsearch也使用Java开发并使用Lucene作为其核心来实现所有索引和搜索的功能，但是它的目 的是通过简单的
RESTful API来隐藏Lucene的复杂性，从而让全文搜索变得简单。Elasticsearch的中文网址：https://www.elastic.co/cn/products/elasticsearch
1.1 正向索引和倒排索引
正向索引与倒排索引，这是在搜索领域中非常重要的两个名词，正向索引通常用于数据库中，在搜 索引擎领域使用的最多的就是倒排索引，我们根据如下两个网页来对这两个概念进行阐述：
html1
我爱我的祖国，我爱编程
html2
我爱编程，我是个快乐的小码农
1.1.1 正向索引
假设我们使用mysql的全文检索，会对如上两句话分别进行分词处理，那么预计得到的结果如下：
我 爱 爱我 祖国 我的祖国 编程 爱编程 我爱编程
我 我爱 爱 编程 爱编程 我爱编程 快乐 码农 小码农
假设我们现在使用正向索引搜索 编程 这个词，那么会到第一句话中去查找是否包含有
编程这 个关键词，如果有则加入到结果集中；第二句话也是如此。假设现在有成千上百个网页，每个网页 非常非常多的分词，那么搜索的效率将会非常非常低些。
1.1.2 倒排索引
倒排索引是按照分词与文档进行映射，我们来看看如果按照倒排索引的效果：
| 关键词 | 文档名 |
|---|---|
| 我 | html1,html2,html3 |
| 爱 | html1,html2 |
| 爱我 | html1 |
| 我爱 | html2 |
| 祖国 | html1 |
| 我的祖国 | html1 |
| 编程 | html1,html2 |
| 我爱编程 | html1,html2 |
| 爱编程 | html1,html2 |
| 快乐 | html2 |
| 码农 | html2 |
| 小码农 | html2 |
如果采用倒排索引的方式搜索
编程这个词，那么会直接找到关键词中查找到编程，然后查找 到对应的文档，这就是所谓的倒排索引。
二. 软件简介以及启动
2.1 相关软件下载地址(ELK)
| 软件名 | 下载地址 |
|---|---|
| Elasticsearch | https://www.elastic.co/cn/start |
| Logstash | https://www.elastic.co/cn/downloads/logstash |
| Kibana | https://www.elastic.co/cn/start |
2.2 Elasticsearch安装
进入到
elasticsearch解压目录下的 bin 目录下，双击elasticsearch.bat即可启动。 在浏览器地址栏输入: http://localhost:9200/ ，如果出现如下页面表示 elasticsearch 启动成功
2.3 Kibana
2.3.1 Kibana简介
Kibana是世界上最受欢迎的开源日志分析平台ELK Stack中的“K” ，它为用户提供了一个工具，用于 在存储于Elasticsearch集群中的日志数据进行检索，可视化和构建仪表板。
Kibana的核心功能是数据查询和分析。使用各种方法，用户可以搜索Elasticsearch中索引的数据， 以查找其数据中的特定事件或字符串，以进行根本原因分析和诊断。基于这些查询，用户可以使用 Kibana的可视化功能，允许用户使用图表，表格，地理图和其他类型的可视化以各种不同的方式可 视化数据。
2.3.2 Kibana的启动
进入到 kibana 解压目录下的 bin 目录下，双击 kibana.bat 即可启动 kibana. 在浏览器地址栏输入：http://localhost:5601，出现如下页面代表 kibana 启动成功。
2.4 Logstash
2.4.1 logstash简介
Logstash是一个开源的服务器端数据处理管道，可以同时从多个数据源获取数据，并对其进行转 换，然后将其发送到你最喜欢的“存储”。创建于2009年，于2013年被elasticsearch收购。
2.4.2 logstash导入数据
虽然
kibana提供了一些数据集供我们使用，为了加深对logstash的理解，我们movielens的电影数据集。movielens 数据集的下载地址为：http://files.grouplens.org/datasets/movielens，进入该网页只 用下载 ml-latest.zip 数据即可，如下图所示：
将
ml-latest.zip加压文件中的movies.csv文件拷贝到logstash的家目录下； 再将logstash的config目录下新建名为logstash.conf的文件，文件内容如下:
input {
file {
# 引号的的内容为 movies.csv 的实际路径，根据实际情况而定
path => "D:/environment/logstash-7.8.0/movies.csv"
start_position => "beginning"
sincedb_path => "D:/environment/logstash-7.8.0/db_path.log"
}
}
filter {
csv {
separator => ","
columns => ["id","content","genre"]
}
mutate {
split => { "genre" => "|" }
remove_field => ["path", "host","@timestamp","message"]
}
mutate {
split => ["content", "("]
add_field => { "title" => "%{[content][0]}"}
add_field => { "year" => "%{[content][1]}"}
}
mutate {
convert => {
"year" => "integer"
}
strip => ["title"]
remove_field => ["path", "host","@timestamp","message","content"]
}
}
output {
elasticsearch {
# 双引号中的内容为ES的地址，视实际情况而定
hosts => "http://localhost:9200"
index => "movies"
document_id => "%{id}"
}
stdout {}
}
movies.csv数据
打开dos命令行，进入到
logstash的bin目录下，执行如下命令导入movielens的数据集。备注：如果要重新导入，需先删除
db_path.log文件
logstash.bat -f D:\environment\logstash-7.8.0\logstash.conf
2.4.3 验证
进入到 kibana 的命令行页面，执行
GET _cat/indices验证数据是否成功导入
三. Elasticsearch的基本概念
3.1 索引
Elasticsearch中的索引有多层的意思：
a. 某一类文档的集合就构成了一个索引，类比到数据库就是一个数据库(或者数据库表);
b.它还描述了一个动作，就是将某个文档保存在elasticsearch的过程也叫索引;
c. 倒排索引。
3.2 文档
具体的一条数据，类比到数据库就是一条记录。
GET movies/_search
3.4 mapping
mapping是ES每一个文档的约束信息，例如属性的类型，是否能被索引等。
3.5 DSL
DSL 是 ES 的查询语言。
3.6 类比
我们通过大家比较熟悉的
DBMS与ES的基本概念进行类比，加深大家的理解。
| DBMS | Elasticsearch |
|---|---|
| database | Index |
| table | type(在7.0之后type为固定值_doc) |
| Row | Document |
| Column | Field |
| Schema | Mapping |
| SQL | DSL(Descriptor Structure Language) |
在7.0之前，一个
Index可以创建多个类型，从7.0开始，一个索引只能创建一个类型，也就是_doc
四. RestAPI
4.1 基本CRUD
4.1.1 查询movies的数据
GET movies/_search
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32225",
"_score" : 1.0,
"_source" : {
"title" : "Project Grizzly",
"genre" : [
"Documentary"
],
"year" : 1996,
"id" : "32225",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32230",
"_score" : 1.0,
"_source" : {
"title" : "Snow Queen, The",
"genre" : [
"Children",
"Fantasy"
],
"year" : 0,
"id" : "32230",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32234",
"_score" : 1.0,
"_source" : {
"title" : "Julia",
"genre" : [
"Drama"
],
"year" : 1977,
"id" : "32234",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32239",
"_score" : 1.0,
"_source" : {
"title" : "Save the Green Planet!",
"genre" : [
"Comedy",
"Drama",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 0,
"id" : "32239",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32243",
"_score" : 1.0,
"_source" : {
"title" : "Stealing Rembrandt",
"genre" : [
"Action",
"Comedy",
"Crime"
],
"year" : 0,
"id" : "32243",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32250",
"_score" : 1.0,
"_source" : {
"title" : "Snake of June, A",
"genre" : [
"Drama",
"Mystery"
],
"year" : 0,
"id" : "32250",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32261",
"_score" : 1.0,
"_source" : {
"title" : "Thirty Seconds Over Tokyo",
"genre" : [
"Drama",
"War"
],
"year" : 1944,
"id" : "32261",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32263",
"_score" : 1.0,
"_source" : {
"title" : "Vares: Private Eye",
"genre" : [
"Action",
"Comedy",
"Crime",
"Thriller"
],
"year" : 0,
"id" : "32263",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32277",
"_score" : 1.0,
"_source" : {
"title" : "Girl Crazy",
"genre" : [
"Comedy",
"Musical",
"Romance"
],
"year" : 1943,
"id" : "32277",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32280",
"_score" : 1.0,
"_source" : {
"title" : "The 3 Penny Opera",
"genre" : [
"Comedy",
"Drama",
"Musical"
],
"year" : 1931,
"id" : "32280",
"@version" : "1"
}
}
]
}
}
4.1.2 查询movies的总数
GET movies/_count
{
"count" : 58099,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
4.1.3 查看所有的索引
GET _cat/indices
green open .kibana-event-log-7.8.0-000001 IPfj1sfGQWCv3wiBM5nfXA 1 0 2 0 10.4kb 10.4kb
yellow open movies BdLadipoRa2MfeKeFkkjiw 1 1 58099 0 7mb 7mb
green open .kibana-event-log-7.8.0-000002 mtvXKAziSWGWwtxTMHCnmw 1 0 1 0 5.3kb 5.3kb
yellow open product 9Hovl5eKR0Gg1qET5rlDcw 3 1 0 0 624b 624b
green open .apm-custom-link j-margrYSpylvIEI8pVa-w 1 0 0 0 208b 208b
green open .kibana_task_manager_1 aDRY7wE3TJqArR7VSzyHGQ 1 0 5 8 73.7kb 73.7kb
green open kibana_sample_data_ecommerce j84eHNbtRkGeA8UDcGurkg 1 0 4675 0 4.5mb 4.5mb
green open .apm-agent-configuration DKoxjNaMSFa78gU_t2c_9g 1 0 0 0 208b 208b
green open kibana_sample_data_logs AYGZxvG5R7K7R6L4g0KT1w 1 0 14074 0 11.3mb 11.3mb
green open .kibana_1 r7H-MJHtSbWB-nyYTio9-w 1 0 232 0 1.3mb 1.3mb
green open kibana_sample_data_flights aZEbUyVwTUeinyzpRPZ4Ng 1 0 13059 0 6.2mb 6.2mb
yellow open users 1RU70EgyTKinRJbcG-QigQ 1 1 2 0 8kb 8kb
yellow open shopping TXu8AmJCS-2nj1oHdljATw 1 1 6 0 16.1kb 16.1kb
4.1.4 查询id为24的数据
GET movies/_doc/24
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "24",
"_version" : 1,
"_seq_no" : 249,
"_primary_term" : 1,
"found" : true,
"_source" : {
"title" : "Powder",
"genre" : [
"Drama",
"Sci-Fi"
],
"year" : 1995,
"id" : "24",
"@version" : "1"
}
}
4.1.5 加一条数据到索引users
POST users/_doc
{
"firstname": "will",
"lastname": "smith"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
}
]
}
}
4.1.6 添加指定id为1的文档
POST users/_doc/1
{
"firstname": "jack",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 3,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1
}
GET users/_search
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.7 添加指定相同id为1的文档，会把之前的数据覆盖(比较危险，尽量少用)
POST users/_doc/1
{
"firstname": "rod",
"lastname": "johnson"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 4,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1
}
GET users/_search
{
"took" : 1006,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
}
]
}
}
4.1.8创建id为1的文档，如果索引中已存在相同id，会报错(可以替代4.1.7命令)
POST users/_create/1
{
"firstname": "jack",
"lastname": "ma"
}
{
"error" : {
"root_cause" : [
{
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [4])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
}
],
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [4])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
},
"status" : 409
}
POST users/_create/2
{
"firstname": "jack",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 5,
"_primary_term" : 1
}
GET users/_search
{
"took" : 215,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.9 添加一个属性age(会覆盖原有属性)
POST users/_doc/2
{
"age":53
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 6,
"_primary_term" : 1
}
GET users/_search
{
"took" : 994,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53
}
}
]
}
}
4.1.10 添加一个属性age(不会覆盖原有属性)
POST users/_update/2
{
"doc": {
"firstname": "jack",
"lastname": "ma"
}
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 7,
"_primary_term" : 1
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.11 PUT添加一条数据(用法与POST相同)
PUT users/_doc/1
{
"firstname": "pony",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 5,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 8,
"_primary_term" : 1
}
GET users/_search
{
"took" : 771,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
}
]
}
}
PUT users/_create/2
{
"firstname": "harry",
"lastname": "potter"
}
{
"error" : {
"root_cause" : [
{
"type" : "version_conflict_engine_exception",
"reason" : "[2]: version conflict, document already exists (current version [3])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
}
],
"type" : "version_conflict_engine_exception",
"reason" : "[2]: version conflict, document already exists (current version [3])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
},
"status" : 409
}
PUT users/_create/3
{
"firstname": "harry",
"lastname": "potter"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 9,
"_primary_term" : 1
}
4.1.12 删除指定id为3的数据
GET users/_search
{
"took" : 308,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"firstname" : "harry",
"lastname" : "potter"
}
}
]
}
}
DELETE users/_doc/3
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 2,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 10,
"_primary_term" : 1
}
GET users/_search
{
"took" : 530,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
}
]
}
}
4.1.13 删除整个users索引
DELETE users
{
"acknowledged" : true
}
GET _cat/indices
green open .kibana-event-log-7.8.0-000001 IPfj1sfGQWCv3wiBM5nfXA 1 0 2 0 10.4kb 10.4kb
yellow open movies BdLadipoRa2MfeKeFkkjiw 1 1 58099 0 7mb 7mb
green open .kibana-event-log-7.8.0-000002 mtvXKAziSWGWwtxTMHCnmw 1 0 1 0 5.3kb 5.3kb
yellow open product 9Hovl5eKR0Gg1qET5rlDcw 3 1 0 0 624b 624b
green open .apm-custom-link j-margrYSpylvIEI8pVa-w 1 0 0 0 208b 208b
green open kibana_sample_data_ecommerce j84eHNbtRkGeA8UDcGurkg 1 0 4675 0 4.5mb 4.5mb
green open .kibana_task_manager_1 aDRY7wE3TJqArR7VSzyHGQ 1 0 5 8 73.7kb 73.7kb
green open .apm-agent-configuration DKoxjNaMSFa78gU_t2c_9g 1 0 0 0 208b 208b
green open kibana_sample_data_logs AYGZxvG5R7K7R6L4g0KT1w 1 0 14074 0 11.3mb 11.3mb
green open .kibana_1 r7H-MJHtSbWB-nyYTio9-w 1 0 254 20 1.3mb 1.3mb
yellow open user -U8L42soTjiCygg8Cr3M9g 1 1 1 0 4.2kb 4.2kb
green open kibana_sample_data_flights aZEbUyVwTUeinyzpRPZ4Ng 1 0 13059 0 6.2mb 6.2mb
yellow open shopping TXu8AmJCS-2nj1oHdljATw 1 1 6 0 16.1kb 16.1kb
4.1.14 批量添加数据
POST users/_bulk
{"index":{"_id":1}}
{"firstname": "a", "lastname": "A"}
{"index": {"_id": 2}}
{"firstname": "x", "lastname": "X"}
{"index": {}}
{"firstname": "y", "lastname": "Y"}
{"index": {"_id": 3}}
{"firstname": "z", "lastname": "Z"}
{
"took" : 560,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "cX_dSnwBXvXiB6zYeylY",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1,
"status" : 201
}
}
]
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "cX_dSnwBXvXiB6zYeylY",
"_score" : 1.0,
"_source" : {
"firstname" : "y",
"lastname" : "Y"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"firstname" : "z",
"lastname" : "Z"
}
}
]
}
}
4.1.15 批量查询多个指定的id的数据，也可以批量查询
GET _mget
{
"docs": [
{"_index": "users", "_id": 1},
{"_index": "users", "_id": 2},
{"_index": "users", "_id": 3}
]
}
{
"docs" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"_seq_no" : 3,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "z",
"lastname" : "Z"
}
}
]
}
GET _mget
{
"docs": [
{"_index": "users", "_id": 1},
{"_index": "users", "_id": 2},
{"_index": "movies", "_id": 38701}
]
}
{
"docs" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "38701",
"_version" : 1,
"_seq_no" : 10500,
"_primary_term" : 1,
"found" : true,
"_source" : {
"title" : "Don't Come Knocking",
"genre" : [
"Drama",
"Western"
],
"year" : 2005,
"id" : "38701",
"@version" : "1"
}
}
]
}
4.2 URI查询
4.2.1 查询所有的属性中只要包含2012的所有的数据，泛查询(没有指定特定字段的查找)
GET movies/_search?q=2012
{
"took" : 29,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1867,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : 11.507438,
"_source" : {
"title" : "2012",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
],
"year" : 2009,
"id" : "72378",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "2012",
"_score" : 10.564456,
"_source" : {
"title" : "Back to the Future Part III",
"genre" : [
"Adventure",
"Comedy",
"Sci-Fi",
"Western"
],
"year" : 1990,
"id" : "2012",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "80505",
"_score" : 9.692035,
"_source" : {
"title" : "2012: Supernova",
"genre" : [
"Action",
"Sci-Fi"
],
"year" : 2009,
"id" : "80505",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "102848",
"_score" : 9.692035,
"_source" : {
"title" : "Armageddon 2012",
"genre" : [
"Sci-Fi"
],
"year" : 2012,
"id" : "102848",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "84017",
"_score" : 7.367466,
"_source" : {
"title" : "2012: Time for Change",
"genre" : [
"Animation",
"Documentary"
],
"year" : 2010,
"id" : "84017",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "189451",
"_score" : 7.367466,
"_source" : {
"title" : "Macross: Flash Back 2012",
"genre" : [
"Animation",
"Sci-Fi"
],
"year" : 1987,
"id" : "189451",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "144456",
"_score" : 6.578554,
"_source" : {
"title" : "All's Well, Ends Well 2012",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2012,
"id" : "144456",
"@version" : "1"
}
}
]
}
}
4.2.2 查询title中包含2012的所有的电影，df(default field)
GET movies/_search?q=2012&df=title
GET movies/_search?q=title:2012
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 13,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : 11.507438,
"_source" : {
"title" : "2012",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
],
"year" : 2009,
"id" : "72378",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "80505",
"_score" : 9.692035,
"_source" : {
"title" : "2012: Supernova",
"genre" : [
"Action",
"Sci-Fi"
],
"year" : 2009,
"id" : "80505",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "102848",
"_score" : 9.692035,
"_source" : {
"title" : "Armageddon 2012",
"genre" : [
"Sci-Fi"
],
"year" : 2012,
"id" : "102848",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "84017",
"_score" : 7.367466,
"_source" : {
"title" : "2012: Time for Change",
"genre" : [
"Animation",
"Documentary"
],
"year" : 2010,
"id" : "84017",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "189451",
"_score" : 7.367466,
"_source" : {
"title" : "Macross: Flash Back 2012",
"genre" : [
"Animation",
"Sci-Fi"
],
"year" : 1987,
"id" : "189451",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "144456",
"_score" : 6.578554,
"_source" : {
"title" : "All's Well, Ends Well 2012",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2012,
"id" : "144456",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "160329",
"_score" : 6.578554,
"_source" : {
"title" : "I Love Hong Kong 2012",
"genre" : [
"Comedy"
],
"year" : 2012,
"id" : "160329",
"@version" : "1"
}
}
]
}
}
4.2.3 查询title中包含Beautiful或者Mind的所有的数据
GET movies/_search?q=title:Beautiful Mind
GET movies/_search?q=title:(Beautiful Mind)
GET movies/_search?q=title:(+Beautiful +Mind)
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 145,
"relation" : "eq"
},
"max_score" : 13.474831,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474831,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "74064",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2009,
"id" : "74064",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "113760",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "113760",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "3912",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2000,
"id" : "3912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "47404",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Game",
"genre" : [
"Adventure",
"Animation",
"Comedy",
"Fantasy",
"Romance",
"Sci-Fi"
],
"year" : 2004,
"id" : "47404",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "124129",
"_score" : 8.210646,
"_source" : {
"title" : "Menno's Mind",
"genre" : [
"Sci-Fi",
"Thriller"
],
"year" : 1997,
"id" : "124129",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "150048",
"_score" : 8.210646,
"_source" : {
"title" : "Dirty Mind",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2009,
"id" : "150048",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "153272",
"_score" : 8.210646,
"_source" : {
"title" : "Mirrored Mind",
"genre" : [
"(no genres listed)"
],
"year" : 2006,
"id" : "153272",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "175563",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Blown",
"genre" : [
"Action",
"Sci-Fi",
"Thriller"
],
"year" : 2016,
"id" : "175563",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "188239",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Ripper",
"genre" : [
"Horror",
"Sci-Fi"
],
"year" : 1995,
"id" : "188239",
"@version" : "1"
}
}
]
}
}
4.2.4 查询title中包含Beautiful但是不包含mind的所 有的数据
GET movies/_search?q=title:(Beautiful NOT Mind)
GET movies/_search?q=title:(Beautiful -Mind)
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 97,
"relation" : "eq"
},
"max_score" : 8.774164,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "74064",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2009,
"id" : "74064",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "113760",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "113760",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "3912",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2000,
"id" : "3912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "66701",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Ohio",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2006,
"id" : "66701",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "89449",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Lies",
"genre" : [
"Comedy",
"Drama",
"Romance"
],
"year" : 0,
"id" : "89449",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "90353",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Boy",
"genre" : [
"Drama"
],
"year" : 2010,
"id" : "90353",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "110123",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Darling",
"genre" : [
"Documentary"
],
"year" : 2010,
"id" : "110123",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "114126",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Losers",
"genre" : [
"Documentary"
],
"year" : 2008,
"id" : "114126",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "117899",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Girl",
"genre" : [
"Drama"
],
"year" : 2014,
"id" : "117899",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "27232",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Joe",
"genre" : [
"Comedy",
"Drama",
"Romance"
],
"year" : 2000,
"id" : "27232",
"@version" : "1"
}
}
]
}
}
4.2.5 查询title中既包含Mind又包含Beautiful的所有 的数据，与顺序没有关系
GET movies/_search?q=title:(mind AND beautiful)
GET movies/_search?q=title:(beautiful AND mind)
#如果and变小写这表示搜索包含mind或and或beautiful的数据
#GET movies/_search?q=title:(mind and beautiful)
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 13.474831,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474831,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
}
]
}
}
4.2.6 查询title中包含 "Beautiful Mind"这个短语的所 有的数据
GET movies/_search?q=title:"beautiful mind"
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 13.474829,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474829,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
}
]
}
}
GET movies/_search?q=title:"beautiful AND mind"
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
4.2.7 查询title中包含2012，每页3条从第二页开始，查询3条 数据
GET movies/_search?q=title:2012&from=3&size=3
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 13,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
}
]
}
}
4.2.8 查询2018年之后上映的电影
GET movies/_search?q=year:>=2018
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 859,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122910",
"_score" : 1.0,
"_source" : {
"title" : "Captain Marvel",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2018,
"id" : "122910",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122912",
"_score" : 1.0,
"_source" : {
"title" : "Avengers: Infinity War - Part I",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2018,
"id" : "122912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "135448",
"_score" : 1.0,
"_source" : {
"title" : "Avatar 4",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Sci-Fi"
],
"year" : 2018,
"id" : "135448",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "143347",
"_score" : 1.0,
"_source" : {
"title" : "Aquaman",
"genre" : [
"Action",
"Fantasy",
"Sci-Fi"
],
"year" : 2018,
"id" : "143347",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182369",
"_score" : 1.0,
"_source" : {
"title" : "Journey's End",
"genre" : [
"War"
],
"year" : 2018,
"id" : "182369",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182715",
"_score" : 1.0,
"_source" : {
"title" : "Annihilation",
"genre" : [
"Adventure",
"Mystery",
"Sci-Fi",
"Thriller"
],
"year" : 2018,
"id" : "182715",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182835",
"_score" : 1.0,
"_source" : {
"title" : "The 15:17 to Paris",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2018,
"id" : "182835",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183011",
"_score" : 1.0,
"_source" : {
"title" : "The Commuter",
"genre" : [
"Crime",
"Drama",
"Mystery",
"Thriller"
],
"year" : 2018,
"id" : "183011",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183055",
"_score" : 1.0,
"_source" : {
"title" : "The Beyond",
"genre" : [
"Horror",
"Sci-Fi"
],
"year" : 2018,
"id" : "183055",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183155",
"_score" : 1.0,
"_source" : {
"title" : "Lean on Pete",
"genre" : [
"Adventure",
"Drama"
],
"year" : 2018,
"id" : "183155",
"@version" : "1"
}
}
]
}
}
4.2.9查询在2012到2017年上映的电影
GET movies/_search?q=year:(>=2012 AND <2018)
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 2.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "89745",
"_score" : 2.0,
"_source" : {
"title" : "Avengers, The",
"genre" : [
"Action",
"Adventure",
"Sci-Fi",
"IMAX"
],
"year" : 2012,
"id" : "89745",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121124",
"_score" : 2.0,
"_source" : {
"title" : "StreetDance 2",
"genre" : [
"Drama",
"Romance"
],
"year" : 2012,
"id" : "121124",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121129",
"_score" : 2.0,
"_source" : {
"title" : "The Hungover Games",
"genre" : [
"Comedy"
],
"year" : 2014,
"id" : "121129",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121135",
"_score" : 2.0,
"_source" : {
"title" : "Nurse 3D",
"genre" : [
"Horror",
"Thriller"
],
"year" : 2013,
"id" : "121135",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121137",
"_score" : 2.0,
"_source" : {
"title" : "Wrong Turn 5: Bloodlines",
"genre" : [
"Horror",
"Thriller"
],
"year" : 2012,
"id" : "121137",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121143",
"_score" : 2.0,
"_source" : {
"title" : "Flu",
"genre" : [
"Action",
"Drama",
"Sci-Fi"
],
"year" : 2013,
"id" : "121143",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121145",
"_score" : 2.0,
"_source" : {
"title" : "McCullin",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "121145",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121152",
"_score" : 2.0,
"_source" : {
"title" : "The White Haired Witch of Lunar Kingdom",
"genre" : [
"Action",
"Fantasy"
],
"year" : 2014,
"id" : "121152",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121155",
"_score" : 2.0,
"_source" : {
"title" : "Kevin Hart: Let Me Explain",
"genre" : [
"Comedy",
"Documentary"
],
"year" : 2013,
"id" : "121155",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121157",
"_score" : 2.0,
"_source" : {
"title" : "Tad, the Lost Explorer",
"genre" : [
"Adventure",
"Animation",
"Children",
"Comedy"
],
"year" : 2012,
"id" : "121157",
"@version" : "1"
}
}
]
}
}
4.2.10 查询title中包含beautiful或mind，并且电影上映年份在[1990,1992]的所有的数据
GET movies/_search?q=year:(>=1=990 AND <=1992) AND title:beautiful mind
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
4.2.11 查询2016年到2017年上映的电影，必须以 ] 结尾
#前开后闭 大于2015，小于等于2017
GET movies/_search?q=year:{2015 TO 2017]
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4099,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122888",
"_score" : 1.0,
"_source" : {
"title" : "Ben-hur",
"genre" : [
"(no genres listed)"
],
"year" : 2016,
"id" : "122888",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122890",
"_score" : 1.0,
"_source" : {
"title" : "Warcraft",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2016,
"id" : "122890",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122894",
"_score" : 1.0,
"_source" : {
"title" : "Avatar 2",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2016,
"id" : "122894",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122896",
"_score" : 1.0,
"_source" : {
"title" : "Pirates of the Caribbean: Dead Men Tell No Tales",
"genre" : [
"(no genres listed)"
],
"year" : 2017,
"id" : "122896",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122898",
"_score" : 1.0,
"_source" : {
"title" : "Justice League",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122898",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122904",
"_score" : 1.0,
"_source" : {
"title" : "Deadpool",
"genre" : [
"Action",
"Adventure",
"Comedy",
"Sci-Fi"
],
"year" : 2016,
"id" : "122904",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122906",
"_score" : 1.0,
"_source" : {
"title" : "Black Panther",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122906",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122916",
"_score" : 1.0,
"_source" : {
"title" : "Thor: Ragnarok",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122916",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122918",
"_score" : 1.0,
"_source" : {
"title" : "Guardians of the Galaxy 2",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122918",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122920",
"_score" : 1.0,
"_source" : {
"title" : "Captain America: Civil War",
"genre" : [
"Action",
"Sci-Fi",
"Thriller"
],
"year" : 2016,
"id" : "122920",
"@version" : "1"
}
}
]
}
}
4.2.12 占位符搜索
#查询title中包含以 Min开头的字母的电影
GET movies/_search?q=title:Min*
#?代表一个字母
GET movies/_search?q=title:Min?
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 248,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32653",
"_score" : 1.0,
"_source" : {
"title" : "This Land Is Mine",
"genre" : [
"Drama",
"War"
],
"year" : 1943,
"id" : "32653",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32830",
"_score" : 1.0,
"_source" : {
"title" : "44 Minutes: The North Hollywood Shoot-Out",
"genre" : [
"Action",
"Crime"
],
"year" : 2003,
"id" : "32830",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "38824",
"_score" : 1.0,
"_source" : {
"title" : "Mother of Mine",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "38824",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "40962",
"_score" : 1.0,
"_source" : {
"title" : "Yours, Mine and Ours",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2005,
"id" : "40962",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "46115",
"_score" : 1.0,
"_source" : {
"title" : "Forever Mine",
"genre" : [
"Crime",
"Drama",
"Romance",
"Thriller"
],
"year" : 1999,
"id" : "46115",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "47404",
"_score" : 1.0,
"_source" : {
"title" : "Mind Game",
"genre" : [
"Adventure",
"Animation",
"Comedy",
"Fantasy",
"Romance",
"Sci-Fi"
],
"year" : 2004,
"id" : "47404",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "49371",
"_score" : 1.0,
"_source" : {
"title" : "Min and Bill",
"genre" : [
"Comedy",
"Drama"
],
"year" : 1930,
"id" : "49371",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "49642",
"_score" : 1.0,
"_source" : {
"title" : "Unaccompanied Minors",
"genre" : [
"Children",
"Comedy"
],
"year" : 2006,
"id" : "49642",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "53138",
"_score" : 1.0,
"_source" : {
"title" : "Librarian: Return to King Solomon's Mines, The",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2006,
"id" : "53138",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "53207",
"_score" : 1.0,
"_source" : {
"title" : "88 Minutes",
"genre" : [
"Crime",
"Drama",
"Mystery",
"Thriller"
],
"year" : 2008,
"id" : "53207",
"@version" : "1"
}
}
]
}
}
五. Analysis
analysis(只是一个概念)，文本分析是将全文本转换为一系列单词的过程，也叫分词。analysis是通 过analyzer(分词器)来实现的，可以使用Elasticsearch内置的分词器，也可以自己去定制一些分词 器。 除了在数据写入的时候进行分词处理，也会在查询的时候也可以使用分析器对查询语句进行分词。 anaylzer是由三部分组成，例如有：
<p>Hello a World, the world is beautifu</p>
- Character Filter: 将文本中html标签剔除掉。
- Tokenizer: 按照规则进行分词，在英文中按照空格分词。
- Token Filter: 去掉stop world(停顿词，a, an, the, is, are等)，然后转换小写
5.1 内置分词器
| 分词器名称 | 处理过程 |
|---|---|
| Standard Analyzer | 默认的分词器，按词切分，小写处理 |
| Simple Analyzer | 按照非字母切分(符号被过滤)，小写处理 |
| Stop Analyzer | 小写处理，停用词过滤(the, a, this) |
| Whitespace Analyzer | 按照空格切分，不转小写 |
| Keyword Analyzer | 不分词，直接将输入当做输出 |
| Pattern Analyzer | 正则表达式，默认是\W+(非字符串分隔) |
5.2 内置分词器示例
5.2.1 Standard Analyzer
GET _analyze
{
"analyzer": "standard",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "<NUM>",
"position" : 0
},
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "<ALPHANUM>",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "<ALPHANUM>",
"position" : 2
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "<ALPHANUM>",
"position" : 3
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "<ALPHANUM>",
"position" : 4
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "<ALPHANUM>",
"position" : 5
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "<ALPHANUM>",
"position" : 6
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "<ALPHANUM>",
"position" : 7
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "<ALPHANUM>",
"position" : 8
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "<ALPHANUM>",
"position" : 9
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "<ALPHANUM>",
"position" : 10
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "<ALPHANUM>",
"position" : 11
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "<ALPHANUM>",
"position" : 12
}
]
}
5.2.2 Simple Analyzer
GET _analyze
{
"analyzer": "simple",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 0
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 1
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 2
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 8
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 9
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.3 Stop Analyzer
GET _analyze
{
"analyzer": "stop",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 0
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 1
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 2
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.4 Whitespace Analyzer
GET _analyze
{
"analyzer": "whitespace",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "word",
"position" : 0
},
{
"token" : "Running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 2
},
{
"token" : "brown-foxes",
"start_offset" : 16,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 8
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 9
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.5 Keyword Analyzer
GET _analyze
{
"analyzer": "keyword",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2 Running quick brown-foxes leap over lazy dog in the summer evening",
"start_offset" : 0,
"end_offset" : 68,
"type" : "word",
"position" : 0
}
]
}
5.2.6 Pattern Analyzer
GET _analyze
{
"analyzer": "pattern",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "word",
"position" : 0
},
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 2
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 3
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 4
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 5
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 6
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 7
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 8
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 9
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 10
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 11
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 12
}
]
}
