[es]elasticSearch常用的api
索引管理
- 查看es的全部索引
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
- 创建一个索引
curl -XPUT 'localhost:9200/customer?pretty&pretty'
- 创建一个索引，并指定shard个数和副本数
curl -X PUT -H "Content-Type:application/json" http://10.12.26.208:9200/op_test1 -d '
{
"settings": {
"index": {
"number_of_shards": 5,
"number_of_replicas": 1
}
}
}'
- 删除索引
curl -XDELETE 'localhost:9200/{索引名}?pretty&pretty'
- 索引起别名
curl -XPUT localhost:9200/索引名/_alias/别名
- 查看别名
curl -XPUT localhost:9200/_cat/aliases?v
- 查看es索引下的数据
curl -X GET http://localhost:9200/hadoop-impala-2021.04/_search?size=1000&scroll=1m&pretty=true'-d '{"size": 0,"query":{"query_string":{ "match_all" : {}}}}
- 查看单个索引的文档数
curl -X GET 'http://localhost:9200/_cat/count/索引?v&pretty'
- 查看索引映射类型
curl -X GET "localhost:9200/索引?pretty"
- 清空索引数据但不删除索引
curl -X POST 'localhost:9200/索引/_delete_by_query' -d '
{
"query": {
"match_all": {}
}
}'
- 迁移索引
curl -u elastic:elastic -X POST "10.11.3.63:9200/_reindex" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "souce_index"
},
"dest": {
"index": "dest_index"
}
}'
- 迁移shard
curl -H "Content-Type: application/json" -X POST http://10.11.3.63:39200/_cluster/reroute -d '
{
"commands": [
{
"move": {
"index": "test-rongapp_wf-2021.09.05",
"shard": 0,
"from_node": "dx-elk14.dx",
"to_node": "dx-elk16.dx"
}
}
]
}'
创建索引
- 创建一个索引
curl -X PUT "localhost:9200/commodity?pretty"
索引创建成功后，返回如下结果
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "commodity"
}
- 创建带有类型、映射的索引
curl -X PUT "localhost:9200/commodity?pretty" -d '
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 2
},
"mapping": {
"shop": {
"properties": {
"commodity_id": {
"type": "long"
},
"commodity_name": {
"type": "text"
},
"picture_url": {
"type": "keyword"
},
"price": {
"type": "double"
}
}
}
}
}'
新增mapping字段
ES不支持修改Mapping，但可以新增Mapping字段。
PUT http://host/<index_name>/_mapping/<type>
{
"properties":{
"last_time_used":{"type" : "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}
}
}
插入
curl -X PUT 'localhost:9200/commodity/shop/1' -d '
{
"commodity_id":1,
"commodity_name":"zhang san",
"picture_url":"zhang san li si zhang san wang wu",
"price":10.0
}'
查询
- 前缀查询
curl -XPOST http://localhost:9200/commodity/_search -d '
{
"query": {
"prefix": {
"picture_url": {
"value": "zhang san"
}
}
}
}
}'
- 条件查询es文档
案例一：
select * from impala where queryState = "FINISHED" and startTime > 1619070995;
结果
curl -XPOST http://localhost:9200/hadoop_impala-2021.04/_search -d '
{
"query": {
"bool": {
"must": [
{
"match": {
"queryState": "FINISHED"
}
},
{
"range": {
"startTime": {
"gt": '1619070995'
}
}
}
]
}
}
}'
案例二：
查询queryState 为 EXCEPTION 或者 durationMillis 大于20000 并且开始时间戳大于1619070995的数据。
select * from impala where (queryState = "FINISHED" or durationMillis > 20000) and startTime > 1619070995;
结果：
curl -XPOST http://localhost:9200/hadoop_impala-2021.04/_search -d '
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{"match": {"queryState": "EXECPTION"}},
{"range": {"durationMillis": {"gt": 20000}}}
]
}
},
{
"range": {
"startTime": {
"gt": 1619070995
}
}
}
]
}
}
}'
聚合查询
- count + group by
sql语句如下：
SELECT COUNT(user_id) FROM table GROUP BY user_id_type;
映射结果如下：
{
"aggs": {
"user_type": {
"terms": {
"field": "user_id_type"
}
}
}
}
segment
查看索引segment
GET /my-index-000001/_segments
查看节点segment
curl 'http://10.11.3.63:39200/_cat/nodes?v&h=name,segments.count,segments.memory,segments.index_writer_memory,segments.version_map_memory,segments.fixed_bitset_memory'
五、运维：
1. 集群状态
GET _cluster/health
- 查看es的健康状态
curl -XGET 'localhost:9200/_cat/health?v&pretty'
- 查看es的节点状态
curl -XGET 'localhost:9200/_cat/nodes?v&pretty'
2. hot_threads
GET /_nodes/hot_threads
GET /_nodes/{node}/hot_threads
3. nodes
GET /_cat/nodes
- 节点状态信息统计 API
统计节点的 jvm，http，IO 统计信息。
GET _nodes/stats?pretty
查看节点使用内存情况
curl -sXGET "http://10.11.3.63:39200/_cat/nodes?h=name,port,segments.memory,segments.index_writer_memory,segments.version_map_memory,segments.fixed_bitset_memory,fielddata.memory_size,query_cache.memory_size,request_cache.memory_size&v"
4. thread_pool
curl http://dx-selk00.dx:39200/_cat/thread_pool?v
5. segment
查看索引的segment
GET _cat/segments/hadoop_new_presto_2021-07?v&h=size,index,segment,size.memory,searchable,id
合并segment
# 期望合并到 1 个 segments
POST {index_name}/_forcemerge?max_num_segments=1
查看未分配的shard
GET /_cluster/allocation/explain
POST /_cluster/reroute?retry_failed=true
