ES数据迁移方案，-自己总结 基于REINDEX_API`

 

跨集群ES数据迁移:

1. 设置白名单在目标ES库（新库）
reindex.remote.whitelist: ["192.168.9.201:9200"]

[或者下面参数:
reindex.remote.whitelist: "10.*:*"
http.cors.enabled: true
http.cors.allow-origin: "*"
]
 
 
2.利用kibana修改新ES参数，防止刷新影响性能.

    PUT _settings?pretty
    {
      "index.refresh_interval": -1,
      "index.number_of_replicas": 0
    }

或者:
PUT http://172.16.1.236:9201/topic-new/_settings
{
  "refresh_interval": "-1",
  "number_of_replicas": 0
}

3.需要在目标ES库创建新索引,必须创建，否则无法指定字段属性.
 [如下案例1 ]reindex命令 如下是 更改索引字段的方式迁移，等于是给表改字段属性，需要删index再重建的方法.
POST _reindex
{
  "source": {
    "index": "old_index"
  },
  "dest": {
    "index": "new_index"
  }
}
 或者迁移命令：
POST http://172.16.1.236:9201/_reindex
{
  "source": {
    "index": "topic"
  },
  "dest": {
    "index": "topic-new"
  }
}

4. [如下案例2] 以下是跨集群迁移方案.
POST http://172.16.1.236:9201/_reindex
{
  "source": {
    "index": "topic",
    "remote": {
      "host": "http://172.16.1.236:9200",
      "username": "username",
      "password": "password"
    },
    "query": {
      "match_all": {}
    }
  },
  "dest": {
    "index": "topic-new"
  }
}

5. [如下案例3,增加参数]
POST _reindex
{
  "source": {
    "index": "old_index",
    "size":1000   //可选，每次批量提交1000个，可以提高效率，建议每次提交5-15M的数据
  },
  "dest": {
    "index": "new_index"
  }
}
 
6. 更新配置.[这时候去看数据,是看不到数据的,因为还要刷新才行]

PUT http://172.16.1.236:9201/topic-new/_settings
{
  "refresh_interval": "1s",
  "number_of_replicas": 1
}
 
 
curl -XPUT "http://localhost:9200/my_index/_doc/1" -H 'Content-Type: application/json' -d '
{
  "name": "John Doe",
  "age": 30,
  "city": "New York"
}'

7. 查看参数相关命令.
GET /_settings/

 #查看index数据，默认只能检索10条.
GET /dumpindex/_search


#查看所有数据.size为查询条数.
GET /dumpindex/_search
{
  "query": {
    "match_all": {}
  }
  ,
  "size": 1000
}


##################测试案例###############



#查询所有数据，默认还是10条,需要加条件
GET /dumpindex/_search
{
  "query": {
    "match_all": {}
  },
   "size": 5,   # 这个是增加的size条件
}

#指定size
http://localhost:9200/_search?q=test&size=100


#创建索引在新库中.
PUT dumpindex
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age": {
        "type": "integer"
      }
    }
  }
}
 
#在老库中添加数据.
 POST _bulk
   {"index":{"_index":"dumpindex"}}
 {"name": "qrqp", "age": 28}
 
  {"index":{"_index":"dumpindex"}}
 {"name": "gvuo", "age": 27}
  {"index":{"_index":"dumpindex"}}
 {"name": "kznv", "age": 29}
  {"index":{"_index":"dumpindex"}}
 {"name": "smcg", "age": 24}
  {"index":{"_index":"dumpindex"}}
 {"name": "myrg", "age": 25}
  {"index":{"_index":"dumpindex"}}
 {"name": "yzol", "age": 29}
  {"index":{"_index":"dumpindex"}}
 {"name": "swzx", "age": 26}
  {"index":{"_index":"dumpindex"}}
 {"name": "dtoo", "age": 22}
  {"index":{"_index":"dumpindex"}}
 {"name": "rguu", "age": 22}
  {"index":{"_index":"dumpindex"}}
 {"name": "sfod", "age": 24}
  {"index":{"_index":"dumpindex"}}
 {"name": "rdnc", "age": 29}
  {"index":{"_index":"dumpindex"}}
 {"name": "lzpm", "age": 27}
  {"index":{"_index":"dumpindex"}}
 {"name": "qtay", "age": 19}
  {"index":{"_index":"dumpindex"}}
 {"name": "utkw", "age": 24}