es部署
部署单机版es
tar xf elasticsearch-7.16.0-linux-x86_64.tar.gz
./bin/elasticsearch -d
can not run elasticsearch as root
su - machangwei
./bin/elasticsearch -d #切换用户后没有权限访问jdk目录
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
could not find java in JAVA_HOME at /opt/jdk/bin/java
ls: cannot open directory /opt/jdk/: Permission denied
退回到root
vim bin/elasticsearch-env
JAVA_HOME=/application/elasticsearch-7.16.0/jdk
chown -R machangwei.machangwei elasticsearch-7.16.0
su - machangwei
[machangwei@mcw1 ~]$ cd /application/elasticsearch-7.16.0/bin/
[machangwei@mcw1 /application/elasticsearch-7.16.0/bin]$ ./elasticsearch -d
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
只能本机访问：
[root@mcw1 /application]$ curl http://127.0.0.1:9200/
{
"name" : "mcw1",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "00voBHA3RUKvKy_Wl-VyEQ",
"version" : {
"number" : "7.16.0",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "6fc81662312141fe7691d7c1c91b8658ac17aa0d",
"build_date" : "2021-12-02T15:46:35.697268109Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
[root@mcw1 /application]$
[root@mcw1 /application]$ curl http://10.0.0.131:9200/
curl: (7) Failed connect to 10.0.0.131:9200; Connection refused
添加配置，重启es报错，
[machangwei@mcw1 /application/elasticsearch-7.16.0/config]$ grep network.host elasticsearch.yml
network.host: 10.0.0.131
====错误信息：
ERROR: [3] bootstrap checks failed. You must address the points described in the following [3] lines before starting Elasticsearch.
bootstrap check failure [1] of [3]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]
bootstrap check failure [2] of [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
bootstrap check failure [3] of [3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ERROR: Elasticsearch did not exit normally - check the logs at /application/elasticsearch-7.16.0/logs/elasticsearch.log
添加配置启动
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ vim config/elasticsearch.yml
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ tail config/elasticsearch.yml
# To protect your data, we strongly encourage you to enable the Elasticsearch security features.
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html
discovery.seed_hosts: ["10.0.0.131"]
cluster.initial_master_nodes: ["esmcw1"]
cluster.name: escluster
node.name: esmcw1
network.host: 0.0.0.0
http.port: 9200
启动后还是报错：最大线程数太少，最大虚拟内存太少
ERROR: [2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.
bootstrap check failure [1] of [2]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]
bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /application/elasticsearch-7.16.0/logs/escluster.log ls^C
[root@mcw1 /application]$ tail /application/elasticsearch-7.16.0/logs/escluster.log
[2021-12-22T09:26:13,015][ERROR][o.e.b.Bootstrap ] [esmcw1] node validation exception
[2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.
bootstrap check failure [1] of [2]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]
bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[root@mcw1 /application]$ tail -1 /etc/sysctl.conf #添加系统配置vm.max_map_count，解决了虚拟内存问题，但是还有报错
vm.max_map_count=655360
==还报错：
[2021-12-22T09:41:48,876][ERROR][o.e.b.Bootstrap ] [esmcw1] node validation exception
[1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
bootstrap check failure [1] of [1]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]
设置/etc/security/limits.conf
* soft nproc 5000
* hard nproc 5000
root soft nproc 5000
root hard nproc 5000
重启服务器生效.应该有不需要重启服务器的方法，回头研究
这次再启动就不报错退出进程了。并且可以使用这个ip进行访问了
[root@mcw1 ~]$ curl http://10.0.0.131:9200/
{
"name" : "esmcw1",
"cluster_name" : "escluster",
"cluster_uuid" : "00voBHA3RUKvKy_Wl-VyEQ",
"version" : {
"number" : "7.16.0",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "6fc81662312141fe7691d7c1c91b8658ac17aa0d",
"build_date" : "2021-12-02T15:46:35.697268109Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
浏览器也可以访问了
安装访问客户端工具
谷歌添加插件
添加es-head插件：连接es访问：
命令行访问es集群状况
curl -XGET -H "Content-Type: application/json" http://10.0.0.131:9200/_cat/health?v
python调用es
查询出来刚刚插入的数据
Elasticsearch 安全配置
vi elasticsearch.yml
添加配置:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
生成证书文件
生成证书文件，执行以下指令，将生成的文件复制到 elasticsearch-7.16.0/config/certs目录下. 这个不需要es运行即可以操作
cd elasticsearch-7.16.0
bin/elasticsearch-certutil ca
输入证书文件名：elastic-certificates.p12
输入密码：mcw_es_passwd123
mkdir -p config/certs
cp elastic-certificates.p12 config/certs/elastic-certificates.p12
3.将文件elastic-certificates.p12 提交到其他节点elasticsearch-7.16.0/config/certs下。在其他节点执行以下指令，设置生成p12文件时输入的密码.
bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
输入密码：mcw_es_passwd123
bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
输入密码：mcw_es_passwd123
4.启动各个节点的es。
cd elasticsearch-7.16.0/bin
elasticsearch -d
5.添加访问账户和密码,执行指令, 需要为4个用户分别设置密码，
默认用户设置密码
elasticmcw_es_passwd123
kibana,mcw_es_passwd123
logstash_system,mcw_es_passwd123
beats_systemmcw_es_passwd123
cd elasticsearch-7.16.0
./bin/elasticsearch-setup-passwords interactive
//输入密码
//输入密码
…
总是报错，重试了很多次生成证书文件，后来没报错，启动成功后，设置密码也可以了
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ ./bin/elasticsearch-setup-passwords interactive
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
带用户密码的curl命令查看集群状态
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ curl -XGET -u elastic:mcw_es_passwd123 -H "Content-Type: application/json" http://10.0.0.131:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1640213613 22:53:33 escluster yellow 1 1 5 5 0 0 1 0 - 83.3%
带用户密码的浏览器和es-head访问：
带密码的python访问：
from elasticsearch import Elasticsearch es = Elasticsearch(['10.0.0.131:9200'],http_auth=('elastic', 'mcw_es_passwd123')) #print(es.index(index='mcw', doc_type='doc', id='1', body={"name":"小马过河", "age": 18})) print(es.search(index='mcw', doc_type='doc'))
方法二：
from elasticsearch import Elasticsearch
es = Elasticsearch(['10.0.0.131'],http_auth=('elastic', 'mcw_es_passwd123'), port=9200, timeout=50000)
#print(es.index(index='mcw', doc_type='doc', id='1', body={"name":"小马过河", "age": 18}))
print(es.search(index='mcw', doc_type='doc'))
问题
刚刚配置了xpack没有重启
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ ./bin/elasticsearch-setup-passwords interactive
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Unexpected response code [500] from calling GET http://10.0.0.131:9200/_security/_authenticate?pretty
It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.
Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.
ERROR: X-Pack Security is disabled by configuration.
Caused by: java.io.IOException: keystore password was incorrect
ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize SSL TrustManager]; nested: IOException[keystore password was incorrect]; nested: UnrecoverableKeyException[failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.];
Likely root cause: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
部署单机版estar xf elasticsearch-7.16.0-linux-x86_64.tar.gz./bin/elasticsearch -dcan not run elasticsearch as root
su - machangwei./bin/elasticsearch -d #切换用户后没有权限访问jdk目录warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOMEcould not find java in JAVA_HOME at /opt/jdk/bin/java
ls: cannot open directory /opt/jdk/: Permission denied
退回到rootvim bin/elasticsearch-envJAVA_HOME=/application/elasticsearch-7.16.0/jdkchown -R machangwei.machangwei elasticsearch-7.16.0su - machangwei
[machangwei@mcw1 ~]$ cd /application/elasticsearch-7.16.0/bin/[machangwei@mcw1 /application/elasticsearch-7.16.0/bin]$ ./elasticsearch -dwarning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOMEwarning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
只能本机访问：[root@mcw1 /application]$ curl http://127.0.0.1:9200/{ "name" : "mcw1", "cluster_name" : "elasticsearch", "cluster_uuid" : "00voBHA3RUKvKy_Wl-VyEQ", "version" : { "number" : "7.16.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "6fc81662312141fe7691d7c1c91b8658ac17aa0d", "build_date" : "2021-12-02T15:46:35.697268109Z", "build_snapshot" : false, "lucene_version" : "8.10.1", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search"}[root@mcw1 /application]$
[root@mcw1 /application]$ curl http://10.0.0.131:9200/curl: (7) Failed connect to 10.0.0.131:9200; Connection refused
添加配置，重启es报错，[machangwei@mcw1 /application/elasticsearch-7.16.0/config]$ grep network.host elasticsearch.yml network.host: 10.0.0.131====错误信息：ERROR: [3] bootstrap checks failed. You must address the points described in the following [3] lines before starting Elasticsearch.bootstrap check failure [1] of [3]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]bootstrap check failure [2] of [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]bootstrap check failure [3] of [3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configuredERROR: Elasticsearch did not exit normally - check the logs at /application/elasticsearch-7.16.0/logs/elasticsearch.log
添加配置启动[machangwei@mcw1 /application/elasticsearch-7.16.0]$ vim config/elasticsearch.yml [machangwei@mcw1 /application/elasticsearch-7.16.0]$ tail config/elasticsearch.yml# To protect your data, we strongly encourage you to enable the Elasticsearch security features. # Refer to the following documentation for instructions.## https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.htmldiscovery.seed_hosts: ["10.0.0.131"]cluster.initial_master_nodes: ["esmcw1"]cluster.name: esclusternode.name: esmcw1network.host: 0.0.0.0http.port: 9200
启动后还是报错：最大线程数太少，最大虚拟内存太少ERROR: [2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.bootstrap check failure [1] of [2]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]ERROR: Elasticsearch did not exit normally - check the logs at /application/elasticsearch-7.16.0/logs/escluster.log ls^C[root@mcw1 /application]$ tail /application/elasticsearch-7.16.0/logs/escluster.log [2021-12-22T09:26:13,015][ERROR][o.e.b.Bootstrap ] [esmcw1] node validation exception[2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.bootstrap check failure [1] of [2]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[root@mcw1 /application]$ tail -1 /etc/sysctl.conf #添加系统配置vm.max_map_count，解决了虚拟内存问题，但是还有报错vm.max_map_count=655360==还报错：[2021-12-22T09:41:48,876][ERROR][o.e.b.Bootstrap ] [esmcw1] node validation exception[1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.bootstrap check failure [1] of [1]: max number of threads [3827] for user [machangwei] is too low, increase to at least [4096]
设置/etc/security/limits.conf
* soft nproc 5000* hard nproc 5000root soft nproc 5000root hard nproc 5000重启服务器生效.应该有不需要重启服务器的方法，回头研究
这次再启动就不报错退出进程了。并且可以使用这个ip进行访问了[root@mcw1 ~]$ curl http://10.0.0.131:9200/{ "name" : "esmcw1", "cluster_name" : "escluster", "cluster_uuid" : "00voBHA3RUKvKy_Wl-VyEQ", "version" : { "number" : "7.16.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "6fc81662312141fe7691d7c1c91b8658ac17aa0d", "build_date" : "2021-12-02T15:46:35.697268109Z", "build_snapshot" : false, "lucene_version" : "8.10.1", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search"}
=========== Elasticsearch 安全配置 vi elasticsearch.yml添加配置:
xpack.security.enabled: truexpack.security.transport.ssl.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
生成证书文件，执行以下指令，将生成的文件复制到 elasticsearch-7.16.0/config/certs目录下. 这个不需要es运行即可以操作cd elasticsearch-7.16.0bin/elasticsearch-certutil ca 输入证书文件名：elastic-certificates.p12输入密码：mcw_es_passwd123mkdir -p config/certs cp elastic-certificates.p12 config/certs/elastic-certificates.p123.将文件elastic-certificates.p12 提交到其他节点elasticsearch-7.16.0/config/certs下。在其他节点执行以下指令，设置生成p12文件时输入的密码.bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password输入密码：mcw_es_passwd123bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password输入密码：mcw_es_passwd123 4.启动各个节点的es。cd elasticsearch-7.16.0/binelasticsearch -d5.添加访问账户和密码,执行指令, 需要为4个用户分别设置密码，
默认用户设置密码elasticmcw_es_passwd123kibana,mcw_es_passwd123logstash_system,mcw_es_passwd123beats_systemmcw_es_passwd123
cd elasticsearch-7.16.0./bin/elasticsearch-setup-passwords interactive//输入密码//输入密码 …
总是报错，重试了很多次生成证书文件，后来没报错，启动成功后，设置密码也可以了
[machangwei@mcw1 /application/elasticsearch-7.16.0]$ ./bin/elasticsearch-setup-passwords interactivewarning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOMEInitiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.You will be prompted to enter passwords as the process progresses.Please confirm that you would like to continue [y/N]y
Enter password for [elastic]: Reenter password for [elastic]: Enter password for [apm_system]: Reenter password for [apm_system]: Enter password for [kibana_system]: Reenter password for [kibana_system]: Enter password for [logstash_system]: Reenter password for [logstash_system]: Enter password for [beats_system]: Reenter password for [beats_system]: Enter password for [remote_monitoring_user]: Reenter password for [remote_monitoring_user]: Changed password for user [apm_system]Changed password for user [kibana_system]Changed password for user [kibana]Changed password for user [logstash_system]Changed password for user [beats_system]Changed password for user [remote_monitoring_user]Changed password for user [elastic]
curl命令查看集群状态
刚刚配置了xpack没有重启[machangwei@mcw1 /application/elasticsearch-7.16.0]$ ./bin/elasticsearch-setup-passwords interactivewarning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Unexpected response code [500] from calling GET http://10.0.0.131:9200/_security/_authenticate?prettyIt doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.
ERROR: X-Pack Security is disabled by configuration.
Caused by: java.io.IOException: keystore password was incorrect
ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize SSL TrustManager]; nested: IOException[keystore password was incorrect]; nested: UnrecoverableKeyException[failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.];Likely root cause: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
