Elasticsearch8 单机以及集群部署
本文以 Elasticsearch8.8.0 为例，介绍一下单机安装 ES ,使用 docker-compose 方式进行便捷管理
需要准本docker 环境：一键安装脚本 https://www.cnblogs.com/chxlay/p/15433473.html
需要准备 docker-compose 环境：一键安装脚本 https://www.cnblogs.com/chxlay/p/15433907.html
关于证书自签请查阅： https://www.cnblogs.com/chxlay/p/17477191.html
官网阅读参考
**** 以下所有命名 均为 my-... 实际使用中，请自行全局替换修改，如：my-es ，myes 等，自行修改为自己的项目名称
开始正题：
一、介绍：
1、以下部署分三种模式，单机节点部署，双节点部署，多节点集群部署， 其中单节点，双节点 均由多节点 模式改造而来， 三种模式均 包含 了 kibana 的部署，三种模式均已经测试验证，
2、三种模式中，第一个服务 my-es-setup是作为自动签发证书相关服务，如果自己通过其他方式已经获得证书，可以不需要启动 my-es-setup 这个服务，或者非首次启动也是，已经持有相关证书文件了，可以不启动 me-es-setup 服务，可将 es 服务中 depends_on 中对 my-es-setup的依赖去除，或注释
depends_on: my-es-setup: condition: service_healthy
关于怎样使用自己的证书，而不使用默认证书，请参考 elastic社区文章
3、部署环境，使用的是 阿里云 ecs 单服务器 8GB 内存 进行的测试部署，三节点服务器要求，单服务 8GB 内存以上才能流畅的测试，否则会在此过程中，部分节点 停机，导致集群不能全部启动
4、环境说明，本案例使用的 阿里云 ECS 非 root 用户，出于安全考虑，购买时直接设置了 ecs-user 用户，所以省去了 新创建 非 root 用户的麻烦，如果是 ECS 使用者也推荐购买时直接选择 ecs-user（非root 用户），虚拟机用户请留意数据卷 volumes 挂载相关的路径权限问题，往下会有具体说明
二、常见问题：
1、非root用户 docker 命令问题
docker 非 root 用户 无法访问 docker 命令问题，请执行如下命令，并重起虚拟机，其中命令 $USER 代表读取当前用户，或者 声明指定 比如 改为声明用户 ecs-user
sudo usermod -aG docker $USER
2、非 root 用户常见权限问题：
如： 给 用户 ecs-user 授权 路径 /home/appdata
sudo chown -R ecs-user /home/appdata
sudo chmod -R 775 /home/appdata
或者也可以将其编写成执行脚本：
$1 为接收执行时输入的参数，也就是授权的路径
:docker 是可选的，虚拟机的话建议不要加 :docker虚拟机和自己命令创建的用户建议不要增加 :docker 了，减少麻烦，我加 :docker 是因为我的系统是 阿里云 ECS ,购买时选择的就是 ecs-user 用户，
使用脚本的好处就是后续赋权操作都比较方便
#!/bin/bash # 文件路径授权 ecs-user,虚拟机和自己命令创建的用户不建议增加 :docker 了，减少麻烦，我加 :docker 是因为我的系统是 阿里云 ECS ,购买时选择的就是 ecs-user 用户 sudo chown -R ecs-user:docker $1 sudo chmod -R 775 $1
记得给脚本文件赋权 chmod +x 文件名.sh
使用：脚本路径及命名 /opt/auth.sh ，测试赋权给 路径 /home/appdata
/opt/auth.sh /home/appdata
3、报错：Error: Could not create the Java Virtual Machine.
基本是挂载文件的权限问题,没有权限,比如日志，数据文件挂在路径没有权限，由于 Elasticsearch 禁止使用 非 root 权限用户执行，
使用上一步的方式给相关的挂载路径赋予权限即可，按照上一步执行给容器挂载的路径授权即可：比如我的用户为 ecs-user ，/home/appdata 是我挂载数据的根路径
sudo chown -R ecs-user /home/appdata
sudo chmod -R 775 /home/appdata
kibana 启动是报错：
FATAL Error: Unable to write to UUID file at /usr/share/kibana/data/uuid. Ensure Kibana has sufficient permissions to read / write to this file. Error was: EACCES
问题原因也是数据挂载路径权限的问题，只需要使用以上步骤，多指定的路径给用赋予权限即可
4、报错 vm.max_map_count [65530] is too low
elasticsearch用户拥有的内存权限太小，至少需要262144, 执行；如下配置，启动完之后，可以再进行还原回来，后续都不影响正常安装和启动的
sudo sysctl -w vm.max_map_count=262144
或者直接修改文件 /etc/sysctl.conf
# 执行命令修改文件 sudo vim /etc/sysctl.conf
配置文件中追加一个配置 vm.max_map_count=262144
# 追加配置文件 vm.max_map_count=262144
使之生效，sudo system -p
# 使之生效 sudo sysctl -p
5、报错： memory locking requested for elasticsearch process but memory is not locked
请在 config/elasticsearch.yml 中 修改 bootstrap.memory_lock 值为 false ，默认值 true
# ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: 内存锁定 # bootstrap.memory_lock: false
以上是我测试中遇到的问题，其他问题没有发生，所以无法给出提示
三、环境变量文件准备
.evn 环境变量文件
BASE_DIR=/home/appdata # Password for the 'elastic' user (at least 6 characters) elastic 用户名的密码 ELASTIC_PASSWORD=myes_7j1TEQyVyoVLJ5G4SXM3NcH6Z # Password for the 'kibana_system' user (at least 6 characters) kibana 密码 KIBANA_PASSWORD=myes_ZFzBxCF1Hrz5Gp5UAElBLnNFS # ES 8.x 的版本标识（可根据个人项目需求修改） STACK_VERSION=8.8.0 # Set the cluster name CLUSTER_NAME=my-es-cluster # Set to 'basic' or 'trial' to automatically start the 30-day trial 设置为 “基本” 或 “试用” 以自动开始30天的试用 #LICENSE=trial LICENSE=basic # Port to expose Elasticsearch HTTP API to the host ES_PORT=9200 #ES_PORT=127.0.0.1:9200 # 集群间内部通讯 TRANSPORT_PORT=9300 # Port to expose Kibana to the host KIBANA_PORT=5601 #KIBANA_PORT=80 # Increase or decrease based on the available host memory (in bytes) # 1GB MEM_LIMIT=1073741824 # Project namespace (defaults to the current folder name if not set)项目命名空间 (如果未设置，默认为当前文件夹名称) COMPOSE_PROJECT_NAME=my-es
四、部署
1、单机节点部署
docker-compose.yml 文件编写，为了文件名不冲突，我其名为 docker-compose-standlone.yml
version: '3.8' services: # my-es-setup 初始化创建 签名证书等相关的 my-es-setup: env_file: - .env image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-setup volumes: - '${BASE_DIR}/elastic/certs:/usr/share/elasticsearch/config/certs' user: "0" command: > bash -c ' if [ x${ELASTIC_PASSWORD} == x ]; then echo "Set the ELASTIC_PASSWORD environment variable in the .env file"; exit 1; elif [ x${KIBANA_PASSWORD} == x ]; then echo "Set the KIBANA_PASSWORD environment variable in the .env file"; exit 1; fi; if [ ! -f config/certs/ca.zip ]; then echo "Creating CA"; bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip; unzip config/certs/ca.zip -d config/certs; fi; if [ ! -f config/certs/certs.zip ]; then echo "Creating certs"; echo -ne \ "instances:\n"\ " - name: my-es\n"\ " dns:\n"\ " - my-es\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ > config/certs/instances.yml; bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key; unzip config/certs/certs.zip -d config/certs; fi; echo "Setting file permissions" chown -R root:root config/certs; find . -type d -exec chmod 750 \{\} \;; find . -type f -exec chmod 640 \{\} \;; echo "Waiting for Elasticsearch availability"; until curl -s --cacert config/certs/ca/ca.crt https://my-es:9200 | grep -q "missing authentication credentials"; do sleep 30; done; echo "Setting kibana_system password"; until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://my-es:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done; echo "All done!"; ' healthcheck: test: ["CMD-SHELL", "[ -f config/certs/my-es/my-es.crt ]"] interval: 1s timeout: 5s retries: 120 networks: - my-network my-es: env_file: - .env image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es volumes: - '${BASE_DIR}/elastic/certs:/usr/share/elasticsearch/config/certs' - '${BASE_DIR}/elasticsearch/plugins:/usr/share/elasticsearch/plugins' - '${BASE_DIR}/elasticsearch/data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/logs:/usr/share/elasticsearch/logs' ports: - ${ES_PORT}:9200 - ${TRANSPORT_PORT}:9300 environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es - discovery.seed_hosts=my-es - ELASTIC_PASSWORD=${ELASTIC_PASSWORD} - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es/my-es.key - xpack.security.http.ssl.certificate=certs/my-es/my-es.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es/my-es.key - xpack.security.transport.ssl.certificate=certs/my-es/my-es.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-kibana: env_file: - .env depends_on: my-es: condition: service_healthy image: docker.elastic.co/kibana/kibana:${STACK_VERSION} container_name: my-kibana volumes: - '${BASE_DIR}/elastic/certs:/usr/share/kibana/config/certs' - '${BASE_DIR}/kibana/data:/usr/share/kibana/data' ports: - ${KIBANA_PORT}:5601 environment: - SERVERNAME=kibana - ELASTICSEARCH_HOSTS=https://my-es:9200 - ELASTICSEARCH_USERNAME=kibana_system - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD} - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt deploy: resources: limits: memory: ${MEM_LIMIT} healthcheck: test: [ "CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network # 自定义网桥 my networks: my-network: # 启动时不自动创建网桥,需要提前手动创建 网桥 docker network create -d bridge my-network external: true driver: bridge
2、双节点模式部署
一个 master node 一个 work node
docker-compose.yml 文件，我起名为 docker-compose-2node.yml
version: '3.8' services: my-es-setup: env_file: - .env image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-setup volumes: - certs:/usr/share/elasticsearch/config/certs user: "0" command: > bash -c ' if [ x${ELASTIC_PASSWORD} == x ]; then echo "Set the ELASTIC_PASSWORD environment variable in the .env file"; exit 1; elif [ x${KIBANA_PASSWORD} == x ]; then echo "Set the KIBANA_PASSWORD environment variable in the .env file"; exit 1; fi; if [ ! -f config/certs/ca.zip ]; then echo "Creating CA"; bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip; unzip config/certs/ca.zip -d config/certs; fi; if [ ! -f config/certs/certs.zip ]; then echo "Creating certs"; echo -ne \ "instances:\n"\ " - name: my-es-master\n"\ " dns:\n"\ " - my-es-master\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ " - name: my-es-node1\n"\ " dns:\n"\ " - my-es-node1\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ > config/certs/instances.yml; bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key; unzip config/certs/certs.zip -d config/certs; fi; echo "Setting file permissions" chown -R root:root config/certs; find . -type d -exec chmod 750 \{\} \;; find . -type f -exec chmod 640 \{\} \;; echo "Waiting for Elasticsearch availability"; until curl -s --cacert config/certs/ca/ca.crt https://my-es-master:9200 | grep -q "missing authentication credentials"; do sleep 30; done; echo "Setting kibana_system password"; until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://my-es-master:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done; echo "All done!"; ' healthcheck: test: [ "CMD-SHELL", "[ -f config/certs/my-es-master/my-es-master.crt ]" ] interval: 1s timeout: 5s retries: 120 networks: - my-network my-es-master: env_file: - .env depends_on: my-es-setup: condition: service_healthy image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-master volumes: - certs:/usr/share/elasticsearch/config/certs - 'pluginis:/usr/share/elasticsearch/plugins' - '${BASE_DIR}/elasticsearch/master-data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/master-logs:/usr/share/elasticsearch/logs' ports: - ${ES_PORT}:9200 - ${TRANSPORT_PORT}:9300 environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es-master - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es-master,my-es-node1 - discovery.seed_hosts=my-es-node1 - ELASTIC_PASSWORD=${ELASTIC_PASSWORD} - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es-master/my-es-master.key - xpack.security.http.ssl.certificate=certs/my-es-master/my-es-master.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es-master/my-es-master.key - xpack.security.transport.ssl.certificate=certs/my-es-master/my-es-master.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-es-node1: env_file: - .env depends_on: - my-es-master image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-node1 volumes: - 'certs:/usr/share/elasticsearch/config/certs' - 'pluginis:/usr/share/elasticsearch/plugins' - '${BASE_DIR}/elasticsearch/node1-data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/node1-logs:/usr/share/elasticsearch/logs' environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es-node1 - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es-master,my-es-node1 - discovery.seed_hosts=my-es-master - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es-node1/my-es-node1.key - xpack.security.http.ssl.certificate=certs/my-es-node1/my-es-node1.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es-node1/my-es-node1.key - xpack.security.transport.ssl.certificate=certs/my-es-node1/my-es-node1.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-kibana: env_file: - .env depends_on: my-es-master: condition: service_healthy my-es-node1: condition: service_healthy image: docker.elastic.co/kibana/kibana:${STACK_VERSION} container_name: my-kibana volumes: - certs:/usr/share/kibana/config/certs - '${BASE_DIR}/elasticsearch/kibana/data:/usr/share/kibana/data' ports: - ${KIBANA_PORT}:5601 environment: - SERVERNAME=kibana - ELASTICSEARCH_HOSTS=https://my-es-master:9200 - ELASTICSEARCH_USERNAME=kibana_system - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD} - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt deploy: resources: limits: memory: ${MEM_LIMIT} healthcheck: test: [ "CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network # 自定义网桥 my-network networks: my-network: # 启动时不自动创建网桥,需要提前手动创建 网桥 docker network create -d bridge my-network external: true driver: bridge # https://www.w3cschool.cn/doc_docker_1_11/docker_1_11-engine-reference-commandline-volume_create-index.html # 创建的 volume 将存储到 /var/lib/docker/volumes/ 路径下 volumes: # CA 证书 挂载 certs: driver: local # 插件挂载 pluginis: driver: local
3、集群模式部署
以下示例以 一个 master 两个 work node 为例，实际中，有多个请自行修改扩展（复制修改）
docker-compose.yml 文件，为了文件名不冲突，我起名为 docker-compose-cluster.yml
version: '3.8' services: my-es-setup: env_file: - .env image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-setup volumes: - certs:/usr/share/elasticsearch/config/certs user: "0" command: > bash -c ' if [ x${ELASTIC_PASSWORD} == x ]; then echo "Set the ELASTIC_PASSWORD environment variable in the .env file"; exit 1; elif [ x${KIBANA_PASSWORD} == x ]; then echo "Set the KIBANA_PASSWORD environment variable in the .env file"; exit 1; fi; if [ ! -f config/certs/ca.zip ]; then echo "Creating CA"; bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip; unzip config/certs/ca.zip -d config/certs; fi; if [ ! -f config/certs/certs.zip ]; then echo "Creating certs"; echo -ne \ "instances:\n"\ " - name: my-es-master\n"\ " dns:\n"\ " - my-es-master\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ " - name: my-es-node1\n"\ " dns:\n"\ " - my-es-node1\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ " - name: my-es-node2\n"\ " dns:\n"\ " - my-es-node2\n"\ " - localhost\n"\ " ip:\n"\ " - 127.0.0.1\n"\ > config/certs/instances.yml; bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key; unzip config/certs/certs.zip -d config/certs; fi; echo "Setting file permissions" chown -R root:root config/certs; find . -type d -exec chmod 750 \{\} \;; find . -type f -exec chmod 640 \{\} \;; echo "Waiting for Elasticsearch availability"; until curl -s --cacert config/certs/ca/ca.crt https://my-es-master:9200 | grep -q "missing authentication credentials"; do sleep 30; done; echo "Setting kibana_system password"; until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://my-es-master:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done; echo "All done!"; ' healthcheck: test: [ "CMD-SHELL", "[ -f config/certs/my-es-master/my-es-master.crt ]" ] interval: 1s timeout: 5s retries: 120 networks: - my-network my-es-master: env_file: - .env depends_on: my-es-setup: condition: service_healthy image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-master volumes: - certs:/usr/share/elasticsearch/config/certs - '${BASE_DIR}/elasticsearch/master-data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/master-logs:/usr/share/elasticsearch/logs' ports: - ${ES_PORT}:9200 - ${TRANSPORT_PORT}:9300 environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es-master - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es-master,my-es-node1,my-es-node2 - discovery.seed_hosts=my-es-node1,my-es-node2 - ELASTIC_PASSWORD=${ELASTIC_PASSWORD} - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es-master/my-es-master.key - xpack.security.http.ssl.certificate=certs/my-es-master/my-es-master.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es-master/my-es-master.key - xpack.security.transport.ssl.certificate=certs/my-es-master/my-es-master.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-es-node1: env_file: - .env depends_on: - my-es-master image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-node1 volumes: - certs:/usr/share/elasticsearch/config/certs - 'pluginis:/usr/share/elasticsearch/plugins' - '${BASE_DIR}/elasticsearch/node1-data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/node1-logs:/usr/share/elasticsearch/logs' environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es-node1 - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es-master,my-es-node1,my-es-node2 - discovery.seed_hosts=my-es-master,my-es-node2 - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es-node1/my-es-node1.key - xpack.security.http.ssl.certificate=certs/my-es-node1/my-es-node1.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es-node1/my-es-node1.key - xpack.security.transport.ssl.certificate=certs/my-es-node1/my-es-node1.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-es-node2: env_file: - .env depends_on: - my-es-node1 image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION} container_name: my-es-node2 volumes: - certs:/usr/share/elasticsearch/config/certs - 'pluginis:/usr/share/elasticsearch/plugins' - '${BASE_DIR}/elasticsearch/node2-data:/usr/share/elasticsearch/data' - '${BASE_DIR}/elasticsearch/node2-logs:/usr/share/elasticsearch/logs' environment: - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - node.name=my-es-node2 - cluster.name=${CLUSTER_NAME} - cluster.initial_master_nodes=my-es-master,my-es-node1,my-es-node2 - discovery.seed_hosts=my-es-master,my-es-node1 - bootstrap.memory_lock=true - xpack.security.enabled=true - xpack.security.http.ssl.enabled=true - xpack.security.http.ssl.key=certs/my-es-node2/my-es-node2.key - xpack.security.http.ssl.certificate=certs/my-es-node2/my-es-node2.crt - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.enabled=true - xpack.security.transport.ssl.key=certs/my-es-node2/my-es-node2.key - xpack.security.transport.ssl.certificate=certs/my-es-node2/my-es-node2.crt - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt - xpack.security.transport.ssl.verification_mode=certificate - xpack.license.self_generated.type=${LICENSE} deploy: resources: limits: memory: ${MEM_LIMIT} # 句柄数配置 ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 healthcheck: test: [ "CMD-SHELL", "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network my-kibana: env_file: - .env depends_on: my-es-master: condition: service_healthy my-es-node1: condition: service_healthy my-es-node2: condition: service_healthy image: docker.elastic.co/kibana/kibana:${STACK_VERSION} container_name: my-kibana volumes: - certs:/usr/share/kibana/config/certs - '${BASE_DIR}/elasticsearch/kibana/data:/usr/share/kibana/data' ports: - ${KIBANA_PORT}:5601 environment: - SERVERNAME=kibana - ELASTICSEARCH_HOSTS=https://my-es-master:9200 - ELASTICSEARCH_USERNAME=kibana_system - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD} - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt deploy: resources: limits: memory: ${MEM_LIMIT} healthcheck: test: [ "CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'", ] interval: 10s timeout: 10s retries: 120 networks: - my-network # 自定义网桥 my networks: my-network: # 启动时不自动创建网桥,需要提前手动创建 网桥 docker network create -d bridge my-network external: true driver: bridge # https://www.w3cschool.cn/doc_docker_1_11/docker_1_11-engine-reference-commandline-volume_create-index.html # 创建的 volume 将存储到 /var/lib/docker/volumes/ 路径下 volumes: # 插件挂载 pluginis: driver: local # CA 证书 挂载 certs: driver: local # 声明指令的卷名，compose会自动创建卷名 project_tomcat_volume01;project 为docker-compose所在的目录的名称， # docker volume create certs 提前手动创建定义的数据卷,docker volume create /home/appdata/certs #external: true # master-data: # driver: local # master-logs: # driver: local # # node1-data: # driver: local # node1-logs: # driver: local # # node2-data: # driver: local # node2-logs: # driver: local # # kibana-data: # driver: local
4、Linux 物理机部署(单节点)
首先下载 elasticsearch 安装文件压缩包，解压，移动到自己设定的 linux 路径下，作为软件安装路径(以下演示中，/opt/elasticsearch 为工作目录 )，需要准本号证书文件，证书签发步骤从以上 容器启动的方式中，my-es-setup 中可以看到，先配置一个 instances.yml 文件，然后通过 bin/elasticsearch-certutil 进行生成，这里就不赘述了，参考容器中的方式，前置条件
1、配置 elasticsearch.yml 配置文件：
# ---------------------------------- Cluster ----------------------------------- # 配置您的句群名称 cluster.name: my-es-cluster # ------------------------------------ Node ------------------------------------ # 当前节点名称 node.name: my-es # ----------------------------------- Paths ------------------------------------ # 数据存储的路径，根据自己需求配置 path.data: /mnt/appdata/elasticsearch/data # # 日志文件存储路径，根据自己需求配置 path.logs: /mnt/appdata/elasticsearch/logs # ----------------------------------- Memory ----------------------------------- # 自定义内存jvm 堆大小时需要开启,内存锁定(个人按需配置) bootstrap.memory_lock: false # ---------------------------------- Network ----------------------------------- # 网络配置，这里我为了测试方便，0.0.0.0 为开放所有IP,云服务中，通常配置为 内网IP地址，非弹性IP network.host: 192.168.0.100 # 这个默认值就行，如果没有特殊需求的话 # http.port: 9200 # --------------------------------- Discovery ---------------------------------- # 默认值 ["127.0.0.1", "[::1]"] # 如果单节点的话，将这个内容清空 discovery.seed_hosts: [] # --------------------------------- 其他自定义的配置---------------------------------- # 数据迁移 reindex reindex.remote.whitelist: ["192.168.0.103:9200"] # 日志级别 WARN / ERROR，首次启动，最好使用 INFO,启动后控制台会打印 密码 logger.level: WARN # 启动安全功能 xpack.security.enabled: true xpack.security.enrollment.enabled: true # 为 HTTP API 客户端连接（例如 Kibana、Logstash 和 Agents）启用加密 # 证书配置千万注意，证书文集必须放在 config/下的路径，如： config/certs/my-es.crt ； 否则无法启动 xpack.security.http.ssl: enabled: true key: /opt/elasticsearch/config/certs/my-es.key certificate: /opt/elasticsearch/config/certs/my-es.crt certificate_authorities: /opt/elasticsearch/config/certs/ca.crt verification_mode: certificate # 启用集群节点之间的加密和相互认证 xpack.security.transport.ssl: enabled: true key: /opt/elasticsearch/config/certs/my-es.key certificate: /opt/elasticsearch/config/certs/my-es.crt certificate_authorities: /opt/elasticsearch/config/certs/ca.crt verification_mode: certificate
常遇到的问题，和 Docker 启动的差不多，无非就是 权限、堆空间等，
启动，进到 elasticsearch 工作目录下的 bin 路径下
./elasticsearch
配置开机启动
创建自启动文件 /etc/systemd/system/elasticsearch.service ； 记得一下示例中， User 修改为自己真实的用户名
[Unit] # 服务的简短描述，用于帮助用户理解服务的功能 Description=you know for search [Service] ExecStart=/opt/elasticsearch/bin/elasticsearch Restart=always User=非root 用户的用户名 [Install] WantedBy=multi-user.target
启用服务:
# 系统重载 sudo systemctl daemon-reload # 开启开机自启动服务 sudo systemctl enable elasticsearch.service # 启动服务 sudo systemctl start elasticsearch.service
禁用服务：
# 禁用服务开机自启动 sudo systemctl disable elasticsearch.service # 停止服务 sudo systemctl stop elasticsearch.service # 状态查看验证 sudo systemctl status elasticsearch.service
五、启动
1、创建网桥
docker network create -d bridge my-network
2、自签证书
容器服务中第一个服务，my-es-setup 服务左右就是脚本生成证书文件，确保后续执行启动 elasticsearch 服务由可用的证书，
此步骤非必须，也可以通过其他手段获得 CA 证书，挂载到容器服务 内路径 /usr/share/kibana/config/certs ，关于自签证书：https://www.cnblogs.com/Alay/p/17477191.html
以单机部署为例： -f 是指定我的compose.yml 文件，如果命名为 docker-compose.yml 文件这不需要申明定义 -f ，默认就是这个文件名的文件
docker-compose -f docker-compose-standlone.yml up my-es-setup
3、启动 ES 服务
启动过程中可能会有报错，根据 报错信息进行处理，比如发现启动日志中 {"@timestamp":"2023-06-13T07:51:43.000Z", "log.level":"ERROR","message":"............... 则根据 message 信息进行处理问题 ，
系统设置和权限问题请参考本文中的常见问题
docker-compose -f docker-compose-standlone.yml up my-es
4、启动 Kibana
同样，遇到问题根据日志进行处理，
FATAL Error: Unable to write to UUID file at /usr/share/kibana/data/uuid. Ensure Kibana has sufficient permissions to read / write to this file. Error was: EACCES 这个也是权限问题，和上面的解决方式一样
docker-compose -f docker-compose-standlone.yml up my-kibana
六：访问测试
注意，一定要使用 https 而非 http ，如： https://192.168.1.188:9200
{ "name": "my-es", "cluster_name": "my-es-cluster", "cluster_uuid": "QePUQj_bR2e7k4ZevwTFYA", "version": { "number": "8.7.0", "build_flavor": "default", "build_type": "docker", "build_hash": "09520b59b6bc1057340b55750186466ea715e30e", "build_date": "2023-03-27T16:31:09.816451435Z", "build_snapshot": false, "lucene_version": "9.5.0", "minimum_wire_compatibility_version": "7.17.0", "minimum_index_compatibility_version": "7.0.0" }, "tagline": "You Know, for Search" }
七、配置文件补充
如果需要自定义配置文件的可以自行配置，但是一定要注意，记得挂在配置文件
cluster.name: "my-search-cluster" # bind 网络设置，默认值 0.0.0.0 代表任意IP (IPV4格式)，云服务器中推荐配置为 私网IP #network.host: 0.0.0.0 # 数据迁移 reindex，如果没需要不需要配置 reindex.remote.whitelist: ["192.168.1.100:9200","192.168.1.101:9200"] # 日志级别，默认是 INFO，如果不想要控制台打印太多，可以配置为 WARN / ERROR logger.level: ERROR # 启动安全功能 xpack.security.enabled: true xpack.security.enrollment.enabled: true # 为 HTTP API 客户端连接（例如 Kibana、Logstash 和 Agents）启用加密 xpack.security.http.ssl: enabled: true key: /usr/share/elasticsearch/config/certs/my-es/my-es.key certificate: /usr/share/elasticsearch/config/certs/my-es/my-es.crt certificate_authorities: /usr/share/elasticsearch/config/certs/ca/ca.crt verification_mode: certificate # 启用集群节点之间的加密和相互认证 xpack.security.transport.ssl: enabled: true key: /usr/share/elasticsearch/config/certs/my-es/my-es.key certificate: /usr/share/elasticsearch/config/certs/my-es/my-es.crt certificate_authorities: /usr/share/elasticsearch/config/certs/ca/ca.crt verification_mode: certificate
本文来自博客园，作者：Vermeer，转载请注明原文链接：https://www.cnblogs.com/chxlay/p/17477153.html
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步