ES单节点部署及集群部署实战

1.下载指定的ES版本

ELK中文社区下载地址链接:
https://elasticsearch.cn/download/

 

elk官网经常性的打不开,为此记录以下url,根据所需要的版本,更换版本号,即可下载。

 

 - https://artifacts.elastic.co/downloads/kibana/kibana-6.7.2-linux-x86_64.tar.gz

 

 - https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.7.2-linux-x86_64.tar.gz

 

 -  https://artifacts.elastic.co/downloads/logstash/logstash-6.7.2.tar.gz

 

 - https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.2.tar.gz

 

 ###########elasticsearch单节点部署############

1.下载elasticsearch软件

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.3-linux-x86_64.tar.gz

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.3-x86_64.rpm

 

2.部署jdk环境

java -version

3.安装elasticserach软件 ,启动程序服务

yum -y localinstall elasticsearch-7.17.3-x86_64.rpm

###########启动elasticsearch服务#########

[root@elk01 elasticsearch]# systemctl start elasticsearch
[root@elk01 elasticsearch]# netstat -ntulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      941/sshd            
tcp        0      0 0.0.0.0:20000           0.0.0.0:*               LISTEN      943/1panel          
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      4246/java          
tcp6       0      0 ::1:9200                :::*                    LISTEN      4246/java          
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      4246/java          
tcp6       0      0 ::1:9300                :::*                    LISTEN      4246/java          
tcp6       0      0 :::22                   :::*                    LISTEN      941/sshd            
udp        0      0 127.0.0.1:323           0.0.0.0:*                           3300/chronyd        
udp6       0      0 ::1:323                 :::*                                3300/chronyd        
[root@elk01 elasticsearch]# ss -ntulp |grep -i elastic[root@elk01 elasticsearch]# systemctl start elasticsearch
[root@elk01 elasticsearch]# netstat -ntulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      941/sshd            
tcp        0      0 0.0.0.0:20000           0.0.0.0:*               LISTEN      943/1panel          
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      4246/java          
tcp6       0      0 ::1:9200                :::*                    LISTEN      4246/java          
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      4246/java          
tcp6       0      0 ::1:9300                :::*                    LISTEN      4246/java          
tcp6       0      0 :::22                   :::*                    LISTEN      941/sshd            
udp        0      0 127.0.0.1:323           0.0.0.0:*                           3300/chronyd        
udp6       0      0 ::1:323                 :::*                                3300/chronyd        
[root@elk01 elasticsearch]# ss -ntulp |grep -i elastic
 #######访问9200,9300端口############
[root@elk01 elasticsearch]# curl 127.0.0.1:9200
{
  "name" : "elk01",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Ik7H-6GDQO6C3cRsgRv7LA",
  "version" : {
    "number" : "7.17.3",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "5ad023604c8d7416c9eb6c0eadb62b14e766caff",
    "build_date" : "2022-04-19T08:11:19.070913226Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
[root@elk01 elasticsearch]# curl 127.0.0.1:9300
This is not an HTTP port[root@elk01 elasticsearch]#
客户端无法访问,只能本机访问9200端口

4.修改配置文件

##查看配置文件的目录

 

 

elasticsearch配置文件的目录:

/etc/elasticsearch/
elasticsearch.yml主配置文件
要修改的内容:
cluster.name:
node.name
network.host: 0.0.0.0
discoveryseed.host ["elk01"]
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
#####检查文件的正文有2行(过滤注释及空行内容就是正文内容)
[root@elk01 elasticsearch]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

 

 

 浏览器访问9200端口

 

 

@@@@@@@@@@@@@@@@@@@

#################ES集群部署########################

1.将elk01上面的elasticsearch软件同步到其他2个节点上面:

[root@elk01 ~]# data_rsync.sh  /opt/elasticsearch/elasticsearch-7.17.3-
elasticsearch-7.17.3-linux-x86_64.tar.gz  elasticsearch-7.17.3-x86_64.rpm          
[root@elk01 ~]# data_rsync.sh  /opt/elasticsearch/elasticsearch-7.17.3-x86_64.rpm
=========== rsyncing elk01:elasticsearch-7.17.3-x86_64.rpm ====================
命令执行成功!
=========== rsyncing elk02:elasticsearch-7.17.3-x86_64.rpm ====================
命令执行成功!
=========== rsyncing elk03:elasticsearch-7.17.3-x86_64.rpm ====================
命令执行成功!
[root@elk01 ~]#
2.elk02,elk03节点安装elasticsearch软件:
yum -y localinstall /opt/elasticsearch/elasticsearch-7.17.3-x86_64.rpm
3.修改elk01的配置文件,再将配置文件传到其他2个节点
[root@elk01 ~]# vim /etc/elasticsearch/elasticsearch.yml
[root@elk01 ~]# egrep -v '^#|^$' /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk01
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["elk01","elk02","elk03"]
[root@elk01 ~]# systemctl restart elasticsearch


[root@elk01 ~]#
[root@elk01 ~]#
[root@elk01 ~]# data_rsync.sh  /etc/elasticsearch/elasticsearch.yml
=========== rsyncing elk01:elasticsearch.yml ====================
命令执行成功!
=========== rsyncing elk02:elasticsearch.yml ====================
命令执行成功!
=========== rsyncing elk03:elasticsearch.yml ====================
命令执行成功!
[root@elk01 ~]#
[root@elk01 ~]#
 
#######修改节点名称elk02,elk03
[root@elk02 ~]# vi /etc/elasticsearch/elasticsearch.yml
[root@elk02 ~]# egrep -v '^#|^$' /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk02
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["elk01","elk02","elk03"]
[root@elk02 ~]#
 
[root@elk03 ~]# vim /etc/elasticsearch/elasticsearch.yml
[root@elk03 ~]# egrep -v '^#|^$' /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk03
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["elk01","elk02","elk03"]
[root@elk03 ~]#
 
启动elasticsearch启动失败遇到的坑:

 

文章类似的分析错误:

服务器重新安装了系统,所有工作都要重新来过。之前没意识到要装Elasticsearch服务,直到有人问起CirrusSearch的问题,我才想起我竟然都没安装这个服务。
按照我之前的文章《为MediaWiki安装高级搜索(AdvancedSearch)插件》中的方法,一路都很顺利,安装也成功了,但是启动服务会失败。

Job for elasticsearch.service failed because a timeout was exceeded. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.

systemctl status elasticsearch.service命令检查结果如下:
Elasticsearch-Starting-Failed.png
错误代码143。用journalctl -xe检查的结果,主要错误如下:

Jan 29 16:04:49 mindseed systemd[1]: elasticsearch.service start operation timed out. Terminating.
Jan 29 16:05:01 mindseed systemd[1]: Failed to start Elasticsearch.

在网上查了一圈,各种说法都有,修改超时时间的:

sudo systemctl edit --full elasticsearch.service

然后将其中的TimeoutStartSec值改为900,但是没用。
还有将143作为成功退出的代码,源配置本来就已经设置好了。还有其他一堆乱七八糟的,总之我试了一圈都没成功,也就不一一列举了。
无意中翻到一个回答,提到了elasticsearch可能失败的原因——没有足够的内存让服务运行。

作为JVM应用程序,Elasticsearch主服务器进程仅利用专用于JVM的内存。所需的内存可能取决于所使用的JVM(32位或64位)。
sudo vim /etc/elasticsearch/jvm.options

找到Xms行如下:

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

按照文章中的内容,-Xms和-Xmx的值不应该超过服务器物理内存的50%,很显然是我那只有1G内存的服务器拖了后腿。将值改为128m后重新启动服务,终于成功了。

我解决的:

问题得到解决:

 4.访问测试

 浏览器访问:

 ##################################################################################
集群排查日志目录:

/var/log/elasticsearch

[root@elk02 elasticsearch]# wc -l elkcluster.log
29558 elkcluster.log
[root@elk02 elasticsearch]# wc -l elkcluster.log
29624 elkcluster.log
[root@elk02 elasticsearch]# wc -l elkcluster.log
29690 elkcluster.log
[root@elk02 elasticsearch]# wc -l elkcluster.log
29723 elkcluster.log
[root@elk02 elasticsearch]# wc -l elkcluster.log
29773 elkcluster.log
[root@elk02 elasticsearch]# pwd
/var/log/elasticsearch
[root@elk02 elasticsearch]#
 坑:只有主节点,没有从节点
[root@elk01 ~]# curl 192.168.8.101:9200/_cat/health
1705392436 08:07:16 elkcluster green 1 1 2 2 0 0 0 0 - 100.0%
[root@elk01 ~]# curl 192.168.8.101:9200/_cat/nodes
192.168.8.101 33 93 5 0.09 0.27 0.35 cdfhilmrstw * elk01
[root@elk01 ~]#

 

重启elasticsearch后,在浏览器上 192.168.8.101:9200/_cluster/health?pretty 查看健康状况
发现

"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
1
2
结果明显是不对的,两者的数值应该都是3,因为我们配置了3个node:分别为elk01,elk02,elk03

 

 

 

排查:

connection failed org.elasticsearch.transport.ConnectTransportException: [elk03][172.17.0.1:9300] handshake failed.

 三台主机都有docker,则存在多个网卡地址,导致elasticsearch集群一直起不来,解决方法:每个节点都指明映射地址 是谁

删除临时数据

三个节点都执行

rm -rf /var/{lib,log}/elasticsearch/* /tmp/*
 
将2个地方主机名改为IP地址:
network.bind_host: ["192.168.56.101"]
network.publish_host: 192.168.56.101
三台主机最后的配置如下:
[root@elk01 elasticsearch]# egrep -v "^$|^#" /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk01
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.8.101","192.168.8.102","192.168.8.103"]
cluster.initial_master_nodes: ["192.168.8.101","192.168.8.102","192.168.8.103"]
network.bind_host: ["192.168.8.101"]
network.publish_host: 192.168.8.101
[root@elk01 elasticsearch]#
##################################################################
[root@elk02 elasticsearch]# egrep -v "^$|^#" /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk02
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.8.101","192.168.8.102","192.168.8.103"]
network.bind_host: ["192.168.8.102"]
network.publish_host: 192.168.8.102
cluster.initial_master_nodes: ["192.168.8.101","192.168.8.102","192.168.8.103"]
[root@elk02 elasticsearch]#
#####################################################################
[root@elk03 ~]# egrep -v "^$|^#" /etc/elasticsearch/elasticsearch.yml
cluster.name: elkcluster
node.name: elk03
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.8.101","192.168.8.102","192.168.8.103"]
cluster.initial_master_nodes: ["192.168.8.101","192.168.8.102","192.168.8.103"]
network.bind_host: ["192.168.8.103"]
network.publish_host: 192.168.8.103
[root@elk03 ~]#
由以下截图可以看出,问题得到解决:

 *代表master节点,其他为slave节点

 

end#############################至此,elasticsearch集群安装成功!!!!!

检查集群状态:3个节点数量成功正确

curl elk01:9200/_cluster/health?pretty

 

 

 

 

 
 
 
 
 
 
 
 
 
 

 

posted @ 2024-01-16 09:13  往事已成昨天  阅读(392)  评论(0编辑  收藏  举报