elasticsearch配置集群+elk报错总结

配置ELK的时候,我平常遇到了以下几种报错情况,整理如下(持续更新中):
  1. elasticsearch启动失败

    # systemctl start elasticsearch
    Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.
    
    #这个时候,直接查看系统日志,因为elasticsearch没有专门的日志审计
    tail -f /var/log/messages
    

    出现如下报错

    Dec 13 10:16:30 oldboy elasticsearch: ERROR: [1] bootstrap checks failed
    Dec 13 10:16:30 oldboy elasticsearch: [1]: initial heap size [536870912] not equal to maximum heap size [775946240]; this can cause resize pauses and prevents mlockall from locking the entire heap
    
    

    其实提示已经很明显了,jvm给的内存不足,那么我们直接把内存调大就可以了

    #修改jvm内存大小
    # vim /etc/elasticsearch/jvm.options
    -Xms1500m 
    -Xms1500m
    #因为刚才把内存改的很小,改回来就行了
    

    如果不是使用的systemd方法启动,直接调用bin/elasticsearch 启动,那么有几点需要注意

    #1.不能使用root进行登录
    useradd elk #创建用户elk
    
    #2.将涉及的用户权限赋予elk
    
  2. kibana显示中文乱码

    #首先查看要拉取的日志的格式是什么
    file file.txt  #在linux上查看
    
    以记事本打开log文件,点击另存为查看,如果显示为ANSI,那么就是gbk  #在windows上查看
    
    #在filebeat中配置字符集
    
    # vim /etc/filebeat/filebeat.yml
    
    filebeat.inputs:
    
    - type: log
    
     
      enabled: true
    
      paths:
        - c:\work\CA*
      encoding: gbk   #此处加入字符格式,如果是utf8,那么不需要添加
    

    继续生成测试日志,登录kibana查看,发现中文字符已经正常显示,没有乱码了。

  3. es集群配置xpack启动后,创建密码失败

    [root@db01 elasticsearch]# bin/elasticsearch-setup-passwords interactive
    
    Failed to determine the health of the cluster running at http://10.0.0.200:9200
    Unexpected response code [503] from calling GET http://10.0.0.200:9200/_cluster/health?pretty
    Cause: master_not_discovered_exception
    
    It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
    It is very likely that the password changes will fail when run against an unhealthy cluster.
    
    Do you want to continue with the password setup process [y/N]y
    
    Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
    You will be prompted to enter passwords as the process progresses.
    Please confirm that you would like to continue [y/N]y
    
    
    #错误原因,因为脏数据的原因,当开始xpack的时候,集群链接失败
    
    #终极大招(只适用于初始创建集群,或者测试环境)
    
    1.停止服务
    2.删除数据目录
    3.三个节点只配置xpack.security.enabled: true,启动
    4.设置密码
    
    #配置文件(三台除了ip之外都一样)
    cluster.name: think
    node.name: node-1
    path.data: /var/lib/elasticsearch
    path.logs: /var/log/elasticsearch
    bootstrap.memory_lock: true
    network.host: 10.0.0.200,127.0.0.1
    http.port: 9200
    discovery.seed_hosts: ["10.0.0.200", "10.0.0.201"]
    cluster.initial_master_nodes: ["10.0.0.200", "10.0.0.201","10.0.0.202"]
    http.cors.enabled: true
    http.cors.allow-origin: "*"
    xpack.security.enabled: true
    
    
    #测试效果
    [root@db01 elasticsearch]# bin/elasticsearch-setup-passwords interactive
    Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
    You will be prompted to enter passwords as the process progresses.
    Please confirm that you would like to continue [y/N]y
    
    
    Enter password for [elastic]: 
    Reenter password for [elastic]: 
    Enter password for [apm_system]: 
    Reenter password for [apm_system]: 
    Enter password for [kibana]: 
    Reenter password for [kibana]: 
    Enter password for [logstash_system]: 
    Reenter password for [logstash_system]: 
    Enter password for [beats_system]: 
    Reenter password for [beats_system]: 
    Enter password for [remote_monitoring_user]: 
    Reenter password for [remote_monitoring_user]: 
    Changed password for user [apm_system]
    Changed password for user [kibana]
    Changed password for user [logstash_system]
    Changed password for user [beats_system]
    Changed password for user [remote_monitoring_user]
    Changed password for user [elastic]
    
    #成功
    

4.隔天上班又出现和标题3同样的情况,如下解决方案

#直接配上ca证书验证,开启ssl

# 设置默认的角色密码
bin/elasticsearch-setup-passwords interactive  #这一步我是不成功的,不过标题3已经创建过了,所以跳过

再elasticsearch.yml加入如下
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate # 证书验证级别
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 # 节点证书路径
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

#创建证书
# 创建keystore文件
# bin/elasticsearch-keystore create # config文件夹下有的话这一步就不用再执行了

# 生成CA证书,一直回车
bin/elasticsearch-certutil ca (CA证书:elastic-stack-ca.p12)

# 生成节点使用的证书,一直回车
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12  (节点证书:elastic-certificates.p12)

# 创建证书保存目录,并移动到config文件下
mkdir -p /etc/elasticsearch/certs
mv elastic-certificates.p12 /etc/elasticsearch/certs 
chmod 777 /etc/elasticsearch/certs   #不给授权就无法登录,可以自己测测到底给多少合适

#重启
posted @ 2019-12-16 15:48  大葱丁  阅读(3469)  评论(0编辑  收藏  举报