ELK容器化部署

centos7容器

docker pull centos7 启动一个基础容器

docker cp elastic.tar.gz ab33344ef:/root

docker exec -ti ab33344ef /bin/bash

instances:
  - name: 'node1'
    dns: [ 'node1.elastic.test.com' ]
  - name: 'my-kibana'
    dns: [ 'kibana.local' ]
  - name: 'logstash'
    dns: [ 'logstash.local' ]

instance.yml

生成CA和服务器证书(Elasticsearch安装完毕之后）

bin/elasticsearch-certutil cert ca --pem --in ~/tmp/cert_blog/instance.yml --out ~/tmp/cert_blog/certs.zip

unzip certs.zip -d ./certs

配置elasticsearch.yml

node.name: node1
network.host: node1.elastic.test.com
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: certs/node1.key
xpack.security.http.ssl.certificate: certs/node1.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.key: certs/node1.key
xpack.security.transport.ssl.certificate: certs/node1.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca.crt
discovery.seed_hosts: [ "node1.elastic.test.com" ]
cluster.initial_master_nodes: [ "node1" ]

elasticsearch.yml

设置内置用户的密码

bin/elasticsearch-setup-passwords auto -u "https://node1.elastic.test.com:9200"

启动es

/usr/share/elasticsearch/bin/elasticsearch

root@siem-cluster-node-09 /]# id elsearch
uid=1000(elsearch) gid=1000(elsearch) groups=1000(elsearch)

docker commit 2c7add484489 192.168.30.13/aabb/elastic

docker push 192.168.30.13/aabb/elastic

Elastic容器设置

Kibana容器设置

kibana.yml中证书来源于在elasticsearch容器中生成certs和key以及生成的es内置用户账号

server.name: "my-kibana"
server.host: "kibana.local"
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/config/certs/my-kibana.crt
server.ssl.key: /etc/kibana/config/certs/my-kibana.key
elasticsearch.hosts: ["https://node1.elastic.test.com:9200"]
elasticsearch.username: "kibana"
elasticsearch.password: "uFrZj1AW82vwSMeW02Cc"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/config/certs/ca.crt" ]
monitoring.enabled: false

kibana.yml

可忽略的错误不影响kibana的使用

访问kibana

示例参考地址

https://www.elastic.co/cn/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash#create-ssl

问题

elasticsearch工作负载一旦重新创建它生成的内置用户账号就会失效 kibana和logstash就会连接不上新启动的es Pod

所以elasticsearch一旦创建后不能随意进行创建

替换容器中的环境变量

sed -i 's/redis_ip="[0-9.]*"/redis_ip="'$redis_ip'"/' config.ini

docker run -d --restart=always --ulimit core=-1 --privileged=true -e redis_ip=$REDIS_IP -e redis_port=$REDIS_PORT

容器使用环境变量传递动态参数

1.找到对应容器

2.登录容器

3.创建启动脚本start.sh

sed -i 's/elasticsearch.password: ".*"/elasticsearch.password: "'$espasswd'"/' /root/kibana-7.8.1-linux-x86_64/config/kibana.yml

1.动态使用环境变量修改配置文件

2.启动相关应用

4.保存镜像

docker commit 1d008b3396f6 192.168.30.11/9999/kibana:v2

docker push 192.168.30.11/9999/kibana:v2

5.启动容器时候的设置

Logstash容器设置

node.name: logstash.local
path.config: /etc/logstash/conf.d/*.conf
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: 'elastic'
xpack.monitoring.elasticsearch.password: 'AokJY29yc13rvdKuFzsM'
xpack.monitoring.elasticsearch.hosts: [ 'https://node1.elastic.test.com:9200' ]
xpack.monitoring.elasticsearch.ssl.certificate_authority: /etc/logstash/config/certs/ca.crt

logstash.yml

input {
  beats {
    port => 5044
    ssl => true
    ssl_key => '/etc/logstash/config/certs/logstash.pkcs8.key'
    ssl_certificate => '/etc/logstash/config/certs/logstash.crt'
  }
}
output {
  elasticsearch {
    hosts => ["https://node1.elastic.test.com:9200"]
    cacert => '/etc/logstash/config/certs/ca.crt'
    user => 'elastic'
    password => 'AokJY29yc13rvdKuFzsM'
  }
}

/etc/logstash/conf.d/example.conf

vi start.sh 

sed -i 's/xpack.monitoring.elasticsearch.password: ".*"/xpack.monitoring.elasticsearch.password: "'$espasswd'"/' /root/logstash-7.8.1/config/logstash.yml

sed -i 's/password => ".*"/password => "'$espasswd'"/' /etc/logstash/conf.d/example.conf

/root/logstash-7.8.1/bin/logstash