ELK容器化部署
centos7容器
docker pull centos7 启动一个基础容器
docker cp elastic.tar.gz ab33344ef:/root
docker exec -ti ab33344ef /bin/bash
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
instances: - name: 'node1' dns: [ 'node1.elastic.test.com' ] - name: 'my-kibana' dns: [ 'kibana.local' ] - name: 'logstash' dns: [ 'logstash.local' ]
生成CA和服务器证书(Elasticsearch安装完毕之后)
bin/elasticsearch-certutil cert ca --pem --in ~/tmp/cert_blog/instance.yml --out ~/tmp/cert_blog/certs.zip
unzip certs.zip -d ./certs
配置elasticsearch.yml
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
node.name: node1 network.host: node1.elastic.test.com xpack.security.enabled: true xpack.security.http.ssl.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.http.ssl.key: certs/node1.key xpack.security.http.ssl.certificate: certs/node1.crt xpack.security.http.ssl.certificate_authorities: certs/ca.crt xpack.security.transport.ssl.key: certs/node1.key xpack.security.transport.ssl.certificate: certs/node1.crt xpack.security.transport.ssl.certificate_authorities: certs/ca.crt discovery.seed_hosts: [ "node1.elastic.test.com" ] cluster.initial_master_nodes: [ "node1" ]
设置内置用户的密码
bin/elasticsearch-setup-passwords auto -u "https://node1.elastic.test.com:9200"
启动es
/usr/share/elasticsearch/bin/elasticsearch
root@siem-cluster-node-09 /]# id elsearch
uid=1000(elsearch) gid=1000(elsearch) groups=1000(elsearch)
docker commit 2c7add484489 192.168.30.13/aabb/elastic
docker push 192.168.30.13/aabb/elastic
Elastic容器设置
Kibana容器设置
kibana.yml中证书来源于在elasticsearch容器中生成certs和key以及生成的es内置用户账号
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
server.name: "my-kibana" server.host: "kibana.local" server.ssl.enabled: true server.ssl.certificate: /etc/kibana/config/certs/my-kibana.crt server.ssl.key: /etc/kibana/config/certs/my-kibana.key elasticsearch.hosts: ["https://node1.elastic.test.com:9200"] elasticsearch.username: "kibana" elasticsearch.password: "uFrZj1AW82vwSMeW02Cc" elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/config/certs/ca.crt" ] monitoring.enabled: false
可忽略的错误 不影响kibana的使用
访问kibana
示例参考地址
https://www.elastic.co/cn/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash#create-ssl
问题
elasticsearch工作负载一旦重新创建 它生成的内置用户账号就会失效 kibana和logstash就会连接不上新启动的es Pod
所以elasticsearch一旦创建后不能随意进行创建
替换容器中的环境变量
sed -i 's/redis_ip="[0-9.]*"/redis_ip="'$redis_ip'"/' config.ini
docker run -d --restart=always --ulimit core=-1 --privileged=true -e redis_ip=$REDIS_IP -e redis_port=$REDIS_PORT
容器使用环境变量传递动态参数
1.找到对应容器
2.登录容器
3.创建启动脚本start.sh
sed -i 's/elasticsearch.password: ".*"/elasticsearch.password: "'$espasswd'"/' /root/kibana-7.8.1-linux-x86_64/config/kibana.yml
1.动态使用环境变量修改配置文件
2.启动相关应用
4.保存镜像
docker commit 1d008b3396f6 192.168.30.11/9999/kibana:v2
docker push 192.168.30.11/9999/kibana:v2
5.启动容器时候的设置
Logstash容器设置
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
node.name: logstash.local path.config: /etc/logstash/conf.d/*.conf xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.username: 'elastic' xpack.monitoring.elasticsearch.password: 'AokJY29yc13rvdKuFzsM' xpack.monitoring.elasticsearch.hosts: [ 'https://node1.elastic.test.com:9200' ] xpack.monitoring.elasticsearch.ssl.certificate_authority: /etc/logstash/config/certs/ca.crt
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
input { beats { port => 5044 ssl => true ssl_key => '/etc/logstash/config/certs/logstash.pkcs8.key' ssl_certificate => '/etc/logstash/config/certs/logstash.crt' } } output { elasticsearch { hosts => ["https://node1.elastic.test.com:9200"] cacert => '/etc/logstash/config/certs/ca.crt' user => 'elastic' password => 'AokJY29yc13rvdKuFzsM' } }
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
vi start.sh sed -i 's/xpack.monitoring.elasticsearch.password: ".*"/xpack.monitoring.elasticsearch.password: "'$espasswd'"/' /root/logstash-7.8.1/config/logstash.yml sed -i 's/password => ".*"/password => "'$espasswd'"/' /etc/logstash/conf.d/example.conf /root/logstash-7.8.1/bin/logstash
搭建成功 elk
elasticsearch的pod重新创建后需要进入es容器重新生成密码 然后给kibana和logstashpod重新设置espasswd的环境变量
Flink容器安装
flink容器启动失败 经过分析是因为启动脚本是以后台进程的方式启动的导致脚本一结束容器便自动结束
flink以前台进程方式启动是需要手动修改start-cluster.sh文件
vi start-cluster.sh 修改文件内容
启动shell的方式不同导致结果不同
正确的启动脚本方式 ./start-cluster.sh
错误的启动脚本方式 sh start-cluster.sh 会提示语法错误
jobmanager.sh start 以后台进程方式启动
jobmanager.sh start-foreground 以前台进程的方式启动
以docker容器的方式启动主进程必须要以前台的方式运行
kafka部署
docker pull kafkamanager/kafka-manager
docker tag 14a7c1e556f7 192.168.30.113/library/kafka
docker push 192.168.30.113/library/kafka
kafka依赖zookeeper 必须先部署zookeeper
kafaka容器设置
kafka成功启动
本文来自博客园,作者:不懂123,转载请注明原文链接:https://www.cnblogs.com/yxh168/p/13517392.html