- 1. 安装Elasticsearch
1.1 安装及配置Elasticsearch
1.1.1 安装
官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
cd /usr/local #进入local目录 mkdir elasticsearch #创建elasticsearch文件夹 cd elasticsearch #进入elasticsearch文件夹 wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.rpm #开始下载 rpm -ivh elasticsearch-6.6.0.rpm #开始安装
使用Docker安装ElasticSearch
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html #获取镜像 docker pull docker.elastic.co/elasticsearch/elasticsearch:6.7.0 #运行容器 docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:6.7.0
1.1.2 配置
whereis elasticsearch #查找安装目录 vi /etc/elasticsearch/elasticsearch.yml #编辑配置文件
主要配置Network.host（本机ip）和http.port（默认9200）（目前单节点模式，其他参数请参考官方文档）
1.1.3 启动服务
firewall-cmd --add-port=9200/tcp --permanent #开启端口9200 firewall-cmd --reload #重新加载配置 systemctl enable elasticsearch #设置服务开机启动 systemctl start elasticsearch #启动服务
在浏览器打开http://192.168.30.128:9200,如下图所示表示启动成功了
1.1.4 卸载Elasticsearch
systemctl stop elasticsearch #停止elasticsearch rpm -e elasticsearch #卸载elasticsearch ps aux | grep elasticsearch #查看gitlab进程
1.1.5 相关问题
l 安装完毕后，elasticSearch无法启动
解决方案：修改jvm配置，减少elasticSearch占用内存
vi /etc/elasticsearch/jvm.options #修改为：-Xms256m -Xmx256m
#保存后，重启elasticsearch服务
l elasticSearch启动报错：elasticsearch: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
解决方案：修改jvm配置（/etc/elasticsearch/jvm.options）
vi /etc/elasticsearch/jvm.options #文件中加入：-XX:-AssumeMP
l 其他
1.2 安装及配置logstash
1.2.1 安装
官方文档：https://www.elastic.co/guide/en/logstash/current/index.html
cd /usr/local/elasticsearch #进入elasticsearch目录 wget https://artifacts.elastic.co/downloads/logstash/logstash-6.6.0.rpm rpm -ivh logstash-6.6.0.rpm #安装rpm包
使用Docker安装logstash
https://www.elastic.co/guide/en/logstash/current/docker.html
docker pull docker.elastic.co/logstash/logstash:6.7.0
1.2.2 配置
vi /etc/logstash/conf.d/nlog.conf #新增配置信息
input：采用TCP监控本机8001端口的消息
filter：使用grok 插件，自定义消息格式，推荐使用grokdebug在线进行调试
output：使用elasticsearch作为数据存储
1.2.3 启动服务
firewall-cmd --add-port=8001/tcp --permanent #开启端口8001 firewall-cmd --reload #重载配置 systemctl enable logstash #设置开机启动 systemctl start logstash #启动logstash
1.2.4 相关问题
l 设置开机启动时报：Failed to start logstash.service: Unit not found
vi /etc/logstash/startup.options #修改启动配置，如下图
cd /usr/share/logstash/bin /usr/share/logstash/bin/system-install /etc/logstash/startup.options
https://www.centos.org/forums/viewtopic.php?t=67591
https://discuss.elastic.co/t/logstash-service-unit-not-found-centos-7/138446
https://stackoverflow.com/questions/41986441/unable-to-start-logstash-service-on-centos7
l 其他
1.3 安装及配置Kibana
1.3.1 安装Kibana
cd /usr/local/elasticsearch #进入elasticsearch目录 wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.0-x86_64.rpm rpm -ivh kibana-6.6.0-x86_64.rpm #安装Kibana
1.3.2 配置Kibana
cd /etc/kibana #进入安装目录 vi kibana.yml #编辑配置文件
设置端口号：5601，Host地址:"192.168.30.128" ，elasticsearch服务地址为：http://192.168.30.128:9200
1.3.3 启动服务
firewall-cmd --add-port=5601/tcp --permanent #开启端口5601 firewall-cmd --reload #重新加载配置 systemctl enable kibana #设置服务开机启动 systemctl start kibana #启动服务
在浏览器打开http://192.168.30.128:5601,将进入到Kibana管理界面
1.4 Nginx使用logstash输出日志
1.4.1 Nginx设置日志格式
vi /usr/local/nginx/conf/nginx.conf
#在http模块中添加
log_format json '{"@timestamp":"$time_iso8601",'
'"@version":"1",'
'"client":"$remote_addr",'
'"url":"$uri",'
'"status":"$status",'
'"domain":"$host",'
'"host":"$server_addr",'
'"size":$body_bytes_sent,'
'"responsetime":$request_time,'
'"referer": "$http_referer",'
'"ua": "$http_user_agent"'
'}';
#在server模块中添加
access_log /var/log/nginx/access.log json;
修改后的Nignx.conf文件
1.4.2 使用logstash
1.4.2.1 配置logstash
vi /etc/logstash/conf.d/nginx_log.conf
配置内容如下：
input {
file {
path => "/var/log/elasticsearch/elasticsearch.log"
type => "es-error"
start_position => "beginning"
codec => multiline {
pattern => "^\["
negate => true
what => "previous"
}
}
file {
path => "/usr/local/nginx/logs/access.log"
codec => json
start_position => "beginning"
type => "nginx-log"
}
}
output {
if [type] == "es-error"{
elasticsearch {
hosts => ["192.168.56.99:9200"]
index => "es-error-%{+YYYY.MM.dd}"
}
}
if [type] == "nginx-log"{
elasticsearch {
hosts => ["192.168.56.99:9200"]
index => "nginx-log-%{+YYYY.MM.dd}"
}
}
}
1.4.2.2 检测logstash配置是否正确
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx_log.conf --configtest
1.4.2.3 使用kibana查看日志
l 使用浏览器访问kibana：http://192.168.56.99:5601
l 选择management/Index Patterns创建nginx索引
l 单击“Discover”查看日志
