使用的elk是6.4.3版本,安装java8,可以使用yum install tomcat就会自动安装了
1、首先下载elasticsearch-6.4.3、logstash-6.4.3、kibana-6.4.3,不要放在root目录下
2、解压完成后做如下修改
解压后进入config目录:cd elasticsearch-6.4.2/config,vim elasticsearch.yml 修改配置文件,在最下面加入如下几行:
network.host: 0.0.0.0 http.port: 9200 http.cors.enabled: true http.cors.allow-origin: "*"
3、不能使用root用户启动,需要创建新用户
创建elsearch用户组及elsearch用户 groupadd elsearch useradd elsearch -g elsearch -p elasticsearch 更改elasticsearch-6.4.3文件夹及内部文件的所属用户及组为elsearch:elsearch cd切换到elasticsearch-6.4.3的父路径下 chown -R elsearch:elsearch elasticsearch-6.4.3 切换到elsearch用户再启动 su elsearch cd elasticsearch-6.4.3/bin sh elasticsearch &
4、完成之后启动
在bin目录下执行./elasticsearch &
5、可能会遇到错误,如下解决
启动失败,检查没有通过,报错 [2018-05-18T17:44:59,658][INFO ][o.e.b.BootstrapChecks ] [gFOuNlS] bound or publishing to a non-loopback address, enforcing bootstrap checks ERROR: [2] bootstrap checks failed [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] [1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536] 编辑 /etc/security/limits.conf,追加以下内容; * soft nofile 65536 * hard nofile 65536
centos7.6版本需要添加
* soft nofile 655350
* hard nofile 655350
此文件修改后需要重新登录用户,才会生效 [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] 编辑 /etc/sysctl.conf,追加以下内容: vm.max_map_count=655360 保存后,执行: sysctl -p 重新启动,成功。 bin/elasticsearch &
6、kibana启动
按照上面的操作,可以先切回root用户了:su root 需要输入root密码
然后cd kibana-6.4.3-linux-x86_64/config/ 进入kibana的配置文件夹,
vim kibana.yml 编辑配置文件,在最后面加上如下配置就行:
server.port: 5601 server.host: "0.0.0.0" elasticsearch.url: "http://localhost:9200" kibana.index: ".kibana"
cd命令进入bin目录,
sh kibana & 启动kibana
7、配置logstash
cd logstash-6.4.2/bin/进入bin目录
新建文件 vim logstash1.conf ,写入内容(监听tomcat的日志):
input { file { path => "/usr/share/tomcat/logs/*.log" start_position => beginning } } filter { } output { elasticsearch { hosts => "localhost:9200" } }
启动logstash:sh logstash -f logstash.conf &
如果提示--path.data的问题,则需要指定path.data的路径,随便找个路径就行,
我的是这样启动:sh logstash -f logstash1.conf --path.data=/home/elk/logstash-6.4.3/logs &
完了可以看到kibana上面有logstash推送过去的日志了
配合mysql存储
wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.49.tar.gz -O /home/liyong/mysql-connector.tar.gz
可以把解压后的mysql-connector-java-5.1.46-bin.jar复制到bin目录,方便启动
在bin目录新建配置文件 vim mysql-logstash.cnf ,写入:
input { jdbc { jdbc_driver_library => "mysql-connector-java-5.1.46-bin.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://ip:3306/db_name" jdbc_user => "jdbc_user" jdbc_password => "jdbc_password" schedule => "* * * * *" statement => "SELECT * FROM table WHERE 时间字段 >= :sql_last_value" use_column_value => true tracking_column_type => "timestamp" tracking_column => "时间字段" last_run_metadata_path => "syncpoint_table" } } output { elasticsearch { hosts => "localhost:9200" index => "索引名称" document_id => "%{主键字段}" } }
需要先把logstash停了,在重启
netstat -ntlp 命令查看端口为9600的进程
重新执行即可
sh /usr/elk/logstash-6.4.3/bin/logstash -f /usr/elk/logstash-6.4.3/bin/mysql-logstash.cnf --path.data=/usr/elk/logstash-6.4.3/logs &