logstash5安装并实现mariadb数据写入到elasticsearch
java环境这里默认安装了 ，一般源码安装，这里就不说了
一、安装logstash
安装logstash可以用yum安装，也可以用源码安装：
yum安装：
1.导入GPG:
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
2.创建repo文件，vim /etc/yum.repos.d/logstash.repo
[logstash-5.x] name=Elastic repository for 5.x packages baseurl=https://artifacts.elastic.co/packages/5.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md
3.安装logstash
yum install logstash
二、源码安装（推荐）
1.官网下载需要的稳定版本
2.解压：
unzip logstash-5.5.2.zip
3.移到安装目录,创建软链接，我这里安装到/opt/app目录
mv logstash-5.5.2 /opt/app/ ln -s logstash-5.5.2/ logstash
4.修改配置文件
vim /opt/app/logstash/config/logstash.yml
path.data: /opt/app/logstash/data/ pipeline.workers: 32 pipeline.output.workers: 4 pipeline.batch.size: 125 path.config: /opt/app/logstash/etc #放配置文件 config.reload.automatic: false #这里yes的话后面会报错，后面有提到这个问题 config.reload.interval: 3 path.logs: /data/logs/logstash
vim /opt/app/logstash/config/startup.options 修改部分选项如下：
JAVACMD=/opt/app/jdk/bin/java LS_HOME=/opt/app/logstash LS_JAVA_OPTS="/opt/app/jdk" LS_PIDFILE=/opt/app/logstash/run/logstash.pid LS_USER=logstash LS_GROUP=logstash LS_GC_LOG_FILE=/data/logs/logstash/gc.log
5.创建目录和用户
useradd -g 549 -u 549 logstash -s /sbin/nologin mkdir /opt/app/logstash/run -p mkdir /data/logs/logstash -p mkdir /opt/app/logstash/data/ -p mkdir /opt/app/logstash/etc -p chown logstash:logstash /opt/app/logstash -R chown logstash:logstash /data/logs/logstash
6.创建启动脚本，通过systemd管理：
vim /etc/systemd/system/logstash.service
[Unit] Description=logstash [Service] Type=simple User=logstash Group=logstash EnvironmentFile=-/etc/default/logstash #ExecStart=/opt/app/logstash/bin/logstash ExecStart=/opt/app/logstash/bin/logstash --path.settings /opt/app/logstash/config ExecStop=/bin/kill -s QUIT $MAINPID Restart=always WorkingDirectory=/ Nice=19 LimitNOFILE=16384 [Install] WantedBy=multi-user.target
vim /etc/default/logstash
JAVACMD="/opt/app/jdk/bin/java" LS_HOME="/opt/app/logstash" LS_SETTINGS_DIR="/opt/app/logstash/config" LS_PIDFILE="/opt/app/logstash/run/logstash.pid" LS_USER="logstash" LS_GROUP="logstash" LS_GC_LOG_FILE="/data/logs/logstash/gc.log" LS_OPEN_FILES="16384" LS_NICE="19" SERVICE_NAME="logstash" SERVICE_DESCRIPTION="logstash"
这样logstash就可以启动了
三、实现logstash写入mariadb数据到elasticsearch
1.安装插件：（这是一个基于jdbc的数据导入插件，可以从各种能对接此驱动的数据库导入数据到ES,还可以自行设定导入执行频率，规则等。
cd /opt/app/logstash bin/logstash-plugin install logstash-input-jdbc
出现success表明安装成功
tips:
安装插件国外源太慢，解决办法：
（1）先安装gem
yum install gem
(2)替换国内的镜像
gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/
（3）验证是否成功
#gem sources -l ***CURRENTSOURCES*** https://gems.ruby-china.org/
（4）修改Gemfile数据源地址
cd /opt/app/logstash 修改Gemfile文件的source的值为：“https://gems.ruby-china.org/” 修改Gemfile.jruby-1.9.lock文件的remote的值为：“https://gems.ruby-china.org/
(5)执行安装插件
bin/logstash-plugin install logstash-input-jdbc
2.安装所需的java驱动包：
解压到自定义目录，这个路径会在logstash配置文件中配置使用，我这里是：
/opt/app/logstash/lib/mysql-connector-java-5.1.44
3.生成配置文件，这里有工具可以生成
、、、、、、
tips:
logstash 启动后报错：
Logstash is not able to start since configuration auto reloading was enabled but the configuration contains plugins that don't support it. Quitting... {:pipeline_id=>"main", :plugins=>[LogStash::Inputs::Stdin]}
解决办法：
config.reload.automatic: false #关闭这个选项
