es连接mysql,并且配置中文分词

es连接mysql导入数据

---

下载 logstash

logstash的版本需要与es的版本一致,安装完成后添加两个文件:mysql.conf(存放sql查询结果导入es的配置)和mysql.sql(存放sql语句);

mysql.sql:这儿有根据修改的时间来判断最后一条数据,避免重复导入

select * FROM commodity where update_time > :sql_last_value

mysql:conf

input {
    jdbc {
      jdbc_connection_string => "jdbc:mysql://localhost:3306/database?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&lower_case_table_names=1&allowMultiQueries=true&autoReconnect=true"
      jdbc_user => "root"
      jdbc_password => "数据库的密码"
      jdbc_driver_library => "/home/mysql-connector-java-8.0.21.jar"
      jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      jdbc_default_timezone => "UTC"
      record_last_run => true
      use_column_value => true
      tracking_column => "update_time"
      tracking_column_type => "timestamp"
      clean_run => false
      lowercase_column_names => false
      statement_filepath => "/home/logstash-7.8.0/mysql/mysql.sql"#mysql.sql的存放路径
      schedule => "*/3 * * * *" #多长时间执行一次
    }
}

filter {
    json {
        source => "message"
        remove_field => ["message"]
    }
}

output {
    elasticsearch {
        hosts => ["localhost:9200"] #es的路径
        user => "elastic" #如果es设置了密码,那么有user和password,没有就不设置
	password => "@13980053017qq"
        index => "good" #这儿是索引,如果想要数据库中的某个字段设置中文分词器,可以先建好索引,这儿填写建好的索引即可
	document_id => "%{id}" 
	template_overwrite => true
	template => "/home/logstash-7.8.0/templete/logstash.json"
        manage_template => false
    }
    stdout {
        codec => json_lines
    }
}

然后执行.\logstash -f mysql.conf即可