ElasticSearch
一、ElasticSearch
1、Installation
(1)uncompress elasticsearch-5.6.8
(2)command line-->elasticsearch(at \bin directory)
2、usage
(1)create index
(2)create document
(3)search
(4)modify
(5)delete
二、Head plugin
1、uncompress elasticsearch-head-master
2、command line-->npm install -g grunt-cli(must be npm,not cnpm)
3、command line-->cnpm install
4、grunt server
5、cross origin
(1)add cross origin configuration in server end("D:\documents\code\java\tensquare\elasticsearch-5.6.8\config")
(1.1)http.cors.enabled: true
(1.2)http.cors.allow-origin: "*"
6、create index
7、select
8、add\delete\update
三、IK participle
1、uncompress elasticsearch-analysis-ik-5.6.8
2、put ik into elasticsearch/plugins/
3、usage
(1)ik_smart
(2)ik_max_word
4、extended thesuarus
(1)add dictionary
(2)registry into "IKAnalyzer.cfg.xml"
四、java
1、pom.xml
1 <dependency> 2 <groupId>org.springframework.data</groupId> 3 <artifactId>spring-data-elasticsearch</artifactId> 4 <version>3.2.1.RELEASE</version> 5 </dependency>
2、SearchApplication.java
1 public class SearchApplication { 2 public static void main(String[] args) { 3 SpringApplication.run(SearchApplication.class,args); 4 } 5 6 @Bean 7 public IdWorker idWorker(){ 8 return new IdWorker(1,1); 9 } 10 }
3、application.yml
1 server: 2 port: 9007 3 spring: 4 application: 5 name: tensquare-search 6 data: 7 elasticsearch: 8 cluster-nodes: 127.0.0.1:9300
4、Article.java
1 @Document(indexName = "tensquare_article",type = "article") 2 public class Article implements Serializable { 3 @Id 4 private String id; 5 6 //index or not 7 //ik participle or not 8 //store or not 9 @Field(index = true,analyzer = "ik_max_word",searchAnalyzer = "ik_max_word") 10 private String title; 11 12 @Field(index = true,analyzer = "ik_max_word",searchAnalyzer = "ik_max_word") 13 private String content; 14 15 private String state; 16 17 18 public String getId() { 19 return id; 20 } 21 22 public void setId(String id) { 23 this.id = id; 24 } 25 26 public String getTitle() { 27 return title; 28 } 29 30 public void setTitle(String title) { 31 this.title = title; 32 } 33 34 public String getContent() { 35 return content; 36 } 37 38 public void setContent(String content) { 39 this.content = content; 40 } 41 42 public String getState() { 43 return state; 44 } 45 46 public void setState(String state) { 47 this.state = state; 48 } 49 }
5、ArticleDao.java
1 public interface ArticleDao extends ElasticsearchRepository<Article,String> { 2 3 public Page<Article> findByTitleOrContentLike(String title, String content, Pageable pageable); 4 }
6、ArticleService.java
1 @Service 2 @Transactional 3 public class ArticleService { 4 5 @Autowired 6 private ArticleDao articleDao; 7 8 @Autowired 9 private IdWorker idWorker; 10 11 public void save(Article article){ 12 article.setId(idWorker.nextId()+""); 13 articleDao.save(article); 14 } 15 16 public Page<Article> findByKey(String key, int page, int size) { 17 PageRequest pageRequest = PageRequest.of(page-1,size); 18 return articleDao.findByTitleOrContentLike(key,key,pageRequest); 19 } 20 }
7、ArticleController.java
1 @RestController 2 @CrossOrigin 3 @RequestMapping(value = "/article") 4 public class ArticleController { 5 6 @Autowired 7 private ArticleService articleService; 8 9 @RequestMapping(method = RequestMethod.POST) 10 public Result save(@RequestBody Article article){ 11 articleService.save(article); 12 return new Result(true, StatusCode.OK,"保存成功!"); 13 } 14 15 @RequestMapping(value = "/{key}/{page}/{size}",method = RequestMethod.GET) 16 public Result findByKey(@PathVariable(name = "key") String key,@PathVariable(name="page") int page,@PathVariable(name = "size") int size){ 17 Page<Article> pageData = articleService.findByKey(key,page,size); 18 return new Result(true,StatusCode.OK,"搜索成功!",new PageResult<Article>(pageData.getTotalElements(),pageData.getContent())); 19 } 20 }
五、LogStash
1、uncompress logstash-5.6.8
2、modify logstash.bat-->add "SET JAVA_HOME=D:\software\java\installDocuments"(java install path)
3、command line
(1)logstash -e 'command'
(2)logstash -f 'file':mysql-->elasticsearch
(2.1)mysql.conf
1 input { 2 jdbc { 3 # mysql jdbc connection string to our backup databse 4 jdbc_connection_string => "jdbc:mysql://192.168.1.66:3306/tensquare_article?characterEncoding=UTF8" 5 # the user we wish to excute our statement as 6 jdbc_user => "root" 7 jdbc_password => "xxxxxx" 8 # the path to our downloaded jdbc driver 9 jdbc_driver_library => "D:\documents\code\java\tensquare\logstash-5.6.8\mysqletc\mysql-connector-java-5.1.46.jar" 10 # the name of the driver class for mysql 11 jdbc_driver_class => "com.mysql.jdbc.Driver" 12 jdbc_paging_enabled => "true" 13 jdbc_page_size => "50" 14 #以下对应着要执行的sql的绝对路径。 15 #statement_filepath => "" 16 statement => "SELECT tb_article.id,tb_article.title,tb_article.content,tb_article.state FROM tb_article" 17 #定时字段 各字段含义(由左至右)分、时、天、月、年,全部为*默认含义为每分钟都更新(测试结果,不同的话请留言指出) 18 schedule => "* * * * *" 19 } 20 } 21 22 output { 23 elasticsearch { 24 #ESIP地址与端口 25 hosts => "127.0.0.1:9200" 26 #ES索引名称(自己定义的) 27 index => "tensquare_article" 28 #自增ID编号 29 document_id => "%{id}" 30 document_type => "article" 31 } 32 stdout { 33 #以JSON格式输出 34 codec => json_lines 35 } 36 }
五、elasticsearch docker
1、download-->docker pull docker.elastic.co/elasticsearch/elasticsearch:7.8.1
2、create container--> docker run -di --name=tensquare_es -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.8.1
3、start-->docker start 8e420c522025
4、IK analyzer(must be installed before use,otherwise there will be error.Because IK can not find type=FieldType.xxx which had been created!!!)
(1)on line
(1.1)docker exec -it 8e420c522025 /bin/bash
(1.2)./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.8.11/elasticsearch-analysis-ik-6.8.11.zip
(1.3)exit && docker restart 8e420c522025
(2)off line
(2.1)upload elasticsearch-analysis-ik-6.8.11.zip
(2.2)docker cp ~/elasticsearch-analysis-ik-6.8.11.zip elasticsearch:/usr/share/elasticsearch/plugins
(2.3)docker exec -it 8e420c522025 /bin/bash
(2.4)mkdir /usr/share/elasticsearch/plugins/ik
(2.5)mv /usr/share/elasticsearch/plugins/elasticsearch-analysis-ik-6.8.11.zip /usr/share/elasticsearch/plugins/ik
(2.6)cd /usr/share/elasticsearch/plugins/ik && unzip elasticsearch-analysis-ik-6.8.11.zip && rm -rf elasticsearch-analysis-ik-6.8.11.zip
(2.7)exit
(2.8)docker restart 8e420c522025
5、open firewall
(1)firewall-cmd --zone=public --add-port=9200/tcp --permanent
(2)firewall-cmd --zone=public --add-port=9300/tcp --permanent
(3)firewall-cmd --reload
6、vim application.yml
1 server: 2 port: 9007 3 spring: 4 application: 5 name: tensquare-search 6 data: 7 elasticsearch: 8 cluster-nodes: 192.168.1.66:9300 9 cluster-name: docker-cluster
7、elasticsearch version
8、system configure
(1)docker exec ‐it tensquare_es/bin/bash
(2)docker cp tensquare_es:/usr/share/elasticsearch/config/elasticsearch.yml /usr/share/elasticsearch.yml
(3)docker stop tensquare_es
(4)docker rm tensquare_es
(5)docker run ‐di ‐‐name=tensquare_es‐p 9200:9200 ‐p 9300:9300 ‐v /usr/share/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml elasticsearch:6.8.11
(6)docker restart tensquare_es
(7)/etc/security/limits.conf
(7.1)* soft nofile 65536
(7.2)* hard nofile 65536
(8)/etc/sysctl.conf
(8.1)vm.max_map_count=655360
(9)sysctl ‐p
(10)systemctl reboot
8、elasticsearch-head
(1) docker pull docker.io/mobz/elasticsearch-head:5
(2)config/elasticsearch.yml
(2.1)http.cors.enabled: true
(2.2)http.cors.allow-origin: "*"
(3)docker run -di --name tensquare_header -p 9100:9100 docker.io/mobz/elasticsearch-head:5
(4)firewall-cmd --zone=public --permanent --add-port=9100/tcp
(5)firewall-cmd --reload