ElasticSearch快速入门
ElasticSearch快速入门
一、简介
ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。
其他搜索服务器：Solr
ElasticSearch与Solr区别：ElasticSearch对进行实时更新的数据进行搜索比较快，Solr对进行不经常更改的数据比较搜索比较快，遇到实时更新的数据会出现线程阻塞。
传统数据库为特定列增加一个索引，例如B-Tree索引来加速检索。Elasticsearch和Lucene使用一种叫做倒排索引(inverted index)的数据结构来达到相同目的，通俗点来说，倒排索引就是通过Value获取key，就是先把数据导入搜索服务器中，搜索服务器对数据进行分词，然后对分词进行归类，每一个文档（行）就声明一个词的出现频次、位置。
非必要最新版本，建议使用ElasticSearch7版本，ElasticSearch8版本对javaApI及java High Level Rest Cliet语法有所改变，但是java-client7.0与ElasticSearch8.0兼容，具体见官网:https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/installation.html
二、倒排索引
每一列表示字段值，每一行表示一个文档，每个类型相当于数据库中的数据表，但是在ElasticSearch7.0以后，为了提升查询效率，类型只是有`_doc`一种。
举例解析：比如谷歌这个词，文档频率是5，表示出现在5行中(见文档内容图)都有谷歌这个词,(1;1),(2;1)
,(3;2)...表示第一行出现一次，第二行出现一次，第三行出现二次，<1>表示出现在第一个单词位置上，<1;6>表示出现在第一个和第六个单词位置上，搜索服务器根据倒排列表的信息快速定位到单词位置，提升查询效率。
- ElasticSearch与MySQL概率对比：
三、环境搭建(单机)
- 下载软件(官网下载 选择相应的版本进行下载)
- 把下载的Linux包上传Linux中相应的文件夹中
- 解压压缩包
tar -zxvf elasticsearch-8.10.2-linux-x86_64.tar.gz
进入解压后的目录中 并修改elasticserach.yml
cd elasticsearch-8.10.2/config
vim elasticserach.yml
以下配置要启动服务器(
cd elasticsearch-8.10.2/bin./elasticsearch)后才能生成，生成以后就修改如下配置,外部浏览器才能访问或者直接添加xpack.security.enabled: false，记住开放端口号9200和9300。
配置Linux中允许创建的最大线程（root用户下）
vim /etc/security/limits.conf
- 配置信息如下：
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096
配置Linux中最大虚拟内存区域（root用户下）
vim /etc/sysctl.conf
- 配置信息如下：
vm.max_map_count = 262144
重新加载虚拟内存（root用户下）
sysctl -p
注意要在非root用户下操作启动（root用户下）
groupadd es #创建用户组
useradd es -g es #创建用户赋予es用户组
chown es:es -Rf elasticsearch-8.10.2 #为新建的用户授权 防止权限不足无法启动
su es #切换用户启动服务器
cd elasticsearch-8.10.2/bin #进入启动目录
./elasticsearch #启动服务器 （非root用户下操作）
./elasticsearch -d #守护进程后台启动（非root用户下操作）
测试访问
http://192.168.147.110:9200
四、搭建集群（三台服务器）
- 注意事项
如果配置过单机版的es服务器，配置集群时，要重新解压安装包，否则容易出错，因为单机版es服务器启动后，会产生一些新的缓存和配置文件信息。
- 上传文件并解压linux压缩包到三个不同服务器中（
192.168.147.110;192.168.147.120;192.168.147.130）。
tar -zxvf elasticsearch-8.10.2-linux-x86_64.tar.gz
- 进入解压文件中config，配置elasticSearch.yml文件(三台服务器要配置)
cluster.name: my-application #三个节点集群一样
node.name: node-2 #每个节点名不一样
network.host: 0.0.0.0 #所有的远程主机都可以访问
http.port: 9200 #端口号三个节点一样
discovery.seed_hosts: ["192.168.147.110", "192.168.147.120","192.168.147.130"] #配置三个主机IP
cluster.initial_master_nodes: ["node-1", "node-2","node-3"] #配置集群节点
discovery.request_peers_timeout: 120s #节点与节点之间连接超时等待时间
http.cors.enabled: true #开启第三方工具可以连接
http.cors.allow-origin: "*" #允许所有的第三方工具连接
xpack.security.enabled: false #关闭安全连接 否则单机 浏览器无法访问 集群会报错
配置Linux中允许创建的最大线程（root用户下）
vim /etc/security/limits.conf
- 配置信息如下：
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096
配置Linux中最大虚拟内存区域（root用户下）
vim /etc/sysctl.conf
- 配置信息如下：
vm.max_map_count = 262144
重新加载虚拟内存（root用户下）
sysctl -p
分别启动三台不同服务器
groupadd es
useradd es -g es
chown es:es -Rf /elasticsearch-8.10.2
su es #切换非root用户
cd /elasticsearch-8.10.2/bin #进入启动文件夹
./elasticsearch #启动服务
查看集群的状态：访问
http://192.168.147.130:9200/_cluster/health?pretty
所有分片都正常，健康状态显示green，如果只是其中一个分片坏了，服务器还是能完整查询，如：node-1中2、3分片坏了，其他分别还是可以组件0,1,2,3,4完整查询，健康状态为yellow，如果分片坏了，无法组件完整数据，如：三个节点中所有的0坏了，无法排列组合完整，健康状态为red。
五、elasticSearch可视化工具安装
第三方插件
常用的第三方插件有：head、cerebro、elasticHD，这里只说明head插件安装。
安装git
yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm #安装endpoint镜像仓库 里面有新版git
yum install -y git
git --version #验证安装
安装npm centOs7支持nodejs16.X及以下版本，版本太高可能安装失败！
- 下载官方源：
# root用户执行
curl -fsSL https://rpm.nodesource.com/setup_16.x | bash
# 非root用户执行
curl -fsSL https://rpm.nodesource.com/setup_16.x | sudo bash -
- 安装nodejs
yum install -y nodejs
- 验证安装
node -v
npm -v
- 更换镜像
npm config set registry https://registry.npmmirror.com/ #下载速度更快
安装head第三方插件
- 进入head插件github开源项目站
https://github.com/mobz/elasticsearch-head
- 配置三台主机允许跨域才能外部浏览器访问
- git 拉取和启动elasticsearch-head插件
git clone https://github.com/mobz/elasticsearch-head.git #如果没网络去官网下载tag.gz文件再丢进服务器 然后解压进入项目根目录。 注意地址是https://开头
cd elasticsearch-head
npm install phantomjs-prebuilt --ignore-scripts #这步要执行（重要）
npm install
npm run start
访问 http://192.168.147.110:9100/
出现以上界面表示集群搭建成功！
- elasticSearch 基础语法
GET http://192.168.147.110:9200/test/_doc/1/ #查看id=1的值
put/post http://192.168.147.110:9200/test/_doc/5/ #id值不存在就添加数据 id不设置 随机生成id
delete http://192.168.147.110:9200/test/_doc/5/ #根据id删除
安装谷歌插件
六、安装Kibana可视化软件
二、创建文件夹并解压
mkdir -p /usr/local/kibana #c创建自定义文件夹
tar -zxvf kibana-*****-linux-x86_64.tar.gz -C /usr/local/kib
三、修改配置文件
vi config/kibana.yml
- 配置文件如下：
# 服务端口，默认5601
server.port: 5601
# 允许访问IP
server.host: "0.0.0.0"
# 设置 elasticsearch 节点及端口
elasticsearch.hosts: ["http://192.168.147.110:9200", "http://192.168.147.120:9200",
"http://192.168.147.130:9200"]
四、启动kibana软件(进入kibana bin目录下 注意开放端口号)
./kibana --allow-root #允许root用户启动
./kibana #非root用户启动
五、访问请求路径(前提要启动es)
http://192.168.147.110:5601
七、安装IK分词器插件
找到官网下载：下载地址： https://github.com/medcl/elasticsearch-analysis-ik/releases将文件上传至服务器
方式一：创建ik目录，然后将ik分词器解压至ik目录(三个节点都需要操作)
方式二：在bin目录下执行命令来安装插件(三个节点都需要安装)
mkdir -p /usr/local/elasticsearch-/plugins/ik # 解压至ik目录
解压插件到 /usr/local/elasticsearch-***/plugins/ik/
unzip elasticsearch-analysis-ik-.zip -d /usr/local/elasticsearch-***/plugins/ik/
- 注意：如果版本不匹配修改ik中的plugin-descriptor.properties配置
version=8.10.2 #要与elasticSearch版本一致
java.version=20 #要与elasticSearch加载的jdk版本一致
version=8.10.2 #要与elasticSearch版本一致
测试(在linux上)
curl -XPUT http://localhost:9200/index #创建索引
#创建索引映射 text可以分词 key_word不分词 ik_smart：细粒度分词 ik_max_word：粗粒度分词
curl -XPOST http://localhost:9200/index/_mapping -H 'Content-Type:application/json' -d'
{
"properties": {
"content": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_smart"
}
}
}'
#添加若干数据
curl -XPOST http://localhost:9200/index/_create/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'
curl -XPOST http://localhost:9200/index/_create/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'
curl -XPOST http://localhost:9200/index/_create/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'
curl -XPOST http://localhost:9200/index/_create/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'
#查询中国显示高亮
curl -XPOST http://localhost:9200/index/_search -H 'Content-Type:application/json' -d'
{
"query" : { "match" : { "content" : "中国" }},
"highlight" : {
"pre_tags" : ["<tag1>", "<tag2>"],
"post_tags" : ["</tag1>", "</tag2>"],
"fields" : {
"content" : {}
}
}
}
'
八、mysql密码和授权
mysql -u root –p #登录mysql成功后 执行以下操作
set password = password('root'); #修改密码
SHOW VARIABLES LIKE 'validate_password%'; #展示密码当前修改策略
set global validate_password_policy=0; #设置密码安全策略 有三级:0表示low 1表示medium 2表示height
set global validate_password_length=4; #设置密码长度为4位
然后再按照对应的加密策略进行密码修改！
# 允许所有连接访问所有库的所有表(*.*)，连接密码为1234
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '1234' WITH GRANT OPTION;
# 刷新权限
FLUSH PRIVILEGES;
九、安装Logstash（数据导入工具）
安装数据导入工具兼容多个数据库(mysql，oracle)，自动把数据库中的数据导入搜索服务器的索引库中。
下载并上传到虚拟机服务器并解压
mkdir -p /usr/local/logstash
tar -zxvf logstash-****.tar.gz -C /usr/local/logstash/
创建jdbc.conf配置文件在解压包根目录下
touch jdbc.conf
- 配置信息如下：
input {
stdin {
}
jdbc {
# 配置数据库信息
jdbc_conn句文件
statement_filection_string => "jdbc:mysql://localhost:3306/shop?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai"
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
jdbc_user => "root"
jdbc_password => "root@123456"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
jdbc_default_timezone => "Asia/Shanghai"
# 执行 sql 语epath => "/usr/local/logstash/logstash-8.10.2/jdbc.sql"
# 定时字段 各字段含义（由左至右）分、时、天、月、年，全部为*默认含义为每分钟都更
新
schedule => "* * * * *"
# 是否将 sql 中 column 名称转小写
lowercase_column_names => false
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "shop"
# 文档_id，%{goods_id}意思是取查询出来的goods_id的值，并将其映射到es的_id字段中
# 文档_id，%{goodsId}如果是别名，意思是取查询出来的goodsId的值，并将其映射到es的_id字段中
document_id => "%{goodsId}"
}
stdout {
codec => json_lines
}
}
创建jdbc.sql 在解压包根目录下
SELECT
goods_id goodsId, -- 设置别名映射搜索服务器中的字段名
goods_name goodsName,
market_price marketPrice,
original_img originalImg
FROM
t_goods
把mysql-connection-8.0.*.jar包上传到
../logstash-core/lib/jars目录中。
检查jdbc.conf配置文件是否可用
bin/logstash -f /usr/local/logstash/logstash-*.*.*/jdbc.conf -t
创建索引 （索引名为shop）
curl -X PUT http://localhost:9200/shop -H 'Content-Type:application/json' -d'{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
}
}'
创建字段分词器类型映射（设置goodsName字段名类型是
text,使用ik分词器中的细粒度分词ik_max_word,也就是穷尽一切排列组合分词。）
ik_smart：粗粒度分词，比如中华人民共和国国歌，会拆分为中华人民共和国，国歌；
ik_max_word：细粒度分词，比如中华人民共和国国歌，会拆分为中华人民共和国，中华人民，中华，华人，人民共和国，人民，人，民，共和国，共和，和，国国，国歌，会穷尽各种可能的组
curl -XPOST http://localhost:9200/shop/_mapping -H 'Content-Type:application/json' -d'{
"properties": {
"goodsName": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
}
}
}'
执行数据导入,执行时会和es保持联系，一直执行导入数据操作进行数据更新！
cd /usr/local/logstash/logstash-***/
bin/logstash -f /usr/local/logstash/logstash-*** /jdbc.conf #执行数据插入
十、Elasticsearch的JavaAPI
创建Maven基本工程
导入依赖
<!--elasticsearch服务器-->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.17.13</version>
</dependency>
<!--client 客户端包-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.17.13</version>
</dependency>
<!--client 客户端依赖包-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.17.13</version>
</dependency>
编写测试类(操作索引)
package com.zwf;
import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.client.indices.GetIndexResponse;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
/**
* Unit test for simple App.
*/
public class IndexTest {
//默认是Http请求协议
private static final HttpHost[] HTTPS={
new HttpHost("192.168.147.110",9200),
new HttpHost("192.168.147.120",9200),
new HttpHost("192.168.147.130",9200),
};
private RestHighLevelClient client=null;
@Before
public void beforeEnv(){
//版本一致时使用
// client=new RestHighLevelClient(RestClient.builder(HTTPS));
//高版本ElasticSearch 兼容低版本RestHighLevelClient使用，可以防止爆红异常信息。
client= new RestHighLevelClientBuilder(RestClient.builder(HTTPS).build())
.setApiCompatibilityMode(true)
.build();
}
@After
public void afterEnv(){
if (client!=null){
try {
client.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
//创建索引
@Test
public void test() throws IOException {
CreateIndexRequest index = new CreateIndexRequest("client_index");
//发送请求
CreateIndexResponse response = client.indices().create(index, RequestOptions.DEFAULT);
boolean flag = response.isAcknowledged();
//操作状态
System.out.println("操作状态："+flag);
}
//查看索引
@Test
public void getIndex(){
GetIndexRequest request=new GetIndexRequest("client_index");
try {
GetIndexResponse IndexResponse = client.indices().get(request, RequestOptions.DEFAULT);
System.out.println("别名："+IndexResponse.getAliases());
System.out.println("映射："+IndexResponse.getMappings());
System.out.println("时间："+IndexResponse.getSettings());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
//删除索引
@Test
public void deleteIndex() throws IOException {
DeleteIndexRequest del = new DeleteIndexRequest("client_index");
AcknowledgedResponse delete = client.indices().delete(del, RequestOptions.DEFAULT);
boolean b = delete.isAcknowledged();
System.out.println(b);
}
}
编写测试类（操作数据）
package com.zwf;
import org.apache.http.HttpHost;
import org.elasticsearch.action.DocWriteResponse;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.text.Text;
import org.elasticsearch.index.engine.Engine;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.fetch.subphase.highlight.Highlighter;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.xcontent.XContent;
import org.elasticsearch.xcontent.XContentType;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
/**
* @author Mr Zeng
* @version 1.0
* @date 2023-10-05 11:32
*/
public class EsData_CRUD {
//默认是Http请求协议
private static final HttpHost[] HTTPS={
new HttpHost("192.168.147.110",9200),
new HttpHost("192.168.147.120",9200),
new HttpHost("192.168.147.130",9200),
};
private RestHighLevelClient client=null;
@Before
public void beforeEnv(){
client=new RestHighLevelClient(RestClient.builder(HTTPS));
}
@After
public void afterEnv(){
if (client!=null){
try {
client.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
@Test
public void SaveData() throws IOException {
Map<String,Object> jsons=new HashMap<>();
jsons.put("username","password");
jsons.put("age","18");
jsons.put("sex","femal");
IndexRequest index = new IndexRequest("client_index").id("1").source(jsons);
IndexResponse response = client.index(index, RequestOptions.DEFAULT);
DocWriteResponse.Result status = response.getResult();
System.out.println(status.name());
}
@Test
public void QueryData() throws IOException {
//指定索引库和id查询数据
GetRequest index = new GetRequest("client_index", "1");
GetResponse response = client.get(index, RequestOptions.DEFAULT);
System.out.println(response.getSource());
}
@Test
public void UpdateData() throws IOException {
//声明要修改的文档
Map<String,Object> map=new HashMap<>();
map.put("username","zhangsan");
map.put("age","30");
map.put("sex","mal");
//根据索引和id修改数据
UpdateRequest index = new UpdateRequest("client_index","1").doc(map);
UpdateResponse response = client.update(index, RequestOptions.DEFAULT);
DocWriteResponse.Result result = response.getResult();
System.out.println(result.getLowercase());
}
@Test
public void deleteData() throws IOException {
DeleteRequest clientIndex = new DeleteRequest("client_index", "1");
DeleteResponse response = client.delete(clientIndex, RequestOptions.DEFAULT);
System.out.println(response.status());
}
//批量增删改
@Test
public void makeBlur() throws IOException {
BulkRequest request = new BulkRequest();
request.add(
new IndexRequest("client_index").id("1").source(XContentType.JSON,"username","root","passwd","123456","sex","female"),
new IndexRequest("client_index").id("2").source(XContentType.JSON,"username","admin","passwd","1234785","sex","male"),
new IndexRequest("client_index").id("3").source(XContentType.JSON,"username","administor","password","1872694955","sex","female"),
new IndexRequest("client_index").id("4").source(XContentType.JSON,"username","admin","password","789456123","sex","male"));
//批量更新
request.add(new UpdateRequest("client_index","1").doc(XContentType.JSON,"username","black","password","987456","sex","male"));
//批量删除
request.add(new DeleteRequest("client_index","3"));
BulkResponse bulk = client.bulk(request, RequestOptions.DEFAULT);
System.out.println(bulk.hasFailures());
}
//批量查询
@Test
public void queryDoc() throws IOException {
//指定搜 索引库 shop和index两个索引库
SearchRequest searchRequest=new SearchRequest("shop","index");
//构建查询对象
SearchSourceBuilder searchSourceBuilder=new SearchSourceBuilder();
//添加查询条件
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
//执行请求
searchRequest.source(searchSourceBuilder);
SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
//打印记录总数数值
System.out.println(search.getHits().getTotalHits().value);
//查询记录大于10条 默认显示10条 打印每个json对象 只能操作对象
search.getHits().forEach(System.out::println);
System.out.println("$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$");
//获取每个对象的具体数值
//结果值 查询记录大于10条 默认显示10条 可以操作对象的属性值
SearchHit[] hits = search.getHits().getHits();
for (SearchHit hit:hits){
System.out.println(hit); //每个对象
System.out.println(hit.getScore()); //分数
System.out.println(hit.getSourceAsMap());//内容
System.out.println(hit.getIndex()); //索引库名
System.out.println("******************************");
}
}
//匹配查询
@Test
public void matchQuery() throws IOException {
//指定查询的索引库
SearchRequest request = new SearchRequest("index", "shop");
//指定匹配查询条件
SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
//指定查询的关键字
String keyWord="中国移动";
//指定多条件匹配查询 goodName和content字段值进行了ik分词
SearchSourceBuilder query = sourceBuilder.query(QueryBuilders.multiMatchQuery(keyWord, "goodsName", "content"));
request.source(query);
//进行查询操作
SearchResponse search = client.search(request, RequestOptions.DEFAULT);
//获取查询到的总记录数
System.out.println("查询的记录数："+search.getHits().getTotalHits().value);
//获取查询的具体属性值
SearchHit[] hits = search.getHits().getHits();
for (SearchHit hit:hits){
System.out.println(hit.getIndex());
System.out.println(hit.getScore());
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
System.out.println("********************");
Set<Map.Entry<String, Object>> entries = sourceAsMap.entrySet();
for (Map.Entry<String,Object> entry:entries){
System.out.println("查询内容："+entry.getValue());
}
}
}
//分页 排序查询
@Test
public void getResultInfo() throws IOException {
SearchRequest request=new SearchRequest("shop");
SearchSourceBuilder builder=new SearchSourceBuilder();
String keyword="中国移动联通";
//goodsName字段值 进行分词
builder.query(QueryBuilders.multiMatchQuery(keyword, "goodsName"));
//进行score值升序排序
builder.sort(SortBuilders.scoreSort().order(SortOrder.ASC));
//进行分页 从0开始显示30条数据 分页是： 参数一：(currentPage-1)*pageSize 参数二：pageSize
builder.from(0).size(30);
request.source(builder); //构建查询条件
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
System.out.println("查询记录数："+response.getHits().getTotalHits().value);
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit:hits){
System.out.println(hit.getScore());
System.out.println(hit.getSourceAsMap());
System.out.println("*************************");
}
}
//高亮查询
@Test
public void highLightSearch() throws IOException {
//指定查询索引库
SearchRequest request = new SearchRequest("shop");
//构建查询条件 高亮字段名和查询字段名
SearchSourceBuilder builder=new SearchSourceBuilder();
//显示0到29记录数
builder.from(0).size(30);
//构建查询关键字 goodsName进行了ik分词 细粒度分词
String keyWord="中国移动联通";
builder.query(QueryBuilders.multiMatchQuery(keyWord,"goodsName"));
//构建高亮字段查询
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.field("goodsName").preTags("<span color='red'>").postTags("<span/>");
builder.highlighter(highlightBuilder);
//构建根据价格降序排序
builder.sort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC));
request.source(builder);
SearchResponse search = client.search(request, RequestOptions.DEFAULT);
long value = search.getHits().getTotalHits().value;
System.out.println("查询记录数："+value);
SearchHit[] hits = search.getHits().getHits();
for (SearchHit hit:hits){
hit.getSourceAsMap().forEach((s, o) -> {
System.out.println(o);
});
System.out.println("******************");
HighlightField goodsName = hit.getHighlightFields().get("goodsName");
Text text = goodsName.fragments()[0];
System.out.println("高亮字段html文本:"+text.string());
}
}
}
十一、springboot整合elasticSearch
导入依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
编写配置文件
spring:
elasticsearch:
uris: 192.168.147.110:9200,192.168.147.120:9200,192.168.147.130:9200
编写实体类，ORM映射elasticSearch中的索引对象
//createIndex = true默认自定创建索引分片5+5=10， createIndex = false手动创建索引在添加数据时创建索引
//如果在查询时 indexName="shop" 与elasticSearch索引库建立映射查询 如果插入数据时 会插入实体类指定的 索引中，没有索引就创建设置的索引。
@Document(indexName = "shop",shards=5,replicas = 5,createIndex = false)
public class Shop implements Serializable {
@Id
private Integer goodsId;
/**
* 商品名称 对产品名字进行细粒度分词 Text允许分词
*/
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String goodsName;
/**
* 市场价
*/
@Field(type = FieldType.Double)
private BigDecimal marketPrice;
/**
* 商品上传原始图 Keyword 不允许分词
*/
@Field(type = FieldType.Keyword)
private String originalImg;
/**
* t_goods
*/
private static final long serialVersionUID = 1L;
public Integer getGoodsId() {
return goodsId;
}
public void setGoodsId(Integer goodsId) {
this.goodsId = goodsId;
}
public String getGoodsName() {
return goodsName;
}
public void setGoodsName(String goodsName) {
this.goodsName = goodsName == null ? null : goodsName.trim();
}
public BigDecimal getMarketPrice() {
return marketPrice;
}
public void setMarketPrice(BigDecimal marketPrice) {
this.marketPrice = marketPrice;
}
public String getOriginalImg() {
return originalImg;
}
public void setOriginalImg(String originalImg) {
this.originalImg = originalImg == null ? null : originalImg.trim();
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(getClass().getSimpleName());
sb.append(" [");
sb.append("Hash = ").append(hashCode());
sb.append(", goodsId=").append(goodsId);
sb.append(", goodsName=").append(goodsName);
sb.append(", marketPrice=").append(marketPrice);
sb.append(", originalImg=").append(originalImg);
sb.append("]");
return sb.toString();
}
public Shop() {
}
public Shop(Integer goodsId, String goodsName, BigDecimal marketPrice, String originalImg) {
this.goodsId = goodsId;
this.goodsName = goodsName;
this.marketPrice = marketPrice;
this.originalImg = originalImg;
}
}
编写repositry接口类，根据泛型可以生成常用的增删改查的方法
//接口继承 第一个泛型是实体类 第二个泛型是id的数据类型 自动进行映射！
@Repository
public interface EsDataRepositry extends ElasticsearchRepository<Shop, Integer> {
List<Shop> findByGoodsName(String shopName);
/**
* 根据id查询商品
* ？0为占位符
* @param id
* @return
*/
@Query("{\"match\": {\"goodsId\":{ \"query\": \"?0\"}}}")
Shop findByIdValue(Integer id);
}
编写测试类
@SpringBootTest
class SpringBootEsApplicationTests {
//可以自定义方法对数据进行增删改查 跟数据库操作方法类似 注入继承ElasticsearchRepository子接口！
@Autowired
private EsDataRepositry esDataRepositry;
//可以操作索引 匹配查询 复杂查询 显示高亮查询
@Autowired
private ElasticsearchRestTemplate template;
@Test
public void testSave(){
//批量插入
List<Shop> list=new ArrayList<>();
list.add(new Shop(200,"测试数据",new BigDecimal("1563"),"1.jpg"));
list.add(new Shop(210,"测试数据1",new BigDecimal("1780"),"2.jpg"));
list.add(new Shop(220,"测试数据2",new BigDecimal("1630"),"3.jpg"));
try {
esDataRepositry.saveAll(list);
}catch (Exception e){
}
//查询所有
Iterable<Shop> all=esDataRepositry.findAll();
all.forEach(System.out::println);
}
@Test
public void queryByName(){
//根据商品名称查询信息
List<Shop> byName = esDataRepositry.findByGoodsName("%中国%");
byName.forEach(System.out::println);
}
//根据Id查询对象
@Test
public void quryById(){
//配置repository层的自定义方法
System.out.println(esDataRepositry. findByIdValue(100));
}
//创建索引
@Test
public void createIndex(){
//设置实体类索引信息 返回索引操作对象
IndexOperations indexOperations = template.indexOps(Shop.class);
indexOperations.create();
//创建索引映射
Document mapping = indexOperations.createMapping();
//将映射写入索引
indexOperations.putMapping(mapping);
//获取索引
Map<String, Object> map = indexOperations.getMapping();
map.forEach((k,v)->{
System.out.println(k+"--------->"+v);
});
//索引是否存在
boolean exists = indexOperations.exists();
System.out.println(exists);
//删除索引
indexOperations.delete();
}
//操作文档 匹配查询
@Test
public void testDocument(){
//删除指定索引和id 并返回Id
String goodsId = template.delete("200", IndexCoordinates.of("shop1"));
System.out.println(goodsId);
//根据索引名和字段 匹配查询删除
ByQueryResponse delete = template.delete(new NativeSearchQueryBuilder().withQuery(new MatchQueryBuilder("goodsName", "测试数据1")).build(), Shop.class, IndexCoordinates.of("shop1"));
//id存在就更新 不存在就新增数据
List<Shop> list=new ArrayList<>();
list.add(new Shop(200,"测试数据2",new BigDecimal("1213"),"123.jpg"));
list.add(new Shop(210,"测试数据3",new BigDecimal("89642"),"456.jpg"));
template.save(list);
}
@Test
//本地匹配查询
public void testSearchMatch(){
NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
NativeSearchQuery query = queryBuilder.withQuery(QueryBuilders.multiMatchQuery("中国移动联通电信", "goodsName")).build();
SearchHits<Shop> search = template.search(query, Shop.class);
search.forEach(shopSearchHit -> {
System.out.println(shopSearchHit.getContent());
});
}
//测试高亮效果 分页 排序
@Test
public void testHighLight(){
NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
//构建查询条件
NativeSearchQuery query = builder.withQuery(QueryBuilders.multiMatchQuery("中国", "goodsName"))
//根据市场价 降序显示 从0到4记录显示
.withPageable(Pageable.ofSize(5).withPage(0)).withSort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC))
.withHighlightBuilder(new HighlightBuilder().field("goodsName").preTags("<span color='red'").postTags("</span>")).build();
SearchHits<Shop> search = template.search(query, Shop.class);
List<SearchHit<Shop>> searchHits = search.getSearchHits();
searchHits.forEach(System.out::println);
System.out.println("*******************");
for (SearchHit<Shop> searchHit:searchHits){
//结果集
System.out.println(searchHit.getContent());
//高亮信息
System.out.println(searchHit.getHighlightField("goodsName").get(0));
System.out.println("*******************");
}
}
}
springboot解决es-8.10.8高版本兼容低版本java high level client-7.17.13解决方案
- 导入依赖
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.16</version>
<relativePath/>
</parent>
<groupId>com.zwf</groupId>
<artifactId>springBoot-es</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>springBoot-es</name>
<description>springBoot-es</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>4.4.16</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
</exclusion>
<exclusion>
<artifactId>transport-netty4-client</artifactId>
<groupId>org.elasticsearch.plugin</groupId>
</exclusion>
<exclusion>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<groupId>org.elasticsearch.client</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<groupId>org.elasticsearch.client</groupId>
<version>7.17.13</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.17.13</version>
</dependency>
<dependency>
<artifactId>elasticsearch</artifactId>
<groupId>org.elasticsearch</groupId>
<version>7.17.13</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>transport-netty4-client</artifactId>
<version>7.17.13</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>7.17.13</version>
<exclusions>
<exclusion>
<artifactId>elasticsearch</artifactId>
<groupId>org.elasticsearch</groupId>
</exclusion>
<exclusion>
<artifactId>transport-netty4-client</artifactId>
<groupId>org.elasticsearch.plugin</groupId>
</exclusion>
<exclusion>
<artifactId>elasticsearch-rest-client</artifactId>
<groupId>org.elasticsearch.client</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
编写配置类(yml文件不要配置Elasticsearch配置了！)
//高版本ElasticSearch兼容低版本java-client
@Configuration
public class ElasticsearchConfig {
//默认是Http请求协议
private static final HttpHost[] HTTPS={
new HttpHost("192.168.147.110",9200),
new HttpHost("192.168.147.120",9200),
new HttpHost("192.168.147.130",9200),
};
// 使用ES http客户端
@Bean
public RestHighLevelClient restHighLevelClient() {
// IP以及port可以写入到application.yml配置文件中
return new RestHighLevelClientBuilder(RestClient.builder(HTTPS).build())
.setApiCompatibilityMode(true)
.build();
}
// 引入ElasticsearchRestTemplate，方法名和Bean_name必须是elasticsearchTemplate
@Bean
public ElasticsearchRestTemplate elasticsearchTemplate() {
return new ElasticsearchRestTemplate(restHighLevelClient());
}
}
编写启动类（去除自动装配）
@SpringBootApplication(exclude = {ElasticsearchRepositoriesAutoConfiguration.class, ElasticsearchDataAutoConfiguration.class})
//扫描自定义实行继承的 ElasticsearchRepository<Shop,Integer> 子接口包
@EnableElasticsearchRepositories(value ="com.zwf.springbootes.Respositry")
public class SpringBootEsApplication {
public static void main(String[] args) {
SpringApplication.run(SpringBootEsApplication.class, args);
}
}
自定义继承ElasticsearchRepository的接口 （在com.zwf.springbootes.Respositry包下）
@Repository
public interface EsDataRepositry extends ElasticsearchRepository<Shop,Integer> {
List<Shop> findByGoodsName(String shopName);
/**
* 根据id查询商品
* ？0为占位符
* @param id
* @return
*/
@Query("{\"match\": {\"goodsId\":{ \"query\": \"?0\"}}}")
Shop findByIdValue(Integer id);
}
测试代码
@SpringBootTest
class SpringBootEsApplicationTests {
@Autowired
private EsDataRepositry esDataRepositry;
@Resource
private ElasticsearchRestTemplate elasticsearchTemplate;
@Test
public void testSave(){
//批量插入
List<Shop> list=new ArrayList<>();
list.add(new Shop(200,"测试数据",new BigDecimal("1563"),"1.jpg"));
list.add(new Shop(210,"测试数据1",new BigDecimal("1780"),"2.jpg"));
list.add(new Shop(220,"测试数据2",new BigDecimal("1630"),"3.jpg"));
esDataRepositry.saveAll(list);
//查询所有
Iterable<Shop> all=esDataRepositry.findAll();
all.forEach(System.out::println);
}
@Test
public void queryByName(){
//根据商品名称查询信息
List<Shop> byName = esDataRepositry.findByGoodsName("%测试%");
byName.forEach(System.out::println);
}
//根据Id查询对象
@Test
public void quryById(){
//配置repository层的自定义方法
System.out.println(esDataRepositry. findByIdValue(100));
}
// //创建索引
@Test
public void createIndex(){
//设置实体类索引信息 返回索引操作对象
IndexOperations indexOperations = elasticsearchTemplate.indexOps(Shop.class);
indexOperations.create();
//创建索引映射
Document mapping = indexOperations.createMapping();
//将映射写入索引
indexOperations.putMapping(mapping);
//获取索引
Map<String, Object> map = indexOperations.getMapping();
map.forEach((k,v)->{
System.out.println(k+"--------->"+v);
});
//索引是否存在
boolean exists = indexOperations.exists();
System.out.println(exists);
//删除索引
indexOperations.delete();
}
// //操作文档 匹配查询
@Test
public void testDocument(){
// //删除指定索引和id 并返回Id
String goodsId = elasticsearchTemplate.delete("200", IndexCoordinates.of("shop1"));
System.out.println(goodsId);
//根据索引名和字段 匹配查询删除
ByQueryResponse delete = elasticsearchTemplate.delete(new NativeSearchQueryBuilder().withQuery(new MatchQueryBuilder("goodsName", "测试数据1")).build(), Shop.class, IndexCoordinates.of("shop1"));
// //id存在就更新 不存在就新增数据
List<Shop> list=new ArrayList<>();
list.add(new Shop(200,"测试数据2",new BigDecimal("1213"),"123.jpg"));
list.add(new Shop(210,"测试数据3",new BigDecimal("89642"),"456.jpg"));
elasticsearchTemplate.save(list);
}
//
@Test
// //本地匹配查询
public void testSearchMatch(){
NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
NativeSearchQuery query = queryBuilder.withQuery(QueryBuilders.multiMatchQuery("中国移动联通电信", "goodsName")).build();
SearchHits<Shop> search = elasticsearchTemplate.search(query, Shop.class);
search.forEach(shopSearchHit -> {
System.out.println(shopSearchHit.getContent());
});
}
// //测试高亮效果 分页 排序
@Test
public void testHighLight(){
NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
//构建查询条件
NativeSearchQuery query = builder.withQuery(QueryBuilders.multiMatchQuery("中国", "goodsName"))
//根据市场价 降序显示 从0到4记录显示
.withPageable(Pageable.ofSize(5).withPage(0)).withSort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC))
.withHighlightBuilder(new HighlightBuilder().field("goodsName").preTags("<span color='red'").postTags("</span>")).build();
SearchHits<Shop> search = elasticsearchTemplate.search(query, Shop.class);
List<SearchHit<Shop>> searchHits = search.getSearchHits();
searchHits.forEach(System.out::println);
System.out.println("*******************");
for (SearchHit<Shop> searchHit:searchHits){
//结果集
System.out.println(searchHit.getContent());
//高亮信息
System.out.println(searchHit.getHighlightField("goodsName").get(0));
System.out.println("*******************");
}
}
}
本文来自博客园，作者：戴莫先生Study平台，转载请注明原文链接：https://www.cnblogs.com/smallzengstudy/p/17746719.html
