即时查询工具| Druid
Druid是一个快速的列式分布式的支持实时分析的数据存储系统，在处理PB级别数据、毫秒级查询、数据实时处理方面，比传统的OLAP系统有了显著的性能改进。
Druid数据结构
与Druid架构相辅相成的是其基于DataSource与Segment的数据结构，它们共同成就了Druid的高性能优势。
Druid安装（单机版）
从https://imply.io/get-started 下载最新版本安装包
安装部署
imply集成了Druid，提供了Druid从部署到配置到各种可视化工具的完整的解决方案，imply有点类似于我们之前学过的Cloudera Manager
1.解压
tar -zxvf imply-2.7.10.tar.gz -C /opt/module
目录说明如下：
- bin/ - run scripts for included software.
- conf/ - template configurations for a clustered setup.
- conf-quickstart/* - configurations for the single-machine quickstart.
- dist/ - all included software.
- quickstart/ - files related to the single-machine quickstart.
2.修改配置文件
1）修改Druid的ZK配置
[kris@hadoop101 _common]$ vim /opt/module/imply/conf/druid/_common/common.runtime.properties
druid.zk.service.host=hadoop101:2181,hadoop102:2181,hadoop103:2181
2）修改启动命令参数，使其不校验不启动内置ZK
[kris@hadoop101 supervise]$ vim /opt/module/imply/conf/supervise/quickstart.conf
:verify bin/verify-java
#:verify bin/verify-default-ports
#:verify bin/verify-version-check
:kill-timeout 10
#!p10 zk bin/run-zk conf-quickstart
3.启动
1）启动zookeeper
2）启动imply
[kris@hadoop101 imply-2.7.10]$ bin/supervise -c conf/supervise/quickstart.conf
[Fri Jan 3 13:37:51 2020] Running command[coordinator], logging to[/opt/module/imply-2.7.10/var/sv/coordinator.log]: bin/run-druid coordinator conf-quickstart
[Fri Jan 3 13:37:51 2020] Running command[broker], logging to[/opt/module/imply-2.7.10/var/sv/broker.log]: bin/run-druid broker conf-quickstart
[Fri Jan 3 13:37:51 2020] Running command[historical], logging to[/opt/module/imply-2.7.10/var/sv/historical.log]: bin/run-druid historical conf-quickstart
[Fri Jan 3 13:37:51 2020] Running command[overlord], logging to[/opt/module/imply-2.7.10/var/sv/overlord.log]: bin/run-druid overlord conf-quickstart
[Fri Jan 3 13:37:51 2020] Running command[middleManager], logging to[/opt/module/imply-2.7.10/var/sv/middleManager.log]: bin/run-druid middleManager conf-quickstart
[Fri Jan 3 13:37:51 2020] Running command[imply-ui], logging to[/opt/module/imply-2.7.10/var/sv/imply-ui.log]: bin/run-imply-ui-quickstart conf-quickstart
说明：每启动一个服务均会打印出一条日志。可以通过/opt/module/imply-2.7.10/var/sv/查看服务启动时的日志信息
[kris@hadoop101 sv]$ ll
总用量 904
-rw-rw-r-- 1 kris kris 77528 1月 3 13:38 broker.log
-rw-rw-r-- 1 kris kris 588001 1月 3 14:07 coordinator.log
-rw-rw-r-- 1 kris kris 76857 1月 3 13:38 historical.log
-rw-rw-r-- 1 kris kris 11599 1月 3 14:06 imply-ui.log
-rw-rw-r-- 1 kris kris 70725 1月 3 13:38 middleManager.log
-rw-rw-r-- 1 kris kris 93658 1月 3 14:07 overlord.log
查看端口号9095的启动情况：
[kris@hadoop101 sv]$ netstat -anp | grep 9095
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 :::9095 :::* LISTEN 6118/imply-ui-linux
tcp 0 0 ::ffff:192.168.1.101:9095 ::ffff:192.168.1.1:54884 TIME_WAIT -
tcp 0 0 ::ffff:192.168.1.101:9095 ::ffff:192.168.1.1:54883 TIME_WAIT -
登录hadoop101:9095查看
停止服务
按Ctrl + c中断监督进程，如果想中断服务后进行干净的启动，请删除/opt/module/imply-2.7.10/var/目录。
从hadoop加载数据
加载数据
批量摄取维基百科样本数据，文件位于quickstart/wikipedia-2016-06-27-sampled.json。使用quickstart/wikipedia-index-hadoop.json 摄取任务文件。
bin/post-index-task --file quickstart/wikipedia-index-hadoop.json
此命令将启动Druid Hadoop摄取任务。 摄取任务完成后，数据将由历史节点加载，并可在一两分钟内进行查询。
[kris@hadoop101 imply]$ bin/post-index-task --file quickstart/wikipedia-index-hadoop.json
Beginning indexing data for wikipedia
Task started: index_hadoop_wikipedia_2020-02-08T09:25:37.204Z
Task log: http://localhost:8090/druid/indexer/v1/task/index_hadoop_wikipedia_2020-02-08T09:25:37.204Z/log
Task status: http://localhost:8090/druid/indexer/v1/task/index_hadoop_wikipedia_2020-02-08T09:25:37.204Z/status
Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...
Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...
Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...
Task index_hadoop_wikipedia_2020-02-08T09:25:37.204Z still running...
Task finished with status: SUCCESS
Completed indexing data for wikipedia. Now loading indexed data onto the cluster...
wikipedia loading complete! You may now query your data
SQL查询
select * from wikipedia
[kris@hadoop101 imply]$ curl -XPOST -H'Content-Type: application/json' -d @quickstart/wikipedia-kafka-supervisor.json http://hadoop101:8090/druid/indexer/v1/supervisor
{"id":"wikipedia-kafka"}
加载实时数据：
[kris@hadoop101 imply]$ curl -O https://static.imply.io/quickstart/wikiticker-0.4.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 30.0M 100 30.0M 0 0 535k 0 0:00:57 0:00:57 --:--:-- 623k
