单机安装部署doris
环境:
OS:Centos 7
doris:1.2.2
1.下载安装介质
https://doris.apache.org/zh-CN/download/
根据实际情况进行下载,我这里下载的是
apache-doris-fe-1.2.2-bin-x86_64.tar.xz
apache-doris-be-1.2.2-bin-x86_64.tar.xz
apache-doris-dependencies-1.2.2-bin-x86_64.tar.xz
2.安装java环境
安装部署请参考
https://www.cnblogs.com/hxlasky/p/14775706.html
3.设置系统最大打开文件句柄数
vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
4.关闭交换分区
swapoff -a ##临时关闭
永久删除Swap挂载
vim /etc/fstab
# 注释 swap 行
重启系统
5.关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl status firewalld.service
###############################安装PE##############################
1.安装fe
tar -xvf apache-doris-fe-1.2.2-bin-x86_64.tar.xz
mv apache-doris-fe-1.2.2-bin-x86_64 /opt/doris-fe
2.修改配置
vi /opt/doris-fe/conf/fe.conf
priority_networks = 192.168.1.0/24 ## 服务器的网段
meta_dir = /opt/doris-fe/data/doris-meta ##元数据存放目录,该目录需要提前创建好
qe_query_timeout_second=3000 ##超时设置，最大只能设置259200秒,即72小时
enable_batch_delete_by_default=true ##启用批量删除
3.启动
[root@host134 bin]# cd /opt/doris-fe/bin
./bin/start_fe.sh --daemon
4.查看 FE 运行状态
你可以通过下面的命令来检查 Doris 是否启动成功
curl http://127.0.0.1:8030/api/bootstrap
通过IE登录界面,账号为root,默认密码为空
http://192.168.1.34:8030
5.连接FE
[root@host134 bin]# /opt/mysql5640/bin/mysql -uroot -P9030 -h127.0.0.1
mysql> show frontends\G;
*************************** 1. row ***************************
Name: 192.168.1.134_9010_1677565195689
IP: 192.168.1.134
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: FOLLOWER
IsMaster: true
ClusterId: 1965633392
Join: true
Alive: true
ReplayedJournalId: 1236
LastHeartbeat: 2023-02-28 15:28:37
IsHelper: true
ErrMsg:
Version: doris-1.2.2-rc01-Unknown
CurrentConnected: Yes
1 row in set (0.03 sec)
ERROR:
No query specified
如果 IsMaster,Join 和 Alive 三列均为true,则表示节点正常.
######################安装BE#############################
1.安装be
tar -xvf apache-doris-be-1.2.2-bin-x86_64.tar.xz
mv apache-doris-be-1.2.2-bin-x86_64 /opt/doris-be
2.修改配置
priority_networks = 192.168.1.0/24
storage_root_path=/path/your/data_dir ##可以使用默认的,默认目录在 BE安装目录的 storage 目录下,BE 配置的存储目录必须先创建好
3.配置 JAVA_HOME 环境变量
以在start_be.sh启动脚本第一行添加export JAVA_HOME=your_java_home_path 来添加环境变量
如下:
export JAVA_HOME=/usr/local/java/jdk1.8.0_361
4.拷贝java udf jar包到be的lib目录
tar -xvf apache-doris-dependencies-1.2.2-bin-x86_64.tar.xz
cp java-udf-jar-with-dependencies.jar /opt/doris-be/lib/
5.启动be
cd /opt/doris-be/bin
./start_be.sh --daemon
[root@host135 bin]# ./start_be.sh --daemon
Please set vm.max_map_count to be 2000000 under root using 'sysctl -w vm.max_map_count=2000000'.
[root@host135 bin]# sysctl -w vm.max_map_count=2000000
vm.max_map_count = 2000000
####################添加BE#################################
1.在fe上添加be并创建库和表
[root@host134 bin]# /opt/mysql5640/bin/mysql -uroot -P9030 -h127.0.0.1
mysql> ALTER SYSTEM ADD BACKEND "192.168.1.134:9050";
Query OK, 0 rows affected (0.13 sec)
创建数据库
create database demo;
建表
use demo;
CREATE TABLE IF NOT EXISTS demo.example_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
查看表结构
show create table example_tbl;
2.导入数据
vi /tmp/mydata.txt
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
这里我们通过Stream load 方式将上面保存到文件中的数据导入到我们刚才创建的表里.
curl --location-trusted -u root: -T /tmp/mydata.txt -H "column_separator:," http://127.0.0.1:8030/api/demo/example_tbl/_stream_load
[root@host134 /]# curl --location-trusted -u root: -T /tmp/mydata.txt -H "column_separator:," http://127.0.0.1:8030/api/demo/example_tbl/_stream_load
{
"TxnId": 2,
"Label": "34740bfc-3c13-46bc-a978-8ee88880c40c",
"TwoPhaseCommit": "false",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 7,
"NumberLoadedRows": 7,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 399,
"LoadTimeMs": 890,
"BeginTxnTimeMs": 68,
"StreamLoadPutTimeMs": 318,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 364,
"CommitAndPublishTimeMs": 135
}
查看
mysql> select * from example_tbl;
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date | city | age | sex | last_visit_date | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10000 | 2017-10-01 | 北京 | 20 | 0 | 2017-10-01 07:00:00 | 35 | 10 | 2 |
| 10001 | 2017-10-01 | 北京 | 30 | 1 | 2017-10-01 17:05:45 | 2 | 22 | 22 |
| 10002 | 2017-10-02 | 上海 | 20 | 1 | 2017-10-02 12:59:12 | 200 | 5 | 5 |
| 10003 | 2017-10-02 | 广州 | 32 | 0 | 2017-10-02 11:20:00 | 30 | 11 | 11 |
| 10004 | 2017-10-01 | 深圳 | 35 | 0 | 2017-10-01 10:00:15 | 100 | 3 | 3 |
| 10004 | 2017-10-03 | 深圳 | 35 | 0 | 2017-10-03 10:20:22 | 11 | 6 | 6 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
6 rows in set (0.17 sec)
3.多个be的情况，按照如下方法添加
在fe上执行
ALTER SYSTEM ADD BACKEND "192.168.1.176:9050";
ALTER SYSTEM ADD BACKEND "192.168.1.177:9050";
ALTER SYSTEM ADD BACKEND "192.168.1.178:9050";
然后查看be情况
mysql> show proc '/backends';
+-----------+-----------------+---------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster | IP | HostName | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+---------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 10003 | default_cluster | 192.168.1.176 | 192.168.1.176 | 9050 | 9060 | 8040 | 8060 | 2023-03-01 10:26:50 | 2023-03-01 10:40:40 | true | false | false | 0 | 0.000 | 1020.098 GB | 1023.499 GB | 0.33 % | 0.33 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-01 10:40:01","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 10004 | default_cluster | 192.168.1.177 | 192.168.1.177 | 9050 | 9060 | 8040 | 8060 | 2023-03-01 10:30:59 | 2023-03-01 10:40:40 | true | false | false | 0 | 0.000 | 1020.102 GB | 1023.499 GB | 0.33 % | 0.33 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-01 10:40:05","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 10005 | default_cluster | 192.168.1.178 | 192.168.1.178 | 9050 | 9060 | 8040 | 8060 | 2023-03-01 10:33:34 | 2023-03-01 10:40:40 | true | false | false | 0 | 0.000 | 1020.102 GB | 1023.499 GB | 0.33 % | 0.33 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-01 10:40:08","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
+-----------+-----------------+---------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
3 rows in set (0.10 sec)
4.修改root账号密码
通过fe登录，执行如下语句修改root密码
mysql> SET PASSWORD FOR 'root' = PASSWORD('mysql');
Query OK, 0 rows affected (0.01 sec)
