时序数据库tdengine部署说明

TDengine 是一款开源、高性能、云原生的时序数据库（Time Series Database, TSDB）。

参考文档: https://docs.taosdata.com/

目录

单节点部署

docker-compose启动

version: '3.9'
services:
    tdengine:
        image: tdengine/tdengine:3.3.4.3
        container_name: tdengine
        ports:
            - '6060:6060' # taosExplorer (web控制台)
            - '6044-6045:6044-6045/udp'
            - '6044-6049:6044-6049' # 三方数据接入端口
            - '6043:6043' # taosKeeper指标监控端口
            - '6041:6041' # taosAdapter Restful API
            - '6030:6030' # taosd(核心服务)
        volumes:
            - './log:/var/log/taos'
            - './data:/var/lib/taos'

容器内包含tdengine的taosd、taosAdapter 、taosKeeper 、taosExplorer 等组件。

连接测试

  • 进入容器

    docker exec -it tdengine bash

  • 测试数据
    使用官方提供的taosBenchmark工具生成测试数据

    taosBenchmark -y

    系统将自动在数据库 test 下创建一张名为 meters的超级表。这张超级表将包含 10,000 张子表，表名从 d0 到 d9999，每张表包含 10,000条记录。每条记录包含 ts（时间戳）、current（电流）、voltage（电压）和 phase（相位）4个字段，每张表还带有 location 和 groupId 两个标签。

  • 进入taos交互模式

    taos
> use test;
> select count(*) from meters;

  • 修改root用户密码

    > alter user root pass '123456';
> exit;

  • 使用用户名密码登录

    taos -uroot -p123456

  • taosExplorer

    使用数据库用户名密码登录taosExplorer (6060)，可以可视化管理数据库。
    taosExplorer

集群部署

集群规划

使用3台服务器，分别启动docker容器，组成集群。

服务器名称 服务器地址 部署组件 端口
node0 192.168.4.211 tdengine 6030,6041,6043,6060,6044-6049,6044-6045/udp
node1 192.168.4.115 tdengine 6030,6041,6043,6060,6044-6049,6044-6045/udp
node2 192.168.0.37 tdengine,nginx 6030,6041,6043,6060,6044-6049,6044-6045/udp，16041，16043，16060

部署过程

初始化配置文件

3台服务器分别执行:

  1. 启动一个临时容器

    docker run --rm -it --name=tdengine tdengine/tdengine:3.3.4.3 taos

  2. 开启另一个终端窗口，拷贝容器内配置文件到当前目录下的conf

    docker cp tdengine:/etc/taos conf

  3. 替换配置文件中默认的主机名(buildkitsandbox)为当前服务器ip(或hostname)

    # 当前服务器ip，获取失败可以手动设置
ip=$(ifconfig $(ip route show default |head -1 |awk '{print $5}') |grep -oP '(?<=inet\s)\d+(\.\d+){3}')
cd conf
sed -i "s/buildkitsandbox/$ip/g" explorer.toml
sed -i "s/buildkitsandbox/$ip/g" taosadapter.toml
sed -i "s/buildkitsandbox/$ip/g" taos.cfg
sed -i "s/buildkitsandbox/$ip/g" taoskeeper.toml

设置firstEp

编辑taos.cfg，将firstEp配置为集群中首个启动的节点地址(3台服务器配置相同)

# The end point of the first dnode in the cluster to be connected to when this dnode or the CLI utility is started
firstEp                   192.168.4.211:6030

启动集群

3台服务器分别启动tdengine，docker-compose配置如下:

version: '3.9'
services:
    tdengine:
        image: tdengine/tdengine:3.3.4.3
        container_name: tdengine
        network_mode: host
        volumes:
            - './log:/var/log/taos'
            - './data:/var/lib/taos'
            - './conf:/etc/taos'

验证

任意一台服务器执行docker exec -it tdengine taos进入交互模式，然后执行show dnodes;指令查看集群中的数据节点，输出如下:

     id      |            endpoint            | vnodes | support_vnodes |    status    |       create_time       |       reboot_time       |              note              |
=============================================================================================================================================================================
           1 | 192.168.4.211:6030             |      2 |             37 | ready        | 2024-11-26 01:57:30.422 | 2024-11-26 02:03:50.247 |                                |
           2 | 192.168.4.115:6030             |      2 |             37 | ready        | 2024-11-26 02:21:51.044 | 2024-11-26 02:21:50.700 |                                |
           3 | 192.168.0.37:6030              |      1 |             37 | ready        | 2024-11-26 02:21:58.352 | 2024-11-26 02:21:59.149 |                                |
Query OK, 3 row(s) in set (0.002399s)

添加管理节点冗余

在创建 TDengine 集群时，首个 dnode 将自动成为集群的 mnode，负责集群的管理和协调工作。为了实现 mnode 的高可用性，后续添加的 dnode 需要手动创建 mnode。

create mnode on dnode 2;

然后执行指令show mnodes;, 输出如下:

     id      |            endpoint            |      role      |   status    |       create_time       |        role_time        |
==================================================================================================================================
           1 | 192.168.4.211:6030             | leader         | ready       | 2024-11-26 01:57:30.440 | 2024-11-26 02:03:50.318 |
           2 | 192.168.4.115:6030             | follower       | ready       | 2024-11-26 02:28:23.321 | 2024-11-26 02:28:27.385 |
Query OK, 2 row(s) in set (0.002358s)

nginx负载均衡部署

taosAdapter 为 TDengine 集群提供 RESTful 和 WebSocket 接入能力，通过nginx代理集群中多个taosAdapter 节点，实现负载均衡。

  1. nginx配置文件

    user root;
worker_processes auto;
# error_log /var/log/nginx_error.log;


events {
        use epoll;
        worker_connections 1024;
}

http {

    access_log off;

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    server {
        listen 6041;
        location ~* {
            proxy_pass http://dbserver;
            proxy_read_timeout 600s;
            proxy_send_timeout 600s;
            proxy_connect_timeout 600s;
            proxy_next_upstream error http_502 non_idempotent;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $http_connection;
        }
    }
    server {
        listen 6043;
        location ~* {
            proxy_pass http://keeper;
            proxy_read_timeout 60s;
            proxy_next_upstream error  http_502 http_500  non_idempotent;
        }
    }

    server {
        listen 6060;
        location ~* {
            proxy_pass http://explorer;
			
			if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';

                add_header 'Access-Control-Allow-Credentials' 'true';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';

                add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';

                add_header 'Access-Control-Max-Age' 86400;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204; break;
            }

            if ($request_method = 'POST') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Credentials' 'true';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
            }
            if ($request_method = 'GET') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Credentials' 'true';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
            }

            proxy_set_header Host      $host:$server_port;
            proxy_set_header X-Real-IP $remote_addr;

            #proxy_http_version 1.1;
            proxy_read_timeout 60s;
            proxy_next_upstream error  http_502 http_500  non_idempotent;
		}
    }
    upstream dbserver {
        least_conn;
        server 192.168.4.211:6041 max_fails=0;
        server 192.168.4.115:6041 max_fails=0;
        server 192.168.0.37:6041 max_fails=0;
    }
    upstream keeper {
        ip_hash;
        server 192.168.4.211:6043 ;
        server 192.168.4.115:6043 ;
        server 192.168.0.37:6043 ;
    }
    upstream explorer{
        ip_hash;
        server 192.168.4.211:6060 ;
        server 192.168.4.115:6060 ;
        server 192.168.0.37:6060 ;
    }
}

  2. docker-compose启动

    version: '3.9'
services:
  nginx:
    restart: always
    image: nginx:1.23.1
    container_name: nginx-tdengine
    ports:
      - 16043:6043
      - 16060:6060
      - 16041:6041
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./logs:/var/log/nginx

  3. 验证
    访问taosAdapter Restful接口，查看响应结果:

    curl -L 'http://192.168.0.37:16041/rest/sql/test' \
-H 'Content-Type: text/plain' \
-H 'Authorization: Basic cm9vdDp0YW9zZGF0YQ==' \
-d 'select * from meters limit 1'

    输出如下:

    {
    "code": 0,
    "column_meta": [
        [
            "ts",
            "TIMESTAMP",
            8
        ],
        [
            "current",
            "FLOAT",
            4
        ],
        [
            "voltage",
            "INT",
            4
        ],
        [
            "phase",
            "FLOAT",
            4
        ],
        [
            "groupid",
            "INT",
            4
        ],
        [
            "location",
            "VARCHAR",
            24
        ]
    ],
    "data": [
        [
            "2017-07-14T02:40:00.000Z",
            6.535898,
            252,
            146.5,
            8,
            "California.SanJose"
        ]
    ],
    "rows": 1
}
posted @ 2024-11-26 14:29  伊丽莎白菜  阅读(326)  评论(0)  编辑  收藏  举报