随笔- 575 文章- 31 评论- 37 阅读- 60万

9: SQL client kafka demo

1: Flink: flink-1.11.2-bin-scala_2.11.tgz 需要包含的jars:

2: kafka: （安裝流程参考：https://www.cnblogs.com/liufei1983/p/15706019.html）

　　docker pull wurstmeister/kafka:2.12-2.8.1
　　docker pull wurstmeister/zookeeper:3.7.0

3：docker安装flink的流程参考 https://www.cnblogs.com/liufei1983/p/15705999.html，

但是compose文件替换成如下的，主要是替换版本，和挂载。

version: "2.1"
services:
  jobmanager:
    image: flink:1.11.2-scala_2.11
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    volumes:
      - C:/flink-1.11.2/lib:/opt/flink/lib
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
 
  taskmanager:
    image: flink:1.11.2-scala_2.11
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    volumes:
      - C:/flink-1.11.2/lib:/opt/flink/lib
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

4： SQL-client:

CREATE TABLE CustomerStatusChangedEvent(
request_time BIGINT,
response_size BIGINT
) with (
'connector.type' = 'kafka',
'connector.version' = 'universal',
'connector.properties.group.id' = 'group1',
'connector.properties.bootstrap.servers' = '192.168.5.6:9092',
'connector.properties.zookeeper.connect' = '192.168.5.6:2181',
'connector.topic' = 'cdn_events',
'connector.startup-mode' = 'earliest-offset',
'format.type' = 'json');