Hudi-Flink SQL实时读取Hudi表数据
代码如下(hudi表实时写入参考上一篇[Hudi-Flink消费kafka将增量数据实时写入Hudi])
package com.zhen.hudi; import org.apache.flink.table.api.EnvironmentSettings; import org.apache.flink.table.api.TableEnvironment; /** * @Author FengZhen * @Date 3/10/22 8:33 PM * @Description 基于Flink SQL Connector实现:从hudi表中加载数据,编写SQL查询 */ public class FlinkSQLReadDemo { public static void main(String[] args) { //1.获取表的执行环境 EnvironmentSettings settings = EnvironmentSettings .newInstance() .inStreamingMode() .build(); TableEnvironment tableEnv = TableEnvironment.create(settings); //2.创建输入表,TODO:加载hudi表数据 tableEnv.executeSql( "CREATE TABLE order_hudi(\n" + " `orderId` STRING PRIMARY KEY NOT ENFORCED,\n" + " `userId` STRING,\n" + " `orderTime` STRING,\n" + " `ip` STRING,\n" + " `orderMoney` DOUBLE,\n" + " `orderStatus` INT,\n" + " `ts` STRING,\n" + " `partition_day` STRING\n" + ")\n" + "PARTITIONED BY (partition_day)\n" + "WITH(\n" + " 'connector' = 'hudi',\n" + " 'path'='hdfs://localhost:9000/hudi-warehouse/flink_hudi_order',\n" + " 'table.type' = 'MERGE_ON_READ',\n" + " 'read.streaming.enabled' = 'true',\n" + " 'read.streaming.check-interval' = '4'\n" + ")" ); //3.执行查询语句,流式读取hudi表数据 tableEnv.executeSql( "SELECT orderId, userId, orderTime, ip, orderMoney, orderStatus, ts, partition_day FROM order_hudi" ).print(); } }
分类:
Hudi
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 分享4款.NET开源、免费、实用的商城系统
· 全程不用写代码,我用AI程序员写了一个飞机大战
· MongoDB 8.0这个新功能碉堡了,比商业数据库还牛
· 白话解读 Dapr 1.15:你的「微服务管家」又秀新绝活了
· 记一次.NET内存居高不下排查解决与启示
2017-03-10 java 获取某路径下的子文件/子路径