|NO.Z.00044|——————————|BigDataEnd|——|Hadoop&实时数仓.V24|——|项目.v24|需求二:数据处理&增量统计.V2|——|编程实现|

一、编程实现:工具类:每隔5分钟统计最近1小时内的订单交易情况显示城市/省份/交易总金额/订单总数
### --- 编程实现:工具类一:ConnHBase

package myutils

import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.hbase.{HBaseConfiguration, HConstants}
import org.apache.hadoop.hbase.client.{Connection, ConnectionFactory}

class ConnHBase {
  def connToHbase:Connection ={
    val conf : Configuration = HBaseConfiguration.create()
    conf.set("hbase.zookeeper.quorum","hadoop01,hadoop02,hadoop03")
    //    conf.set("hbase.zookeeper.quorum","hadoop01,hadoop02,hadoop03")
    conf.set("hbase.zookeeper.property.clientPort","2181")
    conf.setInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,30000)
    conf.setInt(HConstants.HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD,30000)
    val connection = ConnectionFactory.createConnection(conf)
    connection
  }

}
### --- 编程实现:工具类二:SourceKafka

package myutils

import java.util.Properties
import org.apache.flink.api.common.serialization.SimpleStringSchema
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer

class SourceKafka {
  def getKafkaSource(topicName: String) : FlinkKafkaConsumer[String] = {
    val props = new Properties()
    props.setProperty("bootstrap.servers","hadoop01:9092,hadoop02:9092,hadoop03:9092");//3,4
    props.setProperty("group.id","consumer-group")
    props.setProperty("key.deserializer","org.apache.kafka.common.serialization.StringDeserializer")
    props.setProperty("value.deserializer","org.apache.kafka.common.serialization.StringDeserializer")
    props.setProperty("auto.offset.reset","la")

    new FlinkKafkaConsumer[String](topicName, new SimpleStringSchema(),props);
  }
}
二、编程实现:样例类
### --- 编程实现:样例类一:CityOrder

package modes

case class CityOrder(city:String, province:String, totalMoney:Double, totalCount:Int)
### --- 编程实现:样例类二:TableObject

package modes

case class TableObject (database:String, tableName:String, typeInfo: String, dataInfo: String) extends Serializable
### --- 编程实现:样例类三:TradeOrder

package dw.dws

case class TradeOrder(orderId:Int,orderNo:String,userId:Int,status:Int,totalMoney:Double,areaId:Int)

 
 
 
 
 
 
 
 
 

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
                                                                                                                                                   ——W.S.Landor

 

 

posted on   yanqi_vip  阅读(39)  评论(0编辑  收藏  举报

相关博文:
阅读排行:
· 无需6万激活码!GitHub神秘组织3小时极速复刻Manus,手把手教你使用OpenManus搭建本
· Manus爆火,是硬核还是营销?
· 终于写完轮子一部分:tcp代理 了,记录一下
· 别再用vector<bool>了!Google高级工程师:这可能是STL最大的设计失误
· 单元测试从入门到精通
< 2025年3月 >
23 24 25 26 27 28 1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31 1 2 3 4 5

导航

统计

点击右上角即可分享
微信分享提示