spark - 随笔分类 - jaysonteng

python生产和消费kafka数据

摘要：安装kafka-python pip install kafka-python 生产者 from kafka import KafkaProducer # 有时候导入包会报错，使用pip uninstall kafka-python，卸载后重装可以解决 import json # 创建produce 阅读全文

posted @ 2020-12-24 09:57 jaysonteng 阅读(2353) 评论(0) 推荐(0) 编辑

spark streaming updatestatebykey scala

摘要：import org.apache.spark.streaming._ import org.apache.spark.SparkConf import org.apache.spark.sql.SparkSession import org.apache.spark.streaming.dstre 阅读全文

posted @ 2020-12-15 13:16 jaysonteng 阅读(90) 评论(0) 推荐(0) 编辑

first spark streaming

摘要：import org.apache.spark.streaming._ import org.apache.spark.SparkConf import org.apache.spark.sql.SparkSession val conf = new SparkConf().setAppName(" 阅读全文

posted @ 2020-12-15 11:31 jaysonteng 阅读(85) 评论(0) 推荐(0) 编辑

随笔分类 - spark