Alamps 沁园春

摘要： package dataframe import org.apache.spark.sql.{DataFrame, Dataset, SparkSession} //// Explore interoperability between DataFrame and Dataset. Note tha 阅读全文

摘要： Hadoop、Storm系统和组件接口对比表： package storm; import org.apache.storm.Config;import org.apache.storm.StormSubmitter;import org.apache.storm.generated.StormTo 阅读全文

摘要： [root@alamps TestDir]# hadoop fs -chown hadoop:hadoop hdfs://alamps:9000/library/SparkStreaming/data/[root@alamps TestDir]# hadoop fs -chown hadoop:ha 阅读全文

摘要： 1: Exception in thread "main" org.apache.spark.SparkException: org.apache.spark.streaming.dstream.MappedDStream@5a69b104 has not been initialized 原因： StreamingContext.checkpoint(...)指定的checkpoint目录下... 阅读全文

摘要： package stream.scala import java.io.PrintWriter import java.net.ServerSocket class LoggerSimulation { } object LoggerSimulation{ /** * 生成一个字母 * @param 字母的下标 * @return 生成的字母 ... 阅读全文

摘要： package sql import org.apache.spark.sql.SparkSession import org.apache.spark.SparkContext object Parquet extends App { val spark = SparkSession .builder() .appName("Spark SQL basic exampl... 阅读全文

摘要： import java.util.concurrent.CountDownLatch; import java.util.concurrent.CyclicBarrier; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; public class CountdownLatch... 阅读全文

摘要： package sql; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaSparkContext; import org.apache.spark.sql.DataFrame; import org.apache.spark.sql.SQLContext; /** * */ public... 阅读全文

摘要： package core.java; import java.util.Arrays; import java.util.List; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaPairRDD; import org.apache.spark.api.java.JavaSparkContext;... 阅读全文

摘要： #移动数据 UPDATE dcs_organize_user AS a, dcs_organize_user AS b SET a.SHORTTELNO=b.USERTELNO WHERE a.id=b.id; #字段加前缀 UPDATE dcs_organize_user SET USERTELNO=CONCAT('0038089819',USERTELNO); 阅读全文