build jar(sbt)

project 中遇到

  1. log file & project file
  2. build.sbt
  3. startup.sh
  4. then command line:　　> sh startup.sh

example:

.sbt

name := "Simple Project" //project name
version := "1.0"  //project version
scalaVersion := "2.11.8" //scala version
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming-kafka-0-8_2.11" % "2.1.0"
libraryDependencies += "org.json4s" %% "json4s-jackson" % "3.2.11"
//all Dependencies should be aware of the version compatibility




.sh




 /usr/local/spark/bin/spark-submit --driver-class-path /usr/local/spark/jars/*:/usr/local/spark/jars/kafka/* --class "org.apache.spark.examples.streaming.KafkaWordCount" /usr/local/spark/mycode/kafka/target/scala-2.11/simple-project_2.11-1.0.jar 127.0.0.1:2181 1 sex 1




spark-submit目录 
参数：


--driver-class-path　　spark中的所有jars以及kafka中所有jars

--class　　工程文件main所在class + 生成的jar的位置


127.0.0.1:2181 ： zookeeper-server kafka-server ip+port

1 sex 1： consumer group；topics； number of consumers

REFERENCE:
http://dblab.xmu.edu.cn




 


 


 







        

        

            posted on 
2017-09-26 23:14 
satyrs 
阅读(129) 
评论(0) 
编辑 
收藏 
举报

        

    










    
    

    
    

    

        

    

        

            
                
                
            
        

    

        

            
                
            
        

    

    

    





            

        

        

            

导航