本地Mac

Spark-Shell

  1. sc
  2. .textFile("/Users/yangyangyang/app/spark-3.0.0-bin-hadoop3.2/bin/input/word.txt")
  3. .flatMap(_.split(" "))
  4. .map((_,1))
  5. .reduceByKey(_+_)
  6. .collect

提交任务到Spark

  1. /Users/yangyangyang/app/spark-3.0.0-bin-hadoop3.2/bin/spark-submit 
  2. --class org.apache.spark.examples.SparkPi 
  3. --master local[2] ../examples/jars/spark-examples_2.12-3.0.0.jar 10

复制

  1. /Users/yangyangyang/app/spark-3.0.0-bin-hadoop3.2/bin/spark-submit --class org.apache.spark.examples.SparkPi --master local[2] ../examples/jars/spark-examples_2.12-3.0.0.jar 10