Apache Zeppelinで単純なsparkタスクを実行中に例外が発生しました

Question

Apache Zeppelinで単純なsparkコマンドを実行しようとしているときにInvocationTargetExceptionが発生しました。あなたは2つのタスクが実行しようとしているのを見ることができます。どちらも私に同じ例外を与えます。

ツェッペリンからのスタックトレース：

DEBUG [2017-02-22 10:14:34,351] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - ERROR [2017-02-22 10:14:34,297] ({pool-2-thread-5} Utils.java[invokeMethod]:40) - 
DEBUG [2017-02-22 10:14:34,353] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - java.lang.reflect.InvocationTargetException 
DEBUG [2017-02-22 10:14:34,354] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
DEBUG [2017-02-22 10:14:34,356] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
DEBUG [2017-02-22 10:14:34,357] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
DEBUG [2017-02-22 10:14:34,358] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.lang.reflect.Method.invoke(Method.java:498) 
DEBUG [2017-02-22 10:14:34,361] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38) 
DEBUG [2017-02-22 10:14:34,361] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33) 
DEBUG [2017-02-22 10:14:34,365] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:350) 
DEBUG [2017-02-22 10:14:34,368] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:222) 
DEBUG [2017-02-22 10:14:34,368] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:815) 
DEBUG [2017-02-22 10:14:34,371] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) 
DEBUG [2017-02-22 10:14:34,371] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482) 
DEBUG [2017-02-22 10:14:34,375] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.scheduler.Job.run(Job.java:175) 
DEBUG [2017-02-22 10:14:34,375] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) 
DEBUG [2017-02-22 10:14:34,377] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
DEBUG [2017-02-22 10:14:34,378] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
DEBUG [2017-02-22 10:14:34,380] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 
DEBUG [2017-02-22 10:14:34,381] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 
DEBUG [2017-02-22 10:14:34,381] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
DEBUG [2017-02-22 10:14:34,385] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
DEBUG [2017-02-22 10:14:34,386] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.lang.Thread.run(Thread.java:745) 
DEBUG [2017-02-22 10:14:34,387] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - Caused by: java.lang.NullPointerException 
DEBUG [2017-02-22 10:14:34,387] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012) 
DEBUG [2017-02-22 10:14:34,389] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.hadoop.util.Shell.runCommand(Shell.java:482) 
DEBUG [2017-02-22 10:14:34,389] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.hadoop.util.Shell.run(Shell.java:455) 
DEBUG [2017-02-22 10:14:34,389] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) 
DEBUG [2017-02-22 10:14:34,391] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:873) 
DEBUG [2017-02-22 10:14:34,392] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.hadoop.fs.FileUtil.chmod(FileUtil.java:853) 
DEBUG [2017-02-22 10:14:34,394] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.util.Utils$.fetchFile(Utils.scala:475) 
DEBUG [2017-02-22 10:14:34,394] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext.addFile(SparkContext.scala:1477) 
DEBUG [2017-02-22 10:14:34,395] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext.addFile(SparkContext.scala:1425) 
DEBUG [2017-02-22 10:14:34,397] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:470) 
DEBUG [2017-02-22 10:14:34,398] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext$$anonfun$12.apply(SparkContext.scala:470) 
DEBUG [2017-02-22 10:14:34,400] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at scala.collection.immutable.List.foreach(List.scala:381) 
DEBUG [2017-02-22 10:14:34,401] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext.<init>(SparkContext.scala:470) 
DEBUG [2017-02-22 10:14:34,401] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) 
DEBUG [2017-02-22 10:14:34,403] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868) 
INFO [2017-02-22 10:14:34,403] ({pool-2-thread-3} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1487752190357_-397170656 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session969841713 
DEBUG [2017-02-22 10:14:34,404] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860) 
DEBUG [2017-02-22 10:14:34,407] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at scala.Option.getOrElse(Option.scala:121) 
DEBUG [2017-02-22 10:14:34,407] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860) 
DEBUG [2017-02-22 10:14:34,409] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - ... 20 more 
DEBUG [2017-02-22 10:14:34,409] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - INFO [2017-02-22 10:14:34,319] ({pool-2-thread-5} SparkInterpreter.java[createSparkSession]:351) - Created Spark session with Hive support 
DEBUG [2017-02-22 10:14:34,410] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - ERROR [2017-02-22 10:14:34,319] ({pool-2-thread-5} Job.java[run]:181) - Job failed 
DEBUG [2017-02-22 10:14:34,412] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - java.lang.NullPointerException 
DEBUG [2017-02-22 10:14:34,412] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38) 
DEBUG [2017-02-22 10:14:34,412] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33) 
DEBUG [2017-02-22 10:14:34,414] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:380) 
DEBUG [2017-02-22 10:14:34,415] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:369) 
DEBUG [2017-02-22 10:14:34,415] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:144) 
DEBUG [2017-02-22 10:14:34,417] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:817) 
DEBUG [2017-02-22 10:14:34,418] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) 
DEBUG [2017-02-22 10:14:34,420] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482) 
DEBUG [2017-02-22 10:14:34,423] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.scheduler.Job.run(Job.java:175) 
DEBUG [2017-02-22 10:14:34,423] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) 
DEBUG [2017-02-22 10:14:34,425] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
DEBUG [2017-02-22 10:14:34,425] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
DEBUG [2017-02-22 10:14:34,427] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 
DEBUG [2017-02-22 10:14:34,428] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 
DEBUG [2017-02-22 10:14:34,428] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
DEBUG [2017-02-22 10:14:34,430] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
DEBUG [2017-02-22 10:14:34,431] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - at java.lang.Thread.run(Thread.java:745) 
DEBUG [2017-02-22 10:14:34,431] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - INFO [2017-02-22 10:14:34,320] ({pool-2-thread-5} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1487754873203 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter1927937575 
DEBUG [2017-02-22 10:14:34,433] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - DEBUG [2017-02-22 10:14:34,321] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.spark.concurrentSQL, value: false 
DEBUG [2017-02-22 10:14:34,433] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:189) - DEBUG [2017-02-22 10:14:34,321] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.spark.concurrentSQL, value: false 

私はスパーク "火花2.1.0-ビンhadoop2.7"、Hadoopの "Hadoopの-2.7.3" およびツェッペリン「ツェッペリン-0.7を使用しています.0-bin-all "と入力します。

Answer 1

あなたがHadoopのためツェッペリンを構築する必要が2.7

• ツェッペリン0.7.0-ビン全てが火花2.1.0とHadoopの2.6
• ツェッペリン0.7.0リリーススクリプトのために構築されています：https://github.com/apache/zeppelin/blob/branch-0.7/dev/create_release.sh#L109

ここで

はbuilt from source

# make sure that you are using scala 2.11.x, maven 3.1.x+ and JDK 1.7+ 
# also, you need nodejs 6.9.1+ (will be removed from build requirements in 0.7.1) 

$ ./dev/change_scala_version.sh 2.11 
$ mvn clean package -DskipTests -Pspark-2.1 -Phadoop-2.7 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11 

私はこの答えをチェックしようとしているが、