SAP HANA VORA - Sparkコントローラの問題

Question

Ambariを使用してVARA 1.2にSAP HANA Sparkコントローラをインストールしようとしています。

ただし、Sparkコントローラを起動するときに、以下の例外が発生します。

親切にここに助ける...

[hanaes@ip-172-30-2-218 bin]$ ./hanaes start 
Starting HANA Spark Controller ... 
Class path is /usr/sap/spark/controller/bin/../conf:/usr/hdp/2.3.4.7-4/hadoop/conf:/etc/hive/conf:../*:../lib/*:../lib/external/*:/usr/hdp/2.3.4.7-4/hadoop/*:/usr/hdp/2.3.4.7-4/hadoop/lib/*:/usr/hdp/2.3.4.7-4/hadoop-hdfs/*:/usr/hdp/2.3.4.7-4/hadoop-hdfs/lib/* 
STARTED 
[hanaes@ip-172-30-2-218 bin]$ clear 
[hanaes@ip-172-30-2-218 bin]$ tail -1000f /var/log/hanaes/hana_controller.log 
SLF4J: Class path contains multiple SLF4J bindings. 
SLF4J: Found binding in [jar:file:/usr/sap/spark/controller/lib/external/spark-assembly-1.5.2.2.3.4.7-4-hadoop2.7.1.2.3.4.7-4.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.4.7-4/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 
16/05/09 12:04:43 INFO HanaESConfig: Loaded HANA Extended Store Configuration Found Spark Libraries. Proceeding with Current Class Path 
16/05/09 12:04:44 INFO Server: Starting Spark Controller 
16/05/09 12:04:52 ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master. 
     at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:125) 
     at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:65) 
     at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144) 
     at org.apache.spark.SparkContext.<init>(SparkContext.scala:523) 
     at com.sap.hana.spark.network.CommandRouter.initializeHanaContext(CommandRouter.scala:125) 
     at com.sap.hana.spark.network.CommandRouter.<init>(CommandRouter.scala:38) 
     at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96) 
     at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96) 
     at akka.actor.TypedCreatorFunctionConsumer.produce(Props.scala:343) 
     at akka.actor.Props.newActor(Props.scala:252) 
     at akka.actor.ActorCell.newActor(ActorCell.scala:552) 
     at akka.actor.ActorCell.create(ActorCell.scala:578) 
     at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456) 
     at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) 
     at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263) 
     at akka.dispatch.Mailbox.run(Mailbox.scala:219) 
     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397) 
     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 
     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) 
     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) 
     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 16/05/09 12:04:52 ERROR Utils: Uncaught exception in thread SAPHanaSpark-akka.actor.default-dispatcher-2 java.lang.NullPointerException 
     at org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152) 
     at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228) 
     at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100) 
     at org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749) 
     at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185) 
     at org.apache.spark.SparkContext.stop(SparkContext.scala:1748) 
     at org.apache.spark.SparkContext.<init>(SparkContext.scala:593) 
     at com.sap.hana.spark.network.CommandRouter.initializeHanaContext(CommandRouter.scala:125) 
     at com.sap.hana.spark.network.CommandRouter.<init>(CommandRouter.scala:38) 
     at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96) 
     at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96) 
     at akka.actor.TypedCreatorFunctionConsumer.produce(Props.scala:343) 
     at akka.actor.Props.newActor(Props.scala:252) 
     at akka.actor.ActorCell.newActor(ActorCell.scala:552) 
     at akka.actor.ActorCell.create(ActorCell.scala:578) 
     at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456) 
     at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) 
     at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263) 
     at akka.dispatch.Mailbox.run(Mailbox.scala:219) 
     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397) 
     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 
     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) 
     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) 
     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 

Answer 1

スパークコントローラログは、糸で問題を示し、障害が発生した火花コントローラ仕事を担当して糸のログをチェックする必要があります。

Ambari - >糸 - >クイックリンク - >リソースマネージャーUI - >失敗したSpark Controllerジョブを見つける - >左側のアプリケーションIDをクリック - >ログをクリックする