自分のマシンでpySparkを実行しようとしたときにエラーが発生しました

Question

Sparkを自分のMac（バージョン10.11.6）で実行してみました。私はSpark 2.0.0をダウンロードし、./bin/pysparkを実行しようとしました。

しかし、私は次のエラーを取得しています：

Python 2.7.12 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:43:17) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin 
Type "help", "copyright", "credits" or "license" for more information. 
Anaconda is brought to you by Continuum Analytics. 
Please check out: http://continuum.io/thanks and https://anaconda.org 
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 
Setting default log level to "WARN". 
To adjust logging level use sc.setLogLevel(newLevel). 
16/09/13 15:27:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
16/09/13 15:27:47 ERROR SparkContext: Error initializing SparkContext. 
java.net.UnknownHostException: huey: huey: nodename nor servname provided, or not known 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1473) 
    at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:846) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:839) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:839) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.util.Utils$.localHostName(Utils.scala:896) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:388) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:240) 
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
    at py4j.Gateway.invoke(Gateway.java:236) 
    at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
    at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
    at py4j.GatewayConnection.run(GatewayConnection.java:211) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.net.UnknownHostException: huey: nodename nor servname provided, or not known 
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) 
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) 
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1469) 
    ... 20 more 
16/09/13 15:27:47 WARN SparkContext: Another SparkContext is being constructed (or threw an exception in its constructor). This may indicate an error, since only one SparkContext may be running in this JVM (see SPARK-2243). The other SparkContext was created at: 
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:240) 
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
py4j.Gateway.invoke(Gateway.java:236) 
py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
py4j.GatewayConnection.run(GatewayConnection.java:211) 
java.lang.Thread.run(Thread.java:745) 
16/09/13 15:27:47 ERROR SparkContext: Error initializing SparkContext. 
java.net.UnknownHostException: huey: huey: nodename nor servname provided, or not known 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1473) 
    at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:846) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:839) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:839) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.util.Utils$.localHostName(Utils.scala:896) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:388) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:240) 
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
    at py4j.Gateway.invoke(Gateway.java:236) 
    at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
    at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
    at py4j.GatewayConnection.run(GatewayConnection.java:211) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.net.UnknownHostException: huey: nodename nor servname provided, or not known 
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) 
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) 
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1469) 
    ... 20 more 
Traceback (most recent call last): 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/shell.py", line 47, in <module> 
    spark = SparkSession.builder.getOrCreate() 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/sql/session.py", line 169, in getOrCreate 
    sc = SparkContext.getOrCreate(sparkConf) 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py", line 294, in getOrCreate 
    SparkContext(conf=conf or SparkConf()) 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py", line 115, in __init__ 
    conf, jsc, profiler_cls) 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py", line 168, in _do_init 
    self._jsc = jsc or self._initialize_context(self._conf._jconf) 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/pyspark/context.py", line 233, in _initialize_context 
    return self._jvm.JavaSparkContext(jconf) 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/lib/py4j-0.10.1-src.zip/py4j/java_gateway.py", line 1183, in __call__ 
    File "/Users/hkwik/Downloads/spark-2.0.0-bin-hadoop2.7/python/lib/py4j-0.10.1-src.zip/py4j/protocol.py", line 312, in get_return_value 
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. 
: java.net.UnknownHostException: huey: huey: nodename nor servname provided, or not known 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1473) 
    at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:846) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:839) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:839) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:896) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.util.Utils$.localHostName(Utils.scala:896) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:388) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:240) 
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
    at py4j.Gateway.invoke(Gateway.java:236) 
    at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
    at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
    at py4j.GatewayConnection.run(GatewayConnection.java:211) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.net.UnknownHostException: huey: nodename nor servname provided, or not known 
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) 
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) 
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1469) 
    ... 20 more 

私は無駄にlocalhostにSPARK_LOCAL_IPを設定してみました。何か案は？

Answer 1

/etc/hostsマッピング127.0.0.1にhueyが問題を修正する行を追加したようです。