2016-07-15 16 views
0

私はPigバージョン0.16.0でUbuntu 14.04 LTSに疑似分散Hadoop 2.7.2をインストールしたPigラーニングを開始しました。以下のPIGとHadoopのための私の構成です -Hadoop 2.7.2でPig-0.16.0 - エラー1002:エイリアスを保存できません

ファイル:

#=============================================================== 
# Hadoop Variable List 

export JAVA_HOME=/usr/lib/jvm/java-9-oracle 
export HADOOP_INSTALL=/home/hadoop/hadoop 
export PATH=$PATH:$HADOOP_INSTALL/bin 
export PATH=$PATH:$HADOOP_INSTALL/sbin 
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL 
export HADOOP_COMMON_HOME=$HADOOP_INSTALL 
export HADOOP_HDFS_HOME=$HADOOP_INSTALL 
export HADOOP_HOME=$HADOOP_INSTALL 
export YARN_HOME=$HADOOP_INSTALL 
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native 
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib/native" 

#=============================================================== 
# PIG variable 
export PIG_HOME="/home/hadoop/pig" 
export PIG_INSTALL="$PIG_HOME" 
export PIG_CONF_DIR="$PIG_HOME/conf" 
export PIG_CLASSPATH="$HADOOP_INSTALL/conf" 
export HADOOPDIR="$HADOOP_INSTALL/conf" 
export PATH="$PIG_HOME/bin:$PATH" 

=======================

の.bashrc

と次は私があまりにも

[email protected]:~$ hadoop fs -ls -R /user/hadoop 
-rw-r--r-- 1 hadoop supergroup  540117 2016-07-15 12:48 /user/hadoop/myfile.txt 
HDFSにこのファイルをコピーし

-rw-rw-r-- 1 hadoop hadoop 540117 Jul 15 12:41 myfile.txt 
[email protected]:~$ pwd 
/home/hadoop 

ディレクトリそこから私が実行していブタであります

今、Gruntシェルで次のコマンドを実行すると、エラーが発生します。

grunt> a = load 'myfile.txt' as line; 
grunt> store a into 'c.out'; 

2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a 
2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],a[-1,-1] C: R: 
2016-07-15 12:56:38,684 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
2016-07-15 12:56:38,685 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:57:25,722 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1468556821972_0006 has failed! Stop running all dependent jobs 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 
2016-07-15 12:57:25,726 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,786 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,839 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed! 
2016-07-15 12:57:25,841 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: 

HadoopVersion PigVersion UserId StartedAt FinishedAt Features 
2.7.2 0.16.0 hadoop 2016-07-15 12:56:36 2016-07-15 12:57:25 UNKNOWN 

Failed! 

Failed Jobs: 
JobId Alias Feature Message Outputs 
job_1468556821972_0006 a MAP_ONLY Message: Job failed!  hdfs://localhost:9001/user/hadoop/c.out, 

Input(s): 
Failed to read data from "hdfs://localhost:9001/user/hadoop/myfile.txt" 

Output(s): 
Failed to produce result in "hdfs://localhost:9001/user/hadoop/c.out" 

Counters: 
Total records written : 0 
Total bytes written : 0 
Spillable Memory Manager spill count : 0 
Total bags proactively spilled: 0 
Total records proactively spilled: 0 

Job DAG: 
job_1468556821972_0006 


2016-07-15 12:57:25,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! 

私はただのMapReduceモードでローカルモードでPIGを実行しないことにより、他の手段によってそれを解決しようとしたが、何も動作しているようだありません。これらの単純な2つのコマンドが失敗するたびに

メッセージ

2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a 
2016-07-15 12:56:38,670 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],a[-1,-1] C: R: 
2016-07-15 12:56:38,684 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
2016-07-15 12:56:38,685 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete 
2016-07-15 12:56:53,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1468556821972_0006] 
2016-07-15 12:57:25,722 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1468556821972_0006 has failed! Stop running all dependent jobs 
2016-07-15 12:57:25,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete 
2016-07-15 12:57:25,726 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,786 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032 
2016-07-15 12:57:25,839 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed! 
2016-07-15 12:57:25,841 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: 

HadoopVersion PigVersion UserId StartedAt FinishedAt Features 
2.7.2 0.16.0 hadoop 2016-07-15 12:56:36 2016-07-15 12:57:25 UNKNOWN 

Failed! 

Failed Jobs: 
JobId Alias Feature Message Outputs 
job_1468556821972_0006 a MAP_ONLY Message: Job failed!  hdfs://localhost:9001/user/hadoop/c.out, 

Input(s): 
Failed to read data from "hdfs://localhost:9001/user/hadoop/myfile.txt" 

Output(s): 
Failed to produce result in "hdfs://localhost:9001/user/hadoop/c.out" 

Counters: 
Total records written : 0 
Total bytes written : 0 
Spillable Memory Manager spill count : 0 
Total bags proactively spilled: 0 
Total records proactively spilled: 0 

Job DAG: 
job_1468556821972_0006 


2016-07-15 12:57:25,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! 

を次のエラー・ログ・ファイルのプリントは、あなたの助けをリクエスト!

答えて

0

ロード先のフィールドの絶対パスとデータ型を指定します。

a = load 'hdfs://localhost:9001/user/hadoop/myfile.txt' AS (line:chararray); 
store a into 'hdfs://localhost:9001/user/hadoop/c.out'; 
+0

ありがとうございます。私はすべてのhadoopファイルと豚ファイルを再インストールする必要がありました。私がそれをしたときにあなたのコメントと一緒に問題を解決しました。ありがとう! –

関連する問題