I have a simple pig script used to import data file.
My data file is located in : /home/fs188
It's a csv file which contains data as :
011958029,00000024,,1,20100209,1
011951228,00000036,,1,20100209,1
011964431,00000814,,1,20100227,1
003526500,00000863,,1,20080122,1
011950864,00001478,,1,20100209,1
011999168,00002495,X0,1,20100331,0
001684881,00002641,,1,19861126,1
001677981,00003165,,1,19861119,1
001677457,00003311,,1,19870114,1
001677161,00003440,,1,19870116,1
002594705,00003475,,1,19870122,1
011958074,00004327,,1,20100210,1
I just want to execute my script pig named PigScript and test it in local mode.
It contains this code :
ENEE_ENR_FILTER = LOAD '/home/fs188/DataExempleUdf.csv' USING PigStorage(',') AS (idt_gcp:chararray,idt_ent_pse:chararray,cd_not:chararray,idc_pse_pci:chararray,da_pram_ett:chararray,idc_cd_not:chararray);
DUMP ENEE_ENR_FILTER;
So I call my script :
pig -x local PigScript.pig
I get this error :
2019-08-07 12:03:14,277 [main] ERROR org.apache.pig.tools.grunt.Grunt
- ERROR 1000: Error during parsing. Encountered " "-x "" at line 1, column 2.
This is weird because I don't have any synthax error
Getting Error:
13:59:17.372 [GatlingSystem-akka.actor.default-dispatcher-7] INFO com.intuit.karate - >> lock acquired, begin callonce: read('CLASSPATH:CreateUsers.feature');
13:59:17.372 [GatlingSystem-akka.actor.default-dispatcher-7] ERROR com.intuit.karate.FileUtils - feature relative path resolution failed: Illegal char <:> at index 9: CL
I tried replacing ":" by "=" but it didn't work for me
have you tried it as classpath:, It is case sensitive.
callonce read('classpath:CreateUsers.feature')
I'm using Hue for PIG scripts on amazon EMR. I am using the declare and default statements as mentioned in the documentation.
I have some %default and %declare statements and it looks like they are
not preprocessed within Hue. Therefore, although the parameters are defined
in my script, the editor keeps popping in a parameter configuration window. If I leave the parameter blank, the job fails with an error.
Sample Script
%declare OUTPUT_FOLDER 'testingOutput01';
ts = LOAD 's3://testbucket1/input/testdata-00000.gz' USING PigStorage('\t');
STORE ts INTO 's3://testbucket1/$OUTPUT_FOLDER' USING PigStorage('\t');
Upon execution, it shows the pop-up window asking for values for OUTPUT_FOLDER. If I leave it blank it fails with the following error:
2015-06-23 20:15:54,908 [main] ERROR org.apache.pig.Main - ERROR 2997:
Encountered IOException. org.apache.pig.tools.parameters.ParseException:
Encountered "<EOF>" at line 1, column 12.
Was expecting one of:
<IDENTIFIER> ...
<OTHER> ...
<LITERAL> ...
<SHELLCMD> ...
Is that the expected behavior? Is this a known issue or am I missing something?
Configuration details:
AMI version:3.7.0
Hadoop distribution:Amazon 2.4.0
Applications:Hive 0.13.1, Pig 0.12.0, Impala 1.2.4, Hue
The same behavior is seen with default instead of declare.
If you need any clarifications then please do comment on this question. I will update it as needed.
Hue does not support %declare with a default statement. It will be fixed with: https://issues.cloudera.org/browse/HUE-2508
The current temporary workaround is to put any value in the popup.
Pig exists with exit code 7 after printing these 3 lines:
2014-07-16 21:57:37,271 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.0-cdh4.6.0 (rexported) compiled Feb 26 2014, 03:01:22
2014-07-16 21:57:37,272 [main] INFO org.apache.pig.Main - Logging error messages to: ..../pig_1405562257268.log
2014-07-16 21:57:37,627 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/sam/.pigbootup not found
what does this mean?
The INFO messages are normal
The only unusual bit is the exit code (7, see above)
The pig_*.log file does not exist
Is this documented somewhere?
EDIT: the problem was eliminated when I removed the semicolon from the end of the %declare line.
go figure...
You may take a look at the return codes in the source code.
The book Programming Pig also contains a list of their meaning in chapter two.
I copy them here for reference:
0 Success
1 Retriable failure
2 Failure
3 Partial failure - Used with multiquery; see “Nonlinear Data Flows”
4 Illegal arguments passed to Pig
5 IOException thrown - Would usually be thrown by a UDF
6 PigException thrown - Usually means a Python UDF raised an exception
7 ParseException thrown (can happen after parsing if variable substitution
is being done)
8 Throwable thrown (an unexpected exception)
inputData = LOAD '$input' AS (line:chararray);
statusLineFilter = FILTER smallData BY (line MATHCES '^.* AppWrite-Dispatcher: Status code: [0-9]+$');
This code, when I run it, yields this error: ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Syntax error, unexpected symbol at or near 'line'
The log file says the exact same thing. I'm at a loss, because the exact same syntax is working in other scripts I've written.
In order to avoid misspelling of key words I recommend you to use an IDE or a Text-Editor like emacs with the pig-mode.el which add syntax highlight ;)