How to write a function in Pig? - apache-pig

Question

A Pig newbie here. I have a relation A with multiple fields (f1,f2...). I want to quickly see all the distinct values that are there in each field.
Right now, I am doing this:
f1 = FOREACH A GENERATE f1;
f1 = DISTINCT f1;
dump f1;
I don't want to do this for each field. It is too elaborate. Is it possible instead to write some kind of a function in Pig to do this. I've looked at UDFs in the documentation, but I don't want to switch to another language like Java or Python. I think Pig is fine for what I am doing.

Answer 1

What you are looking for is a Macro. This is the equivalent to a function.
DEFINE MY_MACRO(relation,field) RETURNS selected_field_distinct {
selected_field = FOREACH $relation GENERATE $field;
$selected_field_distinct = DISTINCT selected_field;
};
A = LOAD 'input.txt' USING PigStorage(',') AS (f1:chararray, f2:chararray);
F1 = MY_MACRO(A,'f1');
F2 = MY_MACRO(A,'f2');
DUMP F1
DUMP F2
Please note that:
You have to declare the macro above the location where you use it.
You can also write your macro on a different file and import it to your script.
There is no way to use the command DUMP within the macro as described here.

Answer 2

A thought worth contemplating ....
If the values seen in f1 will not occur in f2 then you can try this approach. In this case we are performing DISTINCT only once.
f1 = FOREACH A GENERATE f1;
f2 = FOREACH A GENERATE f2;
...
f10 = FOREACH A GENERATE f10;
all_values = UNION f1,f2,..., f10;
uniq_values = DISTINCT all_values;
DUMP uniq_values;