IN operator in Apache Pig - apache-pig

Question

Is there an IN operator equivalent for Apache Pig? I'm currentl using Apache Pig 0.10.0
I want to do something similar to this:
select count(distinct(o.order_id)),count(od.prod_id),count(od.prod_id)/count(distinct(o.order_id))
from orders o
inner join order_details od
on od.order_id=o.order_id
where o.order_id
in (
select *
from (select o.order_id
from orders o
inner join order_details od
on od.order_id = o.order_id
where(o.order_date between '2013-05-01' and '2013-05-31') and (od.prod_id=1274348)
) as subq
);

Answer 1

This is probably the equivalent script in Pig. You can create as many interim relations as you like to get just the data you need before generating the counts. Note I've treated dates as timestamps; you could use the built-in ToDate UDF which can convert between UNIX timestamps or dates as chararrays to native Pig DateTime types.
-- Load in all of your data
-- Replace with actual paths
-- You may need to supply a delimiter value
o = LOAD 'orders' USING PigStorage() AS (
order_date:long,
order_id:chararray
);
od = LOAD 'order_details' USING PigStorage() AS (
order_id:chararray,
prod_id:chararray
);
-- Filter like WHERE in SQL
-- Replace 1000 and 2000 with actual timestamps
o_filtered = FILTER o BY order_date <= 2000 AND order_date >= 1000;
od_filtered = FILTER od BY prod_id == '1274348';
-- Inner join - only needed once in Pig
subq = JOIN o_filtered BY order_id, od_filtered BY order_id;
-- Drop fields not needed for final counts
subq_renamed = FOREACH subq GENERATE
o_filtered::order_id AS order_id,
od_filtered::prod_id AS prod_id;
-- To do the counts, need to group the data
subq_counts = FOREACH (GROUP subq_renamed ALL) {
dist_order_id = DISTINCT subq_renamed.order_id;
GENERATE
COUNT(dist_order_id) AS dist_order_id_count,
COUNT(subq_renamed.prod_id) AS prod_id_count;
}
-- Calculate the ratio count(od.prod_id)/count(distinct(o.order_id))
final_counts = FOREACH subq_counts GENERATE *,
(float)prod_id_count/dist_order_id_count AS prod_order_ratio;