Filter inner bag by other field (not constant) - apache-pig

Question

I am trying to run the following scenario but fail.
I start with a list of movies, and group it by {year, rating}.
movies = LOAD '/movies_data.csv'
USING PigStorage(',') AS (id:int, name:chararray, year:int, rating:double, duration:int);
grouped = GROUP movies BY (year, rating);
The resulting schema is:
DESCRIBE grouped;
grouped: {group: (year: int,rating: double),movies: {(id: int,name: chararray,
year: int,rating: double,duration: int)}}
Now, for each group I would like to get a list of movie names that contain the year (which is the part of the group name).
So I try the following:
model =
FOREACH grouped {
listNames = DISTINCT movies.name;
listNamesFiltered = FILTER listNames BY name MATCHES group::year;
GENERATE
group.year AS year
,group.rating AS rating
,listNamesFiltered AS listNamesFiltered
,COUNT(listNamesFiltered) AS countNamesFiltered
;};
but fail with the message:
Invalid field projection. Projected field [group::year] does not exist in schema: name:chararray.
Using a constant (like in the following line) works:
listNamesFiltered = FILTER listNames BY name MATCHES '.*2010.*';
results in:
(2010,2.6,{(2010: Moby Dick)},1)
(2010,3.8,{(Saturday Night Live: The 2010s)},1)
Any help will be greatly appreciated.

Answer 1

This seems like it would be a lot easier if you did all the filtering and then did all GROUP/DISTINCT/COUNToperations.
Data:
1 2010: Moby Dick 2010 2.6 128
2 Saturday Night Live: The 2010s 2010 3.8 127
3 2001: A Space Odyssey 2001 4.0 145
4 Forrest Gump 1994 4.9 334
Query:
movies = LOAD 'movie_data.csv' USING PigStorage(',') AS (id:int,
name:chararray, year:int, rating:double, duration:int);
filtered = FILTER movies BY name MATCHES StringConcat('.*', (chararray)year, '.*');
dump filtered;
Output:
(1,2010: Moby Dick,2010,2.6,128)
(2,Saturday Night Live: The 2010s,2010,3.8,127)
(3,2001: A Space Odyssey,2001,4.0,145)
Then do whatever else you were going to do (COUNT etc ...).