Google's Big Query using SQL: Associate the assignee name and harmonized assignee name when there are multiple assignees - sql

Question

My goal is to create a table from Google's Big Query patents-public-data.patents.publications_201710 table using standard SQL that has one row for the publication_number, assignee and assignee_harmonized.name where the publication_number is repeated for records that have multiple assignees. Here's an example of my desired output:
publication_number|assignee|assignee_harm
US-6044964-A|Sony Corporation|SONY CORP
US-6044964-A|Digital Audio Disc Corporation|DIGITAL AUDIO DISC CORP
US-8746747-B2|IPS Corporation—Weld-On Division|IPS CORPORATION—WELD ON DIVISION
US-8746747-B2|null|MCPHERSON TERRY R
I've tried the following query based off of the UNNEST suggestion found in this post
#standard SQL
SELECT
p.publication_number,
p.assignee,
a.name AS assignee_harm
FROM
`patents-public-data.patents.publications_201710` AS p,
UNNEST(assignee_harmonized) AS a
WHERE
p.publication_number IN ('US-6044964-A',
'US-8746747-B2')
However, the output appears as follows:
row|publication_number|assignee|assignee_harm
1|US-6044964-A|Sony Corporation|SONY CORP
||Digital Audio Disc Corporation|
2|US-6044964-A|Sony Corporation|DIGITAL AUDIO DISC CORP
||Digital Audio Disc Corporation|
3|US-8746747-B2|IPS Corporation—Weld-On Division|MCPHERSON TERRY R
4|US-8746747-B2|IPS Corporation—Weld-On Division|IPS CORPORATION—WELD ON DIVISION
You can see that the "Sony Corporation" assignee is inappropriately associated with the "DIGITAL AUDIO DISC CORP" harmonized name in row 2 with a similar issue appearing in row 3. Also, rows 1 and 2 contain two lines each but don't repeat the publication_number identifier. I don't see a straightforward way to do this because the number of "assignee" doesn't always equal the number of "assignee_harmonized.name" and they don't always appear in the same order (otherwise I could try creating two tables and merging them somehow). On the other hand, there has to be a way to associate the "assignee" variable with its harmonized value "assignee_harmonized.name", otherwise the purpose of having a harmonized value is lost. Could you please suggest a query (or set of queries) that will produce the desired output when there are either multiple "assignee" or multiple "assignee_harmonized.name" or both?

Answer 1

You're querying for a string and two arrays - the whole thing basically looks like this:
{
"publication_number": "US-8746747-B2",
"assignee": [
"IPS Corporation—Weld-On Division"
],
"assignee_harm": [
"MCPHERSON TERRY R",
"IPS CORPORATION—WELD ON DIVISION"
]
}
So that's the data and you somehow need to decide how to treat the combination of them ... either you cross join everything:
#standard SQL
SELECT
p.publication_number,
assignee,
assignee_harmonized.name AS assignee_harm
FROM
`patents-public-data.patents.publications_201710` AS p
,p.assignee assignee
,p.assignee_harmonized AS assignee_harmonized
WHERE
p.publication_number IN ('US-6044964-A','US-8746747-B2')
.. which gives you relational data .. or you leave it as two separate arrays:
#standard SQL
SELECT
p.publication_number,
assignee,
ARRAY( (SELECT name FROM p.assignee_harmonized)) AS assignee_harm
FROM
`patents-public-data.patents.publications_201710` AS p
WHERE
p.publication_number IN ('US-6044964-A','US-8746747-B2')
You can save this nested result as a table in bq as well.