BigQuery Legacy SQL - How to insert into tables that has nested fields? - google-bigquery

Question

I'm trying to insert records into tables that has Nested and Repeated fields. I know that STRUCT and ARRAY keyword can be used respectively in Standard SQL.
What is equivalent of STRUCT and ARRAY keyword in Legacy SQL to insert records into Nested and Repeated fields?

Answer 1

I am reusing example you provided in bq command line tool - How to insert into Big query tables that has nested fields?
Try below, it is for Legacy SQL and using in-line version of Javascript UDF for Legacy SQL
Note: by default BigQuery Legacy SQL flattens any result, so make sure you set destination table and set Allow Large Results to true (or check it in Web UI) and Flatten Results to false (or uncheck it in Web UI)
SELECT Employee_id, Name, Age, Department.*, Location.* FROM JS((
SELECT Employee_id, Name, Age, Department_id, Department_Name, Department_Code, e.Location_id AS Location_id, Country, State, City
FROM (SELECT e.Employee_id AS Employee_id, e.Name AS Name, e.Age AS Age,
e.Department_id AS Department_id, d.Department_Name AS Department_Name, d.Department_Code AS Department_Code, e.Location_id AS Location_id
FROM Employee e JOIN Department d ON e.Department_id = d.Department_id ) AS e
JOIN Location l ON e.Location_id = l.Location_id
),
// input columns
Employee_id, Name, Age, Department_id, Department_Name, Department_Code, Location_id, Country, State, City,
// output schema
"[
{'name': 'Employee_id', 'type': 'INTEGER', 'mode': 'NULLABLE'},
{'name': 'Name', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'Age', 'type': 'INTEGER', 'mode': 'NULLABLE'},
{'name': 'Department', 'type': 'RECORD', 'mode': 'NULLABLE', 'fields': [
{'name': 'Department_id', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'Department_Name', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'Department_Code', 'type': 'STRING', 'mode': 'NULLABLE'}
]},
{'name': 'Location', 'type': 'RECORD', 'mode': 'NULLABLE', 'fields': [
{'name': 'Location_id', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'Country', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'State', 'type': 'STRING', 'mode': 'NULLABLE'},
{'name': 'City', 'type': 'STRING', 'mode': 'NULLABLE'}
]}
]",
// function
"function(r, emit){
emit({
Employee_id: r.Employee_id, Name: r.Name, Age: r.Age,
Department: {Department_id:r.Department_id, Department_Name:r.Department_Name, Department_Code:r.Department_Code},
Location: {Location_id:r.Location_id, Country:r.Country, State:r.State, City:r.City}
});
}"
)
Please note: i am using in-line version of UDF here for the purpose of easy showing and testing. In-line version is not recommended and not officially supported. But you can easily convert it to supported version - see more for User-Defined Functions in Legacy SQL
P.S. even though above works and helped a lot before the Standard SQL was an option - now what is the big reason for you to use Legacy SQL where Standard SQL is more elegant and gives you much more flexibility especially when it is about dealing with nested and repeated fields