I loaded nested json file to a table and got nested table.
Now i want to filter the table data by where clause and insert it to another table.
so i will do destination table - new table
and source table = select * from old_nested_table where status=1
The problem is that the select * is flatten the old nested table while i want to keep it nested.
How can i query nested table in order to get nested results?
You should
check on "Allow Large Results"
and
check off "Flatten Results"
as below
These settings allow you to save result preserving schema of original table
Related
I need to create a separate table/view from a partition table with daily data.
I used this query:
SELECT * FROM `awantec-gws-bigquery2.gmail_log_dataset_copy.daily_*`
but I get this error:
Views cannot be queried through prefix. First view awantec-gws-bigquery2:gmail_log_dataset_copy.daily_gmail_log.
I have a partitioned table in big query and I wanted to change the schema of that table.
I have changed the schema of table before using the following sql from the web UI
'SELECT * REPLACE ((SELECT AS STRUCT whatever.* EXCEPT (columnName)) AS whatever) FROM `a:b.c`'
but this causes all the previous partitions to be lost and when I look for the partitions of this newly created table using the following command it gives me today's date
SELECT _PARTITIONTIME as pt, FORMAT_TIMESTAMP("%Y%m%d", _PARTITIONTIME) as partition_id
FROM `a.b.c`
GROUP BY _PARTITIONTIME
ORDER BY _PARTITIONTIME
Is it possible to change the schema of a table and also keep its partitions in BigQuery?
Currently, it is stated in the documentation ( link 1, link 2) all the possible modifications that are available and changing a partitioned table to a non-partitioned table is not listed. However, there is a work around.
In order to change a Partitioned table to a Non-partitioned table, you can use the Console to query your data and overwrite your current table or copy to a new one. As an example, I have a table in BigQuery partitioned by _PARTITIONTIME. I used the following query to create a non-partitioned table,
SELECT *, _PARTITIONTIME as pt FROM `project.dataset.table`
With the above code, you will query the data among all table's partitions and create an extra column to show which partition it came from. Then, before executing it, there are two options, save the view in a new non-partitioned table or overwrite the current table:
Creating a new table go to: More(under the query editor) > Query Settings > Check the box "Set a destination table for query results" > Choose your project, dataset and write your new table's name > Under Destination table write preference check Write if empty.
Overwriting the current table: More(under the query editor) > Query Settings > Check the box "Set a destination table for query results" > Choose the same project and dataset for your current table > Write the same table's name as the one you want to overwrite > Under Destination table write preference check Overwrite table.
I have 2 tables in my SQL database:
And I want to merge them in a way the result will be:
This is just an example for 2 tables which need to be merged into one new table (The tables contain an example data, the statement should work for any amount of data inside the tables).
The ID which got different value in CSV should be updated into the new table for example:
ID 3's value is 'KKK' and in table T is 'CCC', then what should be updated is the CSV table.
You seem to want a left join and to match to the second table if available:
select t.id, coalesce(csv.value, t.value) as value
from t left join
csv
on t.id = csv.id;
If you want this in a new table, use the appropriate construct for your database, or use insert to insert into an existing table.
I'm looking for a solution to extract data from multiple tables and insert it into another table automatically running a single script. I need to query many tables, so I want to make a loop to select from those table's names dynamically.
I wonder if I could have a table with table names, and execute a loop like:
foreach(i in table_names)
insert into aggregated_table select * from table_names[i]
end
Below is for BigQuery Standard SQL
#standardSQL
SELECT * FROM `project.dataset1.*`
WHERE _TABLE_SUFFIX IN (SELECT table_name FROM `project.dataset2.list`)
This approach will work if below conditions are met
all to be processed table from list have exact same schema
one of those tables is the most recent table - this table will define schema that will be used for all the rest tables in the list
to meet above bullet - ideally list should be hosted in another dataset
Obviously, you can add INSERT INTO ... to insert result into whatever destination is to be
Please note: Filters on _TABLE_SUFFIX that include subqueries cannot be used to limit the number of tables scanned for a wildcard table, so make sure your are using longest possible prefix - for example
#standardSQL
SELECT * FROM `project.dataset1.source_table_*`
WHERE _TABLE_SUFFIX IN (SELECT table_name FROM `project.dataset2.list`)
So, again - even though you will select data from specific tables (set in project.dataset2.list) the cost will be for scanning all tables that match project.dataset1.source_table_* woldcard
While above is purely in BigQuery SQL - you can use any client of your choice to script exacly the logic you need - read table names from list table and then select and insert in loop - this option is simplest and most optimal I think
In SQL Server 2008, I need to create a child table from parent big table copying it's values in two columns and updates automatically. These values must be distinct and copy into corresponding two columns in new table. The whole idea would be making reference table with these distinct values and make relation with parent table. Thanks
Enkhzul,
You'll want to create a VIEW. A VIEW will simply run a SELECT query in the background against any tables you request in the query and display as instructed. Syntax will be:
USE [DATABASE]
GO
CREATE VIEW [VIEWNAME] AS
(
select statement here
)
After that you just need to do a SELECT query against the newly created VIEW.