I have 2 table value functions in 2 different projects. I can run these 2 individual queries and its working fine.
SELECT *
FROM `project1.analytics_1.TableValueFunc1`('2022-01-01', '2023-02-02')
SELECT *
FROM `project2.analytics_2.TableValueFunc2`('2023-01-01', '2023-01-02')
But when I union these queries, like this:
SELECT *
FROM `project1.analytics_1.TableValueFunc1`('2022-01-01', '2023-02-02')
UNION ALL
SELECT *
FROM `project2.analytics_2.TableValueFunc2`('2023-01-01', '2023-01-02')
I get this error:
Table-valued function not found: project1.analytics_1.TableValueFunc1
I tried this solution but it's not working
BigQuery JOINs between tables in different projects
Related
I have over 40 tables I want to append in BigQuery using standard SQL. I have already formatted them to have the exact same schema. When I try to use the '*' wildcard at the end of table name in my FROM clause, I get the following error:
Syntax error: Expected end of input but got "*" at [95:48]
I ended up manually doing a UNION DISTINCT on all my tables. Is this the best way to do this? Any help would be appreciated. Thank you!
CREATE TABLE capstone-320521.Trips.Divvy_Trips_All AS
SELECT * FROM capstone-320521.Trips.Divvy_Trips_*;
--JOIN all 2020-21 trips data
CREATE TABLE capstone-320521.Trips.Divvy_Trips_Raw_2020_2021 AS
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_04
UNION DISTINCT
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_05
UNION DISTINCT
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_06
UNION DISTINCT
Syntax error: Expected end of input but got "*"
I think the problem is in missing ticks around the table references. Try below
CREATE TABLE `capstone-320521.Trips.Divvy_Trips_All` AS
SELECT * FROM `capstone-320521.Trips.Divvy_Trips_*`
Note: The wildcard table name contains the special character (*), which means that you must enclose the wildcard table name in backtick (`) characters. See more at Enclose table names with wildcards in backticks
I am unaware of any such UNION DISTINCT syntax. If your intention is to do a union of the 3 tables and remove any duplicate records in the process, then just using UNION should suffice:
CREATE TABLE capstone-320521.Trips.Divvy_Trips_Raw_2020_2021 AS
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_04
UNION
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_05
UNION
SELECT * FROM capstone-320521.Trips.Divvy_Trips_2020_06;
Note that in general it is bad practice to use SELECT * with union queries. The reason has to do with that a union between two (or more) tables is generally only valid if the two queries involved have the number and types of columns. Using SELECT * obfuscates what columns are actually being selected, and so it is preferable to always explicitly list out the columns.
I want to query multiple tables, all with the same column names in the same order, and combine the results.
SELECT SUBSTR(arrest_date, 0, 4) arrest_year, *
FROM
`OBTS.circuit11`,
`OBTS.circuit15`,
`OBTS.circuit17`,
`OBTS.circuit19`
WHERE
init_statute LIKE '%3%22%32%' OR
init_statute LIKE '%3%22%34%' OR
LOWER(init_charge_descrip) LIKE '%suspend%';
When I run this BigQuery gives me the following error.
Column name init_statute is ambiguous at [8:3]
How do I query these tables and combine all the resulting rows into one set of results?
I think you are looking for UNION ALL vs CROSS JOIN (note: comma in BigQuery Standard SQL is used for express CROSS JOIN)
So, you most likely looking for below
SELECT SUBSTR(arrest_date, 0, 4) arrest_year, *
FROM (
SELECT * FROM `OBTS.circuit11` UNION ALL
SELECT * FROM `OBTS.circuit15` UNION ALL
SELECT * FROM `OBTS.circuit17` UNION ALL
SELECT * FROM `OBTS.circuit19`
)
WHERE
init_statute LIKE '%3%22%32%' OR
init_statute LIKE '%3%22%34%' OR
LOWER(init_charge_descrip) LIKE '%suspend%'
We have a table that has claim number, amount, and code for writeoffs. We do multiple writeoffs per record so we have 4 separate instances labeled as:
WOCLAIMNO1,WOAMT1,WOCODE1
WOCLAIMNO2,WOAMT2,WOCODE2
WOCLAIMNO3,WOAMT3,WOCODE3
WOCLAIMNO4,WOAMT4,WOCODE4
Currently we need to run 4 separate queries and then just copy and paste them all into one spreadsheet. We need to get the results of every record that has the WO code including the word 'Warehouse'.
So if you take the example table below and run query with that criteria you should get the output expected. What I need is a way to run just one query on all the columns instead of running each query separately, meaning query WO1, then WO2, then WO3, and then WO4 and then combining all the results together manually.
Sample Table
Output
You could just use UNION ALL :
SELECT
WOCLAIMNO1 AS WOCLAIMNO,
WOAMT1 AS WOAMT,
WOCODE1 AS WOCODE
FROM
mytable
WHERE
WOCODE1 LIKE '%Warehouse%'
UNION ALL
SELECT WOCLAIMNO2, WOAMT2, WOCODE2 FROM mytable WHERE WOCODE2 LIKE '%Warehouse%'
UNION ALL
SELECT WOCLAIMNO3, WOAMT3, WOCODE3 FROM mytable WHERE WOCODE3 LIKE '%Warehouse%'
UNION ALL
SELECT WOCLAIMNO4, WOAMT4, WOCODE4 FROM mytable WHERE WOCODE4 LIKE '%Warehouse%'
I have this in CL and would like to make it into a view. I first run 2 queries which are creating these 2 tables. How would the code here for the UNION ALL be? the 2 tables are (Logical) views
CPYF FROMFILE(JETDTA/EODDETAILH) +
TOFILE(JETDTA/EODDETAILS) MBROPT(*ADD) +
FMTOPT(*NOCHK)
MONMSG CPF0000
Besides the structure of the queries result must match, you can do it with:
CREATE VIEW yourViewName AS
SELECT *
FROM JETDTA.EODDETAILH
UNION
SELECT *
FROM JETDTA.EODDETAILS
You can find more information about the CREATE VIEW command here and about the UNION clause here, both of those are part of SQL/400.
I've been playing around with the sample on Jeff' Server blog to compare two tables to find the differences.
In my case the tables are a backup and the current data. I can get what I want with this SQL statement (simplified by removing most of the columns). I can then see the rows from each table that don't have an exact match and I can see from which table they come.
SELECT
MIN(TableName) as TableName
,[strCustomer]
,[strAddress1]
,[strCity]
,[strPostalCode]
FROM
(SELECT
'Old' as TableName
,[JAS001].[dbo].[AR_CustomerAddresses].[strCustomer]
,[JAS001].[dbo].[AR_CustomerAddresses].[strAddress1]
,[JAS001].[dbo].[AR_CustomerAddresses].[strCity]
,[JAS001].[dbo].[AR_CustomerAddresses].[strPostalCode]
FROM
[JAS001].[dbo].[AR_CustomerAddresses]
UNION ALL
SELECT
'New' as TableName
,[JAS001new].[dbo].[AR_CustomerAddresses].[strCustomer]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strAddress1]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strCity]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strPostalCode]
FROM
[JAS001new].[dbo].[AR_CustomerAddresses]) tmp
GROUP BY
[strCustomer]
,[strAddress1]
,[strCity]
,[strPostalCode]
HAVING
COUNT(*) = 1
This Stack Overflow Answer gives me a much cleaner SQL query but does not tell me from which table the rows come.
SELECT * FROM [JAS001new].[dbo].[AR_CustomerAddresses]
UNION
SELECT * FROM [JAS001].[dbo].[AR_CustomerAddresses]
EXCEPT
SELECT * FROM [JAS001new].[dbo].[AR_CustomerAddresses]
INTERSECT
SELECT * FROM [JAS001].[dbo].[AR_CustomerAddresses]
I could use the first version but I have many tables that I need to compare and I think that there has to be an easy way to add the source table column to the second query. I've tried several things and googled to no avail. I suspect that maybe I'm just not searching for the correct thing since I'm sure it's been answered before.
Maybe I'm going down the wrong trail and there is a better way to compare the databases?
Could you use the following setup to accomplish your goal?
SELECT 'New not in Old' Descriptor, *
FROM
(
SELECT * FROM [JAS001new].[dbo].[AR_CustomerAddresses]
EXCEPT
SELECT * FROM [JAS001].[dbo].[AR_CustomerAddresses]
) a
UNION
SELECT 'Old not in New' Descriptor, *
FROM
(
SELECT * FROM [JAS001].[dbo].[AR_CustomerAddresses]
EXCEPT
SELECT * FROM [JAS001new].[dbo].[AR_CustomerAddresses]
) b
You can't add the table name there because union, except, and intersection all compare all columns. This means you can't differentiate between them by adding the table name to the query. A group by gives you control over what columns are considered in finding duplicates so you can exclude the table name.
To help you with the large number of tables you need to compare you could write a sql query off the metadata tables that hold table names and columns and generate the sql commands dynamically off those values.
Derive one column using table names like below
SELECT MIN(TableName) as TableName
,[strCustomer]
,[strAddress1]
,[strCity]
,[strPostalCode]
,table_name_came
FROM
(SELECT 'Old' as TableName
,[JAS001].[dbo].[AR_CustomerAddresses].[strCustomer]
,[JAS001].[dbo].[AR_CustomerAddresses].[strAddress1]
,[JAS001].[dbo].[AR_CustomerAddresses].[strCity]
,[JAS001].[dbo].[AR_CustomerAddresses].[strPostalCode]
,'[JAS001].[dbo].[AR_CustomerAddresses]' as table_name_came
FROM [JAS001].[dbo].[AR_CustomerAddresses]
UNION ALL
SELECT 'New' as TableName
,[JAS001new].[dbo].[AR_CustomerAddresses].[strCustomer]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strAddress1]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strCity]
,[JAS001new].[dbo].[AR_CustomerAddresses].[strPostalCode]
,'[JAS001new].[dbo].[AR_CustomerAddresses]' as table_name_came
FROM [JAS001new].[dbo].[AR_CustomerAddresses]
) tmp
GROUP BY [strCustomer]
,[strAddress1]
,[strCity]
,[strPostalCode]
,table_name_came
HAVING COUNT(*) = 1