I have a requirement to query different tables a once to save my time. Tables names like
abc_yyyymmdd
can be easily query using the
table_date_range(abc_,timestamp('2016-01-01'),timestamp('2016-03-12'))
but I have different format table name
abc_mm_dd_yyyy
is there a way to query in these tables using table_date_range.
In Legacy SQL you can use TABLE_QUERY for this
So it can be something like below
SELECT *
FROM (
TABLE_QUERY(YourDataset, 'LEFT(table_id, 4) = "abc_" AND LENGTH(table_id) = 14
AND CONCAT(SUBSTR(table_id,11,4),'-',SUBSTR(table_id,5,2), -",SUBSTR(table_id,8,2))
BETWEEN "2016-01-01" AND "2016-03-12"')
)
If you can use Standard SQL, you can use the _TABLE_SUFFIX pseudo column to work with any table name format.
Is there an equivalent of table wildcard functions in BigQuery with standard SQL?
In this case, it would be something like:
SELECT ... FROM `mydataset.abc_2016_*` WHERE _TABLE_SUFFIX = '01-01' or _TABLE_SUFFIX = '03-12'
Related
I have a table which is like - 'numberedlanding_20210606_storefront'
this table is generated every day and changes in the time stamp. For example, nextday table name would be 'numberedlanding_20210607_storefront'
I am writing a query that uses this table as a base table and does further does aggregations such as -
For example -
with df1 as (select * from 'numberedlanding_20210606_storefront' where col1>0),
df2 as (select *,col1/col2 as test from df1),
select * from df1 a join df2 b on a.name=b.name
Now since this table name changes everyday how do I make it dynamic in this query?
I tried looking into Bigquery's wildcard but I think it applies on to variable at the end of the name, I want to change the name which is in the middle.
Also, here is something which I tried but was not successful -
CREATE TEMP FUNCTION
getTableName() AS ((CONCAT("numberedlanding_",FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)),"_storefront")));
SELECT
getTableName()
This just gives me the table name I want to use. Any suggestions would be really helpful.
SELECT * FROM numberedlanding_* WHERE _TABLE_SUFFIX = CURRENT_DATE() || '_storefront'
Below is how I structured the data in BigQuery database.
test
-> sales
-> monthly-2015
-> monthly-2016
-> ...
I want to combine the data of all tables with the table name , monthly-*, and below is how I wrote the sql from examples I found.
Running this sql leads an error like following Scalar subquery produced more than one element. How could I fix it to error?
SELECT
*,
(
SELECT
table_id
FROM
`test.sales.__TABLES_SUMMARY__`
WHERE
table_id LIKE 'monthly-%')
FROM
`test.sales.monthly*`
I want to combine the data of all tables with the table name , monthly-*
Try below
SELECT *, 'monthly_' || _TABLE_SUFFIX as table_name
FROM `test.sales.monthly_*`
I have dataset containing multiple tables with similar names:
e.g.
affilinet_4221_first_20180911_204956
affilinet_4221_first_20180911_160004
affilinet_4221_first_20180911_085559
affilinet_4221_first_20180910_201323
affilinet_4221_first_20180910_201042
affilinet_4221_first_20180910_080006
affilinet_4221_first_20180909_160707
This query identifies the latest dataset (according to yyyymmdd_hhmmss naming convention) with __TABLES_SUMMARY__ method
SELECT max(table_id) as table_id FROM `modemutti-8d8a6.feed_first.__TABLES_SUMMARY__`
where table_id LIKE "affilinet_4221_first_%"
query result
this query extracts all values from a specific table with _TABLE_SUFFIX method
SELECT * FROM `modemutti-8d8a6.feed_first.*`
WHERE _TABLE_SUFFIX = "affilinet_4221_first_20180911_204956"
query result
This query combines __TABLES_SUMMARY__ (which returns affilinet_4221_first_20180911_204956) and _TABLE_SUFFIX
SELECT * FROM `modemutti-8d8a6.feed_first.*`
WHERE _TABLE_SUFFIX = (
SELECT max(table_id) FROM `modemutti-8d8a6.feed_first.__TABLES_SUMMARY__`
where table_id LIKE "affilinet_4221_first_%")
this query fails:
Error: Cannot read field 'modemio_cat_level' of type INT64 as STRING
error screenshot
any idea why is this happening or how I could solve the issue?
------------EDIT------------
#Mikhail solution works correctly but processes a huge amount of data. See explicit call Vs the suggested Method. Another solution would have been
SELECT * FROM `modemutti-8d8a6.feed_first.affilinet_4221_first_*` WHERE _TABLE_SUFFIX =
(
SELECT MAX(_TABLE_SUFFIX) FROM`modemutti-8d8a6.feed_first.affilinet_4221_first_*`
)
but this processes also a much bigger amount of data compared to the explicit query. Is there are way to achieve through a view in the UI or should I rather use the Python / Java SDK via API?
Try below
#standardSQL
SELECT * FROM `modemutti-8d8a6.feed_first.affilinet_4221_first_*`
WHERE _TABLE_SUFFIX = (
SELECT REPLACE(MAX(table_id), 'affilinet_4221_first_', '')
FROM `modemutti-8d8a6.feed_first.__TABLES_SUMMARY__`
WHERE table_id LIKE "affilinet_4221_first_%"
)
I am in a situation where I need to return results if some conditions on the string/character are met.
For example: to return only the names that contain 'F' character from the Person table.
How to create an SQL query based on such conditions? Is there any link to a documentation that explains how can SQL perform such queries?
Thanks in advance
The most basic approach is to use LIKE operator:
-- name starts with 'F'
SELECT * FROM person WHERE name LIKE 'F%'
-- name contains 'F'
SELECT * FROM person WHERE name LIKE '%F%'
(% is a wildcard)
Most RDBMS offer string operations which are able to perform that required task in one way or the other.
In MySQL you might use INSTR:
SELECT *
FROM yourtable
WHERE INSTR(Person, 'F') > 0;
In Oracle, this can be done, too.
In PostgreSQL, you can use STRPOS:
SELECT *
FROM yourtable
WHERE strpos(Person, 'F') > 0;
Usually there are several approaches to solve this, many would choose the LIKE operator. For more details, please refer to the documentation of the RDBMS of your choice.
Update
As requested by the questioner a few words about the LIKE operator, which are used not only in MySQL or Oracle, but in other RDBMS, too.
The use of LIKE will in some cases make your RDBMS try to use an index, it usually does not not try to do so if you use a string functions.
Example:
SELECT *
FROM yourtable
WHERE Person LIKE 'F%';
The query may look like this:
SELECT * FROM Person WHERE FirstName LIKE '%F%' OR LastName LIKE '%F%'
I'm trying to search a SQL database using this SQL query:
SELECT * FROM Reservations WHERE fDate=06/12/13
Here is an image of my SQL Reservations table with dummy data:
What am I doing wrong here?
Thanks,
C.
You'll need to put the date inside ''
SELECT * FROM Reservations WHERE fDate='06/12/13'
But, if what you're showing us a view, then you'll need to use the standard format for dates:
SELECT * FROM Reservations WHERE fDate='2013-12-06'
Try changing date format from 'dd/mm/yyyy' to 'yyyy/mm/dd'.
Like this:
SELECT * FROM Reservations WHERE fDate='2013/11/08'
See sample SQL Fiddle I created
You are missing single quotes around the date ''
It should be simply:
SELECT * FROM Reservations WHERE fDate = '06/12/13'