What I'm trying to do is to pull out a subset of tables from a Google Big Query Dataset based on the name of those tables and then add those tables to a Tableau datasource without having to join or union any of the tables.
I want to pull out all the tables beginning with System_1, from the below dataset
System_1_Start
System_1_Middle
System_1_End
System_2_Start
System_2_Middle
System_2_End
System_3_Start
System_3_Middle
System_3_End
I have been able to do a wildcard search using Information_Schema.Tables to get all the names of the tables that begin with System_1, but I cannot figure out a way to then get all of the tables with those names as the output of the query (SQL below)
SELECT table_name
AS matchingTables
FROM dataset.INFORMATION_SCHEMA.TABLES
WHERE table_name LIKE 'System_1%'
How do I go about extracting those tables and not just the names of those tables?
~~~~~~~~~~~EDIT~~~~~~~~~~~~~
This is the best approximation as to how I could have done this but I'm getting a dataset not found error which is strange
SELECT *
FROM dataset
WHERE (SELECT table_name FROM dataset.INFORMATION_SCHEMA.TABLES)
LIKE 'System_1%'
Error Recieved:
Not found: Dataset dataset:dataset was not found in location EU
If I understand correctly you want the list of the table names in your dataset that start with "System_1_". You can obtain that with the following:
SELECT CONCAT("System_1_", _TABLE_SUFFIX) AS table_name
FROM `dataset.System_1_*`
Related
I have a look-up table containing a list of fully qualified table paths in a Big Query table called all_tables. For example
|table_list|
|----------|
|project_name.dataset_name1.table_1|
|project_name.dataset_name2.table_1|
|project_name.dataset_name3.table_1|
|project_name.dataset_name4.table_1|
|project_name.dataset_name5.table_1|
I am trying to iterate through these tables to pull out elements I need for another procedure using the for-in syntax in Big Query. This is a simplified version of the query I am using
```
FOR table IN (select * from my_project.my_dataset.all_tables)
DO
select * from table;
END FOR;
```
This isn't working. It picks up the list of tables correctly, but when it substitutes the dataset name in the line 3 select statement, it says
**Invalid value: Table "table" must be qualified with a dataset (e.g. dataset.table)**
I know what the error is, but I am not sure how to make it 'see' the value of table as a table path.
All paths are correct, and I am doing it this way as I am querying multiple tables across multiple datasets for a table creation query.
You should a dynamic sql to refer the table name as a variable, so consider below query:
FOR table IN (select * from my_project.my_dataset.all_tables)
DO
EXECUTE IMMEDIATE FORMAT("""
SELECT * FROM %s;
""", table.table_list);
END FOR;
Is there any query that results from source table names and column table names using a mapping or mapping Id in informatica. This has been very hard and challenging
Like when we search up SELECT * FROM opb_mapping WHERE mapping_name LIKE '%CY0..%'
It is resulting in some details but I cannot find source table names and target table names. Help if you can.
Thanks.
You can use below view to get the data.
(Assuming you have full access to metadata tables.
select source_name, source_field_name,
target_name, target_column_name,
mapping_name, subject_name folder
from REP_FLD_MAPPING
where mapping_name like '%xx%';
Only issue i can see is, if you have overwrite sql, then you need to check sql for true source.
Is there a query I can write to search all the column names for a particular database in Netezza?
Within the same database you can use the following query:
select *
from _v_odbc_columns1
where column_name like '%columnname%'
or a less Netezza specific query
select *
from information_schema.columns
where column_name like '%columnname%'
The important catalog views in netezza system are listed below
_V_USER: the user view gives information about the users in the netezza system.
_V_TABLE: the table view contains the list of tables created in the netezza performance system.
_V_RELATION_COLUMN: the relation column system catalog view contains the columns available in a table.
_V_TABLE_INDEX: this system catalog contains the information about the
indexes created on table. netezza does not support creating indexes on a table as of now.
_V_OBJECTS: lists the different objects like tables, view, functions etc. available in the netezza.
Example:
SELECT *
FROM _V_RELATION_COLUMN
WHERE
ATTNAME like '%GEO%' --SPECIFY COLUMN NAME
AND type = 'TABLE'
You would access something similar to an information_schema.
Column Name, %COW%', would use % as a wildcard...gathering any column that has 'COW' in the name
SELECT *
FROM _V_SYS_COLUMNS
WHERE
COLUMN_NAME like '%COW%'
AND TABLE_SCHEMA = 'DEV'
ORDER BY TABLE_NAME
;
I have a dataset that contains several tables that have suffixes in their name:
table_src1_serie1
table_src1_serie2
table_src2_opt1
table_src2_opt2
table_src3_type1_v1
table_src3_type2_v1
table_src3_type2_v2
I know that i can use this type of queries in BQ:
select * from `project.dataset.table_*`
to get all the rows from theses different tables.
What i am trying to achieve is to have a column that will contain for instance the type of source (src1, src2, src3)
Assuming the schema of all tables the same - you can add below to your select list (for BigQuery Standard SQL)
SPLIT(_TABLE_SUFFIX, '_')[SAFE_OFFSET(0)] AS src
My overall goal is to create new tables by selecting columns in existing tables with certain patterns/tags in their column names. This is in SQL Server.
For example, I have a common need to get all contact information out of a company table, and into its own contact table.
I haven't been able to find a programmatic approach in SQL to express excluding columns from a SELECT statement based on string.
When looking to options like the COL_NAME function, those require an ID arg, which kills that option for me.
Wishing there was something built in that could work like the following:
SELECT *
FROM TABLE
WHERE COL_NAME() LIKE 'FLAG%'
Any ideas? Open to anything! Thanks!!
One trick could be to firstly using the following code to get the column names you desire:
select * from information_schema.columns
where table_name='tbl' and column_name like 'FLAG%'
Then concatenate them into a comma-delimited string and finally create a dynamic sql query using the created string as the column names.