How can I use a dynamic query as an input source for a larger query?
There is a query I'm getting the union of values in different datasets/tables scattered around and the list is growing so I'm thinking of using the dynamic query than to write queries for each tables like this
SET QUERY = "";
SET tables = ["table1", "table2"...];
SET tables_size = ARRAY_LENGTH(tables);
WHILE i < tables_size DO
IF (i = tables_size -1) THEN
BEGIN
SET query = CONCAT(query, " SELECT id, name FROM ", tables[OFFSET(i)]);
BREAK;
END;
ELSE
SET query = CONCAT(query, " SELECT id, name FROM ", tables[OFFSET(i)], ' UNION ALL ');
END IF;
SET i = i + 1;
END WHILE;
EXECUTE IMMEDIATE query;
My goal is to use the output of the executed query as a FROM clause for a larger query.
It will be something like
Select A, B, C, D ... From *EXECUTE IMMEDIATE query* LEFT JOIN ... ON..
Is there a way to inject an output of a dynamic query as a table for another query?
I don't see TABLE as a variable type for bigquery so that was not my option.
I'm getting a bit tired of copy pasting table names to the exact query every time a new table is introduced to this logic.
SELECT id, name FROM table1 UNION ALL
SELECT id, name FROM table1 UNION ALL
SELECT id, name FROM table3...
If there is a simple way to do this? or maybe a reason to not use dynamic queries for performance reasons?
Hope one of these are helpful:
1. Wildcard tables
If tables you want to union have a common prefix, you can consider to use a wildcard table like below. I think this is more concise form rather than union-all:
-- Sample Tables
CREATE TABLE IF NOT EXISTS testset.table1 AS SELECT 1 AS id, 'aaa' AS name;
CREATE TABLE IF NOT EXISTS testset.table2 AS SELECT 2 AS id, 'bbb' AS name;
CREATE TABLE IF NOT EXISTS testset.table3 AS SELECT 3 AS id, 'ccc' AS name;
--- Wildcard tables
SELECT * FROM `testset.table*` WHERE _TABLE_SUFFIX IN ('1', '2', '3');
2. Dynamic SQL & Temp Table
You can't inject a dynamic SQL directly into another query but you can use a temp table to emulate it.
2.1 Dynamic SQL
More concise dynamic query to union all tables:
DECLARE tables DEFAULT ["testset.table1", "testset.table2", "testset.table3"];
SELECT ARRAY_TO_STRING(ARRAY_AGG(FORMAT('SELECT id, name FROM %s', t)), ' UNION ALL\n')
FROM UNNEST(tables) t;
2.2 Using a temp table
I thinks you can modify your larger query to use a dynamically generated temp table.
DECLARE tables DEFAULT ["testset.table1", "testset.table2", "testset.table3"];
CREATE TABLE IF NOT EXISTS testset.table1 AS SELECT 1 AS id, 'aaa' AS name;
CREATE TABLE IF NOT EXISTS testset.table2 AS SELECT 2 AS id, 'bbb' AS name;
CREATE TABLE IF NOT EXISTS testset.table3 AS SELECT 3 AS id, 'ccc' AS name;
EXECUTE IMMEDIATE (
SELECT "CREATE TEMP TABLE IF NOT EXISTS union_tables AS \n"
|| ARRAY_TO_STRING(ARRAY_AGG(FORMAT('SELECT id, name FROM %s', t)), ' UNION ALL\n') FROM UNNEST(tables) t
);
-- your larger query using a temp table
SELECT * FROM union_tables;
output:
Related
I have a stored procedure that uses a variable ID, I have a list of valid IDs in a table.
I'm trying to write a stored procedure that runs a specific piece of code if the ID exists in table. I'm just not sure of the syntax.
Below is my pseudo-code of what I'm attempting to do.
IF
#ID = possible id IN (SELECT DISTINCT ID FROM [dbo].ID_TABLE WHERE ID = 'valid')
SELECT * FROM dbo.[results]
ELSE
SELECT * FROM dbo.[otherresults]
I'm using SQL Server
Typically, this is the case where you would use EXISTS; as in....
IF EXISTS (SELECT * FROM ID_TABLE WHERE ID = #ID)
While #ID IN (SELECT DISTINCT would work, that query requires going through the table data to assemble a result set that is then checked for #ID's inclusion. EXISTS queries do not create result sets, and return early on the first row fitting the criteria.
If your query
SELECT DISTINCT ID FROM [dbo].ID_TABLE WHERE ID = 'valid'
always returns only one id, then below solution is enough
#ID = (SELECT DISTINCT ID FROM [dbo].ID_TABLE WHERE ID = 'valid')
If it returns list of ids, then you need to create a temp table and store those id's like below
create table #temp_table(id int)
insert into #temp_table SELECT DISTINCT ID FROM [dbo].ID_TABLE WHERE ID = 'valid'
Trying to union two tables with the same field into one master table but for some reason im getting a weird result.
select count(*)
from staging.sandoval_parcels
where parcel_id = 0;
returns 0
select count(*)
from staging.bernalillo_parcels
where parcel_id = 0;
returns 0
but when i merge the tables using
CREATE TABLE staging.master_parcels
AS
SELECT * FROM bernalillo_parcels
UNION ALL
SELECT * FROM sandoval_parcels
;
then
select count(*)
from staging.master_parcels
where parcel_id = 0;
returns 85553
both tables have the same fields and the fields are the same data type,also, no of the values for any field are missing, thus no nulls, why am i getting ids = 0 when either of the table have parcel_ids = 0?
The order of the fields matter, replace the * for the explicit name, other wise the second query field will be inserted on the first query position. But not necessarily on the same field you want.
CREATE TABLE staging.master_parcels
AS
SELECT parcel_id, field1 ... FROM bernalillo_parcels
UNION ALL
SELECT parcel_id, field1 ... FROM sandoval_parcels
;
Union will merge tables even if the column order is not the same. If all of the columns match and are in the same order, it will union distinct values and not create duplicates if the rows are the same for each table. Having the order and data type be the same is important.
I have a 50 tables with similar structures with name TABLE_1, TABLE_2, TABLE_3 etc. I want to select some information like SELECT * FROM TABLE WHERE CRM_ID = 100 but I dont know which table consists this ID. I guess that I should make a union with this tables and make a query from this union but I am afraid that it is not the best solution. Could anybody help me?
To find the table containing the field CRM_ID, you can use:
select TABLE_NAME
from SYS.ALL_TAB_COLUMNS
where COLUMN_NAME = 'CRM_ID'
From here, you can query the relevant tables with your union
select 'table_1', count(*) from table_1 where CMR_ID = 100
union all
select 'table_2', count(*) from table_2 where CMR_ID = 100
[.....]
If you have 50 tables, you may want to use some advanced text editing so you don't have to type the same line 50 times.
I can't see anything different from a UNION to get what you need.
However, I would like to avoid repeating all the code every time you need to query your tables, for example by creating a view:
create view all_my_tables as
select * from table1
union
select * from table2
...
and then
select *
from all_my_tables
where crm_id = 100
I would
create a view table_all as
select * from table_1 union all
select * from table_2 ...
but I guess you have 50 tables due to some performance reason like creating partitioning without partitioned table. If that is the case you need some table that simulates index and you need to have data "sorted". Then create table that contains: table_name, min_val, max_val:
table_1, 1, 1000
table_2, 1001, 2000
and procedure selecting looks like:
procedure sel(crmid) is
a varchar2(10);
value table%rowtype;
begin
select table_name into a from lookup where crmid between min_val and max_val;
execute immediate 'select * from ' || a || ' where crm_id = ' || crmid into value;
--do logic with value
end;
but if data is not ordered you need to iterate selects in loop. In such case use view.
Try this:
Select tmp.tablename from (
select 'table1' as tablename from table1 where CRM_ID=100
union all
select 'table2' as tablename from table2 where CRM_ID=100
) as tmp
I want a SQL statement which can run on all the platforms and give the result in tree structure (not a real tree structure though)format.i.e all the related columns appears together. is it possible to achieve the following format using a sql. I have a simple table with three columns(GROUP_STEP,PREDECESSOR,COLUMNNUM). expected output
Platform supported : Oracle, SQL Server, DB2 and Sybase. I am looking for a SELECT statement from the table having following data in different format.
After #diaho suggestion , following is the output
select groupnum,predecessor ,count(groupnum) as columnum group by groupnum,predecessor
if you give more details on table schema answer may change !!!
Based on your table, I'm assuming that you only have the Group_Step and Predecessor columns in your raw table and that the ColumnNum column represents the total number of levels for a leaf in your tree. For example, since PC_Wrap1 has the Predecessor BENEFITS which has the Predecessor COPY_BUDG it has a total of 3 'levels'. If that is the case, you need a recursive query to calculate the ColumnNum value. I can't speak to all platforms, but for SQL Server, you can use a CTE:
EDIT: Removed 'dreaded non-standard square brackets' per a_horse_with_no_name's suggestion :)
-- Setup table
CREATE TABLE #Temp
(
Group_Step VARCHAR(100),
Predecessor VARCHAR(100)
)
-- Setup dummy data
INSERT INTO #Temp
(
Group_Step,
Predecessor
)
SELECT 'ACT_BD_ACT', '' UNION
SELECT 'COPY_BUDG', '' UNION
SELECT 'COPY_BUDG2', '' UNION
SELECT 'BENEFITS', 'COPY_BUDG' UNION
SELECT 'BENEFITS', 'COPY_BUDG2' UNION
SELECT 'PC_WRAP1', 'BENEFITS' UNION
SELECT 'PC_WRAP2', 'BENEFITS' UNION
SELECT 'ALLC1', '' UNION
SELECT 'ALLC2', '' UNION
SELECT 'ALLC3', 'ALLC2' UNION
SELECT 'TCP1', 'ALLC3' UNION
SELECT 'TCP1', 'ALLC4' UNION
SELECT 'COPY_BUDG3', '' UNION
SELECT 'COPY_BUDG4', '';
-- Actual solution starts here:
WITH Result
(
Group_Step,
Predecessor,
ColumnNum
)
AS
(
-- Anchor member definition
SELECT
Group_Step,
Predecessor AS Predecessor,
1 AS ColumnNum
FROM
#Temp
WHERE
Predecessor = ''
UNION ALL
-- Recursive member definition
SELECT
t.Group_Step,
t.Predecessor,
ColumnNum + 1 AS ColumnNum
FROM
#Temp AS t
JOIN
Result AS r
ON
t.Predecessor = r.Group_Step
)
-- Statement that executes the CTE
SELECT DISTINCT
Group_Step,
Predecessor,
ColumnNum
FROM
Result
-- EDIT #2: Adding ORDER BY per Op's comment
ORDER BY
ColumnNum
DROP TABLE #Temp
I have some tables with the same structure and I want to make a select in a group of them.
Rather than just making a loop to all of those tables, I would like to put a subquery after the FROM of the main query.
Is it possible or it will fail?
Thanks!
(Using Oracle)
Additional info: I don't have the name of the table right away! They're stored in another table. Is it possible to have a subquery that I could put after the FROM of my main query?
"I don't have the name of the table
right away! They're stored in another
table"
Oracle doesn't do this sort of thing in SQL. You'll need to use PL/SQL and assemble a dynamic query.
create or replace function get_dynamic_rows
return sys_refcursor
is
stmt varchar2(32767) := null;
return_value sys_refcursor;
begin
for r in ( select table_name from your_table )
loop
if stmt is not null then
stmt := stmt||' union all ';
end if;
stmt := stmt||'select * from '||r.table_name;
end loop;
open return_value for stmt;
return return_value;
end;
/
This will assemble a query like this
select * from table_1 union all select * from table_2
The UNION ALL is a set operator which combines the output of several queries in a single result set without removing duplicates. The columns in each query must match in number and datatype.
Because the generated statement will be executed automatically there's no real value in formatting it (unless the actual bits of the query are more complicated and you perhaps need to debug it).
Ref Cursors are PL/SQL contructs equivalent to JDBC or .Net ResultSets. Find out more.
Sure, just union them together:
select * from TableA
union all
select * from TableB
union all
select * from TableC
You can union in a subquery:
select *
from (
select * from TableA
union all
select * from TableB
) sub
where col1 = 'value1'
Use union if you're only interested in unique rows, and union all if you want all rows including duplicates.