I am hoping to write one SQL query that goes through all 20+ schemas without the need to constantly replace the search_path. I've tried UNION ALL but in most situations separating the query might take all the time I saved by not hard writing schemas. The query itself can be very basic such as:
SELECT *FROM schm1.table1
UNION ALL
SELECT *FROM schm2.table1
Thank you for your assistance!
"The impossible will be completed as you wait; please allow two days for the delivery of miracles".
I'm afraid what you want to achieve can only be done by SQL generating SQL:
SELECT
CASE ROW_NUMBER() OVER(ORDER BY table_schema)
WHEN 1 THEN ''
ELSE 'UNION ALL '
END
||'SELECT * FROM '
||table_schema
||'.'
||table_name
|| CASE ROW_NUMBER() OVER(ORDER BY table_schema DESC)
WHEN 1 THEN ';'
ELSE CHR(10)
END
FROM tables
WHERE table_name='d_teas_scd'
ORDER BY table_schema
;
What I get with d_teas_scd as table_name, is this:
SELECT * FROM flatt.d_teas_scd
UNION ALL SELECT * FROM public.d_teas_scd
UNION ALL SELECT * FROM star.d_teas_scd;
It can't guarantee that all tables with the same name have the same structure, though, that's why the resulting query could fail - that's your responsibility...
Happy playing
Marco the Sane
Related
I have a table in BigQuery with hundreds of columns, and it just happens that I want to select all of them except for those that begin with an underscore. I know how to do a query to select the columns beginning with an underscore using the INFORAMTION_SCHEMA.COLUMNS table, but I can't figure out how I would use this query to select the columns I want. I know BigQuery has EXCEPT but I want to avoid writing out each column that begins with an underscore, and I can't seem to pass to it a subquery or even something like a._*.
Consider below approach
execute immediate (select '''
select * except(''' || string_agg(col) || ''') from your_table
'''
from (
select col
from (select * from your_table limit 1) t,
unnest([struct(translate(to_json_string(t), '{}"', '') as kvs)]),
unnest(split(kvs)) kv,
unnest([struct(split(kv, ':')[offset(0)] as col)])
where starts_with(col, '_')
));
if apply to table like below
it generates below statement
select * except(_c,_e) from your_table
and produces below output
I have beginner knowledge on SQL and I am wondering whether this is possible in SQL.
SQL query 1 >>
select distinct(id) as active_pod from schema_naming
Query 1 output >>
active_pod
DB_1
DB_2
...
DB_20
SQL query 2 >>
select * from DB_1.mapping UNION
select * from DB_2.mapping UNION
....
select * from DB_20.mapping UNION
Due to my limited knowledge on SQL, I'm currently running #1 query first and change DB_1, DB2,.. DB_20 in query 2 everytime and run #2.
However, I was wondering whether there's way to this in one query so I don't have to manually change DB number in the #2 query and don't have to union every line.
something like this..(but not sure what to do with union)
select * from {
select distinct id from schema_naming}.user_map
It will be great if someone can shed light on this. (I'm trying to do this on Oracle SQL)
thank you in advance.
Are you trying to get something like this?
SELECT 'SELECT * FROM ' || active_pod || '.' || 'Mapping UNION'
FROM
(
select distinct(id) as active_pod from schema_naming
) as DT;
Alternatively, use PL/SQL block:
BEGIN
For i in (SELECT 'SELECT * FROM ' || ACTIVE_POD || '.MAPPING UNION' AS QUERY
FROM SCHEMA_NAMING) loop
dbms_output.put_line(i.query);
end loop;
END
Your queries will appear in the output window on your IDE.
This is a definitely a hack but it might make your life easier until a better solution is proposed. Basically use a query to generate your 2nd query, only manual edit needed would to remove the unecessary UNION on the final line.
SELECT 'SELECT * FROM ' || ACTIVE_POD || '.MAPPING UNION' AS QUERY
FROM SCHEMA_NAMING
Results:
SELECT * FROM DB_1.MAPPING UNION
SELECT * FROM DB_2.MAPPING UNION
SELECT * FROM DB_3.MAPPING UNION
SELECT * FROM DB_4.MAPPING UNION
SELECT * FROM DB_5.MAPPING UNION
SELECT * FROM DB_6.MAPPING UNION
SELECT * FROM DB_7.MAPPING UNION
SELECT * FROM DB_8.MAPPING UNION
SELECT * FROM DB_9.MAPPING UNION
SELECT * FROM DB_10.MAPPING UNION
SELECT * FROM DB_11.MAPPING UNION
SELECT * FROM DB_12.MAPPING UNION
SELECT * FROM DB_13.MAPPING UNION
SELECT * FROM DB_14.MAPPING UNION
SELECT * FROM DB_15.MAPPING UNION
SELECT * FROM DB_16.MAPPING UNION
SELECT * FROM DB_17.MAPPING UNION
SELECT * FROM DB_18.MAPPING UNION
SELECT * FROM DB_19.MAPPING UNION
SELECT * FROM DB_20.MAPPING UNION
I have a query that has multiple union statements where the last union statements results are derived from a function. I want to know how to sort the resultset based on the vchLastName of the function results. Here is the basis of the stored procedure.
select ID, objtype, Descr=coalesce(v.acctName, x.Descr), x.ihhid
from (
-----select statements----
UNION ALL
SELECT
pn.iPartyID,
'CONTACT',
ISNULL(vchFirstName,'') + ' ' + ISNULL(vchLastName,'') + ' ' +
CASE WHEN vchGoesBy IS NOT NULL THEN '(' + vchGoesBy + ')' ELSE ISNULL(vchGoesBy,'') END,
pn.iHHID
FROM CRM.dbo.cfn_PartyNameSearch(1, 'test') pn
WHERE pn.cPartyType = 'C'
)
x left join tableName v on v.id = x.id
and x.ObjType='ACCOUNT'
I want to sort by the pn.vchLastName for CONTACT. the other results sorting isn't a big deal honestly, and I am not able to edit the function that gives the results.
This general idea will work. You can work out the details
select 2 sortfield1
, whatever aliasName
from etc
union all
select 1 sortfield1
, yourFunctionResult aliasName
from etc
order by sortfield1, aliasName
Ordering takes place towards the end of the query execution, at least after all the data has been selected. Therefore you will have to order the entire result set at the end of the query. IIRC there is not conditonal ordering in T-SQL.
Is there a way, through the information_schema or otherwise, to calculate how many percent of each column of a table (or a set of tables, better yet) are NULLs?
Your query has a number of problems, most importantly you are not escaping identifiers (which could lead to exceptions at best or SQL injection attacks in the worst case) and you are not taking the schema into account.
Use instead:
SELECT 'SELECT ' || string_agg(concat('round(100 - 100 * count(', col
, ') / count(*)::numeric, 2) AS ', col_pct), E'\n , ')
|| E'\nFROM ' || tbl
FROM (
SELECT quote_ident(table_schema) || '.' || quote_ident(table_name) AS tbl
, quote_ident(column_name) AS col
, quote_ident(column_name || '_pct') AS col_pct
FROM information_schema.columns
WHERE table_name = 'my_table_name'
ORDER BY ordinal_position
) sub
GROUP BY tbl;
Produces a query like:
SELECT round(100 - 100 * count(id) / count(*)::numeric, 2) AS id_pct
, round(100 - 100 * count(day) / count(*)::numeric, 2) AS day_pct
, round(100 - 100 * count("oDd X") / count(*)::numeric, 2) AS "oDd X_pct"
FROM public.my_table_name;
Closely related answer on dba.SE with a lot more details:
Check whether empty strings are present in character-type columns
In PostgreSQL, you can easily compute it using the statistics tables if your autovacuum setting is on (check it by SHOW ALL;). You can also set the vacuum interval to configure how fast your statistics tables should be updated. You can then compute the NULL percentage (aka, null fraction) simply using the query below:
select attname, null_frac from pg_stats where tablename = 'table_name'
Think there is not built-in features for this. You can make this self
Just walk thorough each column in table and calc count() for all rows and count() for rows where column is null.
There is possible and optimize this for one query for one table.
OK, I played around a little and made a query that returns a query--or queries if you use LIKE 'my_table%' instead of = 'my_table_name':
SELECT 'select '|| string_agg('(count(*)::real-count('||column_name||')::real)/count(*)::real as '||column_name||'_percentage ', ', ') || 'from ' || table_name
FROM information_schema.columns
WHERE table_name LIKE 'my_table_name'
GROUP BY table_name
It returns a ready-to-run SQL query, like:
"SELECT (count(*)::real-count(id)::real)/count(*)::real AS id_percentage , (count(*)::real-count(value)::real)/count(*)::real AS value_percentage FROM my_table_name"
id_percentage;value_percentage
0;0.0177515
(The caps didn't go exactly right for readability.)
I am having 3 sys_refcursor for 3 different query inside a same loop.
I want to combine the o/p of these 3 cursor and insert into a table.
I am using the below code.here "l_ssc" and "l_nwo" are records.
SELECT shipper_short_code bulk collect
INTO l_ssc
FROM prepayment_ssc
ORDER BY shipper_short_code;
SELECT DISTINCT pmd_client_owner_nwo bulk collect
INTO l_nwo
FROM pp_meter_data ORDER pmd_client_owner_nwo;
FOR i IN 1.. l_ssc.count
LOOP
FOR j IN 1.. l_nwo.count
LOOP
l_query_a70 := 'select * from USER_PAYS_SI_A70 where shipper_short_code =''' || l_ssc(i).ssc ||''' and NWO_SHORT_CODE =''' ||l_nwo(j).nwo ||'''''';
l_query_O35 := 'select * from USER_PAYS_SI_O35 where NWO_SHORT_CODE=''' || l_nwo(j).nwo ||'';
l_query_R21 := 'select * from user_pays_si where SHIPPER_SHORT_CODE =''' || l_ssc(i).ssc ||''' and CLIENT_OWNER=''' || l_nwo(j).nwo ||'';
OPEN o_cursor1 FOR l_query_a70;
OPEN o_cursor2 FOR l_query_035;
OPEN o_cursor3 FOR l_query_r21;
END LOOP;
END LOOP;
/
You will run 3 queries per iteration of the loops, and this is nested inside two loops. So you will run 3 * l_ssc.count * l_nwo.count queries.
A basic principle in writing good plsql is to let the database do the processing for you.
Finding another way to write the queries into one and letting the database do the work of the loops will create a better solution.
I don’t have your tables but a rough stab at the query could be:
select * from (
select upsa.*
from USER_PAYS_SI_A70 upsa
where upsa.shipper_short_code in (SELECT shipper_short_code FROM prepayment_ssc)
and upsa.NWO_SHORT_CODE in (SELECT DISTINCT pmd_client_owner_nwo FROM pp_meter_data)
union all
select upso.*
from USER_PAYS_SI_O35 upso
where upso.NWO_SHORT_CODE in (SELECT DISTINCT pmd_client_owner_nwo FROM pp_meter_data)
union all
select ups.*
from user_pays_si ups
where ups.SHIPPER_SHORT_CODE in (SELECT shipper_short_code FROM prepayment_ssc)
and ups.CLIENT_OWNER in (SELECT DISTINCT pmd_client_owner_nwo FROM pp_meter_data)
) so_far
order by so_far.SHIPPER_SHORT_CODE, so_far.NWO_SHORT_CODE
This won’t work but with some experimentation I am sure you can build a single query which gets the required results.