How to use pl/pgSQL to handle 'comma separated list' returns? - sql

Question

I'am trying UNION ALL many tables into a new table.The columns of the old tables are the same, but the order of the columns is different, so the below SQL statement will get wrong result:
CREATE TABLE sum_7_2018_xia_weijian
AS
(
SELECT * FROM huiwen
UNION
SELECT * FROM penglai
UNION
SELECT * FROM baoluo
UNION
SELECT * FROM dongge
UNION
SELECT * FROM resultdonglu
UNION
SELECT * FROM resultwencheng
UNION
SELECT * FROM tan_illeg
);
I finally corrected it, but the SQL statements is too redundant:
step 1. get column names of one of the old tables named huiwen
SELECT string_agg(column_name, ',')
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'huiwen';
results:
> string_agg
> ----------------------------------------------------------------------
>
> gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
step 2. union tables as a new table. I copy the string_agg of table huiwen to each SELECT-UNION to keep the order of columns, this is clumsy.
CREATE TABLE sum_2018_xia_weijian
AS
(
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM huiwen
UNION ALL
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM penglai
UNION ALL
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM baoluo
);
results:
> Query returned successfully: 2206 rows affected, 133 msec execution time.
I tried to do some optimization by pl/pgSQL using Declarations of variable to handle column names, but failed to find any SQL data type can handle this. Using of RECORD result Pseudo-Types ERROR：
CREATE or replace FUNCTION ct() RETURNS RECORD AS $$
DECLARE
clms RECORD;
BEGIN
SELECT column_name INTO clms
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'huiwen';
RETURN clms;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE sum_2018_xia_weijian
AS
(
SELECT ct() FROM huiwen
UNION ALL
SELECT ct() FROM penglai
UNION ALL
SELECT ct() FROM baoluo
UNION ALL
SELECT ct() FROM dongge
UNION ALL
SELECT ct() FROM resultdonglu
UNION ALL
SELECT ct() FROM resultwencheng
UNION ALL
SELECT ct() FROM tan_illeg
);

Answer 1

You may use STRING_AGG twice for getting the UNION ALL. You can get all the columns in specific order by explicitly ordering it by column_name in the string_agg.
Here's a generic function which takes an array of tables and a final table name.
CREATE or replace FUNCTION fn_create_tab(tname_arr TEXT[], p_tab_name TEXT)
RETURNS VOID AS $$
DECLARE
l_select TEXT;
BEGIN
select STRING_AGG(query,' UNION ALL ' ) INTO l_select
FROM
(
SELECT 'select ' || string_agg( column_name,','
ORDER BY column_name ) || ' from ' || table_name as query
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = ANY (tname_arr)
GROUP BY table_name
) s;
IF l_select IS NOT NULL
THEN
EXECUTE format ('DROP TABLE IF EXISTS %I',p_tab_name);
EXECUTE format ('create table %I AS %s',p_tab_name,l_select);
END IF;
END;
$$ LANGUAGE plpgsql;
Now, run the function like this:
select fn_create_tab(ARRAY['huiwen','penglai'],'sum_2018_xia_weijian');

Answer 2

Instead of making the programming block complex you can follow some below concepts from the documentation of Union or Union All as it says :
The number of columns in all queries must be the same.
The corresponding columns must have the compatible data type.
The column names of the first query determine the column names of the combined result set.
The GROUP BY and HAVING clauses are applied to each individual query, not the final result set.
The ORDER BY clause is applied to the combined result set, not within the individual result set.
By following the 3rd point make your Union query adjusted to refer to the table whose column order is expected in the result.