Given two arrays how to get items that are not in both? - sql

I have two arrays in postgresql:
CREATE OR REPLACE FUNCTION func()
RETURNS void AS
$BODY$
declare
first integer[];
second integer[];
array_vb integer[];
array_vb2 integer[];
begin
code....
select array_agg(id) into first
from a
where id = any (array_vb);
select array_agg(id) into second
from a
where id = any (array_vb2);
end;
$BODY$
LANGUAGE plpgsql VOLATILE
I would like to add a raise notice that will print all items that are in first but not in second
for example:
first = [1,10,15,3,7]
second = [1,3,15,4]
it will print 10,7
How do I do that?

You can use PostgreSQL unsent function:
SELECT ARRAY(SELECT unnest(first) EXCEPT SELECT unnest(second))
Example:
SELECT ARRAY(SELECT unnest(ARRAY[1,10,15,3,7]) EXCEPT SELECT unnest(array[1,3,15,4]))
Gives:
array
--------
{10,7}
See SQLFiddle here.

You can use intarray extension:
create extension if not exists intarray;
select array[1,10,15,3,7] - array[1,3,15,4] as result;
result
--------
{7,10}
(1 row)

create or replace function f(a1 int[], a2 int[])
returns void as $body$
begin
raise notice '%', (
select string_agg(i::text, ',')
from
unnest (a1) a1(i)
left join
unnest (a2) a2(i) using (i)
where a2.i is null
);
end;
$body$ language plpgsql volatile
;
select f(array [1,10,15,3,7], array [1,3,15,4]);
NOTICE: 10,7
f
---
(1 row)
Notice that the above function, as it is, can be marked immutable

You can create a function that returns the desired array result:
create or replace function array_except(a1 int[], a2 int[])
returns int[]
as
$$
select array_agg(i)
from (
select i
from unnest(a1) i
except
select a2
from unnest(a2) a2
) t;
$$
language sql;
The function returns an array that contains the elements that are in the first, but not in the second array.
You can use the above function in your actual function:
create or replace function foo()
returns void
as
$BODY$
declare
_first integer[];
_second integer[];
begin
_first := array[1,10,15,3,7];
_second := array[1,3,15,4];
raise notice 'Result %', array_except(_first, _second);
end;
$BODY$
LANGUAGE plpgsql;

Related

Replacing Placeholder values with another table's data

I have 2 tables .The first table contains rows with placeholders and the second table contains those placeholders values.
I want a query which fetches data from the first table and replaces placeholders with actual values which are stored in the second table.
Ex:
Table1 Data
id value
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://{POSTGRESIP}:{POSTGRESPORT}/{TESTDB}
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://{SERVICEIP}/svc/{TESTSERVICE}/
Table2 Data
id placeolder value
201FEBFE-DF92-4474-A945-A592D046CA02 POSTGRESIP 1.2.3.4
20D9DE14-643F-4CE3-B7BF-4B7E01963366 POSTGRESPORT 5432
45611605-F2D9-40C8-8C0C-251E300E183C TESTDB mytest
FA8E2E4E-014C-4C1C-907E-64BAE6854D72 SERVICEIP 10.90.30.40
45B76C68-8A0F-4FD3-882F-CA579EC799A6 TESTSERVICE mytest-service
Required output is
id value
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://1.2.3.4:5432/mytest
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://10.90.30.40/svc/mytest-service/
If you want to use Python-like named placeholders then you need the helper function written on plpythonu:
create extension plpythonu;
create or replace function formatpystring( str text, a json ) returns text immutable language plpythonu as $$
import json
d = json.loads(a)
return str.format(**d)
$$;
Then simple test:
select formatpystring('{foo}.{bar}', '{"foo": "win", "bar": "amp"}');
formatpystring
----------------
win.amp
Finally you need to compose those arguments from your tables. It is simple:
select t1.id, formatpystring(t1.value, json_object_agg(t2.placeholder, t2.value)) as value
from table1 as t1, table2 as t2
group by t1.id, t1.value;
(Query was not tested but you have the direction)
(Clumsy) dynamic SQL implementation, featuring an outer join, but generating a recursive function call:
This function will not be very efficient, but probably the translation table is relatively small.
CREATE TABLE xlat_table (aa text ,bb text);
INSERT INTO xlat_table (aa ,bb ) VALUES( 'BBB', '/1.2.3.4/')
,( 'ccc', 'OMG') ,( 'ddd', '/4.3.2.1/') ;
CREATE FUNCTION dothe_replacements(_arg1 text) RETURNS text
AS
$func$
DECLARE
script text;
braced text;
res text;
found record; -- (aa text, bb text, xx text);
BEGIN
script := '';
res := format('%L', _arg1);
for found IN SELECT xy.aa,xy.bb
, regexp_matches(_arg1, '{\w+}','g' ) AS xx
FROM xlat_table xy
LOOP
-- RAISE NOTICE '#xx=%', found.xx[1];
-- RAISE NOTICE 'aa=%', found.aa;
-- RAISE NOTICE 'bb=%', found.bb;
braced := '{'|| found.aa || '}';
IF (found.xx[1] = braced ) THEN
-- RAISE NOTICE 'Res=%', res;
script := format ('replace(%s, %L, %L)'
,res,braced,found.bb);
res := format('%s', script);
END IF;
END LOOP;
if(length(script) =0) THEN return res; END IF;
script :='Select '|| script;
-- RAISE NOTICE 'script=%', script;
EXECUTE script INTO res;
return res;
END;
$func$
LANGUAGE plpgsql;
SELECT dothe_replacements( 'aaa{BBB}ccc{ddd}eee' );
SELECT dothe_replacements( '{AAA}bbb{CCC}DDD}{EEE}' );
Results:
CREATE TABLE
INSERT 0 3
CREATE FUNCTION
dothe_replacements
-----------------------------
aaa/1.2.3.4/ccc/4.3.2.1/eee
(1 row)
dothe_replacements
--------------------------
'{AAA}bbb{CCC}DDD}{EEE}'
(1 row)
The above method has quadratic behaviour(wrt the numberof xlat-entries); which is horrible.
But,we could dynamically create a function (once) and call it multiple times
(a poor man's generator)
Selecting only the relevant entries from the xlat table should probably be added.
And, you should of course re-create the function everytime the xlat table is changed.
CREATE FUNCTION create_replacement_function(_name text) RETURNS void
AS
$func$
DECLARE
argname text;
res text;
script text;
braced text;
found record; -- (aa text, bb text, xx text);
BEGIN
script := '';
argname := '_arg1';
res :=format('%I', argname);
for found IN SELECT xy.aa,xy.bb
FROM xlat_table xy
LOOP
-- RAISE NOTICE 'aa=%', found.aa;
-- RAISE NOTICE 'bb=%', found.bb;
-- RAISE NOTICE 'Res=%', res;
braced := '{'|| found.aa || '}';
script := format ('replace(%s, %L, %L)'
,res,braced,found.bb);
res := format('%s', script);
END LOOP;
script :=FORMAT('CREATE FUNCTION %I (_arg1 text) RETURNS text AS
$omg$
BEGIN
RETURN %s;
END;
$omg$ LANGUAGE plpgsql;', _name, script);
RAISE NOTICE 'script=%', script;
EXECUTE script ;
return ;
END;
$func$
LANGUAGE plpgsql;
SELECT create_replacement_function( 'my_function');
SELECT my_function('aaa{BBB}ccc{ddd}eee' );
SELECT my_function( '{AAA}bbb{CCC}DDD}{EEE}' );
And the result:
CREATE FUNCTION
NOTICE: script=CREATE FUNCTION my_function (_arg1 text) RETURNS text AS
$omg$
BEGIN
RETURN replace(replace(replace(_arg1, '{BBB}', '/1.2.3.4/'), '{ccc}', 'OMG'), '{ddd}', '/4.3.2.1/');
END;
$omg$ LANGUAGE plpgsql;
create_replacement_function
-----------------------------
(1 row)
my_function
-----------------------------
aaa/1.2.3.4/ccc/4.3.2.1/eee
(1 row)
my_function
------------------------
{AAA}bbb{CCC}DDD}{EEE}
(1 row)
The following offers a plpgsql solution in a with a single function.
You'll notice I've 'renamed' the value column. It's bad practice using rserved/key words as object names. Also soq is the schema I use for all SO code.
The process first takes the holder-values from table2 and generates a set of key-value pairs (in this case hstore, but jsonb would also work). It then builds an array from the value column (my column name: val_string) containing the place_holder name from the value. Finally, it iterates that array replacing the actual holder-name with the value from the key-values using the array value as the lookup key.
The performance would not be great with a larger volume from either table. If you need to process a large volume at a time to a single row temp table may yield better performance.
create or replace function soq.replace_holders( place_holder_line_in text)
returns text
language plpgsql
as $$
declare
l_holder_values hstore;
l_holder_line text;
l_holder_array text[];
l_indx integer;
begin
-- transform cloumns to key-value pairs of holder-value
select string_agg(place,',')::hstore
into l_holder_values
from (
select concat( '"',place_holder,'"=>"',place_value,'"') place
from soq.table2
) p;
-- raise notice 'holder_array_in==%',l_holder_values;
-- extract the text line and build array of place_holder names
select phv, string_to_array (string_agg(v,','),',')
into l_holder_line,l_holder_array
from (
select replace(replace(place_holder_line_in,'{',''),'}','') phv
, replace(replace(replace(regexp_matches(place_holder_line_in,'({[^}]+})','g')::text ,'{',''),'}',''),'"','') v
) s
group by phv;
-- raise notice 'Array==%',l_holder_array::text;
-- replace each key from text line with the corresponding value
for l_indx in 1 .. array_length(l_holder_array,1)
loop
l_holder_line = replace(l_holder_line,l_holder_array[l_indx],l_holder_values -> l_holder_array[l_indx]);
end loop;
-- done
return l_holder_line;
end;
$$;
-- Test driver
select id, soq.replace_holders(val_string) result_value from soq.table1;
I have created a simple query for this solution and it working as required.
WITH RECURSIVE cte(id, value, level) AS (
SELECT id,value, 0 as level
FROM Table1
UNION
SELECT ts.id,replace(ts.value,'{'||tp.placeholder||'}',tp.value) as value, level+1
FROM cte ts, Table2 tp WHERE ts.value LIKE CONCAT('%',tp.placeholder, '%')
)
SELECT id, value FROM cte c
where level =
(
select Max(level)
from cte c2 where c.id=c2.id
)
Output is
id value
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://10.90.30.40/svc/mytest-service/
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://1.2.3.4:5432/mytest

Multiple ALTER TABLE ADD COLUMN in one SQL function call

I came across some weird behaviour I'd like to understand.
I create a plpgsql function doing nothing except of ALTER TABLE ADD COLUMN. I call it 2 times on the same table:
A) In a single SELECT sentence
B) In a SQL function with same SELECT as in A)
Results are different: A) creates two columns, while B) creates only one column. Why?
Code:
CREATE FUNCTION add_text_column(table_name text, column_name text) RETURNS VOID
LANGUAGE plpgsql
AS $fff$
BEGIN
EXECUTE '
ALTER TABLE ' || table_name || '
ADD COLUMN ' || column_name || ' text;
';
END;
$fff$
;
-- this function is called only in B
CREATE FUNCTION add_many_text_columns(table_name text) RETURNS VOID
LANGUAGE SQL
AS $fff$
WITH
col_names (col_name) AS (
VALUES
( 'col_1' ),
( 'col_2' )
)
SELECT add_text_column(table_name, col_name)
FROM col_names
;
$fff$
;
-- A)
CREATE TABLE a (id integer);
WITH
col_names (col_name) AS (
VALUES
( 'col_1' ),
( 'col_2' )
)
SELECT add_text_column('a', col_name)
FROM col_names
;
SELECT * FROM a;
-- B)
CREATE TABLE b (id integer);
SELECT add_many_text_columns('b');
SELECT * FROM b;
Result:
CREATE FUNCTION
CREATE FUNCTION
CREATE TABLE
add_text_column
-----------------
(2 rows)
id | col_1 | col_2
----+-------+-------
(0 rows)
CREATE TABLE
add_many_text_columns
-----------------------
(1 row)
id | col_1
----+-------
(0 rows)
I'm using PostgreSQL 10.4. Please note that this is only a minimal working example, not the full functionality I need.
CREATE OR REPLACE FUNCTION g(i INTEGER)
RETURNS VOID AS $$
BEGIN
RAISE NOTICE 'g called with %', i;
END
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS VOID AS $$
SELECT g(id)
FROM generate_series(1, i) id;
$$ LANGUAGE SQL;
What do you think happens when I run SELECT t(4)? The only statement printed from g() is g called with 1.
The reason for this is your add_many_text_columns function returns a single result (void). Because it's SQL and is simply returning the result of a SELECT statement, it seems to stop executing after getting the first result, which makes sense if you think of it - it can only return one result after all.
Now change the function to:
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS SETOF VOID AS $$
SELECT g(id)
FROM generate_series(1, i) id;
$$ LANGUAGE SQL;
And run SELECT t(4) again, and now this is printed:
g called with 1
g called with 2
g called with 3
g called with 4
Because the function now returns SETOF VOID, it doesn't stop after the first result and executes it fully.
So back to your functions, you could change your SQL function to return SETOF VOID, but it doesn't really make much sense - better I think to change it to plpgsql and have it do a PERFORM:
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS VOID AS $$
BEGIN
PERFORM g(id)
FROM generate_series(1, i) id;
END
$$ LANGUAGE plpgsql;
That will execute the statement fully and it still returns a single VOID.
eurotrash provided a good explanation.
Alternative solution 1
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS VOID AS
$func$
SELECT g(id)
FROM generate_series(1, i) id;
SELECT null::void;
$func$ LANGUAGE sql;
Because, quoting the manual:
SQL functions execute an arbitrary list of SQL statements, returning
the result of the last query in the list. In the simple (non-set)
case, the first row of the last query's result will be returned.
By adding a dummy SELECT at the end we avoid that Postgres stops after processing the the first row of the query with multiple rows.
Alternative solution 2
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS VOID AS
$func$
SELECT count(g(id))
FROM generate_series(1, i) id;
$func$ LANGUAGE sql;
By using an aggregate function, all underlying rows are processed in any case. The function returns bigint (that's what count() returns), so we get the number of rows as result.
Alternative solution 3
If you need to return void for some unknown reason, you can cast:
CREATE OR REPLACE FUNCTION t(i INTEGER)
RETURNS VOID AS
$func$
SELECT count(g(id))::text::void
FROM generate_series(1, i) id;
$func$ LANGUAGE sql;
The cast to text is a stepping stone because the cast from bigint to void is not defined.

Getting an array return on function

I would like to have this function return as an array which contains all ID (integer) from this query, but i am stuck here:
CREATE OR REPLACE FUNCTION public.all_id(
prm_id integer)
RETURNS SETOF integer
LANGUAGE 'plpgsql'
COST 100.0
AS $function$
DECLARE
all_id integer;
-- all_id integer[]; gives an error, while integer only returns last value
BEGIN
SELECT id
COLLECT INTO all_id
FROM subject_data
WHERE sab_subject = (
SELECT sab_subject
FROM subject_data
WHERE id = prm_id
);
RETURN NEXT all_id;
END;
$function$;
SELECT * FROM public.all_id(1);
here's an example of fn:
t=# create or replace function ar() returns int[] as $$
declare ia int[];
begin
select array_agg(oid::int) into ia from pg_database;
return ia;
end;
$$ language plpgsql;
CREATE FUNCTION
t=# select * from ar();
ar
-----------------------------------------------------------
{13505,16384,1,13504,16419,16816,17135,25542,25679,25723}
(1 row)
SETOF would return "table", language is not literal and thus does not require single quotes, and so on...
as a_horse_with_no_name correctly brings out, you don't need plpgsql for this. A simple query will work:
SELECT array_agg(id)
FROM subject_data
WHERE sab_subject = (
SELECT sab_subject
FROM subject_data
WHERE id = PRM_ID_VALUE
)

cannot pass dynamic query to sql-function

I cannot seem to find a way to pass my query as a parameter to my sql-function. My problem is table 'my_employees1' could be dynamic.
DROP FUNCTION function_test(text);
CREATE OR REPLACE FUNCTION function_test(text) RETURNS bigint AS '
DECLARE ret bigint;
BEGIN
SELECT count(mt.id) INTO ret
FROM mytable as mt
WHERE mt.location_id = 29671
--and mt.employee_id in (SELECT id from my_employees1);
--and mt.employee_id in ($1);
$1;
RETURN ret;
END;
' LANGUAGE plpgsql;
select function_test('and mt.employee_id in (SELECT id from my_employees1)');
select function_test('SELECT id from my_employees1');
It must be dynamically built:
DROP FUNCTION function_test(text);
CREATE OR REPLACE FUNCTION function_test(text) RETURNS bigint AS $$
DECLARE
ret bigint;
BEGIN
execute(format($q$
SELECT count(mt.id) INTO ret
FROM mytable as mt
WHERE mt.location_id = 29671
%s; $q$, $1)
);
RETURN ret;
END;
$$ LANGUAGE plpgsql;
The $$ and $q$ are dollar quotes. They can be nested as long as the inner identifier is different. In addition to the obvious advantages of permitting the use of unquoted quotes and being nestable it also let the syntax highlighting do its work.

How do I pass in the array output of SQL query into a PostgreSQL (PL/pgSQL) function?

I am able to do the following in SQL where an "array" of user_ids are passed into the where clause of a SQL query.
select * from users where id in (select user_id from profiles);
I would like to do the same thing but pass the "array" into a PostgreSQL (PL/pgSQL) function as shown below. How do I declare the function and work with the "array" within the function?
select * from users_function(select user_id from profiles);
CREATE OR REPLACE FUNCTION users_function(....)
RETURNS void AS
$BODY$
....
Declare an array datatype [] in the function then use the aggregate function array_agg to transform the select statement into an array.
CREATE OR REPLACE FUNCTION users_function(myints integer[])
$$
BEGIN
-- you need to find the bounds with array_lower and array_upper
FOR i in array_lower(myints, 1) .. array_upper(myints, 1) LOOP
Raise Notice '%', myints[i]::integer;
END LOOP;
END;
$$
select * from users_function(array_agg((select user_id from profiles)));
I could not get the nate c's array_agg approach as I described above. This is an option:
select * from test_array('{1,2}');
CREATE OR REPLACE FUNCTION test_array(user_ids integer[])
RETURNS void AS
$$
declare
begin
FOR i in array_lower(user_ids, 1) .. array_upper(user_ids, 1) LOOP
RAISE NOTICE '%', user_ids[i]::integer;
END LOOP;
end
$$
LANGUAGE plpgsql;