i want to setup a function on PostgreSQL which returns a table. This is the source code of the function:
CREATE OR REPLACE FUNCTION feiertag(inDate Date)
RETURNS TABLE (eingabeDatum DATE, f_heute INT, f_1 INT, f_2 INT, f_3 INT, f_5 INT)
AS $$
DECLARE
f_heute integer := 0;
f_1 integer := 0;
f_2 integer := 0;
f_3 integer := 0;
f_5 integer := 0;
BEGIN
SELECT 1 INTO f_heute FROM feiertage where datum = inDate;
SELECT 1 INTO f_1 FROM feiertage where datum = (inDate + interval '1' day);
SELECT 1 INTO f_2 FROM feiertage where datum = (inDate + interval '2' day);
SELECT 1 INTO f_3 FROM feiertage where datum = (inDate + interval '3' day);
SELECT 1 INTO f_5 FROM feiertage where datum = (inDate + interval '5' day);
RETURN QUERY SELECT inDate as eingabeDatum, coalesce(f_heute, 0) as f_heute, coalesce(f_1,0) as f_1, coalesce(f_2,0) as f_2, coalesce(f_3,0) as f_3, coalesce(f_5,0) as f_5 ;
END;
$$ LANGUAGE plpgsql;
Calling the function returns only one column with ',' separated values:
psql (9.5.12)
Type "help" for help.
tarec=> select feiertag('2017-01-01');
feiertag
------------------------
(2017-01-01,1,0,0,0,0)
(1 row)
I expected differnt columns (one for each value as the table is specified at the beginning of the function) and not only one with all values. Does anybody know why this is happening and how i could fix this?
Thanks
Timo
Use
SELECT *
FROM feiertag('2017-01-01');
instead of
SELECT feiertag('2017-01-01');
to get the result as a table.
(Treat the function as if it were a table.)
Related
I am writing an SQL query for creating the partitions which looks like:
DO
$$
DECLARE
table_name text := 'table_1';
start_date date := (SELECT MIN(create_date)
FROM db.table);
end_date date := (SELECT MAX(create_date)
FROM db.table);
partition_interval interval := '1 day';
partition_column_value text;
BEGIN
FOR partition_column_value IN SELECT start_date +
(generate_series * extract(day from partition_interval)::integer)::date
FROM generate_series(0, extract(day from end_date - start_date::date) /
extract(day from partition_interval))
LOOP
EXECUTE format(
'create table if not exists %1$s_%2$s partition of %1$s for values in (%2$s) partition by list (create_date)',
table_name, partition_column_value::date);
END LOOP;
END
$$;
I get an error:
[42883] ERROR: function pg_catalog.extract(unknown, integer) does not exist
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Where: PL/pgSQL function inline_code_block line 9 at FOR over SELECT rows
The immediate cause of the error msg is this:
extract(day from end_date - start_date::date)
It's nonsense to cast start_date::date, start_date being type date to begin with. More importantly, date - date yields integer (not interval like you might assume). And extract() does not operate on integer input.
I removed more confusion and noise to arrive at this:
DO
$do$
DECLARE
table_name text := 'table_1';
partition_interval integer := 1; -- given in days!!
start_date date;
end_date date;
partition_column_value text;
BEGIN
SELECT INTO start_date, end_date -- two assignments for the price of one
min(create_date), max(create_date)
FROM db.table;
FOR partition_column_value IN
SELECT start_date + g * partition_interval -- date + int → date
FROM generate_series(0, (end_date - start_date) -- date - date → int
/ partition_interval) g
LOOP
EXECUTE format(
'CREATE TABLE IF NOT EXISTS %1$I PARTITION OF %1$I
FOR VALUES IN (%3$L) PARTITION BY LIST (create_date)'
, table_name || to_char(partition_column_value, '"_"yyyymmdd') -- !
, table_name
, partition_column_value::text -- only covers single day!!
);
END LOOP;
END
$do$;
This should work.
But it only makes sense for the example interval of '1 day'. For longer intervals, concatenate the list of days per partition or switch to range partitioning ...
I have 2 procedures: the first one is to fill a table "dm.dm_account_turnover_f" with data on sertain day and the second one is to log this procces in a table dm.lg_messages. The first one calls the second one and when I call my procedure several times (I need that to fill my table with all days in a month) it causes the error that the transaction wasn't finished correctly.
When I call my procdure only once there's no errors. How can I do it in a loop without error like that? Is there's something wrong in procedures?
PROCEDURE ds.fill_account_turnover_f()
declare
v_RowCount int;
begin
call dm.writelog( '[BEGIN] fill(i_OnDate => date '''
|| to_char(i_OnDate, 'yyyy-mm-dd')
|| ''');', 1
);
call dm.writelog( 'delete on_date = '
|| to_char(i_OnDate, 'yyyy-mm-dd'), 1
);
delete
from dm.dm_account_turnover_f f
where f.on_date = i_OnDate;
call dm.writelog('insert', 1);
insert
into dm.dm_account_turnover_f
( on_date
, account_rk
, credit_amount
, credit_amount_rub
, debet_amount
, debet_amount_rub
)
with wt_turn as
( select p.credit_account_rk as account_rk
, p.credit_amount as credit_amount
, p.credit_amount * nullif(er.reduced_cource, 1) as credit_amount_rub
, cast(null as numeric) as debet_amount
, cast(null as numeric) as debet_amount_rub
from ds.ft_posting_f p
join ds.md_account_d a
on a.account_rk = p.credit_account_rk
left
join ds.md_exchange_rate_d er
on er.currency_rk = a.currency_rk
and i_OnDate between er.data_actual_date and er.data_actual_end_date
where p.oper_date = i_OnDate
and i_OnDate between a.data_actual_date and a.data_actual_end_date
and a.data_actual_date between date_trunc('month', i_OnDate) and (date_trunc('MONTH', to_date(i_OnDate::TEXT,'yyyy-mm-dd')) + INTERVAL '1 MONTH - 1 day')
union all
select p.debet_account_rk as account_rk
, cast(null as numeric) as credit_amount
, cast(null as numeric) as credit_amount_rub
, p.debet_amount as debet_amount
, p.debet_amount * nullif(er.reduced_cource, 1) as debet_amount_rub
from ds.ft_posting_f p
join ds.md_account_d a
on a.account_rk = p.debet_account_rk
left
join ds.md_exchange_rate_d er
on er.currency_rk = a.currency_rk
and i_OnDate between er.data_actual_date and er.data_actual_end_date
where p.oper_date = i_OnDate
and i_OnDate between a.data_actual_date and a.data_actual_end_date
and a.data_actual_date between date_trunc('month', i_OnDate) and (date_trunc('MONTH', to_date(i_OnDate::TEXT,'yyyy-mm-dd')) + INTERVAL '1 MONTH - 1 day')
)
select i_OnDate as on_date
, t.account_rk
, sum(t.credit_amount) as credit_amount
, sum(t.credit_amount_rub) as credit_amount_rub
, sum(t.debet_amount) as debet_amount
, sum(t.debet_amount_rub) as debet_amount_rub
from wt_turn t
group by t.account_rk;
GET DIAGNOSTICS v_RowCount = ROW_COUNT;
call dm.writelog('[END] inserted ' || to_char(v_RowCount,'FM99999999') || ' rows.', 1);
commit;
end
PROCEDURE dm.writelog(, )
declare
log_NOTICE constant int := 1;
log_WARNING constant int := 2;
log_ERROR constant int := 3;
log_DEBUG constant int := 4;
c_splitToTable constant int := 4000;
c_splitToDbmsOutput constant int := 900;
v_logDate timestamp;
v_callerType varchar;
v_callerOwner varchar;
v_caller varchar;
v_line numeric;
v_message varchar;
begin
v_logDate := now();
-- split to log table
v_message := i_message;
i_messageType := log_NOTICE;
while length(v_message) > 0 loop
insert into dm.lg_messages (
record_id,
date_time,
pid,
message,
message_type,
usename,
datname,
client_addr,
application_name,
backend_start
)
select
nextval('dm.seq_lg_messages'),
now(),
pid,
substr(v_message, 1, c_splitToTable),
i_messageType,
usename,
datname,
client_addr,
application_name,
backend_start
from pg_stat_activity
where pid = pg_backend_pid();
v_message := substr(v_message, c_splitToTable + 1);
end loop;
commit;
end
I need fill my table with data for a month so I need to call my procedure 31 times. I tried this with python
my loop:
date_dt = datetime.strptime(input('Please input the last day of month in format "yyyy-mm-dd" '), "%Y-%m-%d")
while (date_dt - timedelta(days=1)).strftime("%m") == date_dt.strftime("%m"):
date_str = str(date_dt)
new_table.fill_account_turnover_f(date_str)
date_dt -= timedelta(days=1)
function that calls the procedure:
def fill_account_turnover_f(self, date_str):
cur = self.con.cursor()
cur.execute(f"CALL ds.fill_account_turnover_f('{date_str}');")
self.con.commit()
cur.close()
but it causes the error about invalid transaction termitation! it says something about COMMIT, string 51 and string 6 operator CALL
I have this code where I try to load a dimension table if the date of the current day has not been loaded yet. It is not loading any record, is my comparison wrong?
CREATE OR REPLACE PROCEDURE load_dimDate()
LANGUAGE plpgsql AS
$$
DECLARE
_date date := get_fecha();
_year int = get_year();
_month int = get_month();
_day int = get_day();
BEGIN
if _date <> (SELECT MAX(date) from dimDate) then
INSERT INTO dimfechas(date, year, month, day)
VALUES(_date, _year, _month, _day);
end if;
END
$$;
Both the variable _date and the select statement are of type DATE.
My function named stat() reads from 2 tables on PostgreSQL 11.
Table T has ~1,000,000 rows, the table D has ~3,000 rows.
My function stat() runs 1.5 secs and it is slow for my use-case:
select * from stat('2019-01-01', '2019-10-01','UTC');
To improve performance I tried to create different indexes (code below), but it did not help.
I was able to improve performance when I put the hardcoded numbers '2019-01-01', '2019-10-01' instead time_start and time_end in the body of stat().
In this case it runs 0.5 sec. But this is not the solution.
CREATE TABLE T(
id SERIAL PRIMARY KEY,
time TIMESTAMP WITH TIME zone NOT NULL,
ext_id INTEGER
);
CREATE TABLE D(
id SERIAL PRIMARY KEY,
time TIMESTAMP WITH TIME zone NOT NULL,
ext_id INTEGER NOT NULL
);
CREATE INDEX t_time_idx ON T(time);
CREATE INDEX d_time_idx ON D(time);
CREATE INDEX t_ext_idx ON T(ext_id);
CREATE INDEX d_ext_idx ON D(ext_id);
CREATE OR REPLACE FUNCTION stat(time_start varchar, time_end varchar, tz varchar)
RETURNS TABLE (result float)
AS $$
DECLARE
time_points INTEGER;
capacity INTEGER;
BEGIN
time_points := 1000;
capacity := 12;
RETURN QUERY
SELECT (total::float / (capacity * time_points))::float as result
FROM (
SELECT count(*)::float AS total FROM T
INNER JOIN (
SELECT * FROM (
SELECT ext _id, name, ROW_NUMBER() OVER(PARTITION BY ext_id ORDER BY time desc) AS rk
FROM D WHERE time at time zone tz < time_end::timestamp
) InB WHERE rk = 1
) D_INFO
ON T.ext_id = D_INFO.ext_id
WHERE T.time at time zone tz between time_start::timestamp and time_end::timestamp
) B;
END;
$$
LANGUAGE plpgsql;
Usage:
select * from stat('2019-01-01', '2019-10-01','UTC'); --> takes 1.5 sec, too slow
What I tried:
ANALYZE T;
ANALYZE D;
I created different indexes for T and D tables
CREATE INDEX covering_t_time_ext_idx ON t(ext_id) INCLUDE (time);
CREATE INDEX t_time_ext_idx ON T(time) INCLUDE (ext_id);
CREATE INDEX t_time_ext_multicolumn_idx ON t(time, ext_id);
CREATE INDEX t_time_ext_multicolumn2_idx ON t(ext_id, time);
but it did not help to improve performance.
function.
CREATE OR REPLACE FUNCTION stat(time_start varchar, time_end varchar, tz varchar)
RETURNS TABLE (result float)
AS $$
DECLARE
time_points INTEGER;
capacity INTEGER;
BEGIN
time_points := 1000;
capacity := 12;
RETURN QUERY
SELECT (total::float / (capacity * time_points))::float as result
FROM (
SELECT count(*)::float AS total
FROM T
WHERE T.time at time zone tz between time_start::timestamp and time_end::timestamp
AND EXISTS (
SELECT 1
FROM D
WHERE D.ext_id = T.ext_id
AND D.time at time zone tz < time_end::timestamp
)
) B;
END;
$$
LANGUAGE plpgsql;
I solve this by casting the input parameters:
(time_start varchar, time_end varchar)
into intermediate variables with type timestamp:
DECLARE
start_time timestamp;
end_time timestamp;
BEGIN
start_time := time_start::timestamp;
end_time := time_end::timestamp;
and using these intermediate variables in the SQL instead doing this casting in SQL.
I am doing the migration from Oracle to pgsql and I've the the oracle sql like below:
select code,product_no,qty qty from t_ma_tb_inventory
where code is not null and status=2
and update_type in (select * from table(splitstr(:types,',')))
and tb_shop_id=:shopId
order by update_time
and the splitstr function like below:
CREATE OR REPLACE FUNCTION splitstr (p_string text, p_delimiter text) RETURNS SETOF STR_SPLIT AS $body$
DECLARE
v_length bigint := LENGTH(p_string);
v_start bigint := 1;
v_index bigint;
BEGIN
WHILE(v_start <= v_length)
LOOP
v_index := INSTR(p_string, p_delimiter, v_start);
IF v_index = 0
THEN
RETURN NEXT SUBSTR(p_string, v_start);
v_start := v_length + 1;
ELSE
RETURN NEXT SUBSTR(p_string, v_start, v_index - v_start);
v_start := v_index + 1;
END IF;
END LOOP;
RETURN;
END
$body$
LANGUAGE PLPGSQL
SECURITY DEFINER
;
-- REVOKE ALL ON FUNCTION splitstr (p_string text, p_delimiter text) FROM PUBLIC;
can someone help me to write the equivalent code in pgsql?Thank you very much
You don't need to write your own function - Postgres already has a built-in function for that: string_to_array
select code, product_no, qty qty
from t_ma_tb_inventory
where code is not null and status=2
and update_type = any ( string_to_array(:types,',') )
and tb_shop_id = :shopId
order by update_time;
If you are passing a text, but you need to compare that to an integer, you need to cast the resulting array:
select code, product_no, qty qty
from t_ma_tb_inventory
where code is not null and status=2
and update_type = any ( string_to_array(:types,',')::int[] )
and tb_shop_id = :shopId
order by update_time;
However: that is a pretty ugly workaround that is only necessary in Oracle because it has no support for real arrays in SQL (only in PL/SQL).
In Postgres it would be much better to pass an array of integers directly (e.g. make :types an int[]). Then no parsing or casting would be necessary:
select code, product_no, qty qty
from t_ma_tb_inventory
where code is not null and status=2
and update_type = any ( :types )
and tb_shop_id = :shopId
order by update_time;