Effective way how to handle application settings in PL/pgSQL functions - sql

Consider following situation: I have PL/pgSQL function which checks, If given auditor has some prerequisites for QS Auditor function. Thresholds of this prerequisites are defined in separate table quasar_settings. Every time, If is the function called, is executed SELECT which retrieves these prerequisites. This is quite inefficient, because this SELECT is called for every row. This quasar_settings table contains only one row. Is there any other more effective solution (global variable, caching, etc)?
Table quasar_settings has only one row.
Using PostgreSQL 9.3
PL/pgSQL function
CREATE OR REPLACE FUNCTION qs_auditor_training_auditing(auditor quasar_auditor) RETURNS boolean AS $$
DECLARE
settings quasar_settings%ROWTYPE;
BEGIN
SELECT s INTO settings
FROM quasar_settings s LIMIT 1;
RETURN auditor.nb1023_procedures_hours >= settings.qs_auditor_nb1023_procedures AND
-- MD Training
auditor.mdd_hours + auditor.ivd_hours >= settings.qs_auditor_md_training AND
-- ISO 9001 Trainig
(
auditor.is_aproved_for_iso13485 OR
(auditor.is_aproved_for_iso9001 AND auditor.iso13485_hours >= settings.qs_auditor_iso13485_training) OR
(auditor.iso13485_hours + auditor.iso9001_hours >= settings.qs_auditor_class_room_training)
);
END;
$$ LANGUAGE plpgsql;
Example of usage:
SELECT auditor.id, qs_auditor_training_auditing(auditor) FROM quasar_auditor auditor;

Do a cross join to the settings table in instead of calling the function at every row
select
a.id,
a.nb1023_procedures_hours >= s.qs_auditor_nb1023_procedures and
-- md training
a.mdd_hours + a.ivd_hours >= s.qs_auditor_md_training and
-- iso 9001 trainig
(
a.is_aproved_for_iso13485 or
(
a.is_aproved_for_iso9001 and
a.iso13485_hours >= s.qs_auditor_iso13485_training
) or
(a.iso13485_hours + a.iso9001_hours >= s.qs_auditor_class_room_training)
)
from
quasar_auditor a
cross join
quasar_settings s

Related

Declare variables in scheduled query BigQuery;

I am developing a scheduled query where I am using the WITH statement to join and filtrate several tables from BigQuery. To filtrate the dates, I would like to declare the following variables:
DECLARE initial, final DATE;
SET initial = DATE_TRUNC(DATE_TRUNC(CURRENT_DATE(), MONTH)+7,ISOWEEK);
SET final = LAST_DAY(DATE_TRUNC(CURRENT_DATE(), MONTH)+7, ISOWEEK);
However, when executing this query, I am getting two results; one for the variables declared (which I am not interested in having them as output), and the WITH statement that is selected at the end (which as the results that I am interested in).
The principal problem is that, whenever I try t connect this scheduled query to a table in Google Data Studio I get the following error:
Invalid value: configuration.query.destinationTable cannot be set for scripts;
How can I declare a variable without getting it as a result at the end?
Here you have a sample of the code I am trying work in:
DECLARE initial, final DATE;
SET initial = DATE_TRUNC(DATE_TRUNC(CURRENT_DATE(), MONTH)+7,ISOWEEK);
SET final = LAST_DAY(DATE_TRUNC(CURRENT_DATE(), MONTH)+7, ISOWEEK);
WITH HelloWorld AS (
SELECT shop_date, revenue
FROM fulltable
WHERE shop_date >= initial
AND shop_date <= final
)
SELECT * from HelloWorld;
with initial1 as ( select DATE_TRUNC(DATE_TRUNC(CURRENT_DATE(), MONTH)+7,ISOWEEK) as initial2),
final1 as ( select LAST_DAY(DATE_TRUNC(CURRENT_DATE(), MONTH)+7, ISOWEEK) as final2),
HelloWorld AS (
SELECT shop_date, revenue
FROM fulltable
WHERE shop_date >= (select initial2 from initial1) AND shop_date <= (select final2 from final1)
)
SELECT * from HelloWorld;
With config table having just 1 row and cross-joining it with your table, your query can be written like below.
WITH config AS (
SELECT DATE_TRUNC(DATE_TRUNC(CURRENT_DATE(), MONTH)+7,ISOWEEK) AS initial,
LAST_DAY(DATE_TRUNC(CURRENT_DATE(), MONTH)+7, ISOWEEK) AS final
),
HelloWorld AS (
SELECT * FROM UNNEST([DATE '2022-06-06']) shop_date, config
WHERE shop_date >= config.initial AND shop_date <= config.final
)
SELECT * FROM HelloWorld;
A few patterns I've used:
If you have many that have the same return type (STRING)
CREATE TEMP FUNCTION config(key STRING)
RETURNS STRING AS (
CASE key
WHEN "timezone" THEN "America/Edmonton"
WHEN "something" THEN "Value"
END
);
Then use config(key) to retrieve the value.
Or,
Create a function for each constant
CREATE TEMP FUNCTION timezone()
RETURNS STRING AS ("America/Edmonton");
Then use timezone() to get the value.
It would execute the function each time, so don't do something expensive in there (like SELECT from another table).

In PostgreSQL what does the CREATE AGGREGATE option SORTOP do?

From the Postgres documentation (https://www.postgresql.org/docs/9.6/sql-createaggregate.html) I find it hard to deduce what the parameter SORTOP does.
Is this option only applicable to an ordered-set aggregate?
Concretely I'm trying to create an aggregate function that finds the most frequent number in a column of numbers. I thought specifying the SORTOP option would sort the data before executing my self defined aggregate function, but this doesn't seem to be the case.
Here is my current implementation that only works when the input data is sorted.
It loops over the rows and keeps track of the largest sequence of previous numbers (largfreq variables in state) and the amount of repetitions seen so far of the number that it's currently on (currfreq variables in state).
CREATE TYPE largfreq_state AS (
largfreq_val INT,
largfreq INT,
currfreq_val INT,
currfreq INT
);
CREATE FUNCTION slargfreq(state largfreq_state, x INT) RETURNS largfreq_state AS $$
BEGIN
if state.currfreq_val <> x then
if state.currfreq >= state.largfreq then
state.largfreq = state.currfreq;
state.largfreq_val = state.currfreq_val;
end if;
state.currfreq = 1;
state.currfreq_val = x;
else
state.currfreq = state.currfreq + 1;
end if;
return state;
END;
$$ language plpgsql;
CREATE FUNCTION flargfreq(state largfreq_state) RETURNS INT AS $$
BEGIN
if state.currfreq >= state.largfreq then
return state.currfreq_val;
else
return state.largfreq_val;
end if;
END;
$$ language plpgsql;
CREATE AGGREGATE largfreq(INT) (
SFUNC = slargfreq,
STYPE = largfreq_state,
FINALFUNC = flargfreq,
INITCOND = '(0, 0, 0, 0)',
SORTOP = <
);
This is well explained in the documentation:
Aggregates that behave like MIN or MAX can sometimes be optimized by looking into an index instead of scanning every input row. If this aggregate can be so optimized, indicate it by specifying a sort operator. The basic requirement is that the aggregate must yield the first element in the sort ordering induced by the operator; in other words:
SELECT agg(col) FROM tab;
must be equivalent to:
SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
So you need that for aggregates that can be calculated using an index scan.

postgres access a cte inside a view

is it possible, in postgresql - to access a cte defined in a view?
By that I mean - if you have the following:
create view my_view as
with
blah as (select 1 as x, 2 as y, 3 as z)
select
x*x as x_squared,
y*y as y_squared,
z*z as z_squared
from
blah
is there any way from outside of getting at blah? eg looking for something like:
select * from my_view.blah
Basically we have large views that use a number of complicated CTE's - and it's quite difficult sometimes to troubleshoot them without splitting them all out into separate smaller views [yes, I would prefer to just keep it like that, but I don't have that option]
I know I will be able to do it by making a stored proc that pulls out the view definition - extracts the with clauses, parses up to the blah definition, changes that to the main select, gets rid of the rest, and then does the query - but that all seems like a lot of work. Am hoping there's a built-in way?
You can create the CTE as a view by itself. For example:
create table a (b int);
insert into a (b) values (1), (50), (200), (350), (1000);
create view blah as
select * from a where b > 100;
Anf then base your original view on this new intermediate one to avoid repeating code:
create view my_view as
select * from blah where b < 500;
See running example at DB Fiddle.
ok - so I have a sort of solution - it's a not a function - it's a procedure that turns a cte into a materialized_view.
I first wanted to formulate it as function, so you could say:
select * from cte_from( 'my_real_view', 'the_cte' )
but it appears that a function needs its schema defined in advance, which is obviously impossible in this case. If anyone can suggest a hack to make it closer to above, I'd apperciate it. But anyway - bottom line this works:
create procedure from_cte(view_schema text, view_name text, cte_name text, materialized_view_name text) as
$func$
declare
_code text;
_others text;
_script text;
begin
execute format('drop materialized view if exists %s', materialized_view_name);
with recursive
string_provider as (
select view_definition as the_string,
position(concat(cte_name, ' as (') in lower(view_definition)) + length(cte_name) + 5 as start_location
from information_schema.views
where table_name = view_name
and table_schema = view_schema
),
string_walk as (
select start_location as x,
1 as brackets
from string_provider
union
select x + 1,
new_brackets
from string_provider,
string_walk,
lateral (select case
when substring(the_string from x + 1 for 1) = '(' then brackets + 1
when substring(the_string from x + 1 for 1) = ')' then brackets - 1
else brackets
end as new_brackets
) calculated
where new_brackets != 0
and x < length(the_string)
)
select substring(the_string from start_location for 1 + max(x) - start_location),
trim(substring(the_string from 0 for start_location - length(cte_name) - 5))
into _code, _others
from string_walk,
string_provider
group by the_string, start_location;
if length(_others) < 5 then
select _code into _script;
else
select concat(substring(_others from 0 for length(_others)), ' ', _script) into _script;
end if;
execute format('create materialized view %s as ( %s )', materialized_view_name, _code);
end
$func$ language plpgsql;

Problem in calling postgres function from .net

I have a function of day closing in postgresql
CREATE OR REPLACE FUNCTION transactions.start_eod_operation(
value_date_ date,
user_id_ integer)
RETURNS void
LANGUAGE 'plpgsql'
COST 100
VOLATILE
AS $BODY$
BEGIN
IF NOT EXISTS(SELECT * FROM core.day_operation WHERE is_eod_completed = false AND value_date = value_date_) THEN
RAISE EXCEPTION 'BOD for this date was never started.';
END IF;
IF EXISTS(SELECT * FROM transactions.transactions WHERE verification_status_id = 0) THEN
RAISE EXCEPTION 'Voucher(s) pending for verification, cannot perform EOD!';
END IF;
DELETE FROM history.balance_holder WHERE value_date::date = value_date_;
INSERT INTO history.balance_holder(office_id, account_number_id, value_date, balance, minimum_balance, audit_user_id)
SELECT 1, t.account_number_id, value_date_, SUM(COALESCE(t.credit,'0') - COALESCE(t.debit, '0')), p.minimum_balance, user_id_
FROM deposit.transaction_view t
INNER JOIN deposit.account_holders a ON a.account_number_id = t.account_number_id
INNER JOIN core.deposit_products p ON a.deposit_product_id = p.deposit_product_id
WHERE t.value_date <= value_date_
GROUP BY t.account_number_id, p.minimum_balance;
PERFORM deposit.post_interest_transit(value_date_, user_id_);
PERFORM deposit.auto_transfer_interest(value_date_, user_id_);
PERFORM deposit.auto_transfer_fd($1, $2);
PERFORM loan.post_interest_transit(value_date_);
UPDATE core.day_operation SET is_eod_completed = true, eod_started_on=NOW(), eod_user_id = user_id_ WHERE value_date = value_date_;
END
$BODY$;
When user begin day close process from software
PERFORM deposit.post_interest_transit(value_date_, user_id_); this function or command in this function run twice a day not every day when there is problem in connection or server computer performance or in normal condition too. Inside this
PERFORM deposit.post_interest_transit(value_date_, user_id_); function there is delete command if there is data exist in that day and only begin insert command but it inserting 2 times in table. There is not any problem if I ran this function directly from database twice or more. What is the solution is this postgres bug or web browser bug ?

Creating Dynamic Dates as Variable (Column Names) in SQL

First, I have read about similar posts and have read the comments that this isn't an ideal solution and I get it but the boss (ie client) wants it this way. The parameters are as follows (for various reasons too bizarre to go into but trust me):
1. SQL Server Mgmt Studio 2016
2. NO parameters or pass throughs or temp tables. All has to be within contained code.
So here we go:
I need to create column headings that reflect dates:
1. Current date
2. Most recent quarter end prior to current date
3. Most recent quarter end prior to #2
4. Most recent quarter end prior to #3
5. Most recent quarter end prior to #4
6. Most recent quarter end prior to #5
So if using today's date, my column names would be as follows
12/18/2016 9/30/2016 6/30/2016 3/31/2016 12/31/2016 9/30/2015
I can easily do it in SAS but can't in SQL given the requirements stated above.
Help please with same code.
Thank you
Paula
Seems like a long way to go for something which really belongs in the presentation layer. That said, consider the following:
Let's assume you maintain a naming convention for your calculated fields, for example [CurrentDay], [QtrMinus1], [QtrMinus2], [QtrMinus3], [QtrMinus4],[QtrMinus5]. Then we can wrap your complicated query in some dynamic SQL.
Just as an illustration, let's assume your current query results looks like this
After the "wrap", the results will then look like so:
The code - Since you did NOT exclude Dynamic SQL.
Declare #S varchar(max)='
Select [CustName]
,['+convert(varchar(10),GetDate(),101)+'] = [CurrentDay]
,['+Convert(varchar(10),EOMonth(DateFromParts(Year(DateAdd(QQ,-1,GetDate())),DatePart(QQ,DateAdd(QQ,-1,GetDate()))*3,1)),101)+'] = [QtrMinus1]
,['+Convert(varchar(10),EOMonth(DateFromParts(Year(DateAdd(QQ,-2,GetDate())),DatePart(QQ,DateAdd(QQ,-2,GetDate()))*3,1)),101)+'] = [QtrMinus2]
,['+Convert(varchar(10),EOMonth(DateFromParts(Year(DateAdd(QQ,-3,GetDate())),DatePart(QQ,DateAdd(QQ,-3,GetDate()))*3,1)),101)+'] = [QtrMinus3]
,['+Convert(varchar(10),EOMonth(DateFromParts(Year(DateAdd(QQ,-4,GetDate())),DatePart(QQ,DateAdd(QQ,-4,GetDate()))*3,1)),101)+'] = [QtrMinus4]
,['+Convert(varchar(10),EOMonth(DateFromParts(Year(DateAdd(QQ,-5,GetDate())),DatePart(QQ,DateAdd(QQ,-5,GetDate()))*3,1)),101)+'] = [QtrMinus5]
From (
-- Your Complicated Query --
Select * from YourTable
) A
'
Exec(#S)
If it helps the visualization, the generated SQL is as follows:
Select [CustName]
,[12/18/2016] = [CurrentDay]
,[09/30/2016] = [QtrMinus1]
,[06/30/2016] = [QtrMinus2]
,[03/31/2016] = [QtrMinus3]
,[12/31/2015] = [QtrMinus4]
,[09/30/2015] = [QtrMinus5]
From (
-- Your Complicated Query --
Select * from YourTable
) A
Here is one way using dynamic query
DECLARE #prior_quarters INT = 4,
#int INT =1,
#col_list VARCHAR(max)=Quotename(CONVERT(VARCHAR(20), Getdate(), 101))
WHILE #int <= #prior_quarters
BEGIN
SELECT #col_list += Concat(',', Quotename(CONVERT(VARCHAR(20), Eomonth(Getdate(), ( ( ( ( Month(Getdate()) - 1 ) % 3 ) + 1 ) * -1 ) * #int), 101)))
SET #int+=1
END
--SELECT #col_list -- for debugging
EXEC ('select '+#col_list+' from yourtable')