In PostgreSQL what does the CREATE AGGREGATE option SORTOP do? - sql

From the Postgres documentation (https://www.postgresql.org/docs/9.6/sql-createaggregate.html) I find it hard to deduce what the parameter SORTOP does.
Is this option only applicable to an ordered-set aggregate?
Concretely I'm trying to create an aggregate function that finds the most frequent number in a column of numbers. I thought specifying the SORTOP option would sort the data before executing my self defined aggregate function, but this doesn't seem to be the case.
Here is my current implementation that only works when the input data is sorted.
It loops over the rows and keeps track of the largest sequence of previous numbers (largfreq variables in state) and the amount of repetitions seen so far of the number that it's currently on (currfreq variables in state).
CREATE TYPE largfreq_state AS (
largfreq_val INT,
largfreq INT,
currfreq_val INT,
currfreq INT
);
CREATE FUNCTION slargfreq(state largfreq_state, x INT) RETURNS largfreq_state AS $$
BEGIN
if state.currfreq_val <> x then
if state.currfreq >= state.largfreq then
state.largfreq = state.currfreq;
state.largfreq_val = state.currfreq_val;
end if;
state.currfreq = 1;
state.currfreq_val = x;
else
state.currfreq = state.currfreq + 1;
end if;
return state;
END;
$$ language plpgsql;
CREATE FUNCTION flargfreq(state largfreq_state) RETURNS INT AS $$
BEGIN
if state.currfreq >= state.largfreq then
return state.currfreq_val;
else
return state.largfreq_val;
end if;
END;
$$ language plpgsql;
CREATE AGGREGATE largfreq(INT) (
SFUNC = slargfreq,
STYPE = largfreq_state,
FINALFUNC = flargfreq,
INITCOND = '(0, 0, 0, 0)',
SORTOP = <
);

This is well explained in the documentation:
Aggregates that behave like MIN or MAX can sometimes be optimized by looking into an index instead of scanning every input row. If this aggregate can be so optimized, indicate it by specifying a sort operator. The basic requirement is that the aggregate must yield the first element in the sort ordering induced by the operator; in other words:
SELECT agg(col) FROM tab;
must be equivalent to:
SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
So you need that for aggregates that can be calculated using an index scan.

Related

Postgresql column reference is ambiguous

I want to call my function but I get this error:
ERROR: column reference "list" is ambiguous LINE 3: SET
list = ARRAY_APPEND(list, input_list2),
the error is on the second list inside array_append function.
My function:
CREATE OR REPLACE FUNCTION update_order(input_id uuid,input_sku text,input_store_id uuid,input_order_date bigint,input_asin text,input_amount int,input_list text[],input_price real,input_list2 text) RETURNS void LANGUAGE plpgsql AS
$body$
#variable_conflict use_column
BEGIN
INSERT INTO orders_summary(id,sku,store_id,order_date,asin,amount,list,price)
VALUES(input_id,input_sku,input_store_id,to_timestamp(input_order_date / 1000.0),input_asin,input_amount,input_list,input_price) ON CONFLICT(sku,order_date) DO UPDATE
SET list = ARRAY_APPEND(list, input_list2),
amount = amount + input_amount,
price = input_price
WHERE NOT list #> input_list;
END
$body$;
You have to use the alias name in the insert query because list has two references, one reference in EXCLUDED.list and another reference to the column for an update statement.
Please check the below query (I append the alias with name os in query):
CREATE OR REPLACE FUNCTION update_order(input_id uuid,input_sku text,input_store_id uuid,input_order_date bigint,input_asin text,input_amount int,input_list text[],input_price real,input_list2 text) RETURNS void LANGUAGE plpgsql AS
$body$
#variable_conflict use_column
BEGIN
INSERT INTO orders_summary as os (id,sku,store_id,order_date,asin,amount,list,price)
VALUES(input_id,input_sku,input_store_id,to_timestamp(input_order_date / 1000.0),input_asin,input_amount,input_list,input_price) ON CONFLICT(sku,order_date) DO UPDATE
SET list = ARRAY_APPEND(os.list, input_list2),
amount = os.amount + input_amount,
price = input_price
WHERE NOT os.list #> input_list;
END
$body$;
Or you can use table name:
CREATE OR REPLACE FUNCTION update_order(input_id uuid,input_sku text,input_store_id uuid,input_order_date bigint,input_asin text,input_amount int,input_list text[],input_price real,input_list2 text) RETURNS void LANGUAGE plpgsql AS
$body$
#variable_conflict use_column
BEGIN
INSERT INTO orders_summary (id,sku,store_id,order_date,asin,amount,list,price)
VALUES(input_id,input_sku,input_store_id,to_timestamp(input_order_date / 1000.0),input_asin,input_amount,input_list,input_price) ON CONFLICT(sku,order_date) DO UPDATE
SET list = ARRAY_APPEND(orders_summary.list, input_list2),
amount = orders_summary.amount + input_amount,
price = input_price
WHERE NOT orders_summary.list #> input_list;
END
$body$;

How to compare two oracle sql object type with common base super class

Example:
Figure_t base class (super class)
sphere_t under figure_t
pyramid_t under figure_t
both has volume.
How to do objects comparison using map or order function?
What I am doing is using the map member function in the super class for comparing using the volume. I have tried with/without override of the map function in the subclass but still no luck. I can compare if I create the same object twice but not if I create different ones.
In example below I paste just the sphere since it is almost the same for both sphere and pyramid.
This is my super class:
CREATE OR REPLACE TYPE figure_t AS OBJECT (
v_volume NUMBER,
v_area NUMBER,
MAP MEMBER FUNCTION compare RETURN NUMBER, PRAGMA restrict_references ( compare, wnds, trust )
);
/
CREATE OR REPLACE TYPE BODY figure_t AS
MAP MEMBER FUNCTION compare RETURN NUMBER IS
BEGIN
RETURN v_volume;
END;
END;
/
ALTER TYPE figure_t NOT FINAL
CASCADE;
/
Then, this is my subtype:
CREATE OR REPLACE TYPE sphere_t UNDER figure_t (
v_radio NUMBER,
CONSTRUCTOR FUNCTION sphere_t (
radio NUMBER
) RETURN SELF AS RESULT,
MEMBER FUNCTION get_volume RETURN NUMBER,
MEMBER FUNCTION get_area RETURN NUMBER,
OVERRIDING MAP MEMBER FUNCTION compare RETURN NUMBER
);
/
CREATE OR REPLACE TYPE BODY sphere_t AS
CONSTRUCTOR FUNCTION sphere_t (
radio NUMBER
) RETURN SELF AS RESULT IS
BEGIN
self.v_radio := radio;
self.v_volume := ( 4 / 3 ) * 3.141592654 * power(radio, 3);
self.v_area := 4 * 3.141592654 * power(radio, 2);
return;
END;
MEMBER FUNCTION get_volume RETURN NUMBER IS
BEGIN
RETURN v_volume;
END;
MEMBER FUNCTION get_area RETURN NUMBER IS
BEGIN
RETURN v_area;
END;
OVERRIDING
MAP MEMBER FUNCTION compare RETURN NUMBER IS
BEGIN
RETURN self.v_volume;
END;
END;
/
For doing the comparison it looks like:
DECLARE
sphere_v sphere_t;
pyramid_v pyramid_t;
BEGIN
pyramid_v := pyramid_t(120, 90, 30);
sphere_v := sphere_t(10);
IF ( sphere_v != pyramid_v ) THEN
dbms_output.put_line('NOT EQUAL');
END IF;
END;
There should be a way for this comparison since figures have a super class in common.
There should be a way for this comparison since figures have a super
class in common
Am not very sure as what you wanted to achieve here. Also what is definition of pyramid_t which you are using in your comparison.
IF ( sphere_v != pyramid_v ) THEN
The above condition looks dicey to me as this will always be the case.
When you do sphere_v := sphere_t(10); means you try to get all the return of the sphere_t to sphere_v.
So it would be good if you could compare the volume and area of the Sphere and pyramid separately. See below demo how you could take these value:
DECLARE
sphere_v sphere_t;
-- pyramid_v pyramid_t;
BEGIN
sphere_v := sphere_t(10);
dbms_output.put_line('Input Radio -->'||sphere_v.v_radio);
dbms_output.put_line('Volume of Sphere-->'||sphere_v.v_volume);
dbms_output.put_line('Area Of Sphere -->'||sphere_v.v_area);
--Similarly you can take the values of `volume` and `area`
--of pyramid and get it compared with that of Sphere.
-- pyramid_v := pyramid_t(120, 90, 30);
-- dbms_output.put_line('Input Radio Pyramid -->'||pyramid_v.v_radio);
-- dbms_output.put_line('Volume of Pyramid -->'||pyramid_v.v_volume);
-- dbms_output.put_line('Area Of Pyramid -->'||pyramid_v.v_area);
-- If sphere_v.v_volume = pyramid_v.v_volume then
-- dbms_output.put_line('Equal');
-- Else
-- dbms_output.put_line('Not Equal');
END;
Assumption: pyramid_t also have the same Object Body definition having volume and area calculation.
There should be a way for this comparison since figures have a super class in common.
There is a way, it's just not obvious.
When the types are exactly the same e.g. two instances of the same subtype we can invoke the map function implicitly. So we can compare two spheres like this:
IF ( sphere_1 != sphere_2 ) THEN ...
However, to compare two different subtypes we need to invoke the parent map function, and to make this happen we must reference it explicitly:
IF ( sphere_v.compare() != pyramid_v.compare() ) THEN ...
Yes, this is clunky. But Oracle is an RDBMS not an ORDBMS (whatever they claimed back in the version 8.0 days).

Conditional WHERE expression in dynamic query

I have the following function
CREATE OR REPLACE FUNCTION match_custom_filter(filters text[], id text)
RETURNS boolean LANGUAGE plpgsql as $$
DECLARE
r boolean;
BEGIN
execute format(
'SELECT 1 FROM trackings t LEFT JOIN visitors v ON v.id = t.visitor_id
WHERE v.id = ''%s'' AND %s',
id,
array_to_string(filters, ') AND ('))
into r;
RETURN r;
END $$;
select v.*, array_agg(g.name) as groups from visitors v join groups g on match_custom_filter(g.formatted_custom_filters, v.id)
where v.id = 'cov4pisw00000sjctfyvwq126'
group by v.id
This works fine when the filters are not empty. But it is also possible that a filter is empty, in which case I will have an dangling AND with no right hand side.
Error:
ERROR: syntax error at end of input
LINE 2: ... WHERE v.id = 'cov4pisw00000sjctfyvwq126' AND
^
QUERY: SELECT 1 FROM trackings t LEFT JOIN visitors v ON v.id = t.visitor_id
WHERE v.id = 'cov4pisw00000sjctfyvwq126' AND
CONTEXT: PL/pgSQL function match_custom_filter(text[],text) line 5 at EXECUTE statement
What's the best way to handle this?
UPDATE:
Example of how I generate the array of string filters based off JSONB array of filter objects
def build_condition(%{"filter" => filter, "field" => field, "value" => value}) when field in #default_values do
case filter do
"greater_than" -> "#{field} > #{value}"
"less_than" -> "#{field} < #{value}"
"is" -> "#{field} = '#{value}'"
"is_not" -> "#{field} <> '#{value}'"
..
First, a warning. What you are doing here gives you in-stored-proc sql injection. I highly recommend you reconsider so you can properly parameterize.
Now, having said this, the obvious option is to declare a text variable and then pre-process it.
In your DECLARE block you add:
filterstring text;
then in your body, you add:
filterstring := array_to_string(filters, ') AND ('))
IF filterstring = '' or filterstring is null THEN
filterstring := 'TRUE';
END IF;
Then you use filterstring in place of the array_to_string call in the format() call.
Note that any time you assemble a query anywhere by string interpolation you have the possibility of sql injection.
To protect against SQL injection you will need to rethink your approach a little bit. Your best option is not to use format() for your query to the extent possible. So:
execute 'SELECT 1 FROM trackings t
LEFT JOIN visitors v ON v.id = t.visitor_id
WHERE v.id = $1'
USING id;
That causes planning and filling in the value to happen on two different points. That works well in the case of a simple parameter. However it doesn't work well in the case of the dynamic filters.
Instead of passing a one-dimensional array in, you could pass a two dimensional (nx3 array) with three elements per line. These would be column name, operator, and value. You can sanitize the column name by passing it through quote_ident and the value by passing it through quote_literal but sanitizing the operators is likely to be a problem so my recommendation would be to whitelist these and throw an exception if the operator is not found. Something like:
DECLARE
...
op TEXT;
allowed_ops TEXT[] := ARRAY['=', '<=', '>='];
BEGIN
...
IF not(op = ANY(allowed_ops)) THEN
RAISE EXCEPTION 'Illegal operator in function, %', op;
END IF;
...
END;
This is not going to be easy but it is doable.
Since you have your filters in the form of a jsonb array to begin with, you should use that as a function parameter instead of a text[]. For one thing, it will allow you to protect against SQL-injection.
CREATE OR REPLACE FUNCTION match_custom_filter(filters jsonb, id text)
RETURNS boolean LANGUAGE plpgsql AS $$
DECLARE
f text;
r boolean;
BEGIN
IF jsonb_array_length(filters) = 0 THEN
-- If no filters are specified then run a straight SQL query against trackings
PERFORM * FROM trackings WHERE visitor_id = quote_literal(id);
RETURN FOUND;
ELSE
-- Build the filters from the jsonb array
SELECT string_agg(
-- Concatenate the parts from a single json object into a filter
quote_ident(j->>'field') || -- avoid SQL injection on column name
CASE j->>'type'
WHEN 'greater_than' THEN ' > '
...
END ||
quote_literal(j->>'value'), -- avoid SQL injection on value
-- Aggregate individual filters with the AND operator
' AND ') INTO f
FROM jsonb_array_elements(filters) j;
-- Run a dynamic query with the filters
EXECUTE format('SELECT true FROM trackings t
LEFT JOIN visitors v ON v.id = t.visitor_id
WHERE v.id = %L AND %s LIMIT 1', id, f) INTO r;
RETURN r;
END IF;
END $$;
You should call this function passing in the jsonb array, like so:
SELECT v.*, array_agg(g.name) AS groups
FROM visitors v JOIN groups g ON match_custom_filter(g.group->'filter', v.id)
WHERE v.id = 'cov4pisw00000sjctfyvwq126'
GROUP BY v.id;

How to add only checked rows in APEX tabular forms to another table?

I've created a tabular form from a view - so only the checked items be added to a table. So far I can only insert all the records not individually selected rows.
This is the code I used:
DECLARE
v_insertcount NUMBER := 0;
BEGIN
FOR i IN 1 .. apex_application.g_f02.COUNT
LOOP
IF apex_application.g_f01 (i) IS NOT NULL THEN -- this is the checkbox
insert into my_table (pme_id, MYREF, STAC, START_DATE)
values ( SEQ_PME.NEXTVAL,
:P5_MYREF
apex_application.g_f02(i),
apex_application.g_f03(i)
)
v_insertcount := v_insertcount + 1;
END IF;
END LOOP;
END;
First you have to remember that only checked checkboxes are submitted. So you cannot test for NULL in order to identify unchecked boxes. That are simply not in the array.
So, as the checkbox array size is lower or equal to the other fields array size, you cannot use index to find corresponding values of the same row.
There are some clever tricks to overcome that limitations. However, for simple cases, a solution is simply to use a look-up function to map id to index.
FUNCTION array_search(value IN VARCHAR2, arr IN apex_application_global.vc_arr2)
RETURN PLS_INTEGER
IS
BEGIN
FOR i IN 1 .. arr.COUNT
LOOP
IF arr(i) = value THEN
RETURN i;
END IF;
END LOOP;
return 0;
END;
This is a linear search. So the performances are not very good. Anyway for form processing it is probably sufficient. Given that function, you can now write:
Region source
SELECT APEX_ITEM.CHECKBOX(1,pme_id) " ",
...
on-submit process
FOR i IN 1 .. apex_application.g_f01.COUNT -- your checkboxes
LOOP
-- identify the "row index"
idx := array_search(apex_application.g_f01(i), apex_application.g_f02);
-- ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
-- checkboxes array id array
-- assuming now idx > 0
-- (in production code, should check and log in case of `0`)
-- do whatever you want, using `idx` as index
insert into my_table
(pme_id,
MYREF,
STAC,
START_DATE
)
values (
SEQ_PME.NEXTVAL,
:P5_MYREF
apex_application.g_f02 (idx),
apex_application.g_f03 (idx)
);
END LOOP;

Effective way how to handle application settings in PL/pgSQL functions

Consider following situation: I have PL/pgSQL function which checks, If given auditor has some prerequisites for QS Auditor function. Thresholds of this prerequisites are defined in separate table quasar_settings. Every time, If is the function called, is executed SELECT which retrieves these prerequisites. This is quite inefficient, because this SELECT is called for every row. This quasar_settings table contains only one row. Is there any other more effective solution (global variable, caching, etc)?
Table quasar_settings has only one row.
Using PostgreSQL 9.3
PL/pgSQL function
CREATE OR REPLACE FUNCTION qs_auditor_training_auditing(auditor quasar_auditor) RETURNS boolean AS $$
DECLARE
settings quasar_settings%ROWTYPE;
BEGIN
SELECT s INTO settings
FROM quasar_settings s LIMIT 1;
RETURN auditor.nb1023_procedures_hours >= settings.qs_auditor_nb1023_procedures AND
-- MD Training
auditor.mdd_hours + auditor.ivd_hours >= settings.qs_auditor_md_training AND
-- ISO 9001 Trainig
(
auditor.is_aproved_for_iso13485 OR
(auditor.is_aproved_for_iso9001 AND auditor.iso13485_hours >= settings.qs_auditor_iso13485_training) OR
(auditor.iso13485_hours + auditor.iso9001_hours >= settings.qs_auditor_class_room_training)
);
END;
$$ LANGUAGE plpgsql;
Example of usage:
SELECT auditor.id, qs_auditor_training_auditing(auditor) FROM quasar_auditor auditor;
Do a cross join to the settings table in instead of calling the function at every row
select
a.id,
a.nb1023_procedures_hours >= s.qs_auditor_nb1023_procedures and
-- md training
a.mdd_hours + a.ivd_hours >= s.qs_auditor_md_training and
-- iso 9001 trainig
(
a.is_aproved_for_iso13485 or
(
a.is_aproved_for_iso9001 and
a.iso13485_hours >= s.qs_auditor_iso13485_training
) or
(a.iso13485_hours + a.iso9001_hours >= s.qs_auditor_class_room_training)
)
from
quasar_auditor a
cross join
quasar_settings s