PostgreSQL 11 - using format to assign to variable - sql

I have been awake for well beyond my schedule and I have been stuck with this issue for a long time, I don't even know what I am looking for to solve, but I wish to use format to insert values that I'll be using for column names, and then executing it... but it keeps giving me errors no matter how much I try changing it :c
Heres the part that im trying to do something that doesnt work, but i think you get the idea what im trying to achieve
ratelimit := EXECUTE format('(SELECT %I
FROM users.ratelimits
WHERE user_id = $2)
', $1);
and heres the full code for the brave
CREATE OR REPLACE FUNCTION users.consume_ratelimit(_name text,__user_id integer)
RETURNS boolean
LANGUAGE 'plpgsql'
VOLATILE
PARALLEL UNSAFE
COST 100
AS $BODY$DECLARE
ratelimit INTEGER;
reset_timeout timestamptz;
premium BOOLEAN;
BEGIN
ratelimit := EXECUTE format('(SELECT %I
FROM users.ratelimits
WHERE user_id = $2)
', $1);
reset_timeout := EXECUTE format('(SELECT %I_refresh
FROM users.ratelimits
WHERE user_id = $2)
', $1);
premium := (SELECT users.is_premium($2));
IF premium THEN
RETURN TRUE;
ELSIF reset_timeout <= NOW() THEN
UPDATE users.ratelimits
SET image_refresh = NOW() + '1 hour'::interval,
image = DEFAULT
WHERE user_id = $2;
RAISE NOTICE 'reset';
RETURN TRUE;
ELSE
IF ratelimit > 0 THEN
EXECUTE format('UPDATE users.ratelimits
SET %I = %I - 1
WHERE user_id = $2', $1, $1);
RAISE NOTICE 'decrement';
RETURN TRUE;
ELSIF ratelimit <= 0 THEN
RAISE NOTICE 'out of credits';
RETURN FALSE;
ELSE
EXECUTE format('INSERT INTO users.ratelimits(user_id) VALUES ($2)
ON CONFLICT DO UPDATE SET
%I = excluded.%I,
%I_refresh = excluded.%I_refresh', $1, $1, $1, $1);
RAISE NOTICE 'create';
RETURN TRUE;
END IF;
END IF;
END;$BODY$;

As documented in the manual you need to use into together with EXECUTE to store the result into a variable. This can handle multiple columns/variables as well, so you only need a single EXECUTE to get both values.
For clarity you should reference parameters by name, not by position.
EXECUTE format('SELECT %I, %I_refresh
FROM users.ratelimits WHERE user_id = $1'),
_name, _name)
USING __user_id
INTO ratelimit, reset_timeout;
Note the $1 inside the string for format() is a parameter placeholder used when the SQL statement is executed, and will be replaced with the value of the variable specified in the USING clause.
Variable assignment is also more efficient without a SELECT:
premium := users.is_premium(__user_id);

It seems some_var := EXECUTE ... will not work. Additionally, params inside of an EXECUTE statement are not the same as those in the function - you need to supply them to the execute statement.
I have not tested this, but perhaps
-- q declared above as text
q := format('
SELECT %I
FROM users.ratelimits
WHERE user_id = %s;
', $1, $2);
EXECUTE q INTO ratelimit;
will work? I also removed the parens from the query, those are unnecessary and may the problem themselves.
I have tested the function
CREATE OR REPLACE FUNCTION test_sum (a int, b int)
RETURNS int
AS $$
DECLARE
q text;
result int;
BEGIN
q := FORMAT ('SELECT %s + %s', $1, $2);
EXECUTE q INTO result;
RETURN result;
END
$$ LANGUAGE plpgsql
and that does work. If my suggestion above does not work, perhaps you can use the above as a starting point.

Related

Find SUM and MAX in custom aggregate function

I've been working recently with custom aggregate function.
In this custom aggregate, the first function, doesn't compute the sum and the max value correctly.
I'm using a composite type to return the sum and the max value.
I've tried with appending everything with the array, but it's not an efficient way to work
CREATE TYPE sum_max_complex AS (sum real, max_v real);
CREATE OR REPLACE FUNCTION calculateSum(sum_max_complex, real) RETURNS sum_max_complex AS $$
DECLARE
sumValue real := 0;
max_v real := $2;
output sum_max_complex;
BEGIN
RAISE NOTICE '-------------------';
RAISE NOTICE 'IL PRIMO VALORE DI INPUT E: % ... %',$1.sum,$1.max_v;
RAISE NOTICE 'IL SECONDO VALORE DI INPUT E: %',$2;
IF $2 IS NOT NULL THEN
sumValue := calculateSumAggregate(sumValue,$2) + sumValue;
ELSE
sumValue := sumValue;
END IF;
max_v := searchmaximumvalue(max_v,$2);
output.sum := sumValue;
output.max_v := max_v;
RAISE NOTICE '-------------------';
RAISE NOTICE 'IL VALORE DI OUTPUT SONO: % ... %',output.sum,output.max_v;
RETURN output;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION addLaplacianNoiseSum(sum_max_complex) RETURNS real AS $$
DECLARE
epsilon real := 0.005;
sensivity real := $1.max_v;
laplaceDistribution real;
BEGIN
laplaceDistribution := generaterandomvalues(sensivity / (epsilon));
RETURN $1.sum + laplaceDistribution;
END;
$$ LANGUAGE plpgsql;
CREATE AGGREGATE SUM_LAPLACE(real)
(
SFUNC = calculateSum,
STYPE = sum_max_complex,
FINALFUNC = addLaplacianNoiseSum
);
In my table column, I have as values: 19,22,22.5,27.
It takes the correct value in the $2 parameter method, in the 1st function, but doesn't accumulate and sum every value.
It doesn't look like you are ever adding to the values stored in the sum_max_complex type. Here's a simplified example that shows approximately what you should do. I don't know what calculateSumAggregate or generaterandomvalues do, so I wasn't able to reproduce those.
CREATE TYPE sum_max_complex AS (sum real, max_v real);
CREATE OR REPLACE FUNCTION calculateSum(sum_max_complex, real) RETURNS sum_max_complex AS $$
select ROW(
$1.sum + coalesce($2, 0),
greatest($1.max_v, $2)
)::sum_max_complex;
$$ LANGUAGE SQL IMMUTABLE;
CREATE OR REPLACE FUNCTION addLaplacianNoiseSum(sum_max_complex) RETURNS real AS $$
select $1.sum + ($1.max_v/0.005);
$$ LANGUAGE SQL IMMUTABLE;
CREATE AGGREGATE SUM_LAPLACE(real)
(
SFUNC = calculateSum,
STYPE = sum_max_complex,
FINALFUNC = addLaplacianNoiseSum,
INITCOND = '(0, 0)'
);
with a as (select a from (values (19), (22), (22.5), (27)) v(a))
select sum_laplace(a) from a;
sum_laplace
-------------
5490.5

PostgreSQL how to disable plan caching to fix my base62 hashing?

From the documentation i gather, that PostgreSQL somehow employs its own cache and prepares satements ahead...
This is really bad for my base62 hash values. At some point, after 2-3 tries, they start returning the same number:
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
Here is my function:
CREATE OR REPLACE FUNCTION base62_id() RETURNS character varying
LANGUAGE plpgsql IMMUTABLE
AS $$
DECLARE
chars char[];
ret varchar;
val bigint;
BEGIN
chars := ARRAY['0','1','2','3','4','5','6','7','8','9'
,'A','B','C','D','E','F','G','H','I','J','K','L','M'
,'N','O','P','Q','R','S','T','U','V','W','X','Y','Z'
,'a','b','c','d','e','f','g','h','i','j','k','l','m'
,'n','o','p','q','r','s','t','u','v','w','x','y','z'];
val := (CEIL(EXTRACT(EPOCH FROM now()) * 1000))::bigint;
RAISE LOG 'base62_id.val %', val;
ret := '';
IF val < 0 THEN
val := val * -1;
END IF;
WHILE val != 0 LOOP
ret := chars[(val % 62)+1] || ret;
val := val / 62;
END LOOP;
RETURN ret;
END;$$;
In theory, this should work...
Any ideas?
Edit: How i use the function:
DECLARE
copied_article_id text := base62_id();
duplication_check int := 1;
copied_article articles;
BEGIN
WHILE duplication_check IS NOT NULL LOOP
SELECT COUNT(*) INTO duplication_check FROM articles WHERE id = copied_article_id;
END LOOP;
INSERT ... INTO ... SELECT ...
FROM
articles
WHERE
id = base_id;
SELECT * INTO copied_article FROM articles WHERE id = copied_article_id LIMIT 1;
Your explanation is pretty bogus. Plan caching has nothing to do with this outcome, the plancache doesn't care about individual function outputs.
There's at least two giant gaping bugs in the function:
You declare it IMMUTABLE but you call now() which is not immutable. IMMUTABLE functions must return the same result for every call with the same inputs. In fact, your function must be declared VOLATILE if it's allowed to return a different value for each call with the same inputs.
now() is STABLE. It actually returns the same value for each call within a transaction. So presumably using it when you want unique values makes no sense at all. I imagine you actually want clock_timestamp().
The latter problem with now() (a.k.a. current_timestamp) being STABLE across a transaction is likely the cause for the results you report.
BTW, the function will also probably be amazingly slow implemented in plpgsql. If you can turn it into a set operation plus string_agg it might be more tolerable, but still slow.

Postgresql function auto add 's' character to the end of string

I have a function in PostgreSQL database. However, every time I run the function in PostgreSQL, it auto-adda the character s behind of the query, therefore, I receive an error when it executes.
An example query ends up looking like this:
WHAT IN HERE: query SELECT uom_id FROM ps_quantity where id = 11s
My version is PostgreSQL 9.2.13. How can I solve this? Thanks.
CREATE OR REPLACE FUNCTION select_field(
selected_table text,
selected_field text,
field_type_sample anyelement,
where_clause text DEFAULT ''::text)
RETURNS anyelement AS
$BODY$ DECLARE
-- Log
ME constant text := 'selected_field()';
-- Local variables
_qry varchar := '';
_result_value ALIAS FOR $0;
where_clause1 varchar := 'asdasdsad';
BEGIN
RAISE NOTICE 'FUNCTION: SELECT_FIELD';
-- BANGPH
RAISE NOTICE 'BANGPH - CHANGE 11s to 11';
_qry := 'SELECT uom_id FROM ps_quantity where id = 11';
RAISE NOTICE 'WHERE = %s', where_clause1;
RAISE NOTICE 'WHAT IN HERE: query %s', _qry;
As the manual explains:
instead of:
RAISE NOTICE 'WHAT IN HERE: query %s', _qry;
you need to use:
RAISE NOTICE 'WHAT IN HERE: query %', _qry;
Placeholders for the RAISE statement don't have a "type modifier", it's a plain %

Calling postgres function with multiple params

This is my sql function in postgresql:
FUNCTION test(year integer)
RETURNS SETOF json AS
$BODY$
SELECT ARRAY_TO_JSON(ARRAY_AGG(T))
FROM table t
WHERE year = $1;
$BODY$
This works quite good. But now I want specify more parameters and
I'd like to get a return with the condition if parameters are set. For example following function call:
test(year := 2014, location := 'Belo Horizonte')
How should the function look like and where to set conditions? Here is my (wrong) suggestion:
FUNCTION test(year integer, location text)
RETURNS SETOF json AS
$BODY$
SELECT ARRAY_TO_JSON(ARRAY_AGG(T))
FROM table t
IF $1 IS SET THEN
WHERE year = $1
ELSIF $2 THEN
UNION
WHERE location = $2
END IF;
$BODY$
A further challenge is a return of the function for this statements:
test(year := 1584)
-- should return all entries with year 1584
test(location := 'Cambridge')
-- should return all entries with location Cambridge
test(year := 1584, location := 'Cambridge')
-- should return all entries with year 2014 AND location Belo Horizonte
Thanks in advance!
You may try to do something like that, adding default values, and working with OR clauses
FUNCTION test(year integer DEFAULT -1, location text DEFAULT 'noLocation')
RETURNS SETOF json AS
$BODY$
SELECT ARRAY_TO_JSON(ARRAY_AGG(T))
FROM table t
WHERE ($1 = -1 OR year = $1)
AND ($2 = 'noLocation' OR location = $2);
$BODY$

How fix double encoding in PostgreSQL?

I have a table in PostgreSQL with words, but some words have invalid UTF-8 chars like 0xe7e36f and 0xefbfbd.
How I can identify all chars inside words that are invalid and replace they with some symbol like ??
EDIT: My database is in UTF-8, but I think there are double encoding from various other encodings. I think this because when I tried to convert to one type as LATIN1, I get an error saying that some char don't exists in that encoding, when I change to LATIN2 I get
the same error, but with another character.
So, what is possible to do to solve this?
Usage
It's a solution for my specific case, but maybe with some modifications can help another people.
Usage
SELECT fix_wrong_encoding('LATIN1');
Function
-- Convert words with wrong encoding
CREATE OR REPLACE FUNCTION fix_wrong_encoding(encoding_name VARCHAR)
RETURNS VOID
AS $$
DECLARE
r RECORD;
counter INTEGER;
token_id INTEGER;
BEGIN
counter = 0;
FOR r IN SELECT t.id, t.text FROM token t
LOOP
BEGIN
RAISE NOTICE 'Converting %', r.text;
r.text := convert_from(convert_to(r.text,encoding_name),'UTF8');
RAISE NOTICE 'Converted to %', r.text;
RAISE NOTICE 'Checking existence.';
SELECT id INTO token_id FROM token WHERE text = r.text;
IF (token_id IS NOT NULL) THEN
BEGIN
RAISE NOTICE 'Token already exists. Updating ids in textblockhastoken';
IF(token_id = r.id) THEN
RAISE NOTICE 'Token is the same.';
CONTINUE;
END IF;
UPDATE textblockhastoken SET tokenid = token_id
WHERE tokenid = r.id;
RAISE NOTICE 'Removing current token.';
DELETE FROM token WHERE id = r.id;
END;
ELSE
BEGIN
RAISE NOTICE 'Token don''t exists. Updating text in token';
UPDATE token SET text = r.text WHERE id = r.id;
END;
END IF;
EXCEPTION WHEN untranslatable_character THEN
--do nothing
WHEN character_not_in_repertoire THEN
--do nothing
END;
counter = counter + 1;
RAISE NOTICE '% token converted', counter;
END LOOP;
END
$$
LANGUAGE plpgsql;