Timeout on advisory locks in postgresql - sql

I'm migrating from ORACLE. Currently I'm trying to port this call:
lkstat := DBMS_LOCK.REQUEST(lkhndl, DBMS_LOCK.X_MODE, lktimeout, true);
This function tries to acquire lock lkhndl and returns 1 if it fails to get it after timeout seconds.
In postgresql I use
pg_advisory_xact_lock(lkhndl);
However, it seems that it waits for lock forever. pg_try_advisory_xact_lock returns immediately if fails. Is there a way to implement timeout version of lock acquiring?
There is lock_timeout setting, but I'm not sure is it applicable to advisory locks and how pg_advisory_xact_lock would behave after timeout.

This is a prototype of a wrapper that poorly emulates DBMS_LOCK.REQUEST - constrained to only one type of lock (transaction-scope advisory lock).
To make function fully compatible with Oracle's, it would need several hundreds lines. But that's a start.
CREATE OR REPLACE FUNCTION
advisory_xact_lock_request(p_key bigint, p_timeout numeric)
RETURNS integer
LANGUAGE plpgsql AS $$
/* Imitate DBMS_LOCK.REQUEST for PostgreSQL advisory lock.
Return 0 on Success, 1 on Timeout, 3 on Parameter Error. */
DECLARE
t0 timestamptz := clock_timestamp();
BEGIN
IF p_timeout NOT BETWEEN 0 AND 86400 THEN
RAISE WARNING 'Invalid timeout parameter';
RETURN 3;
END IF;
LOOP
IF pg_try_advisory_xact_lock(key) THEN
RETURN 0;
ELSIF clock_timestamp() > t0 + (p_timeout||' seconds')::interval THEN
RAISE WARNING 'Could not acquire lock in % seconds', p_timeout;
RETURN 1;
ELSE
PERFORM pg_sleep(0.01); /* 10 ms */
END IF;
END LOOP;
END;
$$;
Test it using this code:
SELECT CASE
WHEN advisory_xact_lock_request(1, 2.5) = 0
THEN pg_sleep(120)
END; -- and repeat this in parallel session
/* Usage in Pl/PgSQL */
lkstat := advisory_xact_lock_request(lkhndl, lktimeout);

Related

Toad oracle 10.5 [duplicate]

I have this anonymous PL/SQL block which calculates and prints a value return from a table.
DECLARE
U_ID NUMBER :=39;
RETAIL BINARY_FLOAT:=1;
FLAG NUMBER;
BEGIN
SELECT NVL(RETAIL_AMOUNT,1),UNIT_ID INTO RETAIL, FLAG FROM UNITS WHERE UNIT_ID=U_ID;
LOOP
SELECT NVL(MAX(UNIT_ID),U_ID) INTO FLAG FROM UNITS WHERE FATHER_ID=FLAG;
IF FLAG=U_ID THEN EXIT; END IF;
SELECT RETAIL* RETAIL_AMOUNT INTO RETAIL FROM UNITS WHERE UNIT_ID=FLAG;
EXIT WHEN FLAG=U_ID;
END LOOP;
DBMS_OUTPUT.PUT_LINE( RETAIL);
END;
This block work correctly, but I wanted to do the same thing using a PL/SQL Function
I wrote the function as follow:
CREATE OR REPLACE FUNCTION GET_UNIT_RETAIL(U_ID NUMBER)
RETURN NUMBER
IS
RETAIL BINARY_FLOAT:=1;
FLAG NUMBER;
BEGIN
SELECT NVL(RETAIL_AMOUNT,1),UNIT_ID
INTO RETAIL, FLAG
FROM UNITS
WHERE UNIT_ID=U_ID;
LOOP
SELECT NVL(MAX(UNIT_ID),U_ID)
INTO FLAG
FROM UNITS
WHERE FATHER_ID=FLAG;
IF FLAG=U_ID THEN
EXIT;
END IF;
SELECT RETAIL* RETAIL_AMOUNT
INTO RETAIL
FROM UNITS
WHERE UNIT_ID=FLAG;
EXIT WHEN FLAG=U_ID;
END LOOP;
RETURN NUMBER;
END;
/
When I try to execute the above code to save the function to the database, the environment (SQL*PLUS) hangs for a long time and at the end returns this error:
ERROR at line 1:
ORA-04021: timeout occurred while waiting to lock object
What is the problem ??? Please !
Sounds like ddl_lock problem
Take a look at
dba_ddl_locks to see who is "blocking" a create or replace.
Also try to create under different name - and see what happens.
The problem was because the Object GET_UNIT_RETAIL was busy by other environment
Here is the answer:
https://community.oracle.com/thread/2321256

PostgreSQL how to disable plan caching to fix my base62 hashing?

From the documentation i gather, that PostgreSQL somehow employs its own cache and prepares satements ahead...
This is really bad for my base62 hash values. At some point, after 2-3 tries, they start returning the same number:
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
CONTEXT: PL/pgSQL function copy_article(text) line 23 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: copied_article_id QQZCFzm | article_count 1
CONTEXT: PL/pgSQL function copy_article(text) line 37 at RAISE
STATEMENT: select to_json("public"."copy_article"($1)) as value
LOG: base62_id.val 1501145675089
Here is my function:
CREATE OR REPLACE FUNCTION base62_id() RETURNS character varying
LANGUAGE plpgsql IMMUTABLE
AS $$
DECLARE
chars char[];
ret varchar;
val bigint;
BEGIN
chars := ARRAY['0','1','2','3','4','5','6','7','8','9'
,'A','B','C','D','E','F','G','H','I','J','K','L','M'
,'N','O','P','Q','R','S','T','U','V','W','X','Y','Z'
,'a','b','c','d','e','f','g','h','i','j','k','l','m'
,'n','o','p','q','r','s','t','u','v','w','x','y','z'];
val := (CEIL(EXTRACT(EPOCH FROM now()) * 1000))::bigint;
RAISE LOG 'base62_id.val %', val;
ret := '';
IF val < 0 THEN
val := val * -1;
END IF;
WHILE val != 0 LOOP
ret := chars[(val % 62)+1] || ret;
val := val / 62;
END LOOP;
RETURN ret;
END;$$;
In theory, this should work...
Any ideas?
Edit: How i use the function:
DECLARE
copied_article_id text := base62_id();
duplication_check int := 1;
copied_article articles;
BEGIN
WHILE duplication_check IS NOT NULL LOOP
SELECT COUNT(*) INTO duplication_check FROM articles WHERE id = copied_article_id;
END LOOP;
INSERT ... INTO ... SELECT ...
FROM
articles
WHERE
id = base_id;
SELECT * INTO copied_article FROM articles WHERE id = copied_article_id LIMIT 1;
Your explanation is pretty bogus. Plan caching has nothing to do with this outcome, the plancache doesn't care about individual function outputs.
There's at least two giant gaping bugs in the function:
You declare it IMMUTABLE but you call now() which is not immutable. IMMUTABLE functions must return the same result for every call with the same inputs. In fact, your function must be declared VOLATILE if it's allowed to return a different value for each call with the same inputs.
now() is STABLE. It actually returns the same value for each call within a transaction. So presumably using it when you want unique values makes no sense at all. I imagine you actually want clock_timestamp().
The latter problem with now() (a.k.a. current_timestamp) being STABLE across a transaction is likely the cause for the results you report.
BTW, the function will also probably be amazingly slow implemented in plpgsql. If you can turn it into a set operation plus string_agg it might be more tolerable, but still slow.

Trying to Understand Oracle Deterministic Function Speed Test Results

This question arose as a spin-off from this one: Adding Many (UDFs) Validation Functions to Oracle - Which Method Run Fastest
I am debating between putting a low level function that will be used everywhere in my application in an object as a CONSTRUCTOR FUNCTION so it's encapsulated with it's type that it returns, or just making the function STAND-ALONE. I know most would thing these test result are inconsequential because the loop count is so high. But in our app this function could be used to check multiple columns in each large loop iteration. So it really wouldn't be hard for us to reach 3 million checks in a single process. And this all reports back to a web page so the user is waiting on these results, so every msec matters.
So I was running some results and this is what I found...
/*****
isValid is a CONSTRUCTOR FUNCTION of an OBJECT
and it is deterministic
passing in:
'blah' -> 12 seconds
x -> 464 msecs
changing x to varchar2(7)
x -> 2 seconds (this is how we would use it in most cases within loops)
*****/
declare
x number;
--x varchar2(7);
begin
for i in 1 .. 3000000 loop
x := x + 1;
if (isValid(x,'number').result = 1) then
null;
end if;
end loop;
end;
/
/*****
isValid2 is a STAND-ALONE FUNCTION
and it is deterministic
passing in:
'blah' -> 407 msecs
x -> 4 seconds
changing x to varchar2(7)
x -> 4 seconds (this is how we would use it in most cases within loops)
*****/
declare
x number;
--x varchar2(7);
begin
for i in 1 .. 3000000 loop
x := x + 1;
if (isValid2(x,'number').result = 1) then
null;
end if;
end loop;
end;
/
Based on these result I think I'm going to go with the CONSTRUCTOR FUNCTION of an OBJECT approach.
So my question is: Why are the result between 'blah' and x completely reversed between the two different methods?
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
CORE 11.2.0.2.0 Production
TNS for 64-bit Windows: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
PLSQL_OPTIMIZE_LEVEL = 2 (I've never read up on this, this is a test box, I'm not sure what it *should* be set to yet but I will read up on it)
PLSQL_CODE_TYPE = INTERPRETED (I know NATIVE is faster but they won't change it.)
Here is the stand-alone function...
create or replace type valObj as object (
result number(1),
resulttext varchar2(32000) );
/
create or replace function isValid2(v in varchar2, f in varchar2)
return valObj
deterministic
is
test PLS_INTEGER;
begin
if f = 'number' then
begin
test := to_number(v);
return valObj(1,null);
exception when VALUE_ERROR then return valObj(0,'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...');
end;
elsif f = 'phone' then
null; --TO DO
elsif f = 'integer' then
null; --TO DO
elsif f = 'email' then
null; --TO DO
elsif f = 'IPv4' then
null; --TO DO
elsif f = 'IPv6' then
null; --TO DO
end if;
--dozens of others to follow....
end;
/
And here is the object/function...
create or replace type isValid as object (
result number(1),
resulttext varchar2(32000),
constructor function isValid(v varchar, f varchar) return self as result deterministic );
/
create or replace type body isValid as
constructor function isValid(v varchar, f varchar) return self as result deterministic is
test PLS_INTEGER;
begin
if f = 'number' then
begin
test := to_number(v);
self.result := 1;
self.resulttext := null;
return;
exception when VALUE_ERROR then
self.result := 0;
self.resulttext := 'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...';
return;
end;
elsif f = 'phone' then
null; --TO DO
elsif f = 'integer' then
null; --TO DO
elsif f = 'email' then
null; --TO DO
elsif f = 'IPv4' then
null; --TO DO
elsif f = 'IPv6' then
null; --TO DO
end if;
--dozens of others to follow....
end;
end;
/

Adding Many (UDFs) Validation Functions to Oracle - Which Method Run Fastest

I have to move around 50+ validation functions into Oracle. I'm looking for the approach that runs fastest, but also would like to get around a boolean issue if possible. The return object for them all needs to be the same so that the application can react off the result in a consistent fashion and alert the user or display whatever popups, messages we may need. I created a valObj for this, but not sure yet if that is the best approach. The return format can be changed because the front-end that reacts off of it is not developed yet. In the end it will contain many different validation functions, from integer, number, phone, email, IPv4, IPv6, etc... This is what I have so far...
/***
This is the validation object.
It stores 1 for valid, 0 for not valid and some helper text that can be relayed back to the user.
***/
create or replace type valObj as object (
result number(1),
resultText varchar(32000)
);
/***
Coming from ColdFusion this seems clean to me but the function
will end up being a couple thousand lines long.
***/
create or replace function isValid(v in varchar2, format in varchar2)
return valObj
is
test number;
begin
if format = 'number' then
begin
test := to_number(v);
return valObj(1,null);
exception when VALUE_ERROR then return valObj(0,'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...');
end;
elsif format = 'integer' then
null; --TO DO
elsif format = 'email' then
null; --TO DO
elsif format = 'IPv4' then
null; --TO DO
elsif format = 'IPv6' then
null; --TO DO
end if;
--dozens of others to follow....
end;
/
/* Example Usage in SQL */
select isValid('blah','number') from dual; -- returns: (0, Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...)
select isValid('blah','number').result from dual; -- returns: 0
select isValid('blah','number').resulttext from dual; -- returns: Valid formats are: 12345, 12345.67, -12345, etc...
select isValid(1234567890.123,'number') from dual; -- returns: 1,{null}
select isValid(1234567890.123,'number').result from dual; -- returns: 1
select isValid(1234567890.123,'number').resulttext from dual; -- returns: {null}
/* Example Usage in PL/SQL */
declare
temp valObj;
begin
temp := isValid('blah','number');
if (temp.result = 0) then
dbms_output.put_line(temp.resulttext);
else
dbms_output.put_line('Valid');
end if;
end;
/
My questions are:
When using it in PL/SQL I would love to be able to do boolean checks instead like this: if (temp.result) then but I can't figure out a way, cause that won't work in SQL. Should I just add a 3rd boolean attribute to the valObj or is there another way I don't know of?
These validation functions could end up being called within large loops. Knowing that, is this the most efficient way to accomplish these validations?
I'd appreciate any help. Thanks!
UPDATE: I forgot about MEMBER FUNCTIONS. Thanks #Brian McGinity for reminding me. So I'd like to go with this method since it keeps the type and its functions encapsulated together. Would there be any speed difference between this method and a stand-alone function? Would this be compiled and stored the same as a stand-alone function?
create or replace type isValid as object (
result number(1),
resulttext varchar2(32000),
constructor function isValid(v varchar, format varchar) return self as result );
/
create or replace type body isValid as
constructor function isValid(v varchar, format varchar) return self as result as
test number;
begin
if format = 'number' then
begin
test := to_number(v);
self.result := 1;
self.resulttext := null;
return;
exception when VALUE_ERROR then
self.result := 0;
self.resulttext := 'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...';
return;
end;
elsif format = 'phone' then
null; --TO DO
end if;
--and many others...
end;
end;
/
/* Example Usage in SQL */
select isValid('a','number') from dual;
/* Example Usage in PL/SQL */
declare
begin
if (isValid('a','number').result = 1) then
null;
end if;
end;
/
TEST RESULTS:
/* Test isValid (the object member function), this took 7 seconds to run */
declare
begin
for i in 1 .. 2000000 loop
if (isValid('blah','number').result = 1) then
null;
end if;
end loop;
end;
/* Test isValid2 (the stand-alone function), this took 16 seconds to run */
declare
begin
for i in 1 .. 2000000 loop
if (isValid2('blah','number').result = 1) then
null;
end if;
end loop;
end;
Both isValid and isValid2 do the same exact code, they just run this line test := to_number(v); then do the exception if it fails and return the result. Does this appear to be a valid test? The Object member function method is actually faster than a stand-alone function???
The stand-alone function can be much faster if you set it to DETERMINISTIC and if the data is highly repetitive. On my machine this setting decreased run time from 9 seconds to 0.1 seconds. For reasons I don't understand that setting does not improve performance of the object function.
create or replace function isValid2(v in varchar2, format in varchar2)
return valObj
deterministic --<< Hit the turbo button!
is
test number;
begin
if format = 'number' then
begin
test := to_number(v);
return valObj(1,null);
exception when VALUE_ERROR then return valObj(0,'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...');
end;
end if;
end;
/
May also want to consider utilizing pls_integer over number. Don't know if it will buy you much, but documents suggest some gain will be had.
http://docs.oracle.com/cd/B10500_01/appdev.920/a96624/03_types.htm states,
"You use the PLS_INTEGER datatype to store signed integers. Its magnitude range is -2*31 .. 2*31. PLS_INTEGER values require less storage than NUMBER values. Also, PLS_INTEGER operations use machine arithmetic, so they are faster than NUMBER and BINARY_INTEGER operations, which use library arithmetic. For efficiency, use PLS_INTEGER for all calculations that fall within its magnitude range."

Monitoring long-running PL/SQL block

I have a fairly time intensive PL/SQL block that builds fingerprints from molecular structures. I would like to print output to SQL*Plus console to provide feedback on how many structures have been processed. I can do this with dbms_output.put_line
However everytime that is called a new line is written. I want to overwrite the line.
For example, currently I have the below.
Structure x of y processed
Structure x of y processed
Structure x of y processed
Structure x of y processed
Eventually I fill up the buffer as I'm dealing with thousands of structure records.
Is there a method I can use that will just overwrite the last output line?
Using DBMS_OUTPUT means that SQL*Plus will display nothing until the entire PL/SQL block is complete and will then display all the data currently in the buffer. It is not, therefore, an appropriate way to provide an ongoing status.
On the other hand, Oracle does provide a package DBMS_APPLICATION_INFO that is specifically designed to help you monitor your running code. For example, you could do something like
CREATE PROCEDURE process_structures
AS
<<other variable declarations>>
rindex BINARY_INTEGER;
slno BINARY_INTEGER;
totalwork NUMBER := y; -- Total number of structures
worksofar NUMBER := 0; -- Number of structures processed
BEGIN
rindex := dbms_application_info.set_session_longops_nohint;
FOR i IN (<<select structures to process>>)
LOOP
worksofar := worksofar + 1;
dbms_application_info.set_session_longops(
rindex => rindex,
slno => slno,
op_name => 'Processing of Molecular Structures',
sofar => worksofar ,
totalwork => totalwork,
target_desc => 'Some description',
units => 'structures');
<<process your structure with your existing code>>
END LOOP;
END;
From a separate SQL*Plus session, you can then monitory progress by querying the V$SESSION_LONGOPS view
SELECT opname,
target_desc,
sofar,
totalwork,
units,
elapsed_seconds,
time_remaining
FROM v$session_longops
WHERE opname = 'Processing of Molecular Structures';
You may also send messages to a named pipe and have another process read the message from the pipe.
procedure sendmessage(p_pipename varchar2
,p_message varchar2) is
s number(15);
begin
begin
sys.dbms_pipe.pack_message(p_message);
exception
when others then
sys.dbms_pipe.reset_buffer;
end;
s := sys.dbms_pipe.send_message(p_pipename, 0);
if s = 1
then
sys.dbms_pipe.purge(p_pipename);
end if;
end;
function receivemessage(p_pipename varchar2
,p_timeout integer) return varchar2 is
n number(15);
chr varchar2(200);
begin
n := sys.dbms_pipe.receive_message(p_pipename, p_timeout);
if n = 1
then
return null;
end if;
sys.dbms_pipe.unpack_message(chr);
return(chr);
end;
I don't think you can. As far as I understood the dbms_output it just doesn't work that way.
I recommend you use put to echo a single dot and a newline every 1000 or so entries to see that something is happening and write into a table or sequence the current position so you can have a look if you want to know.