PostgreSQL - Start A Transaction block IN Function - sql

I'm trying to use create a transaction block inside a function, so my goal is to use this function one at time, so if some one use this Function and another want to use it, he can't until the first one is finish i create this Function :
CREATE OR REPLACE FUNCTION my_job(time_to_wait integer) RETURNS INTEGER AS $$
DECLARE
max INT;
BEGIN
BEGIN;
SELECT MAX(max_value) INTO max FROM sch_lock.table_concurente;
INSERT INTO sch_lock.table_concurente(max_value, date_insertion) VALUES(max + 1, now());
-- Sleep a wail
PERFORM pg_sleep(time_to_wait);
RETURN max;
COMMIT;
END;
$$
LANGUAGE plpgsql;
But it seams not work, i have a mistake Syntax error BEGIN;
Without BEGIN; and COMMIT i get a correct result, i use this query to check :
-- First user should to wait 10 second
SELECT my_job(10) as max_value;
-- First user should to wait 3 second
SELECT my_job(3) as max_value;
So the result is :
+-----+----------------------------+------------+
| id | date | max_value |
+-----+----------------------------+------------+
| 1 | 2017-02-13 13:03:58.12+00 | 1 |
+-----|----------------------------+------------+
| 2 | 2017-02-13 13:10:00.291+00 | 2 |
+-----+----------------------------+------------+
| 3 | 2017-02-13 13:10:00.291+00 | 2 |
+-----+----------------------------+------------+
But the result should be :
+-----+----------------------------+------------+
| id | date | max_value |
+-----+----------------------------+------------+
| 1 | 2017-02-13 13:03:58.12+00 | 1 |
+-----|----------------------------+------------+
| 2 | 2017-02-13 13:10:00.291+00 | 2 |
+-----+----------------------------+------------+
| 3 | 2017-02-13 13:10:00.291+00 | 3 |
+-----+----------------------------+------------+
so the third one id = 3 should have the max_value = 3 and not 2, this happen because the first user Select the max = 1 and wait 10 sec and the second user Select the max = 1 and wait 3 sec before Insertion, but the right solution is : I can't use this Function Until the First one finish, for that i want to make something secure and protected.
My questions is :
how can i make a Transaction block inside a function?
Do you have any suggestion how can we make this, with a secure way?
Thank you.

Ok so you cannot COMMIT in a function. You can have a save point and roll back to the save point however.
Your smallest possible transaction is a single statement parsed and executed by the server from the client, so every transaction is a function. Within a transaction, however, you can have save points. In this case you would look at the exception handling portions of PostgreSQL to handle this.
However that is not what you want here. You want (I think?) data to be visible during a long-running server-side operation. For that you are kind of out of luck. You cannot really increment your transaction ids while running a function.
You have a few options, in order of what I would consider to be good practices (best to worst):
Break down your logic into smaller slices that each move the db from one consistent state to another, and run those in separate transactions.
Use a message queue (like pg_message_queue)in the db, plus an external worker, and something which runs a step and yields a message for the next step. Disadvantage is this adds more maintenance.
Use a function or framework like dblink or pl/python, or pl/perlu to connect back to the db and run transactions there. ick....

You can use dblink for this. Something like :
CREATE OR REPLACE FUNCTION my_job(time_to_wait integer) RETURNS INTEGER AS $$
DECLARE
max INT;
BEGIN
SELECT INTO RES dblink_connect('con','dbname=local');
SELECT INTO RES dblink_exec('con', 'BEGIN');
...
SELECT INTO RES dblink_exec('con', 'COMMIT');
SELECT INTO RES dblink_disconnect('con');
END;
$$
LANGUAGE plpgsql;

I don't know if this is a good way or not but what if we use LOCK TABLE for example like this :
CREATE OR REPLACE FUNCTION my_job(time_to_wait integer) RETURNS INTEGER AS $$
DECLARE
max INT;
BEGIN
-- Lock table so no one will use it until the first one is finish
LOCK TABLE sch_lock.table_concurente IN ACCESS EXCLUSIVE MODE;
SELECT MAX(max_value) INTO max FROM sch_lock.table_concurente;
INSERT INTO sch_lock.table_concurente(max_value, date_insertion) VALUES(max + 1, now());
PERFORM pg_sleep(time_to_wait);
RETURN max;
END;
$$
LANGUAGE plpgsql;
It gives me the right result.

Related

What is maximum rows count in oracles nested table

CREATE TYPE nums_list AS TABLE OF NUMBER;
What is maximum possible rows count in oracle's nested table ?
UPDATE
CREATE TYPE nums_list AS TABLE OF NUMBER;
CREATE OR REPLACE FUNCTION generate_series(from_n NUMBER, to_n NUMBER)
RETURN nums_list AS
ret_table nums_list := nums_list();
BEGIN
FOR i IN from_n..to_n LOOP
ret_table.EXTEND;
ret_table(i) := i;
END LOOP;
RETURN ret_table;
END;
SELECT count(*) FROM TABLE ( generate_series(1,4555555) );
This gives error: ORA-22813 operand value exceeds system limits, Object or Collection value was too large
The range of subscripts for a nested table is 1..2**31 so you can have 2**31 elements in the collection. That limit hasn't changed since at least 8.1.6 though, of course, it might change in the future.
Just as an additional observation, it isn't the nested table itself that is too large or using too much memory. With an exception handler you can see that the error is not being thrown by your function. You can populate the same thing in an anonymous block:
DECLARE
ret_table nums_list := nums_list();
BEGIN
FOR i IN 1..4555555 LOOP
ret_table.EXTEND;
ret_table(i) := i;
END LOOP;
dbms_output.put_line(ret_table.count);
END;
/
anonymous block completed
4555555
And you can call your function from a block too:
DECLARE
ret_table nums_list;
BEGIN
ret_table := generate_series(1,4555555);
dbms_output.put_line(ret_table.count);
END;
/
anonymous block completed
4555555
It's only when you use it as table collection expression that you get an error:
SQL Error: ORA-22813: operand value exceeds system limits
22813. 00000 - "operand value exceeds system limits"
*Cause: Object or Collection value was too large. The size of the value
might have exceeded 30k in a SORT context, or the size might be
too big for available memory.
*Action: Choose another value and retry the operation.
The cause text refers to the SORT context, and a sort is being done by your query:
------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 2 | 29 (0)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 2 | | |
| 2 | COLLECTION ITERATOR PICKLER FETCH| GENERATE_SERIES | 8168 | 16336 | 29 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------------
As #a_horse_with_no_name suggested, you can avoid the problem by making your function pipelined:
CREATE OR REPLACE FUNCTION generate_series(from_n NUMBER, to_n NUMBER)
RETURN nums_list PIPELINED AS
BEGIN
FOR i IN from_n..to_n LOOP
PIPE ROW (i);
END LOOP;
RETURN;
END;
/
SELECT count(*) FROM TABLE ( generate_series(1,4555555) );
COUNT(*)
----------
4555555
That still does a SORT AGGREGATE but it doesn't seem to mind any more. Not really sure why it does that in either case; perhaps someone else will be able to explain what it's doing. (I'm doing this in an 11gR2 instance by the way; I don't have a 12c instance to verify the behaviour is the same, but your symptoms suggest it will be). Or maybe it isn't the SORT context that's the issue, perhaps it is available memory. In my environment your version seems to consistently work up to 4177918 elements - which doesn't seem to be a significant number, so is likely to be environment related?
But it depends how you intend to use the collection; from a PL/SQL context your original version might be more suitable.

sql server 2008 r2 cursor is not forwarded by fetch

I stumbled upon a very strange behaviour while working on some T-SQL Code.
I am working on a SQL Server 2008 R2 SP2 (build nr.: 10.50.4000).
My question to you guys is if anybody has seen such a behaviour before or if anybody might be able to explain it to me.
So,
What's the situation?
We have a table, which looks like that:
product_number | id_object | position_in_product
---------------------------------------------------
1 | 101 | 1
1 | 102 | 1
1 | 103 | 1
2 | 201 | 1
2 | 202 | 1
2 | 203 | 1
Multiple object ids are allocated to one product number. The order should be defined by the position_in_product column. The funny part lies exactly in establishing that order.
Of course, after doing that the table should look like this:
product_number | id_object | position_in_product
---------------------------------------------------
1 | 101 | 1
1 | 102 | 2
1 | 103 | 3
2 | 201 | 1
2 | 202 | 2
2 | 203 | 3
What's going on?
To update the order column we create a cursor with the following statement:
DECLARE
table_runner CURSOR LOCAL FORWARD_ONLY FOR
SELECT id_object, product_number
FROM table
WHERE ident = #ident
ORDER BY product_number
By using this cursor and counting the rows with the same product_number we should be able to update the position_in_product column. (This has worked in every installation until now)
To move the cursor to the next row we use this:
FETCH next from table_runner
INTO #table_runner$id_object, #table_runner$product_number
The whole function looks like this:
OPEN table_runner
FETCH next from table_runner
INTO #table_runner$id_object, #table_runner$product_number
while ##FETCH_STATUS = 0
BEGIN
/* update_logic */
FETCH next from table_runner
INTO #table_runner$id_object, #table_runner$product_number
END
CLOSE table_runner
And that is the part, that does not work as expected.
The fetch will not give me the next row. I am getting always the same result row.
The while loop does never end, the fetch_status is always 0, but the result stays the same.
The Workaround
After searching the web for quite a while without any results i decided to try a more pragmatical way and put another FETCH statement in.
I know that the id_object variable is unique and has to change in every loop cycle,
so i remembered the last fetched id and put this under the loop fetch statement:
if #id_object_memory = #table_runner$id_object
begin
FETCH next from table_runner
INTO #table_runner$id_object, #table_runner$product_number
set #id_object_memory = #table_runner$id_object
end
else
set #id_object_memory = #table_runner$id_object
With that the loop works as expected, the column in question is updated as it should and the cursor will reach the end of the result set.
The big ?
Has anyone any explanation for that?
There are more cursor defined in the same procedure and they all work as expected.
I have absolute no clue how to explain this.
So, thanks for reading ;)
I can't help with the cursor issue, I've never seen this before, but should point out you don't need a cursor at all to do this update. You can simply use:
WITH CTE AS
( SELECT Product_Number,
ID_Object,
Position_in_Product,
RowNumber = ROW_NUMBER() OVER(PARTITION BY Product_Number
ORDER BY id_object)
FROM T
WHERE ident = #ident
)
UPDATE CTE
SET Position_in_Product = RowNumber;
Example on SQL Fiddle
You possibly don't even need to store this column, and can just use ROW_NUMBER in a query where the position_in_product is required.
Cursors are so 2000 ;-)
Seriously though; avoid cursors at all costs. Set-based operations > looping.
Just create a view with the following:
CREATE VIEW your_view
AS
SELECT product_number
, id_object
, Row_Number() OVER (PARTITION BY product_number ORDER BY id_object) As position_in_product
FROM your_table
;
No need to ever perform the update; the row numbers will "automatically" recalculate.

SQL - Is this possible?

I have a table that looks like this:
--------------------------------------------
| Date | House# | Subscription |
--------------------------------------------
| 3/02/10 | x | Monthly |
--------------------------------------------
| 3/03/10 | y | Weekly |
--------------------------------------------
| 3/04/10 | z | Daily |
--------------------------------------------
I need a command that will take a column name and an int and shift the values in those columns up so many levels. So (house, 1) would put z where y is, y where x is, and z would go to 0/Null. Whereas (house, 2) would put z where x is and y and z would go to 0/null.
I understand that SQL does not actually extract ables row by row, so is this possible?
Thanks ahead of time!
You can do this in a stored procedure using cursors.
You should use PL/SQL, here is an example (not for this particular example):
DECLARE
CURSOR cpaises
IS
SELECT CO_PAIS, DESCRIPCION, CONTINENTE
FROM PAISES;
co_pais VARCHAR2(3);
descripcion VARCHAR2(50);
continente VARCHAR2(25);
BEGIN
OPEN cpaises;
LOOP
FETCH cpaises INTO co_pais,descripcion,continente;
EXIT WHEN cpaises%NOTFOUND;
dbms_output.put_line(descripcion);
END LOOP;
CLOSE cpaises;
END;
I think you could use a variable to indicate which column to select and to update, and inside a loop, you can have an array, with the last n values.
You can use PL/SQL routine .Take the column name and number as input and then implement the logic as you want. Cursors as suggested above is one of the options that you have.
I would think adding a column that contains a value to use as a sort order you could then update that column as needed and then ordered by that column. If it is not possible to change that table perhaps you could create a new table to hold the sort column and join the two

Oracle performance using functions in where clause

In a stored procedure (which has a date parameter named 'paramDate' ) I have a query like this one
select id, name
from customer
where period_aded = to_char(paramDate,'mm/yyyy')
will Oracle convert paramDate to string for each row?
I was sure that Oracle wouldn't but I was told that Oracle will.
In fact I thought that if the parameter of the function was constraint (not got a fierld nor a calculated value inside the query) the result should be allways the same, and that's why Oracle should perform this conversion only once.
Then I realized that I've sometimes executed DML sentences in several functions, and perhaps this could cause the resulting value to change, even if it does not change for each row.
This should mean that I should convert such values before I add them to the query.
Anyway, perhaps well 'known functions' (built in) are evaluated once, or even my functions would also be.
Anyway, again...
Will oracle execute that to_char once or will Oracle do it for each row?
Thanks for your answers
I do not think this is generally the case, as it would prevent an index from being used.
At least for built-in functions, Oracle should be able to figure out that it could evaluate it only once. (For user-defined functions, see below).
Here is a case where an index is being used (and the function is not evaluated for every row):
SQL> select id from tbl_table where id > to_char(sysdate, 'YYYY');
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 35 | 140 | 1 (0)| 00:00:01 |
|* 1 | INDEX RANGE SCAN| SYS_C004274 | 35 | 140 | 1 (0)| 00:00:01 |
--------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("ID">TO_NUMBER(TO_CHAR(SYSDATE#!,'YYYY')))
For user-defined functions check out this article. It mentions two ways to ensure
that your function gets called only once:
Since Oracle 10.2, you can define the function as DETERMINISTIC.
On older versions you can re-phrase it to use "scalar subquery caching":
SELECT COUNT(*)
FROM EMPLOYEES
WHERE SALARY = (SELECT getValue(1) FROM DUAL);
Looking at write-ups on the DETERMINISTIC keyword (here is one, here is another), it was introduced to allow the developer to tell Oracle that the function will return the same value for the same input params. So if you want your functions to be called only once, and you can guarantee they will always return the same value for the same input params you can use the keyword DETERMINISTIC.
With regards to built-in functions like to_char, I defer to those who are better versed in the innards of Oracle to give you direction.
The concern about to_char does not ring a bell with me. However, in your pl/sql,
you could have
create or replace procedure ........
some_variable varchar2(128);
begin
some_variable := to_char(paramDate,'mm/yyyy');
-- and your query could read
select id, name from customer where period_aded = some_variable;
.
.
.
end;
/
Kt

cloning hierarchical data

let's assume i have a self referencing hierarchical table build the classical way like this one:
CREATE TABLE test
(name text,id serial primary key,parent_id integer
references test);
insert into test (name,id,parent_id) values
('root1',1,NULL),('root2',2,NULL),('root1sub1',3,1),('root1sub2',4,1),('root
2sub1',5,2),('root2sub2',6,2);
testdb=# select * from test;
name | id | parent_id
-----------+----+-----------
root1 | 1 |
root2 | 2 |
root1sub1 | 3 | 1
root1sub2 | 4 | 1
root2sub1 | 5 | 2
root2sub2 | 6 | 2
What i need now is a function (preferrably in plain sql) that would take the id of a test record and
clone all attached records (including the given one). The cloned records need to have new ids of course. The desired result
would like this for example:
Select * from cloningfunction(2);
name | id | parent_id
-----------+----+-----------
root2 | 7 |
root2sub1 | 8 | 7
root2sub2 | 9 | 7
Any pointers? Im using PostgreSQL 8.3.
Pulling this result in recursively is tricky (although possible). However, it's typically not very efficient and there is a much better way to solve this problem.
Basically, you augment the table with an extra column which traces the tree to the top - I'll call it the "Upchain". It's just a long string that looks something like this:
name | id | parent_id | upchain
root1 | 1 | NULL | 1:
root2 | 2 | NULL | 2:
root1sub1 | 3 | 1 | 1:3:
root1sub2 | 4 | 1 | 1:4:
root2sub1 | 5 | 2 | 2:5:
root2sub2 | 6 | 2 | 2:6:
root1sub1sub1 | 7 | 3 | 1:3:7:
It's very easy to keep this field updated by using a trigger on the table. (Apologies for terminology but I have always done this with SQL Server). Every time you add or delete a record, or update the parent_id field, you just need to update the upchain field on that part of the tree. That's a trivial job because you just take the upchain of the parent record and append the id of the current record. All child records are easily identified using LIKE to check for records with the starting string in their upchain.
What you're doing effectively is trading a bit of extra write activity for a big saving when you come to read the data.
When you want to select a complete branch in the tree it's trivial. Suppose you want the branch under node 1. Node 1 has an upchain '1:' so you know that any node in the branch of the tree under that node must have an upchain starting '1:...'. So you just do this:
SELECT *
FROM table
WHERE upchain LIKE '1:%'
This is extremely fast (index the upchain field of course). As a bonus it also makes a lot of activities extremely simple, such as finding partial trees, level within the tree, etc.
I've used this in applications that track large employee reporting hierarchies but you can use it for pretty much any tree structure (parts breakdown, etc.)
Notes (for anyone who's interested):
I haven't given a step-by-step of the SQL code but once you get the principle, it's pretty simple to implement. I'm not a great programmer so I'm speaking from experience.
If you already have data in the table you need to do a one time update to get the upchains synchronised initially. Again, this isn't difficult as the code is very similar to the UPDATE code in the triggers.
This technique is also a good way to identify circular references which can otherwise be tricky to spot.
The Joe Celko's method which is similar to the njreed's answer but is more generic can be found here:
Nested-Set Model of Trees (at the middle of the article)
Nested-Set Model of Trees, part 2
Trees in SQL -- Part III
#Maximilian: You are right, we forgot your actual requirement. How about a recursive stored procedure? I am not sure if this is possible in PostgreSQL, but here is a working SQL Server version:
CREATE PROCEDURE CloneNode
#to_clone_id int, #parent_id int
AS
SET NOCOUNT ON
DECLARE #new_node_id int, #child_id int
INSERT INTO test (name, parent_id)
SELECT name, #parent_id FROM test WHERE id = #to_clone_id
SET #new_node_id = ##IDENTITY
DECLARE #children_cursor CURSOR
SET #children_cursor = CURSOR FOR
SELECT id FROM test WHERE parent_id = #to_clone_id
OPEN #children_cursor
FETCH NEXT FROM #children_cursor INTO #child_id
WHILE ##FETCH_STATUS = 0
BEGIN
EXECUTE CloneNode #child_id, #new_node_id
FETCH NEXT FROM #children_cursor INTO #child_id
END
CLOSE #children_cursor
DEALLOCATE #children_cursor
Your example is accomplished by EXECUTE CloneNode 2, null (the second parameter is the new parent node).
This sounds like an exercise from "SQL For Smarties" by Joe Celko...
I don't have my copy handy, but I think it's a book that'll help you quite a bit if this is the kind of problems you need to solve.