I found a previous post (Cursors in BigQuery) where there exists a way to declare cursors on bigquery. This works well and good when the cursor subquery is not present as a parameter.
Currently I was going through FOR..IN EXECUTE construct in Netezza. It behaves extactly like a cursor construct except here, the sql present is a dynamic sql. This dynamic sql is first executed, after that the construct boils down to a simple cursor statement.
Consider the following use case, where the sub-query is present as a parameter.
CREATE or replace PROCEDURE myproc(varchar(256))
RETURNS INT4
LANGUAGE NZPLSQL
AS
BEGIN_PROC
declare
sqlstr alias for $1; ---- sqlStr is a parameter
r1 record;
begin
FOR r1 IN EXECUTE sqlstr ---- sqlStr is evaluated after that it boils down to cursor statement.
loop
insert into t1 values r1.c1;
end loop;
end;
END_PROC#
Is there a similiar way to declare cursors with subquery as a parameter on BigQuery ?
Formally, both LOOPs and EXECUTE IMMEDIATE are available in BigQuery and well documented.
Practically - using LOOP to mimic cursor for real table is extremely ineffective and quite a no-no in BigQuery unless you have use-case where this is the only way to go
But it is almost in 100% cases you can express your logic to be done in batch way (as opposed to using cursor)
At the same time if Table is not that big (like lookup tab le for example) - you can select table rows into array and than loop through arrays elements running execute immediate getting the result and inserting it into target table
Related
I am facing a problem in PL/pgSQL while trying to convert some procedures from Oracle to Postgres RDBMS. At the original procedure in PL/SQL, one of these procedures has declared within the DECLARE clause: a bounded cursor, a TYPE which is a TABLE OF ROWTYPE of that bounded cursor and a variable which takes the datatype of this predefined TYPE.
In a few words (as far as I've understood), a bounded cursor and an ARRAY of its type - meaning that each position of that array is actually a ROWTYPE of that cursor - are declared. I have tried to research ways to get around this in Postgres but no success, yet.
This isn't the original I am trying to migrate from Oracle to Postgres, but I think it clarifies what I am trying to reach:
CREATE OR REPLACE PROCEDURE schema.procedure_name (arguments)
AS
$BODY$
DECLARE
CUR_FILMS2 CURSOR (year INTEGER) for
SELECT *
FROM film
WHERE release_year = year;
TYPE TYPE_EXAMPLE IS TABLE OF CUR_FILMS2%ROWTYPE;
V_TYPE_EXAMPLE TYPE_EXAMPLE;
...
BEGIN
OPEN CUR_FILMS2(2022);
FETCH CUR_FILMS2 BULK COLLECT INTO V_TYPE_EXAMPLE;
CLOSE CUR_FILMS2;
...
-- Later on, that variable is iterated to execute some code beneath.
FOR I IN 1 .. V_TYPE_EXAMPLE.COUNT LOOP
...
END LOOP;
END;
$BODY$
I have already thought about the RECORD datatype using, but it only accepts one record per time, and as far as I know there isn't a sort of TABLE variable in Postgres. The Postgres version I am using is PostgreSQL 14.3.
What is the easiest way to get around this problem? Is there a way?
Postgres does not have table variables - and no BULK COLLECT for cursors in PL/pgSQL. You might work with temporary tables.
Seems like you can replace all the overhead with a plain FOR loop (and its implicit cursor) using a generic record variable for the row type:
CREATE OR REPLACE PROCEDURE schema.procedure_name (_year int) AS
$func$
DECLARE
_rec record;
BEGIN
FOR _rec IN
SELECT *
FROM film
WHERE release_year = _year
LOOP
...
END LOOP;
END
$func$;
See:
Cursor based records in PostgreSQL
Loop on tables with PL/pgSQL in Postgres 9.0+
We run user submitted queries which can potentially return a large result set.
In order to avoid memory issues, we would like to detect these cases and cancel the query. The user then is expected to modify the query.
We already use PreparedStatement#setFetchSize() to scroll the result set and process a large result set incrementally.
However, when the result set is too large, we would like to avoid bringing even the first results over the network or any other unnecessary work as much as possible on the client side and on the database side.
Doing a SELECT COUNT(*)... beforehand just degrades the performance of the expected case where the queries behave nicely in general.
Is there a way for postgres to tell the expected result set size?
Take a look here.
They are doing an estimate with a database procedure:
CREATE FUNCTION count_estimate(query text) RETURNS INTEGER AS
$func$
DECLARE
rec record;
ROWS INTEGER;
BEGIN
FOR rec IN EXECUTE 'EXPLAIN ' || query LOOP
ROWS := SUBSTRING(rec."QUERY PLAN" FROM ' rows=([[:digit:]]+)');
EXIT WHEN ROWS IS NOT NULL;
END LOOP;
RETURN ROWS;
END
$func$ LANGUAGE plpgsql;
It uses the EXPLAIN command of PGSQL to estimate the returned rowcount.
I would like to execute a SQL statement based on a string. I have created the SQL statement through a function which returns a string. Can anybody explain how I can execute the statement that is returned? I know that you can't do it in plain SQL, so I was thinking about putting it in a function. The only issue is that the columns in the statement aren't always the same, so I don't know which data types to use for the columns. I'm using Postgres 9.1.0.
For example, suppose the SQL string returned from my function the is:
Select open, closed, discarded from abc
But, it can also be:
Select open from abc
Or
Select open, closed from abc
How can I execute any of these strings, so that the results would be returned as a table with only the columns listed in the statement?
Edit: the function is written in PL/pgSQL. And the results will be used for reporting where they don't want to see columns that have no values. So the function that I wrote returns the names of all columns that have values and then add it to the SQL statement.
Thanks for your help!
I don't think you can return the rows directly from a function, because its return type would be unknown. Even if you specified the return type as RECORD, you'd have to list the returned columns at call time. Based on Wayne Conrad's idea, you could do this:
CREATE FUNCTION my_create(cmd TEXT) RETURNS VOID AS $$
BEGIN
EXECUTE 'CREATE TEMPORARY TABLE temp_result AS ' || cmd;
END;
$$ VOLATILE LANGUAGE plpgsql;
Then use the function like this:
BEGIN;
SELECT my_create(...);
SELECT * FROM temp_result;
ROLLBACK; -- or COMMIT
I have 2 procedures inside a package. I am calling one procedure to get a comma separated list of user ids.
I am storing the result in a VARCHAR variable. Now when I am using this comma separated list to put inside an IN clause in it is throwing "ORA-01722:INVALID NUMBER" exception.
This is how my variable looks like
l_userIds VARCHAR2(4000) := null;
This is where i am assigning the value
l_userIds := getUserIds(deptId); -- this returns a comma separated list
And my second query is like -
select * from users_Table where user_id in (l_userIds);
If I run this query I get INVALID NUMBER error.
Can someone help here.
Do you really need to return a comma-separated list? It would generally be much better to declare a collection type
CREATE TYPE num_table
AS TABLE OF NUMBER;
Declare a function that returns an instance of this collection
CREATE OR REPLACE FUNCTION get_nums
RETURN num_table
IS
l_nums num_table := num_table();
BEGIN
for i in 1 .. 10
loop
l_nums.extend;
l_nums(i) := i*2;
end loop;
END;
and then use that collection in your query
SELECT *
FROM users_table
WHERE user_id IN (SELECT * FROM TABLE( l_nums ));
It is possible to use dynamic SQL as well (which #Sebas demonstrates). The downside to that, however, is that every call to the procedure will generate a new SQL statement that needs to be parsed again before it is executed. It also puts pressure on the library cache which can cause Oracle to purge lots of other reusable SQL statements which can create lots of other performance problems.
You can search the list using like instead of in:
select *
from users_Table
where ','||l_userIds||',' like '%,'||cast(user_id as varchar2(255))||',%';
This has the virtue of simplicity (no additional functions or dynamic SQL). However, it does preclude the use of indexes on user_id. For a smallish table this shouldn't be a problem.
The problem is that oracle does not interprete the VARCHAR2 string you're passing as a sequence of numbers, it is just a string.
A solution is to make the whole query a string (VARCHAR2) and then execute it so the engine knows he has to translate the content:
DECLARE
TYPE T_UT IS TABLE OF users_Table%ROWTYPE;
aVar T_UT;
BEGIN
EXECUTE IMMEDIATE 'select * from users_Table where user_id in (' || l_userIds || ')' INTO aVar;
...
END;
A more complex but also elegant solution would be to split the string into a table TYPE and use it casted directly into the query. See what Tom thinks about it.
DO NOT USE THIS SOLUTION!
Firstly, I wanted to delete it, but I think, it might be informative for someone to see such a bad solution. Using dynamic SQL like this causes multiple execution plans creation - 1 execution plan per 1 set of data in IN clause, because there is no binding used and for the DB, every query is a different one (SGA gets filled with lots of very similar execution plans, every time the query is run with a different parameter, more memory is needlessly used in SGA).
Wanted to write another answer using Dynamic SQL more properly (with binding variables), but Justin Cave's answer is the best, anyway.
You might also wanna try REF CURSOR (haven't tried that exact code myself, might need some little tweaks):
DECLARE
deptId NUMBER := 2;
l_userIds VARCHAR2(2000) := getUserIds(deptId);
TYPE t_my_ref_cursor IS REF CURSOR;
c_cursor t_my_ref_cursor;
l_row users_Table%ROWTYPE;
l_query VARCHAR2(5000);
BEGIN
l_query := 'SELECT * FROM users_Table WHERE user_id IN ('|| l_userIds ||')';
OPEN c_cursor FOR l_query;
FETCH c_cursor INTO l_row;
WHILE c_cursor%FOUND
LOOP
-- do something with your row
FETCH c_cursor INTO l_row;
END LOOP;
END;
/
I need to run big queries (that was a part of SP) and look at their results (just trying to find a bug in a big SP with many unions. I want to break it into parts and run them separately).
How can I do that if this SP have few parameters? I don't want to replace them in code, it would be great just to add declare in a header with a hardcode for this parameter.
I've tried something like this:
DECLARE
p_asOfDate DATE := '22-Feb-2011';
BEGIN
SELECT * from myTable where dateInTable < p_asOfDate;
END
But it says that I should use INTO keyword. How can I view this results in my IDE? (I'm using Aqua data studio)
I need to do that very often, so will be very happy if will find a simple solution
You are using an anonymous block of pl/sql code.
In pl/sql procedures you need to specify a target variable for the result.
So you first need to define a variable to hold the result in the declare section
and then insert the result data into it.
DECLARE
p_asOfDate DATE := '22-Feb-2011';
p_result myTable%ROWTYPE;
BEGIN
select * into p_result from myTable where dateInTable < p_asOfDate;
END
That said you will probaply get more than one row returned, so I would use
a cursor to get the rows separately.
DECLARE
CURSOR c_cursor (asOfDate IN DATE) is
select * from myTable where dateInTable < asOfDate;
p_asOfDate DATE := '22-Feb-2011';
p_result myTable%ROWTYPE;
BEGIN
OPEN c_cursor(p_asOfDate);
loop
FETCH c_cursor into p_result;
exit when c_cursor%NOTFOUND;
/* do something with the result row here */
end loop;
CLOSE c_cursor;
END
To output the results you can use something like this for example:
dbms_output.put_line('some text' || p_result.someColumn);
Alternatively you can execute the query on an sql command-line (like sqlplus)
and get the result as a table immediately.
I hope I understood your question correctly...
update
Here is a different way to inject your test data:
Use your tools sql execution environemnt to submit your sql statement directly without a pl/sql block.
Use a "&" in front of the variable part to trigger a prompt for the variable.
select * from myTable where dateInTable < &p_asOfDate;
The Result should be displayed in a formatted way by your tool this way.
I do not know about Aqua, but some tools have functions to define those parameters outside the sql code.