I have 6 TEST environments and 1 production environment. I create quite a few different reports as Oracle views, and need a way to sync these between environments.
I am trying to make a script that I can run, which will basically output a list of commands that I can copy and paste into my different environment to create the necessary views/public synonyms and privileges.
I have to put the resultant text into a database table as dbms_output.put_line has a certain limitation on how many characters it can show.
I have the following, but if I try to insert the data, I get ORA-06502: PL/SQL: numeric or value error. I am guessing this is probably got to do with character literals not being escaped and what not.
CREATE OR REPLACE PROCEDURE EXPORT_REPORTS AS
statements CLOB;
tmp_statement CLOB;
CURSOR all_views IS
SELECT
OWNER,
VIEW_NAME,
TEXT
FROM
ALL_VIEWS
WHERE
OWNER = 'PAS'
;
BEGIN
FOR v IN all_views LOOP
tmp_statement := 'CREATE OR REPLACE FORCE VIEW "' || v.OWNER || '"."' || v.VIEW_NAME || '" AS ' || CHR(13) || CHR(10) || v.TEXT;
statements := statements || tmp_statement;
END LOOP;
EXECUTE IMMEDIATE 'INSERT INTO VIEW_EXPORTS VALUES ('''|| statements || ''')';
END EXPORT_REPORTS;
Any idea what I can do to try and fix this?
If it is because some of the text in the statements variable contains single quotes, how can I escape this before inserting the data into a table?
This sounds like a job for Data Pump.
Oracle Data Pump technology enables very high-speed movement of data and metadata from one database to another.
http://docs.oracle.com/cd/B28359_01/server.111/b28319/dp_overview.htm
For getting database object DDL I would recommend using the DBMS_METADATA package.
http://docs.oracle.com/cd/B28359_01/appdev.111/b28419/d_metada.htm#BGBDJAHH
you cannot use CLOB as a datatype for the local variables. Use VARCHAR instead
Related
I am working with vendor application that uses Oracle database. To save the content it uses database tables, which are queried by using views. I do not have any control over that code.
Because of security I gave access to those views to special reporting user, which can only select entries from it.
Whenever some major change is made in the application it drops the appropriate view and creates it anew. Of course all grants are lost and since the changes are made rarely it is easy to forget to backup up and restore them afterwards.
I consulted DBA and he suggested to write a trigger to save grants in temporary table, after which the entries can be used to restore grants. Saving part works fine as expected:
create or replace TRIGGER RECORD_GRANTS_ONDROP
BEFORE DROP ON MYUSER.SCHEMA
BEGIN
IF ora_dict_obj_owner = 'MYUSER' and ora_dict_obj_name not like 'TEMP_PRIV%' and ora_dict_obj_type='VIEW' then
EXECUTE IMMEDIATE 'CREATE TABLE TEMP_PRIV AS SELECT ''GRANT '' || PRIVILEGE || '' ON MYUSER.'' || TABLE_NAME || '' TO '' || GRANTEE PRIVILEGE_x FROM USER_TAB_PRIVS WHERE GRANTEE not in (''MYUSER'',''PUBLIC'') AND TABLE_NAME=''' || ora_dict_obj_name || '''';
ELSE null;
END IF;
END;
As a result I get a table with all grants assigned to the said view.
For restoring I wanted to run similar trigger:
create or replace TRIGGER RESTORE_GRANTS_AFTERCREATE
AFTER CREATE ON MYUSER.SCHEMA
BEGIN
IF ora_dict_obj_owner = 'MYUSER' and ora_dict_obj_type='VIEW' then
FOR loop_counter IN (select '''' || privilege_x || '''' AS privilege_x from temp_priv)
LOOP
EXECUTE IMMEDIATE loop_counter.privilege_x;
DBMS_OUTPUT.PUT_LINE(loop_counter.privilege_x);
END LOOP;
ELSE null;
END IF;
NULL;
END;
I will note here that is just basic test of concept without any proper checks, so just focus on the big issue here.
When I try to create a view now I get an error:
Error report -
ORA-00604: error occurred at recursive SQL level 1
ORA-00900: invalid SQL statement
ORA-06512: at line 5
00604. 00000 - "error occurred at recursive SQL level %s"
*Cause: An error occurred while processing a recursive SQL statement
(a statement applying to internal dictionary tables).
*Action: If the situation described in the next error on the stack
can be corrected, do so; otherwise contact Oracle Support.
This can only mean that the view is not created at the time when trigger tries to add grants. Syntax wise I successfully ran the command:
BEGIN
EXECUTE IMMEDIATE 'GRANT SELECT ON MYUSER.CR_STATUS_TABLE TO SOMEUSER';
END
But when I try to run it using for, or just by itself in 'after create' trigger I get the same error.
Does anyone knows how to approach this, and I would like to avoid jobs at all costs if possible.
Ignoring whether this is a good idea... you're getting that error because you're overthinking your string manipulation. What you're putting into your table looks OK. The problem is when you get it back out. The value in the table is already a string, so you don't need to then enclose it in another set of quotes.
What you're actually running is the equivalent of:
EXECUTE IMMEDIATE '''GRANT SELECT ON MYUSER.CR_STATUS_TABLE TO SOMEUSER''';
which will also throw "ORA-00900: invalid SQL statement", rather than your standalone, working, version:
EXECUTE IMMEDIATE 'GRANT SELECT ON MYUSER.CR_STATUS_TABLE TO SOMEUSER';
If you swapped the order of your EXECUTE IMMEDIATE and DBMS_OUTPUT calls you'd see the problem statement before it ran, which would be more helpful - you'd see those quotes as part of the string.
So in your second trigger, instead of doing:
FOR loop_counter IN (select '''' || privilege_x || '''' AS privilege_x from temp_priv)
LOOP
EXECUTE IMMEDIATE loop_counter.privilege_x;
DBMS_OUTPUT.PUT_LINE(loop_counter.privilege_x);
END LOOP;
just do:
FOR loop_counter IN (select privilege_x from temp_priv)
LOOP
DBMS_OUTPUT.PUT_LINE(loop_counter.privilege_x);
EXECUTE IMMEDIATE loop_counter.privilege_x;
END LOOP;
However, this still won't work; it will now get ORA-30511: invalid DDL operation in system triggers. This is presumably because of the restrictions shown in the documentation:
Trigger cannot do DDL operations on object that caused event to be generated.
DDL on other objects is limited to compiling an object, creating a trigger, and creating, altering, and dropping a table.
You said "it is easy to forget to backup up and restore them afterwards" but you're going to have to put a robust process around your upgrades to make sure that does happen.
You can change your process to have a separate step at the end of every upgrade that is always run, which executes all of those stored statements for all objects - maybe skipping or ignoring errors from anything that wasn't recreated - and then drops the temp_priv table.
But you don't really want to (try to) create that temporary table in a trigger anyway - if two views are dropped, the first creates it, the second fails because it already exists. A perhaps more realistic approach might be to create that table once now:
create table TEMP_PRIV (PRIVILEGE_X VARCHAR2(4000));
and then utilise it for all subsequent upgrades, either by populating it with all grants for all views as a single step before the upgrade starts:
INSERT INTO TEMP_PRIVS (PRIVILEGE_X)
SELECT 'GRANT ' || PRIVILEGE || ' ON MYUSER.' || TABLE_NAME || ' TO ' || GRANTEE
FROM USER_VIEWS UV
JOIN USER_TAB_PRIVS UTP ON UTP.TABLE_NAME = UV.VIEW_NAME
WHERE UTP.GRANTEE not in ('MYUSER','PUBLIC');
or if you're still worried about possibly forgetting that step then with a trigger to do it one view at a time as they are dropped:
create or replace TRIGGER RECORD_GRANTS_ONDROP
BEFORE DROP ON MYUSER.SCHEMA
BEGIN
IF ora_dict_obj_owner = 'MYUSER' and ora_dict_obj_name not like 'TEMP_PRIV%' and ora_dict_obj_type='VIEW' then
INSERT INTO TEMP_PRIV
SELECT 'GRANT ' || PRIVILEGE || ' ON MYUSER.' || TABLE_NAME || ' TO ' || GRANTEE
FROM USER_TAB_PRIVS
WHERE GRANTEE not in ('MYUSER','PUBLIC')
AND TABLE_NAME = ora_dict_obj_name;
END IF;
END;
/
Then at the end of the upgrade process reissue all the statements in the table and clear it down ready for next time:
DECLARE
missing_view EXCEPTION;
PRAGMA EXCEPTION_INIT(missing_view, -942);
BEGIN
FOR loop_counter IN (select privilege_x from temp_priv)
LOOP
BEGIN
DBMS_OUTPUT.PUT_LINE(loop_counter.privilege_x);
EXECUTE IMMEDIATE loop_counter.privilege_x;
EXCEPTION
WHEN missing_view THEN
-- report but otherwise ignore
DBMS_OUTPUT.PUT_LINE(SQLERRM);
END;
END LOOP;
END;
/
TRUNCATE TABLE temp_priv;
If you go with the simpler non-trigger approach then it will re-grant existing privileges, but that's OK. And the exception handler means it'll report but skip any views that were dropped and not recreated, if that ever happens. (You'll still have to deal with any new view of course; your after-create trigger wouldn't have helped with that anyway.) And note that I've truncated the table, rather than dropped it - so it's still there, empty, when the next upgrade comes around and wants to populate it.
I have one table with id, name and complex queries. Below is just a sample of that table..
ID name Query
1 advisor_1 "Select * from advisor"
2 student_1 "Select * from student where id = 12"
3 faculty_4 "Select * from student where id = 12"
I want to iterate over this table and save each record into the csv file
Is there any way I can do it though Anonymous block automatically.
I don't want to do this manually as table has lots of rows.
Can anyone please help?
Not being superuser means the export can't be done in a server-side DO block.
It could be done client-side in any programming language that can talk to the database, or assuming a psql-only environment, it's possible to generate a list of \copy statements with an SQL query.
As an example of the latter, assuming the unique output filenames are built from the ID column, something like this should work:
SELECT format('\copy (%s) TO ''file-%s.csv'' CSV', query, id)
FROM table_with_queries;
The result of this query should be put into a file in a format such that it can be directly included into psql, like this:
\pset format unaligned
\pset tuples_only on
-- \g with an argument treats it as an output file.
SELECT format('\copy (%s) TO ''file-%s.csv'' CSV', query, id)
FROM table_with_queries \g /tmp/commands.sql
\i /tmp/commands.sql
As a sidenote, that process cannot be managed with the \gexec meta-command introduced in PG 9.6, because \copy itself is a meta-command. \gexec iterates only on SQL queries, not on meta-commands. Otherwise the whole thing could be done by a single \gexec invocation.
You may use a function like: (IF your problem is the code)
DECLARE
rec RECORD;
BEGIN
FOR rec IN SELECT id, query FROM table_name
LOOP
EXECUTE('COPY (' || rec.query || ') TO ' || QUOTE_LITERAL('d:/csv' || rec.id || '.csv') || ' CSV');
END LOOP;
END;
for permission problem, You should use some places on server that you have writing access to them (or request from vendor).
I have Oracle database with many schemas. I'd like to find xyz string in all the tables in one specific schema. I've tried to do it as in accepted answer here (Search All Fields In All Tables For A Specific Value (Oracle)). Unfortunately, I get error:
3 ORA-00903: invalid table name
ORA-06512: at line 8 SQL.sql 30 2
where line 8 is
SELECT COUNT(*) FROM ' || t.owner || '.' || t.table_name ||
How can I do this full search? I've also tried other googled solutions but none of them worked for me.
Adapted from the other answer, run this query:
SELECT
'SELECT '''|| owner||table_name||column_name|| ''' as locn, COUNT(*) FROM "' || owner || '"."' || table_name ||'" WHERE "'||column_name||'" = ''[VALUE YOURE LOOKING FOR HERE]'' UNION ALL'
FROM
all_tab_columns
WHERE owner <> 'SYS' and data_type LIKE '%CHAR%'
Replace the [VALUE YOURE LOOKING FOR HERE] with the value you're looking for. Replace the square brackets too. Do not touch any apostrophes.
Then run the query, and it'll produce a huge number of sql statements. Copy them out of your query tool results grid, paste them into the query area panel, delete the last union all and then run them. And wait. For a very long time. Eventually it'll produce a list of every table and column name, together with the count of the number of times that value you're looking for appears in that column
You can adapt the original code to cope with tables (and schemas, and columns) created with case-sensitive or otherwise invalid object identifiers, by treating everything as a quoted identifier.
EXECUTE IMMEDIATE
'SELECT COUNT(*) FROM "' || t.owner || '"."' || t.table_name || '"' ||
' WHERE "'||t.column_name||'" = :1'
INTO match_count
USING '1/22/2008P09RR8';
but using whatever string you're actually looking for, of course.
In the dynamic SQL that generates, the owner, table name and column name are now all enclosed in double quotes - which is what #CaiusJard is doing, but this still executes separate queries inside an anonymous block.
Here's a question for you all, is there any way to make the PL/SQL code below work by dynamically replacing the column_name in the cursor.column_name syntax? I know I'm confusing the pl/sql engine, but I'm not entirely sure on how to fix this...
Currently I get the error below, but I think the actual issue is that the pl/sql engine doesn't know how to interpret TRIM (e_rec.v_temp_column_name) :
"[Error] PLS-00302 (133: 26): PLS-00302: component
'V_TEMP_COLUMN_NAME' must be declared"
Parameter: x_person_rec IN OUT xxsome_table%ROWTYPE
v_temp_column_name dba_tab_columns.column_name%TYPE;
...
BEGIN
FOR e_rec IN (SELECT * FROM xxsome_table WHERE ..)
LOOP
--LOG (3, 'Loading/Sanitizing Header Record');
FOR col IN (SELECT column_name
FROM dba_tab_columns
WHERE table_name = UPPER ('xxsome_table'))
LOOP
--LOG (3, 'Sanitizing Column Name: ' || col.column_name);
v_temp_column_name := col.column_name;
x_person_rec.v_temp_column_name := TRIM (e_rec.v_temp_column_name);
END LOOP;
END LOOP;
...
I've tried doing this (which results in different error): x_person_rec.col.column_name := TRIM (e_rec.col.column_name);
No, you can't and you are indeed confusing the PL/SQL engine. The problem is that v_temp_column_name is a character, so TRIM (e_rec.v_temp_column_name) is evaluated as TRIM (e_rec.'v_temp_column_name'), which doesn't make any sense.
The best things to do, if trailing whitespace is a problem, is to ensure that all data being put into your database is trimmed by your application/ETL processes at the time. If you can't do this then use a trigger to ensure that it happens inside the database. If it's a really bad problem you can even enforce check constraints to stop it from ever happening.
Now, to sort of answer your question, there's no need to do anything dynamically here. Your cursor may be implicit but it's not been dynamically generated. You know every column in the table so be a little less lazy and type them all out.
FOR e_rec IN (SELECT trim(col1) as col1
, trim(col2) as col2
FROM xxsome_table WHERE ...
If you can't fix your data (or if it's not broken!) then this is easily the simplest way to do it.
To actually answer your question, you can dynamically build your SELECT statement using the same techniques you're using here...
declare
l_cols varchar2(4000);
l_curs sys_refcursor;
begin
select wm_concat('trim(' || column_name || ')')
into l_cols
from user_tab_columns
where table_name = 'XXSOME_TABLE'
;
open l_curs for
' select ' || l_cols || ' from xxsome_table where ...';
loop
...
end loop;
end;
/
As you're on 10g you can't use the excellent LISTAGG(), but there are plenty of other string aggregation techniques. Please note that if the resulting string is greater than 4,000 bytes, you'll have to loop rather than generating the column list in a single SQL statement.
P.S., if these columns are CHAR(n) then you'll have to trim them every time you select. It might be worth changing them to VARCHAR2(n)
Background: I'm converting a database table to a format that doesn't support null values. I want to replace the null values with an arbitrary number so my application can support null values.
Question: I'd like to search my whole table for a value ("999999", for example) to make sure that it doesn't appear in the table. I could write a script to test each column individually, but I wanted to know if there is a way I could do this in pure sql without enumerating each field. Is that possible?
You can use a special feature of the PostgreSQL type system:
SELECT *
FROM tbl t
WHERE t::text LIKE '%999999%';
There is a composite type of the same name for every table that you create in PostgreSQL. And there is a text representation for every type in PostgreSQL (to input / output values).
Therefore you can just cast the whole row to text and if the string '999999' is contained in any column (its text representation, to be precise) it is guaranteed to show in the query above.
You cannot rule out false positives completely, though, if separators and / or decorators used by Postgres for the row representation can be part of the search term. It's just very unlikely. And positively not the case for your search term '999999'.
There was a very similar question on codereview.SE recently. I added some more explanation in my answer there.
create or replace function test_values( real ) returns setof record as
$$
declare
query text;
output record;
begin
for query in select 'select distinct ''' || table_name || '''::text table_name, ''' || column_name || '''::text column_name from '|| quote_ident(table_name)||' where ' || quote_ident(column_name) || ' = ''' || $1::text ||'''::' || data_type from information_schema.columns where table_schema='public' and numeric_precision is not null
loop
raise notice '%1 qqqq', query;
execute query::text into output;
return next output;
end loop;
return;
end;$$ language plpgsql;
select distinct * from test_values( 999999 ) as t(table_name text ,column_name text)