Have PL/SQL Outputs in Real Time - sql

Is it possible to have Outputs from PL/SQL in real time? I have a pretty huge package that runs for more than an hour and I'd like to see where the package is at a particular time.
Anyways, I currently do this with a log table, which gets filled up with hundreds of log descriptions per run, I'm just curious if this is possible.
Thanks!

This is the kind of thing I use (output can be seen in v$session and v$session_longops)...
DECLARE
lv_module_name VARCHAR2(48);
lv_action_name VARCHAR2(32);
gc_MODULE CONSTANT VARCHAR2(48) := 'MY_PROC';
-- For LONGOPS
lv_rindex BINARY_INTEGER;
lv_slno BINARY_INTEGER;
lc_OP_NAME CONSTANT VARCHAR2(64) := '['||gc_MODULE||']';
lv_sofar NUMBER;
-- This is a guess as to the amount of work we will do
lv_totalwork NUMBER;
lc_TARGET_DESC CONSTANT VARCHAR2(64) := 'Tables';
lc_UNITS CONSTANT VARCHAR2(64) := 'Rows';
CURSOR tab_cur
IS
SELECT owner, table_name
FROM all_tables;
BEGIN
<<initialisation>>
BEGIN
-- To preserve the calling stack, read the current module and action
DBMS_APPLICATION_INFO.READ_MODULE( module_name => lv_module_name
, action_name => lv_action_name );
-- Set our current module and action
DBMS_APPLICATION_INFO.SET_MODULE( module_name => gc_MODULE
, action_name => NULL );
END initialisation;
<<main>>
BEGIN
DBMS_APPLICATION_INFO.SET_ACTION( action_name => 'Part 01' );
NULL;
DBMS_APPLICATION_INFO.SET_ACTION( action_name => 'Part 02' );
FOR tab_rec IN tab_cur
LOOP
DBMS_APPLICATION_INFO.SET_CLIENT_INFO( client_info => 'Rows = ['||TO_CHAR( tab_cur%ROWCOUNT, '999,999,999' )||']' );
NULL;
END LOOP;
DBMS_APPLICATION_INFO.SET_ACTION( action_name => 'Part 03' );
--Initialising longops
lv_rindex := DBMS_APPLICATION_INFO.SET_SESSION_LONGOPS_NOHINT;
lv_sofar := 0;
lv_totalwork := 5000; -- This is a guess, but could be actual if the query is quick
FOR tab_rec IN tab_cur
LOOP
DBMS_APPLICATION_INFO.SET_CLIENT_INFO( client_info => 'Rows = ['||TO_CHAR( tab_cur%ROWCOUNT, '999,999,999' )||']' );
lv_sofar := lv_sofar + 1;
-- Update our totalwork guess
IF lv_sofar > lv_totalwork
THEN
lv_totalwork := lv_totalwork + 500;
END IF;
DBMS_APPLICATION_INFO.SET_SESSION_LONGOPS( rindex => lv_rindex
, slno => lv_slno
, op_name => lc_OP_NAME
, sofar => lv_sofar
, totalwork => lv_totalwork
, target_desc => lc_TARGET_DESC
, units => lc_UNITS
);
END LOOP;
-- Clean up longops
DBMS_APPLICATION_INFO.SET_SESSION_LONGOPS( rindex => lv_rindex
, slno => lv_slno
, op_name => lc_OP_NAME
, sofar => lv_sofar
, totalwork => lv_sofar
, target_desc => lc_TARGET_DESC
, units => lc_UNITS
);
END main;
<<finalisation>>
BEGIN
-- Reset the module and action to the values that may have called us
DBMS_APPLICATION_INFO.SET_MODULE( module_name => lv_module_name
, action_name => lv_action_name );
-- Clear the client info, preventing any inter process confusion for anyone looking at it
DBMS_APPLICATION_INFO.SET_CLIENT_INFO( client_info => NULL );
END finalisation;
END;
/

I don't know if this is exactly what you want but I use dbms_application_info.set_module to see where my package is.
dbms_application_info.set_module(module_name => 'Conversion job',
action_name => 'updating table_x');
A query on v$session will show you which part of the procedure is running.

you could use autonomous transactions (as suggested in this SO for example).
This would allow you to write and commit in a log table without commiting the main transaction. You would then be able to follow what happens in your main script while it is running (incidentally, it will also allow you to time/tune your batch).

Use DBMS_PIPE to write a message to a named pipe. In another session you can read the messages from the pipe. Very simple, works like a charm !
procedure sendmessage(p_pipename varchar2
,p_message varchar2) is
s number(15);
begin
begin
sys.dbms_pipe.pack_message(p_message);
exception
when others then
sys.dbms_pipe.reset_buffer;
end;
s := sys.dbms_pipe.send_message(p_pipename, 0);
if s = 1
then
sys.dbms_pipe.purge(p_pipename);
end if;
end;
function receivemessage(p_pipename varchar2
,p_timeout integer) return varchar2 is
n number(15);
chr varchar2(200);
begin
n := sys.dbms_pipe.receive_message(p_pipename, p_timeout);
if n = 1
then
return null;
end if;
sys.dbms_pipe.unpack_message(chr);
return(chr);
end;

If your long-running job is processing a large number of fairly evenly sized tasks, you may find session longops a good way of monitoring the job progress, as well as allowing you to estimate how long the job will take to finish.
DBMS_APPLICATION_INFO.set_session_longops

If you have access to shell from PL/SQL environment you can call netcat:
BEGIN RUN_SHELL('echo "'||v_msg||'" | nc '||v_host||' '||v_port||' -w 5'); END;
/
v_host is a host running python script that reads data from socket on port v_port.
I used this design when I wrote aplogr for shell and pl/sql logs monitoring.

Related

How to Write Blob from Oracle Column to the File System

my_images table consists of a blob column called images. I need to write these images to my image_dir which is 'C:\TEMP'.
When the following PL/SQL code is executed, only the first image is written to the directory as an image. The second blob is written as 0 byte (empty) and there is no other (Should be a total number of 8).
So the loop does not seem to work correctly. I am using Oracle 11g Express Edition (XE) and SQL Developer. Here is the error and the code:
Error starting at line : 53 in command -
BEGIN write_blob_to_file_v5; END;
Error report -
ORA-01403: no data found
ORA-06512: at "SYS.DBMS_LOB", line 1056
ORA-06512: at "SYS.WRITE_BLOB_TO_FILE_V5", line 40
ORA-06512: at line 1
01403. 00000 - "no data found"
*Cause: No data was found from the objects.
*Action: There was no data from the objects which may be due to end of fetch.
PL/SQL code
CREATE OR REPLACE PROCEDURE write_blob_to_file_v5
AS
v_lob_image_name VARCHAR (100);
v_lob_image_id NUMBER;
v_blob BLOB;
v_buffer RAW (32767);
v_buffer_size BINARY_INTEGER;
v_amount BINARY_INTEGER;
v_offset NUMBER (38) := 1;
v_chunksize INTEGER;
v_out_file UTL_FILE.file_type;
BEGIN
FOR i IN (SELECT DBMS_LOB.getlength (v_blob) v_len,
image_id v_lob_image_id,
"IMAGE_NAME" v_lob_image_name,
image v_blob
FROM sys.MY_IMAGES)
LOOP
v_chunksize := DBMS_LOB.getchunksize (i.v_blob);
IF (v_chunksize < 32767)
THEN
v_buffer_size := v_chunksize;
ELSE
v_buffer_size := 32767;
END IF;
v_amount := v_buffer_size;
DBMS_LOB.open (i.v_blob, DBMS_LOB.lob_readonly);
v_out_file :=
UTL_FILE.fopen (
location => 'IMAGE_DIR',
filename => ( ''
|| i.v_lob_image_id
|| '_'
|| i.v_lob_image_name
|| '.JPG'),
open_mode => 'wb',
max_linesize => 32767);
WHILE v_amount >= v_buffer_size
LOOP
DBMS_LOB.read (i.v_blob,
v_amount,
v_offset,
v_buffer);
v_offset := v_offset + v_amount;
UTL_FILE.put_raw (file => v_out_file,
buffer => v_buffer,
autoflush => TRUE);
UTL_FILE.fflush (file => v_out_file);
--utl_file.new_line(file => v_out_file);
END LOOP;
UTL_FILE.fflush (v_out_file);
UTL_FILE.fclose (v_out_file);
DBMS_LOB.close (i.v_blob);
END LOOP;
END;
The main problem is related NOT to re-initialize the parameter v_offset to 1 ( as in the declaration section ) :
v_offset := 1;
for every image id just before
v_chunksize := dbms_lob.getchunksize(i.v_blob);
assignment.
Moreover, the problem may arise about not yet closed or already opened blobs. To prevent these,
replace
dbms_lob.open(i.v_blob,dbms_lob.lob_readonly);
with
if ( dbms_lob.isopen(i.v_blob)=0 ) then
dbms_lob.open(i.v_blob,dbms_lob.lob_readonly);
end if;
and
replace
dbms_lob.close(i.v_blob);
with
if ( dbms_lob.isopen(i.v_blob)=1 ) then
dbms_lob.close(i.v_blob);
end if;

Modify DBMS_SCHEDULER job to stop running between certain hours and minutes

I need to alter a job to run between 23:00PM up till 19:45PM. I have tried the below which achieves this, but the problem with the below is that the job will always stop for 15minutes per hour, between for example 10:45 till 11:00 etc. I'm not sure if I can modify the below to solve this issue.
BEGIN
DBMS_SCHEDULER.SET_ATTRIBUTE (
name => 'MY_TEST_JOB',
attribute => 'repeat_interval',
value => 'FREQ=SECONDLY;INTERVAL=5;
BYHOUR=23,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19;
BYDAY=mon,tue,wed,thu,fri,sat,sun;
BYMINUTE=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40'
);
END;
Create two schedules and combine them:
exec DBMS_SCHEDULER.CREATE_SCHEDULE('JOB_PERIOD_1',
repeat_interval => 'FREQ=SECONDLY;INTERVAL=5;BYHOUR=23,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18');
exec DBMS_SCHEDULER.CREATE_SCHEDULE('JOB_PERIOD_2',
repeat_interval => 'FREQ=SECONDLY;INTERVAL=5;BYHOUR=19;BYMINUTE=0,1,2,3,4,5,[...],43,44');
BEGIN
DBMS_SCHEDULER.SET_ATTRIBUTE (
name => 'MY_TEST_JOB',
attribute => 'repeat_interval',
value => 'JOB_PERIOD_1,JOB_PERIOD_2'
);
END;
Actually I tested it only with this PL/SQL block.
DECLARE
next_run_date TIMESTAMP := TIMESTAMP '2018-02-06 19:44:00';
res INTEGER := 0;
BEGIN
FOR i IN 1..30 LOOP
DBMS_SCHEDULER.EVALUATE_CALENDAR_STRING('JOB_PERIOD_1,JOB_PERIOD_2', NULL, next_run_date, next_run_date);
DBMS_OUTPUT.PUT_LINE(next_run_date);
END LOOP;
END;

pdf to text conversion using oracle package

i m seeing some strange behavior for my pdf to text conversion using oracle
bellow is the code of a sql file.
create or replace directory pdf_dir as '&1';
create or replace directory l_curr_dir as '&3';
declare
ll_clob CLOB;
l_bfile BFILE;
l_filename VARCHAR2(100) := '&2';
begin
begin
ctx_ddl.drop_policy('test_policy');
exception
when others then
null;
end;
ctx_ddl.create_policy('test_policy','ctxsys.auto_filter');
l_bfile := bfilename('PDF_DIR',l_filename);
dbms_lob.createtemporary(ll_clob, true);
ctx_doc.policy_filter(
policy_name => 'test_policy'
, document => l_bfile
, restab => ll_clob
, plaintext => true
);
ll_clob := REPLACE(TRIM(ll_clob), chr(13), chr(10));
ll_clob := REPLACE(ll_clob, chr(10), chr(32) || '<<EOL>>' || chr(10)||'<<BOL>>');
INSERT into tempclob_op(filename, data) VALUES(l_filename, ll_clob);
DBMS_XSLPROCESSOR.clob2file (ll_clob,'L_CURR_DIR' , 'plaintext.text');
dbms_lob.freeTemporary( ll_clob );
end;
/
problem is i have run this code for 10000 files and it gives correct results for almost all but for almost 10 files it corrupts the output in plaintext.text file. And i dont know why is it happening? Also when i run this sql code for individual files it gives me correct results.
I have added some delay of 2 seconds in every execution while in loop for each file. and seems it resolved the problem strangely ..no concrete answers though.

Trying to Understand Oracle Deterministic Function Speed Test Results

This question arose as a spin-off from this one: Adding Many (UDFs) Validation Functions to Oracle - Which Method Run Fastest
I am debating between putting a low level function that will be used everywhere in my application in an object as a CONSTRUCTOR FUNCTION so it's encapsulated with it's type that it returns, or just making the function STAND-ALONE. I know most would thing these test result are inconsequential because the loop count is so high. But in our app this function could be used to check multiple columns in each large loop iteration. So it really wouldn't be hard for us to reach 3 million checks in a single process. And this all reports back to a web page so the user is waiting on these results, so every msec matters.
So I was running some results and this is what I found...
/*****
isValid is a CONSTRUCTOR FUNCTION of an OBJECT
and it is deterministic
passing in:
'blah' -> 12 seconds
x -> 464 msecs
changing x to varchar2(7)
x -> 2 seconds (this is how we would use it in most cases within loops)
*****/
declare
x number;
--x varchar2(7);
begin
for i in 1 .. 3000000 loop
x := x + 1;
if (isValid(x,'number').result = 1) then
null;
end if;
end loop;
end;
/
/*****
isValid2 is a STAND-ALONE FUNCTION
and it is deterministic
passing in:
'blah' -> 407 msecs
x -> 4 seconds
changing x to varchar2(7)
x -> 4 seconds (this is how we would use it in most cases within loops)
*****/
declare
x number;
--x varchar2(7);
begin
for i in 1 .. 3000000 loop
x := x + 1;
if (isValid2(x,'number').result = 1) then
null;
end if;
end loop;
end;
/
Based on these result I think I'm going to go with the CONSTRUCTOR FUNCTION of an OBJECT approach.
So my question is: Why are the result between 'blah' and x completely reversed between the two different methods?
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
PL/SQL Release 11.2.0.2.0 - Production
CORE 11.2.0.2.0 Production
TNS for 64-bit Windows: Version 11.2.0.2.0 - Production
NLSRTL Version 11.2.0.2.0 - Production
PLSQL_OPTIMIZE_LEVEL = 2 (I've never read up on this, this is a test box, I'm not sure what it *should* be set to yet but I will read up on it)
PLSQL_CODE_TYPE = INTERPRETED (I know NATIVE is faster but they won't change it.)
Here is the stand-alone function...
create or replace type valObj as object (
result number(1),
resulttext varchar2(32000) );
/
create or replace function isValid2(v in varchar2, f in varchar2)
return valObj
deterministic
is
test PLS_INTEGER;
begin
if f = 'number' then
begin
test := to_number(v);
return valObj(1,null);
exception when VALUE_ERROR then return valObj(0,'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...');
end;
elsif f = 'phone' then
null; --TO DO
elsif f = 'integer' then
null; --TO DO
elsif f = 'email' then
null; --TO DO
elsif f = 'IPv4' then
null; --TO DO
elsif f = 'IPv6' then
null; --TO DO
end if;
--dozens of others to follow....
end;
/
And here is the object/function...
create or replace type isValid as object (
result number(1),
resulttext varchar2(32000),
constructor function isValid(v varchar, f varchar) return self as result deterministic );
/
create or replace type body isValid as
constructor function isValid(v varchar, f varchar) return self as result deterministic is
test PLS_INTEGER;
begin
if f = 'number' then
begin
test := to_number(v);
self.result := 1;
self.resulttext := null;
return;
exception when VALUE_ERROR then
self.result := 0;
self.resulttext := 'Invalid number. Valid formats are: 12345, 12345.67, -12345, etc...';
return;
end;
elsif f = 'phone' then
null; --TO DO
elsif f = 'integer' then
null; --TO DO
elsif f = 'email' then
null; --TO DO
elsif f = 'IPv4' then
null; --TO DO
elsif f = 'IPv6' then
null; --TO DO
end if;
--dozens of others to follow....
end;
end;
/

Monitoring long-running PL/SQL block

I have a fairly time intensive PL/SQL block that builds fingerprints from molecular structures. I would like to print output to SQL*Plus console to provide feedback on how many structures have been processed. I can do this with dbms_output.put_line
However everytime that is called a new line is written. I want to overwrite the line.
For example, currently I have the below.
Structure x of y processed
Structure x of y processed
Structure x of y processed
Structure x of y processed
Eventually I fill up the buffer as I'm dealing with thousands of structure records.
Is there a method I can use that will just overwrite the last output line?
Using DBMS_OUTPUT means that SQL*Plus will display nothing until the entire PL/SQL block is complete and will then display all the data currently in the buffer. It is not, therefore, an appropriate way to provide an ongoing status.
On the other hand, Oracle does provide a package DBMS_APPLICATION_INFO that is specifically designed to help you monitor your running code. For example, you could do something like
CREATE PROCEDURE process_structures
AS
<<other variable declarations>>
rindex BINARY_INTEGER;
slno BINARY_INTEGER;
totalwork NUMBER := y; -- Total number of structures
worksofar NUMBER := 0; -- Number of structures processed
BEGIN
rindex := dbms_application_info.set_session_longops_nohint;
FOR i IN (<<select structures to process>>)
LOOP
worksofar := worksofar + 1;
dbms_application_info.set_session_longops(
rindex => rindex,
slno => slno,
op_name => 'Processing of Molecular Structures',
sofar => worksofar ,
totalwork => totalwork,
target_desc => 'Some description',
units => 'structures');
<<process your structure with your existing code>>
END LOOP;
END;
From a separate SQL*Plus session, you can then monitory progress by querying the V$SESSION_LONGOPS view
SELECT opname,
target_desc,
sofar,
totalwork,
units,
elapsed_seconds,
time_remaining
FROM v$session_longops
WHERE opname = 'Processing of Molecular Structures';
You may also send messages to a named pipe and have another process read the message from the pipe.
procedure sendmessage(p_pipename varchar2
,p_message varchar2) is
s number(15);
begin
begin
sys.dbms_pipe.pack_message(p_message);
exception
when others then
sys.dbms_pipe.reset_buffer;
end;
s := sys.dbms_pipe.send_message(p_pipename, 0);
if s = 1
then
sys.dbms_pipe.purge(p_pipename);
end if;
end;
function receivemessage(p_pipename varchar2
,p_timeout integer) return varchar2 is
n number(15);
chr varchar2(200);
begin
n := sys.dbms_pipe.receive_message(p_pipename, p_timeout);
if n = 1
then
return null;
end if;
sys.dbms_pipe.unpack_message(chr);
return(chr);
end;
I don't think you can. As far as I understood the dbms_output it just doesn't work that way.
I recommend you use put to echo a single dot and a newline every 1000 or so entries to see that something is happening and write into a table or sequence the current position so you can have a look if you want to know.