I am using the following stored procedures in an Oracle database to export the output of the query to a CSV file.
CREATE OR REPLACE PROCEDURE run_query(p_tbl_name IN VARCHAR2
) IS
Select_Stmt VARCHAR2(100) := 'select * from '||p_tbl_name;
p_dir VARCHAR2 (100) := 'DATA_PUMP_DIR';
v_finaltxt VARCHAR2(4000);
v_v_val VARCHAR2(4000);
v_n_val NUMBER;
v_d_val DATE;
v_ret NUMBER;
c NUMBER;
d NUMBER;
col_cnt INTEGER;
f BOOLEAN;
rec_tab DBMS_SQL.DESC_TAB;
col_num NUMBER;
v_fh UTL_FILE.FILE_TYPE;
BEGIN
c := DBMS_SQL.OPEN_CURSOR;
DBMS_SQL.PARSE(c, Select_Stmt, DBMS_SQL.NATIVE);
d := DBMS_SQL.EXECUTE(c);
DBMS_SQL.DESCRIBE_COLUMNS(c, col_cnt, rec_tab);
FOR j in 1..col_cnt
LOOP
CASE rec_tab(j).col_type
WHEN 1 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
WHEN 2 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_n_val);
WHEN 3 THEN DBMS_SQL.DEFINE_COLUMN(c,j,v_d_val);
ELSE
DBMS_SQL.DEFINE_COLUMN(c,j,v_v_val,2000);
END CASE;
END LOOP;
-- This part outputs the HEADER
v_fh := UTL_FILE.FOPEN(upper(p_dir),p_tbl_name||'.csv','w',32767);
FOR j in 1..col_cnt
LOOP
v_finaltxt := ltrim(v_finaltxt||','||lower(rec_tab(j).col_name),',');
END LOOP;
UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
-- This part outputs the DATA
LOOP
v_ret := DBMS_SQL.FETCH_ROWS(c);
EXIT WHEN v_ret = 0;
v_finaltxt := NULL;
FOR j in 1..col_cnt
LOOP
CASE rec_tab(j).col_type
WHEN 1 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_v_val);
v_finaltxt := ltrim(v_finaltxt||',"'||v_v_val||'"',',');
WHEN 2 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_n_val);
v_finaltxt := ltrim(v_finaltxt||','||v_n_val,',');
WHEN 3 THEN DBMS_SQL.COLUMN_VALUE(c,j,v_d_val);
v_finaltxt := ltrim(v_finaltxt||','||to_char(v_d_val,'DD/MM/YYYY HH24:MI:SS'),',');
ELSE
DBMS_SQL.COLUMN_VALUE(c,j,v_v_val);
v_finaltxt := ltrim(v_finaltxt||',"'||v_v_val||'"',',');
END CASE;
END LOOP;
-- DBMS_OUTPUT.PUT_LINE(v_finaltxt);
UTL_FILE.PUT_LINE(v_fh, v_finaltxt);
END LOOP;
UTL_FILE.FCLOSE(v_fh);
DBMS_SQL.CLOSE_CURSOR(c);
END;
/
The above stored procedure is working perfectly and I got the output by using the following script to run this stored procedure
exec run_query person_list;
Here stored_procedure_name is run_query and the table_name is person_list.
Now my question is that how can I use the WHERE clause after the select statement on "line 3" -
Select_Stmt VARCHAR2(100) := 'select * from '||p_tbl_name;
or are there any other ways I can use the where clause.
Thanks in advance.
Select_Stmt VARCHAR2(100) := 'select * from '||p_tbl_name||' where name=''david''';
You have to have the character delimiters; and since you have it inside quotes you have to duplicate them.
I have the following problem:
I have a table Source_Data(actually, quite many different source tables), and I need to export some data from them into Result_Table. For each source table I have some sql that returns the data, and it needs to be transformed into csv-like string.
For example, Source_Data1 table has PK - ID, some foreign keys - A_ID, B_ID, CREATION_DATE date, and some other columns.
I need to export the result of SQL query that selects data from Source_Data1 table, it usually returns the primary key, some foreign keys which are not null and a date which is also not null, and transform it into cvs-like string, which should be saved later in some other table as a clob.
CREATE TABLE Source_Data1 (
ID NUMBER(3) NOT NULL,
A_ID NUMBER(10) NOT NULL,
B_ID NUMBER(10) NOT NULL,
CREATION_DATE DATE NOT NULL,
some other columns );
and I have SQL query:
select ID, A_ID, B_ID, CREATION_DATE
from Source_Data1
where <expression>
Based on this query, I need to generate string like this:
'ID,A_ID,B_ID,CREATION_DATE
1,200,10,2018-03-01 00:00:00.0
7,202,11,2018-03-02 00:00:00.0
8,205,11,2018-03-02 00:00:00.0'
and I need to save it into the Result table as a clob:
Insert into Result_Table (ID, SOURCE_NAME, DATA) values
(result_seq.nextval, 'Source_Data1', result of sqlquery 1);
And this needs to be done for many source tables.
I was wondering if there is some general way to do it? Because for each source table I have its own sql query, different set of selected columns, and generating this csv string manually looks not very nice.
Thank you for your advice.
Step 1 should be to download SQLcl, you can easily save the output of SQL select statements in CSV format via "SET SQLFORMAT CSV" and SPOOL, it does the work for you. So you can use that to loop through a bunch of table names / select statements, spool and save the output of each to a directory via a shell script.
Next, here's a package that has a bunch of file utilities that all work with oracle DIRECTORY objects and the files within them. With this and some PL/SQL, you could easily pull those files that you've saved off into a table. There may be easier ways to do what you're trying to do, and if there is, I look forward to hearing about them for others. It's kind of a big problem to solve.
1) file_into_blob - To get the file into the database
2) convert_blob_to_clob - To convert to clob
3) You can then insert that into your table.
Package spec
Create or replace package fileutils as
--
-- This procedure deletes a file, and depends on an Oracle DIRECTORY object being passed
--
Procedure delete_os_file (i_directory varchar2, i_filename varchar2);
--
-- This procedure moves and optionally renames a file,
-- and depends on an Oracle DIRECTORY object being passed
--
Procedure move_os_file ( i_source_directory in varchar2, i_source_file in varchar2, i_target_directory in varchar2, i_target_file in varchar2);
--
-- This procedure takes a blob variable and writes it to a file,
-- and depends on an Oracle DIRECTORY object being passed
--
Procedure blob_into_file (i_directory in varchar2, i_file_name in varchar2, i_blob in blob);
--
-- This procedure takes a file and uploads it into a blob variable
-- and depends on an Oracle DIRECTORY object being passed
--
Procedure file_into_blob(i_directory in varchar2, i_file_name in varchar2, o_blob out blob);
--
-- This procedure converts a clob to a blob
--
Procedure convert_clob_to_blob (i_clob in clob, o_blob out blob);
--
-- This procedure converts a blob to a clob
--
Procedure convert_blob_to_clob (i_blob in blob, o_clob out clob);
--
-- This one checks for file existence without Java
--
Function file_exists (i_directory in varchar2, i_filename in varchar2) return boolean;
--
-- Returns the basename of a filename
-- Works with Windows and UNIX pathnames
--
Function basename (i_filename in varchar2) return varchar2;
--
-- This takes a Base64 string and converts it to a binary BLOB
--
Procedure base64_string_to_blob (i_clob in clob, o_blob out blob);
Function base64_string_to_blob (i_clob in clob) return blob;
--
-- This takes a binary BLOB and converts it to a Base64 string
--
Procedure blob_to_base64_string (i_blob in blob, o_clob out clob);
Function blob_to_base64_string (i_blob in blob) return clob;
End fileutils;
/
Show error;
Package body
Set define off;
Create or replace package body fileutils as
Procedure delete_os_file (i_directory varchar2, i_filename varchar2)
is
Begin
utl_file.fremove(i_directory,i_filename);
End;
Procedure move_os_file
(
i_source_directory in varchar2,
i_source_file in varchar2,
i_target_directory in varchar2,
i_target_file in varchar2
)
is
srcdir varchar2(255) := upper(i_source_directory);
tgtdir varchar2(255) := upper(i_target_directory);
begin
--
-- NOTE: If you're getting the all-too-familiar
-- ORA-29292: file rename operation failed
-- and you're SURE that your directory names are correct,
-- and you're SURE that your privileges are correct, both at the
-- OS level, and within the database, there's one last thing that
-- can get you. I learned the hard way that this command will NOT
-- work successfully renaming a file from one filesystem to another,
-- at least when those filesystems are NFS mounted. That is all.
--
utl_file.frename(srcdir,i_source_file,tgtdir,i_target_file,TRUE);
end move_os_file;
Procedure blob_into_file (i_directory in varchar2, i_file_name in varchar2, i_blob in blob)
is
l_file utl_file.file_type;
l_buffer raw(32767);
l_amount binary_integer := 32767;
l_pos integer := 1;
i_blob_len integer;
Begin
i_blob_len := dbms_lob.getlength(i_blob);
l_pos:= 1;
-- Open the destination file.
l_file := utl_file.fopen(i_directory,i_file_name,'wb', 32767);
-- Read chunks of the BLOB and write them to the file
-- until complete.
while l_pos < i_blob_len loop
dbms_lob.read(i_blob, l_amount, l_pos, l_buffer);
utl_file.put_raw(l_file, l_buffer, TRUE);
l_pos := l_pos + l_amount;
end loop;
-- Close the file.
utl_file.fclose(l_file);
End blob_into_file;
Procedure file_into_blob(i_directory in varchar2, i_file_name in varchar2, o_blob out blob)
is
src_loc bfile := bfilename(i_directory, i_file_name);
Begin
-- Initialize the dest blob
o_blob := empty_blob();
-- Open source binary file from OS
dbms_lob.open(src_loc, dbms_lob.lob_readonly);
-- Create temporary LOB object
dbms_lob.createtemporary(
lob_loc => o_blob
, cache => true
, dur => dbms_lob.session
);
-- Open temporary lob
dbms_lob.open(o_blob, dbms_lob.lob_readwrite);
-- Load binary file into temporary LOB
dbms_lob.loadfromfile(
dest_lob => o_blob
, src_lob => src_loc
, amount => dbms_lob.getLength(src_loc));
-- Close lob objects
dbms_lob.close(o_blob);
dbms_lob.close(src_loc);
End file_into_blob;
Function basename (i_filename in varchar2) return varchar2
is
v_basename varchar2(1024);
Begin
--
-- If the regex's below don't match, then it's already at its base name
-- Return what was passed.
--
v_basename := i_filename;
if regexp_like(i_filename,'^.*\\') then
dbms_output.put_line('This is a Windows file');
v_basename := regexp_substr(i_filename,'[^\]*$');
dbms_output.put_line('Basename is : '||v_basename);
end if;
if regexp_like(i_filename,'^/') then
dbms_output.put_line('This is a UNIX file');
v_basename := regexp_substr(i_filename,'[^/]*$');
dbms_output.put_line('Basename is : '||v_basename);
end if;
return v_basename;
End basename;
Function file_exists (i_directory in varchar2, i_filename in varchar2) return boolean
is
v_exists boolean;
v_file_length number;
v_block_size number;
Begin
utl_file.fgetattr(upper(i_directory), i_filename, v_exists, v_file_length, v_block_size);
if (v_exists) then
dbms_output.put_line('File '||i_filename||' exists, '||v_file_length||' bytes');
else
dbms_output.put_line('File '||i_filename||' does not exist');
end if;
return v_exists;
end file_exists;
Procedure convert_clob_to_blob (i_clob in clob, o_blob out blob)
is
v_in pls_Integer := 1;
v_out pls_Integer := 1;
v_lang pls_Integer := 0;
v_warning pls_Integer := 0;
Begin
dbms_lob.createtemporary(o_blob,TRUE);
dbms_lob.converttoblob(o_blob,i_clob,DBMS_lob.getlength(i_clob),v_in,v_out,dbms_lob.default_csid,v_lang,v_warning);
End convert_clob_to_blob;
Procedure convert_blob_to_clob (i_blob in blob, o_clob out clob)
is
v_in pls_Integer := 1;
v_out pls_Integer := 1;
v_lang pls_Integer := 0;
v_warning pls_Integer := 0;
Begin
dbms_lob.createtemporary(o_clob,TRUE);
dbms_lob.converttoclob(o_clob,i_blob,DBMS_lob.getlength(i_blob),v_in,v_out,dbms_lob.default_csid,v_lang,v_warning);
End convert_blob_to_clob;
Procedure blob_to_base64_string (i_blob in blob, o_clob out clob)
is
v_out_cl clob;
file_len pls_integer;
modulo pls_integer;
pieces pls_integer;
amt binary_integer := 23808;
buf raw (32767);
buf_tx varchar2(32767);
pos pls_integer := 1;
filepos pls_integer := 1;
counter pls_integer := 1;
Begin
dbms_lob.createtemporary (v_out_cl, true, dbms_lob.call);
file_len := dbms_lob.getlength (i_blob);
modulo := mod (file_len, amt);
pieces := trunc (file_len / amt);
while (counter <= pieces) loop
dbms_lob.read (i_blob, amt, filepos, buf);
buf_tx:=utl_raw.cast_to_varchar2 (utl_encode.base64_encode (buf));
dbms_lob.writeappend (v_out_cl,length(buf_tx),buf_tx);
filepos := counter * amt + 1;
counter := counter + 1;
end loop;
if (modulo <> 0) THEN
dbms_lob.read (i_blob, modulo, filepos, buf);
buf_tx:=utl_raw.cast_to_varchar2 (utl_encode.base64_encode (buf));
dbms_lob.writeappend (v_out_cl,length(buf_tx),buf_tx);
end if;
o_clob := v_out_cl;
End blob_to_base64_string;
Function blob_to_base64_string (i_blob in blob) return clob
is
v_out_cl clob;
file_len pls_integer;
modulo pls_integer;
pieces pls_integer;
amt binary_integer := 23808;
buf raw (32767);
buf_tx varchar2(32767);
pos pls_integer := 1;
filepos pls_integer := 1;
counter pls_integer := 1;
Begin
dbms_lob.createtemporary (v_out_cl, true, dbms_lob.call);
file_len := dbms_lob.getlength (i_blob);
modulo := mod (file_len, amt);
pieces := trunc (file_len / amt);
while (counter <= pieces) loop
dbms_lob.read (i_blob, amt, filepos, buf);
buf_tx:=utl_raw.cast_to_varchar2 (utl_encode.base64_encode (buf));
dbms_lob.writeappend (v_out_cl,length(buf_tx),buf_tx);
filepos := counter * amt + 1;
counter := counter + 1;
end loop;
if (modulo <> 0) THEN
dbms_lob.read (i_blob, modulo, filepos, buf);
buf_tx:=utl_raw.cast_to_varchar2 (utl_encode.base64_encode (buf));
dbms_lob.writeappend (v_out_cl,length(buf_tx),buf_tx);
end if;
return v_out_cl;
End blob_to_base64_string;
Procedure base64_string_to_blob (i_clob in clob, o_blob out blob)
is
v_out_bl blob;
clob_size number;
pos number;
charBuff varchar2(32767);
dBuffer RAW(32767);
v_readSize_nr number;
v_line_nr number;
begin
dbms_lob.createTemporary (v_out_bl, true, dbms_lob.call);
v_line_nr:=greatest(65, instr(i_clob,chr(10)), instr(i_clob,chr(13)));
v_readSize_nr:= floor(32767/v_line_nr)*v_line_nr;
clob_size := dbms_lob.getLength(i_clob);
pos := 1;
while (pos < clob_size) loop
dbms_lob.read (i_clob, v_readSize_nr, pos, charBuff);
dBuffer := UTL_ENCODE.base64_decode (utl_raw.cast_to_raw(charBuff));
dbms_lob.writeAppend (v_out_bl,utl_raw.length(dBuffer),dBuffer);
pos := pos + v_readSize_nr;
end loop;
o_blob := v_out_bl;
end base64_string_to_blob;
Function base64_string_to_blob (i_clob in clob) return blob
is
v_out_bl blob;
clob_size number;
pos number;
charBuff varchar2(32767);
dBuffer RAW(32767);
v_readSize_nr number;
v_line_nr number;
begin
dbms_lob.createTemporary (v_out_bl, true, dbms_lob.call);
v_line_nr:=greatest(65, instr(i_clob,chr(10)), instr(i_clob,chr(13)));
v_readSize_nr:= floor(32767/v_line_nr)*v_line_nr;
clob_size := dbms_lob.getLength(i_clob);
pos := 1;
while (pos < clob_size) loop
dbms_lob.read (i_clob, v_readSize_nr, pos, charBuff);
dBuffer := UTL_ENCODE.base64_decode (utl_raw.cast_to_raw(charBuff));
dbms_lob.writeAppend (v_out_bl,utl_raw.length(dBuffer),dBuffer);
pos := pos + v_readSize_nr;
end loop;
return v_out_bl;
end base64_string_to_blob;
end fileutils;
/
Show error;
I think you can use LISTAGG function
select listagg(ONE_LINE) WITHIN GROUP (ORDER BY ROW_NUM)
from (
select 'ID,A_ID,B_ID,CREATION_DATE' || CHR(10) as ONE_LINE,
-1 as ROW_NUM
from dual
union all
select ID ||','|| A_ID ||','|| B_ID ||','|| CREATION_DATE || CHR(10) as ONE_LINE,
ROWNUM as ROW_NUM
from Source_Data1
where <expression>
);
The following code will create a CSV version of a table as a CLOB row, using DATA_DUMP.SQL and the function DBMS_XSLPROCESSOR.READ2CLOB.
First, install the DATA_PUMP procedure on your schema. The program is a single command in a single file, you can run it through SQL*Plus or just copy and paste it into your IDE.
Next, create a directory on the server to temporarily store the files. Most existing utilities are meant to create files. It's easier to write a file and read it as a CLOB, than to modify the utility to write directly into a table.
create or replace directory temp_dir as 'C:\temp';
Then this PL/SQL block will read and write the data:
--Store a table as a single CSV clob.
--TODO: Delete the files from the server when done.
declare
v_clob clob;
begin
for tables in
(
--Query that returns the table names.
--(Doesn't have to be hard-coded, maybe a query on DBA_TABLES would help?)
select column_value table_name
from table(sys.odcivarchar2list('source_data'))
) loop
data_dump
(
query_in => 'select * from source_data1',
file_in => 'source_data1.csv',
directory_in => 'temp_dir',
nls_date_fmt_in => 'YYYY-MM-DD HH24:MI:SS',
delimiter_in => ',',
header_row_in => true
);
v_clob := dbms_xslprocessor.read2clob
(
flocation => 'TEMP_DIR',
fname => 'source_data1.csv'
);
insert into result_table (id, source_name, data) values
(result_seq.nextval, 'source_data1', v_clob);
--I wouldn't normally commit after each row, but when exporting large
--tables the script may run too long and there's a good chance of an
--error, so partial results may be helpful.
commit;
end loop;
end;
/
Reading and writing to CSV files is not rocket science but it's not as trivial as most people think. 99% of the CSV programs out there can't handle things like commas in the data, or adding a header row. So it's best to look for preexisting code instead of writing your own utility.
Does anybody know how to split string separated by ';' in a trigger, faster than this code:
SELECT regexp_substr('asd;asaaaad;dd;', '([^;]*)(;)', 1, level, null, 1)
BULK COLLECT INTO array_TREATMENT_TR_CD
FROM DUAL
CONNECT BY LEVEL < REGEXP_COUNT('asd;asaaaad;dd;', '[;]') + 1 ;
Regexp functions are usually way slower than standard db functions. When I run your code 100.000 times it takes around 12 seconds (on my db, YMMV).
Then I created this small test:
declare
type t_vc2 is table of varchar2(250) index by pls_integer;
procedure split_str(
p_str1 in out nocopy varchar2
, p_str2 out varchar2
, p_separator in varchar2 default ';'
)
is
l_pos pls_integer;
begin
l_pos := instr(p_str1, p_separator);
if nvl(l_pos, 0) = 0
then
p_str2 := p_str1;
p_str1 := null;
else
p_str2 := substr(p_str1, 1, l_pos-1);
p_str1 := substr(p_str1, l_pos+1);
end if;
end split_str;
begin
for l_i in 1 .. 1000000
loop
declare
array_TREATMENT_TR_CD t_vc2;
l_input varchar2(250);
l_return varchar2(250);
begin
l_input := 'asd;asaaaad;dd;';
loop
split_str(l_input, l_return);
if l_return is not null
then
array_TREATMENT_TR_CD(array_TREATMENT_TR_CD.count+1) := l_return;
end if;
exit when l_return is null;
end loop;
end;
end loop;
end;
/
This parses your example string 100.000 times to an array with standard db functions. It takes around 0.5 seconds. That's a significant improvement if "real time" is what you aim for.
I am a newbie when it comes to PL/SQL. I have the following block of PL/SQL code:
FUNCTION evaluate(p_sid IN NUMBER, p_min_len IN NUMBER, p_max_len IN NUMBER) RETURN VARCHAR2
IS
v_res VARCHAR2(255);
BEGIN
v_res := evaluate_batch(CURSOR(SELECT sid, stext FROM maintable WHERE sid = p_sid), p_min_len, p_max_len);
RETURN v_res;
END eval_ocr_quality;
where
TYPE text_rec_t IS RECORD (
rec_id NUMBER,
text CLOB);
TYPE text_rec_cur_t IS REF CURSOR RETURN text_rec_t;
FUNCTION evaluate(p_sid IN NUMBER, p_min_len IN NUMBER := 3, p_max_len IN NUMBER := 3) RETURN VARCHAR2;
FUNCTION evaluate_batch(p_cur IN text_rec_cur_t, p_min_len IN NUMBER := 3, p_max_len IN NUMBER := 3) RETURN VARCHAR2;
I get PLS-00405: subquery not allowed in this context error at the v_res := evaluate_batch line
What am I doing wrong?
Open the cursor separately and pass it in:
FUNCTION evaluate(p_sid IN NUMBER, p_min_len IN NUMBER, p_max_len IN NUMBER) RETURN VARCHAR2
IS
v_res VARCHAR2(255);
c TEXT_REC_CUR_T;
BEGIN
OPEN c FOR SELECT TEXT_REC_T(sid, stext) FROM maintable WHERE sid = p_sid;
v_res := evaluate_batch(c, p_min_len, p_max_len);
-- May need to close cursor c here if not closed in evaluate_batch
RETURN v_res;
END eval_ocr_quality;
Share and enjoy.
I want to pass a multiple values like '1,2,3,4,5,6,7,8,9,10,11,12' on as a single parameter and insert them into a table.
I have a scenario like saving a bill for particular customer, the customer might have purchased multiple items.
You can pass the values comma separated. This PL/SQL procedure breaks up a comma-separated string (passed as P_VALUE) and inserts each token into a table as a separate row, along with some other values.
PROCEDURE XXXXGL_INST_PARAMS (P_VALUE VARCHAR2) IS
i number;
n number;
r number;
pos number;
L_NAT_ACCT varchar2(10);
L_UID number;
BEGIN
L_UID := fnd_global.user_id;
n := length(P_VALUE);
i := 1;
r := 1;
WHILE (i < n) LOOP
pos := (INSTR(P_VALUE,',',1,r)-i);
IF pos < 0 THEN
pos := n;
END IF;
SELECT substr(P_VALUE,i,pos) INTO L_NAT_ACCT FROM dual;
INSERT INTO xxXXX.xxXXXgl_natural_accts_t
(NATURAL_ACCT, CREATION_DATE, CREATED_BY, LAST_UPDATE_LOGIN,
LAST_UPDATE_DATE, LAST_UPDATED_BY)
VALUES(L_NAT_ACCT,SYSDATE,L_UID,L_UID,SYSDATE,L_UID);
i := i + length(L_NAT_ACCT)+1;
r := r+1;
END LOOP;
END;
Note: I found this here.