Export all The extended ASCII codes (character code 128-255) in XML file from oracle store procedure - sql

I have created a Store Procedure to generate an xml file of Table Data,
but in my database some table have "extended ASCII codes (character code 128-255)".
When I generate xml file then it shows error "ORA-31061: XDB error: special char to escaped char conversion failed."
So I replaced these Char to space but I need all ASCII codes (character code 128-255) in XML files.
Please help
My Store Procedure is below:
create or replace
PROCEDURE Export_project6
(
V_TABLE_NAME1 IN varchar2,
v_FLAG OUT NUMBER
)
AS
BEGIN
----- Export table data
DECLARE
v_file UTL_FILE.file_type;
qryCtx DBMS_XMLGEN.ctxHandle;
result CLOB;
v_FILENAME varchar2(50);
V_TABLE_NAME varchar2(50);
xt_data xmltype;
v_ctx dbms_xmlgen.ctxHandle;
rc_data sys_refcursor;
BEGIN
V_TABLE_NAME := UPPER(V_TABLE_NAME1) ;
v_file := UTL_FILE.fopen('MYXML',V_TABLE_NAME||'.xml', 'W');
OPEN rc_data FOR
'select * FROM '||V_TABLE_NAME||' ORDER BY 1' ;
v_ctx := dbms_xmlgen.newContext (rc_data);
DBMS_XMLGEN.USEITEMTAGSFORCOLL (v_ctx);
DBMS_XMLGEN.SETNULLHANDLING(v_ctx, 1);
DBMS_XMLGEN.setrowsettag(v_ctx,'root');
DBMS_XMLGEN.setrowtag(v_ctx,V_TABLE_NAME );
result:= DBMS_XMLGEN.getXML(v_ctx);
result := REPLACE( result, '<?xml version="1.0"?>','<?xml version="1.0" encoding="UTF-8" standalone ="yes"?>');
-- DBMS_XMLGEN.RESTARTQUERY (v_ctx);
-- xt_data := dbms_xmlgen.getXMLType (v_ctx);
dbms_xslprocessor.clob2file( result, 'MYXML', ''||V_TABLE_NAME||'.xml',1);
dbms_xmlgen.closeContext (v_ctx);
v_FLAG := 1;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE(SQLERRM);
DBMS_XMLGEN.closeContext (v_ctx);
v_FLAG := 0;
END ;
Please Advices and help here
END Export_project6;

At least part of the problem is that you're specifying the file should be US7ASCII, which only allows the first 128 ASCII characters, not the extended values from 128-255. You're doing that in this line:
dbms_xslprocessor.clob2file( result, 'MYXML', ''||V_TABLE_NAME||'.xml',1);
You're passing 1 as the fourth parameter, csid. That value represents US7ASCII:
SQL> select nls_charset_name(1) from dual;
NLS_CHAR
--------
US7ASCII
Your XML is UTF-8, but specifying that with encoding="UTF-8" has no bearing on how the file is written. Any unrecognised characters are replaced with ?. So you might want to use the same setting for the file:
SQL> select nls_charset_id('UTF8') from dual;
NLS_CHARSET_ID('UTF8')
----------------------
871
So:
dbms_xslprocessor.clob2file( result, 'MYXML', ''||V_TABLE_NAME||'.xml',871);
or to be clearer:
dbms_xslprocessor.clob2file( result, 'MYXML', ''||V_TABLE_NAME||'.xml',
nls_charset_id('UTF8'));
But leaving it as the default might be OK - by not specifying csid at all, or by explicitly setting it to zero - depending on our database environment.
You mentioned you avoid ORA-31061 error if you "replace all ASCIICHAR (0-30) like ♂ : 11 ♀ : 12 ♫ : 14 ☼ : 15 ► : 16 ◄ : 17 ↕ : 18 ‼ : 19 ¶ : 20". Those symbols aren't what you'd expect from ASCII, so your character set or client or something seems to be interpreting them differently.
I get the error with all the ASCII control characters, 0 through to 31, except the printable ones: 9, 10 or 13. But that's what's expected, the other characters in that range are not valid in XML 1.0:
U+0009, U+000A, U+000D: these are the only C0 controls accepted in XML 1.0;
The same page shows that more, but still not all, control characters are allowed in XML 1.1, but as far as I'm aware Oracle only supports 1.0. If you do really have control characters in your data you'll need to strip them (retaining tabs, new lines and carriage returns); the rest would be meaningless in the final XML anyway, and perhaps of limited use in your existing data. I'm not sure if this is real data, or if you'd generated those values as a test.

Related

Transforming Strings

I need to accept the following strings and put them into a type collection and pass it to a procedure
String Cars = Dodge Charger||Ford Mustang||Chevy Camro||Ford GT
String Cost = 35,000||25,000||29,000|55,000
String CarDesc = Power House||Sweet Ride||Too Cool||Blow your door off
How do I transform the records so they will be like the following?
Cars:
Dodge Charger||35,000||Power House
Ford Mustang||25,00||Sweet Ride
Chevy Camro||29,000||Too Cool
Ford GT||55,000||Blow your door off
How do I parse them into an array?
The types:
create or replace TYPE "CAR_OBJ"
AS
OBJECT (CAR_NAME VARCHAR2 (50),
Price Number,
CarDesc VARCHAR2 (100));
/
create or replace TYPE "CAR_IN_ARR"
IS
TABLE OF CAR_OBJ;
/
procedure car_values (
p_in_upd_car car_in_arr,
p_out_upd_results out car_out_cur
)
as
I have tried all kinds of for loops and I just can't get it in the right order
Thank you soo much
The double delimiter || makes this hard, so I cheated by replacing them with ~. Probably there is a neater way to handle this with a single regex, or a more elaborate approach using substr and instr.
Also I've assumed the cost example should be 35,000||25,000||29,000||55,000.
The real code should probably confirm that all the strings contain the same number of delimiters. Also you might want to parse the cost value into a number.
declare
inCars long := 'Dodge Charger||Ford Mustang||Chevy Camro||Ford GT';
inCost long := '35,000||25,000||29,000||55,000';
inCarDesc long := 'Power House||Sweet Ride||Too Cool||Blow your door off';
type varchar2_tt is table of varchar2(50);
cars varchar2_tt := varchar2_tt();
costs varchar2_tt := varchar2_tt();
carDescs varchar2_tt := varchar2_tt();
begin
inCars := replace(inCars,'||','~');
inCost := replace(inCost,'||','~');
inCarDesc := replace(inCarDesc,'||','~');
cars.extend(regexp_count(inCars,'~') +1);
costs.extend(regexp_count(inCost,'~') +1);
carDescs.extend(regexp_count(inCarDesc,'~') +1);
for i in 1..cars.count loop
cars(i) := regexp_substr(inCars,'[^~]+', 1, i);
costs(i) := regexp_substr(inCost,'[^~]+', 1, i);
carDescs(i) := regexp_substr(inCarDesc,'[^~]+', 1, i);
dbms_output.put_line(cars(i) || '||' || costs(i) || '||' || carDescs(i));
end loop;
end;

UTL_FILE.WRITE_ERROR when calling utl_file.put in a loop

I have below code in my PL/SQL procedure, which I called in API_XXX.put(it calls utl_file.put) in a while loop. And the l_xmldoc is CLOB from a function of getReportXML, which returns the xml clob.
the code I write to write xml into a file is like:
l_offset := 1;
WHILE (l_offset <= l_length)
LOOP
l_char := dbms_lob.substr(l_xmldoc,1,l_offset);
IF (l_char = to_char(10)) ---I also tried if (l_char=chr(10)) but it did not work
THEN
API_XXXX.new_line(API_XXX.output, 1);
ELSE
API_XXXX.put(fnd_API_XXX.output, l_char);
END IF;
l_offset := l_offset + 1;
END LOOP;
Please note that the API_XXX is the existing package which I am not able to modify, and this api calls fflush in the end of put.
API_XXX.put's part is like below("WHICH" is the first param):
elsif WHICH = API_XXX.OUTPUT then
temp_file := OUT_FNAME;
utl_file.put(F_OUT, BUFF);
utl_file.fflush(F_OUT);
API_XXX.new_line is like(LINES is the number of lines to write):
elsif WHICH = API_XXX.OUTPUT then
temp_file := OUT_FNAME;
utl_file.new_line(F_OUT, LINES);
utl_file.fflush(F_OUT);
I notice a that the put/new_line procedure in my customer's side will sometimes raise UTL_FILE.WRITE_ERROR for unknown reason(maybe due to the l_length is too large(up to 167465)) in the while loop from my customer.
I read Oracle PL/SQL UTL_FILE.PUT buffering
. And I found that this is the same cause, my l_xmldoc is really large and when I loop it, I found that it is without a new line terminator so the buffer is up to 32767 even though I fflush every time.
So, how should I convert the l_xmldoc into a varchar with new line terminator.
PS: I confirmed that my customer is using Oralce 11g
Post the Oracle Version you are using! Or we can just guess around...
Your fflush will not work as you expect - From the documentation:
FFLUSH physically writes pending data to the file identified by the file handle. Normally, data being written to a file is buffered. The FFLUSH procedure forces the buffered data to be written to the file. The data must be terminated with a newline character.
tbone is abolutely right the line TO_CHAR(10) is wrong! Just try SELECT TO_CHAR(10) FROM DUAL; you will get 10 which you then compare to a single character. A single character will never be '10' since 10 has two characters!
Your problem is most likely a buffer-overflow with too large XML-Files, but keep in mind, also other problems on the target system can lead to write_errors, which should be handled.
Solutions
Quick&Dirty: Since you don't seem to care about performance anyways you can just close the file every X byte and reopen it with A for append. So just add to the loop:
IF MOD( l_offset, 32000 ) = 0
THEN
UTL_FILE.FCLOSE( f_out );
UTL_FILE.FOPEN( out_fpath, out_fname, f_out, 'a', 32767 );
END IF;
Use the right tool for the right job: UTL_FILE is not suited for handling complex data. The only usecase for UTL_FILE are small newline-separated lines of text. For everything else you should write RAW bytes! (Which will also allow you porper control over ENCODING, which is currently just mini-vanilly-lucky-guess)
Write a Java-Stored-Procedure with NIO-Filechannels - fast, safe, nice... But be careful, your program might run 10 times as fast!
Just a guess, but instead of "to_char(10)" you might try chr(10) to determine/write a newline. Not sure if this will solve your problem, but sometimes very long lines (without newlines) can cause issues.
For example:
declare
l_clob clob;
l_char char;
begin
l_clob := 'Line 1' || chr(10) || 'Line 2' || chr(10);
for i in 1 .. DBMS_LOB.GETLENGTH(l_clob)
loop
l_char := dbms_lob.substr(l_clob, 1, i);
if (l_char = chr(10)) then
--if (l_char = to_char(10)) then
dbms_output.put_line('Found a newline at position ' || i);
end if;
end loop;
end;
Notice the difference between chr(10) and to_char(10). Easy enough to test if this solves your problem anyway.

How to Convert Output of listagg() function into XMLTYPE

I have one stored Procedure P_FP_GET_PATTERN.I expect output in following format
PATTERN_ID |PATTERN_NAME | SHIFT
1 Pattern 1 A,B,C,B,A
2 Pattern 2 C,B,A,C
I'm getting this output in SYS_REFCURSOR ALL_RESULT_SET.I need to pass it in XML format.But the moment i put that output in ALL_RESULT_SET_XML which is my out parameter of stored procedure of XMLTYPE using ALL_RESULT_SET_XML:= XMLTYPE(ALL_RESULT_SET);
Im getting an error as
Error encountered: ORA-31061: XDB error: special char to escaped char conversion failed.
I`m getting this error due to column shown using LISTAGG() function.Anybody can please tell me how to handle this ?
My stored Procedure
create or replace
PROCEDURE P_FP_GET_PATTERN
(
ALL_RESULT_SET_XML OUT XMLTYPE,
P_MESSAGE_ALL OUT VARCHAR2
)
AS
V_ERROR VARCHAR2(2000);
ALL_RESULT_SET SYS_REFCURSOR;
BEGIN
OPEN ALL_RESULT_SET FOR
SELECT PM.PATTERN_ID ,PM.PATTERN_NAME,
LISTAGG(SM.SHIFT_NUMBER) WITHIN GROUP (ORDER BY PD.INSTANCE_DAY) "SHIFT"
FROM T_FP_PATTERN_MASTER PM,
T_FP_PATTERN_DETAILS PD,
T_FP_SHIFT_MASTER SM
WHERE SM.SHIFT_ID= PD.SHIFT_ID
AND PM.PATTERN_ID = PD.PATTERN_ID
GROUP BY PM.PATTERN_NAME,PM.PATTERN_ID;
--Adding output in XML output parameter
ALL_RESULT_SET_XML:= XMLTYPE(ALL_RESULT_SET);
EXCEPTION
WHEN OTHERS
THEN
V_ERROR := SUBSTR(SQLERRM,1,1000);
P_MESSAGE_ALL := 'Error encountered: '||V_ERROR ;
END;

PL/SQL error LPX-00249 while parsing XML file from web

I'm writing a programm in PL/SQL for downloading XML files from internet, parsing them and storing values in Oracle database. I have one large XML file, which includes links for a huge amount of smaller XMLs. There are about 6253 files to be parsed. I have function, which downloads XML file into CLOB and saves data into XmlType. This value is returned into programm and is further processed. This is the function:
create or replace function get_xml_by_url
( v_url VARCHAR2
)
RETURN XMLType
AS
req SYS.UTL_HTTP.REQ;
resp SYS.UTL_HTTP.RESP;
xmlClob CLOB;
x XmlType;
l_offset number := 1;
value VARCHAR2(3999); -- URL to post to
BEGIN
BEGIN
UTL_HTTP.SET_PROXY('http://10.1.250.233:8080');
req := UTL_HTTP.BEGIN_REQUEST (url=> v_url, method => 'GET');
UTL_HTTP.SET_HEADER(req, 'User-Agent', 'Mozilla/4.0');
UTL_HTTP.SET_HEADER
( r => req
, name => 'Content-Type'
, value => 'text/xml;charset=UTF-8'
);
resp := UTL_HTTP.GET_RESPONSE(req);
DBMS_LOB.CREATETEMPORARY(xmlClob, true);
-- Loading first line
UTL_HTTP.READ_LINE(resp,value,false);
DBMS_LOB.WRITE(xmlClob,length(value),l_offset,value);
l_offset := l_offset + length(value);
-- Loading and adjusting second line
UTL_HTTP.READ_LINE(resp,value,true);
value := rtrim(value,'xmlns="http://seznam.gov.cz/ovm/datafile/seznamovm/v1">')||'>';
DBMS_LOB.WRITE(xmlClob, length(value), l_offset,value);
l_offset := l_offset + length(value);
-- Filling CLOB
LOOP
UTL_HTTP.READ_LINE(resp,value,false);
DBMS_LOB.WRITE(xmlClob,length(value),l_offset,value);
l_offset := l_offset + length(value);
END LOOP;
EXCEPTION
when UTL_HTTP.END_OF_BODY
then
UTL_HTTP.END_RESPONSE(resp);
when others
then
utl_http.end_response(resp);
END;
x := XMLType.createXML(xmlClob);
DBMS_LOB.FREETEMPORARY(xmlClob);
RETURN x;
END;
I was calling this function in a loop for all 6.253 XMl files and every time I've got an error, but every time in different file, and when I ran the script again for only the one XML file which raised an error, it ran fine. I think that the problem is something about memory, but I don't know where a nd why it occurs.
I'm getting following error:
Error report:
ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00249: invalid external ID declaration
Error at line 1
ORA-06512: in "SYS.XMLTYPE", line 5
ORA-06512: in "GET_XML_BY_URL", line 47
ORA-06512: in line 29
31011. 00000 - "XML parsing failed"
*Cause: XML parser returned an error while trying to parse the document.
*Action: Check if the document to be parsed is valid.
GET_XML_BY_URL is a name given to described function. Does anybody have any experience wuth this kind of problem?
Best regards, Michal

More than 4000 chars gives string literal too long error on oracle

I am currently using oci8 driver on Codeigniter.
While updating a field that will have more than 4000 chars, I was given a error :
ORA-01704: string literal too long
So, going through few blogs, I got this:
declare
vClobVal varchar2(32767) := 'long text'
begin
update FMS_K_OFFICEWISE_LETTER set FKOL_LETTER_BODY=vClobVal
where FKOL_OFFICEWISE_LETTER_ID=240;
end;
This worked for me when fired at Toad.
Now, I created a stored procedure and compiled as :
CREATE OR REPLACE PROCEDURE FMIS3.UPDATE_LETTER_BODY ( body_text IN FMS_K_OFFICEWISE_LETTER.FKOL_LETTER_BODY%type,condition_id in FMS_K_OFFICEWISE_LETTER.FKOL_OFFICEWISE_LETTER_ID%type)IS
begin
update FMS_K_OFFICEWISE_LETTER set FKOL_LETTER_BODY=body_text
end;
and this does not work for more than 4000 chars again. Can't I define the size of varchar2 as it gave me error. Any suggestions ?
Even tried using PDO by binding parameters, works only when the string is of size less than 4000 chars :(
$conn = new PDO("oci:dbname=".$this->db->hostname,$this->db->username,$this->db->password);
$params = array(
':body_text' => "Long String"
);
$sth = $conn->prepare("update FMS_K_OFFICEWISE_LETTER set FKOL_LETTER_BODY = :body_text
where FKOL_OFFICEWISE_LETTER_ID=241");
$sth->execute($params) or die('error occured');
Check this out :
declare
vClobVal varchar2(32767) := 'long text'
begin
update FMS_K_OFFICEWISE_LETTER set FKOL_LETTER_BODY=vClobVal
where FKOL_OFFICEWISE_LETTER_ID=240;
end;
Are you sure this is not supported ?
In PL/SQL, a VARCHAR2 can have 32767 bytes, but in SQL only 4000 bytes. Therefore the BEGIN ... END; block worked, as it is PL/SQL, and the procedure didn't, as it is SQL.
varchar2 has a limit of 4000 chars. Use CLOB instead.