Oracle query parses XML from blob with header - sql

I have a table in an oracle 10 database, containing a field with a BLOB (not CLOB). This BLOB contains a fixed-size header of about 300 bytes, followed by an XML document. The blob might have a size up to about 1 megabyte. I'd like to create an SQL query which uses XQUERY on this document to extract information from the XML.
So the structure is, roughly:
create table tbl(id integer, data blob);
insert into tbl(id,data) value(1,'HEADER <?xml version="1.0"><data>
<row key="k1" value="v11"/>
<row key="k2" value="v12"/></data>');
insert into tbl(id,data) value(2,'HEADER <?xml version="1.0"><data>
<row key="k1" value="v21"/>
<row key="k1" value="v21B"/>
<row key="k2" value="v22"/></data>');
I'd like a query on this table which, when given key k1, returns values v11,v21 and v21B
I know this data organisation is suboptimal, but it can't be changed.

OK, so first you have to get the XML part. Assuming that the header and XML are both character data, and the header is a fixed length, I'd probably use a combination of
dbms_lob.converttoclob to turn the blob into a clob
dbms_lob.substr to get a clob that has the XML portion
xmltype.createXML (clob) to assign the XML to your xmltype
xmltype.extract to apply your xpath expression
If the header isn't character data, you can still use dbms_lob.substr but it will return a RAW which you'll need to convert. If the header isn't fixed length, you can search for the location of the
So, based on the comments, use something like this to build a clob that has what you want, where offset is the number of bytes to the start of your actual XML. Modify to pass in your blob or clob. Then apply your xpath at the end instead of my dbms_output.
declare
v_buffer varchar2(32767);
v_offset integer := 5;
v_xml xmltype;
v_clob clob;
v_input clob := 'xxxx<?xml version="1.0" encoding="UTF-8"?><test>This is a test</test>';
i integer := 0;
begin
dbms_lob.createtemporary (v_clob,true);
v_buffer := dbms_lob.substr(v_input,32767,v_offset);
while length (v_buffer) > 0 loop
v_clob := v_clob || v_buffer;
i := i + 1;
v_buffer := dbms_lob.substr(v_input,32767, v_offset + i * 32767);
end loop;
dbms_output.put_line ('x'||v_clob||'x');
v_xml := xmltype.createXML (v_clob);
dbms_lob.freetemporary (v_clob);
dbms_output.put_line (v_xml.getclobval);
end;

Related

String literal too long while inserting a XML into XMLTYPE Column in Oracle

i have a text XML with more than 100K Characters , I am trying to write a PLSQL Block to insert this XML into an XMLTYPE Oracle Table. Below is my PLSQL Block and it wont allow to insert the xml and getting error saying string literal is too long because Oracle's SQL can only handle up to 4000 characters . I cannot change the datatype in Oracle. How i can insert this XML ?
Table Column is REQUEST_XML XMLTYPE
I was trying like this but it seems that also not working.
INSERT INTO XYZ.ABC(ID,REQUEST_XML) values(123,yourXmlStr);
PLSQL is below,
DECLARE
yourXmlStr xmltype := xmltype('<DRIVEResponse TimeZone="EDT">
<Condition1 ActionStep="ABCABC" /> // This can be more than 100K Characters
</DRIVEResponse>');
BEGIN
INSERT INTO XYZ.ABC(ID,REQUEST_XML) values(123,XMLTYPE.CREATEXML(yourXmlStr));
COMMIT;
END;
You're creating an xmltype twice, which is unnecessary. Your source data is a string literal (anything between two single quotes), which in PL/SQL has a maximum length of 4000 characters (or 32767, see MAX_STRING_SIZE). Fortunately, the xmltype constructor can also operate on a clob. You don't say where the source data is coming from, so I'm not sure of the best way to construct that:
declare
l_clob clob;
l_xml xmltype;
begin
l_clob := '<DRIVEResponse TimeZone="EDT">';
l_clob := l_clob || '<Condition1 ActionStep="ABCABC" />';
... repeat in chunks of less than 32768 characters ...
l_clob := l_clob || '</DRIVEResponse>';
l_xml := xmltype.createxml( l_clob );
INSERT INTO XYZ.ABC(ID,REQUEST_XML) values(123, l_xml);
end;

Oracle fetch XML from BLOB

I have tried lots of methods and still can't get my full XML document from DB. What I want to achieve is displaying the XML in Oracle Apex (Display only element) but I can't manage to get the full XML out from my blob.
SELECT
utl_raw.cast_to_varchar2(dbms_lob.substr(<blob_column>, 2000, 1))
FROM
<my_table>
WHERE <some_id> = 123
Also tried to fetch it with mimetype but had no luck. Thank you.
First, you shouldn't convert it to varchar on the server side since Oracle SQL has a 4K limitation on a varchar string size. You can utilize a PL\SQL block for retrieving your data but in this case you will have a limitation in 32K. There is a special way how to get round this issue: http://mayo-tech-ans.blogspot.com/2013/06/displaying-large-clobs-in-oracle-apex.html .
Hoping, I understood the question correctly.
I think, below query will help you.
First, convert the blob column to XMLTYPE, this will also help to check, if XML is valid or not.
http://www.dba-oracle.com/t_convert_blob_to_xml_type.htm
Then, use EXTRACTVALUE to fetch the data from the XML.
select EXTRACTVALUE(xml_data,'/note/to')
from
(
select XMLTYPE.createXML('<note>
<to>Tove</to>
<from>Jani</from>
<heading>Reminder</heading>
<body>Test body</body>
</note>') xml_data from dual
)
;
In order to get full BLOB content:
Converted BLOB into CLOB
create or replace FUNCTION blob_to_clob (blob_in IN BLOB)
RETURN CLOB
AS
v_clob CLOB;
v_varchar VARCHAR2(32767);
v_start PLS_INTEGER := 1;
v_buffer PLS_INTEGER := 32767;
BEGIN
DBMS_LOB.CREATETEMPORARY(v_clob, TRUE);
FOR i IN 1..CEIL(DBMS_LOB.GETLENGTH(blob_in) / v_buffer)
LOOP
v_varchar := UTL_RAW.CAST_TO_VARCHAR2(DBMS_LOB.SUBSTR(blob_in, v_buffer, v_start));
DBMS_LOB.WRITEAPPEND(v_clob, LENGTH(v_varchar), v_varchar);
v_start := v_start + v_buffer;
END LOOP;
RETURN v_clob;
END blob_to_clob;
In Oracle Apex created Display Only (element) called ex: P5_XML
PLSQL code
declare
v_clob clob;
begin
SELECT blob_to_clob(<blob_column>) INTO v_clob FROM <table> WHERE <some_column> = <something>;
:P5_XML := v_clob;
end;

How to update CLOB column from a physical file?

I have a CLOB column in a table which holds very large amount of XML data. I need to update this column's value for one row of table. How can I do this?
I have tried googling, but this visibly simple stuff is not available in a simple language anywhere. Can anybody please suggest?
If I use the normal update query syntax and paste the huge xml content inside the single quotes (the single quote in xml content replaced with 2 single quotes), then the sql developer just disables the execute query button.
update tableName t
set t.clobField = 'How to specify physical file data'
where t.anotherField='value';
You need to create a function that reads the data from the file into a variable of the CLOB data type and returns it as the result.
Try this working example:
create table tclob (id number, filename varchar2 (64), doc clob)
/
insert into tclob values (1, 'test.xml', empty_clob ());
commit;
create or replace function clobLoader (filename varchar2) return clob is
bf bfile := bfilename ('TEMPFILES', filename);
cl clob;
begin
if dbms_lob.fileexists (bf) = 1 then
dbms_lob.createtemporary (cl, true);
dbms_lob.fileopen (bf, dbms_lob.file_readonly);
dbms_lob.loadfromfile (cl, bf, dbms_lob.getlength (bf));
dbms_lob.fileclose (bf);
else cl := empty_clob ();
end if;
return cl;
end;
/
Usage:
update tclob t
set t.doc = clobLoader (t.filename)
where t.id = 1;
1 row updated.
Search the internet for "load clob from file" and you will find a couple of examples. Such as this (http://www.anujparashar.com/blog/loading-text-file-into-clob-field-in-oracle):
DECLARE
v_bfile BFILE;
v_clob CLOB;
BEGIN
v_bfile := BFILENAME (p_file_directory, p_file_name);
IF DBMS_LOB.FILEEXISTS (v_bfile) = 1 THEN
DBMS_LOB.OPEN (v_bfile);
DBMS_LOB.CREATETEMPORARY (v_clob, TRUE, DBMS_LOB.SESSION);
DBMS_LOB.LOADFROMFILE (v_clob, v_bfile, DBMS_LOB.GETLENGTH (v_bfile));
DBMS_LOB.CLOSE (v_bfile);
INSERT INTO tbl_clob (clob_col)
VALUES (v_clob);
END IF;
COMMIT;
END;
/

how to sort the contents of CLOB field

I have a table and some fields are CLOB type and the content in the CLOB was delimited by some separator such as '|' and usually the content in the filed looks like this : name2|name1|name3..., actually the length of the content is more than 40000 characters, so is there any way to sort the content by asc? I want to look the content like this: name1|name2|name3...
can any body help me?
If it's even remotely possible, I'd strongly suggest you change your data model - add a details table for the names. This will solve you a lot of pain in the future.
Anyhow, if you absolutely need to store a pipe-separated list of names in your CLOB field, I'd suggest this approach:
break the CLOB into separate rows (using a pipelined function)
sort the rows
aggregate the rows into a new CLOB
A (somewhat naive and untested) implementation of this approach:
create type stringtabletype as table of varchar2(4000);
create or replace function split_CLOB(p_Value in CLOB,
p_Separator in varchar2 default '|')
return stringtabletype
pipelined as
l_Offset number default 1;
l_Str varchar2(4000);
idx number;
begin
idx := dbms_lob.instr(lob_loc => p_Value,
pattern => p_Separator,
offset => l_Offset);
dbms_output.put_line(idx);
while (idx > 0)
loop
l_Str := dbms_lob.substr(p_Value,
idx - l_Offset,
l_Offset);
pipe row(l_Str);
l_Offset := idx+1;
idx := dbms_lob.instr(p_Value,
p_Separator,
l_Offset);
dbms_output.put_line(idx);
end loop;
-- pipe remainder of string
l_Str := dbms_lob.substr(p_Value,
dbms_lob.getlength(p_Value) - l_Offset + 1,
l_Offset);
pipe row(l_str);
return;
end;
create or replace function sort_stringtabletype(p_Values in stringtabletype)
return stringtabletype as
l_Result stringtabletype;
begin
select column_value bulk collect
into l_Result
from table(p_Values)
order by column_value;
return l_Result;
end;
create or replace function stringtabletype_to_CLOB(p_Values in stringtabletype,
p_Separator in varchar2 default '|')
return CLOB as
l_Result CLOB;
begin
dbms_lob.createtemporary(l_Result, false);
for i in 1 .. p_Values.count - 1
loop
dbms_lob.writeappend(l_Result,
length(p_Values(i)),
p_Values(i));
dbms_lob.writeappend(l_Result,
length(p_Separator),
p_Separator);
end loop;
dbms_lob.writeappend(l_Result,
length(p_Values(p_Values.count)),
p_Values(p_Values.count));
return l_Result;
end;
Example usage:
select stringtabletype_to_CLOB (
sort_stringtabletype(
split_CLOB('def|abc|ghic', '|')
)
) from dual
You could then use an UPDATE statement like
update my_table
set clob_field = stringtabletype_to_CLOB (
sort_stringtabletype(
split_CLOB(my_table, '|')
)

Converting small-ish Oracle long raw values to other types

I have an Oracle table that contains a field of LONG RAW type that contains ASCII character data. How can I write a query or view that will convert this to a more easily consumed character string? These are always going to be single-byte characters, FWIW.
Maybe
select ...., to_lob(long_raw) from old_table
(http://www.psoug.org/reference/convert_func.html)
or
UTL_RAW.CAST_TO_VARCHAR2(b)
(http://www.dbasupport.com/forums/showthread.php?t=5342).
I found this quote:
In Oracle9i, you can even:
alter table old_table modify ( c clob
);
to convert it.
See here: http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:1037232794454
Edit
The max length of a varchar2 column is 4000. Is that too short?
I have found this works well on CLOB data types. I would believe the same would hold true for LOB types.
create or replace function lob2char(clob_col clob) return varchar2 IS
buffer varchar2(4000);
amt BINARY_INTEGER := 4000;
pos INTEGER := 1;
l clob;
bfils bfile;
l_var varchar2(4000):='';
begin
LOOP
if dbms_lob.getlength(clob_col)<=4000 THEN
dbms_lob.read (clob_col, amt, pos, buffer);
l_var := l_var||buffer;
pos:=pos+amt;
ELSE
l_var:= 'Cannot convert. Exceeded varchar2 limit';
exit;
END IF;
END LOOP;
return l_var;
EXCEPTION
WHEN NO_DATA_FOUND THEN
return l_var;
END;
INSERT INTO NEWTABLE (NEWCOLUMN) SELECT RTRIM(lob2char(OLDCOLUMN)) FROM OLDTABLE;