Extract data from a XML and load it into a table - sql

How would I fetch the data from a XML (loaded into a Oracle table) and create a table to load that extracted data.
I'm trying to create a table, with columns as follows, the below mentioned columns are tags in the xml
ID, Provider_Name, Index, Provider_ID_description, Provider_ID
1, Provider_P107, 1, GNRCN, GNRCN
1, Provider_P107, 2, INDF1, INDF1
2, Provider_EGUT, 1, EGUT, EGUT
The XML is:
<?xml version="1.0" encoding="us-ascii"?>
<AuxiliaryType auxiliaryTypeId="1617309" base="Value list" hasImplementation="false" isShared="true" masteredIn="false" name="ID list" type="String">
<AuxiliaryObject id="1" name="Provider_P107">
<Row>
<Index>1</Index>
<Provider_ID_description>GNRCN</Provider_ID_description>
<Provider_ID>GNRCN</Provider_ID>
</Row>
<Row>
<Index>2</Index>
<Provider_ID_description>INDF1</Provider_ID_description>
<Provider_ID>INDF1</Provider_ID>
</Row>
</AuxiliaryObject>
<AuxiliaryObject id="2" name="Provider_EGUT">
<Row>
<Index>1</Index>
<Provider_ID_description>EGUT</Provider_ID_description>
<Provider_ID>EGUT</Provider_ID>
</Row>
</AuxiliaryObject>
</AuxiliaryType>
I've tried using the following query:
SELECT w.ID,
p.Name,
s.Index,
q.P_Desc,
q.P_ID
from traptabclob t
left join xmltable ('/AuxiliaryType/AuxiliaryObject/'
passing t.testclob
columns ID XmlType path '/AuxiliaryType/AuxiliaryObject/#id',
columns Name XmlType path '/AuxiliaryType/AuxiliaryObject/#name',
columns Index XmlType path 'Row/Index',
columns P_Desc varchar2(201) path 'Row/Provider_ID_description',
columns P_ID varchar2(21) path 'Row/Provider_ID',
) q
on (1=1)
left join xmltable('/Index'
passing q.Index
columns Index number path '.') s
on (1=1)
left join xmltable('/AuxiliaryType/AuxiliaryObject/'
passing q.ID
columns ID number path '.') w
on (1=1)
left join xmltable('/AuxiliaryType/AuxiliaryObject/'
passing q.Name
columns Name varchar2(21) path '.') p
on (1=1);
I tried this http://www.sqlfiddle.com/#!4/1a672/7/0. Here is the new where I am able to fetch more data http://www.sqlfiddle.com/#!4/1a672/8/0, but I'm still struggling. This does not work when I have more than one index or row. How to do it If I have the count of AuxiliaryObject id ?

Related

How to extract only the subtag in this xml element

with this query I can select the content of tag
select xmltype(
'<?xml version="1.0"?>
<ROWSET>
<ROW>
<PERSON_NUMBER>1000142</PERSON_NUMBER>
<LOAN_1>25000</LOAN_1>
</ROW>
<ROW>
<PERSON_NUMBER>1000142</PERSON_NUMBER>
<LOAN_1>25000</LOAN_1>
</ROW>
</ROWSET>'
).extract( '/ROWSET/ROW' ) .getstringval() p#
from dual;
<PERSON_NUMBER>1000142</PERSON_NUMBER><LOAN_1>25000</LOAN_1><PERSON_NUMBER>1000142</PERSON_NUMBER><LOAN_1>25000</LOAN_1>
but what I want is to have a list of the sub-tag.
the expected result for this example would be:
PERSON_NUMBER, LOAN_1
code
Similar to an answer to your previous question, you can get the name() XPATH value:
SELECT DISTINCT name
from XMLTABLE(
'/ROWSET/ROW/*'
PASSING xmltype(
'<?xml version="1.0"?>
<ROWSET>
<ROW>
<PERSON_NUMBER>1000142</PERSON_NUMBER>
<LOAN_1>25000</LOAN_1>
<LOAN_2>26000</LOAN_2>
</ROW>
<ROW>
<PERSON_NUMBER>1000142</PERSON_NUMBER>
<LOAN_1>25000</LOAN_1>
<LOAN_2>26000</LOAN_2>
</ROW>
</ROWSET>'
)
COLUMNS
name VARCHAR2(200) PATH './name()'
);
Outputs:
NAME
PERSON_NUMBER
LOAN_1
LOAN_2
db<>fiddle here

oracle XML to Database table

I have XML like this . I am unable to parse it into a table
<REQMST>
<ROW>
<REQ_ID>668 </REQ_ID>
<RequestDetails>
<REQ_DTL_ID>845</REQ_DTL_ID>
<INTERFACE_REFNUM>1</INTERFACE_REFNUM>
</RequestDetails>
<RequestDetails>
<REQ_DTL_ID>846</REQ_DTL_ID>
<INTERFACE_REFNUM>2</INTERFACE_REFNUM>
</RequestDetails>
</ROW>
</REQMST>
I am trying to parse it into table using
select *
FROM XMLTABLE('/REQMST/ROW'
PASSING
xmltype('
<REQMST>
<ROW>
<REQ_ID>668 </REQ_ID>
<RequestDetails>
<REQ_DTL_ID>845</REQ_DTL_ID>
<INTERFACE_REFNUM>1</INTERFACE_REFNUM>
</RequestDetails>
<RequestDetails>
<REQ_DTL_ID>846</REQ_DTL_ID>
<INTERFACE_REFNUM>2</INTERFACE_REFNUM>
</RequestDetails>
</ROW>
</REQMST>')
COLUMNS
--describe columns and path to them:
REQ_ID varchar2(20) PATH './REQ_ID',
REQ_DTL_ID varchar2(20) PATH './RequestDetails/REQ_DTL_ID',
INTERFACE_REFNUM varchar2(20) PATH './RequestDetails/INTERFACE_REFNUM'
) xmlt
;
I am getting error ORA-19279: XPTY0004 - XQuery dynamic type mismatch: expected singleton sequence - got multi-item sequence
it works if there is only one RequestDetails
You need to go one more level down to get the details from RequestDetails tag using the same technique as parent.
Used column name xml to get the xml as xmltype column and then use xmltable on it to get the further details:
Try this:
select xmlt.req_id, x.REQ_DTL_ID, x.INTERFACE_REFNUM
FROM XMLTABLE('/REQMST/ROW'
PASSING
xmltype('
<REQMST>
<ROW>
<REQ_ID>668 </REQ_ID>
<RequestDetails>
<REQ_DTL_ID>845</REQ_DTL_ID>
<INTERFACE_REFNUM>1</INTERFACE_REFNUM>
</RequestDetails>
<RequestDetails>
<REQ_DTL_ID>846</REQ_DTL_ID>
<INTERFACE_REFNUM>2</INTERFACE_REFNUM>
</RequestDetails>
</ROW>
</REQMST>')
COLUMNS
--describe columns and path to them:
REQ_ID varchar2(20) PATH 'REQ_ID',
xml xmltype PATH 'RequestDetails'
) xmlt, xmltable('/RequestDetails'
passing xmlt.xml
columns
REQ_DTL_ID varchar2(20) PATH 'REQ_DTL_ID',
INTERFACE_REFNUM varchar2(20) PATH 'INTERFACE_REFNUM'
) x;
Produces:
REQ_ID REQ_DTL_ID INTERFACE_REFNUM
668 845 1
668 846 2

convert XML to table in oracle

I have a XML in the below format, store in column of XMLType in oracle database
<a>
<c>1</c>
<c>2</c>
</a>
I need to convert this in table format as
c
1
2
Any idea how to do this using SQL?
with data as
(select '<a><c>1</c><c>2</c></a>' xmlval
from dual)
( select c
from data d,
xmltable('/a/*' passing xmltype(d.xmlval)
columns
c varchar2(254) path '/c'
))
There is answer in this url.
For instance user daggett did this:
select *
FROM XMLTABLE('/person/row'
PASSING
xmltype('
<person>
<row>
<name>Tom</name>
<Address>
<State>California</State>
<City>Los angeles</City>
</Address>
</row>
<row>
<name>Jim</name>
<Address>
<State>California</State>
<City>Los angeles</City>
</Address>
</row>
</person>
')
COLUMNS
--describe columns and path to them:
name varchar2(20) PATH './name',
state varchar2(20) PATH './Address/State',
city varchar2(20) PATH './Address/City'
) xmlt
;
Thanks, it's work.
pISLEMLER CLOB;
select *
FROM XMLTABLE('/mydata/UCRET'
PASSING
xmltype('<mydata>'||pISLEMLER||'</mydata>
')
COLUMNS
--describe columns and path to them:
F_M_IS_ID varchar2(20) PATH './FMISID',
ISKONTO varchar2(20) PATH 'ISK'
) xmlt;

Extract xml data using oracle query

Oracle version 11g
HI ,
When trying to read the XML in a LOOP the query is giving an error but if, I change this to a non existing path then the query is running fine . However the #Name is not fetching is the expected . What should I correct here to make it give the desired output given below:
sqlfiddle link
Non exsisting path described above, removed the E from the name : for $i in AuxiliaryObject/Row return <C>{$i}<R>{AuxiliaryObject/#NAM}
code :
SELECT *
FROM XMLTABLE (
'<C> {for $i in AuxiliaryObject/Row return <C>{$i}<R>{AuxiliaryObject/#NAM}</R></C>}</C>/C'
PASSING xmltype(
'<AuxiliaryType>
<AuxiliaryObject id="1" NAME="Provider_P107">
<Row>
<Index_id>1</Index_id>
<Provider_ID_description>GNRCN</Provider_ID_description>
<Provider_ID>GNRCN</Provider_ID>
</Row>
<Row>
<Index_id>2</Index_id>
<Provider_ID_description>EGUT12</Provider_ID_description>
<Provider_ID>EGUT12 </Provider_ID>
</Row>
</AuxiliaryObject>
<AuxiliaryObject id="2" NAME="Provider_P108">
<Row>
<Index_id>1</Index_id>
<Provider_ID_description>GNRCN</Provider_ID_description>
<Provider_ID>GNRCN</Provider_ID>
</Row>
<Row>
<Index_id>2</Index_id>
<Provider_ID_description>EGUT</Provider_ID_description>
<Provider_ID>EGUT </Provider_ID>
</Row>
</AuxiliaryObject>
</AuxiliaryType>'
).EXTRACT ('AuxiliaryType/*')
COLUMNS
Name varchar (30) Path 'R/#NAME',
Index_Id VARCHAR2 (10) PATH 'Row/Index_id',
Provider_id_description VARCHAR2 (30) PATH 'Row/Provider_ID_description',
provider_id VARCHAR2 (30) PATH 'Row/Provider_ID')
Output : Desired :
ID, Provider_Name, Index, Provider_ID_description, Provider_ID
1 Provider_P107 1 GNRCN GNRCN
1 Provider_P107 2 INDF1 INDF1
2 Provider_P108 2 EGUT12 EGUT12
2 Provider_P108 1 EGUT EGUT
Output coming is in the sqlfiddle link.
The above Queation is a link to this :
Extract data from a XML and load it into a table
When I run the query on Toad : Output is :
NAME INDEX_ID PROVIDER_ID_DESCRIPTION PROVIDER_ID
Provider_P107Provider_P108 1 GNRCN GNRCN
Provider_P107Provider_P108 2 EGUT12 EGUT12
Provider_P107Provider_P108 1 GNRCN GNRCN
Provider_P107Provider_P108 2 EGUT EGUT
I would extract the data in stages:
SELECT xobjects.id, xobjects.name, xrows.index_id,
xrows.provider_id_description, xrows.provider_id
FROM XMLTABLE(
'/AuxiliaryType/AuxiliaryObject'
PASSING xmltype(
'<AuxiliaryType>
<AuxiliaryObject id="1" NAME="Provider_P107">
<Row>
<Index_id>1</Index_id>
<Provider_ID_description>GNRCN</Provider_ID_description>
<Provider_ID>GNRCN</Provider_ID>
</Row>
<Row>
<Index_id>2</Index_id>
<Provider_ID_description>EGUT12</Provider_ID_description>
<Provider_ID>EGUT12 </Provider_ID>
</Row>
</AuxiliaryObject>
<AuxiliaryObject id="2" NAME="Provider_P108">
<Row>
<Index_id>1</Index_id>
<Provider_ID_description>GNRCN</Provider_ID_description>
<Provider_ID>GNRCN</Provider_ID>
</Row>
<Row>
<Index_id>2</Index_id>
<Provider_ID_description>EGUT</Provider_ID_description>
<Provider_ID>EGUT </Provider_ID>
</Row>
</AuxiliaryObject>
</AuxiliaryType>'
)
COLUMNS
name VARCHAR2(30) PATH '#NAME',
id VARCHAR2(10) PATH '#id',
xrows XMLTYPE PATH 'Row') xobjects,
XMLTABLE(
'/Row'
PASSING xobjects.xrows
COLUMNS
index_id VARCHAR2(10) PATH 'Index_id',
provider_id_description VARCHAR2(30) PATH 'Provider_ID_description',
provider_id VARCHAR2(30) PATH 'Provider_ID') xrows;
The XMLTable xobjects contains each of the AuxiliaryObject instances within the AuxiliaryType, from your original XML text. It has the attributes name and id, plus a sub-XMLType containing the nested rows. The second XMLTable, xrows, expands that so the elements can be extracted. The joins and passing of the XML types creates the hierarchy that gives the output you want:
ID NAME INDEX_ID PROVIDER_ID_DESCRIPTION PROVIDER_ID
---------- ------------------------------ ---------- ------------------------------ ------------------------------
1 Provider_P107 1 GNRCN GNRCN
1 Provider_P107 2 EGUT12 EGUT12
2 Provider_P108 1 GNRCN GNRCN
2 Provider_P108 2 EGUT EGUT
This works in SQL Developer against an 11.2.0.3 database, and in SQL Fiddle.
An earlier CTE-based version of this answer also worked in SQL Developer but SQL Fiddle got an ORA-600 error; that along with the issue you had in the question suggests maybe SQL Fiddle is on an unpatched, or at least differently patched, version of 11gR2 which has bugs in the XML handling.

How to extract element-path from XMLType Node?

I would like to have a select statement on an XML document and one column should return me the path of each node.
For example, given the data
SELECT *
FROM TABLE(XMLSequence(
XMLTYPE('<?xml version="1.0"?>
<users><user><name>user1</name></user>
<user><name>user2</name></user>
<group>
<user><name>user3</name></user>
</group>
<user><name>user4</name></user>
</users>').extract('/*//*[text()]'))) t;
Which results in
column_value
--------
<user><name>user1</name></user>
<user><name>user2</name></user>
<user><name>user3</name></user>
<user><name>user4</name></user>
I'd like to have a result like this:
path value
------------------------ --------------
/users/user/name user1
/users/user/name user2
/users/group/user/name user3
/users/user/name user4
I can not see how to get to this. I figure there are two thing that have to work together properly:
Can I extract the path from an XMLType with a single operation or method, or do I have to do this with string-magic?
What is the correct XPath expression so that I do get the whole element path (if thats possible), eg. <users><group><user><name>user3</name></user></group></user> insead of <user><name>user3</name></user>?
Maybe I am not understanding XMLType fully, yet. It could be I need a different approach, but I can not see it.
Sidenotes:
In the final version the XML document will be coming from CLOBs of a table, not a static document.
The path column can of course also use dots or whatever and the initial slash is not the issue, any representation would do.
Also I would not mind if every inner node also gets a result row (possibly with null as value), not only the ones with text() in it (which is what I am really interested in).
In the end I will need the tail element of path separate (always "name" in the example here, but this will vary later), i.e. ('/users/groups/user', 'name', 'user3'), I can deal with that separately.
You can achieve that with help of XMLTable function from Oracle XML DB XQuery function set:
select * from
XMLTable(
'
declare function local:path-to-node( $nodes as node()* ) as xs:string* {
$nodes/string-join(ancestor-or-self::*/name(.), ''/'')
};
for $i in $rdoc//name
return <ret><name_path>{local:path-to-node($i)}</name_path>{$i}</ret>
'
passing
XMLParse(content '
<users><user><name>user1</name></user>
<user><name>user2</name></user>
<group>
<user><name>user3</name></user>
</group>
<user><name>user4</name></user>
</users>'
)
as "rdoc"
columns
name_path varchar2(4000) path '//ret/name_path',
name_value varchar2(4000) path '//ret/name'
)
For me XQuery looks at least more intuitive for XML data manipulation than XSLT.
You can find useful set of XQuery functions here.
Update 1
I suppose that you need totally plain dataset with full data at last stage.
This target can be reached by complicated way, constructed step-by-step below, but this variant is very resource-angry. I propose to review final target (selecting some specific records, count number of elements etc.) and after that simplify this solution or totally change it.
Update 2
All steps deleted from this Update except last because #A.B.Cade proposed more elegant solution in comments.
This solution provided in Update 3 section below.
Step 1 - Constructing dataset of id's with corresponding query results
Step 2 - Aggregating to single XML row
Step 3 - Finally get full plain dataset by querying constracted XML with XMLTable
with xmlsource as (
-- only for purpose to write long string only once
select '
<users><user><name>user1</name></user>
<user><name>user2</name></user>
<group>
<user><name>user3</name></user>
</group>
<user><name>user4</name></user>
</users>' xml_string
from dual
),
xml_table as (
-- model of xmltable
select 10 id, xml_string xml_data from xmlsource union all
select 20 id, xml_string xml_data from xmlsource union all
select 30 id, xml_string xml_data from xmlsource
)
select *
from
XMLTable(
'
for $entry_user in $full_doc/full_list/list_entry/name_info
return <tuple>
<id>{data($entry_user/../#id_value)}</id>
<path>{$entry_user/name_path/text()}</path>
<name>{$entry_user/name_value/text()}</name>
</tuple>
'
passing (
select
XMLElement("full_list",
XMLAgg(
XMLElement("list_entry",
XMLAttributes(id as "id_value"),
XMLQuery(
'
declare function local:path-to-node( $nodes as node()* ) as xs:string* {
$nodes/string-join(ancestor-or-self::*/name(.), ''/'')
};(: function to construct path :)
for $i in $rdoc//name return <name_info><name_path>{local:path-to-node($i)}</name_path><name_value>{$i/text()}</name_value></name_info>
'
passing by value XMLParse(content xml_data) as "rdoc"
returning content
)
)
)
)
from xml_table
)
as "full_doc"
columns
id_val varchar2(4000) path '//tuple/id',
path_val varchar2(4000) path '//tuple/path',
name_val varchar2(4000) path '//tuple/name'
)
Update 3
As mentioned by #A.B.Cade in his comment, there are really simple way to join ID's with XQuery results.
Because I don't like external links in answers, code below represents his SQL fiddle, a little bit adapted to the data source from this answer:
with xmlsource as (
-- only for purpose to write long string only once
select '
<users><user><name>user1</name></user>
<user><name>user2</name></user>
<group>
<user><name>user3</name></user>
</group>
<user><name>user4</name></user>
</users>' xml_string
from dual
),
xml_table as (
-- model of xmltable
select 10 id, xml_string xml_data from xmlsource union all
select 20 id, xml_string xml_data from xmlsource union all
select 30 id, xml_string xml_data from xmlsource
)
select xd.id, x.* from
xml_table xd,
XMLTable(
'declare function local:path-to-node( $nodes as node()* ) as xs:string* {$nodes/string-join(ancestor-or-self::*/name(.), ''/'') }; for $i in $rdoc//name return <ret><name_path>{local:path-to-node($i)}</name_path>{$i}</ret> '
passing
XMLParse(content xd.xml_data
)
as "rdoc"
columns
name_path varchar2(4000) path '//ret/name_path',
name_value varchar2(4000) path '//ret/name'
) x
This is not perfect but can be a start:
Here is a sqlfiddle
with xslt as (
select '<?xml version="1.0" ?><xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<records>
<xsl:apply-templates/>
</records>
</xsl:template>
<xsl:template match="//name">
<columns>
<path>
<xsl:for-each select="ancestor-or-self::*">
<xsl:call-template name="print-step"/>
</xsl:for-each>
</path>
<value>
<xsl:value-of select="."/>
</value>
<xsl:apply-templates select="*"/>
</columns>
</xsl:template>
<xsl:template name="print-step">
<xsl:text>/</xsl:text>
<xsl:value-of select="name()"/>
<xsl:text>[</xsl:text>
<xsl:value-of select="1+count(preceding-sibling::*)"/>
<xsl:text>]</xsl:text>
</xsl:template>
</xsl:stylesheet>'
xsl from dual)
, xmldata as
(select xmltransform(xmltype('<?xml version="1.0"?>
<users><user><name>user1</name></user>
<user><name>user2</name></user>
<group>
<user><name>user3</name></user>
</group>
<user><name>user4</name></user>
</users>'), xmltype(xsl)) xd from xslt)
select XT.*
from xmldata c,
xmltable('$x//columns' passing c.xd
as "x"
columns
path_c VARCHAR2(4000) PATH 'path',
value_c VARCHAR2(4000) PATH 'value'
) as XT
This is what I tried to do:
Since you want the "path" I had to use xslt (credits to this post)
Then I used xmltransform for transforming your original xml with the xsl to the
desired output (path, value)
Then I used xmltable to read it as a table
This improves on above answer by A.B.Cade:
<xsl:template name="print-step">
<xsl:variable name="name" select="name()" />
<xsl:text>/</xsl:text>
<xsl:value-of select="$name"/>
<xsl:text>[</xsl:text>
<xsl:value-of select="1+count(preceding-sibling::*[name()=$name])"/>
<xsl:text>]</xsl:text>
</xsl:template>
With result:
/users[1]/user[1]/name[1] user1
/users[1]/user[2]/name[1] user2
/users[1]/group[1]/user[1]/name[1] user3
/users[1]/user[3]/name[1] user4
Instead of:
/users[1]/user[1]/name[1] user1
/users[1]/user[2]/name[1] user2
/users[1]/group[3]/user[1]/name[1] user3
/users[1]/user[4]/name[1] user4