How do I produce an XML response from PostgreSQL 1:n tables? - sql

I have the table "client" with:
id name registered_on status
-- ------- ------------- ------
1 Alice 2020-03-04 a
2 Vincent 2020-03-05 p
3 Anne 2020-03-06 a
And the table "account" with:
client_id account_number type balance
--------- -------------- ---- -------
1 300-1 CHK 100
2 307-5 SAV 24
2 307-6 CHK 350
I created them in DB Fiddle (for a similar question I asked before about producing JSON).
Now, I need a SQL query to produce the 1:n XML document:
<client id="1" name="Alice" registered_on="2020-03-04" status="a">
<account account_number="300-1" type="CHK" balance="100" />
</client>
<client id="2" name="Vincent" registered_on="2020-03-05" status="p">
<account account_number="307-5" type="SAV" balance="24" />
<account account_number="307-6" type="CHK" balance="350" />
</client>
<client id="3" name="Anne" registered_on="2020-03-06" status="a" />
There's a 1:n relationship between the tables and some clients may not have an account (such as "Anne"). The result is a simple join (probably an outer join) that I know how to do. I just don't get how to produce a XML document from it.
If it's makes it easier/shorter I'm open to an alternative XML result, as long as it represents the same data; using tags, instead of attributes, for example.

After trying a bunch of options I was able to find the answer(s).
Original Format: With Attributes
It's possible to produce the XML result using an outer join:
select
xmlserialize(content -- remove this line to keep as XML instead of VARCHAR
xmlagg(r)
as text) -- remove this line to keep as XML instead of VARCHAR
from (
select
xmlelement(name client,
xmlattributes(c.id, c.name, c.registered_on, c.status),
case when count(a.client_id) > 0 then
xmlagg(xmlelement(name account,
xmlattributes(a.account_number, a.type, a.balance) ))
end
) as r
from client c
left join account a on a.client_id = c.id
group by c.id
) s
Or using subqueries (shorter but less performant):
select
xmlserialize(content -- remove this line to keep as XML instead of VARCHAR
xmlagg(
xmlelement(name client, xmlattributes(id, name, registered_on, status),
( select xmlagg(xmlelement(name account,
xmlattributes(a.account_number, a.type, a.balance)
)) from account a where a.client_id = c.id
)
))
as text) -- remove this line to keep as XML instead of VARCHAR
from client c;
Result:
<client id="1" name="Alice" registered_on="2020-03-04" status="a">
<account account_number="300-1" type="CHK" balance="100.00" />
</client>
<client id="2" name="Vincent" registered_on="2020-03-05" status="p">
<account account_number="307-5" type="SAV" balance="24.00" />
<account account_number="307-6" type="CHK" balance="350.00" />
</client>
<client id="3" name="Anne" registered_on="2020-03-06" status="a" />
Alternative Format: Without Attributes
Some people prefer to avoid attributes altogether and always use tags. That can also be done, using:
select
xmlserialize(content -- remove this line to keep as XML instead of VARCHAR
xmlagg(xmlelement(name client,
xmlforest(id, name, registered_on, status),
( select xmlagg(xmlelement(name account,
xmlforest(a.account_number, a.type, a.balance)))
from account a where a.client_id = c.id
)
))
as text) -- remove this line to keep as XML instead of VARCHAR
from client c;
Result:
<client>
<id>1</id>
<name>Alice</name>
<registered_on>2020-03-04</registered_on>
<status>a</status>
<account>
<account_number>300-1</account_number>
<type>CHK</type>
<balance>100.00</balance>
</account>
</client>
<client>
<id>2</id>
<name>Vincent</name>
<registered_on>2020-03-05</registered_on>
<status>p</status>
<account>
<account_number>307-5</account_number>
<type>SAV</type>
<balance>24.00</balance>
</account>
<account>
<account_number>307-6</account_number>
<type>CHK</type>
<balance>350.00</balance>
</account>
</client>
<client>
<id>3</id>
<name>Anne</name>
<registered_on>2020-03-06</registered_on>
<status>a</status>
</client>

Related

How to extract a list of values from a SQL XML column

Here is my XML:
<Triggers>
<Trigger>
<Name>DrugName</Name>
<Values>
<Value>Meclofenamate</Value>
<Value>Meloxicam</Value>
<Value>Vimovo</Value>
<Value>Nabumetone</Value>
<Value>Qmiiz</Value>
<Value>Tolmetin</Value>
</Values>
</Trigger>
<Trigger>
<Name>State</Name>
<Values>
<Value>MI</Value>
</Values>
</Trigger>
<Trigger>
<Name>BenefitType</Name>
<Values>
<Value>Pharmacy</Value>
</Values>
</Trigger>
<Trigger>
<Name>LineOfBusiness</Name>
<Values>
<Value>Medicaid</Value>
</Values>
</Trigger>
</Triggers>
My goal is to get output that looks like this:
ID DrugName State BenefitType LineOfBusiness
6500 Meclofenamate MI Pharmacy Medicaid
6501 Meloxicam MI Pharmacy Medicaid
6502 Vimovo MI Pharmacy Medicaid
6503 Nabumetone MI Pharmacy Medicaid
6504 Qmiiz MI Pharmacy Medicaid
6505 Tolmetin MI Pharmacy Medicaid
I can't find any examples on stackoverflow after extensive searches where XML is organized this way, and the examples I have found, tweaked and applied result in my getting a list of all the Values in one column (State values, BenefitType values, etc. mixed in with DrugName values).
The ID column is not part of the XML, but I need to have that in my output.
Here what the table looks like that has the XML column.
You needs the .nodes XML function to break out the Trigger nodes, then again for Values rows.
To get the value of a node instead of it's name, we use text().
To verify we are grabbing the right Trigger node for each column, we use the [] predicate to check (a bit like a where).
.value requires a single value, so we use [1] to get the first node.
SELECT
DrugName = drugs.DrugName.value('text()[1]','nvarchar(100)'),
State = tr.Trigg.value('Trigger[Name/text()="State"][1]/Values[1]/Value[1]/text()[1]', 'nvarchar(100)'),
BenefitType = tr.Trigg.value('Trigger[Name/text()="BenefitType"][1]/Values[1]/Value[1]/text()[1]', 'nvarchar(100)'),
LineOfBusiness = tr.Trigg.value('Trigger[Name/text()="LineOfBusiness"][1]/Values[1]/Value[1]/text()[1]', 'nvarchar(100)')
FROM #xml.nodes('/Triggers') tr(Trigg)
OUTER APPLY tr.Trigg.nodes('Trigger[Name/text()="DrugName"][1]/Values/Value') drugs(DrugName)

SQL to XML transfer of data

Ok have been trying to learn SQL to XML the last few days and this is what I have been able to teach my self thus far.
`SELECT distinct StudentItem.foldername AS "foldername", StudentItem.status, StudentItem.vhrid, StudentItem.firstname, StudentItem.middleinitial, StudentItem.lastname,
dbo.getEnumDescript(StudentType, 'StudentType') AS title,
StudentItem.email,
dbo.getEnumDescript(OfficeLocation, 'OfficeLocation') AS Office,
practices.id as 'StudentItem/practices/practice/id',
practices.name as 'StudentItem/practices/practice/name',
schoolItem.Name as 'StudentItem/bio/schools/schoolItem/schoolname',
schoolItem.schoolYear as 'lawyerItem/bio/schools/schoolItem/schoolyear'
FROM [dbo].[Student] as lawyerItem
LEFT JOIN [dbo].[StudentGroups] as aprac on StudentItem.vhrid = aprac.vhrid
INNER JOIN [dbo].[PracticeGroups] as practices on aprac.PracticeGroupID = practices.ID
LEFT JOIN [dbo].[StudentEducation] as schoolItem on StudentItem.vhrid = schoolItem.vhrid
where StudentItem.vhrid='50330'
FOR XML path, ROOT ('StudentItem'), ELEMENTS;`
What I get is this
`<StudentItems>
<row>
<foldername>susan.wissink</foldername>
<status>1</status>
<vhrid>50330</vhrid>
<firstname>Susan</firstname>
<middleinitial>M.</middleinitial>
<lastname>Wissink</lastname>
<title>Student leader</title>
<email>swissink#blank.com</email>
<Office>Phoenix</Office>
<StudentItem>
<practices>
<practice>
<id>681</id>
<name>Real Estate Finance and Lending</name>
</practice>
</practices>
<bio>
<schools>
<schoolItem>
<schoolname><i>Best in America®</i>, ASU</schoolname>
<schoolyear>2016</schoolyear>
</schoolItem>
</schools>
</bio>
</StudentItem>
</row>
<row>
<foldername>susan.wissink</foldername>
<status>1</status>
<vhrid>50330</vhrid>
<firstname>Susan</firstname>
<middleinitial>M.</middleinitial>
<lastname>Wissink</lastname>
<title>Student leader</title>
<email>swissink#blank.com</email>
<Office>Phoenix</Office>
<StudentItem>
<practices>
<practice>
<id>681</id>
<name>Real Estate Finance and Lending</name>
</practice>
</practices>
<bio>
<schools>
<schoolItem>
<schoolname><i>Best in America®</i>, UOP</schoolname>
<schoolyear>2011-2015</schoolyear>
</schoolItem>
</schools>
</bio>
</StudentItem>
</row>`
But I'm trying to get the all the practices and schools to show up as one entry for the guy that. More or less I'm trying to get it to look like below.
`<StudentItems>
<row>
<foldername>susan.wissink</foldername>
<status>1</status>
<vhrid>50330</vhrid>
<firstname>Susan</firstname>
<middleinitial>M.</middleinitial>
<lastname>Wissink</lastname>
<title>Student leader</title>
<email>swissink#blank.com</email>
<Office>Phoenix</Office>
<StudentItem>
<practices>
<practice>
<id>681</id>
<name>Real Estate Finance and Lending</name>
<id>683</id>
<name>Business and Finance</name>
</practice>
</practices>
<bio>
<schools>
<schoolItem>
<schoolname><i>Best in America®</i>, UOP</schoolname>
<schoolyear>2011-2015</schoolyear>
<schoolname><i>Best in America®</i>, ASU</schoolname>
<schoolyear>2016</schoolyear>
</schoolItem>
</schools>
</bio>
</StudentItem>
</row>`
Any help would be welcome. Thank You.
Without sample data, it's difficult to write code and test for. But generally what you need to do is to create sub-queries to create your practice and schoolItem XML nodes. Something like this:
SELECT distinct StudentItem.foldername AS "foldername",
StudentItem.status,
StudentItem.vhrid,
StudentItem.firstname,
StudentItem.middleinitial,
StudentItem.lastname,
dbo.getEnumDescript(StudentType, 'StudentType') AS title,
StudentItem.email,
dbo.getEnumDescript(OfficeLocation, 'OfficeLocation') AS Office,
(
select practices.id, practices.name
from [dbo].[StudentGroups] as aprac
INNER JOIN [dbo].[PracticeGroups] as practices
on aprac.PracticeGroupID = practices.ID
where StudentItem.vhrid = aprac.vhrid
FOR XML path(''), type
) 'StudentItem/practices/practice',
(
select Name schoolname, schoolYear
from [dbo].[StudentEducation] schoolItem
where StudentItem.vhrid = schoolItem.vhrid
FOR XML path(''), type
) 'StudentItem/bio/schools/schoolItem'
FROM [dbo].[Student] as StudentItem
where StudentItem.vhrid='50330'
FOR XML path, ROOT ('StudentItem');

Using SQL FOR XML PATH, how can I achieve the following formatted XML output?

The scope of this project is much larger than this one question. I've been tasked with a project and I'll not bore you with the intimate details. Ultimately what I'm needing to do is get the data out of the database and into XML so I can convert to JSON and create a simple web app that will allow me to parse and format the data in way that will meet the customer's needs.
I'm sure there's a better way to do this, but this is the path I've settled on..
I have about 46,000 records dumped into a Temp Table. To Give you an idea of how this data is structured running the following query:
SELECT
TransactionID,
OwnerID,
Date,
TransactionType,
ChargeCode,
Description,
DebitAmount
FROM #OwnerHistoryTemp
WHERE OwnerID = '11111111'
Returns this:
TransactionID OwnerID Date TransactionType ChargeCode Description DebitAmount
28727 11111111 2014-12-01 E A1 APPLY CHARGES 210.00
28728 11111111 2014-12-03 C A1 DB11111111 210.00
28729 11111111 2015-01-01 E A1 APPLY CHARGES 183.37
What I'm looking to do here is use the SQL FOR XML PATH (open to any other suggestions) to output the data like so:
<OwnerHistory>
<OwnerID OwnerID="11111111">
<Transactions>
<TransactionID ID="28727">
<Date>2014-12-01</Date>
<TransactionType>E</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>210.00</DebitAmount>
</TransactionID>
<TransactionID ID="28728">
<Date>2014-12-03</Date>
<TransactionType>C</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>DB11111111</Description>
<DebitAmount>210.00</DebitAmount>
</TransactionID>
<TransactionID ID="28729">
<Date>2015-1-01</Date>
<TransactionType>E</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>183.37</DebitAmount>
</TransactionID>
</Transactions>
</OwnerID>
</OwnerHistory>
The Query I have, gets me close, but not QUITE there. Because the same OwnerID appears multiple times (once for each TransactionID), running the following query:
SELECT
OwnerID AS "#OwnerID",
TransactionID AS "Transaction/#RecordID",
Date AS "Transaction/Date",
TransactionType AS "Transaction/TransactionType",
ChargeCode AS "Transaction/ChargeCode",
Description AS "Transaction/Description",
DebitAmount AS "Transaction/DebitAmount"
FROM #OwnerHistoryTemp
WHERE OwnerID = '11111111'
GROUP BY OwnerID, RecordID, Date, ChargeCode, Description, DebitAmount
order by OwnerID
FOR XML PATH ('OwnerID'), ROOT('OwnerHistory')
Returns the folowing:
<OwnerHistory>
<OwnerID OwnerID="11111111">
<Transaction RecordID="28727">
<Date>2014-12-01</Date>
<TransactionType>E</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>210.0000</DebitAmount>
</Transaction>
</OwnerID>
<OwnerID OwnerID="11111111">
<Transaction RecordID="28728">
<Date>2014-12-03</Date>
<TransactionType>C</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>DB11111111</Description>
<DebitAmount>210.0000</DebitAmount>
</Transaction>
</OwnerID>
<OwnerID OwnerID="11111111">
<Transaction RecordID="28729">
<Date>2015-01-01</Date>
<TransactionType>E</TransactionType>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>183.3700</DebitAmount>
</Transaction>
</OwnerID>
</OwnerHistory>
Any thoughts on how to only pull the OwnerID one time as the Parent and group all transactions underneath it?
Might be something simple I'm just not understanding, or it might be impossible. Feel free to publicly cane me if the former is the case...
Ready for your caning?
Nest your queries to get nested XML, something like this:
SELECT TOP 1
OwnerID AS "#ID",
(SELECT
TransactionID AS "Transaction/#ID",
[Date] AS "Transaction/Date",
TransactionType AS "Transaction/Type",
ChargeCode AS "Transaction/ChargeCode",
[Description] AS "Transaction/Description",
DebitAmount AS "Transaction/DebitAmount"
FROM OwnerHistory
WHERE OwnerID = [Owner].OwnerID
FOR XML PATH(''), TYPE) Transactions
FROM OwnerHistory [Owner]
WHERE OwnerID = '11111111'
FOR XML PATH('Owner'), ROOT('OwnerHistory'), TYPE
Resulting XML:
<OwnerHistory>
<Owner ID="11111111">
<Transactions>
<Transaction ID="28727">
<Date>2015-03-26</Date>
<Type>E</Type>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>210.0000</DebitAmount>
</Transaction>
<Transaction ID="28728">
<Date>2015-03-26</Date>
<Type>C</Type>
<ChargeCode>A1</ChargeCode>
<Description>DB11111111</Description>
<DebitAmount>210.0000</DebitAmount>
</Transaction>
<Transaction ID="28729">
<Date>2015-03-26</Date>
<Type>E</Type>
<ChargeCode>A1</ChargeCode>
<Description>APPLY CHARGES</Description>
<DebitAmount>183.3700</DebitAmount>
</Transaction>
</Transactions>
</Owner>
</OwnerHistory>
Note that the TOP 1 is only included to avoid repeating the entire set of transactions for each row containing the OwnerID. This could be handled in a number of ways; normally this sort of nesting would be the result of a join on two normalized tables so that only one instance of Owner would occur.
MSDN has some good examples to demonstrate this technique.

Solr delta import does not work

I am using solr 4.2. Note that full import works but somehow delta import doesn't. Delta import does not give any error but never fetches any changes. Here's the data config file.
<dataConfig>
<dataSource type="JdbcDataSource"
driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver:testsql;databaseName=test"
user="dba"
password="dba"/>
<script>
<![CDATA[
function metadataTransformer (row) {
var attributeName = row.get("attribute_name");
var attributeValue = row.get("attribute_value");
row.put(attributeName, attributeValue);
return row;
}
]]>
</script>
<document name="PRODUCTS">
<entity name="product" query="select distinct p.product_id as id from products p
inner join products_meta pm on p.product_id = pm.product_id
inner join meta m on pm.meta_id = m.meta_id
where m.meta_type_id = 11 order by id desc"
deltaImportQuery="select distinct p.product_id as id from products p
inner join products_meta pm on p.product_id = pm.product_id
inner join meta m on pm.meta_id = m.meta_id
where m.meta_type_id = 11 and p.product_id='${dih.delta.product_id}'"
deltaQuery= "select distinct product_id as id from products
where updtime > '${dih.last_index_time}'">
<field column="id" name="id"/>
<entity name="attribute" query="select attribute_name,attribute_value from solr_import
where id =${product.id}" transformer= "script:metadataTransformer">
</entity>
</entity>
</document>
</dataConfig>
Here's what I have tried without any luck.
changing p.product_id='${dih.delta.product_id} to p.product_id='${dih.delta.id} and other way also.
changing where updtime > '${dih.last_index_time}'" to where updtime > '${dih.last_index_time}'"
Please help.
This is fixed. The issue was solr box was on UTC timezone. Once I converted that to ET timezone, delta import started working fine.
Hope this helps someone else.
If you don't want to use ${dih.last_index_time} (server's time config problem), you can try mysql query like:
deltaImportQuery="SELECT * FROM table_name where id='${dataimporter.delta.id}'"
deltaQuery="select id from table_name where update_time > SUBTIME( NOW( ) , '0:15:0' )
I used with crontab and worked fine!

How to get xml from many records?

So I have the below that will allow me to get xml from one record in the Products table.
However, I need to to be able to get the XML from ALL records in the Products table, along with its ProductId.
I'm confused on how to start this.
DECLARE #MyXML XML
SET #MyXML =
(SELECT ProductsXML
FROM Products
WHERE ProductId= 1)
SELECT
a.b.value('#upccode','int') as UPC,
a.b.value('#dateadded','date') as DateAdded
FROM
#MyXML.nodes('xml/Product/UPC')a(b);
I'm using SQL Server 2008.
Test Data:
ProductId: 1
ProductsXML:
<xml>
<Product>
<UPC upccode="1237" dateadded="10/9/2012"/>
<UPC upccode="1236" dateadded="10/8/2012"/>
<UPC upccode="1235" dateadded="10/7/2012"/>
<UPC upccode="1234" dateadded="10/6/2012"/>
</Product>
</xml>
ProductId: 2
ProductsXML:
<xml>
<Product>
<UPC upccode="9876" dateadded="9/9/2012"/>
<UPC upccode="9877" dateadded="9/8/2012"/>
<UPC upccode="0998" dateadded="9/7/2012"/>
<UPC upccode="7877" dateadded="9/6/2012"/>
</Product>
</xml>
The Result I'm looking for is something like this:
ProductId UPC DateAdded
--------- --- ---------
1 1237 10/9/2012
1 1236 10/8/2012
....
2 9876 9/9/2012
2 9877 9/8/2012
Right now I can get the above but ONLY by specifying one PoductId at a time. I want to be able to run all of the `Products' without specifying each entry.
I guess you are looking for something like this.
SELECT ProductId,
a.b.value('#upccode','int') as UPC,
a.b.value('#dateadded','date') as DateAdded
FROM Products
CROSS APPLY ProductsXML.nodes('xml/Product/UPC')a(b);
There is no need to have a XML variable in there. Query the Products table directly and use CROSS APPLY against ProductsXML field to shred your XML.
hi u can try using Linq to xml to create an entity from the xml, see the sample code below
XDocument somedoc = XDocument.Load("your xml path");
var somelist = from somevariable in somedoc.Descendants("Product")
select new Product {
UPC = somevariable.Attribute("upccode").value
DateAdded = somevariable.Attribute("dateadded").value
}
so now somelist is a collection thru which u can iterate and add to List<Product>, create product b4 u start tryinh this