I have a WCF SQL receive location and am able to get the relevant details from the database.
The xml looks like this:
<TypedPolling xmlns="http://schemas.microsoft.com/Sql/2008/05/TypedPolling/EmailNotifications">
<TypedPollingResultSet0>
<TypedPollingResultSet0>
<strPortName>port name 1</strPortName>
<LastRun_UTC>2016-01-29T10:20:10.083Z</LastRun_UTC>
</TypedPollingResultSet0>
<TypedPollingResultSet0>
<strPortName>portname 2</strPortName>
<LastRun_UTC>2016-01-29T11:37:38.82Z</LastRun_UTC>
</TypedPollingResultSet0>
<TypedPollingResultSet0>
<strPortName>portname3</strPortName>
<LastRun_UTC>2016-01-29T11:37:39.353Z</LastRun_UTC>
</TypedPollingResultSet0>
</TypedPollingResultSet0>
</TypedPolling>
The associated xsd is like this:
<?xml version="1.0" encoding="utf-16" ?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
targetNamespace="http://schemas.microsoft.com/Sql/2008/05/TypedPolling/EmailNotifications"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="TypedPolling">
<xs:complexType>
<xs:sequence>
<xs:element name="TypedPollingResultSet0">
<xs:complexType>
<xs:sequence>
<xs:element name="TypedPollingResultSet0">
<xs:complexType>
<xs:sequence>
<xs:element name="strPortName"
type="xs:string" />
<xs:element name="LastRun_UTC"
type="xs:dateTime" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
I want to move data from this schema to a simpler one, whose xsd looks like this:
<?xml version="1.0" encoding="utf-16" ?>
<xs:schema xmlns="http://_024_EmailNotifications_Schemas.BizTalkDTADBExtractMod"
xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
targetNamespace="http://_024_EmailNotifications_Schemas.BizTalkDTADBExtractMod"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Root">
<xs:complexType>
<xs:sequence>
<xs:element maxOccurs="unbounded"
name="Notification">
<xs:complexType>
<xs:sequence>
<xs:element name="strPortName"
type="xs:string" />
<xs:element name="LastRun_UTC"
type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
I created a transform shape and did a one to one mapping.
The source has 4 records but after the mapping I only get one record.
Can somebody please help me to figure out what am doing wrong?
Add a looping functoid to your map, with the input from the repeating TypedPollingResultSet0 and the output going to Notification.
Also, avoid copying and pasting XML from a browser -it will add undesireable characters (all the -s in your question), and it'd be a good idea to include some details about your map.
Related
I am receiving a document in Biztalk, which contain some base64 encoded content inside an xml tag.
What I need, is to decode this content (which is a seperate XML document) and insert it into an in an envelope schema, to then be processed further.
The structure of the received document is like so:
<Polling xmlns="http://schemas.microsoft.com/Sql/2008/05/Polling/">
<PolledData>
<DataSet xmlns="http://schemas.datacontract.org/2004/07/System.Data">
<xs:schema id="NewDataSet" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element msdata:IsDataSet="true" name="NewDataSet">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="NewTable">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="ID" type="xs:string"/>
<xs:element minOccurs="0" name="hostUTC" type="xs:dateTime"/>
<xs:element minOccurs="0" name="msgType" type="xs:string"/>
<xs:element minOccurs="0" name="acknowledgment" type="xs:string"/>
<xs:element minOccurs="0" name="sendLog" type="xs:string"/>
<xs:element minOccurs="0" name="msgFormat" type="xs:string"/>
<xs:element minOccurs="0" name="msgbody" type="xs:base64Binary"/>
<xs:element minOccurs="0" name="fromID" type="xs:string"/>
<xs:element minOccurs="0" name="toID" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
<diffgr:diffgram xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1">
<NewDataSet xmlns="">
<NewTable>
<ID>123</ID>
<hostUTC>2018-11-08T14:53:24.11Z</hostUTC>
<msgType>INVOIC</msgType>
<msgFormat>Edifact</msgFormat>
<msgbody>base64encodedmessage...</msgbody>
<fromID>DKT:28504861</fromID>
<toID>KMDoioUBL</toID>
</NewTable>
</NewDataSet>
</diffgr:diffgram>
</DataSet>
</PolledData>
</Polling>
The base64 encoded content is located inside the tag.
I've already created the envelope schema, which has the following structure:
The message is received using a WCF-SQL adapter on a receive location. Currently no pipeline has been set up to process the message. In the 'Messages' tab of the adapter configuration it seems there is an option for base64 encoding, though I am not sure what this does or how it works exactly. When using the body setting here I get returned the above XML.
I'm not sure if this can be done without writing any code.
Reading this post on the subject, it seems that I need to write a custom pipeline component to handle the decoding. But first I would need to extract the content of the specific XML node.
I'm unsure how to approach this problem. I'd like to avoid using orchestrations if at all possible. Can this be done in standard Biztalk, or will I need to create custom functionality to handle this process?
Edit: XPath attempts
/Polling/PolledData[1]/*[namespace-uri()='http://schemas.datacontract.org/2004/07/System.Data' and local-name()='DataSet'][1]/*[namespace-uri()='urn:schemas-microsoft-com:xml-diffgram-v1' and local-name()='diffgram'][1]/*[namespace-uri()='' and local-name()='NewDataSet'][1]/*[namespace-uri()='' and local-name()='NewTable'][1]/*[namespace-uri()='' and local-name()='msgbody'][1]
/*[local-name()='Polling']/*[local-name()='PolledData']/*[local-name()='DataSet']/*[local-name()='diffgram']/*[local-name()='NewDataSet']/*[local-name()='NewTable']/*[local-name()='msgbody']
/Polling/PolledData/DataSet/diffgr:diffgram/NewDataSet/NewTable/msgbody
//*[msgbody]/text()
I have a complex XML file that I am trying to import into SQL Server using SQL Server Data Tools. The issue is that there are many nested elements. The SSIS XML Source identifies each collection of children elements as their own table.
For example, I have the following (simplified) XML and XSD:
<?xml version="1.0" encoding="UTF-8"?>
<TrafficReport>
<TrafficElement>
<counts>
<vehicles>
<vehiclesElement>
<vehicleType>car</vehicleType>
<vehicleCount>15</vehicleCount>
</vehiclesElement>
<vehiclesElement>
<vehicleType>truck</vehicleType>
<vehicleCount>5</vehicleCount>
</vehiclesElement>
</vehicles>
<pedestrian>4</pedestrian>
</counts>
<description>
<type>Manual</type>
</description>
</TrafficElement>
</TrafficReport>
XSD:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:complexType name="Vehicle">
<xs:sequence>
<xs:element name="vehicleType" type="xs:string"/>
<xs:element name="vehicleCount" type="xs:short"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="TrafficCounts">
<xs:sequence>
<xs:element name="vehicles">
<xs:complexType>
<xs:sequence minOccurs="0" maxOccurs="unbounded">
<xs:element name="vehiclesElement" type="Vehicle"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="pedestrian" type="xs:short"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="TrafficType">
<xs:sequence>
<xs:element name="type" type="xs:string"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="Traffic">
<xs:sequence>
<xs:element name="counts" type="TrafficCounts"/>
<xs:element name="description" type="TrafficType"/>
</xs:sequence>
</xs:complexType>
<xs:element name="TrafficReport">
<xs:complexType>
<xs:sequence minOccurs="0" maxOccurs="unbounded">
<xs:element name="TrafficElement" type="Traffic"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Using the above XML and XSD, I have the following output tables from SSIS:
My problem with how SSIS creates the tables is that there are so many intermediate tables linking a single child element to the parent element.
My current solution for this is to create an XSL file that transforms the source data into the following XML:
<?xml version="1.0"?>
<TrafficReport>
<TrafficElement>
<vehiclesElement>
<vehicleType>car</vehicleType>
<vehicleCount>15</vehicleCount>
</vehiclesElement>
<vehiclesElement>
<vehicleType>truck</vehicleType>
<vehicleCount>5</vehicleCount>
</vehiclesElement>
<pedestrian>4</pedestrian>
<type>Manual</type>
</TrafficElement>
</TrafficReport>
Here is the XSL file I am using:
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:template match="TrafficReport">
<TrafficReport>
<xsl:for-each select="TrafficElement">
<TrafficElement>
<xsl:for-each select="counts/vehicles/vehiclesElement">
<vehiclesElement>
<vehicleType><xsl:value-of select="vehicleType" /></vehicleType>
<vehicleCount><xsl:value-of select="vehicleCount" /></vehicleCount>
</vehiclesElement>
</xsl:for-each>
<pedestrian><xsl:value-of select="counts/pedestrian" /></pedestrian>
<type><xsl:value-of select="description/type" /></type>
</TrafficElement>
</xsl:for-each>
</TrafficReport>
</xsl:template>
</xsl:stylesheet>
With the new XML, SSIS wants to create the following tables which are much easier to work with:
My solution is to pull any children elements to their parent elements, as long as it is a 1:1 relationship. The problem is that creating the XSL file is time consuming (over 750 elements) and I need to do this for multiple files.
Is there an automated way to compress XML to the minimum elements like I am doing manually? (Preferably within SSDT.)
Am I using the XML Source data task incorrectly with the first XML? When trying with the actual data, SSDT identifies over 100 output tables.
Any general suggestions on importing complex XML into SQL Server?
Since you have number of XML files, it's seems you need to use script component to process your XML files, where you can put your logic to extract required data from XML. Also you can LINQ to XML to process your files which is quite faster.
Just trying to figure out if below scenario can be achieved through XSD schema level validation.
E.g-
<PreferredPlan>
This tag can have Values - Family or Single
if I got the value as "Family", I should be having below xml elements.
<Family>
<tag1></tag1>
<tag2></tag2>
If "Single" value is captured, I should have below tags.
<Single>
<tag3></tag3>
<tag4></tag4>
</Single>
One simple way to define this would look something like this:
<xs:element name="PreferredPlan">
<xs:complexType>
<xs:choice>
<xs:element ref="Family"/>
<xs:element ref="Single"/>
</xs:choice>
</xs:complexType>
</xs:element>
<xs:element name="Family">
<xs:complexType>
<xs:sequence>
<xs:element ref="tag1"/>
<xs:element ref="tag2"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Single">
<xs:complexType>
<xs:sequence>
<xs:element ref="tag3"/>
<xs:element ref="tag4"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="tag1" type="xs:string"/>
<xs:element name="tag2" type="xs:string"/>
<xs:element name="tag3" type="xs:string"/>
<xs:element name="tag4" type="xs:string"/>
This allows
<PreferredPlan>
<Family>
<tag1/>
<tag2/>
</Family>
</PreferredPlan>
and
<PreferredPlan>
<Single>
<tag3/>
<tag4/>
</Single>
</PreferredPlan>
and forbids
<PreferredPlan>
<Family>
<tag3/>
<tag4/>
</Family>
</PreferredPlan>
is it possible to create a schema where the children of an element would be partialy in a sequence and partiali not? something like this:
<xs:element name="rootElement">
<xs:complexType>
<xs:all>
<xs:element name="value1" type="xs:string"/>
<xs:element name="value2" type="xs:string"/>
</xs:all>
<xs:sequence>
<xs:element name="value3" type="xs:string"/>
<xs:element name="value4" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
Thanks in advance!
OK. I think I understand you know. Your XML looks like:
<?xml version="1.0"?>
<house>
<ownerName>aaa</ownerName>
<ownerLastName>bbb</ownerLastName>
<rooms>
<room1>kitchen</room1>
<room2>living room</room2>
<room3></room3>
<room4></room4>
<room5></room5>
</rooms>
</house>
You want the ownerName and ownerLastName elements to occur in any order although they must both be present. It's not clear from your description it the position of the rooms element is important. I have assumed it is. It's import that the child elements of rooms remain in order.
Splitting this up, we can define a complexType for rooms (so the model is complete)
<xs:complexType name="RoomType">
<xs:simpleContent>
<xs:extension base="xs:string"/>
</xs:simpleContent>
</xs:complexType>
<xs:element name="RoomsType">
<xs:complexType>
<xs:sequence>
<xs:element name="room1" type="RoomType"/>
<xs:element name="room2" type="RoomType"/>
<xs:element name="room3" type="RoomType"/>
<xs:element name="room4" type="RoomType"/>
<xs:element name="room5" type="RoomType"/>
</xs:sequence>
</xs:complexType>
</xs:element>
This gives us the sequence of rooms in order.
There are quite a few limitations on the use of xs:all so I've avoided it here (you can't use in a sequence, for example). In order to allow the owner names to occur in any order, I've used a choice of two sequences. The choice is embedded in another sequence to ensure that the rooms element comes last.
<xs:complexType name="HouseType">
<xs:sequence>
<xs:choice>
<xs:sequence>
<xs:element ref="ownerName"/>
<xs:element ref="ownerLastName"/>
</xs:sequence>
<xs:sequence>
<xs:element ref="ownerLastName"/>
<xs:element ref="ownerName"/>
</xs:sequence>
</xs:choice>
<xs:element name="rooms" type="HouseType"/>
</xs:sequence>
</xs:complexType>
<xs:element name="house" type="HouseType"/>
Now, if you actually don't care about the order or ownerLastName, ownerName and house then you can use xs:all without a problem:
<xs:complexType name="HouseType">
<xs:all>
<xs:element ref="ownerName"/>
<xs:element ref="ownerLastName"/>
<xs:element name="rooms" type="HouseType"/>
</xs:all>
</xs:complexType>
which requires that you have all three elements present in any order.
I'm trying to create an element in a schema where its sub-types can appear in any order and as many times as necessary:
<xs:element name="workflowNodes">
<xs:complexType>
<xs:sequence minOccurs="0" maxPO>
<xs:element ref="nodeType1" />
<xs:element ref="nodeType2" />
<xs:element ref="nodeType3" />
<xs:element ref="nodeType4" />
</xs:sequence>
</xs:complexType>
</xs:element>
where the nodes nodeType(n) can be in any order and be repetitive as necessary.
Also, I tried :
But the compiler is not working complaining the nodeType2 is not valid when there is not nodeType1.
What am I missing here?
Thanks in advance.
You should be able to get your results by doing this:
<xs:element name="workflowNodes">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element ref="nodeType1" />
<xs:element ref="nodeType2" />
<xs:element ref="nodeType3" />
<xs:element ref="nodeType4" />
</xs:choice>
</xs:complexType>
</xs:element>
<xs:choice> gives you the option to pick one of the elements, and making the xs:choice appear many times allows you to pick each element as many times as you like.
Marc