I'm trying to use the Xml Source to shred an XML source file however I do not want the entire document shredded into tables. Rather I want to import the xml Nodes into rows of Xml.
a simplified example would be to import the document below into a table called "people" with a column called "person" of type "xml". When looking at the XmlSource --- it seem that it suited to shredding the source xml, into multiple records --- not quite what I'm looking for.
Any suggestions?
<people>
<person>
<name>
<first>Fred</first>
<last>Flintstone</last>
</name>
<address>
<line1>123 Bedrock Way</line>
<city>Drumheller</city>
</address>
</person>
<person>
<!-- more of the same -->
</person>
</people>
I didn't think that SSIS 2005 supported the XML datatype at all. I suppose it "supports" it as DT_NTEXT.
In any case, you can't use the XML Source for this purpose. You would have to write your own. That's not actually as hard as it sounds. Base it on the examples in Books Online. The processing would consist of moving to the first child node, then calling XmlReader.ReadSubTree to return a new XmlReader over just the next <person/> element. Then use your favorite XML API to read the entire <person/>, convert the resulting XML to a string, and pass it along down the pipeline. Repeat for all <person/> nodes.
Could you perhaps change your xml output so that the content of person is seen as a string? Use escape chars for the <>.
You could use a script task to parse it as well, I'd imagine.
Related
I have an XML document that needs to get stored in an SQL db (Postgres).
I've already seen how that's done, but I have a question: do I just create a single table with a xml field and place the whole document there? This is a document about movies and so (movies, actors...) that has information to be later retrieved.
I've never worked with XML in databases, so I'm a little confused.
Here's an example of my XML:
<?xml version="1.0" encoding="UTF-8"?>
<cinema xmlns="movies"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="movies file:/C:/Users/Fabio/git/LAPD/movies.xsd">
<persons>
<person id="P1">
<name>Samuel L. Jackson</name>
<birth>1948-12-21</birth>
</person>
<person id="P2">
<name>Leonardo Di Caprio</name>
<birth>1974-11-11</birth>
</person>
<person id="P3">
<name>Quentin Tarantino</name>
<birth>1963-03-27</birth>
</person>
</persons>
<movies>
<movie id="M1">
<title>Pulp Fiction</title>
<length>154</length>
<year>1994</year>
<description>The lives of two mob hit men,
a boxer, a gangster's wife, and a pair
of diner bandits intertwine in four tales of violence and redemption</description>
<crew>
<director ref="P3"/>
<writer ref="P3"/>
</crew>
<cast>
<actor ref="P1"/>
</cast>
<rate>
<imdb>8.9</imdb>
<rottentomatoes>9</rottentomatoes>
<moviedb>7.8</moviedb>
<average>8.57</average>
</rate>
<numOscars>1</numOscars>
</movie>
<movie id="M2">
<title>Django Unchained</title>
<length>165</length>
<year>2012</year>
<description>With the help of a German bounty hunter,
a freed slave sets out to rescue his wife
from a brutal Mississippi plantation owner.</description>
<crew>
<director ref="P3"/>
<writer ref="P3"/>
</crew>
<cast>
<actor ref="P1"/>
<actor ref="P2"/>
</cast>
<rate>
<imdb>8.5</imdb>
<rottentomatoes>8</rottentomatoes>
<moviedb>7.4</moviedb>
<average>7.97</average>
</rate>
<numOscars>2</numOscars>
</movie>
</movies>
You can store a whole XML document as value in a single xml column or you can extract data and store it in a more or less normalized form.
Which is better, depends on all the details of your application that are unknown to us.
Here is a related answer discussing pros and cons of storing document types vs. db normalization:
Does JSONB make PostgreSQL arrays useless?
Save XML as a text column of DB so that you can also apply equality operator easily. You may find some error on insertion for " or ' so try to replace them with other characters like ~ or `, both.
I am being returned some XML from a web service. Basically, the xml looks like this:
<response>
<data>
More XML here but the it's escaped by XML entities
</data>
</response>
so, as you can see, I have xml that is valid, but the stuff inside data tag is escaped with XML entities. what's the best (most efficient) way for me to feed this into the parser?
What I am doing right now is, when I get the data from web service, I convert it into NSString....then replace the "XML escaped entities" with real ones.....then convert it back into NSData...then feed it into the parser. This doesn't seem like a very good solution so I was wondering if there's a better way to do it?
Thanks.
Alright, here's the xml that I am getting:
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Header><ActivityId CorrelationId="d39007b5-ee69-41c7-a61d-831b456f9ea3" xmlns="http://schemas.microsoft.com/2004/09/ServiceModel/Diagnostics">aa88d1cd-253c-48d1-abeb-62a880bea806</ActivityId></s:Header><s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><LoginResponse xmlns="http://MSS"><LoginResult><LoginInformation>
<User>
<UserID>612</UserID>
<UserName>Demo User</UserName>
<Email>mssdev#mss-mail.com</Email>
<CompanyID>17034</CompanyID>
<CompanyName>PlanET Demonstration Agency</CompanyName>
</User>
</LoginInformation></LoginResult></LoginResponse></s:Body></s:Envelope>
As you can see, everything in is escaped.
That's kind of horrifying. Why don't you just take the contents of that tag (e.g. the data with the entities) and just pass that through another NSXMLParser? The first parser will have decoded the entities, and the second parser will be presented with the decoded version of the contents of that tag.
I have successfully imported an XML file parsing elements info table attributes using this xml data formating:
<PN>
<guid>aaaa</guid>
<dataInput>0</dataInput>
<deleted>false</deleted>
<customField1></customField1>
<customField2></customField2>
<customField3></customField3>
<description></description>
<name>name1></name>
<ccid>CC007814</ccid>
<productIds>bbbb</productIds>
</PN>
but it errors whwen I input an XML in this format:
<PN guid="aaaa"
deleted="false"
customField1=""
customField2=""
customField3=""
description=""
modified="2010-10-20T00:00:00.001"
created="2010-05-20T18:07:10.416"
name="name1"
ccid="CC006035"
productIds="bbbb"/>
Is this later form usable? Any help would be appreciated. Thanks.
It's usable, but you're looking at the difference between using tags (your first example) and attributes (your second example). Your processing is slightly different.
I have a table with a mix of escaped and non-escaped XML. Of course, the data I need is escaped. For example, I have:
<Root>
<InternalData>
<Node>
<ArrayOfComment>
<Comment>
<SequenceNo>1</SequenceNo>
<IsDeleted>false</IsDeleted>
<TakenByCode>397</TakenByCode>
</Comment>
</ArrayOfComment>
</Node>
</InternalData>
</Root>
As you can see, the data in the Node tag is all escaped. I can use a query to obtain the Node data, but how can I convert it to XML in SQL so that it can be parsed and broken up? I'm pretty new to using XML in SQL, and I can't seem to find any examples of this.
Thanks
You have not given enough information about your end goal, but this will get you very close. FYI - You had two missing ; both after comment>
declare #xml xml
set #xml = '
<Root>
<InternalData>
<Node>
<ArrayOfComment>
<Comment>
<SequenceNo>1</SequenceNo>
<IsDeleted>false</IsDeleted>
<TakenByCode>397</TakenByCode>
</Comment>
</ArrayOfComment>
</Node>
</InternalData>
</Root>
'
select convert(xml, n.c.value('.', 'varchar(max)'))
from #xml.nodes('Root/InternalData/Node/text()') n(c)
Output
<ArrayOfComment>
<Comment>
<SequenceNo>1</SequenceNo>
<IsDeleted>false</IsDeleted>
<TakenByCode>397</TakenByCode>
</Comment>
</ArrayOfComment>
The result is an XML column that you can put into a variable or cross-apply into directly to get data from the XML fragment.
Your best bet might be to look into a HTML Decoding UDF. I did a quick search and found this one:
http://www.andreabertolotto.net/Articles/HTMLDecodeUDF.aspx
You may want to modify it so it only decodes > and <. The one above seems to go above and beyond your needs.
UPDATE
#Cyberkiwi's solution seems to be a bit cleaner. I will leave this up in case the version of SQL Server you are running doesn't support his solution.
In my SQL 2008 database table, I have one column name AUTHOR that contains XML data. The XML is not well formed and has data like below
<Author>
<ID>172-32-1176</ID>
<LastName>White</LastName>
<FirstName>Johnson</FirstName>
<Address>
<Street>10932 Bigge Rd.</Street>
<City>Menlo Park</City>
<State>CA</State>
</Address>
</Author>
Some XML have all of above data and some have just one tag.
<ID>172-32-1176</ID>
I want to write query that returns me a column as identiry.
I tried using AUTHOR.query('data(/Author/ID)') as identity but it fails when XML does not have Author node.
Thanks,
Vijay
Have you tried something like /Author/ID|/ID ? i.e. try for the first scenario, and with no match, the second ?
(note that the | operator is a set union operator, as described here)
In case that nothing "certain" can be maintained about the XML, except that a unique ID element contains the required identity value, then the following XPath expression selects the ID element:
//ID