I'm a beginner when it comes to SQL and have no experience with XML so I'm after a little bit of help.
At the moment I am looking at a single table and just using the query below
select
name,
convert(xml, convert(varbinary(max), orders)) ClientOrders
from client;
In the second columns of SQL output, I have a very lengthy bit of XML similar to the example below. I've used "..." just to skip over some of the output and give a general idea.
Name
ClientOrders
Client1
<report ... ><QueryParameter></QueryParameter Name = "#hello1"><commandtext> ...<value>Example1</value>....<value>Example2</value>...<value>Example3</value>...</commandtext></report>
Client2
<report ... ><QueryParameter></QueryParameter Name = "#hello2"><commandtext> ...<value>Example4</value>....<value>Example5</value>...<value>Example6</value>...</commandtext></report>
I have this for a lot of rows and this output is so long that it exceeds the Excel cell character limit. I'm only looking for the values Example1 through to Example6 in the example given above. Is there an SQL command I can run to get the above string between the open and close value?
I am using SSMS version 18.9.1
Cheers
Related
I'm trying to put together a report for a badge program. Some the data for custom fields we created are stored in a single column in the table as XML. I need to return a couple of the items in there to the report and am having a hard time getting the proper syntax to get it to parse out.
Aside from the XML, the query itself is simple:
SELECT
Person1.firstName
,Person1.lastName
,Person1.idNumber
,Person1.idNumber2
,Person1.idNumber3
,Person1.status
,Person1.customdata
FROM
Person1
the field "customdata" is the XML field that I need to pull Title, and 2 different dates out of. This is what the XML looks like:
<person1_7:CustomData xmlns:person1_7="http://www.badgepass.com/Person1_7">
<Title>IT Director</Title>
<Gaming_x0020_Level>Level 1</Gaming_x0020_Level>
<Gaming_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Gaming_x0020_Issue_x0020_Date>
<Gaming_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Gaming_x0020_Expire_x0020_Date>
<Betting_x0020_Level>Level 1</Betting_x0020_Level>
<Betting_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Betting_x0020_Issue_x0020_Date>
<Betting_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Betting_x0020_Expire_x0020_Date>
<BadgeType>Dual Employee</BadgeType>
<Gaming_x0020_Status>TEMP</Gaming_x0020_Status>
<Betting_x0020_Status>TEMP</Betting_x0020_Status>
</person1_7:CustomData>
I have tried a couple of different methods trying to follow the advice from How to query for Xml values and attributes from table in SQL Server? and then tried declaring a XML namespace with the following query:
WITH XMLNAMESPACES ('http://www.badgepass.com/Person1_7' as X)
SELECT
Person1.firstName
,Person1.lastName
,Person1.idNumber
,Person1.idNumber2
,Person1.idNumber3
,Person1.status
,Person1.customdata.value('(/X:person1_7:customdata/X:Title)[1]', 'varchar(100)') AS Title
FROM
Person1
So far all of my results keep returning "XQuery [Person1.customData.value()]: ")" was expected.
" I'm assuming I have a syntax issue that I'm overlooking as I've never had to manipulate XML with SQL before. Thank you in advance for any help.
Please try the following solution.
Notable points:
XQuery .nodes() method establishes a context so you can access any
XML element right away without long XPath expressions.
Use of the text() in the XPath expressions is for performance
reasons.
SQL
-- DDL and sample data population, start
DECLARE #person1 TABLE (firstname varchar(50), customdata xml);
INSERT INTO #person1(firstname, customdata) VALUES
('John', '<person1_7:CustomData xmlns:person1_7="http://www.badgepass.com/Person1_7">
<Title>IT Director</Title>
<Gaming_x0020_Level>Level 1</Gaming_x0020_Level>
<Gaming_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Gaming_x0020_Issue_x0020_Date>
<Gaming_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Gaming_x0020_Expire_x0020_Date>
<Betting_x0020_Level>Level 1</Betting_x0020_Level>
<Betting_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Betting_x0020_Issue_x0020_Date>
<Betting_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Betting_x0020_Expire_x0020_Date>
<BadgeType>Dual Employee</BadgeType>
<Gaming_x0020_Status>TEMP</Gaming_x0020_Status>
<Betting_x0020_Status>TEMP</Betting_x0020_Status>
</person1_7:CustomData>');
-- DDL and sample data population, end
WITH XMLNAMESPACES ('http://www.badgepass.com/Person1_7' as person1_7)
SELECT firstName
, c.value('(Title/text())[1]', 'VARCHAR(100)') AS Title
, c.value('(Gaming_x0020_Issue_x0020_Date/text())[1]', 'DATETIME') GamingIssueDate
FROM #person1
CROSS APPLY customdata.nodes('/person1_7:CustomData') AS t(c);
Output
+-----------+-------------+-------------------------+
| firstName | Title | GamingIssueDate |
+-----------+-------------+-------------------------+
| John | IT Director | 2021-02-18 12:00:00.000 |
+-----------+-------------+-------------------------+
This question already has answers here:
SQL Server Xml query with multiple namespaces
(3 answers)
Closed 3 years ago.
The problem
I need to take data from XML files provided by another application. These XMLs contain data that is required to be inserted into tables for a database for a different application.
The problem is this, I can't get the values out of the XML file. I keep getting blank/null values instead.
I am using SQL Server 2014 Management Studio.
What I've tried
I've looked into how to read an XML and have that placed into a table. I've gone as far as looking at tutorials that work. But, whenever I try to apply the same process to my tables and using my XML data I keep getting blank values.
Some help would be appreciated as I have been looking into this for a day and researching how to approach this using MS SQL. And I cannot for the life of me understand why I keep getting nulls.
The XML
<DistributionOrder xmlns="a-url">
<Date>2019-03-07T14:38:00</Date>
<Order_Number>ACME01</Order_Number>
<Comment/>
<Status>Active</Status>
<Lines>
<Line>
<Order_Line_Number>1</Order_Line_Number>
<Product>22</Product>
<Status_Line>Active</Status_Line>
<Comment_Line/>
<Location>ENT_78</Location>
</Line>
</Lines>
</DistributionOrder>
The query
DECLARE #xmlData XML;
INSERT INTO dbo.XMLwithOpenXML(XML_data)
SELECT *
FROM OPENROWSET(BULK 'C:\test_file.xml', SINGLE_BLOB) AS ImportSource;
SELECT * FROM XMLwithOpenXML;
SELECT
element.value('(Order_Line_Number/text())[1]', 'nvarchar(80)') AS Order_Line_Number,
element.value('(Product/text())[1]', 'nvarchar(80)') AS Product
FROM
XMLwithOpenXML
CROSS APPLY
XML_data.nodes('DistributionOrder/Lines/Line') AS DO(element)
GO
What I expect
Order_Line_Number | Product
------------------+----------
1 | 22
What I Get
Order_Line_Number | Product
------------------+----------
|
Figured out that the,
<DistributionOrder xmlns="a-url">
</DistributionOrder>
was the problem. I deleted the xmlns="a-url" from the xml and found that my code is working.
Discovered that you can use
;WITH XMLNAMESPACES(DEFAULT 'a-url')
To take the namespace into account.
I want to find the latest instance of an expression, then keep looking to find there a better match and then choose the best match.
The cell I am looking at is a repeatedly apended log with notes followed by the username and timestamp.
Example cell contents:
Starting the investigation.
JWAYNE entered the notes above on 08/12/1976 12:01
Taking over the case. Not a lot of progress recently.
CEASTWOOD entered the notes above on 03/14/2001 09:04
No wonder this case is not progressing, the whole town is covering up some shenanigans!
CEASTWOOD entered the notes above on 03/21/2001 05:23
Star command was right, this investigation has been tossed around like a hot potato for a long time!
BLIGHTYEAR entered the notes above on 08/29/2659 08:01
I am not an expert on database normal form rules but it is annoying that the entries are jammed together into one cell making my job of isolating and checking the notes for specific words, especially when the cell is duplicated for multiple rows until the investigation is closed which puts the notes from future phases into the note column of past events and on top of that the time stamps making a timestamp PATINDEX with even a few minute margin unreliable like this:
CaseID, Username, Notes, Phase, Timestamp
E18902, JWAYNE, Starting....08:01, E1, 03/14/2001 09:13
E18902, CEASTWOOD, Starting....08:01, E2, 03/14/2001 09:13
E18902, CEASTWOOD, Starting....08:01, E3, 03/21/2001 05:34
E18902, BLIGHTYEAR,Starting....08:01, E4, 08/29/2659 07:58
Right now I am doing a reverse on the whole string then a patindex to find the username then substringing to select only the note for that phase of the investigation and the problem is when the same user enters notes for multiple phases my simple "look for the first match staring at the end of the string moving to the top" picks up the wrong entry. My first thought is to search for the username and then check again to see if an entry further up is a better match (note time stamp vs column time stamp) but I am not sure how to code that...
Do i have to get into complicated string splits or is there a more simple solution?
Here's my suggestion. This is for one record, but you can convert it to a user-defined table-valued function, if you like.
I'm going to use the example data you had above.
declare #sourceText nvarchar(max)
, #workText nvarchar(max)
, #xml xml
set #sourceText = <your example text in your question>
set #workText = #sourceText
-- We're going to replace all the carriage returns and line feeds with
-- characters unlikely to appear in your text. (If they are, use some
-- other character.)
set #workText = REPLACE(#workText, char(10), '|')
set #workText = REPLACE(#workText, char(13), '|')
-- Now, we're going to turn your text into XML. Our first target is
-- the string of four "|" characters that the blank lines between entries
-- will be turned into. (If you've got 3, or 6, or blanks in between,
-- adjust accordingly.)
set #workText = REPLACE(#workText, '||||', '</line></entry><entry><line>')
-- Now we replace every other "|".
set #workText = REPLACE(#workText, '|', '</line><line>')
-- Now we construct the rest of the XML and convert the variable to an
-- actual XML variable.
set #workText = '<entry><line>' + #workText + '</line></entry>'
set #workText = REPLACE(#workText, '<line></line>','') -- Get rid of any empty nodes.
set #xml = CONVERT(xml, #workText)
We should now have an XML fragment that looks like this. (You can see it if you insert select #xml into the SQL at this point.)
<entry>
<line>Starting the investigation.</line>
<line>JWAYNE entered the notes above on 08/12/1976 12:01</line>
</entry>
<entry>
<line>Taking over the case. Not a lot of progress recently.</line>
<line>CEASTWOOD entered the notes above on 03/14/2001 09:04</line>
</entry>
<entry>
<line>No wonder this case is not progressing, the whole town is covering up some shenanigans!</line>
<line>CEASTWOOD entered the notes above on 03/21/2001 05:23</line>
</entry>
<entry>
<line>Star command was right, this investigation has been tossed around like a hot potato for a long time!</line>
<line>BLIGHTYEAR entered the notes above on 08/29/2659 08:01</line>
</entry>
We can now transform this XML into XML we like better:
set #xml = #xml.query(
'for $entry in /entry
return <entry><data>
{
for $line in $entry/line[position() < last()]
return string($line)
}
</data>
<timestamp>{ data($entry/line[last()]) }</timestamp>
</entry>
')
This gives us XML that looks like this (just one entry shown, for length reasons):
<entry>
<data>Starting the investigation.</data>
<timestamp>JWAYNE entered the notes above on 08/12/1976 12:01</timestamp>
</entry>
You can convert this back to tabular data with this query:
select EntryData = R.lines.value('data[1]', 'nvarchar(max)')
, EntryTimestamp = R.lines.value('timestamp[1]', 'nvarchar(MAX)')
from #xml.nodes('/entry') as R(lines)
... and get data that looks like this.
And from there, you can do whatever you need to do.
I have a table of people which build from two columns the first one is id and the second one is XML which counties the person details.
For example the person XML file can be:
<Person>
<name>First Last</name>
<address>5 Champ de Mars</address>
<city>Paris</city>
<country>France</country>
<phone-number>+3312345</phone-number>
</Person>
I would like to know how I can extract the value of each one of the parameters in the XML using regexp_substr, I have tried to use the function EXTRACTVALUE which caused me too many problems.
I am ruining oracle.
For example if querying the XML I gave:
select regexp_substr(XML_file, 'pattern name') from peoples where id = 1;
will result in: First Last.
Thank you in advance
Yonatan
What kind of problems did you run into with EXTRACTVALUE?
Because it seems to be what you need...
Something along the lines of SELECT EXTRACTVALUE(my_xml_column, '/Person/name') FROM my_table WHERE id=; should return 'First Last' as asked...
I have a table that has a column called RAW DATA of type NVARCHAR MAX, which is a dump from a web service. Here is a sample of 1 data line:
<CourtRecordEventCaseHist>
<eventDate>2008-02-11T06:00:00Z</eventDate>
<eventDate_TZ>-0600</eventDate_TZ>
<histSeqNo>4</histSeqNo>
<countyNo>1</countyNo>
<caseNo>xxxxxx</caseNo>
<eventType>WCCS</eventType>
<descr>Warrant/Capias/Commitment served</descr>
<tag/>
<ctofcNameL/>
<ctofcNameF/>
<ctofcNameM/>
<ctofcSuffix/>
<sealCtofcNameL/>
<sealCtofcNameF/>
<sealCtofcNameM/>
<sealCtofcSuffix/>
<sealCtofcTypeCodeDescr/>
<courtRptrNameL/>
<courtRptrNameF/>
<courtRptrNameM/>
<courtRptrSuffix/>
<dktTxt>Signature bond set</dktTxt>
<eventAmt>0.00</eventAmt>
<isMoneyEnabled>false</isMoneyEnabled>
<courtRecordEventPartyList>
<partyNameF>Name</partyNameF>
<partyNameM>A.</partyNameM>
<partyNameL>xxxx</partyNameL>
<partySuffix/>
<isAddrSealed>false</isAddrSealed>
<isSeal>false</isSeal>
</courtRecordEventPartyList>
</CourtRecordEventCaseHist>
It was suppose to go in a table, with the node names representing the column names. The table it's going to is created, I just need to exract the data from this row to the table. I have 100's of thousands records like this. I was going to copy to a xml file, then import. But there is so much data, I would rather try and do the work within the DB.
Any ideas?
First, create the table with all the required columns.
Then, use your favorite scripting language to load the table! Mine being groovy, here is what I'd do:
def sql = Sql.newInstance(/* SQL connection here*/)
sql.eachRow("select RAW_DATA from TABLE_NAME") { row ->
String xmlData = row."RAW_DATA"
def root = new XmlSlurper().parseText(xmlData)
def date = root.eventDate
def histSeqNo = root.histSeqNo
//Pull out all the data and insert into new table!
}
I did find an answer to this, I'm sure there is more than one way of doing this. But this is what I got to work. Thanks for everyone's help.
SELECT
pref.value('(caseNo/text())[1]', 'varchar(20)') as CaseNumber,
pref.value('(countyNo/text())[1]', 'int') as CountyNumber
FROM
dbo.CaseHistoryRawData_10 CROSS APPLY
RawData.nodes('//CourtRecordEventCaseHist') AS CourtRec(pref)