Need to Revise version of SQL Server generated XML - sql

The following lines of code generate me an XML file looking like the below set, which is almost acceptable to my client. I say ALMOST because the one change I need to to have <?xml version="1.0" encoding="UTF-8"?> versus the standard <?xml version="1.0"> which is at the top of every XML file I generate using the EXEC xp_cmdshell command below. I essentially need <?xml version="1.0" encoding="UTF-8"?> instead of <?xml version="1.0">. Can someone please tell me how this can be accomplished?
-- SQL CODE USED TO GENERATE XML FILE - Using XML Path
SET #FileString = #FileName + '.xml" -S ALSCG-JPATHIL\SQLEXPRESS -T -c -t,'
SET #SQLSTRING = 'bcp ";WITH XMLNAMESPACES (DEFAULT ''urn:CP-xml'') select A.TargetSystem AS ''Header/Target'' from [Header] A FOR XML PATH(''Qty'')" queryout "C:\Program Files\'
SET #SQLSTRING = #SQLSTRING + #FileString
EXEC xp_cmdshell #SQLSTRING
-- XML FILE CONTENTS GENERATED - Missing the Encoding Condition here
<?xml version="1.0">
<Qty xmlns="urn:CP-xml">
<Header>
<Target></Target>
</Header>
</Qty>
-- XML FILE CONTENTS DESIRED - Note only difference is the Encoding!
<?xml version="1.0" encoding="UTF-8"?>
<Qty xmlns="urn:CP-xml">
<Header>
<Target></Target>
</Header>
</Qty>

The very last response in this link has the only "solution" to this problem that I've been able to find
Hi, (late response but might help someone in future) VARBINARY did not
work for me, probably 'coz of the my datasource didn't comply. Heres
what worked for me at the SQL end;
1) Store your raw xml data as VARCHAR or TEXT (instead of NVARCHAR or
NTEXT) into a variable,
2) Read this variable into the xml data type using utf-8 encoding.
Something like:
DECLARE #TempHTMLText
SET #TempHTMLText = --your raw xml data DECLARE #XMLDataText XML
SELECT #XMLDataType = '<?xml version="1.0" encoding="utf-8" ?>' + #TempHTMLText
I don't think you'd need to do exactly that, rather use the following query:
DECLARE #Xml XML;
WITH XMLNAMESPACES(DEFAULT 'urn:CP-xml')
SELECT #Xml = (select A.TargetSystem AS 'Header/Target' from [Header] A FOR XML PATH('Qty'));
SELECT CONCAT('<?xml version="1.0" encoding="utf-8" ?>', CAST(#Xml AS VARCHAR(MAX)))
Note that in my testing, I was unable to convert this data back to XML while preserving the encoding tag. That seems to be enforced explicitly by the XML data type

Related

Liquibase giving syntax error for an included SQL file,which is otherwise correct

I have a change log file,part of which looks like:
<include file="ChangeSets/00_SessionAuth.xml"/>
<include file="ChangeSets/01_Legacy_Baseline_V197_ANB.xml" context="legacy"/>
<include file="ChangeSets/02_V198_ANB.xml" context="non-legacy"/>
The change set 01_Legacy_Baseline_V197_ANB.xml has 197 sql scripts included,which are part of legacy database.
The change set 02_V198_ANB.xml is defined as:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<include file="ChangeSets/sql/Dummy_schema/V198_AddColumn.sql"/>
</databaseChangeLog>
When i run the Liquibase update with context as non legacy, i get the following error :
Unexpected error running Liquibase: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601, SQLERRMC=SET SCEHMA = Dummy_schema;;BEGIN-OF-STATEMENT;<space>, DRIVER=3.62.56 [Failed SQL: SET SCEHMA = Dummy_schema;
The SQL file itself is correct.
I think this is an issue related to end delimiter or splitStatements.
The SQL goes like this:
SET SCEHMA = Dummy_schema;
ALTER TABLE T1 ADD COLUMN C1 VARCHAR(35)
ADD COLUMN C2 INTEGER;
Call Sysproc.admin_cmd ('REORG TABLE dummy_schema.T1');
ALTER TABLE T2 ADD COLUMN CT1 INTEGER;
Call Sysproc.admin_cmd ('REORG TABLE dummy_schema.T2');
We are using DB2 V10.5.
Can someone also provide the list of attributes,we can use with tag.
Regards
I think your assertion that
the SQL file itself is correct
is incorrect. That is valid SQL if it was run through the db2 command line application, but it is not valid SQL to be sent through a JDBC query, which is what Liquibase does.

Missing encode info on XML header using XMLDOM

i want to print xml header with xmldom. The problem is, it prints only the xml version but missing the encode information.
What i got :
<?xml version="1.0"?>
What i want:
<?xml version="1.0" encoding="UTF-8"?>
im using ORACLE 11g
And here is what i got so far:
doc := xmldom.newdomdocument;
xmldom.setversion(doc,'1.0');
xmldom.setCharset(doc,'UTF-8');
mainNode := xmldom.makeNode(doc);
rootElmt := xmldom.createElement(doc,'Dokument');
rootNode := xmldom.appendChild (mainNode,xmldom.makeNode(rootElmt));
SetCurNode (rootNode);
dbms_lob.createTemporary(vClob,true);
dbms_xmldom.writeToClob (doc,vClob);
xmldom.freedocument (doc);
Thanks in advance,
Ivan
Not entirely sure why charset isn't included in the output, but it seems to be a common issue. One alternative (shown here: https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:2791321000346652231) is to do the following:
xmldom.setVersion(doc,'1.0" encoding="UTF-8');
Edit: Apparently, the character set you specify is ignored unless you use the writeToFile procedure, this would explain why it is not being included in the output when using writeToClob

Converting Complex XML to CSV

I have some (complex to me) XML code that I need to convert into CSV, I need absolutely every value added to the CSV for every submission, I have tried a few basic things however I cant get past the deep nesting and the different structures of this file.
Could someone please help me with a powershell script that would, I have started but cannot get the output of all data out I only get Canvas Results
Submissions.xml To large to post here (102KB)
$d=([xml](gc submissions.xml)).CANVASRESULTS | % {
foreach ($i in $_.CANVASRESULTS) {
$o = New-Object Object
Add-Member -InputObject $o -MemberType NoteProperty -Name Submissions -Value $_.Submission
Add-Member -InputObject $o -MemberType NoteProperty -Name Submission -Value $i
$o
}
}
$d | ConvertTo-Csv -NoTypeInformation -Delimiter ","
Anytime a complex XML has deeply nested structures and you require migration into a flat file format (i.e., txt, csv, xlsx, sql), consider using XSLT to simplify your XML format. As information, XSLT is a declarative, special-purpose programming language used to style, re-format, re-structure XML/HTML and other SGML markup documents for various end-use purposes. Aside - SQL is also a declarative, special-purpose programming language.
For most softwares to import XML into flat file formats in two dimensions of rows and columns, XML files must follow repeating elements (i.e., rows/records) with one level of children for columns/fields:
<data>
<row>
<column1>value</column1>
<column1>value</column1>
<column1>value</column1>
...
</row>
<row>
...
</data>
Nearly every programming language maintains an XSLT processor including PowerShell, Java, C#, Perl, PHP, Python, SAS, even VBA with your everyday MS Excel. For your complex XML, below is an example XSLT stylesheet with following output. Do note I manually create nodes based on values from original XML:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:strip-space elements="*"/>
<xsl:output method="xml" indent="yes"/>
<xsl:template match="CanvasResult">
<Data>
<xsl:for-each select="//Responses">
<Submission>
<Fitter><xsl:value-of select="Response[contains(Label, 'Fitter Name')]/Value"/></Fitter>
<Date><xsl:value-of select="Response[Label='Date']/Value"/></Date>
<Time><xsl:value-of select="Response[Label='Time']/Value"/></Time>
<Client><xsl:value-of select="Response[Label='Client']/Value"/></Client>
<Machine><xsl:value-of select="Response[Label='Machine']/Value"/></Machine>
<Hours><xsl:value-of select="Response[Label='Hours']/Value"/></Hours>
<Signature><xsl:value-of select="Response[Label='Signature']/Value"/></Signature>
<SubmissionDate><xsl:value-of select="Response[Label='Submission Date:']/Value"/></SubmissionDate>
<SubmissionTime><xsl:value-of select="Response[Label='Submission Time:']/Value"/></SubmissionTime>
<Customer><xsl:value-of select="Response[Label='Customer:']/Value"/></Customer>
<PlantLocation><xsl:value-of select="Response[Label='Plant Location']/Value"/></PlantLocation>
<PlantType><xsl:value-of select="Response[Label='Plant Type:']/Value"/></PlantType>
<PlantID><xsl:value-of select="Response[Label='Plant ID:']/Value"/></PlantID>
<PlantHours><xsl:value-of select="Response[Label='Plant Hours:']/Value"/></PlantHours>
<RegoExpiryDate><xsl:value-of select="Response[Label='Rego Expiry Date:']/Value"/></RegoExpiryDate>
<Comments><xsl:value-of select="Response[Label='Comments:']/Value"/></Comments>
</Submission>
</xsl:for-each>
</Data>
</xsl:template>
</xsl:stylesheet>
Output
<?xml version='1.0' encoding='UTF-8'?>
<Data>
...
<Submission>
<Fitter>Damian Stewart</Fitter>
<Date/>
<Time/>
<Client/>
<Machine/>
<Hours/>
<Signature/>
<SubmissionDate>28/09/2015</SubmissionDate>
<SubmissionTime>16:30</SubmissionTime>
<Customer>Dicks Diesels</Customer>
<PlantLocation/>
<PlantType>Dozer</PlantType>
<PlantID>DZ09</PlantID>
<PlantHours>2213.6</PlantHours>
<RegoExpiryDate>05/03/2016</RegoExpiryDate>
<Comments>Moving tomorrow from Daracon BOP to KCE BOP S6A Dam
Cabbie to operate</Comments>
</Submission>
...
</Data>
From there, you can import the two-dimensional XML into a usable rows/columns format. Below are the same import into an MS Access Database and MS Excel spreadsheet. You will notice gaps in the data due to XML content not populating the created nodes (handled in XSLT). A simple SQL cleanup can render final dataset.
Database Import

How to avoid empty xml file

Apologizing in advance if i am posting this in the wrong place. I have a windows batch script that calls a sql query to produce an xml file which gets transferred to a 3rd party. It is scheduled to run every 15 min. My issue is that there are times when there are no records and it creates an empty file. This causes issues when it reaches the 3rd party.
This is the section that is in my .sql file (which runs fine) but I cant figure out how to only produce the xml if there are records in it.
select #cmd = ' bcp "select * from dbo.WelcomeEmail CustomerDetail for xml auto, root(''CustomerDetails''), elements" ' + 'queryout "d:\sample.xml"
I tried this but it gave me an error bcp failed:
select #cmd = ' bcp "select * from dbo.WelcomeEmail where email is not null CustomerDetail for xml auto, root(''CustomerDetails''), elements" ' + 'queryout "d:\sample.xml"
if you need me to provide additional info, please let me know. Thanks!
See this answer: How can I check the size of a file in a Windows batch script?
You could check to see if the file contains zero bytes (or is whatever the size of the file is when there's no data) and then just delete the file.

What’s the easiest way to preview data from an image column?

I have some columns with image data type and I want to preview (or browse) the data in those tables. When I use Select top 1000 rows in SQL Server Management Studio, the value of image columns is displayed in hexadecimal. What’s the easiest way to preview those images since the hex-value is not useful to me?
PS.: database is not under my control, so changing data type is not an option.
If you have LinqPad installed, previewing images is simple. Query your record, convert the binary data to an image, then dump the output to the preview window.
Edit: If you aren't aware, LinqPad is a free utility that can be used for many things, such as a replacement for management studio. Most of the time I use it as a scratch pad for .Net for throw-away programs, test code, and samples.
var entity = // fetch data
using (var ms = new MemoryStream(entity.Image.ToArray()))
{
System.Drawing.Image.FromStream(ms).Dump();
}
Here's what the result looks like:
I would write a proc (or query; see below) to export the binary out to the file system and then use any old off the shelf photo management utility (i.e. Windows Photo Viewer) to take a look at what's inside.
If your clever in your file naming you could give yourself enough information about each image in the name to quickly find it in the database again once you've visually located what your looking for.
Here is a proc that will export binary to the file system. I modified from this sample code. It's untested but should be extremely close for concept. It's using BCP to export your binary. Check here for the full docs on the BCP utility.
The proc also gives you the ability to export everything in the table, or only a single row based on a the passed primarykey. It uses a cursor (yuck), as well as some dynamic sql (yuck, yuck) but sometimes you gotta do what you gotta do.
CREATE PROCEDURE ExportMyImageFiles
(
#PriKey INT,
#OutputFilePath VARCHAR(500)
)
AS
BEGIN
DECLARE #sql VARCHAR(8000)
IF #PriKey IS NULL /* export all images */
BEGIN
DECLARE curExportBinaryImgs CURSOR FAST_FORWARD FOR
SELECT 'BCP "SELECT MyImage FROM [dbo].[MyTable]
WHERE PrimaryKey =' + CAST(PrimaryKey AS VARCHAR(25)) +
'" queryout ' + #OutputFilePath + MyImageName + '.' +
MyImageType + ' -S MyServer\MyInstance -T -fC:\Documents.fmt'
FROM [dbo].[MyTable]
OPEN curExportBinaryImgs
FETCH NEXT FROM curExportBinaryImgs INTO #sql
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC xp_cmdshell #sql, NO_OUTPUT
FETCH NEXT FROM curExportBinaryImgs INTO #sql
END
CLOSE curExportBinaryImgs
DEALLOCATE curExportBinaryImgs
END
ELSE /* Export only the primary key provided */
BEGIN
SELECT #sql = 'BCP "SELECT MyImage FROM [dbo].[MyTable]
WHERE PrimaryKey =' + CAST(PrimaryKey AS VARCHAR(25)) +
'" queryout ' + #OutputFilePath
+ MyImageName + '.' + MyImageType +
' -S MyServer\MyInstance -T -fC:\Documents.fmt'
FROM [dbo].[MyTable]
WHERE PrimaryKey = #PriKey
EXEC xp_cmdshell #sql,NO_OUTPUT
END
END
This is all assuming of course that what is stored in your Image column is actually an image and not some other file type. Hopefully if it is an image you also know the type, bmp, jpg, png, gif, etc.
If you don't want the hassle or reusability of a full blown proc try single query like this:
DECLARE #OutputFilePath VarChar(500) = /* put output dir here */
DECLARE #sql VARCHAR(8000)
DECLARE curExportBinaryImgs CURSOR FAST_FORWARD FOR
SELECT 'BCP "SELECT MyImage FROM [dbo].[MyTable]
WHERE PrimaryKey =' + CAST(PrimaryKey AS VARCHAR(25)) +
'" queryout ' + #OutputFilePath + MyImageName + '.' +
MyImageType + ' -S MyServer\MyInstance -T -fC:\Documents.fmt'
FROM [dbo].[MyTable]
OPEN curExportBinaryImgs
FETCH NEXT FROM curExportBinaryImgs INTO #sql
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC xp_cmdshell #sql, NO_OUTPUT
FETCH NEXT FROM curExportBinaryImgs INTO #sql
END
CLOSE curExportBinaryImgs
DEALLOCATE curExportBinaryImgs
The image type isn't for storing images, it's just 'variable-length binary data'. This type is deprecated and you should now use varbinary(max) for variable length binary data.
Since the SQL Server has no knowledge of what type of binary data has been stored (.zip, .exe, .jpg, .anything) it's not surprising Management Studio doesn't provide a preview.
You definitely can't preview these data types in Managment Studio, but I like the solution given by #RTomas.
There is a really great add-in for SSMS SSMSBoost which which provides plenty of useful features, and of course the simplest way to preview images stored in SQL(at least in my opinion)
NOTE : You must restart SSMS after installing this add-in.
Install it and enjoy previewing images just with : RightClick > Visualize As > Picture
I don't know of a way to accomplish this in Management Studio. You'd probably be better server writing a simple application that can query the database and then convert the hex into the correct image type (.jpg, .png, etc). There are also commercial applications that will do this for you.
Using linqpad the code could become even simpler
Ones you have the entity/type with you on the binary data column you would see .ToImage() method
For my case I am looping through all the rows and publishing all binary columns to images.
Hope it helps.
var yourData_Or_List = // fetch data
DataItem_Or_ListItem.BinaryDataColumn.ToImage().Dump();