Using CHAR(13) in a FOR XML SELECT - sql

I'm trying to use CHAR(13) to force a new line, but the problem is, I'm doing this within a FOR XML Select statement:
SELECT
STUFF((SELECT CHAR(13) + Comment
FROM
myTable
FOR XML PATH ('')) , 1, 1, '')
The problem with this, is that I don't get an actual new line. Instead, I get:
#x0D;
So the data literally looks like this:
#x0D;First Line of Data#x0D;Second Line of Data#x0D;Etc
So I tried to just replace #x0D; with CHAR(13) outside of the FOR XML:
REPLACE(SELECT
STUFF((SELECT CHAR(13) + Comment
FROM
myTable
FOR XML PATH ('')) , 1, 1, '')), '#x0D;', CHAR(13))
This gets me close. It DOES add in the line breaks, but it also includes an & at the end of each line, and the start of each line after the first:
First Line of Data&
&Second Line of Data&
&Etc

Your approach is not real XML:
Try this with "output to text":
DECLARE #tbl TABLE(TestText VARCHAR(100));
INSERT INTO #tbl VALUES('line 1'),('line 2'),('line 3');
SELECT STUFF
(
(
SELECT CHAR(10) + tbl.TestText
FROM #tbl AS tbl
FOR XML PATH('')
),1,1,''
)
With CHAR(13)
#x0D;line 1
line 2
line 3
See that your STUFF just took away the ampersand?
With CHAR(10)
line 1
line 2
line 3
But what you really need is:
SELECT STUFF
(
(
SELECT CHAR(10) + tbl.TestText --you might use 13 and 10 here
FROM #tbl AS tbl
FOR XML PATH(''),TYPE
).value('.','nvarchar(max)'),1,1,''
)
The ,TYPE will return real XML and with .value() you read this properly.
Some background
You have a misconception of "So the data literally looks like this"
It does not "look like this", it is escaped to fit to the rules within XML. And it will be back encoded implicitly, when you read it correctly.
And you have a misconception of line breaks:
In (almost) ancient times you needed a CR = Carriage Return, 13 or x0D to move back the printing sledge and additionally you needed a LF = Line Feed, 10 or x0A to turn the platen to move the paper. Hence the widely used need to have a line break coded with two characters (13/10 or 0D/0A).
Today the ASCII 10 (0A) is often seen alone...
But back to your actual problem: Why do you bother about the look of your data? Within XML some string might look ugly, but - if you read this properly - the decoding back to the original look is done implicitly...
Your residues are not more than part of the encoding as this starts with an ampersand and ends with a semicolon: ≶ or 
. Your attempt to replace this is just one character to short. But anyway: You should not do this!
Just try:
SELECT CAST('<x>Hello</x>' AS XML).value('/x[1]','nvarchar(max)')

Thanks everyone for your help.
The ultimate goal here was to present the data in Excel as part of a report. I'm sure there is a more elegant way to do this, but I at least got the results I wanted by doing this:
REPLACE (
REPLACE(
REPLACE(
(SELECT Comment FROM CallNotes WHERE ForeignId = a.ForeignId FOR XML PATH (''))
, '<Comment>', '')
, '</Comment>', CHAR(13) + CHAR(10))
, '
', '') AS Comments
The select statement all by itself returns XML as we would expect:
<comment>This is a comment</comment><comment>This is another comment</comment>
The inner most REPLACE just gets rid of the opening tag:
<comment>
The middle REPLACE removes the closing tag:
</comment>
and replaces it with CHAR(13) + CHAR(10). And the outer most REPLACE gets rid of this:
(I still don't understand where that's coming from.)
So, when the results are sent to Excel, it looks like this inside the cell:
This is a comment.
This is another comment.
Which is exactly what I want. Again, I'm sure there is a better solution. But this at least is working for now.

I think this is cleaner. Basically start with line feeds (or some other special character) then replace them with carriage returns plus line feeds if you want.
Select REPLACE(STUFF((SELECT CHAR(10) + Comment
FROM myTable FOR XML PATH ('')) , 1, 1, ''),
CHAR(10), CHAR(13)+CHAR(10))

I suppose since you need to group on the FK you can use something like this... just replace #TempT with your table...
Select Pri.ForeignKey,
Replace(Left(Pri.Notes,Len(Pri.Notes)-1),',',CHAR(13)) As Notes
From
(
Select distinct T2.ForeignKey,
(
Select T1.Note + ',' AS [text()]
From #TempT T1
Where T1.ForeignKey = T2.ForeignKey
ORDER BY T1.ForeignKey
For XML PATH ('')
) Notes
From #TempT T2
) Pri
Also in the OP that you listed in the comments, you have a duplicate PrimaryKey. I found that odd. Just a heads up.

If you use below query and results to text option you will see line breaks. Line breaks can't be shown using the results to grid functionality.
SELECT
STUFF((SELECT CHAR(10) + Comment
FROM
myTable
FOR XML PATH ('')) , 1, 1, '')

I suggest Comment + Char(10) + Char(13)
The "Carriage Return" "Line feed" should be at the end of the line.

I believe this could help:
REPLACE(STUFF ((SELECT CHAR(13)+CHAR(10) + Field1 + Field2
FROM
((table
WHERE
field3= 'condition1'
FOR XML PATH ('')), 1, 0, '') , '
' , '')

Related

Seperate XML node by comma in SQL

I have a Column content like this:
CustomTags
<CustomTagsSerialiser>
<custom-tags>
<tag>Visas and travel</tag>
<tag>Explore Options</tag>
<tag>Consider – feasibility</tag>
</custom-tags>
</CustomTagsSerialiser>
I can query g.[CustomTags].value('(/CustomTagsSerialiser//custom-tags)[1]', 'nvarchar(500)') as Custom_Tag to get result like
Visas and travelExplore OptionsConsider – feasibility
But I want the result to have a tag separated by comma (in the same column), like the following:
Visas and travel,Explore Options,Consider – feasibility
Ideally, I would like this to be implemented by using XML functionality/node
instead of breaking it into + ',' + or coalesce
You may refere How Stuff and 'For Xml Path' work in Sql Server this answer.
try below
SELECT
STUFF((SELECT
',' + CTS.tag.value('(.)[1]', 'nvarchar(500)')
FROM
Temp12345
CROSS APPLY
col1.nodes('/CustomTagsSerialiser/custom-tags/tag') AS CTS(tag)
FOR XML PATH('')
), 1, 1, '')
should be this without using cross apply
STUFF((SELECT ',' + x.t.value('.', 'varchar(50)') FROM
[g].CustomTags.nodes('//tag') x(t) FOR XML PATH('')), 1, 1, '') AS 'Custom Tags'

Escape XML special characters upon convert

I have working csv splitter for my needs.
You can just grab and run it as is:
declare #t table(data varchar(max))
insert into #t select 'a,b,c,d'
insert into #t select 'e,,,h'
;with cte(xm) as
(
select convert(xml,'<f><e>' + replace(data,',', '</e><e>') + '</e></f>') as xm
from #t
)
select
xm.value('/f[1]/e[1]','varchar(32)'),
xm.value('/f[1]/e[2]','varchar(32)'),
xm.value('/f[1]/e[3]','varchar(32)'),
xm.value('/f[1]/e[4]','varchar(32)')
from cte
Only issue is, that if I introduce an XML sensitive character in the data, like &:
insert into #t select 'i,j,&,k'
It fails with error: character 24, illegal character
One solution is to replace & character to &amp on the fly, like this:
select convert(xml,'<f><e>' + replace(replace(data,'&','&amp'),',', '</e><e>') + '</e></f>') as xm
but there are several dozens of special XML characters which I need to escape upon convert, and I can't really nest dozens replace(replace(replace(... functions in there. That's what i did and it is messy.
How the above code can be modified to escape XML sensitive characters, and produce the same result?
Thanks!
You have got your answer by Martin Smith already. But I think, it is worth to place an answer here for followers. Want to provide some explanantion and furthermor, the rextester-link might not be reachable in future...
If you think of a string in a table like this ...
DECLARE #mockup TABLE(SomeXMLstring VARCHAR(100));
INSERT INTO #mockup VALUES('This is a string with forbidden characters like "<", ">" or "&"');
-- ... you can easily add XML-tags:
SELECT '<root>' + SomeXMLstring + '</root>'
FROM #mockup ;
--The result would look like XML
<root>This is a string with forbidden characters like "<", ">" or "&"</root>
--But it is not! You can test this, the CAST( AS XML) will fail:
SELECT CAST('<root>This is a string with forbidden characters like "<", ">" or "&"</root>' AS XML);
--Sometimes people try to do their own replaces and start to replace <, > and & with the corresponding entities <, > and &. But this will need a lot of replacements in order to be safe.
--But XML is doing all this for us implicitly
SELECT SomeXMLstring
FROM #mockup
FOR XML PATH('')
--This is the result
<SomeXMLstring>This is a string with forbidden characters like "<", ">" or "&"</SomeXMLstring>
--And the funny thing is: We can easily create a nameless element with AS [*]:
SELECT SomeXMLstring AS [*]
FROM #mockup
FOR XML PATH('')
--The result is the same, but without the tags:
This is a string with forbidden characters like "<", ">" or "&"
--Although this is looking like XML in SSMS, this will be implicitly casted to NVARCHAR(MAX) when used as a string.
--You can use this for implicit escaping of a string wherever you feel the need to build a XML with string concatenation:
SELECT CAST('<root>' + (SELECT SomeXMLstring AS [*] FOR XML PATH('')) + '</root>' AS XML)
FROM #mockup ;
To finally answer your question
This line must use the trick:
select convert(xml,'<f><e>' + replace((SELECT data AS [*] FOR XML PATH('')),',', '</e><e>') + '</e></f>') as xm

Delete spaces and special characters

This question is linked to :concatenate a column on one line depends on a id
I used this query (thanks Mihai) :
SELECT id, date,MAX(docline),
Ids=Stuff((SELECT ' ' + doctext FROM documentation d WHERE d.id=documentation.id
FOR XML PATH (''))
, 1, 1, '' )
from documentation where date in (02/14/2017)
GROUP BY id,date
I received this line :
Backup :
D:\Bas.bac
test testtest
tesdttest testtest
I would like to have something like :
Backup : D:\Bas.bac test testtest tesdttest testtest
So no space and no special characters like ('
')
Like you can see I have some special characters and too much space...
I tried to use RTrim() and LTRIM like :
SELECT id, date,MAX(docline),
Ids=Stuff(RTRIM(SELECT ' ' + doctext FROM documentation d WHERE d.id=documentation.id
FOR XML PATH ('')))
, 1, 0, '' )
from documentation where date in (02/14/2017)
GROUP BY id,date
But I don't see any difference and I have the same behavior.
Thanks,
EDIT : I tried too (without success) :
Stuff((SELECT ' ' + LTRIM(RTRIM(doctext)) FROM documentation d WHERE d.id=documentation.id FOR XML PATH ('')),1,0,'')
If you just need to remove multiple spaces you could try
REPLACE (<string>,' ', '')
But in your case it also remove double space between Backup and :

Using Upper to Capitalize the first letter of City name

I am doing some data clean-up and need to Capitalize the first letter of City names. How do I capitalize the second word in a City Like Terra Bella.
SELECT UPPER(LEFT([MAIL CITY],1))+
LOWER(SUBSTRING([MAIL CITY],2,LEN([MAILCITY])))
FROM masterfeelisting
My results is this 'Terra bella' and I need 'Terra Bella'. Thanks in advance.
Ok, I know I answered this before, but it bugged me that we couldn't write something efficient to handle an unknown amount of 'text segments'.
So re-thinking it and researching, I discovered a way to change the [MAILCITY] field into XML nodes where each 'text segment' is assigned it's own Node within the xml field. Then those xml fields can be processed node by node, concatenated together, and then changed back to a SQL varchar. It's convoluted, but it works. :)
Here's the code:
CREATE TABLE
#masterfeelisting (
[MAILCITY] varchar(max) not null
);
INSERT INTO #masterfeelisting VALUES
('terra bellA')
,(' terrA novA ')
,('chicagO ')
,('bostoN')
,('porT dE sanTo')
,(' porT dE sanTo pallo ');
SELECT
RTRIM
(
(SELECT
UPPER([xmlField].[xmlNode].value('.', 'char(1)')) +
LOWER(STUFF([xmlField].[xmlNode].value('.', 'varchar(max)'), 1, 1, '')) + ' '
FROM [xmlNodeRecordSet].[nodeField].nodes('/N') as [xmlField]([xmlNode]) FOR
xml path(''), type
).value('.', 'varchar(max)')
) as [MAILCITY]
FROM
(SELECT
CAST('<N>' + REPLACE([MAILCITY],' ','</N><N>')+'</N>' as xml) as [nodeField]
FROM #masterfeelisting
) as [xmlNodeRecordSet];
Drop table #masterfeelisting;
First I create a table and fill it with dummy values.
Now here is the beauty of the code:
For each record in #masterfeelisting, we are going to create an xml field with a node for each 'text segment'.
ie. '<N></N><N>terrA</N><N>novA</N><N></N>'
(This is built from the varchar ' terrA novA ')
1) The way this is done is by using the REPLACE function.
The string starts with a '<N>' to designate the beginning of the node. Then:
REPLACE([MAILCITY],' ','</N><N>')
This effectively goes through the whole [MAILCITY] string and replaces each
' ' with '</N><N>'
and then the string ends with a '</N>'. Where '</N>' designates the end of each node.
So now we have a beautiful XML string with a couple of empty nodes and the 'text segments' nicely nestled in their own node. All the 'spaces' have been removed.
2) Then we have to CAST the string into xml. And we will name that field [nodeField]. Now we can use xml functions on our newly created record set. (Conveniently named [xmlNodeRecordSet].)
3) Now we can read the [xmlNodeRecordSet] into the main sub-Select by stating:
FROM [xmlNodeRecordSet].[nodeField].nodes('/N')
This tells us we are reading the [nodeField] as nodes with a '/N' delimiter.
This table of node fields is then parsed by stating:
as [xmlField]([xmlNode]) FOR xml path(''), type
This means each [xmlField] will be parsed for each [xmlNode] in the xml string.
4) So in the main sub-select:
Each blank node '<N></N>' is discarded. (Or not processed.)
Each node with a 'text segment' in it will be parsed. ie <N>terrA</N>
UPPER([xmlField].[xmlNode].value('.', 'char(1)')) +
This code will grab each node out of the field and take its contents '.' and only grab the first character 'char(1)'. Then it will Upper case that character. (the plus sign at the end means it will concatenate this letter with the next bit of code:
LOWER(STUFF([xmlField].[xmlNode].value('.', 'varchar(max)'), 1, 1, ''))
Now here is the beauty... STUFF is a function that will take a string, from a position, for a length, and substitute another string.
STUFF(string, start position, length, replacement string)
So our string is:
[xmlField].[xmlNode].value('.', 'varchar(max)')
Which grabs the whole string inside the current node since it is 'varchar(max)'.
The start position is 1. The length is 1. And the replacement string is ''. This effectively strips off the first character by replacing it with nothing. So the remaining string is all the other characters that we want to have lower case. So that's what we do... we use LOWER to make them all lower case. And this result is concatenated to our first letter that we already upper cased.
But wait... we are not done yet... we still have to append a + ' '. Which adds a blank space after our nicely capitalized 'text segment'. Just in case there is another 'text segment' after this node is done.
This main sub-Select will now parse each node in our [xmlField] and concatenate them all nicely together.
5) But now that we have one big happy concatenation, we still have to change it back from an xml field to a SQL varchar field. So after the main sub-select we need:
.value('.', 'varchar(max)')
This changes our [MAILCITY] back to a SQL varchar.
6) But hold on... we still are not done. Remember we put an extra space at the end of each 'text segment'??? Well the last 'text segment still has that extra space after it. So we need to Right Trim that space off by using RTRIM.
7) And dont forget to rename the final field back to as [MAILCITY]
8) And that's it. This code will take an unknown amount of 'text segments' and format each one of them. All using the fun of XML and it's node parsers.
Hope that helps :)
Here's one way to handle this using APPLY. Note that this solution supports up to 3 substrings (e.g. "Phoenix", "New York", "New York City") but can easily be updated to handle more.
DECLARE #string varchar(100) = 'nEW yoRk ciTY';
WITH DELIMCOUNT(String, DC) AS
(
SELECT #string, LEN(RTRIM(LTRIM(#string)))-LEN(REPLACE(RTRIM(LTRIM(#string)),' ',''))
),
CIPOS AS
(
SELECT *
FROM DELIMCOUNT
CROSS APPLY (SELECT CHARINDEX(char(32), string, 1)) CI1(CI1)
CROSS APPLY (SELECT CHARINDEX(char(32), string, CI1.CI1+1)) CI2(CI2)
)
SELECT
OldString = #string,
NewString =
CASE DC
WHEN 0 THEN UPPER(SUBSTRING(string,1,1))+LOWER(SUBSTRING(string,2,8000))
WHEN 1 THEN UPPER(SUBSTRING(string,1,1))+LOWER(SUBSTRING(string,2,CI1-1)) +
UPPER(SUBSTRING(string,CI1+1,1))+LOWER(SUBSTRING(string,CI1+2,100))
WHEN 2 THEN UPPER(SUBSTRING(string,1,1))+LOWER(SUBSTRING(string,2,CI1-1)) +
UPPER(SUBSTRING(string,CI1+1,1))+LOWER(SUBSTRING(string,CI1+2,CI2-(CI1+1))) +
UPPER(SUBSTRING(string,CI2+1,1))+LOWER(SUBSTRING(string,CI2+2,100))
END
FROM CIPOS;
Results:
OldString NewString
--------------- --------------
nEW yoRk ciTY New York City
This will only capitalize the first letter of the second word. A shorter but less flexible approach. Replace #str with [Mail City].
DECLARE #str AS VARCHAR(50) = 'Los angelas'
SELECT STUFF(#str, CHARINDEX(' ', #str) + 1, 1, UPPER(SUBSTRING(#str, CHARINDEX(' ', #str) + 1, 1)));
This is a way to use imbedded Selects for three City name parts.
It uses CHARINDEX to find the location of your separator character. (ie a space)
I put an 'if' structure around the Select to test if you have any records with more than 3 parts to the city name. If you ever get the warning message, you could add another sub-Select to handle another city part.
Although... just to be clear... SQL is not the best language to do complicated formatting. It was written as a data retrieval engine with the idea that another program will take that data and massage it into a friendlier look and feel. It may be easier to handle the formatting in another program. But if you insist on using SQL and you need to account for city names with 5 or more parts... you may want to consider using Cursors so you can loop through the variable possibilities. (But Cursors are not a good habit to get into. So don't do that unless you've exhausted all other options.)
Anyway, the following code creates and populates a table so you can test the code and see how it works. Enjoy!
CREATE TABLE
#masterfeelisting (
[MAILCITY] varchar(30) not null
);
Insert into #masterfeelisting select 'terra bella';
Insert into #masterfeelisting select ' terrA novA ';
Insert into #masterfeelisting select 'chicagO ';
Insert into #masterfeelisting select 'bostoN';
Insert into #masterfeelisting select 'porT dE sanTo';
--Insert into #masterfeelisting select ' porT dE sanTo pallo ';
Declare #intSpaceCount as integer;
SELECT #intSpaceCount = max (len(RTRIM(LTRIM([MAILCITY]))) - len(replace([MAILCITY],' ',''))) FROM #masterfeelisting;
if #intSpaceCount > 2
SELECT 'You need to account for more than 3 city name parts ' as Warning, #intSpaceCount as SpacesFound;
else
SELECT
cThird.[MAILCITY1] + cThird.[MAILCITY2] + cThird.[MAILCITY3] as [MAILCITY]
FROM
(SELECT
bSecond.[MAILCITY1] as [MAILCITY1]
,SUBSTRING(bSecond.[MAILCITY2],1,bSecond.[intCol2]) as [MAILCITY2]
,UPPER(SUBSTRING(bSecond.[MAILCITY2],bSecond.[intCol2] + 1, 1)) +
SUBSTRING(bSecond.[MAILCITY2],bSecond.[intCol2] + 2,LEN(bSecond.[MAILCITY2]) - bSecond.[intCol2]) as [MAILCITY3]
FROM
(SELECT
SUBSTRING(aFirst.[MAILCITY],1,aFirst.[intCol1]) as [MAILCITY1]
,UPPER(SUBSTRING(aFirst.[MAILCITY],aFirst.[intCol1] + 1, 1)) +
SUBSTRING(aFirst.[MAILCITY],aFirst.[intCol1] + 2,LEN(aFirst.[MAILCITY]) - aFirst.[intCol1]) as [MAILCITY2]
,CHARINDEX ( ' ', SUBSTRING(aFirst.[MAILCITY],aFirst.[intCol1] + 1, LEN(aFirst.[MAILCITY]) - aFirst.[intCol1]) ) as intCol2
FROM
(SELECT
UPPER (LEFT(RTRIM(LTRIM(mstr.[MAILCITY])),1)) +
LOWER(SUBSTRING(RTRIM(LTRIM(mstr.[MAILCITY])),2,LEN(RTRIM(LTRIM(mstr.[MAILCITY])))-1)) as [MAILCITY]
,CHARINDEX ( ' ', RTRIM(LTRIM(mstr.[MAILCITY]))) as intCol1
FROM
#masterfeelisting as mstr -- Initial Master Table
) as aFirst -- First Select Shell
) as bSecond -- Second Select Shell
) as cThird; -- Third Select Shell
Drop table #masterfeelisting;

Using SQL Server to replace line breaks in columns with spaces

I have a large table of data where some of my columns contain line breaks. I would like to remove them and replace them with some spaces instead.
Can anybody tell me how to do this in SQL Server?
Thanks in advance
SELECT REPLACE(REPLACE(#str, CHAR(13), ''), CHAR(10), '')
This should work, depending on how the line breaks are encoded:
update t
set col = replace(col, '
', ' ')
where col like '%
%';
That is, in SQL Server, a string can contain a new line character.
#Gordon's answer should work, but in case you're not sure how your line breaks are encoded, you can use the ascii function to return the character value. For example:
declare #entry varchar(50) =
'Before break
after break'
declare #max int = len(#entry)
; with CTE as (
select 1 as id
, substring(#entry, 1, 1) as chrctr
, ascii(substring(#entry, 1, 1)) as code
union all
select id + 1
, substring(#entry, ID + 1, 1)
, ascii(substring(#entry, ID + 1, 1))
from CTE
where ID <= #max)
select chrctr, code from cte
print replace(replace(#entry, char(13) , ' '), char(10) , ' ')
Depending where your text is coming from, there are different encodings for a line break. In my test string I put the most common.
First I replace all CHAR(10) (Line feed) with CHAR(13) (Carriage return), then all doubled CRs to one CR and finally all CRs to the wanted replace (you want a blank, I put a dot for better visability:
Attention: Switch the output to "text", otherwise you wont see any linebreaks...
DECLARE #text VARCHAR(100)='test single 10' + CHAR(10) + 'test 13 and 10' + CHAR(13) + CHAR(10) + 'test single 13' + CHAR(13) + 'end of test';
SELECT #text
DECLARE #ReplChar CHAR='.';
SELECT REPLACE(REPLACE(REPLACE(#text,CHAR(10),CHAR(13)),CHAR(13)+CHAR(13),CHAR(13)),CHAR(13),#ReplChar);
I have the same issue, means I have a column having values with line breaks in it. I use the query
update `your_table_name` set your_column_name = REPLACE(your_column_name,'\n','')
And this resolves my issue :)
Basically '\n' is the character for Enter key or line break and in this query, I have replaced it with no space (which I want)
Keep Learning :)
zain