I'm trying to convert a string to rows using T-SQL. I've found some people using XML for this but I'm running into troubles.
The original record:
A new line seperated string of data
New In Progress Left Message On Hold Researching Researching (2nd Level) Researching (3rd Level) Resolved Positive False Positive Security Respond
Using the following statement converts this string into XML:
select
cast('<i>'+REPLACE(convert(varchar(max), list_items), CHAR(13) + CHAR(10),'</i><i>')+'</i>' as xml)
from
field
where
column_name = 'state' and table_name = 'sv_inquiry'
XML string:
<i>Unassigned</i><i>Assigned</i><i>Transferred</i><i>Accepted</i><i>Closed</i><i>Reactivated</i>
Now I would like to convert every 'i' node into a separate row. I've constructed the query below, but I can't get it working in the way that it returns all the rows...
select x.i.value('i[1]', 'varchar(30)')
from (
select cast('<i>'+REPLACE(convert(varchar(max), list_items), CHAR(13) + CHAR(10),'</i><i>')+'</i>' as xml)
from field
where column_name='state' and table_name='sv_inquiry'
) x(i)
This will return
Unassigned
To be clear, when i change 'i[1]' into 'i[2]' it will return 'Assigned'. I've tried '.', this will return the whole string in a single record...
How about using the nodes method on an XML datatype.
declare #xml xml
set #xml = '<i>Unassigned</i><i>Assigned</i><i>Transferred</i><i>Accepted</i><i>Closed</i><i>Reactivated</i>'
select
t.c.value('.', 'nvarchar(100)') as [Word]
from
#xml.nodes('/i') as t(c)
You can split a string into rows without XML, see for example the fnSplitString function at SQL Server Central.
Here's an example using the nodes() function of the xml type. I'm using a space as the delimiter because SQL Fiddle doesn't play well with line feeds:
select node_column.value('.', 'varchar(max)')
from (
select cast('<i>' + replace(list_items, ' ', '</i><i>') +
'</i>' as xml) xml_value
from field
) f
cross apply
xml_value.nodes('/i') node_table(node_column);
Live example at SQL Fiddle.
Related
There is a comma separated string in a column which looks like
test=1,value=2.2,system=321
I want to extract value out from the string. I can use select PatIndex('%value=%',columnName) then use left, but this only find the beginning of the patindex.
How to identify the end of pattern value=%, so we can extract the value out?
Chain a few SUBSTRING with CHARINDEX and your PATHINDEX.
DECLARE #text VARCHAR(100) = 'test=1,value=2.21954,system=321'
SELECT
Original = #text,
Parsed = SUBSTRING( -- Get a portion of the original value
#text,
PATINDEX('%value=%',#text) + 6, -- ... starting from the 'value=' (without the 'value=')
-1 + CHARINDEX( -- ... and get as many characters until the first comma
',',
SUBSTRING( -- ... (find the comma starting from the 'value=' onwards)
#text,
PATINDEX('%value=%',#text) + 6,
100)))
Result:
Original Parsed
test=1,value=2.2,system=321 2.2
Note that the CHARINDEX will fail if there is no comma after your value=. You can filter this with a WHERE.
I strongly suggest to store your values already split on a proper table and you wont have to deal with string nightmares like this.
You can use CHARINDEX with starting position to find the first comma after the pattern. CROSS APPLY is used to keep the query easier to read:
WITH tests(str) AS (
SELECT 'test=1,value=2.2,system=321'
)
SELECT str, substring(str, pos1, pos2 - pos1) AS match
FROM tests
CROSS APPLY (SELECT PATINDEX('%value=%', str) + 6) AS ca1(pos1)
CROSS APPLY (SELECT CHARINDEX(',', str, pos1 + 1)) AS ca2(pos2)
-- 2.2
First of all, don't store denormalized data in this way, if you want to query them. SQL, the language, isn't good at string manipulation. Parsing and splitting strings can't take advantage of indexes either, which means any query that tried to find eg all records that refer to system 321 would have to scan and parse all rows.
SQL Server 2016 and JSON
SQL Server 2016 added suppor for JSON and the STRING_SPLIT function. Earlier versions already provided the XML type. It's better to store complex values as JSON or XML instead of trying to parse the string.
One option is to convert the string into a JSON object and retrieve the value contents, eg :
DECLARE #text VARCHAR(100) = 'test=1,value=2.2,system=321'
select json_value('{"' + replace(replace(#text,',','","'),'=','":"') + '"}','$.value')
This returns 2.2.
The replacements converted the original string into
{"test":"1","value":"2.2","system":"321"}
JSON_VALUE(#json,'$.') will return the value property of that object
Earlier SQL Server versions
In earlier SQL Server version, you can convert that string into an XML element the same way and use XQuery :
DECLARE #text VARCHAR(100) = 'test=1,value=2.2,system=321';
declare #xml varchar(100)='<r ' + replace(replace(#text,',','" '),'=',' ="') + '" />';
select #xml
select cast(#xml as xml).value('(/r[1]/#value)','varchar(20)')
In this case #xml contains :
<r test ="1" value ="2.2" system ="321" />
The query result is 2.2
You can try like following.
DECLARE #xml AS XML
SELECT #xml = Cast(( '<X>' + Replace(txt, ',', '</X><X>') + '</X>' ) AS XML)
FROM (VALUES ('test=1,value=2.2,system=321')) v(txt)
SELECT LEFT(value, Charindex('=', value) - 1) AS LeftPart,
RIGHT(value, Charindex('=', Reverse(value)) - 1) AS RightPart
FROM (SELECT n.value('.', 'varchar(100)') AS value
FROM #xml.nodes('X') AS T(n))T
Online Demo
Output
+----------+-----------+
| LeftPart | RightPart |
+----------+-----------+
| test | 1 |
+----------+-----------+
| value | 2.2 |
+----------+-----------+
| system | 321 |
+----------+-----------+
You can try the below query if you are using SQL Server (2016 or above)
SELECT RIGHT(Value,CHARINDEX('=',REVERSE(Value))-1) FROM YourTableName
CROSS APPLY STRING_SPLIT ( ColumnName , ',' )
WHERE Value Like 'Value=%'
I have a string column in my table that contains 'Character-separated' data such as this:
"Value|Data|4|Z|11/06/2012"
This data is fed into a 'parser' and deserialised into a particular object. (The details of this aren't relevant and can't be changed)
The structure of my object has changed and now I would like to get rid of some of the 'sections' of data
So I want the previous value to turn into this
"Value|Data|11/06/2012"
I was hoping I might be able to get some help on how I would go about doing this in T-SQL.
The data always has the same number of sections, 'n' and I will want to remove the same sections for all rows , 'n-x and 'n-y'
So far I know I need an update statement to update my column value.
I've found various ways of splitting a string but I'm struggling to apply it to my scenario.
In C# I would do
string RemoveSecitons(string value)
{
string[] bits = string.split(value,'|');
List<string> wantedBits = new List<string>();
for(var i = 0; i < bits.Length; i++)
{
if ( i==2 || i==3) // position of sections I no longer want
{
continue;
}
wantedBits.Add(bits[i]);
}
return string.Join(wantedBits,'|');
}
But how I would do this in SQL I'm not sure where to start. Any help here would be appreciated
Thanks
Ps. I need to run this SQL on SQL Server 2012
Edit: It looks like parsing to xml in some manner could be a popular answer here, however I can't guarantee my string won't have characters such as '<' or '&'
Using NGrams8K you can easily write a nasty fast customized splitter. The logic here is based on DelimitedSplit8K. This will likely outperform even the C# code you posted.
DECLARE #string VARCHAR(8000) = '"Value|Data|4|Z|11/06/2012"',
#delim CHAR(1) = '|';
SELECT newString =
(
SELECT SUBSTRING(
#string, split.pos+1,
ISNULL(NULLIF(CHARINDEX(#delim,#string,split.pos+1),0),8000)-split.pos)
FROM
(
SELECT ROW_NUMBER() OVER (ORDER BY d.Pos), d.Pos
FROM
(
SELECT 0 UNION ALL
SELECT ng.position
FROM samd.ngrams8k(#string,1) AS ng
WHERE ng.token = #delim
) AS d(Pos)
) AS split(ItemNumber,Pos)
WHERE split.ItemNumber IN (1,2,5)
ORDER BY split.ItemNumber
FOR XML PATH('')
);
Returns:
newString
----------------------------
"Value|Data|11/06/2012"
Not the most elegant way, but works:
SELECT SUBSTRING(#str,1, CHARINDEX('|',#str,CHARINDEX('|',#str,1)+1)-1)
+ SUBSTRING(#str, CHARINDEX('|',#str,CHARINDEX('|',#str,CHARINDEX('|',#str,CHARINDEX('|',#str,1)+1)+1)+1), LEN(#str))
----------------------
Value|Data|11/06/2012
You might try some XQuery:
DECLARE #s VARCHAR(100)='Value|Data|4|Z|11/06/2012';
SELECT CAST('<x>' + REPLACE(#s,'|','</x><x>') + '</x>' AS XML)
.value('concat(/x[1],"|",/x[2],"|",/x[5])','nvarchar(max)');
In short: The value is trasformed to XML by some string replacements. Then we use the XQuery-concat to bind the first, the second and the fifth element together again.
This version is a bit less efficient but safe with forbidden characters:
SELECT CAST('<x>' + REPLACE((SELECT #s AS [*] FOR XML PATH('')),'|','</x><x>') + '</x>' AS XML)
.value('concat(/x[1],"|",/x[2],"|",/x[5])','nvarchar(max)')
Just to add a non-xml option for fun:
Edit and Caveat - In case anyone tries this for a different solution and doesn't read the comments...
HABO rightly noted that this is easily broken if any of the columns have a period (".") in them. PARSENAME is dependent on a 4 part naming structure and will return NULL if that is exceeded. This solution will also break if any values ever contain another pipe ("|") or another delimited column is added - the substring in my answer is specifically there as a workaround for the dependency on the 4 part naming. If you are trying to use this solution on, say, a variable with 7 delimited columns, it would need to be reworked or scrapped in favor of one of the other answers here.
DECLARE
#a VARCHAR(100)= 'Value|Data|4|Z|11/06/2012'
SELECT
PARSENAME(REPLACE(SUBSTRING(#a,0,LEN(#a)-CHARINDEX('|',REVERSE(#a))+1),'|','.'),4)+'|'+
PARSENAME(REPLACE(SUBSTRING(#a,0,LEN(#a)-CHARINDEX('|',REVERSE(#a))+1),'|','.'),3)+'|'+
SUBSTRING(#a,LEN(#a)-CHARINDEX('|',REVERSE(#a))+2,LEN(#a))
Here is a quick way to do it.
CREATE FUNCTION [dbo].StringSplitXML
(
#String VARCHAR(MAX), #Separator CHAR(1)
)
RETURNS #RESULT TABLE(id int identity(1,1),Value VARCHAR(MAX))
AS
BEGIN
DECLARE #XML XML
SET #XML = CAST(
('<i>' + REPLACE(#String, #Separator, '</i><i>') + '</i>')
AS XML)
INSERT INTO #RESULT
SELECT t.i.value('.', 'VARCHAR(MAX)')
FROM #XML.nodes('i') AS t(i)
WHERE t.i.value('.', 'VARCHAR(MAX)') <> ''
RETURN
END
GO
SELECT * FROM dbo.StringSplitXML( 'Value|Data|4|Z|11/06/2012','|')
WHERE id not in (3,4)
Note that using a UDF will slow things down, so this solution should be considered only if you have a reasonably small data set to work with.
I want to remove leading zeros for a varchar column. Actually we are storing version information in a column. Find below example versions.
2.00.001
The output would be : 2.0.1
Input : 2.00.00.001
The output would be: 2.0.0.1
Input : 2.00
The output would be : 2.0
The dots in the version column not constant. It may be two or three or four
I found some solutions in google but those are not working. Find below are the queries I tried.
SELECT SUBSTRING('2.00.001', PATINDEX('%[^0 ]%', '2.00.001' + ' '), LEN('2.00.001'))
SELECT REPLACE(LTRIM(REPLACE('2.00.001', '0', ' ')),' ', '0')
Please suggest me the best approach in sqlserver.
One way is to use a string splitting function with cross apply, for xml path, and stuff.
For an explanation on how stuff and for xml works together to concatenate a string from selected rows, read this SO post.
Using a string splitting function will enable you to convert each number part of the string to int, that will remove the leading zeroes. Executing a select statement on the result of the string splitting function will enable you to get your int values back into a varchar value, seperated by dot.
The stuff function will remove the first dot.
Create the string splitting function:
CREATE FUNCTION SplitStrings_XML
(
#List NVARCHAR(MAX),
#Delimiter NVARCHAR(255)
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
SELECT Item = y.i.value('(./text())[1]', 'nvarchar(4000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(#List, #Delimiter, '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
);
GO
I've chosen to use an xml based function because it's fairly simple. If you are using 2016 version you can use the built in string_split function. For earlier versions, I would stronly suggest reading Aaron Bertrand's Split strings the right way – or the next best way.
Create and populate sample table (Please save us this step in your future questions)
DECLARE #T AS TABLE
(
col varchar(20)
)
INSERT INTO #T VALUES
('2.00.001'),
('2.00.00.001'),
('2.00')
The query:
SELECT col, result
FROM #T
CROSS APPLY
(
SELECT STUFF(
(
SELECT '.' + CAST(CAST(Item as int) as varchar(20))
FROM SplitStrings_XML(col, '.')
FOR XML PATH('')
)
, 1, 1, '') As result
) x
Results:
col result
2.00.001 2.0.1
2.00.00.001 2.0.0.1
2.00 2.0
You can see it in action on this link on rextester
No need for Split/Parse Function, and easy to expand if there could be more than 5 groups
Declare #YourTable table (YourCol varchar(25))
Insert Into #YourTable Values
('2.00.001'),
('2.00.00.001'),
('2.00')
Update #YourTable
Set YourCol = concat(Pos1,'.'+Pos2,'.'+Pos3,'.'+Pos4,'.'+Pos5)
From #YourTable A
Cross Apply (
Select Pos1 = ltrim(rtrim(xDim.value('/x[1]','int')))
,Pos2 = ltrim(rtrim(xDim.value('/x[2]','int')))
,Pos3 = ltrim(rtrim(xDim.value('/x[3]','int')))
,Pos4 = ltrim(rtrim(xDim.value('/x[4]','int')))
,Pos5 = ltrim(rtrim(xDim.value('/x[5]','int')))
From (Select Cast('<x>' + replace((Select replace(A.YourCol,'.','§§Split§§') as [*] For XML Path('')),'§§Split§§','</x><x>')+'</x>' as xml) as xDim) as A
) B
Select * from #YourTable
Returns
YourCol
2.0.1
2.0.0.1
2.0
Easy, fast, compatible and readable way – without tables or XML tricks.
Correctly handles all cases including empty string, NULL, or numbers like 00100.
Supports unlimited number of groups. Runs on all SQL Server versions.
Step 1: Remove leading zeros from all groups.
Step 2: Place single zero to groups where no digits remained.
[Edit: Not sure why it was downvoted twice. Check the solution: ]
The function:
CREATE FUNCTION dbo.fncGetNormalizedVersionNumber(#Version nvarchar(200))
RETURNS nvarchar(200) AS
BEGIN
-- Preprocessing: Surround version string by dots so all groups have the same format.
SET #Version = '.' + #Version + '.';
-- Step 1: Remove any leading zeros from groups as long as string length decreases.
DECLARE #PreviousLength int = 0;
WHILE #PreviousLength <> LEN(#Version)
BEGIN
SET #PreviousLength = LEN(#Version);
SET #Version = REPLACE(#Version, '.0', '.');
END;
-- Step 2: Insert 0 to any empty group as long as string length increases.
SET #PreviousLength = 0;
WHILE #PreviousLength <> LEN(#Version)
BEGIN
SET #PreviousLength = LEN(#Version);
SET #Version = REPLACE(#Version, '..', '.0.');
END;
-- Strip leading and trailing dot added by preprocessing.
RETURN SUBSTRING(#Version, 2, LEN(#Version) - 2);
END;
Usage:
SELECT dbo.fncGetNormalizedVersionNumber('020.00.00.000100');
20.0.0.100
Performance per 100,000 calculations:
solution using helper function + helper tables + XML: 54519 ms
this solution (used on table column): 2574 ms (→ 21 times faster) (UPDATED after comment.)
For SQL Server 2016:
SELECT
STUFF
((SELECT
'.' + CAST(CAST(value AS INT) AS VARCHAR)
FROM STRING_SPLIT('2.00.001', '.')
FOR XML PATH (''))
, 1, 1, '')
According to this: https://sqlperformance.com/2016/03/sql-server-2016/string-split
It's the fastest way :)
Aaron Bertrand knows it's stuff.
For an interesting and deep read about splitting strings on SQL Server plese read this gem of knowledge: http://www.sqlservercentral.com/articles/Tally+Table/72993/
It has some clever strategies
I am not sure this is what you are looking for but you can give a go, it should handle up to 4 zeros.
DECLARE #VERSION NVARCHAR(20) = '2.00.00.001'
SELECT REPLACE(REPLACE(REPLACE(#VERSION, '0000','0'),'000','0'),'00','0')
2.0.0.01
SET #VERSION = '2.00.00.01'
SELECT REPLACE(REPLACE(REPLACE(#VERSION, '0000','0'),'000','0'),'00','0')
2.0.0.01
SET #VERSION = '2.000.0000.0001'
SELECT REPLACE(REPLACE(REPLACE(#VERSION, '0000','0'),'000','0'),'00','0')
2.0.0.01
Try this one
SUBSTRING(str_col, PATINDEX('%[^0]%', str_col+'.'), LEN(str_col))
Here is another sample:
CREATE TABLE #tt(s VARCHAR(15))
INSERT INTO #tt VALUES
('2.00.001'),
('2.00.00.001'),
('2.00')
SELECT t.s,STUFF(c.s,1,1,'') AS news FROM #tt AS t
OUTER APPLY(
SELECT '.'+LTRIM(z.n) FROM (VALUES(CONVERT(XML,'<n>'+REPLACE(t.s,'.','</n><n>')+'</n>'))) x(xs)
CROSS APPLY(SELECT n.value('.','int') FROM x.xs.nodes('n') AS y(n)) z(n)
FOR XML PATH('')
) c(s)
s news
--------------- -----------
2.00.001 2.0.1
2.00.00.001 2.0.0.1
2.00 2.0
I have several data concatenated in one cell delimited by "," separator. Below is the screen shot for data and output i required. I know how to convert from output to concatenate format by using For XML but i am unable to convert concatenate to the output format.
I am using Sql server 2008. Kindly help to accomplish this.
Regards,
Ratan
as you have composite string in both columns then I prefer to use cross joins with convert to xml, twice :
SELECT MemberId = y.i.value('(./text())[1]', 'nvarchar(1000)'),
TokenId = u.j.value('(./text())[1]', 'nvarchar(1000)')
FROM
(
SELECT
m = CONVERT(XML, '<i>'
+ REPLACE(MemberId, ',' , '</i><i>')
+ '</i>').query('.'),
t= CONVERT(XML, '<j>'
+ REPLACE(TokenId, ',' , '</j><j>')
+ '</j>').query('.')
FROM member_tokens
) AS a
CROSS APPLY m.nodes('i') AS y(i)
CROSS APPLY t.nodes('j') AS u(j)
SQLFIDDLE DEMO
So I have this bit of sql that grabs a XML nodes whose content is encoded html. I then converted that into a varchar and "decode" it and cast it back into an XML data type. The problem that I have is when I call the nodes function it says that "nodes" is not a valid function, property, or field.. The weird thing is that if use the other functions query, value,exist, and modify it does not complain. Any ideas as to why?
Declare #XmlEl As XML
DECLARE #htmlString As varchar(max)
Select #XmlEl = CAST(replace(CAST(Body AS VARCHAR(MAX)), 'utf-16', 'utf-8') AS xml) FROM Templates where TemplateID = 3119
Set #XmlEl = #XmlEl.query('/PdfTemplate/PdfBody').query('string(/)')
Select CAST(#XmlEl As varchar(max))
Set #htmlString = Replace(CAST(#XmlEl As varchar(max)), '<', '<')
Set #XmlEl = CAST(Replace(#htmlString, '>', '>') AS XML)
Select #XmlEl.nodes('p/span')
I do not think you can just do a SELECT off of #XmlEl.nodes like that. I believe you need something more like
Select x.a.query('.') FROM #XmlEl.nodes('p/span') x(a)
Referenced from MSDN nodes()