Concatenate sql values to a variable - sql

On a SQL Server 2008 I'm trying to get a comma separated list of all selected values into a variable.
SELECT field
FROM table
returns:
+-------+
| field |
+-------+
| foo |
+-------+
| bar |
+-------+
I'd like to get:
"foo, bar, "
I tried:
DECLARE #foo NVARCHAR(MAX)
SET #foo = ''
SELECT #foo = #foo + field + ','
FROM TABLE
PRINT #foo
Which returns nothing. What am I doing wrong?

You'll need to change NULLs
SELECT #foo = #foo + ISNULL(field + ',', '')
FROM TABLE
or remove them
SELECT #foo = #foo + field + ','
FROM TABLE
WHERE field IS NOT NULL

That happens if you have even a SINGLE field in the table that is NULL. In SQL Server, NULL + <any> = NULL. Either omit them
SELECT #foo = #foo + field + ','
FROM TABLE
WHERE field is not null
Or work around them
SELECT #foo = #foo + isnull(field + ',', '')
FROM TABLE
You can write the whole thing without the leading SET statement which is more common. This query below returns "foo,bar" with no trailing comma
DECLARE #foo NVARCHAR(MAX)
SELECT #foo = isnull(#foo + ',', '') + field
FROM TABLE
WHERE field is not null
PRINT #foo

Don't forget to use LTRIM and RTRIM around #foo (when data type is char/varchar) in the concatenation other it will not give expected results in SQL 2008 R2.

As per the comment Lukasz Szozda made on one of the answers here, you should not use your indicated method to aggregate string values in SQL Server, as this is not supported functionality. While this tends to work when no order clause is used (and even if no exception to this tendency has ever been documented), Microsoft does not guarantee that this will work, and there's always a chance it could stop working in the future. SQL is a declarative language; you cannot assume that behaviour that is not explicitly defined as being the correct behaviour for interpreting a given statement will continue working.
Instead, see the examples below, or see this page for a review of valid ways to achieve the same result, and their respective performance: Optimal way to concatenate/aggregate strings
Doing it in a valid way, whichever way you end up using, still has the same considerations as in the other answers here. You either need to exclude NULL values from your result set or be explicit about how you want them to be added to the resulting string.
Further, you should probably use some kind of explicit ordering so that this code is deterministic - it can cause all sorts of problems down the line if code like this can produce a different result when running on the same data, which may happen without an explicit ordering specified.
--Null values treated as empty strings
SET #Foo =
STUFF /*Stuff is used to remove the seperator from the start of the string*/
( (SELECT N','/*separator*/ + ISNULL(RTRIM(t.Field), '' /*Use an emptry string in the place of NULL values*/) /*Thing to List*/
FROM TABLE t
ORDER BY t.SomeUniqueColumn ASC /*Make the query deterministic*/
FOR XML PATH, TYPE).value(N'.[1]',N'varchar(max)')
,1
,1 /*Length of separator*/
,N'');
--Null values excluded from result
SET #Foo =
STUFF /*Stuff is used to remove the seperator from the start of the string*/
( (SELECT N','/*separator*/ + RTRIM(t.Field) /*Thing to List*/
FROM TABLE t
WHERE t.Field IS NOT NULL
ORDER BY t.SomeUniqueColumn ASC /*Make the query deterministic*/
FOR XML PATH, TYPE).value(N'.[1]',N'varchar(max)')
,1
,1 /*Length of separator*/
,N'');

Related

T-SQL get substring

I am looking to get an order number from a column named KEY_Ref, this ref column have various contents, but some rows look like this
LINE_NO=15^ORDER_NO=176572^RELEASE_NO=1^
Now I am interested in getting the value for ORDER_NO (176572 in this case)
How would I (In SQL Server) go about getting this (Or other) value from the main string
The logic is always
key1=value1^key2=value2^key3=value3^
You can use string_split():
select t.*, s.orderno
from t outer apply
(select stuff(s.value, 1, 9, '') as orderno
from string_split(t.key_ref, '^') s
where s.value like 'ORDER_NO=%'
) s;
Here is a db<>fiddle.
this is going to be a bit lengthy answer however if your SQL server version doesn't support string_split function you may use this.
declare #str varchar(100) = 'LINE_NO=15^ORDER_NO=176572^RELEASE_NO=1^'
declare #substr1 varchar(50) = substring(#str,charindex('^',#str)+1,len(#str))
declare #substr2 varchar(50) = substring(#substr1,charindex('=',#substr1)+1,charindex('^',#substr1)-charindex('=',#substr1)-1)
select #substr2 as 'order number'
the final variable will produce the desired value and you must merge the above queries to a single query that can fetch the value from the table in a single select statement.
this will work only if the pattern doesn't deviate from the one you've mentioned.

How might I concatenate all values in a row into a string?

Suppose I have a row of data, store such as the following:
------------------------
| Col 1 | Col 2 | Col 3 |
|------------------------|
| Foo | Bar | Foobar |
How might I concatinate this into a single string, such as the below?
Foo-Bar-Foobar
The column headings (and number of column headings) in this table will not be known, so selecting by column name is not an option(?).
Please note that I am not trying to concatinate a list of values in a column, I am trying to concatinate the values stores in one single row. I would also prefer to avoid using pivots, as I will be working with large sets of data and do not want to take the hit to performance.
In such cases I really adore the mighty abilities of XML in dealing with generic sets:
SELECT STUFF(b.query('
for $element in ./*
return
<x>;{$element/text()}</x>
').value('.','nvarchar(max)'),1,1,'')
FROM
(
SELECT TOP 3 * FROM sys.objects o FOR XML PATH('row'),ELEMENTS XSINIL,TYPE
) A(a)
CROSS APPLY a.nodes('/row') B(b);
The result
sysrscols;3;4;0;S ;SYSTEM_TABLE;2017-08-22T19:38:02.860;2017-08-22T19:38:02.867;1;0;0
sysrowsets;5;4;0;S ;SYSTEM_TABLE;2009-04-13T12:59:05.513;2017-08-22T19:38:03.197;1;0;0
sysclones;6;4;0;S ;SYSTEM_TABLE;2017-08-22T19:38:03.113;2017-08-22T19:38:03.120;1;0;0
Remarks
Some things to mention
I use the ; as delimiter, as the - might break with values containing hyphens (e.g. DATE)
I use TOP 3 from sys.objects to create an easy-cheesy-stand-alone sample
Thx to Zohard Peled I added ELEMENTS XSINIL to force the engine not to omit NULL values.
UPDATE Create JSON in pre-2016 versions
You can try this to create a JSON-string in versions before 2016
SELECT '{'
+ STUFF(b.query('
for $element in ./*
return
<x>,"{local-name($element)}":"{$element/text()}"</x>
').value('.','nvarchar(max)'),1,1,'')
+ '}'
FROM
(
SELECT TOP 3 * FROM sys.objects o FOR XML PATH('row'),TYPE
) A(a)
CROSS APPLY a.nodes('/row') B(b);
The result
{"name":"sysrscols","object_id":"3","schema_id":"4","parent_object_id":"0","type":"S ","type_desc":"SYSTEM_TABLE","create_date":"2017-08-22T19:38:02.860","modify_date":"2017-08-22T19:38:02.867","is_ms_shipped":"1","is_published":"0","is_schema_published":"0"}
{"name":"sysrowsets","object_id":"5","schema_id":"4","parent_object_id":"0","type":"S ","type_desc":"SYSTEM_TABLE","create_date":"2009-04-13T12:59:05.513","modify_date":"2017-08-22T19:38:03.197","is_ms_shipped":"1","is_published":"0","is_schema_published":"0"}
{"name":"sysclones","object_id":"6","schema_id":"4","parent_object_id":"0","type":"S ","type_desc":"SYSTEM_TABLE","create_date":"2017-08-22T19:38:03.113","modify_date":"2017-08-22T19:38:03.120","is_ms_shipped":"1","is_published":"0","is_schema_published":"0"}
Hint
You might add ELEMENTS XSINIL to this query as well. This depends, if you'd like NULLs to simply miss, or if you want to include them as "SomeColumn":""
I use UnitE and this is what I would use to select the columns dynamically from the person table.
INFORMATION_SCHEMA.COLUMNS stores the column list for the table and the SELECT statement is built around that.
Declare #Columns NVARCHAR(MAX)
Declare #Table varchar(15) = 'capd_person'
SELECT #Columns=COALESCE(#Columns + ',', '') + COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (TABLE_NAME=#Table )
EXEC('SELECT DISTINCT ' + #Columns + ' FROM ' + #Table)
You would need to change the EXEC command to suit your needs, using CONCAT as described before.
Simply do a SELECT CONCAT(col1,col2,col3) FROM table
However if you wish to make it neat
Use:
SELECT CONCAT(col1,'-',col2,'-',col3) FROM table.
Find more help here.
An improved version of JonTout's answer:
Declare #Columns NVARCHAR(MAX)
Declare #Table varchar(15) = 'TableName'
SELECT #Columns=COALESCE(#Columns + '+', '') +'CONVERT(varchar(max),ISNULL('+ COLUMN_NAME+',''''))+''-'''
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (TABLE_NAME=#Table )
EXEC('SELECT ' + #Columns + ' FROM ' + #Table)

Understanding the Syntax for COALESCE

I am trying to understand the below syntax. Can I get some help with this.
DECLARE #StringList VARCHAR(2500);
SELECT COALESCE(#StringList + ',','') + CAST(apID as VARCHAR) AS ApIdList FROM testTable
As a result you will get all apID from testTable in VARCHAR
COALESCE
checks if first parameter is NULL then the second parameter will returned. In this line #StringList is always equals NULL
COALESCE(#StringList + ',','')
So, NULL + ',' = NULL and you will get empty string ('')
Then empty string + CAST(apID as VARCHAR) and you will get apID as VARCHAR
Coalesce returns the first non-null element provided in the list supplied. See - https://msdn.microsoft.com/en-us/library/ms190349.aspx.
In your case, if #StringList is not null, then it's contents will be prepended with a comma to appID for each row in testTable.
Your code is returning ApID as a string for all rows in the table. Why? Because #StringList is NULL so the first expression evaluates to '' and the second to a string representation of the ApId in some row in the table.
I caution you about the conversion to VARCHAR with no length. Don't do this! The default length varies by context, and you can introduce very hard-to-debug errors without a length.
A related expression is more common, I think:
SELECT #StringList = COALESCE(#StringList + ',', '') + CAST(apID as VARCHAR(8000)) AS ApIdList
FROM testTable;
This does string concatenation, so all the values of apID are concatenated together in a comma-delimited string.
What this is doing is looping on the result set to assign the variable. This type of assignment of a variable across multiple rows is discouraged. I don't think that SQL Server guarantees that it actually works (i.e. is a documented feature), but it appears to work in practice across all versions.

Execute table valued function from row values

Given a table as below where fn contains the name of an existing table valued functions and param contains the param to be passed to the function
fn | param
----------------
'fn_one' | 1001
'fn_two' | 1001
'fn_one' | 1002
'fn_two' | 1002
Is there a way to get a resulting table like this by using set-based operations?
The resulting table would contain 0-* lines for each line from the first table.
param | resultval
---------------------------
1001 | 'fn_one_result_a'
1001 | 'fn_one_result_b'
1001 | 'fn_two_result_one'
1002 | 'fn_two_result_one'
I thought I could do something like (pseudo)
select t1.param, t2.resultval
from table1 t1
cross join exec sp_executesql('select * from '+t1.fn+'('+t1.param+')') t2
but that gives a syntax error at exec sp_executesql.
Currently we're using cursors to loop through the first table and insert into a second table with exec sp_executesql. While this does the job correctly, it is also the heaviest part of a frequently used stored procedure and I'm trying to optimize it. Changes to the data model would probably imply changes to most of the core of the application and that would cost more then just throwing hardware at sql server.
I believe that this should do what you need, using dynamic SQL to generate a single statement that can give you your results and then using that with EXEC to put them into your table. The FOR XML trick is a common one for concatenating VARCHAR values together from multiple rows. It has to be written with the AS [text()] for it to work.
--=========================================================
-- Set up
--=========================================================
CREATE TABLE dbo.TestTableFunctions (function_name VARCHAR(50) NOT NULL, parameter VARCHAR(20) NOT NULL)
INSERT INTO dbo.TestTableFunctions (function_name, parameter)
VALUES ('fn_one', '1001'), ('fn_two', '1001'), ('fn_one', '1002'), ('fn_two', '1002')
CREATE TABLE dbo.TestTableFunctionsResults (function_name VARCHAR(50) NOT NULL, parameter VARCHAR(20) NOT NULL, result VARCHAR(200) NOT NULL)
GO
CREATE FUNCTION dbo.fn_one
(
#parameter VARCHAR(20)
)
RETURNS TABLE
AS
RETURN
SELECT 'fn_one_' + #parameter AS result
GO
CREATE FUNCTION dbo.fn_two
(
#parameter VARCHAR(20)
)
RETURNS TABLE
AS
RETURN
SELECT 'fn_two_' + #parameter AS result
GO
--=========================================================
-- The important stuff
--=========================================================
DECLARE #sql VARCHAR(MAX)
SELECT #sql =
(
SELECT 'SELECT ''' + T1.function_name + ''', ''' + T1.parameter + ''', F.result FROM ' + T1.function_name + '(' + T1.parameter + ') F UNION ALL ' AS [text()]
FROM
TestTableFunctions T1
FOR XML PATH ('')
)
SELECT #sql = SUBSTRING(#sql, 1, LEN(#sql) - 10)
INSERT INTO dbo.TestTableFunctionsResults
EXEC(#sql)
SELECT * FROM dbo.TestTableFunctionsResults
--=========================================================
-- Clean up
--=========================================================
DROP TABLE dbo.TestTableFunctions
DROP TABLE dbo.TestTableFunctionsResults
DROP FUNCTION dbo.fn_one
DROP FUNCTION dbo.fn_two
GO
The first SELECT statement (ignoring the setup) builds a string which has the syntax to run all of the functions in your table, returning the results all UNIONed together. That makes it possible to run the string with EXEC, which means that you can then INSERT those results into your table.
A couple of quick notes though... First, the functions must all return identical result set structures - the same number of columns with the same data types (technically, they might be able to be different data types if SQL Server can always do implicit conversions on them, but it's really not worth the risk). Second, if someone were able to update your functions table they could use SQL injection to wreak havoc on your system. You'll need that to be tightly controlled and I wouldn't let users just enter in function names, etc.
You cannot access objects by referencing their names in a SQL statement. One method would be to use a case statement:
select t1.*,
(case when fn = 'fn_one' then dbo.fn_one(t1.param)
when fn = 'fn_two' then dbo.fn_two(t1.param)
end) as resultval
from table1 t1 ;
Interestingly, you could encapsulate the case as another function, and then do:
select t1.*, dbo.fn_generic(t1.fn, t1.param) as resultval
from table1 t1 ;
However, in SQL Server, you cannot use dynamic SQL in a user-defined function (defined in T-SQL), so you would still need to use case or similar logic.
Either of these methods is likely to be much faster than a cursor, because they do not require issuing multiple queries.

String manipulation SQL

I have a row of strings that are in the following format:
'Order was assigned to lastname,firsname'
I need to cut this string down into just the last and first name but it is always a different name for each record.
The 'Order was assigned to' part is always the same.......
Thanks
I am using SQL Server. It is multiple records with different names in each record.
In your specific case you can use something like:
SELECT SUBSTRING(str, 23) FROM table
However, this is not very scalable, should the format of your strings ever change.
If you are using an Oracle database, you would want to use SUBSTR instead.
Edit:
For databases where the third parameter is not optional, you could use SUBSTRING(str, 23, LEN(str))
Somebody would have to test to see if this is better or worse than subtraction, as in Martin Smith's solution but gives you the same result in the end.
In addition to the SUBSTRING methods, you could also use a REPLACE function. I don't know which would have better performance over millions of rows, although I suspect that it would be the SUBSTRING - especially if you were working with CHAR instead of VARCHAR.
SELECT REPLACE(my_column, 'Order was assigned to ', '')
For SQL Server
WITH testData AS
(
SELECT 'Order was assigned to lastname,firsname' as Col1 UNION ALL
SELECT 'Order was assigned to Bloggs, Jo' as Col1
)
SELECT SUBSTRING(Col1,23,LEN(Col1)-22) AS Name
from testData
Returns
Name
---------------------------------------
lastname,firsname
Bloggs, Jo
on MS SQL Server:
declare #str varchar(100) = 'Order was assigned to lastname,firsname'
declare #strLen1 int = DATALENGTH('Order was assigned to ')
declare #strLen2 int = len(#str)
select #strlen1, #strLen2, substring(#str,#strLen1,#strLen2),
RIGHT(#str, #strlen2-#strlen1)
I would require that a colon or some other delimiter be between the message and the name.
Then you could just search for the index of that character and know that anything after it was the data you need...
Example with format changing over time:
CREATE TABLE #Temp (OrderInfo NVARCHAR(MAX))
INSERT INTO #Temp VALUES ('Order was assigned to :Smith,Mary')
INSERT INTO #Temp VALUES ('Order was assigned to :Holmes,Larry')
INSERT INTO #Temp VALUES ('New Format over time :LootAt,Me')
SELECT SUBSTRING(OrderInfo, CHARINDEX(':',OrderInfo)+1, LEN(OrderInfo))
FROM #Temp
DROP TABLE #Temp