I have a requirement where I need to check authorization in DB. However the authorization field contains comma-separated values. So, to extract and add a where clause I used CROSS-APPLY, as demonstrated at split-comma-separated-list, which is working fine.
The query goes something like
SELECT EmployeeID,
LTRIM(RTRIM(m.n.value('.[1]','varchar(8000)'))) AS Certs
FROM
(
SELECT EmployeeID,CAST('<XMLRoot><RowData>' + REPLACE(Certs,',','</RowData><RowData>') + '</RowData></XMLRoot>' AS XML) AS x
FROM some_table
)t
CROSS APPLY x.nodes('/XMLRoot/RowData')m(n)
Later I need to get the column "Certs" dynamically from another table. So, I added one more select with it to get the column name. But when I write that in REPLACE function, it returns the value i.e. column name. I need to treat it as a column name and get the values corresponding to it.
The modified query looks like
SELECT EmployeeID,
LTRIM(RTRIM(m.n.value('.[1]','varchar(8000)'))) AS Certs
(SELECT DatabaseColumnName FROM column_name_table WHERE ID = 98) YY ---Returns "Certs"
CROSS APPLY
FROM
(
SELECT EmployeeID,CAST('<XMLRoot><RowData>' + REPLACE(YY.DatabaseColumnName,',','</RowData><RowData>') + '</RowData></XMLRoot>' AS XML) AS x
FROM some_table
)t
CROSS APPLY x.nodes('/XMLRoot/RowData')m(n)
Any help is deeply appreciated!
Repeating my earlier comment: you need to use dynamic sql, and you need to be very careful with dynamic sql to avoid sql injection if this code is exposed to user input.
If the DatabaseColumnName will be the same for all EmployeeID, then you can get that column name and concatenate it to the executed sql like so:
declare #col sysname;
declare #sql nvarchar(max);
select #col = DatabaseColumnName from column_name_table where id = 98);
set #sql = '
SELECT EmployeeID,
LTRIM(RTRIM(m.n.value(''.[1]'',''varchar(8000)''))) AS '+#col+'
FROM
(
SELECT EmployeeID,CAST(''<XMLRoot><RowData>''
+ REPLACE('+#col+','','',''</RowData><RowData>'')
+ ''</RowData></XMLRoot>'' AS XML) AS x
FROM some_table
)t
CROSS APPLY x.nodes(''/XMLRoot/RowData'')m(n)
'
exec sp_executesql #sql;
Dynamic Sql Reference:
The curse and blessings of dynamic SQL - Erland Sommarskog
sp_executesql
If you want more options for splitting strings other than the method you are using now:
Splitting Strings:
Tally OH! An Improved SQL 8K “CSV Splitter” Function - Jeff Moden
Split strings the right way – or the next best way - Aaron Bertrand
Splitting Strings : A Follow-Up - Aaron Bertrand
Splitting Strings : Now with less T-SQL - Aaron Bertrand
string_split() in SQL Server 2016 : Follow-Up #1 - Aaron Bertrand
Related
In Sql Server you can run the following query to combine values from multiple rows into a single string variable.
DECLARE #x NVARCHAR(MAX) = N'';
SELECT TOP 5 #x = #x + name + ', '
FROM sys.objects
SELECT #x
Which results in
"sysrscols, sysrowsets, sysclones, sysallocunits, sysfiles1,"
Is there a way to do something similar in PostgreSql? I've tried creating a PostgreSql function that does a SELECT INTO stringVariable but that is only resulting in a single row's worth of values.
Not exactly. The code you have in SQL Server is not actually guaranteed to work. In Postgres, you would use:
select string_agg(name, ', ')
from sys.objects;
If you have a sys.objects table.
Note that in SQL Server, you should do the same thing:
select string_agg(name, ', ')
from sys.objects;
as demonstrated in this SQL Fiddle.
As a note: Your use of the variable is directly contradicted by the documentation:
If a SELECT statement returns more than one row and the variable references a non-scalar expression, the variable is set to the value returned for the expression in the last row of the result set.
That said, I know what you are doing actually works in practice. But I would recommend using string_agg() or in earlier versions using XML for string concatenation.
I have a table like below which has several columns along with series of numbers as well like the below:
Name: JLEDG
name
user_val_1
user_val_2
user_val_3
user_val_4
One
Two
Three
Three
Three
DECLARE #myvar int = 3;
So I would like to do the following which is not working:
SELECT * FROM JLEDG WHERE ('user_val_' + #myvar) IS NULL;
Expect the sql should be
SELECT * FROM JLEDG WHERE user_val_3 IS NULL;
You can only do that in dynamic SQL. You seem to have a problem with your data model. You shouldn't be storing values splayed across columns like that. You should have another table with one row per value.
One thing you can do is unpivot (using apply) and then filter:
select j.*
from jledg j cross apply
(values (1, user_val_1), (2, user_val_2), . . .
) v(which, user_val)
where which = #myvar;
The alternative is to use dynamic SQL (sp_executesql), but that seems quite cumbersome when you could just fix the data model.
SQL Server is declarative by design, and does not support macro substitution. As Gordon mentioned in his solution (+1), Dynamic SQL is just another option
Example
Declare #myvar int = 3
Declare #SQL varchar(max) = concat('SELECT * FROM JLEDG WHERE user_val_',#myvar,' IS NULL;')
Exec(#SQL)
I am looking to get an order number from a column named KEY_Ref, this ref column have various contents, but some rows look like this
LINE_NO=15^ORDER_NO=176572^RELEASE_NO=1^
Now I am interested in getting the value for ORDER_NO (176572 in this case)
How would I (In SQL Server) go about getting this (Or other) value from the main string
The logic is always
key1=value1^key2=value2^key3=value3^
You can use string_split():
select t.*, s.orderno
from t outer apply
(select stuff(s.value, 1, 9, '') as orderno
from string_split(t.key_ref, '^') s
where s.value like 'ORDER_NO=%'
) s;
Here is a db<>fiddle.
this is going to be a bit lengthy answer however if your SQL server version doesn't support string_split function you may use this.
declare #str varchar(100) = 'LINE_NO=15^ORDER_NO=176572^RELEASE_NO=1^'
declare #substr1 varchar(50) = substring(#str,charindex('^',#str)+1,len(#str))
declare #substr2 varchar(50) = substring(#substr1,charindex('=',#substr1)+1,charindex('^',#substr1)-charindex('=',#substr1)-1)
select #substr2 as 'order number'
the final variable will produce the desired value and you must merge the above queries to a single query that can fetch the value from the table in a single select statement.
this will work only if the pattern doesn't deviate from the one you've mentioned.
I would really like some advice here, to give some background info I am working with inserting Message Tracking logs from Exchange 2007 into SQL. As we have millions upon millions of rows per day I am using a Bulk Insert statement to insert the data into a SQL table.
In fact I actually Bulk Insert into a temp table and then from there I MERGE the data into the live table, this is for test parsing issues as certain fields otherwise have quotes and such around the values.
This works well, with the exception of the fact that the recipient-address column is a delimited field seperated by a ; character, and it can be incredibly long sometimes as there can be many email recipients.
I would like to take this column, and split the values into multiple rows which would then be inserted into another table. Problem is anything I am trying is either taking too long or not working the way I want.
Take this example data:
message-id recipient-address
2D5E558D4B5A3D4F962DA5051EE364BE06CF37A3A5#Server.com user1#domain1.com
E52F650C53A275488552FFD49F98E9A6BEA1262E#Server.com user2#domain2.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user3#domain3.com;user4#domain4.com;user5#domain5.com
I would like this to be formatted as followed in my Recipients table:
message-id recipient-address
2D5E558D4B5A3D4F962DA5051EE364BE06CF37A3A5#Server.com user1#domain1.com
E52F650C53A275488552FFD49F98E9A6BEA1262E#Server.com user2#domain2.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user3#domain3.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user4#domain4.com
4fd70c47.4d600e0a.0a7b.ffff87e1#Server.com user5#domain5.com
Does anyone have any ideas about how I can go about doing this?
I know PowerShell pretty well, so I tried in that, but a foreach loop even on 28K records took forever to process, I need something that will run as quickly/efficiently as possible.
Thanks!
If you are on SQL Server 2016+
You can use the new STRING_SPLIT function, which I've blogged about here, and Brent Ozar has blogged about here.
SELECT s.[message-id], f.value
FROM dbo.SourceData AS s
CROSS APPLY STRING_SPLIT(s.[recipient-address], ';') as f;
If you are still on a version prior to SQL Server 2016
Create a split function. This is just one of many examples out there:
CREATE FUNCTION dbo.SplitStrings
(
#List NVARCHAR(MAX),
#Delimiter NVARCHAR(255)
)
RETURNS TABLE
AS
RETURN (SELECT Number = ROW_NUMBER() OVER (ORDER BY Number),
Item FROM (SELECT Number, Item = LTRIM(RTRIM(SUBSTRING(#List, Number,
CHARINDEX(#Delimiter, #List + #Delimiter, Number) - Number)))
FROM (SELECT ROW_NUMBER() OVER (ORDER BY s1.[object_id])
FROM sys.all_objects AS s1 CROSS APPLY sys.all_objects) AS n(Number)
WHERE Number <= CONVERT(INT, LEN(#List))
AND SUBSTRING(#Delimiter + #List, Number, 1) = #Delimiter
) AS y);
GO
I've discussed a few others here, here, and a better approach than splitting in the first place here.
Now you can extrapolate simply by:
SELECT s.[message-id], f.Item
FROM dbo.SourceData AS s
CROSS APPLY dbo.SplitStrings(s.[recipient-address], ';') as f;
Also I suggest not putting dashes in column names. It means you always have to put them in [square brackets].
SQL Server 2016 include a new table function string_split(), similar to the previous solution.
The only requirement is Set compatibility level to 130 (SQL Server 2016)
You may use CROSS APPLY (available in SQL Server 2005 and above) and STRING_SPLIT function (available in SQL Server 2016 and above):
DECLARE #delimiter nvarchar(255) = ';';
-- create tables
CREATE TABLE MessageRecipients (MessageId int, Recipients nvarchar(max));
CREATE TABLE MessageRecipient (MessageId int, Recipient nvarchar(max));
-- insert data
INSERT INTO MessageRecipients VALUES (1, 'user1#domain.com; user2#domain.com; user3#domain.com');
INSERT INTO MessageRecipients VALUES (2, 'user#domain1.com; user#domain2.com');
-- insert into MessageRecipient
INSERT INTO MessageRecipient
SELECT MessageId, ltrim(rtrim(value))
FROM MessageRecipients
CROSS APPLY STRING_SPLIT(Recipients, #delimiter)
-- output results
SELECT * FROM MessageRecipients;
SELECT * FROM MessageRecipient;
-- delete tables
DROP TABLE MessageRecipients;
DROP TABLE MessageRecipient;
Results:
MessageId Recipients
----------- ----------------------------------------------------
1 user1#domain.com; user2#domain.com; user3#domain.com
2 user#domain1.com; user#domain2.com
and
MessageId Recipient
----------- ----------------
1 user1#domain.com
1 user2#domain.com
1 user3#domain.com
2 user#domain1.com
2 user#domain2.com
for table = "yelp_business", split the column categories values separated by ; into rows and display as category column.
SELECT unnest(string_to_array(categories, ';')) AS category
FROM yelp_business;
I am passing a comma-delimited list of values into a stored procedure. I need to execute a query to see if the ID of an entity is in the comma-delimited list. Unfortunately, I think I do not understand something.
When I execute the following stored procedure:
exec dbo.myStoredProcedure #myFilter=N'1, 2, 3, 4'
I receive the following error:
"Conversion failed when converting the varchar value '1, 2, 3, 4' to data type int."
My stored procedure is fairly basic. It looks like this:
CREATE PROCEDURE [dbo].[myStoredProcedure]
#myFilter nvarchar(512) = NULL
AS
SET NOCOUNT ON
BEGIN
-- Remove the quote marks so the filter will work with the "IN" statement
SELECT #myFilter = REPLACE(#myFilter, '''', '')
-- Execute the query
SELECT
t.ID,
t.Name
FROM
MyTable t
WHERE
t.ID IN (#myFilter)
ORDER BY
t.Name
END
How do I use a parameter in a SQL statement as described above? Thank you!
You could make function that takes your parameter, slipts it and returns table with all the numbers in it.
If your are working with lists or arrays in SQL Server, I recommend that you read Erland Sommarskogs wonderful stuff:
Arrays and Lists in SQL Server 2005
You need to split the string and dump it into a temp table. Then you join against the temp table.
There are many examples of this, here is one at random.
http://blogs.microsoft.co.il/blogs/itai/archive/2009/02/01/t-sql-split-function.aspx
Absent a split function, something like this:
CREATE PROCEDURE [dbo].[myStoredProcedure]
#myFilter varchar(512) = NULL -- don't use NVARCHAR for a list of INTs
AS
SET NOCOUNT ON
BEGIN
SELECT
t.ID,
t.Name
FROM
MyTable t
WHERE
CHARINDEX(','+CONVERT(VARCHAR,t.ID)+',',#myFilter) > 0
ORDER BY
t.Name
END
Performance will be poor. A table scan every time. Better to use a split function. See: http://www.sommarskog.se/arrays-in-sql.html
I would create a function that takes your comma delimited string and splits it and returns a single column table variable with each value in its own row. Select that column from the returned table in your IN statement.
I found a cute way of doing this - but it smells a bit.
declare #delimitedlist varchar(8000)
set #delimitedlist = '|1|2|33|11|3134|'
select * from mytable where #delimitedlist like '%|' + cast(id as varchar) + '|%'
So... this will return all records with an id equal to 1, 2, 33, 11, or 3134.
EDIT:
I would also add that this is not vulnerable to SQL injection (whereas dynamic SQL relies on your whitelisting/blacklisting techniques to ensure it isn't vulnerable). It might have a performance hit on large sets of data, but it works and it's secure.
I have a couple of blog posts on this as well, with a lot of interesting followup comments and dialog:
More on splitting lists
Processing list of integers