Suppose I have a table a with one column b and three rows(1,2,3), I would like to create a function that will return '1,2,3' that would be called like this : SELECT FUNC(f), ... FROM ...
In other words, I have a linked table that have more than one rows linked to each rows of the first table and would like to concatenate the content of one column from the second table. In this case, it's a list of names associated with a specific observation.
I was thinking of using a SQL function for that, but I can't remember how... :(
Thanks
Here is an example for SQL Server:
CREATE FUNCTION ConcatenateMyTableValues
(#ID int)
RETURNS varchar(max)
AS
BEGIN
declare #s as varchar(max);
select #s = isnull(#s + ',', '') + MyColumn from MyTable where ID = #ID;
return #s
end
And then you could use it like this:
select t.ID, t.Name, dbo.ConcatenateMyTableValues(t.ID)
from SomeTable t
you can use COALESCE to convert values in one column to csv
http://www.sqlteam.com/article/using-coalesce-to-build-comma-delimited-string
I am not sure how you will be able to create a function
unless you do not mind creating a dynamic sql which can accept a column name and then build sql accordingly at run time.
edit.
this works for SQL Server only.
didn't realize that you have not mentioned any db.
DECLARE #list AS varchar(MAX)
SELECT #list = ISNULL(#list + ',', '') + b
FROM MyTable
SELECT #list AS Result
Here a way to do this recursive (ms sql)
Depends on your technology.
If it is Oracle, check out the stragg function.
http://www.sqlsnippets.com/en/topic-11591.html
If it is MSSQL use the XML PATH trick
http://sqlblogcasts.com/blogs/tonyrogerson/archive/2009/03/29/creating-an-output-csv-using-for-xml-and-multiple-rows.aspx
I guess I'm going to answer my own question since I actually got a way to do it using CURSOR (that keyword I was missing in my thoughs..)
ALTER FUNCTION [dbo].[GET_NOM_MEDECIN_REVISEURS] (#NoAs810 int)
RETURNS NVARCHAR(1000)
AS
BEGIN
IF #NoAs810 = 0
RETURN ''
ELSE
BEGIN
DECLARE #NomsReviseurs NVARCHAR(1000)
DECLARE #IdReviseurs AS TABLE(IdReviseur int)
DECLARE #TempNomReviseur NVARCHAR(50)
SET #NomsReviseurs = ''
DECLARE CurReviseur CURSOR FOR
SELECT DISTINCT Nom FROM T_Ref_Reviseur R INNER JOIN T_Signature S ON R.IdReviseur = S.idReviseur WHERE NoAs810 = #NoAs810
OPEN CurReviseur
FETCH FROM CurReviseur INTO #TempNomReviseur
WHILE ##FETCH_STATUS = 0
BEGIN
SET #NomsReviseurs = #NomsReviseurs + #TempNomReviseur
FETCH NEXT FROM CurReviseur INTO #TempNomReviseur
IF ##FETCH_STATUS = 0
SET #NomsReviseurs = #NomsReviseurs + ' - '
END
CLOSE CurReviseur
RETURN #NomsReviseurs
END
RETURN ''
END
Related
I have created a stored procedure as shown below, but it's returning only one row instead of 3:
CREATE PROCEDURE [dbo].[tempsp]
(#RecycleIds NVARCHAR(MAX) = NULL)
AS
BEGIN
DECLARE #Err INT
DECLARE #WhereClause NVARCHAR(MAX)
DECLARE #SQLText1 NVARCHAR(MAX)
DECLARE #SQLText NVARCHAR(MAX)
SET #SQLText1 = 'SELECT FROM dbo.SKU '
IF #RecycledSkuIds IS NOT NULL
BEGIN
SET #SQLText = 'SELECT FROM dbo.SKU WHERE SKU.SkuId IN (#RecycleIds)'
EXEC sp_executesql #SQLText, N'#RecycleSkuIds nvarchar', #RecycleIds
END
ELSE
BEGIN
EXEC(#SQLText1)
END
SET #Err = ##ERROR
RETURN #Err
END
-------end of stored procedure--------
EXEC tempsp #RecycleIds = '5,6,7'
After running this SQL statement, it only returns one row instead of 3, with the id's of 5, 6, 7.
Can anyone tell me what I am doing wrong?
i wanted to use sp_executesql, so that it can be safe against sql injection with strong type defined.
Use a table type parameter, with a strongly typed column:
CREATE TYPE dbo.IDs AS table (ID int);
GO
CREATE PROCEDURE [dbo].[tempsp] #RecycleIds dbo.IDs READONLY AS
BEGIN
IF EXISTS (SELECT 1 FROM #RecycleIds)
SELECT * --Replace with needed columns
FROM dbo.SKU S
--Using EXISTS in case someone silly puts in the same ID twice.
WHERE EXISTS (SELECT 1
FROM #RecycleIds R
WHERE R.ID = S.SkuID);
ELSE
SELECT * --Replace with needed columns
FROM dbo.SKU S
END;
GO
Then you could execute it like so:
EXEC dbo.tempsp; --All Rows
GO
DECLARE #RecycleIds dbo.IDs;
INSERT INTO #RecycleIds
VALUES(1),(40),(182);
EXEC dbo.tempsp #RecycleIds;
I was trying to retrive the rows whose id matches within the IN clause.
SET #INClauseIds='''' + replace(#Ids, ',', ''',''') + ''''
Above statement would convert the ID's ='1,2,3' to '1','2','3' which i can directly place in the IN clause.
SET #SQLText1 ='EXEC(''SELECT Name,SEOFriendlyName FROM SKU Where Id IN ( ''+ #Ids+'' ) )'
EXEC sp_executesql #SQLText1 ,N'#INClauseIds nvarchar(max)',#Ids=#INClauseIds
If you want to avoid the usage of Temp Table which would add extra caliculation time. you can you the above strategy to retrive n number of records. Safe with strongly coupled with sp_executesql and without any sql injection.
You cannot use IN. Or, more accurately, you have a string and you are confusing it with a list. One method is to instead use LIKE:
SET #SQLText = '
SELECT *
FROM dbo.SKU
WHERE CONCAT('','', #RecycleIds, '','') LIKE CONCAT(''%,'', SKU.SkuId, '',%'')
';
I have problem in optimizing an SQL query to do some data cleansing.
In fact, I have a table which is a sort of referential of a multiple special characters and word. Let's call it ABNORMAL(ID,PATTERN)
I have also another table INDIVIDUALS containing a column (NAME) which I want to clean by removing from it all characters that exist in the table ABNORMAL.
Currently, I have tried to use update statements, but I'm not sure if there is a better way to do this.
Approach one
Use a while loop to build a replace containing all characters from ABNORMALS by a blank '' and do one update using the built-in REPLACE
DECLARE #REPLACE_EXPRESSION nvarchar(max) ='REPLACE(NAME,'''','''')'
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- NEW REPLACE EXPRESSION TO IMBRICATE INTO THE REPLACE EXPRESSION VARIABLE
DECLARE #CURR_REP NVARCHAR(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
SET #REPLACE_EXPRESSION = REPLACE(#REPLACE_EXPRESSION ,'NAME','REPLACE(NAME,'+''''+#CURRENT_CHARAC+''''+','''')')
set #i=#i+1
END
SET #UPDATE_QUERY = 'UPDATE INDIVIDUAL SET NAME ='+ #REPLACE_EXPRESSION
EXEC sp_executesql #UPDATE_QUERY
Approach two
Use a while loop to select every character in abnormal and do an update using replace containing the characters to remove:
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
UPDATE INDIVIDUAL
SET NAME = REPLACE(NAME,#CURRENT_CHARAC,'')
SET #i=#i+1
END
I already tested both approaches for 2 millions records, and I found that the first approach is faster than the second. I would know if you have already done something similar and new (better) ideas to try.
If you are using SQL Server 2017 you could use TRANSLATE and avoid dynamic SQL:
SELECT i.*
, REPLACE(TRANSLATE(i.NAME, f, REPLICATE('!', s.l)), '!', '') AS cleansed
FROM INDIVIDUALS i
OUTER APPLY (SELECT STRING_AGG(PATTERN, '') AS f
,LEN(STRING_AGG(PATTERN,'')) AS l
FROM ABNORMAL) AS s
DBFiddle Demo
Anyway 1st approach is better becasue you do one UPDATE, with second approach you remove characters one character at time (so you will have multiple UPDATE).
I would also track transaction log growth with both approaches.
If there's not too many characters that to be cleaned, then this trick might work.
Basically, you build 1 big update statement with a replace for each value in the table with the characters to be removed.
Example code:
Test data (using temp tables)
create table #ABNORMAL_CHARACTERS (id int identity(1,1), chr varchar(30));
insert into #ABNORMAL_CHARACTERS (chr) values ('!'),('&'),('#');
create table #INDIVIDUAL (id int identity(1,1), name varchar(30));
insert into #INDIVIDUAL (name) values ('test 1 &'),('test !'),('test 3');
Code:
declare #FieldName varchar(30) = 'name';
declare #Replaces varchar(max) = #FieldName;
declare #UpdateSQL varchar(max);
select #Replaces = concat('replace('+#Replaces+', ', ''''+chr+''','''')') from #ABNORMAL_CHARACTERS order by id;
set #UpdateSQL = 'update #INDIVIDUAL
set name = '+#Replaces + '
where exists (select 1 from #ABNORMAL_CHARACTERS where charindex(chr,name)>0)';
exec (#UpdateSQL);
select * from #INDIVIDUAL;
A test here on rextester
And if you would have a UDF that can do a regex replace.
For example here
Then the #Replaces variable could be simplified with only 1 RegexReplace function and a pattern.
I am having a small problem with the IN SQL statement. I was just wondering if anyone could help me?
#Ids = "1,2,3,4,5"
SELECT * FROM Nav WHERE CONVERT(VARCHAR,NavigationID) IN (CONVERT(VARCHAR,#Ids))
This is coming back with the error below, I am sure this is pretty simple!
Conversion failed when converting the varchar value '1,' to data type int.
The SQL IN clause does not accept a single variable to represent a list of values -- no database does, without using dynamic SQL. Otherwise, you could use a Table Valued Function (SQL Server 2000+) to pull the values out of the list & return them as a table that you can join against.
Dynamic SQL example:
EXEC('SELECT *
FROM Nav
WHERE NavigationID IN ('+ #Ids +')')
I recommend reading The curse and blessings of dynamic SQL before using dynamic SQL on SQL Server.
Jason:
First create a function like this
Create FUNCTION [dbo].[ftDelimitedAsTable](#dlm char, #string varchar(8000))
RETURNS
--------------------------------------------------------------------------*/
/*------------------------------------------------------------------------
declare #dlm char, #string varchar(1000)
set #dlm=','; set #string='t1,t2,t3';
-- tHIS FUNCION RETUNRS IN THE ASCENDING ORDER
-- 19TH Apr 06
------------------------------------------------------------------------*/
--declare
#table_var TABLE
(id int identity(1,1),
r varchar(1000)
)
AS
BEGIN
declare #n int,#i int
set #n=dbo.fnCountChars(#dlm,#string)+1
SET #I =1
while #I <= #N
begin
insert #table_var
select dbo.fsDelimitedString(#dlm,#string,#i)
set #I= #I+1
end
if #n =1 insert #TABLE_VAR VALUES(#STRING)
delete from #table_var where r=''
return
END
And then
set quoted_identifier off
declare #ids varchar(max)
select #Ids = "1,2,3,4,5"
declare #nav table ( navigationid int identity(1,1),theother bigint)
insert #nav(theother) select 10 union select 11 union select 15
SELECT * FROM #Nav WHERE CONVERT(VARCHAR,NavigationID) IN (select id from dbo.ftDelimitedAsTable(',',#Ids))
select * from dbo.ftDelimitedAsTable(',',#Ids)
What you're doing is not possible with the SQL IN statement. You cannot pass a string to it and expect that string to be parsed. IN is for specific, hard-coded values.
There are two ways to do what you want to do here.
One is to create a 'dynamic sql' query and execute it, after substituting in your IN list.
DECLARE #query varchar(max);
SET #query = 'SELECT * FROM Nav WHERE CONVERT(VARCHAR,NavigationID) IN (' + #Ids + ')'
exec (#query)
This can have performance impacts and other complications. Generally I'd try to avoid it.
The other method is to use a User Defined Function (UDF) to split the string into its component parts and then query against that.
There's a post detailing how to create that function here
Once the function exists, it's trivial to join onto it
SELECT * FROM Nav
CROSS APPLY dbo.StringSplit(#Ids) a
WHERE a.s = CONVERT(varchar, Nav.NavigationId)
NB- the 'a.s' field reference is based on the linked function, which stores the split value in a column named 's'. This may differ based on the implementation of your string split function
This is nice because it uses a set based approach to the query rather than an IN subquery, but a CROSS JOIN may be a little complex for the moment, so if you want to maintain the IN syntax then the following should work:
SELECT * FROM Nav
WHERE Nav.NavigationId IN
(SELECT CONVERT(int, a.s) AS Value
FROM dbo.StringSplit(#Ids) a
Does T-SQL allow a variable number of arguments to a stored procedure like params in C#?
EDIT: I'm using SQL Server 2005. That 2008 answer makes me wish we were using it...
In SQL 2008 there's Table-Valued Parameters (TVPs)
Your stored proc can accept lists of parameters..
Finally we're able to do a IN clause without relying on XML!
Mike
No, not for things like UDFs or stored procedures. That's what tables are for. Put the values in a table somewhere (with a common key) and pass the correct key to your procedure.
Typically
CREATE PROCEDURE dbo.sptest
( #xml TEXT )
AS
BEGIN
DECLARE #flag1 INT
DECLARE #flag2 VARCHAR(50)
DECLARE #flag3 DATETIME
DECLARE #idoc INT
exec sp_xml_preparedocument #idoc OUTPUT, #xml
SELECT #flag1 = firstparam, flag2 = secondparam, flag3 = thirdparam
FROM OPENXML(#idoc, '/root', 2) WITH
( firstparam INT, secondparam VARCHAR(50), thirdparam DATETIME) as x
END
exec sptest '<root><firstparam>5</firstparam><secondparam>Joes Bar</secondparam><thirdparam>12/30/2010</thirdparam></root>'
Extend as necessary
Another approach I've seen to passing in params or arrays is to pass in an XML string, dump that to a temporary table/table variable and work with it from that point. Not the easiest when you want to manually run a stored procedure, but it works as a work around to the lack of array/dynamic param support.
I've used a little function to separate a CSV string into a table
That way I could go
SELECT col1, col2
FROM myTable
WHERE myTable.ID IN (SELECT ID FROM dbo.SplitIDs('1,2,3,4,5...'))
My function is below:
CREATE FUNCTION [dbo].[SplitIDs]
(
#IDList varchar(500)
)
RETURNS
#ParsedList table
(
ID int
)
AS
BEGIN
DECLARE #ID varchar(10), #Pos int
SET #IDList = LTRIM(RTRIM(#IDList))+ ','
SET #Pos = CHARINDEX(',', #IDList, 1)
IF REPLACE(#IDList, ',', '') <> ''
BEGIN
WHILE #Pos > 0
BEGIN
SET #ID = LTRIM(RTRIM(LEFT(#IDList, #Pos - 1)))
IF #ID <> ''
BEGIN
INSERT INTO #ParsedList (ID)
VALUES (CAST(#ID AS int)) --Use Appropriate conversion
END
SET #IDList = RIGHT(#IDList, LEN(#IDList) - #Pos)
SET #Pos = CHARINDEX(',', #IDList, 1)
END
END
RETURN
END
I'm sure there are better ways to implement this, this is one way I found online and it works well for what I'm doing. If there are some improvement that can be made please comment.
[Update: Using SQL Server 2005]
Hi, what I want to do is query my stored procedure with a comma-delimited list of values (ids) to retrieve rows of data.
The problem I am receiving is a conversion error:
Conversion failed when converting the varchar value ' +
#PassedInIDs + ' to data type int.
The statement in my where-clause and error is:
...
AND (database.ID IN (' + #PassedInIDs + '))
Note: database.ID is of int type.
I was following the article at:
http://www.sql-server-helper.com/functions/comma-delimited-to-table.aspx
but did not complete because of the error.
In my execution script I have:
...
#PassedInIDs= '1,5'
Am I doing something wrong here?
Thank you for your help.
I would strongly suggest that you use the second method from that link. Create a user-defined function that turns your comma-delimited string into a table, which you can then select from easily.
If you do a Google on Erland and "Dynamic SQL" he has a good writeup of the pitfalls that it entails.
For one, you are passing a string to the IN function in SQL. If you look back at the original article, you'll see that instead of issuing a direct SQL statement, it instead is building a string which is the SQL statement to execute.
There is no string evaluation in SQL. This:
database.ID IN (' + #PassedInIDs + ')
will not be turned to:
database.ID IN (1,2,3)
just because the #PassedInIDs parameter happens to contain '1,2,3'. The parameter is not even looked at, because all you have is a string containing " + #PassedInIDs + ". Syntactically, this is equivalent to:
database.ID IN ('Bob')
To make it short, you can't do what you attempt here in SQL. But there are four other possibilities:
you construct the SQL string in the calling language and abandon the stored procedure altogether
you use a dynamic prepared statement with as many parameters in the IN clause as you pan to use
you use a fixed prepared statement with, say, 10 parameters: IN (?,?,?,?,?,?,?,?,?,?), filling only as many as you need, setting the others to NULL
you create a stored procedure with, say, 10 parameters and pass in as many as you need, setting the others to NULL: IN (#p1, #p2, ..., #p10).
I would create a CLR table-valued function:
http://msdn.microsoft.com/en-us/library/ms131103.aspx
In it, you would parse the string apart and perform a conversion to a set of rows. You can then join on the results of that table, or use IN to see if an id is in the list.
You need to treat ufn_CSVToTable like it's a table. So you can join the function:
JOIN ufn_CSVToTable(#PassedInIDs) uf ON database.ID = uf.[String]
I suggest using XML for this in SQL 2005. Somewhat bulkier, but it can be easier. It allows you to select the XML into a table which can then be joined or inserted etc.
Look at Sql Server's OPENXML() if you haven't already.
For example, you could pass in something like:
'12...'
and then use:
exec sp_xml_preparedocument #doc OUTPUT, #xmlParam
SELECT element
FROM OPENXML (#doc, 'Array/Value', 2) WITH (element varchar(max) 'text()')
That should be a start
this may be solved by 6 ways as mentioned in Narayana's article Passing a list/array to an SQL Server stored procedure
And my most strait forward implementation is
declare #statement nvarchar(256)
set #statement = 'select * from Persons where Persons.id in ('+ #PassedInIDs +')'
exec sp_executesql #statement
-
Here is what I have found and tested:
SET QUOTED_IDENTIFIER ON
SET ANSI_NULLS ON
GO
CREATE FUNCTION [dbo].[SplitStrings] ( #IDsList VARCHAR(MAX) )
RETURNS #IDsTable TABLE ( [ID] VARCHAR(MAX) )
AS
BEGIN
DECLARE #ID VARCHAR(MAX)
DECLARE #Pos VARCHAR(MAX)
SET #IDsList = LTRIM(RTRIM(#IDsList)) + ','
SET #Pos = CHARINDEX(',', #IDsList, 1)
IF REPLACE(#IDsList, ',', '') <> ''
BEGIN
WHILE #Pos > 0
BEGIN
SET #ID = LTRIM(RTRIM(LEFT(#IDsList, #Pos - 1)))
IF #ID <> ''
BEGIN
INSERT INTO #IDsTable
( [ID] )
VALUES ( CAST(#ID AS VARCHAR) )
END
SET #IDsList = RIGHT(#IDsList, LEN(#IDsList) - #Pos)
SET #Pos = CHARINDEX(',', #IDsList, 1)
END
END
RETURN
END
GO
Here is how function Call:
SELECT * FROM dbo.SplitStrings('123,548,198,547,965')
Try this:
DECLARE #Ids varchar(50);
SET #Ids = '1,2,3,5,4,6,7,98,234';
SELECT *
FROM sometable
WHERE ','+#Ids+',' LIKE '%,'+CONVERT(VARCHAR(50),tableid)+',%';