I have created a user defined table type with a single varchar(max) column with the hopes that I could call my function by passing in an actual query. Of course it didn't work.
My function looks like this:
create function tfnTableToString(#strings TVP_Strings READONLY)
returns varchar(max)
as
begin
return SUBSTRING((SELECT ',' + string FROM #strings FOR XML PATH('')), 2 , 9999)
end
go
As you can see, I'm am just trying to make the code a little simpler by refactoring the process of converting query results into a comma delimited string. Is there a way to do this or am I just stuck having to write the ugly SUBSTRING(... FOR XML PATH...) wherever I need it?
Related
I have a stored procedure it takes comma separated string as input. Which might be too large some times approximately more than 8 thousand characters or more. In that situation, query performance goes down sometimes. And I think there is a limitation for the character length inside the IN clause. For that, sometimes I get errors. Now, I need to know is it better to use a Custom TABLE TYPE as parameter and use Inner JOIN to find the result. If it is then why is it. Here are my 2 stored procedures (minimal code):
CREATE TYPE [dbo].[INTList] AS TABLE(
[ID] [int] NULL
)
Procedure 1
CREATE PROCEDURE [report].[GetSKU]
#list [INTList] READONLY,
AS
Select sk.SKUID,sk.Code SCode,sk.SName
FROM SKUs sk
INNER JOIN #list sst ON sst.ID=sk.SKUID
Procedure 2
CREATE PROCEDURE [report].[GetSKU]
#params varchar(max),
AS
Select sk.SKUID,sk.Code SCode,sk.SName
FROM SKUs sk
WHere CHARINDEX(','+cast( sk.SKUID as varchar(MAX))+',', #params) > 0
Now, which procedures is better to use.
Note: Original Stored Procedures does have few more Joins.
As this question did raise quite some discussion in comments but did not get any viable answer, I'd like to add the major points in order to help future research.
This question is about: How do I pass a (large) list of values into a query?
In most cases, people need this either in a WHERE SomeColumn IN(SomeValueList)-filter or to JOIN against this with something like FROM MyTable INNER JOIN SomeValueList ON....
Very important is the SQL-Server's version, as with v2016 we got two great tools: native STRING_SPLIT() (not position-safe!) and JSON support.
Furthermore, and rather obvious, we have to think about the scales and values.
Do we pass in a simple list of some IDs or a huge list with thousands of values?
Do we talk about simple integers or GUIDs?
And what's about text values, where we have to think about dangerous characters (like [ { " in JSON or < & in XML - there are many more...)?
What about CSV-lists, where the separating character might appear within the content (quoting / escaping)?
In some cases we might even want to pass several columns at once...
There are several options:
Table valued parameter (TVP, CREATE TYPE ...),
CSV together with string splitting functions (native since v2016, various home brewed, CLR...),
and text-based containers: XML or JSON (since v2016)
Table valued paramter (TVP - the best choice)
A table valued parameter (TVP) must be created in advance (this might be a draw back) but will behave as any other table once created. You can add indexes, you can use it in various use cases and you do not have to bother about anything under the hood.
Sometimes we cannot use this due to missing rights to use CREATE TYPE...
Character separated values (CSV)
With CSV we see three approaches
Dynamic Sql: Create a statement, where the CSV list is simply stuffed into the IN() and execute this dynamically. This can be a very efficient approach, but will be open to various obstacles (no ad-hoc-usage, injection threat, breaking on bad values...)
String splitting functions: There are tons of examples around... All of them have in common that the separated string will be returned as a list of items. Common issues here: performance, missing ordinal position, limits for the separator, handling of duplicate or empty values, handling of quoted or escaped values, handling of separators within the content. Aaron Bertrand did some great research about the various approaches of string splitting. Similar to TVPs one draw back might be, that this function must exist in the database in advance or that we need to be allowed to execute CREATE FUNCTION if not.
ad-hoc-splitters: Before v2016 the most used approach was XML based, since then we have moved to JSON based splitters. Both use some string methods to transform the CSV string to 1) separated elements (XML) or 2) into a JSON-array. The result is queried by 1) XQuery (.value() and .nodes()) or 2) JSON's OPENJSON() or JSON_VALUE().
Text based containers
We can pass the list as string, but within a defined format:
Using ["a","b","c"] instead of a,b,c allows for immediate usage of OPENJSON().
Using <x>a</x><x>b</x><x>c</x> instead allows for XML queries.
The biggest advantage here: Any programming language provides support for these formats.
Common obstacles like date and number formatting is solved implicitly. Passing JSON or XML is - in most cases - just some few lines of code.
Both approaches allow for type- and position-safe queries.
We can solve our needs without the need to rely on anything existing in advance.
For the very best performance you can use this function:
CREATE FUNCTION [dbo].StringSplit
(
#String VARCHAR(MAX), #Separator CHAR(1)
)
RETURNS #RESULT TABLE(Value VARCHAR(MAX))
AS
BEGIN
DECLARE #SeparatorPosition INT = CHARINDEX(#Separator, #String ),
#Value VARCHAR(MAX), #StartPosition INT = 1
IF #SeparatorPosition = 0
BEGIN
INSERT INTO #RESULT VALUES(#String)
RETURN
END
SET #String = #String + #Separator
WHILE #SeparatorPosition > 0
BEGIN
SET #Value = SUBSTRING(#String , #StartPosition, #SeparatorPosition- #StartPosition)
IF( #Value <> '' )
INSERT INTO #RESULT VALUES(#Value)
SET #StartPosition = #SeparatorPosition + 1
SET #SeparatorPosition = CHARINDEX(#Separator, #String , #StartPosition)
END
RETURN
END
This function return table - select * from StringSplit('12,13,14,15,16', ',') so you can join this function to your table or can use IN on the where clause.
I have a SQL view that calls a scalar function with a string parameter. The problem is that the string occasionally has special characters which causes the function to fail.
The view query looks like this:
SELECT TOP (100) PERCENT
Id, Name, StartDate, EndDate
,dbo.[fnGetRelatedInfo] (Name) as Information
FROM dbo.Session
The function looks like this:
ALTER FUNCTION [dbo].[fnGetRelatedInfo]( #Name varchar(50) )
RETURNS varchar(200)
AS
BEGIN
DECLARE #Result varchar(200)
SELECT #Result = ''
SELECT #Result = #Result + Info + CHAR(13)+CHAR(10)
FROM [SessionInfo]
WHERE SessionName = #Name
RETURN #Result
END
How do I escape the name value so it will work when passed to the function?
I am guessing that the problem is non-unicode characters in dbo.Session.Name. Since the parameter to the function is VARCHAR, it will only hold unicode characters, so the non-unicode characters are lost when being passed to the function. The solution for this would be to change the parameter to be NVARCHAR(50).
However, if you care about performance, and more importantly consistent, reliable results stop using this function immediately. Alter your view to simply be:
SELECT s.ID,
s.Name,
s.StartDate,
s.EndDate,
( SELECT si.Info + CHAR(13)+CHAR(10)
FROM SessionInfo AS si
WHERE si.SessionName = s.Name
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)') AS Information
FROM dbo.Session AS s;
Using variable concatenation can lead to unexpected results which are dependent on the internal pathways of the execution plan. So I would rule this out as a solution immediately. Not only this, the RBAR nature of a scalar UDF means that this will not scale well at all.
Various ways of doing this grouped concatenation have been benchmarked here, where CLR is actually the winner, but this is not always an option.
I am using FOR XML PATH in my stored procedure to concatenate rows.But it gives me performance issue.can any one please tell me a function that i can use instead of for xml path.
You can create a CLR Aggregate function to do concatenation.
Example here: http://www.mssqltips.com/sqlservertip/2022/concat-aggregates-sql-server-clr-function/
You can concatenate storing the string in a variable, just with a simple select.
Example concatenating with a comma:
DECLARE #MyConcatenateData VARCHAR(1000)
SELECT #MyConcatenateData = COALESCE(#MyConcatenateData + ',', '') + MyTextColumn
FROM MyTable
I'm trying to query a table with a column which is xml data with the query and value functions. When using regular string literals it's all okay, but if I put that in a stored procedure and try to use variables it doesn't work.
I suppose I'm not using the correct datatype, but after some searching I can't figure out what datatype the query function wants.
Example:
table contains
| Id | xmldata |
| 1 | <data><node>value</node></data> |
now, using the select query
select id
from table
where xmldata.query('/data/node').value('.', 'VARCHAR(50)') = 'value'
gets me the data I want. But, if I use this in a stored procedure and use a parameter #xpath varchar(100) and pass that to the query method as xmldata.query(#xpath)
i get the error
The argument 1 of the xml data type method "query" must be a string literal.
I guess varchar(100) is not correct, but what datatype can I use that would make MSSQL happy?
Update:
Okay, so. Apparently you can't pass a parameter to the query method "just like that", but one can use the sql:variable in conjunction with local-name to work a part of it out. So, for instance, this will work
declare #xpath VarChar(100)
set #xpath='node'
select objectData.query('/data/*[local-name() = sql:variable("#xpath")]')
.value('.', 'varchar(100)') as xmldata
from table
and value is selected in the column xmldata. But(!) it requires that the root node is the first value in the query function. The following will not work
declare #xpath VarChar(100)
set #xpath='/data/node'
select objectData.query('*[local-name() = sql:variable("#xpath")]')
.value('.', 'varchar(100)') as xmldata
from table
notice how the query path is "moved up" to the variable. I will continue my investigations..
A literal is the opposite of a variable. The message means that you cannot pass a variable as the first argument to query.
One way around that is dynamic SQL:
declare #sql varchar(max)
set #sql = 'select id from table where xmldata.query(''' + #path +
''').value(''.'', ''VARCHAR(50)'') = ''value'''
exec #sql
As you can see, dynamic SQL does not result in very readable code. I would certainly investigate alternatives.
EDIT: Your suggestion of local-name() works for node names.
declare #nodename varchar(max)
set #nodename = 'node'
...
where xmldata.query('//*[local-name()=sql:variable("#nodename")]')
.value('.', 'varchar(50)') = 'value'
There doesn't seem to be an equivalent for paths.
I made the following function in SQL Server 2008 earlier this week that takes two parameters and uses them to select a column of "detail" records and returns them as a single varchar list of comma separated values. Now that I get to thinking about it, I would like to take this table and application-specific function and make it more generic.
I am not well-versed in defining SQL functions, as this is my first. How can I change this function to accept a single "column" worth of data, so that I can use it in a more generic way?
Instead of calling:
SELECT ejc_concatFormDetails(formuid, categoryName)
I would like to make it work like:
SELECT concatColumnValues(SELECT someColumn FROM SomeTable)
Here is my function definition:
FUNCTION [DNet].[ejc_concatFormDetails](#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000) AS
BEGIN
DECLARE #returnData VARCHAR(1000)
DECLARE #currentData VARCHAR(75)
DECLARE dataCursor CURSOR FAST_FORWARD FOR
SELECT data FROM DNet.ejc_FormDetails WHERE formuid = #formuid AND category = #category
SET #returnData = ''
OPEN dataCursor
FETCH NEXT FROM dataCursor INTO #currentData
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #returnData = #returnData + ', ' + #currentData
FETCH NEXT FROM dataCursor INTO #currentData
END
CLOSE dataCursor
DEALLOCATE dataCursor
RETURN SUBSTRING(#returnData,3,1000)
END
As you can see, I am selecting the column data within my function and then looping over the results with a cursor to build my comma separated varchar.
How can I alter this to accept a single parameter that is a result set and then access that result set with a cursor?
Others have answered your main question - but let me point out another problem with your function - the terrible use of a CURSOR!
You can easily rewrite this function to use no cursor, no WHILE loop - nothing like that. It'll be tons faster, and a lot easier, too - much less code:
FUNCTION DNet.ejc_concatFormDetails
(#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000)
AS
RETURN
SUBSTRING(
(SELECT ', ' + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
FOR XML PATH('')
), 3, 1000)
The trick is to use the FOR XML PATH('') - this returns a concatenated list of your data columns and your fixed ', ' delimiters. Add a SUBSTRING() on that and you're done! As easy as that..... no dogged-slow CURSOR, no messie concatenation and all that gooey code - just one statement and that's all there is.
You can use table-valued parameters:
CREATE FUNCTION MyFunction(
#Data AS TABLE (
Column1 int,
Column2 nvarchar(50),
Column3 datetime
)
)
RETURNS NVARCHAR(MAX)
AS BEGIN
/* here you can do what you want */
END
You can use Table Valued Parameters as of SQL Server 2008, which would allow you to pass a TABLE variable in as a parameter. The limitations and examples for this are all in that linked article.
However, I'd also point out that using a cursor could well be painful for performance.
You don't need to use a cursor, as you can do it all in 1 SELECT statement:
SELECT #MyCSVString = COALESCE(#MyCSVString + ', ', '') + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
No need for a cursor
Your question is a bit unclear. In your first SQL statement it looks like you're trying to pass columns to the function, but there is no WHERE clause. In the second SQL statement you're passing a collection of rows (results from a SELECT). Can you supply some sample data and expected outcome?
Without fully understanding your goal, you could look into changing the parameter to be a table variable. Fill a table variable local to the calling code and pass that into the function. You could do that as a stored procedure though and wouldn't need a function.