Note: This is in SQL Server 2008.
I'm trying to use the APPLY operator to allow me to call user-defined in-line table-valued functions on each other in a sensible manner, but it doesn't seem to faithfully do so. I'm using OUTER APPLY, but CROSS APPLY does the same thing. Here's the relevant chunk:
SELECT addresses.Book,addresses.Page,foo.BookInput,foo.PageInput
FROM [dbo].bookPageFromAddress(#address) addresses
outer apply [dbo].[imageFileFromBookPage](addresses.Book, addresses.Page) foo
Nothing is bizarre:
GO
CREATE FUNCTION [dbo].imageFileFromBookPage (#book nvarchar(max), #page nvarchar(max))
RETURNS TABLE AS RETURN(
WITH ids AS (
SELECT right('00000000'+ltrim(str(c.DocID)),8) as VarString,
right('0000000000'+ltrim(str(i.ImageID)),12) as PathVar6
FROM [Resolution].[dbo].[Constant] c, [Resolution].[dbo].[Images] i
WHERE ltrim(c.[Book]) like #book
AND ltrim(c.[Page]) like #page
AND i.DocID = c.DocID
)
SELECT '/Images/' +
substring(ids.VarString,1,2)+'/' +
substring(ids.VarString,3,2)+'/' +
substring(ids.VarString,5,2)+'/' +
right(ids.VarString,8)+'/' +
PathVar6 + '.tif' as ImageLocation
,#book as BookInput, #page as PageInput
FROM ids
);
So, in essence,imageFileFromBookPage outputs its input to BookInput and PageInput. Here's the result of that outer apply on sample input:
Book 4043
Page 125
BookInput NULL
PageInput NULL
Note that Book and Page are strings, not integers; they just happen to be holding numeric characters here, but BookInput and PageInput are really NULL, not strings. I thought at first it was a typing issue; both Book and Page are nchar(10) in their original table, and both of my functions expect nvarchar(max) inputs. I tried CASTing the arguments as nvarchar(max) and got identical results, however.
Am I barking up the wrong tree? I'm not familiar with APPLY, but it sure seems like what I want, here. How do I get APPLY or something like it to actually pass along values to the next function?
EDIT: Modified code above to include more information.
Looking at the APPLY docs, shouldn;t the function be declared as:
CREATE FUNCTION [dbo].imageFileFromBookPage (#book nvarchar(max), #page nvarchar(max))
returns #Data Table
(
book nvartchar(max),
page nvarchar(max)
)
as
begin
insert into #Data
select #book, #page;
end
In other words, it's not a problem with APPLY, it's a problem that your function needs to return a Table.
Cheers -
Related
I have a stored procedure it takes comma separated string as input. Which might be too large some times approximately more than 8 thousand characters or more. In that situation, query performance goes down sometimes. And I think there is a limitation for the character length inside the IN clause. For that, sometimes I get errors. Now, I need to know is it better to use a Custom TABLE TYPE as parameter and use Inner JOIN to find the result. If it is then why is it. Here are my 2 stored procedures (minimal code):
CREATE TYPE [dbo].[INTList] AS TABLE(
[ID] [int] NULL
)
Procedure 1
CREATE PROCEDURE [report].[GetSKU]
#list [INTList] READONLY,
AS
Select sk.SKUID,sk.Code SCode,sk.SName
FROM SKUs sk
INNER JOIN #list sst ON sst.ID=sk.SKUID
Procedure 2
CREATE PROCEDURE [report].[GetSKU]
#params varchar(max),
AS
Select sk.SKUID,sk.Code SCode,sk.SName
FROM SKUs sk
WHere CHARINDEX(','+cast( sk.SKUID as varchar(MAX))+',', #params) > 0
Now, which procedures is better to use.
Note: Original Stored Procedures does have few more Joins.
As this question did raise quite some discussion in comments but did not get any viable answer, I'd like to add the major points in order to help future research.
This question is about: How do I pass a (large) list of values into a query?
In most cases, people need this either in a WHERE SomeColumn IN(SomeValueList)-filter or to JOIN against this with something like FROM MyTable INNER JOIN SomeValueList ON....
Very important is the SQL-Server's version, as with v2016 we got two great tools: native STRING_SPLIT() (not position-safe!) and JSON support.
Furthermore, and rather obvious, we have to think about the scales and values.
Do we pass in a simple list of some IDs or a huge list with thousands of values?
Do we talk about simple integers or GUIDs?
And what's about text values, where we have to think about dangerous characters (like [ { " in JSON or < & in XML - there are many more...)?
What about CSV-lists, where the separating character might appear within the content (quoting / escaping)?
In some cases we might even want to pass several columns at once...
There are several options:
Table valued parameter (TVP, CREATE TYPE ...),
CSV together with string splitting functions (native since v2016, various home brewed, CLR...),
and text-based containers: XML or JSON (since v2016)
Table valued paramter (TVP - the best choice)
A table valued parameter (TVP) must be created in advance (this might be a draw back) but will behave as any other table once created. You can add indexes, you can use it in various use cases and you do not have to bother about anything under the hood.
Sometimes we cannot use this due to missing rights to use CREATE TYPE...
Character separated values (CSV)
With CSV we see three approaches
Dynamic Sql: Create a statement, where the CSV list is simply stuffed into the IN() and execute this dynamically. This can be a very efficient approach, but will be open to various obstacles (no ad-hoc-usage, injection threat, breaking on bad values...)
String splitting functions: There are tons of examples around... All of them have in common that the separated string will be returned as a list of items. Common issues here: performance, missing ordinal position, limits for the separator, handling of duplicate or empty values, handling of quoted or escaped values, handling of separators within the content. Aaron Bertrand did some great research about the various approaches of string splitting. Similar to TVPs one draw back might be, that this function must exist in the database in advance or that we need to be allowed to execute CREATE FUNCTION if not.
ad-hoc-splitters: Before v2016 the most used approach was XML based, since then we have moved to JSON based splitters. Both use some string methods to transform the CSV string to 1) separated elements (XML) or 2) into a JSON-array. The result is queried by 1) XQuery (.value() and .nodes()) or 2) JSON's OPENJSON() or JSON_VALUE().
Text based containers
We can pass the list as string, but within a defined format:
Using ["a","b","c"] instead of a,b,c allows for immediate usage of OPENJSON().
Using <x>a</x><x>b</x><x>c</x> instead allows for XML queries.
The biggest advantage here: Any programming language provides support for these formats.
Common obstacles like date and number formatting is solved implicitly. Passing JSON or XML is - in most cases - just some few lines of code.
Both approaches allow for type- and position-safe queries.
We can solve our needs without the need to rely on anything existing in advance.
For the very best performance you can use this function:
CREATE FUNCTION [dbo].StringSplit
(
#String VARCHAR(MAX), #Separator CHAR(1)
)
RETURNS #RESULT TABLE(Value VARCHAR(MAX))
AS
BEGIN
DECLARE #SeparatorPosition INT = CHARINDEX(#Separator, #String ),
#Value VARCHAR(MAX), #StartPosition INT = 1
IF #SeparatorPosition = 0
BEGIN
INSERT INTO #RESULT VALUES(#String)
RETURN
END
SET #String = #String + #Separator
WHILE #SeparatorPosition > 0
BEGIN
SET #Value = SUBSTRING(#String , #StartPosition, #SeparatorPosition- #StartPosition)
IF( #Value <> '' )
INSERT INTO #RESULT VALUES(#Value)
SET #StartPosition = #SeparatorPosition + 1
SET #SeparatorPosition = CHARINDEX(#Separator, #String , #StartPosition)
END
RETURN
END
This function return table - select * from StringSplit('12,13,14,15,16', ',') so you can join this function to your table or can use IN on the where clause.
How can I create a function like this?
function FN_something (#entrada char(50))
declare #consulta table
declare #notificacao varchar(50)
declare #multa float
declare #saida varchar(50)
set #consulta as = (select num_notificacao,num_multa from table where field = #entrada)
set #notificacao = #consulta.num_notificacao
set #multa = #consulta.num_multa
set #saida = "resultado: "+ #notificacao +";"+#multa
return #saida
Thanks in advance
I would not use a function... Scalar functions tend to be a real performance killer. Try to use something like this inline
SELECT 'resultado: '
+ ISNULL(CAST(t.num_notificacao AS VARCHAR(MAX)),'???')
+ ';'
+ ISNULL(CAST(t.num_multa AS VARCHAR(MAX)),'???')
FROM SomeTable AS t WHERE t.SomeField=#entrada;
If you need a function it was much better to use an inlined TVF (syntax without BEGIN...END and bind it into your query with CROSS APPLY.
Might be simplified:
If your columns are NOT NULL you can go without ISNULL()-function. If your columns are strings, you can do without CAST()... My code is defensive proramming :-D
Hint
If this is something you need more often, you might introduce a VIEW carrying this calculated column and use it instead of your table. You might include this value into your table as computed column as well...
UPDATE
Great, the VIEW you show in the comment is an inline TVF actually, which is very good!
My magic crystall ball tells me, that you might need something like this:
SELECT cl.*
,'resultado: ' + t.num_notificacao + ';' + t.num_multa AS CalculatedResult
FROM dbo.[CampoLivre876]('SomeParameter') AS cl
LEFT JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField --should be only one related row per main row!
This will call the iTFV and join it to the other Table, where the two columns are living. I Assume, that the CampoLivre876-row knows its entrada key.
Hint 2:
If this works for you, you might include this approach directly into your existing iTVF.
UPDATE 2
You might try to change your function like here:
ALTER FUNCTION [dbo].[CampoLivre876] ()
RETURNS TABLE
RETURN
Select cl.mul_numero_notificacao + ';' + CAST(cl.mul_valor_multa as varchar(max)) AS ExistingColumn
,'resultado: ' + t.num_notificacao + ';' + CAST(t.num_multa AS varchar(max)) AS CalculatedResult
From Campo_Livre AS cl With(NoLock)
INNER JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField;
This should read all lines in one go. Reading 1 row after the other is - in almost all cases - something really, really bad...
Here is an example of a function with correct SQL Server syntax:
create function FN_something (
#entrada char(50) -- should probably be `varchar(50)` rather than `char(50)`
) returns varchar(50)
begin
declare #saida varchar(50);
select #saida = 'resultado: ' + num_notificacao + ';' + num_multa
from table
where field = #entrada;
return #saida;
end;
Note: This assumes that the num_ columns are strings, not numbers. If they are numbers, you need to convert them or use concat().
EDIT:
A function really isn't appropriate for this. Probably the best solution is a computed column:
alter table t add something as (concat('resultado: ', num_notificacao, ';', num_multa);
Then you can get the value directly from the table. In earlier versions of SQL Server, you would use a view rather than computed column.
My first xQuery, may be a bit basic, but it's the only time I need to use it at the moment)
DECLARE #IdList XML;
SET #IdList =
'<Ids>
<Id>6faf5db8-b434-437f-99c8-70299f82dab4</Id>
<Id>5b3ddaf1-3412-471a-a6cf-71f8e1c31168</Id>
<Id>1da6136d-2ff5-44cc-8510-4713451aac4d</Id>
</Ids>';
What I want to do is:
SELECT * FROM [MyTable] WHERE [Id] IN ( /* This list of Id's in the XML */ );
What is the desired way to do this?
Note: The format of the XML (passed into a Stored Procedure from C#) is also under my control, so if there is a better structure (for performance), then please include details.
Also, there could be 1000's of Id's in the real list if that makes a difference..
Thanks
Rob
If you're passing 1000's of Id's, I don't think this will be a stellar performer, but give it a try:
DECLARE #IdList XML;
SET #IdList =
'<Ids>
<Id>6faf5db8-b434-437f-99c8-70299f82dab4</Id>
<Id>5b3ddaf1-3412-471a-a6cf-71f8e1c31168</Id>
<Id>1da6136d-2ff5-44cc-8510-4713451aac4d</Id>
</Ids>';
select * from mytable where id in
(
select cast(T.c.query('text()') as varchar(36)) as result from #idlist.nodes('/Ids/Id') as T(c)
)
You might look at table valued parameters as a better way, unless you already have your data in XML.
I am using SQL Server 2008, and I am struggling with learning how to correctly call a User-defined In-line Table-Valued Function on a User-defined In-line Table-Valued Function (that is, since each expects a scalar or scalars as input and outputs a table, I want to learn how to correctly call one by passing it another table, whereupon each row is treated as its scalar inputs).
I posted a couple questions related to this recently, but I think I was not clear enough, and did not sufficiently encapsulate the problem to cleanly demonstrate it. I have now prepared the proper statements to provide anyone interested in helping the necessary tables, views, functions, and SELECT outputs to see the problem occur in front of them by executing the query below.
There are several ways I can phrase the core question, and from here and other forums, I can tell I have difficulty clearly expressing it. I am going to phrase it several ways here, but these are all meant to be the same question, phrased differently so people from different backgrounds can more easily understand me.
How do I correctly write the "imageFileNameFromAddress" function below so it works as intended; to wit, the intent is that it takes the same input as "bookAndPageFromAddress" and, using bookAndPageFromAddress and imageFileNameFromBookPage, passing the input to the first, then its output to the second, and returns the second's output?
Why does the third SELECT statement at the bottom below provide different results from the second one, and how do I fix the underlying function(s) to provide identical results, without repeating code from the other functions?
What is the correct syntax for the OUTER APPLY call in imageFileNameFromAddress so that its output is not null?
WARNING: The code below constructs the necessary tables, views, and functions to demonstrate the problem by dropping them first if they exist, so please please please check first to make sure you don't drop anything of your own! The final three SELECTS demonstrate the problem; the final two SELECTS should have identical output, but do not - the first one (of the final two, so the middle of the three) is a three row table of strings, and the final one is a one row table containing only a NULL.
USE [TOM_GIS]
GO
IF OBJECT_ID(N'[dbo].[constant]', N'U') IS NOT NULL
DROP TABLE [dbo].[constant]
CREATE TABLE [dbo].[constant]
(
ID INT IDENTITY(1,1) PRIMARY KEY CLUSTERED,
BOOK varchar(5),
PAGE varchar(5),
DocID numeric(8, 0)
)
INSERT INTO [dbo].[constant]
VALUES(' 4043',' 125', 576030)
GO
IF OBJECT_ID(N'[dbo].[images]', N'U') IS NOT NULL
DROP TABLE [dbo].[images]
CREATE TABLE [dbo].[images]
(
ID INT IDENTITY(1,1) PRIMARY KEY CLUSTERED,
DocID numeric(8, 0),
ImageID numeric(12,0)
)
INSERT INTO [dbo].[images] VALUES(576030, 1589666);
INSERT INTO [dbo].[images] VALUES(576030, 1589667);
INSERT INTO [dbo].[images] VALUES(576030, 1589668);
GO
IF OBJECT_ID(N'[dbo].[addressBookPage]', N'U') IS NOT NULL
DROP TABLE [dbo].[addressBookPage]
CREATE TABLE [dbo].[addressBookPage]
(
ID INT IDENTITY(1,1) PRIMARY KEY CLUSTERED,
PARCEL_ADDRESS nvarchar(50),
BOOK nchar(10),
PAGE nchar(10),
)
INSERT INTO [dbo].[addressBookPage]
VALUES('155 CENTER STREET','4043', '125')
GO
IF OBJECT_ID(N'[dbo].[vw_quindraco]') IS NOT NULL
DROP VIEW [dbo].[vw_quindraco]
GO
CREATE VIEW [dbo].[vw_quindraco]
AS
WITH files AS (SELECT RIGHT('00000000' + LTRIM(STR(c.DocID)), 8) AS PathInfo
,RIGHT('0000000000' + LTRIM(STR(i.ImageID)), 12) AS FileName
,ltrim(c.Book) as Book
,ltrim(c.Page) as Page
FROM [dbo].[constant] AS c INNER JOIN
[dbo].[images] AS i ON c.DocID = i.DocID)
SELECT 'Images/' + SUBSTRING(PathInfo, 1, 2) + '/' + SUBSTRING(PathInfo, 3, 2) + '/' + SUBSTRING(PathInfo, 5, 2)
+ '/' + RIGHT(PathInfo, 8) + '/' + FileName + '.tif' AS FullFileName
,Book
,Page
FROM files AS files_1
GO
IF OBJECT_ID(N'[dbo].[bookAndPageFromAddress]') IS NOT NULL
DROP FUNCTION [dbo].[bookAndPageFromAddress];
GO
CREATE FUNCTION [dbo].[bookAndPageFromAddress] (#address NVARCHAR(max))
RETURNS TABLE AS RETURN(
SELECT PARCEL_ADDRESS AS Address, Book, Page
FROM [dbo].[addressBookPage]
WHERE PARCEL_ADDRESS like '%' + #address + '%'
);
GO
IF OBJECT_ID(N'[dbo].[imageFileNameFromBookPage]') IS NOT NULL
DROP FUNCTION [dbo].[imageFileNameFromBookPage];
GO
CREATE FUNCTION [dbo].[imageFileNameFromBookPage] (#book nvarchar(max), #page nvarchar(max))
RETURNS TABLE AS RETURN(
SELECT i.FullFileName
FROM [dbo].[vw_quindraco] i
WHERE i.Book like #book
AND i.Page like #page
);
GO
IF OBJECT_ID(N'[dbo].[imageFileNameFromAddress]') IS NOT NULL
DROP FUNCTION [dbo].[imageFileNameFromAddress];
GO
CREATE FUNCTION [dbo].[imageFileNameFromAddress] (#address NVARCHAR(max))
RETURNS TABLE AS RETURN(
SELECT *
FROM [dbo].[bookAndPageFromAddress](#address) addresses
OUTER APPLY [dbo].[imageFileNameFromBookPage](addresses.Book, addresses.Page) foo
);
GO
SELECT Book,Page FROM [dbo].[bookAndPageFromAddress]('155 Center Street');
SELECT FullFileName FROM [dbo].[imageFileNameFromBookPage]('4043','125');
SELECT FullFileName FROM [dbo].[imageFileNameFromAddress]('155 Center Street')
You have your table fields as nchars, and you are using Like.
Because it's nchar, the value is padded with spaces to the declared length (10).
Because it's Like, the spaces are considered essential part of a match, whereas the equality operator, =, would ignore trailing spaces.
Because data types in the table and in the function parameters do not match, implicit conversions happen in the background, ultimately causing comparison to fail because of spaces.
Use = instead of Like inside imageFileNameFromBookPage to quickly fix it.
Better yet, use correct data types in all functions and views to avoid any conversions.
I made the following function in SQL Server 2008 earlier this week that takes two parameters and uses them to select a column of "detail" records and returns them as a single varchar list of comma separated values. Now that I get to thinking about it, I would like to take this table and application-specific function and make it more generic.
I am not well-versed in defining SQL functions, as this is my first. How can I change this function to accept a single "column" worth of data, so that I can use it in a more generic way?
Instead of calling:
SELECT ejc_concatFormDetails(formuid, categoryName)
I would like to make it work like:
SELECT concatColumnValues(SELECT someColumn FROM SomeTable)
Here is my function definition:
FUNCTION [DNet].[ejc_concatFormDetails](#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000) AS
BEGIN
DECLARE #returnData VARCHAR(1000)
DECLARE #currentData VARCHAR(75)
DECLARE dataCursor CURSOR FAST_FORWARD FOR
SELECT data FROM DNet.ejc_FormDetails WHERE formuid = #formuid AND category = #category
SET #returnData = ''
OPEN dataCursor
FETCH NEXT FROM dataCursor INTO #currentData
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #returnData = #returnData + ', ' + #currentData
FETCH NEXT FROM dataCursor INTO #currentData
END
CLOSE dataCursor
DEALLOCATE dataCursor
RETURN SUBSTRING(#returnData,3,1000)
END
As you can see, I am selecting the column data within my function and then looping over the results with a cursor to build my comma separated varchar.
How can I alter this to accept a single parameter that is a result set and then access that result set with a cursor?
Others have answered your main question - but let me point out another problem with your function - the terrible use of a CURSOR!
You can easily rewrite this function to use no cursor, no WHILE loop - nothing like that. It'll be tons faster, and a lot easier, too - much less code:
FUNCTION DNet.ejc_concatFormDetails
(#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000)
AS
RETURN
SUBSTRING(
(SELECT ', ' + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
FOR XML PATH('')
), 3, 1000)
The trick is to use the FOR XML PATH('') - this returns a concatenated list of your data columns and your fixed ', ' delimiters. Add a SUBSTRING() on that and you're done! As easy as that..... no dogged-slow CURSOR, no messie concatenation and all that gooey code - just one statement and that's all there is.
You can use table-valued parameters:
CREATE FUNCTION MyFunction(
#Data AS TABLE (
Column1 int,
Column2 nvarchar(50),
Column3 datetime
)
)
RETURNS NVARCHAR(MAX)
AS BEGIN
/* here you can do what you want */
END
You can use Table Valued Parameters as of SQL Server 2008, which would allow you to pass a TABLE variable in as a parameter. The limitations and examples for this are all in that linked article.
However, I'd also point out that using a cursor could well be painful for performance.
You don't need to use a cursor, as you can do it all in 1 SELECT statement:
SELECT #MyCSVString = COALESCE(#MyCSVString + ', ', '') + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
No need for a cursor
Your question is a bit unclear. In your first SQL statement it looks like you're trying to pass columns to the function, but there is no WHERE clause. In the second SQL statement you're passing a collection of rows (results from a SELECT). Can you supply some sample data and expected outcome?
Without fully understanding your goal, you could look into changing the parameter to be a table variable. Fill a table variable local to the calling code and pass that into the function. You could do that as a stored procedure though and wouldn't need a function.