I have a SQL view that calls a scalar function with a string parameter. The problem is that the string occasionally has special characters which causes the function to fail.
The view query looks like this:
SELECT TOP (100) PERCENT
Id, Name, StartDate, EndDate
,dbo.[fnGetRelatedInfo] (Name) as Information
FROM dbo.Session
The function looks like this:
ALTER FUNCTION [dbo].[fnGetRelatedInfo]( #Name varchar(50) )
RETURNS varchar(200)
AS
BEGIN
DECLARE #Result varchar(200)
SELECT #Result = ''
SELECT #Result = #Result + Info + CHAR(13)+CHAR(10)
FROM [SessionInfo]
WHERE SessionName = #Name
RETURN #Result
END
How do I escape the name value so it will work when passed to the function?
I am guessing that the problem is non-unicode characters in dbo.Session.Name. Since the parameter to the function is VARCHAR, it will only hold unicode characters, so the non-unicode characters are lost when being passed to the function. The solution for this would be to change the parameter to be NVARCHAR(50).
However, if you care about performance, and more importantly consistent, reliable results stop using this function immediately. Alter your view to simply be:
SELECT s.ID,
s.Name,
s.StartDate,
s.EndDate,
( SELECT si.Info + CHAR(13)+CHAR(10)
FROM SessionInfo AS si
WHERE si.SessionName = s.Name
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)') AS Information
FROM dbo.Session AS s;
Using variable concatenation can lead to unexpected results which are dependent on the internal pathways of the execution plan. So I would rule this out as a solution immediately. Not only this, the RBAR nature of a scalar UDF means that this will not scale well at all.
Various ways of doing this grouped concatenation have been benchmarked here, where CLR is actually the winner, but this is not always an option.
Related
Is there any way to make following query Work?
declare #t nvarchar(20)
set #t='حس'
SELECT [perno] ,[pName]
FROM [dbo].[People]
Where [pName] like N''+#t +'%'
I cann't use like this:
Where [pName] like N'حس%'
Or using an stored procedure :
ALTER PROCEDURE [dbo].[aTest]
(#t nvarchar(20))
AS
BEGIN
SELECT [perno] ,[pName]
FROM [dbo].[People]
WHERE ([People].[pName] LIKE N'' +#t + '%')
END
You don't need to use N prefix in the WHERE clause since your variable is already nvarchar, and you are passing a variable not a literal string.
Here is an example:
CREATE TABLE People
(
ID INT,
Name NVARCHAR(45)
);
INSERT INTO People VALUES
(1, N'حسام'),
(2, N'حسان'),
(3, N'حليم');
DECLARE #Name NVARCHAR(45) = N'حس';--You need to use N prefix when you pass the string literal
SELECT *
FROM People
WHERE Name LIKE #Name + '%'; --You can use it here when you pass string literal, but since you are passing a variable, you don't need N here
Live demo
You may have seen Transact-SQL code that passes strings around using an N prefix. This denotes that the subsequent string is in Unicode (the N actually stands for National language character set). Which means that you are passing an NCHAR, NVARCHAR or NTEXT value, as opposed to CHAR, VARCHAR or TEXT.
From docs
Prefix Unicode character string constants with the letter N. Without the N prefix, the string is converted to the default code page of the database. This default code page may not recognize certain characters.
To answer your question in the comment with a simple answer, you are using the wrong datatype, so ALTER the stored procedure and change the datatype of your parameter from VARCHAR to NVARCHAR.
UPDATE:
Since you are using an SP, you can create your SP (according to your comment) as
CREATE PROCEDURE MyProc
(
#Var NVARCHAR(45)
)
AS
BEGIN
SELECT *
FROM People
WHERE Name LIKE ISNULL(#Var, Name) + '%';
--Using ISNULL() will return all rows if you pass NULL to the stored procedure
END
and call it as
EXEC MyProc N'حس'; --If you don't use N prefix then you are pass a varchar string
If you see, you need to use the N prefix when you pass literal string to your SP not inside the SP or the WHERE clause neither.
Demo for the SP
in these lines
declare #t nvarchar(20)
set #t='حس'
the 'حس' is a varchar constant that you then assign to an nvarchar variable. But you already lost data with the original conversion to that varchar constant and you cannot get that back.
The solution is to use an nvarchar constant:
set #t=N'حس'
It might be much simpler:
Try this
declare #t nvarchar(20)
set #t='حس';
SELECT #t; --the result is "??"
You are declaring the variable as NVARCHAR correctly. But the literal does not know its target. Without the N it is taken as a VARCHAR with the default collation.
The following line
Where [pName] like N''+#t +'%'
will search for a pName LIKE '??%'.
The solution should be
set #t=N'حس'; --<-- N-prefix
How can I create a function like this?
function FN_something (#entrada char(50))
declare #consulta table
declare #notificacao varchar(50)
declare #multa float
declare #saida varchar(50)
set #consulta as = (select num_notificacao,num_multa from table where field = #entrada)
set #notificacao = #consulta.num_notificacao
set #multa = #consulta.num_multa
set #saida = "resultado: "+ #notificacao +";"+#multa
return #saida
Thanks in advance
I would not use a function... Scalar functions tend to be a real performance killer. Try to use something like this inline
SELECT 'resultado: '
+ ISNULL(CAST(t.num_notificacao AS VARCHAR(MAX)),'???')
+ ';'
+ ISNULL(CAST(t.num_multa AS VARCHAR(MAX)),'???')
FROM SomeTable AS t WHERE t.SomeField=#entrada;
If you need a function it was much better to use an inlined TVF (syntax without BEGIN...END and bind it into your query with CROSS APPLY.
Might be simplified:
If your columns are NOT NULL you can go without ISNULL()-function. If your columns are strings, you can do without CAST()... My code is defensive proramming :-D
Hint
If this is something you need more often, you might introduce a VIEW carrying this calculated column and use it instead of your table. You might include this value into your table as computed column as well...
UPDATE
Great, the VIEW you show in the comment is an inline TVF actually, which is very good!
My magic crystall ball tells me, that you might need something like this:
SELECT cl.*
,'resultado: ' + t.num_notificacao + ';' + t.num_multa AS CalculatedResult
FROM dbo.[CampoLivre876]('SomeParameter') AS cl
LEFT JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField --should be only one related row per main row!
This will call the iTFV and join it to the other Table, where the two columns are living. I Assume, that the CampoLivre876-row knows its entrada key.
Hint 2:
If this works for you, you might include this approach directly into your existing iTVF.
UPDATE 2
You might try to change your function like here:
ALTER FUNCTION [dbo].[CampoLivre876] ()
RETURNS TABLE
RETURN
Select cl.mul_numero_notificacao + ';' + CAST(cl.mul_valor_multa as varchar(max)) AS ExistingColumn
,'resultado: ' + t.num_notificacao + ';' + CAST(t.num_multa AS varchar(max)) AS CalculatedResult
From Campo_Livre AS cl With(NoLock)
INNER JOIN SomeOtherTable AS t ON cl.entrada=t.SomeField;
This should read all lines in one go. Reading 1 row after the other is - in almost all cases - something really, really bad...
Here is an example of a function with correct SQL Server syntax:
create function FN_something (
#entrada char(50) -- should probably be `varchar(50)` rather than `char(50)`
) returns varchar(50)
begin
declare #saida varchar(50);
select #saida = 'resultado: ' + num_notificacao + ';' + num_multa
from table
where field = #entrada;
return #saida;
end;
Note: This assumes that the num_ columns are strings, not numbers. If they are numbers, you need to convert them or use concat().
EDIT:
A function really isn't appropriate for this. Probably the best solution is a computed column:
alter table t add something as (concat('resultado: ', num_notificacao, ';', num_multa);
Then you can get the value directly from the table. In earlier versions of SQL Server, you would use a view rather than computed column.
I would like to know the most efficient way of removing any occurrence of characters like , ; / " from a varchar column.
I have a function like this but it is incredibly slow. The table has about 20 million records.
CREATE FUNCTION [dbo].[Udf_getcleanedstring] (#s VARCHAR(255))
returns VARCHAR(255)
AS
BEGIN
DECLARE #o VARCHAR(255)
SET #o = Replace(#s, '/', '')
SET #o = Replace(#o, '-', '')
SET #o = Replace(#o, ';', '')
SET #o = Replace(#o, '"', '')
RETURN #o
END
Whichever method you use it is probably worth adding a
WHERE YourCol LIKE '%[/-;"]%'
Except if you suspect that a very large proportion of rows will in fact contain at least one of the characters that need to be stripped.
As you are using this in an UPDATE statement then simply adding the WITH SCHEMABINDING attribute can massively improve things and allow the UPDATE to proceed row by row rather than needing to cache the entire operation in a spool first for Halloween Protection
Nested REPLACE calls in TSQL are slow anyway though as they involve multiple passes through the strings.
You could knock up a CLR function as below (if you haven't worked with these before then they are very easy to deploy from an SSDT project as long as CLR execution is permitted on the server). The UPDATE plan for this too does not contain a spool.
The Regular Expression uses (?:) to denote a non capturing group with the various characters of interest separated by the alternation character | as /|-|;|\" (the " needs to be escaped in the string literal so is preceded by a slash).
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Text.RegularExpressions;
public partial class UserDefinedFunctions
{
private static readonly Regex regexStrip =
new Regex("(?:/|-|;|\")", RegexOptions.Compiled);
[SqlFunction]
public static SqlString StripChars(SqlString Input)
{
return Input.IsNull ? null : regexStrip.Replace((string)Input, "");
}
}
I want to show the huge performance differences between the using with 2 types of USER DIFINED FUNCTIONS:
User TABLE function
User SCALAR function
See the test example :
use AdventureWorks2012
go
-- create table for the test
create table dbo.FindString (ColA int identity(1,1) not null primary key,ColB varchar(max) );
declare #text varchar(max) = 'A web server can handle a Hypertext Transfer Protocol request either by reading
a file from its file ; system based on the URL <> path or by handling the request using logic that is specific
to the type of resource. In the case that special logic is invoked the query string will be available to that logic
for use in its processing, along with the path component of the URL.';
-- init process in loop 1,000,000
insert into dbo.FindString(ColB)
select #text
go 1000000
-- use one of the scalar function from the answers which post in this thread
alter function [dbo].[udf_getCleanedString]
(
#s varchar(max)
)
returns varchar(max)
as
begin
return replace(replace(replace(replace(#s,'/',''),'-',''),';',''),'"','')
end
go
--
-- create from the function above new function an a table function ;
create function [dbo].[utf_getCleanedString]
(
#s varchar(255)
)
returns table
as return
(
select replace(replace(replace(replace(#s,'/',''),'-',''),';',''),'"','') as String
)
go
--
-- clearing the buffer cach
DBCC DROPCLEANBUFFERS ;
go
-- update process using USER TABLE FUNCTIO
update Dest with(rowlock) set
dest.ColB = D.String
from dbo.FindString dest
cross apply utf_getCleanedString(dest.ColB) as D
go
DBCC DROPCLEANBUFFERS ;
go
-- update process using USER SCALAR FUNCTION
update Dest with(rowlock) set
dest.ColB = dbo.udf_getCleanedString(dest.ColB)
from dbo.FindString dest
go
AND these are the execution plan :
As you can see the UTF is much better the USF ,they 2 doing the same thing replacing string, but one return scalar and the other return as a table
Another important parameter for you to see (SET STATISTICS IO ON ;)
How about nesting them together in a single call:
create function [dbo].[udf_getCleanedString]
(
#s varchar(255)
)
returns varchar(255)
as
begin
return replace(replace(replace(replace(#s,'/',''),'-',''),';',''),'"','')
end
Or you may want to do an UPDATE on the table itself for the first time. Scalar functions are pretty slow.
Here is a similar question asked previously, I like this approach mentioned here.
How to Replace Multiple Characters in SQL?
declare #badStrings table (item varchar(50))
INSERT INTO #badStrings(item)
SELECT '>' UNION ALL
SELECT '<' UNION ALL
SELECT '(' UNION ALL
SELECT ')' UNION ALL
SELECT '!' UNION ALL
SELECT '?' UNION ALL
SELECT '#'
declare #testString varchar(100), #newString varchar(100)
set #teststring = 'Juliet ro><0zs my s0x()rz!!?!one!#!#!#!'
set #newString = #testString
SELECT #newString = Replace(#newString, item, '') FROM #badStrings
select #newString -- returns 'Juliet ro0zs my s0xrzone'
I have a problem with SQL Server 2008 full text search I have the following query
SELECT *
FROM cigars
WHERE CONTAINS(cigars.*, #Search )
#Search is a value that is passed in through a stored procedure. But, thats not the problem.
What is is if someone searches for say 'Punch' it works fine and I get all cigars that match.
Where the problem lays is if they search for 'Punch Cigar' I get this error.
Syntax error near 'Cigar' in the full-text search condition 'Punch Cigar'.
Any idea on how I can get it to allow for it to search for that phrase?
Why are you searching by all columns in the CIGAR table? Surely some of them do not use a string/text based data type...
After looking at the CONTAINS documentation, I'd look at a function to properly escape the words for the FTS searching:
CREATE FUNCTION [dbo].[escapeFTSSearch] (
#SearchParameter NVARCHAR(MAX)
)
RETURNS NVARCHAR(MAX)
AS
BEGIN
-- Declare the return variable here
DECLARE #result NVARCHAR(MAX)
SELECT #result = '"'+ REPLACE(REPLACE(#SearchParameter,'"',''), ' ', '" AND "') +'"'
-- Return the result of the function
RETURN #result
END
Test:
SELECT [example].[dbo].[escapeFTSSearch] ('Punch Cigar')
...which gives me:
"Punch" AND "Cigar"
Usage:
WHERE CONTAINS(cigars.*, dbo.escapeFTSSearch(#Search) )
Addendum
The function is simplistic:
it assumes you want all words provided
doesn't support fuzzy searching
assumes double quotes aren't in the parameter value
Tweak as needed.
You need to ensure you have leading and trailing double quotes ans spaces. i.e. the value of #Search should be ' "Punch Cigar" '
Further to OMG's comments about escaping you would definitely need to strip out any embedded double quotes.
declare #Search varchar(1000)
set #Search = 'punch and" cigar'
set #Search = ' "' + REPLACE(#Search,'"','') + '" '
select * from sys.dm_fts_parser(#Search,1033,null,0)
I made the following function in SQL Server 2008 earlier this week that takes two parameters and uses them to select a column of "detail" records and returns them as a single varchar list of comma separated values. Now that I get to thinking about it, I would like to take this table and application-specific function and make it more generic.
I am not well-versed in defining SQL functions, as this is my first. How can I change this function to accept a single "column" worth of data, so that I can use it in a more generic way?
Instead of calling:
SELECT ejc_concatFormDetails(formuid, categoryName)
I would like to make it work like:
SELECT concatColumnValues(SELECT someColumn FROM SomeTable)
Here is my function definition:
FUNCTION [DNet].[ejc_concatFormDetails](#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000) AS
BEGIN
DECLARE #returnData VARCHAR(1000)
DECLARE #currentData VARCHAR(75)
DECLARE dataCursor CURSOR FAST_FORWARD FOR
SELECT data FROM DNet.ejc_FormDetails WHERE formuid = #formuid AND category = #category
SET #returnData = ''
OPEN dataCursor
FETCH NEXT FROM dataCursor INTO #currentData
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #returnData = #returnData + ', ' + #currentData
FETCH NEXT FROM dataCursor INTO #currentData
END
CLOSE dataCursor
DEALLOCATE dataCursor
RETURN SUBSTRING(#returnData,3,1000)
END
As you can see, I am selecting the column data within my function and then looping over the results with a cursor to build my comma separated varchar.
How can I alter this to accept a single parameter that is a result set and then access that result set with a cursor?
Others have answered your main question - but let me point out another problem with your function - the terrible use of a CURSOR!
You can easily rewrite this function to use no cursor, no WHILE loop - nothing like that. It'll be tons faster, and a lot easier, too - much less code:
FUNCTION DNet.ejc_concatFormDetails
(#formuid AS int, #category as VARCHAR(75))
RETURNS VARCHAR(1000)
AS
RETURN
SUBSTRING(
(SELECT ', ' + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
FOR XML PATH('')
), 3, 1000)
The trick is to use the FOR XML PATH('') - this returns a concatenated list of your data columns and your fixed ', ' delimiters. Add a SUBSTRING() on that and you're done! As easy as that..... no dogged-slow CURSOR, no messie concatenation and all that gooey code - just one statement and that's all there is.
You can use table-valued parameters:
CREATE FUNCTION MyFunction(
#Data AS TABLE (
Column1 int,
Column2 nvarchar(50),
Column3 datetime
)
)
RETURNS NVARCHAR(MAX)
AS BEGIN
/* here you can do what you want */
END
You can use Table Valued Parameters as of SQL Server 2008, which would allow you to pass a TABLE variable in as a parameter. The limitations and examples for this are all in that linked article.
However, I'd also point out that using a cursor could well be painful for performance.
You don't need to use a cursor, as you can do it all in 1 SELECT statement:
SELECT #MyCSVString = COALESCE(#MyCSVString + ', ', '') + data
FROM DNet.ejc_FormDetails
WHERE formuid = #formuid AND category = #category
No need for a cursor
Your question is a bit unclear. In your first SQL statement it looks like you're trying to pass columns to the function, but there is no WHERE clause. In the second SQL statement you're passing a collection of rows (results from a SELECT). Can you supply some sample data and expected outcome?
Without fully understanding your goal, you could look into changing the parameter to be a table variable. Fill a table variable local to the calling code and pass that into the function. You could do that as a stored procedure though and wouldn't need a function.