I got a searchbox, the sql statment is like:
SELECT ..... WHERE ... LIKE '%A%'
but when user entered "A B" (space between A and B)
I want to select all records which content contains both A and B.
But I can not write:
.... LIKE '%A%B%'
because it won't match string that B occurs first like: "ilikeBnotA"
It should be
...LIKE '%A%' AND ...LIKE '%B%'
But the user may input more than one spance, like "A B C ...", so i need to write a loop, contacting every keyword into:
AND ...LIKE '%keyword%'
I don't think it's a good idea, i doubt it may cause performance issues when searching large string (like a blog post content).
Is there any good solution to this problem? Dow do search engines do that?
First you need a split function:
CREATE function [dbo].[f_split]
(
#param nvarchar(max),
#delimiter char(1)
)
returns #t table (val nvarchar(max), seq int)
as
begin
set #param += #delimiter
;with a as
(
select cast(1 as bigint) f, charindex(#delimiter, #param) t, 1 seq
union all
select t + 1, charindex(#delimiter, #param, t + 1), seq + 1
from a
where charindex(#delimiter, #param, t + 1) > 0
)
insert #t
select substring(#param, f, t - f), seq from a
option (maxrecursion 0)
return
end
Then you can search like this:
-- declaring a tablevariable to represent your table
declare #yourtable table(id int identity(1,1), searchcol varchar(50))
insert #yourtable values('abc')
insert #yourtable values('za')
insert #yourtable values('az')
insert #yourtable values('zz')
declare #input varchar(50)
set #input = 'a b c'
-- show if one or more match exists
select * from #yourtable a
where exists (select 1 from f_split(#input, ' ') b
where a.searchcol like '%'+ b.val + '%')
--show only if all matches exists
select * from #yourtable a
where not exists (select 1 from clausens_base.dbo.f_split(#input, ' ') b
where not a.searchcol like '%'+ b.val + '%')
You can use regular expressions with SQL and write a regex to match all given strings:
(?=.*one)(?=.*two)(?=.*three)
Below written code should do in case of one space char is there and your search string is in a variable called #a
...LIKE LEFT(#a,CHARINDEX(' ',#a)) AND ... LIKE RIGHT(#a,CHARINDEX(' ',#a))
Create another column in your table, say keywords, and when inserting data also create the keyword of your data by removing whitespace characters and insert to this column.
When searching search the keyword in this column.
create function createKeyword(#data nvarchar(256))
returns nvarchar(256)
as
begin
-- remove whitespace characters
return #kword
end
And when searching use
t.keywords like '%' + dbo.createKeyword(#data) + '%'
Related
I have a query like below
SELECT vals.*
FROM table AS rels
INNER JOIN table AS vals ON vals.Id IN (
SELECT value
FROM STRING_SPLIT(rels.relatedID, ',')
)
WHERE rels.Id = 'x'
I want to avoid using STRING_SPLIT method here. Is there any possible way for it?
You can use LIKE:
SELECT vals.*
FROM table rels INNER JOIN
table vals
ON CONCAT(',', rels.relatedID, ',') LIKE CONCAT('%,', vals.Id, ',%')
WHERE rels.Id = 'x';
However, your efforts should be put to fixing the data model. Storing multiple values in a string column is just the wrong way to store data in a SQL database.
You can create function like below. Here in below CTE f is starting index and t is ending index of string each one separated by Delimiter.
Then loop over CTE and applied SUBSTRING to your original string and returned the result.
CREATE FUNCTION [dbo].[SplitString]
(
#String nvarchar(max),
#Delimiter char(1)
)
RETURNS #t table (value nvarchar(max), sequence int)
AS
BEGIN
SET #String += #Delimiter
;WITH a AS
(
SELECT CAST(1 AS bigint) f, CHARINDEX(#Delimiter, #String) t, 1 seq
UNION ALL
SELECT t + 1, CHARINDEX(#Delimiter, #String, t + 1), seq + 1
FROM a
WHERE CHARINDEX(#Delimiter, #String, t + 1) > 0
)
INSERT #t
SELECT SUBSTRING(#String, f, t - f), seq
FROM a
OPTION (maxrecursion 0)
RETURN
END
So I am trying to write a query which, among other things, brings back the first character in a Varchar field, then returns the first character which appears after each / character throughout the rest of the field.
The field I am refrering too will contain a group of last names, separated by a '/'. For example: Fischer-Costello/Korbell/Morrison/Pearson
For the above example, I would want my select statement to return: FKMP.
So far, I have only been able to get my code to return the first character + the first character after the FIRST (and only the first) '/' character.
So for the above example input, my select statement would return: FK
Here is the code that I have written so far:
select rp.CONTACT_ID, ra.TRADE_REP, c.FIRST_NAME, c.LAST_NAME,
UPPER(LEFT(FIRST_NAME, 1)) + SUBSTRING(c.first_name,CHARINDEX('/',c.first_name)+1,1) as al_1,
UPPER(LEFT(LAST_NAME, 1)) + SUBSTRING(c.LAST_name,CHARINDEX('/',c.LAST_name)+1,1) as al_2
from dbo.REP_ALIAS ra
inner join dbo.REP_PROFILE rp on rp.CONTACT_ID = ra.CONTACT_ID
inner join dbo.CONTACT c on rp.CONTACT_ID = c.CONTACT_ID
where
rp.CRD_NUMBER is null and
ra.TRADE_REP like '%DNK%' and
(c.LAST_NAME like '%/%' or c.FIRST_NAME like '%/%') and
ra.TRADE_FIRM in
(
'xxxxxxx',
'xxxxxxx'
)
If you read the code, it's obvious that I am attempting to perform the same concatenation on the first_name column as well. However, I realize that a solution which will work for the Last_name column (used in my example), will also work for the first_name column.
Thank you.
Some default values
DECLARE #List VARCHAR(50) = 'Fischer-Costello/Korbell/Morrison/Pearson'
DECLARE #SplitOn CHAR(1) = '/'
This area just splits the string into a list
DECLARE #RtnValue table
(
Id int identity(1,1),
Value nvarchar(4000)
)
While (Charindex(#SplitOn, #List)>0)
Begin
Insert Into #RtnValue (value)
Select
Value = ltrim(rtrim(Substring(#List,1,Charindex(#SplitOn,#List)-1)))
Set #List = Substring(#List,Charindex(#SplitOn,#List)+len(#SplitOn+',')-1,len(#List))
End
Insert Into #RtnValue (Value)
Select Value = ltrim(rtrim(#List))
Now lets grab the first character of each name and stuff it back into a single variable
SELECT STUFF((SELECT SUBSTRING(VALUE,1,1) FROM #RtnValue FOR XML PATH('')),1,0,'') AS Value
Outputs:
Value
FKMP
Here is another way to do this would be a lot faster than looping. What you need is a set based splitter. Jeff Moden at sql server central has one that is awesome. Here is a link to the article. http://www.sqlservercentral.com/articles/Tally+Table/72993/
Now I know you have to signup for an account to view this but it is free and the logic in that article will change the way you look at data. You might also be able to find his code posted if you search for DelimitedSplit8K.
At any rate, here is how you could implement this type of splitter.
declare #Table table(ID int identity, SomeValue varchar(50))
insert #Table
select 'Fischer-Costello/Korbell/Morrison/Pearson'
select ID, STUFF((select '' + left(x.Item, 1)
from #Table t2
cross apply dbo.DelimitedSplit8K(SomeValue, '/') x
where t2.ID = t1.ID
for xml path('')), 1, 0 , '') as MyResult
from #Table t1
group by t1.ID
I would like to do something like this :
DECLARE #list nvarhcar(200) SET #list = 'VALUE1,VALUE2,VALUE3'
Select * from foo where field in (#list)
Until today I solved using sp_executeSQL procedure, my own database function csv2table and subselect, but I think that is not very clean.
Is there any way to solve without use dynamic sql ? Is there a direct method?
I'm using Microsoft Sql Server 2005.
Thanks!
Would you please try as below: thanks
DECLARE #list nvarchar(200)
SET #list = 'VALUE1,VALUE2,VALUE3'
SELECT * FROM foo WHERE CHARINDEX(',' + field + ',', ',' + #list + ',') > 0
Create Part A as an UDF
/* Part A */
DECLARE #list Varchar(max) = 'VALUE1,VALUE2,VALUE3'
DECLARE #tmpTbl_Values Table (ID varchar(50))
Set #list = #list + ','
-- Indexes to keep the position of searching
Declare #Pos1 Int
Declare #pos2 Int
-- Start from first character
Set #Pos1=1
Set #Pos2=1
While #Pos1<Len(#list)
Begin
Set #Pos1 = CharIndex(',',#list,#Pos1)
Insert #tmpTbl_Values Select Cast(Substring(#list,#Pos2,#Pos1-#Pos2) As varchar(50))
-- Go to next non comma character
Set #Pos2=#Pos1+1
-- Search from the next charcater
Set #Pos1 = #Pos1+1
End
/* Part B */
SELECT *
FROM foo A
INNER JOIN #tmpTbl_Values B
ON AB.ID = B.ID
One alternative:
Select * from foo where '%,' + field + ',%' like ',' + (#list) + ','
This is even "dirtier" than the existing approach.
Ideally, I would suggest changing the query string to:
Select * from foo where field in ()
then reading through the list of values in #list, inserting a ? for each value (separated by commas) and binding each value to that parameter marker. However, I don't know how to achieve this in dynamic SQL alone.
If the datatype for the field is nchar or varchar, the IN() operator will look for value like
IN('VALUE1','VALUE2','VALUE3', etc..)
So you set your #list based on that
A possible approach is to do the following:
SELECT * FROM foo WHERE CHARINDEX(',' + field + ',', ',' + #list + ',') > 0
This is assuming that the data in the field has no comma in it
If you deal with a string field, you could use something like this..
DECLARE #csv varchar(50) = 'item1,item2, item3, item4 ,item5'
SELECT * FROM foo WHERE PATINDEX('%' + Field + '%', #csv) > 0
In SQL 2005 you can use a custom UDF to parse a delimited list into a table - please see this article http://www.sommarskog.se/arrays-in-sql-2005.html
How can I do this kind of selection:
SELECT *
FROM Street
WHERE StreetName LIKE IN ('% Main Street', 'foo %')
Please don't tell me that I can use OR because these actually comes from a query.
There is no combined LIKE and IN syntax but you can use LIKE to JOIN onto your query as below.
;WITH Query(Result) As
(
SELECT '% Main Street' UNION ALL
SELECT 'foo %'
)
SELECT DISTINCT s.*
FROM Street s
JOIN Query q ON StreetName LIKE q.Result
Or to use your example in the comments
SELECT DISTINCT s.*
FROM Street s
JOIN CarStreets cs ON s.StreetName LIKE cs.name + '%'
WHERE cs.Streets = 'offroad'
You don't have a lot of choices here.
SELECT * FROM Street Where StreetName LIKE '% Main Street' OR StreetName LIKE 'foo %'
If this is part of an existing, more complicated query (which is the impression I'm getting), you could create a table value function that does the checking for you.
SELECT * FROM Street Where StreetName IN (dbo.FindStreetNameFunction('% Main Street|foo %'))
I'd recommend using the simplest solution (the first). If this is nested inside a larger, more complicated query, post it and we'll take a look.
I had a similar conundrum but due to only needing to match the start of a string, I changed my 'like' to SUBSTRING as such:
SELECT *
FROM codes
WHERE SUBSTRING(code, 1, 12) IN ('012316963429', '012315667849')
You can resort to Dynamic SQL and wrapping up all in a stored procedure.
If you get the LIKE IN param in a string as tokens with a certain separator, like
'% Main Street,foo %,Another%Street'
first you need to create a function that receives a list of LIKE "tokens" and returns a table of them.
CREATE FUNCTION [dbo].[SplitList]
(
#list nvarchar(MAX),
#delim nvarchar(5)
)
RETURNS #splitTable table
(
value nvarchar(50)
)
AS BEGIN
While (Charindex(#delim, #list)>0) Begin
Insert Into #splitTable (value)
Select ltrim(rtrim(Substring(#list, 1, Charindex(#delim, #list)-1)))
Set #list = Substring(#list, Charindex(#delim, #list)+len(#delim), len(#list))
End
Insert Into #splitTable (value) Select ltrim(rtrim(#list))
Return
END
Then in the SP you have the following code
declare
#sql nvarchar(MAX),
#subWhere nvarchar(MAX)
#params nvarchar(MAX)
-- prepare the where sub-clause to cover LIKE IN (...)
-- it will actually generate where sub clause StreetName Like option1 or StreetName Like option2 or ...
set #subWhere = STUFF(
(
--(**)
SELECT ' OR StreetName like ''' + value + '''' FROM SplitList('% Main Street,foo %,Another%Street', ',')
FOR XML PATH('')
), 1, 4, '')
-- create the dynamic SQL
set #sql ='select * from [Street]
where
(' + #subWhere + ')
-- and any additional query params here, if needed, like
AND StreetMinHouseNumber = #minHouseNumber
AND StreetNumberOfHouses between (#minNumberOfHouses and #maxNumberOfHouses)'
set #params = ' #minHouseNumber nvarchar(5),
#minNumberOfHouses int,
#minNumberOfHouses int'
EXECUTE sp_executesql #sql, #params,
#minHouseNumber,
#minNumberOfHouses,
#minNumberOfHouses
Of course, if you have your LIKE IN parameters in another table or you gather it through a query, you can replace that in line (**)
I believe I can clarify what he is looking for, but I don't know the answer. I'll use my situation to demonstrate. I have a table with a column called "Query" that holds SQL queries. These queries sometimes contain table names from one of my databases. I need to find all Query rows that contain table names from a particular database. So, I can use the following code to get the table names:
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
I'm trying to use a WHERE IN clause to identify the Query rows that contain the table names I'm interested in:
SELECT *
FROM [DatasourceQuery]
WHERE Query IN LIKE
(
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
)
I believe the OP is trying to do something like that.
This is my way:
First create a table function:
create function [splitDelimeter](#str nvarchar(max), #delimeter nvarchar(10)='*')
returns #r table(val nvarchar(max))
as
begin
declare #x nvarchar(max)=#str
set #x='<m>'+replace(#x, #delimeter, '</m><m>')+'</m>'
declare #xx xml=cast(#x as xml)
insert #r(val)
SELECT Tbl.Col.value('.', 'nvarchar(max)') id
FROM #xx.nodes('/m') Tbl(Col)
return
end
Then split the search text with your preference delimeter. After that you can do your select with left join as below:
declare #s nvarchar(max)='% Main Street*foo %'
select a.* from street a
left join gen.splitDelimeter(#s, '*') b
on a.streetname like b.val
where val is not null
What I did when solving a similar problem was:
SELECT DISTINCT S.*
FROM Street AS S
JOIN (SELECT value FROM String_Split('% Main Street,foo %', N',')) T
ON S.StreetName LIKE T.value;
Which is functionally similar to Martin's answer but a more direct answer to the question.
Note: DISTINCT is used because you might get multiple matches for a single row.
I have some column EntityName, and I want to have users to be able to search names by entering words separated by space. The space is implicitly considered as an 'AND' operator, meaning that the returned rows must have all of the words specified, and not necessarily in the given order.
For example, if we have rows like these:
abba nina pretty balerina
acdc you shook me all night long
sth you are me
dream theater it's all about you
when the user enters: me you, or you me (the results must be equivalent), the result has rows 2 and 3.
I know I can go like:
WHERE Col1 LIKE '%' + word1 + '%'
AND Col1 LIKE '%' + word2 + '%'
but I wanted to know if there's some more optimal solution.
The CONTAINS would require a full text index, which (for various reasons) is not an option.
Maybe Sql2008 has some built-in, semi-hidden solution for these cases?
The only thing I can think of is to write a CLR function that does the LIKE comparisons. This should be many times faster.
Update: Now that I think about it, it makes sense CLR would not help. Two other ideas:
1 - Try indexing Col1 and do this:
WHERE (Col1 LIKE word1 + '%' or Col1 LIKE '%' + word1 + '%')
AND (Col1 LIKE word2 + '%' or Col1 LIKE '%' + word2 + '%')
Depending on the most common searches (starts with vs. substring), this may offer an improvement.
2 - Add your own full text indexing table where each word is a row in the table. Then you can index properly.
Function
CREATE FUNCTION [dbo].[fnSplit] ( #sep CHAR(1), #str VARCHAR(512) )
RETURNS TABLE AS
RETURN (
WITH Pieces(pn, start, stop) AS (
SELECT 1, 1, CHARINDEX(#sep, #str)
UNION ALL
SELECT pn + 1, stop + 1, CHARINDEX(#sep, #str, stop + 1)
FROM Pieces
WHERE stop > 0
)
SELECT
pn AS Id,
SUBSTRING(#str, start, CASE WHEN stop > 0 THEN stop - start ELSE 512 END) AS Data
FROM
Pieces
)
Query
DECLARE #FilterTable TABLE (Data VARCHAR(512))
INSERT INTO #FilterTable (Data)
SELECT DISTINCT S.Data
FROM fnSplit(' ', 'word1 word2 word3') S -- Contains words
SELECT DISTINCT
T.*
FROM
MyTable T
INNER JOIN #FilterTable F1 ON T.Col1 LIKE '%' + F1.Data + '%'
LEFT JOIN #FilterTable F2 ON T.Col1 NOT LIKE '%' + F2.Data + '%'
WHERE
F2.Data IS NULL
Source: SQL SELECT WHERE field contains words
http://msdn.microsoft.com/en-us/magazine/cc163473.aspx
You're going to end up with a full table scan anyway.
The collation can make a big difference apparently. Kalen Delaney in the book "Microsoft SQL Server 2008 Internals" says:
Collation can make a huge difference
when SQL Server has to look at almost
all characters in the strings. For
instance, look at the following:
SELECT COUNT(*) FROM tbl WHERE longcol LIKE '%abc%'
This may execute 10 times faster or more with a binary collation than a nonbinary Windows collation. And with varchar data, this executes up to seven or eight times faster with a SQL collation than with a Windows collation.
WITH Tokens AS(SELECT 'you' AS Token UNION ALL SELECT 'me')
SELECT ...
FROM YourTable AS t
WHERE (SELECT COUNT(*) FROM Tokens WHERE y.Col1 LIKE '%'+Tokens.Token+'%')
=
(SELECT COUNT(*) FROM Tokens) ;
This should ideally be done with the help of Full text search as mentioned above.
BUT,
If you don't have full text configured for your DB, here is a performance intensive solution for doing a prioritized string search.
-- table to search in
drop table if exists dbo.myTable;
go
CREATE TABLE dbo.myTable
(
myTableId int NOT NULL IDENTITY (1, 1),
code varchar(200) NOT NULL,
description varchar(200) NOT NULL -- this column contains the values we are going to search in
) ON [PRIMARY]
GO
-- function to split space separated search string into individual words
drop function if exists [dbo].[fnSplit];
go
CREATE FUNCTION [dbo].[fnSplit] (#StringInput nvarchar(max),
#Delimiter nvarchar(1))
RETURNS #OutputTable TABLE (
id nvarchar(1000)
)
AS
BEGIN
DECLARE #String nvarchar(100);
WHILE LEN(#StringInput) > 0
BEGIN
SET #String = LEFT(#StringInput, ISNULL(NULLIF(CHARINDEX(#Delimiter, #StringInput) - 1, -1),
LEN(#StringInput)));
SET #StringInput = SUBSTRING(#StringInput, ISNULL(NULLIF(CHARINDEX
(
#Delimiter, #StringInput
),
0
), LEN
(
#StringInput)
)
+ 1, LEN(#StringInput));
INSERT INTO #OutputTable (id)
VALUES (#String);
END;
RETURN;
END;
GO
-- this is the search script which can be optionally converted to a stored procedure /function
declare #search varchar(max) = 'infection upper acute genito'; -- enter your search string here
-- the searched string above should give rows containing the following
-- infection in upper side with acute genitointestinal tract
-- acute infection in upper teeth
-- acute genitointestinal pain
if (len(trim(#search)) = 0) -- if search string is empty, just return records ordered alphabetically
begin
select 1 as Priority ,myTableid, code, Description from myTable order by Description
return;
end
declare #splitTable Table(
wordRank int Identity(1,1), -- individual words are assinged priority order (in order of occurence/position)
word varchar(200)
)
declare #nonWordTable Table( -- table to trim out auxiliary verbs, prepositions etc. from the search
id varchar(200)
)
insert into #nonWordTable values
('of'),
('with'),
('at'),
('in'),
('for'),
('on'),
('by'),
('like'),
('up'),
('off'),
('near'),
('is'),
('are'),
(','),
(':'),
(';')
insert into #splitTable
select id from dbo.fnSplit(#search,' '); -- this function gives you a table with rows containing all the space separated words of the search like in this e.g., the output will be -
-- id
-------------
-- infection
-- upper
-- acute
-- genito
delete s from #splitTable s join #nonWordTable n on s.word = n.id; -- trimming out non-words here
declare #countOfSearchStrings int = (select count(word) from #splitTable); -- count of space separated words for search
declare #highestPriority int = POWER(#countOfSearchStrings,3);
with plainMatches as
(
select myTableid, #highestPriority as Priority from myTable where Description like #search -- exact matches have highest priority
union
select myTableid, #highestPriority-1 as Priority from myTable where Description like #search + '%' -- then with something at the end
union
select myTableid, #highestPriority-2 as Priority from myTable where Description like '%' + #search -- then with something at the beginning
union
select myTableid, #highestPriority-3 as Priority from myTable where Description like '%' + #search + '%' -- then if the word falls somewhere in between
),
splitWordMatches as( -- give each searched word a rank based on its position in the searched string
-- and calculate its char index in the field to search
select myTable.myTableid, (#countOfSearchStrings - s.wordRank) as Priority, s.word,
wordIndex = CHARINDEX(s.word, myTable.Description) from myTable join #splitTable s on myTable.Description like '%'+ s.word + '%'
-- and not exists(select myTableid from plainMatches p where p.myTableId = myTable.myTableId) -- need not look into myTables that have already been found in plainmatches as they are highest ranked
-- this one takes a long time though, so commenting it, will have no impact on the result
),
matchingRowsWithAllWords as (
select myTableid, count(myTableid) as myTableCount from splitWordMatches group by(myTableid) having count(myTableid) = #countOfSearchStrings
)
, -- trim off the CTE here if you don't care about the ordering of words to be considered for priority
wordIndexRatings as( -- reverse the char indexes retrived above so that words occuring earlier have higher weightage
-- and then normalize them to sequential values
select s.myTableid, Priority, word, ROW_NUMBER() over (partition by s.myTableid order by wordindex desc) as comparativeWordIndex
from splitWordMatches s join matchingRowsWithAllWords m on s.myTableId = m.myTableId
)
,
wordIndexSequenceRatings as ( -- need to do this to ensure that if the same set of words from search string is found in two rows,
-- their sequence in the field value is taken into account for higher priority
select w.myTableid, w.word, (w.Priority + w.comparativeWordIndex + coalesce(sequncedPriority ,0)) as Priority
from wordIndexRatings w left join
(
select w1.myTableid, w1.priority, w1.word, w1.comparativeWordIndex, count(w1.myTableid) as sequncedPriority
from wordIndexRatings w1 join wordIndexRatings w2 on w1.myTableId = w2.myTableId and w1.Priority > w2.Priority and w1.comparativeWordIndex>w2.comparativeWordIndex
group by w1.myTableid, w1.priority,w1.word, w1.comparativeWordIndex
)
sequencedPriority on w.myTableId = sequencedPriority.myTableId and w.Priority = sequencedPriority.Priority
),
prioritizedSplitWordMatches as ( -- this calculates the cumulative priority for a field value
select w1.myTableId, sum(w1.Priority) as OverallPriority from wordIndexSequenceRatings w1 join wordIndexSequenceRatings w2 on w1.myTableId = w2.myTableId
where w1.word <> w2.word group by w1.myTableid
),
completeSet as (
select myTableid, priority from plainMatches -- get plain matches which should be highest ranked
union
select myTableid, OverallPriority as priority from prioritizedSplitWordMatches -- get ranked split word matches (which are ordered based on word rank in search string and sequence)
),
maximizedCompleteSet as( -- set the priority of a field value = maximum priority for that field value
select myTableid, max(priority) as Priority from completeSet group by myTableId
)
select priority, myTable.myTableid , code, Description from maximizedCompleteSet m join myTable on m.myTableId = myTable.myTableId
order by Priority desc, Description -- order by priority desc to get highest rated items on top
--offset 0 rows fetch next 50 rows only -- optional paging