How to apply trim function inside this query [duplicate] - sql

Below is simple sql query to select records using in condition.
--like this I have 6000 usernames
select * from tblUsers where Username in ('abc ','xyz ',' pqr ',' mnop ' );
I know there are LTrim & Rtrim in sql to remove the leading trailing spaces form left & right respectively.
I want to remove the spaces from left & right in all the usernames that I am supplying to the select query.
Note:-
I want to trim the values that I am passing in the in clause.(I don't want to pass LTrim & RTrim to each value passed).
There are no trailing space in the records but value that I am passing in the clause is copied from excel & then pasted in Visual Studio. Then using ALT key I put '(single quote) at the left & right sides of the string. Due to this some strings has spaces in the right side trailing.
How to use the trim function in the select query?
I am using MS SQL Server 2012

If I understand your question correctly you are pasting from Excel into an IN clause in an adhoc query as below.
The trailing spaces don't matter. It will still match the string foo without any trailing spaces.
But you need to ensure that there are no leading spaces.
As the source of the data is Excel why not just do it all there?
You can use formula
= CONCATENATE("'",TRIM(SUBSTITUTE(A1,"'","''")),"',")
Then copy the result (from column B in the screenshot above) and just need to trim off the extra comma from the final entry.

You can do like this:
select * from tblUsers where LTRIM(RTRIM(Username)) in (ltrim(rtrim('abc')),ltrim(rtrim('xyz')),ltrim(rtrim('pqr')),ltrim(rtrim('mnop')));
However, if you have permission to update the database. Please remove all the spaces in your Username field. It is really not good to do the query like this.

One way to tackle your problem and still be able to benefit from an index on username is to use a persisted computed column:
Setup
-- drop table dbo.tblUsers
create table dbo.tblUsers
(
UserId INT NOT NULL IDENTITY(1, 1) CONSTRAINT PK_UserTest PRIMARY KEY,
Username NVARCHAR(64) NOT NULL,
UsernameTrimmed AS LTRIM(RTRIM(Username)) PERSISTED
)
GO
-- other columns may be included here with INCLUDE (col1, col2)
CREATE INDEX IDX_UserTest ON dbo.tblUsers (UsernameTrimmed)
GO
insert into dbo.tblUsers (Username) VALUES ('abc '),('xyz '),(' pqr '), (' mnop '), ('abc'), (' useradmin '), ('etc'), (' other user ')
GO
-- some mock data to obtain a large number of records
insert into dbo.tblUsers (Username)
select top 20000 SUBSTRING(text, 1, 64) from sys.messages
GO
Test
-- this will use the index (index seek)
select * from tblUsers where UsernameTrimmed in (LTRIM(RTRIM('abc')), LTRIM(RTRIM(' useradmin ')));
This allows for faster retrievals at the expense of extra space.
In order to get rid of query construction (and the ugliness of many LTRIMs and RTRIMs), you can push searched users in a table that looks like tblUsers.
create table dbo.searchedUsers
(
Username NVARCHAR(64) NOT NULL,
UsernameTrimmed AS LTRIM(RTRIM(Username)) PERSISTED
)
GO
Push raw values into dbo.searchedUsers.Username column and the query should look like this:
select U.*
from tblUsers AS U
join dbo.searchedUsers AS S ON S.UsernameTrimmed = U.UsernameTrimmed
The big picture
It is way better to properly trim your data in the service layer of your application (C#) so that future clients of your table may rely on decent information. So, trimming should be performed both when inserting information into tblUsers and when searching for users (IN values)

select *
from tblUsers
where RTRIM(LTRIM(Username)) in ('abc','xyz','pqr','mnop');

Answer: SELECT * FROM tblUsers WHERE LTRIM(RTRIM(Username)) in ('abc','xyz','pqr','mnop');
However, please note that if you have functions in your WHERE clause it defeats the purpose of having an indexes on that column and will use a
scan than a seek.
I would propose you clean your data before inserting into tblUsers

I think you can try this:
Just replace the table2 with you table name form where you are getting the username
select * from tblUsers where Username in ((select distinct
STUFF((SELECT distinct ', ' + RTRIM(LTRIM(t1.Username))
from table2 t1
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,2,'') UserName
from table2 t) );

I'd do it in two step:
1) populate a temp table with all your strings with blanks
2) do a select with a subselect
create table a (a char(1))
insert into a values('a')
insert into a values('b')
insert into a values('c')
insert into a values('d')
create table #b (atmp char(5))
insert into #b values ('a ')
insert into #b values (' b')
insert into #b values (' c ')
select * from a where a in (select ltrim(rtrim(atmp)) from #b)

Related

How to manipulate comma-separated list in SQL Server

I have a list of values such as
1,2,3,4...
that will be passed into my SQL query.
I need to have these values stored in a table variable. So essentially I need something like this:
declare #t (num int)
insert into #t values (1),(2),(3),(4)...
Is it possible to do that formatting in SQL Server? (turning 1,2,3,4... into (1),(2),(3),(4)...
Note: I can not change what those values look like before they get to my SQL script; I'm stuck with that list. also it may not always be 4 values; it could 1 or more.
Edit to show what values look like: under normal circumstances, this is how it would work:
select t.pk
from a_table t
where t.pk in (#place_holder#)
#placeholder# is just a literal place holder. when some one would run the report, #placeholder# is replaced with the literal values from the filter of that report:
select t.pk
from a_table t
where t.pk in (1,2,3,4) -- or whatever the user selects
t.pk is an int
note: doing
declare #t as table (
num int
)
insert into #t values (#Placeholder#)
does not work.
Your description is a bit ridicuolus, but you might give this a try:
Whatever you mean with this
I see what your trying to say; but if I type out '#placeholder#' in the script, I'll end up with '1','2','3','4' and not '1,2,3,4'
I assume this is a string with numbers, each number between single qoutes, separated with a comma:
DECLARE #passedIn VARCHAR(100)='''1'',''2'',''3'',''4'',''5'',''6'',''7''';
SELECT #passedIn; -->: '1','2','3','4','5','6','7'
Now the variable #passedIn holds exactly what you are talking about
I'll use a dynamic SQL-Statement to insert this in a temp-table (declared table variable would not work here...)
CREATE TABLE #tmpTable(ID INT);
DECLARE #cmd VARCHAR(MAX)=
'INSERT INTO #tmpTable(ID) VALUES (' + REPLACE(SUBSTRING(#passedIn,2,LEN(#passedIn)-2),''',''','),(') + ');';
EXEC (#cmd);
SELECT * FROM #tmpTable;
GO
DROP TABLE #tmpTable;
UPDATE 1: no dynamic SQL necessary, all ad-hoc...
You can get the list of numbers as derived table in a CTE easily.
This can be used in a following statement like WHERE SomeID IN(SELECT ID FROM MyIDs) (similar to this: dynamic IN section )
WITH MyIDs(ID) AS
(
SELECT A.B.value('.','int') AS ID
FROM
(
SELECT CAST('<x>' + REPLACE(SUBSTRING(#passedIn,2,LEN(#passedIn)-2),''',''','</x><x>') + '</x>' AS XML) AS AsXml
) as tbl
CROSS APPLY tbl.AsXml.nodes('/x') AS A(B)
)
SELECT * FROM MyIDs
UPDATE 2:
And to answer your question exactly:
With this following the CTE
insert into #t(num)
SELECT ID FROM MyIDs
... you would actually get your declared table variable filled - if you need it later...

How to replace underscore with a blank space with a regular expression in SQL

I'm trying to insert post codes into my database but getting rid of the underscores.
I have a table called FeedDataSetMapping that is used to map the fields before they get inserted:
INSERT INTO FeedDataSetMapping (
[source_field]
,[database_field]
,[template_id]
,[conversion_id]
,[order_id]
,[values_group]
,[direct_value]
,[value_regex]
,[condition_regex]
,[split_separator]
,[enclosing_character]
,[cumulative_field]
,[cumulative_format])
VALUES
('manufacturerId','manufacturer_Id',#template_id,0,0,null,null,null,null,null,null,null,null),
('dealership','leasing_broker_name',#template_id,0,0,null,null,null,null,null,null,null,null),
('manufacturersDealerId','supplier_ref',#template_id,0,0,null,null,19,null,null,null,null,null),
('address1','address1',#template_id,0,0,null,null,null,null,null,null,null,null),
('address2','address2',#template_id,0,0,null,null,null,null,null,null,null,null),
('postcode','post_code',#template_id,0,0,null,null,null,null,null,null,null,null),
('telephone','telephone',#template_id,0,0,null,null,null,null,null,null,null,null),
('fax','fax_number',#template_id,0,0,null,null,null,null,null,null,null,null),
('email','email',#template_id,0,0,null,null,null,null,null,null,null,null),
('website','web_address',#template_id,0,0,null,null,null,null,null,null,null,null),
('NewCarSales','service_mask',#template_id,0,0,null,1,null,'^(?!(?i:^0$|^n$|^no$|^f$|^false$|^$))',null,null,1,null),
('UsedCarSales','service_mask',#template_id,0,0,null,2,null,'^(?!(?i:^0$|^n$|^no$|^f$|^false$|^$))',null,null,1,null),
('Servicing','service_mask',#template_id,0,0,null,8,null,'^(?!(?i:^0$|^n$|^no$|^f$|^false$|^$))',null,null,1,null),
('Repairs','service_mask',#template_id,0,0,null,16,null,'^(?!(?i:^0$|^n$|^no$|^f$|^false$|^$))',null,null,1,null),
('Longitude','longitude',#template_id,0,0,null,null,null,null,null,null,null,null),
('Latitude','latitude',#template_id,0,0,null,null,null,null,null,null,null,null)
This already contains some condition regex that in case that this field contains some text it converts it to true or false respectively.
What I need is a condition_regex that gets rid of these underscores and replaces it with a blank space i.e: 'GDB_A45' to 'GDB A45'. I don't know much about regex so any idea would be greatly appreciated. Thanks in advance!
SQL Server does not have much of regular expression support, but in this case I don't think you need it. You can do a simple replace:
UPDATE mytable
SET mycolumn = REPLACE(mycolumn, '_', ' ')
WHERE mycolumn LIKE '%[_]%'
To do this while updating you can use INSERT ... SELECT instead of INSERT ... VALUES:
INSERT INTO mytable (mycolumn)
SELECT REPLACE('my data 1', '_', ' ') UNION
SELECT REPLACE('my data 2', '_', ' ') UNION
SELECT REPLACE('my_data_3', '_', ' ') UNION
...
There will be some maximum number of unions you can do, so you should split your inserts into batches with this method.
Or, you could define a trigger on the target table that will do the job for you:
CREATE TRIGGER mytrigger ON mytable
AFTER INSERT AS
BEGIN
UPDATE mytable
SET mytable.mycolumn = REPLACE(i.mycolumn, '_', ' ')
FROM mytable
INNER JOIN inserted i
ON i.id = mytable.id
AND i.mycolumn LIKE '%[_]%'
END
... where it is assumed your table has a primary key named id.
Well after been thinking a while I got to the conclusion that would be easier if I replace the underscore from the scraped data during the scraping (in the c# code) before I generate the XML file. That would avoid me a lot of headaches. Anyway thank you for your help guys ;)

How to minimize sql select?

I have an array of words like this one:
$word1 = array('test1','test2','test3','test4','test5',...,'test20');
I need to search in my table every row that has at least one of these words in the text column. So far, I have this sql query:
SELECT * FROM TABLE WHERE text LIKE '$word1[0]' OR text LIKE '$word1[1]'
OR ... OR text LIKE '$word1[20]'
But I see that this design isn't very efficient. Is there any way I can shorten this query, in such a way that I don't need to write out every word in the where clause?
Example SELECT * FROM TABLE WHERE text IN ($word1)
P.S.: this is an example of what I'm looking for, not an actual query I can run.
If you use a table variable instead of a list to store your words then you can use something like:
DECLARE #T TABLE (Word VARCHAR(255) NOT NULL);
INSERT #T (Word)
VALUES ('test1'), ('test2'), ('test3'), ('test4'), ('test5'), ('test20');
SELECT *
FROM TABLE t
WHERE EXISTS
( SELECT 1
FROM #T
WHERE t.Text LIKE '%' + word + '%'
);
You can also create a table type to store this, then you can pass this as a parameter to a stored procedure if required:
CREATE TYPE dbo.StringList (Value VARCHAR(MAX) NOT NULL);
GO
CREATE PROCEDURE dbo.YourProcedures #Words dbo.StringList READONLY
AS
SELECT *
FROM TABLE t
WHERE EXISTS
( SELECT 1
FROM #Words w
WHERE t.Text LIKE '%' + w.word + '%'
);
GO
DECLARE #T dbo.StringList;
INSERT #T (Value)
VALUES ('test1'), ('test2'), ('test3'), ('test4'), ('test5'), ('test20');
EXECUTE dbo.YourProcedure #T;
For more on this see table-valued Parameters on MSDN.
EDIT
I may have misunderstood your requirements as you used LIKE but with no wild card operator, in which case you can just use IN, however I would still recommend using a table to store your values:
DECLARE #T TABLE (Word VARCHAR(255) NOT NULL);
INSERT #T (Word)
VALUES ('test1'), ('test2'), ('test3'), ('test4'), ('test5'), ('test20');
SELECT *
FROM TABLE t
WHERE t.Text IN (SELECT Word FROM #T);
You can use a SELECT like this without declaring an array:
SELECT * FROM TABLE WHERE text IN ('test1', 'test2', 'test3', 'test4', 'test5')
One solution could be :
Create a table in the database with the searched words in a column called word (by example)- by using wildcard if you need
use this kind of request
SELECT *
FROM TABLE, FILTER_TABLE
WHERE TABLE.text LIKE FILTER_TABLE.word
Although I don't have access to SQL Server 2008 at the moment and SQLfiddle seems sick, it would seem you can use a table value constructor to simplify the expression somewhat;
SELECT * FROM test
JOIN (SELECT w FROM (VALUES('word1'), ('word2'), ('word3'), ('word4')) AS a(w)) a
ON test.text LIKE '%'+a.w+'%';
...which will search the text column in the test table for the words listed as values. If you don't want duplicates of rows where multiple words match, you can just add a DISTINCT to the select.
Note though that you may want to look into fulltext indexing if you're doing extensive searches, a LIKE query to find words in a string in this way will not use any indexes, and will most likely be quite slow unless the data is already in memory.

Look for trailing spaces in a table

I'd like to know how I can identify trailing spaces in a table. I'm using SQL Server 2008 and create the following table as a test
CREATE TABLE first_test_name
(
firstName varchar(255)
)
And then inserted a record like this:
insert into first_test_name (firstName)
values('Bob')
I then tried inserting a space and then adding a new record like this:
insert into first_test_name (firstName)
values('Bob ') -- just 1 space
And for a 3rd time,
insert into first_test_name (firstName)
values('Bob ') -- two spaces used this time.
Now if I query for for 'Bob' (no spaces), I still get a count of 3.
My query was:
select count(*) from first_test_name WHERE firstName = 'Bob'
Shouldn't the answer have been 1?
Also, I used sp_help on this table and the value for "Trim Trailing Blanks" is set to no.
So why am I getting a count of 3? I was expecting just 1.
On a related note, if I search using this query
select * from first_test_name
where firstName like '% '
I then get the right answer of two rows found.
So just to reiterate, the question is why I get a count of 3 when searching for 'Bob'.
Also, what does "Trim Trailing Blanks" mean in this case?
Why I get a count of 3 when searching for 'Bob'?
SQL Server ignores trailing spaces in most string comparisons.
Also, what does "Trim Trailing Blanks" mean in this case?
This tells you the ANSI_PADDING option set when the table was created.
How can I identify those two with 1 or 2 trailing spaces?
Here's one way.
SELECT *
FROM first_test_name
WHERE firstName LIKE 'Bob '
And to find ones with no trailing space
SELECT *
FROM first_test_name
WHERE firstName LIKE 'Bob' AND firstName NOT LIKE 'Bob '
SQL Server will expand strings with whitespace during comparisons.
This is what I would do:
SELECT COUNT(*)
FROM first_test_name
WHERE REPLACE(firstName, ' ', '_') = 'Bob'
SELECT *
FROM USERS
WHERE DATALENGTH(Username) <> DATALENGTH(RTrim(Username))
Another way might be to append something on the string.
declare #test table(
id varchar(4) not null,
firstname varchar(255) not null)
insert into #test values('1', 'Bob')
insert into #test values('2', 'Bob ')
insert into #test values('3', 'Bob ')
insert into #test values('4', ' Bob')
select count((firstname + 'end')) from #test
where (firstname + 'end') not like '% %'
The query will return a count of 1.
A good clean way to do this would be to compare your original string against an Rtrim version of itself where they don't match e.g.:
SELECT *
FROM First_Test_Name
WHERE Firstname <> RTrim(Firstname)
This should return all records where Firstname has trailing spaces (I think ...)
I was looking recently and couldn't find the answer to this. Thought I'd share since coming across one.
https://stackoverflow.com/a/14188944/1953837
LEN trims trailing whitespaces by default. Using the below they are moved to the front and then the field length is counted.
Hope this helps anyone searching in the future.
(LEN(REVERSE(FieldName))

String manipulation SQL

I have a row of strings that are in the following format:
'Order was assigned to lastname,firsname'
I need to cut this string down into just the last and first name but it is always a different name for each record.
The 'Order was assigned to' part is always the same.......
Thanks
I am using SQL Server. It is multiple records with different names in each record.
In your specific case you can use something like:
SELECT SUBSTRING(str, 23) FROM table
However, this is not very scalable, should the format of your strings ever change.
If you are using an Oracle database, you would want to use SUBSTR instead.
Edit:
For databases where the third parameter is not optional, you could use SUBSTRING(str, 23, LEN(str))
Somebody would have to test to see if this is better or worse than subtraction, as in Martin Smith's solution but gives you the same result in the end.
In addition to the SUBSTRING methods, you could also use a REPLACE function. I don't know which would have better performance over millions of rows, although I suspect that it would be the SUBSTRING - especially if you were working with CHAR instead of VARCHAR.
SELECT REPLACE(my_column, 'Order was assigned to ', '')
For SQL Server
WITH testData AS
(
SELECT 'Order was assigned to lastname,firsname' as Col1 UNION ALL
SELECT 'Order was assigned to Bloggs, Jo' as Col1
)
SELECT SUBSTRING(Col1,23,LEN(Col1)-22) AS Name
from testData
Returns
Name
---------------------------------------
lastname,firsname
Bloggs, Jo
on MS SQL Server:
declare #str varchar(100) = 'Order was assigned to lastname,firsname'
declare #strLen1 int = DATALENGTH('Order was assigned to ')
declare #strLen2 int = len(#str)
select #strlen1, #strLen2, substring(#str,#strLen1,#strLen2),
RIGHT(#str, #strlen2-#strlen1)
I would require that a colon or some other delimiter be between the message and the name.
Then you could just search for the index of that character and know that anything after it was the data you need...
Example with format changing over time:
CREATE TABLE #Temp (OrderInfo NVARCHAR(MAX))
INSERT INTO #Temp VALUES ('Order was assigned to :Smith,Mary')
INSERT INTO #Temp VALUES ('Order was assigned to :Holmes,Larry')
INSERT INTO #Temp VALUES ('New Format over time :LootAt,Me')
SELECT SUBSTRING(OrderInfo, CHARINDEX(':',OrderInfo)+1, LEN(OrderInfo))
FROM #Temp
DROP TABLE #Temp