I am trying to find a performance smart resolution to searching on data that is coming into an application with character spaces removed, but could exist in the DB system in many ways with spaces. I was thinking the following, but it seems to fail:
select top 1 * where REPLACE(Mfr, ' ', '') = #Mfr and REPLACE(Model, ' ', '') = #Model
Am I doing this completely wrong and is there a better way? It appears to time out only only a DB with 150k records at the moment.
You can implement the expressions as indexed virtual columns for fast lookup.
First you'll need to create the virtual columns:
alter table t add clean_mfr as replace(Mfr, ' ', '');
alter table t add clean_model as replace(Model, ' ', '');
Then you'll index them:
create index ix1 on t (clean_mfr, clean_model);
Now, you can try your query again. Since you are using equality (=) in both virtual columns lookups the query should be instantaneous.
Try:
select top 1 *
from t
where REPLACE(Mfr, ' ', '') = #Mfr and REPLACE(Model, ' ', '') = #Model
Or:
select top 1 *
from t
where clean_mfr = #Mfr and clean_model = #Model
Related
I have the following problem with a query on SQL Server
I have this simple query:
SELECT *
FROM [PROT_UTENTI]
WHERE
Nome = 'Paolo D''Aannuntis'
that finds a user by a Nome field (representing the name and the surname, I know, it is horrible but I can't change the DB because it is an old legacy application in production).
As you can see the surname (D'Annuntis) contains the ' character (I escaped it by doubling the '').
The problem is that I am performing this query using the Nome value extracting it from another databse (it is involved into a migration process) where the same Nome field is saved without the ' character into the surname.
So I am performing this query:
SELECT *
FROM [PROT_UTENTI]
WHERE
Nome = 'Paolo DAannuntis'
instead
SELECT *
FROM [PROT_UTENTI]
WHERE
Nome = 'Paolo D''Aannuntis'
finding 0 result.
My question is: there is a way to ignore the ' so the where clause Nome = 'Paolo DAannuntis' can find the same result Nome = 'Paolo D''Aannuntis'?
you can replace any ' by nothing in your search query
SELECT *
FROM [PROT_UTENTI]
WHERE replace(Nome, '''', '') = 'Paolo DAannuntis'
you can also do the replace on both sides
WHERE replace(Nome, '''', '') = replace('Paolo D''Aannuntis', '''', '')
You wrote that the value is extraced from another database in a migration,
so I guess you will have a variable.
declare #nome varchar(100) = 'Paolo D''Aannuntis' -- filled by your process
SELECT *
FROM [PROT_UTENTI]
WHERE replace(Nome, '''', '') = replace(#nome, '''', '')
this will find your row, regardless if there are quotes or not in the variable or in [PROT_UTENTU].nome
this will have affect on the performance of the query, so I hope there are not millions of rows in your table
I'm creating sprocs with some calculations and I want to make sure I'm not missing something simple.
Say I'm finding a SUM() of a column that might have NULLs. Is there a single set statement that will convert NULL to Zero automatically without having to COALESCE each time? Or do I have to manually check for NULL each time?
I've looked through MSDN SET but I don't see anything useful.
There's a way to make NULL work with concatenation but I don't see anything for calculations.
For example:
SET ANSI_NULLS ON
SET CONCAT_NULL_YIELDS_NULL ON
--Calc
SELECT SUM(CONVERT(decimal(10,2), NULL))
SELECT SUM(CONVERT(decimal(10,2), Coalesce(NULL,0)))
--Concat
SELECT NULL + ', ' + 'Isaak' AS Name
SELECT COALESCE(NULL + ', ' + 'Isaak','') AS Name
SELECT COALESCE(NULL,'') + ', ' + 'Isaak' AS Name
--Change Concat NULL to OFF
SET ANSI_NULLS ON
SET CONCAT_NULL_YIELDS_NULL OFF
--Calc
SELECT SUM(CONVERT(decimal(10,2), NULL))
SELECT SUM(CONVERT(decimal(10,2), Coalesce(NULL,0)))
--Concat
SELECT NULL + ', ' + 'Isaak' AS Name
SELECT COALESCE(NULL + ', ' + 'Isaak','') AS Name
SELECT COALESCE(NULL,'') + ', ' + 'Isaak' AS Name
No, there is no magic way to do this. However there are multiple workarounds:
Stop allowing NULLs in the first place - add a default of 0 and if you can't update the DML logic then add a trigger (but far preferable to do this as part of the original insert/update).
Put the COALESCE into a view, and then reference the view in your queries.
Persist a zero (using COALESCE of course) into a separate, computed column, and change the calculation to use the computed column instead of the original column.
Im trying to update a field by appending data to it.
if it contains the data already i wont update it otherwise I will.
if it already contains data i want it to append a comma and space followed by the word. e.g.
update myTable
set Prefixes = convert(nvarchar(max),Prefixes) + ', abc'
where MyCol='xyz' and Prefixes not like '%abc%'
im trying to get this to work so that if the prefixes column is empty initially it only includes the word 'abc'
and not ', abc'
How can i do this?
Sounds like you need a CASE:
update myTable
set Prefixes =
case
when Prefixes is null or Prefixes = ''
then 'abc'
else convert(nvarchar(max),Prefixes) + ', abc'
end
where MyCol='xyz' and (Prefixes not like '%abc%' or Prefixes is null)
See SQL Fiddle with Demo
You need to check for null values of Prefixes before filtering as NOT LIKE in the WHERE clause as well. Sql-Demo
update myTable
set Prefixes = isnull(nullif(rtrim(Prefixes),'') + ', abc','abc')
where MyCol='xyz' and isnull(Prefixes,'') not like ', abc%'
A number of times over the last month I've had to replace 'null' fields with '0' to every column returned from a query.
to save a lot of time (some of these are returning a high number of columns) I've been using the following and then pasting the results for relevant columns into a new query:
select ', isnull(' + COLUMN_NAME + ', 0)' + ' as ' + COLUMN_NAME
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = 'summary_by_scca_sales_category '
and TABLE_SCHEMA = 'property''
Essentially I'm wondering if there's a better way that I can do this? Ideally a method where I could automatically apply isnull to all columns being returned in a query (without using two queries).
For example:
I want to take a query like:
select *
from tablename
And for every column returned by * replace null results with 0 without having to write an isnull() line for each column.
edit:
Will accomplish this with a view (doh, should have thought of that). For interests / educations sake is there a way to do something like this with code also?
You could create a VIEW against the tables in question where the ISNULL logic you want is set up. Then queries against the views would return the data you want.
EDIT:
As requested, some sample code to accomplish creating the VIEWs automatically. This is pretty gross, but for something that only has to be run once it will work. Beware of type issues (you stated everything should transmute to 0 so I assume all your columns are of a suitable numeric type):
DECLARE #table_def varchar(max)
SET #table_def = 'CREATE VIEW <tname>_NoNull AS SELECT '
SELECT #table_def = REPLACE(#table_def, '<tname>', t.name) +
'ISNULL(' + c.name + ', 0) AS ' + c.name + ', '
FROM sys.tables t
INNER JOIN sys.columns c ON t.object_id = c.object_id
WHERE t.name = <<table name>>
SELECT #table_def
Simple concept we are basically doing some auditing, comparing what came in, and what actually happened during processing. I am looking for a better way to execute a query that can do side by side table comparisons with columns that are slightly differnt in name and potentialy type.
DB Layout:
Table (* is the join condition)
Log (Un-altered data record.)
- LogID
- RecordID*
- Name
- Date
- Address
- Products
- etc.
Audit (post processing record)
- CardID*
- CarName
- DeploymentDate
- ShippingAddress
- Options
- etc.
For example this would work if you look past the annoying complexity to write, and performance issues.
The query just joins the left and right and selects them as strings. Showing each field matched up.
select
cast(log.RecordID as varchar(40)) + '=' + cast(audit.CardID as varchar(40),
log.Name+ '=' + audit.Name ,
cast(log.Date as varchar(40)) + '=' + cast(audit.DeploymentDate as varchar(40),
log.Address + '=' + audit.ShippingAddress,
log.Products+ '=' + audit.Options
--etc
from Audit audit, Log log
where audit.CardID=log.RecordId
Which would output something like:
1=1 Test=TestName 11/09/2009=11/10/2009 null=My Address null=Wheels
This works but is extremely annoying to build. Another thing I thought of was to just alias the columns, union the two tables, and order them so they would be in list form. This would allow me to see the column comparisons. This comes with the obvious overhead of the union all.
ie:
Log 1 Test 11/09/2009 null, null
Audit 1 TestName 11/10/2009 My Address Wheels
Any suggestions on a better way to audit this data?
Let me know what other questions you may have.
Additional notes. We are going to want to reduce the unimportant information so in some cases we might null the column if they are equal (but i know its too slow)
case when log.[Name]<>audit.[CarName] then (log.[Name] + '!=' + audit.[CarName]) else null end
or if we are doing the second way
nullif(log.[Name], audit.[CarName]) as [Name]
,nullif(audit.[CarName], log.[Name]) as [Name]
I've found the routine given here by Jeff Smith to be helpful for doing table comparisons in the past. This might at least give you a good base to start from. The code given on that link is:
CREATE PROCEDURE CompareTables(#table1 varchar(100),
#table2 Varchar(100), #T1ColumnList varchar(1000),
#T2ColumnList varchar(1000) = '')
AS
-- Table1, Table2 are the tables or views to compare.
-- T1ColumnList is the list of columns to compare, from table1.
-- Just list them comma-separated, like in a GROUP BY clause.
-- If T2ColumnList is not specified, it is assumed to be the same
-- as T1ColumnList. Otherwise, list the columns of Table2 in
-- the same order as the columns in table1 that you wish to compare.
--
-- The result is all records from either table that do NOT match
-- the other table, along with which table the record is from.
declare #SQL varchar(8000);
IF #t2ColumnList = '' SET #T2ColumnList = #T1ColumnList
set #SQL = 'SELECT ''' + #table1 + ''' AS TableName, ' + #t1ColumnList +
' FROM ' + #Table1 + ' UNION ALL SELECT ''' + #table2 + ''' As TableName, ' +
#t2ColumnList + ' FROM ' + #Table2
set #SQL = 'SELECT Max(TableName) as TableName, ' + #t1ColumnList +
' FROM (' + #SQL + ') A GROUP BY ' + #t1ColumnList +
' HAVING COUNT(*) = 1'
exec ( #SQL)
Would something like this work for you:
select
(Case when log.RecordID = audit.CardID THEN 1 else 0) as RecordIdEqual,
(Case when log.Name = audit.Name THEN 1 else 0) as NamesEqual ,
(Case when log.Date = audit.DeploymentDate THEN 1 else 0) as DatesEqual,
(Case when log.Address = audit.ShippingAddress THEN 1 else 0) as AddressEqual,
(Case when log.Products = audit.Options THEN 1 else 0) as ProductsEqual
--etc
from Audit audit, Log log
where audit.CardID=log.RecordId
This will give you a break down of what's equal based on the column name. Seems like it might be easier than doing all the casting and having to interpret the resulting string...