I've written a crazy INSERT/SELECT statement that works pretty well, except that I think it can be tuned a wee bit more if I can avoid using the LEN([stats].[dbo].[Domain].[DomainName]) twice. Right now it takes 20 seconds to run, and if I replace these LEN sections with actual numbers for a test, it literally takes a second, hence my hope!
I've been racking my brain on how to get that into a variable so I can use it twice but only have one performance hit. I'm on SQL Server 2008 R2 for what it is worth.
Thanks much!
INSERT INTO [stats].[dbo].[5MinStats] (Qty, MsgRequest, MsgRecType, MsgDateTime, DomainID)
SELECT
COUNT([stats].[dbo].[RawMsgLog].[MsgRequest]) AS Qty,
[stats].[dbo].[RawMsgLog].[MsgRequest],
[stats].[dbo].[RawMsgLog].[MsgRecType],
DATEADD(minute, DATEDIFF(minute, 0, [stats].[dbo].[RawMsgLog].[MsgDateTime])/ 5 * 5, 0) AS MsgDateTime,
[stats].[dbo].[Domain].[DomainID]
FROM
[stats].[dbo].[RawMsgLog], [stats].[dbo].[Domain]
WHERE
RIGHT([stats].[dbo].[Domain].[DomainName], LEN([stats].[dbo].[Domain].[DomainName])) = RIGHT([stats].[dbo].[RawMsgLog].[MsgRequest], LEN([stats].[dbo].[Domain].[DomainName]))
AND [stats].[dbo].[RawMsgLog].[switch] = 1
GROUP BY
[stats].[dbo].[RawMsgLog].[MsgRequest],
[stats].[dbo].[RawMsgLog].[MsgRecType],
DATEADD(minute, DATEDIFF(minute, 0, [stats].[dbo].[RawMsgLog].[MsgDateTime]) / 5 * 5, 0),
[stats].[dbo].[Domain].[DomainID]
ORDER BY
MsgDateTime ASC
Change your WHERE to the following:
WHERE stats.dbo.Domain.DomainName LIKE '%' + stats.dbo.RawMsgLog.MsgRequest
This avoids the RIGHT() and LEN() functions, and allows your query to use any available indexes... applying functions to your indexed columns will obfuscate them and cause a scan to be used instead.
I would also make a few other changes...
Alias your tables.
Don't use [] if you don't need to...
Use explicit JOIN.
Which would give you the following query:
INSERT INTO stats.dbo.5MinStats (
Qty,
MsgRequest,
MsgRecType,
MsgDateTime,
DomainID
)
SELECT
COUNT(rml.MsgRequest) as Qty,
rml.MsgRequest,
rml.MsgRecType,
DATEADD(minute,
DATEDIFF(minute, 0, rml.MsgDateTime)/ 5 * 5, 0) as MsgDateTime,
d.DomainID
FROM
stats.dbo.RawMsgLog rml
JOIN stats.dbo.Domain d
ON d.DomainName LIKE '%' + rml.MsgRequest
WHERE rml.switch=1
GROUP BY
rml.MsgRequest,
rml.MsgRecType,
dateadd(minute, datediff(minute, 0, rml.MsgDateTime)/ 5 * 5, 0),
d.DomainID
ORDER BY MsgDateTime ASC
You could use CHARINDEX to speed this up:
WHERE CHARINDEX([stats].[dbo].[Domain].[DomainName], [stats].[dbo].[RawMsgLog].[MsgRequest]) > 0
This will return true if the value in DomainName is found anywhere in the MsgRequest column.
In the first half of your condition, you do a "right" with the length of the substring being the same as length of the field itself (i.e. right(field, len(field)). You can remove this and just use the field itself. Then, your comparison can be wildcard text:
WHERE stats.dbo.RawMsgLog.MsgRequest like '%' + stats.dbo.Domain.DomainName
you can also use this substring comparison, but I imagine it would be slower.
right(stats.dbo.RawMsgLog.MsgRequest, LEN(stats.dbo.Domain.DomainName)) = stats.dbo.Domain.DomainName
Related
I posted one question today but that was too broad. So, I worked on that and now narrow down to some of parts. However, my query is returning syntax error while just copy paste the same text.
(
SELECT
TOP (100) PERCENT v_UpdateComplianceStatus.ResourceID, v_UpdateComplianceStatus.Status, CAST(DATEPART(yyyy, v_UpdateInfo.DatePosted) AS varchar(255)) + '-' + RIGHT('0' + CAST(DATEPART(mm, v_UpdateInfo.DatePosted) AS VARCHAR(255)), 2) AS MonthPosted, COUNT(1) AS Count
FROM
v_UpdateComplianceStatus
INNER JOIN
v_UpdateInfo
ON v_UpdateComplianceStatus.CI_ID = v_UpdateInfo.CI_ID
INNER JOIN
v_R_System
ON v_UpdateComplianceStatus.ResourceID = v_R_System.ResourceID
inner join
v_FullCollectionMembership fcm
on v_UpdateComplianceStatus.ResourceID = fcm.ResourceID
WHERE
(
v_R_System.Operating_System_Name_and0 like '%Workstation 6.1%'
and v_R_System.Obsolete0 = 0
)
AND
(
v_UpdateInfo.Severity IN
(
8,
10
)
)
AND
(
v_UpdateInfo.IsSuperseded = 0
)
AND
(
v_UpdateInfo.IsEnabled = 1
)
and fcm.CollectionID = 'ABC00328'
GROUP BY
v_UpdateComplianceStatus.ResourceID, v_UpdateComplianceStatus.Status, CAST(DATEPART(yyyy, v_UpdateInfo.DatePosted) AS varchar(255)) + '-' + RIGHT('0' + CAST(DATEPART(mm, v_UpdateInfo.DatePosted) AS VARCHAR(255)), 2)) FF
where Status =2
Group By MonthPosted ) E
where E.MonthPosted = E.MonthPosted
order by MonthPosted Desc
If I run the above query it is throwing error at
Group By MonthPosted ) EE
Not sure why it is giving error.
Msg 102, Level 15, State 1, Line 123
Incorrect syntax near 'EE'.
Some Important things which I discover.
This query works fine if I run some part of it.
(SELECT TOP (100) PERCENT v_UpdateComplianceStatus.ResourceID, v_UpdateComplianceStatus.Status, CAST(DATEPART(yyyy,
v_UpdateInfo.DatePosted) AS varchar(255)) + '-' + RIGHT('0' + CAST(DATEPART(mm, v_UpdateInfo.DatePosted) AS VARCHAR(255)), 2)
AS MonthPosted, COUNT(1) AS Count
FROM v_UpdateComplianceStatus INNER JOIN
v_UpdateInfo ON v_UpdateComplianceStatus.CI_ID = v_UpdateInfo.CI_ID INNER JOIN
v_R_System ON v_UpdateComplianceStatus.ResourceID = v_R_System.ResourceID
inner join v_FullCollectionMembership fcm on v_UpdateComplianceStatus.ResourceID=fcm.ResourceID
WHERE (v_R_System.Operating_System_Name_and0 like '%Workstation 6.1%' and v_R_System.Obsolete0 = 0) AND (v_UpdateInfo.Severity IN (8, 10)) AND (v_UpdateInfo.IsSuperseded = 0) AND (v_UpdateInfo.IsEnabled = 1)
and fcm.CollectionID='ABC00328'
GROUP BY v_UpdateComplianceStatus.ResourceID, v_UpdateComplianceStatus.Status, CAST(DATEPART(yyyy,
v_UpdateInfo.DatePosted) AS varchar(255)) + '-' + RIGHT('0' + CAST(DATEPART(mm, v_UpdateInfo.DatePosted) AS VARCHAR(255)), 2))
But if I Put Alias (FF) then it is throwing a syntax error.
Too long for a comment. Here is your first snippet condensed into something readable. Note - learn to write and post readable code. You have gone with FAR too much line spacing, indentation, and white space for anyone to easily read your code. The harder it is to read, the harder it is to understand. I've done my best to condense your first query into the essential elements.
(
SELECT blah blah,
CAST(DATEPART(yyyy, v_UpdateInfo.DatePosted) AS varchar(255)) + '-' +
RIGHT('0' + CAST(DATEPART(mm, v_UpdateInfo.DatePosted) AS VARCHAR(255)), 2) AS MonthPosted,
<aggregated column>
GROUP BY blah blah) as FF
where Status =2
Group By MonthPosted
) as E
where E.MonthPosted = E.MonthPosted
order by MonthPosted Desc
So what do YOU see wrong here. The where clause is pointless - does nothing useful. You probably introduced that error with all the editing. And apparently you cast numbers to large strings for no reason. That is just sloppy. It is concerning that you feel a need to cast a number to zero-filled string in the first place - that is probably an inefficient approach. If you really need to do that, look up the documentation for convert. Style 112 does what you need - all you need to do is take the first 6 characters of the converted string. Note 6 characters, not 255 characters or MAX characters. That will declutter your code significantly.
And now that you have edited your post multiple times, it logically makes no sense. There is no "EE" alias in your first query at all - so the error you posted cannot come from that snippet. Most likely the problem comes from the code you left out.
So now it is time to divide and conquer - a technique you can use to build complicated queries. Focus on that snippet ONLY. Write it as a complete query and run it, test it, validate that it works. When it does execute without errors and returns the correct results (results that you have actually verified and not just glanced at to see if numbers/values look "reasonable"), you can then add additional logic as needed. Usually it is best to add joins 1 by 1 to avoid creating a monster problem that is difficult to understand and diagnose. Often the use of CTEs can help. Put your starting query in a cte, get it working correctly. Example:
with cte1 as (...)
select * from cte1
order by ...;
Then add another cte to this first one and write it to use the first one, get it working. Example:
with cte1 as (...),
cte2 as (select ... from cte1 inner join ...)
select * from cte2
order by ...;
Repeat that as needed. Once everything works you can try to bring it all together and "beautify" it if needed.
And start thinking about your code. Use the appropriate datatypes, do NOT try to prematurely optimize things, learn and understand your schema, and stop using tricks. As I mentioned, "select top 100 percent" is generally pointless. Rhetorical question - why do you think this is needed as a part of this derived table. And use meaningful alias names. "E" and "EE" are not meaningful. Remember that someone will need to maintain this code, perhaps even modify it. And, of course, create an alias for every table/view and use it. Something short (but not too short) but meaningful. This will vastly improve readability - especially with those very long view (presumably) names.
Lastly, You said "This query works fine if i run some part of it." That just is not a useful thing to write since it means nothing to the reader. Which part? All of it? The first line? Just lines 2 through 5? It is difficult to have a technical discussion - do not add confusion by using terms that are imprecise.
Have a ncarchar(MAX) field in SQL table. It has numbers such as 717.08064182582, 39.0676048113, etc. in which I need to only have 3 places after decimal. For instance 717.080, 39.067.
Without converting the field type, would like to get rid of those last n characters, however every row has different number of characters. I believe I could use ROUND (correct me if wrong), but would rather not.
Try this
SELECT CAST(ColumnName AS DECIMAL(18,3))
Without converting it data type As per #vkp Comment
SELECT SUBSTRING(ColumnName ,0,CHARINDEX('.', ColumnName )+4)
select CASE WHEN CHARINDEX('.', Your_column) > 0
THEN SUBSTRING(Your_column, 1, CHARINDEX('.', Your_column) + 3)
ELSE Your_column
END
this is similar as previous answers but more faster and safer
Try this:
SELECT SUBSTRING(Your_column, 1, PATINDEX('%.%', Your_column) + 3)
FROM Your_Table
I need to examine ACCT_NUMS values om TABLE_1. If the ACCT_NUM is prefixed by "GF0", then I need to disregard the "GF0" prefix and take the rightmost 7 characters of the remaining string. If this resulting value is not found in account_x_master or CW_CLIENT_STAGE, then, the record is to be flagged as an error.
The following seems to do the trick, but I have a concern...
UPDATE
table_1
SET
Error_Ind = 'GW001'
WHERE
LEFT(ACCT_NUM, 3) = 'GF0'
AND RIGHT(SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3), 7) NOT IN
(
SELECT
acct_num
FROM
account_x_master
)
AND RIGHT(SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3), 7) NOT IN
(
SELECT
CW_CLIENT_STAGE.AGS_NUM
FROM
dbo.CW_CLIENT_STAGE
)
My concern is that SQL Server may attempt to perform a SUBSTRING operation
SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3)
that results in a computed negative value and causing the SQL to fail. Of course, this wouldn't fail is the SUBSTRING operation were only applied to those records that we at least 3 characters long, which would always be the case if the
LEFT(ACCT_NUM, 3) = 'GF0'
were applied first. If possible, I'd like to avoid adding new columns to the table. Bonus points for simplicity and less overhead :-)
How can I rewrite this UPDATE SQL to protect against this?
As other people said, your concern is valid.
I'd make two changes to your query.
1) To avoid having negative value in the SUBSTRING parameter we can rewrite it using STUFF:
SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3)
is equivalent to:
STUFF(ACCT_NUM, 1, 3, '')
Instead of extracting a tail of a string we replace first three characters with empty string. If the string is shorter than 3 characters, result is empty string.
By the way, if your ACCT_NUM may end with space(s), they will be trimmed by the SUBSTRING version, because LEN doesn't count trailing spaces.
2) Instead of
LEFT(ACCT_NUM, 3) = 'GF0'
use:
ACCT_NUM LIKE 'GF0%'
If you have an index on ACCT_NUM and only relatively small number of rows start with GF0, then index will be used. If you use a function, such as LEFT, index can't be used.
So, the final query becomes:
UPDATE
table_1
SET
Error_Ind = 'GW001'
WHERE
ACCT_NUM LIKE 'GF0%'
AND RIGHT(STUFF(ACCT_NUM, 1, 3, ''), 7) NOT IN
(
SELECT
acct_num
FROM
account_x_master
)
AND RIGHT(STUFF(ACCT_NUM, 1, 3, ''), 7) NOT IN
(
SELECT
CW_CLIENT_STAGE.AGS_NUM
FROM
dbo.CW_CLIENT_STAGE
)
You have a very valid concern, because SQL Server will rearrange the order of evaluation of expressions in the WHERE.
The only way to guarantee the order of operations in a SQL statement is to use case. I don't think there is a way to catch failing calls to substring() . . . there is no try_substring() analogous to try_convert().
So:
WHERE
LEFT(ACCT_NUM, 3) = 'GF0' AND
(CASE WHEN LEN(ACCT_NUM) > 3 THEN RIGHT(SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3), 7) END) NOT IN (SELECT acct_num
FROM account_x_master
) AND
(CASE WHEN LEN(ACCT_NUM) > 3 THEN RIGHT(SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3), 7) END) NOT IN (SELECT CW_CLIENT_STAGE.AGS_NUM
FROM dbo.CW_CLIENT_STAGE
)
This is uglier. And, there may be ways around it, say by using LIKE with wildcards rather than string manipulation. But, the case will guarantee that the SUBSTRING() is only run on strings long enough so no error is generated.
Please try the below query.
Since there is no short circuit and or in SQL WHERE clause, only way to achieve is via CASE syntax.
I noticed that you had two NOT IN comparisons in different parts of WHERE which I combined into one.
Note that CASE condition is >=3 and not >3, as RIGHT('',x) is allowed.
Also note the proper use of CASE with NOT IN
UPDATE table_1
SET
Error_Ind = 'GW001'
select * from table_1
WHERE
LEFT(ACCT_NUM, 3) = 'GF0'
AND CASE
WHEN LEN(ACCT_NUM)>=3
THEN RIGHT(SUBSTRING(ACCT_NUM, 4, LEN(ACCT_NUM) - 3), 7)
ELSE NULL END NOT IN
(
SELECT acct_num as num
FROM account_x_master
UNION
SELECT CW_CLIENT_STAGE.AGS_NUM as num
FROM dbo.CW_CLIENT_STAGE
)
SQL SERVER 2005
SQL Sorting :
Datatype varchar
Should sort by
1.aaaa
5.xx
11.bbbbbb
12
15.
how can i get this sorting order
Wrong
1.aaaa
11.bbbbbb
12
15.
5.xx
On Oracle, this would work.
SELECT
*
FROM
table
ORDER BY
to_number(regexp_substr(COLUMN,'^[0-9]+')),
regexp_substr(column,'\..*');
You could do this by calculating a column based on what's on the left hand side of the period('.').
However this method will be very difficult to make robust enough to use in a production system, unless you can make a lot of assertions about the content of the strings.
Also handling strings without periods could cause some grief
with r as (
select '1.aaaa' as string
union select '5.xx'
union select '11.bbbbbb'
union select '12'
union select '15.' )
select *
from r
order by
CONVERT(int, left(r.string, case when ( CHARINDEX('.', r.string)-1 < 1)
then LEN(r.string)
else CHARINDEX('.', r.string)-1 end )),
r.string
If all the entries have this form, you could split them into two parts and sort be these, for example like this:
ORDER BY
CONVERT(INT, SUBSTRING(fieldname, 1, CHARINDEX('.', fieldname))),
SUBSTRING(fieldname, CHARINDEX('.', fieldname) + 1, LEN(fieldname))
This should do a numeric sort on the part before the . and an alphanumeric sort for the part after the ., but may need some tuning, as I haven't actually tried it.
Another way (and faster) might be to create computed columns that contain the part before the . and after the . and sort by them.
A third way (if you can't create computed columns) could be to create a view over the table that has two additional columns with the respective parts of the field and then do the select on that view.
I want to truncate a column to a max of 100 characters. How do you do this in SQL Server?
Try this:
SELECT LEFT (your_column, 100) FROM your_table
Edit:
you can also try something like this:
SELECT LEFT (your_column, LEN(your_column)-5) FROM your_table
for say if you want to trim the last 5 characters from a record.
You can also use the LEFT() function.
LEFT(col, 100)
SUBSTRING(myColumn, 1, 100)
See the docs: http://msdn.microsoft.com/en-us/library/ms187748.aspx
substring is the method:
SUBSTRING ( value_expression ,start_expression , length_expression )
from the help.
SELECT SUBSTR(COLUMN_NAME, 1, LENGTH) FROM TABLENAME where LENGTH(COLUMN_NAME) > LENGTH
Ex:
SELECT SUBSTR(DESCRIPTION,1,100) FROM STOREDETAILS where LENGTH(DESCRIPTION)>100
For those records, with length less than 100, the actual value would be shown.
Otherwise, some databases induce blank characters in the resultant records.