I am running a query that returns the last execution time for a stored procedure:
SELECT
o.name,
ps.last_execution_time
FROM
sys.dm_exec_procedure_stats ps
INNER JOIN
sys.objects o ON ps.object_id = o.object_id
ORDER BY
ps.last_execution_time DESC
I am getting the correct results, but if I run the query again in around 30 seconds, I don't get any results.
Is there a setting or command I need to set or add to persist the results?
My goal is to find out what stored procedures ran in the past 3 days. I'm running this query against SQL Server 2019 Express.
I would suggest extended events for this. First, the session definition:
CREATE EVENT SESSION [ProcExecutions] ON SERVER
ADD EVENT sqlserver.module_end
ADD TARGET package0.event_file(
SET filename = N'ProcExecutions',
max_file_size = 10,
max_rollover_files = 5
)
WITH (
MAX_MEMORY = 4096 KB,
EVENT_RETENTION_MODE = ALLOW_MULTIPLE_EVENT_LOSS,
MAX_DISPATCH_LATENCY = 30 SECONDS,
MAX_EVENT_SIZE = 0 KB,
MEMORY_PARTITION_MODE = NONE,
TRACK_CAUSALITY = OFF,
STARTUP_STATE = ON
)
GO
ALTER EVENT SESSION [ProcExecutions] ON SERVER
STATE = START;
You may want to modify the session definition to suit your needs. Examples would be:
Filtering by a particular user (e.g. your application's login)
Grabbing just a sample (e.g. "one in a hundred executions")
Grab additional data (e.g. "what user called the proc?", "what was the whole statement (including parameters)?", etc)
Here's how to read the data:
IF object_id('tempdb.dbo.#events') IS NOT NULL
DROP TABLE #events;
select cast(event_data as xml) as [event]
into #events
from sys.fn_xe_file_target_read_file('ProcExecutions*.xel', null, null, null);
WITH XEData AS (
SELECT
[event].value('(event/#timestamp)[1]', 'datetime2') AS [timestamp],
db_name([event].value('(event/data[#name="source_database_id"])[1]', 'int')) AS [database],
[event].value('(event/data[#name="object_name"])[1]', 'sysname') AS [object_name],
[event].query('.') AS [event]
from #events
)
SELECT *
FROM XEData
ORDER BY [timestamp];
Again, this is very basic (returning only the timestamp, database, and procedure name). When I set about munging a new event session's data, I'll use that event column as a reference for what the XML looks like so I can write appropriate xpath expressions to pull the data that I need.
Related
We are trying to implement pagination in coldfusion.So that I need to make changes in sql query to fetch only a perticular number of data and when clicking on next button, next set of data should be queried.
SELECT History.*,User.FirstName,User.LastName
FROM History
LEFT OUTER JOIN User ON History.UserID = User.UserID
WHERE History.UserID = <CFQUERYPARAM VALUE="#UserID#" CFSQLTYPE="CF_SQL_INTEGER">
AND Type IS NOT NULL
AND SubType IS NOT NULL
ORDER BY #OrderBy# #sort#
Can anyone help me to do the same with above given query.
example query :
SELECT * FROM (
SELECT ROW_NUMBER() OVER(ORDER BY SearchID) AS NUMBER,
* FROM search_history
WHERE UserID=111
AND Date >= #DateVal#
) AS TBL
WHERE NUMBER BETWEEN ((#pageNum# - 1) * #MaxRows# + 1) AND (#pageNum# * #MaxRows#)
ORDER BY #OrderBy# #sort#
First, this is an issue with SQL Server and not ColdFusion. You're just generating the query inline with ColdFusion. It would be better if you create this query as a stored procedure, which by nature would perform better than an inline query.
Second, this answer is the basis for the example I'm posting. SQL Server famously has not supported OFFSET and LIMIT, which makes paging dead simple. These have been in MySQL and postgreSql forever.
Third, as of SQL Server 2012, there is now support for OFFSET and FETCH (essentially LIMIT). Here is a good example of how that works.
This example uses SQL #parameters. Ideally you would declare these in a stored procedure, but you can declare and set them in your inline query. Just make sure to use cfqueryparam to protect against SQL Injection attacks.
SELECT *
FROM (
SELECT
h.column1
, h.column2
, u.FirstName
, u.LastName
, SELECT ROW_NUMBER() OVER ( ORDER BY #ORDER_BY #SORT_ORDER ) AS RowNum
FROM
dbo.History h
INNER JOIN
dbo.User u ON h.UserID = u.UserID
WHERE
h.UserID = #USER_ID
AND
h.Type IS NOT NULL
AND
h.SubType IS NOT NULL
) AS HistorySearch
WHERE
RowNum >= #ROW_START
AND
RowNum < #ROW_END
ORDER BY
RowNum
You'll want to calculate the values of ROW_START and ROW_END based on the page number and a page size variable.
Finally, don't use SELECT * in your main select query. It's a bad practice.
Depending on how much data you are returning you can do your query normally and do pagination on the output with cold fusion. And if you can avoid using * in the query that is a good idea.
<cfparam name="StartAt" default="1">
<cfparam name="MaxRows" default="100">
<cfoutput name="QueryName" startrow="#StartAt#" maxrows="#maxRows#" >
The variables MaxRows and StartAt would be set in your form, and passed when the user clicks Next or previous.
If you have a very large result set then limiting the data returned would be a better idea as you have put forward.
I have two tables Backup and Requests.
Below is the script for both the tables
Backup
CREATE TABLE UserBackup(
FileName varchar(70) NOT NULL,
)
File name is represented by a guid. Sometimes there is some additional information related to the file. Hence we have entries like guid_ADD entried in table.
Requests
CREATE TABLE Requests(
RequestId UNIQUEIDENTIFIER NOT NULL,
Status int Not null
)
Here are some sample rows :
UserBackup table:
FileName
15b993cc-e8be-405d-bb9f-0c58b66dcdfe
4cffe724-3f68-4710-b785-30afde5d52f8
4cffe724-3f68-4710-b785-30afde5d52f8_Add
7ad22838-ddee-4043-8d1f-6656d2953545
Requests table:
RequestId Status
15b993cc-e8be-405d-bb9f-0c58b66dcdfe 1
4cffe724-3f68-4710-b785-30afde5d52f8 1
7ad22838-ddee-4043-8d1f-6656d2953545 2
What I need is to return all the rows from userbackup table whose name (the guid) is matches RequestId in the Requests table and the status is 1. So here is the query I wrote
Select *
from UserBackup
inner join Requests on UserBackup.FileName = Requests.RequestId
where Requests.Status = 1
And this works fine. It returns me the following result
FileName RequestId Status
15b993cc-e8be-405d-bb9f-0c58b66dcdfe 15b993cc-e8be-405d-bb9f-0c58b66dcdfe 1
4cffe724-3f68-4710-b785-30afde5d52f8 4cffe724-3f68-4710-b785-30afde5d52f8 1
4cffe724-3f68-4710-b785-30afde5d52f8_Add 4cffe724-3f68-4710-b785-30afde5d52f8 1
This is exactly what I want. But what I don't understand is how it is working. If you notice the result is returning 4cffe724-3f68-4710-b785-30afde5d52f8_Add row as well. The inner join is on varchar and uniqueidentifier, and this join instead of working like "Equals to" comparison works like "contains" comparison. I want to know how this works so that I can be sure to use this code without any unexpected scenarios.
The values on both sides of a comparison have to be of the same data type. There's no such thing as, say, comparing a uniqueidentifier and a varchar.
uniqueidentifier has a higher precedence than varchar so the varchars will be converted to uniqueidentifiers before the comparison occurs.
Unfortunately, you get no error or warning if the string contains more characters than are needed:
select CONVERT(uniqueidentifier,'4cffe724-3f68-4710-b785-30afde5d52f8_Add')
Result:
4CFFE724-3F68-4710-B785-30AFDE5D52F8
If you want to force the comparison to occur between strings, you'll have to perform an explicit conversion:
Select *
from UserBackup
inner join Requests
on UserBackup.FileName = CONVERT(varchar(70),Requests.RequestId)
where Requests.Status = 1
When you compare two columns of different data types SQL Server will attempt to do implicit conversion on lower precedence.
The following comes from MSDN docs on uniqueidentifier
The following example demonstrates the truncation of data when the
value is too long for the data type being converted to. Because the
uniqueidentifier type is limited to 36 characters, the characters that
exceed that length are truncated.
DECLARE #ID nvarchar(max) = N'0E984725-C51C-4BF4-9960-E1C80E27ABA0wrong';
SELECT #ID, CONVERT(uniqueidentifier, #ID) AS TruncatedValue;
http://msdn.microsoft.com/en-us/library/ms187942.aspx
Documentation is clear that data is truncated
When ever you are unsure about your join operation you can verify Actual Execution Plan.
Here is test sample that you can run inside SSMS or SQL Sentry Plan Explorer
DECLARE #userbackup TABLE ( _FILENAME VARCHAR(70) )
INSERT INTO #userbackup
VALUES ( '15b993cc-e8be-405d-bb9f-0c58b66dcdfe' ),
( '4cffe724-3f68-4710-b785-30afde5d52f8' ),
( '4cffe724-3f68-4710-b785-30afde5d52f8_Add' )
, ( '7ad22838-ddee-4043-8d1f-6656d2953545' )
DECLARE #Requests TABLE
(
requestID UNIQUEIDENTIFIER
,_Status INT
)
INSERT INTO #Requests
VALUES ( '15b993cc-e8be-405d-bb9f-0c58b66dcdfe', 1 )
, ( '4cffe724-3f68-4710-b785-30afde5d52f8', 1 )
, ( '7ad22838-ddee-4043-8d1f-6656d2953545', 2 )
SELECT *
FROM #userbackup u
JOIN #Requests r
ON u.[_FILENAME] = r.requestID
WHERE r.[_Status] = 1
Instead of regular join operation SQL Server is doing HASH MATCH with EXPR 1006 in SSMS it is hard to see what is doing but if you open XML file you will find this
<ColumnReference Column="Expr1006" />
<ScalarOperator ScalarString="CONVERT_IMPLICIT(uniqueidentifier,#userbackup.[_FILENAME] as [u].[_FILENAME],0)">
When ever in doubt check execution plan and always make sure to match data types when comparing.
This is great blog Data Mismatch on WHERE Clause might Cause Serious Performance Problems from Microsoft engineer on exact problem.
What is happening here is the FileName is being converted from varchar to a UniqueIdentifier, and during that process it ignores anything after the first 36 characters.
You can see it in action here
Select convert(uniqueidentifier, UserBackup.FileName), FileName
from UserBackup
It works, but to reduce confusion for the next person to come along, you might want to store the RequestId associated with the UserBackup as a GUID in the UserBackup table and join on that.
At the very least put a comment in ;)
I am facing a problem and cant find any solution to this. I have a source table (T) where I get data from field. The data may contain duplicate records with time stamp. My objective is to take the field data and store it into a final table (F) having the same structure.
Before inserting I check whether key field exists or not in the F if yes I update the the record in F with the latest one from T. Other wise I Insert the record in F from T. This works fine as long as there is no duplicate record in T. In case T has two records of the same key with different time stamp. It always inserts both the record (In case the key is primary key the insert operation fails). I am using following code for the operation -
IF EXISTS(SELECT * FROM [Final_Table] F, TMP_Source T WHERE T.IKEy =F.IKEY)
begin
print 'Update'
UPDATE [Final_Table]
SET [FULLNAME] = T.FULLNAME
,[FATHERNAME] = T.FATHERNAME
,[MOTHERNAME] = T.MOTHERNAME
,[SPOUSENAME] = T.SPOUSENAME
from TMP_Source T
WHERE Final_Table.IKEy = T.IKEy
and [Final_Table].[RCRD_CRN_DATE] < T.RCRD_CRN_DATE
--Print 'Update'
end
else
begin
INSERT INTO [Final_Table]
([IKEy],[FTIN],[FULLNAME],[FATHERNAME],[MOTHERNAME],[SPOUSENAME]
)
Select IKEy,FTIN,FULLNAME,FATHERNAME,MOTHERNAME,SPOUSENAME
from TMP_Source
end
The problem comes when I my T table has entries like -
IKey RCRD_CRN_DATE ...
123 10-11-2013-12.20.30
123 10-11-2013-12.20.35
345 10-11-2013-01.10.10
All three are inserted in the F table.
Please help.
Remove all but the latest row as a first step (well, in a CTE) using ROW_NUMBER() before attempting to perform the insert:
;WITH UniqueRows AS (
SELECT IKey,RCRD_CRN_DATE,FULL_NAME,FATHER_NAME,MOTHER_NAME,SPOUSENAME,FTIN,
ROW_NUMBER() OVER (PARTITION BY IKey ORDER BY RCRD_CRN_DATE desc) as rn
FROM TMP_Source
)
MERGE INTO Final_Table t
USING (SELECT * FROM UniqueRows WHERE rn = 1) s
ON t.IKey = s.IKey
WHEN MATCHED THEN UPDATE
SET [FULLNAME] = s.FULLNAME
,[FATHERNAME] = s.FATHERNAME
,[MOTHERNAME] = s.MOTHERNAME
,[SPOUSENAME] = s.SPOUSENAME
WHEN NOT MATCHED THEN INSERT
([IKEy],[FTIN],[FULLNAME],[FATHERNAME],[MOTHERNAME],[SPOUSENAME]) VALUES
(s.IKEy,s.FTIN,s.FULLNAME,s.FATHERNAME,s.MOTHERNAME,s.SPOUSENAME);
(I may not have all the columns entirely correct, they seem to keep switching around in your question)
(As you may have noticed, I've also switched to using MERGE since it allows us to express everything as a single declarative statement rather than writing procedural code)
I have some IDs inserted into a temp table #A as follows:
SELECT DISTINCT ID
INTO #A
FROM LocalDB.dbo.LocalTable1
WHERE ID NOT IN (SELECT DISTINCT ID FROM LocalDB.dbo.LocalTable2)
GO
CREATE INDEX TT ON #A(ID)
GO
I am trying to obtain some information from a remote linked server using the identifiers I gathered in the previous stage:
Query 1:
SELECT ID, Desc
FROM RemoteLinkedServer.DB.dbo.RemoteTable X
WHERE ID IN (SELECT ID FROM #A)
Query 2:
SELECT ID, Desc
FROM RemoteLinkedServer.DB.dbo.RemoteTable X
INNER JOIN #A Y
ON X.ID = Y.ID
Now in the following query, what I am doing is obtain the output of the temp table, copy the rows and format them properly into a comma-separated list and manually putting it in the query.
Query 3:
SELECT ID, Desc
FROM RemoteLinkedServer.DB.dbo.RemoteTable X
WHERE ID IN (-- Put all identifiers here --)
Queries 1 and 2 take 2 hours to execute and query 3 takes 0 seconds (my temp table contains about 200 rows). I don't know what's going on and do not have permissions to check if the remote server has the relevant indexes on ID but it is simply baffling to see that a manually constructed query runs in no time indicating that there is something that is going wrong at the query optimization phase.
Any ideas on what's going wrong here or how I could speed up my query?
Queries 1 and 2 cause ALL of the data in the RemoteTable to be pulled into your local database in order to perform the join operation. This is going to eat RAM, network bandwidth and generally be very slow while the query is executing.
Query 3 allows the remote server to filter down the results to send just those matches you want.
Basically, it boils down to who does the work. Queries 1/2 require your local DB to do it; Query 3 lets the remote one do it.
If you have a lot of data in that remote table, then you'll likely run into network congestion etc.
The best approach to querying linked servers is to construct your queries such as the remote server does all the work and just sends results back to your local one. This will optimize the amount of network, memory and disk resources required to get the data you want.
Any time you have to join across server boundaries (using a linked server) it's going to be a disaster.
For reference, this is how I solved the problem based on #ChrisLively's suggestions:
SELECT DISTINCT ID
INTO #A
FROM LocalDB.dbo.LocalTable1
WHERE ID NOT IN (SELECT DISTINCT ID FROM LocalDB.dbo.LocalTable2)
GO
CREATE INDEX TT ON #A(ID)
GO
DECLARE #IDList VARCHAR(MAX)
SELECT #IDList=(SELECT TOP 1 REPLACE(RTRIM((
SELECT DISTINCT CAST(ID AS VARCHAR(MAX)) + ' '
FROM #A AS InnerTable
FOR XML PATH (''))),' ',', '))
FROM #A AS OuterResults
DECLARE #sql AS varchar(MAX)
SET #sql = 'SELECT * FROM RemoteLinkedServer.RemoteDB.dbo.RemoteTable X WHERE ID IN (' + #IDList + ')'
EXEC (#sql)
DROP TABLE #A
GO
I have a table which looks something like
Event ID Date Instructor
1 1/1/2000 Person 1
1 1/1/2000 Person 2
Now what I want to do is return this data so that each event is on one row and the Instructors are all in one column split with a <br> tag like 'Person 1 <br> Person 2'
Currently the way I have done this is to use a function
CREATE FUNCTION fnReturnInstructorNamesAsHTML
(
#EventID INT
)
RETURNS VARCHAR(max)
BEGIN
DECLARE #Result VARCHAR(MAX)
SELECT
#result = coalesce(#result + '<br>', '') + inst.InstructorName
FROM
[OpsInstructorEventsView] inst
WHERE
inst.EventID = #EventID
RETURN #result
END
Then my main stored procedure calls it like
SELECT
ev.[BGcolour],
ev.[Event] AS name,
ev.[eventid] AS ID,
ev.[eventstart],
ev.[CourseType],
ev.[Type],
ev.[OtherType],
ev.[OtherTypeDesc],
ev.[eventend],
ev.[CourseNo],
ev.[Confirmed],
ev.[Cancelled],
ev.[DeviceID] AS resource_id,
ev.Crew,
ev.CompanyName ,
ev.Notes,
dbo.fnReturnInstructorNamesAsHTML(ev.EventID) as Names
FROM
[OpsSimEventsView] ev
JOIN
[OpsInstructorEventsView] inst
ON
ev.EventID = inst.EventID
This is very slow, im looking at 4seconds per call to the DB. Is there a way for me to improve the performance of the function? Its a fairly small function so im not sure what I can do here, and I couldnt see a way to work the COALESCE into the SELECT of the main procedure.
Any help would be really appreciated, thanks.
You could try something like this.
SELECT
ev.[BGcolour],
ev.[Event] AS name,
ev.[eventid] AS ID,
ev.[eventstart],
ev.[CourseType],
ev.[Type],
ev.[OtherType],
ev.[OtherTypeDesc],
ev.[eventend],
ev.[CourseNo],
ev.[Confirmed],
ev.[Cancelled],
ev.[DeviceID] AS resource_id,
ev.Crew,
ev.CompanyName ,
ev.Notes,
STUFF((SELECT '<br>'+inst.InstructorName
FROM [OpsInstructorEventsView] inst
WHERE ev.EventID = inst.EventID
FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)'), 1, 4, '') as Names
FROM
[OpsSimEventsView] ev
Not sure why you have joined OpsInstructorEventsView in the main query. I removed it here but if you needed you can just add it again.
A few things to look at:
1) The overhead of functions makes them expensive to call, especially in the select statement of a query that could potentially be returning thousands of rows. It will have to execute that function for every one of them. Consider merging the behavior of the function into your main stored procedure, where the SQL Server can make better use of its optimizer.
2) Since you are joining on event id in both tables, make sure you have an index on those two columns. I would expect that you do, given that those both appear to be primary key columns, but make sure. An index can make a huge difference.
3) Convert your coalesce call into its equivalent case statements to remove the overhead of calling that function.
Yes make it an INLINE Table-Valued SQL function:
CREATE FUNCTION fnReturnInstructorNamesAsHTML
( #EventID INT )
RETURNS Table
As
Return
SELECT InstructorName + '<br>' result
FROM OpsInstructorEventsView
WHERE EventID = #EventID
Go
Then, in your SQL Statement, use it like this
SELECT ]Other stuff],
(Select result from dbo.fnReturnInstructorNamesAsHTML(ev.EventID)) as Names
FROM OpsSimEventsView ev
JOIN OpsInstructorEventsView inst
ON ev.EventID = inst.EventID
I'm not exactly clear how the query you show in your question is concatenating data from multiple rows in one row of the result, but the problem is that ordinary UDFs are compiled on use, on EVERY use, so for each row in your output result the Query processopr has to recompile the UDF again. THis is NOT True for an "inline table valued" UDF, as it's sql is folded into the outer sql before it is passed to the SQL optimizer, (the subsystem that generates the statement cache plan) and so the UDF is only compiled once.