Is it possible to pick one result set to return from a batch execution? - sql

I have to retrieve some data from a MSSQL 2014 server through a propriatery application which then uses an odbc data source. The only thing I can modify is the query the application uses. I cannot modify the application or how the application handles the results.
The following query is doing what I want if I execute it directly e.g. in Heidi.
USE MY_DB;
BEGIN TRANSACTION
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
DECLARE #myvar1 INT = 2;
DECLARE #myvar2 INT = 2;
PRINT #myvar1;
SELECT TOP 20 [vwPGIA].[OrNum],[vwPGIA].[DBM],[vwPGIA].[MBM],[vwPGIA].[MN],[NOMID],[Priority],SUBSTRING([Comment],0,254) AS Comment,[TLSAP],[Box],[SequenceNumber]
INTO #tmp_tbl
FROM [MY_DB].[dbo].[vwPGIA]
INNER JOIN [MY_DB].[dbo].[tblDLA] ON [dbo].[tblDLA].[OrNum]=[dbo].[vwPGIA].[OrNum]
INNER JOIN [dbo].[tblMDM] ON [vwPGIA].[MBM]=[tblMDM].[MBM]
WHERE ([TLSAP] = #myvar1)
AND [vwPGIA].[MBM] NOT IN (SELECT [MBM] FROM [MY_DB].[dbo].[vwDPS])
AND [vwPGIA].[OrNum] NOT IN (SELECT [OrNum] FROM [MY_DB].[dbo].[vwDPS] WHERE [MY_DB].[dbo].[vwDPS].[TLR] <> #myvar1)
ORDER BY [SequenceNumber];
SELECT TOP 1 [OrNum],[DBM],[MBM],[MN],[NOMID],[Priority],[Comment],[TLSAP],[Box],[WTT],[SequenceNumber]
FROM #tmp_tbl
INNER JOIN [dbo].[tblTBN] ON [Box]=[BoxN]
WHERE ([WTT]=#myvar2)
ORDER BY [SequenceNumber];
INSERT INTO [dbo].[tblDPS]
(OrNum,DBM,MBM,State,StateStartTime,Info,TLR)
SELECT TOP 1 [OrNum],[DBM],[MBM],'1',GETDATE(),'info',#myvar1
FROM #tmp_tbl
INNER JOIN [dbo].[tblTBN] ON [Box]=[BoxN]
WHERE ([WTT]=#myvar2)
ORDER BY [SequenceNumber]
;
DROP TABLE #tmp_tbl;
COMMIT TRANSACTION
Running this through the ODBC interface results in an empty result. The problem seems to be, that I am doing a batch request which results in multiple result sets. The application probably only handles the first result set or maybe cannot handle more than one result set.
Finally the question: Is there a way or workaround to reduce the result sets to only the one returned by the SELECT TOP 1 ... part?

Related

SQL Locking - Timeout issue

I have a SQL job which kind of process a queue of requests. It updates Table1. It is long process that takes like 15 minutes.
Meanwhile my application tries to read records from Table1 and displays them in a grid. The corresponding get proc has set tran isolation level read uncommited.
When the SQL job is running, my application always time outs while populating the grid. If the SQL job is not running it works fine.
My proc has appropriate isolation level, so I'm not getting why it still time out.
Thoughts?
Here is how my get proc looks like:
CREATE PROCEDURE dbo.MyGetProc(...)
AS
BEGIN
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
SET NOCOUNT ON;
SELECT
...
FROM
(
SELECT
...
FROM
dbo.Table1
LEFT JOIN dbo.OtherTable1
WHERE ...
GROUP BY
...
) X
INNER JOIN dbo.OtherTable2
LEFT JOIN dbo.OtherTable3
LEFT JOIN dbo.OtherTable4
LEFT JOIN dbo.OtherTable5
LEFT JOIN dbo.OtherTable6
LEFT JOIN dbo.OtherTable6
ORDER BY
...
END
use
Select * from table1 with (nolock)
which reads commited data.

Azure stored procedure out value

I've been trying to assemble a stored procedure on my Azure database that when it runs the query, it returns one output value from one specific column.
The likelihood of multiple results is zero since the the table being queried has 3 columns, and the query must mach 2. Then it grabs data from another table. The key is I need the first query to output the value in order to commence the second query.
At present I have 2 procedures, I would like to have one.
Query is as such for the moment:
select
customers_catalogs_define.catalog_id
from
customers_catalogs
left outer join
customers_catalogs_define on customers_catalogs.catalog_id = customers_catalogs_define.catalog_id
where
customers_catalogs.catalog_unique_identifier = #catalog_unique
AND customers_catalogs_define.customer_id = #customer_id
The output of course is the catalog_id. From that I take it into another query which I have that does the actual list retrieval. At the very least I would like to add a line that simply states #catalog_id = output
Thanks
You have two options basically to have this work :
Since I haven't seen your second query, just make sure its querying the correct table listed below as CUSTOMERS_CATALOGS_DEFINE
Using a variable as you suggested:
DECLARE #CATALOG_ID INT
SET #CATALOG_ID = (
SELECT CUSTOMERS_CATALOGS_DEFINE.CATALOG_ID
FROM CUSTOMERS_CATALOGS
LEFT OUTER JOIN CUSTOMERS_CATALOGS_DEFINE
ON CUSTOMERS_CATALOGS.CATALOG_ID = CUSTOMERS_CATALOGS_DEFINE.CATALOG_ID
WHERE CUSTOMERS_CATALOGS.CATALOG_UNIQUE_IDENTIFIER = #CATALOG_UNIQUE
AND CUSTOMERS_CATALOGS_DEFINE.CUSTOMER_ID = #CUSTOMER_ID )
SELECT *
FROM CUSTOMERS_CATALOGS_DEFINE
WHERE CATALOG_ID = #CATALOG_ID
Second option would be to do it in one query:
SELECT *
FROM CUSTOMERS_CATALOGS_DEFINE
WHERE CATALOG_ID IN (
SELECT CUSTOMERS_CATALOGS_DEFINE.CATALOG_ID
FROM CUSTOMERS_CATALOGS
LEFT OUTER JOIN CUSTOMERS_CATALOGS_DEFINE
ON CUSTOMERS_CATALOGS.CATALOG_ID = CUSTOMERS_CATALOGS_DEFINE.CATALOG_ID
WHERE CUSTOMERS_CATALOGS.CATALOG_UNIQUE_IDENTIFIER = #CATALOG_UNIQUE
AND CUSTOMERS_CATALOGS_DEFINE.CUSTOMER_ID = #CUSTOMER_ID
)

SQL 2005 Query Optimisation

I have a SQL 2005 table consisting of around 10million records (dbo.Logs).
I have another table, dbo.Rollup that matches distinct dbo.Logs.URL to a FileId column in a third table, dbo.Files. The dbo.Rollup table forms the basis of various aggregate reports we run at a later stage.
Suffice to say for now, the problem I am having is in populating dbo.Rollup efficiently.
By definition, dbo.Logs has potentially tens of thousands of rows which all share the same URL field value. In our application, one URL can be matched to one dbo.Files.FileId. I.E. There is a many-to-one relationship between dbo.Logs.URL and dbo.Files.FileId (we parse the values of dbo.Logs to determine what the appropriate FileId is for a given URL).
My goal is to significantly reduce the amount of time it takes the first of three stored procedures that run in order to create meaningful statistics from our raw log data.
What I need is a specific example of how to refactor this SQL query to be much more efficient:
sp-Rollup-Step1:
INSERT INTO dbo.Rollup ([FileURL], [FileId])
SELECT
logs.RequestedFile As [URL],
FileId = dbo.fn_GetFileIdFromURL(l.RequestedFile, l.CleanFileName)
FROM
dbo.Logs l (readuncommitted)
WHERE
NOT EXISTS (
SELECT
FileURL
FROM
dbo.Rollup
WHERE
FileUrl = RequestedFile
)
fn_GetFileIdFromURL():
CREATE FUNCTION [dbo].[fn_GetFileIdFromURL]
(
#URL nvarchar(500),
#CleanFileName nvarchar(255)
)
RETURNS uniqueidentifier
AS
BEGIN
DECLARE #id uniqueidentifier
if (exists(select FileURL from dbo.[Rollup] where [FileUrl] = #URL))
begin
-- This URL has been seen before in dbo.Rollup.
-- Retrieve the FileId from the dbo.Rollup table.
set #id = (select top 1 FileId from dbo.[Rollup] where [FileUrl] = #URL)
end
else
begin
-- This is a new URL. Hunt for a matching URL in our list of files,
-- and return a FileId if a match is found.
Set #id = (
SELECT TOP 1
f.FileId
FROM
dbo.[Files] f
INNER JOIN
dbo.[Servers] s on s.[ServerId] = f.[ServerId]
INNER JOIN
dbo.[URLs] u on
u.[ServerId] = f.[ServerId]
WHERE
Left(u.[PrependURLProtocol],4) = left(#URL, 4)
AND #CleanFileName = f.FileName
)
end
return #id
END
Key considerations:
dbo.Rollup should contain only one entry for each DISTINCT/unique URL found in dbo.tLogs.
I would like to omit records from being inserted into dbo.[Rollup] where the FileId is NULL.
In my own observations, it seems the slowest part of the query by far is in the stored procedure: the "NOT EXISTS" clause (I am not sure at this point whether that continually refreshes the table or not).
I'm looking for a specific solution (with examples using either pseudo-code or by modifying my procedures shown here) - answer will be awarded to those who provide it!
Thanks in advance for any assistance you can provide.
/Richard.
Short answer is you have a CURSOR here. The scalar UDF is run per row of output.
The udf could be 2 LEFT JOINs onto derived tables. A rough outline:
...
COALESCE (F.xxx, L.xxx) --etc
...
FROM
dbo.Logs l (readuncommitted)
LEFT JOIN
(select DISTINCT --added after comment
FileId, FileUrl from dbo.[Rollup]) R ON L.FileUrl = R.FileUrl
LEFT JOIN
(SELECT DISTINCT --added after comment
f.FileId,
FileName ,
left(#PrependURLProtocol, 4) + '%' AS Left4
FROM
dbo.[Files] f
INNER JOIN
dbo.[Servers] s on s.[ServerId] = f.[ServerId]
INNER JOIN
dbo.[URLs] u on
u.[ServerId] = f.[ServerId]
) F ON L.CleanFileName = R.FileName AND L.FileURL LIKE F.Left4
...
I'm also not sure if you need the NOT EXISTS because of how the udf works. If you do, make sure the columns are indexed.
I think your hotspot is located here:
Left(u.[PrependURLProtocol],4) = left(#URL, 4)
This will cause the server to do a scan on the url table. You should not use a function on a field in a join clause. try to rewrite that to something like
... where PrependURLProtocol like left(#URL, 4) +"%"
And make sure you have an index on the field.
INSERT INTO dbo.Rollup ([FileURL], [FileId])
SELECT
logs.RequestedFile As [URL],
FileId = dbo.fn_GetFileIdFromURL(l.RequestedFile, l.CleanFileName)
FROM dbo.Logs l (readuncommitted) LEFT OUTER JOIN dbo.Rollup
on FileUrl = RequestedFile
WHERE FileUrl IS NULL
The logic here is that if dbo.Rollup does not exist for the given FileUrl, then the left outer join will turn up null. The NOT EXISTS now becomes an IS NULL, which is faster.

SQL2000 to SQL2005. Query now a lot slower

This query used to take 3secs in SQL2000, now it takes about 70secs. Both databases give the same results. The 2005 database is not running in compatibility mode.
Currently we're rebuilding the query to run in SQL2005.. by a process of elimination and understanding the logic.
However - can anyone see anything obvious that we've missed.
And/or are there any tools that could help here?
We've been looking at the Execution plan... and profiler. And index tuning wizard.
Profiler points to a massive number more records being queried to get the same results.
I know that this is a very hard question to debug without the data... another pair of eyes is always good if there is anything obvious!
Cheers
Dave
ALTER PROCEDURE [dbo].[GetNodeList]
#ViewID int,
#UserID int = null
as
Select ProcessList.*,
A.NDOC_DOC_ID,
A.NDOC_Order,
A.OMNIBOOK_ID,
A.Node_Order
from (
(SELECT N.NOD_ID,
N.NOD_Name,
N.NOD_Procname,
N.NOD_Xpos,
N.NOD_Ypos,
N.NOD_Zpos,
VN.VNOD_VIE_ID
FROM Node N
INNER JOIN View_NODe VN
ON N.NOD_ID = VN.VNOD_NOD_ID
Where VN.VNOD_VIE_ID = #ViewID) ProcessList
Left Join
(
SELECT N.NOD_ID,
N.NOD_Name,
N.NOD_Procname,
N.NOD_Xpos as NOD_Xpos,
N.NOD_Ypos as NOD_Ypos,
N.NOD_Zpos as NOD_Zpos,
VN.VNOD_VIE_ID,
ND.NDOC_DOC_ID as NDOC_DOC_ID,
ND.NDOC_Order as NDOC_Order,
null as OMNIBOOK_ID,
null as Node_Order
FROM Node N
INNER JOIN View_NODe VN
ON N.NOD_ID = VN.VNOD_NOD_ID
LEFT JOIN NODe_DOCument ND
ON N.NOD_ID = ND.NDOC_NOD_ID
WHERE VN.VNOD_VIE_ID=#ViewID
and ND.NDOC_DOC_ID is not null
and (#UserID is null
or exists (Select 1
from Document D
where Doc_ID = ND.NDOC_DOC_ID
and dbo.fn_UserCanSeeDoc(#UserID,D.Doc_ID)<>0
)
)
UNION
SELECT N.NOD_ID,
N.NOD_Name,
N.NOD_Procname,
N.NOD_Xpos,
N.NOD_Ypos,
N.NOD_Zpos,
VN.VNOD_VIE_ID,
null,
null,
NOM.OMNIBOOK_ID,
NOM.Node_Order
FROM Node N
INNER JOIN View_NODe VN
ON N.NOD_ID = VN.VNOD_NOD_ID
LEFT JOIN NODe_OMNIBOOK NOM
ON N.NOD_ID = NOM.NODE_ID
WHERE VN.VNOD_VIE_ID=#ViewID
and NOM.OMNIBOOK_ID is not null
and exists (select 1 from Omnibook_Doc where OmnibookID = NOM.OMNIBOOK_ID)
) A
--On ProcessList.NOD_ID = A.NOD_ID
ON ProcessList.NOD_Xpos = A.NOD_Xpos
And ProcessList.NOD_Ypos = A.NOD_Ypos
And ProcessList.NOD_Zpos = A.NOD_Zpos
And ProcessList.VNOD_VIE_ID = A.VNOD_VIE_ID
)
ORDER BY
ProcessList.NOD_Xpos,
ProcessList.NOD_Zpos,
ProcessList.NOD_Ypos,
Coalesce(A.NDOC_Order,A.Node_Order),
Coalesce(A.NDOC_DOC_ID,A.OMNIBOOK_ID)
I've seen this before when the statistics haven't kept up with the data. It's possible in this instance that SQL Server 2005 uses the statistics differently to SQL Server 2000. Try rebuilding your statistics for the tables used in the query; so for each table:
UPDATE STATISTICS <table> WITH FULLSCAN
Yes, I'd add the FULLSCAN unless you know your data well enough to think that a sample of records will give good enough results. It'll slow down the stats creation, but will make it more accurate.
Is it possible that your statistics haven't come across? in the 2k5 dbase? So the dbase doesn't have the info needed to make a good plan? As opposed to your old database which has good stats on the table and can choose a better plan for the data?
Could it be an issue with "parameter sniffing", i.e. SQL Server caching a query plan optimized for the parameters supplied for the first execution?
Microsoft technet has more
A college has come up with a solution... regarding bringing the function fn_UserCanSeeDoc back into the SQL.
Shown below is the old commented out function code, then the new inline SQL below it. The code now runs super quick (from over 1 minute to about a second)
Looking at the old SQL I'm surprised how good a job SQL2000 did of running it!
Cheers
--and dbo.fn_UserCanSeeDoc(#UserID,D.Doc_ID)<>0
-- if exists(Select 1 from Omnibook where Omnibook_ID = #DocID)
-- Begin
-- Set #ReturnVal = 1
-- End
--
-- else
-- Begin
-- if exists(
-- Select 1
-- from UserSecurityModule USM
-- Inner join DocSecurity DS
-- On USM.SecurityModuleID = DS.SecurityModuleID
-- where USM.UserID = #UserID
-- and DS.DocID = #DocID
-- )
--
-- Set #ReturnVal = 1
--
-- else
--
-- Set #ReturnVal = 0
-- End
AND D.Doc_ID IN (select DS.DocID from UserSecurityModule USM
Inner join DocSecurity DS
On USM.SecurityModuleID = DS.SecurityModuleID
where USM.UserID = #UserID)

T-SQL looping to create a recordset

I have some stored procedures that I need to write against a nasty beast of an database. I need to loop through a table (application) and pull values out of other tables (some are aggerate / averages /etc values) using the application_id from the application table.
So far I have:
declare #id INT
declare app cursor for
SELECT application_id from application
OPEN app
FETCH NEXT FROM app
INTO #id
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT a.NAME_LAST, a.NAME_FIRST, ca.status, (SELECT AVG(score) FROM reviews WHERE application_id = #id), (SELECT count(*) FROM reviews WHERE application_id = #id) FROM application a, committee_applications ca WHERE a.application_id = ca.application_id AND a.application_id = #id
FETCH NEXT FROM app INTO #id
END
CLOSE app
DEALLOCATE app
Which is giving me the results I want, but I'm sure there is a cleaner way of doing this, and I can't seem to make the mental jump today to do this correctly. Could someone point out a better way of doing this as this seems really ugly to me.
Also, it seems like I should be storing these values into a temp table then returning the full results instead of running the SELECT statement one by one.
Any suggestions would be greatly appreciated.
Thanks.
Removing the cursor will probably speed this up a lot. By using a derived table, you can get the counts and averages with one query and join that back to the other tables to get the remaining columns. Like this....
SELECT a.NAME_LAST,
a.NAME_FIRST,
ca.status,
Scores.AverageScore,
Scores.CountScore
FROM application a
Inner Join committee_applications ca
On a.application_id = ca.application_id
Left Join (
SELECT application_id,
AVG(score) As AverageScore,
Count(*) As CountScore
FROM reviews
Group By application_id
) As Scores
On a.application_id = Scores.application_id