What is the proper way to fill a form with multiple data fields? - vb.net

I have been asked to create a vb.net WinForms app that automates the creation of this spreadsheet:
Obviously, creating that spreadsheet manually is a back breaking task.
I am very comfortable writing SQL to pull each individual field of data, however I've got to believe that there is a better way. Such a method would require 144 queries !!
I've considered creating a form with individual labels for each field, or using a gridview. Either would be acceptable to the end user, but I'm really not sure how I would write a query to produce a dataset that look like the end product anyway.
I'm not asking anyone to write any code for me, what I'm asking for help with is the concept of how I should attack this task.
Here is a query that I wrote which returns the fields for Business Unit #1 for the first week on the spreadsheet. I don't think it helps much, but I'm adding it to show that I've put some effort into this.
DECLARE #WeekEnding AS DATE
DECLARE #DIV AS INT
SET #WeekEnding = '11/13/2015'
SET #DIV = 20
-- A/R downpayments OPCH
SELECT
SUM(OPCH.DocTotal - OPCH.VatSum - OPCH.TotalExpns) AS 'ARDownPaymentInvoice'
FROM OPCH
LEFT JOIN INV1 ON OPCH.DocEntry = INV1.DocEntry AND INV1.VisOrder = 0
LEFT JOIN OITM ON INV1.ItemCode = OITM.ItemCode
LEFT JOIN OITB ON OITM.ItmsGrpCod = OITB.ItmsGrpCod
LEFT JOIN OACT AS CurrCode (NOLOCK) ON OITB.RevenuesAc = CurrCode.AcctCode
WHERE DATEPART(WEEK, OPCH.DocDate) = DATEPART(WEEK, #WeekEnding) AND YEAR(OPCH.DocDate) = YEAR(#WeekEnding)
AND CurrCode.Segment_4 = #DIV
-- Credits ORIN
SELECT
SUM(ORIN.DocTotal - ORIN.VatSum - ORIN.TotalExpns) * -1 AS 'Credit'
FROM ORIN
LEFT JOIN INV1 ON ORIN.DocEntry = INV1.DocEntry AND INV1.VisOrder = 0
LEFT JOIN OITM ON INV1.ItemCode = OITM.ItemCode
LEFT JOIN OITB ON OITM.ItmsGrpCod = OITB.ItmsGrpCod
LEFT JOIN OACT AS CurrCode (NOLOCK) ON OITB.RevenuesAc = CurrCode.AcctCode
WHERE DATEPART(WEEK, ORIN.DocDate) = DATEPART(WEEK, #WeekEnding) AND YEAR(ORIN.DocDate) = YEAR(#WeekEnding)
AND CurrCode.Segment_4 = #DIV
--Invoices
SELECT
SUM(OINV.DocTotal - OINV.VatSum - OINV.TotalExpns) AS 'Invoice'
FROM OINV
LEFT JOIN INV1 ON OINV.DocEntry = INV1.DocEntry AND INV1.VisOrder = 0
LEFT JOIN OITM ON INV1.ItemCode = OITM.ItemCode
LEFT JOIN OITB ON OITM.ItmsGrpCod = OITB.ItmsGrpCod
LEFT JOIN OACT AS CurrCode (NOLOCK) ON OITB.RevenuesAc = CurrCode.AcctCode
WHERE DATEPART(WEEK, OINV.DocDate) = DATEPART(WEEK, #WeekEnding) AND YEAR(OINV.DocDate) = YEAR(#WeekEnding)
AND CurrCode.Segment_4 = #DIV`
Thanks for any ideas.

I would do a pivot rather than doing multiple queries. However I think you should ask yourself or the end user: "Do you really want to see an aging report by date as columns?" If so do I add one manually each time the report ages another increment? Here is a self extracting example FYI. The first set is just displaying my data where I mock up a user, a department they belong to, and their last access. Then I pretend I want a report of aging to see how many people access the system in total, and by each fictitious department. Doing multiple queries in SQL is expensive and can often be alleviated with a little know how of pivots, unions, and holding your data creatively with SQL.
DECLARE #Example TABLE (PersonId INT IDENTITY, PersonName VARCHAR(128), DepartmentName VARCHAR(128), AccessedOn Date);
INSERT INTO #Example (PersonName, DepartmentName, AccessedOn) VALUES ('Brett', 'Dev', '1-1-2017'), ('John', 'Dev', '1-6-2017'), ('Mark', 'Dev', '1-8-2017'), ('Shawn', 'Ops', '1-15-2017'), ('Ryan', 'Ops', '1-16-2017'), ('Kevin', 'Ops', '1-21-2017');
--Data as is
SELECT *
From #Example
--I would use Date By row as it is far easier to maintain
SELECT
DATEADD(WEEK, DATEDIFF(WEEK, 0, AccessedOn), 0) AS Grouping
, COUNT(PersonId) AS PeopleAccessedAtTime
, SUM(CASE WHEN DepartmentName = 'Dev' THEN 1 ELSE 0 END) AS DevCounts
, SUM(CASE WHEN DepartmentName = 'Ops' THEN 1 ELSE 0 END) AS OpsCounts
FROM #Example
GROUP BY DATEADD(WEEK, DATEDIFF(WEEK, 0, AccessedOn), 0)
--Aging Report you are asking for trouble as you need to manually pivot ELSE go down the road of doing dynamic sql to do a pivot which is No Bueno
; WITH x AS
(
SELECT
DATEADD(WEEK, DATEDIFF(WEEK, 0, AccessedOn), 0) AS Grouping
, COUNT(PersonId) AS PeopleAccessedAtTime
, SUM(CASE WHEN DepartmentName = 'Dev' THEN 1 ELSE 0 END) AS DevCounts
, SUM(CASE WHEN DepartmentName = 'Ops' THEN 1 ELSE 0 END) AS OpsCounts
FROM #Example
GROUP BY DATEADD(WEEK, DATEDIFF(WEEK, 0, AccessedOn), 0)
)
Select
'PeopleAccessed' AS Header
, MAX([2017-01-02]) AS [2017-01-02]
, Max([2017-01-09]) AS [2017-01-09]
, Max([2017-01-16]) AS [2017-01-16]
From x
PIVOT(MAX(PeopleAccessedAtTime) FOR Grouping IN ([2017-01-02], [2017-01-09], [2017-01-16])) AS pvt
UNION
Select
'DevDivisionAccessed'
, MAX([2017-01-02]) AS [2017-01-02]
, Max([2017-01-09]) AS [2017-01-09]
, Max([2017-01-16]) AS [2017-01-16]
From x
PIVOT(MAX(DevCounts) FOR Grouping IN ([2017-01-02], [2017-01-09], [2017-01-16])) AS pvt
UNION
Select
'OpsDivisionAccessed'
, MAX([2017-01-02]) AS [2017-01-02]
, Max([2017-01-09]) AS [2017-01-09]
, Max([2017-01-16]) AS [2017-01-16]
From x
PIVOT(MAX(OpsCounts) FOR Grouping IN ([2017-01-02], [2017-01-09], [2017-01-16])) AS pvt

One way to achieve this:
CREATE TABLE #tmpData
(column1 varchar(50),
column2 varchar(50),
column3 varchar(50),
column4 varchar(50),
column5 varchar(50),
column6 varchar(50),
column7 varchar(50),
column8 varchar(50),
column9 varchar(50),
column10 varchar(50),
column11 varchar(50),
column12 varchar(50))
Then you could insert a row displaying the dates of each column
INSERT INTO #tmpData
--Write your select query here for filling in all the dates
Then you would insert the data for each line item in the rows.
INSERT INTO #tmpdata
SELECT 'Business Unit 1',
-- insert remaining column data here

Related

Not does not exclude query info

I have a really long query and I'm finding that my NOT is not excluding what's in parenthesis after the NOT.
I saw Exclude and where not exists, but I'd have to re-select for that, and there's too many complicatedly joined tables in what I selected already, plus one table is very big and takes a long time to select what I have already, so I can't re-select because it will make the query take too long. How do I get this exclusion to work?
INSERT INTO #UNeedingC(id, CASEID, firstname, lastname, userid, AGEOFNOTIFICATION, DATETIMEDECISIONMADE, DELEGATESYSTEM, Person_id, request_type_id, service_place_id, status_summary, externalUserId, subject, onDate, externalPersonId, externalSystemId)
select distinct
c.id
,uc.case_id
,t_case.FIRSTNAME as first
,t_case.LASTNAME as last
,t_case.user_id as userid
,CONVERT(VARCHAR, DATEDIFF(dd, SC.status_change_date, GETDATE())) + ' Day(s) ' + CONVERT(VARCHAR, DATEDIFF(hh, SC.status_change_date, GETDATE()) % 24) + ' Hour(s) ' as [AGE OF NOTIFICATION]
,SC.status_change_date AS [DATE TIME DECISION MADE]
,[ckoltp_sys].DBO.ckfn_GetStringLocaleValue(152,9,uc.delegate_system,50,0) AS [DELEGATESYSTEM]
,c.person_id
,uc.request_type_id ------
,uc.service_place_id
,uc.status_summary
,eou.external_id
,c.tzix_id+' '+[ckoltp_sys].dbo.ckfn_GetStringLocaleValue(148,9,uc.status_summary,5,0)+' type' AS subject
,dateadd( hour,41,dateadd(day,0,datediff(d,0,sc.status_change_date)) ) AS onDate
,emd.externalId externalPersonId
,eou.system_id as externalSystemId
--,u.disable
from
#tempC t_case with (NOLOCK)
inner join dbo.org_case c with (nolock) ON t_case.Person_id=c.Person_id
INNER JOIN dbo.org_2_case uc with (NOLOCK) ON uc.case_id=c.id
inner JOIN dbo.ORG_LOS S WITH (NOLOCK) ON S.case_id = UC.case_id
inner JOIN dbo.ORG_EXTENSION SC WITH (NOLOCK) ON SC.los_id= S.id
inner join dbo.org_user u with (NOLOCK) on u.id=t_case.user_id
inner join dbo.org_person op with (NOLOCK) on op.id=c.Person_id
inner JOIN dbo.u_person_concept_value MC ON MC.CID = op.cid --this is the slow table
inner join dbo.EXTERNAL_ORG_USER_DATA eou with (NOLOCK) ON eou.org_user_id = t_case.user_id
inner join dbo.EXTERNAL_person_DATA emd with (NOLOCK) ON emd.CID = op.cid --op.id --?
WHERE
DATEDIFF(day, SC.status_change_date , GETDATE()) <= 2
AND
u.disable <> 1
AND
( --(denied/approved)
dbo.ckfn_GetStringLocaleValue(148,9,uc.status_summary,5,0) = 'Denied'
OR
(dbo.ckfn_GetStringLocaleValue(148,9,uc.status_summary,5,0) in( 'Fully Approved', 'Partially Approved'))
)
AND
(
(
ISNULL(uc.request_type_id,'') in( 12)
AND DATEDIFF(month, SC.status_change_date , GETDATE()) <= 2
)
OR
(
ISNULL(uc.request_type_id,'') in( 10,11)
)
--OR
--(
-- --exclude this
-- (
-- MC.concept_id = '501620' --general val1 (1000/1001)
-- AND
-- (C.ID in (select case_id from #CASES where str_value in ('1000','1001'))
-- AND (uc.service_place_id = 31 OR uc.service_place_id = 32))
-- ) --not
--) --or
)--AND
AND
(t_case.firstname not like '%external%' and t_case.lastname not like '%case manager%')
AND
(
C.ID in (select case_id from #CASES where concept_id='501620')--MC.concept_id = '501620'
)
--overall around AND (denied/approved)--
and DBO.ckfn_GetStringLocaleValue(152,9,uc.delegate_system,50,0) in ('AP','CA')
AND NOT --this not is not working...this appears in query results
(
--exclude these
(
MC.concept_id = '501620'
AND
(C.ID in (select case_id from #CASES where str_value in ('1000','1001'))
AND (uc.service_place_id = 31 OR uc.service_place_id = 32))
) --not
) --
select * from #UNeedingC
results show what is excluded:
id caseid firstname lastname userid ageofNotification Datetimedecisionmade DelegateSys Person_id request_type_id service_place_id status_summary externalUserId subject
onDate externalPersonId externalSystemId
000256200 256200 Sree Par 1234 0 Apr 5 CA
4270000 11 31 3 sparee 000256200 Fully Approved tested Ad 2021-04-06 17:00 363000 2
My question: do you know why the NOT is not working and how I can get this to exclude without another select? See "this not is not working" comment. I searched online but only found exclude and where not exists, which require another select, which I don't want.
I think I figured it out: "NOT acts on one condition. To negate two or more conditions, repeat the NOT for each condition,"
from not on two things.
This seems to work:
...
AND
--exclude these
(
MC.concept_id = '501620' --general val1 (1000/1001)
AND
(C.ID not in (select case_id from #CASES where str_value in ('1000','1001'))
AND (uc.service_place_id not in ('31','32')))
) --not

SQL Server / T-SQL : query optimization assistance

I have this QA logic that looks for errors into every AuditID within a RoomID to see if their AuditType were never marked Complete or if they have two complete statuses. Finally, it picks only the maximum AuditDate of the RoomIDs with errors to avoid showing multiple instances of the same RoomID, since there are many audits per room.
The issue is that the AUDIT table is very large and takes a long time to run. I was wondering if there is anyway to reach the same result faster.
Thank you in advance !
IF object_ID('tempdb..#AUDIT') is not null drop table #AUDIT
IF object_ID('tempdb..#ROOMS') is not null drop table #ROOMS
IF object_ID('tempdb..#COMPLETE') is not null drop table #COMPLETE
IF object_ID('tempdb..#FINALE') is not null drop table #FINALE
SELECT distinct
oc.HotelID, o.RoomID
INTO #ROOMS
FROM dbo.[rooms] o
LEFT OUTER JOIN dbo.[hotels] oc on o.HotelID = oc.HotelID
WHERE
o.[status] = '2'
AND o.orderType = '2'
SELECT
t.AuditID, t.RoomID, t.AuditDate, t.AuditType
INTO
#AUDIT
FROM
[dbo].[AUDIT] t
WHERE
t.RoomID IN (SELECT RoomID FROM #ROOMS)
SELECT
t1.RoomID, t3.AuditType, t3.AuditDate, t3.AuditID, t1.CompleteStatus
INTO
#COMPLETE
FROM
(SELECT
RoomID,
SUM(CASE WHEN AuditType = 'Complete' THEN 1 ELSE 0 END) AS CompleteStatus
FROM
#AUDIT
GROUP BY
RoomID) t1
INNER JOIN
#AUDIT t3 ON t1.RoomID = t3.RoomID
WHERE
t1.CompleteStatus = 0
OR t1.CompleteStatus > 1
SELECT
o.HotelID, o.RoomID,
a.AuditID, a.RoomID, a.AuditDate, a.AuditType, a.CompleteStatus,
c.ClientNum
INTO
#FINALE
FROM
#ROOMS O
LEFT OUTER JOIN
#COMPLETE a on o.RoomID = a.RoomID
LEFT OUTER JOIN
[dbo].[clients] c on o.clientNum = c.clientNum
SELECT
t.*,
Complete_Error_Status = CASE WHEN t.CompleteStatus = 0
THEN 'Not Complete'
WHEN t.CompleteStatus > 1
THEN 'Complete More Than Once'
END
FROM
#FINALE t
INNER JOIN
(SELECT
RoomID, MAX(AuditDate) AS MaxDate
FROM
#FINALE
GROUP BY
RoomID) tm ON t.RoomID = tm.RoomID AND t.AuditDate = tm.MaxDate
One section you could improve would be this one. See the inline comments.
SELECT
t1.RoomID, t3.AuditType, t3.AuditDate, t3.AuditID, t1.CompleteStatus
INTO
#COMPLETE
FROM
(SELECT
RoomID,
COUNT(1) AS CompleteStatus
-- Use the above along with the WHERE clause below
-- so that you are aggregating fewer records and
-- avoiding a CASE statement. Remove this next line.
--SUM(CASE WHEN AuditType = 'Complete' THEN 1 ELSE 0 END) AS CompleteStatus
FROM
#AUDIT
WHERE
AuditType = 'Complete'
GROUP BY
RoomID) t1
INNER JOIN
#AUDIT t3 ON t1.RoomID = t3.RoomID
WHERE
t1.CompleteStatus = 0
OR t1.CompleteStatus > 1
Just a thought. Streamline your code and your solution. you are not effectively filtering your datasets smaller so you continue to query the entire tables which is taking a lot of your resources and your temp tables are becoming full copies of those columns without the indexes (PK, FK, ++??) on the original table to take advantage of. This by no means is a perfect solution but it is an idea of how you can consolidate your logic and reduce your overall data set. Give it a try and see if it performs better for you.
Note this will return the last audit record for any room that has either not had an audit completed or completed more than once.
;WITH cte AS (
SELECT
o.RoomId
,o.clientNum
,a.AuditId
,a.AuditDate
,a.AuditType
,NumOfAuditsComplete = SUM(CASE WHEN a.AuditType = 'Complete' THEN 1 ELSE 0 END) OVER (PARTITION BY o.RoomId)
,RowNum = ROW_NUMBER() OVER (PARTITION BY o.RoomId ORDER BY a.AuditDate DESC)
FROm
dbo.Rooms o
LEFT JOIN dbo.Audit a
ON o.RoomId = a.RoomId
WHERE
o.[Status] = 2
AND o.OrderType = 2
)
SELECT
oc.HotelId
,cte.RoomId
,cte.AuditId
,cte.AuditDate
,cte.AuditType
,cte.NumOfAuditsComplete
,cte.clientNum
,Complete_Error_Status = CASE WHEN cte.NumOfAuditsComplete > 1 THEN 'Complete More Than Once' ELSE 'Not Complete' END
FROM
cte
LEFT JOIN dbo.Hotels oc
ON cte.HotelId = oc.HotelId
LEFT JOIN dbo.clients c
ON cte.clientNum = c.clientNum
WHERE
cte.RowNum = 1
AND cte.NumOfAuditsComplete != 1
Also note I changed your
WHERE
o.[status] = '2'
AND o.orderType = '2'
TO
WHERE
o.[status] = 2
AND o.orderType = 2
to be numeric without the single quotes. If the data type is truely varchar add them back but when you query a numeric column as a varchar it will do data conversion and may not take advantage of indexes that you have built on the table.

Using results of JOIN statement to join on the results of another JOIN statement

I have the following 2 Join Statements:
--Get Total Hrs
DECLARE #BeginDate datetime, #EndDate datetime
set #BeginDate = '01-01-2013'
set #EndDate = '12-31-2013'
BEGIN
SELECT F.Type, E.Product, SUM(F.Hours * E.Amount) AS 'Total Hours'
FROM Hours H
INNER JOIN Equipment E
ON F.SN = E.SN
WHERE (F.Date BETWEEN #BeginDate AND #EndDate)
GROUP BY F.Type, E.Product
ORDER BY Product ASC
END
--Get Number of Unscheduled Removals
DECLARE #BeginDate1 datetime, #EndDate1 datetime
set #BeginDate1 = '01-01-2013'
set #EndDate1 = '12-31-2013'
BEGIN
SELECT LEFT(dbo.fn_GetPartName(R.PartID),CHARINDEX('-',dbo.fn_GetPartName(R.PartID), 1) - 1) AS 'Part No',
Count(s.status) AS NumberUnscheduledRemovals
FROM Repair R
INNER JOIN Conversion C
ON R.Performed = C.Performed
AND R.Confirmed = C.Confirmed
INNER JOIN Status S
ON C.StatusID = S.StatusID
WHERE (R.Received BETWEEN #BeginDate1 AND #EndDate1)
AND (S.Status = 'UNSCHEDULED')
GROUP BY LEFT(dbo.fn_GetPartName(R.PartID),CHARINDEX('-',dbo.fn_GetPartName(R.PartID), 1) - 1)
ORDER BY LEFT(dbo.fn_GetPartName(R.PartID),CHARINDEX('-',dbo.fn_GetPartName(R.PartID), 1) - 1) ASC
END
Both queries have results including part numbers (these have the same values). I want to INNER JOIN the results from both queries on the resulting part numbers. have been trying for a while but cant seem to get the syntax right to do it.
Use a temp table using CREATE TABLE #TempPartNum1 & #TempPartNum2.
Grab all the relevant data from the first two queries and put them in the temp tables, then join the temp tables.
You could also use a CTE ("Common Table Expression"):
;WITH QueryOne AS (
... put your first query here
), QueryTwo AS (
... put your second query here
) SELECT blah blah blah
FROM QueryOne INNER JOIN QueryTwo ON foo = bar
CTEs are very handy for things like this.

Showing all values in Group By with inclusion of CASE

I have the following data
CREATE TABLE #Test
(
Severity NVARCHAR(50)
,WorkId INT
)
INSERT INTO #Test VALUES('High',1)
INSERT INTO #Test VALUES('Critical',2)
SELECT
CASE
WHEN Severity IN ('High','Critical') THEN 'Critical'
WHEN Severity IN ('Low','Medium') THEN 'Medium'
END AS 'Severity'
,COUNT(*) AS 'Total'
FROM #Test
GROUP BY
CASE
WHEN Severity IN ('High','Critical') THEN 'Critical'
WHEN Severity IN ('Low','Medium') THEN 'Medium'
END
This results in output -
Severity | Total
---------+-------
Critical | 2
I am expecting the following output -
Severity | Total
---------+-------
Critical | 2
Medium | 0
I have looked into the following two links which details a similar case but not same and am still unable to get the result -
http://ask.sqlservercentral.com/questions/47705/showing-null-values-as-well-in-group-by-in-sql.html
How to return empty groups in SQL GROUP BY clause
Any help or links?
Further update.
Having tried the solution below, the results are still not appearing. Pasting here the actual code wherein I would need to apply the logic
SELECT s.NewSeverity AS 'Severity'
,COUNT(WI.microsoft_vsts_common_severity) AS 'Total'
FROM ( VALUES
('Critical','I-High')
,('High','I-High')
,('Medium','I-Low')
,('Low','I-Low')
)s(OldSeverity,NewSeverity)
LEFT JOIN DimWorkItem WI (NOLOCK)
ON WI.microsoft_vsts_common_severity = s.OldSeverity
JOIN dbo.DimPerson P
ON p.personsk = WI.system_assignedto__personsk
JOIN DimTeamProject TP
ON WI.TeamProjectSK = TP.ProjectNodeSK
JOIN DimIteration Itr (NOLOCK)
ON Itr.IterationSK = WI.IterationSK
JOIN DimArea Ar (NOLOCK)
ON Ar.AreaSK = WI.AreaSK
WHERE TP.ProjectNodeName = 'ABC'
AND WI.System_WorkItemType = 'Bug'
AND WI.Microsoft_VSTS_CMMI_RootCause <> 'Change Request'
AND Itr.IterationPath LIKE '%\ABC\R1234\Test\IT%'
AND WI.System_State NOT IN ( 'Rejected', 'Closed' )
AND WI.System_RevisedDate = CONVERT(datetime, '9999', 126)
GROUP BY s.NewSeverity
In actual there are only two 'Low' items, hence the output I am getting is I-Low, 2 whereas I want even I-High to appear with 0 count.
The way I would go about this is to create your own table of values using a table value constructor:
SELECT OldSeverity, NewSeverity
FROM (VALUES
('Critical', 'Critical'),
('High', 'Critical'),
('Medium', 'Medium'),
('Low', 'Medium')
) s (OldSeverity, NewSeverity);
This gives a table you can select from, then left join to your existing table:
SELECT Severity = s.NewSeverity,
Total = COUNT(t.Severity)
FROM (VALUES
('Critical', 'Critical'),
('High', 'Critical'),
('Medium', 'Medium'),
('Low', 'Medium')
) s (OldSeverity, NewSeverity)
LEFT JOIN #Test t
ON t.Severity = s.OldSeverity
GROUP BY s.NewSeverity;
This will give the desired results.
Example on SQL Fiddle
EDIT
The problem you have with the way that you are implimenting the query, is that although you have immediately left joined to DimWorkItem you then inner join to subsequent tables and refer to columns in WorkItem in the where clause, which undoes your left join and turns it back into an inner join. You need to place your whole logic into a subquery, and left join to this:
SELECT s.NewSeverity AS 'Severity'
,COUNT(WI.microsoft_vsts_common_severity) AS 'Total'
FROM ( VALUES
('Critical','I-High')
,('High','I-High')
,('Medium','I-Low')
,('Low','I-Low')
)s(OldSeverity,NewSeverity)
LEFT JOIN
( SELECT wi.Severity
FROM DimWorkItem WI (NOLOCK)
JOIN dbo.DimPerson P
ON p.personsk = WI.system_assignedto__personsk
JOIN DimTeamProject TP
ON WI.TeamProjectSK = TP.ProjectNodeSK
JOIN DimIteration Itr (NOLOCK)
ON Itr.IterationSK = WI.IterationSK
JOIN DimArea Ar (NOLOCK)
ON Ar.AreaSK = WI.AreaSK
WHERE TP.ProjectNodeName = 'ABC'
AND WI.System_WorkItemType = 'Bug'
AND WI.Microsoft_VSTS_CMMI_RootCause <> 'Change Request'
AND Itr.IterationPath LIKE '%\ABC\R1234\Test\IT%'
AND WI.System_State NOT IN ( 'Rejected', 'Closed' )
AND WI.System_RevisedDate = CONVERT(datetime, '9999', 126)
) WI
ON WI.Severity = s.OldSeverity
GROUP BY s.NewSeverity;

Multiple Recordsets Issue

I have the below SQL Script which is run from Excel VBA using ADO against a SQL Server 2000 database.
The issue I have is that although there is only one SELECT statement in the script, I sometimes receive three recordsets back when executing the .Open method. I say sometimes as on occassion and on other 'parallel' databases I only receive one recordset back.
I know all about the .NextRecordset() method etc. but I am trying to understand why I get three recordsets back some times and other times I only receive one. I'm shortly going to run a SQL trace to see if that throws up any ideas but as usual, any help or advice would be greatly appreciated.
SET NOCOUNT ON
DECLARE #RunDate VARCHAR(8)
SET #RunDate = CONVERT(VARCHAR(8), DATEADD(d, -1 * 1, GETDATE()), 112)
IF OBJECT_ID('tempdb..#ActiveOrders') IS NOT NULL DROP TABLE #ActiveOrders
IF OBJECT_ID('tempdb..#ApplicableOrders') IS NOT NULL DROP TABLE #ApplicableOrders
IF OBJECT_ID('tempdb..#FilterOut') IS NOT NULL DROP TABLE #FilterOut
/*Temp table created as it has a self-join in the below query */
CREATE TABLE #ActiveOrders(
order_id VARCHAR(30)
, instrument_id VARCHAR(30)
, side CHAR(1)
)
CREATE INDEX idx_ActiveOrders_orderId ON #ActiveOrders(order_id)
/*Build dataset of all orders which have had activity on the run date or are in an Open status. Ignoring Program Trades.*/
INSERT INTO #ActiveOrders
SELECT o1.order_id COLLATE Latin1_General_CI_AS
, o1.instrument_id
, o1.side
FROM orders o1
INNER JOIN desk d1 ON d1.desk_id = o1.investment_desk
INNER JOIN (SELECT o0.order_id
FROM orders o0
WHERE ((LEFT(o0.added_datetime, 8) = #RunDate
OR LEFT(o0.approved_datetime, 8) = #RunDate)
OR (LEFT(o0.added_datetime, 8) <= #RunDate
AND o0.summary_status IN (1, 2, 3, 5, 8, 9))) /*Approved, Assigned, Acknowledged, Working, Partial, WorkingPartial*/
UNION
(SELECT r0.order_id
FROM releases r0
WHERE LEFT(r0.added_datetime, 8) = #RunDate)
UNION
(SELECT e0.order_id
FROM executions e0
WHERE LEFT(e0.execution_datetime, 8) = #RunDate
OR LEFT(e0.allocated_datetime, 8) = #RunDate)
) t1 ON o1.order_id = t1.order_id
WHERE d1.location_id = 'LDEQ'
AND o1.summary_status <> 4
AND o1.list_id IS NULL /*Ignore program trades*/
/*This is now the actual dataset we are interested in.
This is everything which could be a contender for aggregation.*/
CREATE TABLE #ApplicableOrders(
order_id VARCHAR(30)
, instrument_id VARCHAR(30)
, side CHAR(1)
, approved_datetime DATETIME
, acknowledged_datetime DATETIME
, last_allocation_datetime DATETIME
, latest_status INT
, merged_orders VARCHAR(500)
, dealer VARCHAR(100)
, manager VARCHAR(100)
, limit_price FLOAT
, original_qty FLOAT
, executed_qty FLOAT
, trader_instruction TEXT
, dealer_note TEXT
)
CREATE INDEX idx_ApplicableOrders_orderId ON #ApplicableOrders(order_id)
CREATE INDEX idx_ApplicableOrders_lastAllocation ON #ApplicableOrders(last_allocation_datetime)
CREATE INDEX idx_ApplicableOrders_approved ON #ApplicableOrders(approved_datetime)
/*All orders from #ActiveOrders where there are two or more orders which are for the same instrument and in the same direction.*/
INSERT INTO #ApplicableOrders
SELECT o.order_id
, o.instrument_id
, o.side
, dbo.mglz_datetime(o.approved_datetime)
, dbo.mglz_datetime(o.ack_datetime)
, MAX(dbo.mglz_datetime(e.allocated_datetime)) "Last Allocation DateTime"
, o.summary_status
, o.merged_orders
, o.ack_id
, o.approver_id
, o.limit_price
, o.original_qty
, o.executed_qty_at
, CONVERT(VARCHAR(900), o.trader_instruction)
, CONVERT(VARCHAR(900), o.dealer_note)
FROM orders o
INNER JOIN #ActiveOrders t ON o.order_id = t.order_id COLLATE Latin1_General_CI_AS
INNER JOIN #ActiveOrders s ON s.order_id <> o.order_id COLLATE Latin1_General_CI_AS
AND s.instrument_id = o.instrument_id COLLATE Latin1_General_CI_AS
AND s.side = o.side COLLATE Latin1_General_CI_AS
LEFT JOIN executions e ON e.order_id = o.order_id
GROUP BY o.order_id
, o.instrument_id
, o.side
, o.approved_datetime
, o.ack_datetime
, o.summary_status
, o.merged_orders
, o.ack_id
, o.approver_id
, o.limit_price
, o.original_qty
, o.executed_qty_at
, CONVERT(VARCHAR(900), o.trader_instruction)
, CONVERT(VARCHAR(900), o.dealer_note)
/*Filter out any orders where Order2.Approved_Date > Order1.Last_Release_Date AND Order1.Is_Complete
Order1 is defined as the order which was approved first.*/
SELECT t1.*
INTO #FilterOut
FROM #ApplicableOrders t1
WHERE EXISTS (SELECT 1
FROM
(SELECT order2.order_id
FROM (SELECT b.order_id
, b.instrument_id
, b.side
, b.approved_datetime
, b.last_allocation_datetime
, b.latest_status
, b.executed_qty
, b.original_qty
FROM #ApplicableOrders b
WHERE b.approved_datetime = (SELECT MIN(b1.approved_datetime) FirstApproval
FROM #ApplicableOrders b1
WHERE b1.instrument_id = b.instrument_id
AND b1.side = b.side)
) order1
INNER JOIN
(SELECT c.order_id
, c.instrument_id
, c.side
, c.approved_datetime
FROM #ApplicableOrders c
WHERE c.approved_datetime > (SELECT MIN(c1.approved_datetime) FirstApproval
FROM #ApplicableOrders c1
WHERE c1.instrument_id = c.instrument_id
AND c1.side = c.side)
) order2
ON order1.instrument_id = order2.instrument_id
AND order1.side = order2.side
AND order2.approved_datetime > order1.last_allocation_datetime
AND (order1.latest_status = 6 OR order1.executed_qty = order1.original_qty)) filter1
WHERE t1.order_id = filter1.order_id)
/*Filter out any orders where Order2.Acknowledged_Date > Order1.Last_Allocation_Date.*/
INSERT INTO #FilterOut
SELECT t1.*
FROM #ApplicableOrders t1
WHERE EXISTS (SELECT 1
FROM
(SELECT order2.order_id
FROM (SELECT b.order_id
, b.instrument_id
, b.side
, b.approved_datetime
, b.last_allocation_datetime
FROM #ApplicableOrders b
WHERE b.approved_datetime = (SELECT MIN(b1.approved_datetime) FirstApproval
FROM #ApplicableOrders b1
WHERE b1.instrument_id = b.instrument_id
AND b1.side = b.side)
) order1
INNER JOIN
(SELECT c.order_id
, c.instrument_id
, c.side
, c.approved_datetime
, c.acknowledged_datetime
FROM #ApplicableOrders c
WHERE c.approved_datetime > (SELECT MIN(c1.approved_datetime) FirstApproval
FROM #ApplicableOrders c1
WHERE c1.instrument_id = c.instrument_id
AND c1.side = c.side)
) order2
ON order1.instrument_id = order2.instrument_id
AND order1.side = order2.side
AND order2.acknowledged_datetime > order1.last_allocation_datetime) filter2
WHERE t1.order_id = filter2.order_id)
AND NOT EXISTS (SELECT 1
FROM #FilterOut a1
WHERE a1.order_id = t1.order_id)
/*Filter any 'single' orders. I.e. all 'matching' orders have been excluded so the instrument/direction combination is not applicable for Aggregation.*/
INSERT INTO #FilterOut
SELECT t1.*
FROM #ApplicableOrders t1 INNER JOIN (SELECT DISTINCT t.instrument_id
, t.side
FROM #ApplicableOrders t
INNER JOIN #FilterOut a ON t.instrument_id = a.instrument_id
AND t.side = a.side
GROUP BY t.instrument_id
, t.side
HAVING COUNT(t.instrument_id) > 1) t2 ON t1.instrument_id = t2.instrument_id
AND t1.side = t2.side
WHERE NOT EXISTS (SELECT 1
FROM #FilterOut a1
WHERE a1.order_id = t1.order_id)
/*Final Report*/
/*A list of all orders where aggregation could have possibly occurred but didn't.*/
SELECT t1.order_id "Order ID"
, i.name "Name"
, t1.side "B/S"
, userDealer.short_name "Dlr"
, userManager.short_name "FM"
, t1.limit_price "Limit"
, t1.approved_datetime "Order Approved"
, t1.acknowledged_datetime "Order Acknowledged"
, t1.last_allocation_datetime "Last Execution"
, t1.merged_orders "Merged Orders"
, m.description "Status"
, t1.dealer_note "Dealer Note"
, t1.trader_instruction "Trader Instruction"
FROM #ApplicableOrders t1
INNER JOIN instrument i ON t1.instrument_id = i.instrument_id COLLATE Latin1_General_CI_AS
INNER JOIN mnemonics m ON t1.latest_status = m.value AND m.attribute = 'order_summary_status'
LEFT JOIN users userDealer ON userDealer.user_id = t1.dealer COLLATE Latin1_General_CI_AS
LEFT JOIN users userManager ON userManager.user_id = t1.manager COLLATE Latin1_General_CI_AS
WHERE NOT EXISTS (SELECT 1
FROM #FilterOut t2
WHERE t1.order_id = t2.order_id)
ORDER BY t1.name
, t1.side
, t1.approved_datetime
IF OBJECT_ID('tempdb..#ActiveOrders') IS NOT NULL DROP TABLE #ActiveOrders
IF OBJECT_ID('tempdb..#ApplicableOrders') IS NOT NULL DROP TABLE #ApplicableOrders
IF OBJECT_ID('tempdb..#FilterOut') IS NOT NULL DROP TABLE #FilterOut
I would add an isotope column (dummy column) to the end of your temp tables so that when you see the three recordsets you can interrogate the columns names to see where they're coming from. And what happens when you take the SQL from profile and run directly in SSMS? Do you get just one resultset?
I finally found the issue and thought I would report back. The problem was with NULL values in two of the Aggregation Functions in the script. The reason this only occurred occasionally is because the data was constantly changing.
Although SQL Server handles the issue 'silently' it does produce warnings which ADO sees as recordsets although in my case (might be every case) these recordsets are closed and so impossible to actually see what they contain or what produced them.
Instead of re-writing the SQL script with ISNULL() logic, I have opted simply to include the
SET ANSI_WARNINGS OFF
statement at the top of the script. This prevents the warnings from being reported and thus ADO does not produce these additional, unwanted recordsets.
Disappointed I didn't spot this earlier but at least I've learnt something new. :)