Slow MS Access Query (Using DSum & DCount Functions) - sql

Question

I'm having an issue in Microsoft Access where my query calculates extremely slow (it takes hours and hours). This query is reading a table that has 150,000 records and each record belongs to one of 4,000 unique groups (called API_10).
The goal of the query is to calculate a running cumulative production value (organized by API_10 and Date) such that the running cumulative production starts over at each new API_10 group. Each record in the table has a field called No which is an autonumber that MS Access calculates so that the table has a Primary Key. An example of what I'm describing is shown below:
MyTable:
No API_10 Date Production
1 1 1/1/2010 1000
2 1 2/1/2010 500
3 2 7/1/2014 300
4 2 8/1/2014 400
MyQuery:
No API_10 Date Production Cumulative_Production
1 1 11/1/2010 1000 1000
2 1 12/1/2010 500 1500
3 2 27/1/2014 300 300
4 2 28/1/2014 400 700
Here is a sample of the code (typed in the Expression Builder on MS Access) used to create the Cumulative_Production column in MyQuery:
Cumulative_Production:
DSum("[Production]","[MyTable]","[API_10]='" & [API_10] & "' AND [No]<=" & [No])
Do note that this is a simplified version of the actual query/table. The real query also computes another field called Normalized_Prod_Month which counts the number of production dates (starting at 1) for each unique API_10 as shown below:
NORMALIZED_PROD_MONTH:
DCount("[Date]","[MyTable]","[API_10]='" & [API_10] & "' AND [No]<=" & [No])
Any tips for improving these types of calculations would greatly help!!

Answer 1

If you apply this query to each record, then you must access n * (n + 1) / 2 records. If all 4000 groups have about the same size of 38 records, you get 4000 * 38 * (38 + 1) / 2 = ~ 3 Mio accesses. But this is the best case, since larger groups have an over-proportional cost because of the quadratic nature of n * (n + 1) / 2.
You are better off by creating the running sum in a loop in VBA, and accessing each record only once.
Dim db As DAO.Database, rs As DAO.Recordset
Dim lastNoApi As Long, runningSum As Long
Set db = CurrentDb
Set rs = db.OpenRecordset("SELECT * FROM MyTable ORDER BY NoAPI_10, Date")
Do Until rs.EOF
If rs!NoAPI_10 <> lastNoApi Then
runningSum = 0
lastNoApi = rs!NoAPI_10
End If
runningSum = runningSum + rs!Production
'TODO: insert the result into a temporary table
rs.MoveNext
Loop
rs.Close: Set rs = Nothing
db.Close: Set db = Nothing
Or use the following query. It still has a quadratic cost, but a single query is always more performing than multiple calls to DCount, DSum or DLookup.
SELECT
A.API_10,
A.Date,
A.Production,
(Select Sum(B.Production)
FROM MyTable B
WHERE B.API_10 = A.API_10 And B.[No] <= A.[No]) AS Cumulative_Production
FROM MyTable AS A
ORDER BY A.API_10, A.Date;
Assuming that the No column is consistent with the date sequence. If the dates are unique, you can also replace B.[No] <= A.[No] with B.[Date] <= A.[Date].