How to improve performance of left join in AX Query - sql

Question

I'm trying to create a query to fetch dynamics AX salestable records with their history log records in a left join.
If I filter on the salesid I naturally get the results in a split second, but if I leave out the salesid it takes forever even though SalesID SN16129492 is the only 'valid' record for 2017/03/08?:
Declare #Bedrijf NVARCHAR(3) = 'dat';
Declare #leverdatum_van DATE = convert(datetime, '2017/03/08');
Declare #leverdatum_tot DATE = convert(datetime, '2017/03/08');
Declare #bedrag decimal = 1;
SELECT SUM(SALESLINE.LINEAMOUNT) AS waarde, SALESTABLE.CUSTACCOUNT, SALESTABLE.SALESID as ordernummer, CUSTTABLE.NAME as klantnaam,
MAX(SalesTable.createdBy) as ingevoerd_door, log.dader, log.veldnummer, log.huidigestatus, log.vorigestatus
FROM SALESLINE INNER JOIN
SALESTABLE ON SALESLINE.SALESID = SALESTABLE.SALESID AND SALESLINE.DATAAREAID = SALESTABLE.DATAAREAID
INNER JOIN CUSTTABLE ON
SALESTABLE.DATAAREAID=CUSTTABLE.DATAAREAID AND SALESTABLE.CUSTACCOUNT=CUSTTABLE.ACCOUNTNUM
LEFT JOIN
(select dbo.CONPEEK(CAST(dbo.CONPEEK(data, 2) AS varbinary(8000)), 2) AS DADER,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 1) AS VELDNUMMER,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 2) AS HUIDIGESTATUS,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 3) AS VORIGESTATUS, LOGRECID
from SYSDATABASELOG
where TABLE_=366 AND DATAAREAID=#bedrijf
) log
ON SALESTABLE.DATAAREAID=#bedrijf AND SALESTABLE.RECID=log.LOGRECID
WHERE (SALESLINE.DATAAREAID = #bedrijf) AND SALESTABLE.SALESTYPE=4 --retourorder
AND (SALESTABLE.SHIPPINGDATEREQUESTED between #leverdatum_van and #leverdatum_tot)
--AND (Salestable.SALESID = 'SN16129492')
AND (SALESTABLE.SalesOrderStatus in (0,1,2,3,4,5))
GROUP BY SALESTABLE.CUSTACCOUNT, SALESTABLE.SALESID, CUSTTABLE.NAME, log.dader, log.veldnummer, log.huidigestatus, log.vorigestatus
HAVING (SUM(ABS(SALESLINE.LINEAMOUNT))> #bedrag)
Is there a way to improve performance / redesign this query?

Answer 1

You must explicitly include dataareaid AND partition in conditions for all tables (you can either add it as a filter, or in join conditions), even if you just have a single partition.
By default, AX2012 implicitly adds the dataareaid and partition fields to all indexes created in the AOT as the first two fields.
As we found, SQL server is notoriously bad with handling these kinds of indexes, apart from the fact that it can't just "skip" the first level if you don't provide it explicitly (partition), it is also really bad at estimating index statistics (SQL server does a histogram for the first field in the index, and assumes the rest of the fields in the index have a normal distribution).

Answer 2

This is not an answer, but I couldn't write code so long in a comment.
The result could be different, but it's just to help us understand if a part of your query degrade performance (GROUP BY). I moved part of group fields outside.
Can you try this and see if it is "going forever" (using your words)?
SELECT A.*
,log.dader
,log.veldnummer
,log.huidigestatus
,log.vorigestatus
FROM (
SELECT SUM(SALESLINE.LINEAMOUNT) AS waarde
,SALESTABLE.CUSTACCOUNT
,SALESTABLE.SALESID AS ordernummer
,CUSTTABLE.NAME AS klantnaam
,MAX(SalesTable.createdBy) AS ingevoerd_door
,SALESTABLE.RECID /* ADDED THIS */
FROM SALESLINE
INNER JOIN SALESTABLE ON SALESLINE.SALESID = SALESTABLE.SALESID AND SALESLINE.DATAAREAID = SALESTABLE.DATAAREAID
INNER JOIN CUSTTABLE ON SALESTABLE.DATAAREAID = CUSTTABLE.DATAAREAID AND SALESTABLE.CUSTACCOUNT = CUSTTABLE.ACCOUNTNUM
WHERE SALESLINE.DATAAREAID = #bedrijf
AND SALESTABLE.SALESTYPE = 4 --retourorder
AND SALESTABLE.SHIPPINGDATEREQUESTED BETWEEN #leverdatum_van AND #leverdatum_tot
--AND (Salestable.SALESID = 'SN16129492')
AND SALESTABLE.SalesOrderStatus IN (0, 1, 2, 3, 4, 5)
GROUP BY SALESTABLE.CUSTACCOUNT
,SALESTABLE.SALESID
,CUSTTABLE.NAME
,SALESTABLE.RECID /* ADDED THIS */
HAVING SUM(ABS(SALESLINE.LINEAMOUNT)) > #bedrag
) A
LEFT JOIN (SELECT dbo.CONPEEK(CAST(dbo.CONPEEK(data, 2) AS VARBINARY(8000)), 2) AS DADER
,dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS VARBINARY(8000)), 1) AS VELDNUMMER
,dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS VARBINARY(8000)), 2) AS HUIDIGESTATUS
,dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS VARBINARY(8000)), 3) AS VORIGESTATUS
,LOGRECID
FROM SYSDATABASELOG
WHERE TABLE_ = 366
AND DATAAREAID = #bedrijf) log ON SALESTABLE.RECID = log.LOGRECID

Answer 3

By adding filters for createddatetime for bot left and right table I've decreased processing time from 'forever' to < 30 seconds for a period of one month.
Optimized Query:
Declare #Bedrijf NVARCHAR(3) = 'dat';
Declare #leverdatum_van DATETIME = convert(datetime, '2017/03/01');
Declare #leverdatum_tot DATETIME = convert(datetime, '2017/04/01');
Declare #bedrag decimal = 1;
SELECT SUM(SALESLINE.LINEAMOUNT) AS waarde, SALESTABLE.CUSTACCOUNT, SALESTABLE.SALESID as ordernummer, CUSTTABLE.NAME as klantnaam,
MAX(SalesTable.createdBy) as ingevoerd_door, log.dader, log.veldnummer, log.huidigestatus, log.vorigestatus
FROM SALESLINE
INNER JOIN SALESTABLE
ON SALESLINE.SALESID = SALESTABLE.SALESID AND SALESLINE.DATAAREAID = SALESTABLE.DATAAREAID
INNER JOIN CUSTTABLE
ON SALESTABLE.DATAAREAID=CUSTTABLE.DATAAREAID AND SALESTABLE.CUSTACCOUNT=CUSTTABLE.ACCOUNTNUM
LEFT JOIN (
select dbo.CONPEEK(CAST(dbo.CONPEEK(data, 2) AS varbinary(8000)), 2) AS DADER,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 1) AS VELDNUMMER,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 2) AS HUIDIGESTATUS,
dbo.CONPEEK(CAST(dbo.CONPEEK(data, 5) AS varbinary(8000)), 3) AS VORIGESTATUS, LOGRECID
from SYSDATABASELOG
where TABLE_=366 AND DATAAREAID=#bedrijf
and createddatetime between #leverdatum_van and #leverdatum_tot
) log
ON SALESTABLE.DATAAREAID=#bedrijf AND SALESTABLE.RECID=log.LOGRECID
WHERE (SALESLINE.DATAAREAID = #bedrijf) AND SALESTABLE.SALESTYPE=4 --retourorder
AND (SALESTABLE.createddatetime between #leverdatum_van and #leverdatum_tot)
--AND (Salestable.SALESID = 'SN16129492')
AND (SALESTABLE.SalesOrderStatus in (0,1,2,3,4,5))
GROUP BY SALESTABLE.CUSTACCOUNT, SALESTABLE.SALESID, CUSTTABLE.NAME, log.dader, log.veldnummer, log.huidigestatus, log.vorigestatus
HAVING (SUM(ABS(SALESLINE.LINEAMOUNT))> #bedrag)
More clustered index seeks in the new execution plan:
Thnx for helping me out ;)