Below I will be describing all the tables which I will be using.
The first table is the Base table on which we have to calculate the cost.
Del_Line table
The second Table is Outbound_Cost_Table
The third table contains the column which has a calculation for each countrycityplant_code
Calculation Table
There can be more countries and cities with different plant codes and calculations.
What I want to do is to write the generic query to calculate the outbound cost at each delivery line since the Calculation table will be dynamic and can be updated by the business users at any time.
I will show you the sample query calculating the outbound cost for one city.
with cte as (
Select A.plant_code, B.Cost/(Sum(A.Packs)*2) cost_per_pack
from #del_line A left join Outbound_Cost B
on A.dl_plant_code = B.Plant_code and A.city = B.City
group by A.[DL_Plant_Code],B.Cost
)
Select A.*,
B.cost_per_pack*A.Packs*2 as Outbound_Handling_Cost
from #del_line A left join CTE B
On A.dl_plant_code = B.dl_plant_code
In this above query, I am calculating for Dallas.
Now I want to write the generic query in which I dont have to harcode for every city.
I want something like this
with cte as (
Select A.plant_code, C.Calc cost_per_pack
from #del_line A left join Outbound_Cost B
on A.dl_plant_code = B.Plant_code and A.city = B.City left Join #Calc C on A.Market = C.Market and C.kpi = 'Outbound Handling'and C.kpi_level = 'Delivery Line'
group by A.[DL_Plant_Code],B.Cost
)
Select A.*,
C.calc as Outbound_Handling_Cost
from #del_line A left join CTE B
On A.dl_plant_code = B.dl_plant_code left Join #Calc C on A.Market = C.Market and C.Handling = 'Outbound Handling' and C.[KPI Level] = 'Delivery Line Drilldown'
Please avoid using the loop since the base table del_line is very big.
Could anyone help me speed this query up? It currently take 17 minutes to run but does return the correct data and it populates a subform in MS Access. Functions in the rest of the VBA are declared as long to try to speed up more.
Here's the full query:
SELECT lots of things
FROM (((((((((((((((ngstest
INNER JOIN patients
ON ngstest.internalpatientid = patients.internalpatientid)
INNER JOIN referral
ON ngstest.referralid = referral.referralid)
INNER JOIN checker
ON ngstest.bookby = checker.check1id)
INNER JOIN ngspanel
ON ngstest.ngspanelid = ngspanel.ngspanelid)
LEFT JOIN ngspanel AS ngspanel_1
ON ngstest.ngspanelid_b = ngspanel_1.ngspanelid)
INNER JOIN status
ON ngstest.statusid = status.statusid)
INNER JOIN dbo_patient_table
ON patients.patientid = dbo_patient_table.patienttrustid)
LEFT JOIN dna
ON ngstest.dna = dna.dnanumber)
INNER JOIN status AS status_1
ON patients.s_statusoverall = status_1.statusid)
LEFT JOIN gw_gendertable
ON dbo_patient_table.genderid = gw_gendertable.genderid)
LEFT JOIN ngswesbatch
ON ngstest.wesbatch = ngswesbatch.ngswesbatchid)
LEFT JOIN checker AS checker_1
ON ngstest.check1id = checker_1.check1id)
LEFT JOIN checker AS checker_2
ON ngstest.check2id = checker_2.check1id)
LEFT JOIN checker AS checker_3
ON ngstest.check3id = checker_3.check1id)
LEFT JOIN ngspanel AS ngspanel_2
ON ngstest.ngspanelid_c = ngspanel_2.ngspanelid)
LEFT JOIN checker AS checker_4
ON ngstest.check4id = checker_4.check1id
WHERE ((ngstest.referralid IN
(SELECT referralid FROM referral
WHERE grouptypeid = 14)
AND ngstest.ngstestid IN
(SELECT ngstest.ngstestid
FROM ngsanalysis
INNER JOIN ngstest
ON ngsanalysis.ngstestid = ngstest.ngstestid
WHERE ngsanalysis.pedigree = 3302) )
AND status.statusid = 1202218800)
ORDER BY ngstest.priority,
ngstest.daterequested;
The two nested queries are strings from elsewhere in the code so are called in the vba as " & includereferralls & " And " & ParentsStatusesFilter & "
They are:
ParentsStatusesFilter = "NGSTest.NGSTestID in
(SELECT NGSTest.NGSTestID
FROM NGSAnalysis
INNER JOIN NGSTest
ON NGSAnalysis.NGSTestID = NGSTest.NGSTestID
WHERE NGSAnalysis.Pedigree IN (3302,3303,3304)"
And
includereferrals = "NGSTest.ReferralID
(SELECT referralid FROM referral WHERE referral.grouptypeid = 14)"
The query needs to remain readable (and therefore editable) so can't use things like Distinct, Group By or contain any Unions. Have tried Exists instead of In for the nested queries but that stops it from actually filtering the results.
WHERE EXISTS (SELECT NGSTest.NGSTestID
FROM NGSAnalysis
INNER JOIN NGSTest
ON NGSAnalysis.NGSTestID = NGSTest.NGSTestID
WHERE NGSAnalysis.Pedigree IN (3302,3303,3304)
So the exist clause you have there isn't tied to the outer query which would run similar to just added 1 = 1 to the where clause. I took your where clause and converted it. It should look something like this...
WHERE EXISTS (
SELECT referralid
FROM referral
WHERE grouptypeid = 14 AND ngstest.referralid = referral.referralid)
AND EXISTS (
SELECT ngsanalysis.ngstestid
FROM ngsanalysis
WHERE ngsanalysis.pedigree IN (3302,3303,3304) AND ngstest.ngstestid = ngsanalysis.ngstestid
)
AND status.statusid = 1202218800
Adding exists will speed it up a bit, but the the bulk of the slowness is the left joins. Access does not handle the left joins as well as SQL Server does. Change all your joins to inner joins and you will see the query runs very fast. This is obviously not ideal since some relationships are optional. What I have done to get around this is add a default record that replaces a null relationship.
Here is what that looks like for you: In the checker table you could add a record that represents a null value. So put a record into the checker table with check1id of -1 or 0. Then default check1id, check2id, check3id on ngstest to -1 or 0. You will need to do that type of thing for all tables you need to left join on.
I am trying to write a query that gets Income Statements for Investments and calculates how much each has made dependent on the time frame.
SELECT
iis.GrossIncome, dates.[Month], dates.[Year], s.TotalGrossIncome
FROM
dbo.InvestmentIncomeStatements AS iis
JOIN
dbo.IssueDates AS dates ON iis.IssueDateID = dates.ID
JOIN
(SELECT
f.InvestmentID, SUM(f.GrossIncome) AS TotalGrossIncome
FROM
(SELECT
stment.InvestmentID, stment.GrossIncome, inment.[Name] as InvestmentName
FROM
dbo.InvestmentIncomeStatements stment
JOIN
dbo.IssueDates AS dts ON stment.IssueDateID = dts.ID
JOIN
dbo.Investments inment ON stment.InvestmentID = inment.ID
WHERE
dts.[Month] + dts.[Year] < '???') AS f
GROUP BY
f.InvestmentID, f.InvestmentName) AS s ON iis.InvestmentID = s.InvestmentID
In the place of the '???' I would like to write 'dates.[Year] + dates.[Month]'.
However I can't refer it. What should i do?
I'm trying to combine two different SQL queries into one table. I've tried various joins and Union but it either duplicates rows or doesn't show all of them.
The first query is
Select
HW.DisplayName,
HW.LocationDetails_0B39A057_2BE8_11B2_BBE2_1E03564AA5CA,
HW.Notes_5CFC0E2A_AB82_5830_D4BB_0596CBED1984
FROM MT_Cireson$AssetManagement$HardwareAsset HW
where HardwareAssetStatus_3019ADDF_4F3D_2C55_2024_72C22E11F4CF = '866879DF-8FB6-E521-F0E3-FEF86EE1BC92'
This gives all of my hardware assets that have the status I'm looking for.
The second query is:
SELECT
hw.DisplayName,
HW.LocationDetails_0B39A057_2BE8_11B2_BBE2_1E03564AA5CA,
HW.Notes_5CFC0E2A_AB82_5830_D4BB_0596CBED1984,
UB.UPN_7641DFF7_7A20_DC04_FC1C_B6FA8715DA02
FROM MT_Cireson$AssetManagement$HardwareAsset HW
inner join Relationship Rel on HW.BaseManagedEntityId = Rel.SourceEntityId
inner join RelationshipType RT on RT.RelationshipTypeId = Rel.RelationshipTypeId
inner join MT_Microsoft$AD$UserBase UB on UB.BaseManagedEntityId = Rel.TargetEntityId
where RT.RelationshipTypeName = 'Cireson.AssetManagement.HardwareAssetHasPrimaryUser'
and HardwareAssetStatus_3019ADDF_4F3D_2C55_2024_72C22E11F4CF = '866879DF-8FB6-E521-F0E3-FEF86EE1BC92'
This gives all of the hardware assets I'm looking for that have a primary user configured, but doesn't give the assets without a primary user. I'm not sure how to either A: combine the results just putting in NULL as a primary user for records that don't have one, or B: actually query all the assets at one time and include the primary user column.
I didn't write the second query and I'm not sure exactly how it works. I've tried doing union between the queries but that duplicates the rows because the first query already contains all the elements in the second.
Edit: The PrimaryUser comes from the MT_Microsoft$AD$UserBase table. I've tried adding another column to the first and just setting it as null like:
null as primaryUser,
How about a LEFT JOIN to include all records from HW that are not in UB:
SELECT
hw.DisplayName,
HW.LocationDetails_0B39A057_2BE8_11B2_BBE2_1E03564AA5CA,
HW.Notes_5CFC0E2A_AB82_5830_D4BB_0596CBED1984,
UB.UPN_7641DFF7_7A20_DC04_FC1C_B6FA8715DA02
FROM
MT_Cireson$AssetManagement$HardwareAsset HW
INNER JOIN
Relationship Rel
ON
HW.BaseManagedEntityId = Rel.SourceEntityId
INNER JOIN
RelationshipType RT
ON
RT.RelationshipTypeId = Rel.RelationshipTypeId
LEFT JOIN
MT_Microsoft$AD$UserBase UB
ON
UB.BaseManagedEntityId = Rel.TargetEntityId
WHERE
RT.RelationshipTypeName = 'Cireson.AssetManagement.HardwareAssetHasPrimaryUser'
AND HardwareAssetStatus_3019ADDF_4F3D_2C55_2024_72C22E11F4CF = '866879DF-8FB6-E521-F0E3-FEF86EE1BC92'
UPDATE:
If the null primary users is what you want, I would recraft the query like:
SELECT
hw.DisplayName,
HW.LocationDetails_0B39A057_2BE8_11B2_BBE2_1E03564AA5CA,
HW.Notes_5CFC0E2A_AB82_5830_D4BB_0596CBED1984,
UB.UPN_7641DFF7_7A20_DC04_FC1C_B6FA8715DA02
FROM
MT_Cireson$AssetManagement$HardwareAsset HW
LEFT JOIN
MT_Microsoft$AD$UserBase UB
ON
HW.BaseManagedEntityId = UB.SourceEntityId
INNER JOIN
RelationshipType RT
ON
RT.RelationshipTypeId = Rel.RelationshipTypeId
INNER JOIN
Relationship Rel
ON
UB.BaseManagedEntityId = Rel.TargetEntityId
WHERE
RT.RelationshipTypeName = 'Cireson.AssetManagement.HardwareAssetHasPrimaryUser'
AND HardwareAssetStatus_3019ADDF_4F3D_2C55_2024_72C22E11F4CF = '866879DF-8FB6-E521-F0E3-FEF86EE1BC92'
I LEFT JOIN'ed HW and UB tables.
As I said earlier, you'll have to tweak the joins. I would try a LEFT JOIN on all tables.
you have an extra column in your 1st query that's why you have duplicates in your union. I would suggest using CTE, there might be other better efficient solutions out there.
;WITH query1
AS
(
SELECT col1, col2
FROM table
),
query2 AS
(
SELECT col1, col2
FROM table
)
SELECT *
FROM cteTable1
UNION ALL
SELECT *
FROM cteTable2
WHERE NOT EXISTS(SELECT * FROM cteTable1 WHERE cteTable1.col1 = cteTable2.col2)
-- WITH POD was causing the issue, removing this code reduced 2 year pull to 3 mins.
Will post new question to figure out best way to include POD data.
--Edit for clarity, I am a read only user to these tables.
I wrote the below query, but it takes a very long time to execute (20min).
It is currently limited to 1 month, but user wants at least 1 year preferably 2. I assume this would scale time to hours.
Can anyone take a look at let me know if there is a BKM I am not using to improve performance?
Or if there is a better method for a report of this size? At 2 years, it would return ~100K rows from 17 tables.
WITH
POD AS
(
SELECT SHIPMENTS.Delivery
,SHIPMENTS.Shipment_Number
,PROOF_OF_DELIVERY.Shipping_Carrier
,PROOF_OF_DELIVERY.Tracking_Number
,PROOF_OF_DELIVERY.Ship_Method
,PROOF_OF_DELIVERY.POD_Signature
,PROOF_OF_DELIVERY.POD_Date
,PROOF_OF_DELIVERY.POD_Time
FROM
SHIPMENTS
LEFT JOIN PROOF_OF_DELIVERY
ON SHIPMENTS.Shipment_Number = PROOF_OF_DELIVERY.Delivery_Or_Shipment
WHERE Load_Date IN
(
SELECT MAX(Load_Date)
FROM PROOF_OF_DELIVERY
GROUP BY Delivery_Or_Shipment
)
)
SELECT DISTINCT GI.GOODS_ISSUE_DOCUMENT_ID
,GI.SALES_ORDER_ID
,GI.SALES_ORDER_LINE_ID
,GI.SALES_ORDER_TYPE_CODE
,GI.DELIVERY_HEADER_ID
,GI.DELIVERY_ITEM_ID
,FD.FISCAL_MONTH_CODE
,GI.MATERIAL_NUMBER
,GI.SHIPPED_QTY
,SO.ORDERER_NAME
,SO.CREATED_BY
,SO.CONTACT_PERSON
,GI.SOLD_TO_CUSTOMER_ID
,GI.SHIP_TO_CUSTOMER_ID
,GI.ORIGINAL_COMMIT_DATE
,GI.SHIP_FROM_PLANT_ID
,GI.ACTUAL_PGI_DATE
,GI.CUSTOMER_PO_NUMBER
,GI.SHIPPED_PRICE
,(GI.SHIPPED_PRICE * GI.SHIPPED_QTY) AS EXT_SHIPPED_PRICE
,GI.SALES_ORGANIZATION_CODE
,GI.DELIVERY_NOTE_PRIORITY_CODE
,FD.FISCAL_WEEK_CODE
,DV.DIVISION_CODE
,DN.Delivery_Item_Creation_Date
,SOLD.CUSTOMER_SHORT_NAME AS SOLD_TO_CUSTOMER_SHORT_NAME
,SHIP.CUSTOMER_SHORT_NAME AS SHIP_TO_CUSTOMER_SHORT_NAME
,SHIP.Customer_Site_Name
,SHIP.REGION_NAME
,MATD.MATERIAL_DESCRIPTION
,MATD.STANDARD_COST
,(MATD.STANDARD_COST * GI.SHIPPED_QTY) AS EXT_STANDARD_COST
,MATD.GLOBAL_EVENT
,PLT.LEAD_TIME_FOR_ORIGINAL_COMMIT
,OPRM.BASE_PART_CODE
,MATD.PRODUCT_INSP_MEMO
,MATD.MATERIAL_PRICING_GROUP_CODE
,MATD.MATERIAL_STATUS AS MMPP
,PIM.PIM_PBG_GROUPING
,SOL.SHIPPING_CONDITION
,SVO.SERVICE_ORDER_NUM
,SO.CREATION_TIME AS SO_CREATION_TIME
,SOL.CREATED_TIME AS SO_LINE_CREATED_TIME
,SOL.SHIPPING_POINT
,SDT.SALES_DOCUMENT_TYPE_CODE AS SVO_DOCUMENT_TYPE_CODE
,EQU.EQUIPMENT_NUM
,EQU.SERIAL_NUMBER
,EQU.CUSTOMERTOOLID
,POD.Shipment_Number
,POD.Shipping_Carrier
,POD.Tracking_Number
,POD.Ship_Method
,POD.POD_Signature
,POD.POD_Date
,POD.POD_Time
,DATEDIFF(dd,SO.CREATION_TIME,GI.ACTUAL_PGI_DATE) AS Cycle_Time_to_PGI_Days
,DATEDIFF(hh,SO.CREATION_TIME,GI.ACTUAL_PGI_DATE) AS Cycle_Time_to_PGI_Hours
FROM GOODS_ISSUE AS GI
INNER JOIN dbo.Delivery_Notes AS DN
ON GI.DELIVERY_HEADER_ID = DN.DELIVERY_HEADER_CODE AND GI.DELIVERY_ITEM_ID = DN.DELIVERY_ITEM_CODE
INNER JOIN dbo.Customer_View AS SOLD
ON GI.SOLD_TO_CUSTOMER_ID = SOLD.CUSTOMER_CODE
INNER JOIN dbo.Customer_View AS SHIP
ON GI.SOLD_TO_CUSTOMER_ID = SHIP.CUSTOMER_CODE
INNER JOIN dbo.MATERIAL_DETAILS AS MATD
ON GI.MATERIAL_NUMBER = MATD.MATERIAL_NUMBER
INNER JOIN dbo.OPR_MATERIAL_DIM AS OPRM
ON OPRM.MATERIAL_NUMBER = GI.MATERIAL_NUMBER
LEFT JOIN dbo.SM_DATE_DIM AS FD
ON CAST(FD.CALENDAR_DAY AS DATE) = CAST(GI.ACTUAL_PGI_DATE AS DATE)
LEFT JOIN dbo.DIM_PUBLISHED_LEAD_TIME_COMMIT AS PLT
ON PLT.MATERIAL_NUMBER = OPRM.BASE_PART_CODE
LEFT JOIN dbo.PRODUCT_INSP_MEMO_DIM AS PIM
ON PIM.PRODUCT_INSP_MEMO = MATD.PRODUCT_INSP_MEMO
INNER JOIN dbo.SM_SALES_ORDER_LINE_FACT AS SOL
ON SOL.SALES_ORDER_CODE = GI.SALES_ORDER_ID AND SOL.SALES_ORDER_LINE_CODE = GI.SALES_ORDER_LINE_ID
INNER JOIN dbo.SM_SALES_ORDER_FACT AS SO
ON SO.SALES_ORDER_CODE = GI.SALES_ORDER_ID
INNER JOIN dbo.SM_DIVISION_DIM AS DV
ON SO.DIVISION_SID = DV.DIVISION_SID
LEFT JOIN dbo.SERVICE_ORDER_FACT AS SVO
ON SVO.SERVICE_ORDER_NUM = SO.SERVICE_ORDER_NUMBER
LEFT JOIN dbo.SM_SALES_DOCUMENT_TYPE_DIM AS SDT
ON SDT.SALES_DOCUMENT_TYPE_SID = SVO.SALES_DOCUMENT_TYPE_SID
LEFT JOIN dbo.SM_EQUIPMENT_DIM AS EQU
ON EQU.EQUIPMENT_SID = SVO.EQUIPMENT_SID
LEFT JOIN POD
ON POD.Delivery = GI.DELIVERY_HEADER_ID
WHERE GI.ACTUAL_PGI_DATE > GETDATE()-32
AND SOLD_TO_CUSTOMER_ID IN (0010000252,0010000898,0010001121,0010001409,0010001842,0010001852,0010001879,0010001977,0010001978,0010002021,0010002202,0010002227,0010002982,0010003118,0010003176,0010003294,0010005492,0010006904,0010007048,0010007080,0010010381,0010010572,0010010905,0010011999,0010012014,0010012048,0010012571,0010013124,0010013711,0010013713,0010013824,0010014180,0010014188,0010014333,0010015059,0010015313,0010015414,0010015541,0010015544,0010015550)
A CTE is just syntax
I suspect that CTE is evaluated many times
Materialze the CTE to #temp with indexe(s) so it is run once
This cast will hurt it
Make those columns true dates and index them
ON CAST(FD.CALENDAR_DAY AS DATE) = CAST(GI.ACTUAL_PGI_DATE AS DATE)
That where negates the left so you can just do a join
Also that MAX(Load_Date) could match on another shipment
SELECT SHIPMENTS.Delivery
,SHIPMENTS.Shipment_Number
,PROOF_OF_DELIVERY.Shipping_Carrier
,PROOF_OF_DELIVERY.Tracking_Number
,PROOF_OF_DELIVERY.Ship_Method
,PROOF_OF_DELIVERY.POD_Signature
,PROOF_OF_DELIVERY.POD_Date
,PROOF_OF_DELIVERY.POD_Time
FROM SHIPMENTS
JOIN PROOF_OF_DELIVERY
ON SHIPMENTS.Shipment_Number = PROOF_OF_DELIVERY.Delivery_Or_Shipment
WHERE PROOF_OF_DELIVERY.Load_Date IN
(
SELECT MAX(Load_Date)
FROM PROOF_OF_DELIVERY
GROUP BY Delivery_Or_Shipment
)
Pull this up into the join
INNER JOIN dbo.Customer_View AS SOLD
ON GI.SOLD_TO_CUSTOMER_ID = SOLD.CUSTOMER_CODE
AND GI.SOLD_TO_CUSTOMER_ID IN (0010000252,0010000898,0010001121,0010001409,0010001842,0010001852,0010001879,0010001977,0010001978,0010002021,0010002202,0010002227,0010002982,0010003118,0010003176,0010003294,0010005492,0010006904,0010007048,0010007080,0010010381,0010010572,0010010905,0010011999,0010012014,0010012048,0010012571,0010013124,0010013711,0010013713,0010013824,0010014180,0010014188,0010014333,0010015059,0010015313,0010015414,0010015541,0010015544,0010015550)