I'm trying to build one SQL query for Access that links tables with myodbc connection to retrive the data from internet, but the time to finish the query is too long about five minutes, so I think the problem is with the query:
SELECT COUNT([o].[orders_id]) AS howmany_orders,
(SELECT SUM([op1].[products_quantity]) FROM orders_total AS ot1, orders AS o1, orders_products AS op1
WHERE [o1].[date_purchased] >=date()-30 and [o1].[orders_id] = [op1].[orders_id] and [ot1].[orders_id] = [op1].[orders_id] and [ot1].[class]="ot_total" and [o1].[orders_status] = 1 and [op1].[products_id]=[op].[products_id]
GROUP BY [op1].[products_id]
) AS pendiente,
[op].[products_model],
Round((((7+1)*(howmany_orders/90))+1)-(p.stock_real- IIF(pendiente>0,pendiente,0)), 0) AS pedir,
p.ref_id
FROM orders_total AS ot, orders AS o, orders_products AS op INNER JOIN Productos AS p ON Mid([op].[products_model],4) LIKE p.ref_id
WHERE [o].[date_purchased] >=date()-90 and [o].[orders_id] = [op].[orders_id] and [ot].[orders_id] = [op].[orders_id] and [ot].[class]="ot_total" and [o].[orders_status] IN (7, 1) and ((p.fuera_de_stock)=False) and ((p.suspendido)=False) and ((p.quitar_de_la_web)=False)
GROUP BY [op].[products_model], p.ref_id, p.stock_real, [op].[products_id];
At a glance I see that the "LIKE" operator could be one of the problems here:
INNER JOIN Productos AS p ON Mid([op].[products_model],4) LIKE p.ref_id
but I have not way to substitute for an = operator
Thanks for your help!
EDITING:
I have reduced the query to that but is the same time:
SELECT COUNT(o.orders_id) AS howmany_orders, (
SELECT SUM(opz.products_quantity) FROM orders AS oz, orders_products AS opz WHERE oz.date_purchased >=date()-30 and oz.orders_id = opz.orders_id and oz.orders_status = 1 and opz.products_id=op.products_id GROUP BY opz.products_id
) AS pendiente, op.products_model, Round((((7+1)*(howmany_orders/90))+1)-(p.stock_real-IIf(pendiente>0,pendiente,0)),0) AS pedir, p.ref_id
FROM orders AS o, orders_products AS op INNER JOIN Productos AS p ON op.products_model=p.cod
WHERE o.date_purchased>=date()-90 And o.orders_id=op.orders_id And o.orders_status In (7,1) And ((p.suspendido)=False) And ((p.quitar_de_la_web)=False)
GROUP BY op.products_model, p.ref_id, p.stock_real, op.products_id;
Yep there’s your problem. The database can not do the join using any indexes so it has to do a table scan. Is there anyway you could persist this data so you don’t have to do the MID statement and just join on that? i.e. in your products_model table have an extra column with the MID data stored in there with the join on that column
Related
I have the following SQL select statement that I use to get a subset of products, or wines:
SELECT pv.SkProdVariantId AS id,
pa.Colour AS colour,
FROM Dim.ProductVariant AS pv
JOIN ProductAttributes_new AS pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
The length of this table generated is 3,905. I want to get all the transactional data for these products.
At the moment I'm using this select statement
SELECT c.CalDate AS timestamp,
f.SkProductVariantId AS sku_id,
f.Quantity AS quantity
FROM fact.FTransactions AS f
LEFT JOIN Dim.Calendar AS c
ON f.SkDateId = c.SkDateId
LEFT JOIN (
SELECT pv.SkProdVariantId AS id,
pa.Colour AS colour,
FROM Dim.ProductVariant AS pv
JOIN ProductAttributes_new AS pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
) AS s
ON s.id = f.SkProductVariantId
WHERE c.CalDate LIKE '%2019%'
The calendar dates are correct, but the number of unique products returned is 5,648, rather than the expected 3,905 from the select query.
Why does my LEFT JOIN on the first select query not work as I expect it to, please?
Thanks for any help!
If you want all the rows form your query, it needs to be the first reference in the LEFT JOIN. Then, I am guessing that you want transaction in 2019:
select . . .
from (SELECT pv.SkProdVariantId AS id, pa.Colour AS colour,
FROM Dim.ProductVariant pv JOIN
ProductAttributes_new pa
ON pv.SkProdVariantId = pa.SkProdVariantId
WHERE pv.ProdTypeName = 'Wines'
) s LEFT JOIN
(fact.FTransactions f JOIN
Dim.Calendar c
ON f.SkDateId = c.SkDateId AND
c.CalDate >= '2019-01-01' AND
c.CalDate < '2020-01-01'
)
ON s.id = f.SkProductVariantId;
Note that this assumes that CalDate is really a date and not a string. LIKE should only be used on strings.
You misunderstand somehow how outer joins work. See Gordon's answer and my request comment on that.
As to the task: It seems you want to select transactions of 2019, but you want to restrict your results to wine products. We typically restrict query results in the WHERE clause. You can use IN or EXISTS for that.
SELECT
c.CalDate AS timestamp,
f.SkProductVariantId AS sku_id,
f.Quantity AS quantity
FROM fact.FTransactions AS f
INNER JOIN Dim.Calendar AS c ON f.SkDateId = c.SkDateId
WHERE DATEPART(YEAR, c.CalDate) = 2019
AND f.SkProductVariantId IN
(
SELECT pv.SkProdVariantId
FROM Dim.ProductVariant AS pv
WHERE pv.ProdTypeName = 'Wines'
);
(I've removed the join to ProductAttributes_new, because it doesn't seem to play any part in this query.)
I have the below query which takes a while to run, since ir_sales_summary is ~ 2 billion rows:
select c.ChainIdentifier, s.SupplierIdentifier, s.SupplierName, we.Weekend,
sum(sales_units_cy) as TY_unitSales, sum(sales_cost_cy) as TY_costDollars, sum(sales_units_ret_cy) as TY_retailDollars,
sum(sales_units_ly) as LY_unitSales, sum(sales_cost_ly) as LY_costDollars, sum(sales_units_ret_ly) as LY_retailDollars
from ir_sales_summary i
left join Chains c
on c.ChainID = i.ChainID
inner join Suppliers s
on s.SupplierID = i.SupplierID
inner join tmpWeekend we
on we.SaleDate = i.saledate
where year(i.saledate) = '2017'
group by c.ChainIdentifier, s.SupplierIdentifier, s.SupplierName, we.Weekend
(Worth noting, it takes roughly 3 hours to run since it is using a view that brings in data from a legacy service)
I'm thinking there's a way to speed up the filtering, since I just need the data from 2017. Should I be filtering from the big table (i) or be filtering from the much smaller weekending table (which gives us just the week ending dates)?
Try this. This might help, joining a static table as first table in query onto a fact/dynamic table will impact query performance i believe.
SELECT c.ChainIdentifier
,s.SupplierIdentifier
,s.SupplierName
,i.Weekend
,sum(sales_units_cy) AS TY_unitSales
,sum(sales_cost_cy) AS TY_costDollars
,sum(sales_units_ret_cy) AS TY_retailDollars
,sum(sales_units_ly) AS LY_unitSales
,sum(sales_cost_ly) AS LY_costDollars
,sum(sales_units_ret_ly) AS LY_retailDollars
FROM Suppliers s
INNER JOIN (
SELECT we
,weeekend
,supplierid
,chainid
,sales_units_cy
,sales_cost_cy
,sales_units_ret_cy
,sales_units_ly
,sales_cost_ly
,sales_units_ret_ly
FROM ir_sales_summary i
INNER JOIN tmpWeekend we
ON we.SaleDate = i.saledate
WHERE year(i.saledate) = '2017'
) i
ON s.SupplierID = i.SupplierID
INNER JOIN Chains c
ON c.ChainID = i.ChainID
GROUP BY c.ChainIdentifier
,s.SupplierIdentifier
,s.SupplierName
,i.Weekend
I have 2 queries. Individually, both run well. How when I combine them, they run terribly slow and time out on me. The second query is returning a list of invoices that I want to use in the IN clause in the first query. Not sure if I'm doing something wrong. Thanks for your help....
select
ba.bill_acct_nbr,
ba.acct_name,
i.inv_id,
i.prnt_tmsp,
i.due_dt,
d.dvr_frst_name,
d.dvr_srnm,
r.ecr_ticket_no,
r.co_tmsp,
i.dr_note_amt ,
ht.amt HT,
vat.amt VAT
from rfs.rnt_agr_inv_notes i
inner join rfs.rnt_Agrs r on r.rnt_agr_nbr = i.rea_rnt_agr_nbr
inner join rfs_rv.dvr_rras d on r.rnt_agr_nbr = d.rdy_rnt_agr_nbr and d.main_dvr_flg = 'MR'
inner join rfs_rv.bus_acnts ba on ba.acct_id = i.bac_acc_id
inner join (select sum(chg_amt) amt, rain.inv_id
from rfs.rra_chgs rrc
inner join rfs.rnt_agr_inv_note_lns rainl on rrc.rrc_id=rainl.rrc_rrc_id
inner join rfs.rnt_agr_inv_notes rain on rainl.rai_inv_id=rain.inv_id
inner join rfs.rnt_agrs ra on rain.rea_rnt_agr_nbr=ra.rnt_agr_nbr
inner join rfs.rra_chg_typs rct on rrc.rct_chg_typ=rct.chg_typ
where rrc.rct_chg_typ not in ('TAX', 'SUR', 'VAT') and not ( ra.stn_system='ECR' and (rrc.rct_chg_typ ='02000' or rrc.rct_chg_typ ='02201'))
group by rain.inv_id) ht on ht.inv_id=i.inv_id
inner join (select sum(chg_amt) amt, rain.inv_id
from rfs.rra_chgs rrc
inner join rfs.rnt_agr_inv_note_lns rainl on rrc.rrc_id=rainl.rrc_rrc_id
inner join rfs.rnt_agr_inv_notes rain on rainl.rai_inv_id=rain.inv_id
inner join rfs.rnt_agrs ra on rain.rea_rnt_agr_nbr=ra.rnt_agr_nbr
inner join rfs.rra_chg_typs rct on rrc.rct_chg_typ=rct.chg_typ and not ( ra.stn_system='ECR' and rrc.rct_chg_typ ='02200')
where rrc.rct_chg_typ in ('TAX', 'SUR', 'VAT')
group by rain.inv_id) vat on vat.inv_id=i.inv_id
where
i.inv_id in (425001975206,550008226812,425002005105, 425002046396, 42500190929)
I commented out the last line above and replaced it with this code; which runs well by itself.
i.inv_id in (select
q.inv_id
from
rfs.rnt_agr_inv_notes q,
rfs_rv.bus_acnts ba
where
ba.acct_id = q.bac_acc_id
and ba.bill_acct_nbr IN ('16785616')
AND extract(MONTH from q.prnt_tmsp) = 5
AND extract(YEAR from q.prnt_tmsp) = 2015)
I can give you a solution for combining the two which has worked well for our product. It looks horrible but had a vastly huge performance improvement by generating the list of Ids up front in a temp table.
(As an aside, you really need to take a look at the huge amount of INNER JOIN in your query as this is not helping your performance).
You'd want something like this:
CREATE TABLE #invIds (id INT)
INSERT INTO #invIds (id) SELECT
q.inv_id
from
rfs.rnt_agr_inv_notes q,
rfs_rv.bus_acnts ba
where
ba.acct_id = q.bac_acc_id
and ba.bill_acct_nbr IN ('16785616')
AND extract(MONTH from q.prnt_tmsp) = 5
AND extract(YEAR from q.prnt_tmsp) = 2015
Then the query would do this
d.dvr_frst_name,
d.dvr_srnm,
r.ecr_ticket_no,
r.co_tmsp,
i.dr_note_amt ,
ht.amt HT,
vat.amt VAT
from rfs.rnt_agr_inv_notes I .....
.....where
i.inv_id in #invIds
I have 3 tables:
CP_carthead (idOrder)
CP_cartrows (idOrder, idCartRow)
CP_shipping (idCartRow, idShipping, dateShipped)
There can be multiple idCartRows per idOrder.
I want to get all orders where all its idCartRows exist in CP_shipping. This seems like it should be simple, but I haven't found much on the web.
Here's my query now:
SELECT
s.idOrder
, s.LatestDateShipped
FROM
CP_carthead o
LEFT OUTER JOIN (
SELECT
MAX(s.dateShipped) [LatestDateShipped]
, r.idOrder
FROM
CP_shipping s
LEFT OUTER JOIN CP_cartrows r ON s.idCartRow = r.idCartRow
GROUP BY
r.idOrder
) s ON o.idOrder = s.idOrder
Your query is returning rows from "s" and not the orders. Based on your question, I came up with this query:
select o.*
from CP_Carthead o
where o.orderId in (select cr.idOrder
from cp_cartrows cr left outer join
cp_shipping s
on cr.idCartRow = s.IdCartrow
group by cr.idOrder
having count(s.idCartRow) = COUNT(*)
)
The subquery in the in statement is getting orders all of whose cartrows are in shipping.
I have this strange error in SQL Server 2005 where I take a working query, add the UNION keyword below it and then copy the query again. In my opinion, this should always be working, but it is not. I get the message 'Incorrect syntax near the keyword 'union'.
What could create this problem ?
To be more specific, here is the complete query :
select distinct deliveries.id, orders.id, 20 + sum(orders.mass1) as allowed_duration
from features_resources
inner join features on features.id = featureid
inner join orders on orders.id = features_resources.resourceid
inner join orderinformations on orders.id = orderinformations.orderid
inner join deliveries on orderinformations.deliveryid = deliveries.id
where features.name = 'O_FRAIS'
and (deliveries.ID IN
(SELECT ID
FROM dbo.DeliveriesInExportedSchedule))
group by deliveries.id, features.name ,orders.id order by deliveries.id
union
select distinct deliveries.id, orders.id, 20 + sum(orders.mass1) as allowed_duration
from features_resources
inner join features on features.id = featureid
inner join orders on orders.id = features_resources.resourceid
inner join orderinformations on orders.id = orderinformations.orderid
inner join deliveries on orderinformations.deliveryid = deliveries.id
where features.name = 'O_FRAIS'
and (deliveries.ID IN
(SELECT ID
FROM dbo.DeliveriesInExportedSchedule))
group by deliveries.id, features.name ,orders.id order by deliveries.id
I have tried to reproduce the error on a smaller query, by starting from a simple query and adding features one by one (inner join, nested queryes, group by, sum,....) but failed to reproduce the error again.
Any idea ?
It is actually the order by deliveries.id in the top half that causes the problem.
The order by needs to apply to the whole query.
Example Syntax
SELECT v1.number
FROM master.dbo.spt_values v1
WHERE v1.number > 2000
UNION
SELECT v2.number
FROM master.dbo.spt_values v2
WHERE v2.number < 10
ORDER BY v1.number
Try putting the individual SELECTs in parentheses:
(SELECT ... )
UNION
(SELECT ... )
The way you have it now, the second WHERE and GROUP BY clauses are ambiguous - should that apply to the SELECT, or to the UNION? I don't have any way to tell, and neither has your DB server.