Related
I have this MSSQL Query which works
SELECT c.CategoryName + ' (' + cast(count(ic.CategoryId) as varchar(255)) + ')' AS
CategoryName, count(ic.CategoryId) AS NumPhotos,
c.Slug, c.ParentCategoryId, c.Id
FROM Categories
c LEFT JOIN
ItemCategories ic
on ic.CategoryId = c.Id
GROUP BY c.CategoryName, c.slug, c.ParentCategoryId, c.id
ORDER BY ParentCategoryId DESC
And I want to return only rows, WHERE NumPhotos>0 but temporary columns are not allowed on SQL WHERE clause
The having clause is the canonical solution to your problem. However, I suspect that an inner join would also be appropriate:
SELECT c.CategoryName + ' (' + cast(count(ic.CategoryId) as varchar(255)) + ')' AS CategoryName,
count(ic.CategoryId) AS NumPhotos,
c.Slug, c.ParentCategoryId, c.Id
FROM Categories c INNER JOIN
ItemCategories ic
on ic.CategoryId = c.Id
GROUP BY c.CategoryName, c.slug, c.ParentCategoryId, c.id
ORDER BY ParentCategoryId DESC;
Without sample data, it is hard to be sure, but I'm pretty sure this does the same thing.
The advantage of an inner join is that it might be more efficient, because less (maybe only slightly less) data would be processed and the optimizer has more opportunities to pick the best join algorithm.
Existing comments have given adequate answers, but here is another solution, using a "virtual table":
SELECT * FROM (
SELECT c.CategoryName + ' (' + cast(count(ic.CategoryId) as varchar(255)) + ')' AS
CategoryName, count(ic.CategoryId) AS NumPhotos,
c.Slug, c.ParentCategoryId, c.Id
FROM Categories
c LEFT JOIN
ItemCategories ic
on ic.CategoryId = c.Id
GROUP BY c.CategoryName, c.slug, c.ParentCategoryId, c.id
)
WHERE NumPhotos>0
ORDER BY ParentCategoryId DESC
This query takes dynamic input in the place of cg.ownerid IN (294777,228649 ,188464).when the input increases in the IN condition the query is taking too much time to execute. Please suggest me a way to optimize it.
For example, the below query is taking 4 seconds, if I reduce the list to just IN(188464) its just taking 1 second.
SELECT *
FROM
(SELECT *,
Row_number() OVER(
ORDER BY datecreated DESC) AS rownum
FROM
(SELECT DISTINCT c.itemid,
(CASE WHEN (Isnull(c.password, '') <> '') THEN 1 ELSE 0 END) AS password,
c.title,
c.encoderid,
c.type,
(CASE WHEN c.author = 'education' THEN 'Discovery' ELSE c.type END) AS TYPE,
c.publisher,
c.description,
c.author,
c.duration,
c.copyright,
c.rating,
c.userid,
Stuff(
(SELECT DISTINCT ' ' + NAME AS [text()]
FROM firsttable SUB
LEFT JOIN secondtable AS rgc ON thirdtable = rgc.id
WHERE SUB.itemid = c.itemid
FOR xml path('')), 1, 1, '')AS [Sub_Categories]
FROM fourthtable AS cg
LEFT JOIN item AS c ON c.itemid = cg.itemid
WHERE Isnull(title, '') <> ''
AND c.active = '1'
AND c.systemid = '20'
AND cg.ownerid IN (294777,
228649,
188464)) AS a) AS b
WHERE rownum BETWEEN 1 AND 32
ORDER BY datecreated DESC
As I haven't further information, I just would suggest a first change of your where clause. They should be moved to a subquery as you left join those columns.
SELECT *
FROM(
SELECT *,
Row_number() OVER(
ORDER BY datecreated DESC) AS rownum
FROM
(SELECT DISTINCT c.itemid,
(CASE WHEN (Isnull(c.password, '') <> '') THEN 1 ELSE 0 END) AS password,
c.title,
c.encoderid,
c.type,
(CASE WHEN c.author = 'education' THEN 'Discovery' ELSE c.type END) AS TYPE,
c.publisher,
c.description,
c.author,
c.duration,
c.copyright,
c.rating,
c.userid,
Stuff(
(
SELECT DISTINCT ' ' + NAME AS [text()]
FROM firsttable SUB
LEFT JOIN secondtable AS rgc ON thirdtable = rgc.id
WHERE SUB.itemid = c.itemid
FOR xml path('')
), 1, 1, ''
) AS [Sub_Categories]
FROM (
SELECT cg.itemid
FROM fourthtable as cg
WHERE cg.ownerid IN (294777,228649, 188464)
) AS cg
LEFT JOIN (
SELECT DISTINCT c.itemid, c.type, c.author, c.title, c.encoderid, c.type, c.publisher, c.description, c.author, c.duration, c.copyright, c.rating,c.userid
FROM item as c
WHERE Isnull(c.title, '') <> ''
AND c.active = '1'
AND c.systemid = '20'
) AS c
ON c.itemid = cg.itemid
) AS a
) AS b
WHERE rownum BETWEEN 1 AND 32
ORDER BY datecreated DESC
But not quite sure if everything is connected right away, your missing some aliases which makes it hard for me to get through your query. But I thing you'll get my idea. :-)
With this little information it's impossible to give any specific ideas, but the normal general things apply:
Turn on statistics io and check what's causing most of the logical I/O and try to solve that
Look at the actual plan and check if there's something that doesn't look ok, for example:
Clustered index / table scans (new index could solve this)
Key lookups with a huge amount of rows (adding more columns to index could solve this, either as normal or included fields)
Spools (new index could solve this)
Big difference between estimated and actual number of rows (10x, 100x and so on)
To give any better hints you should really include the actual plan, table / index structure at least on the essential parts and tell what is too much time (seconds, minutes, hours?)
select a.Hall, a.Title,
STUFF((SELECT ', ' + '[' + CONVERT(varchar(2),DATEPART(Hour, b.StartFilm))
+ ':' + CONVERT(varchar(2),DATEPART(Minute, b.StartFilm))
+ ' ' + CONVERT(varchar(2),DATEPART(Hour, b.EndTime))
+ ':' + CONVERT(varchar(2),DATEPART(Minute, b.EndTime))
+ ']'
FROM (select c.Name as Hall, b.Title,
Convert(time,a.StartFilmTime) as StartFilm,
Convert(time,a.EndFilmTime) as EndTime
from FilmSchedule a
left join Film b on a.FilmId = b.Id
left join Room c on a.RoomId = c.Id
where a.ApproveStatus = 1 and a.Status = 1
and CONVERT(date, a.StartFilmTime) = '05-06-2015'
) b
Where a.Hall = b.Hall and a.Title = b.Title
FOR XML PATH('')),1,1,'') As ShowTime
from (select c.Name as Hall, b.Title,
Convert(time,a.StartFilmTime) as StartFilm,
Convert(time,a.EndFilmTime) as EndTime
from FilmSchedule a
left join Film b on a.FilmId = b.Id
left join Room c on a.RoomId = c.Id
where a.ApproveStatus = 1 and a.Status = 1
and CONVERT(date, a.StartFilmTime) = '05-06-2015'
Order by a.StartFilmTime
) a
group by a.Hall, a.Title
I get the error:
The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP or FOR XML is also specified.
Help please! (I have used FOR XML?)
Although your query does use FOR XML (for the GROUP_CONCAT workaround), you are applying the order by outside of the derived table that uses FOR XML, hence the error.
Given that you aren't including start date directly in the final select (although you are composing it as part of the STUFF ShowTime column), you also can't ORDER BY StartFilm in the final GROUP BY either, as the column would otherwise need to be included in the GROUP BY or as an aggregated column.
What you can do is move the ORDER BY into the STUFF and then order by the derived column ShowTime (since your query only runs for one given day, and StartFilmTime is the first part of the STUFFED composed column).
At the same time, I would DRY up the repetition on the derived table with a CTE:
WITH cteFiltered AS
(select c.Name as Hall, b.Title,
Convert(time,a.StartFilmTime) as StartFilm,
Convert(time,a.EndFilmTime) as EndTime
from FilmSchedule a
left join Film b on a.FilmId = b.Id
left join Room c on a.RoomId = c.Id
where a.ApproveStatus = 1 and a.Status = 1
and CONVERT(date, a.StartFilmTime) = '05-06-2015'
)
select
a.Hall,
a.Title,
STUFF((SELECT ', ' + '[' + CONVERT(varchar(2),DATEPART(Hour, b.StartFilm))
+ ':' + CONVERT(varchar(2),DATEPART(Minute, b.StartFilm))
+ ' ' + CONVERT(varchar(2),DATEPART(Hour, b.EndTime))
+ ':' + CONVERT(varchar(2),DATEPART(Minute, b.EndTime))
+ ']'
FROM
cteFiltered b
Where
a.Hall = b.Hall and a.Title = b.Title
order by b.StartFilm -- ***
FOR XML PATH('')),1,1,'') As ShowTime
from
cteFiltered a
group by a.Hall, a.Title
order by ShowTime; -- *** Hour is first (assuming 24H format) and only one day
I'm trying to add inner joins to old SQL code to make it run more efficiently. But when I added them and tried to execute I get this error:
1016, Line 12 Outer join operators cannot be specified in a query containing joined tables
Here's the query:
select a.s_purchase_order as order_id,
a.order_type,
a.nobackorder,
a.order_note,
a.note,
a.rqst_dlvry_date,
b.customer_name ,
c.store_name,
(c.store_name + ',' + isnull(c.address1 + ',', ' ') + isnull(c.city + ',', ' ') + isnull(c.state_cd+ ',', ' ') + isnull( c.zipcode, ' ')) as store_info,
d.supplier_account
from VW_CustomerOrder a, Customer b, Store c, eligible_supplier d
where a.customer = c.customer
and a.store = c.store
and a.customer = b.customer
and c.customer *= d.customer
and c.store *= d.store
and a.supplier *= d.supplier
and a.purchase_order = #order_id
and a.customer = #customer_id
and a.store=#store_id
and a.supplier = #supplier_id
Any idea what's causing it? I'm guessing it has something to do with the isnull?
Did you try this? It replaces your commas between your tables with INNER JOIN and LEFT JOIN
select a.s_purchase_order as order_id,
a.order_type,
a.nobackorder,
a.order_note,
a.note,
a.rqst_dlvry_date,
b.customer_name ,
c.store_name,
(c.store_name + ',' + isnull(c.address1 + ',', ' ') + isnull(c.city + ',', ' ') + isnull(c.state_cd+ ',', ' ') + isnull( c.zipcode, ' ')) as store_info,
d.supplier_account
from VW_CustomerOrder a
INNER JOIN Customer b
ON a.customer = b.customer
INNER JOIN Store c
ON a.customer = c.customer
and a.store = c.store
LEFT JOIN eligible_supplier d
ON c.customer = d.customer
and c.store = d.store
and a.supplier = d.supplier
where a.purchase_order = #order_id
and a.customer = #customer_id
and a.store=#store_id
and a.supplier = #supplier_id
If you left your "*=" join operators in the code after you converted it to ANSI syntax, that would explain your error. Use = for all equality tests when using ANSI syntax -- the type of your JOIN should be explicit in the JOIN declaration itself (INNER, LEFT, RIGHT, etc.)
I've been working with databases for a long time but I'm new to query optimization. I have the following query (some of it code-generated):
SELECT DISTINCT COALESCE(gi.start_time, '') start_time,
COALESCE(b.name, '') bank,
COALESCE(a.id, '') account_id,
COALESCE(a.account_number, '') account_number,
COALESCE(at.code, '') account_type,
COALESCE(a.open_date, '') open_date,
COALESCE(a.interest_rate, '') interest_rate,
COALESCE(a.maturity_date, '') maturity_date,
COALESCE(a.opening_balance, '') opening_balance,
COALESCE(a.has_e_statement, '') has_e_statement,
COALESCE(a.has_bill_pay, '') has_bill_pay,
COALESCE(a.has_overdraft_protection, '') has_overdraft_protection,
COALESCE(a.balance, '') balance,
COALESCE(a.business_or_personal, '') business_or_personal,
COALESCE(a.cumulative_balance, '') cumulative_balance,
COALESCE(c.customer_number, '') customer_number,
COALESCE(c.social_security_number, '') social_security_number,
COALESCE(c.name, '') customer_name,
COALESCE(c.phone, '') phone,
COALESCE(c.deceased, '') deceased,
COALESCE(c.do_not_mail, '') do_not_mail,
COALESCE(cdob.date_of_birth, '') date_of_birth,
COALESCE(ad.line1, '') line1,
COALESCE(ad.line2, '') line2,
COALESCE(ad.city, '') city,
COALESCE(s.name, '') state,
COALESCE(ad.zip, '') zip,
COALESCE(o.officer_number, '') officer_number,
COALESCE(o.name, '') officer_name,
COALESCE(po.line1, '') po_box,
COALESCE(po.city, '') po_city,
COALESCE(po_state.name, '') po_state,
COALESCE(po.zip, '') zip,
COALESCE(br.number, '') branch_number,
COALESCE(cd_type.code, '') cd_type,
COALESCE(mp.product_number, '') macatawa_product_number,
COALESCE(mp.product_name, '') macatawa_product_name,
COALESCE(pt.name, '') macatawa_product_type,
COALESCE(hhsc.name, '') harte_hanks_service_category,
COALESCE(mp.hoh_hierarchy, '') hoh_hierarchy,
COALESCE(cft.name, '') core_file_type,
COALESCE(oa.line1, '') original_address_line1,
COALESCE(oa.line2, '') original_address_line2,
COALESCE(uc.code, '') use_class
FROM account a
JOIN customer c ON a.customer_id = c.id
JOIN officer o ON a.officer_id = o.id
JOIN account_address aa ON aa.account_id = a.id
LEFT JOIN account_po_box apb ON apb.account_id = a.id
JOIN address ad ON aa.address_id = ad.id
JOIN original_address oa ON oa.address_id = ad.id
LEFT JOIN address po ON apb.address_id = po.id
JOIN state s ON s.id = ad.state_id
LEFT JOIN state po_state ON po_state.id = po.state_id
LEFT JOIN branch br ON a.branch_id = br.id
JOIN account_import ai ON a.account_import_id = ai.id
JOIN generic_import gi ON gi.id = ai.generic_import_id
JOIN import_bundle ib ON gi.import_bundle_id = ib.id
JOIN bank b ON b.id = ib.bank_id
LEFT JOIN customer_date_of_birth cdob ON cdob.customer_id = c.id
LEFT JOIN cd_type ON a.cd_type_id = cd_type.id
LEFT JOIN account_macatawa_product amp ON amp.account_id = a.id
LEFT JOIN macatawa_product mp ON mp.id = amp.macatawa_product_id
LEFT JOIN product_type pt ON pt.id = mp.product_type_id
LEFT JOIN harte_hanks_service_category hhsc
ON hhsc.id = mp.harte_hanks_service_category_id
LEFT JOIN core_file_type cft ON cft.id = mp.core_file_type_id
LEFT JOIN use_class uc ON a.use_class_id = uc.id
LEFT JOIN account_type at ON a.account_type_id = at.id
WHERE 1
AND gi.active = 1
AND b.id = 8 AND ib.is_finished = 1
ORDER BY a.id
LIMIT 10
And it's pretty slow. On my dev server it takes about a minute to run and on my production server, where there's more data, I can't get it to even finish. Here's what an EXPLAIN looks like:
http://i.stack.imgur.com/eR6lq.png
I know the basics of EXPLAIN. I know that it's good that I have something other than NULL for everything under key. But I don't know, overall, how much room for improvement my query has. I do know that Using temporary; Using filesort under Extra is bad, but I have no idea what to do about it.
It looks like you don't have indexes on most of your JOIN fields. Make sure every field that you use as a JOIN key has an index on both tables.
With 23 joins and what looks like only 2 relevant indexes, poor performance can be expected.
With no index to reference, the query engine is checking every row in both tables to compare them, which is obviously very inefficient.
edit:
For example, in your query you have
JOIN customer c ON a.customer_id = c.id
Make sure you have an index on a.customer_id AND customer.id. Having an index on both tables (on the JOINed fields) will exponentially speed up the query.
In addition to what #JNK mentioned in his answer about ensuring you have indexes, I have restructured your query and added the "STRAIGHT_JOIN" clause at the top which tells the optimizer to do the query in the order the tables are presented to it.
Since your query is based on the generic import, to import bundle to bank, I've moved THOSE to the front of the list... The where will pre-qualify THOSE records first instead of looking at all accounts that may never be part of the result. So, the join is now reversed from the generic import back to the account following the same relationships you started with.
I've also associated the respective JOIN / ON conditions directly under the table they were joining against for readability and following table relationships. I've also made it so the ON clause has Table1.ID = JoinedTable.ID... although some reversed and otherwise no big deal, knowing how something is based on the join INTO the other just allows easier readability.
So, ensure respective tables have indexes on whatever key column is the join, and from this sample query, make sure your GI table (alias) has an index on "Active", and your IB (alias) has an index on Is_Finished.
Lastly, your WHERE clause had WHERE 1 AND... no purpose of the "1", so I stripped that out.
SELECT STRAIGHT_JOIN DISTINCT
COALESCE(gi.start_time, '') start_time,
COALESCE(b.name, '') bank,
COALESCE(a.id, '') account_id,
COALESCE(a.account_number, '') account_number,
COALESCE(at.code, '') account_type,
COALESCE(a.open_date, '') open_date,
COALESCE(a.interest_rate, '') interest_rate,
COALESCE(a.maturity_date, '') maturity_date,
COALESCE(a.opening_balance, '') opening_balance,
COALESCE(a.has_e_statement, '') has_e_statement,
COALESCE(a.has_bill_pay, '') has_bill_pay,
COALESCE(a.has_overdraft_protection, '') has_overdraft_protection,
COALESCE(a.balance, '') balance,
COALESCE(a.business_or_personal, '') business_or_personal,
COALESCE(a.cumulative_balance, '') cumulative_balance,
COALESCE(c.customer_number, '') customer_number,
COALESCE(c.social_security_number, '') social_security_number,
COALESCE(c.name, '') customer_name,
COALESCE(c.phone, '') phone,
COALESCE(c.deceased, '') deceased,
COALESCE(c.do_not_mail, '') do_not_mail,
COALESCE(cdob.date_of_birth, '') date_of_birth,
COALESCE(ad.line1, '') line1,
COALESCE(ad.line2, '') line2,
COALESCE(ad.city, '') city,
COALESCE(s.name, '') state,
COALESCE(ad.zip, '') zip,
COALESCE(o.officer_number, '') officer_number,
COALESCE(o.name, '') officer_name,
COALESCE(po.line1, '') po_box,
COALESCE(po.city, '') po_city,
COALESCE(po_state.name, '') po_state,
COALESCE(po.zip, '') zip,
COALESCE(br.number, '') branch_number,
COALESCE(cd_type.code, '') cd_type,
COALESCE(mp.product_number, '') macatawa_product_number,
COALESCE(mp.product_name, '') macatawa_product_name,
COALESCE(pt.name, '') macatawa_product_type,
COALESCE(hhsc.name, '') harte_hanks_service_category,
COALESCE(mp.hoh_hierarchy, '') hoh_hierarchy,
COALESCE(cft.name, '') core_file_type,
COALESCE(oa.line1, '') original_address_line1,
COALESCE(oa.line2, '') original_address_line2,
COALESCE(uc.code, '') use_class
FROM
generic_import gi
JOIN import_bundle ib
ON gi.import_bundle_id = ib.id
JOIN bank b
ON ib.bank_id = b.id
JOIN account_import ai
ON gi.id = ai.generic_import_id
JOIN account a
ON ai.id = a.account_import_id
JOIN customer c
ON a.customer_id = c.id
LEFT JOIN customer_date_of_birth cdob
ON c.id = cdob.customer_id
JOIN officer o
ON a.officer_id = o.id
LEFT JOIN branch br
ON a.branch_id = br.id
LEFT JOIN cd_type
ON a.cd_type_id = cd_type.id
LEFT JOIN account_macatawa_product amp
ON a.id = amp.account_id
LEFT JOIN macatawa_product mp
ON amp.macatawa_product_id = mp.id
LEFT JOIN product_type pt
ON mp.product_type_id = pt.id
LEFT JOIN harte_hanks_service_category hhsc
ON mp.harte_hanks_service_category_id = hhsc.id
LEFT JOIN core_file_type cft
ON mp.core_file_type_id = cft.id
LEFT JOIN use_class uc
ON a.use_class_id = uc.id
LEFT JOIN account_type at
ON a.account_type_id = at.id
JOIN account_address aa
ON a.id = aa.account_id
JOIN address ad
ON aa.address_id = ad.id
JOIN original_address oa
ON ad.id = oa.address_id
JOIN state s
ON ad.state_id = s.id
LEFT JOIN account_po_box apb
ON a.id = apb.account_id
LEFT JOIN address po
ON apb.address_id = po.id
LEFT JOIN state po_state
ON po.state_id = po_state.id
WHERE
gi.active = 1
AND ib.is_finished = 1
AND b.id = 8
ORDER BY
a.id
LIMIT
10