Need help optimizing this tSQL Query - sql

I'm definitely not a DBA and unfortunately we don't have a DBA to consult within at our company. I was wondering if someone could give me a recommendation on how to improve this query, either by changing the query itself or adding indexes to the database.
Looking at the execution plan of the query it seems like the outer joins are killing the query. This query only returns 350k results, but it takes almost 30 seconds to complete. I don't know much about DB's, but I don't think this is good? Perhaps I'm wrong?
Any suggestions would be greatly appreciated. Thanks in advance.
As a side note this is obviously being create by an ORM and not me directly. We are using Linq-to-SQL.
SELECT
[t12].[value] AS [DiscoveryEnabled],
[t12].[value2] AS [isConnected],
[t12].[Interface],
[t12].[Description] AS [InterfaceDescription],
[t12].[value3] AS [Duplex],
[t12].[value4] AS [IsEnabled],
[t12].[value5] AS [Host],
[t12].[value6] AS [HostIP],
[t12].[value7] AS [MAC],
[t12].[value8] AS [MACadded],
[t12].[value9] AS [PortFast],
[t12].[value10] AS [PortSecurity],
[t12].[value11] AS [ShortHost],
[t12].[value12] AS [SNMPlink],
[t12].[value13] AS [Speed],
[t12].[value14] AS [InterfaceStatus],
[t12].[InterfaceType],
[t12].[value15] AS [IsUserPort],
[t12].[value16] AS [VLAN],
[t12].[value17] AS [Code],
[t12].[Description2] AS [Description],
[t12].[Host] AS [DeviceName],
[t12].[NET_OUID],
[t12].[DisplayName] AS [Net_OU],
[t12].[Enclave]
FROM (
SELECT
[t1].[DiscoveryEnabled] AS [value],
[t1].[IsConnected] AS [value2],
[t0].[Interface],
[t0].[Description],
[t2].[Duplex] AS [value3],
[t0].[IsEnabled] AS [value4],
[t3].[Host] AS [value5],
[t6].[Address] AS [value6],
[t3].[MAC] AS [value7],
[t3].[MACadded] AS [value8],
[t2].[PortFast] AS [value9],
[t2].[PortSecurity] AS [value10],
[t4].[Host] AS [value11],
[t0].[SNMPlink] AS [value12],
[t2].[Speed] AS [value13],
[t2].[InterfaceStatus] AS [value14],
[t8].[InterfaceType],
[t0].[IsUserPort] AS [value15],
[t2].[VLAN] AS [value16],
[t9].[Code] AS [value17],
[t9].[Description] AS [Description2],
[t7].[Host], [t7].[NET_OUID],
[t10].[DisplayName],
[t11].[Enclave],
[t7].[Decommissioned]
FROM [dbo].[IDB_Interface] AS [t0]
LEFT OUTER JOIN [dbo].[IDB_InterfaceLayer2] AS [t1] ON [t0].[IDB_Interface_ID] = [t1].[IDB_Interface_ID]
LEFT OUTER JOIN [dbo].[IDB_LANinterface] AS [t2] ON [t1].[IDB_InterfaceLayer2_ID] = [t2].[IDB_InterfaceLayer2_ID]
LEFT OUTER JOIN [dbo].[IDB_Host] AS [t3] ON [t2].[IDB_LANinterface_ID] = [t3].[IDB_LANinterface_ID]
LEFT OUTER JOIN [dbo].[IDB_Infrastructure] AS [t4] ON [t0].[IDB_Interface_ID] = [t4].[IDB_Interface_ID]
LEFT OUTER JOIN [dbo].[IDB_AddressMapIPv4] AS [t5] ON [t3].[IDB_AddressMapIPv4_ID] = ([t5].[IDB_AddressMapIPv4_ID])
LEFT OUTER JOIN [dbo].[IDB_AddressIPv4] AS [t6] ON [t5].[IDB_AddressIPv4_ID] = [t6].[IDB_AddressIPv4_ID]
INNER JOIN [dbo].[ART_Asset] AS [t7] ON [t7].[ART_Asset_ID] = [t0].[ART_Asset_ID]
LEFT OUTER JOIN [dbo].[NSD_InterfaceType] AS [t8] ON [t8].[NSD_InterfaceTypeID] = [t0].[NSD_InterfaceTypeID]
INNER JOIN [dbo].[NSD_InterfaceCode] AS [t9] ON [t9].[NSD_InterfaceCodeID] = [t0].[NSD_InterfaceCodeID]
INNER JOIN [dbo].[NET_OU] AS [t10] ON [t10].[NET_OUID] = [t7].[NET_OUID]
INNER JOIN [dbo].[NET_Enclave] AS [t11] ON [t11].[NET_EnclaveID] = [t10].[NET_EnclaveID]
) AS [t12]
WHERE ([t12].[Enclave] = 'USMC') AND (NOT ([t12].[Decommissioned] = 1))
LINQ-TO-SQL Query:
return from t in db.IDB_Interfaces
join v in db.IDB_InterfaceLayer3s on t.IDB_Interface_ID equals v.IDB_Interface_ID
join u in db.ART_Assets on t.ART_Asset_ID equals u.ART_Asset_ID
join c in db.NET_OUs on u.NET_OUID equals c.NET_OUID
join w in
(from d in db.IDB_InterfaceIPv4s
select new { d.IDB_InterfaceIPv4_ID, d.IDB_InterfaceLayer3_ID, d.IDB_AddressMapIPv4_ID, d.IDB_AddressMapIPv4.IDB_AddressIPv4.Address })
on v.IDB_InterfaceLayer3_ID equals w.IDB_InterfaceLayer3_ID
join h in db.NET_Enclaves on c.NET_EnclaveID equals h.NET_EnclaveID into enclaveLeftJoin
from i in enclaveLeftJoin.DefaultIfEmpty()
join m in
(from z in db.IDB_StandbyIPv4s
select new
{
z.IDB_InterfaceIPv4_ID,
z.IDB_AddressMapIPv4_ID,
z.IDB_AddressMapIPv4.IDB_AddressIPv4.Address,
z.Preempt,
z.Priority
})
on w.IDB_InterfaceIPv4_ID equals m.IDB_InterfaceIPv4_ID into standbyLeftJoin
from k in standbyLeftJoin.DefaultIfEmpty()
where t.ART_Asset.Decommissioned == false
select new NetIDBGridDataResults
{
DeviceName = u.Host,
Host = u.Host,
Interface = t.Interface,
IPAddress = w.Address,
ACLIn = v.InboundACL,
ACLOut = v.OutboundACL,
VirtualAddress = k.Address,
VirtualPriority = k.Priority,
VirtualPreempt = k.Preempt,
InterfaceDescription = t.Description,
Enclave = i.Enclave
};

As a rule (and this is very general), you want an index on:
JOIN fields (both sides)
Common WHERE filter fields
Possibly fields you aggregate
For this query, start with checking your JOIN criteria. Any one of those missing will force a table scan which is a big hit.

Looking at the execution plan of the query it seems like the outer joins are killing the query.
This query only returns 350k results, but it takes almost 30 seconds to complete. I don't know
much about DB's, but I don't think this is good? Perhaps I'm wrong?
A man has got to do waht a mana has got to do.
The joins may kill you, but when you need them YOU NEED THEM. Some tasks take long.
Make sure you ahve all indices you need.
Make sure your sql server is not a sad joke hardware wise.
All you can do.
I woudl bet someone has no clue about SQL and needs to be enlighted to the power of indices.

Related

script optimization in sql

I am running simple select script, which inner join with other 3 table . all the tables are big ( lots of data ) its taking around 20 sec to run. want to optimized it.
I tried to used nolock , but not much deference
SELECT RR.ReportID,
RR.RequestFormat,
RRP.SequenceNumber,
RRP.ParameterName,
RRP.ParameterValue
CASE WHEN RP.ParameterLabelOvrrd IS NULL THEN P.ParameterLabel ELSE .ParameterLabelOvrrd END AS ParameterLabelChosen,
RRP.ParameterValueEntered
FROM ReportRequestParameters AS RRP WITH (NOLOCK)
INNER JOIN ReportRequests AS RR WITH (NOLOCK) ON RRP.RequestID = RR.RequestID
INNER JOIN ReportParameter AS RP WITH (NOLOCK) ON RP.ReportID = RR.ReportID
AND RP.SequenceNumber = RRP.SequenceNumber
INNER JOIN Parameter AS P WITH (NOLOCK) ON P.ParameterID = RP.ParameterID
WHERE RRP.RequestID = '2226765'
ORDER BY SequenceNumber;
Please advice.
This is your query:
SELECT RR.ReportID, RR.RequestFormat, RRP.SequenceNumber,
RRP.ParameterName, RRP.ParameterValue
COALESCE(RP.ParameterLabelOvrrd, P.ParameterLabel) as ParameterLabelChosen,
RRP.ParameterValueEntered
FROM ReportRequestParameters RRP JOIN
ReportRequests RR
ON RRP.RequestID = RR.RequestID JOIN
ReportParameter RP
ON RP.ReportID = RR.ReportID AND
RP.SequenceNumber = RRP.SequenceNumber JOIN
Parameter P
ON P.ParameterID = RP.ParameterID
WHERE RRP.RequestID = 2226765
ORDER BY RRP.SequenceNumber;
I have removed the single quotes on 2226765, assuming that the id is a number. Mixing types can impede the optimizer.
Then, I recommend an index on ReportRequestParameters(RequestID, SequenceNumber). I assume the other tables have indexes on the appropriate columns, but these are:
ReportRequests(RequestID, ReportID, SequenceNumber)
ReportParameter(ReportID, SequenceNumber, ParameterID)
Parameter(ParameterID)
I strongly advise you not to use nolock, unless you know what you are doing. Aaron Bertrand has a good blog post on this subject.
I would suggest running with the execution plan turned on and see if SSMS can advise you on additional indexing.
Other than that your query looks straight-forward, nothing code wise that is going to help make it faster, other than perhaps getting rid of the case statement and definitely getting rid of the NOLOCK statements.

Best way to optimize this SQL Query

I have to optimize this query and I am really in a hurry here. The following query searches by client. The input value RIF.keyvaluechar
LIKE 'V%10553790 ' is because in some old registers in the database some IDs when missing characters it used to be V0012345678 but it should have been V12345678 as that's the maximum amount of characters the ID can have. I know 12345678 should have been numeric and the V a char and then compare, but that's another issue.
Anyway, the query is this one:
SELECT DISTINCT idata.itemnum AS [ID],
LTRIM(RTRIM(ISNULL(CONTRATO.keyvaluechar,'N/A'))) AS [Contrato],
idata.datestored AS [Fecha],
NUMERO.keyvaluesmall AS [Numero],
TIPO.keyvaluechar AS [Tipo],
LTRIM(RTRIM(ISNULL(LC.lifecyclename,'N/A'))) AS [Flujo],
LTRIM(RTRIM(ISNULL(LC.lcnum,-1))) AS [FlujoID],
LTRIM(RTRIM(ISNULL(LCS.statename,'N/A'))) AS [Cola],
LTRIM(RTRIM(ISNULL(LCS.statenum,-1))) AS [ColaID],
CASE
WHEN PC.NombreProceso IN('PTD','PV2','PV3') THEN 1
ELSE 0
END AS [Portada]
FROM OnBase.hsi.itemdata idata WITH (NOLOCK)
INNER JOIN OnBase.hsi.keyitem109 TIPO WITH (NOLOCK) ON TIPO.itemnum = idata.itemnum
INNER JOIN OnBase.hsi.keyitem113 NUMERO WITH (NOLOCK) ON NUMERO.itemnum = idata.itemnum
LEFT JOIN OnBase.hsi.keyitem132 CONTRATO WITH (NOLOCK) ON CONTRATO.itemnum = idata.itemnum
LEFT JOIN OnBase.hsi.keyitem114 CLIENTE WITH (NOLOCK) ON CLIENTE.itemnum = idata.itemnum
LEFT JOIN OnBase.hsi.keyitem111 RIF WITH (NOLOCK) ON RIF.itemnum = idata.itemnum
INNER JOIN OnBase.hsi.doctype DOC WITH (NOLOCK) ON DOC.itemtypenum = idata.itemtypenum
INNER JOIN BD_WorkFlow.dbo.BBVA_ProcesosConfig PC WITH (NOLOCK) ON PC.ID_Documento = idata.itemtypenum
LEFT JOIN Onbase.hsi.itemlc ILC WITH (NOLOCK) ON ILC.itemnum = idata.itemnum
LEFT JOIN Onbase.hsi.lcstate LCS WITH (NOLOCK) ON LCS.statenum = ILC.statenum
LEFT JOIN Onbase.hsi.lifecycle LC WITH (NOLOCK) ON LC.lcnum = ILC.lcnum
WHERE PC.NombreProceso <> 'XXX' AND
PC.NombreProceso NOT IN('PTD','PV2','PV3') AND
TIPO.keyvaluechar = 'CCD' AND
RIF.keyvaluechar LIKE 'V%10553790 '
As you can see it is this way so it finds V0012345678 or V12345678 but this is not the right way or I feel it is the best optimization, although I am no expert in databases.
Anyways, I've though about something like this instead of last line
AND LEFT ('RIF.Keyvaluechar, 1) ="V"
AND SUBSTRING (RIF.Keyvaluechar, 2, LEN(RIF.Keyvaluechar)) = "12345678"
What do you guys think? Is there any other better way to improve upon this?
First, your query has a logic problem. You have this:
LEFT JOIN OnBase.hsi.keyitem111 RIF WITH(NOLOCK) ON RIF.itemnum = idata.itemnum
and then this in your where clause:
AND RIF.keyvaluechar LIKE 'V%10553790 '
Putting that filter in your where clause effectively changes your left join to an inner join. To fix this, move the filter to the join.
In terms of optimizing it, I assume that means to make it run faster. What you were thinking about will probably slow things down because you are filtering on function results instead of fields. A better approach, no matter how much of a hurry you are in, is to look at the indexes in your database and try to filter on those. In fact, it might be appropriate to add new ones.
Is the Keyvaluechar always a number from the second character onwards and you want to treat it as a number (=remove leading zeros). You could try to add a persisted column convert(int, SUBSTRING (Keyvaluechar, 2, 10)) to the table, then index that, and use it as a search criteria. At least I would assume that should help a lot.
In addition to that, looking at statistics IO output might be a good idea too, to see what table is actually responsible for the biggest I/O amounts.
Just a note, I hope you also know the problems using NOLOCK can cause you.

Oracle Query Optimization for Multiple Joins

Following query is taking too long (30 seconds).
I think bottleneck is in where clause. Is it possible to optimize ?
SELECT
COUNT(*) AS IN_HELP
FROM
HCM.TableA
INNER JOIN
HCM.EMPLOYEE_POSITION_BOX EPB
ON
TableA.POSITION_BOX_CODE = EPB.POSITION_BOX_CODE AND NVL (V_DATE,SYSDATE) BETWEEN EPB.EFFECTIVE_DATE AND EPB.END_DATE
LEFT OUTER JOIN
HCM.NODE_LEVEL_MASTER NLM
ON
HCM.F_EMPLOYEE_ACTUAL_NODE(EPB.EMPLOYEE_CODE,TRUNC(SYSDATE)) = NLM.NODE_NO
LEFT JOIN
HCM.POSITION_STATUS_MASTER PSM
ON
TableA.STATUS = PSM.POSITION_STATUS_NO
WHERE
NLM.NODE_NO_LEVEL2 = V_LOOP.NODE_NO_LEVEL2
AND HCM.F_EMPLOYEE_ACTUAL_NODE(EPB.EMPLOYEE_CODE,TRUNC (SYSDATE)) <> TableA.NODE_NO
AND PSM.FLAG = 'Y'
AND TableA.STATUS <> '0108'
First of all, check the indexes for your tables from WHERE clause:
NLM.NODE_NO_LEVEL2
V_LOOP.NODE_NO_LEVEL2
TableA.NODE_NO
TableA.STATUS
PSM.FLAG shouldn't be added to the index as it will not provide some efficient filtering the data in tables.
Also, am I right that HCM.F_EMPLOYEE_ACTUAL_NODE is a function? If so, then you should consider to remove it from WHERE clause, as the execution plan won't examine the inside query, and it doesn't being optimized at all, which lead to performance issue.
The main problem is your function HCM.F_EMPLOYEE_ACTUAL_NODE. first move the function call to a subquery (epbx):
SELECT
COUNT(*) AS IN_HELP
FROM
HCM.TableA
INNER JOIN
(SELECT HCM.F_EMPLOYEE_ACTUAL_NODE(EPB.EMPLOYEE_CODE,TRUNC(SYSDATE)) node, epb.* from HCM.EMPLOYEE_POSITION_BOX EPB) epbx
ON
TableA.POSITION_BOX_CODE = EPBx.POSITION_BOX_CODE AND NVL (V_DATE,SYSDATE) BETWEEN EPBx.EFFECTIVE_DATE AND EPBx.END_DATE
LEFT OUTER JOIN
HCM.NODE_LEVEL_MASTER NLM
ON
epbx.node = NLM.NODE_NO
LEFT JOIN
HCM.POSITION_STATUS_MASTER PSM
ON
TableA.STATUS = PSM.POSITION_STATUS_NO
WHERE
NLM.NODE_NO_LEVEL2 = V_LOOP.NODE_NO_LEVEL2
AND epbx.node <> TableA.NODE_NO
AND PSM.FLAG = 'Y'
AND TableA.STATUS <> '0108'
Then use explain plan to find out whether your query is written efficiently.

Ignore null values in select statement

I'm trying to retrieve a list of components via my computer_system, BUT if a computer system's graphics card is set to null (I.e. It has an onboard), the row isn't returned by my select statement.
I've been trying to use COALESCE without results. I've also tried with and OR in my WHERE clause, which then just returns my computer system with all different kinds of graphic cards.
Relevant code:
SELECT
computer_system.cs_id,
computer_system.cs_name,
motherboard.name,
motherboard.price,
cpu.name,
cpu.price,
gfx.name,
gfx.price
FROM
public.computer_case ,
public.computer_system,
public.cpu,
public.gfx,
public.motherboard,
public.ram
WHERE
computer_system.cs_ram = ram.ram_id AND
computer_system.cs_cpu = cpu.cpu_id AND
computer_system.cs_mb = motherboard.mb_id AND
computer_system.cs_case = computer_case.case_id AND
computer_system.cs_gfx = gfx.gfx_id; <-- ( OR computer_system.cs_gfx IS NULL)
Returns:
1;"Computer1";"Fractal Design"; 721.00; "MSI Z87"; 982.00; "Core i7 I7-4770K "; 2147.00; "Crucial Gamer"; 1253.00; "ASUS GTX780";3328.00
Should I use Joins? Is there no easy way to say return the requested row, even if there's a bloody NULL value. Been struggling with this for at least 2 hours.
Tables will be posted if needed.
EDIT: It should return a second row:
2;"Computer2";"Fractal Design"; 721.00; "MSI Z87"; 982.00; "Core i7 I7-4770K "; 2147.00; "Crucial Gamer"; 1253.00; "null/nothing";null/nothing
You want a LEFT OUTER JOIN.
First, clean up your code so you use ANSI joins so it's readable:
SELECT
computer_system.cs_id,
computer_system.cs_name,
motherboard.name,
motherboard.price,
cpu.name,
cpu.price,
gfx.name,
gfx.price
FROM
public.computer_system
INNER JOIN public.computer_case ON computer_system.cs_case = computer_case.case_id
INNER JOIN public.cpu ON computer_system.cs_cpu = cpu.cpu_id
INNER JOIN public.gfx ON computer_system.cs_gfx = gfx.gfx_id
INNER JOIN public.motherboard ON computer_system.cs_mb = motherboard.mb_id
INNER JOIN public.ram ON computer_system.cs_ram = ram.ram_id;
Then change the INNER JOIN on public.gfx to a LEFT OUTER JOIN:
LEFT OUTER JOIN public.gfx ON computer_system.cs_gfx = gfx.gfx_id
See PostgreSQL tutorial - joins.
I very strongly recommend reading an introductory tutorial to SQL - at least the PostgreSQL tutorial, preferably some more material as well.
It looks like it's just a bracket placement issue. Pull the null check and the graphics card id comparison into a clause by itself.
...
computer_system.cs_case = computer_case.case_id AND
(computer_system.cs_gfx IS NULL OR computer_system.cs_gfx = gfx.gfx_id)
Additionally, you ask if you should use joins. You are in fact using joins, by virtue of having multiple tables in your FROM clause and specifying the join criteria in the WHERE clause. Changing this to use the JOIN ON syntax might be a little easier to read:
FROM sometable A
JOIN someothertable B
ON A.somefield = B.somefield
JOIN somethirdtable C
ON A.somefield = C.somefield
etc
Edit:
You also likely want to make the join where you expect the null value to be a left outer join:
SELECT * FROM
first_table a
LEFT OUTER JOIN second_table b
ON a.someValue = b.someValue
If there is no match in the join, the row from the left side will still be returned.

Optimize SQL query with many left join

I have a SQL query with many left joins
SELECT COUNT(DISTINCT po.o_id)
FROM T_PROPOSAL_INFO po
LEFT JOIN T_PLAN_TYPE tp ON tp.plan_type_id = po.Plan_Type_Fk
LEFT JOIN T_PRODUCT_TYPE pt ON pt.PRODUCT_TYPE_ID = po.cust_product_type_fk
LEFT JOIN T_PROPOSAL_TYPE prt ON prt.PROPTYPE_ID = po.proposal_type_fk
LEFT JOIN T_BUSINESS_SOURCE bs ON bs.BUSINESS_SOURCE_ID = po.CONT_AGT_BRK_CHANNEL_FK
LEFT JOIN T_USER ur ON ur.Id = po.user_id_fk
LEFT JOIN T_ROLES ro ON ur.roleid_fk = ro.Role_Id
LEFT JOIN T_UNDERWRITING_DECISION und ON und.O_Id = po.decision_id_fk
LEFT JOIN T_STATUS st ON st.STATUS_ID = po.piv_uw_status_fk
LEFT OUTER JOIN T_MEMBER_INFO mi ON mi.proposal_info_fk = po.O_ID
WHERE 1 = 1
AND po.CUST_APP_NO LIKE '%100010233976%'
AND 1 = 1
AND po.IS_STP <> 1
AND po.PIV_UW_STATUS_FK != 10
The performance seems to be not good and I would like to optimize the query.
Any suggestions please?
Try this one -
SELECT COUNT(DISTINCT po.o_id)
FROM T_PROPOSAL_INFO po
WHERE PO.CUST_APP_NO LIKE '%100010233976%'
AND PO.IS_STP <> 1
AND po.PIV_UW_STATUS_FK != 10
First, check your indexes. Are they old? Did they get fragmented? Do they need rebuilding?
Then, check your "execution plan" (varies depending on the SQL Engine): are all joins properly understood? Are some of them 'out of order'? Do some of them transfer too many data?
Then, check your plan and indexes: are all important columns covered? Are there any outstandingly lengthy table scans or joins? Are the columns in indexes IN ORDER with the query?
Then, revise your query:
- can you extract some parts that normally would quickly generate small rowset?
- can you add new columns to indexes so join/filter expressions will get covered?
- or reorder them so they match the query better?
And, supporting the solution from #Devart:
Can you eliminate some tables on the way? does the where touch the other tables at all? does the data in the other tables modify the count significantly? If neither SELECT nor WHERE never touches the other joined columns, and if the COUNT exact value is not that important (i.e. does that T_PROPOSAL_INFO exist?) then you might remove all the joins completely, as Devart suggested. LEFTJOINs never reduce the number of rows. They only copy/expand/multiply the rows.