Inadvertent cross join occurring in SQL merge - sql

I have two tables that I would like to join with the following columns:
Tables: #IndexDecomposition; Constituent.
Rows: Identifier; CUSIP; ISIN; SEDOL.
This is all part of a larger MERGE statement, and now I think that the cross join is happening on the INSERT statement:
MERGE indexdecomp.ConstituentWeighting targ
USING (#IndexConstituents src
INNER JOIN indexDecomp.Constituent c WITH (NOLOCK) on
((c.Identifier = src.Identifier) OR (c.Identifier IS NULL AND src.Identifier IS NULL))
AND ((c.CUSIP = src.CUSIP) OR (c.CUSIP IS NULL AND src.CUSIP IS NULL))
AND ((c.ISIN = src.ISIN) OR (c.ISIN IS NULL AND src.ISIN IS NULL))
AND ((c.SEDOL = src.SEDOL) OR (c.SEDOL IS NULL AND src.SEDOL IS NULL)))
ON ( targ.ConstituentId = c.Id
AND targ.AsOfDate = src.Date
AND ((targ.Weighting = src.Weighting) OR (targ.Weighting IS NULL AND src.Weighting IS NULL))
AND ((targ.TotalSharesHeld = src.TotalSharesHeld) OR (targ.TotalSharesHeld IS NULL AND src.TotalSharesHeld IS NULL))
AND ((targ.SharesOutstanding = src.SharesOutstanding) OR (targ.SharesOutstanding IS NULL AND src.SharesOutstanding IS NULL))
AND ((targ.NotionalValue = src.NotionalValue) OR (targ.NotionalValue IS NULL AND src.NotionalValue IS NULL))
AND ((targ.MarketValue = src.MarketValue) OR (targ.MarketValue IS NULL AND src.MarketValue IS NULL))
AND ((targ.MarketCap = src.MarketCap) OR (targ.MarketCap IS NULL AND src.MarketCap IS NULL))
AND ((targ.LastTrade = src.LastTrade) OR (targ.LastTrade IS NULL AND src.LastTrade IS NULL))
AND ((targ.Earnings = src.Earnings) OR (targ.Earnings IS NULL AND src.Earnings IS NULL))
AND ((targ.PeRatio = src.PeRatio) OR (targ.PeRatio IS NULL AND src.PeRatio IS NULL))
AND ((targ.Face = src.Face) OR (targ.Face IS NULL AND src.Face IS NULL)))
WHEN NOT MATCHED BY TARGET THEN
INSERT (
ConstituentID
,AsOfDate
,Weighting
,TotalSharesHeld
,SharesOutstanding
,NotionalValue
,MarketValue
,MarketCap
,LastTrade
,Earnings
,PeRatio
,Face
,ModifiedBy
,ModifiedDate
,CreatedBy
,CreatedDate
)
VALUES (
c.Id
,src.Date
,src.Weighting
,src.TotalSharesHeld
,src.SharesOutstanding
,src.NotionalValue
,src.MarketValue
,src.MarketCap
,src.LastTrade
,src.Earnings
,src.PeRatio
,src.Face
,'user'
,getdate()
,'user'
,getdate()
)
;

I think the only problem with your latest attempt is that it does not weed out combinations of all nulls (i.e. false positives where all four IDs are null). You can fix it by requiring that at least one of the four identifiers mus be non-null, like this:
SELECT *
FROM xTradeCapture.staging.IndexDecomposition src
INNER JOIN indexDecomp.Constituent c WITH (NOLOCK) on
(((c.Identifier = src.Identifier) OR (c.Identifier IS NULL AND src.Identifier IS NULL))
AND ((c.CUSIP = src.CUSIP) OR (c.CUSIP IS NULL AND src.CUSIP IS NULL))
AND ((c.ISIN = src.ISIN) OR (c.ISIN IS NULL AND src.ISIN IS NULL))
AND ((c.SEDOL = src.SEDOL) OR (c.SEDOL IS NULL AND src.SEDOL IS NULL))
AND (c.Identifier IS NOT NULL OR c.CUSIP IS NOT NULL OR c.ISIN IS NOT NULL OR c.SEDOL IS NOT NULL))

The second code snippet of yours is correct. The optimizer recognizes this pattern and converts it into an equi-join. So you don't get a cartesian product.

You could also use ISNULL if you have dummy values you know are not in the tables them selves. The example below assumes integers for Identifier & CUSIP and varchar for ISIN and SEDOL...
SELECT *
FROM xTradeCapture.staging.IndexDecomposition src
INNER JOIN indexDecomp.Constituent c WITH (NOLOCK) on
ISNULL(c.Identifier,-1) = ISNULL(src.Identifier,-1)
AND ISNULL(c.CUSIP,-1) = ISNULL(src.CUSIP,-1)
AND ISNULL(c.ISIN,'ABC') = ISNULL(src.ISIN,'ABC')
AND ISNULL(c.SEDOL,'ABC') = ISNULL(src.SEDOL,'ABC')

Related

SQL Query to apply join on basis of case Condition

I have a requirement where I need to fetch the Dimension Key of Region table on basis of the following preference.
Fetch dimension key on basis of Zipcode of Physical address(PA)
If the first condition is not satisfied that fetch dimension key on basis of the Zip Code of the Mailing address
If the second condition is also not satisfied than fetch the dimension key on basis of the Parish Code of Physical address
Else fetch dimension key on basis of parish Code of Mailing address.
I am trying to use the below query but is giving multiple records since all left joins are getting evaluated. I want that it should not go on the second condition if the first condition is satisfied.
select REGION_DIM_SK, CASE_NUM
from (
select distinct COALESCE(RDIM.REGION_DIM_SK, RDIM1.REGION_DIM_SK, RDIM2.REGION_DIM_SK, RDIM3.REGION_DIM_SK) AS REGION_DIM_SK
, DC.CASE_NUM, ADDR_TYPE_CD
FROM rpt_dm_ee_intg.CASE_PERSON_ADDRESS dc
left join rpt_dm_ee_prsnt.REGION_DIM RDIM on dc.ZIP_CODE = RDIM.ZIP_CODE and RDIM.REGION_EFF_END_DT IS NULL and dc.addr_type_cd='PA' AND dc.EFF_END_DT IS NULL
left join rpt_dm_ee_prsnt.REGION_DIM RDIM1 ON dc.ZIP_CODE = RDIM1.ZIP_CODE AND RDIM1.REGION_EFF_END_DT IS NULL AND dc.addr_type_cd='MA' AND DC.EFF_END_DT IS NULL
left join (
select PARISH_CD, min(REGION_DIM_SK) as REGION_DIM_SK
from rpt_dm_ee_prsnt.REGION_DIM
where REGION_EFF_END_DT is null
group by PARISH_CD
) RDIM2 ON dc.addr_type_cd='PA' and dc.PARISH_CD = RDIM2.PARISH_CD AND DC.EFF_END_DT IS NULL
left join (
select PARISH_CD, min(REGION_DIM_SK) as REGION_DIM_SK
from rpt_dm_ee_prsnt.REGION_DIM
where REGION_EFF_END_DT is null
group by PARISH_CD
) RDIM3 ON dc.addr_type_cd='MA' and dc.PARISH_CD = RDIM3.PARISH_CD AND DC.EFF_END_DT IS NULL
) A
where REGION_DIM_SK is not null
) RD on RD.case_num = rpt_dm_ee_intg.CASE_PERSON_ELIGIBILITY.CASE_NUM
Use multiple left joins. Your query is rather hard to follow -- it has other tables and references not described in the problem.
But the idea is:
select t.*,
coalesce(rpa.dim_key, rm.dim_key, rpap.dim_key, rmp.dim_key) as dim_key
from t left join
dim_region rpa
on t.physical_address_zipcode = rpa.zipcode left join
dim_region rm
on t.mailing_address_zipcode = rm.zipcode and
rpa.zipcode is null left join
dim_region rpap
on t.physical_addresss_parishcode = rpap.parishcode and
rm.zipcode is null left join
dim_region rmp
on t.physical_addresss_parishcode = rmp.parishcode and
rpap.zipcode is null
The trick is to put the conditions in CASE WHEN:
SELECT *
FROM table1 a
JOIN table2 b
ON CASE
WHEN a.code is not null and a.code = b.code THEN 1
WHEN a.type = b.type THEN 1
ELSE 0
END = 1
For your example you can reduce the code to just two joins, it can't be done in one as you are joining two different tables.
SELECT CASE WHEN RDIM.addres IS NULL THEN RDIM2.addres ELSE RDIM.addres
FROM rpt_dm_ee_intg.CASE_PERSON_ADDRESS dc
LEFT JOIN rpt_dm_ee_prsnt.REGION_DIM RDIM ON CASE
WHEN (dc.ZIP_CODE = RDIM.ZIP_CODE
AND RDIM.REGION_EFF_END_DT IS NULL
AND dc.addr_type_cd='PA'
AND dc.EFF_END_DT IS NULL) THEN 1
WHEN (dc.ZIP_CODE = RDIM1.ZIP_CODE
AND RDIM1.REGION_EFF_END_DT IS NULL
AND dc.addr_type_cd='MA'
AND DC.EFF_END_DT IS NULL) THEN 1
ELSE 0
END = 1
LEFT JOIN
(SELECT PARISH_CD,
min(REGION_DIM_SK) AS REGION_DIM_SK
FROM rpt_dm_ee_prsnt.REGION_DIM
WHERE REGION_EFF_END_DT IS NULL
GROUP BY PARISH_CD) RDIM2 ON CASE
WHEN (dc.addr_type_cd='PA'
AND dc.PARISH_CD = RDIM2.PARISH_CD
AND DC.EFF_END_DT IS NULL
AND RDIM.ZIP_CODE IS NULL) THEN 1
WHEN (dc.addr_type_cd='MA'
AND dc.PARISH_CD = RDIM3.PARISH_CD
AND DC.EFF_END_DT IS NULL
AND RDIM.ZIP_CODE IS NULL) THEN 1
ELSE 0
END = 1
edit
If you don't want to have nulls from RDIM2 table if RDIM1 zip code is present the logic could be easily extended to support that. You just need to add AND RDIM.ZIP_CODE IS NULL to CASE WHEN conditions.

Use if in where statement

I have 4 table:
Table_op_type
Table_maintenancetype
Table_repair_time
Table_repair_type
Each table has one column that can true or false and I have one Table_maintenancereport that has many columns and 4 columns in Table_maintenancereport foreign key given from 4 up table when I select Table_maintenancereport and one row in each table is true code working fine but when more than one row returned from table give this error.
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
How can I fix it?
My code is:
SELECT *
FROM Table_maintenancereport
WHERE mtypeid IN (IIF(#smtype = 1, (SELECT Table_maintenancetype.id FROM Table_maintenancetype WHERE enable_search = 1), mtypeid))
AND op_typeid IN (IIF(#sop IN (1), (SELECT Table_op_type.id FROM Table_op_type WHERE enable_search = 1), op_typeid))
AND repaire_timeid IN (IIF(#stime IN (1), (SELECT Table_repair_time.id FROM Table_repair_time WHERE enable_search = 1), repaire_timeid))
AND repaire_typeid IN (IIF(#stype = 1, (SELECT Table_repair_type.id FROM Table_repair_type WHERE enable_search = 1), repaire_typeid));
You need to fix the where conditions. You can do this with basic logic operators. Sets cannot be returned by iif() or by case() expressions.
So:
WHERE (#smtype <> 1 OR
mtypeid IN (SELECT Table_maintenancetype.id FROM Table_maintenancetype WHERE enable_search = 1)
) AND
(#sop <> 1 OR
op_typeid IN (SELECT Table_op_type.id FROM Table_op_type WHERE enable_search = 1)
) AND
(#stime <> 1 OR
repaire_timeid IN (SELECT Table_repair_time.id FROM Table_repair_time WHERE enable_search = 1)
) AND
(#stype <> 1 OR
repaire_typeid IN (SELECT Table_repair_type.id FROM Table_repair_type WHERE enable_search = 1)
);
If I am interpreting what you are trying to do correctly:
SELECT *
FROM Table_maintenancereport mr
LEFT OUTER JOIN Table_maintenancetype mt ON mr.mtypeid = mt.id AND enable_search = 1
LEFT OUTER JOIN Table_op_type ot ON mr.op_typeid = ot.id AND enable_search = 1
LEFT OUTER JOIN Table_repair_time rt ON mr.repaire_timeid = rt.id AND enable_search = 1
LEFT OUTER JOIN Table_repair_type rty ON mr.repaire_typeid = rty.id AND enable_search = 1
WHERE (ISNULL(#smtype,-1) != 1 OR mt.id IS NOT NULL)
AND (ISNULL(#sop,-1) != 1 OR ot.id IS NOT NULL)
AND (ISNULL(#stime,-1) != 1 OR rt.id IS NOT NULL)
AND (ISNULL(#stype,-1) != 1 OR rty.id IS NOT NULL)
so for each #variable, if it is set to 1, the corresponding field has to be one of the search enabled ids to appear in the output. If a variable is not set to 1, then it will not filter the corresponding field.
Maybe you can use CASE WHEN statement in where clause like:
select
*
from
Table_maintenancereport
where
case
when #smtype = 1 then mtypeid in (
select Table_maintenancetype.id
from
Table_maintenancetype
where
enable_search = 1)
else mtypeid = mtypeid end
....

My INNER JOIN is filtering out too much

My INNER JOIN is filtering out too much or I can't see what the difference is.
I am trying to consolidate 2 tables from 2 different databases on the same server.
My first step is to find all the values that match:
SELECT intl.*
FROM tblData intl
INNER JOIN [db].dbo.tblData us
ON intl.DataID = us.DataID
AND intl.AnotherID = us.AnotherID
AND intl.DataValue = us.DataValue
AND intl.TextValue = us.TextValue
AND intl.DefaultValue = us.DefaultValue
AND intl.RateValue = us.RateValue
AND intl.YetAnotherID = us.YetAnotherID
I've checked all the columns and they are the same type and same varchar values. So this query should bring back all the matching values.
However... That is not the case. It is only returning 200+ records, where if I run this query after I insert the 200+ records into a temp table:
SELECT intl.DataID
FROM tblData intl
LEFT JOIN TempDataTable TDT
ON TDT.DataID = intl.DataID
AND TDT.AnotherID = intl.AnotherID
AND TDT.DataValue = intl.DataValue
AND TDT.TextValue = intl.TextValue
AND TDT.DefaultValue = intl.DefaultValue
AND TDT.RateValue = intl.RateValue
AND TDT.YetAnotherID = intl.YetAnotherID
WHERE TDT.DataID IS NULL
This query returns 1500+ rows.
I've run a similar query with the US data and found the same discrepancy (1500+ rows returned).
I looked at 1 record just to see if I could identify something whacky, and I found that the records from each table (INTL and US) are EXACTLY the same!
DataID: 1
AnotherID: 1
DataValue: NULL
TextValue: NORMAL
DefaultValue: 0
RateValue: NULL
YetAnotherID: 1
I imagine my inner join is finding a difference between the 2 tables structurally but I can not see it.
Any ideas on what would cause this?
This is likely caused by the NULL values that you have in the table failing the equality checks of your LEFT JOIN.
NULL values aren't equal to anything, not even to another NULL of the same datatype.
To remedy this, you can add a check to the ON clause for NULL values on both sides in addition to the existing JOIN conditions:
SELECT intl.DataID
FROM tblData intl
LEFT JOIN TempDataTable TDT
ON (
(TDT.DataID IS NULL AND intl.DataID IS NULL)
OR TDT.DataId = intl.DataId
)
AND (
(TDT.AnotherID IS NULL AND intl.AnotherID IS NULL)
OR TDT.AnotherId = intl.AnotherId
)
AND (
(TDT.DataValue IS NULL AND intl.DataValue IS NULL)
OR TDT.DataValue = intl.DataValue
)
AND (
(TDT.TextValue IS NULL AND intl.TextValue IS NULL)
OR TDT.TextValue = intl.TextValue
)
AND (
(TDT.DefaultValue IS NULL AND intl.DefaultValue IS NULL)
OR TDT.DefaultValue = intl.DefaultValue
)
AND (
(TDT.RateValue IS NULL AND intl.RateValue IS NULL)
OR TDT.RateValue = intl.RateValue
)
AND (
(TDT.YetAnotherID IS NULL AND intl.YetAnotherID IS NULL)
OR TDT.YetAnotherID = intl.YetAnotherID
)
WHERE TDT.DataID IS NULL
I like this approach to dealing with NULL equality comparisons.
More details about it here
SELECT intl.*
FROM tblData intl
INNER JOIN [db].dbo.tblData us
ON EXISTS (SELECT intl.DataID,
intl.AnotherID,
intl.DataValue,
intl.TextValue,
intl.DefaultValue,
intl.RateValue,
intl.YetAnotherID,
INTERSECT
SELECT us.DataID,
us.AnotherID,
us.DataValue,
us.TextValue,
us.DefaultValue,
us.RateValue,
us.YetAnotherID)
Values null not take in your query. Use isnull(value, 0) for type value number and isnull(value, '') for type value chararctere
SELECT intl.DataID
FROM tblData intl
inner JOIN TempDataTable TDT
ON isnull(TDT.DataID, 0) = isnull(intl.DataID, 0)
AND isnull(TDT.AnotherID, 0) = isnull(intl.AnotherID, 0)
AND isnull(TDT.DataValue, '') = isnull(intl.DataValue, '')
AND isnull(TDT.TextValue, '') = isnull(intl.TextValue, '')
AND isnull(TDT.DefaultValue, 0) = isnull(intl.DefaultValue, 0)
AND isnull(TDT.RateValue, '') = isnull(intl.RateValue, '')
AND isnull(TDT.YetAnotherID, 0) = isnull(intl.YetAnotherID, 0)
You INNER JOIN is doing what it should and that is returning only rows from tblData (us) and tblData (international) where all of your tdt = intl condtions apply. If it is a NULL value then coalesce() would be what you want to use. Consider this:
select 1 where null = null
select 2 where coalesce(null,2) = coalesce(null,2)
EXISTS can be good for this query.
SELECT
intl.DataID
FROM
tblData intl
WHERE
NOT EXIST
(
SELECT TOP 1 FROM TempDataTable TDT
WHERE
TDT.DataID = intl.DataID AND
TDT.AnotherID = intl.AnotherID AND
TDT.DataValue = intl.DataValue AND
TDT.TextValue = intl.TextValue AND
TDT.DefaultValue = intl.DefaultValue AND
TDT.RateValue = intl.RateValue AND
TDT.YetAnotherID = intl.YetAnotherID
)

WHERE IS NULL Not Working in Nested Select

INSERT INTO AutomatedTest_XMLReconciliation(TaskID, TestCaseID, TestInstanceID, XMLHierarchyLevelID, XMLNode, XMLValue_Truth, XMLValue_Test)
(
(SELECT A.TaskID, A.TestCaseID, NULL, A.XMLHierarchyLevelID, A.XMLNode, A.XMLValue_Truth, NULL
FROM AutomatedTest_XMLTruthData A
LEFT JOIN AutomatedTest_Auth B
ON A.TestCaseID = B.TestCaseID
FULL Join AutomatedTest_XMLTestData C
ON B.TestInstanceID = C.TestInstanceID AND A.TruthIdentity = C.TestIdentity AND A.XMLHierarchyLevelID = C.XMLHierarchyLevelID AND A.XMLNode = C.XMLNode AND A.TaskID = C.TaskID
WHERE (B.ReconcileDate IS NULL AND A.TaskID IS NOT NULL)
EXCEPT
(Select A.TaskID, A.TestCaseID, NULL, A.XMLHierarchyLevelID, A.XMLNode, A.XMLValue_Truth, NULL
FROM AutomatedTest_XMLTruthData A
INNER JOIN AutomatedTest_Auth B
ON A.TestCaseID = B.TestCaseID
INNER JOIN AutomatedTest_XMLTestData C
ON B.TestInstanceID = C.TestInstanceID AND A.TruthIdentity = C.TestIdentity AND A.XMLHierarchyLevelID = C.XMLHierarchyLevelID AND A.XMLNode = C.XMLNode AND A.TaskID = C.TaskID)
)
UNION
(SELECT C.TaskID, NULL, C.TestInstanceID, C.XMLHierarchyLevelID, C.XMLNode, NULL, C.XMLValue_Test
FROM AutomatedTest_XMLTruthData A
LEFT JOIN AutomatedTest_Auth B
ON A.TestCaseID = B.TestCaseID
FULL JOIN AutomatedTest_XMLTestData C
ON B.TestInstanceID = C.TestInstanceID AND A.TruthIdentity = C.TestIdentity AND A.XMLHierarchyLevelID = C.XMLHierarchyLevelID AND A.XMLNode = C.XMLNode AND A.TaskID = C.TaskID
WHERE B.ReconcileDate IS NULL AND C.TaskID IS NOT NULL)
EXCEPT
(Select C.TaskID, NULL, C.TestInstanceID, C.XMLHierarchyLevelID, C.XMLNode, NULL, C.XMLValue_Test
FROM AutomatedTest_XMLTruthData A
INNER JOIN AutomatedTest_Auth B
ON A.TestCaseID = B.TestCaseID
INNER JOIN AutomatedTest_XMLTestData C
ON B.TestInstanceID = C.TestInstanceID AND A.TruthIdentity = C.TestIdentity AND A.XMLHierarchyLevelID = C.XMLHierarchyLevelID AND A.XMLNode = C.XMLNode AND A.TaskID = C.TaskID)
))
UPDATE AutomatedTest_Auth
SET ReconcileDate = GETDATE()
WHERE (TestCaseID IN (Select TestCaseID FROM AutomatedTest_DataReconciliation WHERE TestCaseID IS NOT NULL)
OR TestInstanceID IN (Select TestInstanceID FROM AutomatedTest_DataReconciliation WHERE TestInstanceID IS NOT NULL))
AND ReconcileDate IS NULL
So I've got this insert query that is meant to check two tables, one test and one auth, and compare their rows. It matches a TestInstance in the test table with a TestCase in a truth table through the AutomatedTest_Auth table.
Once it is done, it updates the AutomatedTest_Auth table with the GETDATE() stamp to show that the reconciliation process is complete. The insert checks this in the WHEREs to make sure that it does not insert anything that is already inserted by making sure the ReconcileDate IS NULL (it defaults to null on upon a new entry into the AutomatedTest_Auth table).
My problem, however, is that the second "B.Reconcile IS NULL" is not working. Upon running this twice, it will pick up the same rows from AutomatedTest_XMLTestData it picked up the first time and insert them again. (Just a note, there is no PK violation, as the Reconciliation columns could overlap at any time; it has an identity column instead).
Is this an order of operations issue? ... a bug? Any help would be nice.
Also, when I change the B.ReconcileDate IS NULL to IS NOT NULL, the insert swaps, and it will instead insert the same rows from AutomatedTest_XMLTruthData and not AutomatedTest_XMLTestData, so I know the inserts from AutomatedTest_XMLTruthData are correctly recognizing the WHERE clause, and that AutomatedTest_XMLTestData is incorrectly inverting the B.ReconcileData IS NULL to IS NOT NULL.
You are inserting into a table called AutomatedTest_XMLReconciliation
The update where condition looks at a table called AutomatedTest_DataReconciliation which is not referenced elsewhere in your script. Is this correct? I think the Update should be:
UPDATE AUTOMATEDTEST_AUTH
SET RECONCILEDATE = GETDATE()
WHERE ( TESTCASEID IN (SELECT TESTCASEID
FROM AUTOMATEDTEST_XMLRECONCILIATION
WHERE TESTCASEID IS NOT NULL)
OR TESTINSTANCEID IN (SELECT TESTINSTANCEID
FROM AUTOMATEDTEST_XMLRECONCILIATION
WHERE TESTINSTANCEID IS NOT NULL) )
AND RECONCILEDATE IS NULL

SQL SP runs Slow

Hi
I am using an SP which takes 7 minutes in a server which has 7336 recrds
and 6seconds in another server which has 3500 records.
Can anybody help me to know why is it happening?
Thanks,
-Divya
THE SP:
SELECT WORKSHEET_ID
FROM PERSON PER
INNER JOIN PERSON EMPLEE
ON EMPLEE.PERSON_ID = PER.PERSON_ID
AND
dbo.FN_CHECKRPTSECURITY(EMPLEE.PERSON_ID, #p_SEC_ACCOUNT_ID) > 0
LEFT JOIN SEARCH_ASSIGNMENT_VW PERSON_ASGN
ON PERSON_ASGN.ASSIGNMENT_ID =
dbo.FN_GETRPTASSIGNMENTID(EMPLEE.PERSON_ID)
LEFT JOIN LOOKUP EMPLEE_ASGN_STAT
ON EMPLEE_ASGN_STAT.TYPE_ = 'ASSIGNMMENT_STATUS_CODE'
AND EMPLEE_ASGN_STAT.CODE = PERSON_ASGN.ASGN_STAT_CODE
INNER JOIN
(SELECT w1.ASSIGNMENT_ID, w1.WORKSHEET_ID, w1.EFFECTIVE_DATE, w1.APPROVED_BY, w3.CREATED_BY
FROM WORKSHEET_PAYROLL_VW w1
INNER JOIN WORKSHEET w3
ON w3.WORKSHEET_ID = w1.WORKSHEET_ID
WHERE w1.EFFECTIVE_DATE = CASE
WHEN #p_MOST_RECENT_ONLY = 'Y'
THEN (SELECT MAX(w2.EFFECTIVE_DATE)
FROM WORKSHEET_PAYROLL_VW w2
WHERE w1.ASSIGNMENT_ID = w2.ASSIGNMENT_ID
AND (ISNULL(#p_WKS_EFFECTIVE_DATE,w2.EFFECTIVE_DATE) =w2.EFFECTIVE_DATE))
ELSE ISNULL(#p_WKS_EFFECTIVE_DATE,w1.EFFECTIVE_DATE)
END
)
PERSON_WKS
ON PERSON_WKS.ASSIGNMENT_ID = dbo.FN_GETRPTASSIGNMENTID(EMPLEE.PERSON_ID)
INNER JOIN
(SELECT ASSIGNMENT_ID, VALUE
FROM ASSIGNMENT_HISTORY AH
WHERE FIELD_NAME ='HOME PAYROLL GROUP'
AND EFFECTIVE_DATE = (SELECT MAX(EFFECTIVE_DATE)
FROM ASSIGNMENT_HISTORY
WHERE ASSIGNMENT_ID = AH.ASSIGNMENT_ID
AND EFFECTIVE_DATE <=getDate()
AND FIELD_NAME = 'HOME PAYROLL GROUP')
)HOME_PAYROLL
ON HOME_PAYROLL.ASSIGNMENT_ID = dbo.FN_GETRPTASSIGNMENTID(EMPLEE.PERSON_ID)
WHERE
(#p_SELECTED_PERSON_ONLY = 'N' OR EMPLEE.PERSON_ID = #p_PERSON_ID)
AND
(#p_ASGN_STAT_CODE IS NULL OR PERSON_ASGN.ASGN_STAT_CODE = SUBSTRING(#p_ASGN_STAT_CODE,1,1)
OR PERSON_ASGN.ASGN_STAT_CODE = SUBSTRING(#p_ASGN_STAT_CODE,2,1))
AND
(#p_POLICY_ID IS NULL OR PERSON_ASGN.PROGRAM_CODE = #p_POLICY_ID)
AND
(#p_HOME_COUNTRY_ID IS NULL OR PERSON_ASGN.HOMECOUNTRYID=#p_HOME_COUNTRY_ID)
AND
(#p_HOME_CITY_ID IS NULL OR PERSON_ASGN.HOMECITYID=#p_HOME_CITY_ID )
AND
(#p_HOME_COMPANY_ID IS NULL OR PERSON_ASGN.HOMEBUSINESSID=#p_HOME_COMPANY_ID )
AND
(#p_HOME_DIVISION_ID IS NULL OR PERSON_ASGN.HOMECOMPONENTID=#p_HOME_DIVISION_ID )
AND
(#p_HOST_COUNTRY_ID IS NULL OR PERSON_ASGN.HOSTCOUNTRYID=#p_HOST_COUNTRY_ID )
AND
(#p_HOST_CITY_ID IS NULL OR PERSON_ASGN.HOSTCITYID=#p_HOST_CITY_ID )
AND
(#p_HOST_COMPANY_ID IS NULL OR PERSON_ASGN.HOSTBUSINESSID=#p_HOST_COMPANY_ID )
AND
(#p_HOST_DIVISION_ID IS NULL OR PERSON_ASGN.HOSTCOMPONENTID=#p_HOST_DIVISION_ID )
AND
(#p_CREATED_BY IS NULL OR PERSON_WKS.CREATED_BY=#p_CREATED_BY )
AND
(#p_APPROVED_BY IS NULL OR PERSON_WKS.APPROVED_BY=#p_APPROVED_BY )
AND
(#p_payroll_code IS NULL OR HOME_PAYROLL.VALUE=#p_payroll_code )
ORDER BY PER.LAST_NAME ASC,
PER.FIRST_NAME ASC,
PERSON_WKS.EFFECTIVE_DATE DESC
The Function in the 5th line is the one which is running slow. rest of the part is running in 4secs
The FUNCTION:
BEGIN
DECLARE
#v_ASGN_COUNT INT,
#v_RESULT INT
SELECT #v_ASGN_COUNT = COUNT(ASSIGNMENT_ID) --to find out if this employee has any assignment
FROM ASSIGNMENT
WHERE EXPATRIATE_PERSON_ID = #p_PERSON_ID AND
ASGN_STAT_CODE IN ('PD','A','I')
IF(#v_ASGN_COUNT > 0) --yes assignment, check against SECURITY_ASSIGNMENT_VW
BEGIN
SELECT #v_RESULT = COUNT(ASSIGNMENT_ID)
FROM SECURITY_ASSIGNMENT_VW
WHERE SEC_ACCOUNT_ID = #p_SEC_ACCOUNT_ID AND
ASSIGNMENT_ID IN (SELECT ASSIGNMENT_ID
FROM ASSIGNMENT
WHERE EXPATRIATE_PERSON_ID = #p_PERSON_ID AND
ASGN_STAT_CODE IN ('PD','A','I'))
END
ELSE --no assignment, so check against SECURITY_PERSON_VW
BEGIN
SELECT #v_RESULT = COUNT(PERSON_ID)
FROM SECURITY_PERSON_VW
WHERE SEC_ACCOUNT_ID = #p_SEC_ACCOUNT_ID AND
PERSON_ID = #p_PERSON_ID
END
RETURN #v_RESULT
END
Do the schemas match exactly... in particular check for missing indexes.
Well to begin with you have scalar functions which will run significantly slower as the number of records increase becasue they process row-by-agonizing-row. Not only that you've used the functions in joins which is a horrible practice if you need performance. You have a bunch of OR conditions which tend to slowness. And while it is too hard to actually read the code you posted (please try to format and only use all caps for keywords), I would suspect that some of those conditions are not sargable.
To know what is actually happening check the Execution plan (SQL Server) or Explain Plan (mySQL and others I think) or the equivalent feature in your database. Likely you wil find table scans which of course are going to get significantly slower as the number of records increases.
You may also have a problem with parameter sniffing. Please google to see how to fix that.
One improvement would be to make sure that dbo.FN_GETRPTSSIGNMENTID only gets executed once.
Currently, it gets executed three times.
You can replace two of those calls by joining to the field of the (one) remaing call.
Something like
SELECT WORKSHEET_ID
FROM PERSON PER
INNER JOIN PERSON EMPLEE ON EMPLEE.PERSON_ID = PER.PERSON_ID AND dbo.FN_CHECKRPTSECURITY(EMPLEE.PERSON_ID, #p_SEC_ACCOUNT_ID) > 0
INNER JOIN (
SELECT w1.ASSIGNMENT_ID
, w1.WORKSHEET_ID
, w1.EFFECTIVE_DATE
, w1.APPROVED_BY
, w3.CREATED_BY
FROM WORKSHEET_PAYROLL_VW w1
INNER JOIN WORKSHEET w3 ON w3.WORKSHEET_ID = w1.WORKSHEET_ID
WHERE w1.EFFECTIVE_DATE =
CASE WHEN #p_MOST_RECENT_ONLY = 'Y'
THEN (
SELECT MAX(w2.EFFECTIVE_DATE)
FROM WORKSHEET_PAYROLL_VW w2
WHERE w1.ASSIGNMENT_ID = w2.ASSIGNMENT_ID AND (ISNULL(#p_WKS_EFFECTIVE_DATE,w2.EFFECTIVE_DATE) = w2.EFFECTIVE_DATE)
)
ELSE ISNULL(#p_WKS_EFFECTIVE_DATE,w1.EFFECTIVE_DATE)
END
) PERSON_WKS ON PERSON_WKS.ASSIGNMENT_ID = dbo.FN_GETRPTASSIGNMENTID(EMPLEE.PERSON_ID)
INNER JOIN (
SELECT ASSIGNMENT_ID
, VALUE
FROM ASSIGNMENT_HISTORY AH
WHERE FIELD_NAME ='HOME PAYROLL GROUP'
AND EFFECTIVE_DATE = (
SELECT MAX(EFFECTIVE_DATE)
FROM ASSIGNMENT_HISTORY
WHERE ASSIGNMENT_ID = AH.ASSIGNMENT_ID
AND EFFECTIVE_DATE <=getDate()
AND FIELD_NAME = 'HOME PAYROLL GROUP'
)
LEFT JOIN SEARCH_ASSIGNMENT_VW PERSON_ASGN ON PERSON_ASGN.ASSIGNMENT_ID = PERSON_WKS.ASSIGNMENT_ID
LEFT JOIN LOOKUP EMPLEE_ASGN_STAT ON EMPLEE_ASGN_STAT.TYPE_ = 'ASSIGNMMENT_STATUS_CODE' AND EMPLEE_ASGN_STAT.CODE = PERSON_ASGN.ASGN_STAT_CODE
) HOME_PAYROLL ON HOME_PAYROLL.ASSIGNMENT_ID = PERSON_WKS.ASSIGNMENT_ID
WHERE (#p_SELECTED_PERSON_ONLY = 'N' OR EMPLEE.PERSON_ID = #p_PERSON_ID)
AND (#p_ASGN_STAT_CODE IS NULL OR PERSON_ASGN.ASGN_STAT_CODE = SUBSTRING(#p_ASGN_STAT_CODE,1,1) OR PERSON_ASGN.ASGN_STAT_CODE = SUBSTRING(#p_ASGN_STAT_CODE,2,1))
AND (#p_POLICY_ID IS NULL OR PERSON_ASGN.PROGRAM_CODE = #p_POLICY_ID)
AND (#p_HOME_COUNTRY_ID IS NULL OR PERSON_ASGN.HOMECOUNTRYID=#p_HOME_COUNTRY_ID)
AND (#p_HOME_CITY_ID IS NULL OR PERSON_ASGN.HOMECITYID=#p_HOME_CITY_ID )
AND (#p_HOME_COMPANY_ID IS NULL OR PERSON_ASGN.HOMEBUSINESSID=#p_HOME_COMPANY_ID )
AND (#p_HOME_DIVISION_ID IS NULL OR PERSON_ASGN.HOMECOMPONENTID=#p_HOME_DIVISION_ID )
AND (#p_HOST_COUNTRY_ID IS NULL OR PERSON_ASGN.HOSTCOUNTRYID=#p_HOST_COUNTRY_ID )
AND (#p_HOST_CITY_ID IS NULL OR PERSON_ASGN.HOSTCITYID=#p_HOST_CITY_ID )
AND (#p_HOST_COMPANY_ID IS NULL OR PERSON_ASGN.HOSTBUSINESSID=#p_HOST_COMPANY_ID )
AND (#p_HOST_DIVISION_ID IS NULL OR PERSON_ASGN.HOSTCOMPONENTID=#p_HOST_DIVISION_ID )
AND (#p_CREATED_BY IS NULL OR PERSON_WKS.CREATED_BY=#p_CREATED_BY )
AND (#p_APPROVED_BY IS NULL OR PERSON_WKS.APPROVED_BY=#p_APPROVED_BY )
AND (#p_payroll_code IS NULL OR HOME_PAYROLL.VALUE=#p_payroll_code )
ORDER BY
PER.LAST_NAME ASC
, PER.FIRST_NAME ASC
, PERSON_WKS.EFFECTIVE_DATE DESC