Oracle query running too slow (approximately taking 5 hrs) - sql

SELECT t1.member_id ,
SUM(t1.paid_amt) AS paid_amt
FROM
(SELECT DISTINCT fm.member_id,
fc.claim_skey_no,
fc.claim_id,
fc.claim_line_no ,
CASE
WHEN fc.claim_type_cd = 'RX'
THEN NVL(fc.rx_paid_amt,0) -- For RX claims use rx_paid_amt as paid amount
ELSE NVL(fc.approved_amt,0)
END AS paid_amt -- For all other claims use approved_amt
,
CASE
WHEN fc.claim_type_cd = 'RX'
THEN fc.submit_dt --For RX claims use submit_dt as paid date
ELSE NVL(fc.paid_dt,NVL(fc.edi_eob_dt,NVL(fc.eob_run_dt,fc.outsource_vndr_paid_dt)))
END AS paid_dt --For all other claims use paid_dt
FROM dwprod.fct_claim fc ,
dwprod.fct_member fm
WHERE fc.mbr_skey_no = fm.member_skey_no
-- The service_from_dt on the claim must be between the reimbursement time period.
AND fc.service_from_dt BETWEEN '31-MAY-2013' AND '30-Jun-2014'
-- The follwong 2 conditions make sure that the calims selected are final-status (unadjusted)
-- For non-RX claims, the adjust_type_cd must be Null and the dw_backout_tag must be Null or 'N'
-- For RX claims only the dw_backout_tag must be Null or 'N', the adjust_type_cd is ignored
AND
CASE
WHEN fc.claim_type_cd = 'RX'
THEN 1
WHEN fc.claim_type_cd <> 'RX'
AND fc.adjust_type_cd IS NULL
THEN 1
ELSE 0
END = 1
AND NVL(fc.dw_backout_tag,'N') = 'N'
-- The claim must be in an 'Approved' status, indicated by a status_type_cd = 'A'
AND fc.status_type_cd = 'A'
-- QNXT claims must be in a 'PAID' status
-- Non QNXT claims in the warehouse are assumed to be paid - There are no pended RX claims.
AND
CASE
WHEN fc.dw_source_cd <> 'QNXT'
THEN 1
WHEN fc.dw_source_cd = 'QNXT'
AND fc.last_status_nm = 'PAID'
THEN 1
ELSE 0
END = 1
-- Dental claims are excluded
AND fc.dw_source_cd <> 'DBP'
-- Excludes any Medicare Non-RCI claims
AND
CASE
WHEN NVL(fc.program_nm,'OTHER') = 'MEDICAID'
AND NVL(fc.enroll_ratecode,'RCI') IN ('RCII','RCV','RCVII')
THEN 0
ELSE 1
END = 0
-- It Fits! claims are excluded
AND NVL(fc.expense_cat_nm,'Other') <> 'FITNESS'
AND NVL(fc.proc1_skey_no,12345) NOT IN (21586,21588,21589)
--
AND
CASE
WHEN NVL(fc.program_nm,'OTHER') = 'MEDICAID'
AND NVL(fc.enroll_ratecode,'RCI') IN ('RCII','RCV','RCVII')
THEN 1
WHEN EXISTS
(SELECT 1
FROM dwprod.fct_member_enroll me
WHERE fm.member_skey_no = me.mbr_skey_no
AND fc.service_from_dt BETWEEN me.segment_effect_dt AND me.segment_term_dt
AND me.program_nm = 'MEDICAID'
AND me.enroll_ratecode IN ('RCII','RCV','RCVII')
)
THEN 1
ELSE 0
END = 1
) t1
--Where t1.paid_dt < '31-JAN-2014'
GROUP BY t1.member_id
HAVING SUM(t1.paid_amt) > 175000

Run an explain plan to see what's causing the slowdown. From the top of my head, this is what's 'killing' you:
WHEN EXISTS
(SELECT 1
FROM dwprod.fct_member_enroll me
WHERE fm.member_skey_no = me.mbr_skey_no
AND fc.service_from_dt BETWEEN me.segment_effect_dt AND me.segment_term_dt
AND me.program_nm = 'MEDICAID'
AND me.enroll_ratecode IN ('RCII','RCV','RCVII')
)
See if you can somehow change this exists logic to something with better performance. The explain plan is a must though!

I'm gonna make a semi-blind guess here, based on similar queries in the DW I'm working with.
Oracle's optimizer gets easily confused by predicates such as:
where (case when ... then ... else ... end) = 1;
The reason is that Oracle grossly over estimate the selectivity.
Check the explain plan like others have said. If you find that the estimated cardinality of table dwprod.fct_claim seems way too low, try unrolling the case statements.
For example, instead of:
AND CASE WHEN fc.dw_source_cd <> 'QNXT' THEN 1
WHEN fc.dw_source_cd = 'QNXT' AND fc.last_status_nm = 'PAID' THEN 1
ELSE 0
END = 1
Write:
and ( fc.dw_source_cd <> 'QNXT'
or (fc.dw_source_cd = 'QNXT' and fc.last_status_nm = 'PAID')
)
Final note. This seems to be less of a problem in version 11, but I have not yet had time to investigate why.

FROM dwprod.fct_claim fc ,
dwprod.fct_member fm
WHERE fc.mbr_skey_no = fm.member_skey_no
This cross join is effectively an inner join. I can't say whether Oracle will optimize this, but there's no reason not to make its job easier:
FROM dwprod.fct_claim fc ,
INNER JOIN dwprod.fct_member fm
ON fc.mbr_skey_no = fm.member_skey_no

Related

Deriving values based on result of the query and grouping the data

I am writing a sql where I am trying to pull out information of the status of the courses the user has enrolled. I am trying to return single record for each user. Two fields in the select list would derive the value in the following manner
CourseResultStatusId -
If the status of all the courses is passed then return the status as passed otherwise.
If any status is fail, the overall status is fail.
If any of the status is expired, then overall status is expired.
If any of the status is in-progress then overall status is in-progress
ExpiryDateTime - Training expiring (nearest date)
I need to apply the following logic on courses he has been assigned.
cr.ExpiryDateTime > GetDate() and cr.ExpiryDateTime <= dateadd(dd,30,getdate()) )
If you see below , the query I have written so far pulls the courses that each user has been enrolled but it is not a cumulative result. Do I need to group, if yes would need help.
DECLARE #Rep1 INT;
SET #Rep1 = 13119;
SELECT
cr.[CourseID]
,cr.[UserID]
,u.[Code]
,u.[DisplayName]
,t.[Name]
,cr.[CourseResultStatusID] AS [CourseResultStatusID]
,crs.[Description] AS [CourseResultStatusDescription]
,c.[PointsRequired]
,cr.[ExpiryDateTime]
FROM [training].[CourseResult] cr
INNER JOIN [training].[Course] c
ON cr.[CourseID] = c.[ID] and c.[IsOptional] = 0 -- and cr.ExpiryDateTime > GetDate() and cr.ExpiryDateTime <= dateadd(dd,30,getdate())
INNER JOIN [training].[CourseResultStatus] crs
ON cr.[CourseResultStatusID] = crs.[ID]
INNER JOIN org.RepresentativeTierHistory rth on rth.RepresentativeID = cr.[UserID] and GetDate() between rth.StartDate and rth.EndDate
INNER JOIN org.tier t on t.ID = rth.TierID
LEFT JOIN [org].[User] u
ON u.[ID] = cr.[UserID]
WHERE cr.[UserID] IN (
SELECT hd.DescendantId FROM org.HierarchyDescendant hd WHERE hd.RepresentativeId = #Rep1 UNION ALL SELECT #Rep1 -- for management exchange info
)
order by UserID
The result of the query is as follows. I have circled to show you records that belong to a particular user and the columns that I am interested in . I need help in getting single record for each user based on the of logic that I mentioned above.
If I followed you correctly, you can implement the priorization rules on the overall result of each user using conditional aggregation.
Starting from your existing query, the logic would be:
select
cr.[UserID],
case
when min(case when crs.[Description] = 'Complete' then 1 else 0 end) = 1
then 'Complete'
when max(case when crs.[Description] = 'Fail' then 1 else 0 end) = 1
then 'Fail'
when max(case when crs.[Description] = 'Expired' then 1 else 0 end) = 1
then 'Expired'
when max(case when crs.[Description] = 'In Progress' then 1 else 0 end) = 1
then 'In Progress'
end as ResultStatus
from ...
where ...
group by cr.[UserID]
As for the date filtering logic, you should be able to implement it directly in the where clause.
It is possible that other parts of your query can be optimized - you might want to ask a new question for this, providing proper sample data and desired results.

Query timeout increased but script fails to execute

When i execute this script on my remote database it gives me query timeout error. I've increased the timeout on my database but still have this error. I've been told if i'm able to optimized the script to make it simple it might work.
SELECT TOP 8 MIN( CASE WHEN pic_alb_love.pic=users_pics.pic
AND pic_alb_love.email = 'try#mail.com' THEN 'User' ELSE 'Guest' END)AS answer_one,
MIN ( CASE WHEN favorites.pic=users_pics.pic AND favorites.email = 'try#mail.com' THEN 'good' ELSE 'Bad'
END)AS answer2,
(CASE WHEN RTRIM (users_pics.upload_type) = 'wow' THEN 'loaded' ELSE
CASE WHEN RTRIM (users_pics.upload_type)= 'hey' THEN 'added' ELSE
CASE WHEN RTRIM (users_pics.upload_type) = 'check' THEN 'Changed' END END END)as up_ans,
(CASE WHEN RTRIM (users_pics.upload_type) = 'sample1' THEN 'new' ELSE
CASE WHEN RTRIM (users_pics.upload_type) = 'sample2' THEN 'existing' ELSE
CASE WHEN RTRIM (users_pics.upload_type) = 'sample3' THEN 'Profile Picture' END END END) as exs,
COUNT(DISTINCT users_pics.pic) as total,RTRIM (users_pics.wardrobe) as wardrobe,
fname,users_pics.wardrobe,
MIN (make)as make,MIN (htags)as htags, RTRIM (profile.profile_id) as profile_id,
users_pics.email,profile.profile_pix, RTRIM (profile.gender) as gender,
users_pics.time_group,profile.fpage,up_user_id, MIN (u_pic_id) as u_pic_id, MIN (users_pics.pic) as pic
FROM users_pics
LEFT join profile on users_pics.email = profile.email
LEFT join favorites on users_pics.pic = favorites.pic
LEFT JOIN pic_alb_love on users_pics.pic = pic_alb_love.pic
left join friends on users_pics.email = friends.resp_email
WHERE req_email = 'try#mail.com' and pic_enable='enable' or pic_view='Public'
GROUP BY users_pics.upload_type,profile.fname,profile.profile_id,users_pics.wardrobe,
users_pics.email, profile.gender,users_pics.time_group,profile.profile_pix, profile.fpage,up_user_id
ORDER BY MIN (users_pics.u_pic_id) DESC
Increasing timeout can help, but you should also check if your query isn't blocked by others operations like INSERT/UPDATE or open transaction.
The easiest way is to install and use sp_whoisactive procedure.
Second you don't need to nest CASE like you did:
(CASE WHEN RTRIM (users_pics.upload_type) = 'wow' THEN 'loaded' ELSE
CASE WHEN RTRIM (users_pics.upload_type)= 'hey' THEN 'added' ELSE
CASE WHEN RTRIM (users_pics.upload_type) = 'check' THEN 'Changed' END END END)as up_ans,
to
CASE RTRIM (user_pics.upload_type)
WHEN 'wow' THEN 'loaded'
WHEN 'hey' THEN 'added'
WHEN 'check' THEN 'changed'
ELSE NULL /* or your value like 'unknown' */
END AS up_ans
Next thing: you RTRIM almost on every string value, you should sanitize your input during inserting, unless you need spaces/tabs/newline and so on.
This way your query won't need RTRIM and can utilize index if exists any.
/* New values */
INSERT INTO table_name(...) VALUES (LTRIM(RTRIM(value...)))
/* Existing ones */
UPDATE table_name
SET col = LTRIM(RTRIM(col))
SQL Parser will understand wall of text, human will need time to do it.
I know we can argue about code style but remember you write code for people. Good readable code allow you to spot errors earlier and it is a hell easier to maintain in the future for you and your successors:
1) One selected value one line
2) The same order in SELECT and GROUP BY
3) Aggregated columns at end
4) You can use aliases no need for fully qualified names
5) No ambiguous column names, always specify from which table
6) SQL syntax UPPER CASE
7) Allign your code
Your query in more human readable from:
SELECT TOP 8
[up_user_id] /* Always add from which table even if it is unique column name, because in future you may get ambigous column */
,[fname]
,[profile_id] = RTRIM(profile.profile_id)
,[up_ans] = CASE RTRIM(users_pics.upload_type)
WHEN 'wow' THEN 'loaded'
WHEN 'hey' THEN 'added'
WHEN 'check' THEN 'changed'
ELSE NULL
END
,[exs] = CASE RTRIM(users_pics.upload_type)
WHEN 'sample1' THEN 'new'
WHEN 'sample2' THEN 'existing'
WHEN 'sample3' THEN 'Profile Picture'
ELSE NULL
END
,[wardrobe] = RTRIM(users_pics.wardrobe)
,users_pics.email
,[gender] = RTRIM(profile.gender)
,users_pics.time_group
,profile.profile_pix
,profile.fpage
,[answer_one] = MIN(CASE
WHEN pic_alb_love.pic=users_pics.pic THEN 'User'
ELSE 'Guest'
END)
,[answer2] = MIN(CASE
WHEN favorites.pic = users_pics.pic AND favorites.email = 'try#mail.com' WHEN 'good'
ELSE 'Bad'
END)
,[total] = COUNT(DISTINCT users_pics.pic)
,[make] = MIN(make)
,[htags] = MIN(htags)
,[u_pic_id] = MIN(u_pic_id)
,[pic] = MIN(users_pics.pic)
FROM users_pics /* you can use alias like AS up */
LEFT JOIN profile
ON users_pics.email = profile.email
LEFT JOIN favorites
ON users_pics.pic = favorites.pic
LEFT JOIN pic_alb_love
ON users_pics.pic = pic_alb_love.pic
LEFT JOIN friends
ON users_pics.email = friends.resp_email
WHERE
req_email = 'try#mail.com'
AND pic_enable = 'enable'
OR pic_view = 'Public'
GROUP BY
up_user_id
,profile.fname
,profile.profile_id
,users_pics.upload_type
,users_pics.wardrobe
,users_pics.email
,profile.gender
,users_pics.time_group
,profile.profile_pix
,profile.fpage
ORDER BY MIN(users_pics.u_pic_id) DESC
After you check that your query is not blocked during selecting data you can think about:
checking indexes on your tables
add WHERE condition to fetch smaller set, maybe you can use some update_date > current_date - 2 weeks
think to optimize query because now it does grouping and ordering which needs time to complete.
your WHERE condition, are you sure it shouldn't be:
.
WHERE (req_email = 'try#mail.com'
AND pic_enable = 'enable')
OR pic_view = 'Public'

How can I write this select query in SQL Server?

I need to extract some data to analyse exceptions/logs, and I'm stuck at a point.
I have a table with a column called CallType, and a status which can be Success or Failure. This table also has a column called SessionId.
I need to do this:
Select all the SessionId's where all the CallType = 'A' are marked as Success, but there is at least one CallType = 'B' having a Failure for that session.
There will be a where clause to filter out some stuff.
I'm thinking something like:
select top 10 *
from Log nolock
where ProviderId=48 -- add more conditions here
group by SessionId
having --? what should go over here?
I would do this with conditional aggregation in the having clause:
select top 10 *
from Log nolock
where ProviderId=48 -- add more conditions here
group by SessionId
having sum(case when CallType = 'A' and Status = 'Failure' then 1 else 0 end) = 0 and
sum(case when CallType = 'B' and Status = 'Failure' then 1 else 0 end) > 0 and
sum(case when CallType = 'A' and Status = 'Success' then 1 else 0 end) > 0;
The having clause checks for three conditions by counting the number of rows that meet each one. If = 0, then no records are allowed. If > 0 then records are required.
That CallType A has no failures.
That CallType B has at least one failure.
That at least one CallType A success exists.
The third condition is ambiguous -- if is not clear if you actually need CallType As to be in the data, based on the question.
SELECT *
FROM Log L WITH(NOLOCK)
WHERE L.CallType='A'
AND L.[Status] = 'Success'
AND L.ProviderId = 48
AND EXISTS (SELECT 1
FROM Log
WHERE L.SessionID = SessionID
AND CallType='B'
AND [Status] = 'Failure')
Having clause can only operate on aggregates within the group so this isn't the correct way to go about it since you are filtering out other rows you want to check against. I'd use EXISTS for this e.g.
edit: corrected the query
SELECT *
FROM Log L WITH(NOLOCK)
WHERE ProviderId = 48
AND CallType = 'A'
AND Status = 'Success'
AND EXISTS(SELECT * FROM Log WHERE L.SessionId = SessionId AND CallType = 'B' AND Status = 'Failure')
You can essentially filter out rows in the EXISTS part of the query using the aliased Log table (aliased L), matching all rows with the same session ID and seeing if any match the filters you required (failed with call type B)

Are there way/s for me to extract data that only contain a certain values in sql?

Are there way/s for me to extract data that only contain a certain values.
Ex:
Contact Asset Status
AB 1 Cancelled
AB 2 Cancelled
AB 3 Cancelled
AB 4 Cancelled
CD 5 Cancelled
CD 6 Active
CD 7 Cancelled
CD 8 Active
What I want to get are only those contacts that does contain cancelled assets ONLY (like Contact AB). And not those with both cancelled and active assets (like Contact CD).
You can do this with group by and a having clause:
select contact
from table t
group by contact
having min(status) = 'Cancelled' and max(status) = 'Cancelled';
This works for the data in your example. If status could be NULL and you want to count that as a different value, then the logic would be slightly more complicated.
The pure relational logic way is easier to understand, but will perform less well, requiring some kind of join to work. Let's satisfy the conditions
There is at least one status Cancelled for each Contact
But there are 0 statuses for that same Contact that aren't Cancelled
in a query like so:
SELECT DISTINCT
CS.Contact
FROM
ContactStatus CS
WHERE
CS.Status = 'Cancelled' -- at least one cancelled
AND NOT EXISTS ( -- but there are none of...
SELECT *
FROM ContactStatus CS2 -- contacts in the same table
WHERE
CS.Contact = CS2.Contact -- for that same contact
AND CS.Status <> 'Cancelled' -- that aren't cancelled
)
;
But we can do this with an aggregate, that will take only a single scan of the table, by using a little thought:
SELECT
Contact
FROM
ContactStatus
GROUP BY
Contact
HAVING
Count(*) = Count(CASE WHEN Status = 'Cancelled' THEN 1 END)
;
Other aggregate expressions in the HAVING clause are possible, such as:
Count(CASE WHEN Status <> 'Cancelled' THEN 1 END) = 0 -- or Min()
Min(Status) = 'Cancelled' AND Max(Status) = 'Cancelled'
Max(CASE WHEN Status = 'Cancelled' THEN 0 ELSE 1 END) = 0
Sum(SELECT 1 WHERE Status <> 'Cancelled') = 0
All of these would do the trick in this case; pick the one that makes the most sense to you.
Select contact from contacts a
where a.status = 'Canceled' AND contact = 'AB'

Complex query - IF ELSE or CASE or both (Replicate part of ETL process)

Not sure how to apporach this query (The aim of this query is to replicate part of an ETL process, thus validating it):
When a bill_type = C or M and payer type = C (client) or S (Subsidiary) or T (third party)
The payer type is defined as follows:
If BLT_BILLP.PAYR_CLIENT_UNO <> HBM_MATTER.CLIENT_UNO for Lead Matter
and TBM_MATTER._HS_3PTY = 0 Then value = S ('Subsidiary')
If BLT_BILLP.PAYR_CLIENT_UNO <> HBM_MATTER.CLIENT_UNO for Lead Matter
and TBM_MATTER._HS_3PTY = 1 Then value = T ('Third Party')
If BLT_BILLP.PAYR_CLIENT_UNO = HBM_MATTER.CLIENT_UNO for Lead Matter
Then value = C ( 'Client')
and
BLT_BILL._HS_CNTYPE = blank
Then the transaction code = BC or BS or BT.
A case query would be best:
SELECT CASE BILL_TYPE
WHEN 'C' THEN 'BC'
END
FROM DBO.SRC_BLT.BILL
Below is my attempt to create a query, is there a better way of doing this:
SELECT CASE BILL_TYPE
WHEN 'C' THEN
(CASE
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO <> DBO.SRC_HBM_MATTER.CLIENT_UNO THEN
CASE DBO.SRC_TBM_MATTER._HS_3PTY
WHEN 0 then 'S'
WHEN 1 then 'T'
END
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO = DBO.SRC_HBM_MATTER.CLIENT_UNO THEN 'C'
END)
WHEN 'M' THEN
(CASE
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO <> DBO.SRC_HBM_MATTER.CLIENT_UNO THEN
CASE DBO.SRC_TBM_MATTER._HS_3PTY
WHEN 0 then 'S'
WHEN 1 then 'T'
END
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO = DBO.SRC_HBM_MATTER.CLIENT_UNO THEN 'C'
END)
END AS TRANS_CODE
FROM DBO.SRC_BLT_BILLP, DBO.SRC_HBM_MATTER, DBO.SRC_TBM_MATTER, DBO.SRC_BLT_BILL
AND WHERE _HS_CNTYPE = ''
I'd say neither. I think "procedural" when I see "if" or "case". SQL works best when it's set-based and declarative.
Unless this is a stored procedure, I'd rework this to eliminate "if" and "case". Just my opinion.
If there are disparate data sets for each bill and player type, maybe you can access them as views and eliminate this logic from your queries.
A slightly more DRY approach to the CASE clause:
SELECT CASE
WHEN BILL_TYPE IN ('C','M') THEN
CASE
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO <> DBO.SRC_HBM_MATTER.CLIENT_UNO THEN
CASE DBO.SRC_TBM_MATTER._HS_3PTY
WHEN 0 then 'S'
WHEN 1 then 'T'
END
WHEN DBO.SRC_BLT_BILLP.PAYR_CLIENT_UNO = DBO.SRC_HBM_MATTER.CLIENT_UNO THEN 'C'
END
END AS TRANS_CODE
...