Please assume I have built a query in MS Sqlserver, it has the following structure:
WITH issues_a AS
(
SELECT a_prop
FROM ds_X x
)
, issues_b AS
(
SELECT key
, z.is_flagged as is_flagged
, some_prop
FROM ds_Z z
JOIN issues_a i_a
ON z.a_diff = i_a.a_prop
)
-- {{ run }}
UPDATE samples
SET error =
CASE
WHEN i_b.some_prop IS NULL THEN '#1 ...'
WHEN UPPER(i_b.is_flagged) != 'Y' THEN '#2 ...'
END
FROM samples s
left join issues_b i_b ON s.key = i_b.key;
Now I want enhance the whole thing, updating one more table in a consecutive way by enclosing parts of the query in BEGIN TRANSACTION and COMMIT, but don't get my head around the how of it. Tried enclosing the whole expression with the transaction parenthesis, but that didn't bring me any further.
Are there any other ways to achieve the above task - even without concatenating the consecutive updates in a transactional manner, though better it would be?
For abbreviation the task again: WITH <...>(...), <...>(...) UPDATE <... Using data from latter WITH> UPDATE <... using data from latter WITH>?
Hope you don't mind my poor grammar...
Related
This select statement takes a long time running, after my investigation I found that the problem un subquery, stored procedure, please I appreciate your help.
SELECT DISTINCT
COKE_CHQ_NUMBER,
COKE_PAY_SUPPLIER
FROM
apps.Q_COKE_AP_CHECKS_SIGN_STATUS_V
WHERE
plan_id = 40192
AND COKE_SIGNATURE__A = 'YES'
AND COKE_SIGNATURE__B = 'YES'
AND COKE_AUDIT = 'YES'
AND COKE_CHQ_NUMBER NOT IN (SELECT DISTINCT COKE_CHQ_NUMBER_DELIVER
FROM apps.Q_COKE_AP_CHECKS_DELIVERY_ST_V
WHERE UPPER(COKE_CHQ_NUMBER_DELIVER_STATUS) <> 'DELIVERED')
AND COKE_CHQ_NUMBER NOT IN (SELECT COKE_CHQ_NUMBER_DELIVER
FROM apps.Q_COKE_AP_CHECKS_DELIVERY_ST_V)
Well there are a few issues with your SELECT statement that you should address:
First let's look at this condition:
COKE_CHQ_NUMBER NOT IN (SELECT DISTINCT COKE_CHQ_NUMBER_DELIVER
FROM apps.Q_COKE_AP_CHECKS_DELIVERY_ST_V
WHERE UPPER(COKE_CHQ_NUMBER_DELIVER_STATUS) <> 'DELIVERED')
First you select DISTINCT cheque numbers with a not delivered status then you say you don't want this. Rather than saying I don't want non delivered it is much more readable to say I want delivered ones. However this is not really an issue but rather it would make your SELECT easier to read and understand.
Second let's look at your second cheque condition:
COKE_CHQ_NUMBER NOT IN (SELECT COKE_CHQ_NUMBER_DELIVER
FROM apps.Q_COKE_AP_CHECKS_DELIVERY_ST_V)
Here you want to exclude all cheques that have an entry in Q_COKE_AP_CHECKS_DELIVERY_ST_V. This makes your first DISTINCT condition redundant as whatever cheques numbers will bring back would be rejected by this second condition of yours. I do't know if Oracle SQL engine is clever enough to work out this redundancy but this could cause your slowness as SELECT distinct can take longer to run
In addition to this if you don't have them already I would recommend adding the following indexes:
CREATE INDEX index_1 ON q_coke_ap_checks_sign_status_v(coke_chq_number, coke_pay_supplier);
CREATE INDEX index_2 ON q_coke_ap_checks_sign_status_v(plan_id, coke_signature__a, coke_signature__b, coke_audit);
CREATE INDEX index_3 ON q_coke_ap_checks_delivery_st_v(coke_chq_number_deliver);
I called the index_1,2,3 for easy to read obviously not a good naming convention.
With this in place your select should be optimized to retrieve you your data in an acceptable performance. But of course it all depends on the size and the distribution of your data which is hard to control without performing specific data analysis.
looking to you code .. seems you have redundant where condition the second NOT IN implies the firts so you could avoid
you could also transform you NOT IN clause in a MINUS clause .. join the same query with INNER join of you not in subquery
and last be careful you have proper composite index on table
Q_COKE_AP_CHECKS_SIGN_STATUS_V
cols (plan_id,COKE_SIGNATURE__A , COKE_SIGNATURE__B, COKE_AUDIT, COKE_CHQ_NUMBER, COKE_PAY_SUPPLIER)
SELECT DISTINCT
COKE_CHQ_NUMBER,
COKE_PAY_SUPPLIER
FROM
apps.Q_COKE_AP_CHECKS_SIGN_STATUS_V
WHERE
plan_id = 40192
AND COKE_SIGNATURE__A = 'YES'
AND COKE_SIGNATURE__B = 'YES'
AND COKE_AUDIT = 'YES'
MINUS
SELECT DISTINCT
COKE_CHQ_NUMBER,
COKE_PAY_SUPPLIER
FROM apps.Q_COKE_AP_CHECKS_SIGN_STATUS_V
INNER JOIN (
SELECT COKE_CHQ_NUMBER_DELIVER
FROM apps.Q_COKE_AP_CHECKS_DELIVERY_ST_V
) T ON T.COKE_CHQ_NUMBER_DELIVER = apps.Q_COKE_AP_CHECKS_SIGN_STATUS_V
WHERE
plan_id = 40192
AND COKE_SIGNATURE__A = 'YES'
AND COKE_SIGNATURE__B = 'YES'
AND COKE_AUDIT = 'YES'
I have this below Oracle query. It takes ages to execute.
Select Distinct Z.WH_Source,
substr(Z.L_Y_Month,0,4) || '-' || substr(Z.L_Y_Month,5) Ld_Yr_Mth,
m.model_Name, p.SR, p.PLATE_NO, pp.value, z.CNT_number, z.platform_SR_number,
z.account_name, z.owner_name, z.operator_name, z.jetcare_expiry_date, z.wave,
z.address, z.country, substr(z.CNT_status, 10) ctstatus,
ALLOEM.GET_CNT_TYRE_SR#TNS_GG(z.CNT_number, Z.WH_Source, Z.L_Y_Month,
z.platform_SR_number, '¿')
product_SR_number
From MST.ROLE p
inner join MST.model m on m.model_id = p.model_id
left join MST.ROLEproperty pp on pp.ROLE_id = p.ROLE_id
and pp.property_lookup = 'SSG-WH-ENROLL'
left join alloem.Z_SSG_HM_LOG#TNS_GG z on z.camp_ac_ROLE_id = p.ROLE_id
Where
1 = 1 or z.L_Y_Month = 1
Order By 1, 2 desc, 3,4
If i remove this line,
ALLOEM.GET_CNT_TYRE_SR#TNS_GG(z.CNT_number, Z.WH_Source, Z.L_Y_Month,
z.platform_SR_number, '¿')
it executes very fast. But, I can't remove the line. Is there any way to make this query to execute fast.?
If i remove this line,
ALLOEM.GET_CNT_TYRE_SR#TNS_GG(z.CNT_number, Z.WH_Source,
Z.L_Y_Month, z.platform_SR_number, '¿')
it executes very fast. But, I can't remove the line. Is there any way to make this query to execute fast.?
Query tuning is a complex thing. Without table structures, indexes, execution plan or statistics it is very hard to provide one universal answer.
Anyway I would try scalar subquery caching(if applicable):
ALLOEM.GET_CNT_TYRE_SR#TNS_GG(z.CNT_number, Z.WH_Source, Z.L_Y_Month,
z.platform_SR_number, '¿')
=>
(SELECT ALLOEM.GET_CNT_TYRE_SR#TNS_GG(z.CNT_number, Z.WH_Source,Z.L_Y_Month,
z.platform_SR_number, '¿') FROM dual)
Also usage of DISTINCT may indicate some problems with normalization. If possible please fix underlying problem and remove it.
Finally you should avoid using positional ORDER BY (it is commom anti-pattern).
This:
alloem.Z_SSG_HM_LOG#TNS_GG
suggests that you fetch data over a database link. It is usually slower than fetching data locally. So, if you can afford it & if your query manipulates "static" data (i.e. nothing changes in Z_SSG_HM_LOG table frequently) and - even if it does - the amount of data isn't very high, consider creating a materialized view (MV) in schema you're connected to while running that query. You can even create index(es) on a MV so ... hopefully, everything will run faster without too much effort.
Ok I have a rather unique situation and I can't believe there is not a better way of doing this than my solution.
Requirements:
Table 2 - EpmTask_UserView_RM is a subset of table 1 -
MSP_EpmTask_UserView So while all the fields match Table 1 has many
more rows than table 2
Table 2 needs to get updated from table 1 based on the date a task has changed (We can't do a drop and replace) There are three cases:
Task updates where something has changed about the task (We will know based on the task date stamp)
Task Deletes where a task has been deleted
Task Adds where a new task exists
I have 3 different queries that do this and am thinking there is a better way.
**** DELETE Tasks from ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM table if no longer present in Production***/
USE [ProjectWebApp]
GO
DELETE FROM [dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM]
WHERE [dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM].TaskUID IN
(SELECT
/*Subquery to select all records in ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM NOT found in MSP_EpmTask_UserView_RM */
[ProjectWebApp].[dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM].[TaskUID]
FROM [ProjectWebApp].[dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM]
LEFT JOIN [MSPSPRO].[ProjectWebApp].[dbo].[MSP_EpmTask_UserView] as Prod
on Prod.TaskUID = [ProjectWebApp].[dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM].TASKuid
where Prod.TaskUID is NULL)
Query 2 the Update
UPDATE [dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM]
SET
[ProjectUID] = Source.[ProjectUID]
,[TaskUID] = Source.[TaskUID]
,[TaskName] = Source.[TaskName]
,[TaskIndex] = Source.[TaskIndex]
,[TaskOutlineLevel] = Source.[TaskOutlineLevel]
,[TaskOutlineNumber] = Source.[TaskOutlineNumber]
,[TaskStartDate] = Source.[TaskStartDate]
,[TaskFinishDate] = Source.[TaskFinishDate]
,[TaskActualStartDate] = Source.[TaskActualStartDate]
,[TaskActualFinishDate] = Source.[TaskActualFinishDate]
,[TaskPercentCompleted] = Source.[TaskPercentCompleted]
,[Health] = Source.[Health]
,[Milestone Significance Level] = Source.[Milestone Significance Level]
,[TaskModifiedDate] = Source.[TaskModifiedDate]
,[TaskBaseline1StartDate] = Source.[TaskBaseline1StartDate]
,[TaskBaseline1FinishDate] = Source.[TaskBaseline1FinishDate]
,[TaskBaseline1Duration] = Source.[TaskBaseline1Duration]
,[QueryTimestamp] = GetDate()
FROM [MSPSPRO].[ProjectWebApp].[dbo].[MSP_EpmTask_UserView] AS Source
WHERE Source.TaskUID = [dbo].[ZZZ_TEST_OF_UPDATE_MSP_EpmTask_UserView_RM].TaskUID
AND GetDate() - Source.TaskModifiedDate <= .01 -- Update any task changed in last 14 minutes (14 minutes = 1% of a full day, ie '.01')
GO
Task 3 the add
SELECT
[MSP_EpmProject_UserView].[ProjectUID]
,[TaskUID]
,[TaskName]
,[TaskIndex]
,[TaskOutlineLevel]
,[TaskOutlineNumber]
,[TaskStartDate]
,[TaskFinishDate]
,[TaskActualStartDate]
,[TaskActualFinishDate]
,[TaskPercentCompleted]
,[Health]
,[Milestone Significance Level]
,[TaskModifiedDate]
,[TaskBaseline1StartDate]
,[TaskBaseline1FinishDate]
,[TaskBaseline1Duration]
,GetDate() as QueryTimestamp
INTO [ProjectWebApp].[dbo].[MSP_EpmTask_UserView_RM]
FROM [MSPSPRO].[ProjectWebApp].[dbo].[MSP_EpmTask_UserView]
Inner Join [MSPSPRO].[ProjectWebApp].[dbo].[MSP_EpmProject_UserView]
on [MSP_EpmProject_UserView].projectUID = [MSP_EpmTask_UserView].ProjectUID
WHERE [SMO Programs] = 'SMO Day 1 Release Management'
AND [Milestone Significance Level] is not null
/*AND [TaskModifiedDate] > (getdate() - 1)*/
Thoughts?
This looks like an ideal situation for a MERGE statement. If you haven't used them much or at all, I'd strongly suggest this site as a primer.
A MERGE can carry out INSERT, UPDATE, and DELETE in one shot, in the right conditions. The basic idea is that you compare rows in two tables, your source and destination, and from that comparison (and potentially other conditions) you then take the appropriate action.
MERGE can perform very well because it carries out these actions in bulk - but do test it out. Sometimes people have found them to be slower than using the separate statements in some situations. Indexing correctly (Microsoft suggest indexing the columns used to join in both tables) can help immensely. Writing the MERGE statement correctly and well is important in terms of both getting the right result, and getting good performance - so definitely do your reading up if you haven't used them before. The above link is a good starter, but there are plenty of other articles around.
Firstly, I know ORA-000937 is a common issue, with an obvious answer, but I am yet to find any results that could point to a possible solution.
Quick Spec;
National TB/HIV report, based on patient medical records/encounters/visits and drug's provided. This is only a tiny portion of the report, which loops all patient drugs, and calculates most of it's figures off date calculations, we do not store historic/aggregated data, everything is aggregated when requested. I mention this because I expect a few suggestions to move away from GTT's and to rather use MVIEW's - I hear you, but no, not a solution.
Here is my problem, this is one of my queries populating a GTT, within a function, which stores aggregated results. I have structured my data collection in such away as to reduce server load as the medical table exceeds 12 million records. (Each patient has 3 per default).
Here is the GTT
CREATE GLOBAL TEMPORARY TABLE EKAPAII.TEMP_ART_VISIT_MEDS
(
EPISODE_ID NUMBER,
LAST_MEDS_DATE DATE
)
ON COMMIT DELETE ROWS
RESULT_CACHE (MODE DEFAULT)
NOCACHE;
CREATE UNIQUE INDEX EKAPAII.TEMP_ART_VISIT_MEDS_PK ON EKAPAII.TEMP_ART_VISIT_MEDS
(EPISODE_ID);
ALTER TABLE EKAPAII.TEMP_ART_VISIT_MEDS ADD (
CONSTRAINT TEMP_ART_VISIT_MEDS_PK
PRIMARY KEY
(EPISODE_ID)
USING INDEX EKAPAII.TEMP_ART_VISIT_MEDS_PK
ENABLE VALIDATE);
And my simple insert query
INSERT INTO temp_art_visit_meds (EPISODE_ID, LAST_MEDS_DATE)
SELECT episode_id, encounter_date + number_of_days
FROM ( SELECT enc_meds.episode_id,
MAX (enc_meds.encounter_date) encounter_date,
MAX (
CASE
WHEN (NVL (meds.number_of_days, 0) > 150)
THEN
90
ELSE
NVL (meds.number_of_days, 0)
END)
number_of_days
FROM temp_art_visit enc_meds,
vd_medication meds,
dl_drugs_episode_class dlc,
( SELECT latest_meds_visit.episode_id,
MAX (latest_meds_visit.encounter_date)
encounter_date
FROM temp_art_visit latest_meds_visit,
vd_medication latest_meds,
dl_drugs_episode_class dc
WHERE latest_meds_visit.encounter_id =
latest_meds.encounter_id
AND latest_meds.drug_id = dc.drug_id
AND dc.sd_drug_application_id = 8401
GROUP BY latest_meds_visit.episode_id) latest_meds
WHERE enc_meds.encounter_id = meds.encounter_id
AND enc_meds.episode_id = latest_meds.episode_id
AND enc_meds.encounter_date =
latest_meds.encounter_date
AND meds.drug_id = dlc.drug_id
AND dlc.sd_drug_application_id = 8401
AND meds.active_flag = 'Y'
GROUP BY enc_meds.episode_id);
Now my error, is ORA-000937 not a single-group group function, but if I run this query in a normal editor window it works, but I get ORA-000937 when executing the select query in the package body itself, calling the function does not return any error, even though I have an exception block to handle any errors.
Any help will do, I do understand that this errors could occur only at runtime, and not at compile time? Or is it the fact that I am running the query from the pl/sql block?
Toad for Oracle version 12.5 - in all it's glory. (sarcasm)
Again, pardon me if this has already been asked/answered.
EDIT - SOLUTION
So, after a few hours of trouble shooting, I was able to understand why this error is being generated. Firstly, the fixed query;
INSERT INTO temp_art_visit_meds (EPISODE_ID, LAST_MEDS_DATE)
SELECT enc_meds.episode_id ,
TRUNC( MAX (enc_meds.encounter_date)) + MAX (CASE WHEN (NVL (meds.number_of_days, 0) > 150) THEN 90 ELSE NVL (meds.number_of_days, 0) END) last_meds_date
FROM temp_art_visit enc_meds,
vd_medication meds,
dl_drugs_episode_class dlc,
( SELECT latest_meds_visit.episode_id,
MAX (latest_meds_visit.encounter_date) encounter_date
FROM temp_art_visit latest_meds_visit,
vd_medication latest_meds,dl_drugs_episode_class dc
WHERE latest_meds_visit.encounter_id = latest_meds.encounter_id
AND latest_meds.drug_id = dc.drug_id
AND dc.sd_drug_application_id = 8401
GROUP BY latest_meds_visit.episode_id) latest_meds
WHERE enc_meds.encounter_id = meds.encounter_id
AND enc_meds.episode_id = latest_meds.episode_id
AND enc_meds.encounter_date = latest_meds.encounter_date
AND meds.drug_id = dlc.drug_id
AND dlc.sd_drug_application_id = 8401
AND meds.active_flag = 'Y'
GROUP BY enc_meds.episode_id, meds.number_of_days, enc_meds.encounter_date;
It would appear that problem was due to the amount of sub-queries, I attempted to use different hint optimizers with no avail. If you look closely at the first query, you will notice I am basically aggregating results from aggregated results, so the column's encounter_date, episode_id, number_of_days are no longer 'available', so even if I added the appropriate GROUP BY clause to my last (outer) subquery, Oracle would not be able to group on those column names/identifiers.
I am not sure why this would fail only in a PACKAGE BODY, and did not return any SQLERR or SQLCODE when executed.
Happy days.
Keep in mind I am using SQL 2000
I have two tables.
tblAutoPolicyList contains a field called PolicyIDList.
tblLossClaims contains two fields called LossPolicyID & PolicyReview.
I am writing a stored proc that will get the distinct PolicyID from PolicyIDList field, and loop through LossPolicyID field (if match is found, set PolicyReview to 'Y').
Sample table layout:
PolicyIDList LossPolicyID
9651XVB19 5021WWA85, 4421WWA20, 3314WWA31, 1121WAW11, 2221WLL99 Y
5021WWA85 3326WAC35, 1221AXA10, 9863AAA44, 5541RTY33, 9651XVB19 Y
0151ZVB19 4004WMN63, 1001WGA42, 8587ABA56, 8541RWW12, 9329KKB08 N
How would I go about writing the stored proc (looking for logic more than syntax)?
Keep in mind I am using SQL 2000.
Select LossPolicyID, * from tableName where charindex('PolicyID',LossPolicyID,1)>0
Basically, the idea is this:
'Unroll' tblLossClaims and return two columns: a tblLossClaims key (you didn't mention any, so I guess it's going to be LossPolicyID) and Item = a single item from LossPolicyID.
Find matches of unrolled.Item in tblAutoPolicyList.PolicyIDList.
Find matches of distinct matched.LossPolicyID in tblLossClaims.LossPolicyID.
Update tblLossClaims.PolicyReview accordingly.
The main UPDATE can look like this:
UPDATE claims
SET PolicyReview = 'Y'
FROM tblLossClaims claims
JOIN (
SELECT DISTINCT unrolled.LossPolicyID
FROM (
SELECT LossPolicyID, Item = itemof(LossPolicyID)
FROM unrolling_join
) unrolled
JOIN tblAutoPolicyList
ON unrolled.ID = tblAutoPolicyList.PolicyIDList
) matched
ON matched.LossPolicyID = claims.LossPolicyID
You can take advantage of the fixed item width and the fixed list format and thus easily split LossPolicyID without a UDF. I can see this done with the help of a number table and SUBSTRING(). unrolling_join in the above query is actually tblLossClaims joined with the number table.
Here's the definition of unrolled 'zoomed in':
...
(
SELECT LossPolicyID,
Item = SUBSTRING(LossPolicyID,
(v.number - 1) * #ItemLength + 1,
#ItemLength)
FROM tblLossClaims c
JOIN master..spt_values v ON v.type = 'P'
AND v.number BETWEEN 1 AND (LEN(c.LossPolicyID) + 2) / (#ItemLength + 2)
) unrolled
...
master..spt_values is a system table that is used here as the number table. Filter v.type = 'P' gives us a rowset with number values from 0 to 2047, which is narrowed down to the list of numbers from 1 to the number of items in LossPolicyID. Eventually v.number serves as an array index and is used to cut out single items.
#ItemLength is of course simply LEN(tblAutoPolicyList.PolicyIDList). I would probably also declared #ItemLength2 = #ItemLength + 2 so it wasn't calculated every time when applying the filter.
Basically, that's it, if I haven't missed anything.
If the PolicyIDList field is a delimited list, you have to first separate the individual policy IDs and create a temporary table with all of the results. Next up, use an update query on the tblLossClaims with 'where exists (select * from #temptable tt where tt.PolicyID = LossPolicyID).
Depending on the size of the table/data, you might wish to add an index to your temporary table.