I have the following SQL-code in my (SAP IdM) Application:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
I already found the solution for the ORA-01841 problem, as it could either be a solution similar to MSSQLs try_to_date as mentioned here: How to handle to_date exceptions in a SELECT statment to ignore those rows?
or a solution where I change the code to something like this, to work soly on strings:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and v1.searchvalue<= to_char(sysdate+3,'YYYY-MM-DD')
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
So for the actually problem I have a solution.
But now I came into discussion with my customers and teammates why the error happens at all.
Basically for all entries of idmv_value_basic_activ that comply to the requirement of "attrname = 'Start_of_company_change'" we can be sure that those are dates. In addition, if we execute the query to check all values that would be delivered, all are in a valid format.
I learned in university that the DB-Engine could decide in which order it will run individual segments of a query. So for me the most logical explanation would be that, on the development environment (where we face the problem), the section " to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3” is executed before the section “attrname = 'Start_of_company_change'”
Whereas on the productive environment, where everything works like a charm, the segments are executed in the order that is descripted by the SQL Statement.
Now my Question is:
First: do I remember that right, since the teacher said that only once and at that time I could not really make sense out of it
And Second: Is this assumption of mine correct or is there another reason for the problem?
Borderinformation:
The Tool uses a kind of shifted data structure which is why there can be quite a few different types in the actual “Searchvalue” column of the idmv_value_basic_activ view. The datatype on the database layer is always a varchar one.
"the DB-Engine could decide in which order it will run individual segments of a query"
This is correct. A SQL query is just a description of the data you want and where it's stored. Oracle will calculate an execution plan to retrieve that data as best it can. That plan will vary based on any number of factors, like the number of actual rows in the table and the presence of indexes, so it will vary from environment to environment.
So it sounds like you have an invalid date somewhere in your table, so to_date raises an exception. You can use validate_conversion to find it.
Related
We are facing a performance issue while executing a stored procedure. It usually takes between 10-15 minutes to run, but sometimes it takes up to more than 30 minutes to execute.
We captured visualize plan execute files for the Normal run and Long run cases.
By checking the visualized plan we came to know that, one particular Insert block of code takes extra time in the long run. And by checking
"EXPLAIN PLAN FOR SQL PLAN CACHE ENTRY <plan_id> "
the table we found that the order of execution differs in the long run.
This is the block which takes extra time to run sometimes.
INSERT INTO #TMP_DATI_SALDI_LORDI_BASE (
"COD_SCENARIO","COD_PERIODO","COD_CONTO","COD_DEST1","COD_DEST2","COD_DEST3","COD_DEST4","COD_DEST5"
,"IMPORTO","COD_VALUTA","IMPORTO_VALUTA_ORIGINARIA","COD_VALUTA_ORIGINARIA","NOTE"
)
( SELECT
SCEN_P.SCENARIO
,SCEN_P.PERIOD
,ACCOUT_ADJ.ATTRIBUTO1 AS "COD_CONTO"
,DATAS_rev.COD_DEST1
,DATAS_rev.COD_DEST2
,DATAS_rev.COD_DEST3
,__typed_NString__($1, 50)
,'RPT_NON'
,SUM(
CASE WHEN INFO.INCOT = 'FOB' THEN
CASE ACCOUT_rev.ATTRIBUTO1 WHEN 'CalcInsurance' THEN
0
ELSE
DATAS_rev.IMPORTO
END
ELSE
DATAS_rev.IMPORTO
END
* (DATAS_ADJ.IMPORTO - DATAS.IMPORTO)
)
,DATAS_rev.COD_VALUTA
,SUM(
CASE WHEN INFO.INCOT = 'FOB' THEN
CASE ACCOUT_rev.ATTRIBUTO1 WHEN 'CalcInsurance' THEN
0
ELSE
DATAS_rev.IMPORTO_VALUTA_ORIGINARIA
END
ELSE
DATAS_rev.IMPORTO_VALUTA_ORIGINARIA
END
* (DATAS_ADJ.IMPORTO_VALUTA_ORIGINARIA - DATAS.IMPORTO_VALUTA_ORIGINARIA)
)
,DATAS_rev.COD_VALUTA_ORIGINARIA
,'CPM_SP_CACL_FY_E3 Parts Option ADJ'
FROM #TMP_TAGERT_SCEN_P SCEN_P
INNER JOIN #TMP_DATI_SALDI_LORDI_BASE DATAS_rev
ON DATAS_rev.COD_SCENARIO = SCEN_P.SCENARIO
AND DATAS_rev.COD_PERIODO = SCEN_P.PERIOD
AND LEFT(DATAS_rev.COD_DEST3, 1) = 'O'
INNER JOIN CONTO ACCOUT_rev
ON ACCOUT_rev.COD_CONTO = DATAS_rev.COD_CONTO
AND ACCOUT_rev.ATTRIBUTO1 IN ('CalcFOB','CalcInsurance') --FOB,Insurance(Ocean freight is Nothing by Option)
INNER JOIN #DSL DATAS
ON DATAS.COD_SCENARIO = 'LAUNCH'
AND DATAS.COD_PERIODO = 12
AND DATAS.COD_DEST1 = 'NC'
AND DATAS.COD_DEST2 = 'NC'
AND DATAS.COD_DEST3 = 'F001'
AND DATAS.COD_DEST4 = DATAS_rev.COD_DEST4
AND DATAS.COD_DEST5 = 'INP'
INNER JOIN CONTO ACCOUT
ON ACCOUT.COD_CONTO = DATAS.COD_CONTO
AND ACCOUT.ATTRIBUTO2 = 'E3'
INNER JOIN CONTO ACCOUT_ADJ
ON ACCOUT_ADJ.ATTRIBUTO3 = DATAS.COD_CONTO
AND ACCOUT_ADJ.ATTRIBUTO2 = 'HE3'
INNER JOIN #DSL DATAS_ADJ
ON LEFT(DATAS_ADJ.COD_SCENARIO,4) = LEFT(SCEN_P.SCENARIO,4)
AND DATAS_ADJ.COD_PERIODO = 12
AND DATAS_ADJ.COD_DEST1 = DATAS.COD_DEST1
AND DATAS_ADJ.COD_DEST2 = DATAS.COD_DEST2
AND DATAS_ADJ.COD_DEST3 = DATAS.COD_DEST3
AND DATAS_ADJ.COD_DEST4 = DATAS.COD_DEST4
AND DATAS_ADJ.COD_DEST5 = DATAS.COD_DEST5
AND DATAS_ADJ.COD_CONTO = ACCOUT_ADJ.COD_CONTO
LEFT OUTER JOIN #TMP_KDPWT_INCOTERMS INFO
ON INFO.P_CODE = DATAS.COD_DEST4
GROUP BY
SCEN_P.SCENARIO,SCEN_P.PERIOD,ACCOUT_ADJ.ATTRIBUTO1,DATAS_rev.COD_DEST1,DATAS_rev.COD_DEST2
,DATAS_rev.COD_DEST3, DATAS.COD_DEST4,DATAS_rev.COD_VALUTA,DATAS_rev.COD_VALUTA_ORIGINARIA,INFO.INCOT
)
I will share the order of execution details also for normal and long run case.
Could someone please help us to overcome this issue? And also we don't know how to fix the order of the join execution. Is there any way to fix the join order execution, Please guide us.
Thanks in advance
Vinothkumar
Without a lot more detailed information, there is no way to tell exactly why your INSERT statement shows this alternating runtime behaviour.
Based on my experience, such an analysis can take quite some time and there are only few people available that are capable to perform it. If you can get someone like that to look at this, make sure to understand and learn.
What I can tell from the information shared is this
using temporary tables to structure a multi-stage data flow is the wrong thing to do on SAP HANA. Instead, use table variables in SQLScript.
if you insist on using the temporary tables, make them at least column tables; this will allow to avoid a need for some internal data materialisation.
when using joins make sure that the joined columns are of the same data type. The explain plan is full of TO_INT(), TO_DECIMAL(), and other conversion functions. Those take time, memory, and make it hard for the optimiser(s) to estimate cardinalities.
as the statement uses a lot of temporary tables, the different join orders can easily result from different volumes of data that was present when the SQL was parsed, prepared and optimised. One option to avoid this is to have HANA ignore any cached plans for the statement. The documentation has the HINTS for that.
And that is about what I can say about this with the available information.
I am a SQL server guy, with limited Oracle coding. I have inherited a bunch of views that I need to convert to ANSI. What I am looking for is someone to educate me, if possible, on a systematic way to do this in steps. I have junior guys that would read and understand TSQL style code, instead of the way it is. I want to take this knowledge, and have them start working on them; there are many. A conversion tool will not work, we don't have time to play with one, no budget to buy it, and our IT department will not let us install anything. (We have to do this manually)
I would like to know the steps (systematic approach) based on interpreting the code, not learn the data or try to understand why the existing code was written the way it was.
For example, are these the steps:
Reorder the tables to match how they are in the where clause.
Separate the lines in the where clause based on their table, so that it is easier to read.
Replace the Oracle join operators with left, right, outer, etc.
Leave the lines with defined values in the where clause, or should I include them with an "AND" in the table join.
This task will be done by junior SQL server guys, so it has to be as simple and straight forward as possible.
FROM WORK MECHANISMS WM,
WT_MANPOWER_RESOURCES WTMR,
LOGICAL_ITEMS LI,
APSWHS.WMX_LOG_REL_ELEM WLRE,
WMECH_DSGN_COMP_ELEMENTS WDCE,
WORK_TASKS WT,
persons_v per,
APSWHS.WMX_WO_WF_STATUS WFA,
RT_DETAILS RTD
WHERE
WFA.WF_SEQ > 639
AND WTMR.WTASK_WMECH_DB_ID = WM.DB_ID
AND WTMR.WTASK_DB_ID = 0
AND WTMR.WTMANPOW_TYPE = ‘LEAD’
AND WT.WMECH_DB_ID = WM.DB_ID
AND WT. DB_ID= 0
AND WFA.WM_DB_ID = WM.DB_ID
AND PER.per_db_id = wm.assigned_to_per_db_id
AND RTD.WMECH_DB_ID = WM.DB_ID
AND WDCE .WMECH_DB_ID = WFA.WM_DB_ID
AND WDCE.LITM_ID = WLRE.LITM_ID
AND WDCE.LITM_ID = LI.ID
AND WDCE.PRIMARY_DCID_FLAG = ‘Y’
I'm assuming that you want to migrate from table lists (join predicates in WHERE clause) to ANSI JOIN syntax. This has nothing to do with T-SQL.
Ideally, you'll take an ERD and check in what order the tables should be listed, visually. I personally find that easier than from mere text. Although, it is possible to do with plain text as well. Here are the steps:
Take the first table and all its non-join predicates:
FROM WORK_MECHANISMS WM
WHERE 1 = 1 -- No predicates on this table
Take the next table that you can join to the first one, and all of its join and non-join predicates
FROM WORK_MECHANISMS WM
JOIN WT_MANPOWER_RESOURCES WTMR
ON WTMR.WTASK_WMECH_DB_ID = WM.DB_ID
WHERE WTMR.WTASK_DB_ID = 0
AND WTRM.WTMANPOW_TYPE = 'LEAD'
And the next table...
Observe that this isn't the next table in your original table list, but another one, i.e. the next one that can be joined to the existing join graph without creating a cartesian product. In particular, I skipped (for now) LOGICAL_ITEMS and APSWHS.WMX_LOG_REL_ELEM and WMECH_DSGN_COMP_ELEMENTS. I will add them to the graph later.
FROM WORK_MECHANISMS WM
JOIN WT_MANPOWER_RESOURCES WTMR
ON WTMR.WTASK_WMECH_DB_ID = WM.DB_ID
JOIN WORK_TASKS WT
ON WT.WMECH_DB_ID = WM.DB_ID
WHERE WTMR.WTASK_DB_ID = 0
AND WTRM.WTMANPOW_TYPE = 'LEAD'
AND WT.DB_ID = 0
And the next table...
You continue adding tables to your new statement, until you've added all tables. If you ever encounter a (+) operator, "just" get the LEFT JOIN semantics right.
They return the same count of rows, but I'm not sure if one is an accident waiting to happen, or one is simply "the preferred method":
SELECT duckbill.id, duckbill.pack_size, duckbill.description, duckbill.platypus_id, duckbill.department, duckbill.subdepartment, duckbill.unit_cost, duckbill.unit_list, duckbill.open_qty, duckbill.UPC_code, duckbill.UPC_pack_size, duckbill.crv_id, duckbill_platypuss.platypus_item
FROM duckbill
INNER JOIN duckbill_platypuss ON duckbill.platypus_id = duckbill_platypuss.platypus_id
SELECT duckbill.id, duckbill.pack_size, duckbill.description, duckbill.platypus_id, duckbill.department, duckbill.subdepartment, duckbill.unit_cost, duckbill.unit_list, duckbill.open_qty, duckbill.UPC_code, duckbill.UPC_pack_size, duckbill.crv_id, duckbill_platypuss.platypus_item
FROM duckbill, duckbill_platypuss
WHERE (duckbill.platypus_id = duckbill_platypuss.platypus_id)
So the reason there's a difference is due to the order of execution. You may not notice the difference on some table combinations, but on very large tables, you generally want to filter with the JOIN first.
This post gives some good information toward that end it looks like: Order Of Execution of the SQL query
I recently upgraded MySQL to 5.1.41. Before the upgrade the following SQL worked (or at least I thought I remembered it working...it has been a few weeks since designing this...). Now the SQL gives me an error stating that the "archived" column is ambiguous. How can I write this differently, or is there a different problem I'm not aware of?
I simply want to return the "unit_id", "lease_count" (stored in another table with a unit_id that should correspond with the "a.unit_id"), and "lease_archived_count (stored in another table with a unit_id that should correspond with the "a.unit_id").
SELECT a.unit_id,
(SELECT count(*) FROM o_leases WHERE unit_id = a.unit_id AND archived = 0) as lease_count,
(SELECT count(*) FROM o_leases WHERE unit_id = a.unit_id AND archived = 1) as lease_archive_count
FROM p_unit a, properties b, portfolio c
WHERE a.property_id = b.properties_id
AND b.portfolio_id = c.portfolio_id
AND a.archived = 0
Thanks for your help.
There can only be one place where this error is referring to. I suggest to give the tables in the sub select also an alias:
(SELECT count(*) FROM o_leases o WHERE o.unit_id = a.unit_id AND o.archived = 0) as lease_count,
(SELECT count(*) FROM o_leases o WHERE o.unit_id = a.unit_id AND o.archived = 1) as lease_archive_count
It seems that it collides with the archived field of p_unit.
I would guess the error is because o_leases is not the only table that has an archived field. p_unit Or properties or portfolio must have that column as well.
However you have several other things that should also be fixed. First in most databases correlated subqueries are poor performers and should be rewritten as joins (although in this case you probably only need a case statement and a join). You should notgenerally even consider writing correlated subqueries.
Second, you should NEVER use implicit joins. Extremely bad idea that leads to poor maintainability and accidental cross joins as well as a generally poor understanding of joins. Bad bad bad. This kind of code was replaced 18 years ago, would you still be using C# or Java code relaced witha better method even five years ago?
I've got an Access MDB I use for reporting that has linked table views from SQL Server 2005. I built a query that retrieves information off of a PO table and categorizes the line item depending on information from another table. I'm relatively certain the query was fine until approximately a month ago when we shifted from compatibility mode 80 to 90 on the Server as required by our primary application (which creates the data). I can't say this with 100% certainty, but that is the only major change made in the past 90 days. We noticed that suddenly data was not showing up in the query making the reports look odd.
This is a copy of the failing query:
SELECT dbo_porel.jobnum, dbo_joboper.opcode, dbo_porel.jobseqtype,
dbo_opmaster.shortchar01,
dbo_porel.ponum, dbo_porel.poline, dbo_podetail.unitcost
FROM ((dbo_porel
LEFT JOIN dbo_joboper ON (dbo_porel.assemblyseq = dbo_joboper.assemblyseq)
AND (dbo_porel.jobseq = dbo_joboper.oprseq)
AND (dbo_porel.jobnum = dbo_joboper.jobnum))
LEFT JOIN dbo_opmaster ON dbo_joboper.opcode = dbo_opmaster.opcode)
LEFT JOIN dbo_podetail ON (dbo_porel.poline = dbo_podetail.poline)
AND (dbo_porel.ponum = dbo_podetail.ponum)
WHERE (dbo_porel.jobnum="367000003")
It returns the following:
jobnum opcode jobseqtype shortchar01 ponum poline unitcost
367000003 S 6624 2 15
The query normally should have displayed a value for opcode and shortchar01. If I remove the linked table dbo_podetail, it properly displays data for these fields (although I obviously don't have unitcost anymore). At first I thought it might be a data issue, but I found if I nested the query and then linked the table, it worked fine.
For example the following code works perfectly:
SELECT qryTest.*, dbo_podetail.unitcost
FROM (
SELECT dbo_porel.jobnum, dbo_joboper.opcode, dbo_porel.jobseqtype,
dbo_opmaster.shortchar01, dbo_porel.ponum, dbo_porel.poline
FROM (dbo_porel
LEFT JOIN dbo_joboper ON (dbo_porel.jobnum=dbo_joboper.jobnum)
AND (dbo_porel.jobseq=dbo_joboper.oprseq)
AND (dbo_porel.assemblyseq=dbo_joboper.assemblyseq))
LEFT JOIN dbo_opmaster ON dbo_joboper.opcode=dbo_opmaster.opcode
WHERE (dbo_porel.jobnum="367000003")
) As qryTest
LEFT JOIN dbo_podetail ON (qryTest.poline = dbo_podetail.poline)
AND (qryTest.ponum = dbo_podetail.ponum)
I'm at a loss for why it works in the latter case and not in the first case. Worse yet, it seems to work intermittently for some records and not for others (it's consistent about the ones it does and does not work for).
Do any of you experts have any ideas?
You definitely need to use subqueries for multiple left/right joins in Access.
I think it's a limitation of the Jet optimizer that gets confused if you're just chaining left/right joins.
You can see that this is a recurrent problem that surfaces often.
I'm always confused by Access' use of brackets in joins. Try stripping out the extra brackets.
FROM
dbo_porel
LEFT JOIN
dbo_joboper ON (dbo_porel.assemblyseq = dbo_joboper.assemblyseq)
AND (dbo_porel.jobseq = dbo_joboper.oprseq)
AND (dbo_porel.jobnum = dbo_joboper.jobnum)
LEFT JOIN
dbo_opmaster ON (dbo_joboper.opcode = dbo_opmaster.opcode)
LEFT JOIN
dbo_podetail ON (dbo_porel.poline = dbo_podetail.poline)
AND (dbo_porel.ponum = dbo_podetail.ponum)
OK the above doesn't work - Sorry I give up