I'm getting what appears to be a bizarre bug with Oracle 11.2.0.2.0. Everything tells me that this must be a bug, but I haven't been able to find mentions of it and it's a common enough use case that you would think that it would have been found and fixed long before now. Things work fine in the RDS version of Oracle (12-something) which lends credence to my opinion that it must be a bug.
I've been able to reproduce the issue with nothing but selects from dual, so no tables need to be added to test this. Consider the following SQL:
select t2.*
from (
select 1 as id from "DUAL"
) t1
left outer join (
select 2 as id, 10 as foo from "DUAL"
) t2 on t1.id = t2.id
left outer join (
select
1 as id,
-- case when 1 = 1 then 10 else 7 end as foo,
2 as bar
from "DUAL"
) t3 on t1.id = t3.id
It produces exactly the results you'd expect:
id foo
-- ---
null null
Since the first outer join fails (there is no row in t2 with an id of 1), the columns are null as they should be. Notice I'm not even using the results of the second join. But if I uncomment the case statement, the results change!
id foo
-- ---
2 10
Here are some things I've observed so far:
The results change if there are 1 or more additional outer joins in the query which also contain a case statement.
It doesn't matter if the second outer join passes or fails.
It only appears that literals in t2's select statement have the issue. That is, if the select statement were something like select id, 10 as foo from "MY_TABLE", id would be null as you'd expect but foo still is 10.
There are a couple of ways you can answer this question:
If this is actually not a bug, can you explain why this is occurring and cite documentation explaining the behavior?
If this is a bug, can you link to references to the bug and in what version of 11g that it was resolved in (if any)?
What I don't want are "this must be a bug because it violates such-and-such spec" answers. I already know this and I already completely agree with you. I need to know why it's happening (if it's not a bug) or where/when it was fixed if it is a bug.
If you have a version of Oracle where both versions of the query work properly, please leave a comment with the version number. It may be useful to find out what bug this was and when it was fixed.
Related
I have come across two versions of an SQLRPGLE program and saw a change in the code as below:
Before:
Exec Sql SELECT 'N'
INTO :APRFLG
FROM LG751F T1
INNER JOIN LG752F T2
ON T1.ISBOLN = T2.IDBOLN AND
T1.ISITNO = T2.IDMDNO
WHERE T2.IDVIN = :M_VIN AND
T1.ISAPRV <> 'Y';
After:
Exec Sql SELECT case
when T1.ISAPRV <> 'Y' then 'N'
else T1.ISAPRV
end as APRFLG
INTO :APRFLG
FROM LG751F T1
join LG752F T2
ON T1.ISBOLN = T2.IDBOLN AND
T1.ISITNO = T2.IDMDNO
WHERE T2.IDVIN = :M_VIN AND
T1.ISAPRV <> 'Y'
group by T1.ISAPRV;
Could you please tell me if you see any difference in how the codes would work differently? The second SQL has a group by which is supposed to be a fix to avoid -811 SQLCod error. Apart from this, do you guys spot any difference?
They are both examples of poor coding IMO.
The requirement to "remove duplicates" is often an indication of a bad statement design and/or a bad DB design.
You appear to be doing an existence check, in which case you should be making use of the EXISTS predicate.
select 'N' into :APRFLG
from sysibm.sysdummy1
where exists (select 1
FROM LG751F T1
INNER JOIN LG752F T2
ON T1.ISBOLN = T2.IDBOLN
AND T1.ISITNO = T2.IDMDNO
WHERE
T2.IDVIN = :M_VIN
AND T1.ISAPRV <> 'Y');
As far as the original two statements, besides the group by, the only real difference is moving columns from the JOIN clause to the WHERE clause. However, the query engine in Db2 for i will rewrite both statements equivalently and come up with the same plan; since an inner join is used.
EDIT : as Mark points out, there JOIN and WHERE are the same in both the OP's statements. But I'll leave the statement above in as an FYI.
I don't find a compelling difference, other that the addition of the group by, that will have the effect of suppressing any duplicate rows that might have been output.
It looks like the developer intended for the query to be able to vary its output to be sometimes Y and sometimes N, but forgot to remove the WHERE clause that necessarily forces the case to always be true, and hence it to always output N. This kind of pattern is usually seen when the original report includes some spec like "don't include managers in the employee Sakarya report" and that then changes to "actually we want to know if the employee is a manager or not". What was a "where employee not equal manager" becomes a "case when employee is manager then... else.." but the where clause needs removing for it to have any effect
The inner keyword has disappeared from the join statement, but overall this should also be a non-op
Another option is just to use fetch first row only like this:
Exec Sql
SELECT 'N'
INTO :APRFLG
FROM LG751F T1
JOIN LG752F T2
ON T1.ISBOLN = T2.IDBOLN AND
T1.ISITNO = T2.IDMDNO
WHERE T2.IDVIN = :M_VIN AND
T1.ISAPRV <> 'Y'
fetch first row only;
That makes it more obvious that you only want a single row rather than trying to use grouping which necessitates the funky do nothing CASE statement. But I do like the EXISTS method Charles provided since that is the real goal, and having exists in there makes it crystal clear.
If your lead insists on GROUP BY, you can also GROUP BY 'N' and still leave out the CASE statement.
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 8 years ago.
Improve this question
Background
Recently my company upgraded from Oracle 11g to Exadata. All production objects and data were successfully migrated to the new system and were verified to be a 1:1 match between systems. Immediately after the first daily set of ETL processes were completed on our new system we discovered our reporting tables were significantly smaller than expected. Upon further investigation we discovered that a batch ID being added to LEFT OUTER joins were causing the problem where they had worked perfectly on 11g.
Question
Why would the following query be treated as a LEFT OUTER JOIN on 11g, but an INNER JOIN on Exadata?
SELECT
*
FROM DIM_CALL CALLS
LEFT OUTER JOIN FACT_ACTVY_MNGMT ACTVY_MNGMT
ON ACTVY_MNGMT.CALL_KEY = CALLS.CALL_KEY
AND ACTVY_MNGMT.BATCH_ID = 20141112
LEFT OUTER JOIN DIM_ACTVY ACTVY
ON ACTVY.ACTVY_KEY = ACTVY_MNGMT.ACTVY_KEY
AND ACTVY_MNGMT.BATCH_ID = 20141112
Update
It appears there was either a typo in the query used in the ETL process or the original developer just overlooked this case. If you look at the last join you'll notice that the join is on the ACTVY_KEY and BATCH_ID. The issue is that the BATCH_ID it's referencing is from the ACTVY_MNGMT table. The database essentially treats this as a WHERE clause, so any case in which CALL_KEY is NULL results in a failure.
There is a minor programming mistake here, but it also looks like there is an optimizer bug. Think of a LEFT JOIN as operating on the previous set, not just the previous table. Adding a duplicate predicate to a different LEFT JOIN should not make any difference.
My guess is there is an optimizer or parser bug with some fancy Exadata feature, such as smart scan.
This code is similar to your problem. However, it does not reproduce the error for me, on 11.2.0.3 EE, on Solaris. And it may not even reproduce the error for you, on Exadata. It's just meant to show that the problem you've described is a bug. Avoiding the bug, like you're doing, is usually the best approach. But you may also want to create a service request with Oracle support to look into the issue. That same bug may be affecting other code in less obvious ways.
with table1 as (select '1' a, '1' b from dual),
table2 as (select '1' a, '2' b from dual),
table3 as (select '1' a, '2' b from dual)
select *
from table1
left join table2
on table1.a = table2.a
and table2.b = 3
left join table3
on table2.a = table3.a;
with table1 as (select '1' a, '1' b from dual),
table2 as (select '1' a, '2' b from dual),
table3 as (select '1' a, '2' b from dual)
select *
from table1
left join table2
on table1.a = table2.a
and table2.b = 3
left join table3
on table2.a = table3.a
--This predicate is logically redundant but does *not* change results.
and table2.b = 3;
Both queries return this result on Oracle 11.2.0.3, EE, on Solaris:
A B A B A B
- - - - - -
1 1
It appears there was either a typo in the query used in the ETL process or the original developer just overlooked this case. If you look at the last join you'll notice that the join is on the ACTVY_KEY and BATCH_ID. The issue is that the BATCH_ID it's referencing is from the ACTVY_MNGMT table. The database essentially treats this as a WHERE clause, so any case in which CALL_KEY is NULL results in a failure.
I have an update query for an Oracle SQL db. Upon execution the query returns ORA-00904: "t1"."sv_id": invalid identifier
So, why do I get an "invalid identifier" error message although the column exists?
Here is the complete query (replaced actual table and column names by dummies in np++)
UPDATE table_1 t1 SET (type) =
CASE
WHEN
((SELECT COUNT(dateCheck.id) FROM table_2 dateCheck
WHERE dateCheck.s_id = t1.s_id AND dateCheck.sv_id = t1.sv_id) = 0)
THEN
(SELECT sv.type FROM table_3 sv WHERE sv.id = t1.sv_id)
ELSE
(SELECT type FROM
(SELECT d.type as type FROM table_2 d
WHERE d.s_id = t1.s_id AND d.sv_id = t1.sv_id
ORDER BY d.creationTimestamp ASC)
WHERE ROWNUM = 1)
END
Now I don't understand why that error occurs.
Here is what I already know:
The Queries in the CASE statement work when executed separately, provided they are wrapped into a query that provides table_1 t1 for sure.
t1.s_id seems to work since oracle doesn't complain about that. When i change it to a column that really doesn't exist, oracle starts complaining about that non existent column before returning something about t1.sv_id. So somehow the alias might work, although I'm not sure about it.
I'm 100% sure that the column t1.sv_id exists and no typo was made. Executed a query on t1 directly and doublechecked everything in notepad by marking all occurences.
An (completely unrelated) update query like the following works as well (note the alias t1 is used in the select query). Don't assume table_1/2 to be the same as in the update query above, just reused the names. This should just illustrate that I successfully used an alias in an update query before.
update table_1 t1 set (t2_id) = (select id from table_2 t2 where t1.id = t2.t1_id)
UPDATE
Thx a lot for pointing me to the "you don't have access to alises in deeper suquery layers" issue. That got me on track again pretty fast.
So here is the query I ended up with. This seems to work fine. Eliminates the acces to t1 in the deeper layers and selects the oldest row, so that the same result should be returned from the query I expected from the original query in the ELSE part.
UPDATE table_1 t1 SET (type) =
CASE
WHEN
((SELECT COUNT(dateCheck.id) FROM table_2 dateCheck
WHERE dateCheck.s_id = t1.s_id AND dateCheck.sv_id = t1.sv_id) = 0)
THEN
(SELECT sv.type FROM table_3 sv WHERE sv.id = t1.sv_id)
ELSE
(SELECT d.type as type FROM table_2 d
WHERE d.s_id = t1.s_id
AND d.sv_id = t1.sv_id
AND d.creation = (SELECT MIN(id.creation) FROM table_2 id
WHERE d.s_id = id.s_id AND d.sv_id = id.sv_id))
END
You can't reference a table alias in a subquery of a subquery; the alias doesn't apply (or doesn't exist, or isn't in scope, depending on how you prefer to look at it). With the code you posted the error is reported against line 11 character 24, which is:
(SELECT type FROM
(SELECT d.type as type FROM table_2 d
WHERE d.s_id = t1.s_id AND d.sv_id = t1.sv_id
^^^^^^^^
If you change the t1.s_id reference on the same line to something invalid then the error doesn't change and is still reported as ORA-00904: "T1"."SV_ID": invalid identifier. But if you change the same reference on line 5 instead to something like
((SELECT COUNT(dateCheck.id) FROM table_2 dateCheck
WHERE dateCheck.s_id = t1.s_idXXX AND dateCheck.sv_id = t1.sv_id) = 0)
... then the error changes to ORA-00904: "T1"."S_IDXXX": invalid identifier. This is down to how the statement is being parsed. In your original version the subquery in the WHEN clause is value, and you only break it by changing that identifier. The subquery in the ELSE is also OK. But the nested subquery in the ELSE has the problem, and changing the t1.s_id in that doesn't make any difference because the parser reads that part of the statement backwards (I don't know, or can't remember, why!).
So you have to eliminate the nested subquery. A general approach would be to make the whole CASE an inline view which you can then join using s_id and sv_id, but that's complicated as there may be no matching table_2 record (based on your count); and there may be no s_id value to match against as that isn't being checked in table_3.
It isn't clear if there will always be a table_3 record even then there is a table_2 record, or if they're mutually exclusive. If I've understood what the CASE is doing then I think you can use an outer join between those two tables and compare the combined data with the row you're updating, but because of that ambiguity it needs to be a full outer join. I think.
Here's a stab at using that construct with a MERGE instead of an update.
MERGE INTO table_1 t1
USING (
SELECT t2.s_id,
coalesce(t2.sv_id, t3.id) as sv_id,
coalesce(t2.type, t3.type) as type,
row_number() over (partition by t2.s_id, t2.sv_id
order by t2.creationtimestamp) as rn
FROM table_2 t2
FULL OUTER JOIN table_3 t3
ON t3.id = t2.sv_id
) tmp
ON ((tmp.s_id is null OR tmp.s_id = t1.s_id) AND tmp.sv_id = t1.sv_id AND tmp.rn = 1)
WHEN MATCHED THEN UPDATE SET t1.type = tmp.type;
If there will always be a table_3 record then you could use that as the driver and have a left outer join to table_2 instead, but hard to tell which might be appropriate. So this is really just a starting point.
SQL Fiddle with some made-up data that I believe would have hit both branches of your case. More realistic data would expose the flaws and misunderstandings, and suggest a more robust (or just more correct) approach...
Your query and your analysis seems sound to me. I have no solution but a few things you can try to maybe trigger something that explains this odd behavior:
Quote the column (just in case it happens to be a SQL keyword).
Use table_1.sv_id - this works as long as the whole query contains this table only once.
Make sure that the alias t1 exists only once
Run the query with a query tool like SQuirrel SQL - the tool can examine the exact position where Oracle reports the problem. Maybe it's in a different place of the query than you think
Check () and make sure they are around the parts where they should be.
Swap the order of expressions around =
Problem
I need to better understand the rules about when I can reference an outer table in a subquery and when (and why) that is an inappropriate request. I've discovered a duplication in an Oracle SQL query I'm trying to refactor but I'm running into issues when I try and turn my referenced table into a grouped subQuery.
The following statement works appropriately:
SELECT t1.*
FROM table1 t1,
INNER JOIN table2 t2
on t1.id = t2.id
and t2.date = (SELECT max(date)
FROM table2
WHERE id = t1.id) --This subquery has access to t1
Unfortunately table2 sometimes has duplicate records so I need to aggregate t2 first before I join it to t1. However when I try and wrap it in a subquery to accomplish this operation, suddenly the SQL engine can't recognize the outer table any longer.
SELECT t1.*
FROM table1 t1,
INNER JOIN (SELECT *
FROM table2 t2
WHERE t1.id = t2.id --This loses access to t1
and t2.date = (SELECT max(date)
FROM table2
WHERE id = t1.id)) sub on t1.id = sub.id
--Subquery loses access to t1
I know these are fundamentally different queries I'm asking the compiler to put together but I'm not seeing why the one would work but not the other.
I know I can duplicate the table references in my subquery and effectively detach my subquery from the outer table but that seems like a really ugly way of accomplishing this task (what with all the duplication of code and processing).
Helpful References
I found this fantastic description of the order in which clauses are executed in SQL Server: (INNER JOIN ON vs WHERE clause). I'm using Oracle but I would think that this would be standard across the board. There is a clear order to clause evaluation (with FROM being first) so I would think that any clause occuring further down the list would have access to all information previously processed. I can only assume my 2nd query somehow changes that ordering so that my subquery is being evaluated too early?
In addition, I found a similar question asked (Referencing outer query's tables in a subquery
) but while the input was good they never really explained why he couldn't do what he is doing and just gave alternative solutions to his problem. I've tried their alternate solutions but it's causing me other issues. Namely, that subquery with the date reference is fundamental to the entire operation so I can't get rid of it.
Questions
I want to understand what I've done here... Why can my initial subquery see the outer table but not after I wrap the entire statement in a subquery?
That said, if what I'm trying to do can't be done, what is the best way of refactoring the first query to eliminate the duplication? Should I reference table1 twice (with all the duplication that requires)? Or is there (probably) a better way of tackling this problem?
Thanks in advance!
------EDIT------
As some have surmised these queries above are not the actually query I'm refactoring but an example of the problem I'm running into. The query I'm working with is a lot more complicated so I'm hesitant to post it here as I'm afraid it will get people off track.
------UPDATE------
So I ran this by a fellow developer and he had one possible explanation for why my subquery is losing access to t1. Because I'm wrapping this subquery in a parenthesis, he thinks that this subquery is being evaluated before my table t1 is being evaluated. This would definitely explain the 'ORA-00904: "t1"."id": invalid identifier' error I've been receiving. It would also suggest that like arithmetic order of operations, that adding parens to a statement gives it priority within certain clause evaluations. I would still love for an expert to weigh in if they agree/disagree that is a logical explanation for what I'm seeing here.
So I figured this out based on the comment that Martin Smith made above (THANKS MARTIN!) and I wanted to make sure I shared my discovery for anyone else who trips across this issue.
Technical Considerations
Firstly, it would certainly help if I used the proper terminology to describe my problem: My first statement above uses a correlated subquery:
http://en.wikipedia.org/wiki/Correlated_subquery
http://www.programmerinterview.com/index.php/database-sql/correlated-vs-uncorrelated-subquery/
This is actually a fairly inefficient way of pulling back data as it reruns the subquery for every line in the outer table. For this reason I'm going to look for ways of eliminating these type of subqueries in my code:
https://blogs.oracle.com/optimizer/entry/optimizer_transformations_subquery_unesting_part_1
My second statement on the other hand was using what is called an inline view in Oracle also known as a derived table in SQL Server:
http://docs.oracle.com/cd/B19306_01/server.102/b14200/queries007.htm
http://www.programmerinterview.com/index.php/database-sql/derived-table-vs-subquery/
An inline view / derived table creates a temporary unnamed view at the beginning of your query and then treats it like another table until the operation is complete. Because the compiler needs to create a temporary view when it sees on of these subqueries on the FROM line, those subqueries must be entirely self-contained with no references outside the subquery.
Why what I was doing was stupid
What I was trying to do in that second table was essentially create a view based on an ambiguous reference to another table that was outside the knowledge of my statement. It would be like trying to reference a field in a table that you hadn't explicitly stated in your query.
Workaround
Lastly, it's worth noting that Martin suggested a fairly clever but ultimately inefficient way to accomplish what I was trying to do. The Apply statement is a proprietary SQL Server function but it allows you to talk to objects outside of your derived table:
http://technet.microsoft.com/en-us/library/ms175156(v=SQL.105).aspx
Likewise this functionality is available in Oracle through different syntax:
What is the equivalent of SQL Server APPLY in Oracle?
Ultimately I'm going to re-evaluate my entire approach to this query which means I'll have to rebuild it from scratch (believe it or not I didn't create this monstrocity originally - I swear!). A big thanks to everyone who commented - this was definitely stumping me but all of the input helped put me on the right track!
How about the following query:
SELECT t1.* FROM
(
SELECT *
FROM
(
SELECT t2.id,
RANK() OVER (PARTITION BY t2.id, t2.date ORDER BY t2.date DESC) AS R
FROM table2 t2
)
WHERE R = 1
) sub
INNER JOIN table1 t1
ON t1.id = sub.id
In your second example you are trying to pass the t1 reference down 2 levels.. you can't do that, you can only pass it down 1 level (which is why the 1st works). If you give a better example of what you are trying to do, we can help you rewrite your query as well.
I need to perform a query like this:
SELECT *,
(SELECT Table1.Column
FROM Table1
INNER JOIN Table2 ON Table1.Table2Id = Table2.Id
) as tmp
FROM Table2 WHERE tmp = 1
I know I can take a workaround but I would like to know if this syntax is possible as it is (I think) in Mysql.
The query you posted won't work on sql server, because the sub query in your select clause could possibly return more than one row. I don't know how MySQL will treat it, but from what I'm reading MySQL will also yield an error if the sub query returns any duplicates. I do know that SQL Server won't even compile it.
The difference is that MySQL will at least attempt to run the query and if you're very lucky (Table2Id is unique in Table1) it will succeed. More probably is will return an error. SQL Server won't try to run it at all.
Here is a query that should run on either system, and won't cause an error if Table2Id is not unique in Table1. It will return "duplicate" rows in that case, where the only difference is the source of the Table1.Column value:
SELECT Table2.*, Table1.Column AS tmp
FROM Table1
INNER JOIN Table2 ON Table1.Table2Id = Table2.Id
WHERE Table1.Column = 1
Perhaps if you shared what you were trying to accomplish we could help you write a query that does it.
SELECT *
FROM (
SELECT t.*,
(
SELECT Table1.Column
FROM Table1
INNER JOIN
Table2
ON Table1.Table2Id = Table2.Id
) as tmp
FROM Table2 t
) q
WHERE tmp = 1
This is valid syntax, but it will fail (both in MySQL and in SQL Server) if the subquery returns more than 1 row
What exactly are you trying to do?
Please provide some sample data and desired resultset.
I agree with Joel's solution but I want to discuss why your query would be a bad idea to use (even though the syntax is essentially valid). This is a correlated subquery. The first issue with these is that they don't work if the subquery could possibly return more than one value for a record. The second and more critical problem (in my mind) is that they must work row by row rather than on the set of data. This means they will virtually always affect performance. So correlated subqueries should almost never be used in a production system. In this simple case, the join Joel showed is the correct solution.
If the subquery is more complicated, you may want to turn it into a derived table instead (this also fixes the more than one value associated to a record problem). While a derived table looks a lot like a correlated subquery to the uninitated, it does not perform the same way because it acts on the set of data rather than row-by row and thus will often be significantly faster. You are essentially making the query a table in the join.
Below is an example of your query re-written as a derived table. (Of course in production code you would not use select * either especially in a join, spell out the fields you need)
SELECT *
FROM Table2 t2
JOIN
(SELECT Table1.[Column], Table1.Table2Id as tmp
FROM Table1
INNER JOIN Table2 ON Table1.Table2Id = Table2.Id ) as t
ON t.Table2Id = Table2.Id
WHERE tmp = 1
You've already got a variety of answers, some of them more useful than others. But to answer your question directly:
No, SQL Server will not allow you to reference the column alias (defined in the select list) in the predicate (the WHERE clause). I think that is sufficient to answer the question you asked.
Additional details:
(this discussion goes beyond the original question you asked.)
As you noted, there are several workarounds available.
Most problematic with the query you posted (as others have already pointed out) is that we aren't guaranteed that the subquery in the SELECT list returns only one row. If it does return more than one row, SQL Server will throw a "too many rows" exception:
Subquery returned more than 1 value.
This is not permitted when the subquery
follows =, !=, , >= or when the
subquery is used as an expression.
For the following discussion, I'm going to assume that issue is already sufficiently addressed.
Sometimes, the easiest way to make the alias available in the predicate is to use an inline view.
SELECT v.*
FROM ( SELECT *
, (SELECT Table1.Column
FROM Table1
JOIN Table2 ON Table1.Table2Id = Table2.Id
WHERE Table1.Column = 1
) as tmp
FROM Table2
) v
WHERE v.tmp = 1
Note that SQL Server won't push the predicate for the outer query (WHERE v.tmp = 1) into the subquery in the inline view. So you need to push that in yourself, by including the WHERE Table1.Column = 1 predicate in the subquery, particularly if you're depending on that to make the subquery return only one value.
That's just one approach to working around the problem, there are others. I suspect that query plan for this SQL Server query is not going to be optimal, for performance, you probably want to go with a JOIN or an EXISTS predicate.
NOTE: I'm not an expert on using MySQL. I'm not all that familiar with MySQL support for subqueries. I do know (from painful experience) that subqueries weren't supported in MySQL 3.23, which made migrating an application from Oracle 8 to MySQL 3.23 particularly painful.
Oh and btw... of no interest to anyone in particular, the Teradata DBMS engine DOES have an extension that allows for the NAMED keyword in place of the AS keyword, and a NAMED expression CAN be referenced elsewhere in the QUERY, including the WHERE clause, the GROUP BY clause and the ORDER BY clause. Shuh-weeeet
That kind of syntax is basically valid (you need to move the where tmp=... to on outer "select * from (....)", though), although it's ambiguous since you have two sets named "Table2"- you should probably define aliases on at least one of your usages of that table to clear up the ambiguity.
Unless you intended that to return a column from table1 corresponding to columns in table2 ... in which case you might have wanted to simply join the tables?