Oracle 10g DB. I have a table called s_contact. This table has a field called person_uid. This person_uid field is a varchar2 but contains valid numbers for some rows and in-valid numbers for other rows. For instance, one row might have a person_uid of '2-lkjsdf' and another might be 1234567890.
I want to return just the rows with valid numbers in person_uid. The SQL I am trying is...
select person_uid
from s_contact
where decode(trim(translate(person_uid, '1234567890', ' ')), null, 'n', 'c') = 'n'
The translate replaces all numbers with spaces so that a trim will result in null if the field only contained numbers. Then I use a decode statement to set a little code to filter on. n=number, c=char.
This seems to work when I run just a preview, but I get an 'invalid number' error when I add a filter of...
and person_uid = 100
-- or
and to_number(person_uid) = 100
I just don't understand what is happening! It should be filtering out all the records that are invalid numbers and 100 is obviously a number...
Any ideas anyone? Greatly Appreciated!
Unfortunately, the various subquery approaches that have been proposed are not guaranteed to work. Oracle is allowed to push the predicate into the subquery and then evaluate the conditions in whatever order it deems appropriate. If it happens to evaluate the PERSON_UID condition before filtering out the non-numeric rows, you'll get an error. Jonathan Gennick has an excellent article Subquery Madness that discusses this issue in quite a bit of detail.
That leaves you with a few options
1) Rework the data model. It's generally not a good idea to store numbers in anything other than a NUMBER column. In addition to causing this sort of issue, it has a tendency to screw up the optimizer's cardinality estimates which leads to less than ideal query plans.
2) Change the condition to specify a string value rather than a number. If PERSON_UID is supposed to be a string, your filter condition could be PERSON_UID = '100'. That avoids the need to perform the implicit conversion.
3) Write a custom function that does the string to number conversion and ignores any errors and use that in your code, i.e.
CREATE OR REPLACE FUNCTION my_to_number( p_arg IN VARCHAR2 )
RETURN NUMBER
IS
BEGIN
RETURN to_number( p_arg );
EXCEPTION
WHEN others THEN
RETURN NULL;
END;
and then my_to_number(PERSION_UID) = 100
4) Use a subquery that prevents the predicate from being pushed. This can be done in a few different ways. I personally prefer throwing a ROWNUM into the subquery, i.e. building on OMG Ponies' solution
WITH valid_persons AS (
SELECT TO_NUMBER(c.person_uid) 'person_uid',
ROWNUM rn
FROM S_CONTACT c
WHERE REGEXP_LIKE(c.personuid, '[[:digit:]]'))
SELECT *
FROM valid_persons vp
WHERE vp.person_uid = 100
Oracle can't push the vp.person_uid = 100 predicate into the subquery here because doing so would change the results. You could also use hints to force the subquery to be materialized or to prevent predicate pushing.
Another alternative is to combine the predicates:
where case when translate(person_uid, '1234567890', ' ')) is null
then to_number(person_uid) end = 100
When you add those numbers to the WHERE clause it's still doing those checks. You can't guarantee the ordering within the WHERE clause. So, it still tries to compare 100 to '2-lkjsdf'.
Can you use '100'?
Another option is to apply a subselect
SELECT * FROM (
select person_uid
from s_contact
where decode(trim(translate(person_uid, '1234567890', ' ')), null, 'n', 'c') = 'n'
)
WHERE TO_NUMBER(PERSON_UID) = 100
Regular expressions to the rescue!
where regexp_like (person_uid, '^[0-9]+$')
Use the first part of your query to generate a temp table. Then query the temp table based on person_uid = 100 or whatever.
The problem is that oracle tries to convert each person_uid to an int as it gets to it due to the additional and statement in your where clause. This behavior may or may not show up in the preview depending on what records where picked.
Related
Why would someone use WHERE 1=1 AND <conditions> in a SQL clause (Either SQL obtained through concatenated strings, either view definition)
I've seen somewhere that this would be used to protect against SQL Injection, but it seems very weird.
If there is injection WHERE 1 = 1 AND injected OR 1=1 would have the same result as injected OR 1=1.
Later edit: What about the usage in a view definition?
Thank you for your answers.
Still,
I don't understand why would someone use this construction for defining a view, or use it inside a stored procedure.
Take this for example:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1 AND table.Field=Value
If the list of conditions is not known at compile time and is instead built at run time, you don't have to worry about whether you have one or more than one condition. You can generate them all like:
and <condition>
and concatenate them all together. With the 1=1 at the start, the initial and has something to associate with.
I've never seen this used for any kind of injection protection, as you say it doesn't seem like it would help much. I have seen it used as an implementation convenience. The SQL query engine will end up ignoring the 1=1 so it should have no performance impact.
Just adding a example code to Greg's answer:
dim sqlstmt as new StringBuilder
sqlstmt.add("SELECT * FROM Products")
sqlstmt.add(" WHERE 1=1")
''// From now on you don't have to worry if you must
''// append AND or WHERE because you know the WHERE is there
If ProductCategoryID <> 0 then
sqlstmt.AppendFormat(" AND ProductCategoryID = {0}", trim(ProductCategoryID))
end if
If MinimunPrice > 0 then
sqlstmt.AppendFormat(" AND Price >= {0}", trim(MinimunPrice))
end if
I've seen it used when the number of conditions can be variable.
You can concatenate conditions using an " AND " string. Then, instead of counting the number of conditions you're passing in, you place a "WHERE 1=1" at the end of your stock SQL statement and throw on the concatenated conditions.
Basically, it saves you having to do a test for conditions and then add a "WHERE" string before them.
Seems like a lazy way to always know that your WHERE clause is already defined and allow you to keep adding conditions without having to check if it is the first one.
Indirectly Relevant: when 1=2 is used:
CREATE TABLE New_table_name
as
select *
FROM Old_table_name
WHERE 1 = 2;
this will create a new table with same schema as old table. (Very handy if you want to load some data for compares)
I found this pattern useful when I'm testing or double checking things on the database, so I can very quickly comment other conditions:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1
AND Table.Field=Value
AND Table.IsValid=true
turns into:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1
--AND Table.Field=Value
--AND Table.IsValid=true
1 = 1 expression is commonly used in generated sql code. This expression can simplify sql generating code reducing number of conditional statements.
Actually, I've seen this sort of thing used in BIRT reports. The query passed to the BIRT runtime is of the form:
select a,b,c from t where a = ?
and the '?' is replaced at runtime by an actual parameter value selected from a drop-down box. The choices in the drop-down are given by:
select distinct a from t
union all
select '*' from sysibm.sysdummy1
so that you get all possible values plus "*". If the user selects "*" from the drop down box (meaning all values of a should be selected), the query has to be modified (by Javascript) before being run.
Since the "?" is a positional parameter and MUST remain there for other things to work, the Javascript modifies the query to be:
select a,b,c from t where ((a = ?) or (1==1))
That basically removes the effect of the where clause while still leaving the positional parameter in place.
I've also seen the AND case used by lazy coders whilst dynamically creating an SQL query.
Say you have to dynamically create a query that starts with select * from t and checks:
the name is Bob; and
the salary is > $20,000
some people would add the first with a WHERE and subsequent ones with an AND thus:
select * from t where name = 'Bob' and salary > 20000
Lazy programmers (and that's not necessarily a bad trait) wouldn't distinguish between the added conditions, they'd start with select * from t where 1=1 and just add AND clauses after that.
select * from t where 1=1 and name = 'Bob' and salary > 20000
where 1=0, This is done to check if the table exists. Don't know why 1=1 is used.
While I can see that 1=1 would be useful for generated SQL, a technique I use in PHP is to create an array of clauses and then do
implode (" AND ", $clauses);
thus avoiding the problem of having a leading or trailing AND. Obviously this is only useful if you know that you are going to have at least one clause!
Here's a closely related example: using a SQL MERGE statement to update the target tabled using all values from the source table where there is no common attribute on which to join on e.g.
MERGE INTO Circles
USING
(
SELECT pi
FROM Constants
) AS SourceTable
ON 1 = 1
WHEN MATCHED THEN
UPDATE
SET circumference = 2 * SourceTable.pi * radius;
If you came here searching for WHERE 1, note that WHERE 1 and WHERE 1=1 are identical. WHERE 1 is used rarely because some database systems reject it considering WHERE 1 not really being boolean.
Why would someone use WHERE 1=1 AND <proper conditions>
I've seen homespun frameworks do stuff like this (blush), as this allows lazy parsing practices to be applied to both the WHERE and AND Sql keywords.
For example (I'm using C# as an example here), consider the conditional parsing of the following predicates in a Sql query string builder:
var sqlQuery = "SELECT * FROM FOOS WHERE 1 = 1"
if (shouldFilterForBars)
{
sqlQuery = sqlQuery + " AND Bars > 3";
}
if (shouldFilterForBaz)
{
sqlQuery = sqlQuery + " AND Baz < 12";
}
The "benefit" of WHERE 1 = 1 means that no special code is needed:
For AND - whether zero, one or both predicates (Bars and Baz's) should be applied, which would determine whether the first AND is required. Since we already have at least one predicate with the 1 = 1, it means AND is always OK.
For no predicates at all - In the case where there are ZERO predicates, then the WHERE must be dropped. But again, we can be lazy, because we are again guarantee of at least one predicate.
This is obviously a bad idea and would recommend using an established data access framework or ORM for parsing optional and conditional predicates in this way.
Having review all the answers i decided to perform some experiment like
SELECT
*
FROM MyTable
WHERE 1=1
Then i checked with other numbers
WHERE 2=2
WHERE 10=10
WHERE 99=99
ect
Having done all the checks, the query run town is the same. even without the where clause. I am not a fan of the syntax
This is useful in a case where you have to use dynamic query in which in where
clause you have to append some filter options. Like if you include options 0 for status is inactive, 1 for active. Based from the options, there is only two available options(0 and 1) but if you want to display All records, it is handy to include in where close 1=1.
See below sample:
Declare #SearchValue varchar(8)
Declare #SQLQuery varchar(max) = '
Select [FirstName]
,[LastName]
,[MiddleName]
,[BirthDate]
,Case
when [Status] = 0 then ''Inactive''
when [Status] = 1 then ''Active''
end as [Status]'
Declare #SearchOption nvarchar(100)
If (#SearchValue = 'Active')
Begin
Set #SearchOption = ' Where a.[Status] = 1'
End
If (#SearchValue = 'Inactive')
Begin
Set #SearchOption = ' Where a.[Status] = 0'
End
If (#SearchValue = 'All')
Begin
Set #SearchOption = ' Where 1=1'
End
Set #SQLQuery = #SQLQuery + #SearchOption
Exec(#SQLQuery);
Saw this in production code and asked seniors for help.
Their answer:
-We use 1=1 so when we have to add a new condition we can just type
and <condition>
and get on with it.
I do this usually when I am building dynamic SQL for a report which has many dropdown values a user can select. Since the user may or may not select the values from each dropdown, we end up getting a hard time figuring out which condition was the first where clause. So we pad up the query with a where 1=1 in the end and add all where clauses after that.
Something like
select column1, column2 from my table where 1=1 {name} {age};
Then we would build the where clause like this and pass it as a parameter value
string name_whereClause= ddlName.SelectedIndex > 0 ? "AND name ='"+ ddlName.SelectedValue+ "'" : "";
As the where clause selection are unknown to us at runtime, so this helps us a great deal in finding whether to include an 'AND' or 'WHERE'.
Making "where 1=1" the standard for all your queries also makes it trivially easy to validate the sql by replacing it with where 1 = 0, handy when you have batches of commands/files.
Also makes it trivially easy to find the end of the end of the from/join section of any query. Even queries with sub-queries if properly indented.
I first came across this back with ADO and classic asp, the answer i got was: performance.
if you do a straight
Select * from tablename
and pass that in as an sql command/text you will get a noticeable performance increase with the
Where 1=1
added, it was a visible difference. something to do with table headers being returned as soon as the first condition is met, or some other craziness, anyway, it did speed things up.
Using a predicate like 1=1 is a normal hint sometimes used to force the access plan to use or not use an index scan. The reason why this is used is when you are using a multi-nested joined query with many predicates in the where clause where sometimes even using all of the indexes causes the access plan to read each table - a full table scan. This is just 1 of many hints used by DBAs to trick a dbms into using a more efficient path. Just don't throw one in; you need a dba to analyze the query since it doesn't always work.
Here is a use case... however I am not too concerned with the technicalities of why I should or not use 1 = 1.
I am writing a function, using pyodbc to retrieve some data from SQL Server. I was looking for a way to force a filler after the where keyword in my code. This was a great suggestion indeed:
if _where == '': _where = '1=1'
...
...
...
cur.execute(f'select {predicate} from {table_name} where {_where}')
The reason is because I could not implement the keyword 'where' together inside the _where clause variable. So, I think using any dummy condition that evaluates to true would do as a filler.
Why would someone use WHERE 1=1 AND <conditions> in a SQL clause (Either SQL obtained through concatenated strings, either view definition)
I've seen somewhere that this would be used to protect against SQL Injection, but it seems very weird.
If there is injection WHERE 1 = 1 AND injected OR 1=1 would have the same result as injected OR 1=1.
Later edit: What about the usage in a view definition?
Thank you for your answers.
Still,
I don't understand why would someone use this construction for defining a view, or use it inside a stored procedure.
Take this for example:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1 AND table.Field=Value
If the list of conditions is not known at compile time and is instead built at run time, you don't have to worry about whether you have one or more than one condition. You can generate them all like:
and <condition>
and concatenate them all together. With the 1=1 at the start, the initial and has something to associate with.
I've never seen this used for any kind of injection protection, as you say it doesn't seem like it would help much. I have seen it used as an implementation convenience. The SQL query engine will end up ignoring the 1=1 so it should have no performance impact.
Just adding a example code to Greg's answer:
dim sqlstmt as new StringBuilder
sqlstmt.add("SELECT * FROM Products")
sqlstmt.add(" WHERE 1=1")
''// From now on you don't have to worry if you must
''// append AND or WHERE because you know the WHERE is there
If ProductCategoryID <> 0 then
sqlstmt.AppendFormat(" AND ProductCategoryID = {0}", trim(ProductCategoryID))
end if
If MinimunPrice > 0 then
sqlstmt.AppendFormat(" AND Price >= {0}", trim(MinimunPrice))
end if
I've seen it used when the number of conditions can be variable.
You can concatenate conditions using an " AND " string. Then, instead of counting the number of conditions you're passing in, you place a "WHERE 1=1" at the end of your stock SQL statement and throw on the concatenated conditions.
Basically, it saves you having to do a test for conditions and then add a "WHERE" string before them.
Seems like a lazy way to always know that your WHERE clause is already defined and allow you to keep adding conditions without having to check if it is the first one.
Indirectly Relevant: when 1=2 is used:
CREATE TABLE New_table_name
as
select *
FROM Old_table_name
WHERE 1 = 2;
this will create a new table with same schema as old table. (Very handy if you want to load some data for compares)
I found this pattern useful when I'm testing or double checking things on the database, so I can very quickly comment other conditions:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1
AND Table.Field=Value
AND Table.IsValid=true
turns into:
CREATE VIEW vTest AS
SELECT FROM Table WHERE 1=1
--AND Table.Field=Value
--AND Table.IsValid=true
1 = 1 expression is commonly used in generated sql code. This expression can simplify sql generating code reducing number of conditional statements.
Actually, I've seen this sort of thing used in BIRT reports. The query passed to the BIRT runtime is of the form:
select a,b,c from t where a = ?
and the '?' is replaced at runtime by an actual parameter value selected from a drop-down box. The choices in the drop-down are given by:
select distinct a from t
union all
select '*' from sysibm.sysdummy1
so that you get all possible values plus "*". If the user selects "*" from the drop down box (meaning all values of a should be selected), the query has to be modified (by Javascript) before being run.
Since the "?" is a positional parameter and MUST remain there for other things to work, the Javascript modifies the query to be:
select a,b,c from t where ((a = ?) or (1==1))
That basically removes the effect of the where clause while still leaving the positional parameter in place.
I've also seen the AND case used by lazy coders whilst dynamically creating an SQL query.
Say you have to dynamically create a query that starts with select * from t and checks:
the name is Bob; and
the salary is > $20,000
some people would add the first with a WHERE and subsequent ones with an AND thus:
select * from t where name = 'Bob' and salary > 20000
Lazy programmers (and that's not necessarily a bad trait) wouldn't distinguish between the added conditions, they'd start with select * from t where 1=1 and just add AND clauses after that.
select * from t where 1=1 and name = 'Bob' and salary > 20000
where 1=0, This is done to check if the table exists. Don't know why 1=1 is used.
While I can see that 1=1 would be useful for generated SQL, a technique I use in PHP is to create an array of clauses and then do
implode (" AND ", $clauses);
thus avoiding the problem of having a leading or trailing AND. Obviously this is only useful if you know that you are going to have at least one clause!
Here's a closely related example: using a SQL MERGE statement to update the target tabled using all values from the source table where there is no common attribute on which to join on e.g.
MERGE INTO Circles
USING
(
SELECT pi
FROM Constants
) AS SourceTable
ON 1 = 1
WHEN MATCHED THEN
UPDATE
SET circumference = 2 * SourceTable.pi * radius;
If you came here searching for WHERE 1, note that WHERE 1 and WHERE 1=1 are identical. WHERE 1 is used rarely because some database systems reject it considering WHERE 1 not really being boolean.
Why would someone use WHERE 1=1 AND <proper conditions>
I've seen homespun frameworks do stuff like this (blush), as this allows lazy parsing practices to be applied to both the WHERE and AND Sql keywords.
For example (I'm using C# as an example here), consider the conditional parsing of the following predicates in a Sql query string builder:
var sqlQuery = "SELECT * FROM FOOS WHERE 1 = 1"
if (shouldFilterForBars)
{
sqlQuery = sqlQuery + " AND Bars > 3";
}
if (shouldFilterForBaz)
{
sqlQuery = sqlQuery + " AND Baz < 12";
}
The "benefit" of WHERE 1 = 1 means that no special code is needed:
For AND - whether zero, one or both predicates (Bars and Baz's) should be applied, which would determine whether the first AND is required. Since we already have at least one predicate with the 1 = 1, it means AND is always OK.
For no predicates at all - In the case where there are ZERO predicates, then the WHERE must be dropped. But again, we can be lazy, because we are again guarantee of at least one predicate.
This is obviously a bad idea and would recommend using an established data access framework or ORM for parsing optional and conditional predicates in this way.
Having review all the answers i decided to perform some experiment like
SELECT
*
FROM MyTable
WHERE 1=1
Then i checked with other numbers
WHERE 2=2
WHERE 10=10
WHERE 99=99
ect
Having done all the checks, the query run town is the same. even without the where clause. I am not a fan of the syntax
This is useful in a case where you have to use dynamic query in which in where
clause you have to append some filter options. Like if you include options 0 for status is inactive, 1 for active. Based from the options, there is only two available options(0 and 1) but if you want to display All records, it is handy to include in where close 1=1.
See below sample:
Declare #SearchValue varchar(8)
Declare #SQLQuery varchar(max) = '
Select [FirstName]
,[LastName]
,[MiddleName]
,[BirthDate]
,Case
when [Status] = 0 then ''Inactive''
when [Status] = 1 then ''Active''
end as [Status]'
Declare #SearchOption nvarchar(100)
If (#SearchValue = 'Active')
Begin
Set #SearchOption = ' Where a.[Status] = 1'
End
If (#SearchValue = 'Inactive')
Begin
Set #SearchOption = ' Where a.[Status] = 0'
End
If (#SearchValue = 'All')
Begin
Set #SearchOption = ' Where 1=1'
End
Set #SQLQuery = #SQLQuery + #SearchOption
Exec(#SQLQuery);
Saw this in production code and asked seniors for help.
Their answer:
-We use 1=1 so when we have to add a new condition we can just type
and <condition>
and get on with it.
I do this usually when I am building dynamic SQL for a report which has many dropdown values a user can select. Since the user may or may not select the values from each dropdown, we end up getting a hard time figuring out which condition was the first where clause. So we pad up the query with a where 1=1 in the end and add all where clauses after that.
Something like
select column1, column2 from my table where 1=1 {name} {age};
Then we would build the where clause like this and pass it as a parameter value
string name_whereClause= ddlName.SelectedIndex > 0 ? "AND name ='"+ ddlName.SelectedValue+ "'" : "";
As the where clause selection are unknown to us at runtime, so this helps us a great deal in finding whether to include an 'AND' or 'WHERE'.
Making "where 1=1" the standard for all your queries also makes it trivially easy to validate the sql by replacing it with where 1 = 0, handy when you have batches of commands/files.
Also makes it trivially easy to find the end of the end of the from/join section of any query. Even queries with sub-queries if properly indented.
I first came across this back with ADO and classic asp, the answer i got was: performance.
if you do a straight
Select * from tablename
and pass that in as an sql command/text you will get a noticeable performance increase with the
Where 1=1
added, it was a visible difference. something to do with table headers being returned as soon as the first condition is met, or some other craziness, anyway, it did speed things up.
Using a predicate like 1=1 is a normal hint sometimes used to force the access plan to use or not use an index scan. The reason why this is used is when you are using a multi-nested joined query with many predicates in the where clause where sometimes even using all of the indexes causes the access plan to read each table - a full table scan. This is just 1 of many hints used by DBAs to trick a dbms into using a more efficient path. Just don't throw one in; you need a dba to analyze the query since it doesn't always work.
Here is a use case... however I am not too concerned with the technicalities of why I should or not use 1 = 1.
I am writing a function, using pyodbc to retrieve some data from SQL Server. I was looking for a way to force a filler after the where keyword in my code. This was a great suggestion indeed:
if _where == '': _where = '1=1'
...
...
...
cur.execute(f'select {predicate} from {table_name} where {_where}')
The reason is because I could not implement the keyword 'where' together inside the _where clause variable. So, I think using any dummy condition that evaluates to true would do as a filler.
I think I'm encountering a fairly simple problem in PL/SQL on an Oracle Database(10g) and I'm hoping one of you guys can help me out.
I'm trying to explain this as clear as possible, but it's hard for me.
When I try to compare varchar2 values of 2 different tables to check if I need to create a new record or I can re-use the ID of the existing one, the DB (or I) compares these values in a wrong way. All is fine when both the field contain a value, this results in 'a' = 'a' which it understands. But when both fields are NULL (or '' which Oracle will turn into NULL) it can not compare the fields.
I found a 'solution' to this problem but I'm certain there is a better way.
rowTable1 ROWTABLE1%ROWTYPE;
iReUsableID INT;
SELECT * INTO rowTable1
FROM TABLE1
WHERE TABLE1ID = 'someID';
SELECT TABLE2ID INTO iReUsableID
FROM TABLE2
WHERE NVL(SOMEFIELDNAME,' ') = NVL(rowTable1.SOMEFIELDNAME,' ');
So NVL changes the null value to ' ' after which it will compare in the right way.
Thanks in advance,
Dennis
You can use LNNVL function (http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions078.htm) and reverse the condition:
SELECT TABLE2ID INTO iReUsableID
FROM TABLE2
WHERE LNNVL(SOMEFIELDNAME != rowTable1.SOMEFIELDNAME);
Your method is fine, unless one of the values could be a space. The "standard" way of doing the comparison is to explicitly compare to NULL:
WHERE col1 = col2 or col1 is null and col2 is null
In Oracle, comparisons on strings are encumbered by the fact that Oracle treats the empty string as NULL. This is a peculiarity of Oracle and not a problem in other databases.
In Oracle (or any RDBMS I believe), one NULL is not equal to another NULL. Therefore, you need to use the workaround that you have stated if you want to force 2 NULL values to be considered the same. Additionally, you might want to default NULL values to '' (empty) rather than ' ' (space).
From Wikipedia (originally the ISO spec, but I couldn't access it):
Since Null is not a member of any data domain, it is not considered a "value", but rather a marker (or placeholder) indicating the absence of value. Because of this, comparisons with Null can never result in either True or False, but always in a third logical result, Unknown.
As mentioned by Jan Spurny, you can use LNNVL for comparison. However, it would be wrong to say that a comparison is actually being made when both values being compared are NULL.
This is indeed a simple and usable way to compare nulls.
You cannot compare NULLS directly since NULL is not equal NULL.
You must provide your own logic who you would like to compare, what you've done with NVL().
Take in mind, you are treating NULLS as space, so ' ' in one table would be equal to NULL in another table in your case.
There are some other ways (e.g. LNNVL ) but they are not some kind of a "better" way, I think.
"Internal error: An expression services limit has been reached. Please look for potentially complex expressions in your query, and try to simplify them."
Has anyone seen this before and found a good workaround?
I managed to get around this issue by splitting my SQL query into two parts essentially and writing the first SQL select query to a temp table and the second part, a new SQL select statement selects from the temporary table and uses alot of CROSS APPLY operator to Calculate cascading computed columns.
This is an example of how the second part looks but I'm using alot more Cross Applys to produce new columns which are calculations:
Select * from #tempTable
cross apply
(
select HmmLowestSalePrice =
round(((OurSellingPrice + 1.5) / 0.95) - (CompetitorsLowestSalePrice) + 0.08, 2)
) as HmmLowestSalePrice
cross apply
(
select checkLowestSP =
case
when adjust = 'No Room' then 'No Room'
when OrginalTestSalePrice >= CompetitorsLowestSalePrice then 'Minus'
when OrginalTeslSalePrice < CompetitorsLowestSalePrice then 'Ok'
end
) as checkLowestSP
cross apply
(
select AdjustFinalNewTestSP =
case
when FinalNewTestShipping < 0 Then NewTestSalePrice - (FinalNewTestShipping)
when FinalNewTestShipping >= 0 Then NewTestSalePrice
end
) as AdjustFinalNewTestSP
cross apply
(
select CheckFinalSalePriceWithWP =
case
when round(NewAdminSalePrice, 2) >= round(wholePrice, 2) then 'Ok'
when round(NewAdminSalePrice, 2) < round(wholePrice, 2) then 'Check'
end
) as CheckFinalPriceWithWP
DROP TABLE #tempTable
My goal to to put this into a sql report and it work fine if there is 1 user only as the #tempTable will get created and dropped in the same execution and the results are displayed in the report correctly. But in the future if there are concurrent users I'm concerned that they will be writing to the same #tempTable which will affect the results?
I've looked at putting this into stored procedures but still get the error message above.
This issue occurs because SQL Server limits the number of identifiers and constants that can be contained in a single expression of a query. The limit is 65,535. The test for the number of identifiers and constants is performed after SQL Server expands all referenced identifiers and constants. In SQL Server 2005 and above, queries are internally normalized and simplified. And that includes *(asterisk), computed columns etc.
In order to work around this issue, rewrite your query. Reference fewer identifiers and constants in the largest expression in the query. You must make sure that the number of identifiers and constants in each expression of the query does not exceed the limit. To do this, you may have to break down a query into more than one single query. Then, create a temporary intermediate result.
The same issue happens to me when we tried to change the Database Compatibility Level to 150. It is not an issue when it is 140 or lower.
I just had this problem and fixed it by removing the UNIQUE index on my table. For some reason, that seems to trigger this error, although it cannot figure out why.
By the way, the same query does work with several other indexes.
What worked for me was replacing several COALESCE statements with ISNULL whenever was possible
I've read many of the gory details of write consistency and I understand how it works in the simple cases. What I'm not clear on is what this means for nested sub-queries.
Here's a concrete example:
A table with PK id, and other columns state, temp and date.
UPDATE table SET state = DECODE(state, 'rainy', 'snowy', 'sunny', 'frosty') WHERE id IN (
SELECT id FROM (
SELECT id,state,temp from table WHERE date > 50
) WHERE (state='rainy' OR state='sunny') AND temp < 0
)
The real thing was more convoluted (in the innermost query), but this captures the essence.
If we assume the state column is not nullable, can this update ever fail due to concurrent modification (i.e., the DECODE function doesn't find a match, a value of 'rainy' or 'sunny', and so tries to insert null into a non-nullable column)?
Oracle supports "statement level read and write consistency" (as all other serious DBMS)
This means that the statement as a whole will not see any changes to the database that occurred after the statement started.
As your UPDATE is one single statement there shouldn't be a case where the decode returns null.
Btw: the statement can be simplified, you don't need the outer SELECT in the sub-query:
UPDATE table SET state = DECODE(state, 'rainy', 'snowy', 'sunny', 'frosty')
WHERE id IN (
SELECT id
FROM table
WHERE date > 50
AND (state='rainy' OR state='sunny')
AND temp < 0
)
I don't see any reason to be concerned. The subquery explicitly retrieves only IDs of rows with state 'rainy' or 'sunny' and that's what outer DECODE is going to get. Thole thing is one statement, and is going to be executed within transaction boundaries.
Answering my own question: turns out there is a bug in Oracle which can cause this query to fail. Details confirmed by Tom Kyte, in the discussion starting here.