I am trying to write a SQL query with a WHERE clause that does not actually filter anything.
import cx_Oracle
condition = []
#condition = ['a', 'b']
query = ("""
SELECT *
FROM table
WHERE var = {}""".format(tuple(condition)))
with cx_Oracle.connect(dsn=tsn), encoding="UTF-8")) as con:
df = pd.read_sql(con=con, sql=query)
Disclaimer: Still a bit new to SQL. I appreciate corrections of terminology
Edit:
This is often handled using logic like this:
where col = :input_val or :input_val is null
If you don't want to have the WHERE clause filter anything then it needs to contain a condition which is always true; something like
SELECT *
FROM table1
WHERE var = var OR
var IS NULL
db<>fiddle here
If you need to use an index and the column value is not nullable then you'll want to use the NVL trick. I don't know why, but Oracle has better optimizations for NVL than for the ...OR A IS NULL expression.
SELECT *
FROM table
WHERE nvl(anyvalueofvar1, var) = var;
See my answer here for an example of the better execution plan generated by this different syntax.
(I said "and the column value is not nullable" because NULL = NULL is not true in Oracle, so the NVL expression won't work if the column has nulls. I usually hate these cryptic syntaxes with weird behavior, but they are sometimes necessary for performance.)
Related
I am building a SQL query inside a function. The functions parameters are my criterias for the WHERE clause. Parameters can be null too.
foo (1,2,3) => SELECT x FROM y WHERE a=1 AND b=2 AND c=3;
foo (null, 2, null) => SELECT x FROM y WHERE b=2;
My approach to do that in code is to add a very first alltime true in the WHERE-clause (e.g. 1=1 or NULL is NULL or 2 > 1)
So I do not need to handle the problem that the 1st WHERE condition is after a "WHERE" and all others are after a "AND".
String sql="SELECT x FROM y WHERE 1=1";
if (a!=null)
{
sql += " AND a="+a;
}
Is there a better term than 1=1 or my other samples to EXPLICITLY have a always true value? TRUE and FALSE is not working.
Oracle does not support a boolean type in SQL. It exists in PL/SQL, but can't be used in a query. The easiest thing to do is 1=1 for true and 0=1 for false. I'm not sure if the optimizer will optimize these out or not, but I suspect the performance impact is negligible. I've used this where the number of predicates is unknown until runtime.
I think this construct is superior to anything using nvl, coalesce, decode, or case, because those bypass normal indexes and I've had them lead to complicated and slow execution plans.
So, yes I'd do something like you have (not sure what language you're building this query in; this is not valid PL/SQL. Also you should use bind variables but that's another story):
sql = "SELECT x FROM y WHERE 1=1";
if (a != null)
{
sql += " AND a=" + a;
}
if (b != null)
{
sql += " AND b=" + b;
}
... etc.
I think this is better than Gordon Linoff's suggestion, because otherwise this would be more complicated because you'd have to have another clause for each predicate checking if there was a previous predicate, i.e. whether you needed to include the AND or not. It just makes the code more verbose to avoid a single, trivial clause in the query.
I've never really understood the approach of mandating a where clause even when there are no conditions. If you are constructing the logic, then combine the conditions. If the resulting string is empty, then leave out the where clause entirely.
Many application languages have the equivalent of concat_ws() -- string concatenation with a separator (join in Python, for instance). So leaving out the where clause does not even result in code that is much more complicated.
As for your question, Oracle doesn't have boolean values, so 1=1 is probably the most common approach.
use NVL so you can set a default value. Look up NVL or NVL2. Either might work for you.
check out: http://www.dba-oracle.com/t_nvl_vs_nvl2.htm
In sql the standard way to "default null" is to use coalesce. So lets say your input is #p1, #p2, and #p3
Then
select *
from table
where a = coalesce(#p1,a) and
b = coalesce(#p2,b) and
c = coalesce(#p3,c)
Thus if (as an example) #p1 is null then it will compare a = a (which is always true). If #p1 is not null then it will filter on a equals that value.
In this way null becomes a "wildcard" for all records.
you could also use decode to get the results you need since it is oracle
Select
decode(a, 1, x,null),
decode(b, 2, x,null),
decode(c, 3, x,null)
from y
What about using OR inside of the parentheses while ANDs exist at outside of them
SELECT x
FROM y
WHERE ( a=:prm1 OR :prm1 is null)
AND ( b=:prm2 OR :prm2 is null)
AND ( c=:prm3 OR :prm3 is null);
I want to use completely different conditions in a WHERE-clause in an SQL Server 2014 query, not just change the parameter.
I have the following clause
WHERE (dbo.SecurityCheckUserInADGroups.CWID = #cwid2) AND (dbo.SecurityCheckUserInDatabaseRoles.Server = #server)
If the variable #cwid2 is NULL or empty I'd like to replace the whole first part of the AND-Statement with
dbo.SecurityCheckLDAPGroupName.DatabaseUserName = #role2
So here not only the parameter but also the column changes.
I tried If, Case and Iif with something like this, but it isn't accepted:
(IIF (LEN(#cwid1) > '0, 'dbo.SecurityCheckUserInADGroups.CWID = #cwid2','dbo.SecurityCheckLDAPGroupName.DatabaseUserName = #role2'))
It keeps telling me "An expression of non-boolean type specified in a context where a condition is expected"
Any ideas how I can solve this?
Just use basic logic:
WHERE ((#cwid <> '' AND dbo.SecurityCheckUserInADGroups.CWID = #cwid2) OR
(COALESCE(#cwid, '') = '' AND dbo.SecurityCheckLDAPGroupName.DatabaseUserName = #role2)
) AND
(dbo.SecurityCheckUserInDatabaseRoles.Server = #server)
The most general solution would be
Where (#var is null and col1=val1)
Or (#var is not null and col2=val2)
Because query optimizers can sometimes choke on OR, I tend to look for other solutions when performance could be an issue (large data sets, etc.) but the above might be worth trying first.
If you do need something more specific, the trick to iif logic is that it returns a value to compare, not a predicate to evaluate. You could do something like
Where col1 = iif(#var is null, val1, col1)
And col2 = iif(var is null, col2, val2)
The idea here is that your "don't care" cases - like the value of col2 when var is null - resolve as "always true". But there are cases (like when col1 or col2 could be NULL) where this specific solution Will fail.
Which is why the first approach is the most general
WHERE CASE WHEN ISNULL(#cwid2,'') <> '' THEN
dbo.SecurityCheckUserInADGroups.CWID = #cwid2 ELSE
dbo.SecurityCheckLDAPGroupName.DatabaseUserName = #role2
END
AND dbo.SecurityCheckUserInDatabaseRoles.Server = #server
Assume a sqlite database with an integer column.
Now, it tends to happen that the integer field contains NULL values (=unset) as well.
I would like to interpret NULL values as zero (0) when doing queries on that field.
Is there a way to tell sqlite that I like NULL handled like 0 in this case, especially when I do comparisons using a SELECT statement?
Since I construct the queries dynamically in C-like code, I like to be able to write something like this:
query = "SELECT * FROM tbl WHERE intField=" + IntToStr(anIntValue)
Currently, I work around this with code as follow, which I like to avoid:
if (anIntValue == 0) {
query = "SELECT * FROM tbl WHERE intField IS NULL OR intField=0"
} else {
query = "SELECT * FROM tbl WHERE intField=" + IntToStr(anIntValue)
}
Maybe there's an operator I can use in the query that converts NULLs to other values I specify?
Of course, another way would be to make sure one never ends up with NULL values in that field, but one might want to be able to tell when the value hasn't been set at all yet, so I like to be able to keep the NULL values in the database.
In Standard SQL this is called COALESCE:
COALESCE(col, 0)
Why using a proprietary extension like IFNULL/NVL if there's a Standard which is supported by every DBMS?
please, try ifnull function, see doc at http://www.sqlite.org/lang_corefunc.html#ifnull
You can use IFNULL
or
CASE WHEN fieldname IS NULL THEN 0 ELSE fieldname END
This works the same as isnull(fieldname, 0)
You can read more here
Although you could use null coalesce functionality of your RDBMS, a universal approach exists that lets you treat NULLs as zeros in a condition, like this:
String val = IntToStr(anIntValue)
"SELECT * FROM tbl WHERE intField=" + val + " OR (0="+ val+" AND intField IS NULL)"
I am trying to connect a Filemaker DB to Firebird SQL DB in both ways import to FM and export back to Firebird DB.
So far it works using the MBS Plug-in but FM 13 Pro canot handle NULL.
That means that for example Timestamp fields that are empty (NULL) produce a "0" value.
Thats means in Time something like 01.01.1889 00:00:00.
So my idea was to simply ignore fields containing NULL.
But here my poor knowlege stops.
First I thought I can do this with WHERE, but this is ignoring whole records sets:
SELECT * FROM TABLE WHERE FIELD IS NOT NULL
Also I tried to filter it later on like this:
If (IsEmpty (MBS("SQL.GetFieldAsDateTime"; $command; "FIELD") ) = 0 ; MBS("SQL.GetFieldAsDateTime"; $command; "FIELD"))
With no result either.
This is a direct answer to halfbit's suggestion, which is correct but not for this SQL dialect. In a query to provide a replacement value when a field is NULL you need to use COALESCE(x,y). Where if X is null, Y will be used, and if Y is null then the field is NULL. Thats why it is common for me to use it like COALESCE(table.field,'') such that a constant is always outputted if table.field happens to be NULL.
select COALESCE(null,'Hello') as stackoverflow from rdb$database
You can use COALESCE() for more than two arguments, I just used two for conciseness.
I dont know the special SQL dialect, but
SELECT field1, field2, value(field, 0), ...FROM TABLE
should help you:
value gives the first argument, ie, your field if it is NOT NULL or the second argument if it is.
I have a filter where user can select operations like is, contains, etc...
bubu is xoxo translates into WHERE lower(bubu) = 'xoxo' SQL WHERE condition.
bubu contains xoxo translates into WHERE bubu ILIKE '%xoxo%' SQL WHERE condition.
Now I have added the negative variants - is not, does not contain, etc.. I do not want to rewrite the WHERE conditions from scratch, so I prepend NOT to the already existing ones:
bubu is not xoxo translates into WHERE NOT lower(bubu) = 'xoxo' SQL WHERE condition.
bubu does not contain xoxo translates into WHERE NOT bubu ILIKE '%xoxo%' SQL WHERE condition.
However, there is a problem. If bubu is a nullable field and it actually has NULL in it, then the negative WHERE condition does not pick it, although from the human perspective (as opposed to SQL) the NULL value should satisfy the bubu is not xoxo filter.
I solve this problem by modifying the original positive WHERE condition like this:
bubu is xoxo translates into WHERE (lower(bubu) = 'xoxo' AND bubu IS NOT NULL) SQL WHERE condition.
Then, the negation yields:
bubu is not xoxo translates into WHERE NOT (lower(bubu) = 'xoxo' AND bubu IS NOT NULL) SQL WHERE condition.
And this time the NULL values are picked up correctly. The same problem is with the contains filter.
Is there a more elegant solution to resolve this inconsistency between how humans treat NULL and how SQL does it?
I am using PostgreSQL 9.2 and I do not mind having a solution specific to this database.
P.S.
Please, note that I want the negative expression to be of the form NOT positive.
I think you should be able to get away with using COALESCE to convert your NULLs to empty strings:
-- These skip skips NULLs
lower(coalesce(bubu, '')) = 'xoxo'
coalesce(bubu, '') ilike '%xo%'
-- These will find NULLs
not lower(coalesce(bubu, '')) = 'xoxo'
not coalesce(bubu, '') ilike '%xo%'
Of course, this sort of trickery will run into problems if you're searching for empty strings, in such cases you'll need a context-sensitive sentinel value so that you can intelligently choose something that cannot possibly match your search term.
Demo: http://sqlfiddle.com/#!12/8bbd2/3
For the = operator you can use the is [not] distinct from construct
WHERE (lower(bubu) is not distinct from 'xoxo')
http://www.postgresql.org/docs/9.2/static/functions-comparison.html