I have a query where I filter a column based on a where clause, like so:
select * from table where col = val
What value do I set for val so that there is no filtering in the where clause (in which case the where clause is redundant)?
What value do I set for val so that there is no filtering in the where clause?
It's impossible.
You might instead use
query = "SELECT * FROM TABLE"
if val is not None:
query += " WHERE col = :val"
None is a common sentinel value,
but feel free to use another.
Consider switching from = equality to LIKE.
Then a % wildcard will arrange for an unfiltered blind query.
query = "SELECT * FROM table WHERE col LIKE :val"
Pro:
You still get to exploit any index there might be on col, for values of val ending with a % wildcard.
Cons:
The program behavior is clearly different, e.g. there might be a UNIQUE KEY on col, and the revised SELECT can now return multiple rows.
Your data might contain wildcard characters, which now need escaping.
Your users may have more flexibility now to pose queries that you didn't want them to.
Any % characters that are not at the end of val may disable the index, leading to unexpectedly long query times / result set sizes.
If col can't be null you can use col=col?
Related
I have a table with IDs, some have letters, most do not. And associated points with those IDs. The IDs are stored as text.
I would like to add points to a given range of IDS that ARE integers and give them all the same points. 535.
I have found a way using a subquery to SELECT the data I need, but it appears that updating it is another matter. I would love to be able to only get data that is CAST-able without a subquery. however since it errors out when it touches something that isn't a number, that doesn't seem to be possible.
select * from (select idstring, amount from members where idstring ~ '^[0-9]+$') x
WHERE
CAST(x.idstring AS BIGINT) >= 10137377001
and
CAST(x.idstring AS BIGINT) <= 10137377100
What am I doing ineficiently in the above, and how an I update the records that I want to?
In a perfect world my statement would be as simple as:
UPDATE members SET amount = 535
WHERE idstring >= 10137377001
AND idstring <= 10137377100
But since Idstring both contains entries that contain letters and is stored as text, it complicates things significantly. TRY_CAST would be perfect here, however there is no easy equivalent in postgres.
An example of the ids in the table might be
A52556B
36663256
6363632900B
3000525
ETC.
You can use TO_NUMBER together with your regular expression predicate, like so:
UPDATE members
SET amount = 535
WHERE idstring ~ '^[0-9]+$'
AND to_number(idstring, '999999999999') BETWEEN 10137377001 AND 10137377100
Working example on dbfiddle
You can encase the typecast and integer comparison within a CASE statement.
UPDATE members
SET amount = COALESCE(amount, 0) + 535
WHERE CASE WHEN idstring ~ '^[0-9]+$'
THEN idstring::BIGINT BETWEEN 10137377001 AND 10137377100
ELSE FALSE END;
Here I've assumed you might want to add 535 rather than setting it explicitly to 535 based on what you said above, but if my assumption is incorrect then SET amount = 535 is just fine.
I am running queries in a large IBM DB2 database table (let's call it T) and have found that the cells for column Identifier tend to be padded not just on the margins, but in between as well, as in: ' ID1 ID2 '. I do not have rights to update this DB, nor would I, given a number of factors. However, I want a way to ignore the whitespace AT LEAST on the left and right, even if I need to simply add a couple of spaces in between. The following queries work, but are slow, upwards of 20 seconds slow....
SELECT * FROM T WHERE Identifier LIKE '%ID1%ID2%';
SELECT * FROM T WHERE TRIM(Identifier) LIKE 'ID1%ID2';
SELECT * FROM T WHERE TRIM(Identifier) = 'ID1 ID2';
SELECT * FROM T WHERE LTRIM(RTRIM(Identifier)) = 'ID1 ID2';
SELECT * FROM T WHERE LTRIM(Identifier) LIKE 'ID1 ID2%';
SELECT * FROM T WHERE LTRIM(Identifier) LIKE 'ID1%ID2%';
SELECT * FROM T WHERE RTRIM(Identifier) LIKE '%ID1 ID2';
SELECT * FROM T WHERE RTRIM(Identifier) LIKE '%ID1%ID2';
Trying to query something like "Select * FROM T WHERE REPLACE(Identifier, ' ', '')..." of course just freezes up Access until I Ctrl+Break to end the operation. Is there a better, more efficient way to ignore the whitespace?
================================
UPDATE:
As #Paul Vernon describes below, "Trailing spaces are ignored in Db2 for comparison purpose, so you only need to consider the leading and embedded spaces."
This led me to generate combinations of spaces before 'ID1' and 'ID2' and select the records using the IN clause. The number of combinations means that the query is slower than if I knew the exact match. This is how it looks in my Java code with Jdbc (edited to make it more generic to the key issue):
private static final int MAX_LENGTH = 30;
public List<Parts> queryMyTable(String ID1, String ID2) {
String query="SELECT * FROM MYTABLE WHERE ID IN (:ids)";
final Map<String, List<String>> parameters = getIDCombinations(ID1, ID2);
return namedJdbcTemplate.query(query,parameters,new PartsMapper());
}
public static List<String> getIDCombinations(String ID1, String ID2) {
List<String> combinations = new ArrayList<>();
final int literalLength = ID1.length() + ID2.length();
final int maxWhitespace = MAX_LENGTH - literalLength;
combinations.add(ID1+ID2);
for(int x = 1; x <= maxWhitespace; x++){
String xSpace = String.format("%1$"+x+"s", "");
String idZeroSpaceBeforeBase = String.format("%s%s%s",ID1,xSpace,ID2);
String idZeroSpaceAfterBase = String.format("%s%s%s",xSpace,ID1,ID2);
combinations.add(idZeroSpaceBeforeBase);
combinations.add(idZeroSpaceAfterBase);
for(int y = 1; (x+y) <= maxWhitespace; y++){
String ySpace = String.format("%1$"+y+"s", "");
String id = String.format("%s%s%s%s",xSpace,ID1,ySpace,ID2);
combinations.add(id);
}
}
return combinations;
}
Trailing spaces are ignored in Db2 for comparison purpose, so you only need to consider the leading and embedded spaces.
Assuming there is an index on the Identifier, your only option (if you can't change the data, or add a functional index or index a generated column), is probably something like this
SELECT * FROM T
WHERE
Identifier = 'ID1 ID2'
OR Identifier = ' ID1 ID2'
OR Identifier = ' ID1 ID2'
OR Identifier = 'ID1 ID2'
OR Identifier = ' ID1 ID2'
OR Identifier = ' ID1 ID2'
which the Db2 optimize might implement as 6 index lookups, which would be faster than a full index or table scan
You could also try this
SELECT * FROM T
WHERE
Identifier LIKE 'ID1 %ID2'
OR Identifier LIKE ' ID1 %ID2'
OR Identifier LIKE ' ID1 %ID2'
which the Db2 optimize might implement as 3 index range scans,
In both examples add more lines to cover the maximum number of leading spaces you have in your data if needed. In the first example add more lines for the embeded spaces too if needed
Index on the expression REGEXP_REPLACE(TRIM(Identifier), '\s{2,}', ' ') and the following query should make Db2 use this index:
SELECT *
FROM T
WHERE REGEXP_REPLACE(TRIM(Identifier), '\s{2,}', ' ') = 'ID1 ID2'
If you need to search excluding leading and trailing spaces, then no traditional indexes can help you with that, at least as you show the case. To make the query fast, the options I can see are:
Full Text Search
You can use a "full text search" solution. DB2 does include this functionality, but I don't remember if it's included by default in the license or is sold separately. In any case, it requires a bit of indexing or periodic re-indexing of the data to make sure the search is up to date. It's worth the effort if you really need it. You'll need to change your app, since the mechanics are different.
Index on extra, clean column
Another solution is to index the column without the leading or trailing spaces. But you'll need to create an extra column; on a massive table this operation can take some time. The good news is that once is created then there's no more delay. For example:
alter table t add column trimmed_id varchar(100)
generated always as (trim(identifier));
Note: You may need to disable/enable integrity checks on the table before and after this clause. DB2 is picky about this. Read the manual to make sure it works. The creation of this column will take some time.
Then, you need to index it:
create index ix1 on t (trimmed_id);
The creation of the index will also take some time, but it should be faster than the step above.
Now, it's ready. You can query your table by using the new column instead of the original one (that's still there), but this time, you can forget about leading and traling spaces. For example:
SELECT * FROM T WHERE trimmed_id LIKE 'ID1%ID2';
The only wildcard now shows up in the middle. This query will be much faster than reading the whole table. In fact, the longer the string ID1 is, the faster the query will be, since the selectivity will be better.
Now, if ID2 is longer than ID1 then you can reverse the index to make it fast.
I want to use LIKE operator in access 10 sql query with variable.
Example:
temporary variable var contains value bs
var = "bs"
I want to match every String that starts with value of temporary variable followed by zero or more numbers.
I am trying to fire the query:
select * from xyz where variety LIKE "#[tempvars]![var] + [0-9]*"
It is returning 0 records.
Thankz for the help.
You need to refer to your tempvar outside of the quotes, and use & for concatenation:
select * from xyz where variety LIKE "#" & [tempvars]![var] & "[0-9]*"
This will return all records where variety starts with a literal #, then whatever is in [tempvars]![var], then a number, and then any amount of characters.
You can check if that variety is available in your table or not. If that variety is available in your table then don't search with like operator and otherwise use like operator.
I'm facing a strange situation where OledbDataAdapter rearranges the WHERE clause of my query, thus shuffling all parameters. In addition, it doesn't recognize one of the parameters used in the query. Here's how it goes:
A simple table with 4 columns (and infinite rows :)):
Name varchar(50)
Category varchar(20) //One of around 15-20 possible values
YR int //Year
Period int //Month
User will query this table using Year, Month and a comma-separated list of categories that he's interested in. Since .NET doesn't support multi-valued parameters through IN operator, what I do is to accept the list of categories as comma-separated list and then prepend and append a comma to this list within the query and use built-in INSTR() function to filter results for desired categories. User can also supply an empty string for categories filter in which case query will need to return all results (i.e. no filter on categories).
Therefore my query looks like this:
SELECT * FROM [Table1] WHERE
YR = ? AND
Period = ? AND
(LTRIM(RTRIM(?)) = '' OR INSTR(1, ?, ',' + Category + ',') > 0)
This worked with MDBs in the past. But recently I tried it against an ACCDB. The first thing I noted is that when I try to run the query in OledbDataAdapter wizard, Visual Studio rewrites it as:
SELECT * FROM [Table1] WHERE
(YR = ? AND Period = ? AND LTRIM(RTRIM(?)) = '') OR
(YR = ? AND Period = ? AND INSTR(1, ?, ',' + Category + ',') > 0)
In other words, it has rearranged the condition A AND B AND (C OR D) as (A AND B AND C) OR (A AND B AND D).
This would have been fine with me, but the additional problem is that this changes the number of parameters from 4 to 6; the query therefore doesn't return any results even when it should. Moreover, it doesn't recognize that last parameter (the question mark within INSTR function) at all.
So my questions are:
Who is rewriting the query (OledbDataAdapter, Access query parser or something else)?
Why is it doing so? Optimization? Apparently my version of the WHERE clause is more efficient.
Why doesn't it see the last parameter?
How to fix/workaround this issue? (plz do not suggest using 2 separate queries for it).
Assume a sqlite database with an integer column.
Now, it tends to happen that the integer field contains NULL values (=unset) as well.
I would like to interpret NULL values as zero (0) when doing queries on that field.
Is there a way to tell sqlite that I like NULL handled like 0 in this case, especially when I do comparisons using a SELECT statement?
Since I construct the queries dynamically in C-like code, I like to be able to write something like this:
query = "SELECT * FROM tbl WHERE intField=" + IntToStr(anIntValue)
Currently, I work around this with code as follow, which I like to avoid:
if (anIntValue == 0) {
query = "SELECT * FROM tbl WHERE intField IS NULL OR intField=0"
} else {
query = "SELECT * FROM tbl WHERE intField=" + IntToStr(anIntValue)
}
Maybe there's an operator I can use in the query that converts NULLs to other values I specify?
Of course, another way would be to make sure one never ends up with NULL values in that field, but one might want to be able to tell when the value hasn't been set at all yet, so I like to be able to keep the NULL values in the database.
In Standard SQL this is called COALESCE:
COALESCE(col, 0)
Why using a proprietary extension like IFNULL/NVL if there's a Standard which is supported by every DBMS?
please, try ifnull function, see doc at http://www.sqlite.org/lang_corefunc.html#ifnull
You can use IFNULL
or
CASE WHEN fieldname IS NULL THEN 0 ELSE fieldname END
This works the same as isnull(fieldname, 0)
You can read more here
Although you could use null coalesce functionality of your RDBMS, a universal approach exists that lets you treat NULLs as zeros in a condition, like this:
String val = IntToStr(anIntValue)
"SELECT * FROM tbl WHERE intField=" + val + " OR (0="+ val+" AND intField IS NULL)"