Using between operator for string which stores numbers - sql

Question

I have a column in which numbers are stored as string because of the nature of the column where any kind of data type is expected like date, numbers, alpha numeric,
etc.
Now i need to check if the values in that column is in defined range or not here is sample data for testing
create table test (val varchar2(10));
insert into test values ('0');
insert into test values ('67');
insert into test values ('129');
insert into test values ('200');
insert into test values ('1');
Here expected range in which value should be is 0-128 if values are not in range then i need to filter them out for further processing.
For this i have written some queries but none of then is giving requires output.
select *
from test
where val not between '0' and '128';
select *
from test
to_number(val, '9') not between to_number('0', '9') and to_number('128', '9999');
select * from test where
to_number(val, '9') < TO_NUMBER('0', '9')
or
to_number(val, '999') > TO_NUMBER('128', '999')
;
These above queries are producing desired output !! :(
I ma using DB version --
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

Answer 1

Just leave the format out of to_number():
select *
from test
where to_number(val) not between to_number('0') and to_number('128');
The numeric format is needed for conversion to a character. If you pass it in to to_number(), then it expects a number of that format -- and you might get the number of digits wrong.
Or, better yet:
select *
from test
where to_number(val) not between 0 and 128;
Or, even better yet, change the column to contain a number rather than a string.
EDIT:
If the problem is that your value is not a number (which is quite different from your original question), then test for that. This is one situation where case is appropriate in the where clause (because case guarantees the order of evaluation of its arguments:
where (case when regexp_like(val, '[^-0-9]') then 'bad'
when cast(val as number) < 0 then 'bad'
when cast(val as number) > 128 then 'bad'
else 'good'
end) = 'bad'

Answer 2

#GordonLinoff's answer works with the sample data you've shown, but it will error with ORA-01722 "invalid number" if you have any values which do no represent numbers. Your sample data only has good values, but you said that for your real field "any kind of data type is expected like date, numbers, alpha numeric, etc."
You can get around that with a function that attempts to convert the stored string value to a number, and returns null if it gets that exception. A simple example:
create function safe_to_number (p_str varchar2) return number is
begin
return to_number(p_str);
exception
when value_error then
return null;
end;
/
You can then do
select *
from test
where safe_to_number(val) not between 0 and 128;
VAL
----------
129
200
Anything that can't be converted and causes an ORA-06502 value-error exception will be seen as null, which is neither between nor not between any values you supply.
If you need to check date ranges you can do something similar, but there are more errors possible, and you may have dates in multiple formats; you would need to declare exceptions and initialise them to known error numbersto catch the ones you expect to see. This isn't complete, but you could start with something like:
create function safe_to_date (p_str varchar2) return date is
l_formats sys.odcivarchar2list;
format_ex_1 exception;
format_ex_2 exception;
format_ex_3 exception;
format_ex_4 exception;
format_ex_5 exception;
pragma exception_init(format_ex_1, -1840);
pragma exception_init(format_ex_2, -1841);
pragma exception_init(format_ex_3, -1847);
pragma exception_init(format_ex_4, -1858);
pragma exception_init(format_ex_5, -1861);
-- add any others you might get
begin
-- define all expected formats
l_formats := sys.odcivarchar2list('YYYY-MM-DD', 'DD/MM/YYYY', 'DD-MON-RRRR'); -- add others
for i in 1..l_formats.count loop
begin
return to_date(p_str, l_formats(i));
exception
when format_ex_1 or format_ex_2 or format_ex_3 or format_ex_4 or format_ex_5 then
-- ignore the exception; carry on and try the next format
null;
end;
end loop;
-- did not match any expected formats
return null;
end;
/
select *
from test
where safe_to_date(val) not between date '2016-02-01' and date '2016-02-29';
Although I wouldn't normally use between for dates; if you don't have any with times specified then you'd get away with it here.
You could use when others to catch any exception without having to declare them all, but even for this that's potentially dangerous - if something is breaking in a way you don't expect you want to know about it, not hide it.
Of course, this is an object lesson in why you should store numeric data in NUMBER columns and dates in DATE or TIMESTAMP fields - trying to extract useful information when everything is stored as strings is messy, painful and inefficient.

Answer 3

I think the best approach you can try in this condition is use
TRANSLATE function to eliminate the alphanumeric characters. Once its
done all now is OLD school technique to check the data by using NOT
BETWEEN function Hope this helps.
SELECT B.NM
FROM
(SELECT a.nm
FROM
(SELECT '0' AS nm FROM dual
UNION
SELECT '1' AS nm FROM dual
UNION
SELECT '68' AS nm FROM dual
UNION
SELECT '129' AS nm FROM dual
UNION
SELECT '200' AS nm FROM dual
UNION
SELECT '125a' AS nm FROM dual
)a
WHERE TRANSLATE(a.nm, ' +-.0123456789', ' ') IS NULL
)b
WHERE b.nm NOT BETWEEN 1 AND 128;