Oracle throws ORA-01722: invalid number in my SQL query and it is unclear why.
I have a table called "LIGHTS" and I want to get the lights with a WATTAGE <= 3. WATTAGE is stored as a VARCHAR2(40) for some reason, but each character does seem to be an integer or float. When I convert WATTAGE to a number using the query:
SELECT TO_NUMBER(WATTAGE) FROM LIGHTS
There's no problem. I get a result like this:
TO_NUMBER(WATTAGE)
1
7
-1
0
15
17.5
However, when I add a WHERE condition to filter the numbers for those less than 3, I get the ORA-01722: invalid number error:
SELECT WATTAGE FROM LIGHTS
WHERE TO_NUMBER(WATTAGE) <= 3
What could be going wrong?
ORA-01722: invalid number comes from the TO_NUMBER(), not from the conditional. I.e., try this and you'll get the same error:
SELECT TO_NUMBER('test') FROM dual;
This would indicate that at least one of your values is not numeric.
Alas Oracle doesn't have a simple way to check whether a string is in fact representing a number. (One of the many reasons to use the correct data type in the first place!)
However, you can write your own. Here is just a brief demo of this concept. I create a table with a column of VARCHAR2 data type, and populate it with a few strings, one of which is not a number.
create table tbl (nbr varchar2(100));
insert into tbl
select '103' from dual union all
select '-1.3' from dual union all
select 'abc' from dual
;
Then I create a small function with a nested block that should error out if TO_NUMBER fails. The error handler will "do something" specific to errors and then return control to the main function. Then I can use this in a WHERE clause. Here are the function and then how it can be used to find the offending values:
create or replace function not_a_number(str varchar2)
return varchar2
as
x number;
r varchar2(100);
begin
begin
x := to_number(str);
exception
when others then
r := str;
end;
return r;
end;
/
select nbr
from tbl
where not_a_number(nbr) is not null;
NBR
-------
abc
Related
DECLARE
dynaCol varchar2(200);
varCol varchar2(100);
BEGIN
dbms_output.put_line(dynaCol);
select name into varCol from GSSP_ETL.DEMO_TABLE_CHECK where ID='1'|| (SELECT q'$ and optional='N'$' NAME FROM DUAL) ;
dbms_output.put_line(varCol);
END;
Though I have a row for id=1 and optional=N I still get no data found error.
SELECT q'$ and optional='N'$' NAME FROM DUAL gives me and optional='N', so there is nothing wrong to use like this.
Please help if anyone know the reason or better way to append the and in where clause.
Your original query is looking for the row with ID = "1 and optional = 'N'" which causes the problem.
You should use dynamic SQL if you want to have variable WHERE clause:
SQL> create table t (id, name, optional)
2 as
3 select 1, 'XXX','N' from dual
4 /
SQL> select * from t;
ID NAM O
---------- --- -
1 XXX N
SQL> set serveroutput on
SQL> declare
2 dynaCol varchar2(200) := q'[ and optional='N']';
3 varCol varchar2(100);
4 p_id int := 1;
5 begin
6
7 execute immediate
8 ' select name from t where id = :x '||dynaCol
9 into varCol using p_id;
10 dbms_output.put_line(varCol);
11 end;
12 /
XXX
Sorry i couldn't comment due to my low reputation..
No,its not possible to do with out immediate i think so,the best way is what Dmitry did.
Its possible to send whole select statement dynamically but a part with out execute immediate..
...You got close.
Since you're using PL/SQL, why not consider creating a persistent object such as a PL/SQL Stored Procedure instead? There will be less code on the JBOSS side and the database schema layout will be further separated from the code you intended on accessing the database.
The Demonstration Table
This is an interpretation from pieces of your original post. This is the table I used to test out my recommendations.
CREATE TABLE "DEMO_TABLE_CHECK"
( "ID" NUMBER(10,0) NOT NULL ENABLE,
"NAME" VARCHAR2(40) NOT NULL ENABLE,
"OPTIONAL" VARCHAR2(5) NOT NULL ENABLE,
CONSTRAINT "DEMO_TABLE_CHECK_PK" PRIMARY KEY ("ID") ENABLE
)
/
My test data:
The Procedure Source Code
CREATE or REPLACE PROCEDURE proc_example(p_id IN number,
p_opt IN varchar2 default null)
IS
Result varchar2(100);
BEGIN
SELECT name
INTO Result
FROM GSSP_ETL.DEMO_TABLE_CHECK
WHERE id = p_id
AND optional = nvl(p_opt, optional);
dbms_output.put_line(Result);
END proc_example;
Notice that this example solution actually does away with the dynamic SQL calls proposed in previous solutions. The realm of PL/SQL procedural code is likely to accommodate for pushing around and concatenating strings of SQL commands.
An Example Call and the Output
begin
proc_example(p_id=> 1, p_opt=>'N');
end;
-- output:
ALPHA
begin
proc_example(p_id=> 1, p_opt=> null);
proc_example(p_id=> 1);
end;
-- output:
ALPHA
ALPHA
All this PL/SQL code, which was originally in an a block passed from the part of your program that accessed the database will now reside on the database.
Discussion of the PL/SQL Procedure Design
The optional part of the query, represented by p_opt, has a default designation. This means if there is no value for that parameter, then the procedure will ignore it and assume it is equal to the defined default value. No errors will be thrown.
AND optional = nvl(p_opt, optional)
This line is a replacement for the add-on SQL string. the input parameter p_opt, whether it was supplied (such as = 'N') or skipped ( implied to = null ), the SQL script includes or excludes the effect of this operator based on the "switching" parameter supplied.
Closing Comments:
If you want to see a better differentiation of results based on that last, dynamic SQL command, you might want to try scenarios where the first criteria is actually ambiguous, such as multiple instances if the ID column. (If ID = 1 was true for more than one record...) But of that set of results, have the second criteria identify something unique when in combination with the first.
I am running a basic query that retrieves rows based on basic conditional clauses, nothing complex. This works fine:
<cfquery name="courses" datasource="banner">
SELECT *
FROM tjucatalog
WHERE (course_status = 'Active')
AND CONCAT(subject,course_no) IN (#PreserveSingleQuotes(courselist)#)
AND term IN ('Fall 2012')
AND ((end_date > #now()#) OR (course_meeting_info IS NOT NULL))
ORDER BY TYear, TSort, DayNum, start_date, time, title
</cfquery>
However, when I remove the "AND term IN" line from the query, it fails.
<cfquery name="courses" datasource="banner">
SELECT *
FROM tjucatalog
WHERE (course_status = 'Active')
AND CONCAT(subject,course_no) IN (#PreserveSingleQuotes(courselist)#)
AND ((end_date > #now()#) OR (course_meeting_info IS NOT NULL))
ORDER BY TYear, TSort, DayNum, start_date, time, title
</cfquery>
The error I get is: ORA-06502: PL/SQL: numeric or value error: character string buffer too small ORA-06512: at "BANINST1.TJUCATALOG_PACK", line 519
Is this maybe a view that requires the field 'term' to be included, or is there something else at play here I'm entirely unaware of?
This appears to be an error in a package that's being called under the hood, possibly from a view. You're querying against tjucatalog, and it seems plausible that is a view where one of the returned columns is actually a functional call.
It isn't necessarily the term column that's the problem. By removing that condition, more rows will be returned, and the function is being called against a column value from a row that isn't there when the condition is in place. But it could be any column in one of those now-visible rows.
As a simple and contrived example of what might be happening:
create table t42 (id number, foo varchar2(20));
insert into t42 (id, foo) values (1, 'Short');
insert into t42 (id, foo) values (2, 'More than 10');
create package p42 as
function func(p_id in number) return varchar2;
end p42;
/
create package body p42 as
function func(p_id in number) return varchar2 is
l_bar varchar2(10);
begin
select foo into l_bar from t42 where id = p_id;
return l_bar;
end func;
end p42;
/
create view v42 as select id, p42.func(id) as bar from t42;
So we have a table with two rows, one with a foo less than 10 characters, the other more than 10 characters. We have a (silly) package function that takes an id value, looks up foo, and returns it. And a view that uses that function.
This works:
select * from v42 where id = 1;
ID BAR
---------- --------------------
1 Short
But removing the condition causes it to fail:
select * from v42;
ERROR:
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at "SCOTT.P42", line 5
Line 5 of my package body is select foo into l_bar from t42 where id = p_id; and the problem is that I've declare l_bar as varchar2(10), which is too small for the foo value for id=2. I should have declared it as varchar2(20), or even better t42.foo%TYPE.
To see what your problematic function is doing, look at the source code, which you can get from the database (if it isn't wrapped) if you don't have it available:
select line, text from all_source
where owner = 'BANINST1'
and name = 'TJUCATALOG_PACK'
and type = 'PACKAGE BODY'
order by line;
Our Oracle gurus returned and told us they had to change a field type from varchar2 (4000) to CLOB. The lack of the term field as a filter clause was a red herring error. I don't know which field specifically in the query needed to be increased for allowed length, but it works so I'm happy.
I'm trying to run a query that has a few columns that are a CLOB datatype. If i run the query like normal, all of those fields just have (CLOB) as the value.
I tried using DBMS_LOB.substr(column) and i get the error
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
How can i query the CLOB column?
This works
select DBMS_LOB.substr(myColumn, 3000) from myTable
When getting the substring of a CLOB column and using a query tool that has size/buffer restrictions sometimes you would need to set the BUFFER to a larger size. For example while using SQL Plus use the SET BUFFER 10000 to set it to 10000 as the default is 4000.
Running the DBMS_LOB.substr command you can also specify the amount of characters you want to return and the offset from which. So using DBMS_LOB.substr(column, 3000) might restrict it to a small enough amount for the buffer.
See oracle documentation for more info on the substr command
DBMS_LOB.SUBSTR (
lob_loc IN CLOB CHARACTER SET ANY_CS,
amount IN INTEGER := 32767,
offset IN INTEGER := 1)
RETURN VARCHAR2 CHARACTER SET lob_loc%CHARSET;
I did run into another condition with HugeClob in my Oracle database. The dbms_lob.substr only allowed a value of 4000 in the function, ex:
dbms_lob.substr(column,4000,1)
so for my HughClob which was larger, I had to use two calls in select:
select dbms_lob.substr(column,4000,1) part1,
dbms_lob.substr(column,4000,4001) part2 from .....
I was calling from a Java app so I simply concatenated part1 and part2 and sent as a email.
If it's a CLOB why can't we to_char the column and then search normally ?
Create a table
CREATE TABLE MY_TABLE(Id integer PRIMARY KEY, Name varchar2(20), message clob);
Create few records in this table
INSERT INTO MY_TABLE VALUES(1,'Tom','Hi This is Row one');
INSERT INTO MY_TABLE VALUES(2,'Lucy', 'Hi This is Row two');
INSERT INTO MY_TABLE VALUES(3,'Frank', 'Hi This is Row three');
INSERT INTO MY_TABLE VALUES(4,'Jane', 'Hi This is Row four');
INSERT INTO MY_TABLE VALUES(5,'Robert', 'Hi This is Row five');
COMMIT;
Search in the clob column
SELECT * FROM MY_TABLE where to_char(message) like '%e%';
Results
ID NAME MESSAGE
===============================
1 Tom Hi This is Row one
3 Frank Hi This is Row three
5 Robert Hi This is Row five
For big CLOB selects also can be used:
SELECT dbms_lob.substr( column_name, dbms_lob.getlength(column_name), 1) FROM foo
Another option is to create a function and call that function everytime you need to select clob column.
create or replace function clob_to_char_func
(clob_column in CLOB,
for_how_many_bytes in NUMBER,
from_which_byte in NUMBER)
return VARCHAR2
is
begin
Return substrb(dbms_lob.substr(clob_column
,for_how_many_bytes
,from_which_byte)
,1
,for_how_many_bytes);
end;
and call that function as;
SELECT tocharvalue, clob_to_char_func(tocharvalue, 1, 9999)
FROM (SELECT clob_column AS tocharvalue FROM table_name);
To add to the answer.
declare
v_result clob;
begin
---- some operation on v_result
dbms_lob.substr( v_result, 4000 ,length(v_result) - 3999 );
end;
/
In dbms_lob.substr
first parameter is clob which you want to extract .
Second parameter is how much length of clob you want to extract.
Third parameter is from which word you want to extract .
In above example i know my clob size is more than 50000 , so i want last 4000 character .
If you are using SQL*Plus try the following...
set long 8000
select ...
I have been getting an intermittent issue when executing to_number function in the where clause on a varchar2 column if number of records exceed a certain number n. I used n as there is no exact number of records on which it happens. On one DB it happens after n was 1 million on another when it was 0.1. million.
E.g. I have a table with 10 million records say Table Country which has field1 varchar2 containing numberic data and Id
If I do a query as an example
select *
from country
where to_number(field1) = 23
and id >1 and id < 100000
This works
But if I do the query
select *
from country
where to_number(field1) = 23
and id >1 and id < 100001
It fails saying invalid number
Next I try the query
select *
from country
where to_number(field1) = 23
and id >2 and id < 100001
It works again
As I only got invalid number it was confusing, but in the log file it said
Memory Notification: Library Cache Object loaded into SGA
Heap size 3823K exceeds notification threshold (2048K)
KGL object name :with sqlplan as (
select c006 object_owner, c007 object_type,c008 object_name
from htmldb_collections
where COLLECTION_NAME='HTMLDB_QUERY_PLAN'
and c007 in ('TABLE','INDEX','MATERIALIZED VIEW','INDEX (UNIQUE)')),
ws_schemas as(
select schema
from wwv_flow_company_schemas
where security_group_id = :flow_security_group_id),
t as(
select s.object_owner table_owner,s.object_name table_name,
d.OBJECT_ID
from sqlplan s,sys.dba_objects d
It seems its related to SGA size, but google did not give me much help on this.
Does anyone have any idea about this issue with TO_NUMBER or oracle functions for large data?
which has field1 varchar2 containing
numberic data
This is not good practice. Numeric data should be kept in NUMBER columns. The reason is simple: if we don't enforce a strong data type we might find ourselves with non-numeric data in our varchar2 column. If that were to happen then a filter like this
where to_number(field1) = 23
would fail with ORA-01722: invalid number.
I can't for certain sure say this is what is happening in your scenario, because I don't understand why apparently insignificant changes in the filters of ID have changed the success of the query. It would be instructive to see the execution plans for the different versions of the queries. But I think it is more likely to be a problem with your data than a bug in the SGA.
Assuming you know that the given range of ids will always result in field1 containing numeric data, you could do this instead:
select *
from (
select /*+NO_MERGE*/ *
from country
where id >1 and id < 100000
)
where to_number(field1) = 23;
Suggest doing the following to determine for sure whether there are records containing non-numeric data. As others have said, variations in the execution plan and order of evaluation could explain why the error does not appear consistently.
(assuming SQLPlus as the client)
SET SERVEROUTPUT ON
DECLARE
x NUMBER;
BEGIN
FOR rec IN (SELECT id, field1 FROM country) LOOP
BEGIN
x := TO_NUMBER( rec.field1 );
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line( rec.id || ' ' || rec.field1 );
END;
END LOOP;
END;
/
An alternative workaround to your original issue would be to rewrite the query to avoid implicit type conversion, e.g.
SELECT id, TO_NUMBER( field1 )
FROM county
WHERE field1 = '23'
AND <whatever condition on id you want, if any>
Consider writing an IS_NUMBER PL/SQL function:
CREATE OR REPLACE FUNCTION IS_NUMBER (p_input IN VARCHAR2) RETURN NUMBER
AS
BEGIN
RETURN TO_NUMBER (p_input);
EXCEPTION
WHEN OTHERS THEN RETURN NULL;
END IS_NUMBER;
/
SQL> SELECT COUNT(*) FROM DUAL WHERE IS_NUMBER ('TEST') IS NOT NULL;
COUNT(*)
----------
0
SQL> SELECT COUNT(*) FROM DUAL WHERE IS_NUMBER ('123.45') IS NOT NULL;
COUNT(*)
----------
1
Let's say I have a function call on a select or where clause in Oracle like this:
select a, b, c, dbms_crypto.hash(utl_raw.cast_to_raw('HELLO'),3)
from my_table
A similar example can be constructed for MS SQLServer.
What's the expected behavior in each case?
Is the HASH function going to be called once for each row in the table, or DBMS will be smart enough to call the function just once, since it's a function with constant parameters and no side-effects?
Thanks a lot.
The answer for Oracle is it depends. The function will be called for every row selected UNLESS the Function is marked 'Deterministic' in which case it will only be called once.
CREATE OR REPLACE PACKAGE TestCallCount AS
FUNCTION StringLen(SrcStr VARCHAR) RETURN INTEGER;
FUNCTION StringLen2(SrcStr VARCHAR) RETURN INTEGER DETERMINISTIC;
FUNCTION GetCallCount RETURN INTEGER;
FUNCTION GetCallCount2 RETURN INTEGER;
END TestCallCount;
CREATE OR REPLACE PACKAGE BODY TestCallCount AS
TotalFunctionCalls INTEGER := 0;
TotalFunctionCalls2 INTEGER := 0;
FUNCTION StringLen(SrcStr VARCHAR) RETURN INTEGER AS
BEGIN
TotalFunctionCalls := TotalFunctionCalls + 1;
RETURN Length(SrcStr);
END;
FUNCTION GetCallCount RETURN INTEGER AS
BEGIN
RETURN TotalFunctionCalls;
END;
FUNCTION StringLen2(SrcStr VARCHAR) RETURN INTEGER DETERMINISTIC AS
BEGIN
TotalFunctionCalls2 := TotalFunctionCalls2 + 1;
RETURN Length(SrcStr);
END;
FUNCTION GetCallCount2 RETURN INTEGER AS
BEGIN
RETURN TotalFunctionCalls2;
END;
END TestCallCount;
SELECT a,TestCallCount.StringLen('foo') FROM(
SELECT 0 as a FROM dual
UNION
SELECT 1 as a FROM dual
UNION
SELECT 2 as a FROM dual
);
SELECT TestCallCount.GetCallCount() AS TotalFunctionCalls FROM dual;
Output:
A TESTCALLCOUNT.STRINGLEN('FOO')
---------------------- ------------------------------
0 3
1 3
2 3
3 rows selected
TOTALFUNCTIONCALLS
----------------------
3
1 rows selected
So the StringLen() function was called three times in the first case. Now when executing with StringLen2() which is denoted deterministic:
SELECT a,TestCallCount.StringLen2('foo') from(
select 0 as a from dual
union
select 1 as a from dual
union
select 2 as a from dual
);
SELECT TestCallCount.GetCallCount2() AS TotalFunctionCalls FROM dual;
Results:
A TESTCALLCOUNT.STRINGLEN2('FOO')
---------------------- -------------------------------
0 3
1 3
2 3
3 rows selected
TOTALFUNCTIONCALLS
----------------------
1
1 rows selected
So the StringLen2() function was only called once since it was marked deterministic.
For a function not marked deterministic, you can get around this by modifying your query as such:
select a, b, c, hashed
from my_table
cross join (
select dbms_crypto.hash(utl_raw.cast_to_raw('HELLO'),3) as hashed from dual
);
For SQL server, it will be evaluated for every single row.
You will be MUCH better off by running the function once and assigning to a variable and using the variable in the query.
short answer....it depends.
If the function is accessing data ORACLE does not know if it is going to be the same for each row, therefore, it needs to query for each. If, for example, your function is just a formatter that always returns the same value then you can turn on caching (marking it as Deterministic) which may allow for you to only do the function call once.
Something you may want to look into is ORACLE WITH subquery:
The WITH query_name clause lets you
assign a name to a subquery block. You
can then reference the subquery block
multiple places in the query by
specifying the query name. Oracle
optimizes the query by treating the
query name as either an inline view or
as a temporary table
I got the quoted text from here, which has plenty of examples.