I have 3 tables in the database that I'm working on. Out of 3, two of the tables have columns that include dates. When I checked the information schema of the columns I found that dates have the wrong data type. If you see the picture below, the highlighted columns should be stored as DATE data type.
So, I used the following query to change their data type from varchar to DATE:
ALTER TABLE [dbo].[Customer]
ALTER COLUMN DOB DATE;
ALTER TABLE [dbo].[Transactions]
ALTER COLUMN tran_date DATE;
The error that I get is:
Conversion failed when converting date and/or time from character string.
Please let me know how I can fix this error. Thanks!
What you can do is update the value using try_convert() first and then alter the column name. Note: This will set any invalid values to NULL.
update customer
set dob = try_convert(date, dob);
alter table customer alter column dbo date;
If you want to see the bad values, then before you change the table, run:
select c.*
from customer c
where try_convert(date, dob) is null and dob is not null;
You may have other ideas on how to fix the values.
You can't change from varchar to date or time or datetime by altering the column. Why? Because SQL Server does not know if the field contains '1/1/2020' or 'My dog is cute'. You will have to rebuild the table with the proper data types and then CAST() or CONVERT() the values to a true date time.
Underneath the hood, this makes more sense. A char/varchar uses one byte per character. nchar/nvarchar uses 2 bytes per character. A datetime is a number, not a character. This means you need a routine that changes this into the correct number. If you want to get deeper, the number is the number of ticks (nanoseconds) since midnight on January 1, 0001 in the Gregorian Calendar. More info.
I'm working with a rather large Insert Into . . Select . . From . .
I have over 500+ lines of SQL in this script and I'm getting this error:
INSERT INTO MtgeMaster ( [Col1]
,[Col2]
,[Col3]
, etc., etc. )
SELECT [Col1]
,[Col2]
,[Col3]
, etc., etc.
FROM MtgeMktg
When I run the code above I get this error:
Msg 8114, Level 16, State 5, Line 164
Error converting data type varchar to numeric.
It looks like the error comes from line 164, but line 164 is literally my [Col1] field, and this is VARCHAR. So, I'm going from VARCHAR to VARCHAR. There is no VARCHAR to NUMERIC.
Also, if I add a couple of blank lines and re-run the process, I get this:
Msg 8114, Level 16, State 5, Line 166
Error converting data type varchar to numeric.
All it's really doing is going to the line with the INSERT INTO clause.
The error must be coming from another line, but it's hard to tell what's throwing the error when I have 500+ lines of SQL to go through.
SQL Server does not makes this easy. I have found that a brute force approach is necessary. I like to start by loading the data into a staging table where all the columns are strings. This makes it easier to manipulate.
You can use one of two methods to find the error. The first is to use try_convert() on each column to determine where the error is.
The second is to do a binary search to find the offending row. Load the first half of the data to see if the error is there. Then divide that half in half. And so on.
It looks like the error before and after adding few blank lines is same. It is probably a datatype conversion issue. You could try using cast() function and convert the data type of the [Col1] in select field of MtgeMktg table to match with the datatype of [Col1] of the MtgeMaster table.
So, here's my attempt to answer this question, based on what is given.
I would review the table structure of MtgeMaster, and see what columns are supposed to be numeric. Let's just say, for this example, Col3 of MtgeMaster is numeric. You may have multiple numeric columns.
I would then query MtgeMktg and check whether the column you're trying to save from MtgeMktg to MtgeMaster is numeric or not. You may have to do this for each column that is numeric in MtgeMaster. If I were doing it (which I did in SQLFiddle), it would be something to the nature of the following:
select * from MtgeMktg where ISNUMERIC(Col3) = 0
Anything that returns from this query will tell you what rows have a non numeric column.
Simple fiddle listed here:
Obviously, you would have 1 of 2 decisions to make at this point. Fix the data, or filter out the rows that have the non-numeric data in it. I presume you'd need the rows though.
A very easy one for someone,
The following insert is giving me the
ORA-01722: invalid number
why?
INSERT INTO CUSTOMER VALUES (1,'MALADY','Claire','27 Smith St Caulfield','0419 853 694');
INSERT INTO CUSTOMER VALUES (2,'GIBSON','Jake','27 Smith St Caulfield','0415 713 598');
INSERT INTO CUSTOMER VALUES (3,'LUU','Barry','5 Jones St Malvern','0413 591 341');
INSERT INTO CUSTOMER VALUES (4,'JONES','Michael','7 Smith St Caulfield','0419 853 694');
INSERT INTO CUSTOMER VALUES (5,'MALADY','Betty','27 Smith St Knox','0418 418 347');
An ORA-01722 error occurs when an attempt is made to convert a character string into a number, and the string cannot be converted into a number.
Without seeing your table definition, it looks like you're trying to convert the numeric sequence at the end of your values list to a number, and the spaces that delimit it are throwing this error. But based on the information you've given us, it could be happening on any field (other than the first one).
Suppose tel_number is defined as NUMBER - then the blank spaces in this provided value cannot be converted into a number:
create table telephone_number (tel_number number);
insert into telephone_number values ('0419 853 694');
The above gives you a
ORA-01722: invalid number
Here's one way to solve it. Remove non-numeric characters then cast it as a number.
cast(regexp_replace('0419 853 694', '[^0-9]+', '') as number)
Well it also can be :
SELECT t.col1, t.col2, ('test' + t.col3) as test_col3
FROM table t;
where for concatenation in oracle is used the operator || not +.
In this case you get : ORA-01722: invalid number ...
This is because:
You executed an SQL statement that tried to convert a string to a
number, but it was unsuccessful.
As explained in:
Oracle/PLSQL: ORA-01722 Error.
To resolve this error:
Only numeric fields or character fields that contain numeric values
can be used in arithmetic operations. Make sure that all expressions
evaluate to numbers.
As this error comes when you are trying to insert non-numeric value into a numeric column in db it seems that your last field might be numeric and you are trying to send it as a string in database. check your last value.
Oracle does automatic String2number conversion, for String column values! However, for the textual comparisons in SQL, the input must be delimited as a String explicitly: The opposite conversion number2String is not performed automatically, not on the SQL-query level.
I had this query:
select max(acc_num) from ACCOUNTS where acc_num between 1001000 and 1001999;
That one presented a problem: Error: ORA-01722: invalid number
I have just surrounded the "numerical" values, to make them 'Strings', just making them explicitly delimited:
select max(acc_num) from ACCOUNTS where acc_num between '1001000' and '1001999';
...and voilà: It returns the expected result.
edit:
And indeed: the col acc_num in my table is defined as String. Although not numerical, the invalid number was reported. And the explicit delimiting of the string-numbers resolved the problem.
On the other hand, Oracle can treat Strings as numbers. So the numerical operations/functions can be applied on the Strings, and these queries work:
select max(string_column) from TABLE;
select string_column from TABLE where string_column between '2' and 'z';
select string_column from TABLE where string_column > '1';
select string_column from TABLE where string_column <= 'b';
In my case the conversion error was in functional based index, that I had created for the table.
The data being inserted was OK. It took me a while to figure out that the actual error came from the buggy index.
Would be nice, if Oracle could have gave more precise error message in this case.
If you do an insert into...select * from...statement, it's easy to get the 'Invalid Number' error as well.
Let's say you have a table called FUND_ACCOUNT that has two columns:
AID_YEAR char(4)
OFFICE_ID char(5)
And let's say that you want to modify the OFFICE_ID to be numeric, but that there are existing rows in the table, and even worse, some of those rows have an OFFICE_ID value of ' ' (blank). In Oracle, you can't modify the datatype of a column if the table has data, and it requires a little trickery to convert a ' ' to a 0. So here's how to do it:
Create a duplicate table: CREATE TABLE FUND_ACCOUNT2 AS SELECT * FROM FUND_ACCOUNT;
Delete all the rows from the original table: DELETE FROM FUND_ACCOUNT;
Once there's no data in the original table, alter the data type of its OFFICE_ID column: ALTER TABLE FUND_ACCOUNT MODIFY (OFFICE_ID number);
But then here's the tricky part. Because some rows contain blank OFFICE_ID values, if you do a simple INSERT INTO FUND_ACCOUNT SELECT * FROM FUND_ACCOUNT2, you'll get the "ORA-01722 Invalid Number" error. In order to convert the ' ' (blank) OFFICE_IDs into 0's, your insert statement will have to look like this:
INSERT INTO FUND_ACCOUNT (AID_YEAR, OFFICE_ID) SELECT AID_YEAR, decode(OFFICE_ID,' ',0,OFFICE_ID) FROM FUND_ACCOUNT2;
I have found that the order of your SQL statement parameters is also important and the order they are instantiated in your code, this worked in my case when using "Oracle Data Provider for .NET, Managed Driver".
var sql = "INSERT INTO table (param1, param2) VALUES (:param1, :param2)";
...
cmd.Parameters.Add(new OracleParameter("param2", Convert.ToInt32("100")));
cmd.Parameters.Add(new OracleParameter("param1", "alpha")); // This should be instantiated above param1.
Param1 was alpha and param2 was numeric, hence the "ORA-01722: invalid number" error message. Although the names clearly shows which parameter it is in the instantiation, the order is important. Make sure you instantiate in the order the SQL is defined.
For me this error was a bit complicated issue.
I was passing a collection of numbers (type t_numbers is table of number index by pls_integer;) to a stored procedure. In the stored proc there was a bug where numbers in this collection were compared to a varchar column
select ... where ... (exists (select null from table (i_coll) ic where ic.column_value = varchar_column))
Oracle should see that ic.column_value is integer so shouldn't be compared directly to varchar but it didn't (or there is trust for conversion routines).
Further complication is that the stored proc has debugging output, but this error came up before sp was executed (no debug output at all).
Furthermore, collections [<empty>] and [0] didn't give the error, but for example [1] errored out.
The ORA-01722 error is pretty straightforward. According to Tom Kyte:
We've attempted to either explicity or implicity convert a character string to a number and it is failing.
However, where the problem is is often not apparent at first. This page helped me to troubleshoot, find, and fix my problem. Hint: look for places where you are explicitly or implicitly converting a string to a number. (I had NVL(number_field, 'string') in my code.)
This happened to me too, but the problem was actually different: file encoding.
The file was correct, but the file encoding was wrong. It was generated by the export utility of SQL Server and I saved it as Unicode.
The file itself looked good in the text editor, but when I opened the *.bad file that the SQL*loader generated with the rejected lines, I saw it had bad characters between every original character. Then I though about the encoding.
I opened the original file with Notepad++ and converted it to ANSI, and everything loaded properly.
In my case it was an end of line problem, I fixed it with dos2unix command.
In my case I was trying to Execute below query, which caused the above error ( Note : cus_id is a NUMBER type column)
select *
from customer a
where a.cus_id IN ('115,116')
As a solution to the caused error, below code fragment(regex) can be used which is added in side IN clause (This is not memory consuming as well)
select *
from customer a
where a.cus_id IN (select regexp_substr (
com_value,
'[^,]+',
1,
level
) value
from (SELECT '115,116' com_value
FROM dual)rws
connect by level <=
length ( com_value ) - length ( replace ( com_value, ',' ) ) + 1)
try this as well, when you have a invalid number error
In this
a.emplid is number and b.emplid is an varchar2 so if you got to convert one of the sides
where to_char(a.emplid)=b.emplid
You can always use TO_NUMBER() function in order to remove this error.This can be included as INSERT INTO employees phone_number values(TO_NUMBER('0419 853 694');
The question
Is it possible to ask SSIS to cast a value and return NULL in case the cast is not allowed instead of throwing an error ?
My environment
I'm using Visual Studio 2005 and Sql Server 2005 on Windows Server 2003.
The general context
Just in case you're curious, here is my use case. I have to store data coming from somewhere in a generic table (key/value structure with history) witch contains some sort of value that can be strings, numbers or dates. The structure is something like this :
table Values {
Id int,
Date datetime, -- for history
Key nvarchar(50) not null,
Value nvarchar(50),
DateValue datetime,
NumberValue numeric(19,9)
}
I want to put the raw value in the Value column and try to put the same value
in the DateValue column when i'm able to cast it to Datetime
in the NumberValue column when i'm able to cast it to a number
Those two typed columns would make all sort of aggregation and manipulation much easier and faster later.
That's it, now you know why i'm asking this strange question.
============
Thanks in advance for your help.
You could also try a Derived Column component and test the value of the potential date/number field or simply cast it and redirect any errors as being the NULL values for these two fields.
(1) If you just simply cast the field every time with a statement like this in the Derived Column component: (DT_DATE)[MYPOTENTIALDATE] - you can redirect the rows that fail this cast and manipulate the data from there.
OR
(2) You can do something like this in the Derived Column component: ISNULL([MYPOTENTIALDATE]) ? '2099-01-01' : (DT_DATE)[MYPOTENTIALDATE]. I generally send through '2099-01-01' when a date is NULL rather than messing with NULL (works better with Cubes, etc).
Of course (2) won't work if the [MYPOTENTIALDATE] field comes through as other things other than a DATETIME or NULL, i.e., sometimes it is a word like "hello".
Those are the options I would explore, good luck!
In dealing with this same sort of thing I found the error handling in SSIS was not specific enough. My approach has been to actually create an errors table, and query a source table where the data is stored as varchar, and log errors to the error table with something like the below. I have one of the below statements for each column, because it was important for me to know which column failed. Then after I log all errors, I do a INSERT where I select those records in SomeInfo that do not have an errors. In your case you could do more advanced things based on the ColumnName in the errors table to insert default values.
INSERT INTO SomeInfoErrors
([SomeInfoId]
,[ColumnName]
,[Message]
,FailedValue)
SELECT
SomeInfoId,
'PeriodStartDate',
'PeriodStartDate must be in the format MM/DD/YYYY',
PeriodStartDate
FROM
SomeInfo
WHERE
ISDATE(PeriodStartDate) = 0 AND [PeriodStartDate] IS NOT NULL;
Tru using a conditional split and have the records where the data is a date go along one path and the other go along a different path where they are updated to nullbefore being inserted.
I am getting this Error on Insert statement to AS400 database, using Java with JDBC.
I figured it out, I had a numeric field with length 4 and decimal digits 2, In my insert I was trying to insert 100 in this numeric field which gave this error.
I altered the table and made it numeric with length 9 and decimal digits 2 and the insert worked.
even though the error says "Conversion error", actually its the wrong field length.
I ran in to this issue when a table was updated from 2 digits to 3 for user ID's and the query inserts in to a history table where the history table was not updated to 3 digits for the user id. Used Alter Table to correct the history table and all is well.
alter table "Table Name" ALTER COLUMN "COLUMN NAME" SET DATA TYPE NUMERIC(3)
The error message is not intuitive:
Conversion error on variable or parameter *N (#-302) [IBM][System
iAccess ODBC Driver][DB2 for i5/OS]SQL0012 - Correlation without
qualification occurred for column "columnname" to table "Tablename".
(#12)
I got this error today running the following query:
select *
from mytable
where invoice_date >= '2019-01-01'
Turned out invoice "date" is not a date... it's a decimal that impersonates a date. Poor database design, perhaps, but nonetheless the simple fix was:
select *
from mytable
where invoice_date >= 20190101
My issue was I overlooked the fact that I had single quotes around my parameter place holder. So I was getting a, 6 -- Numeric data that is not valid. error.
So, for example:
select someColumn
From favTable
where someOtherColumn = '?'
command.Parameters.Add("decVal", OleDbType.Decimal, 10).Value = model.someDecVal;
Correcting to:
select someColumn
From favTable
where someOtherColumn = ?
solved my issue.
Looks easy to catch here, but with big queries, this can be easily overlooked. Wasted about an hour on this.