My table looks like this:
CREATE TABLE test_ids (id STRING)
When I insert a large numeric value to this string column, it gets rounded:
INSERT INTO test_ids SELECT "12345678901234567890"
When checking the stored value, it comes back like this:
12345678901234567000
Notes: this is on a Hortonworks deployment of hive 3.1.0
I finally realized this rounding is not done in hive, but in Zeppelin UI where I run my SELECT query which returns rounded result. There is an open bug for this issue: https://issues.apache.org/jira/browse/ZEPPELIN-1434
It is only rounding when showing on UI, so underlying data is correct.
Related
I have two a table and a view . The table if of two rows of datatypes nvarchar and money. I have being updating the table by selecting from the view like below.
Insert into MyTable
Select * from MyView
Recently, this update fails due to an error "String or binary data would be truncated." However, when i modified by select statement to something like.
Select * from Myview WHERE Column is not null
OR
Select * from Myview WHERE Column > 0
The above work with a warning saying Warning: Null value is eliminated by an aggregate or other SET operation. . It occurred to me that may may be one of the null value records contain something that's not null. My table column is of money type and accept null. I presumed the error may be due to something that's not of money data type. The record is huge. Is there any way i can filter and return those aliens records?
I also i learnt that i can eliminate the error by turning ANSI WARNING SETTION ON & OFF Here . My concern is wouldn't that result in loss of data. Please any help would be appreciated.
String or binary data would be truncated happened because the data coming from the MyView is larger than the column size in MyTable
Use
Select Max(Len(FieldName)) From MyTable
to check the maximum length of the nvarchar field in the MyTable
Or you can use Left when inserting data something Llike this
Insert into MyTable
Select Left(FieldName,50), Column1 from MyView
Note the 50 should be the size of the nvarchar field in MyTable
String or binary data would be truncated is a very common error. It usually happens when we try to insert any data in string (varchar,nvarchar,char,nchar) data type column which is more than size of the column. So you need to check the data size with respect to the column width and identify which column is creating problem and fix it.
Here is another thread of the same problem as yours in stackoverflow.
string or binary data would be truncated
Hope this will help.
Regards
looks like the data in some column in table MyView exceeds the limit of the corresponding one in table MyTable
I have an imported database which contain lots of float field.
When I imported it, all the float field converted to string and I need to convert it back.
I try alter table to change the data type but I keep getting error (even if the row is empty or null).
EDIT : The server is curently down so I can't get the error message at the moment. My DBMS is SQL Server.
you can do so if all data are really numeric type.
There may be some data which hv null --no problem here.
There may be some data which are blank(('') but not null--problem is because of this.
First you make a select query to retrieve all data which are blank
say,
Select * from table1 where col=''
now update these rows with null
update table1 set col=null where col=''
After this,I think you can easily convert it to float
References: error converting data type varchar column to float.
You can first add a new field in your table with datatype float and then update it using convert function:
UPDATE #tbl
SET
num = convert(FLOAT, text)
SELECT *
FROM
#tbl
I have a query like :
select * from table where varchar_column=Numeric_value
that is fine until I run an insert script. After the new data is inserted, I must use this query:
select * from table where varchar_column='Numeric_value'
Can inserting a certain kind of data cause it to no longer implicitly convert?
After the insert script, the error is Data conversion fails OLEDB Status = 2
And the second query does work
I'm not certain of this... the first may be doing an implicit conversion of the varchar_column to a numeric value. Not the other way around. But when you insert values into that column that's no longer convertable, it fails. However, with the second, you're doing a varchar to varchar comparison and all is right again with the world. My guess.
I have a query that's written dynamically (OO PHP via Joomla) to insert some values into a MySQL database. The form that a user fills out has a field on it for dollar amount, and if they leave that blank I want the value going into the system to be NULL. I've written out the query to the error log as it's running; this is what the query looks like:
INSERT INTO arrc_Voucher
(VoucherNbr,securityCode,sequentialNumber, TypeFlag, CreateDT, ActivatedDT, BalanceInit, BalanceCurrent, clientName)
VALUES
('6032100199108006', '99108006','12','V','2010-10-29 12:50:01','NULL','NULL','NULL','')
When I look in the database table, though, although ActivatedDT is set correctly to NULL, BalanceInit and BalanceCurrent are both 0.00. The ActivatedDT field is a datetime, while the other two are decimal(18,2), and all three are set in the table structure as default value NULL.
If I run a query like this:
UPDATE arrc_Voucher
SET BalanceInit = null
WHERE BalanceInit like "0%"
...it does set the value to null, so why isn't the initial insert query doing so? Is it because null is in quotes? And if so, why is it setting correctly for ActivatedDT?
remove the quotes around NULL. What's actually happening is it's trying to insert the string 'NULL' as a number, and since it can't be converted to a number it uses the default value 0.
As for why ActivatedDT works, I'm guessing that's a date field. Failure to convert a string into a date would normally result in setting the value to 0 (which gets formatted as something like '1969-12-31'), but if you have NO_ZERO_DATE mode enabled, then it would be set to NULL instead.
If you'd like MySQL to throw an error in cases like this, when invalid values are passed, you can set STRICT_ALL_TABLES or STRICT_TRANS_TABLES (make sure you read the part about the difference between them) or one of the emulation modes, like TRADITIONAL.
You can try this with the command SET sql_mode='TRADITIONAL', or by adding sql-mode="TRADITIONAL" in my.cnf.
When you insert NULL into a MySQL database, you cannot insert it with quotes around it. It tries to insert the varchar 'NULL'. If your idea worked, you would never be able to insert the actual word NULL into the DB.
Remove the single quotes when you want to insert NULL.
You are not setting the fields to NULL but to strings ('NULL').
The question
Is it possible to ask SSIS to cast a value and return NULL in case the cast is not allowed instead of throwing an error ?
My environment
I'm using Visual Studio 2005 and Sql Server 2005 on Windows Server 2003.
The general context
Just in case you're curious, here is my use case. I have to store data coming from somewhere in a generic table (key/value structure with history) witch contains some sort of value that can be strings, numbers or dates. The structure is something like this :
table Values {
Id int,
Date datetime, -- for history
Key nvarchar(50) not null,
Value nvarchar(50),
DateValue datetime,
NumberValue numeric(19,9)
}
I want to put the raw value in the Value column and try to put the same value
in the DateValue column when i'm able to cast it to Datetime
in the NumberValue column when i'm able to cast it to a number
Those two typed columns would make all sort of aggregation and manipulation much easier and faster later.
That's it, now you know why i'm asking this strange question.
============
Thanks in advance for your help.
You could also try a Derived Column component and test the value of the potential date/number field or simply cast it and redirect any errors as being the NULL values for these two fields.
(1) If you just simply cast the field every time with a statement like this in the Derived Column component: (DT_DATE)[MYPOTENTIALDATE] - you can redirect the rows that fail this cast and manipulate the data from there.
OR
(2) You can do something like this in the Derived Column component: ISNULL([MYPOTENTIALDATE]) ? '2099-01-01' : (DT_DATE)[MYPOTENTIALDATE]. I generally send through '2099-01-01' when a date is NULL rather than messing with NULL (works better with Cubes, etc).
Of course (2) won't work if the [MYPOTENTIALDATE] field comes through as other things other than a DATETIME or NULL, i.e., sometimes it is a word like "hello".
Those are the options I would explore, good luck!
In dealing with this same sort of thing I found the error handling in SSIS was not specific enough. My approach has been to actually create an errors table, and query a source table where the data is stored as varchar, and log errors to the error table with something like the below. I have one of the below statements for each column, because it was important for me to know which column failed. Then after I log all errors, I do a INSERT where I select those records in SomeInfo that do not have an errors. In your case you could do more advanced things based on the ColumnName in the errors table to insert default values.
INSERT INTO SomeInfoErrors
([SomeInfoId]
,[ColumnName]
,[Message]
,FailedValue)
SELECT
SomeInfoId,
'PeriodStartDate',
'PeriodStartDate must be in the format MM/DD/YYYY',
PeriodStartDate
FROM
SomeInfo
WHERE
ISDATE(PeriodStartDate) = 0 AND [PeriodStartDate] IS NOT NULL;
Tru using a conditional split and have the records where the data is a date go along one path and the other go along a different path where they are updated to nullbefore being inserted.