I am having trouble loading decimal data into a database - specifically, my negative numbers are getting truncated, and I can't figure it out.
Here is what my query looks like:
> CREATE TABLE IF NOT EXISTS mytable (id INT(12) NOT NULL AUTO_INCREMENT,
mydecimal DECIMAL(13,2),PRIMARY KEY(id));
> LOAD DATA INFILE 'data.dat' INTO TABLE mytable FIELDS TERMINATED BY ';';
And the data.dat that I'm loading:
;000000019.50 ;
;000000029.50-;
;000000049.50 ;
When it completes, giving me a warning that "Data truncated for column 'mydecimal' at row 2." And when I look at the data, it's stored as positive number. Any ideas how to fix this?
The best way to handle data abnormalities like this in the input file is to load them into a local variable, then set the actual column value based on a transformation of the local variable.
In your case, you can load the strings into a local variable, then either leave it alone or multiply by negative one depending on whether it ends with a minus sign.
Something like this should work for you:
LOAD DATA INFILE 'data.dat'
INTO TABLE mytable FIELDS TERMINATED BY ';'
(id,#mydecimal)
set mydecimal = IF(#mydecimal like '%-',#mydecimal * -1,#mydecimal);
I'm not sure why you're putting the minus sign after the number rather than before it. Does it work when you place the '-' sign at the start of the line?
you can consider this
CREATE TABLE IF NOT EXISTS mytable (id INT(12) NOT NULL AUTO_INCREMENT,
mydecimal varchar(255),PRIMARY KEY(id));
LOAD DATA INFILE 'data.dat' INTO TABLE mytable FIELDS TERMINATED BY ';';
update mytable set mydecimal =
cast(mydecimal as decimal(13,2))*if (substring(mydecimal, -1)='-', -1, 1);
alter table mytable modify column mydecimal decimal(13,2) signed;
Related
For example, I have a column with type int.
The raw data source has integer values, but the null values, instead of being empty (''), is 'NIL'
How would I handle those values when trying to Bulk Insert into MSSQL?
My code is
create table test (nid INT);
bulk insert test from #FILEPATH with (format="CSV", firstrow=2);
the first 5 rows of my .csv file looks like
1
2
3
NIL
7
You can replace the nil with " (empty string) directly in your data source file or insert the data into a staging table and transform it:
BULK INSERT staging_sample_data
FROM '\\data\sample_data.dat';
INSERT INTO [sample_data]
SELECT NULLIF(ColA, 'nil'), NULLIF(ColB, 'nil'),...
Of course if your field is for example a numeric, the staging table should have a string field. Then, you can do as Larnu offers: 'TRY_CONVERT(INT, ColA)'.
*Note: if there are default constraints you may need to check how to keep nulls
I am trying to convert a column formatted in varchar to decimal(19,12) with the following line of code
ALTER TABLE [tablename]
ALTER COLUMN [columnname][format]
and get the following prompt:
Msg 8114, Level 16, State 5, Line 25
Error converting data type varchar to numeric.
Has worked before like a charm. The issue here seems to be that the values in the column are 19 or so digit numeric values formatted as text.
I tried to create a new column, pasted shortened cell values (used the left() function) into it from the original column but that doesn't seem to do the trick either since the code above ends up occationally with the additional "Arithmetic overflow occurred." message.
When some of the rows have incorrect values, ALTER COLUMN would not work. A typical course of action goes like this:
Add a new column of the desired type
Update the column with values that you would like to keep
Drop the old column
Rename the new column
Step 2 would go like this:
UPDATE MyTable
SET NewColumn =
CASE WHEN ISNUMERIC(OldColumn)=1 AND DATALENGTH(OldColumn) <= 19 THEN
CAST(OldColumn AS decimal(19,12))
ELSE
NULL
END
You could also turn ANSI warnings off with SET ANSI_WARNINGS OFF command, which would let you run ALTER COLUMN ignoring data trunction errors. The drawback of this approach is that potential errors get ignored. On the other hand, when you do conversion explicitly with a CASE expression you have an option to supply an alternative value for the error case (I used NULL above, but you can put any number you want).
Could you try to seperate your problem? This does work on SQL 2012:
set nocount on
if object_id ('tempdb..#t1') is not null drop table #t1
create table #t1 (c1 varchar(100))
insert #t1 values ('1234567.8901234567890')
select * from #t1
alter table #t1
alter column c1 decimal(19,12)
select * from #t1
If you play around a bit with the strings you easily can produce an arimetic overflow error. But 'Error converting data type varchar to numeric' needs character or empty sting.
Maybe you can try with your data?
How do I change the data type float to nvarchar in order to remove the scientific notation and still keep precision? Consider the following:
CREATE TABLE ConversionDataType (ColumnData FLOAT);
INSERT INTO ConversionDataType VALUES (25566685456126);
INSERT INTO ConversionDataType VALUES (12345545546845);
INSERT INTO ConversionDataType VALUES (12345545545257);
When I do a simple read I get the following data, as expected:
select * from ConversionDataType
ColumnData
------------------------------------
25566685456126
12345545546845
12345545545257
Now when I try update the data type to an nvarchar, it gets stored in scientific notation which is something I don't want:
update ConversionDataType
set ColumnData = CAST(ColumnData AS NVARCHAR)
The result set is as follows:
25566700000000
12345500000000
12345500000000
It replaces some digits and adds zeros after the 6th index. How can I go about this? I had a look at the Convert function but that is only for converting date time data types.
Being valid what others said in comment, if you just want to convert float to varchar without scientific notation, you need to convert to numeric. You can try this:
SELECT CAST(CAST(CAST(25566685456126291 AS FLOAT) AS NUMERIC) AS NVARCHAR)
Output:
C1
------------------------------
25566685456126292
Whereas
SELECT CAST(CAST(25566685456126291 AS FLOAT) AS NVARCHAR) AS C1
gives:
C1
------------------------------
2.55667e+016
If you need to change datatype, I think you should add a new column, update it and (if you want) delete the old column and rename the new column at the end.
CREATE TABLE TEST1 (C1 FLOAT)
INSERT INTO TEST1 VALUES (25566685456126291);
ALTER TABLE TEST1 ADD C2 VARCHAR(18)
UPDATE TEST1 SET C2=CAST(CAST(C1 AS NUMERIC) AS VARCHAR)
SELECT * FROM TEST1
Output:
C1 C2
---------------------- ------------------
2.55666854561263E+16 25566685456126292
FLOAT was a very bad decision as this is not a precise data type. If you wanted to store the phone numbers as numbers, you'd have to go for DECIMAL instead.
But you'll have to use NVARCHAR instead. And this is the only reasonable design, as phone numbers can have leading zeros or start with a plus sign. So the first thing is to introduce an NVARCHAR column:
ALTER TABLE ConversionDataType ADD ColumnDataNew NVARCHAR(30);
The function to convert a number into a string in SQL Server is FORMAT. It lets you state the format you want to use for the conversion, which is integer in your case (a simple '0'):
update ConversionDataType set ColumnDataNew = format(ColumnData, '0');
At last remove the old column and then rename the new one with the same name. SQL Server lacks an ALTER TABLE syntax to rename a column, so we must call sp_RENAME instead (at least this is what I have read on the Internet; here is a link to the docs: https://msdn.microsoft.com/de-de/library/ms188351.aspx).
ALTER TABLE ConversionDataType DROP COLUMN ColumnData;
EXEC sp_RENAME 'ConversionDataType.ColumnDataNew', 'ColumnData', 'COLUMN';
Here you can see the results: http://rextester.com/GLLB27702
SELECT CONVERT(NVARCHAR(250), StudentID) FROM TableA
StudentID is your Float Column of database
or Simply use
SELECT CONVERT(NVARCHAR(250), yourFloatVariable)
I have a database table of that I have used to store the data returned from a web spider. I have a column that contains ticket prices for different events all in the varchar type (as the scrapy spider has to scrape the data in unicode). I'm trying to return the min price of the column and since the min() function only works for data of type INT, I tried to convert the column to integers using a solution from this SO post:
ALTER TABLE vs_tickets ALTER COLUMN ticketprice TYPE integer USING (ticketprice::integer);
but I got the error: ERROR: invalid input syntax for integer:
I also tried: change_column :vs_tickets, :ticketprice, 'integer USING CAST(ticketprice AS integer)' but that didn't work either.
What is the proper way to convert the column to type INT?
Edit:
You have decimal places in the string, so a simple cast is not going to work. You can do a double conversion:
cast(cast(ticketprice as decimal(10, 2)) as int)
or:
(ticketprice::decimal(10, 2))::int
(The parens are not strictly necessary.)
EDIT:
Or, as Erwin points out, just use numeric:
(ticketprice::numeric)::int
Postgres is much smarter about numeric than most other databases . . . after all, it supports numbers that are egregiously large ;)
The final query is:
ALTER TABLE vs_tickets
ALTER COLUMN ticketprice TYPE integer USING (ticketprice::numeric::integer);
I'm going to bet on your column have wrong characters.
Also you may want use float or numeric because you will lose decimals if convert to integers.
You need create a function to check if a text is numeric like this isnumeric-with-postgresql
Then check each row like this
select ticketprice
from vs_tickets
where ISNUMERIC(ticketprice) = false;
As your comment you also should try
SELECT ticketprice::float
You will be best off adding an INT column, moving your data with a cast and then removing the old varchar column.
ALTER TABLE vs_tickets ADD COLUMN ticketprice_int TYPE int;
GO
update vs_tickets SET ticketprice_int = cast(ticketprice as int);
// if you fail to cast the varchar to int you can use Gordon's method
// update vs_tickets SET ticketprice_int = cast(cast(ticketprice as decimal(10, 2)) as int);
GO
ALTER TABLE vs_tickets DROP COLUMN ticketprice;
GO
ALTER TABLE vs_tickets RENAME COLUMN ticketprice_int to ticketprice;
GO
With this at minimum you will be able to tell if and where a cast/convert fails and be able to check and recheck at each step before you can't turn back.
I'm working with MS SQL SERVER 2003. I want to change a column in one of my tables to have fewer characters in the entries. This is identical to this question: Altering a Table Column to Accept More Characters except for the fact that I want fewer characters instead of more.
I have a column in one of my tables that holds nine-digit entries. A developer previously working on the table mistakenly set the column to hold ten-digit entries. I need to change the type from CHAR(10) to CHAR(9).
Following the instructions from the discussion linked above, I wrote the statement
ALTER TABLE [MY_TABLE] ALTER COLUMN [MY_COLUMN] CHAR(9);
This returns the error message "String or binary data would be truncated". I see that my nine-digit strings have a space appended to make them ten digits.
How do I tell SQL Server to discard the extra space and convert my column to a CHAR(9) type?
I think you get the error because there are some values in that table that are exactly 10 chars long (with no trailing spaces). Altering the table would thus cut these values to the length 9.
This is not allowed by default. If there only would be strings which would have some trailing spaces, there would be no problem with that.
So, if you are ok with cutting those values, do
UPDATE MY_TABLE SET MY_COLUMN = LEFT(MY_COLUMN, 9)
first, after that do the alter.
Disable Ansi Warnings before you alter your table.
SET ANSI_WARNINGS OFF
Beware that data will be truncated if you happen to have something 10 characters long.
Edit
Check existing lengths before actually changing the column length.
SET ANSI_WARNINGS OFF
GO
CREATE TABLE Test (Value CHAR(10))
INSERT INTO Test SELECT ('1234567890')
IF NOT EXISTS (SELECT * FROM Test WHERE LEN(Value) > 9)
ALTER TABLE Test ALTER COLUMN Value CHAR(9)
ELSE
SELECT LEN(Value), * FROM Test WHERE LEN(Value) > 9
DROP TABLE Test
The above 2 answers didn't work for me because I had a history table attached to the table I was trying to alter. This is what I did:
UPDATE MY_TABLE SET MY_COLUMN = LEFT(MY_COLUMN, 9)
ALTER TABLE MY_TABLE SET (SYSTEM_VERSIONING = OFF)
-- Might need to UPDATE MY_TABLEHistory SET MY_COLUMN = LEFT(MY_COLUMN, 9)
-- I didn't
ALTER TABLE MY_TABLEHistory ALTER COLUMN [MY_COLUMN] [varchar] (9) NULL
ALTER TABLE MY_TABLE SET (SYSTEM_VERSIONING = ON(HISTORY_TABLE = MY_TABLEHistory))
ALTER TABLE MY_TABLE ALTER COLUMN [MY_COLUMN] [varchar] (9) NULL