I am trying to import a csv file into SQL Server 2017 using bulk insert. Most of the int columns, except the primary key have NULLs (as blanks in the csv file) but when I import I get an error that it can't convert a string to type int for the specified target column. Since I don't know which column it is affected, I went and replaced all the blanks with 0s, but I still get the game error.
Here is what I get after I import:
I don't know what to do to make this work, but it doesn't seem to make sense to me.
The issue is not the NULLs. There is a column in your CSV that your are expecting to contain only integer values, but it in fact contains string values that cannot be converted to integers.
Here would be an example:
my_str_to_int_col
1.0
2.0
3.O
4.0
5.0
...
Notice that 3.O should actually be 3.0
So you need to determine which column you are converting to an integer contains non-integer values.
Related
I have a flat CSV file and some fields contain a value like "1.8e-5, 8.139717345049093e-39" (exponent or scientific numbers). I need to store this value in a SQL real data type field (not float). But the maximum exponent supported by real is e-38.
But I need a mechanism to convert this string field to a real number through SSIS. Basically the e-39 or smaller values should be replaced as 0. and the rest should be stored properly.
I tried setting the data type to DT_R4 in flat file connection field mapping and that didn't help. I tried casting it to DT_R4 through a derived column and that didn't help too. When I check through Data Viewer still the value has the unsupported exponent value and it fails when I insert it to the SQL table.
Im working on SSIS 2008.
I have an issue when i tried to import a flat file that is on binary. I had no problems with the normal records but when i tried to read a float number, it didn't recognized. I tried differents ways and also saw for a Unpack Decimal component, nothing works. In the last attempt i get the number in hexadecimal notation but dont know how to convert it into a float number.
I get something like this when i see the record on SQL:
Column 1 Column 2 Column 3
Cx0000000000500013 Cx0000000000000310 Cx0000000000000056
Thanks for help.
I currently have a table that is importing data from provided CSV's on a regular basis. The issue is that I have 6 columns with data type Decimal(5,2) and are nullable, when I import a file that doesn't report any numbers back it appears as ",,,,connectionfailed,,,," within these commas are usually the numbers i'm expecting to import into the table.
When SSIS attempts to import these "blank" csv's I get the following error
Error: 2014-08-04 23:45:01.31 Code: 0xC020901C Source: Data Flow Task OLE DB Destination [9] Description: There was an error with input column "LaunchBBTime" (85) on input "OLE DB Destination Input" (22). The column status returned was: "The value could not be converted because of a potential loss of data.". End Error
Now when I change all the colums to varchar for testing purposes it imports the blank spaces without a problem, the 2nd issue with this is that SSRS cant calculate averages (in this case for performance) from varchar fields.
My question is can I properly get SSIS to import the blank columns into decimal(5,2) fields without needing to modify the datatypes?
It looks like the problem is being caused by spaces i.e. ", ," instead of ",," otherwise the datatype will set the field to its default value (in the case of decimal "0" or NULL, depending on the "retain null values from the source" property). If this is the case, probably the safest and less performance-expensive solution is passing the csv through a pre-process to remove the white spaces.
Pull the columns in as varchar, then in a Derived Column Transformation, create new decimal columns that are populated with something like the following...
LEN(LTRIM([InputValue]))==0 ? NULL(DT_NUMERIC,5,2) : (DT_NUMERIC,5,2)[InputValue]
I am importing data from a text file and have hit a snag. I have a numeric field which occasionally has very large values (10 billion+) and some of these values which are being converted to NULLs.
Upon further testing I have isolated the problem as follows - the first 25 rows of data are used to determine the field size, and if none of the first 25 values are large then it throws out any value >= 2,147,483,648 (2^31) which comes after.
I'm using ADO and the following connection string:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=FILE_ADDRESS;Extended Properties=""text;HDR=YES;FMT=Delimited""
Therefore, can anyone suggest how I can get round this problem without having to get the source data sorted descending on the large value column? Is there some way I could define the data types of the recordset prior to importing rather than let it decide for itself?
Many thanks!
You can use an INI file placed in the directory you are connecting to which describes the column types.
See here for details:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms709353(v=vs.85).aspx
I am using a bulk insert to import data from a CSV file. One of the fields is a number, and I import it as a DECIMAL(4,3). However, this data file has a few values where the number is "3.2342e-05". This obviously throws an error since this is a char. How can I convert this number to zero? For my purposes, I plan to consider any number that small as a zero anyway.
I figure that I will be importing the data into a temp table (staging table) first, so that I can clean it up in there, and then I will be inserting it from there into my final table.
SQL Server 2008
EDIT: One thing I am considering is importing the data as a char and then converting the column type, and then using a CASE statement to set anything greater than 1 to a zero. This field should never be greater than 1, which is why I am able to do this.
This is recognised as float, so you can double CAST via float