I am using the SQL Server Import and Export Wizard to try to import a particular xlsx spreadsheet into an existing table in SQL. The existing table contains a sub-set of the columns in the spreadsheet and I am ignoring the many columns that don't match.
The spreadsheet has 123 columns and 238 lines of data.
Initially when I was importing the spreadsheet the wizard was hanging on 'Executing' and I had to kill the process. Something I have never come across before.
After copy and pasting the data into a new spreadsheet it is now coming up with the following error:
Error 0xc020901c: Data Flow Task 1: There was an error with output column "Confidentiality Clause Comments" (108) on output "Excel Source Output" (9). The column status returned was: "Text was truncated or one or more characters had no match in the target code page.".
(SQL Server Import and Export Wizard)
Error 0xc020902a: Data Flow Task 1: The "output column "Confidentiality Clause Comments" (108)" failed because truncation occurred, and the truncation row disposition on "output column "Confidentiality Clause Comments" (108)" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
(SQL Server Import and Export Wizard)
What I am confused about it, the column "Confidentiality Clause Comments" is one of the columns being ignored - it is not being imported into the database!
I have tried setting "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Jet 4.0\Engines\Excel\TypeGuessRows" to both 0 and higher numbers like 238 and 1000 to increase the sample size. (Although the table does already exist with fields large enough for the data being imported). I also have the "On Truncation (global)" and "On Error (global)" set to Ignore (but this setting seems to be 'ignored').
I have also tried importing the data into a new table, and get the same truncation error message (but on different fields dependant on the data sort).
I thought about importing as a CSV file but there are embedded comma's in many of the fields and it completely messed the data up.
Any ideas on how to get data imported? I have spent over 3 hours on this already, and have got nowhere.
Thanks,Steve
Instead of CSV, you could save as Text (Tab delimited) (*.txt) - that way you won't encounter the comma problem.
Also, if you only need to do this once, and because the dataset is not exactly huge, I'd consider just copying from excel and pasting directly into sql server management studio.
Related
I am facing issues when importing large data values into a pre-defined data table in SQL. Any value larger or equal to 100,000,000 (one hundred million) creates an error and fails upon import. A value of 99,999,999 is fine. Does anyone know what is going on and how to solve this, please?
Notes:
The data table I import data to is pre-defined; I can only append rows. The target column is defined as float(Null). The import wizard seems to set the data type to float automatically. I could not change it to numeric or decimal because a) I do not have the admin rights, and b) there is something funny with the source data when trying to import. But I cannot see anything to be off when looking at it in Excel; all values are numeric and the max length is only 11.
The files I use are in .csv format and many have more than 1 million rows. I tried splitting them up using Python and saving them in .xlsx format to put them through an Excel formatting step, but that did not help.
I am using the import wizard, SMSS v.18.12.1, and the SQL Server Native Client 11.0 of the import wizard.
The error message I get is:
`
Error 0xc02020c5: Data Flow Task 1: Data conversion failed while converting column "ColumnName" (42) to column "ColumnName" (148). The conversion returned status value 6 and status text "Conversion failed because the data value overflowed the specified type.".
(SQL Server Import and Export Wizard)
and
Error 0xc0209029: Data Flow Task 1: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Data Conversion 0 - 0.Outputs[Data Conversion Output].Columns[ColumnName]" failed because error code 0xC0209082 occurred, and the error row disposition on "Data Conversion 0 - 0.Outputs[Data Conversion Output].Columns[ColumnName]" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
`
Initial situation:
I'm stuck in a simple task (in my opinion it should be simple...)
I have a collection of data which should be exported weekly to Excel.
This export contains 104 columns, from which 57 are nvarchar(max) and contains item descriptions and other information in different languages for our sales guys.
The report will have something around 2000 to 8000 rows.
I use a SQL Server 2017 CU 16
My intention:
I intended to do an SSI Job with an Excel template where the columns are predefined (width, data type and so on).
This job would have something like those steps:
Delete existing Excel file
copy Excel template as a new Excel file
DataFlowTask using SQL Server as the source and Excel destination as the target
What I already tried:
If I use the excel template with only headers, I get the following error for each of the nvarchar(max) columns:
[Excel Destination [2]] Error: An error occurred while setting up a
binding for the "ColumnName" column. The binding status was
"DT_NTEXT".
When I prepare the template having it prefilled with one row. This row has a long text (more than 255 characters) for the columns where nvarchar(max) is in the source, everything runs fine but, this dummy line is still existing.
Another try I did was dropping the sheet using an "Execute SQL Task" to the Excel File Connection and recreating the sheet using a create table statement in another "Execute SQL Task" to the same Excel file connection, I get the same error as above. Although I'm using NTEXT as the datatype for the relevant columns.
Question:
How can I export data seamlessly into a preformatted excel file which contains NTEXT?
Thank you very much in advance for any assistance.
I am trying to import a CSV into MSSQL 2008 by using the flat file import method but I am getting an Overflow error. Any ideas on how to go around it?
I used the tool before for files containing up to 10K-15K records but this file has 75K records in it....
These are the error messages
Messages
Error 0xc020209c: Data Flow Task 1: The column data for column "ClientBrandID" overflowed the disk I/O buffer.
(SQL Server Import and Export Wizard)
Error 0xc0202091: Data Flow Task 1: An error occurred while skipping data rows.
(SQL Server Import and Export Wizard)
Error 0xc0047038: Data Flow Task 1: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "Source - Shows_csv" (1) returned error code 0xC0202091. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
This could be a format problem of the csv file e.g. the delimiter. Check if the delimiters are consistent within the file.
It could also be a problem of blank lines. I had a similar problem a while ago. I've solved it by removing all blank lines in the csv file. Worth a try anyway.
You may have one or more bad data elements. Try loading a small subset of your data to determine if it's a small number of bad records or a large one. This will also tell you if your loading scheme is working and your datatypes match.
Sometimes you can quickly spot data issues if you open the csv file in excel.
Another possible reason for this error is that input file has wrong encoding. So, when you manually check data, it seems fine. For example, in my case correct files were in 8-bit ANSI, and wrong files in UTF-16 - you can tell the difference by looking at files size, wrong files were twice bigger than correct files.
I'm trying to insert a large CSV file (several gigs) into SQL Server, but once I go through the Import Wizard and finally try to import the file I get the following error report:
Executing (Error)
Messages
Error 0xc02020a1: Data Flow Task 1: Data conversion failed. The data
conversion for column ""Title"" returned status value 4 and status
text "Text was truncated or one or more characters had no match in the
target code page.".
(SQL Server Import and Export Wizard)
Error 0xc020902a: Data Flow Task 1: The "Source -
Train_csv.Outputs[Flat File Source Output].Columns["Title"]" failed
because truncation occurred, and the truncation row disposition on
"Source - Train_csv.Outputs[Flat File Source Output].Columns["Title"]"
specifies failure on truncation. A truncation error occurred on the
specified object of the specified component.
(SQL Server Import and Export Wizard)
Error 0xc0202092: Data Flow Task 1: An error occurred while processing
file "C:\Train.csv" on data row 2.
(SQL Server Import and Export Wizard)
Error 0xc0047038: Data Flow Task 1: SSIS Error Code
DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Source - Train_csv
returned error code 0xC0202092. The component returned a failure code
when the pipeline engine called PrimeOutput(). The meaning of the
failure code is defined by the component, but the error is fatal and
the pipeline stopped executing. There may be error messages posted
before this with more information about the failure.
(SQL Server Import and Export Wizard)
I created the table to insert the file into first, and I set each column to hold varchar(MAX), so I don't understand how I can still have this truncation issue. What am I doing wrong?
In SQL Server Import and Export Wizard you can adjust the source data types in the Advanced tab (these become the data types of the output if creating a new table, but otherwise are just used for handling the source data).
The data types are annoyingly different than those in MS SQL, instead of VARCHAR(255) it's DT_STR and the output column width can be set to 255. For VARCHAR(MAX) it's DT_TEXT.
So, on the Data Source selection, in the Advanced tab, change the data type of any offending columns from DT_STR to DT_TEXT (You can select multiple columns and change them all at once).
This answer may not apply universally, but it fixed the occurrence of this error I was encountering when importing a small text file. The flat file provider was importing based on fixed 50-character text columns in the source, which was incorrect. No amount of remapping the destination columns affected the issue.
To solve the issue, in the "Choose a Data Source" for the flat-file provider, after selecting the file, a "Suggest Types.." button appears beneath the input column list. After hitting this button, even if no changes were made to the enusing dialog, the Flat File provider then re-queried the source .csv file and then correctly determined the lengths of the fields in the source file.
Once this was done, the import proceeded with no further issues.
I think its a bug, please apply the workaround and then try again: http://support.microsoft.com/kb/281517.
Also, go into Advanced tab, and confirm if Target columns length is Varchar(max).
The Advanced Editor did not resolve my issue, instead I was forced to edit dtsx-file through notepad (or your favorite text/xml editor) and manually replace values in attributes to
length="0" dataType="nText" (I'm using unicode)
Always make a backup of the dtsx-file before you edit in text/xml mode.
Running SQL Server 2008 R2
Goto Advanced tab----> data type of column---> Here change data type from DT_STR to DT_TEXT and column width 255. Now you can check it will work perfectly.
Issue:
The Jet OLE DB provider reads a registry key to determine how many rows are to be read to guess the type of the source column.
By default, the value for this key is 8. Hence, the provider scans the first 8 rows of the source data to determine the data types for the columns. If any field looks like text and the length of data is more than 255 characters, the column is typed as a memo field. So, if there is no data with a length greater than 255 characters in the first 8 rows of the source, Jet cannot accurately determine the nature of the data type.
As the first 8 row length of data in the exported sheet is less than 255 its considering the source length as VARCHAR(255) and unable to read data from the column having more length.
Fix:
The solution is just to sort the comment column in descending order.
In 2012 onwards we can update the values in Advance tab in the Import wizard.
Data conversion failed. The data conversion for column "TIME PERIOD" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
Error: 0xC0209029 at Data Flow Task, Flat File Source [565]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "output column "TIME PERIOD" (590)" failed because error code 0xC0209084 occurred, and the error row disposition on "output column "TIME PERIOD" (590)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
I suggest importing flat files to staging tables (all text fields, no casting) then migrating them to your final table. Importing them without casting will sidestep errors like this, as long as your text fields are sufficiently long as to avoid truncation.
When you migrate the data in SSIS from staging to final table you can direct rows that error to an appropriate error output that you can use to isolate the problematic rows and decide how to handle them. Then you can fix and migrate those rows separately.
To my knowledge there's not an easy way to handle capturing problematic rows when casting on flat-file import in SSIS
You could change your SSIS to not fail the package on error, but then you'd have to dig for the problematic rows in the csv.