I have a .csv file with a currency sign field separator (¤), when I execute this query to bulk load it to a table it raise an error.
The file is UTF-8 encoded.
BULK INSERT dbo.test
FROM 'file.csv'
WITH (DATA_SOURCE = 'MyAzureBlobStorage',
FIRSTROW = 2,
CODEPAGE = 65001, --UTF-8 encoding
FIELDTERMINATOR = '¤', --CSV field delimiter
ROWTERMINATOR = '\n' --Use to shift the control to next row
);
The error I get is:
The bulk load failed. The column is too long in the data file for row 1, column 1. Verify that the field terminator and row terminator are specified correctly.
This is working fine with a semicolon as the separator.
I'm trying to do bulk insert from csv file with the following script:
BULK INSERT x.tableName
FROM 'PATH OF FILE'
WITH (
FIRSTROW = 2,
FIELDTERMINATOR = ',', --CSV field delimiter
ROWTERMINATOR = '\n', --Use to shift the control to next row
ERRORFILE = 'C:\CSVDATA\SchoolsErrorRows.csv',
TABLOCK )
My issue is that I have a column in the database of type Datetime2 and the format of the column in the csv file like this : 2020-12-01 20:15:00 +0300
I got this error:
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 2, column 1 (Date).
How can I solve this issue?
I have a simple data import script I'm running in MSSQL. If I statically assign the data filename to be imported, it works great.
BULK
INSERT TRACKED_IMPORT
FROM 'C:\SQLIMPORT\DATAFILE.CSV'
WITH (FIRSTROW = 2, FIELDTERMINATOR = ';', ROWTERMINATOR = '0x0a')
However, I'd like to use a variable instead of a static filename. Something like this:
DECLARE #IMPORTFILENAME VARCHAR(30) = 'C:\SQLIMPORT\DATAFILE.CSV';
BULK
INSERT TRACKED_IMPORT
FROM #IMPORTFILENAME
WITH (FIRSTROW = 2, FIELDTERMINATOR = ';', ROWTERMINATOR = '0x0a')
When I do that, I get the following errors:
Msg 102, Level 15, State 1, Line 23
Incorrect syntax near '#IMPORTFILENAME'.
Msg 319, Level 15, State 1, Line 24
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
Is there a way to allow for a variable filename?
I just want to ask if there is a way to use the bulk insert feature but to check if the last line is empty and skip it.
I have a text file that is being populated with data but the last line will always be empty cause when it repopulates, it will start from there and the the end of the previous line that is already populated.
My query so far looks like this:
BULK INSERT #TEMP
FROM 'C:\Test\Test.txt'
WITH (FIELDTERMINATOR ='\t',
ROWTERMINATOR = '\r',
FIRSTROW = 2, KEEPNULLS)
It will then be input into a temp table but the query will not go this far because of the last line in the text file. Is there a setting to say, skip the last line if its empty?
If you're expecting one line to be nonimportable then include
MAXERRORS =1
In the command and BULK INSERT should import the other ones fine.
BULK INSERT #TEMP
FROM 'C:\Test\Test.txt'
WITH (FIELDTERMINATOR ='\t',
ROWTERMINATOR = '\r',
MAXERRORS =1,
FIRSTROW = 2, KEEPNULLS)
Unless there's an unexpected problem in another line (in which case you'll want to trap the error anyway)
https://msdn.microsoft.com/en-us/library/ms188365.aspx
Well i found my answer here:Difference between CR LF, LF and CR line break types?
What i did is changed the from ROWTERMINATOR = '\r' to ROWTERMINATOR = '0x0a'
You can read up on the link about the differences in end of line chars which soved my issue. I hope this works for others as well! Best of luck guys!
I'm getting the conversion error when I try to import a text file to my database. Below is the error message I received:
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 4 (Year).
Here is my query code:
CREATE TABLE Students
(
StudentNo Integer NOT NULL Primary Key,
FirstName VARCHAR(40) NOT NULL,
LastName VARCHAR(40) NOT NULL,
Year Integer,
GPA Float NULL
);
Here is the sample data from text file:
100,Christoph,Van Gerwen,2011
101,Anar,Cooke,2011
102,Douglis,Rudinow,2008
I think I know what the problem is..Below is my bulk insert code:
use xta9354
bulk insert xta9354.dbo.Students
from 'd:\userdata\xta9_Students.txt'
with (fieldterminator = ',',rowterminator = '\n')
With the sample data, there is no ',' after the Year attribute even tho there is still another attribute Grade after the Year which is NULL
Can someone please tell me how to fix this?
Try using a format file since your data file only has 4 columns. Otherwise, try OPENROWSET or use a staging table.
myTestFormatFiles.Fmt may look like:
9.0
4
1 SQLINT 0 3 "," 1 StudentNo ""
2 SQLCHAR 0 100 "," 2 FirstName SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 100 "," 3 LastName SQL_Latin1_General_CP1_CI_AS
4 SQLINT 0 4 "\r\n" 4 Year "
(source: microsoft.com)
This tutorial on skipping a column with BULK INSERT may also help.
Your statement then would look like:
USE xta9354
GO
BULK INSERT xta9354.dbo.Students
FROM 'd:\userdata\xta9_Students.txt'
WITH (FORMATFILE = 'C:\myTestFormatFiles.Fmt')
In my case, I was dealing with a file that was generated by hadoop on a linux box. When I tried to import to sql I had this issue. The fix wound up being to use the hex value for 'line feed' 0x0a. It also worked for bulk insert
bulk insert table from 'file'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '0x0a')
We use the bulk insert as well. The file we upload is sent from an external party. After a while of troubleshooting, I realized that their file had columns with commas in it. Just another thing to look for...
The above options works for Google big query file also. I exported a table data to goodle cloud storage and downloaded from there. While loading the same to sql server was facing this issue and could successfully load the file after specifying the row delimiter as
ROWTERMINATOR = '0x0a'
Pay attention to header record as well and specify
FIRSTROW = 2
My final block for data file export from google bigquery looks like this.
BULK INSERT TABLENAME
FROM 'C:\ETL\Data\BigQuery\In\FILENAME.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ',', --CSV field delimiter
ROWTERMINATOR = '0x0a',--Files are generated with this row terminator in Google Bigquery
TABLOCK
)
My guess is it's an encoding problem, for instance your file is UTF-8 but SQL will not read it the way it should, so it attempts to insert 100ÿ or something along these lines into your table.
Possible fixes:
Specify Codepage
Change the Encoding of the source using Powershell
Code samples:
1.
BULK INSERT myTable FROM 'c:\Temp\myfile.csv' WITH (
FIELDTERMINATOR = '£',
ROWTERMINATOR = '\n',
CODEPAGE = 'ACP' -- ACP corresponds to ANSI, also try UTF-8 or 65001 for Unicode
);
2.
get-content "myfile.csv" | Set-content -Path "myfile.csv" -Encoding String
# String = ANSI, also try Ascii, Oem, Unicode, UTF7, UTF8, UTF32
Added MSSQLSERVER full access to the folder, diskadmin and bulkadmin server roles.
In my c# application, when preparing for the bulk insert command,
string strsql = "BULK INSERT PWCR_Contractor_vw_TEST FROM '" + strFileName + "' WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\\n')";
And I get this error - Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 8 (STATUS).
I looked at my logfile and found that the terminator becomes ' ' instead of '\n'.
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error:
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)". Query :BULK INSERT PWCR_Contractor_vw_TEST FROM 'G:\NEWSTAGEWWW\CalAtlasToPWCR\Results\parsedRegistration.csv' WITH (FIELDTERMINATOR = ',', **ROWTERMINATOR = ''**)
So I added extra escape to the rowterminator - string strsql = "BULK INSERT PWCR_Contractor_vw_TEST FROM '" + strFileName + "' WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\\n')";
And now it inserts successfully.
Bulk Insert SQL - ---> BULK INSERT PWCR_Contractor_vw_TEST FROM 'G:\\NEWSTAGEWWW\\CalAtlasToPWCR\\Results\\parsedRegistration.csv' WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\n')
Bulk Insert to PWCR_Contractor_vw_TEST successful... ---> clsDatase.PerformBulkInsert