SQL Insert Out of Sync - sql

Question

I have a bit of SQL here which is throwing an error:
DROP TABLE HACP_TEMP_PIC_HCV_Imported;
CREATE TABLE HACP_TEMP_PIC_HCV_Imported
(
HeadSSN varchar(255) NOT NULL,
HeadFName varchar(255) NOT NULL,
HeadMName varchar(255),
HeadLName varchar(255) NOT NULL,
ModifiedDate varchar(255) NOT NULL,
ActionType varchar(255) NOT NULL,
EffectiveDate varchar(255) NOT NULL
);
BULK INSERT HACP_TEMP_PIC_HCV_Imported
FROM 'C:\Work\MTWAdhocReport.csv'
WITH
(
FIRSTROW = 11,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
ERRORFILE = 'C:\Work\Import_ErrorRows_HCV.csv',
TABLOCK
);
UPDATE HACP_TEMP_PIC_HCV_Imported
SET HeadSSN = REPLACE(HeadSSN, '"', ''),
HeadFName = REPLACE(HeadFName, '"', ''),
HeadMName = REPLACE(HeadMName, '"', ''),
HeadLName = REPLACE(HeadLName, '"', ''),
ModifiedDate = REPLACE(ModifiedDate, '"', ''),
ActionType = REPLACE(ActionType, '"', ''),
EffectiveDate = REPLACE(REPLACE(EffectiveDate, '"', ''),',','');
DROP TABLE HACP_PIC_HCV_Imported;
CREATE TABLE HACP_PIC_HCV_Imported
(
HeadSSN varchar(255) NOT NULL,
HeadFName varchar(255) NOT NULL,
HeadMName varchar(255),
HeadLName varchar(255) NOT NULL,
ModifiedDate varchar(255) NOT NULL,
ActionType int NOT NULL,
EffectiveDate varchar(255) NOT NULL
);
INSERT INTO HACP_PIC_HCV_Imported(HeadSSN, HeadFName, HeadMName, HeadLName, ModifiedDate, ActionType, EffectiveDate)
SELECT
LTRIM(HeadSSN),
LTRIM(HeadFName),
LTRIM(HeadMName),
LTRIM(HeadLName),
LTRIM(ModifiedDate),
CONVERT(int, LTRIM(ActionType)),
LTRIM(EffectiveDate)
FROM
HACP_TEMP_PIC_HCV_Imported;
Stepping through this, creating the temp table and importing the CSV into it works fine. Updating the table to remove quotes and a trailing comma from the EffectiveDate column works. Creating the new table-proper works.
When trying to copy the data into the second table (and converting ActionType into an INT), I get this error message:
Conversion failed when converting the varchar value '4/07/2016' to data type int.
That data is the second row value in ModifiedDate, so the columns are apparently getting out of sync after importing the first row. I have double-checked that all of the data is in the proper columns after being imported into the temp table initially.
Any thoughts? I feel like I'm missing something obvious.

Answer 1

Your code suggests that you are using "proper" CSV format, which allows fields to be enclosed in double quotes. These delimited fields can contain commas. This is the format produced and read by Excel.
My guess is that you have a comma in such a delimited field and this is throwing off the import.
But, this format is not read properly by bulk insert. Ironically, (at least) one database does import the CSV formatted files with commas in the fields.
In the past when I've had this problem, it has only been on smallish files. I simply loaded the data into Excel and then saved in out using tabs or vertical bars as delimiters. This solved the problem in my case.
I'm not sure if there is a more advanced solution now. But I'm pretty sure your problem is that some fields have embedded commas in the text fields.