MERGE Issue In Bulk Insert Sproc - sql

Question

I am having an issue with my "Merging":
The MERGE statement attempted to UPDATE or DELETE the same row more than once.
Can someone help me fix this, as I do not understand how to correct the issue:
ALTER PROCEDURE [Files].[ImportFiles]
AS
-- Create a temporary table for the bulk import
CREATE TABLE #TempImportFileTable(
[fileID] [bigint] IDENTITY(1,1) NOT NULL,
[FileName] [nvarchar](max) NULL,
[FilePath] [nvarchar](max) NULL,
[FullPath] [nvarchar](max) NULL,
[FileSize] [nvarchar](max) NULL,
[FileExtension] [nvarchar](max) NULL,
[FileCreated] [nvarchar](max) NULL,
[FileLastAccessed] [nvarchar](max) NULL,
[FileModified] [nvarchar](max) NULL
CONSTRAINT [PK_fileID1] PRIMARY KEY CLUSTERED
(
[fileID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY];
-- Import into the temp table
BULK INSERT #TempImportFileTable FROM 'C:\Program Files\o7th FileSystem to DB\import.txt'
WITH(KEEPIDENTITY, FIELDTERMINATOR =',', ROWTERMINATOR = '\n');
-- Delete the Duplicate entries
DELETE FROM #TempImportFileTable WHERE fileID NOT IN (SELECT MAX(fileID) FROM #TempImportFileTable GROUP BY FullPath);
-- Now Merge the 2 tables
MERGE [Files].[File] AS TargetTable
USING #TempImportFileTable AS SourceTable
ON (TargetTable.FullPath = SourceTable.FullPath)
WHEN NOT MATCHED BY TARGET
THEN INSERT (FileName, FilePath, FileSize, FileExtension, FileCreated, FileLastAccessed, FileModified)
VALUES(SourceTable.FileName, SourceTable.FilePath, SourceTable.FileSize, SourceTable.FileExtension, SourceTable.FileCreated, SourceTable.FileLastAccessed, SourceTable.FileModified)
WHEN MATCHED
THEN UPDATE SET
TargetTable.FileName = SourceTable.FileName,
TargetTable.FilePath = SourceTable.FilePath,
TargetTable.FileSize = SourceTable.FileSize,
TargetTable.FileExtension = SourceTable.FileExtension,
TargetTable.FileCreated = SourceTable.FileCreated,
TargetTable.FileLastAccessed = SourceTable.FileLastAccessed,
TargetTable.FileModified = SourceTable.FileModified;

Answer 1

So you have duplicates in your #TempFileTable.
IF FileName + FilePath were enough to make a row unique, you could use this condition for your MERGE:
MERGE [Files].[File] AS TargetTable
USING #TempFileTable AS SourceTable
ON (
ISNULL(TargetTable.FileName,'') = ISNULL(SourceTable.FileName,'')
AND ISNULL(TargetTable.FileName,'') = ISNULL(SourceTable.FileName,'')
)
(I don't know why the ISNULL() function didn't work when you tried to call it earlier, but it should definitely work this way, and handle the issues that arise with null values.)
If you really have duplicate rows in your original file and you want to get rid of them, you may use this kind of code:
DELETE FROM #TempFileTable WHERE fileID IN (
SELECT u.fileID FROM(
select fileID, ROW_NUMBER() OVER(PARTITION BY FileName, FilePath OVER fileID) as r_number
FROM #TempFileTable
) where u.r_number>1
)
It is ugly, but when several rows have the same FileName and same FilePath, it will remove the rows with the highest fileID and keep only one.
EDIT: About the performance issue, first try to look at the estimated execution plan and see if you can add indexes to your tables. You can also try to break the MERGE procedure into on INSERT and one UPDATE statement. MERGE should be better, but in some situations it is actually worse than separate statements.

Answer 2

Merge statement has a limitation that it cannot update the same row more than once, or update and delete the same row. So please make sure that you have distinct rows with filename in you source data.
Also, while doing the update you don't have to update the filename column as its going to be same in both source and the target tables.
You may get a work around in this link: http://support.microsoft.com/kb/976316