I am using SSMS and trying to create a stored procedure (because it needs to survive batches) ,so i can bulk insert from multiple csv files (one at a time) into specific tables.
So far I have:
CREATE PROCEDURE AddDataToTable #TableName VARCHAR(25), #DataFolderPath VARCHAR(250),
#DataFile VARCHAR(50), #FieldDeterminator VARCHAR(10)
AS
BEGIN
DECLARE #SQL_BULK VARCHAR(MAX)
SET #SQL_BULK =
'BULK INSERT '+#TableName+'
FROM '''+#DataFolderPath+#DataFile+'''
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = '''+#FieldDeterminator+''',
ROWTERMINATOR = ''0x0A'',
TABLOCK
)'
PRINT #SQL_BULK
EXEC #SQL_BULK
END
GO
And using with
EXEC AddDataToTable 'dbo.People',#DataFolderPath,'\ImdbName.csv',';'
And I get the error:
Msg 911, Level 16, State 4, Procedure AddDataToTable, Line 18 (Batch Start Line 161) Database 'BULK INSERT dbo' does not exist. Make sure that the name is entered correctly.
The thing is that I also added a print statement in the procedure and the print result looks like it should with every quote which is:
BULK INSERT dbo.People
FROM 'C:\Users\PC\Desktop\Data\ImdbName.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ';',
ROWTERMINATOR = '0x0A',
TABLOCK
)
You are using a wrong syntax. Execute the dynamically generated statement like this:
EXEC (#SQL_BULK)
or
DECLARE #err int
EXEC #err = sp_executesql #SQL_BULK
IF #err <> 0 PRINT 'Error found.'
As an additional note, always use QUOTENAME() to prevent possible SQL injection issues, when you generate an SQL Server identifier from an input string:
SET #SQL_BULK = 'BULK INSERT ' + QUOTENAME(#TableName) + ' FROM ... '
Related
The following code gives an error (its part of a T-SQL stored procedure):
-- Bulk insert data from the .csv file into the staging table.
DECLARE #CSVfile nvarchar(255);
SET #CSVfile = N'T:\x.csv';
BULK INSERT [dbo].[TStagingTable]
-- FROM N'T:\x.csv' -- This line works
FROM #CSVfile -- This line will not work
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
FIRSTROW = 2
)
The error is:
Incorrect syntax near the keyword 'with'.
If I replace:
FROM #CSVfile
with:
FROM 'T:\x.csv'
... then it works nicely.
As I know only literal string is required in the from. In that case you have to write a dynamic query to use bulk insert
declare #q nvarchar(MAX);
set #q=
'BULK INSERT [TStagingTable]
FROM '+char(39)+#CSVfile+char(39)+'
WITH
(
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 1
)'
exec(#q)
Have you tried with dynamic SQL?
SET #SQL = "BULK INSERT TmpStList FROM '"+#PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
and then
EXEC(#SQL)
Ref.: http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
you have to engage in string building & then calling EXEC() or sp_executesql BOL
has an example:
DECLARE #bulk_cmd varchar(1000)
SET #bulk_cmd = 'BULK INSERT AdventureWorks2008R2.Sales.SalesOrderDetail
FROM ''<drive>:\<path>\<filename>''
WITH (ROWTERMINATOR = '''+CHAR(10)+''')'
EXEC(#bulk_cmd)
A string literal is required.
http://msdn.microsoft.com/en-us/library/ms188365.aspx
You could use dynamic sql to generate the string literal.
Most of the time the variable i'm looking for in a file name is the date, and this one works perfectly for bulk inserting files with date, for use such as in a daily job. Change as per your need, date format, table name, file path, file name and delimiters.
DECLARE #DT VARCHAR (10)
DECLARE #INSERT VARCHAR (1000)
SET #DT = (CONVERT(VARCHAR(10),GETDATE()-1,120))
SET #INSERT = 'BULK INSERT dbo.table FROM ''C:\FOLDER\FILE'+#DT+'.txt'''+' WITH (FIRSTROW=2, FIELDTERMINATOR=''\t'', ROWTERMINATOR=''\n'')'
EXEC (#INSERT);
Can you try FROM ' + #CSVfile + '
I'm using a bulk insert script to import a number of flat files into SQL Server
A few files end with
-----------------------------------
So what I want to do is or skip last row(s) or remove ------------------ in the bulk insert. Is one of these options possible?
SET #s = N'BULK INSERT ' + #t + '
FROM ''' + #f + '''
WITH (FIELDTERMINATOR = ''|'',
ROWTERMINATOR = ''0x0a'',
FIRSTROW=2) '
lastrow = 1 doesn't work
The only way I can think of is to first bulk insert the whole file into a single column table (as varchar(max)). Than you can identify the last row, and use that value in your actual bulk insert.
This is not a very straight forward approach, but I don't think there is another (unless you write a custom solution in C# or java or whatever). Maybe you can use SQLCMD to first read the number of lines in the file, but I don't know how.
Please note there is a connect item which Microsoft has closed. On that page Microsoft suggests using an openrowset solution, could be worthwhile to try, but I doubt it would work in your situation.
use script like next:
SET NOCOUNT ON;
IF OBJECT_ID('tempdb..#csvData') IS NOT NULL DROP TABLE #csvData
IF OBJECT_ID('tempdb..#csvRowCount') IS NOT NULL DROP TABLE #csvRowCount
CREATE TABLE #csvRowCount
(
v1 nvarchar(max) -- get only first column - for all rows in file
)
BULK INSERT #csvRowCount
FROM 'C:\TEMP\HR_FOR_LOAD\PP_SICKLIST.CSV'
WITH
(
Firstrow=2,
FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n'
);
declare #textRowCount int=(select count(1) from #csvRowCount)
CREATE TABLE #csvData
(
v1 nvarchar(2000),
v2 nvarchar(2000),
v3 nvarchar(2000)
--etc
)
declare #sql varchar(max)
set #sql = '
BULK INSERT #csvData
FROM ''C:\TEMP\HR_FOR_LOAD\PP_SICKLIST.CSV''
WITH
(
Firstrow=2,
FIELDTERMINATOR = ''\t'',
ROWTERMINATOR = ''\n'',
Lastrow = '+cast(#textRowCount as varchar(100))+'
);'
exec (#sql)
select
v1,
v2,
v3
from #csvData
In SQL Server 2014, I have created a simple stored procedure which does a bulk insert; what I'm struggling with is capturing if the bulk insert succeed or failed and returning 0 or 1 respectively.
The stored procedure code is:
ALTER PROCEDURE [dbo].[BulkInsert]
#file_name nvarchar(100)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #bulk_insert NVARCHAR(2000);
SET #bulk_insert =
N'BULK INSERT log_door_access FROM ''' +
#file_name +
N''' WITH (FIELDTERMINATOR = ''\t'', ROWTERMINATOR = ''0x0A'');';
EXEC sp_executesql #bulk_insert;
END
I'm trying to capture the EXEC result and return a 1 if all went well and 0 if an error occurred.
This can be done using TRY...CATCH - 'Implements error handling for Transact-SQL that is similar to the exception handling in the Microsoft Visual C#...'
something like
ALTER PROCEDURE [dbo].[BulkInsert]
#file_name nvarchar(100)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #bulk_insert NVARCHAR(2000);
SET #bulk_insert =
N'BULK INSERT log_door_access FROM ''' +
#file_name +
N''' WITH (FIELDTERMINATOR = ''\t'', ROWTERMINATOR = ''0x0A'');';
declare #r int
EXEC #r = sp_executesql #bulk_insert;
select #r
END
should do, or use an output parameter for sp_executesql, or a try catch block.
I am trying to execute the following SQL in SQL Server 2008-
DECLARE #sql nvarchar, #fullname nvarchar;
SET #fullname='1patents_corrected.csv';
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent from ' + #fullname+ ' WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR = "\n" );'
EXEC(#sql)
However I am getting this error--
Msg 2812, Level 16, State 62, Line 1
Could not find stored procedure 'B'.
What am I doing wrong here?
UPDATE-- I changed the query viz. specified a size for each varchar variable. Now the code is like this--
DECLARE #MyCounter int;
DECLARE #Fileprefix nvarchar(1000), #Filesuffix nvarchar(1000), #fullname nvarchar(1000), #Counter_string nvarchar(1000), #sql nvarchar(1000);
SET #MyCounter = 1;
SET #Fileprefix= 'C:\Arvind_gpd\patents\';
SET #Filesuffix='data_corrected.csv';
WHILE (#MyCounter < 10)
BEGIN;
Set #Counter_string= Cast(#MyCounter AS varchar(1) );
Set #fullname = (#Fileprefix+ #Counter_string + #Filesuffix );
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent from ' + #fullname+
' WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR = "\n" );'
EXEC(#sql);
SET #MyCounter = #MyCounter + 1;
END;
GO
However I am now getting a different error--
Msg 102, Level 15, State 1, Line 1
Incorrect syntax near 'C:'.
Msg 319, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
Now what am I doing incorrectly :( ?
change
DECLARE #sql nvarchar
to
DECLARE #sql nvarchar(2000)
by default the size is 1
The same applies to #fullname nvarchar; give it a size
You need to quote the file name when using BULK INSERT.
Change
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent
from ' + #fullname+
WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR ="\n" );'
to
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent
FROM ''' + #fullname + '''
WITH ( DATAFILETYPE = ''char'', FIELDTERMINATOR = ''^'', ROWTERMINATOR =''\n'' )'
You'll notice that I have instances of multiple single quotes bunched together. Double single quotes stand for escape in SQL Server (ie. I am quoting a quote within another quote).
Also, I see you use a lot of semicolons in your code. No need to put them in when you are writing in T-SQL
The following code gives an error (its part of a T-SQL stored procedure):
-- Bulk insert data from the .csv file into the staging table.
DECLARE #CSVfile nvarchar(255);
SET #CSVfile = N'T:\x.csv';
BULK INSERT [dbo].[TStagingTable]
-- FROM N'T:\x.csv' -- This line works
FROM #CSVfile -- This line will not work
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
FIRSTROW = 2
)
The error is:
Incorrect syntax near the keyword 'with'.
If I replace:
FROM #CSVfile
with:
FROM 'T:\x.csv'
... then it works nicely.
As I know only literal string is required in the from. In that case you have to write a dynamic query to use bulk insert
declare #q nvarchar(MAX);
set #q=
'BULK INSERT [TStagingTable]
FROM '+char(39)+#CSVfile+char(39)+'
WITH
(
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 1
)'
exec(#q)
Have you tried with dynamic SQL?
SET #SQL = "BULK INSERT TmpStList FROM '"+#PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
and then
EXEC(#SQL)
Ref.: http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
you have to engage in string building & then calling EXEC() or sp_executesql BOL
has an example:
DECLARE #bulk_cmd varchar(1000)
SET #bulk_cmd = 'BULK INSERT AdventureWorks2008R2.Sales.SalesOrderDetail
FROM ''<drive>:\<path>\<filename>''
WITH (ROWTERMINATOR = '''+CHAR(10)+''')'
EXEC(#bulk_cmd)
A string literal is required.
http://msdn.microsoft.com/en-us/library/ms188365.aspx
You could use dynamic sql to generate the string literal.
Most of the time the variable i'm looking for in a file name is the date, and this one works perfectly for bulk inserting files with date, for use such as in a daily job. Change as per your need, date format, table name, file path, file name and delimiters.
DECLARE #DT VARCHAR (10)
DECLARE #INSERT VARCHAR (1000)
SET #DT = (CONVERT(VARCHAR(10),GETDATE()-1,120))
SET #INSERT = 'BULK INSERT dbo.table FROM ''C:\FOLDER\FILE'+#DT+'.txt'''+' WITH (FIRSTROW=2, FIELDTERMINATOR=''\t'', ROWTERMINATOR=''\n'')'
EXEC (#INSERT);
Can you try FROM ' + #CSVfile + '