The following code gives an error (its part of a T-SQL stored procedure):
-- Bulk insert data from the .csv file into the staging table.
DECLARE #CSVfile nvarchar(255);
SET #CSVfile = N'T:\x.csv';
BULK INSERT [dbo].[TStagingTable]
-- FROM N'T:\x.csv' -- This line works
FROM #CSVfile -- This line will not work
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
FIRSTROW = 2
)
The error is:
Incorrect syntax near the keyword 'with'.
If I replace:
FROM #CSVfile
with:
FROM 'T:\x.csv'
... then it works nicely.
As I know only literal string is required in the from. In that case you have to write a dynamic query to use bulk insert
declare #q nvarchar(MAX);
set #q=
'BULK INSERT [TStagingTable]
FROM '+char(39)+#CSVfile+char(39)+'
WITH
(
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 1
)'
exec(#q)
Have you tried with dynamic SQL?
SET #SQL = "BULK INSERT TmpStList FROM '"+#PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
and then
EXEC(#SQL)
Ref.: http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
you have to engage in string building & then calling EXEC() or sp_executesql BOL
has an example:
DECLARE #bulk_cmd varchar(1000)
SET #bulk_cmd = 'BULK INSERT AdventureWorks2008R2.Sales.SalesOrderDetail
FROM ''<drive>:\<path>\<filename>''
WITH (ROWTERMINATOR = '''+CHAR(10)+''')'
EXEC(#bulk_cmd)
A string literal is required.
http://msdn.microsoft.com/en-us/library/ms188365.aspx
You could use dynamic sql to generate the string literal.
Most of the time the variable i'm looking for in a file name is the date, and this one works perfectly for bulk inserting files with date, for use such as in a daily job. Change as per your need, date format, table name, file path, file name and delimiters.
DECLARE #DT VARCHAR (10)
DECLARE #INSERT VARCHAR (1000)
SET #DT = (CONVERT(VARCHAR(10),GETDATE()-1,120))
SET #INSERT = 'BULK INSERT dbo.table FROM ''C:\FOLDER\FILE'+#DT+'.txt'''+' WITH (FIRSTROW=2, FIELDTERMINATOR=''\t'', ROWTERMINATOR=''\n'')'
EXEC (#INSERT);
Can you try FROM ' + #CSVfile + '
Related
I am using SSMS and trying to create a stored procedure (because it needs to survive batches) ,so i can bulk insert from multiple csv files (one at a time) into specific tables.
So far I have:
CREATE PROCEDURE AddDataToTable #TableName VARCHAR(25), #DataFolderPath VARCHAR(250),
#DataFile VARCHAR(50), #FieldDeterminator VARCHAR(10)
AS
BEGIN
DECLARE #SQL_BULK VARCHAR(MAX)
SET #SQL_BULK =
'BULK INSERT '+#TableName+'
FROM '''+#DataFolderPath+#DataFile+'''
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = '''+#FieldDeterminator+''',
ROWTERMINATOR = ''0x0A'',
TABLOCK
)'
PRINT #SQL_BULK
EXEC #SQL_BULK
END
GO
And using with
EXEC AddDataToTable 'dbo.People',#DataFolderPath,'\ImdbName.csv',';'
And I get the error:
Msg 911, Level 16, State 4, Procedure AddDataToTable, Line 18 (Batch Start Line 161) Database 'BULK INSERT dbo' does not exist. Make sure that the name is entered correctly.
The thing is that I also added a print statement in the procedure and the print result looks like it should with every quote which is:
BULK INSERT dbo.People
FROM 'C:\Users\PC\Desktop\Data\ImdbName.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ';',
ROWTERMINATOR = '0x0A',
TABLOCK
)
You are using a wrong syntax. Execute the dynamically generated statement like this:
EXEC (#SQL_BULK)
or
DECLARE #err int
EXEC #err = sp_executesql #SQL_BULK
IF #err <> 0 PRINT 'Error found.'
As an additional note, always use QUOTENAME() to prevent possible SQL injection issues, when you generate an SQL Server identifier from an input string:
SET #SQL_BULK = 'BULK INSERT ' + QUOTENAME(#TableName) + ' FROM ... '
The following code gives an error (its part of a T-SQL stored procedure):
-- Bulk insert data from the .csv file into the staging table.
DECLARE #CSVfile nvarchar(255);
SET #CSVfile = N'T:\x.csv';
BULK INSERT [dbo].[TStagingTable]
-- FROM N'T:\x.csv' -- This line works
FROM #CSVfile -- This line will not work
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
FIRSTROW = 2
)
The error is:
Incorrect syntax near the keyword 'with'.
If I replace:
FROM #CSVfile
with:
FROM 'T:\x.csv'
... then it works nicely.
As I know only literal string is required in the from. In that case you have to write a dynamic query to use bulk insert
declare #q nvarchar(MAX);
set #q=
'BULK INSERT [TStagingTable]
FROM '+char(39)+#CSVfile+char(39)+'
WITH
(
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
FIRSTROW = 1
)'
exec(#q)
Have you tried with dynamic SQL?
SET #SQL = "BULK INSERT TmpStList FROM '"+#PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
and then
EXEC(#SQL)
Ref.: http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
you have to engage in string building & then calling EXEC() or sp_executesql BOL
has an example:
DECLARE #bulk_cmd varchar(1000)
SET #bulk_cmd = 'BULK INSERT AdventureWorks2008R2.Sales.SalesOrderDetail
FROM ''<drive>:\<path>\<filename>''
WITH (ROWTERMINATOR = '''+CHAR(10)+''')'
EXEC(#bulk_cmd)
A string literal is required.
http://msdn.microsoft.com/en-us/library/ms188365.aspx
You could use dynamic sql to generate the string literal.
Most of the time the variable i'm looking for in a file name is the date, and this one works perfectly for bulk inserting files with date, for use such as in a daily job. Change as per your need, date format, table name, file path, file name and delimiters.
DECLARE #DT VARCHAR (10)
DECLARE #INSERT VARCHAR (1000)
SET #DT = (CONVERT(VARCHAR(10),GETDATE()-1,120))
SET #INSERT = 'BULK INSERT dbo.table FROM ''C:\FOLDER\FILE'+#DT+'.txt'''+' WITH (FIRSTROW=2, FIELDTERMINATOR=''\t'', ROWTERMINATOR=''\n'')'
EXEC (#INSERT);
Can you try FROM ' + #CSVfile + '
I'm using a bulk insert script to import a number of flat files into SQL Server
A few files end with
-----------------------------------
So what I want to do is or skip last row(s) or remove ------------------ in the bulk insert. Is one of these options possible?
SET #s = N'BULK INSERT ' + #t + '
FROM ''' + #f + '''
WITH (FIELDTERMINATOR = ''|'',
ROWTERMINATOR = ''0x0a'',
FIRSTROW=2) '
lastrow = 1 doesn't work
The only way I can think of is to first bulk insert the whole file into a single column table (as varchar(max)). Than you can identify the last row, and use that value in your actual bulk insert.
This is not a very straight forward approach, but I don't think there is another (unless you write a custom solution in C# or java or whatever). Maybe you can use SQLCMD to first read the number of lines in the file, but I don't know how.
Please note there is a connect item which Microsoft has closed. On that page Microsoft suggests using an openrowset solution, could be worthwhile to try, but I doubt it would work in your situation.
use script like next:
SET NOCOUNT ON;
IF OBJECT_ID('tempdb..#csvData') IS NOT NULL DROP TABLE #csvData
IF OBJECT_ID('tempdb..#csvRowCount') IS NOT NULL DROP TABLE #csvRowCount
CREATE TABLE #csvRowCount
(
v1 nvarchar(max) -- get only first column - for all rows in file
)
BULK INSERT #csvRowCount
FROM 'C:\TEMP\HR_FOR_LOAD\PP_SICKLIST.CSV'
WITH
(
Firstrow=2,
FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n'
);
declare #textRowCount int=(select count(1) from #csvRowCount)
CREATE TABLE #csvData
(
v1 nvarchar(2000),
v2 nvarchar(2000),
v3 nvarchar(2000)
--etc
)
declare #sql varchar(max)
set #sql = '
BULK INSERT #csvData
FROM ''C:\TEMP\HR_FOR_LOAD\PP_SICKLIST.CSV''
WITH
(
Firstrow=2,
FIELDTERMINATOR = ''\t'',
ROWTERMINATOR = ''\n'',
Lastrow = '+cast(#textRowCount as varchar(100))+'
);'
exec (#sql)
select
v1,
v2,
v3
from #csvData
I am doing a bulk insert that I have to skip the last row. Otherwise , I got an error saying "Bulk Insert: Unexpected end-of-file (EOF) encountered in data file."
If I set ROWTERMINATOR='\r\n', then I got 0 rows imported.
I wonder if there is any code that can help me skip the lastrow of the txt file? (The last row is dynamic.) My company currently doesn't have SSIS installed.
My code for bulk insert is
Declare #SQL1 varchar(150), #path varchar(100),
#pathtable varchar(100), #date datetime
set #date = getdate()
-- set path for files
set #path= 'C:\imp\'
set #pathtable = #path + 'importfile.txt'
delete from IDX
-- set sql
set #SQL1 = "BULK INSERT dbo.table FROM '" + #pathtable
+ "' WITH (FIRSTROW = 2, MAXERRORS = 0)"
-- Bulk insert
exec(#sql1)
The issue is that the last row contains a row count from the export process. If you're able to modify the export process, make sure you use the SQL command:
SET NOCOUNT ON;
If you're using a GUI to export the data there should be a place to modify the T-SQL used or an option to set nocount on.
This will prevent the last row from writing out to your file.
If you cannot modify the export process... You can get crazy and right either a console application to read the data and remove the last line or a CLR that does basically that very task.. Open the file, remove the last line, save the file then call your stored procedure above to bulk insert your data.
You need to use single quotes ' multiple times, You have used double quotes " which are treated as identifiers in sql server.
Your query should look like this...
Declare #SQL1 varchar(150)
, #path varchar(100)
, #pathtable varchar(100)
, #date datetime
SET #date = getdate();
SET #path= 'C:\imp\'
SET #pathtable = #path + 'importfile.txt'
SET #SQL1 = 'BULK INSERT dbo.table
FROM ''' + #pathtable + '''
WITH (
FIRSTROW = 2
, MAXERRORS = 0
)';
Now if you print this SQL statement it would look like this...
PRINT #SQL1
RESULT:
BULK INSERT dbo.table
FROM 'C:\imp\importfile.txt'
WITH (
FIRSTROW = 2
, MAXERRORS = 0
)
I am trying to execute the following SQL in SQL Server 2008-
DECLARE #sql nvarchar, #fullname nvarchar;
SET #fullname='1patents_corrected.csv';
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent from ' + #fullname+ ' WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR = "\n" );'
EXEC(#sql)
However I am getting this error--
Msg 2812, Level 16, State 62, Line 1
Could not find stored procedure 'B'.
What am I doing wrong here?
UPDATE-- I changed the query viz. specified a size for each varchar variable. Now the code is like this--
DECLARE #MyCounter int;
DECLARE #Fileprefix nvarchar(1000), #Filesuffix nvarchar(1000), #fullname nvarchar(1000), #Counter_string nvarchar(1000), #sql nvarchar(1000);
SET #MyCounter = 1;
SET #Fileprefix= 'C:\Arvind_gpd\patents\';
SET #Filesuffix='data_corrected.csv';
WHILE (#MyCounter < 10)
BEGIN;
Set #Counter_string= Cast(#MyCounter AS varchar(1) );
Set #fullname = (#Fileprefix+ #Counter_string + #Filesuffix );
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent from ' + #fullname+
' WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR = "\n" );'
EXEC(#sql);
SET #MyCounter = #MyCounter + 1;
END;
GO
However I am now getting a different error--
Msg 102, Level 15, State 1, Line 1
Incorrect syntax near 'C:'.
Msg 319, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
Now what am I doing incorrectly :( ?
change
DECLARE #sql nvarchar
to
DECLARE #sql nvarchar(2000)
by default the size is 1
The same applies to #fullname nvarchar; give it a size
You need to quote the file name when using BULK INSERT.
Change
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent
from ' + #fullname+
WITH ( DATAFILETYPE = "char", FIELDTERMINATOR = "^", ROWTERMINATOR ="\n" );'
to
SET #sql = 'BULK INSERT GooglePatentsIndividualDec2012.dbo.patent
FROM ''' + #fullname + '''
WITH ( DATAFILETYPE = ''char'', FIELDTERMINATOR = ''^'', ROWTERMINATOR =''\n'' )'
You'll notice that I have instances of multiple single quotes bunched together. Double single quotes stand for escape in SQL Server (ie. I am quoting a quote within another quote).
Also, I see you use a lot of semicolons in your code. No need to put them in when you are writing in T-SQL