I am using the below script for exporting data from table, which is working completely fine.
declare #sql varchar(8000)
select #sql = 'bcp "Select * from [CV18].dbo.ZMM002" queryout D:\Share\Vendor_portal_Pending_IBD\bcptest.txt -c -t^| -T -S'+ ##servername
exec master..xp_cmdshell #sql
But I want to add headers to it. I have found out a way to do that by using UNION ALL with header names.
select 'Counter','External_ID','Delivery_Date','Transport_ID','Bill_of_Lading','Delivery_Item','Material','Delivery_QTY',
'Unit','PO_Number','PO_Item'
union all
select * from ZMM002
But how can I add this union all query in my bcp query?
I get a syntax error, because I am using single quotes (') in union all query, which is cutting off the outer single quote, i.e #sql = ' '
In my opinion it is better to do this as a post-op.
Export two files, one with the header, the other with the actual data. Afterwards, concatenate the two files using the COPY command.
Suppose you exported to header.txt and data.txt, the command would be
COPY /b "\\path\header.txt"+"\\path\data.txt" "\\path\data.dat"
Where \\path would be your actual path.
If you're formatting the header SQL, double the qoutes:
SET #sql='select ''Counter'',''External_ID'',''Delivery_Date'',''Transport_ID'',''Bill_of_Lading'',''Delivery_Item'',''Material'',''Delivery_QTY'',''Unit'',''PO_Number'',''PO_Item''';
Related
How to populate a .csv file from a SQL Server stored procedure?
We don't have Office on the Server. The .CSV file has to be populated from a stored procedure result set.
How to export to a .CSV without using SSIS package?
End result, I will have to generate email alert by attaching this CSV file as report.
I will have to use bulk copy program utility (BCP), I am looking on samples for BCP to generate csv file
There are two methods to achieve that:
(1) Using OPENROWSET
Try implementing a similar logic:
INSERT INTO OPENROWSET('Microsoft.ACE.OLEDB.12.0','Text;Database=D:\;HDR=YES;FMT=Delimited','SELECT * FROM [FileName.csv]')
EXEC Sp_TEST
(2) Using bcp
From the third link in References section:
The queryout method allows you to BCP from the result of a stored procedure, which opens up a lot of possibilities and offers a lot of control over the file format. For anything other than a simple table extract I would tend to use this method rather than a view. I would also format each line within the stored procedure. This means that the formatting can be tested independently from the file creation
declare #sql varchar(8000)
select #sql = 'bcp "exec sp_Test"
queryout c:\bcp\sysobjects.csv -c -t, -T -S'
+ ##servername
exec master..xp_cmdshell #sql
References
How To Export Data To the .csv file using Sql server Stored Procedure.
How to produce an csv output file from stored procedure in SQL Server
Creating CSV Files Using BCP and Stored Procedures
Exporting a csv file via stored procedure
Writing select result to a csv file
Look at the code below. A stored procedure is created and then used as the source in the BCP command. Then a text file is generated. Delimiters are discussed in the code sample in the link.
Code sample taken from Creating CSV Files Using BCP and Stored Procedures
use tempdb
go
create proc s_bcpMasterSysobjects
as
select '"' + name + '"'
+ ',' + '"' + convert(varchar(8), crdate, 112) + '"'
+ ',' + '"' + convert(varchar(8), crdate, 108) + '"'
from master..sysobjects
order by crdate desc
go
declare #sql varchar(8000)
select #sql = 'bcp "exec tempdb..s_bcpMasterSysobjects"
queryout c:\bcp\sysobjects.txt -c -t, -T -S'
+ ##servername
exec master..xp_cmdshell #sql
I would use BCP
see example
declare #sql varchar(2000)
select #sql = 'bcp "SQL QUERY HERE" queryout '+#fileLocation+#fileName+'.csv -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql, NO_OUTPUT
if you need to see any errors or whats it's doing, just remove the ", NO_OUTPUT"
Create a little script, the csv is a simple format : csv = comma separate value. Separate your values width comma.
I have a table with bulk data (say 2 Million rows). I need to export this data into a text file.
I've approached Generate Scripts method by which it throws
System.OutOfMemoryExeception.
I need to some how convert the data into text file. Can sqlcmd approach be helpful? If so please suggest the steps.
Turn on xp_cmdshell in Facets and run this -
DECLARE #sql NVARCHAR(4000) = 'bcp "SELECT * FROM sys.schemas" queryout "D:\sample.txt" -S ' + ##servername + ' -T -w -r -t'
EXEC sys.xp_cmdshell #sql
I need to BCP a table into a tab-delimited file, but I need the column names in the first record of the table. Question 1: Am I right that BCP does not have a switch for this? Question 2: If not, why?
I tried to do the following:
BCP "declare #colnames varchar(max); select #colnames=coalesce (#colnames+char(9), '')
+ Column_Name from db.information_Schema.columns where table_name='table1' order by
ordinal_position; select #colnames" queryout Table1_Columns.tsv -S?? -U?? -P?? -f** -e**
The format file looks like this:
9.0
1
1 SQLCHAR 0 100 "\r\n" 1 Column_Names SQL_Latin1_General_CP1_CI_AS
This gets me a file of the column names, then a second BCP command gets me a file of data, and I just DOS-copy the two together. Question 3: Am I clever or what? Question 4: Why doesn't it work? I get the error:
SQLState = S1000, NativeError = 0
Error = [Microsoft][SQL Native Client]Host-file columns may be skipped only when
copying into the Server
bcp does not support exporting the column headers with the data, however there are some workarounds like exporting the headers in a separate file, then merging both the headers and data files as the following:
exec master..xp_cmdshell 'BCP "select 'SETTINGS_ID','GROUP_NAME'" queryout d:\header.csv -c -T -t,'
exec master..xp_cmdshell 'BCP "select SETTINGS_ID,GROUP_NAME from [DB]..[TABLE]" queryout "d:\columns.csv" -c -t, -T '
exec master..xp_cmdshell 'copy /b "d:\header.csv"+"d:\columns.csv" "d:/result.csv"'
You may also delete the unused files:
exec master..xp_cmdshell 'del "d:\header.csv"'
exec master..xp_cmdshell 'del "d:\columns.csv"'
Or maybe you can combine all the data in a view (adding headers) and export it
I have to debug a large stored procedure in SQL Server 2008 (also in 2005).
because I can't go in that procedure StepByStep, I need to debug it using some output files.
actually i use something like
DECLARE #SQL VARCHAR(8000)
SELECT #SQL = 'BCP "SELECT * FROM MY_TABLE" QUERYOUT "D:\TDB\test.txt" -C -T -w'
EXEC MASTER..XP_CMDSHELL #SQL
but this approach has multiple limitations like impossibility to use # tables, and a complex way to use filters (where X='+cast(#MYLocalVar as varchar)+')...
Is there a other way to output a select to a file, like MySql does
SELECT * into outfile '../../htdocs/VIP/Temp/temp.txt' from tmp_Menu2;
You can put the data into a table quite readily:
SELECT *
into anotherdatabase..outtable
from . . .
This creates a new table with whatever columns you want.
To output something into a file requires an extra step, using bulk export.
I am writing a cleanup script. This script will run on weekend and clean up the db. Tables are related to Eamils and path of attachments are being stored in table. In cleanup of tables I also have to delete files from folder.
The path of files is like following.
\\xxx.xxx.xxx.xxx\EmailAttachments\Some Confirmation for xyz Children Centre_9FW4ZE1C57324B70EC79WZ15FT9FA19E.pdf
I can delete multiple files like following.
xp_cmdshell 'del c:\xyz.txt, abc.txt'
BUT when I create a CSV from table using FOR XML PATH('') the string cut off at the end. There might be 1000s of rows to delete so I don't want to use cursor to delete files from folder.
How can I delete files from folder
without using cursor
What permissions do I need on
network folder to delete files using t-sql from sql server
EDIT:
I have used cursor and it looks ok, not taking so much time. One problem which I am facing is
The sql server consider file name with space as two files like following statement
xp_cmdshell 'del E:\Standard Invite.doc'
throws error
Could Not Find E:\Standard
Could Not Find C:\Windows\system32\Invite.doc
NULL
Thanks.
Personally, I wouldn't worry too much about using a cursor here. Cursors are only 'mostly evil'; as your task isn't a set-based operation a cursor may be the most effective solution.
Although you have a comment stating that it will take an "awful lot of time" to use a cursor, in this case the biggest overhead is the actual delete of the file (not the cursor).
Note: The file deletion is done by the Operation System, not by the RDBMS.
As the delete is being done by calling xp_cmdshell, and because it it a procedure (not a function, etc), you can't call it and pass in a table's contents.
What you could do is build up a string, and execute that. But note, you are limitted to a maximum of 8000 characters in this string. As you have already said that you may have thousands of files, you will certaily not fit it within 8000 characters.
This means that you are going to need a loop no matter what.
DECLARE
#command VARCHAR(8000),
#next_id INT,
#next_file VARCHAR(8000),
#total_len INT
SELECT
#command = 'DEL ',
#total_len = 4
SELECT TOP 1
#next_id = id,
#next_file = file_name + ', '
FROM
table_of_files_to_delete
ORDER BY
id DESC
WHILE (#next_file IS NOT NULL)
BEGIN
WHILE ((#total_len + LEN(#next_file)) <= 8000) AND (#next_file IS NOT NULL)
BEGIN
SELECT
#command = #command + #next_file,
#total_len = #total_len + LEN(#next_file)
SELECT
#next_file = NULL
SELECT TOP 1
#next_id = id,
#next_file = file_name + ', '
FROM
table_of_files_to_delete
WHERE
id < #next_id
ORDER BY
id DESC
END
SET #command = SUBSTRING(#command, 1, #total_len - 2) -- remove the last ', '
EXEC xp_cmdshell #command
SELECT
#command = 'DEL ',
#total_len = 4
END
Not pretty, huh?
What you may be able do, depending on what needs deleting, is to use wild-cards. For example:
EXEC xp_cmdshell 'DELETE C:\abc\def\*.txt'
To delete files with space in name you need to enclose the filename with "
xp_cmdshell 'del "E:\Standard Invite.doc"'
DECLARE #deleteSql varchar(500)
,#myPath varchar(500) = '\\DestinationFolder\'
SET #deleteSql = 'EXEC master..xp_cmdshell ''del '+#myPath +'*.csv'''
EXEC(#deleteSql)