I have a table with bulk data (say 2 Million rows). I need to export this data into a text file.
I've approached Generate Scripts method by which it throws
System.OutOfMemoryExeception.
I need to some how convert the data into text file. Can sqlcmd approach be helpful? If so please suggest the steps.
Turn on xp_cmdshell in Facets and run this -
DECLARE #sql NVARCHAR(4000) = 'bcp "SELECT * FROM sys.schemas" queryout "D:\sample.txt" -S ' + ##servername + ' -T -w -r -t'
EXEC sys.xp_cmdshell #sql
Related
I'm trying to use BCP to export my data to a .TXT file, but I needed to do an incremental in the same file, since I will need to export data from two views.
Example:
SET #cmd = '"SELECT * FROM Site"'
SELECT #sql = 'bcp '+#cmd+' queryout D:\mytest.txt -c -t; -T -S SERVER-PC\SQLEXPRESS';
EXEC xp_cmdshell #sql;
SET #cmd = '"SELECT * FROM Customers"'
SELECT #sql = 'bcp '+#cmd+' queryout D:\mytest.txt -c -t; -T -S SERVER-PC\SQLEXPRESS';
EXEC xp_cmdshell #sql;
Data out:
H;04399024100427;20160620
V;04399024100427;CUSTOMER I;STATE;CITY;NAME;75123390;A
I thought of UNION ALL but the structure of the tables are different and I have cases where I will need to export data from up to 5 tables.
I'm having a hard time working out how to write XML to my hard drive in SQL Server. I gather I could do something like this:
EXEC xp_cmdshell 'bcp "###INSERT SELECT/FOR XML QUERY###" queryout "C:\test.xml" -T -c -t'
But my SELECT/FOR XML query runs to about 40 lines so I'd rather not. I tried storing the XML as #storedXML, saving it in a table and then querying that table with bcp:
CREATE TABLE xmlOutput(outXML xml)
INSERT INTO xmlOutput(outXML)
VALUES (#storedXML)
DECLARE #sqlCmd VARCHAR(1000)
SET #sqlCmd = 'bcp "SELECT * FROM xmlOutput" queryout C:\test.xml -c -T -t'
EXEC xp_cmdshell #sqlCmd
But all that gets me is "Unable to open BCP host data-file".
How to populate a .csv file from a SQL Server stored procedure?
We don't have Office on the Server. The .CSV file has to be populated from a stored procedure result set.
How to export to a .CSV without using SSIS package?
End result, I will have to generate email alert by attaching this CSV file as report.
I will have to use bulk copy program utility (BCP), I am looking on samples for BCP to generate csv file
There are two methods to achieve that:
(1) Using OPENROWSET
Try implementing a similar logic:
INSERT INTO OPENROWSET('Microsoft.ACE.OLEDB.12.0','Text;Database=D:\;HDR=YES;FMT=Delimited','SELECT * FROM [FileName.csv]')
EXEC Sp_TEST
(2) Using bcp
From the third link in References section:
The queryout method allows you to BCP from the result of a stored procedure, which opens up a lot of possibilities and offers a lot of control over the file format. For anything other than a simple table extract I would tend to use this method rather than a view. I would also format each line within the stored procedure. This means that the formatting can be tested independently from the file creation
declare #sql varchar(8000)
select #sql = 'bcp "exec sp_Test"
queryout c:\bcp\sysobjects.csv -c -t, -T -S'
+ ##servername
exec master..xp_cmdshell #sql
References
How To Export Data To the .csv file using Sql server Stored Procedure.
How to produce an csv output file from stored procedure in SQL Server
Creating CSV Files Using BCP and Stored Procedures
Exporting a csv file via stored procedure
Writing select result to a csv file
Look at the code below. A stored procedure is created and then used as the source in the BCP command. Then a text file is generated. Delimiters are discussed in the code sample in the link.
Code sample taken from Creating CSV Files Using BCP and Stored Procedures
use tempdb
go
create proc s_bcpMasterSysobjects
as
select '"' + name + '"'
+ ',' + '"' + convert(varchar(8), crdate, 112) + '"'
+ ',' + '"' + convert(varchar(8), crdate, 108) + '"'
from master..sysobjects
order by crdate desc
go
declare #sql varchar(8000)
select #sql = 'bcp "exec tempdb..s_bcpMasterSysobjects"
queryout c:\bcp\sysobjects.txt -c -t, -T -S'
+ ##servername
exec master..xp_cmdshell #sql
I would use BCP
see example
declare #sql varchar(2000)
select #sql = 'bcp "SQL QUERY HERE" queryout '+#fileLocation+#fileName+'.csv -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql, NO_OUTPUT
if you need to see any errors or whats it's doing, just remove the ", NO_OUTPUT"
Create a little script, the csv is a simple format : csv = comma separate value. Separate your values width comma.
I need to BCP a table into a tab-delimited file, but I need the column names in the first record of the table. Question 1: Am I right that BCP does not have a switch for this? Question 2: If not, why?
I tried to do the following:
BCP "declare #colnames varchar(max); select #colnames=coalesce (#colnames+char(9), '')
+ Column_Name from db.information_Schema.columns where table_name='table1' order by
ordinal_position; select #colnames" queryout Table1_Columns.tsv -S?? -U?? -P?? -f** -e**
The format file looks like this:
9.0
1
1 SQLCHAR 0 100 "\r\n" 1 Column_Names SQL_Latin1_General_CP1_CI_AS
This gets me a file of the column names, then a second BCP command gets me a file of data, and I just DOS-copy the two together. Question 3: Am I clever or what? Question 4: Why doesn't it work? I get the error:
SQLState = S1000, NativeError = 0
Error = [Microsoft][SQL Native Client]Host-file columns may be skipped only when
copying into the Server
bcp does not support exporting the column headers with the data, however there are some workarounds like exporting the headers in a separate file, then merging both the headers and data files as the following:
exec master..xp_cmdshell 'BCP "select 'SETTINGS_ID','GROUP_NAME'" queryout d:\header.csv -c -T -t,'
exec master..xp_cmdshell 'BCP "select SETTINGS_ID,GROUP_NAME from [DB]..[TABLE]" queryout "d:\columns.csv" -c -t, -T '
exec master..xp_cmdshell 'copy /b "d:\header.csv"+"d:\columns.csv" "d:/result.csv"'
You may also delete the unused files:
exec master..xp_cmdshell 'del "d:\header.csv"'
exec master..xp_cmdshell 'del "d:\columns.csv"'
Or maybe you can combine all the data in a view (adding headers) and export it
I debug a stored procedure (SQL Server 2005) and I need to find out some values in a datatable.
The procedure is run by an event of the application and I watch just the debugging output.
I do the following my stored procedure (SQL Server 2005), I took a system table (master.dbo.spt_values) as example:
set #logtext = 'select name, type from master.dbo.spt_values where number=6'
--set #logtext = 'master.dbo.spt_values'
SET #cmd = 'bcp ' + #logtext + ' out "c:\spt_values.dat" -U uId -P uPass -c'
EXEC master..XP_CMDSHELL #cmd
So, when I uncomment the second like everything works, a file apprears on the C:\ drive... but if I coment it back leaving only the first line, any output is generated.
How to fix this problem?
bcp out exports tables.
To export a query use queryout instead - you'll need to wrap your query in "double quotes"
set #logtext = '"select name, type from master.dbo.spt_values where number=6"'
--set #logtext = 'master.dbo.spt_values'
SET #cmd = 'bcp ' + #logtext + ' queryout "c:\spt_values.dat" -U uId -P uPass -c'
EXEC master..XP_CMDSHELL #cmd
http://msdn.microsoft.com/en-us/library/ms162802.aspx