All, I need to write a data set from a large SQL table to a .txt file. To do this I have chosen to use xp_cmdshell. The query I have been using to create the Data.txt file is
declare #sql varchar(8000)
select #sql = 'bcp "SELECT /*Lots of field names here*/ ' +
'FROM [SomeDatabase]..TableName WHERE /*Some Long Where Clause*/" ' +
'queryout "M:\\SomeDir\\SomeOtherDirectory\\Data.txt" -c -t -T -S' + ##servername
exec master..xp_cmdshell #sql
the problem I am having is that the SELECT query I am using exceeds the 1024 character limit imposed by the command line. To get around this I have decide to try and use sqlcmd to attempt to execute the SQL Query I need from a file, elliminating the error with the query length. I have tried the following query
DECLARE #DatabaseName VARCHAR(255)
DECLARE #cmd VARCHAR(8000)
SET #DatabaseName = 'SomeDatabase'
SET #CMD = 'SQLCMD -E -S (localhost) -d ' + #DBName +
'i "M:\\SomeDir\\SomeOtherDirectory\\tmpTestQuery.sql"'
EXEC master..xp_cmdshell #CMD
where 'tmpTestQuery.sql' holds the long query I want to execute, but I get the following errors
HResult 0x2AF9, Level 16, State 1
TCP Provider: No such host is known.
NULL
Sqlcmd: Error: Microsoft SQL Server Native Client 10.0 : A network-related or instance-
specific error has occurred while establishing a connection to SQL Server.
Server is not found or not accessible. Check if instance name is correct and
if SQL Server is configured to allow remote connections.
For more information see SQL Server Books Online..
Sqlcmd: Error: Microsoft SQL Server Native Client 10.0 : Login timeout expired.
NULL
I have remote connections enabled.
I would like to know what I am doing wrong, and if there is another way around the problem I am having with the query length when using xp_cmdshell?
Thanks for your time.
Note. This query will eventually be called from C#, so the plan was to write the very long query to a temporary .txt file, execute it using the method outlined and delete when finished.
One way to get around the BCP limitation is to wrap the complex query in a view or stored procedure, then have the BCP command query that object.
Your SQLCMD may not work because of the brackets around localhost. Try:
...
SET #CMD = 'SQLCMD -E -S localhost -d ' + #DBName +
...
You can insert desired data into global temp table (##temp_table) and then use it as source:
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM ##temp_table" ' +
'queryout "M:\\SomeDir\\SomeOtherDirectory\\Data.txt" -c -t -T -S' + ##servername
exec master..xp_cmdshell #sql
Related
I want to use xp_cmdshell to ping servers. I created a procedure but I have a little problem, I need to select the server IP from table that is already created.
I created a cursor to get the server IP from the table but I don't know how to use the #ip varchar variable with ping command.
This syntax didn't work:
execute xp_cmdshell 'ping #ip'
You cannot reference parameters directly within xp_cmdshell, so you have to concatenate the value when creating the command. I recommend reading: https://msdn.microsoft.com/en-us/library/ms175046.aspx
In your example, you would do something like:
DECLARE #cmd nvarchar(4000);
SET #cmd = 'ping ' + #ip;
EXEC xp_cmdshell #cmd;
According to some recommendations i use bcp Utility to write SQL server Table to .cvs file so i could later transfer the data to informix table with the same structure .
My SQLServer Stored Procedure :
ALTER PROCEDURE [dbo].[TestCVS]
AS
BEGIN
declare #sql nvarchar(4000)
select #sql = 'bcp "select * from ML..gmp4vacationbalance" queryout c:\ss\Tom.cvs -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql
END
I have four questions concerning this procedure :
1- How to allow this procedure to write to remote server instead of local server ##servername,because it's not secure to allow specific server to access my sql server ?
2-How to allow to filter the query under specific condition :
say i want to write query like this :
select * from ML..gmp4vacationbalance where balance_date = #date AND emp_num = #empNum
3-when i execute the procedure i get data like this:
Why the third column appear corrupted like this , it's varchar desc written in arabic ?
4-When i want to delimit by pipe | instead of comma , like this
select #sql = 'bcp "select * from ML..gmp4vacationbalance" queryout c:\ss\Tom.cvs -c -t| -T -S' + ##servername
I get the following error :
Question 1: Writing from a remote server
I assume you meant to say "from" not "to". Specify the server name in form ServerName\InstanceName instead of using ##servername, you will need to have permissions to access the other server (since you are using a trusted connection -T)
Question 2: How to add parameters to the BCP statement
BCP is a command line utility not part of Transact-SQL. You can't add parameters. You can format the command line executed. You'll have to make your parameters #date and #empNum strings concatenate them with the rest of the SQL string for the command line
Question 3: Wrong characters in output
Instead of -c use -w to output Unicode characters
Question 4: Pipe not working
A common problem with BCP, simply quote the pipe like this -t"|" to make that the record separator
I have a .sql script with a lot of action queries that work on some staging tables. This script needs to be run twice with some other commands in-between i.e.:
Load the staging table from source A
Use do_stuff.sql to process it
Move the results somewhere.
Repeat Steps 1-3 for source B.
The brute force approach would be to just copy & paste dostuff.sql as needed. While this would technically work, is there a better way?
I'm hoping there's a command like RunThisSQL 'C:\do_stuff.sql' that I haven't discovered yet.
Update
Well, it's been about 5 years and I just re-discovered this old question. I did this recently and made a cursor to loop thru a master table. For each record in that master table, the script runs through an inner script using variables set by the master table.
https://www.mssqltips.com/sqlservertip/1599/sql-server-cursor-example/
If you use visual studio you can create "Sql Server Database" project. Withing the project you can create script that let you execute your *.sql files in a manner
/*
Post-Deployment Script Template
--------------------------------------------------------------------------------------
This file contains SQL statements that will be appended to the build script.
Use SQLCMD syntax to include a file in the post-deployment script.
Example: :r .\myfile.sql
Use SQLCMD syntax to reference a variable in the post-deployment script.
Example: :setvar TableName MyTable
SELECT * FROM [$(TableName)]
--------------------------------------------------------------------------------------
*/
see also. http://candordeveloper.com/2013/01/08/creating-a-sql-server-database-project-in-visual-studio-2012/
Try using xp_cmdshell.
EXEC xp_cmdshell 'sqlcmd -S ' + #ServerName + ' -d ' + #DBName + ' -i ' +#FileName
xp_cmdshell and concatenation do not play together nicely, often resulting in an "Incorrect syntax near '+'" error. So further to Jeotics solution above you will need to make a variable of the entire string you pass to xp_cmdshell (including quotes around anything that may contain a space (eg filepath\filename). This is mentioned in the Microsoft documentation for xp_cmdshell here. Other issues you will have to contend with are the default set up for SQL Server which has xp_cmdshell disabled as outlined here and granting permission to non-system administrators to use xp_cmdshell outlined here. The documentation generally advises against giving xp_cmdshell rights to too many people owing to it being a vehicle for those with malintent but if, like me, you have minimal and trustworthy database users then it seems like a reasonable solution. One last issue that requires correct configuration is the SQL Server Agent as outlined here. Documentation outlines that SQL Agent is responsible for background scheduling (such as back ups) and performance of command line statements, etc..
DECLARE
#Server nvarchar (50)
,#Database nvarchar(50)
,#File nvarchar(100)
,#cmd nvarchar(300);
SET #Server = server_name;
SET #Database = database_name;
SET #File = 'C:\your file path with spaces';
SET #cmd = 'sqlcmd -S ' + #Server + ' -d ' + #Database + ' i "' + #File + '"';
EXEC xp_cmdshell #cmd;
There are some security issues with enabling xp_cmdshell in SQL Server. You can create a CLR Stored procedure, which executes the passed file content. This CLR stored procedure is especially for this purpose, not like xp_cmdshell, which can do anything over the command prompt.
issues with enabling xp_cmdshell
Create CLR stored procedure
I've been trying to work on taking the result of a large and multiply-joined SELECT statement, and email the query result as a CVS file.
I have the query correct and the emailing down, but I'm having trouble automating the export of the result as a CVS file. From what I've been reading, the best bet for auto-exporting query results is a tool called "BCP".
I attempted to use BCP like this in Management Studio:
USE FootPrint;
DECLARE #sql VARCHAR(2048);
DECLARE #dir VARCHAR(50);
SET #dir = 'C:\Users\bailey\Desktop';
SET #sql = 'bcp "SELECT TOP 10 * FROM datex_footprint.Shipments" queryout "' + #dir + '" -c -t, -T';
EXEC master..xp_cmdshell #sql;
FootPrint is the name of a specific database, and datex_footprint a schema.
(This is not the real query, just a test one).
When I run this, the error I get is:
"SQLState=S0002, NativeError = 208"
"Error = [Microsoft][SQL Server Native Client 10.0][SQL Server] Invalid object name 'datex_footprint.Shipments'."
I am 100% positive that datex_footprint.Shipments is the correct schema\table access for the data I'm trying to test on.
Does anyone see what I'm missing or doing wrong in trying to export this result to a CSV file? Specifically, though, I'm trying to automate this process. I know how to export results into a CSV file, but I want to do it in T-SQL so I can automate the generation of the file by time of day.
Any help would be appreciated!
[SOLVED]
I figured out what I was doing wrong. I was not identifying the view in complete form. I was using "schema.Table/View", instead of "database.schema.table/view".
Also, I added a "-S" + ##SERVERNAME flag -- this tells the BCP utility to use the server SQL Server is currently connected to for the query.
The correct code to generate a CSV file of a SELECT-query's results in T-SQL, SQL Server 2008 is:
DECLARE #sql VARCHAR(8000);
SELECT #sql = 'bcp "SELECT * FROM FootPrint.datex_footprint.Shipments" queryout "C:\Users\bailey\Desktop\FlatTables\YamotoShipping.csv" -c -t, -T -S' + ##SERVERNAME;
exec master..xp_cmdshell #sql;
So once I added "FootPrint." to identify the database, it worked.
NOTE: I'm running SQL Server 2008 R2.
I have got same error but I have resolved it in a different way.
I have added the default database to my ID then BCP started looking the tables in my default database and processed the files.
After searching and reading the documentation of bcp, what I found was that when we have copy the data we should be using GLobal temp table i.e. ## instead of #.. because in tempdb it will collide and wont allow to copy the data to the destination file.
Example:
DECLARE #OutputFilePath nvarchar(max); SET #OutputFilePath = 'C:\OutputData'
DECLARE #ExportSQL nvarchar(max); SET #ExportSQL = N'EXEC xp_cmdshell ''bcp
"SELECT * FROM LW_DFS_DIT.dbo.##Mytemptable " queryout "' + #OutputFilePath +
'\OutputData.txt" -T -c -t '''
EXEC(#ExportSQL)
Hope this would help
I am using sqlcmd in a T-SQl script to write a text file to a network location. However SQLCMD is failing to write to that location due to access permission to the network folder. SP is being run under my user account which has access to the network folder.
Could you please help me under which account sqlcmd will run if I do not specify -U and -P option in TSQL Script?
Use this to find the user name:
PRINT Suser_Sname();
If you don't provide credentials with -u/-p it will try to use windows authentication; i.e the windows account of whomever is running it.
I often just use Process Monitor to look at what account is being used and what the permission error is.
You say you are using SQLCMD in a T-SQL script, don't you mean you are using SQLCMD to run a T-SQL script? How is your script writing a text file? Does it work in SQL Manager? My guess is that the user account SQL Server is running under doesn't have access to that location.
If you call an SQL script via xp_cmdshell without User and Password parameters it will run in the environment of the mssqlserver service, which is very much restricted, and without changing security parameters you will get mostly an 'Access is denied' message instead of the results of the script.
To avoid this security conflict situation I use the following trick in my stored procedure create_sql_proc. I read the text of the script file, and wrap it in a procedure by adding a head and a foot to it. Now I have a script creating a stored procedure from the SQL-file called #procname.
If you let now run this stored procedure by EXEC #procname, it will run in your security environment, and delivers the result you would get by running it from a command prompt:
CREATE PROCEDURE create_sql_proc(#procName sysname, #sqlfile sysname) AS
BEGIN
DECLARE #crlf nvarchar(2) = char(10)+char(13)
DECLARE #scriptText nvarchar(max)
DECLARE #cmd nvarchar(max)
= N'SET #text = (SELECT * FROM openrowset(BULK '''+#sqlFile+''', SINGLE_CLOB) as script)'
EXEC sp_executesql #cmd , N'#text nvarchar(max) output', #text = #scriptText OUTPUT
DECLARE #ProcHead nvarchar(max) = N'CREATE or ALTER PROCEDURE '+#procName+ ' AS '+#crlf+'BEGIN'+#crlf
DECLARE #ProcTail nvarchar(max) = #crlf + N'END '
SET #scriptText = #ProcHead + #scriptText + #ProcTail
-- create TestGen stored procedure --
EXEC sys.sp_executesql #scriptText
END