I have a .sql script with a lot of action queries that work on some staging tables. This script needs to be run twice with some other commands in-between i.e.:
Load the staging table from source A
Use do_stuff.sql to process it
Move the results somewhere.
Repeat Steps 1-3 for source B.
The brute force approach would be to just copy & paste dostuff.sql as needed. While this would technically work, is there a better way?
I'm hoping there's a command like RunThisSQL 'C:\do_stuff.sql' that I haven't discovered yet.
Update
Well, it's been about 5 years and I just re-discovered this old question. I did this recently and made a cursor to loop thru a master table. For each record in that master table, the script runs through an inner script using variables set by the master table.
https://www.mssqltips.com/sqlservertip/1599/sql-server-cursor-example/
If you use visual studio you can create "Sql Server Database" project. Withing the project you can create script that let you execute your *.sql files in a manner
/*
Post-Deployment Script Template
--------------------------------------------------------------------------------------
This file contains SQL statements that will be appended to the build script.
Use SQLCMD syntax to include a file in the post-deployment script.
Example: :r .\myfile.sql
Use SQLCMD syntax to reference a variable in the post-deployment script.
Example: :setvar TableName MyTable
SELECT * FROM [$(TableName)]
--------------------------------------------------------------------------------------
*/
see also. http://candordeveloper.com/2013/01/08/creating-a-sql-server-database-project-in-visual-studio-2012/
Try using xp_cmdshell.
EXEC xp_cmdshell 'sqlcmd -S ' + #ServerName + ' -d ' + #DBName + ' -i ' +#FileName
xp_cmdshell and concatenation do not play together nicely, often resulting in an "Incorrect syntax near '+'" error. So further to Jeotics solution above you will need to make a variable of the entire string you pass to xp_cmdshell (including quotes around anything that may contain a space (eg filepath\filename). This is mentioned in the Microsoft documentation for xp_cmdshell here. Other issues you will have to contend with are the default set up for SQL Server which has xp_cmdshell disabled as outlined here and granting permission to non-system administrators to use xp_cmdshell outlined here. The documentation generally advises against giving xp_cmdshell rights to too many people owing to it being a vehicle for those with malintent but if, like me, you have minimal and trustworthy database users then it seems like a reasonable solution. One last issue that requires correct configuration is the SQL Server Agent as outlined here. Documentation outlines that SQL Agent is responsible for background scheduling (such as back ups) and performance of command line statements, etc..
DECLARE
#Server nvarchar (50)
,#Database nvarchar(50)
,#File nvarchar(100)
,#cmd nvarchar(300);
SET #Server = server_name;
SET #Database = database_name;
SET #File = 'C:\your file path with spaces';
SET #cmd = 'sqlcmd -S ' + #Server + ' -d ' + #Database + ' i "' + #File + '"';
EXEC xp_cmdshell #cmd;
There are some security issues with enabling xp_cmdshell in SQL Server. You can create a CLR Stored procedure, which executes the passed file content. This CLR stored procedure is especially for this purpose, not like xp_cmdshell, which can do anything over the command prompt.
issues with enabling xp_cmdshell
Create CLR stored procedure
Related
I have a central management database which collates some information and runs some dynamic SQL for various other tasks when a new database is restored into the environment. One of those tasks is going to be a bit complex to achieve through dynamic SQL so I had the idea of creating a master copy stored procedure in the central DB and copying that over to the new databases after they are restored.
I've seen a few examples of people trying to do that on here but I can't get anything to play ball.
Here's what i am trying to achieve conceptually, note that I'm trying to cater for potentially multiple stored procedures to be created in this way just for future proofing.
declare #sql nvarchar(max), #DatabaseName nvarchar(200)
set #DatabaseName = 'TargetDatabase'
set #sql =
(
SELECT definition + char(13) + 'GO'
FROM sys.sql_modules s
INNER JOIN sys.procedures p
ON [s].[object_id] = [p].[object_id] WHERE p.name LIKE '%mastercopy%'
)
exec #sql
Thanks
Instead of creating dynamic script you could use one script with all the procedures that you want to create (you can script all the procs you want using 2 click in SSMS), you then run this script manually in the context of the database where you want to create these procedures or by passing the file with this script to sqlcmd with -i and passing the correct database name with -d.
Here Use the sqlcmd Utility you can see the examples.
I've been trying to work on taking the result of a large and multiply-joined SELECT statement, and email the query result as a CVS file.
I have the query correct and the emailing down, but I'm having trouble automating the export of the result as a CVS file. From what I've been reading, the best bet for auto-exporting query results is a tool called "BCP".
I attempted to use BCP like this in Management Studio:
USE FootPrint;
DECLARE #sql VARCHAR(2048);
DECLARE #dir VARCHAR(50);
SET #dir = 'C:\Users\bailey\Desktop';
SET #sql = 'bcp "SELECT TOP 10 * FROM datex_footprint.Shipments" queryout "' + #dir + '" -c -t, -T';
EXEC master..xp_cmdshell #sql;
FootPrint is the name of a specific database, and datex_footprint a schema.
(This is not the real query, just a test one).
When I run this, the error I get is:
"SQLState=S0002, NativeError = 208"
"Error = [Microsoft][SQL Server Native Client 10.0][SQL Server] Invalid object name 'datex_footprint.Shipments'."
I am 100% positive that datex_footprint.Shipments is the correct schema\table access for the data I'm trying to test on.
Does anyone see what I'm missing or doing wrong in trying to export this result to a CSV file? Specifically, though, I'm trying to automate this process. I know how to export results into a CSV file, but I want to do it in T-SQL so I can automate the generation of the file by time of day.
Any help would be appreciated!
[SOLVED]
I figured out what I was doing wrong. I was not identifying the view in complete form. I was using "schema.Table/View", instead of "database.schema.table/view".
Also, I added a "-S" + ##SERVERNAME flag -- this tells the BCP utility to use the server SQL Server is currently connected to for the query.
The correct code to generate a CSV file of a SELECT-query's results in T-SQL, SQL Server 2008 is:
DECLARE #sql VARCHAR(8000);
SELECT #sql = 'bcp "SELECT * FROM FootPrint.datex_footprint.Shipments" queryout "C:\Users\bailey\Desktop\FlatTables\YamotoShipping.csv" -c -t, -T -S' + ##SERVERNAME;
exec master..xp_cmdshell #sql;
So once I added "FootPrint." to identify the database, it worked.
NOTE: I'm running SQL Server 2008 R2.
I have got same error but I have resolved it in a different way.
I have added the default database to my ID then BCP started looking the tables in my default database and processed the files.
After searching and reading the documentation of bcp, what I found was that when we have copy the data we should be using GLobal temp table i.e. ## instead of #.. because in tempdb it will collide and wont allow to copy the data to the destination file.
Example:
DECLARE #OutputFilePath nvarchar(max); SET #OutputFilePath = 'C:\OutputData'
DECLARE #ExportSQL nvarchar(max); SET #ExportSQL = N'EXEC xp_cmdshell ''bcp
"SELECT * FROM LW_DFS_DIT.dbo.##Mytemptable " queryout "' + #OutputFilePath +
'\OutputData.txt" -T -c -t '''
EXEC(#ExportSQL)
Hope this would help
Heyy,
I'm trying to use BCP to export a SP result to a text file using this query:
EXEC xp_cmdshell 'bcp "exec asmary..usp_Contract_SelectByEmpId -1,1" queryout "C:\test.txt" -w -C OEM -t$ -T -r ~ -S heba\HEBADREAMNET '
The output of this query is telling this error:
Error = [Microsoft][SQL Server Native Client 10.0][SQL Server]Incorrect syntax near the keyword 'where'.
even thought I'm sure that the stored procedure "usp_Contract_SelectByEmpId" is working correctly.
Anyone faced that kind of error before?
As Lynn suggested, check your stored procedure. It looks like the issue is within that.
Ensure any plain SELECT works (e.g., C: drive is database server's local drive, not necessarily your own local drive).
If the first two items work fine, then add SET FMTONLY OFF as follows:
EXEC xp_cmdshell 'bcp "set fmtonly off exec asmary..usp_Contract_SelectByEmpId -1,1" queryout "C:\test.txt" -w -C OEM -t$ -T -r ~ -S heba\HEBADREAMNET'
I have to admit that when I tried similar on my computer it failed with 'Function sequence error', and I found that it is related to a SQL Server 2008 bug fixed in 2011.
Please note also that even without SET FMTONLY OFF everything works with BCP library (odbcbcp.dll/odbcbcp.lib). So, you can have much more generic ODBC-wide bcp solution if you write your own wrapper executable (for instance, in C or C++).
I also found the following at http://msdn.microsoft.com/en-us/library/ms162802.aspx
The query can reference a stored procedure as long as all tables referenced inside the stored procedure exist prior to executing the bcp statement. For example, if the stored procedure generates a temp table, the bcp statement fails because the temp table is available only at run time and not at statement execution time. In this case, consider inserting the results of the stored procedure into a table and then use bcp to copy the data from the table into a data file.
Please see also my later separate reply - I think the whole concept of using stored procedure for BCP/queryout is wrong.
Try this.
DECLARE #strbcpcmd NVARCHAR(max)
SET #strbcpcmd = 'bcp "EXEC asmary..usp_Contract_SelectByEmpId -1,1" queryout "C:\test.txt" -w -C OEM -t"$" -T -S'+##servername
EXEC master..xp_cmdshell #strbcpcmd
Sorry for flooding your question with multiple answers, but I wanted to find out how much heavier (performance-wise) the use of stored procedure is compared to plain SELECT. And I got a very important information from
http://social.msdn.microsoft.com/Forums/en-US/transactsql/thread/b8340289-7d7e-4a8d-b570-bec7a0d73ead/
This forced me to create another (separate) answer. The post I refer to invalidates the whole concept.
In a few words: stored procedure might be called several (3) times in order to figure out the structure of the resultset, then the actual data.
Therefore (and especially if calling from SQL Server connection rather than client), I think it makes a lot more sense to have a stored procedure or function, which will return SELECT statement. Then you can have another generic stored procedure or function to create and execute full BCP command with that statement embedded. I am pretty sure that in this case BCP might use a lot better execution plan. Unfortunately, I cannot verify that in practice, because of BCP bug in SQL Server 2008 R2 I mentioned in my previous post.
N.B. Please be careful creating dynamic queries and escape all explicit literal strings (i.e. repeat all single quotes twice) in order to avoid notorious SQL injection. Unfortunately, there is another pitfall: you should ensure you are not escaping your queries twice or more times.
I am running SQL Server 2005 on Windows Server 2003 machine.
I have a requirement to accumulate small text files into a bigger one.
So I use
exec xp_cmdshell #sql
where #sql=
'copy /b'+#sourcePath+#sourceFile+' '+#destinationPath+#NewFileName
Both the source and destination paths are on a separate server.
Seldom this process fails and I don't find anything else in the event or SQL Server logs.
The Surface Area Config for xp_cmdshell is also enabled.
Please help.....
I just tested this on my sql server 2005 and EXEC dbo.xp_cmdshell always returns output (even in the case of a bogus command) in the form of a table. For C#, if you call this code with ExecuteNonQuery, then call it with ExecuteReader and read the output. Alternatively, you could dump the output in a table so that you can look at it later at your leisure. Create a table like this :
CREATE TABLE [dbo].[xp_cmdShellOutput](
[errorMsg] [nvarchar](max) NULL
)
and then use this code :
DECLARE #sql AS VARCHAR(600)
SELECT #sql = '<your command>'
INSERT dbo.xp_cmdShellOutput(errorMsg)
EXEC dbo.xp_cmdshell #sql
I am using sqlcmd in a T-SQl script to write a text file to a network location. However SQLCMD is failing to write to that location due to access permission to the network folder. SP is being run under my user account which has access to the network folder.
Could you please help me under which account sqlcmd will run if I do not specify -U and -P option in TSQL Script?
Use this to find the user name:
PRINT Suser_Sname();
If you don't provide credentials with -u/-p it will try to use windows authentication; i.e the windows account of whomever is running it.
I often just use Process Monitor to look at what account is being used and what the permission error is.
You say you are using SQLCMD in a T-SQL script, don't you mean you are using SQLCMD to run a T-SQL script? How is your script writing a text file? Does it work in SQL Manager? My guess is that the user account SQL Server is running under doesn't have access to that location.
If you call an SQL script via xp_cmdshell without User and Password parameters it will run in the environment of the mssqlserver service, which is very much restricted, and without changing security parameters you will get mostly an 'Access is denied' message instead of the results of the script.
To avoid this security conflict situation I use the following trick in my stored procedure create_sql_proc. I read the text of the script file, and wrap it in a procedure by adding a head and a foot to it. Now I have a script creating a stored procedure from the SQL-file called #procname.
If you let now run this stored procedure by EXEC #procname, it will run in your security environment, and delivers the result you would get by running it from a command prompt:
CREATE PROCEDURE create_sql_proc(#procName sysname, #sqlfile sysname) AS
BEGIN
DECLARE #crlf nvarchar(2) = char(10)+char(13)
DECLARE #scriptText nvarchar(max)
DECLARE #cmd nvarchar(max)
= N'SET #text = (SELECT * FROM openrowset(BULK '''+#sqlFile+''', SINGLE_CLOB) as script)'
EXEC sp_executesql #cmd , N'#text nvarchar(max) output', #text = #scriptText OUTPUT
DECLARE #ProcHead nvarchar(max) = N'CREATE or ALTER PROCEDURE '+#procName+ ' AS '+#crlf+'BEGIN'+#crlf
DECLARE #ProcTail nvarchar(max) = #crlf + N'END '
SET #scriptText = #ProcHead + #scriptText + #ProcTail
-- create TestGen stored procedure --
EXEC sys.sp_executesql #scriptText
END