I want to set up the step in release process for azure pipeline where I can run the SQL from the file checked into the repo against multiple databases.
In my environment, I have one central db, one of the table for e.g "Control" table, keeps the connection to all the databases against which I need to run the sql during the deployment.
What I dont want to do is to setup the deployment step for each database. Is Powershell my only friend?
If so, how can I let powershell know what is the SQL in the file checked in during the deployment?
What I dont want to do is to setup the deployment step for each database. Is Powershell my only friend? If so, how can I let powershell know what is the SQL in the file checked in during the deployment?
In your case, Powershell seems to be the most appropriate.
In order to get the SQL in the file checked in during the deployment, we could use the powershell scripts to get the file names and paths, like:
$files=$(git diff HEAD HEAD~ --name-only)
$temp=$files -split ' '
$count=$temp.Length
echo "Total changed $count files"
For ($i=0; $i -lt $temp.Length; $i++)
{
$name=$temp[$i]
echo "this is $name file"
Copy-Item "$name" -Destination "$(System.DefaultWorkingDirectory)\SQLDeploymentFolder"
}
Check the ticket for some more details.
Then, we could use the extension Run SQL Server Scripts or Run SQL / SQLCMD Scripts passing multiple SQLCMD variables to all SQL Scripts in a folder.
Hope this helps.
Related
I have a .sql dump file 20 gb and I am trying to run it on Mysql workbench using run script and after successful execution, using SSMA I'll migrate the data from Mysql workbench to SQL Server. I have migrated the data this way many times successfully however for 20 gb file it seems very time-consuming. Please let me know if there is any alternate way to achieve this quickly. I have followed the following link:
Steps to migrate mysql tables to sql server using SSMA!
From your Title "unable to run .sql file in SSMS" and "I have a .sql dump file 20 gb" are you trying to open a 20GB .sql in SSMS? That's never going to work. SSMS is a 32bit application, so the maximum addressable memory is 2GB. If you want to run your .sql file, I suggest using sqlcmd.
Open up Powershell, and then run the command below replacing the appropriate parts:
sqlcmd -S {Server Name/ServerIP} -U {Your Login} -i {Your full path to your script}
You'll be prompted for your password and then you the file will be run. So, as an example, you might run:
sqlcmd -S svSQL2017 -U Larnu -i \\svFileServer\SQLShare\Scripts\BigBatchFile.sql
If you are using integrated security, then don't pass the -U parameter for the command.
Edit: This answer is no relevant to the OPs question, as they were using "SSMS" as a synonym for SQL Server, which it is not. I have left this here for the moment so the OP can review my comments, and I will likely remove this answer at a later point.
I am trying to run a .sql script on a schedule. I have created a batch file to run the script. The script runs fine in sql server management studio and also when I run the batch file content through cmd.
Contents of the batch file:
sqlcmd -S omfmesql -U OMESRV -P orat -i "\\pvsrv-
fsr14\data\Projects\Stat_Table_Creation_unique.sql"
The sql script is supposed to update a stat table. When I run it though cmd and refresh the stat table, the numbers are updated. But when I run this batch file through Task Scheduler, the only action that seems to be performed is running C:\Windows\SYSTEM32\cmd.exe
The task is stated to be completed successfully but the sql query is just not run.
I am not too experienced with Task Scheduler. Any help here would be very much appreciated. Thanks!
Note: I am not intending to use SQL Server Agent
If you have not done so, you need to set the location in Task Scheduler (TS). In at least some versions of TS, this can only be done when you create a basic task, not from the more general "Create Task..." option. Ensure that all the paths in the batch file are absolute or are based in this location.
I have a very basic script which runs fine on my local PC. It simply backs up some folders and their contents then adds a date stamp on the folder name.
The script backs up folders on two different servers (Server2 & Server3).
Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned
#Part 1
Copy-Item -Path "\\Server3\Example Location" -Destination "\\Server3\Example Location_$(get-date -f yyyyMMdd)" -Recurse
#Part 2
Copy-Item -Path "\\Server2\Example Location" -Destination "\\Server2\Example Location_$(get-date -f yyyyMMdd)" -Recurse
It runs both parts perfectly in the below environments:
On my local PC
When I remotely connect to Server 2 and right click > Run with PowerShell
When I remotely connect to Server 2 and edit > Run script
However, when I try to automate this and create a SQL Server Agent Job (again, on Server 2) only Part 2 actually backs up. The job successfully completes, but Part 1 appears to get ignored (i.e. running on Server 2, backing up on Server 3).
Any ideas why running as a SQL job would cause this?
n.b. The job is set to run as 'SQL Server Agent Service Account'.
Couple things...
If you run your PS script as a step type Operating system (CmdExec) then there will never be an error message returned to the SQL-Server Job.
With that being said, if your SQL-Server Agent Service account does not have access to the \Server3... folders, the copy will fail and as above said, no error message will be transferred to the calling SQL-Agent Job.
I deploy some .bteq and .sql scripts on a TERADATA database. For doing this, I use a client on my desktop called BTEQWin version 13.10.0.03.
I get the .bteq/.sql from a version control like pvcs/svn etc and all I do once the files are in my workspace folder (from Version control tool), to just drag and drop the files from Windows browser to BTEQWin client (which I connect to a database prior to drag/drop for running those scripts).
Now, I have to automate this whole process in UNIX.
I have written a SHELL KSH/BASH script which is getting all the .bteq/.sql from a TAG/LABEL in the version control tool to a given UNIX folder. Now, all I need to do is the pass these files one by one (i'll take care of the order) to Teradata client.
My ?
- what client do I need to tell Unix admin team to install on Unix server - so that I can run something like below:
someTeraDataCommand -u username -p password -h hostname -d database -f filenametoexectue | tee output_filename.log
Where, someTeraDataCommand is the client / executable - which will let me run Teradata scripts (like I was doing using BTEQWin on my desktop - GUI session). Other parameters can be username, password, which database to connect on what server and which file to run or make that file passed to the command using "<" operator at command line.
Any idea?
- What client ?
Assuming the complete Teradata Tools and Utilities package is installed on your UNIX server (which will have the connectivity tools to talk to Teradata), you should have access to bteq from the command line. Something like this:
bteq < script_file > output_file
Your script file should contain a .LOGON statement to establish the connection:
.LOGON yourTDPID/your_account,your_pw
You might also need to use other commands to set your default database or non-default session values.
Another option would be to combine the SQL and call to BTEQ in a Korn shell script:
#!/usr/bin/ksh
##############
SHELL_NAME=`basename $0`
PRG_NAME=`basename $(SHELL_NAME} .ksh`
LOG_FILE=${PRG_NAME}.log
OUT_FILE=${PRG_NAME}.out
#
bteq <<EOBTQ > ${LOG_FILE} 2>$1
.LOGON {TDPID}/{USERID},{PWD};
--.RUN file=${LOGON}
/* Add your SQL/BTEQ commands here */
.QUIT 0;
EOBTQ
Edit
The double hyphen indicates a single line comment. Typically in a UNIX script you do not leave your password in plain text of a KSH script. The .RUN command would reference a text file in a barely sufficient secure location containing the .LOGON {TPDID}/{USERID},{PWD}; command.
The .RUN command in BTEQ allows you to reference another text file containing a series of valid BTEQ commands that you want to run in the current BTEQ script.
Easiest way is to setup the Solaris TTU, is to request root sudo, and run an interactive installation into defaults as a root. That would cure all client issues.
I have looked through the SQL Server questions and answers and I didn't see an answer to this one, if it is out there and I've missed it, please let me know.
Here's the situation:
I write stored procedures and views that are then run as reports (using Crystal) - this is not the problem. Before I am able to release the reports into Production, I need to have the end users run the reports and check them for errors, etc. In a perfect world, I would have a frozen test environment, but I don't live in a perfect world. Every night everything I place into my test environment is wiped out and every morning anything that is in end user testing needs to be re-added. This means that when I come in the first thing I do is run all of the stored procedure, along with a script that unhides the reports in the program we use.
What I'd like to be able to do is to write a package that would find all of the stored procedures in a folder and execute them to add them to the database and, then, run the script that unhides the reports.
I know how to set up an SSIS package to run a stored procedure, but I don't know how to set one up that would run an ever changing list of stored procedures. Is this even possible? And, if it is, how do I go about starting this up?
I should note that while I have more then 10 years of query writing experience, I haven't used VB since VB 6.0 and I very new to the SSIS and SSRS world.
Thanks in advance!
Good old nt shell will do the trick. Run this statement in the folder containing the files.
for %A in (*.sql) DO sqlcmd -i %A -S <myServer> -d <myDb> -E
if you want to include it in a batch file it could look like
#echo off
for %%A in (*.sql) DO sqlcmd -i %%A -S <myServer> -d <myDb> -E
sqlcmd -i script_to_update_config.sql -S <myServer> -d <myDb> -E
This actually sounds like it may be more of a deployment issue than a SQL one. Take a look at Jenkins CI. I believe it's mostly used for code build and deployment, but it can also be used for any automated task.
If you had one SQL file that listed all the changed procs and their associated files you can use that single script to run all the others http://www.devx.com/tips/Tip/15132. For that matter you could just use a scheduled task to run it every morning.
Adding one more step, you could build the file based on the contents of a folder (using a little Powershell script or the like).
I'm not sure trying to do this all within an SSIS package is the right tool for the job.