NetApp SnapManager for SQL Server Update backup script - sql

I was very disappointed that NetApp just doesn't add Feature to automatically update the (MS)SQL Job automatically every time a new Database gets created. (for a Full backup)
So i now want to share with you what I did to automatically add a new Database to out full SQL backup job.
Maybe someone has some ideas on how to make things a little bit more efficient or create not so much temporary files :)
Just Create a SQL job with executes the flowing batch file to update the NetApp SnapManager SQL Job.
#echo off
set SQLinstance=YOUR_SQL_SERVERNAME
REM if something differs
set SQL_SERVER=%SQLinstance%
REM SQL Verify Instalce
set SQL_VERIFY=ANOTHER_SERVERNAME\VERIFY
set db_list=list_%SQL_SERVER%.txt
set db_list_wo_summary=list_new_%SQL_SERVER%.txt
set linecount_file=linecount_%SQL_SERVER%.txt
set output-Full-User-file=output-Full-%SQL_SERVER%.txt
set output-Inc-User-file=output-Inc-%SQL_SERVER%.txt
set SQLCMD="C:\Program Files\Microsoft SQL Server\110\Tools\Binn\sqlcmd.exe"
set arg_start=C:\Program Files\NetApp\SnapManager for SQL Server\SmsqlJobLauncher.exe new-backup -svr ''%SQLinstance%'' -d ''%SQLinstance%'',
set arg_end_full= -ver -verInst ''%SQL_VERIFY%'' -mp -mpdir ''C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint'' -RetainBackupDays 8 -RetainShareBackupDays 17 -cpylgbkshare COPYLOG_TOSHARE -lb -bksif -RetainSnapofSnapInfoDays 17 -rudays 1 -gen -mgmt daily
set arg_end_incremental= -ver -verInst ''%SQL_VERIFY%'' -mp -mpdir ''C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint'' -RetainBackupDays 8 -RetainShareBackupDays 17 -cpylgbkshare COPYLOG_TOSHARE -lgbkonly -bksif -RetainSnapofSnapInfoDays 17 -gen -mgmt daily
set SQL_QUERIE="SELECT NAME FROM master.sys.databases AS dtb WHERE NOT (CAST(case when dtb.name in ('master','model','msdb','tempdb') then 1 else dtb.is_distributor end AS bit)=1) ORDER BY NAME"
set JOBNAME_FULL_BACKUP=jobname_full
set JOBNAME_INCREMENTAL_BACKUP=jobname_incremental
set SQL_UPDATE_COMMAND_START="UPDATE msdb.dbo.sysjobsteps SET command='"
set SQL_UPDATE_COMMAND_END=' FROM msdb.dbo.sysjobs, msdb.dbo.sysjobsteps WHERE msdb.dbo.sysjobs.job_id = msdb.dbo.sysjobsteps.job_id AND name = '%JOBNAME_FULL_BACKUP%'"
set SQL_UPDATE_COMMAND_END_INC=' FROM msdb.dbo.sysjobs, msdb.dbo.sysjobsteps WHERE msdb.dbo.sysjobs.job_id = msdb.dbo.sysjobsteps.job_id AND name = '%JOBNAME_INCREMENTAL_BACKUP%'"
REM Delete Old files (just to be sure)
del %db_list_wo_summary% %db_list% %linecount_file%
REM Get a List af all Databases and write them into a File
%SQLCMD% -E -S %SQLinstance% -o %db_list% -h -1 -s "," -W -Q %SQL_QUERIE%
setLocal EnableDelayedExpansion
REM Remove the last 2 Lines (one is a summary line, ant the other is empty)
set count=
for /f %%x in ('type %db_list% ^| find /c /v ""') do set /a lines=%%x-2
copy /y nul %tmp%\tmp.zzz > nul
for /f "tokens=*" %%x in ('type %db_list% ^| find /v ""') do (
set /a count=count+1
if !count! leq %lines% echo %%x>>%tmp%\tmp.zzz
)
move /y %tmp%\tmp.zzz %db_list_wo_summary% > nul
REM removes all linebreaks and adds a ' on every star and ending of the string (and adds a comma)
for /f "tokens=* delims= " %%a in (%db_list_wo_summary%) do (set s=!s!%%a'', '')
REM removes the last four chars (, '')(comma, space, and 2 '')
set s=!s:~0,-4!
REM counts the lines of the file NOT containing the string XXXYYYZZZ (basicaly a line count)
type %db_list_wo_summary% | find /V /C "XXXYYYZZZ" > %linecount_file%
for /f "tokens=* delims= " %%c in (%linecount_file%) do (set cn=%%c)
REM Creates the final string for the SQL job
set s1=%arg_start% ''%CN%'', ''!s!%arg_end_full%
set s2=%arg_start% ''%CN%'', ''!s!%arg_end_incremental%
REM Update the SQL Job step command with the new string via SQLCMD
%SQLCMD% -E -S %SQLinstance% -Q %SQL_UPDATE_COMMAND_START%%s1%%SQL_UPDATE_COMMAND_END%
%SQLCMD% -E -S %SQLinstance% -Q %SQL_UPDATE_COMMAND_START%%s2%%SQL_UPDATE_COMMAND_END_INC%
REM deleting the temp file
del %db_list_wo_summary% %db_list% %linecount_file%
I hope this can help anyone make things a bit more efficient.

The answer is above :)
Hopefully someone will find that user full, or at least inspiring ;)

Don't specify a database to backup at all ... then SMSQL will backup all databases, also the new ones

Related

batch script for file system: use of for-loop variable in another nested for-loop

the last 4-5 hours I spent trying to get my script fixed, and tried several things, but it still doesn't work as supposed. Here is the working part:
#echo off
rem automatically sort MP3s into an existing structure or add folders if needed
rem example of MP3-file pattern: Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3
SETLOCAL
rem set variables
SET "source=g:\music\folder"
SET "str=Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3"
SET "test=%source%\%str%"
rem split the filename from the remaining path. Take the left part
rem and assign it to %band% while the right part gets assigned to %song%
FOR /f "delims=" %%i IN ("%test%") DO SET "result=%%~nxi"
SET "band=%result: - =" & SET "song=%"
rem If there is no such folder (%band%), create it
IF not exist "%source%\%band%" MD "%source%\%band%"
rem As soon as there definetely is a fitting folder
rem move the file over there.
MOVE /-Y "%source%\%result%" "%source%\%band%"
The next step would be to enhance the code by a for-loop so I don't have to edit the script for every single one of my 3k+ files ;)
I tried really hard to get this code going, but failed:
#echo off
SETLOCAL
SET "source=g:\music\folder"
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
SET "band=!result: - =" & SET "song=%"
IF not exist "%source%\%band%" MD "%source%\%band%"
MOVE /-Y "%source%\%result%" "%source%\%band%"
)
pause
So I added a for-loop and %%f was correctly filled at first and wasn't in the next for-loop.
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
That resulted in: "g:\music\folder\Wu-Tang Clan - Gravel Pit (LP Version Clean).mp3"
Like it was supposed to. But after that
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
result and every following variable was empty, always.
I tried to fix it with a second variable as a 'helper':
FOR %%f IN (%source%\*.mp3) DO (
SET "helper=%%f"
FOR /f "delims=" %%i IN ("%helper%") DO SET "result=%%~nxi"
Even added 'enabledelayedexpansion' after I read about that and went for
FOR /f "delims=" %%i IN ("!helper!") DO SET "result=%%~nxi"
still doesn't work.
Now I could really need some help and would really appreciate it :)
regards Phoenix
Next code snippet could work. Note that the SET "band=%result: - =" & SET "song=%" trick can't be performed using ! delayed expansion unlike %-expansion. Therefore, that command is moved to the :myset subroutine and executed via call command.
#echo off
SETLOCAL EnableExtensions EnableDelayedExpansion
SET "source=g:\music\folder"
FOR %%f IN (%source%\*.mp3) DO (
echo %%f
FOR /f "delims=" %%i IN ("%%f") DO SET "result=%%~nxi"
call :myset
IF not exist "%source%\!band!" MD "%source%\!band!"
MOVE /-Y "%source%\!result!" "%source%\!band!"
)
goto :skipMyset
:myset
SET "band=%result: - =" & SET "song=%"
goto :eof
:skipMyset
pause
Resources (required reading):
(command reference) An A-Z Index of the Windows CMD command line
(additional particularities) Windows CMD Shell Command Line Syntax
(special page) EnableDelayedExpansion

Pass a parameter from one batch file to another batch file

I have two batch files.
First extracts date from and creates folder in \YEAR\MONTH\DATE format.
#echo off
setlocal enabledelayedexpansion
:: Extract date fields
for /f "tokens=1-4 delims=/-. " %%i in ('date /t') do (
set v1=%%i& set v2=%%j& set v3=%%k
if "%%i:~0,1%%" gtr "9" (set v1=%%j& set v2=%%k& set v3=%%l)
for /f "skip=1 tokens=2-4 delims=(-)" %%m in ('echo.^|date') do (
set %%m=!v1!& set %%n=!v2!& set %%o=!v3! ))</br>
:: Final set for language independency
set year=%yy%%aa%
set month=%mm%
set day=%dd%
:: Make Dir
set root=f:\
::Create folder of today's date
if exist %root% goto L2
goto L3
:L2
if not exist %root%\%year% md %root%\%year%
:L3
if exist %root%\%year%\%month% goto L5
:L4
if not exist %root%\%year%\%month% md %root%\%year%\%month%
:L5
md %root%\%year%\%month%\%day%
:: Detete folder older than '3' days
forfiles /p "%root%%year%\%month%" /s /d -3 /c "cmd /c IF #isdir == TRUE rd /S /Q #path"
echo.
pause
creates a backup of database
echo off
cls
echo -- BACKUP DATABASE --
::set db name
set DATABASENAME='db_name'
:: set path and format
set BACKUPFILENAME='path\%DATABASENAME%.bak'
:: set server name
set SERVERNAME='server name'
echo.
::backup execution
sqlcmd -S %SERVERNAME% -Q "BACKUP DATABASE [%DATABASENAME%] TO DISK = N'%BACKUPFILENAME%' WITH INIT , NOUNLOAD , NAME = N'%DATABASENAME% backup', NOSKIP , STATS = 10, NOFORMAT"
echo.
Now I want to call the second batch file from first and also pass the parameters like root, year, month, date from first to second.
I've tried both codes individually and its working perfectly. How can I pass the parameters. Please help
If you call second batch from first one, something like
call backupDP.cmd
both files share the same environment, so, both see the same variables. No need to pass anything, the second batch already has all the data.

robocopy script help - different sources and detinations

I need a robocopy script that can run to replicate data from a source to a destination. the problem I have is I need to replicate certain folders from the source from different drives over to a destination server which will need to go to different drives also.
I'm using this script to copy a single folder over to the destination server (which this is running on)
#echo off
SET Day=%date:~6,4%.%date:~3,2%.%date:~0,2%
SET _source=\\10.10.10.18\R$\User\customer1
SET _target=E:\customer1
SET _log=E:\test_%Day%_Replication_Log.txt
SET _what=/MIR /COPYALL /B
SET _options=/NS /NC /NFL /NDL /R:0 /W:0 /RH:0615-1730 /MON:900 /MOT:30 /LOG+:"%_log%"
robocopy %_source% %_target% %_what% %_options%
I have a number of folders on the R drive as well as other drives that I wish to copy like this but I ideally dont want separate jobs for each and would like to run it under one batch file if possible.
Is there a way of having the source location specified in a text document and having the script read this and pass it to the command same would I also be able to specify the destination for this data i.e.:
customer1 - e drive
customer2 - f drive
etc
I would like this as easy as possible so that folders can be added / removed from the text file so the commands don't need to be altered.
Is this possible?
I think you are going to need to do something like the following..
SET _source=\\10.10.10.18\R$\User\customer1
rem SET _source=C:\windows
SET _target=E:\customer1
SET _log=E:\test_%Day%_Replication_Log.txt
SET _what=/MIR /COPYALL /B
SET _options=/NS /NC /NFL /NDL /R:0 /W:0 /RH:0615-1730 /MON:900 /MOT:30 /LOG+:"%_log%"
for /f "tokens=1*delims=-" %%a in ('dir /B/ad/s %_source%\drive*') do (
robocopy %%a %_target% %_what% %_options%
)
pause
Now, what I suspect you are going to need to do is something like the following..
Notice the oddball: "%~p0" - if you go to CMD (command line) and type in FOR /? - it will explain what these mean.. It like two or three screens in on the help..
SET _source=\\10.10.10.18\R$\User\customer1
rem SET _source=C:\windows
SET _target=E:\customer1
SET _log=E:\test_%Day%_Replication_Log.txt
SET _what=/MIR /COPYALL /B
SET _options=/NS /NC /NFL /NDL /R:0 /W:0 /RH:0615-1730 /MON:900 /MOT:30 /LOG+:"%_log%"
for /f "tokens=1*delims=-" %%a in ('dir /B/ad/s %_source%\drive*') do (
cd "%~p0%_source%\drive"
copy *.txt "%~p0 %_source%\drive"
FOR /F "delims=" %%A IN ('type %~p0ALIST.txt') do (
rem cd "%~p0%%A\%_target%"
rem copy *.txt "%~p0%%A\%_target%"
robocopy %%a %_target% %_what% %_options%
rem cd "%~p0%%A\drive"
)
pause

save the result of a operation into a variable within a .BAT file

I have the following BAT file working ok:
it connects to an specific sql server and select getdate()
what I really wanted is to test whether the server is connectable.
I would like something like:
set is_connectable = call sqlcmd.exe %%SERVITORE%%
is there any way I could achieve this?
thanks and regards
marcelo
#echo off
color fc
cls
echo.
#ECHO --- BEGIN THE SCRIPT 1 ----
echo.
#echo the SERVITORE is "%1"
echo.
echo.
if "%1"=="" GOTO USAGE
set SERVITORE=-Stcp:%1% -Q " USE MASTER select getdate() "
call sqlcmd.exe %%SERVITORE%%
color 6
GOTO THE_END
:USAGE
echo.
echo USAGE:
echo.
ECHO the first parameter is the SERVITORE server.
echo example 1 SERVITORE108
echo.
ECHO the second parameter is optional
echo but if not supplied the default is \\SERVITORE\folder2$
echo.
echo ATB
echo.
:THE_END
color 8
It is easy to run a query and set the result to a DOS environment variable. For instance, you can do the following to get the date/time from the SQL Server instance (SQL Server is running locally in my instance):
for /f "skip=2 delims=" %%i in ('sqlcmd -S localhost -E -Q "set nocount on; select getdate() as [Now]"') do set is_connectable=%%i
However, the is_connectable environment variable set in this example will have an arbitrary value, which will make it hard to evaluate. Since you are just trying to verify that the SQL Server is there, alive, and responsive, you should run a query that creates a more predictable output, like this:
#echo off
:: Make sure the variable is undefined to start with
set is_connectable=
:: Make the connection and run a query that should always return '1'
for /f "skip=2 delims= " %%i in ('sqlcmd -S localhost -E -Q "set nocount on; select 1 as [Rows] into #temp; select ##rowcount as [Rows]; drop table #temp"') do set is_connectable=%%i
:: Verify if SQL Server is avaialble
if not defined is_connectable goto NotFound
if "%is_connectable%"=="1" goto Found
goto UnknownError
:Found
echo SQL Server was found and returned '1' as expected...
goto TheEnd
:NotFound
echo SQL Server was not found...
goto TheEnd
:UnknownError
echo SQLServer was found, but the return value was '%is_connectable%' instead of '1'...
goto TheEnd
:TheEnd

How to set a local variable to the result of sql query in a batch file?

Is there any way to retreive the result set of a sql query in a local variable; the query is to be run in a batch file. What I am trying to do is something like this:
set varname = osql -S%dstserver% -d%dstDB% -Q"SELECT name from table_name where Id = %siteId%" %osqluser% -b
varname is my local variable.
I am quite new in sql so any help would be greatly appreciated1
Write the result to the file and then read the file. In your case you need to read the first line (and possibly trim it).
Add following parameters to your query:
osql -S%dstserver% -d%dstDB% -Q"SET NOCOUNT ON;SELECT name from table_name where Id = %siteId%" %osqluser% -b -w 9999 -h-1 -o tempres.txt
-o ...: output file (which you need to read later)
-h-1: disable header
-w 9999: to ensure that it handles correctly cases when your name is longer then default 80 characters
SET NOCOUNT ON; before real query to disable the status string like (1 row affected)
You can use for /f to iterate over the output of a command:
for /f "usebackq delims=" %%x in (`your command`) do ...
You don't need a temporary file at all; it doesn't gain you anything here (except having to think about where you may have write permissions and remember to delete the file afterwards).
for iterates line-wise over the output, tokenizing as it goes. That's why there is a delims= at the end which effectively disables any tokenizing so you don't get your output split at spaces. There are other options, such as skip=n which will skip n lines before starting processing which you can use to ignore a header or so.
Inside that loop you can then do the following:
for /f "usebackq delims=" %%x in (`your command`) do set VAR=%%x
Be very careful what you do afterwards with that variable, though, as it may contain characters the shell treats as special, such as >, <, &, etc. You may create yourself here some sort of Batch Injection vulnerability when doing something like the following:
echo %VAR%
and someone decides to put the following string in his name:
foo & rd /s q \
If you know that only a single line with usable content returns and the rest is useless junk then you can break the loop prematurely:
for /f "usebackq delims=" %%x in (`your command`) do set VAR=%%x&goto break
:break
Use set /p:
osql -S%dstserver% -d%dstDB% -Q"SET NOCOUNT ON;SELECT name from table_name where Id = %siteId%" %osqluser% -b -w 9999 -h-1 -o tempres.txt
set /p varname=<tempres.txt
(Borrowing osql parameters from van's answer)
You can write the result to a file containing set statements.
#echo off
osql -E -S servername -h-1 -Q "set nocount on; select 'set var=42'" > c:\set.bat
call c:\set.bat
echo %VAR%
This results in 42 being written to the screen. To select a name from a table, use a SQL statement like:
select 'set var=' + name from table_name where Id = %siteId%