How can I customize the the FOR command below to loop through the files inside the Database folder following the order of the number prefixed in the file name?
FOR /R ../Database %%f IN (*.sql) DO sqlcmd -S %1 -d %2 -U %4 -P %5 -i "%%~f" >> Logs/%2_DBInstall.log || goto errors
Database folder contains:
001_usp_procedure1.sql
002_ups_procedure2.sql
Thanks very much,
Please try the following
del temp.txt
del temp1.txt
for /F "usebackq " %%F IN (`dir /b /s /o:n "../Database/*.sql"`) DO (
echo %%~nF / %%F >> temp.txt
)
sort temp.text temp1.text
for /F "tokens=1,2 delims=/" %%i IN (temp1.txt) DO sqlcmd -S %1 -d %2 -U %4 -P %5 -i "%%j" >> Logs/%2_DBInstall.log || goto errors
It should work for recursive directories too (process is sorted by global filename without taking account of the path ... did I understand the problem?)
Please note that two temp files are being used. Be careful with the names (or use real temp files)
Related
I have a set of sql files
file1.sql
file2.sql
....
that contain table inserts INSERT INTO <tablename> (...) VALUES (....); in a directory. I would like to add a command into a existing batch file so that it iterates over each file.
Currently I use the command
<postgreSQLCommandWithParameters> -U <user> -d <database> -f <pathToSQL>\complete.sql -q
I looked into a for loop but I missing the parameter to point to the correct directory.
for %%G in (*.sql) do <postgreSQL> -U <user> -d <database> -E -i"%%G"
Therefore how can I use a for loop to iterate over all sql files which contain insert commands?
As a side question: And what is the syntax in the for loop in batch scripts?
So I used the following command
FOR /F "tokens=4 USEBACKQ" %%F IN (`dir <pathToSQLFiles> ^| findstr .sql `) DO (
<postgreSQL> %1 -U <user> -d <database> -f <pathToSQLFiles>\%%F -q
)
And here are some cliffnotes
"tokens=4 USEBACKQ" takes the numbered items to read from each line and also includes names with space in them
^| findstr .sql filters all SQL files
This script processes all the sql files in a given folder and outputs the result to a csv. Do you have any ideas how could I adapt it to retry the sql file in case of error or failure?
#ECHO OFF
SET SQLCMD="C:\Program Files\Microsoft SQL Server\100\Tools\Binn\SQLCMD.EXE"
SET PATH="C:\Users\username\Desktop\Scripts\Reports\sql"
SET SERVER="localhost"
SET DB="database"
SET LOGIN="username"
SET PASSWORD="password"
SET OUTPUT="C:\Users\username\Desktop\Scripts\Reports\output_%date%-%time:~0,2%-%time:~3,2%-%time:~6,2%.csv"
CD %PATH%
ECHO %date% %time% > %OUTPUT%
for %%f in (*.sql) do (
%SQLCMD% -S %SERVER% -d %DB% -E -i %%~f >> %OUTPUT% -W -w 1024 -s";")
Thank you!
You cant test the ERRORLEVEL environment variable. typically a zero ERRORLEVEL value means success. also you need to enable delayed expansion to check it inside a block.
something like this may help,
SetLocal EnableDelayedExpansion
for %%f in (*.sql) do (
Set /a success=1
for /L %%w in (1,1,5) do ( rem retry five times
if !success! NEQ 0 (
%SQLCMD% -S %SERVER% -d %DB% -E -i %%~f >> %OUTPUT% -W -w 1024 -s";"
if !ERRORLEVEL! EQU 0 set /a success=0
)
)
if !success! NEQ 0 (
rem sql failed, log or advise
)
)
EndLocal
another important point, be careful with the PATH environment variable. you should better use another name as SQL_PATH or MY_PATH.
or you can use PUSHD & POPD to change your working dir
#echo off
SET ...
rem save current dir and jump to...
pushd "C:\Users\username\Desktop\Scripts\Reports\sql"
for %%f in (*.sql) do (
...
...
)
rem restore dir saved by pushd
popd
Is it possible to execute all SQL scripts in a folder in alphabetical order using a batch file ?
Currently I uses the following code but it executes the scripts in the order in which they are kept instead of executing it in alphabetical order
SET Database=<<DatabaseName>>
SET ScriptsPath=<<FolderPath>>
SET ServerName=<<ServerName>>
IF EXIST "%ScriptsPath%output_CCF.txt" del "%ScriptsPath%output_CCF.txt"
type NUL > "%ScriptsPath%output_CCF.txt"
FOR /R "%ScriptsPath%" %%G IN (*.sql *.up) DO (
sqlcmd -d %Database% -S %ServerName% -i "%%G" -o "%%G.txt"
echo .................................................
>> "%ScriptsPath%output_CCF.txt"
echo Executing: "%%G" >> "%ScriptsPath%output_CCF.txt"
echo ...................................>> "%ScriptsPath%output_CCF.txt"
copy "%ScriptsPath%output_CCF.txt"+"%%G.txt" "%ScriptsPath%output_CCF.txt"
del "%%G.txt"
)
If you use the /F option of the FOR command, then you can use another command to generate the set of data to be iterated over.
Use this FOR statement and I believe you'll find the result you're looking for.
FOR /F "usebackq" %%G IN (`dir /ON /B *.sql *.up`) DO ...
This uses the dir command to generate the set of files to use. The /ON argument orders the files by name and the /B argument provides a bare format (no heading information or summary) of the file names.
For more details on the arguments check the help for either of the commands from the command line dir /? and for /?
for /f %%i in ('someprogram %1 2>&1 | find /c "some string"') do ...
it says
2>&1 was unexpected at this time
You will need to escape special characters within the for command:
for /f %%i in ('someprogram %1 2^>^&1 ^| find /c "some string"') do ...
cmd's parser is not the most robust one; this is unfortunately necessary.
For the database in my project I have a drop/create script for the database, a script for creating tables and SPs and an Access 2003 .mdb file with some exported values.
To set up the database from scratch I can use my SQL management studio to first run one script, then the other and lastly manually run the sort of tedious import task.
But I would like to do this as automated as possible. Hopefully something like putting the three files in a folder along with a fourth script to execute. Looking something like:
run script "dropcreate.sql"
run script "createtables.sql"
import "values.mdb"
How is this done? I hope to avoid using SSIS and the like. The tricky this is of course the import of data, where I can't seem to find a simple way. It is also important that the files a left as they are and not embedded into anything.
:: DOC AT THE END
#ECHO OFF
::BOOM BOOM BOOM CHANGE THIS ONE WHEN YOU ARE INSTALLAING DIFFERENT DATABASE
SET DbName=CAS_DEV
ECHO CREATE FIRST BACKUP OF ALL DATABASES ON THE DEFAULT INSTANCE ONES:
ECHO CREATING THE LOG FILES
echo THIS IS THE ERROR LOG OF THE UPDATE OF THE %DbName% ON %DATE% >error.log
echo THIS IS THE INSTALL LOG OF THE UPDATE OF THE %DbName% ON %DATE% >install.log
ECHO STARTTING BACKUP
CD .\0.BackUp
ECHO FOR EACH SQL FILE DO RUN IT THIS WILL TAKE A WHILE
ECHO SINCE WE ARE GOING TO MAKE A BACKUP FOR ALL THE DATABASES ON THE CURRENT HOST
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d MASTER -t 30000 -w 80 -u -p 1 -b -i %%i -r1 1>> "..\install.log" 2>> "..\error.log"
ECHO GO ONE FOLDER UP
ECHO SLEEP FOR 1 SECOND
ping -n 1 127.0.0.1 >NUL
ECHO DONE WITH BACKUP GOING UP
cd ..
ECHO THE BACKUPS ARE IN THE FOLDER
ECHO D:\DATA\BACKUPS
ECHO CLICK A KEY TO CONTINUE
ECHO ========================================================================================================================
PAUSE
ECHO STARTING INSTALLING FUNCTIONS
CD ".\1.Functions"
ECHO FOR EACH SQL FILE DO RUN IT
ping -n 1 127.0.0.1 >NUL
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i "%%i" -r1 1>> "..\install.log" 2>> "..\error.log"
ECHO DONE WITH STORED PROCEDDURES GOING UP
cd ..
ping -n 1 127.0.0.1 >NUL
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO START TO EXECUTE THE MIXED FILES
CD .\1.Mixed
ECHO CREATING THE LOG FILES
echo. >>"..\error.log"
echo. >>"..\install.log"
ECHO FOR EACH SQL FILE DO RUN IT
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i %%i -r1 1>> "..\install.log" 2>> "..\error.log"
ECHO GO ONE FOLDER UP
cd ..
ECHO SLEEP FOR 1 SECOND
ping -n 1 127.0.0.1 >NUL
ECHO DONE WITH MIXED GOING UP
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO STARTING INSTALLING TABLES
CD .\2.Tables
ECHO FOR EACH SQL FILE DO RUN IT
ping -n 1 127.0.0.1 >NUL
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i "%%i" -r1 1>> "..\install.log" 2>> "..\error.log"
ping -n 1 127.0.0.1 >NUL
ECHO DONE WITH TAbles GOING UP
cd ..
ping -n 1 127.0.0.1 >NUL
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO STARTING INSTALLING Views
CD ".\3.Views"
ECHO FOR EACH SQL FILE DO RUN IT
ping -n 1 127.0.0.1 >NUL
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i "%%i" -r1 1>> "..\install.log" 2>> "..\error.log"
ECHO DONE WITH Views GOING UP
cd ..
ping -n 1 127.0.0.1 >NUL
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO STARTING INSTALLING stored procedures
CD ".\5.StoredProcedures"
ECHO FOR EACH SQL FILE DO RUN IT
ping -n 1 127.0.0.1 >NUL
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i "%%i" -r1 1>> "..\install.log" 2>> "..\error.log"
ECHO DONE WITH STORED PROCEDDURES GOING UP
cd ..
ping -n 1 127.0.0.1 >NUL
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO STARTING INSTALLING Triggers
CD ".\6.Triggers"
ECHO FOR EACH SQL FILE DO RUN IT
ping -n 1 127.0.0.1 >NUL
for /f %%i in ('dir *.SQL /s /b /o') DO ECHO %DATE% --- %TIME% RUNNING %%i 1>>"..\install.log"&SQLCMD -U ysg -P pass -H hostname -d %DbName% -t 3000 -w 80 -u -p 1 -b -i "%%i" -r1 1>> "..\install.log" 2>> "..\error.log"
ping -n 1 127.0.0.1 >NUL
ECHO DONE WITH Triggers GOING UP
cd ..
ping -n 1 127.0.0.1 >NUL
ECHO HIT A KEY AFTER PAUSE
PAUSE
ECHO Please , Review the log files and sent them back to Advanced Application Support
set mailadd= yordan.georgiev^#oxit.fi
:: WE USE THE "%cd%\bin\bmail.exe".EXE UTILITY TO SEND OURSELF AN E-MAIL CONTAINING THE TEXT FILE
:: ALTERNATIVE SMTP MIGHT BE company.com, UNCOMMENT THE NEXT LINE FOR ALTERN
::cmd /c "%cd%\bin\bmail.exe" -s company.com -m %computername%.txt -t %mailadd% -a %computername% -h
::"%cd%\bin\bmail.exe" -s smtp.company.com -m install.log -t yordan.georgiev#oxit.fi -a "POC 1.2 install log" -h
::"%cd%\bin\bmail.exe" -s smtp.company.com -m error.log -t yordan.georgiev#oxit.fi -a "POC 1.2 error log" -h
cmd /c start /max INSTALL.LOG
CMD /C start /MAX ERROR.LOG
echo DONE !!!
ECHO HIT A KEY TO EXIT
PAUSE
:: WE GO TROUGH ALL THE FOLDERS AND RUN THE SQL FILES IN ALPHABETIC ORDER
You can run SQL Server Management Studio in SQLCMD mode. In there you can run scripts as follows
:r c:\temp\DropCreate.SQL
:r c:\temp\CreateTables.SQL
Alternately, you can run the whole thing from a batch file using SQLCMD.exe commands.
SQLCMD -S "." -E -i "c:\temp\DropCreate.SQL"
SQLCMD -S "." -E -i "c:\temp\CreateTables.SQL"
Do you have an alternative to SSIS that can import the data for you? Usually to do any kind of transformations and loading you need error handling, lookups, etc that you will have to code yourself unless you use an off-the-shelf product.
You can read a lot about SSIS right here on SO.
We have a similar project (create db, load data, create code). We do all of this inside a Database Project - with Visual Studio Team System Edition 2008 and GDR2.