Escaping special symbols in Sqlcmd - sql

I have a bat file which executes bunch of sql scripts when i install my application. In installation dialogs user sets dbname, login and password which are then used in this bat file. Here it is:
SET _server="(local)"
SET _db=dbname
SET _user=sa
SET _pswr="!#$%^&*()<>"
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_regsql.exe -S %_server% -d %_db% -U %_user% -P %_pswr% -A all
"%systemdrive%\Program Files\Microsoft SQL Server\100\Tools\Binn\SQLCMD.exe" -S %_server% -d %_db% -U %_user% -P %_pswr% -i .\createDB.sql
pause
Everything works great until password doesn't contain any special symbols. But if it does i get following error "Login failed for user 'sa'.". So it's obvious, that somethins wrong with password. And now the question: How can i escape these special symbols in bat file?
PS:
I tried replacing:
< - <
> - >
& - &
But it doesn't help.
Thanks!

You need to escape the percent (%) character and the caret(^) character by doubling them. Here is a list of problematic command line characters
SET _pswr="!#$%%^^&*()<>"

Tested as well with windows batch file on Windows 2008 Server SP2 with sqlcmd. Only needed to escape the % sign as %%. No need to escape the ^.

Related

Errors ("invalid command") when opening a .sql file

I am trying to open a random .sql file off the internet using the following command:
psql -h localhost -d database_name -U postgres < file_name.sql
But when I run this command I just get errors like the following:
invalid command 's
invalid command 's
invalid command 'll
invalid command 'Moving
invalid command 's
invalid command "frequently
It just continuously prints out these invalid command error messages. I thought it might be an encoding problem but I confirmed the file is UTF-8 encoded.
Any suggestions on how I can open this file
To expand and clarify on a_horse_with_no_name's comment - the psql command you are running should be run directly in your shell, not inside pgadmin4.
youruser#yourmachine:~$ psql -h localhost -d database_name -U postgres < file_name.sql
That command should load the contents of file_name.sql in to database_name. Once it's complete, you can use pgadmin4 as normal to interact with the database.
One possibility is that the file contains tabulator keys, which are expanded if you read redirect standard input to the SQL script.
Try using the -f option:
psql -h localhost -d database_name -U postgres -f file_name.sql
Apparently the .sql file was generated through a MySQL dump. I thought it would not matter whether I used PostgreSQL or MySQL but it did. Once I installed MySQL my problem got resolved and I now have a Database ready :)

Running sqlcmd in batch file works but running the same batch file as a scheduled task works and does nothing

I have looked at many SO questions/answers and though some seem similar to my issue they do not seem to be. The answers given fix issues the questions were asking about but will not solve my issue.
I have a batch file...
#ECHO ON
ECHO Disabling the following... >> C:\App\Debug.log
ECHO - V1 >> C:\Apps\Debug.log
FOR /F "tokens=* USEBACKQ" %%F IN (`sqlcmd -j -S DOMAIN\SQLSERVER -U username -P password -d DBNAME -Q "UPDATE [DBNAME].[dbo].[table1] SET ColOne='V1_OFF' WHERE ColOne='V1'"`) DO (
Echo %%F >> C:\Apps\Debug.log
)
EXIT /B
When I run this file at the command prompt it works perfectly fine. When I run it as a scheduled task it show me the echos but nothing for the for loop as expected.
Yes I have made sure the username (using whoami) is the same for the scheduled task set up as the manual run that I do.
Yes I know the user running the script has rights to everything (file access as well as DB access) because it works fine running it from the command prompt.
Scheduled task is set to run wither user is logged on or not.
Any ideas what might be wrong or what I can try for debugging purposes?
Thanks!
sqlcmd is perhaps not enough. cmd.exe in environment of scheduled task may fail to find the executable using local PATHEXT and local PATH environment variables. The executable should be specified with full qualified file name, i.e. drive + path + name + extension. Then the batch file does not anymore depend on the environment variables PATH and PATHEXT because of all files are referenced with full qualified file name.
for executes the specified command line with starting in background one more command process with %ComSpec% /c and the specified command line appended. This means executed is following with Windows installed on drive C::
C:\Windows\System32\cmd.exe /c sqlcmd -j -S DOMAIN\SQLSERVER -U username -P password -d DBNAME -Q "UPDATE [DBNAME].[dbo].[table1] SET ColOne='V1_OFF' WHERE ColOne='V1'"
for captures everything written to handle STDOUT of started command process. The lines of captured output are processed line by line by for after started cmd.exe terminated itself. Error messages output by started cmd.exe or the commands/executables executed by Windows command processor in background to handle STDERR are redirected to handle STDERR of command process processing the batch file and printed to console. But there is no console window on running a batch file as scheduled task. So error messages cannot be seen in this case.
The for command line can be modified easily here to get also error messages written into the C:\Apps\Debug.log.
FOR /F "tokens=* USEBACKQ" %%F IN (`sqlcmd -j -S DOMAIN\SQLSERVER -U username -P password -d DBNAME -Q "UPDATE [DBNAME].[dbo].[table1] SET ColOne='V1_OFF' WHERE ColOne='V1' 2^>^&1"`) DO (
The Microsoft article Using command redirection operators explains 2>&1. The two operators > and & must be escaped with ^ to be interpreted as literal characters on Windows command processor parsing the for command line before executing finally for which executes next %ComSpec% /c with the specified command line on which 2^>^&1 is changed already to 2>&1.
Does the log file C:\App\Debug.log contain with this modification following two lines?
'sqlcmd' is not recognized as an internal or external command,
operable program or batch file.
Yes, then no executable with file name sqlcmd is found by started cmd.exe. The best solution is referencing this executable with full qualified file name. See also: What is the reason for "X is not recognized as an internal or external command, operable program or batch file"?
Otherwise sqlcmd outputs perhaps an error message which should be now also in the log file C:\App\Debug.log.
It would be also possible to use following command line to let background cmd.exe write the error messages into a separate error log file C:\App\Error.log:
FOR /F "tokens=* USEBACKQ" %%F IN (`sqlcmd -j -S DOMAIN\SQLSERVER -U username -P password -d DBNAME -Q "UPDATE [DBNAME].[dbo].[table1] SET ColOne='V1_OFF' WHERE ColOne='V1'" 2^>C:\App\Error.log`) DO (
"tokens=* usebackq" results in first deleting all leading horizontal tabs and normal spaces on non-empty lines by for, then checking if the remaining line starts with ; in which case the line is also ignored and finally assigning the captured line not starting with ; and with leading tabs/spaces removed to loop variable F for further processing.
Better would be using the options usebackq^ delims^=^ eol^= not enclosed in double quotes which requires escaping the two spaces and the two equal signs with caret character ^ to be interpreted as literal characters by cmd.exe on parsing the command line before executing for. The line splitting behavior is disabled completed with delims= because of the definition of an empty list of delimiters. And no line except an empty line is ignored anymore because of end of line character modified from default ; to no character.
Finally a space on an echo line left to redirection operator >> is also output by echo and for that reason written as trailing space into the log file. Therefore no space should be used left to > or >> on printing a line with echo redirected into a file. But care must be taken on omitting the space character left to the redirection operator. The word left to redirection operator should not be 1, 2, ..., 9 as this would result in redirecting the output to these numbered handles into the specified file instead of the character 1, 2, etc. So if unknown text should be written into a file, it is better to specify first the redirection operator > or >> and the full qualified file name and next the echo command with the text to output. See also: Why does ECHO command print some extra trailing space into the file?
The three command lines with echo would be for this batch file:
ECHO Disabling the following...>> C:\App\Debug.log
ECHO - V1>> C:\Apps\Debug.log
>>C:\Apps\Debug.log ECHO %%F
following... is safe for being correct written into the file as also V1. %%F could be just 1 or a string ending with a space and a single digit and so it is better to specify the redirection first on the last echo command line to get finally executed by cmd.exe the command line ECHO %%F 1>>C:\Apps\Debug.log.

ssh to a server and create a directory based off a variable - all in one line

so i have a simple script that lists the folder and file structure of the current directory and spits it out to a file in the current users home directory, then rsyncs that file to a remote server into a specific folder.
the first part of the script SSH's into the remote server and creates a unique folder that the later part of the script transfers the file into.
#ssh -p 12345 sftp.domain.com ' bash -c "mkdir incoming/[foldername]" '
my question is, how can i pass a variable to this? i would usually put this in the script, and then run the script like this "copy.sh $1":
#ssh -p 12345 sftp.domain.com ' bash -c "mkdir incoming/folder-$1" '
however it doesn't work like i might hope it would. all i end up with is a folder on the remote server named "folder-" as it presumably doesn't pass the variable along with the rest when it ssh's in.
is there a better way to make this work?
the rest of the script would also reference the variable $1 to actually copy the file into the folder created on the remote server earlier in the script.
If I understand the problem correctly, the parameter you are trying to reference is set on the local client side (the command line from where you initiate the ssh connection), but you want to reference it in the command line that is to run on the remote server side. This really has nothing to do with ssh and everything to do with shell parameter/variable expansion on the local client side.
The problem is with your usage of single quotes vs. double quotes. Most Unix command shells, including bash which is likely the shell you are running on the local client side, perform environment variable expansion inside of double quotes but not inside of single quotes. So in your command line you should be able to accomplish your goal by changing the single quotes to double quotes and then escaping the embedded double quote characters like this:
#ssh -p 12345 sftp.domain.com " bash -c \"mkdir incoming/folder-$1\" "
Here is a similar example that shows this in action:
$ export EXAMPLE=abc
$ ssh localhost ' bash -c "echo $EXAMPLE def" '
def
$ ssh localhost " bash -c \"echo $EXAMPLE def\" "
abc def

Cannot execute script: Insufficient memory to continue the execution of the program

I have a 123MB sql file which I need to execute in my local PC. But I am getting
Cannot execute script: Insufficient memory to continue the execution of the program
How to solve this issue?
use the command-line tool SQLCMD which is much leaner on memory. It is as simple as:
SQLCMD -d <database-name> -i filename.sql
You need valid credentials to access your SQL Server instance or even to access a database
Taken from here.
It might help you! Please see below steps.
sqlcmd -S server-name -d database-name -i script.sql
Open cmd.exe as Administrator.
Create Documents directory.
Put your SQL Script file(script.sql) in the documents folder.
Type query with sqlcmd, server-name, database-name and script-file-name as like above highlighted query or below command line screen.
For Windows Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
Note: If there is any space in the sql file path then use " (Quotation marks) "
For SQL Server Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -U sa -P sasa -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
-S TestSQLServer\SQLEXPRESS: Here specify SQL Server Name
-U sa: Username (in case of SQL Server Authentication)
-P sasa: Password (in case of SQL Server Authentication)
-d AdventureWorks2018: Database Name come here
-i "d:\document\sql document\script.sql": File Path of SQLFile
You can also simply increase the Minimum memory per query value in server properties. To edit this setting, right click on server name and select Properties > Memory tab.
I encountered this error trying to execute a 30MB SQL script in SSMS 2012. After increasing the value from 1024MB to 2048MB I was able to run the script.
(This is the same answer I provided here)
My database was larger than 500mb, I then used the following
C:\Windows>sqlcmd -S SERVERNAME -U USERNAME -P PASSWORD -d DATABASE -i C:\FILE.sql
It loaded everything including SP's
*NB: Run the cmd as Administrator
If I understand your problem correctly, you are trying to restore (transact sql) xyz.sql - database + schema. You can try this command which worked for me:
SQLCMD -U sa -i xyz.sql
Try this step,
1)Open PowerShell
2)Write this command:
sqlcmd -S PCNAME\SQLEXPRESS -U user -P password -d databanse_name -i C:\script.sql
3)Press Return
:-)
Below script works perfectly:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Symptoms:
When executing a recovery script with sqlcmd utility, the ‘Sqlcmd: Error: Syntax error at line XYZ near command ‘X’ in file ‘file_name.sql’.’ error is encountered.
Cause:
This is a sqlcmd utility limitation. If the SQL script contains dollar sign ($) in any form, the utility is unable to properly execute the script, since it is substituting all variables automatically by default.
Resolution:
In order to execute script that has a dollar ($) sign in any form, it is necessary to add “-x” parameter to the command line.
e.g.
Original:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql
Fixed:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Sometimes, due to the heavy size of the script and data, we encounter this type of error. Server needs sufficient memory to execute and give the result. We can simply increase the memory size, per query.
You just need to go to the sql server properties > Memory tab (left side)> Now set the maximum memory limit you want to add.
Also, there is an option at the top, "Results to text", which consume less memory as compare to option "Results to grid", we can also go for Result to Text for less memory execution.
sqlcmd -S mamxxxxxmu\sqlserverr -U sa -P x1123 -d QLDB -i D:\qldbscript.sql
Open command prompt in run as administrator
enter above command
"mamxxxxxmu" is computer name
"sqlserverr" is server name
"sa" is username of server
"x1123" is password of server
"QLDB" is database name
"D:\qldbscript.sql" is sql script file to execute in database
If you need to connect to LocalDB during development, you can use:
sqlcmd -S "(localdb)\MSSQLLocalDB" -d dbname -i file.sql
As in most answers given here use the command-line tool. In my case the script already has database creation code. If your script contains CREATE DATABASE command, for example
USE [master]
GO
CREATE DATABASE [your-database-name]
Then do not use the -d your-database-name, instead use the following command.
For Windows Authentication use the command
sqlcmd -S ServerName\InstanceName -i "script.sql" -x
For SQL Server Authentication use the command
sqlcmd -S ServerName\InstanceName -U usename -P password -i "script.sql" -x

Importing .sql file on windows to postgresql

I have a .sql file that was created by postgresql a while back. I now want to import this file onto a windows machine running postgresql.
How do I do this. The file is about 1.5gb.
You should use psql command line tool:
psql -h hostname -p port_number -U username -f your_file.sql databasename
click on the SQL Shell and log into the database and use import
Server [localhost]:
Database [postgres]:
Port [5432]:
Username [postgres]:
Password for user postgres:
psql (9.2.4)
WARNING: Console code page (437) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
Type "help" for help.
postgres=# \i c:/data/data01.sql
start you psql command tool, it will give you dialog like the following
Server [localhost]:
Database [postgres]:
Port [5432]:yourport
Username [postgres]:
Password for user postgres:**********
then connect to your database
postgres=# \c yourdatabase;
then import the file
yourdatabase=# \i c:/path/path/data/data01.sql
note the / for directory separator & no spaces in file path
This also works for me:
psql dbname username < file.sql
command prompt
open your cmd window and type the following (make sure the path of postgres is correct)
."C:\Program Files\PostgreSQL\9.4\bin\psql.exe" -h 127.0.0.1 -p 5432 -U postgres -d dbname <./query.sql
psql -U <dbusername>
if the prompt makes you enter password, do that.
\c <yourdatabasename>
\i 'thepathusing/delimiter.sql'
Two points you need to watch out that
Use / as writing path of the file instead of \.
Use single quote
symbol ' instead of ".
If you're doing it with a URI connection string make sure the arguments are before the URI, Powershell examples:
Works on windows:
.\psql -f TestFile.sql $connString
.\psql -c 'SELECT Version();' $connString
Won't work on windows (URI connection before arguments):
.\psql $connString -c 'SELECT Version();'