How can I import a SQL Server RDS backup into a SQL Server Linux Docker instance? - sql

I've followed the directions from the AWS documentation on importing / exporting a database from RDS using their stored procedures.
The command was similar to:
exec msdb.dbo.rds_backup_database
#source_db_name='MyDatabase',
#s3_arn_to_backup_to='my-bucket/myBackup.bak'
This part works fine, and I've done it plenty of times in the past.
However what I want to achieve now; is restoring this database to a local SQL Server instance; however I'm struggling at this point. I'm assuming this isn't a "normal" SQL Server dump - but I'm unsure what the difference is.
I've spun up a new SQL Server for Linux Docker instance; which seems all set. I have made a few changes so that the sqlcmd tool is installed; so technically the image I'm running is comprised of this Dockerfile; not much different.
FROM microsoft/mssql-server-linux:2017-latest
RUN apt-get update && \
apt-get install -y curl && \
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
apt-get update && \
apt-get install -y mssql-tools unixodbc-dev
This image works fine; I'm building it via docker build -t sql . and running it via docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=myPassword1!' -p 1433:1433 -v $(pwd):/backups sql
Within my local folder, I have my backup from RDS downloaded, so this file is now in /backups/myBackup.bak
I now try to run sqlcmd to import the data with the following command; and I'm running into an issue which makes me assume this isn't a traditional SQL dump. Unsure what a traditional SQL dump looks like, but the majority of the file looks garbled with ^#^#^#^# and of course other things.
/opt/mssql-tools/bin/sqlcmd -S localhost -i /backups/myBackup.bak -U sa -P myPassword1! -x
And finally; I get this error:
Sqlcmd: Error: Syntax error at line 56048 near command 'GO' in file '/backups/myBackup.bak'.
Final Answer
My final solution for this mainly came from using -Q and running a RESTORE query rather than importing with the file, but I also needed to include some MOVE options as they were pointing at Windows file paths.
/opt/mssql-tools/bin/sqlcmd -U SA -P myPassword -Q "RESTORE DATABASE MyDatabase FROM DISK = N'/path/to/my/file.bak' WITH MOVE 'mydatabase' TO '/var/opt/mssql/mydatabase.mdf', MOVE 'mydatabase_log' TO '/var/opt/mssql/mydatabase.ldf', REPLACE"

You should use the RESTORE DATABASE command to interact with your backup file instead of specifying it as an input file of commands to the database:
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P myPassword1! -Q "RESTORE DATABASE MyDatabase FROM DISK='/backups/myBackup.bak'"
According to the sqlcmd Docs, the -i flag you used specifies:
The file that contains a batch of SQL statements or stored procedures.
That flag likely won't work properly if given a database backup file as an argument.

Related

Restore Database from sql file docker

I am trying to restore a database with a sql file from a dockerized postgresql using this command:
cat pathfile.sql | docker exec -i dbcontainer psql -U user
But when I run this command the console doesn't do anything, it doesn't even throw an error.
I have verified the database and there is nothing created

Restoring database through docker

Currently learning SQL online. I've been trying to restore the database from this link:
http://app.sixweeksql.com:2000/SqlCourse.bak
when I run SQL Server through Docker (Mac user, can't run SSMS unfortunately). I've been following directions from Microsoft here:
https://learn.microsoft.com/en-us/sql/linux/tutorial-restore-backup-in-sql-server-container?view=sql-server-2017
I moved the file into my container and checked the files listed inside (Course New and CourseNew_log) so I could write out its file path:
sudo docker cp SqlCourse.bak container_name:/var/opt/mssql/backup
followed by:
sudo docker exec -it container_name /opt/mssql-tools/bin/sqlcmd -S localhost \
-U SA -P "Password" \
-Q 'RESTORE FILELISTONLY FROM DISK = "/var/opt/mssql/backup/SqlCourse.bak"'
Yet I just don't know how to restore the database. I've tried this:
sudo docker exec -it container_name /opt/mssql-tools/bin/sqlcmd \
-S localhost -U SA -P "Password" \
-Q 'RESTORE DATABASE SqlCourse FROM DISK = "/var/opt/mssql/backup/SqlCourse.bak" WITH MOVE "CourseNew" TO "/var/opt/mssql/data/SqlCourse.mdf", MOVE "CourseNew_log" TO "/var/opt/mssql/data/SqlCourse.ldf"
and it returns "unexpected argument." Clearly that's not the right call but I'm not sure how else to proceed.
(Running mcr.microsoft.com/mssql/server:2019-CTP3.2-ubuntu)
Single quotes are used to enclose string literals in T-SQL so the resultant RESTORE T-SQL script needs to be:
RESTORE DATABASE SqlCourse
FROM DISK = '/var/opt/mssql/backup/SqlCourse.bak\'
WITH
MOVE 'CourseNew' TO '/var/opt/mssql/data/SqlCourse.mdf'
, MOVE 'CourseNew_log' TO '/var/opt/mssql/data/SqlCourse.ldf';
Since you are passing the command as a bash shell command-line argument, you also need to prefix the argument string with '$' and escape the single quotes within the string by preceding them with a \:
sudo docker exec -it container_name /opt/mssql-tools/bin/sqlcmd \
-S localhost -U SA -P "Password" \
-Q $'RESTORE DATABASE SqlCourse FROM DISK = \'/var/opt/mssql/backup/SqlCourse.bak\' WITH MOVE \'CourseNew\' TO \'/var/opt/mssql/data/SqlCourse.mdf\', MOVE \'CourseNew_log\' TO \'/var/opt/mssql/data/SqlCourse.ldf\';'
You can avoid the escaping ugliness by copying the normal RESTORE script into the container and running with the SQLCMD -i argument.

How to connect to an Azure SQL Server using command prompt?

I have an Azure SQL database that I am able to connect using local SSMS.
Server Name - <Server>.database.windows.net
UserName - zrana
Password - *****
The authentication mode I use is here is Active Directory - Password. Is it possible to connect to the database using the sqlcmd utility on the command line?
I am unable to connect using the following command
sqlcmd -S 910005-sql.database.windows.net -d 900046 -U zrana -P ****
There is another database that I created on Azure and tried to connect through Local SSMS.
This time the authentication method is "SQL Server password". I am able to connect to it using local SSMS and through sqlcmd using the same format mentioned above.
Is it possible to connect to an Azure SQL database using AAD? I tried using -G -I options in my command and didn't work.
This is the error I see:
(Adding an answer that is essentially the process we worked through in the comments)
Install the latest sqlcmd from https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-ver15
run sqlcmd -? to see what version of sqlcmd is actually running - if it isn't the latest version you just downloaded, check your PATH variable etc. You can use the dos command where sqlcmd to find where the command line is searching for sqlcmd
run sqlcmd with the -G switch to specify AAD cred checking, and include your user name an dpassword with the -U and -P options. Be careful with the case of these as -u and -p are different switches entirely.
sqlcmd -S 910005-sql.database.windows.net -d 900046 -G -U zrana -P ****
When you want to use an Azure Active Directory user name and password, you need to provide the -G option and also use the user name and password by providing the -U and -P options. On the -U option please see on my example you need to provide the username#domain,
When using the -G option to connect to SQL Database or SQL Data Warehouse and authenticate using Azure Active Directory, a timeout value of at least 30 seconds is recommended. Use "-l 30". Read more about it on this documentation.
Please see the following example:
sqlcmd -S Target_DB_or_DW.testsrv.database.windows.net -G -U bob#contoso.com -P MyAADPassword -l 30

Import dump/sql file into my postgresql database on Linode

I recently moved my Ruby on Rails 4 app from Heroku to Linode. Everything has been setup correctly, but I need to populate my database with a file, lets call it movies.sql
I am not very familiar with postgresql command and VPS, so having trouble getting this done. I uploaded it to Dropbox since I saw many SO posts that you can use S3/Dropbox.
I saw different commands like this (unsure how to go about it in my situation):
psql -U postgres -d testdb -f /home/you/file.sql
psql -f file.sql dbname
psql -U username -d myDataBase -a -f myInsertFile
So which is the correct one in my situation and how to run when I SSH in Linode? Thanks
You'll need to get the file onto your server or you'll need to use a different command from your terminal.
If you have the file locally, you can restore without sshing in using the psql command:
psql -h <user#ip_address_of_server> -U <database_username> -d <name_of_the_database> -f local/path/to/your/file.sql
Otherwise, the command is:
psql -U <database_username> -d <name_of_the_database> < remote/path/to/your/file.sql
-U sets the db username, -h sets the host, -d sets the name of the database, and -f tells the command you're restoring from a file.

Cannot execute script: Insufficient memory to continue the execution of the program

I have a 123MB sql file which I need to execute in my local PC. But I am getting
Cannot execute script: Insufficient memory to continue the execution of the program
How to solve this issue?
use the command-line tool SQLCMD which is much leaner on memory. It is as simple as:
SQLCMD -d <database-name> -i filename.sql
You need valid credentials to access your SQL Server instance or even to access a database
Taken from here.
It might help you! Please see below steps.
sqlcmd -S server-name -d database-name -i script.sql
Open cmd.exe as Administrator.
Create Documents directory.
Put your SQL Script file(script.sql) in the documents folder.
Type query with sqlcmd, server-name, database-name and script-file-name as like above highlighted query or below command line screen.
For Windows Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
Note: If there is any space in the sql file path then use " (Quotation marks) "
For SQL Server Authentication use this sql cmd
SQLCMD -S TestSQLServer\SQLEXPRESS -U sa -P sasa -d AdventureWorks2018 -i "d:\document\sql document\script.sql"
-S TestSQLServer\SQLEXPRESS: Here specify SQL Server Name
-U sa: Username (in case of SQL Server Authentication)
-P sasa: Password (in case of SQL Server Authentication)
-d AdventureWorks2018: Database Name come here
-i "d:\document\sql document\script.sql": File Path of SQLFile
You can also simply increase the Minimum memory per query value in server properties. To edit this setting, right click on server name and select Properties > Memory tab.
I encountered this error trying to execute a 30MB SQL script in SSMS 2012. After increasing the value from 1024MB to 2048MB I was able to run the script.
(This is the same answer I provided here)
My database was larger than 500mb, I then used the following
C:\Windows>sqlcmd -S SERVERNAME -U USERNAME -P PASSWORD -d DATABASE -i C:\FILE.sql
It loaded everything including SP's
*NB: Run the cmd as Administrator
If I understand your problem correctly, you are trying to restore (transact sql) xyz.sql - database + schema. You can try this command which worked for me:
SQLCMD -U sa -i xyz.sql
Try this step,
1)Open PowerShell
2)Write this command:
sqlcmd -S PCNAME\SQLEXPRESS -U user -P password -d databanse_name -i C:\script.sql
3)Press Return
:-)
Below script works perfectly:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Symptoms:
When executing a recovery script with sqlcmd utility, the ‘Sqlcmd: Error: Syntax error at line XYZ near command ‘X’ in file ‘file_name.sql’.’ error is encountered.
Cause:
This is a sqlcmd utility limitation. If the SQL script contains dollar sign ($) in any form, the utility is unable to properly execute the script, since it is substituting all variables automatically by default.
Resolution:
In order to execute script that has a dollar ($) sign in any form, it is necessary to add “-x” parameter to the command line.
e.g.
Original:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql
Fixed:
sqlcmd -s Server_name -d Database_name -E -i c:\Temp\Recovery_script.sql -x
Sometimes, due to the heavy size of the script and data, we encounter this type of error. Server needs sufficient memory to execute and give the result. We can simply increase the memory size, per query.
You just need to go to the sql server properties > Memory tab (left side)> Now set the maximum memory limit you want to add.
Also, there is an option at the top, "Results to text", which consume less memory as compare to option "Results to grid", we can also go for Result to Text for less memory execution.
sqlcmd -S mamxxxxxmu\sqlserverr -U sa -P x1123 -d QLDB -i D:\qldbscript.sql
Open command prompt in run as administrator
enter above command
"mamxxxxxmu" is computer name
"sqlserverr" is server name
"sa" is username of server
"x1123" is password of server
"QLDB" is database name
"D:\qldbscript.sql" is sql script file to execute in database
If you need to connect to LocalDB during development, you can use:
sqlcmd -S "(localdb)\MSSQLLocalDB" -d dbname -i file.sql
As in most answers given here use the command-line tool. In my case the script already has database creation code. If your script contains CREATE DATABASE command, for example
USE [master]
GO
CREATE DATABASE [your-database-name]
Then do not use the -d your-database-name, instead use the following command.
For Windows Authentication use the command
sqlcmd -S ServerName\InstanceName -i "script.sql" -x
For SQL Server Authentication use the command
sqlcmd -S ServerName\InstanceName -U usename -P password -i "script.sql" -x