I need to be able restore Azure SQL Server databases from one subscription to another (example: Prod to Dev environments that exist in different subscriptions). The databases will have TDE enabled. What is the implication with respect to TDE for the following two scenarios:
Azure SQL databases (PaaS)
SQL databases hosted on Azure VMS (IaaS)
TDE is enabled by default for Azure SQL databases.
We plan to enable TDE on the SQL databases hosted on Azure VMs.
Will there be any issues?
If any issues, how can we mitigate such that the data in the restored data is accessible in the form that it was in on the source servers (and is not in some encrypted state that is not useable).
Similarly, what is the implication with respect to PaaS and IaaS databases that have TDE enabled with DR scenarios where data needs to be accessed in a secondary region.
Thank you.
To restore a TDE-encrypted database to another SQL Server, you need to first restore the certificate to the destination server.
When moving a TDE protected database, you must also move the certificate or asymmetric key that is used to open the DEK. The certificate or asymmetric key must be installed in the master database of the destination server, so that SQL Server can access the database files. For more information, see Transparent Data Encryption (TDE).
You must retain copies of both the certificate file and the private key file in order to recover the certificate. The password for the private key does not have to be the same as the database master key password.
To restore, you need the following permissions:
Backup Operator permissions in the vault where you're doing the restore.
Contributor (write) access to the source VM that's backed up.
Contributor (write) access to the target VM:
If you're restoring to the same VM, this is the source VM.
If you're restoring to an alternate location, this is the new target VM.
For more details and step-by-step implementation, refer Restore SQL Server databases on Azure VMs
Related
We are supporting a legacy system for our organisation. In the current scenario, we receive a SQL Server backup (.bak files) from the application vendor on an FTP location. For every weekend on Sunday it is a Full backup and for every other day its the differential one.
On our side, we have a SQL server instance running which has custom stored procedures written and scheduled to check the location every morning and then restore the backups every day. These restored backups are then used by the organisation for internal reporting purposes. There are 100s of other stored procedures written for different reports in different DBs on the same instance.
Since SQL Server 2008 is now out of support and for cost-saving purposes of running on-premise system, my team has been given a task to look into migrating this whole system to Azure SQL database.
My question is what is the most effective way in which we can move this workflow to the cloud? I have an azure trial account set up for me to try but haven't been successful in restoring the .bak files on Azure SQL instance.
Thanks.
You essentially have two options for Azure, either perform a fairly linear Lift and Shift to SQL Server on an Azure VM or go with a more advanced Azure PaaS offering in Azure SQL Database Managed Instance. The specific deployment Azure SQL Database (Single Instance) will not support your current solution requires with regard to the .bak file support, and I have detailed that below. For further details between the difference between Azure SQL Database Single Instance versus Managed Instance, please see: Features comparison: Azure SQL Database and Azure SQL Managed Instance
The second option, is to leverage the Azure Enterprise Ready Analytics Architecture (AERAA) (link) of Azure (PaaS) Analytics services. With Azure SQL Database (PaaS) services, as opposed to on-premise SQL Server or SQL Server on an Azure VM, there is no Integration Runtime or Analysis Services as a bundled service component. These services are separate PaaS offerings and with the help of the linked AERAA blog, you can gain a better understanding of the Azure Analytics services.
The .bak versus .bacpac support dilemma:
Since the main requirement for your solution is support of .bak files, you need to understand where .bak and where .bacpac files are supported. The term Azure SQL Database applies to both a specific deployment option for an Azure SQL database (PaaS) service and as a general term for Azure SQL cloud databases. As for the specific deployment option, Azure SQL Database (Single Instance nor Elastic Pools) will support your scenario with .bak files. This deployment option will support export/import functionality via .bacpac file format. It will not support full/partial restore functionality. The backup/restore functionality although configurable, is only in scope for the specific database hosted by an Azure SQL (logical) Server instance. Basically, you can not restore an external file. You can import, which is always a full copy. So, for that reason, for an Azure PaaS database service you will need Azure SQL Database Managed Instance for .bak file support or deploy an SQL Server VM image to an Azure VM, and migrate your objects via Azure Database Migration Service.
Regards,
Mike
TDE is not enabled on MI by default. I had manually enabled TDE by setting the encryption to true in the db properties. Encryption of the DB took around 20 mins and below is the output from the sys.dm_database_encryption_keys table.
How do we verify if TDE is enabled in Managed instance or not? As neither Azure provides access to automated backups of MI nor i can manually take the backup when the TDE is turned on in MI.
Would like to access the automated backups (7day backup feature) and verify the files can be restored or not. Provide the procedure.
Hoping the backups are encrypted as TDE is enabled on my DB, so how can these be decrypted as Azure is not providing any keys to decrypt.
Use the following query to find encrypted database:
select name, is_encrypted from sys.databases
You cannot directly access backup files; however, you can run point-in-time restore to verify that any previous backup can be restored as a new database (you have restore button on the portal for each database or you can use PowerShell see - https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/06/28/point-in-time-restore-of-a-database-on-azure-sql-managed-instance-using-azurerm-sql-powershell-library/)
Point-in-time restore is the only way to restore encrypted automatic backups and Azure automatically de-crypts the backup on restore. You cannot restore backup from MI on SQL Server because MI is always higher version than SQL server. Therefore, even if you de-crpt the backup file you cannot use it on SQL Server, and if you want to restore it on Managed Instance, MI will find the key.
We are trying to evaluate possibility of migrating our in-house SQL DB server to Azure SQL as a PaaS.
Our legacy windows application which is written in VB6 and now running on VB.NET Framework 4.5
Clarifications I need if I migrate only DB server to Azure:
We use both trusted / credential based SQL connection from our desktop application to connect to SQL DB. If we migrate to Azure SQL, will it support trusted connection which should authenticate current organizations NT user?
We have lot of cross DB queries, do we need to face any challenge to use the queries as it is?
Run time we take a DB backup / restore for some business cases. Does this work?
Are there any restrictions on number of admin users on Azure DB?
Probably yes if you sync your local AD with an AAD (See: Use Azure Active Directory Authentication for authentication with SQL Database, Managed Instance, or SQL Data Warehouse)
Azure SQL Database (PaaS) doesn't support cross DB queries by default - you have to setup / use Elastic Query for that.
Yes, you can take a DB backup at runtime and also restore it. There is also a point-in-time restore feature available. See: Learn about automatic SQL Database backups.
I think you can only specify one server administrator (at least within the portal) but I doubt you will reach any limit on db users.
Instead of using the single database SQL Server PaaS service you should also consider using Managed Instance (preview)
You will have to extend your active directory to Azure active directory to keep using trusted connections. You will learn how to do it on this documentation and this one.
On Azure SQL Database you have elastic queries that allow you to run cross database queries. Learn how to create elastic queries here.
You can create bacpacs (export your databases) to Azure Storage or to on-premises location very easy.
You can configure one Server Admin or one Azure Active Directory Admin (it can be a group) for your Azure SQL Server. However, at the database level you can add many database users to the dbmanager role. You can have more information about this topic here.
We use Azure as infrastructure for our app and its SQL DBs.
Currently Azure provides automatic backups for all tiers (Basic to Premium), but these settings are individual per DB.
How can I set backup for the entire server, with all DBs inside?
Actually, backups are configured on database level. When you backup all of your databases, the database server is considered backed up. More information on the topic: Azure SQL Database Backup and Restore
I copied a SQL Server database from one system to the next, identical setup, but completely different physical machine. I used Norton Ghost and recoverd files manually, for example, the entire SQL Server 2008 folder found in c:\Program Files after re-installing SQL Server 2008 Express.
One of my databases has AES_256 encryption enabled on a number of one of its tables, columns. I resetup my IIS7 and tried to run the app that access the database, upon retrieving the data, I get this error:
Server Error in '/' Application.
Please create a master key in the
database or open the master key in the
session before performing this
operation. Description: An unhandled
exception occurred during the
execution of the current web request.
Please review the stack trace for more
information about the error and where
it originated in the code.
Exception Details:
System.Data.SqlClient.SqlException:
Please create a master key in the
database or open the master key in the
session before performing this
operation.
Source Error:
An unhandled exception was generated
during the execution of the current
web request. Information regarding the
origin and location of the exception
can be identified using the exception
stack trace below.
I've done some reading and found some links about how the AES encryption is linked with the machine key, but am at a loss as to how to copy it over to the new system. Or perhaps this even isn't the case.
NOTE: I've tried dropping the symmetric key, certificate and the master key and re-creating them. This gets rid of the error, but than the data that in encrypted via AES_256 does not show up. The columns that are NOT encrypted do, however.
Any help would be much appreciated. Thanks in advance!
The database master key is encrypted using the server master key, which is specific to the machine where SQL Server is installed. When you move the database to another server, you lose the ability to automatically decrypt and open the database master key because the local server key will most likely be different. If you can't decrypt the database master key, you can't decrypt anything else that depends on it (certificates, symmetric keys, etc).
Basically, you want to re-encrypt the database master key against the new server key, which can be done with this script (using admin privileges):
-- Reset database master key for server (if database was restored from backups on another server)
OPEN MASTER KEY DECRYPTION BY PASSWORD = '---your database master key password---'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
GO
Note that when you create a database master key, you should always provide a password as well so that you can open the key using the password in the scenario where the service master key cannot be used - hopefully you've got that password stored somewhere!
Alternatively, you can restore a backup of the database master key - but you need one that was created for the target server, not the source server.
If you haven't got either a backup or a password, then I'm not sure you will be able to recover the encrypted data on the new server, as you will have to drop and recreate the database master key with a new password, which will kill any dependent keys and data.
I just had a similar situation, an server rebuild after the OS drives died. I reinstalled SQL and reconnected it to all my old databases on the untouched data drives. Everything worked except for my encrypted columns. But my issue was that the master service key was hosed. I was able to repair my master service key by going back to the same domain credential that had been my SQL server service account before the move.
This article gave me the fix (kudos to Matt Bowler for his excellent article). I knew the local machine key had changed, but my salvation was that I could use the same service account.
Service Master Key: At the top of the key hierarchy is the Service Master Key. There is one per SQL Server instance, it is a symmetric key, and it is stored in the master database. Used to encrypt Database Master Keys, Linked Server passwords and Credentials it is generated at first SQL Server startup.
There are no user configurable passwords associated with this key – it is encrypted by the SQL Server service account and the local machine key. On startup SQL Server can open the Service Master Key with either of these decryptions. If one of them fails – SQL Server will use the other one and ‘fix’ the failed decryption (if both fail – SQL Server will error). This is to account for situations like clusters where the local machine key will be different after a failover. This is also one reason why service accounts should be changed using SQL Server Configuration Manager – because then the Service Master Key encryption is regenerated correctly.
http://mattsql.wordpress.com/2012/11/13/migrating-sql-server-databases-that-use-database-master-keys/