I copied a SQL Server database from one system to the next, identical setup, but completely different physical machine. I used Norton Ghost and recoverd files manually, for example, the entire SQL Server 2008 folder found in c:\Program Files after re-installing SQL Server 2008 Express.
One of my databases has AES_256 encryption enabled on a number of one of its tables, columns. I resetup my IIS7 and tried to run the app that access the database, upon retrieving the data, I get this error:
Server Error in '/' Application.
Please create a master key in the
database or open the master key in the
session before performing this
operation. Description: An unhandled
exception occurred during the
execution of the current web request.
Please review the stack trace for more
information about the error and where
it originated in the code.
Exception Details:
System.Data.SqlClient.SqlException:
Please create a master key in the
database or open the master key in the
session before performing this
operation.
Source Error:
An unhandled exception was generated
during the execution of the current
web request. Information regarding the
origin and location of the exception
can be identified using the exception
stack trace below.
I've done some reading and found some links about how the AES encryption is linked with the machine key, but am at a loss as to how to copy it over to the new system. Or perhaps this even isn't the case.
NOTE: I've tried dropping the symmetric key, certificate and the master key and re-creating them. This gets rid of the error, but than the data that in encrypted via AES_256 does not show up. The columns that are NOT encrypted do, however.
Any help would be much appreciated. Thanks in advance!
The database master key is encrypted using the server master key, which is specific to the machine where SQL Server is installed. When you move the database to another server, you lose the ability to automatically decrypt and open the database master key because the local server key will most likely be different. If you can't decrypt the database master key, you can't decrypt anything else that depends on it (certificates, symmetric keys, etc).
Basically, you want to re-encrypt the database master key against the new server key, which can be done with this script (using admin privileges):
-- Reset database master key for server (if database was restored from backups on another server)
OPEN MASTER KEY DECRYPTION BY PASSWORD = '---your database master key password---'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
GO
Note that when you create a database master key, you should always provide a password as well so that you can open the key using the password in the scenario where the service master key cannot be used - hopefully you've got that password stored somewhere!
Alternatively, you can restore a backup of the database master key - but you need one that was created for the target server, not the source server.
If you haven't got either a backup or a password, then I'm not sure you will be able to recover the encrypted data on the new server, as you will have to drop and recreate the database master key with a new password, which will kill any dependent keys and data.
I just had a similar situation, an server rebuild after the OS drives died. I reinstalled SQL and reconnected it to all my old databases on the untouched data drives. Everything worked except for my encrypted columns. But my issue was that the master service key was hosed. I was able to repair my master service key by going back to the same domain credential that had been my SQL server service account before the move.
This article gave me the fix (kudos to Matt Bowler for his excellent article). I knew the local machine key had changed, but my salvation was that I could use the same service account.
Service Master Key: At the top of the key hierarchy is the Service Master Key. There is one per SQL Server instance, it is a symmetric key, and it is stored in the master database. Used to encrypt Database Master Keys, Linked Server passwords and Credentials it is generated at first SQL Server startup.
There are no user configurable passwords associated with this key – it is encrypted by the SQL Server service account and the local machine key. On startup SQL Server can open the Service Master Key with either of these decryptions. If one of them fails – SQL Server will use the other one and ‘fix’ the failed decryption (if both fail – SQL Server will error). This is to account for situations like clusters where the local machine key will be different after a failover. This is also one reason why service accounts should be changed using SQL Server Configuration Manager – because then the Service Master Key encryption is regenerated correctly.
http://mattsql.wordpress.com/2012/11/13/migrating-sql-server-databases-that-use-database-master-keys/
Related
I need to be able restore Azure SQL Server databases from one subscription to another (example: Prod to Dev environments that exist in different subscriptions). The databases will have TDE enabled. What is the implication with respect to TDE for the following two scenarios:
Azure SQL databases (PaaS)
SQL databases hosted on Azure VMS (IaaS)
TDE is enabled by default for Azure SQL databases.
We plan to enable TDE on the SQL databases hosted on Azure VMs.
Will there be any issues?
If any issues, how can we mitigate such that the data in the restored data is accessible in the form that it was in on the source servers (and is not in some encrypted state that is not useable).
Similarly, what is the implication with respect to PaaS and IaaS databases that have TDE enabled with DR scenarios where data needs to be accessed in a secondary region.
Thank you.
To restore a TDE-encrypted database to another SQL Server, you need to first restore the certificate to the destination server.
When moving a TDE protected database, you must also move the certificate or asymmetric key that is used to open the DEK. The certificate or asymmetric key must be installed in the master database of the destination server, so that SQL Server can access the database files. For more information, see Transparent Data Encryption (TDE).
You must retain copies of both the certificate file and the private key file in order to recover the certificate. The password for the private key does not have to be the same as the database master key password.
To restore, you need the following permissions:
Backup Operator permissions in the vault where you're doing the restore.
Contributor (write) access to the source VM that's backed up.
Contributor (write) access to the target VM:
If you're restoring to the same VM, this is the source VM.
If you're restoring to an alternate location, this is the new target VM.
For more details and step-by-step implementation, refer Restore SQL Server databases on Azure VMs
Contained and login user passwords from the source aren't working in the target after restoring a .bacpac from one Azure SQL Server onto another Azure SQL Server.
The users do exist in the target SQL Server database, but they are unable to login with the same password from the source in the target SQL Server.
How can we keep intact and make the passwords work?
I too faced this issue a year ago and after consulting Microsoft support, they confirmed that this is by design that the passwords are changed in the background for security reasons. I would suggest you to run a script or procedure to alter user passwords back to original ones after restoring the database from bacpac file.
ALTER USER [USERNAME] WITH PASSWORD=[PASSWORD]
Apart from the solution to alter user passwords, you can see the resolution strategy given by Microsoft support -
We are going to use Always Encryption on few of the columns in our database tables. I gone through some of the articles on this subject and stuck on two points.
Say I am going to use same server for my database and my web application. Now I can save the Column Master Key(CMK) in the Windows certificate store of the current machine. How can I ensure that except Administrator no other database user can access the plain text data from SSMS?
I have separate servers for database and web application. In the time of encryption, the CMK certificate get created in the database server. Now I can export the certificate and install it in web application server. Can I remove the certificate from the database server so that nobody can see the plain text data?
Thanks in advance
I am working on a prototype for an upcoming big solution and wish to use Always Encrypted to encrypt certain sensitive database columns.
My setup is a follows:
Database Server: SQL Server 2016 installed
Application Server: Reporting Server 2016 installed pointing to the Database Server engine. IIS, .Net 4.6.2 etc. all setup as well.
The environment is also setup in a way that the DBA can't read the encrypted data even if from Management Studio he will add the 'Column Encryption Setting = Enabled' in the connection. So my certificate is installed on the Application Server while the CMK and CEK are installed on the Database Server database.
I can view the encrypted data from my Web App installed on the Application Server with no problem, and the DBA can't read the encrypted data directly from the database, so I am assuming that my environment is well set up.
As explained I have SSRS 2016 installed on the Application Server but pointing to the database with encrypted columns on the database server. I have done a basic dump report (for testing purposes) using Report Builder of course and all works well EXCEPT that the encrypted data is not displayed - it is remaining blank in the SSRS Table! The encrypted column is just a basic nvarchar(200)
In the datasource connection string I have added 'Column Encryption Setting = Enabled'. Without it the report display #Error as expected. So I am assuming that this is needed as well.
Something also that I noticed is that from the Query Designer I can read the encrypted column. if I remove 'Column Encryption Setting = Enabled' from the datasource the Query Designer displays VarBinary if I remember correctly. I am working with Report Builder and Query Designer directly on the Application server of course.
I tried to search for any tutorials on how to use SSRS with Always Encrypted but I couldn't find anything. All I found is a comment in a post that SSRS supports Always Encrypted.
Can someone please enlighten what I am doing wrong or what I am missing?
Thanks in advance.
Disclaimer: I am a Program Manager at Microsoft.
To troubleshoot your issue, please try to run the problematic query (the one that returns no data in Report Builder) from SQL Server Management Studio on the same machine (that you run Report Builder from) and as the same user, over a connection with 'Column Encryption Setting = Enabled' and see, if you get any error message.
I have seen a query against encrypted columns returning no results in Report Builder, if the certificate is not deployed on the machine, hosting Report Builder, or the user does not have a permission to access the certificate. Do you store the certificate (used as a column master key) in the Current User or Local Machine store? If it is in the Local Machine store, you need to ensure that you (the user who runs Report Builder) have the permissions to access the certificate (you can configure permissions on a certificate using Management Console).
The account that is running the SSRS service needs to have read permission on the Always Encrypted certificate in the Local Machine store. Right click the certificate, select All Tasks – Manage Private Keys and then provide the SSRS service account read permissions on the certificate.
I am trying to test a proof of concept that I can run a distributed transaction across two linked SQL Servers, linked using sp_addlinkedserver - their names are Server1 and Server2, both running under default instances. Each server holds a single database, Source and Destination respectively and the destination database holds a single table called Output, i.e.
Server1.Source
Server2.Destination.Output
The OUTPUT table has the following structure:
OUT_PKEY int identity(1,1) primary key,
OUT_TEXT nvarchar(255)
From Server1 I have called sp_addlinkedserver 'Server2' to link the two databases and I've attempted to run the following query to test that the link does indeed work:
Select *
From Server2.Destination.dbo.Output
I am returned the following exception:
Access to the remote server is denied because no login-mapping exists.
Fair enough, so from Server1, I run sp_addlinkedsrvlogin 'Server2' which according to the documentation says that it should take the user credentials of whomever runs the query remotely (i.e. from Server1) and apply those credentials to Server2. This implies that since I am connected to Server1 using Windows Authentication, this should mean that my Windows Credentials are applied to Server2 also.
Now the exception message changes to:
Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.
Having Googled this exception, I came up with nothing useful that pointed me in the right direction. What am I missing? I would expect [should the login fail at all] the exception to reference my Windows Credentials, not the anonymous logon credentials.
It looks like once I get the link itself working, the distributed transactions themselves should be a fairly simple affair - the documentation implies that I just need to ensure that the DTC Service is running on Server1 and that any queries run on Server1 that will be transacted across the link:
Include SET XACT_ABORT ON prior to initializing my distributed transaction
I use BEGIN DISTRIBUTED TRANSACTION instead of BEGIN TRANSACTION
If I wish to reference a non-default instance of SQL Server on Server2, I replace any instances of the name Server2 in my query with [Server2\InstanceName]
My questions are these:
how do I get past this login issue? The sp_addlinkedsrvlogin stored procedure alone doesn't seem to be doing the trick.
Is it indeed as simple as that to run the distributed transaction as the documentation implies?
TIA
If you're on a domain, then the setting should be "Be made using the login's current security context", but there's one more step - you need to grant an SPN to each of the servers involved in the transaction.
Assuming you're running the SQL Services on both servers as a domain user (which you'll need to in order to make this work - LocalSystem won't do it), here are the instructions you'll need:
http://technet.microsoft.com/en-us/library/bb735885.aspx
Remember that the user will need an SPN for both servers, but not the client - for example, if you are going from client -> server1 -> server2, the SQL Service account will need an SPN for both server1 and server2.
If you're confused (it's a confusing process), post a comment and I'll clarify the instructions.
Assuming these servers are both on the same domain - have you enabled trusted delegation to allow your server to pass the credentials to the targeted server? You would pull up the Active Directory object for the server and go to the Delegation tab and select "Trust this computer for delegation to specified services only" and then enter the SQL Server details that the server is allowed to pass credentials to:
Service Type = MSSQLSvc
User/Computer = YourTargetServer.Your.Domain
Port = 1433
Sadly, a lot of these types of authentication issues with linked servers require a reboot to fully take effect (so if these are production servers it is hard to troubleshoot during the day).
In regards to distributed transactions - if you eventually get the linked server connection up and running correctly then distributed transactions work great. Although the next thing you'll probably run into once you get it working is finding the huge flaw that you can't use any form of SCOPE_IDENTITY(), ##IDENTITY, etc. to retrieve primary keys after inserting something into a linked database. But that's another issue with its own fun workarounds...