Encrypt/decrypt columns, without changing existing functionality - sql

GDPR is causing some headaches in this office. We already have a database table in production, lets call it personal_data, that now requires some columns to be encrypted. We are using SQL Server 2012. I've read that columns can be encrypted and decrypted with a symmetric key stored in the database.
We have dozens of existing queries, stored procedures and views that join to this table, so we'd like to avoid changing them if possible.
Is it possible to encrypt the necessary existing columns and query them without modifying these existing queries?
My thought was that if we renamed the personal_data table to something else, then created a view called personal_data, that queried the personal_data table columns and handled the decryption there, so everything that referenced 'personal_data' would still work as before. But if this is possible, what are the pitfalls with this solution?

I would suggest creating another table, say _personal_data. Encrypt the data in that table and replace the current table with a view on the table that returns acceptable columns.
You can give everyone access to the view, while restricting access to the underlying table.
This is a reasonable interim approach. For GDPR and other privacy initiatives, I prefer stronger restrictions, with personal data being in an entirely separate database -- because that is easier to control access to and to log accesses.

SQL Server 2005 enables developers to encrypt and decrypt sensitive data using EncryptByKey and DecryptByKey functions
You can find a sample case illustrated at SQL Server Database Encryption
But this requires code update for INSERT, UPDATE and READ statements
For example,
SELECT
CONVERT(nvarchar, DecryptByKey(EncryptedData)) AS 'DecryptedData'
FROM myTable;
Instead of direct read as
SELECT EncryptedData AS 'DecryptedData' FROM myTable;
Another encryption method is SQL Server Transparent Data Encryption aka TDE. Once you enable it, you don't need to make any code changes to write and read data. But this is a protection for securing disk files at all not for specific data fields. And once you connect database with a valid connection all data is transparent to you.

Related

Query decryption data SQL SERVER

It would like me know if, exists a native tool of the SQL SERVER for encryption some fields, put when making and execute a query, the SQL SERVER decrypt automatically.
It's called Transparent Data Encryption (TDE).
Here is the official MSDN intro:
https://msdn.microsoft.com/en-us/library/bb934049.aspx
Note this criteria (emphasis mine):
a native tool of the SQL SERVER for encryption [of] some fields
TDE works at the page level, which means you have to use it for all of the fields in a table. If it's important that some fields not be encrypted, you have to accomplish this by putting them into a different table with a 1-to-1 relationship.

How to stop edits on linked tables from changing the actual table data MS Access 2013?

I am not sure what the correct forum is for a question like this, so if it would go better on a different one could you move it there please?
I have split my database into a front and back-end database. The front end is using linked tables which are linked to the back-end real tables. If a user changes something in a table on the front-end database, the changes are carried over to the backend database.
Why is this and how can I prevent this from happening? Is there a way to change the settings to make the database read only? Whether it's through VBA or not, I would accept either answer.
That's a feature, not a bug. You're using a linked table, it's linked.
If you want a separate table, make a separate table, and make some ETL (extract/transform/load) process to keep the two tables in sync as needed, accordingly with whatever business rules you need to implement.
If your Access DB is connecting to SQL Server via SQL authentication, you could have the SQL user on the SQL Server side only authorized to SELECT, and denied UPDATE, DELETE and INSERT permissions. Expect errors on the Access side when the linked table is modified then.

Updating a production database with SQL encryption - mass update shortcuts?

I am trying to encrypt a lot of columns in a lot of tables. I have a basic understanding of how to encrypt and decrypt using keys and certificates, but not how I can implement this on a large scale.
For example, I have a few columns in a table with sql code on our production apps to select to insert or update. Would I need to go back to revise each and every sql query to include a ENCRYPTBYKEY() and DECRYPTBYKEY() method in these queries? Are there any solutions or shortcuts to update a production server to do this?
yes, you would need to update all writes to use encryptbykey and all reads to use decryptbykey
If you are using stored procedures or have CRUD operations centralized and locked down, this can make things easier - but it is still a time consuming process. Another option is Transparent Data Encryption (TDE) which can be applied to an entire database w/o changing code; though there are still several commands that need to be run on the server.

Encryption of entire database or selected tables

I m bit new to this field of DBA i wanted to know is there any codes available to encrypt the entire database as we have a huge database maintained in sqlserver 2005 .
I know that it is not safe to encrypt the entire database but we ha such kind of requirement moreover the in the application end they don't want to encrypt it.
i want the process to be as the data comes through the application end into the database it should be encrypted and stored and while retrieving the data it must be decrypted with a certificate/key as provided and shown. I don't want to use any 3rd party tools as it has been instructed.
i searched through the net and found that we can encrypt columns and stored procedure through asymmetric/symmetric key but i need to encrypt the entire database(selected tables is also ok) can you all help me in that.
I don't think there is an easy way to do it in 2005, you would need to redefine all (or most) of your tables to take encrypted data (varbinary) and then you'd lose the ability to do searches and comparisions and a whole heap of other stuff.
For 2008 there is Transparent Data(base) Encryption, which encrypts at the file level (when SQL server writes data) no changes required to your applications.
Search for SQL Server TDE and have a look around.

Ideas for Combining Thousand Databases into One Database

We have a SQL server that has a database for each client, and we have hundreds of clients. So imagine the following: database001, database002, database003, ..., database999. We want to combine all of these databases into one database.
Our thoughts are to add a siteId column, 001, 002, 003, ..., 999.
We are exploring options to make this transition as smoothly as possible. And we would LOVE to hear any ideas you have. It's proving to be a VERY challenging problem.
I've heard of a technique that would create a view that would match and then filter.
Any ideas guys?
Create a client database id for each of the client databases. You will use this id to keep the data logically separated. This is the "site id" concept, but you can use a derived key (identity field) instead of manually creating these numbers. Create a table that has database name and id, with any other metadata you need.
The next step would be to create an SSIS package that gets the ID for the database in question and adds it to the tables that have to have their data separated out logically. You then can run that same package over each database with the lookup for ID for the database in question.
After you have a unique id for the data that is unique, and have imported the data, you will have to alter your apps to fit the new schema (actually before, or you are pretty much screwed).
If you want to do this in steps, you can create views or functions in the different "databases" so the old client can still hit the client's data, even though it has been moved. This step may not be necessary if you deploy with some downtime.
The method I propose is fairly flexible and can be applied to one client at a time, depending on your client application deployment methodology.
Why do you want to do that?
You can read about Multi-Tenant Data Architecture and also listen to SO #19 (around 40-50 min) about this design.
The "site-id" solution is what's done.
Another possibility that may not work out as well (but is still appealing) is multiple schemas within a single database. You can pull common tables into a "common" schema, and leave the customer-specific stuff in customer-specific schema. In some database products, however, the each schema is -- effectively -- a separate database. In other products (Oracle, DB2, for example) you can easily write queries that work in multiple schemas.
Also note that -- as an optimization -- you may not need to add siteId column to EVERY table.
Sometimes you have a "contains" relationship. It's a master-detail FK, often defined with a cascade delete so that detail cannot exist without the parent. In this case, the children don't need siteId because they don't have an independent existence.
Your first step will be to determine if these databases even have the same structure. Even if you think they do, you need to compare them to make sure they do. Chances are there will be some that are customized or missed an upgrade cycle or two.
Now depending on the number of clients and the number of records per client, your tables may get huge. Are you sure this will not create a performance problem? At any rate you may need to take a fresh look at indexing. You may need a much more powerful set of servers and may also need to partion by client anyway for performance.
Next, yes each table will need a site id of some sort. Further, depending on your design, you may have primary keys that are now no longer unique. You may need to redefine all primary keys to include the siteid. Always index this field when you add it.
Now all your queries, stored procs, views, udfs will need to be rewritten to ensure that the siteid is part of them. PAy particular attention to any dynamic SQL. Otherwise you could be showing client A's information to client B. Clients don't tend to like that. We brought a client from a separate database into the main application one time (when they decided they didn't still want to pay for a separate server). The developer missed just one place where client_id had to be added. Unfortunately, that sent emails to every client concerning this client's proprietary information and to make matters worse, it was a nightly process that ran in the middle of the night, so it wasn't known about until the next day. (the developer was very lucky not to get fired.) The point is be very very careful when you do this and test, test, test, and test some more. Make sure to test all automated behind the scenes stuff as well as the UI stuff.
what I was explaining in Florence towards the end of last year is if you had to keep the database names and the logical layer of the database the same for the application. In that case you'd do the following:
Collapse all the data into consolidated tables into one master, consolidated database (hereafter referred to as the consolidated DB).
Those tables would have to have an identifier like SiteID.
Create the new databases with the existing names.
Create views with the old table names which use row-level security to query the tables in the consolidated DB, but using the SiteID to filter.
Set up the databases for cross-database ownership chaining so that the service accounts can't "accidentally" query the base tables in the consolidated DB. Access must happen through the views or through stored procedures and other constructs that will enforce row-level security. Now, if it's the same service account for all sites, you can avoid the cross DB ownership chaining and assign the rights on the objects in the consolidated DB.
Rewrite the stored procedures to either handle the change (since they are now referring to views and they don't know to hit the base tables and include SiteID) or use InsteadOf Triggers on the views to intercept update requests and put the appropriate site specific information into the base tables.
If the data is large you could look at using a partioned view. This would simplify your access code as all you'd have to maintain is the view; however, if the data is not large, just add a column to identify the customer.
Depending on what the data is and your security requirements the threat of cross contamination may be a show stopper.
Assuming you have considered this and deem it "safe enough". You may need/want to create VIEWS or impose some other access control to prevent customers from seeing each-other's data.
IIRC a product called "Trusted Oracle" had the ability to partition data based on such a key (about the time Oracle 7 or 8 was out). The idea was that any given query would automagically have "and sourceKey = #userSecurityKey" (or some such) appended. The feature may have been rolled into later versions of the popular commercial product.
To expand on Gregory's answer, you can also make a parent ssis that calls the package doing the actual moving within a foreach loop container.
The parent package queries a config table and puts this in an object variable. The foreach loop then uses this recordset to pass variables to the package, such as your database name and any other details the package might need.
You table could list all of your client databases and have a flag to mark when you are ready to move them. This way you are not sitting around running the ssis package on 32,767 databases. I'm hooked on the foreach loop in ssis.