Our company is in the process of adapting TFS for source repository and project management. I am in charge of database part of the project. We are using SQL Server 2008 R2, Visual Studio 2012 and TFS Online. We have a database that is used by several of our applications. So far I have been the only one handling any change to this database. As the company is expending we are going to have multiple dev teams. So I am planning to save the database as as SSDT project to TFS.
At the moment I am maintaining my database like the following:
I have separate folders for UDFs, Stored Procedures, and Config.
Under these folders I have subfolders for each objects. For example, for stored procedures I have subfolders for each stored procedure which contains the SQL script to create the SP. The config folder contains any script similar to SSDT's post deployment script (for example, populating static data).
The SQL script contains code to drop the procedure and create it.
I have a c# app to concatenate all the SQL files into one single SQL file. Let's call it the FINAL script. When creating FINAL script I can specify version number which adds an update statement to update the version table on the database.
FINAL script is made available for customers to download and execute on the database. So the script mainly contains any add/edit to SPs, UDFs, and static data. It does not touch any existing data (data entered by user) in most cases.
As a newbie to TFS and SSDT I am not exactly sure how this can be done using SSDT/TFS or if there is better way of doing something similar. So far what I have understood about SSDT and TFS is:
I can import an existing database to SSDT project.
This will create scripts for all objects including tables.
I can easily do a publish of the database to a local server or to a server I have access to.
Things that seem confusing so far:
How do I supply clients with my latest update script? I am thinking of manually including the FINAL script to the SSDT project but there must be better way of doing it.
How do I publish the changes to a copy of the database without the loss of any user-entered data? My guess is when publishing the tables get created. I can take care of the static data but I am not sure how to handle data entered by users.
May be there is something fundamentally wrong in my understanding of this whole thing. That is why I am here... :)
You want to pull your DB into a SQL Project. Maintain all of your changes there. This tells your system what the schema of your database should be. From there, I'd generate the dacpac files (through building the project) and provide those to your clients along with having them install the SSDT tools that include SQLPackage. They can run SQLPackage to make changes to their database to handle the schema changes automatically. This will bring their database in line with your schema, no matter how far off it might be.
I'd also create a publish profile for them to use. This lets you control some of the settings.
You can choose to not drop any objects not in your project
You can choose to ignore users/permissions
You can set an option to not allow changes if there would be data loss.
You can wrap everything in a transaction so a failed update rolls back
If you give them a batch file to run, you can specify an output file or a Diff report, or have them generate their own script to do the update.
I blogged about this at http://schottsql.blogspot.com/2013/10/all-ssdt-articles.html
(or http://schottsql.blogspot.com/search/label/SSDT if that doesn't work well). That will take you through some basics of why you might want to use SQL Projects, creating them, maintaining them, and publishing the changes to an existing database.
Related
We use TFS Continuous Integration to handle our staging and deployments of code. In our current environment, we (developers) aren't allowed to manually update databases in Production. A script must be staged and then given to a DBA to run.
By default, the database project builds and outputs a database creation script that will create all the tables and stored procedures. However, it does not include checks to see if the object already exists.
For example, when it attempts to create the Customer table, I would like to have the script check if the table already exists, if it does alter the table.
Is this at all possible?
VS can create a script for just the changes. I think this approach will be better than using existence checks because it will be able to handle column changes, and overall it makes for a shorter and more targeted script.
Right-click the project and select Publish.
Click Edit and enter the connection details for your staging database.
Back on the Publish dialog, click Advanced and make sure "Always re-create database" is not checked.
Back on the Publish dialog, click Generate Script.
What this approach does is compare the objects in the database project to your staging database and generates a SQL script for just what is different. You can even save the publish settings to a file to make it easier to generate future scripts.
Keith is right you need to script the changes rather than just using the create statements.
You basically either need a copy of the production database to run a comparison against or you give the DBA's a way to run the comparison and deploy.
The way I prefer to do it is with TFS is to use SSDT in Visual Studio, I then have a custom build step as part of the .sqlproj file that builds the dacpac, uses sqlpackage.exe to compare the dacpac to the mirror of production (or dev, uat, whatever) - this then outputs a script that will take that version of the database to the same version of the code as the dacpac.
You can adjust this slightly to auto-deploy to dev, uat etc and just create the script in production but the choice of exactly what you do it up to you!
If you can't get a mirror of production or a copy of the schema of production etc, you can give the dacpac to the dbas and and either a batch file or powershell script ot drive sqlpackage.exe to create a script or just go ahead and deploy.
Exactly what works depends on the environment you are in!
Ed
I've created an SQL Server Database Project so that I can capture my database schema and add it to source control.
My problem is that the database contains Views which reference external databases. Given the business and project environment, this is an acceptable solution in the short tomedium term.
Sadly, this stops the database project from compiling, (since it don't contain the external database tables).
What are my options for getting around this error? I'm currently storing the schema in a single generated script, which is a pain to update.
Look at creating dacpac files out of the external databases and add them as database references. I did that by using the SQLPackage command line to generate the file, put the files in a "shared" folder (optional, but useful if this pattern persists with other projects), then add a database reference to the project. I recommend removing the variable for the DB name unless it can change in different environments. I blogged a bit about this here:
http://schottsql.blogspot.com/2012/10/ssdt-external-database-references.html
Now if it's a truly breaking change, I've done this through post-deploy scripts. Drop/recreate the view and reapply any permissions necessary. That's not ideal, but it can work.
I am looking for information regarding Visual Studio 2013 and working with SQL Server projects using VS 2013. We are currently working on a project where were're using a database that already exists and is used by an ERP application. We're creating SQL Scripts that would alter and create fields on a table on the target database.
Now, we're not looking to "publish" those scripts, but create postdeploy scripts instead, which contains all the necessary SQL scripts in the order they need to be run. Everything is working fine. When we build the project, we get a fresh copy of the PostDeploy.sql script file that we run across a target database.
At the moment, the script looks at a table, if the column that needs to be added exists, it DROPS it and then recreates it. This is fine for the testing phase, but once we go live, there will be several stages of the databases that the code needs to be tested on. The column may already exist from before and in that case, we wouldn't want to DROP that column, instead, we want to do schema and data level compare and just get over the objects that are DIFFERENT, so that the column doesn't need to be dropped, instead just "updated". I hope I am not being vague when I ask this question.
I found this video: https://www.youtube.com/watch?v=AuVpmu9CKRY and I am not sure if that is what I need to do? I would love any suggestions from you guys..
Have a wonderful day!
Well, this isn't really the best use for SSDT/DB Projects. Ideally, you'd want to pull the schema into a project and tweak that project to look the way you want. Rename columns, change types, etc. Because it sounds like this is a 3rd party app, you'd want some environment that can serve as your baseline - when you run whatever upgrade script is sent by the vendor, it goes against that environment. You'd then want to pull the appropriate changes into your project.
Once you have a project that looks the way you want, you use the publish option against your target database. In your case, I'd likely recommend generating a script. If you're in the VS environment, you can take a look at both the script and a summary of what will be changed.
For data compares, I'd really consider something like Red Gate's SQL Data Compare (pro edition if you can). You can set up a data compare against your baseline and automate pushing the data changes. You can do that through post-deploy scripts, but you'll need to hand-code the data inserts, updates, and deletes yourself.
I've blogged about SSDT before and that may give you some ideas. Jamie Thomson has also written quite a bit about Database/SQL Projects and inspired quite a bit of what I've done.
http://schottsql.blogspot.com/2013/10/all-ssdt-articles.html
My Application Database Without Project and without Source safe, i planned to make my DB to be as project and add it to TFS, but I have no idea how to script the stored procedures, Triggers, Views, Functions, and what is the best practice to Make Update Script for All My stored procedures, Triggers, Views, and Functions to My customers DB.
The best procedure (IMHO) is to manually maintain a strict version of your schemas. Then when you need to make changes you write a delta script to move from one version to the next. I suggest you write the DDL scripts by hand -- you can make them concise and comment them.
You can use a tool like Visual Studio Team System for database architects, take a look at Running static code analysis on SQL Server database with Visual Studio Team System for database architects it will show you how to import the data, disregard the static code analysis that comes later it does not apply to your question
I've found a good way to get SQL scripts into SCM from an existing database is to use SMSS's "export all to script" option or whatever it's called, can't remember now.
Then every other change you add the change script into your SCM with a different version number in the file name.
Every release (or set cycle depending on your development/release methodology) you apply all change scripts, then re-script the entire database, tag it, and start again.
The best way to do it - save the database in TFS as set of database creation script, i.e. MyTable table should be added to TFS as MyTable.sql file (CREATE TABLE...) etc. We are using SQL Examiner to do this - see the following article: How to keep your database under version control
We are working with SVN and I never tested SQL Examiner with TFS, but I know that the tool supports TFS.
I was wondering if there is a way to automatically append to a script file all the changes I am making to my columns, tables, relationships etc...
The thing is I am doing a lot of different changes on a TEST db and the idea will be to apply this change script when I move the test db to production... hence keeping production data but applying all schema and object changes.
Is there an easy way to do this? Can it also migrate database diagram changes?
I have seen how you can create a change script each time I do a change but this means I have to copy and paste into a master file. Actually pretty easy!
I was just wondering if I was missing something?
Do not make changes to the test server using the UI. Write scripts and keep them under source control. You can test your scripts starting from backups of the live data and you can tune yoru scripts untill they achieve the desired result. Then you can check in the scripts for reference and later apply them on the live server. See this article Version Control and Your Database.
BTW, check out the SSMS toolpack, I think it may do what you want (I'm not sure). My advice stand none the less: version your schema, use explicitly created/saved scripts, use source control.
There's no way to directly generate a "delta" script in SSMS.
However, if every time you publish changes, you script out the entire database, including data, to SQL using the SQL Server Database Publishing Wizard you should be able to extract diffs between the versions and get your deltas that way.
If money is no object, you can purchase Visual Studio Team System Database Architect edition and use its fantastic database comparison tools to generate and version control exactly the diffs you want.
Try using TableDiff , that came with SQL Server 2005.
SQL Server 2005 TableDiff Utility
tablediff Utility
We have the process where when a developer gets done with a change, they then script it out and check it into Subversion. In Subversion we have a folder for Tables, Stored Procs, Data, etc. They script it out so it is repeatable (i.e. don’t insert the new data if it is already there.) This is important to do anyway so you keep the history of changes for a given object in the database.
In the past, we would just enter each of the files that we wanted scripted out into a text file (i.e. FileListV102.txt). When we were ready to make a release we would do “get latest” on all of the files (from VSS back then.) We then had a simple utility that would read the “file list” file and open each of those files in turn concatenating them into an output file. That is pretty easy to code.
We outgrew that and now we have a release management tools (which can be found here and will be on sale mid September), that takes all of the files and creates a big SQL script file out of it. It does it in the order that you would expect based on the folder names – so files found in the "Tables" folder are done before those in the "Data" folder, etc.
Either way, once you are done you have a big SQL script file that you can then apply to a fresh copy of production and that is what you test against.
I know I'm way late to the party, but I just wanted to add that there are tens of third party products out there. Some are very good, some are very cheap or free, and some are a mixture. I listed 22 here:
http://bertrandaaron.wordpress.com/2012/04/20/re-blog-the-cost-of-reinventing-the-wheel/
We have been using a relatively new software called Kal Admin.
It has Change Management feature and let distributing selected changes to other databases very easily. We used to do it by comparing two databases but it not satisfy our need for change tracking.
BTW Kal Admin has Metadata and data compare capabilities as well.