hope that everybody here is OK.
We are using VS 2008 as development tool, TFS 2008 as version control as well as build automation. Some of our developer use dbpro for databases changes and some use SQL Server management studio.
I am trying to automate build for Web Application built using C# and VB.Net.
Our scenario is such that we have a central database to which our web application connects.
Whenever we supply our clients with a new functionality or a bug fix, we supply them incremental builds.
The SQL script is checked into source control for every incremental build when they have made and tested there changes on our central DB Server.
I want to generate Differential script that can be run at the client as an incremental update script. Now to come about it is a problem. Sometimes our developers tend to forget the database change-sets and the script in the source control is missing an SP or a two.
Also, sometimes we need to insert default data into some of the tables that have strict stringent values and not test values. Like a table that contains Services provided by the panel, we add a new service name, signature, credentials and service address, etc etc in the ServiceTable. Besides this many other tables may have test data that may not be needed.
If we use DataCompare, it will generate changeset for required data (important for client to enable certain services) and our test data that was added to the database as a result of our testing of the functionality or bug fix.
Currently i am using SQLSchemaCompareTask (from Visual Studio 2008 Team Database Professional Power Tools API) in the TFSBuild.proj file of the build definition for TFS 2008.
Using SQLSchemaCompareTask, the script generated contains database names like [dbo]. etc which are not desired as the script fails when run against SQL Server 2000 databses (Some of our client still use SQL Server 2000) databases as teh backend of the application.
Also default data can't be generated by this process.
To overcome this problem, i have to come up with a solution that can compare databases and generate script automatically that does not have to be manually reviewed again before being sent to the client.
Please suggest effective methodology of such SQL script generation and suggest whether two different databases may be used or something ? Is there any toolkit or api that can enable build automation for SQL Server databases?
Thank you all.
Regards
Steve
Try to use SQL Examiner Suite for this:
http://www.sqlaccessories.com/SQL_Examiner_Suite/
The tool compares both schema and data and produces synchronization scrips (or differentials scripts). You can automate script creation with supplied command-line tool.
Rather than collating many individual change set scripts (and therefore occasionally missing objects out), why not use schema compare and data compare to create a single script from your database project using a database equivalent to your client's on the target? This should create a script tailored to their requirements.
In data compare you can exclude test data records that you don't want pushed to your client by unchecking them in the lower grid.
Related
I work for a customer who has his own database management team and I need to deploy my new web application version within a SQL Server 2008 script (I am not in position to execute any actions on their systems). I can't back up myself their database and I'm not sure they'll do it so if I delete all data it's gonna be terrible.
Therefore, I'm looking for a solution to back-up if possible the database, extract the existing datas, execute the new statements of my script, and re-include the datas saved in the database.
It is possible to do this in a SQL server script ?
More generally, how can I safely update schemas and datas of a SQL Server database within a script without losing all data inside ?
PS :
Currently, I'm using in the dev environment the initial database schema and the newest. So, I use Visual Studio 2012 with Data Tools to make Schema comparison and generate the update script.
You start with a copy of the database schema with sample data against which you can develop and test.
Then you write scripts (perhaps with the aid of a tool like SSDT) that updates the schema to be compatible with the new version of your application, retaining the data in the database.
You deliver these tested schema modification scripts along with the new version of your application for the customer's administrators to test and apply to the target database as part of the application upgrade.
Currently, we are manual running DB scripts (SQL Server 2012) outside of our CI/CD deployment. What are ways (including toolsets) can we automate deployment of DB changes using TFS 2015 Update 3?
There are really two approaches here, both of which work with TFS. Really TFS just facilitates the execution of any scripting that you will use to update your database, including your custom, handcrafted scripts.
There is the state based approach, which uses a comparison technology to look at your VCS/dev/test/staging database and compare this to production. SQL Source Control and the DLM Automation Suite from Redgate Software does this, as do other comparison tools. What you would do is use a command line or programmatic interface to set your source and target, capture the output and then use this as an artifact in your release process. I might include a review of the artifact as a scripting choice in your flow.
Note, there are some problems that State based comparisons don't do well with. Renames, splits, merges, data movement, a few others. Some comparison tools have ways around this, some do not. Be aware this may be an issue. If you have a more mature database, perhaps not, but you should consider this. SQL Source Control allows custom migration scripts, which can handle these issues.
The other approach is a script runner or migration strategy where each change you make to a dev database is captured as an ordered script and a framework executes these in order, if they are needed. This is preferred by some people since you can see exactly what code will be executed at dev and deployment time. ReadyRoll from Redgate Software, Liquibase, Rails Migrations, DBUp, FlywayDB, all use this strategy.
Neither of these is better or worse. Both work, both have pros and cons, but really the choice comes down to your comfort level and preference.
Disclosure: I work for Redgate Software.
If deploy DB changes just mean using SQL Server Database Projects (.sqlproj files) with Team Foundation Build in Team Foundation Server.
There are several ways can achieve this:
Use MSBuild task with some arguments to publish your SQL project
during build.
Add a deploy target in your sqlproj file,run the target after build
completes.
Or add a "Batch Script" step in your build
definition to run "SqlPackage.exe" to publish the .dacpac file.
More details please refer to this blog: Deploying SSDT During Local and Server Build .
As for using TFS2015, you can also try to use SQL Server Database Deployment task.
Use this task to deploy a SQL Server database to an existing SQL
Server instance. The task uses a DACPAC and SqlPackage.exe, which
provides fine-grained control over database creation and upgrades.
I am adding continuous integration testing to an existing Visual Studio 2010 database project. Right now we have a build that deploys an 'empty' database [dbo].[MyDb] with just the reference data needed such as locales and countries. Right now this is performed using sql files containing insert statements that are run in the post deployment sql build task.
I now want to add another test deployment build that will deploy to another database on the same staging server as [dbo].[MyDb].[Test] with the same reference data but with generated test data that will have foreign keys to the reference data. Database integration tests are then run against that. Because the state needs to be restored for each test, this needs to be as fast as possible.
From what I've tried so far, to generate the test data using Visual Studio's data generation plan it seems I need to get the reference data to a form that can be read by the Databound generator so that it can generate the test data in a way that maintains referential integrity.
The possible options I can think of are:
Somehow get the data generation plan to read the reference sql files?
Change the reference sql files to csv files and change the original build to do bulk inserts
Combine the builds so that the MyDb database is always deployed first and set it as the sequential databound generator source for the test db.
Has anyone got a better approach or can point to a good guide?
I'm not an expert on build scripts so would like to take advantage of tools to do as much as possible. I want to keep things as a Visual Studio Database project but I also have a license for RedGate's SQL Tools if that would make the testing easier.
It appears that handling of reference data still isn't supported very well by database projects. This is confirmed by the comments on this post by Barclay Hill.
At the moment I've gone with the option of having a reference database and using that with a sequential databound generator. Since it doesn't change very often I just deploy it manually and have stopped short of having a whole separate project just for that as I've seen elsewhere.
Hopefully reference data handling will be added to SQL Server Data Tools at some point.
Is there a tool that will let me generate a single script containing all tables and views? Sql Publishing Wizard drops everything (so all data is lost) and recreates it. It does have an option to not drop, but in that case, it doesn't update tables that exist (if any columns have changed).
In SQL Server Management Studio 2008 you can right-click on a database in the object explorer, go to Tasks > Generate Scripts..., and that will give you the option to choose not only what object types you want to script, but whether or not you want to script the drop as well.
When you are making changes to existing tables, you should be writing alter table scripts to make the change and then putting them in source control like any other code. Then when you deploy a set of changes, you run the scripts you created for that deployment.
Otherwise, yes use SQL compare.
Should you look at the Red Gate products specifically SQL Compare. They'll handle any situation you could need concerning script generation and database synchronization. (You can get a trial license too, to try it out and see if it is what you need.)
Have a look at this tool can be used which has the capability to generate the create and drop scripts for the SQL server objects, provided in a configuration file.
This tool uses the same mechanism as SSMS tool uses to generate the script.
SQL Server Script Generator Tool (via C#)
Have a look at these:
WinSQL (Lite edition is free, other versions are reasonably priced + free trial)
OpenDbDiff (free)
You can also check out MyDbUtils which can create scripts for:
Stored Procedures
Functions
Views
Triggers
I have been developing in VB.NET and SQL Server 2008 for a while now, but haven't got into live installs yet. In the database system I used be on it had the ability to archive multiple tables into a .dga file, as it was called. I could then restore the .dga file into another database or on another server.
I'm looking for the easiest way to accomplish something similar in SQL Server.
If you want to transfer specific tables, then using Data Transformation Services (right click on the database in SQL Server Management studio and select "Import Data" and it will bring the dialog up for it). Of course, this assumes that you have both databases available to you.
If you are comfortable with replacing the database as a whole, you can easily backup the database and then restore it into a new one through SQL Server Management studio (or through calling the appropriate SP).
I would go for one of the following :
From MS SQL Management Studio, right click on the database / Tasks / Generate scripts
From Visual Studio, in the Server Explorer tab, "publish to provider"
Both will launch a wizard allowing you to export the tables you want the way you want (including data or not, creation scripts or not, etc etc.)
If you want to move tabless without data, the simpliest thing is to script the tables you want and run the script.
We script all our db changes and commit them to subversion and then run them as part of the deplyment process.
If you want to put the whole database on prod including data (scrub out test records first!), then do a backup and restore onthe other server.
For future changes, wescript all our db changes and commit them to subversion and then run them as part of the deployment process. There also are tools that look at the structural differnces bewteen the two servers and creates scripts. REd-Gate's SQL Compare is really good for this.
In addition to HLGEM's suggestions, you can look into SSIS if this is an ongoing process.