Related
I manage a small team of developers who up till recently have all been working on independent projects.
We have now all come together to work on one holistic project and its really tough. People are changing things, without consulting other programmers and its very difficult to manage.
We are also working in a pure production environment (no test/production)
I am a bit lost on where to start, I am looking at products like Team Foundation Server or their is a product by Red gate called "Source control"
Does anyone know any good books on the above subject, I cannot be the only person in this situation? :)
Source Code:
If you're looking for how to use source control (versus how to get your team to a controlled source environment) you should check out Source Control HOWTO by Eric Sink. The disclaimer here is that he has written a source control product but the HOWTO isn't specifically tailored to it.
Database Schema:
If you're looking for how to get your database (structure not data) in source control (since it was mentioned in the title) then you should check out this Coding Horror post: Get your database under version control. It covers some basic concepts and points to a few articles on how to actually implement getting your database in source control.
Source Control Software:
If you are looking for actual source control software, a list of applications can be found here: List of Source Control Systems with Visual Studio Plugins.
Don't make the mistake I did when putting the database definition under source code control by allowing branches and merges. This led developers (and managers) to assume they could have several versions of a table in the same database, leading to programmers overwriting each others changes, and having incompatible versions of source code and table definition all over the place.
As you might well imagine this lead to a great deal of wasted time and many frayed tempers.
Depending if time or money is more important, do you have the option to being someone in for a short time with the specific task of improving your processes? They would be able to set you up a test environment, install a version-control server and get your guys (and you) up to speed.
The first thing that you need to do is to give every developer their own copy of the database.
Secondly, you need each dev to be able re-create a baseline database from the master.
Once each dev is happy with their changes to the database, you will then need them to create a series of patches for their changes.
Lastly, each one of these patches needs to be able to checked in to source-control, and re-distributed to the other devs.
If you are looking for a good source control server, consider Subversion, with Tortoise SVN as your client.
Take a look at DBSourceTools. (http://dbsourcetools.codeplex.com).
It's specifically designed to help developers use source control for database changes.
This tool will allow you to baseline your database at a specific point, and create a named version (v1).
All files are scripted to disk, and very easy to add to a Subversion repository.
Have a look at the patching process for creating and using patches.
Have fun.
I'm wondering if anyone is currently utilizing Microsoft's Master Data Services? How you are utilizing it? Whether you find it useful? When you believe it would be useful? Thanks!
I have been working with MDS since it was first released as part of a feature pack for SQL Server 2008 R2. While MDS has some compelling features - most notably detailed data lineage, I am not confident in recommending it to clients yet.
My reason for this hesitation is the nature of the install and the tendendency to fail on upgrade or system change. I struggled mightily with the both the SQL Server 2012 RC0 MDS and the RTM installs. There are simply too many brittle aspects of the install (such as the hard requirement that the service be installed on a domain-joined machine and the need to install the Silverlight 5.0 SDK for the client to work properly). I also experienced flakiness in the the Excel add-in.
I see where Microsoft is going and I think the product will eventually be useful. Considering it's purpose (master data repository), MDS must be more 'rock solid' before I would use it in production.
We aren't using it currently in our office, however the presentation Microsoft did in town a while back seemed very interesting. I saw it as sort of a competitor to Oracle's OBAW warehouse. You've probably already looked at these, but Microsoft has a decent set of webcasts that cover how to install and use MDS out here:
http://www.msdev.com/Directory/SeriesDescription.aspx?CourseId=155
I'm anxious to see if anyone else is using it as well, we tend to have a hard time talking our management into letting us try these types of services without being able to point to other corporations that have successfully implemented said product.
We're just starting to investigate the use of MDS to support our consulting practice, specifically around data analytics and ETLs to deduplicate, standardize, and sanitize client data. It's probably just scratching the surface of MDS, but we were led to MDS initially for its inclusion of regular expression capabilities in SQL to transform free-form text data.
Before MDS/DQS, part of the sustainability / enduring-success of a custom database application was heavily dependent on one or both of the following items...
Having a full-time technical resource to manually update the master data. Someone who can work with the Business Experts and make the necessary adjustments to the data in the database.
Developing (in addition to the database/application/etc) a custom UI that is intuitive enough for the less-technical Business Experts to use for managing the master data themselves.
Neither of these were ideal from a cost-perspective. With MDS/DQS, a developer/contractor can come in, design an end-to-end Data Warehouse/BI solution including full integration with DQS/MDS (probably via SSIS packages) with relative ease. The Business-Experts can be trained to manage the master data using a UI they are already very familiar with (ex. Excel), and the developer/contractor can move on to the next project/client.
Also, if the business already has other data sources (via acquisitions or silo'd-yet-overlapping efforts or whatever), MDS can be used to manage all the master data in one centralized location.
It might not be the best MDS product available yet however it does come with SQL Server. Compared to most of the bespoke efforts for accommodating meta-data or master data in warehouse loads it's a pretty good option since most of the time is spent concentrating on the warehouse and the mastering of ancillary or other data isn't normally well accommodated for leaving questionable results. I prefer to use it than create some other flaky option that the customers will find it difficult to maintain. If you have budget however I would consider looking around for something more polished.
Like anything though give master data the respect it deserves. If it is going to be used then it's worth spending the time to model the entities, flow of data and usage correctly. The data stewards will need to savvy and will require training (it's not the most usable interface in the world - to say the least).
As we are a small consulting and development company we don't use MDS internally but we do implement it at customers with a focus on managing the Golden Record as the customers have a myriad of databases and applications all using the same data (customer, product …)
I agree with Lynn Langit's comment about installation and SilverLight dependency and the general comments about the UI. There are also a lot of smaller companies that don't run SQL Enterprise Edition but whom could benefit from MDS.
Those are the reasons why we are now developing a modern web application which we will host for our customers (probably on Azure).
If you're thinking about MDS I'd recommend to have a look at the API to replace (parts) of the UI.
Master Data Service is very useful for managing Master Data,
We have used Master Data Services 2012 and 2016, there are not too many features present in 2012 ,2016 is much better than 2012 with some new features , but I think still Microsoft needs to improve Master Data Services, they should include some flexibility in business rule's area.
My development group uses Visual Source Safe for version control; this choice was originally made due to cost and its tight integration with Visual Studio.
As our repository has grown Source Safe has really started to show its limitations and we are considering moving to another solution. Up for discussion are Team Foundation Server, Subversion, Git, and Mercurial.
We are largely a data shop, so another major factor for us is being able to easily version SQL Server 2005/2008 projects. This is one of the benefits of using Source Safe, and also of Team Foundation Server - the integration with Microsoft SQL Server Management Studio.
I'm wondering if anyone has had experience versioning SQL Server with Subversion, Git, or Mercurial and can provide some solid pros/cons for each of these systems, as well as how you went about implementing them.
My honest answer is don't do any integration with your database tooling and SCM if you can avoid it. Use the filesystem where possible. It's another layer of integration which is going to be a pain. Small separate tools are better than a behemoth.
We use Subversion and SQL 2005 together in the following manor:
We use TortoiseSVN only. No VS/SSMS integration at all.
We have a principle of "automate everything", so we never rely on GUI tools to do work.
We keep all scripts inside SVN along with the code. The code, schema and scripts are versioned together.
Schema changes are numbered in order of application i.e. 000-create-table-users.sql. We write down the maximum script number deployed in each environment. Each script performs a migration to the next database r number. When we deploy, we check out the source and run all scripts from the last version number to the highest number.
Any non-schema scripts (sprocs/views) are idempotent (can be executed any number of times with the same result). They are applied via a nant plugin we wrote. These are replaced every time we deploy. Don't forget to refresh your views!
We avoid any scripts where possible anyway as we use NHibernate so there are less problems with script versioning anyway.
From this structure, we can re-create the environment and database at any point in time on any machine which is important.
We do NOT use it for unit testing however - we rely on the NHibernate schema generation to do this on top of an SQLite database.
The only negative point we've encountered has been making sure that developers adhere to the process. Herding cats is a very appropriate description.
Visual Studio Team System 2008 Database Edition (codename "DataDude") is what you need.
It allows you to version your database objects in ways that will blow your mind. (eg upgrade a customer site to a specific version, or rollback to a previous version without destroying any data).
Check out the features at Gert Drapers' blog, starting with this post.
Or if you prefer a podcast, listen to DotNetRocks with Chris Sells in show 494.
I don't know whether you're limited to TFS for source control, when using DataDude -- but it is the undeservedly "underhyped" member of the Visual Studio family.
This might be a useful tool for you:
http://www.liquibase.org/
It's designed so that it's easy to version control in any system, and manages your upgrade scripts in a sane way.
Git and Mercurial are the only ones you should consider IMHO, the other 2 are too old-fashioned. Modern SCMs should treat branches like git does.
For git vs. mercurial comparisons see:
http://rg03.wordpress.com/2009/04/07/mercurial-vs-git/, http://www.russellbeattie.com/blog/distributed-revision-control-systems-git-vs-mercurial-vs-svn.
I have no past experience with SSMS SCM integration though, but AFAIK neither of the systems mentioned (except from TFS) have one. I wouldn't call it a disadvantage tho - git GUI for example is a pretty handy tool, which you'll find more enjoyable than such an integration. This is at least my case when moving from SVN (with VS integration using Ankh) to Git (with no integration at all)...
Mercurial has VS integration with VisualHG, if you think DVCS is the way to go. We use that for C++/C# projects in our shop, and it works well enough. (OTOH, I've never used any "full" integration, so I'm happy to work with the explorer extension and/or command-line for detailed VC work.)
We've now added VSS support to SQL Source Control, which integrates with SSMS to provide fully integrated source control for database development. To try this out, please visit:
http://www.red-gate.com/MessageBoard/viewtopic.php?t=12265
TFS is missing a few features of VSS, notably keyword expansion. If you don't embed revision keyword info within your source files, then it should not be a concern.
There are potentially quite a number of alternatives - SQL Server Management Studio (SSMS) supports integration with any Microsoft Source Code Control Interface MSSCCI Provider. So you can broaden the search to source control systems that feature an MSSCCI compatible provider.
In SSMS, Check out Tools -> Options -> Source Control to see what provider plug-ins are installed on your system.
For example, Team Foundation Server's integration with SQL Management Studio is courtesy of the TFS MSSCCI Provider. I think there's a provider for CVS/Subversion ("Aigenta Unified SCC") and so on.
As to a pros/cons list, I think provided there's a compatible provider, you can open the question up to a wider audience. My main experience is with VSS, TFS and Subversion. It really comes down to your team, and environment. Can you elaborate more on your environment?
E.g.
would you be interested in establishing CI (continuous integration)?
automated builds/automated versioning?
support for multiple environments?
configuration management?
what team size do you have? likely to have lots of merges/branching etc?
do you have a bug tracking system in place already (you get work items/bug tracking as part of a TFS roll out)?
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Okay, I will shortly be starting down the path of windows mobile development. I know nothing about the subject really and I am looking for people with experience to let me know of any gottchas you may know of.
Right now I dont even have a breif of what is requied but the assumption is that the application will be very little more than a bunch of CRUD forms for updating data. The only other requirment knowladge I have is that the application will need to support offline storage when there is no signal avaliable. This in turn will obviously require some kind of syncronization when signal returns.
My initial thoughts are that the application will primarily be a front end to interact with a web service layer. Im assuming that WCF will be an appropriate technology for building these services? I also thought that SQL Server CE would be a good route to go down with regards to the offline storage issues.
Any knowlage that you feel is useful within this domain would be appreciated. Advice, links, books anything appreciated.
EDIT: It has been noted that there are two ways to go with off-line synchronization. To either use some form of message queuing or to use SQL synchronization tools. Could anyone offer a good comparison and introduction to these?
EDIT 2: After a little more digging I get the impression that there are basically 3 different approaches I can use here:
Emmbeded Database to query against then syncronization online, when able
MSMQ along with .NET remoting
WCF with ExchangeWebServiceMailTransport bindings using Exchange Server.
Now, there has been a nice few points raised on the first issue, and I think I understand at some level the issues I would face. But I'd like to get a little more information regarding MSMQ implementations and using WCFs new bindings.
Here a few words from my experience so far (about 9 months) of .net Windows Mobile development.
Well you are occasionally connected. (Or more likely occasionally disconnected). You have to choose whether you are going to use messaging with queues (i.e. WCF/SOAP/XML or something like it) or database synchronisation. I choose the SQL synchronisation route so I can't really comment on messaging. The SQL synchronisation route is not hassle free!
If you go down the sync route with SQL compact like me you basically have two choices. SQL Server merge replication or the newer ADO.NET Synchronisation services. If you choose the former you need to be really careful with your DB design to ensure it can be easily partitioned between mobile subscribers and the publisher. You really need to think about conflicts, and splitting tables that wouldn't normally be split in a normalised DB design is one way of doing that. You have to consider situations where a device goes offline for some time and the publisher DB (i.e. main DB) and/or a subscriber alters the same data. What happens when the device comes back online? It might mean resolving conflicts even if you have partitioned things well. This is where I got burnt. But SQL Merge Replication can work well and reduces the amount of code you have to write.
Roll your own DAL. Don't attempt to use datareaders etc. directly from UI code and don't use typed datasets either. There may be third party DALs that work with Windows Mobile (i.e. I know LLBLGEN does, might be worth a look) but Linq-to-SQL is not supported and anyway you need something lightweight. The chances are the DAL won't be too big so roll it yourself.
If you are using .net you'll probably end up wanting some unimplemented platform features. I recommend using this inexpensive framework to give you what your missing (especially as related to connectivity and power management) - http://www.opennetcf.com/Products/SmartDeviceFramework/tabid/65/Default.aspx
Windows Mobile devices partially switch off to save power when not in use. If you are doing a polling type design you'll need to wake them up every x mins. A normal .net timer class won't do this. You'll need to use a platform feature which can be used from OpenNetCF (above). The timer class is called LargeIntervalTimer and is in the OpenNetCF.WindowsCE assembly/namespace (I think).
Good Luck!
SqlCE is only one of the options available for local data storage on a Windows Mobile device, and although it's an excellent database it has limitations. For one thing, SqlCE will not work (period) under encryption (in other words, if your user encrypts the location where your SDF file is, you will no longer be able to access the data).
The second (and most critical) weakness of SqlCE lies in the RDA/Merge Replication tools. SqlCE Merge Replication is not 100% reliable in situations where the network connection can drop during replication (obviously very common in Windows Mobile devices). If you enjoy trying to explain missing or corrupted data to your clients, go ahead and use SqlCE and merge replication.
Oracle Lite is a good alternative to SqlCE, although it too doesn't work properly under encryption. If encryption is a potential problem, you need to find a database engine that works under encryption (I don't know of one) or else write your own persistence component using XML or something.
Writing a WM application as a front end that primarily interacts with a web service in real time will only work in an always-connected environment. A better approach is to write your application as a front end that primarily interacts with local data (SqlCE, Oracle Lite, XML or whatever), and then create a separate Synchronization component that handles pushing and pulling data.
Again, SqlCE merge replication does this pushing and pulling beautifully and elegantly - it just doesn't work all the time. If you want a replication mechanism that works reliably, you'll have to write your own. Oracle Lite has something called a snapshot table that works very well for this purpose. A snapshot table in Olite tracks changes (like adds, updates and deletes) and allows you to query the changes separately and update the central database (through a web service) to match.
This thread I just posted on SO a few days ago has proven to be a great resource for me thus far.
Also the Windows Mobile MSDN WebCasts are a wealth of information on everything from just getting started up to advanced development.
I would suggest Sqlite for local storage. From the last benchmark I ran it was much better than SqlCe and you don't have to do stupid things like retain an open connection for performance improvements.
Trade-offs being that the toolset is less rich and the integration with other MSSql products is nil. :(
you might want to refer to this:
getting-started-with-windows-mobile-development
You shouldn't be intimidated for windows mobile development. It isn't much different from desktop development. I strongly recommend that you use .NET Compact Framework for development and not C++/MFC.
Some useful links:
Mobile section at the Code
Project. You would find a lot of
articles, a little digging is needed
to find the appropriate one.
Smart
Device Framework from OpenNetCF
offer valuable extensions to the
compact framework.
When you install
the Mobile SDK, you will find under the
Community folder links for the
Windows Mobile and CF framework
blogs. These are also valuable
resources.
Regarding your application, you are right about the WCF and the SQL Server CE. These are the proper ways for handling communication and storage.
Some hints for people coming from a desktop world:
You need to have some sort of power management. The device may automatically go to suspend state. Also, you shouldn't consume power when you don't have to.
Network connectivity is a difficult issue. You can register notifications for when a specific network (Wi-Fi, GPRS) becomes available or unavailable. You can also set the preferred means of communication.
Make the UI as simple as possible. The user uses his thumb and/or a pen and he is probably on the move.
Test in a real device as early as possible.
"24 Hours of Windows Mobile Application Development" from the Windows Mobile Team Blog has some good resources
If you can, try to start from the user use cases and work back to the code, rather than vice versa. It's really easy to spend a lot more time working on the tools than working on the business problem. And thinking through user requirements will help you consider alternate strategies, because a lot of the patterns you know from normal .NET don't apply.
I've done lots of intermittent application development of exactly the type you are describing, and an on-board database works just fine. The MSMQ/WCF stuff just adds conceptual overhead without adding much value. You need a logical datastore locally anyway, and replication at this level is a simple concept that you want to keep simple, so the audit trail is easily monitored and debugged. MSMQ and WCF tend to hide things in unfamiliar places.
I upvoted the SqlLite suggestion BTW. MS doesn't have their persistence story stabilized yet for CE.
For the database replication bit I highly recommend Sybase Ultralite. In terms of flexibility and performance it knocks the socks off SQL CE
I had to do this once. Weird setup with Macs for development, and we were all Java programmers. And a short deadline. PowerPC macs too, so no chance to install Windows for Visual Studio development, never mind that the money for this would never have appeared.
We ended up writing applications using Java, running on the IBM J9 virtual machine, with SWT for a user interface. Entirely free development stack. Easy to deploy. Code ran on any platform we desired, not just PocketPC/WinMob.
Most of the work was on the server side anyway - the database, the web service server. The logic. The reporting engine. The client side wasn't totally simple however - would get the form templates from the server (because they changed frequently), the site details (multi-site deployment), generate a UI from the form template (using some SWT GUI components that are wonderful for PocketPC development, like the ExpandBar), gather data with a point and click interface (minimising keyboard entry where possible), and then submit it back to the server.
For offline storage we used XML files on the device itself. More than enough for our needs, but yours may differ. Maybe consider SQLite?
There are a couple links you can check out to start with:
http://developer.windowsmobile.com
http://msdn.microsoft.com/en-us/windowsmobile/default.aspx
If you have a sticking point while developing, there are also Windows Mobile dedicated chats on MSDN that you can attend and ask your questions. The calendar hasn't been updated yet, but the next ones should be in January. You can find the schedule here: http://msdn.microsoft.com/en-us/chats/default.aspx
I am going to add an additional question to this post, as its been active enough and hopefully will be helpful to others as well as me. Ok, so after playing around I now realize that standard class libraries cannot be included in windows mobile applications.
Now the overwhelming advice here seems to be use an embedded database, though I now do have use cases and it appears that I will need to have document synchronization as well as relational data. With this in mind service layer interaction seems inevitable. So my question is how would I share common domain objects and interfaces between the layers?
"Document synchronization" - does that mean bidirectional? Or cumulative write-only? I can think of mobile architectures that would mainly collect and submit transactions for a shared document - if that's your requirement, then we should discuss offline - it's a long (and interesting) conversation.
Owen you can share code from Compact Framework -> Desktop, it's only Desktop -> Compact Framework that has compatability issues if you use certain objects that are not supported by the CF.
While a desktop lib doesn't work on CF a CF lib WILL work on the desktop, you can also run CF.exes on the desktop!
Just create a CF library as the project that defines your base objects / interfaces etc.
This book sshould e essential reading for all Windows Mobile developers: http://www.microsoft.com/learning/en/us/books/10294.aspx
For developing windows mobile applications you must have the basic tools like silverlight, visual studio, windows phone emulator and sqlite as your database storage.
Assume your .NET-based development team is already using the following set of tools in its processes:
Subversion / TortoiseSVN / VisualSVN (source control)
NUnit (unit testing)
An open source Wiki
A proprietary bug-tracking system that is paid for
You are happy with Subversion and NUnit, but dislike the Wiki and bug-tracking system. You also would like to add some lightweight project-management software (like Fogbugz/Trac) - it does not have to be free, but obviously cheaper is better.
Can you make a compelling argument for adopting VSTS, either to add missing features and replace disliked software or to handle everything (including the source control)? Is the integration of all these features greater than the sum of the parts, or would it simply be better to acquire and replace the parts that you either do not like or do not have?
I remember looking into VSTS a few years ago and thought it was terribly expensive and not really better than many of the free options, but I assume Microsoft has continued to work on it?
VSTS is great, if you do everything in it. Unfortunately the price has not become better over the years. :( The CAL's are still ludicrously expensive. The only improvement is that if a person uses only the work item system, and works only with his/her own work items (no peeking at other person's work items!) then there is no need for a CAL. This makes it a bit easier to use it as an external bugreport system. Still it leaves a lot to be desired in this area.
There is one way to alleviate the cost - become Microsoft Certified Partner. If you are a simple partner, you get 5 VS/TFS licenses for free; if you are a Gold Certifiend Partner, you get 25 (if memory fails me not). That should be enough for most companies. But getting the Gold status might be tricky, depending on what you do.
If you only dislike those two parts, then perhaps it's better just to find a replacement for them instead for everything? There are many wiki systems out there, some should be to your liking. The same goes for bugtracking too.
We are extremely happy with not only the tools, but the integration that Team Foundation Server, and the various Team Editions have given us. We previously used Borland's StarTeam for source control and issue tracking with a 3rd party wiki, the name of which escapes me at the moment.
It came time for us to extend our licensing and support agreement with Borland, only to learn that the cost of adding users to our license and upgrading the product would cost us as much (a little more, actually) than biting the bullet and making the switch. One thing to consider is that you would normally pay for the development tools to begin with, so the cost is partially absorbed by our budget.
We also did not feel the need for getting Team Suite for every person. You might want to consider it for the developers, but other disciplines don't really have a benefit in using all of the tools in most companies.
We were able to get the appropriate team editions for twelve people, enough CALs for 50 users (for Team Explorer, Teamprise, Team Project Portals, Team Web Access), Teamprise for the five Mac Users that we have, and the Team Foundation Server software itself for under six figures. Considering that includes the developer tools that we normally would be buying, it was a good deal.
The upfront cost on new licensing also covered two years, so we could split the budget between the 2008 and 2009 fiscal years. The very important thing is to make sure not to let the licenses lapse, as the renewals on licenses cost a fraction of the initial cost and also include version upgrades.
As to the features, we are in the process of rolling out. About half of our department completed training, and I have already started migrating projects over. The development team absolutely loves the features and tight integration with their workflow. Version control is a snap, and work items (and their related reporting artifacts) are extensible to the nth degree. The fact that TFS relies heavily on bringing sanity to workflow management helps to tie in all of the processes to a level that you just can not get with multiple vendors.
My absolute favorite thing, though, is the extensibility model. Using the Team Foundation Server API, you can easily write check-in policies, write tools to interface with the system, develop plug-ins, and more. We are already seeing gains in productivity and the quality of our products through a minimal implementation.
Still on the horizon, though, is integrating Team Build. I have yet to set up a build project, but it seems to be seamless and painless. Time will tell... :-)
Edit - I forgot to mention that our migration to TFS includes licensing for the Test Load Agent. The load testing functionality within Team Test is one of, if not the absolute best that I have seen.
Where I'm at, we've settled on the following:
SVN for source control
Redmine for bug-tracking and wiki
NUnit for unit testing
CruiseControl.NET for our build server
Redmine is an open source Ruby on Rails application that supports multiple projects much better than Trac and seems to be much easier to administer. It's definitely worth checking out.
VSTS seems to be way too much money compared to other products. As an additional benefit, you also get the souce with open source solutions, which allows you to modify things to fit your need if the capability isn't there yet.
I'd stick with SVN and use trac or bugzilla or fogbugz. You could also do a trial of team server. In my opinion it is not worth the money. MS had their chance with version control and they screwed it up a long time ago. Too late to the party if you ask me and frankly I am not impressed with how they try to control all your development experience in the IDE with "integration" to the source control. I prefer the perforce/SVN and separate defect tracking solution.
With all that said, you probably can't go wrong with any of the following:
bugzilla or trac or fogbugz AND SVN
MS team thingamabob