Release management not triggered when setting configuration to Release - msbuild

I divide this to two issues , but solving any one of them will solve the other.
Issue 1:
When I use the section at the bottom of the following picture and add "configurations to release", the build is not triggering release management.
I queried the build logs for hours, and saw that it stops the release when it founds out that the configuration "does not match current"
If ConfigurationsToRelease Matches Current
Initial Property Values
Condition = False
Final Property Values
Condition = False
Final Property Values
Condition = True
before more words, I will show a picture that sums it up:
the build definition and ms build logs
in a default case, where configuration to release is blank , release management would continue from this point and write in the logs "Release the build" (as a command that happened) and the build would trigger release management.
If you look at my tfs build configurations, you can see it's exactly the same as the upper regular ms build configurations , but still I get this error of a mismatch.
In any case I only have only one dialog to fill my configurations at in build definition as shown in the 'configuration dialog' in the above pic.
I succeeded to release once or twice this way in this same project the other day by adding release configurations, it somehow worked, but then stopped working (worked once or twice as a glitch I think like something is cached there) but 99% of the other attempts failed, it was always stopping my trigger from tfs to release since the first time I tried it a few months ago.
Has someone here experienced it? I looked in a lot of places and spotted only one guy that complained about it. His solution was to remove it (not exactly a solution)
Is there a build argument that can fix this? (/p:something=something)
Issue 2: if anyone can solve it in a way different than rm configurations , then I don't need issue 1 to be solved.
For any one who is interested why I even messed with release configuration section of build definition, it's because I want rm to wait for all transformations to happen before rm intervention, and this seems like a way to tell rm , ok dude , see there? you got 2 configurations to wait for their build.
The thing is that by default when that configuration section is blank, rm is getting in the way of tfs build and tfs build is getting in the way of rm, something like a circular wait. rm expects both transformed folders to exist in build outputs when tfs build is waiting for rm to finish it's run after the first build, tfs wants to continue building the second configuration (and transform) but rm is involved already , seeking for it's second configuration, breaking the build when fails to find it, hence the second configuration will never be created, while rm is still waiting for it and tfs build waits for rm and build breaks. confusing? read again and see the pic below cause it's interesting enough.
more info for clarity:
the next stage of RM is trying to get something from build outputs folder before it's already there.
for example , if i set Release build to true , it will only build the first configuration (a folder is created, picture below) , rm will succeed in first step (QA.Release), and continue strait to try and grab Release for its next stage, but it's not built yet by TFS Build , which waits for rm to finish it's weird intervention in the middle of tfs's build work . and like i said above, I'm sure I've seen it work once or twice in one of my attempted builds.
the tfs output folder whith release build flag on(only one transform happens) \ when off (all two transforms work) + rm error when on (circular wait)

If I understand your situation correctly I think your problem is that you are building multiple configurations. This breaks one of the core tenets of continuous delivery which is that for a given release you should build only once and deploy that same build to each stage in your pipeline, in sequence.
In order to do this your build needs to be stage agnostic, which in practice means all your configuration (eg database connection string) needs to be tokensised so that the correct value can be swapped in for the particular stage (QA, Release etc). I have a blog post series here that explains the full process (for a sample web application) in great detail.

i managed to solve issue 2 , and by doing so, issue 1 is no longer relevant .
i switched to ReleaseTfvcTemplate.12.xaml as a build template (found on your C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\Client\bin)
and then the build finished all configurations before RM intervention .
truly seems like a bug in ReleaseDefaultTemplate.11.1.xaml , or this might be because i was using some additional msbuild arguments (which were needed) , or the fact i'm using slow cheetah to create transforms on a windows service (transformations are only introduced in web applications).
either way , i'm now able to perform advanced tasks , like using the transforms to add \ remove tags that should be different in production , for example , i can leave the use of diagnostics in the configuration file for qa use , and remove it for production to make the verbosity lower there . still am using RM PlaceHolders technique in conjunction to transformation technology , to enjoy both worlds where it comes to changes related to environments , but still keeping the principles by passing the same build (dlls) through all environments .

Related

TFS2010 database size

We've been using TFS since around 2009 when we installed TFS2008. We upgraded to TFS2010 at some point and we've been using it for source control, work item management, builds etc.
Our TFSVersionControl.mdf file is 287,120,000 KB (273GB). We ran some queries and found that our tbl_BuildInformationField table is massive. It has 1,358,430,452 rows which takes up 150,988,624 KB (143GB). We have multiple active products over multiple active builds which more than one solution per build and the solutions aren't free of warning messages.
My questions:
Is it possible to stop MSBuild from spamming the
tbl_BuildInformationField table so much? I.e. only write errors and
general build information and not all the warnings for every
project?
Is there a way to purge or clean up old data from this
table?
Is 273GB for 4 years of TFS use an average size?
Is 143GB for tbl_BuildInformationField a "normal" size?
The table holds the values and output of build process. Take note that build retention policy doesnt actualy delete the build object like everything else in TFS the object is marked deleted and only public visibility and drop location is cleared.
I would suggest if you have retainened same build definitions for very long time (when build definition is deleted the related objects get removed as well) you should query for build info including deleted ones using TFS api, the same api will also alow you to remove them for good. Deleting build definitions probably will not work and will fail with timeout error.
You can consult the following:
http://blogs.msdn.com/b/adamroot/archive/2009/06/12/working-with-deleted-build-data-in-team-foundation-server-2010-beta-1.aspx

TFS Builds, Project Files: Orphaned references to files not being pushed are causing endless build errors

We are using TFS 2010 (Visual Studio) for our deployments and have client code projects (.csproj files) and database projects (.dbproj files) We understand that when our Developers add files to our application there is a corresponding reference to these files in the Project file. If I push a changeset from Dev to QA that includes the project file, and the project file contains a reference to a file that's been added that is not in the changeset, I will receive a build error.
Once we started pushing just changesets (as opposed to performing full builds) this quickly became our number one bottleneck in doing TFS builds. I would deploy the database project and there would be 20 errors. The only way I could correct them was to navigate down the entire solution explorer tree and exclude each orphaned reference individually. This has proved far too time consuming and on the advice of our lead programmer we have returned to doing full builds of QA and UAT.
We are in the early stages of this product, and therefore we will be adding many files for some time. We need a better solution for this problem. Neither the manual exclusions nor asking developers to not check in code until it is ready for qa will suffice for us. Has anybody out there had any experience with this problem and if so how did you deal with it? Thanks!
Jon
Pushing changesets to QA selectively is known as cherry picking and causes the sorts of issues that you are experiencing. This is not the recommended practice, instead setup the Qa build so that successful build is part of checkin. This way that if a part of a fix is missed ( as it may be in multiple change sets ) the build will fail and the checkin cannot be performed.
Second have the developers do the second checkin to QA or merge the dev change sets to Qa and have the team lead coordinate changes to project files by watching for changes by turning on "notify changes made by others " or setting a policy for the dev team. Full builds should always be done as partials do not always pick up the complete pick up the dependency graph.

TFS Continuous Deployment for Windows Service?

I have managed to do Continuous deployment for my Web project using TFS Msbuild.
I have goggled for few hours but couldn't find a relative link to achieve Continuous Deployment for windows service.
Possible to do CD for windows service using TFS Build Definitions? i.e for every check in below steps should be performed, I am using TFS2010 with Windows Server 2008 R2
1] Stop Service,
2] Copy respective Project folder from (Source) Build server to (Destination Server)'staging server1' or 'staging server2'
3] Start Services (willing to do this step manually)
Any blog,tutorial references to achieve this? My guess is need to use Power shell scripts but not sure.
Should be ok, you'll need to install an agent on the box you're deploying to. And you'll need to be able to exit the XAML templates (you'll probably want to copy your existing template that does your build and just add the stop/copy/start stuff onto the end of it).
After your CI build, you'll need to edit it (the XAML template) to start and stop the service you can use the "invoke process" activity (you'll probably want to do something like make it generic and pass in the service name as an argument - note you can change the display names etc in the Metadata argument so it appears meaningful in your build definition).
As far as copying stuff across goes, you can do this fairly easily by accessing properties like the drop location.
Should be fairly straight forward - once you get your head round modifying the templates!
Edit:
Sorry for not responding sooner, I'd have to revise my earlier comment, this isn't as straight forward as it seems unless you really know what you want, I have been thinking about this and like skinning cats, there are more than one ways to achieve this... I've rewritten this a few times so I hope the edit's make sense :)
Boils down to the following:
1) Pass into your template the build agent/machine you want to run this on (this can be done as a simple string, or as an AgentReservationSpec - up to you), since it's unlikely to be the machine that you run your actual CI build on. This is done in the Arguments section of the XAML, as noted before, if you want to edit the display name/description you can edit the Metadata Argument. This machine needs a TFS agent installed of course.
2) Run the task on the remote machine, this is done by adding the Agent Scope activity into your template, you will have to use the info from step 1 to get the ReservationSpec (so would be easier if you add the argument as an AgentReservationSpec or you'll need to resolve this in the template)
2.1) Run the stop/uninstall, this is done via dropping in a (two actually) Invoke Process activity, Invoke Process can take arguments and you need to point it to the executable you're executing, so you'll want to use this, one for the NET command (i.e. NET STOP ), and one for InstallUtil.exe.
2.2) Copy the files from your CI to the remote server, you can use the Copy Directory activity for this, it needs a couple of parameters, the main one is the source location, you should be able to drop in a GetBuildDetail activity, give it a name then reference .DropLocation to get this, destination is wherever you like you're installing to.
2.3) Install the new service as step 2.1, you need to use Invoke Process to install the service, then you can use another to start the service up.
I haven't covered everything, but I haven't set this up myself so I'm sure there are a few pitfalls or things I haven't though of. Off the top of my head this makes sense, but maybe someone that knows better can poke a few holes in it :)

TeamCity: Managing deployment dependencies for acceptance tests?

I'm trying to configure a set of build configurations in TeamCity 6 and am trying to model a specific requirement in the cleanest possible manner way enabled by TeamCity.
I have a set of acceptance tests (around 4-8 suites of tests grouped by the functional area of the system they pertain to) that I wish to run in parallel (I'll model them as build configurations so they can be distributed across a set of agents).
From my initial research, it seems that having a AcceptanceTests meta-build config that pulls in the set of individual Acceptance test configs via Snapshot dependencies should do the trick. Then all I have to do is say that my Commit build config should trigger AcceptanceTests and they'll all get pulled in. So, lets say I also have AcceptanceSuiteA, AcceptanceSuiteB and AcceptanceSuiteC
So far, so good (I know I could also turn it around the other way and cause the Commit config to trigger AcceptanceSuiteA, AcceptanceSuiteB and AcceptanceSuiteC - problem there is I need to manually aggregate the results to determine the overall success of the acceptance tests as a whole).
The complicating bit is that while AcceptanceSuiteC just needs some Commit artifacts and can then live on it's own, AcceptanceSuiteA and AcceptanceSuiteB need to:
DeploySite (lets say it takes 2 minutes and I cant afford to spin up a completely isolated one just for this run)
Run tests against the deployed site
The problem is that I need to be able to ensure that:
the website only gets configured once
The website does not get clobbered while the two suites are running
If I set up DeploySite as a build config and have AcceptanceSuiteA and AcceptanceSuiteB pull it in as a snapshot dependency, AFAICT:
a subsequent or parallel run of AcceptanceSuiteB could trigger another DeploySite which would clobber the deployment that AcceptanceSuiteA and/or AcceptanceSuiteB are in the middle of using.
While I can say Limit the number of simultaneously running builds to force only one to happen at a time, I need to have one at a time and not while the dependent pieces are still running.
Is there a way in TeamCity to model such a hierarchy?
EDIT: Ideas:-
A crap solution is that DeploySite could set a 'in use flag' marker and then have the AcceptanceTests config clear that flag [after AcceptanceSuiteA and AcceptanceSuiteB have completed]. The problem then becomes one of having the next DeploySite down the pipeline wait until said gate has been opened again (Doing a blocking wait within the build, doesnt feel right - I want it to be flagged as 'not yet started' rather than looking like it's taking a long time to do something). However this sort of stuff a flag over here and have this bit check it is the sort of mutable state / flakiness smell I'm trying to get away from.
EDIT 2: if I could programmatically alter the agent configuration, I could set Agent Requirements to require InUse=false and then set the flag when a deploy starts and clear it after the tests have run
Seems you go look on the Jetbrains Devnet and YouTrack tracker first and remember to use the magic word clobber in your search.
Then you install groovy-plug and use the StartBuildPrecondition facility
To use the feature, add system.locks.readLock. or system.locks.writeLock. property to the build configuration.
The build with writeLock will only start when there are no builds running with read or write locks of the same name.
The build with readLock will only start when there are no builds running with write lock of the same name.
therein to manage the fact that the dependent configs 'read' and the DeploySite config 'writes' the shared item.
(This is not a full productised solution hence the tracker item remains open)
EDIT: And I still dont know whether the lock should be under Build Parameters|System Properties and what the exact name format should be, is it locks.writeLock.MYLOCKNAME (i.e., show up in config with reference syntax %system.locks.writeLock.MYLOCKNAME%) ?
Other puzzlers are: how does one manage giving builds triggered by build completion of a writeLock task read access - does the lock get dropped until the next one picks up (which would allow another writer in) - or is it necessary to have something queue up the parent and child dependency at the same time ?

When a TFS build is triggered by a check-in what how does it do the get?

I have a CI process set up in TFS 2008 so that every time a check-in happens a build is triggered but only if a build is not already running. The exact setting used in the configuration is the radio button labeled:
"Accumulate check-ins until the prior build finishes (fewer builds)"
The checkbox "Build no more than every ___ minutes." is not checked
I have other build processes that might also be going on so I can see that a build is queued with a specific "Date Queued". When the build server gets to the queued build, let's say that it takes 10 minutes to get there, does it use the time-stamp of the queued build to do a get or does the build do a "get latest"?
I believe TFS will use the changeset of the last triggering commit (i.e. the build does a 'get specific version' by changeset ID, rather than a 'get latest').
You can check this by looking at the build report and finding the value for the 'Source control version' field, you should see something similar to this:
So first off, I would check that the value you're seeing for this is consistent with what you see in the build's workspace and that the build is referencing the expected changeset.
Assuming that your build is working as advertised (and this is a problem for you), then you may want to consider altering this default behaviour and force your CI build do a 'get latest' by overriding the 'GetVersion' MSBuild property in the build's TFSBuild.rsp file, by adding the following:
/p:GetVersion=T
Where 'T' stands for 'tip' (or 'head' in SVN parlance).