I have written and maintained a lot of Nant/Msbuild files, and have got a few problems in a current project where it seems that rake would make my life a lot easier. However I am a bit stumped on the first hurdle, but this may be because I am looking at the problem through Nant eyes.
To give some context here is how I normally expect my builds to be laid out (forgetting the actual physical structure, just look at what the components of it are):
- Root
| - build-scripts
| - default.build
| - *.build
| - build-properties
| - dev.properties
| - live.properties
| - *.properties
| - tools
| - Nant
| - Nant.exe
| - *.*
| - Nunit
| - Nunit.exe
| - *.*
| - **/*.*
Now as you can see above the main components of a build are the build scripts, which contain the actual build instructions, the build properties, which are files containing purely the properties per environment. i.e
dev.properties may contain web.service.url = "http://some.dev.address"
live.properties may contain web.service.url = "http://some.live.address"
Then there are the tools which are external executables used by the build scripts, such as Nant, Nunit, JsTestDriver etc.
Now focusing on my default.build file, this tends to contain a lot of upfront properties, such as directories used (i.e libs, output, package, project, tests) so they can all be evaluated up front. Then the other *.build files contain relevant build scripts, such as tests.build would deal with Nunit and run Unit/Integration/Acceptance tests etc.
The main problem for me here is that I have these environmental properties which configure the build for each environment, and then a lot of pre-defined properties which are overridden when invoked from the command line, i.e you would pass environment=live if you were building for a live environment, or if you left it blank it would default the environment to dev, and on the CI server it would set the environment to ci.
None of the examples I can find seem to tell me how to have default properties/variables that are overridden by the command line with Rake, I know you can set properties and get them out via the Env[] mechanism, but to use this I would need to set a global variable as normal then have a step which checks any env vars passed in and overwrite the global variables with those properties, also the only way I can see to allow external configuration files is to use global variables within them and include it, which I don't mind doing but I was hoping there was some best practises in this area I could learn from.
Also the final thing I was thinking about was that really the only variable to be passed in would be the environment (dev,ci,live etc) so I could make it so the default build task required an argument to be passed in, which is supported, but not sure if this is best as I would want it to run as "dev" if nothing was set, and this means you would always need to set one (not end of the world).
As you can see there are quite a few questions in this area, taking my existing approach and trying to adapt it to work with Rake. Any advice or information would be great!
This isn't the proper answer, but I have basically kept the same soft of structure as above and used global variables for everything. So I have a properties file which contains something like:
$dir["project"] = "c:/Some/Project/Dir"
$dir["tests"] = "c:/Some/Tests/Dir"
$settings["auto-migrate-deltas"] = true
It is not really ideal, but it allows me to inject different properties based on the environment, it also makes alot of the build files less re-usable as they all use these global variables. Using parameter based tasks may make it more re-usable but there isn't a huge deal of documentation in this area and working examples, so I am just going to stick with the following system until I find a major problem.
Related
My colleagues and I have user specific settings in csproj.user files. They are not checked into the repository. I would like for the build server to use its own set of csproj.user files, overriding certain properties, leaving the "base" project configuration at a decent developer default. But from the looks of it there is no such option in the msbuild command-line for doing that.
Is there really no way, other than copy csproj.user-files to where it'll be picked up by subsequent msbuild invocations?
While writing I realize I'm too much of a prude about these things and should just copy as a step prior build. Still posting in case someone knows a better way, for instance a way that does not modify the source tree.
Passing properties to the MSBuild command line overrides properties in the solution, including dependent projects. Here omitting debug information in build server, otherwise generated for release build to improve profiling:
msbuild MySolution.sln /p:DebugType=none ...
This does not work should I want different properties for different projects. Building projects individually should work nicely though.
Finally, passing arguments on command line can get messy, so to get a more "settings file"-like experience one may instead use #file arguments and MSBuild response files.
I read this post on the contents of a solution file, but still have no clue about the actual purpose of dependencies provided within a solution-file rather than within the project-file itself.
It seems there are two ways of having project 2 depending on project 1:
add a project-reference from p2 to p1. This will alter the csproj-file for p2 by introducing a ProjectReference to p1.csproj, but won´t change the solution, as far as I understand.
add an assembly-reference from p2 to p1. Thill will also alter the csproj-file by using a Reference to a compiled assembly (dll). However, it also adds a ProectDependency into the solution-file, which I do not understand. Why is this second entry within the solution needed in this case? Isn´t the assembly-reference provided within the csproj-file for p2 sufficient?
It's purely historical. The new project files don't really need it anymore, but the .sln file format predates msbuild and thus the solution file has some duplication.
It's used to define the build order, which becomes more important when you have ancient project types in your solution, as these won't be able to declare build order. It's also used to declare and validate build order between unrelated projects (e.g., project that don't reference each other), without the IDE having to load & parse all the projects.
Your second case is one of those cases where the solution file keeps track of build order. It then knows it needs to build P1 prior to P2. Without the solution level reference that information would be lost. It's quite clever that this is automatically detected and added, in the past you needed to manually define such build-order-dependencies.
At compile time the .sln is transformed into an msbuild file which then orchestrates the build. (see an example here). You can set an environment variable to generate yours.
<TL;DR>
The solution file is ancient and has some artefacts left over from pre msbuild. Some things just need to be there for 'reasons'.
If they were to build VS from scratch, the solution file would look very different.
My goal is to create build definitions within Visual Studio Team Services for both test and production environments. I need to update 2 variables in my code which determine which database and which blob storage the environment uses. Up till now, I've juggled this value in a Resource variable, and pulled that value in code from My.Resources.DB for a library, and Microsoft.Azure.CloudConfigurationManager.GetSetting("DatabaseConnectionString") for an Azure worker role. However, changing 4 variables every time I do a release is getting tiring.
I see a lot of posts that get close to what I want, but they're geared towards C#. For reasons beyond my influence, this project is written in VB.NET. It seems I have 2 options. First, I could call the MSBuild process with a couple of defined properties, passing them to the .metaproj build file, but I don't know how to get them to be used in VB code. That's preferable, but, at this point, I'm starting to doubt that this is possible.
I've been able to set some pre-processor constants, to be recognized in #If-#Else directives.
#If DEBUG = True Then
BarStaticItemVersion.Caption = String.Format("Version: {0}", "1.18.0.xxx")
#Else
BarStaticItemVersion.Caption = String.Format("Version: {0}", "1.18.0.133")
#End If
msbuild CalbertNG.sln.metaproj /t:Rebuild /p:DefineConstants="DEBUG=False"
This seems to work, though I need to Rebuild to change the value of that constant. Should I have to? Should Build be enough? Is this normal, or an indication that I don't have something set quite right?
I've seen other posts that talk about pre-processing the source files with some other builder, like Ant, but that seems like overkill. It feels like I'm close here. But I want to zoom out and ask, from a clean sheet of paper, if you're given 2 variables which need to change per environment, you're using VB.NET, and you want to incorporate those variable values in an automated VS Team Services build process upon code check-in, what's the best way to do it? (I want to define the variables in the VSTS panel, but this just passes them to my builder, so I have to know how to parse the call to MSBuild to make these useful.)
I can control picking between 2 static strings, now, via compiler directives, but I'd really like to reference the Build.BuildNumber that comes out of the MSBuild process to display to the user, and, if I can do that, I can just feed the variables for database and blob container via the same mechanism, and skip the pre-processor.
You've already found the way you can pass data from the MsBuild Arguments directly into the code. An alternative is to use the Condition Attribute in your project files to make certain property groups optional, it allows you to even include specific files conditionally. You can control conditions by passing in /p:ConditionalProperty=value on the MsBuild command. This at least ensures people use a set of values that make sense together.
The problem is that when MsBuild is running in Incremental mode it is likely to not process your changes (as you've noticed), the reason for this, is that the input files remain unchanged since the last build and are all older than the last generated output files.
To by-pass this behavior you'd normally create a separate solution configuration and override the output location for all projects to be unique for that configuration. Combined with setting the Compiler constants for that specific configuration you're ensured that when building that Configuration/Platform combination, incremental builds work as intended.
I do want to echo some of the comments from JerryM and Daniel Mann. Some items are better stored in else where or updated before you actually start the compile phase.
Possible solutions:
Store your configuration data in config files and use Configuration Transformation to generate the right config file base don the selected solution configuration. The process is explained on MSDN. To enable configuration transformation on all project types, you can use SlowCheetah.
Store your ocnfiguration data in the config files and use MsDeploy and specify a Parameters.xml file that matches the deploy package. It will perform the transformation on deploy time and will actually allow your solution to contain a standard config file you use at runtime, plus a publish profile which will post-process your configuration. You can use a SetParameters.xml file to override the variables at deploy time.
Create an installer project (such as through Wix) and merge the final configuration at install time (similar to the MsDeploy). You could even provide a UI which prompts for specific values (and can supply default values).
Use a CI server, like the new TFS/VSTS 2015 task based build engine and combine it with a task that can search&replace tokens, like the Replace Tokens task, Tokenization Task, Colin's ALM Corner Build and Release Tasks. And a whole bunch that specifically deal with versioning. Handling these things in the CI server also allows you to do a quick build locally at all times and do these relatively expensive steps on the build server (patching source code breaks incremental build in MsBuild, because there are always newer input files.
When talking specifically about versioning, there are a number of ways to set the AssemblyVersion and AssemblyFileVersion just before compile time, usually it involves overriding the AssemblyInfo.cs file before compilation. Your code could then use reflection to read the value at runtime. You can use the AssemblyInformationalversion to specify something like you do in the example above which contains .xxx or other text. It also ensures that the version displayed always reflects the information obtained when reading the file properties through Windows Explorer.
What reasons could there be for the following strange behaviour, and how might I track down the issues?
We use a combination of make files and msbuild.
I have a project which needs to be strongly named. I was previously setting the snk to use in the project file like this:
<AssemblyOriginatorKeyFile>$(EnvironmentVariable)TheKeyName.snk</AssemblyOriginatorKeyFile>
where EnvironmentVariable was defined in the batch file that launched the shell for the build like this:
set EnvironmentVariable='SomePath'
and this worked ok. Now I need the string name key to be able to be changed, so it can be different on the dev machine and the release build server. There is a variable which exists to hold the full path to the strong name key file, called StrongNameKeyFile. This is defined in the msbuild environment, and if I put some text output in the targets or properties files that are included as part of the msbuild task which build the project then I can see that this StrongNameKeyFile points to the correct location. So I changed the csproj to have this instead:
<AssemblyOriginatorKeyFile>$(StrongNameKeyFile)</AssemblyOriginatorKeyFile>
but when I try and compile this is evaluating to empty and no /keyfile is specified during the build.
We also have variable defined in the make files and these can be accessed in the csproj as well. These are used to point to the locations of referenced dlls, so that they can be different on dev and build machines. I know that these are set as the references come out correctly and everything compiles, but if I try and use one of these variables in the AssemblyOriginatorKeyFile element then it evaluates to empty in that element, but works in the reference element.
Why might this be? Is AssemblyOriginatorKeyFile treated specially somehow? How can I go about tracking the cause of this down?
There's no good reason why this should happen - as you know it normally Just Works; it's likely to be something in the chain dropping it on the floor.
One thing to try is explicitly passing it via /p:StrongNameKeyFile=XX - that would eliminate environment variables and the correct propagation thereof from your inquiries.
Another potential thing is that something is clobbering the variable as the name is used vy something else?
Run with /v:diag and you'll get dumps of all the inputs and/or variables as they change.
Or if on V4, use the MSBuild Debugger
And buy the Hashimi et al MSBuild book
I'm starting to create a MSBuild scripts for my products, and I've encounter a dilema.
The code is divided into around 25 projects, some wll require obfuscation, some will require strong-name signing; others will require linking into a single file.
All these projects should result in 3 products, with 3 setups.
The question at hand is as follow: How do I divide the MSBuild scripts to make most sense?
Do I create a script for each product? do I create a script for each project? Do I have one script for building, another for obfuscation and so on?
I think this is good idea to have script per product.
To minimaze dublication create reusable "sub-scripts" and import them to main script (this could be done with Import directive).
<Import Project="..\Steps\Step1.proj" />
Script per product sounds like the way to go. You may want to consider having any number of shared or base scripts too, to import common build steps. Like Mike Chaliy already mentioned you can then use Import in your product's build script:
<Import Project="..\Shared\Base.proj" />
Another thing you might also want to take advantage of is target and property overriding. It's akin to overriding virtual methods in a .Net class. See the documentation and the MSBuild Team Blog for more details. I know I've taken advantage of this quite often by setting defaults in the included scripts then overriding them as necessary in the product build script in order to customize build behaviour. For instance I often have generated files that are required before the build so I hook those targets into the BuildDependsOn property group. This way my generated files are generated whenever I do an F5 from the IDE, call the build target from the command line or otherwise build the project or solution. Obviously if you have any build steps that run long or only need to be run in special circumstances (like building installers), you'll want to take care about exactly what gets hooked in.