What is the canonical way now to write a custom Target in an sdk-style project to perform operations on files in $(OutputPath) in the context of a multitargeted build? I'm trying to multitarget our build for a migration to net5 and am surprised how messy this is. We have various targets that need to reach into $(OutputPath) to copy files or run exes, etc as part of the build. For instance, a target like this:
<TargetFrameworks>net48;net5</TargetFrameworks>
<AppendTargetFrameworkToOutputPath>true</AppendTargetFrameworkToOutputPath>
...
<ItemGroup>
<MyTargetInputs Include="$(OutputPath)\Foo.exe">
</ItemGroup>
<Target Name="DoPostBuildStuff" AfterTargets="Build" Inputs="#(MyTargetInputs)" Outputs="#(MyTargetOutputs)">
...
</Target>
This worked fine in an sdk-style project without multitargeting as $(OutputPath) is set to bin\debug which is indeed where Foo.exe is. But with the above the ItemGroup gets evaluated before AppendTargetFrameworkToOutputPath does its work to make $(OutputPath) set to bin\debug\net48. So now my ItemGroup is still looking for Foo.exe in bin\debug instead of bin\debug\net48 (this is true even in the inner build where $(TargetFramework) is defined). As an alternative to relying on AppendTargetFrameworkToOutputPath I tried to define $(OutputPath) myself in a Directory.Build.props as bin$(Configuration)$(TargetFramework) but because $(TargetFramework) is not set by the sdk until after my csproj contents are evaluated the same problem occurs (paths in my ItemGroup that use $(OutputPath) evaluate to bin\debug\). The same issue obviously occurs for any PropertyGroup items too.
I can devise some workarounds but they all seem a bit hacky (ex define a preceding target whose only job is to define ItemGroups for inputs/outputs of future targets so that $(OutputPath) is constructed completely by the time it runs). I'm surprised the docs for multi-targeting did not seem to mention that using this feature really messes with your ability to ever refer to build artifacts as part of the build process?
Related
An extension of this question, but slightly different, and the accepted answer does not quite work for this situation.
We've got a process in place on the build of our project which is generating some additional files, these files are getting (correctly) generated into the /bin folder as expected. However they are not getting copied across when this project is referenced as a dependency.
Following the above questions accepted answer (with a little bit of tweaking), I managed to get them copying across to the dependant project however they are all getting put into a /bin sub folder of the dependants /bin folder (i.e. /bin/bin), which is not what I need to have happen.
The process we're running is a 3rd party process (specifically Surviveplus.XmlCommentLocalization), so I have no control over that side of it.
I could do something additional on the dependant project to move them out of the /bin/bin into the level up, but I'd rather have the original project work as I'd expect it to.
This is the ItemGroup I'm using, derived from the other question:
<ItemGroup>
<Content Include="$(OutputPath)\**\*\*.xml">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>
</ItemGroup>
I've also tried setting specifically the TargetPath value, which while un-documented (or I'm blind to it), seems to exist - as per the msbuild output log
<ItemGroup>
<Content Include="$(OutputPath)\**\*\*.xml" KeepMetadata="TargetPath">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<TargetPath>..\</TargetPath>
</Content>
</ItemGroup>
But it appears that when it comes to the Copy task it just ignores it, and resolves a new TargetPath. I've also tried a myriad of combinations of attributes/item types in that item group (i.e. None, EmbeddedResource) but they didn't solve it either.
Ideally I suppose MSBuild needs to mark the generated files as part of the generated assembly? But after getting lost in .target files and MSDN docs, I couldn't figure it out.
MSBuild being used is version 12.0, compiling for .net4.5.
My organization has some huge builds that run on build servers, building lots and and lots of MSBUILD projects that are linked with ProjectReferences. We need to be able to build projects and configurations in parallel with msbuild /m.
My problem is that I have a project that is referenced from a large number of other projects, but the project itself is not reentrant. If more that two or more nodes try to build that project in parallel, it will fail.
How can I wrap this one Project, or it's Target, inside a critical section?
What I really need to do is something like this:
<Target>
<EnterCriticalSection ID=$(ProjectGuid) />
<Exec something />
<LeaveCriticalSection ID=$(ProjectGuid) />
</Target>
The idea being that if multiple MSBUILD nodes tried to build this project in parallel, only one of the nodes could do the execution, and the rest of the nodes would have to wait (or go do something else).
I suppose I could write custom MSBUILD tasks to do this, but isn't there some way of doing this built into the MSBUILD system?
=== EDIT 4/5/13. To clarify, the project is building a 3rd-party library with the build script provided for it. Completely rewriting their build script to make it reentrant -- by insuring that each build used a different set of folders for intermediate files, etc. -- is possible in theory, but not a practical solution. For one thing, all that work would would have to be redone on each new release of that library.
=== EDIT 4/6/13. On further reflection, I don't think it's even theoretically possible to insure a project is reentrant. Let me explain:
Suppose project XYZ is set up to use different temporary directories depending on platform and configuration in the usual way:
XYZ.proj:
<PropertyGroup>
<MyWorkingDir>tmp.$(Platform).$(Configuration)</MyWorkingDir>
</PropertyGroup>
Now suppose some other project GraphicsWindow references project XYZ, either through ProjectReferences or MSBuild tasks. And suppose the GraphicsWindow project can be built to use various graphics APIs. I.e., there's an OpenGL version, a DirectX 9 version, a DirectX 10, version...
So somewhere there's a .proj or .targets file containing a target to build all four versions:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=OpenGL</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D9</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D10</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D11</Properties>
</ProjectToBuild>
</ItemGroup>
<Target Name="All">
<MSBuild Projects="#(ProjectToBuild)" BuildInParallel="true" />
</Target>
</Project>
or the equivalent using batching.
Now MSBuild will build the XYZ project 4 times with the same Platform|Configuration combination and the same working directory.
This will work fine as long as you build without the /m option and MSBuild runs a single thread. Depending on how the XYZ project is written, the 2nd, 3rd, and 4th builds might do nothing because the outputs are up-to-date, or it might do some redundant work, but the final result will be correct and the build will succeed.
But as soon as you start using parallel MSBuild, this build is broken! There is now a race condition where multiple threads can enter the XYZ project's target(s) concurrently, and start building using the same working directories, which will fail.
Irrespective of how you execute MSBuild, with multi-proc option /m or without, it is guaranteed to execute a project once for every configuration requested by the build. Here is a quote from MSDN:
When the Microsoft Build Engine encounters a project-to-project (P2P) reference while it is using parallel builds to build a project, it builds the reference only one time. If two projects have the same P2P reference, the reference is not rebuilt for each project. Instead, the build engine returns the same P2P reference to both projects that depend on it. Future requests in the session for the same target are provided the same P2P reference.
If you see the same project built more than once, it means it was referenced in two (or more) different configurations. By configuration here I mean a set of parameters that was passed to the project, e.g. project platform (x86, x64, AnyCPU, etc.,) flavor(debug/retail), localization language, any other parameters you might use.
More often than not, this is problem with mixture of project platforms. For example, you have project A built for x64, project B built for AnyCPU, and both A and B reference C. Now C has to be built twice -- for x64 and AnyCPU. If C correctly handles both platforms by cleanly separating outputs into separate directories, there is no problem. However if C treats x64 and AnyCPU as the same, it will fail randomly in multi-proc build.
Start by inspecting your solution configuration dialog. Make sure all projects have consistent set of platform/configuration parameters. If you have a need to build same project in different configuraions, make sure it puts output into separate locations.
I want to inhibit the building of certain projects within a solution from building (within a TeamCity Build Configuration in order to optimize the speed of my Commit Build feedback if you must know).
I'm aware of the Solution Configurations mechanism but don't want to have to force lots of .sln files to end up with every permutation of things I want to be able to switch off. I have Convention based rule where I want to say "If I'm doing the Commit Build, I dont want to do the final installer packaging". (And I don't want to break it out into a separate solution).
I'd prefer not to use a solution involving find and replace in the .sln file or in a .proj file created via [MsBuildEmitSolution][1]. I'm aware of questions here which cover the out of the box solution and this slightly related question.
I see MSBuild /v:diag is saying:
2>Target "Build" in file "Z.sln.metaproj" from project "Z.sln" (entry point):
Using "MSBuild" task from assembly "Microsoft.Build.Tasks.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "MSBuild"
Global Properties:
BuildingSolutionFile=true
CurrentSolutionConfigurationContents=<SolutionConfiguration>
<ProjectConfiguration Project="{C83D035D-169B-4023-9BEE-1790C9FE22AB}" AbsolutePath="X.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
<ProjectConfiguration Project="{15E7887D-F1DB-4D85-8454-E4EF5CBDE6D5}" AbsolutePath="Y.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
</SolutionConfiguration>
So the question is:
Is there a neat way of me getting to do an XPath replace or similar to have the effect of changing BuildProjectInSolution="True" to BuildProjectInSolution="False" for Project Y above
Failing that, is there a relatively simple edit I can do within a .ccproj (An Azure 1.4 Package) or a .csproj (a general project) file to cause the effects (including triggering of dependent projects) of the project being enabled within a commandline msbuild Z.sln solution build to be nullified?
Not sure it qualifies as neat, but you can set CustomAfterMicrosoftCommonTargets to import an msbuild file to over-ride the BuildDependsOn property, pointing it to your own custom build task. Basically, by setting CustomAfterMicrosoftCommonTargets you get msbuild to import an msbuild file containing the following:
<PropertyGroup>
<OldBuildDependsOn>$(BuildDependsOn)</OldBuildDependsOn>
<BuildDependsOn>MyBuild</BuildDependsOn>
</PropertyGroup>
<Target Name="OldBuild" DependsOnTargets="$(OldBuildDependsOn)" />
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="<IfIWantThis>" />
</Target>
Edit
You can use the following MyBuild target to Include/Exclude projects based on regular expressions passed in as IncludeInBuild and ExcludeFromBuild properties. (If you want complex regexes, you may fall foul of MSBuild special character escaping, but this works well enough for simple matching)
> msbuild /p:ExcludeFromBuild="Tests|Install|Azure"
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="('$(IncludeInBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(IncludeInBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='True') AND
('$(ExcludeFromBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(ExcludeFromBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='False')" />
</Target>
You could always pass the particular projects you want to build as parameters to the MSBuild.
The MSBuild command line would look like this:
MSBuild /t:<Project Name>:Rebuild;<Another Project Name>:Rebuild
In TeamCity, you would put <Project Name>:<Target Action> in the target field in the MSBuild runner.
I add a system parameter under Parameters
Name: system.ExcludeFromBuild
Kind: System property (system.)
Value: path to your csproj
I have an MSBuild file that manipulates the AssemblyInfo file before the application is compiled. At the end of the build, it restores the AssemblyInfo file. It does this by backing up the file, manipulating it, and then after build time, restoring the file.
This works fairly well except when an error occurs during the build. It then does not restore the original file. Is there a way I can tell MSBuild to execute a target at the end of a build no matter if it succeeded or failed?
Based on your last comment to the original question I would take another approach, and forget the approach you are currently taking. You should know that your version info doesn't have to be in the AssemblyInfo.cs file. It can be in any code file, just as long as you only have attributes AssemblyVersion and AssemblyFileVersion defined once each. With that being said what I would do is follow these steps:
Remove AssemblyVersion & AssemblyFileVersion from AssemblyInfo.cs
Create a new file, name it whatever you want want in my case I put it at Properties\VersionInfo.cs. Do not add this file to the project.
Edit the project file to include that file into the list of file to be compiled only when you want it
Let's expand a bit on #3. When you build a .NET project, the project itself is an MSBuild file. Inside that file you will find an item declared Compile. This is the list of files that will be sent to the compiler to be compiled. You can dynamically include/exclude files from that list. In you case you want to include the VersionInfo.cs file only if you are building on the build server (or whatever other condition you define). For this example I defined that condition to be if the project was building in Release mode. So for Release mode VersionInfo.cs would be sent to the compiler, and for other builds not. Here are the contents of VersionInfo.cs
VersionInfo.cs
[assembly: System.Reflection.AssemblyVersion("1.2.3.4")]
[assembly: System.Reflection.AssemblyFileVersion("1.2.3.4")]
In order to hook this into the build process you have to edit the project file. In that file you will find an element (maybe more than 1 depending on project type). You should add a target similar to the following there.
<Target Name="BeforeCompile">
<ItemGroup Condition=" '$(Configuration)'=='Release' ">
<Compile Include="Properties\VersionInfo.cs" />
</ItemGroup>
</Target>
Here what I've done here is to define a target, BeforeCompile, which is a well-known target that you can override. See this MSDN article about other similar targets. Basically this is a target which will always be called before the compiler is invoked. In this target I add the VersionInfo.cs to the Compile item only if the Configuration property is set to release. You could define that property to be whatever you wanted. For instance if you have TFS as your build server then it could be,
<Target Name="BeforeCompile">
<ItemGroup Condition=" '$(TeamFoundationServerUrl)'!='' ">
<Compile Include="Properties\VersionInfo.cs" />
</ItemGroup>
</Target>
Because we know that TeamFoundationServerUrl is only defined when building through TFS.
If you are building form the command line then something like this
<Target Name="BeforeCompile">
<ItemGroup Condition=" '$(IncludeVersionInfo)'=='true' ">
<Compile Include="Properties\VersionInfo.cs" />
</ItemGroup>
</Target>
And when you build the project just do msbuild.exe YourProject.proj /p:IncludeVersion=true. Note: this will not work when building a solution.
What about changing the problem:
Add a "template" AssemblyInfo.cs.template to version control that represents your "ideal" AssemblyInfo.cs with regex hooks in there
Before build, copy the template to the real and apply your regexes
Add some kind of subversion ignore for AssemblyInfo.cs (I'm no svn expert, but I'm pretty sure there is a way you can tell it to ignore certain files)
In the event that your devs need to add some kind of customization that would normally appear in an AssemblyInfo.cs (eg InternalsVisibleTo), then get them to add it to a different .cs file that IS checked in.
As a further refinement, combine Sayed's solution with mine and remove version info stuff from the actual AssemblyInfo.cs and have a VersionInfo.cs.template that is checked in, that creates a VersionInfo.cs in BeforeBuild.
I never used it, but from the documentation it seems that the OnError Element is useful to what you're trying to achieve.
Causes one or more targets to execute,
if the ContinueOnError attribute is
false for a failed task.
I need a set of tasks that need to be executed exactly once for the entire solution. This will run tasks that will modify each project to run a separate set of tasks for each project. We had done this earlier using a separate project to the solution which had the solution level tasks, but we want to move away from that. Has anyone done this or does anyone have any suggestions on how to implement this?
Since Solution files are not in MSBuild format they are not easily extended or customized. If you want more control over the build process you would have to create a "driver" msbuild file which would replace your solution file. Inside this driver file you would build all the projects that you needed and perform some additional tasks. You would do this using the MSBuild task. Here is a sample showing how to build more than 1 project.
<Project ...>
<ItemGroup>
<Projects Include="proj01.csproj"/>
<Projects Include="proj02.csproj"/>
<Projects Include="proj03.csproj"/>
</ItemGroup>
<Target Name="BuildAll">
<MSBuild Projects="#(Projects)" BuildInParallel="true" />
</Target>
</Project>
So in your case you would just execute the tasks before you build the projects. Also note that I specified the value true for the BuildInParallel indicating that MSBuild can try and build more than one project at once.
An alternative solution is to have a single target that dispatches to an MSBuild invoked target with as many Global properties removed as possible. My team have a target in the InitialTargets of a Directory.Build.props Import'ed props file - something like:
<Target Name="Prebuild">
<MSBuild Projects="$(MSBuildThisFileFullPath)"
Targets="PrebuildWorker"
RemoveProperties="Configuration;Platform;TargetFramework;BuildProjectReferences" />
</Target>
Since MSBuild appears to synchronize parallel builds on the {project file, global properties, target} set, then by removing all of the properties you can synchronize the build and run it once for all projects being built. The downside: you have to maintain the RemoveProperties attribute - MSBuild doesn't have a way to remove all global properties. If something in the build issues a new MSBuild task with a custom property specified, then you'll get a second instance of the Target invoked.
And - of course - your build will be synchronized on this target. You could try hooking the target up by setting, say, CompileDependsOn to depend on the Prebuild target, to allow independent progress in the build. But to have the target run early and ubiquitously using InitialTargets seems like the better option.