DRYing <TargetFramework> sometimes works, sometimes breaks - msbuild

As I write this, we have 2,497 .csproj files in our entire organization.
We have one policy, and that is "mono-everything". It means that for the sake of reducing costs, and increasing quality, we upgrade the whole organization. Everything should be upgraded when we decide to upgrade anything.
Now, after upgrading to .NET Core 3.0 which was and is a painful process, we decided to DRY <TargetFramework> and remove Debug\netcoreapp3.0 from output folder. We want this policy to be applied to ALL of our projects. So we came up with this ProjectDotNetCore.targets file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<OutputPath>.\bin</OutputPath>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
<AppendRuntimeIdentifierToOutputPath>false</AppendRuntimeIdentifierToOutputPath>
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
<PublishWithAspNetCoreTargetManifest>false</PublishWithAspNetCoreTargetManifest>
</PropertyGroup>
</Project>
And we use this .targets file in all of those 2500 .csproj files. This way, we can assure a single behavior across the entire organization. So a typical .csproj file would look like this:
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup Label="Globals">
<SccProjectName>SAK</SccProjectName>
<SccProvider>SAK</SccProvider>
<SccAuxPath>SAK</SccAuxPath>
<SccLocalPath>SAK</SccLocalPath>
</PropertyGroup>
<PropertyGroup>
<AssemblyName>HolismCommerce.Sammerce.AdminService</AssemblyName>
<RootNamespace>HolismCommerce.Sammerce.AdminService</RootNamespace>
</PropertyGroup>
<Import Project="$(OrganizationProjectsRoot)\Framework\ProjectDotNetCore.targets" />
</Project>
The problem is that, sometimes it works, sometimes it doesn't work and we receive this error:
Error NETSDK1013 The TargetFramework value 'netcoreapp3.0' was not recognized. It may be misspelled. If not, then the TargetFrameworkIdentifier and/or TargetFrameworkVersion properties must be specified explicitly.
Why it has non-deterministic behavior? We want to be able to set <TargetFramework> of all projects from a central location. How can we debug it?

Related

With different configurations in a solution, how can you specify different build types (DLL or Static Lib) for a project in the solution?

I have a VisualStudio solution with multiple projects and configurations.
For one of the projects I want to use different configuration types (DLL or Static Lib) for different configurations.
For example for the configuration "Debug|Win32" I want to build a DLL for that project, and for configuration "Static Release|x64" I want to build a Static Library.
When I try to set the configuration type for one of these configurations, this is the type that then is set for all configurations, so it seems always to be "DLL" for all configs or "Static Lib" for all configs.
I have an example where this can be different and I can't work out how this was achieved. Or do you have to 'hack' the vcxproj file ?
Actually, it turns out that this is possible, but you need to edit the project file.
The project file is in xml format and you should find entries like this
<PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">...</PropertyGroup>
Within a PropertyGroup with a condition, add the ConfigurationType you require, so for example
<PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<PlatformToolset>v141</PlatformToolset>
</PropertyGroup>
<PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<PlatformToolset>v141</PlatformToolset>
</PropertyGroup>
becomes
<PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<PlatformToolset>v141</PlatformToolset>
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<PlatformToolset>v141</PlatformToolset>
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
I've found that this works, but as usual be careful when editing the xml by hand, make sure you have a backup in case you mess up syntax or xml nesting.

Is there an equivalent of $(BuildingInsideVisualStudio) which will detect NuGet?

In MSBuild there is a variable $(BuildingInsideVisualStudio) which can be used to detect whether build is running inside Visual Studio, so I can do conditions like this:
<PropertyGroup Condition="'$(BuildingInsideVisualStudio)' != 'true'">
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
</PropertyGroup>
Is there anything similar for NuGet? I want different conditions to run if the project is being used inside package manager.
Your comment to the question makes it sound like your goal is to keep a packages versions consistent across different conditions in a single project, but it's also a common case that you want to keep it consistent across projects in a solution or repo.
I'm going to suggest a different solution. Create a Directory.Build.props in your repo root that looks something like this:
<Project>
<PropertyGroup>
<NewtonsoftJsonVersion>12.0.1</NewtonsoftJsonVersion>
<xunitVersion>2.4.1</xunitVersion>
</PropertyGroup>
</Project>
Now in your projects that need Newtonsoft.json, you change the PackageReference to <PackageReference Include="Newtonsoft.Json" Version="$(NewtonsoftJsonVersion)" />.
If you put your production code in src\ and test code in test\, then you can create a test\Directory.Build.props with the contents:
<Project>
<Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" />
<PropertyGroup>
<PackageReference Include="xunit" Version="$(xunitVersion)" />
</PropertyGroup>
</Project>
Now all of your projects under test\ will get xunit automatically, and it's guaranteed to be the same version.
When you want to upgrade a package version, you can use the Package Manager UI to check for versions, but unfortunately not to upgrade the version. For that, you'll need to manually edit the repo root Directory.Build.props (so add it to your solution for quick access), but you can be confident that every reference to that package will use the same version. It is limited to projects using PackageReference, there's no solution currently for packages.config, but MSBuild conditions only for for PackageReference too.
You can see this pattern often in Microsoft repositories. Certainly NuGet (my team, yay!), and various .NET repos like cli and sdk do it, although in manually imported props files, rather than Directory.Build.props, though the concept is the same.
There is no direct solution for the case. NuGet is just download manager, it loads sources. MSBuild is a build system, it builds sources. They don't exchange any information between.
I would suggest you to move an another way. You can add a props file into your nuget packaging project with
<?xml version="1.0" encoding="utf-8" ?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<ItemGroup>
<PackageUsedFromNuget>true</PackageUsedFromNuget>
</ItemGroup>
</Project>

MsBuild not generating PDB files in Release configuration

<MSBuild Projects="$(ProjectFile)" Targets="_WPPCopyWebApplication;"
Properties="OutDir=..\publish;Configuration=Release;Platform=AnyCPU" />
I am using above script to publish Asp.Net project. In the project settings, I have absolutely made sure debug symbols are generated in release mode. Still MsBuild is not generating the pdb files in the output.
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<DebugType>Full</DebugType>
<DefineDebug>false</DefineDebug>
<DefineTrace>true</DefineTrace>
<Optimize>true</Optimize>
<OutputPath>bin\</OutputPath>
<DocumentationFile>WebProject.xml</DocumentationFile>
<DebugSymbols>true</DebugSymbols>
</PropertyGroup>
After looking at the Microsoft.Web.Publishing.targets source, I have found a variable (ExcludeGeneratedDebugSymbol) being set to True in Release mode. From the comments, it looks like they wanted to exclude symbols from WebSite project, but the condition is not properly set for WebApplication project.
So, I have decided to override my build scrip from the caller arguments and it worked like a charm. I have not yet ascertained any side affects it may cause or using the undocumented property for future stability, but it works for now.
From the Microsoft.Web.Publishing.target file
<!--For website we will always exclude debug symbols from publishing unless it is set explicitly by user in website publish profile-->
<ExcludeGeneratedDebugSymbol Condition="'$(ExcludeGeneratedDebugSymbol)'=='' And '$(_WebProjectType)' == 'WebSite'">True</ExcludeGeneratedDebugSymbol>
<ExcludeGeneratedDebugSymbol Condition="'$(ExcludeGeneratedDebugSymbol)'=='' And '$(Configuration)' == 'Release'">True</ExcludeGeneratedDebugSymbol>
<ExcludeGeneratedDebugSymbol Condition="'$(ExcludeGeneratedDebugSymbol)'==''">False</ExcludeGeneratedDebugSymbol>
I have updated my script as follows.
<MSBuild Projects="$(ProjectFile)" Targets="_WPPCopyWebApplication;"
Properties="OutDir=..\publish;Configuration=Release;Platform=AnyCPU"; ExcludeGeneratedDebugSymbol=false />
You could also updated your publish profile (.pubxml) file to include that property value. I had to do this today with the new build bits in TFS Build 2015 to have the web publishing include the .pdb files. See example contents of file with property added to bottom.
<?xml version="1.0" encoding="utf-8"?>
<!--
This file is used by the publish/package process of your Web project. You can customize the behavior of this process
by editing this MSBuild file. In order to learn more about this please visit http://go.microsoft.com/fwlink/?LinkID=208121.
-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<WebPublishMethod>FileSystem</WebPublishMethod>
<SiteUrlToLaunchAfterPublish />
<publishUrl>C:\Publish</publishUrl>
<DeleteExistingFiles>True</DeleteExistingFiles>
<LastUsedBuildConfiguration>Release</LastUsedBuildConfiguration>
<LastUsedPlatform>Any CPU</LastUsedPlatform>
<ExcludeApp_Data>False</ExcludeApp_Data>
<LaunchSiteAfterPublish>False</LaunchSiteAfterPublish>
<ExcludeGeneratedDebugSymbol>false</ExcludeGeneratedDebugSymbol>
</PropertyGroup>
</Project>
You can put this directly in your *.csproj file, as the last property group section (right before the Import elements):
<PropertyGroup>
<ExcludeGeneratedDebugSymbol Condition="$(DebugSymbols) == true">false</ExcludeGeneratedDebugSymbol>
</PropertyGroup>

Can MsBuild output dependency information similar to makedepend?

Is there a way to force MsBuild to output target dependency information in a structured form similar to makedepend? I need this at the solution level for a solution containing C# and C++ projects. I'm not picky about the output format.
I've considered that the C# dependencies can be determined by processing the .csproj files and building a DAG. Likewise I could run an open-source makedepend on the C++ sources and go from there. I'm really trying not to roll my own here -- this seems like something that MsBuild ought to be able to just do, if even for diagnostic purposes.
I solved this without too much yak shaving. Obviously MsBuild does have the dependency info during the build so my approach is to wrap the build with a custom target that writes the dependency to a .depends file:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- Write project dependencies to a .depends file, one line per dependency -->
<Target Name="OutputProjectDependencies">
<Delete Files="$(OutputPath)\$(TargetFileName).depends"/>
<WriteLinesToFile File="$(OutputPath)\$(TargetFileName).depends"
Lines="#(CscDependencies->'%(FullPath)');#(ReferencePath->'%(FullPath)');#(Content->'%(FullPath)');#(_NoneWithTargetPath->'%(FullPath)')"
Overwrite="false"
Encoding="UTF-8"/>
<WriteLinesToFile File="$(OutputPath)\$(TargetFileName).depends"
Lines="#(ClDependencies->'%(FullPath)')"
Overwrite="false"
Encoding="UTF-8"/>
</Target>
<ItemGroup>
<CscDependencies Include="#(Compile);#(EmbeddedResource)"/>
<ClDependencies Include="#(ClCompile);#(ClInclude)"/>
</ItemGroup>
<PropertyGroup>
<BuildDependsOn>
$(BuildDependsOn);
OutputProjectDependencies;
</BuildDependsOn>
</PropertyGroup>
</Project>
This is not quite as robust as I'd like for C++ projects (it lacks included header and link library dependencies) but could probably be further enhanced. I believe this is a very solid approach for C# -- it includes referenced assemblies, embedded resources and content.

Replace .sln with MSBuild and wrap contained projects into targets

I'd like to create a MSBuild project that reflects the project dependencies in a solution and wraps the VS projects inside reusable targets.
The problem I like solve doing this is to svn-export, build and deploy a specific assembly (and its dependencies) in an BizTalk application.
My question is: How can I make the targets for svn-exporting, building and deploying reusable and also reuse the wrapped projects when they are built for different dependencies?
I know it would be simpler to just build the solution and deploy only the assemblies needed but I'd like to reuse the targets as much as possible.
The parts
The project I like to deploy
<Project DefaultTargets="Deploy" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<ExportRoot Condition="'$(Export)'==''">Export</ExportRoot>
</PropertyGroup>
<Target Name="Clean_Export">
<RemoveDir Directories="$(ExportRoot)\My.Project.Dir" />
</Target>
<Target Name="Export_MyProject">
<Exec Command="svn export svn://xxx/trunk/Biztalk2009/MyProject.btproj --force" WorkingDirectory="$(ExportRoot)" />
</Target>
<Target Name="Build_MyProject" DependsOnTargets="Export_MyProject">
<MSBuild Projects="$(ExportRoot)\My.Project.Dir\MyProject.btproj" Targets="Build" Properties="Configuration=Release"></MSBuild>
</Target>
<Target Name="Deploy_MyProject" DependsOnTargets="Build_MyProject">
<Exec Command="BTSTask AddResource -ApplicationName:CORE -Source:MyProject.dll" />
</Target>
</Project>
The projects it depends upon look almost exactly like this (other .btproj and .csproj).
Wow, this is a loaded question for a forum post. I wrote about 20 pages on creating reusable .targets files in my book, but I'll get you started here with the basics here. I believe that the key to creating reusable build scripts (i.e. .targets files) is three elements:
Place behavior (i.e. targets) into separate files
Place data (i.e. properties and items, these are called .proj files) into their own files
Extensibility
.targets files should validate assumptions
The idea is that you want to place all of your targets into separate files and then these files will be imported by the files which will be driving the build process. These are the files which contain the data. Since you import the .targets files you get all the targets as if they had been defined inline. There will be a silent contract between the .proj and .targets files. This contract is defined in properties and items which both use. This is what needs to be validated.
The idea here is not new. This pattern is followed by .csproj (and other projects generated by Visual Studio). If you take a look your .csproj file you will not find a single target, just properties and items. Then towards the bottom of the file it imports Microsoft.csharp.targets (may differ depending on project type). This project file (along with others that it imports) contains all the targets which actually perform the build.
So it's layed out like this:
SharedBuild.targets
MyProduct.proj
Where MyProdcut.proj might look like:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This uses a .targets file to off load performing the build -->
<PropertyGroup>
<Configuration Condition=" '$(Configuration)'=='' ">Release</Configuration>
<OutputPath Condition=" '$(OutputPath)'=='' ">$(MSBuildProjectDirectory)\BuildArtifacts\bin\</OutputPath>
</PropertyGroup>
<ItemGroup>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary1\ClassLibrary1.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary2\ClassLibrary2.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary3\ClassLibrary3.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\WindowsFormsApplication1\WindowsFormsApplication1.csproj"/>
</ItemGroup>
<Import Project="SharedBuild.targets"/>
</Project>
And SharedBuild.targets might look like:
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This represents a re-usable build file -->
<Target Name="SharedBuild_Validate">
<!-- See http://sedodream.com/2009/06/30/ElementsOfReusableMSBuildScriptsValidation.aspx for more info
about this validation pattern
-->
<ItemGroup>
<_RequiredProperties Include ="Configuration">
<Value>$(Configuration)</Value>
</_RequiredProperties>
<_RequiredProperties Include ="OutputPath">
<Value>$(OutputPath)</Value>
</_RequiredProperties>
<_RequiredItems Include="Projects">
<RequiredValue>%(Projects.Identity)</RequiredValue>
<RequiredFilePath>%(Projects.Identity)</RequiredFilePath>
</_RequiredItems>
</ItemGroup>
<!-- Raise an error if any value in _RequiredProperties is missing -->
<Error Condition="'%(_RequiredProperties.Value)'==''"
Text="Missing required property [%(_RequiredProperties.Identity)]"/>
<!-- Raise an error if any value in _RequiredItems is empty -->
<Error Condition="'%(_RequiredItems.RequiredValue)'==''"
Text="Missing required item value [%(_RequiredItems.Identity)]" />
<!-- Validate any file/directory that should exist -->
<Error Condition="'%(_RequiredItems.RequiredFilePath)' != '' and !Exists('%(_RequiredItems.RequiredFilePath)')"
Text="Unable to find expeceted path [%(_RequiredItems.RequiredFilePath)] on item [%(_RequiredItems.Identity)]" />
</Target>
<PropertyGroup>
<BuildDependsOn>
SharedBuild_Validate;
BeforeBuild;
CoreBuild;
AfterBuild;
</BuildDependsOn>
</PropertyGroup>
<Target Name="Build" DependsOnTargets="$(BuildDependsOn)"/>
<Target Name="BeforeBuild"/>
<Target Name="AfterBuild"/>
<Target Name="CoreBuild">
<!-- Make sure output folder exists -->
<PropertyGroup>
<_FullOutputPath>$(OutputPath)$(Configuration)\</_FullOutputPath>
</PropertyGroup>
<MakeDir Directories="$(_FullOutputPath)"/>
<MSBuild Projects="#(Projects)"
BuildInParallel="true"
Properties="OutputPath=$(_FullOutputPath)"/>
</Target>
</Project>
Don't look too much at the SharedBuild_Validate target yet. I put that there for completeness but don't focus on it. You can find more info on that at my blog at http://sedodream.com/2009/06/30/ElementsOfReusableMSBuildScriptsValidation.aspx.
The important parts to notice are the extensibility points. Even though this is a very basic file, it has all the components of a reusable .targets file. You can customize it's behavior by passing in different properties and items to build. You can extend it's behavior by overriding a target (BeforeBuild, AfterBuild or even CoreBuild) and you can inject your own targets into the build with:
<Project ...>
...
<Import Project="SharedBuild.targets"/>
<PropertyGroup>
<BuildDependsOn>
$(BuildDependsOn);
CustomAfterBuild
</BuildDependsOn>
</PropertyGroup>
<Target Name="CustomAfterBuild">
<!-- Insert stuff here -->
</Target>
</Project>
In your case I would create an SvnExport.targets file which uses the required properties:
SvnExportRoot
SvnUrl
SvnWorkingDirectory
You will use these properties to do the Export.
Then create another one for Biztalk build and deploy. You could split this up into 2 if necessary.
Then inside of your .proj file you just import both and setup the targets to build in the right order, and your off.
This is only really the beginning of creating reusable build elements, but this should get the wheels turning in your head. I am going to post all of this to my blog as well as download links for all files.
UPDATE:
Posted to blog at http://sedodream.com/2010/03/19/ReplacingSolutionFilesWithMSBuildFiles.aspx