Can MsBuild output dependency information similar to makedepend? - msbuild

Is there a way to force MsBuild to output target dependency information in a structured form similar to makedepend? I need this at the solution level for a solution containing C# and C++ projects. I'm not picky about the output format.
I've considered that the C# dependencies can be determined by processing the .csproj files and building a DAG. Likewise I could run an open-source makedepend on the C++ sources and go from there. I'm really trying not to roll my own here -- this seems like something that MsBuild ought to be able to just do, if even for diagnostic purposes.

I solved this without too much yak shaving. Obviously MsBuild does have the dependency info during the build so my approach is to wrap the build with a custom target that writes the dependency to a .depends file:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- Write project dependencies to a .depends file, one line per dependency -->
<Target Name="OutputProjectDependencies">
<Delete Files="$(OutputPath)\$(TargetFileName).depends"/>
<WriteLinesToFile File="$(OutputPath)\$(TargetFileName).depends"
Lines="#(CscDependencies->'%(FullPath)');#(ReferencePath->'%(FullPath)');#(Content->'%(FullPath)');#(_NoneWithTargetPath->'%(FullPath)')"
Overwrite="false"
Encoding="UTF-8"/>
<WriteLinesToFile File="$(OutputPath)\$(TargetFileName).depends"
Lines="#(ClDependencies->'%(FullPath)')"
Overwrite="false"
Encoding="UTF-8"/>
</Target>
<ItemGroup>
<CscDependencies Include="#(Compile);#(EmbeddedResource)"/>
<ClDependencies Include="#(ClCompile);#(ClInclude)"/>
</ItemGroup>
<PropertyGroup>
<BuildDependsOn>
$(BuildDependsOn);
OutputProjectDependencies;
</BuildDependsOn>
</PropertyGroup>
</Project>
This is not quite as robust as I'd like for C++ projects (it lacks included header and link library dependencies) but could probably be further enhanced. I believe this is a very solid approach for C# -- it includes referenced assemblies, embedded resources and content.

Related

Is there an equivalent of $(BuildingInsideVisualStudio) which will detect NuGet?

In MSBuild there is a variable $(BuildingInsideVisualStudio) which can be used to detect whether build is running inside Visual Studio, so I can do conditions like this:
<PropertyGroup Condition="'$(BuildingInsideVisualStudio)' != 'true'">
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
</PropertyGroup>
Is there anything similar for NuGet? I want different conditions to run if the project is being used inside package manager.
Your comment to the question makes it sound like your goal is to keep a packages versions consistent across different conditions in a single project, but it's also a common case that you want to keep it consistent across projects in a solution or repo.
I'm going to suggest a different solution. Create a Directory.Build.props in your repo root that looks something like this:
<Project>
<PropertyGroup>
<NewtonsoftJsonVersion>12.0.1</NewtonsoftJsonVersion>
<xunitVersion>2.4.1</xunitVersion>
</PropertyGroup>
</Project>
Now in your projects that need Newtonsoft.json, you change the PackageReference to <PackageReference Include="Newtonsoft.Json" Version="$(NewtonsoftJsonVersion)" />.
If you put your production code in src\ and test code in test\, then you can create a test\Directory.Build.props with the contents:
<Project>
<Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" />
<PropertyGroup>
<PackageReference Include="xunit" Version="$(xunitVersion)" />
</PropertyGroup>
</Project>
Now all of your projects under test\ will get xunit automatically, and it's guaranteed to be the same version.
When you want to upgrade a package version, you can use the Package Manager UI to check for versions, but unfortunately not to upgrade the version. For that, you'll need to manually edit the repo root Directory.Build.props (so add it to your solution for quick access), but you can be confident that every reference to that package will use the same version. It is limited to projects using PackageReference, there's no solution currently for packages.config, but MSBuild conditions only for for PackageReference too.
You can see this pattern often in Microsoft repositories. Certainly NuGet (my team, yay!), and various .NET repos like cli and sdk do it, although in manually imported props files, rather than Directory.Build.props, though the concept is the same.
There is no direct solution for the case. NuGet is just download manager, it loads sources. MSBuild is a build system, it builds sources. They don't exchange any information between.
I would suggest you to move an another way. You can add a props file into your nuget packaging project with
<?xml version="1.0" encoding="utf-8" ?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<ItemGroup>
<PackageUsedFromNuget>true</PackageUsedFromNuget>
</ItemGroup>
</Project>

How are we supposed to execute package build targets in the new world where nuget packages are consumed through msbuild PackageReference?

I am developing a suite of UI tests using Selenium. One of the run-time dependencies of this suite is the chromedriver.exe, which we are expected to consume through the Selenium.WebDriver.ChromeDriver NuGet package.
The old world
When this NuGet package is imported the following lines are injected into the csproj file:
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\Selenium.WebDriver.ChromeDriver.2.44.0\build\Selenium.WebDriver.ChromeDriver.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\Selenium.WebDriver.ChromeDriver.2.44.0\build\Selenium.WebDriver.ChromeDriver.targets'))" />
</Target>
<Import Project="..\packages\Selenium.WebDriver.ChromeDriver.2.44.0\build\Selenium.WebDriver.ChromeDriver.targets" Condition="Exists('..\packages\Selenium.WebDriver.ChromeDriver.2.44.0\build\Selenium.WebDriver.ChromeDriver.targets')" />
And it is automatic by the Visual Studio. This covers our bases, making sure the build targets provided by the Selenium.WebDriver.ChromeDriver package are there at the time of the build and running them as needed. The logic inside the build targets file copies/publishes the chromedriver.exe to the right location.
All is green.
The new world.
I consume the same NuGet package as PackageReference in the csproj file. Cool. However, the build targets of that package are no longer executed. See https://github.com/NuGet/Home/issues/4013. Apparently, this is by design.
I could import the targets manually, but the problem is that I will have to hard code the location where the package is restored. It is no longer restored in the packages directory in the solution, but under my windows profile. But there is no property pointing to this location and hard coding it sucks.
So, here is the version that works for me and I hate it:
<PropertyGroup>
<MyPackagesPath>$(UserProfile)\.nuget\packages\</MyPackagesPath>
<SeleniumWebDriverChromeDriverTargets>$(MyPackagesPath)selenium.webdriver.chromedriver\2.44.0\build\Selenium.WebDriver.ChromeDriver.targets</SeleniumWebDriverChromeDriverTargets>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="2.44.0" />
</ItemGroup>
<Target Name="EnsureChromeDriver" AfterTargets="PrepareForRun">
<Error Text="chrome driver is missing!" Condition="!Exists('$(OutDir)chromedriver.exe')" />
</Target>
<Import Project="$(SeleniumWebDriverChromeDriverTargets)" Condition="Exists('$(SeleniumWebDriverChromeDriverTargets)') And '$(ExcludeRestorePackageImports)' == 'true'" />
Overall, the Sdk style projects are absolutely great, but this whole business of running targets from the packages is totally broken, even if it is by design.
What am I missing?
EDIT 1
So, here is the content of the generated obj\UITests.csproj.nuget.g.targets:
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<MSBuildAllProjects>$(MSBuildAllProjects);$(MSBuildThisFileFullPath)</MSBuildAllProjects>
</PropertyGroup>
<ImportGroup Condition=" '$(ExcludeRestorePackageImports)' != 'true' ">
<Import Project="$(NuGetPackageRoot)selenium.webdriver.chromedriver\2.44.0\build\Selenium.WebDriver.ChromeDriver.targets" Condition="Exists('$(NuGetPackageRoot)selenium.webdriver.chromedriver\2.44.0\build\Selenium.WebDriver.ChromeDriver.targets')" />
</ImportGroup>
</Project>
Notice the ImportGroup condition is '$(ExcludeRestorePackageImports)' != 'true'. Now, this condition is always false, because ExcludeRestorePackageImports seems to be hard coded to be true in
c:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets
Inspecting binary log confirms this. Plus https://github.com/NuGet/Home/issues/4013 was closed as WontFix.
Or am I still missing something?
If you are running Restore and other targets during the build, you may get unexpected results due to NuGet modifying xml files on disk or because MSBuild files imported by NuGet packages aren't imported correctly.

.NET Core msbuild ProjectReference

I have a solution that contains a console application with a .csproj file like the this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.1</TargetFramework>
</PropertyGroup>
</Project>
I also have a library project that uses the console application to generate a heap of C# code that get compiled into the library, the library .csproj file looks like this.
<Project Sdk="Microsoft.NET.Sdk" InitialTargets="RunGenerator">
<PropertyGroup>
<TargetFramework>netstandard1.4</TargetFramework>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../generator/generator.csproj">
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>
</ItemGroup>
<Target Name="RunGenerator">
<Exec Command="dotnet run -p "../generator/generator.csproj" input output" />
</Target>
</Project>
This fails because the dependency analysis says that a netstandard1.4 assembly cannot reference a netcoreapp1.1 assembly. That is correct except that I am not referencing the assembly.
I can work around that issue by building the generator project like this:
<Project Sdk="Microsoft.NET.Sdk" InitialTargets="RunGenerator">
<PropertyGroup>
<TargetFramework>netstandard1.4</TargetFramework>
</PropertyGroup>
<Target Name="RunGenerator">
<Exec Command="dotnet build "../generator/generator.csproj"" />
<Exec Command="dotnet run -p "../generator/generator.csproj" input output" />
</Target>
</Project>
The problem is that the generator project no longer takes part in the dependency analysis when these projects are built using the containing solution file and the explicit build of the generator project sometimes runs concurrently with another build of the same project initiated by the solution build and this results in errors because files are locked etc.
Is it possible to have a project dependency without checking the target framework?
Can anyone suggest a workaround?
Thanks.
Here are some MSBuild tips. You might need to combine a few of these ideas.
You can use your solution file to add an explicit project dependency. See https://learn.microsoft.com/en-us/visualstudio/ide/how-to-create-and-remove-project-dependencies (This question was originally asked here: Visual Studio 2010: How to enforce build order of projects in a solution?). Unfortunately, this is really hard to do if you don't have VS. The format is .sln files is kinda a nightmare.
To avoid the concurrent build issue, use the MSBuild task instead of the Exec task. See https://learn.microsoft.com/en-us/visualstudio/msbuild/msbuild-task
<Target Name="CompileAnotherProject">
<MSBuild Projects="../generator/generator.csproj" Targets="Build" />
</Target>
dotnet-run invokes "dotnet build" automatically. This is actually problematic in concurrent builds. You can instead add a target to your generator.csproj that runs the app after it has been built. "dotnet filepath.dll" runs the compiled app without building it.
<Target Name="RunCodeGen" AfterTargets="Build">
<Exec Command="dotnet $(AssemblyName).dll input output"
WorkingDirectory="$(OutDir)" />
</Target>

MC.exe in msbuild

How do I compile with mc.exe in the correct way. Currently I have a build step which runs the relevant command but looking at developer network
There seems to be a better way.
I am not a expert with msbuild so please excuse how easy this question is. Googling has revealed no help
<Project
DefaultTargets="Build"
ToolsVersion="14.0"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Message Source Files">
<Extensions>mc;</Extensions>
<UniqueIdentifier>{B796B525-44D3-4260-8C76-705DBADA1043}</UniqueIdentifier>
</Filter>
</ItemGroup>
<ItemGroup>
<MessageCompile Include="a.mc">
<GenerateBaselineResource>true</GenerateBaselineResource>
</MessageCompile>
</ItemGroup>
<Target Name="Build">
<DontKnowWhatGoesHere Sources="#(MessageCompile)"/>
</Target>
</Project>
MSBuild build are usually extended via .targets files, that have to be included in the project, and they extend the existing build proces. The WDK tasks for MSBuild page confirms this:
These command-line tools need to be exposed to MSBuild as tasks (contained in targets) so that they can be run during the build process.
The WDK MSDN page also has a help page on Windows driver targets:
The WindowsDriver.Common.targets, WindowsDriver.masm.targets, and WindowsDriver.arm.targets files provide the targets that are necessary to build a driver.
A quick grep in my C:\Program Files (x86)\Windows Kits\10\build directory showed that the MessageCompile target (the step that actually processes the MessageCompile items) is defined in the build\WindowsDriver.Common.targets file.
After importing the targets in your project you can do one of the following:
<Import
Project="C:\Program Files (x86)\Windows Kits\10\build\build\WindowsDriver.Common.targets" />
<!-- Option A: -->
<Target Name="Build" DependsOnTargets="MessageCompile">
<!-- no need to do anything, the dependency target should do the work -->
</Target>
<!-- Option B: -->
<Target Name="Build" DependsOnTargets="MessageCompile">
<!-- Use the Mc task which is the actual wrapper around the .exe,
see the .common.targets file for the list of all parameters -->
<Mc
Sources ="#(MessageCompile)"
ToolExe ="$(MessageCompileToolExe)"
ToolPath ="$(MessageCompileToolPath)"
Generated
/>
</Target>

Replace .sln with MSBuild and wrap contained projects into targets

I'd like to create a MSBuild project that reflects the project dependencies in a solution and wraps the VS projects inside reusable targets.
The problem I like solve doing this is to svn-export, build and deploy a specific assembly (and its dependencies) in an BizTalk application.
My question is: How can I make the targets for svn-exporting, building and deploying reusable and also reuse the wrapped projects when they are built for different dependencies?
I know it would be simpler to just build the solution and deploy only the assemblies needed but I'd like to reuse the targets as much as possible.
The parts
The project I like to deploy
<Project DefaultTargets="Deploy" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<ExportRoot Condition="'$(Export)'==''">Export</ExportRoot>
</PropertyGroup>
<Target Name="Clean_Export">
<RemoveDir Directories="$(ExportRoot)\My.Project.Dir" />
</Target>
<Target Name="Export_MyProject">
<Exec Command="svn export svn://xxx/trunk/Biztalk2009/MyProject.btproj --force" WorkingDirectory="$(ExportRoot)" />
</Target>
<Target Name="Build_MyProject" DependsOnTargets="Export_MyProject">
<MSBuild Projects="$(ExportRoot)\My.Project.Dir\MyProject.btproj" Targets="Build" Properties="Configuration=Release"></MSBuild>
</Target>
<Target Name="Deploy_MyProject" DependsOnTargets="Build_MyProject">
<Exec Command="BTSTask AddResource -ApplicationName:CORE -Source:MyProject.dll" />
</Target>
</Project>
The projects it depends upon look almost exactly like this (other .btproj and .csproj).
Wow, this is a loaded question for a forum post. I wrote about 20 pages on creating reusable .targets files in my book, but I'll get you started here with the basics here. I believe that the key to creating reusable build scripts (i.e. .targets files) is three elements:
Place behavior (i.e. targets) into separate files
Place data (i.e. properties and items, these are called .proj files) into their own files
Extensibility
.targets files should validate assumptions
The idea is that you want to place all of your targets into separate files and then these files will be imported by the files which will be driving the build process. These are the files which contain the data. Since you import the .targets files you get all the targets as if they had been defined inline. There will be a silent contract between the .proj and .targets files. This contract is defined in properties and items which both use. This is what needs to be validated.
The idea here is not new. This pattern is followed by .csproj (and other projects generated by Visual Studio). If you take a look your .csproj file you will not find a single target, just properties and items. Then towards the bottom of the file it imports Microsoft.csharp.targets (may differ depending on project type). This project file (along with others that it imports) contains all the targets which actually perform the build.
So it's layed out like this:
SharedBuild.targets
MyProduct.proj
Where MyProdcut.proj might look like:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This uses a .targets file to off load performing the build -->
<PropertyGroup>
<Configuration Condition=" '$(Configuration)'=='' ">Release</Configuration>
<OutputPath Condition=" '$(OutputPath)'=='' ">$(MSBuildProjectDirectory)\BuildArtifacts\bin\</OutputPath>
</PropertyGroup>
<ItemGroup>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary1\ClassLibrary1.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary2\ClassLibrary2.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary3\ClassLibrary3.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\WindowsFormsApplication1\WindowsFormsApplication1.csproj"/>
</ItemGroup>
<Import Project="SharedBuild.targets"/>
</Project>
And SharedBuild.targets might look like:
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This represents a re-usable build file -->
<Target Name="SharedBuild_Validate">
<!-- See http://sedodream.com/2009/06/30/ElementsOfReusableMSBuildScriptsValidation.aspx for more info
about this validation pattern
-->
<ItemGroup>
<_RequiredProperties Include ="Configuration">
<Value>$(Configuration)</Value>
</_RequiredProperties>
<_RequiredProperties Include ="OutputPath">
<Value>$(OutputPath)</Value>
</_RequiredProperties>
<_RequiredItems Include="Projects">
<RequiredValue>%(Projects.Identity)</RequiredValue>
<RequiredFilePath>%(Projects.Identity)</RequiredFilePath>
</_RequiredItems>
</ItemGroup>
<!-- Raise an error if any value in _RequiredProperties is missing -->
<Error Condition="'%(_RequiredProperties.Value)'==''"
Text="Missing required property [%(_RequiredProperties.Identity)]"/>
<!-- Raise an error if any value in _RequiredItems is empty -->
<Error Condition="'%(_RequiredItems.RequiredValue)'==''"
Text="Missing required item value [%(_RequiredItems.Identity)]" />
<!-- Validate any file/directory that should exist -->
<Error Condition="'%(_RequiredItems.RequiredFilePath)' != '' and !Exists('%(_RequiredItems.RequiredFilePath)')"
Text="Unable to find expeceted path [%(_RequiredItems.RequiredFilePath)] on item [%(_RequiredItems.Identity)]" />
</Target>
<PropertyGroup>
<BuildDependsOn>
SharedBuild_Validate;
BeforeBuild;
CoreBuild;
AfterBuild;
</BuildDependsOn>
</PropertyGroup>
<Target Name="Build" DependsOnTargets="$(BuildDependsOn)"/>
<Target Name="BeforeBuild"/>
<Target Name="AfterBuild"/>
<Target Name="CoreBuild">
<!-- Make sure output folder exists -->
<PropertyGroup>
<_FullOutputPath>$(OutputPath)$(Configuration)\</_FullOutputPath>
</PropertyGroup>
<MakeDir Directories="$(_FullOutputPath)"/>
<MSBuild Projects="#(Projects)"
BuildInParallel="true"
Properties="OutputPath=$(_FullOutputPath)"/>
</Target>
</Project>
Don't look too much at the SharedBuild_Validate target yet. I put that there for completeness but don't focus on it. You can find more info on that at my blog at http://sedodream.com/2009/06/30/ElementsOfReusableMSBuildScriptsValidation.aspx.
The important parts to notice are the extensibility points. Even though this is a very basic file, it has all the components of a reusable .targets file. You can customize it's behavior by passing in different properties and items to build. You can extend it's behavior by overriding a target (BeforeBuild, AfterBuild or even CoreBuild) and you can inject your own targets into the build with:
<Project ...>
...
<Import Project="SharedBuild.targets"/>
<PropertyGroup>
<BuildDependsOn>
$(BuildDependsOn);
CustomAfterBuild
</BuildDependsOn>
</PropertyGroup>
<Target Name="CustomAfterBuild">
<!-- Insert stuff here -->
</Target>
</Project>
In your case I would create an SvnExport.targets file which uses the required properties:
SvnExportRoot
SvnUrl
SvnWorkingDirectory
You will use these properties to do the Export.
Then create another one for Biztalk build and deploy. You could split this up into 2 if necessary.
Then inside of your .proj file you just import both and setup the targets to build in the right order, and your off.
This is only really the beginning of creating reusable build elements, but this should get the wheels turning in your head. I am going to post all of this to my blog as well as download links for all files.
UPDATE:
Posted to blog at http://sedodream.com/2010/03/19/ReplacingSolutionFilesWithMSBuildFiles.aspx