How can I execute programs in my %PATH% with MSBuild? - msbuild

Note: I'm using Mercurial as an example here, because that's what I'm trying to get to work with MSBuild right now.
But the problem is not limited to Mercurial, it happens with every external program that is somewhere in my %PATH% variable (I tried the same with PowerShell, for example).
So I didn't put the Mercurial tag on this question on purpose, because this is not about Mercurial!
What I actually want to do:
I want my build script to get the current revision number from my Mercurial repository and store it in a file.
The simplest way to do this from the command line is:
hg id -i >rev.txt
Mercurial is installed on my machine and the installation folder is in my %PATH% variable.
So I can run this line from anywhere on my machine (directly from the command line, or from a batch file), and it just works.
The problem occurs when I try to run this line from my build script.
I change the BeforeBuild (or AfterBuild) section of my .csproj file as follows:
<Target Name="AfterBuild">
<Exec Command="hg id -i >rev.txt"/>
</Target>
When I compile my solution with Visual Studio, it works and the rev.txt file is created in the folder where my .csproj is.
But when I compile the exact same solution from the command line with MSBuild, the build fails with the following error message:
The command "hg id -i >rev.txt" exited with code 9009.
I googled "msbuild code 9009" and found some solutions, but all of them propose to provide the full path to the executable.
When I do this, the build succeeds with MSBuild as well.
But this is not an acceptable solution for me, because I can't be sure that everyone using my project (including the build server) has installed Mercurial in the exact same folder.
That's exactly what %PATH% is for...
The same happens when I put the <Exec Command="... line directly into the build script.
If I specify the path to the executable, it works.
If I don't specify the path, it doesn't.
Is there any trick to make MSBuild execute programs in my %PATH% variable without specifying the complete folder?
EDIT:
#leppie:
Output redirection:
You mean the fact that I save the output of my command in a text file inside the command , instead of just running hg id -i as a command and using an output parameter or something like that to get the output?
Doesn't make any difference...the error is the same when I omit >rev.txt.
Command line args:
No, it throws the same error, even if I shorten the command to just hg (without any parameters).
Don't forget: if I run the exact same Exec command in the exact same .csproj file from Visual Studio, or if I just provide the path to the .exe file in the command, everything works.
So IMO output redirection and command line args can't be the problem.

Have you tried this extension pack for mercurial/msbuild?
http://msbuildhg.codeplex.com/documentation
Seems to have a task for returning revision id, which is what your trying to achieve no?
<HgVersion LocalPath="$(MSBuildProjectDirectory)" Timeout="5000">
<Output TaskParameter="Revision" PropertyName="AssemblyRevision" />
</HgVersion>

Okay, I found the solution.
I have to admit, it was a classic case of PEBKAC :-)
I'll explain it anyway, maybe it will help someone who made the same mistake:
Basically everything I have tried (plus what James Woolfenden suggested in his answer) would have been worked...if only the batch file that I use to run the build script wouldn't have looked like this:
path="%windir%\Microsoft.net\Framework\v4.0.30319"
msbuild build.proj
Yes, exactly.
I'm editing the %PATH% variable for the duration of this batch file, and I'm overwriting it with the path to MSBuild instead of just appending it.
So when my build script tries to call Mercurial, it can't find it anymore because its location is not in the %PATH% variable anymore.
No idea why I didn't see this before.
The correct way would be to append the MSBuild path, leaving the other paths intact:
path=%path%;%windir%\Microsoft.net\Framework\v4.0.30319

Related

Generate compile_commands.json from msbuild via command line -- no cmake

I'm looking for a way to integrate clang-tidy into a CI workflow, but the build system being used is MSBuild with dependencies managed by vcpkg in manisfest mode.
Is there some advanced command line that I can pass MSBuild (or some other tool that understands MSBuild process completely) to be able to generate compile_commands.json?
I'm sure I'm not the first one to try that, I've seen a couple of cases of success using SourceTrail and ClangPowerTools, but it has been specially painful in my case because the alternatives cannot detect the include paths exposed by vcpkg.
It turns out that msbuild has a builtin target for clang-tidy since Visual Studio started to support that linter.
To invoke it one can run : msbuild /t:ClangTidy ....
Unfortunately, very few command line options are exposed from the clang-tidy integration to msbuild CLI: the list of checks, the header-filter, additional compiler options, a flag for warnings in system headers and the tool path.
If one wants to export the fixes file generated by clang-tidy, which was my case, it needs to do some extra work.
I achieved my goal by adding one Directory.Build.props file in the project folder which overrides the path to the original clang-tidy.exe with a batch script (.bat), which does some preparation steps and forwards the msbuild target command line to a python script which can invoke clang-tidy with its full command line capabilities.
The props file:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<ClangTidyToolPath>$(where_I_saved_the_bat_file)\</ClangTidyToolPath>
<ClangTidyToolExe>msbuild-clang-tidy-runner.bat</ClangTidyToolExe>
</PropertyGroup>
</Project>
The batch file:
SET PYTHONPATH=...
REM and maybe other stuff
python3 -m my_custom_clang_tidy_runner %*
Interestingly the batch file was the only way other than a real executable file that I could place in the Directory.Build.props that wouldn't result in msbuild compilation error. I tried python and powershell scripts and none were accepted as a suitable clang-tidy executable file. I experimented creating a C++ program which does the same as my Python script and it worked as expected as well.
So, to summarize, the recipe is:
Create the Directory.Build.props file in the root folder of the project you want to lint overriding the clang-tidy path.
A custom .exe or .bat file to replace clang-tidy capable of forwarding the command line options passed by msbuild together with any customisation you wish.
Invoke msbuild /t:ClangTidy ... and Bob's your uncle!

WiX Heat Pre-build Error

I have a WiX installer that was working fine, but it's been a few months since I worked on it... so, I am not attempting to build the installer (on a reconfigured machine) and now it fails to build every time. I've tried to simplify everything to isolate where it's failing, but I still can't seem to figure out why. Essentially, it looks like it's failing at the pre-build process. I had a length Pre-build Event command, but I've simplified it to the bare essentials. Here's my command:
call “$(WIX)bin\heat.exe” dir "$(SolutionDir)MyProj\bin\Release" -out “$(ProjectDir)MyFiles.wxs”
The error I'm getting is:
call “C:\Program Files (x86)\WiX Toolset v3.10\bin\heat.exe” dir "C:\Users\MyName\Documents\GitHub\MyProj\bin\Release" -out “C:\Users\MyName\Documents\GitHub\MyProj\Installer\MyFiles.wxs”
The filename, directory name, or volume label syntax is incorrect.
The code appears to exit with code 1, which seems to suggest that it's failing to locate a filename or path... however, I've checked the paths that are printed in the output log and the heat.exe application does exists under the C:\Program Files (x86) directory. The directory (dir) it's trying to harvest is properly defined and the output location is as well. So, does anyone know why this is failing?
Update
What's incredibly strange is that the pre-build event command seems to fail even when I simplify the command to try to simply call the heat.exe application (with no other parameters). I tried changing my pre-build event to the following:
call “C:\Program Files (x86)\WiX Toolset v3.10\bin\heat.exe”
As you can see from the attached image, the heat.exe application is indeed located in the directory specified in the build command. Yet, I still get an error saying that the MSBuild failed and exited with code 1. If I check the output log, it says that the, "filename, directory name, or volume label syntax is incorrect". However, I can say that this installer was working properly a few months ago with no changes to the build events... so I really don't know why it wouldn't work now, except that I had to reconfigure my machine and reinstall Windows 10 recently. I downloaded and installed the latest WiX toolset, and would assume that would be everything that is needed. But, this error persists. Any ideas?
This is really dumb but I believe you need to put a \ at the end of your dir IIRC. I think I ran into the same issue at one point. The error is not particularly helpful.
call "$(WIX)bin\heat.exe" dir "$(SolutionDir)MyProj\bin\Release\" -out "$(ProjectDir)MyFiles.wxs"
(I think I was remembering backwards and you need to NOT have the trailing \ in the dir.. sorry)
Just in case you can't get the pre-build event to work, you can also call heat like this (requires editing the wixproj file)
<Target Name="BeforeBuild">
<Exec Command=""$(WIX)bin\heat.exe" dir "$(SolutionDir)MyProj\bin\Release" -out "$(ProjectDir)MyFiles.wxs""/>
</Target>
This is how I call heat in some of the installers I've authored. If this also doesn't work I'm out of ideas for why this doesn't work for you.
I know its a little old question but I've faced the issue and spent hours to notice different qoutes here: “$(WIX)bin\heat.exe”. Replace with straight ones "$(WIX)bin\heat.exe" and enjoy.

Use path system environment variable in MSBuild exec task

I'm calling an external tool to do some post processing in my build sequence using the Exec task in MSBuild. When I update the tool to a new version the name of the directory of the executable changes, but the directory is added to the system PATH variable. Is it possible to use the system PATH variable in MSBuild? I tried the following code but it does not work:
<Exec Command=""$(PATH)\mytool.exe" />
If the directory has been added to the PATH variable, I would expect that the file path would be resolved automatically. I.e. you should be able to use:
<Exec Command="mytool.exe"/>
Maybe first check if the resolution is working ok by opening up a command prompt, cd'ing to some root directory that is different to where the 'mytool.exe' lives, and see if it is resolved when you simply enter "mytool.exe" at the prompt.

teamcity building project Failed to start MSBuild.exe. Illegal characters in path

I am trying to build a project within teamcity. It will do the checkout of TFS and the files are locally on the build server. If I manually run the build command on the server it works:
C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\msbuild.exe C:\TeamCity\buildAgent\work\3446a5bd436eea87\Dev\Dev\project\myproj\myproj.csproj /t:build
Within Teamcity I have a Build step of MSBuild:
the build fie path is: Dev\Dev\project\myproj\myproj.csproj"
working directory left blank
mbuild version: microsoft .net framework 4.0
msbuild toolsversion: 4.0
run platform: X86
targets: I have tried build "build" and left blank
command line params left blank
if I create a command line build task and run the above command line it works. I am not sure what differences there are between the command line and msbuild task within teammcity.
EDIT
I have tried a few variations on the working directory:
The output path in project is set to ..\Build\Tools\myproj\
So I have set the working directory in teamcity to: Dev\Dev\project\Build\Tools\myproj\
This didn't help.
Here is the exact output from teamcity:
Starting: C:\TeamCity\buildAgent\plugins\dotnetPlugin\bin\JetBrains.BuildServer.MsBuildBootstrap.exe /workdir:C:\TeamCity\buildAgent\work\3446a5bd436eea87\Dev\Dev\project\Build\Tools\myproj /msbuildPath:C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe
in directory: C:\TeamCity\buildAgent\work\3446a5bd436eea87\Dev\Dev\project\Build\Tools\myproj
Failed to start MSBuild.exe. Illegal characters in path.
System.ArgumentException: Illegal characters in path.
at System.IO.Path.CheckInvalidPathChars(String path)
at System.IO.Path.IsPathRooted(String path)
at JetBrains.TeamCity.Utils.FileUtil.MakeFullPath(String path, String workDir) in c:\BuildAgent\work\c2314fd21f15dc97\src\Utils\src\FileUtil.cs:line 48
at JetBrains.BuildServer.MSBuildBootstrap.RunArgs.get_ProjectFile() in c:\BuildAgent\work\c2314fd21f15dc97\src\MSBuildBootstrap\src\RunArgs.cs:line 156
at JetBrains.BuildServer.MSBuildBootstrap.Impl.MSBuildBootstrapFactory.Create(IClientRunArgs args) in c:\BuildAgent\work\c2314fd21f15dc97\src\MSBuildBootstrap.Core\src\Impl\MSBuildBootstrapFact ory.cs:line 29
at JetBrains.BuildServer.MSBuildBootstrap.Program.Run(String[] _args) in c:\BuildAgent\work\c2314fd21f15dc97\src\MSBuildBootstrap\src\Program.cs:line 67
Try editing the build configuration and remove the quotes from the Build file path option.
If you have:
"Dev\Dev\project\myproj\myproj.csproj"
Just let:
Dev\Dev\project\myproj\myproj.csproj
I hope this help.
In TeamCity, when you are using parameters for your path, or a portion of the path, it is easy to paste in a folder and not notice that it has a New Line or Carriage Return on the end (as I found from painful experience). It could show up in TeamCity as the value for a parameter:
\Folder1\NewlyPastedInFolder
\ExistingFolder\BuildScripts
What looks like a word-wrapped value is actually a parameter with a carriage return in the middle. Check it by editing it, and see if you can delete the character to make it into a single-line value.
This questions seems very similar:
What's the illegal character in this string ? I'm getting MSBUILD : error MSB3095
In addition to this question, take a look at:
What is a dll.refresh file in ASP.Net?
On this question, this answer is of interest:
In an ASP.NET project, adding a file-based reference will add a
.refresh file in the Bin folder. When the project is under source
control, this file is then added to source control. *.dll.refresh
files that litter the bin directory. Every time you add an external
reference, you'll find a dll.refresh file right next to it. These
dll.refresh files are an exception to the rule, and they should go
into source control. Its the only way your web project will know where
its references live.
If you don't have this in source control, it could explain the problem you are seeing. It sounds like the error message is potentially misleading and this is related to a reference not being resolved on the build machine.
With regards to why it works from the command line, is it possible that when you are building from TeamCity it is cleaning the output directory before building? If when you run from the command line binaries have already been output from the partially failed build which are referenced as file references, this would succeed.
To identify if this is the case - if you do a completely clean checkout and then run the MSBuild step from the command line, does it still succeed?
I ran into this error message from TeamCity and it turned out that a space in the solution file was causing it (the solution file referenced in the Visual Studio Build Steps had double quotes around it and that didn't seem to help.) Renamed the solution file without the space and that solved the problem.

Generating an MSBUILD project file from a visual studio solution file and project files

I know that I can pass MSBuild a VS solution file (.sln) and it will build the solution but somewhere in the back of my mind, I remember using a MSBuild command line switch that would take a solution file (and it's referenced project files) and generate a single MSBuild project file from them. However, I now can't find this switch!
Was this all a dream?
I see the /preprocess switch (short form /pp) but I don't think that this was it as this will include all of the imported files (including Microsoft.*.targets) and besides I can't seem to get this switch to work. I when I try the following command line, MSbuild generates the *.out file but its empty!
msbuild /target:rebuild /generate MSBuildCopyTargets.sln
The easiest way to do this is to run MSBuild from the command line, with an environment variable set:
Set MSBuildEmitSolution=1
The output will be in the format SolutionName.metaproj