Generate compile_commands.json from msbuild via command line -- no cmake - msbuild

I'm looking for a way to integrate clang-tidy into a CI workflow, but the build system being used is MSBuild with dependencies managed by vcpkg in manisfest mode.
Is there some advanced command line that I can pass MSBuild (or some other tool that understands MSBuild process completely) to be able to generate compile_commands.json?
I'm sure I'm not the first one to try that, I've seen a couple of cases of success using SourceTrail and ClangPowerTools, but it has been specially painful in my case because the alternatives cannot detect the include paths exposed by vcpkg.

It turns out that msbuild has a builtin target for clang-tidy since Visual Studio started to support that linter.
To invoke it one can run : msbuild /t:ClangTidy ....
Unfortunately, very few command line options are exposed from the clang-tidy integration to msbuild CLI: the list of checks, the header-filter, additional compiler options, a flag for warnings in system headers and the tool path.
If one wants to export the fixes file generated by clang-tidy, which was my case, it needs to do some extra work.
I achieved my goal by adding one Directory.Build.props file in the project folder which overrides the path to the original clang-tidy.exe with a batch script (.bat), which does some preparation steps and forwards the msbuild target command line to a python script which can invoke clang-tidy with its full command line capabilities.
The props file:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<ClangTidyToolPath>$(where_I_saved_the_bat_file)\</ClangTidyToolPath>
<ClangTidyToolExe>msbuild-clang-tidy-runner.bat</ClangTidyToolExe>
</PropertyGroup>
</Project>
The batch file:
SET PYTHONPATH=...
REM and maybe other stuff
python3 -m my_custom_clang_tidy_runner %*
Interestingly the batch file was the only way other than a real executable file that I could place in the Directory.Build.props that wouldn't result in msbuild compilation error. I tried python and powershell scripts and none were accepted as a suitable clang-tidy executable file. I experimented creating a C++ program which does the same as my Python script and it worked as expected as well.
So, to summarize, the recipe is:
Create the Directory.Build.props file in the root folder of the project you want to lint overriding the clang-tidy path.
A custom .exe or .bat file to replace clang-tidy capable of forwarding the command line options passed by msbuild together with any customisation you wish.
Invoke msbuild /t:ClangTidy ... and Bob's your uncle!

Related

How to specify environment variables in .csproj file

How can I, through the .csproj file, specify environment variables to apply during the build, such as when building with Rider?
Specifically, I want to set DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 so that I can build from within the IDE, without resorting to the command line (see below).
Adding <InvariantGlobalization>true</InvariantGlobalization> to the project file did not work, since that does not affect the underlying/imported build target, but executing export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 before dotnet build NAME.csproj worked.
Background: A recent distro upgrade on openSuse Tumbleweed bricked MonoGame project builds with the following error message when invoking mgcb (re-installing ICU with Zypper did not fix the issue):
Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.
(...)
error MSB3073: The command "dotnet (...) exited with code 134
Edit: I have finally gotten a Target to run before the Nopipeline target using InitialTargets. The problem now is the Exec task runs in a discarded scope.
Depending on which task fails, you can hope that the target author added an option to set the environment variables.
E.g. for C# compilation, you could set:
<PropertyGroup>
<CscEnvironment>$(CscEnvironment);DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1</CscEnvironment>
</PropertyGroup>
If the tool is run by a custom target, you would need to author a replacement target that allows setting the environment variable.

Can I specify fileloggerparameters in msbuild project file

I see that you can use the /fileLogger and /fileloggerparameters command line arguments in msbuild to specify things like the location of the log file. Is there any way to specify this information in the Project or PropertyGroup section of the project file? I have all my other project properties imported via an include file. I really don't want to have to one set of properties in an include file and then another set that is specified on the command line.
As far as I'm aware just VC projects has ability log build into separate files. But you have to build it through devenv :-(
and you don't have control over other logging parameters.
Microsoft.Cpp.Default.props
<ItemDefinitionGroup>
<BuildLog>
<Path>$(IntDir)\$(MSBuildProjectName).log</Path>
</BuildLog>
</ItemDefinitionGroup>
Other ugly way is to execute build of each project via
<Exec Command="msbuild.exe project /fl /flp...." />
I guess you want to avid it.
I'm think it could be possible to create custom distributed file logger to do this but I sill don't have it working properly.

MSBUILD - block until a file exists on an ftp server?

As part of a build process .. I would like to block the build until a file gets created (exists) remotely at an ftp location, after which I would continue the build. (Preferably with somekind of time out limit).
Suggestions?
Is this even possible using only the standard msbuild task and/or extensionPack/communitytask?
Your best bet is to build a small custom exe (you can even compile it as a build step) that polls for the file you are looking for. Then you use the PreBuild target, or a custom target in a pre-build step to verify that the file exists.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="WaitOnFTP">
<Exec Command="MyFTPWaiter.exe"/>
</Target>
</Project>
Other more MSBuild oriented suggestions are to remake that exe as a custom task, or even an inline task in MSBuild 4.0.
FWIW, I've encountered a similar solution done by a peer who didn't want large binaries used by integration tests in version control and he required the use of a custom downloader in the build to get the files from a SMB share. It worked well enough.
Custom Tasks
Inline Tasks

How can I execute programs in my %PATH% with MSBuild?

Note: I'm using Mercurial as an example here, because that's what I'm trying to get to work with MSBuild right now.
But the problem is not limited to Mercurial, it happens with every external program that is somewhere in my %PATH% variable (I tried the same with PowerShell, for example).
So I didn't put the Mercurial tag on this question on purpose, because this is not about Mercurial!
What I actually want to do:
I want my build script to get the current revision number from my Mercurial repository and store it in a file.
The simplest way to do this from the command line is:
hg id -i >rev.txt
Mercurial is installed on my machine and the installation folder is in my %PATH% variable.
So I can run this line from anywhere on my machine (directly from the command line, or from a batch file), and it just works.
The problem occurs when I try to run this line from my build script.
I change the BeforeBuild (or AfterBuild) section of my .csproj file as follows:
<Target Name="AfterBuild">
<Exec Command="hg id -i >rev.txt"/>
</Target>
When I compile my solution with Visual Studio, it works and the rev.txt file is created in the folder where my .csproj is.
But when I compile the exact same solution from the command line with MSBuild, the build fails with the following error message:
The command "hg id -i >rev.txt" exited with code 9009.
I googled "msbuild code 9009" and found some solutions, but all of them propose to provide the full path to the executable.
When I do this, the build succeeds with MSBuild as well.
But this is not an acceptable solution for me, because I can't be sure that everyone using my project (including the build server) has installed Mercurial in the exact same folder.
That's exactly what %PATH% is for...
The same happens when I put the <Exec Command="... line directly into the build script.
If I specify the path to the executable, it works.
If I don't specify the path, it doesn't.
Is there any trick to make MSBuild execute programs in my %PATH% variable without specifying the complete folder?
EDIT:
#leppie:
Output redirection:
You mean the fact that I save the output of my command in a text file inside the command , instead of just running hg id -i as a command and using an output parameter or something like that to get the output?
Doesn't make any difference...the error is the same when I omit >rev.txt.
Command line args:
No, it throws the same error, even if I shorten the command to just hg (without any parameters).
Don't forget: if I run the exact same Exec command in the exact same .csproj file from Visual Studio, or if I just provide the path to the .exe file in the command, everything works.
So IMO output redirection and command line args can't be the problem.
Have you tried this extension pack for mercurial/msbuild?
http://msbuildhg.codeplex.com/documentation
Seems to have a task for returning revision id, which is what your trying to achieve no?
<HgVersion LocalPath="$(MSBuildProjectDirectory)" Timeout="5000">
<Output TaskParameter="Revision" PropertyName="AssemblyRevision" />
</HgVersion>
Okay, I found the solution.
I have to admit, it was a classic case of PEBKAC :-)
I'll explain it anyway, maybe it will help someone who made the same mistake:
Basically everything I have tried (plus what James Woolfenden suggested in his answer) would have been worked...if only the batch file that I use to run the build script wouldn't have looked like this:
path="%windir%\Microsoft.net\Framework\v4.0.30319"
msbuild build.proj
Yes, exactly.
I'm editing the %PATH% variable for the duration of this batch file, and I'm overwriting it with the path to MSBuild instead of just appending it.
So when my build script tries to call Mercurial, it can't find it anymore because its location is not in the %PATH% variable anymore.
No idea why I didn't see this before.
The correct way would be to append the MSBuild path, leaving the other paths intact:
path=%path%;%windir%\Microsoft.net\Framework\v4.0.30319

Generating an MSBUILD project file from a visual studio solution file and project files

I know that I can pass MSBuild a VS solution file (.sln) and it will build the solution but somewhere in the back of my mind, I remember using a MSBuild command line switch that would take a solution file (and it's referenced project files) and generate a single MSBuild project file from them. However, I now can't find this switch!
Was this all a dream?
I see the /preprocess switch (short form /pp) but I don't think that this was it as this will include all of the imported files (including Microsoft.*.targets) and besides I can't seem to get this switch to work. I when I try the following command line, MSbuild generates the *.out file but its empty!
msbuild /target:rebuild /generate MSBuildCopyTargets.sln
The easiest way to do this is to run MSBuild from the command line, with an environment variable set:
Set MSBuildEmitSolution=1
The output will be in the format SolutionName.metaproj