Run MSBuild from Command Line task in Azure Devops - msbuild

I'm trying to execute msbuild on Azure Devops. Because of that I cannot use the MSBuild task provided.
When I use a Command Line task the command is not recognised. On my local machine I load vcvarsall.bat before I use msbuild. But I've not been unable to work out how to obtain that path in Azure Devops. Doesn't appear to be a Develop Command Prompt task for Azue Devops either.
Any ideas on how I can use msbuild from a Command Line task or Batch Script task? Using their Hosted VS agent.

The best way to do this in a supported way is to use vswhere. The following bit of script will install vswhere (using chocolatey) and then query the installer registry where msbuild can be found. Replace -latest with a more specific version if you need that:
choco install vswhere
for /f "tokens=*" %%i in ('vswhere -latest -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe -nologo') do set msbuildpath="%%i"
echo "##vso[task.setvariable variable=msbuildpath]%msbuildpath%"
This will save the path to msbuild to the environment variable %msbuildpath% as well as the pipeline variable (for this stage) $(msbuildpath).
You can then either use a second run commandline task and pass in $(msbuildpath) or you can simply call MsBuild from the same piece of script mentioned above by calling:
%msbuildpath%
This will make sure your script will remain working, even if Microsoft upgrades their images and moves some things around (which does happen).
You can also get vswhere using wget or invoke-webrequest -outfile from the following location:
https://github.com/Microsoft/vswhere/releases/latest/download/vswhere.exe
Other samples for vswhere syntax can be found on the project wiki, including the syntax for PowerShell.

If you use Hosted Agent 2017 you can run the msbuild.exe from the Command Line task in this way:
Command Line version 1:
Command Line version 2:
Results:

If you are interested in seeing how the built-in Microsoft task resolves the path, all the Azure Devops tasks are provided open-source. These are the path functions you probably care to review.

Here is the solution I came up with using only built-in pipeline tasks which makes the MSBuild bin directory available on the path environment variable.
Create a PowerShell task to generate an MSBuild project to capture and output to a file the variables you are interested in (ex. MSBuildBinPath)
PowerShell script
"<Project DefaultTargets=`"DetectMsBuild`">
<ItemGroup>
<OutFile Include=`"`$(MsBuildDetectionFile)`" />
<OutFile Condition=`"'`$(OutFile)' == ''`" Include=`"msbuildInfo.json`" />
</ItemGroup>
<Target Name=`"DetectMsBuild`">
<PropertyGroup>
<MsBuildPaths>
[{
`"Name`": `"BinPath`",
`"Value`": `"`$(MSBuildBinPath.Replace('\', '\\'))`"
}]
</MsBuildPaths>
</PropertyGroup>
<WriteLinesToFile
File=`"#(Outfile)`"
Lines=`"`$(MsBuildPaths)`"
Overwrite=`"true`"
Encoding=`"UTF-8`" />
</Target>
</Project>" | Out-File -FilePath "msbuilddetect.proj" -Encoding utf8
Set the working directory and any variables accordingly.
PowerShell task settings screenshot:
Create an MSBuild task to run the project file generated by the previous task. Ensure the MSBuild version is set to the version you want to use.
MSBuild task settings screenshot:
Last, create another PowerShell task that will parse the outputted JSON file of the extracted variables and sets environment variables accordingly.
PowerShell script
Write-Host "Current path: $($env.Path)`n`n"
$msBuildVariables = Get-Content -Path msbuildInfo.json | ConvertFrom-Json
$Path = "$($msBuildVariables[0].Value);$($env:Path)"
Write-Host "##vso[task.setvariable variable=Path;]$Path"
PowerShell task settings screenshot:
Here is a screenshot of the task order in the build pipeline.

Related

USQL msbuild - composite build output

I'm following the instructions on this link https://blogs.msdn.microsoft.com/azuredatalake/2017/10/24/continuous-integration-made-easy-with-msbuild-support-for-u-sql-preview/
It states that
After running MSBuild from command line or as a VSTS task, all scripts in the U-SQL project are built and output to a single file at "Build output path/script name/script name.usql". You can copy this composite U-SQL script to the release folder for further deployment.
Within my Visual Studio project (.usqlproj) I have multiple .usql scripts
CreateDatabase.usql
CreateTable.usql
CreateTvf.usql
when I do clean and msbuild and then check bin\debug folder, all I get is just CreateDatabase.usql and within that there is only CREATE DATBASE statement. As per the blog I would have thought all the 3 usql scripts would have merged into 1 composite usql script. The msbuild command I executed from command prompt on my machine
msbuild TheProject\TheProject.usqlproj /t:Clean /t:Rebuild /property:USQLSDKPath=C:\TheProject\src\packages\Microsoft.Azure.DataLake.USQL.SDK.1.3.1019-preview\build\runtime,USQLTargetType=Merge
I'm using Visual Studio 2017 15.4.3 and Azure Data Lake tools 2.3.0000.1
What am I doing wrong?
Due to some legacy reasons, to make sure all files in the usqlproj are built, the condition are defined in
USqlSDKBuild.targets as below
<ItemGroup>
<FileToBuild Condition="'$(JustOneFile)' == '' and
'$(Build_all_files_in_this_project)' != 'true'" Include="$(ActiveFile)" />
<FileToBuild Condition="'$(JustOneFile)' != '' " Include="$(JustOneFile)" />
<FileToBuild Condition="'$(Build_all_files_in_this_project)' == 'true'"
Include="Build_all_files_in_this_project" />
</ItemGroup>
For current preview release, please add "/property:Build_all_files_in_this_project=true,JustOneFile=''" to the command line to make sure all scripts are built. We will update this in later releases with a easier and simpler condition.
The command msbuild TheProject\TheProject.usqlproj /t:Clean /t:Rebuild /property:USQLSDKPath=C:\TheProject\src\packages\Microsoft.Azure.DataLake.USQL.SDK.1.3.1019-preview\build\runtime,USQLTargetType=Merge actually build the script you selected.
If you are selecting CreateDatabase.usql in VS, it will build the script CreateDatabase.usql in command line.
As below example, since it's selecting test.usql, when execute msbuild command, it will only build test.usql. As the way clicking Build button for the USQL project.
If you want to build all the scripts under the USQL project, you should select Build All Scripts button in VS.

MSBUILD : error MSB1001: Unknown switch Switch: /Y

I'm using team city to make an automatic deploy and MSBuild won't work...
In the build step the the command line parameters look like this:
ProjectName.deploy.cmd /y /M:https://[WebDeployUrl:8172]/MsDeploy.axd /u:username /p:password –allowUntrusted /A:basic
this works fine from my machine, but the build server fails with the following response:
[MSBuild output] MSBUILD : error MSB1001: Unknown switch.
[MSBuild output] Switch: /Y
Anyone has an idea about this?
This is a very old question, and the asker likely sorted out their issue long ago, but here goes anyways:
MSBuild tasks in TeamCity require a commandline that pertain to MSBuild.exe specifically, IIRC.
That is, TeamCity is executing MSBuild.exe with the parameters you've given it like so:
msbuild.exe ProjectName.deploy.cmd /y /M:https://[WebDeployURL]:8172]/MsDeploy.axd /u:username /p:password -allowUntrusted /A:basic
Of course, MSbuild has no idea what those switches are, nor would it be able to process 'ProjectName.deploy.cmd' as an MSBuild file.

Lightswitch - getting it to build in TeamCity

I'm trying to get a Lightswitch Project into Teamcity and have tried the following runner types:
Visual Studio (sln)
MSBuild
Command line (ran MSBuild through the command line)
All 3 runner types gave me the same error when building the Lightswitch solution:
The "UnpackExtensionsToProjectDir" task failed unexpectedly. System.NullReferenceException: Object reference not set to an instance of an object.
Lightswitch has already been installed on the server. Have tried building the solution manually using Visual Studio on the server and it builds fine. Have also tried building the solution via the command line (using MSBuild) and it builds fine too.
Would like to ask if somebody was able to get Lightswitch building nicely on TeamCity. Cheers.
This is how you build via the Command line (using TeamCity)
Pre-requisites)
First make sure you have not checked in the extensions directory, this can cause issues when building.
Check that you have installed any visual studio extensions on the build machine .ie ExtensionsMadeEasy. You can test this by opening the solution in visual studio on the build machine and trying to do a build.
Lastly, in TeamCity do not use the msbuild task, use command line to call msbuild.
Step 1)
msbuild.exe mylightswitchproject.lsproj /p:OutDir=C:\test\stuff\;configuration=Release
Step 2)
Create a bat file to copy your output to the correct folder structure.
robocopy C:\test\stuff\bin C:\test\localrelease\bin *.* /MIR
robocopy C:\test\stuff\Resources C:\test\localrelease\Resources *.* /MIR
robocopy C:\test\stuff\Web C:\test\localrelease\Web *.* /MIR
robocopy C:\test\stuff\ C:\test\localrelease\ ClientAccessPolicy.xml
robocopy C:\test\stuff\ C:\test\localrelease\ default.htm
robocopy C:\test\stuff\ C:\test\localrelease\ Home.aspx
robocopy C:\test\stuff\ C:\test\localrelease\ Login.aspx
robocopy C:\test\stuff\ C:\test\localrelease\ LogOff.aspx
robocopy C:\test\stuff\ C:\test\localrelease\ Silverlight.js
robocopy C:\test\stuff\ C:\test\localrelease\ web.config
You can now take this folder and release it to the next environment.
Finally, if you want to create a web deployment package, out the box visual studio 2010 does not support this. However, you can copy this into an existing website then "Export" your application into a package that is then ready for web deployment via powershell.
The previous answers didn't work for us but Yaegor's answer provided some direction.
The issue we had was extensions are installed at the user level, not the system level. This meant the MSBuild process could not find the required extensions.
Our solution was to use a user account on the build server, log into account, setup VS.NET such that the LS project builds, and then switch the TeamCity agent service to use the new user account.
With this we were able to use the Solution runner (which is preferable to the CLI runner since it provides better logging and reporting).
For not Lightswitch-specific part: If command line works from console, but fails in TeamCity, most probably the issue is in the user or running as a service. You might try running TeamCity agent with the same environment.
When command line works you can then try MSBuild and Solution runners.
I ran into the same error when trying to set up an automated build for a lightswitch application using bamboo. Turned out to be the version of msbuild being called. If the 64bit version is called (from bamboo or the command line) I get the error:
UnpackExtensionsToProjectDir" task failed unexpectedly.
Switching to the 32bit version of msbuild fixes the problem.
32bit Path: 'C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe'
64bit Path: 'C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe'

How to call msbuild from a batch file from msbuild from team build?

I'm using Team Build (2010) to call an msbuild script with an Exec task that calls a batch file that in turn calls msbuild. Like this:
<Exec Command="BatchFileThatCallsMSBuild.bat" />
Of course the batch file does a bunch of other junk or I'd just use the MSBuild task.
The problem is that when the batch file tries to call msbuild it can't find it.
'msbuild' is not recognized as an internal or external command, operable program or batch file.
How do I get the necessary environment set up in the exec task?
I tried changing the command to:
<Exec Command="%22$(VS100COMNTOOLS)..\..\VC\vcvarsall.bat%22&BatchFileThatCallsMSBuild.bat" />
but no dice, still msbuild is not found.
The answer I came up with was to take advantage of the seldom-demonstrated-online multi-line Command string to the Exec task.
<Exec Command="call "%VS100COMNTOOLS%..\..\VC\vcvarsall.bat" x86
set AnotherEnvVar=$(RandomMSBuildProperty)
call BatchFileThatCallsMSBuild.bat
type file_with_output_from_the_msbuilds_in_the_batchfile.log" />
This let me set up the basic build environment (call to vcvarsall), push an msbuild property out to the Exec's environment where the batched msbuilds could see it, call the batch file, and even pull the hidden msbuild output up to the level of the Exec task for clearer logging in Team Build.
I'm not thrilled with having to embed yet another reference to this specific VS version in my code, but it works for now.

How can you publish a ClickOnce application through CruiseControl.NET?

I have CruiseControl.NET Version 1.4 set up on my development server. Whenever a developer checks in code, it makes a compile.
Now we're at a place where we can start giving our application to the testers. We'd like to use ClickOnce to distribute the application, with the idea being that when a tester goes to test the application, they have the latest build.
I can't find a way to make that happen with CruiseControl.NET. We're using MSBUILD to perform the builds.
We've done this and can give you some pointers to start.
2 things you should be aware of:
MSBuild can generate the necessary deployment files for you.
MSBuild won't deploy the files to the FTP or UNC share. You'll need a separate step for this.
To use MSBuild to generate the ClickOnce manifests, here's the command you'll need to issue:
msbuild /target:publish /p:Configuration=Release /p:Platform=AnyCPU; "c:\yourProject.csproj"
That will tell MSBuild to build your project and generate ClickOnce deployment files inside the bin\Release\YourProject.publish directory.
All that's left is to copy those files to the FTP/UNC share/wherever, and you're all set.
You can tell CruiseControl.NET to build using those MSBuild parameters.
You'll then need a CruiseControl.NET build task to take the generated deployment files and copy them to the FTP or UNC share. We use a custom little C# console program for this, but you could just as easily use a Powershell script.
Thanks for all the help. The final solution we implemented took a bit from every answer.
We found it easier to handle working with multiple environments using simple batch files. I'm not suggesting this is the best way to do this, but for our given scenario and requirements, this worked well. Supplement "Project" with your project name and "Environment" with your environment name (dev, test, stage, production, whatever).
Here is the tasks area of our "ccnet.config" file.
<!-- override settings -->
<exec>
<executable>F:\Source\Project\Environment\CruiseControl\CopySettings.bat</executable>
</exec>
<!-- compile -->
<msbuild>
<executable>C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe</executable>
<workingDirectory>F:\Source\Project\Environment\</workingDirectory>
<projectFile>Project.sln</projectFile>
<buildArgs>/noconsolelogger /p:Configuration=Debug /v:diag</buildArgs>
<targets>Rebuild</targets>
<timeout>0</timeout>
<logger>ThoughtWorks.CruiseControl.MsBuild.XmlLogger,ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>
<!-- clickonce publish -->
<exec>
<executable>F:\Source\Project\Environment\CruiseControl\Publish.bat</executable>
</exec>
The first thing you will notice is that CopySettings.bat runs. This copies specific settings for the environment, such as database connections.
Next, the standard MSBUILD task runs. Any compile errors are caught here and handled as normal.
The last thing to execute is Publish.bat. This actually performs a MSBUILD "rebuild" again from command line, and parameters from CruiseControl are automatically passed in and built. Next, MSBUILD is called for the "publish" target. The exact same parameters are given to the publish as the rebuild was issued. This keeps the build numbers in sync. Also, our executables are named differently (i.e. - ProjectDev and ProjectTest). We end up with different version numbers and names, and this allows ClickOnce to do its thing.
The last part of Publish.bat copies the actual files to their new homes. We don't use the publish.htm as all our users are on the network, we just give them a shortcut to the manifest file on their desktop and they can click and always be running the correct executable with a version number that ties out in CruiseControl.
Here is CopySettings.bat
XCOPY "F:\Source\Project\Environment\CruiseControl\Project\app.config" "F:\Source\Project\Environment\Project" /Y /I /R
XCOPY "F:\Source\Project\Environment\CruiseControl\Project\My Project\Settings.Designer.vb" "F:\Source\Project\Environment\Project\My Project" /Y /I /R
XCOPY "F:\Source\Project\Environment\CruiseControl\Project\My Project\Settings.settings" "F:\Source\Project\Environment\Project\My Project" /Y /I /R
And lastly, here is Publish.bat
C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe /target:rebuild "F:\Source\Project\Environment\Project\Project.vbproj" /property:ApplicationRevision=%CCNetLabel% /property:AssemblyName="ProjectEnvironment" /property:PublishUrl="\\Server\bin\Project\Environment\\"
C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe /target:publish "F:\Source\Project\Environment\Project\Project.vbproj" /property:ApplicationVersion="1.0.0.%CCNetLabel%" /property:AssemblyVersion="1.0.0.%CCNetLabel%" /property:AssemblyName="ProjectEnvironment"
XCOPY "F:\Source\Project\Environment\Project\bin\Debug\app.publish" "F:\Binary\Project\Environment" /Y /I
XCOPY "F:\Source\Project\Environment\Project\bin\Debug\app.publish\Application Files" "F:\Binary\Project\Environment\Application Files" /Y /I /S
Like I said, it's probably not done the way that CruiseControl and MSBUILD developers had intended things to work, but it does work. If you need to get this working yesterday, it might be the solution you're looking for. Good luck!
I remember doing this last year for a ClickOnce project I was working on. I remember it taking me forever to figure out but here it is. What I wanted my scripts to do was to generate a different installer that pointed to our dev env and a different one for prod. Not only that but i needed it to inject the right versioning information so the existing clients would 'realize' there is a new version out there which is the whole point of clickOnce.
In this script you have to replace with your own server names etc. The trick is to save the publish.htm and project.publish file and inject the new version number based on the version that is provided to you by CC.NET.
Here is what my build script looked like:
<target name="deployProd">
<exec program="<framework_dir>\msbuild.exe" commandline="<project>/<project>.csproj /property:Configuration=PublishProd /property:ApplicationVersion=${build.label}.*;PublishUrl=\\<prod_location>\binups$\;InstallUrl=\\<prod_location>\binups$\;UpdateUrl=\\<prod_location>\binups$\;BootstrapperComponentsUrl=\\<prod_location>\prereqs$\ /target:publish"/>
<copy todir="<project>\bin\PublishProd\<project>.publish">
<fileset basedir=".">
<include name="publish.htm"/>
</fileset>
<filterchain>
<replacetokens>
<token key="CURRENT_VERSION" value="${build.label}"/>
</replacetokens>
</filterchain>
</copy>
</target>
Hope this helps
Just be able passing the ${CCNetLabel} in the CCNET.config msbuild task would be a great improvement.
You want to use the ClickOnce manifest generation tasks in msbuild. The process is a little long winded, so I am just going to point you to a couple of links. Here is the reference on msdn and a sample article to hopefully get you started.