I've been having an adventure with the new u-sql msbuild support, fighting dragons etc in the lands of "preview build".
I realised many of the gotchas were due to discovering inconsistencies between running the build in VS2017 v.s. running msbuild from the command line.
Against my better judgement I added some diagnostic messages by altering the nuget download directly:
\packages\Microsoft.Azure.DataLake.USQL.SDK.1.3.1019-preview\build\runtime\USqlSDKBuild.targets
I realise now that USQLSDKPath doesn't override what version is used in visual studio.
And that USQLTargetType* isn't supported within visual studio at the time of writing.
*USQLTargetType is needed if you want to compile scripts without checking for pre-existing schema objects, to avoid a chicken and egg scenario.
The versions I've found on my machine are:
C:\Users[user name]\AppData\Roaming\Microsoft\DataLake\MsBuild\1.0\
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\ADL Tools\2.3.3000.5\
And the one installed via nuget relative to my sln.
Can I assume the first in AppData is installed as part of the Visual Studio installation and the second is the visual studio extension from the marketplace?
I've been battling for a few days on various elements so wondered how far others have got in this direction? I also realise this is in preview mode, so has rough edges.
Am I safe to alter:
<Import Project="UsqlSDKBuild.targets" Condition="'$(BuildingInsideVisualStudio)' != 'true' And Exists('UsqlSDKBuild.targets')" />
<Import Project="$(USQLSDKPath)\UsqlSDKBuild.targets" Condition="'$(BuildingInsideVisualStudio)' != 'true' And !Exists('UsqlSDKBuild.targets') And '$(USQLSDKPath)' != '' And Exists('$(USQLSDKPath)\UsqlSDKBuild.targets')" />
<!-- backward compatible with IDE build -->
<Import Project="$(AppData)\Microsoft\DataLake\MsBuild\1.0\Usql.targets" Condition="'$(BuildingInsideVisualStudio)' == 'true'" />
To force using a later version in the IDE as well?
Forcing Visual Studio to use the nuget downloaded USQL SDK doesn't seem viable at the moment.
Even if I deleted all statements at the bottom of the usqlproject and tweaked USqlSDKBuild.targets to allow the project to be loaded without error (I had problems with the UsingTask)....
I discovered that Visual Studio was overriding the USQLSDKPath back to:
USQLSDKPath='C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\Extensions\Microsoft\ADL Tools\2.3.3000.5\U-SQLSDK'
Meaning I'm unable to use the nuget version reliably.
In that same directory I did notice "USqlDBProject.targets" (what sounds like the new upcoming project type) and that USqlSDKBuild.targets has more options. Both indicating that the nuget package is infact older than the extension's SDK.
I'm considering that infact the one bundled with the Visual Studio extension is infact newer than the nuget version!
EDIT
After semi concluding there are different versions at play, I went back to basics and - doh - realised that the nuget package is out of date!
I made this mistake because the u-sql project doesn't support the nuget in Visual Studio so I needed to add it by hand to the packages.config, hardcoding it to "Microsoft.Azure.DataLake.USQL.SDK.1.3.1019-preview" as mentioned in https://blogs.msdn.microsoft.com/azuredatalake/2017/10/24/continuous-integration-made-easy-with-msbuild-support-for-u-sql-preview/
I wrongly assumed the the blog post was up to date, due to it being linked to within the relatively recently promoted spring 2018 release notes. But double checking the blog post obviously pre-dates the release notes. My bad for coding tired.
Related
I work in Visual Studio. It seems for me, that, if I change anything in imported .targets file, the new version of .targets is not reimported to main project while building it. If I reopen my project (I.e. close and open again Visual Studio) and then launch build - the new version of .targets works.
Is it possible to tell Visual Studio to import .targets at any rebuild?
I tried use "rebuild" and "clean" solution and project, without any result regarding the problem.
Re-evaluating imported files is done by the new CPS-based project systems that are used for .NET Core / .NET Standard projects.
For VS versions before VS 2019, you may also need to indicate to MSBuild that the imported targets file should also re-trigger incremental build by adding its path to $(MSBuildAllProjects) (this is no longer needed in MSBuild 16 / VS 2019):
<PropertyGroup>
<MSBuildAllProjects>$(MSBuildAllProjects);$(MSBuildThisFileFullPath)</MSBuildAllProjects>
</PropertyGroup>
But for "classic" .NET Framework / ASP.NET projects, you still need to close and re-open the solution for changes to take effect inside the IDE.
Also see this GitHub tracking issue for the legacy project system.
For Noda Time version 1.1, the main goal is to build a Portable Class Library flavour, primarily to support Windows Phone and Windows Store apps. This means losing some functionality, so we build a desktop configuration and a PCL configuration (for each of debug, release, and "signed release").
To avoid having to work with umpteen project files, all of the 6 configurations exist in the same project file. The project file is customized to generate a property called "Portability", which is set to either "PCL" or "Desktop", like this:
<!-- Set the custom Portability property based on configuration -->
<PropertyGroup>
<Portability Condition="'$(Configuration)' == 'Debug Portable'">PCL</Portability>
<Portability Condition="'$(Configuration)' == 'Release Portable'">PCL</Portability>
<Portability Condition="'$(Configuration)' == 'Signed Release Portable'">PCL</Portability>
<!-- Default to desktop if not explicitly set above -->
<Portability Condition="'$(Portability)' == ''">Desktop</Portability>
</PropertyGroup>
We then have separate property groups for portable vs desktop, based on the above property. This is what defines the project type as "class library" or "portable class library" (along with the OutputType of Library, which is shared):
<!-- Desktop-specific properties -->
<PropertyGroup Condition="'$(Portability)' == 'Desktop'">
<TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
<TargetFrameworkProfile>Client</TargetFrameworkProfile>
</PropertyGroup>
<!-- PCL-specific properties -->
<PropertyGroup Condition="'$(Portability)' == 'PCL'">
<MinimumVisualStudioVersion>10.0</MinimumVisualStudioVersion>
<ProjectGuid>{c78f6992-28d7-45c9-a4c1-6eaa649f3247}</ProjectGuid>
<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>
<TargetFrameworkProfile>Profile2</TargetFrameworkProfile>
<ProjectTypeGuids>{786C830F-07A1-408B-BD7F-6EE04809D6DB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
</PropertyGroup>
This generally works very well - I have the different solution configurations, so I can build and test everything at a moment's notice, and I only need to add each new .cs file to a single project file. So under Visual Studio 2012 Professional (which is what I use) I'm perfectly happy.
The problem comes when I try to load the solution in Visual Studio Express (either VS2010 or VS2012). While the solution is loading, it fails with an error to say that some projects can't be loaded, and the two projects which build PCL versions then have build output like this:
C:\Path\To\NodaTime.csproj : error :
The project file 'C:\Path\To\NodaTime.csproj' cannot be opened.
There is a missing project subtype.
Subtype: '{786C830F-07A1-408B-BD7F-6EE04809D6DB}'
is unsupported by this installation.
(Reformatted for clarity.) The two projects refuse to load, so you can't even browse the source code.
I had really hoped that even if Express users couldn't build the PCL versions, they'd still be able to load up the solution, browse the source, and build non-PCL versions. MSBuild works from the command line, but that's not as friendly.
I've tried removing the solution configurations which refer to the PCL project configurations, and that doesn't help. Weirdly enough, even commenting out the XML element, like this:
<!--
<ProjectTypeGuids>(Guids as before)</ProjectTypeGuids>
-->
doesn't help - although deleting the line does. It's as if Visual Studio isn't actually loading it as a real XML file. (I haven't tried loading the version with the commented out element into VS Pro.)
I could go down the route of generating separate PCL project files if I need to, but I'd really like to avoid it if possible - it would make normal development more painful. Likewise I could generate Express-only PCL and solution files, but again I'd rather not - it just feels wrong.
While ideally I'd like to support VS Express both 2010 and 2012, if there's a solution which only works for 2012, that would be a good start.
So, is there any way of persuading Visual Studio Express that it really can load a project despite a conditional property group (whose condition isn't met) referring to a project type it doesn't know about?
David Kean's comment here gave me the answer I'm using for the moment:
or remove the <ProjectTypeGuid> element entirely - this will opt you of "portable" enhancements, such as a UI for changing the target framework, etc
I've tried that, and it works like a dream. On machines which have everything appropriately installed, you can then even build the PCL version under Express! I've verified that the resulting binary really is a PCL, and it seems fine.
I wouldn't be surprised to find that I ran into some problems later on, but for the moment this works fine for me. I can easily live without the enhancements in Visual Studio - it was already confused by my project having very different build configurations, so I don't think I was getting much benefit anyway.
As answered here:
Visual Studio 2012 Express with Portable Class Library?
and here:
Share functionality using Portable Class Libraries
Portable Class library projects are not supported in the express
SKU...need a higher SKU for the full support. Of course the binary
(e.g., using it as a reference) is, just not the project/source
support.
I can imagine there is a simple reason for that - there are different types of VS2012 Express editions: for Windows Phone development, for Desktop apps, for Windows 8 apps... I bet the Windows Phone Express edition does not know about the Windows Phone project type and vice versa. This could be the simple reason why PCLs are not supported as well.
Although the idea of Portable Class Libraries is really nice, it's still quite limited in many ways, for instance you cannot use conditional compilation using #if xy as far as I know. If you really have to use Visual Studio Express for development, then it might be better to use projects for each platform with referenced source files and conditional compilation.
We current do manual builds/publish from Visual Studio 2010 and we require users to always be running the latest version (check before startup and minimum required version set). I am working on scripting our deployment out and have no issues using msbuild to build/publish. However, I have not found a way to auto-increment the minimum required version when msbuild runs. What are my options to automatically bump this when publishing via msbuild?
I did see quite a few articles on this topic here, but they seemed to be specific to VS and not MSBuild.
Updating the MinimumRequiredVersion Automatically
Introduction to Project Editor
In Solution Explorer, right click on your project and select unload project.
Once the project has become unavailable, right click again and select edit <project_name>.<lang> proj.
Introduction to MSBuild
Properties use key/value pairs to extract information
Using the property name as an alias, you can use $(OutputPath) to obtain the value for the element <OutputPath>.\bin</OutputPath>
We’ll use the following properties generated for a ClickOnce deployment
<MinimumRequiredVersion>1.0.0.6</MinimumRequiredVersion>
<ApplicationRevision>7</ApplicationRevision>
<ApplicationVersion>1.0.0.%2a</ApplicationVersion>
MSBuild Tasks can be specified in the project (*.proj) file and invoked during a build event.
FormatVersion is a built-in task for .NET 4.0 and later that formats the ApplicationVersion and ApplicationRevision into a single version number.
Implementation
Copy and Paste the following code into the opened project file as a child element to the root <Project> element.
<Target Name="AutoSetMinimumRequiredVersion" BeforeTargets="GenerateDeploymentManifest">
<FormatVersion Version="$(ApplicationVersion)" Revision="$(ApplicationRevision)">
<Output PropertyName="MinimumRequiredVersion" TaskParameter="OutputVersion" />
</FormatVersion>
<FormatVersion Version="$(ApplicationVersion)" Revision="$(ApplicationRevision)">
<Output PropertyName="_DeploymentBuiltMinimumRequiredVersion" TaskParameter="OutputVersion" />
</FormatVersion>
</Target>
This code will take ApplicationVersion and ApplicationRevision as parameters in the Format Version task and will save the output by overwriting the MinimumRequiredVersion with the full publish version.
Save and reload your project. Every ClickOnce deployment will now automatically update to the most recently published version.
Many thanks to Kev for their answer which I have basically rehashed here with a little bit of added clarification for any beginners. Here's a blog post I made about the issue that expands even more on my answer here.
Right now, I'm leaning towards updating the MinimumRequiredVersion via a custom command-line utility that will simply read in the project file and increment it. It's the only option I've come up with for scripting out my build.
I'm not sure if you're going about this the best possible way.
I would recommend using a continuous integration (CI) server like Team City that is responsible for deployments. If having the latest version of the source code is a requirement for publishing, then that is probably something you should build into a well-tested CI build configuration, and take away from the hands of the potentially forgetful/occasionally error-prone users.
Since you are wanting to publish using MSBuild and not Visual Studio, I assume you are publishing from a build server or using some sort of script. You can use the Set-ProjectFilesClickOnceVersion PowerShell script to both set the ClickOnce Application Version as well as force the Minimum Required Version to be the latest version. You would want to do this before running MSBuild. My blog describes in more detail how to setup your build server to accommodate publishing ClickOnce applications.
It seems like it's still not possible to build .vdproj on a build-server without having Visual Studio installed. However, using Wix seems to be a lot more complicated.
Are there any other options to do the following task:
Visual Studio 2010 Solution with multiple projects (.csproj)
Many loose content files (not inside assemblies)
Installer must be built on Build-server without Visual Studio on it (devenv.exe / devenv.com)
Installer must create Registry keys
Installer must associate file extensions with installed product
Installer must support upgrades (version upgrades)
Installer should be able to register COM components
Installer should be able to pre-JIT assemblies
My goal is:
Effort to maintain installer is low
Minimal changes if new project (assembly) is added to solution
Ideal: no changes if new content files are added to any of the projects
Maybe I just did not get the point with Wix, but including project output (like in .vdproj) seems very complicated.
Any suggestions very much appreciated!
OK I decided to go with Wix. Found out that using Votive it's possible to include project output like in .vdproj (Binaries, Content, Symbols and even Source Code).
If I run into more complicated situations where this isn't sufficient, I could fall back to some kind of harvester (Heat, Paraffin).
See some more Links in comments below (too little reputation to post more than 1 link in this post).
Check out InstallShield 2010 Limited Edition ( Free for Visual Studio 2010 users ). It has pretty much all the functionality that VDPROJ ( being retired btw ) has and supports silent builds. You can get one license for your dev box and one license for your build machine pretty easily.
We want to create an .MSI package from a web deployment project in Visual Studio 2008.
Now we want to use continuous integration and we would need the .MSI package build in the nightly builds.
Till now we used standard Visual Studio Web Setup project, but this is not compatible with the MSBuild. So we decided to use WiX.
The problem is that I have not found any good tutorial/documentation about this.
Is there a way to do a WiX installer package from a web deployment project? If yes, how?
Also, I tried to use heat.exe to create the XML for the WiX project .wxs file, but it seems that heat.exe doesn't recognize the web deployment project format.
Thank you for your responses.
Regards,
V.
I wrote a blog post about this recently - http://www.chrissurfleet.co.uk/post/2011/07/01/Using-Packaged-Project-Output-in-WiX-and-Visual-Studio.aspx
In short, its fairly easy to use msbuild to package up your web app and then pass it to heat to generate your installer from.
Hope this helps.
You've probably long since found a solution for this, but to elaborate on Tom Cabanski's answer, you can invoke Visual Studio to build the msi on the command line using "devenv.com" via an external process from within your build. It's not a pretty as using msbuild, but it gets the job done. Below is an example of how to invoke Visual Studio:
"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" your.sln /build Release
Where your.sln is the solution file for the solution you wish to build, and Release is the configuration you wish to build, ensuring that the configuration you choose actually builds the vdproj project.
Following the successful execution, you can grab the msi from the appropriate configuration's bin, and do what you want with it.
I'd appreciate your response to this with your findings/approach, as I'm trying to decide whether to adopt WiX or InstallShield as the approach to building msi's for Web Applications within TFS Build, or to continue with the approach I just described. I haven't had to opportunity to try WiX out, and my very limited exposure to InstallShield suggests that this is far to involved for my need, which is to produce a simple deployment aid for some relatively straight-forward web applications to the company intranet via TFS Build.
We used WIX on the installers for our last couple of projects and ended up regretting it. I would stick with the VS built-in projects and just invoke the VS IDE from the command line in the CI build.