dotnet publish failing using .NET Core 1.1 and new VS 2017 project format - asp.net-core

We've recently migrated to Visual Studio 2017 Update 2. We have a .NET Core 1.1 app which targets .NET Framework 4.6.1 only, and has project references to .NET Framework class libraries in the same solution. We are using the 1.0.4 SDK version of dotnet.
One of the project references contains T4 templates which are configured to run on build. This requires importing Microsoft.TextTemplating.targets and related assemblies, so that the T4 templates can be transformed by MSBuild. These assemblies in turn require .NET Framework 4.6 assemblies (System, System.Data, System.Xml, etc).
Visual Studio 2017 has no problems building the code, however when we try to use dotnet build or dotnet publish in our TeamCity builds, the command fails with the following output:
c:\xxx\Microsoft.TextTemplating.targets(340,5): error MSB4018: The "TransformTemplates" task failed unexpectedly.
c:\xxx\Microsoft.TextTemplating.targets(340,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified.
c:\xxx\Microsoft.TextTemplating.targets(340,5): error MSB4018: File name: 'System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
Based on the research I've done so far, the .NET Framework references required to run the T4 transformation task are not being loaded by the dotnet CLI, but they are by Visual Studio, due to entirely separate MSBuild toolchains.
Is there any way to make dotnet publish work in the same manner as Visual Studio? This issue has completely broken our CI / CD trains.
NOTE: I tried including .NET Standard as a target framework to see if that might work, but we use several packages that are not compatible so that isn't an option (Entity Framework 6.1, for example).

The dotnet based commands run on a .NET Core build of MSBuild so it is unable to load full framework assemblies. If there are no .NET Core or .NET Standard versions of these assemblies, loading them during builds of the .NET Core version of MSBuild is impossible at the moment (this may become possible with the move to .NET Core 2.0).
In order to get CI/CD working, you need to use the VS version of MSBuild. dotnet publish only forwards to msbuild so you can get the same behaviour passing parameters to msbuild. For example:
dotnet publish -c Release
becomes
msbuild /m /t:Publish /p:Configuration=Release

Related

Error in Visual Studio CI/CD .Net Core Project

We faced lots of build errors on Visual Studio CI/CD for .net core project but this is our last error and we can't figure it out.
Our project is based on .net core v1.1 and the last error asked us to upgrade to .net core v2.0 and we did that.
All of the CI steps are succeed expect dotnet ef migrations:
This is error message:
Method 'get_Properties' does not have an implementation. in type
'Microsoft.Extensions.Configuration.ConfigurationBuilder' from
assembly 'Microsoft.Extensions.Configuration, Version=1.1.2.0,
Culture=neutral, PublicKeyToken=adb9793829ddae60'.
Note: The project is building local machine and on Visual Studio CI
It looks like you did not upgrade your solution successfully to .net core 2.x., since you're configuration is still depending on on a package from 1.1.
You should upgrade Microsoft.Extensions.Configuration to 2.0 (2.1.1, if you're on .net core 2.1)
Update-Package Microsoft.Extensions.Configuration -version 2.0.0 -reinstall
https://www.nuget.org/packages/Microsoft.Extensions.Configuration/

VS 2017 .NetCore MSBuild custom task fails to load Newtonsoft.Json or other dependents

Environment
Visual Studio 2017
.NET Standard 2.0
.NET Framework 4.6.1
MyCustomTask.csproj contains custom MS build tasks and has multi targets to generate a separate dll for .NET Full and .NEt Core runtime.
There is a 3 level dependency on the Nuget Package "Microsoft.Extensions.Configuration.Json" which in turn has a dependency to Newtonsoft.Json package. as shown below.
The build is all fine and the Nuget package for MyCustomTask is generated successfully.
When I install my MyCustomTask Nuget package on target project then the custom build task is getting successfully executed on .NET Full runtime from Visual Studio 2017, but for .NET Core dotnet build command fail to execute the custom build task...
System.IO.FileLoadException: Could not load file or assembly 'Newtonsoft.Json, Version=10.0.0.0, Culture=neutral.
Even if I package all the dependent DLL and place it in same directory of the .netstandard custom task dll I keep getting same error.
Any thoughts how this can be resolved ?
Posting a solution here for those who are experiencing the same issue.
Credit to #MartinUllrich for posting the solution in a comment above. This saved me a lot of trouble.
https://github.com/AArnott/Nerdbank.MSBuildExtension
Ensure your custom task inherits from ContextIsolatedTask and copy the dependent assemblies to same directory as your task assembly.

Distribute custom MS Build Task with .NET Standard and VS 2017 via Nuget

I have a .NET Standard library (1.4) VS 2017 project that contains custom MS Build task (MyTask) that need to be distributed via Nuget package (Let's say MyCustomTask.dll and it contains MyTask and Portable.targets that will be imported by target project)
This Nuget package with custom build task is then used by target .NET Standard (1.4) project cspro file to import the Portable.targets that invoke the Custom Build task.
However, at this point I keep on getting the build error
Could not load file or assembly 'System.Runtime, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
I tried .NET Standard (1.4, 1.5 and 1.6) but same error.
The problem is that the consuming application, MSBuild.exe in this case, would need to include all the forwarding assemblies necessary to run netstandard tasks (e.g. depend on the NETStandard.Library).
The best solution in this case is multi-targeting the task library to a .net framework and a .net standard target framework:
<TargetFrameworks>netstandard1.6;net46</TargetFrameworks>
The idea is to have 2 dlls that will contain the task. In the project files contained in the NuGet package instead of using a dll path directly in <UsingTask>, the idea is to using a different dll file based on the $(MSBuildRuntimeType) property, which will be Core on the .NET Core version of MSBuild:
<PropertyGroup>
<_CustomTaskAssemblyTFM Condition="'$(MSBuildRuntimeType)' == 'Core'">netstandard1.6</_CustomTaskAssemblyTFM>
<_CustomTaskAssemblyTFM Condition="'$(MSBuildRuntimeType)' != 'Core'">net46</_CustomTaskAssemblyTFM>
<_CustomTaskAssembly>$(MSBuildThisFileDirectory)..\tools\$(_CustomTaskAssemblyTFM)\CustomTaskAssemblyName.dll</_CustomTaskAssembly>
</PropertyGroup>
<UsingTask TaskName="SomeCustomTask" AssemblyFile="$(_CustomTaskAssembly)" />
You can see examples of this in the asp.net core build tools and the .NET Core SDK.

Visual Studio 2017 MSBuild Task Development

With developing an custom MSBuild Task with Visual Studio 2017 RC, I have the following problem: As soon as I add other dependencies than just Microsoft.Build.Utilities.Core (using v15.1.0-preview-000458-02 for .NET Core Support), I cannot load the task into another .csproj MSBuild project as the dependencies are not found.
Is there a way to automatically copy all dependencies to the Debug folder?
Or do I have to publish it every time I want to test it?
Update1:
The problem with publish was something local to my environment and has been fixed.
Update2:
It seems that as soon as I change the TargetFramework from netstandard1.4 to netstandard1.6 it isn't even able to load the task at all. As soon as I use netstandard 1.6 it throws a an exception:
The task could not be loaded from the assembly.
Could not load file or assembly 'System.Runtime, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its
dependencies.
Is there a way to automatically copy all dependencies to the Debug folder? Or do
I have to publish it every time I want to test it?
By default and for good reasons, .NET Core and .NET Standard projects do not copy referenced assemblies into the build folder. Instead, they are resolved them from the NuGet cache.
But if you really need it, this behavior can be changed by overriding the default with the CopyLocalLockFileAssemblies setting.
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
Cref: https://github.com/dotnet/sdk/blob/d20405f91a2959fa91fea6285d9a896286727f2a/src/Tasks/Microsoft.NET.Build.Tasks/build/Microsoft.NET.Sdk.BeforeCommon.targets#L55-L56
Second question
It seems that as soon as I change the TargetFramework from netstandard1.4 to netstandard1.6
To build a task assembly that works on both "MSBuild.exe" and "dotnet.exe msbuild", you should target netstandard1.4 or lower. netstandard1.6 is not compatible with .NET Framework 4.6.1 (which MSBuild.exe runs on.)
If you need API not available in netstandard1.4, you will need to cross-compile your task for .NET Framework and .NET Standard, which is considerably more complex but can be done.

Compile .net 4.0 project on build server with .net 4.5

We're having the seemingly common error
Could not load type 'System.Runtime.CompilerServices.ExtensionAttribute' from assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
in a project that needs to be compiled against .Net 4.0 but is built on a build server running Windows Server 2012 (with .Net 4.5). The project is a web application that gets deployed to a web server running 2003, where installing .Net 4.5 isn't an option. There it runs against "classic" .Net 4.0
From similar questions, we're trying command-line options to MSBuild:
/property:FrameworkPathOverride="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"
We also tried various combinations of
/property:ReferencePath="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"
/property:NoStdLib=true
/property:NoCompilerStandardLib=true
The reference assemblies (including the .dll files) are in fact installed in that place on the build server. But when we deploy the website and visit the home page, we get that error. (Interestingly, on a page reload, the error disappears, and the site operates normally.) What are the MSBuild parameters necessary to compile against the .Net 4.0 assemblies?
Update
I turned on ludicrous-level logging on MSBuild, and I see that apparently it is building against the .Net 4.0 reference assemblies:
Resolved file path is "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll
and I didn't see any mention of assemblies outside that folder or the build server's working directory. So it appears to compile properly, but when deployed on the web server, it throws the exception.
As to the exception going away on a page reload, I wonder if that's related to the markup pre-compile step. We're running aspnet_compile on the build server. Maybe if there's an exception coming from a generated assembly, the web server will re-compile it. And the recompiled assembly is fine, because it was created with true .Net 4.0.
Well the answer turned out to be borderline embarrassing. After we confirmed from detailed MSBuild output that it was in fact building the website project against the right reference assemblies, we realized that there are several in-house NuGet packages in the project that had been built against .Net 4.5. One of them was chock full of extension methods, which is what causes the exception. Rebuilding them against .Net 4.0 fixed the problem.
Which brings up an interesting issue. If a 3rd-party NuGet package is compiled for 4.0 but using 4.5 references, we'd be in the same situation, but not able to fix it. So the lesson for package publishers is to ensure your 4.0 version is compiled against the reference assemblies.