Self-update / shadow-copy with Asp.Net Core - asp.net-core

I'm writing a Asp.Net Core application which should be able to update itself (replace its own binaries while running).
This MSDN article describes shadow copying with the classical .Net framework, which would be exactly what I need. But the whole AppDomain thing is missing in .Net Core.
So my questions are:
Is there an alternative way in .Net Core to enable shadow copying the assemblies?
Are there other mechanisms in .Net Core that allow to build a self-updating application?

Since there is no built in mechanism in .NET Core for doing this, I ended up implementing my own custom solution. It works roughly like this:
The running application downloads and extracts new binaries to a new folder.
The running application starts a small updater process. The following parameters are passed to the updater process via command line:
Process id of the running application
Binary path of the running application
Path of the downloaded binaries
The running application exits itself.
The updater process waits until the running application has exited (using the process id) or forcefully kills the running application if it doesn't exit by itself within a given timeout.
The updater process deletes the existing binaries and copies the new downloaded binaries over.
The updater process starts the new version of the main application.
Make sure you do as much as possible in the main application (downloading, unpacking, validation, ...) and keep the updater process as simple as possible (minimize risk of failing).
This approach has proven to be quite stable.

There's no build in shadow copying facilities in .NET Core

.Net API Browser indicates that the property required to set this up in .Net Core is but AppDomainSetup is not.
To be clear, AppDomain was added in .Net Standard 2.0 but creating a domain is not currently Supported

To save someone having to do what I just did and make this - this only copies files with a different date modified time. I checked and rebuilding your app only changes this on a few files. This makes for a very fast self-loader that then starts the exe in the new location, and exits the exe doing the loading that was running from the old location. This may rely on a few things like your DLL running the code must be named the same as the EXE that starts it.
Works in .Net 5:
using System;
using System.Diagnostics;
using System.IO;
namespace NetworkHelper
{
public static class LocalCopier
{
public static void EnsureRunningLocally(string callingAssemblyDotLocation)
{
var assemblyFileFriendlyName = Path.GetFileName(callingAssemblyDotLocation.Replace(".", "-"));
var assemblyDirToCheck = Path.GetDirectoryName(callingAssemblyDotLocation);
var localLocation = Configuration.Tools.AppsLocation + assemblyFileFriendlyName + "\\";
var assemblyFinalExePath = localLocation + assemblyFileFriendlyName.Replace("-dll", ".exe");
// Check what assembly passed in path starts with
var runningFromNetwork = callingAssemblyDotLocation.ToLower().StartsWith(#"\\w2k3nas1\");
if (callingAssemblyDotLocation.ToLower().StartsWith(#"i:\")) runningFromNetwork = true;
if (!runningFromNetwork) return;
// Check if copied to local already
Directory.CreateDirectory(localLocation);
// Foreach file in source dir, recursively
CopyOnlyDifferentFiles(assemblyDirToCheck, localLocation);
Process.Start(assemblyFinalExePath);
Environment.Exit(0);
}
private static void CopyOnlyDifferentFiles(string sourceFolderPath, string destinationFolderPath)
{
string[] originalFiles = Directory.GetFiles(sourceFolderPath, "*", SearchOption.AllDirectories);
Array.ForEach(originalFiles, (originalFileLocation) =>
{
FileInfo originalFile = new FileInfo(originalFileLocation);
FileInfo destFile = new FileInfo(originalFileLocation.Replace(sourceFolderPath, destinationFolderPath));
if (destFile.Exists)
{
if (originalFile.LastWriteTime != destFile.LastWriteTime)
{
originalFile.CopyTo(destFile.FullName, true);
}
}
else
{
Directory.CreateDirectory(destFile.DirectoryName);
originalFile.CopyTo(destFile.FullName, false);
}
});
}
}
}
Note that "\w2k3nas1" and "i:" are examples of network locations where if it is running from those, it should copy itself to a local directory, I use application data/roaming/localApps and then restart itself from the new directory.
This can all be put into a reference library and be called from any client apps with:
NetworkHelpers.LocalCopier.EnsureRunningLocally(Assembly.GetExecutingAssembly().Location);
(Here, Assembly.GetExecutingAssembly().Location is passed in from the calling app, because if you were to run that from in the reference project, you'd get that library's dll instead.)

I made my own solution with PowerShell Core (available on Windows/Linux/Mac).
You can use the following script to create a powershell script to update the app. IMHO: PowerShell solution is better than an external update app: script is transparent and no additional overhead for background services that lives outside of your app.
Don't forget to inject your variables:
# We don't need progress bars to consume CPU
$ProgressPreference = 'SilentlyContinue'
# Stopping the current app
$appDll = '{assemblyName}.dll'
Stop-Process -Id {processId} -Force
$appFolder = '{folderPath}'
Set-Location -Path $appFolder
# Source file location
$source = '{updateLink}'
# Destination to save the file (folder of the script)
$updateName = Get-Date -Format 'up_dd_MM_yyyy_HH_mm'
$updateNameFile = $updateName + '_update.zip'
$updateZipPath = Join-Path -Path $appFolder -ChildPath $updateNameFile
# Download the update
Invoke-WebRequest -Uri $source -OutFile $updateZipPath
# Classic Unzip
Expand-Archive -Path $updateZipPath -DestinationPath $appFolder -Force
# Cleaning
Remove-Item -Path $updateZipPath
Start-Process -FilePath 'dotnet' -ArgumentList $appDll

Related

Set Environment=Development when xunit testing an ASP.NET Core app

I xUnit test my ASP.NET Core web app, and my test class includes:
this.host = Program.CreateHostBuilder(Array.Empty<string>()).Build();
in order to access host.Services.
I discover that the host has Environment=Production. So the configuration seen in my startup file ignores appsettings.Development.json.
How do I inject or force host to have Environment=Development?
Preferably without any code in the web app itself.
(Context: I'm using JetBrains Rider. I find nothing in Rider setup or configuration that lets me choose an Environment for a UnitTest session. But if there is a solution on that line the question still stands)
I had assumed—wrongly—that I might fix it with this:
this.host = Program.CreateHostBuilder(
"ASPNETCORE_ENVIRONMENT=Development"
).Build();
because the docs say
The default configuration loads environment variables and command line arguments prefixed with DOTNET_ and ASPNETCORE_
but what fixed it was:
this.host = Program.CreateHostBuilder(
"ENVIRONMENT=Development"
).Build();
With this, the Configuration element then picked up the appsettings.Development.json file instead of ignoring it.
( So now I wonder whether part of the env variable processing is done by the dotnet executable before reaching Program.Main() )

Is there a way to force NuGet to restore the latest package version during pipeline execution?

We are using an internal NuGet feed in our Azure DevOps environment to host many different packages, which are consumed in many different projects. I would like to configure the pipeline so that whenever an internal package is referenced, it is always resolved to the latest version. All of the internal references are configured with wildcards in the PackageReference tag of the .vbproj file, like so:
<PackageReference Include="MyPackageName" Version="*" />
And the restore command in the YAML file is configured like so:
- task: NuGetCommand#2
condition: and(succeeded(), eq(variables['ModifiedProject'], 'true'))
displayName: 'NuGet restore'
inputs:
command: 'restore'
restoreSolution: '$(ModifiedProject.Directory)/$(ModifiedProject.Solution)'
feedsToUse: 'select'
vstsFeed: '[guid]/[guid]'
noCache: true
The projects build successfully, but they still end up using the oldest version of the package instead of restoring the newest version. Is there a way to force the restore task to default to the newest package version?
I ended up finding a workaround for this issue by using the Azure DevOps REST API. My objective was to get the pipeline to flow through these basic steps:
After all build steps have been completed successfully, push the new package version into the artifact feed.
Use the REST API to unlist the old package version, leaving only the newest version available.
Because of the use of floating versions in the PackageReference tags, any projects that depend on these packages will then automatically pull the newest version when they are built in their respective pipelines.
I was only comfortable going this route because "deleting" a package in Azure doesn't really delete it, it just gets moved to the Recycle Bin. In the event there is a breaking change that is discovered post-push, we can always restore a previous version to the feed. Given that this process is being used exclusively for our 50+ internal packages with their own testing processes, it definitely seems like a safe route to go, and far more efficient than any other option I can find. However, I don't think I will mark this as a definitive answer to the question, because it still feels a bit hacky and I'd prefer there was a legitimate option to force newest versions in Azure's NuGetCommand#2 task.
Code
As mentioned, I used Azure's well-documented REST API for these functions, particularly the areas governing artifacts. Although there is a page dedicated to deleting a package from a NuGet feed, I couldn't get their specification to work. I ended up inspecting the calls made from the UI and copying those, while still using my own token for authentication. This method does the history "trimming" I needed:
public void TrimPackageFeed(string feedName, string packageName)
{
var packageVersions = GetPackageVersions(feedName, packageName);
var deprecated = packageVersions.Where(x => !x.IsLatest && !x.IsDeleted)?.ToList();
if (deprecated != null && deprecated.Any())
{
foreach (var version in deprecated)
{
var url = $"{version.Links.Feed.Value.Replace("feeds.dev.azure.com", "pkgs.dev.azure.com")}/nuget/packagesBatch";
var payload = new AzurePackagePayload
{
Data = null,
Operation = 2,
Packages = new List<AzurePackagePayloadItem>
{
new AzurePackagePayloadItem
{
ID = packageName,
Version = version.Version
}
}
};
ApiRequest(url, Method.POST, null, JsonConvert.SerializeObject(payload));
}
}
}
I built this as a .NET Core 3.1 command line app, published as a self-contained executable within our build repo. I used C# because it's most familiar to me, but I'm confident this could be scripted in any language (probably even just PowerShell). I then appended the following task to the end of my YAML pipeline definition:
- task: CmdLine#2
condition: and(succeeded(), eq(variables['ModifiedProject'], 'true'))
displayName: 'Trim package feed'
inputs:
script: |
AzureApiClient -action trim-package-feed -feed FeedNameHere -package $(ModifiedProject.AssemblyName)
workingDirectory: 'Azure\AzureApiClient\Output'
failOnStderr: true
The package gets pushed to the feed, then the assembly name is passed to my API client which will trim the historical versions and leaves only the new version available to restore.
If you are using a self-hosted agent to run the pipeline, you can try to clear the local nuget cache, delete all the nuget packages in the global nuget cache under C:\Users\xxx\.nuget\packages or use clean nuget caches.
If the package to be consumed is just pushed into the feed, you need to wait for a while . There has to be a delay in populating packages on the feed.
In addition , you can try to use dotnet restore task to see if this issue still occurs . Here is a ticket with similar issue you can refer to .

Automatically execute migrations when publishing ASP.NET Core app

Question
Is there any ways that I can automatically execute the migration code (EF 7) when publishing my ASP 5 application to IIS using Web Deploy?
I Tried
in the project.json, I added this code in the scripts:
"scripts" : {
"prepublish": ["dnx ef database update", "other commands..."],
"postpublish": ["dnx ef database update"]
}
none worked for me.
Additional Info
I followed the instructions on this link to deploy my ASP 5 RC-1 web application to IIS using web deploy.
After doing so in the publish settings I have:
Using web deploy in ASP 4 applications I have additional database options:
Use context.Database.Migrate()
You can call this from your Startup class:
using (var context = new MyContext(...))
{
context.Database.Migrate();
}
It will migrate your database to the latest version on application startup. But be careful doing it, maybe comment out this code and uncommend only when you want to run your migrations.
Apparently this process does not work now. https://github.com/aspnet/Home/issues/622 After you publish you should find the power shell script with the name of "profile name"-publish.ps1. Then add your commands below these three lines close to the end of this file. You might want to use powershell to make it easier to debug.
'Calling Publish-AspNet' | Write-Verbose
# call Publish-AspNet to perform the publish operation
Publish-AspNet -publishProperties $publishProperties -packOutput $packOutput
So I added the option -environment to my ef database command. Now it works:
"postpublish": ["dnx ef database update -e Staging"]
I have four different appsettings.json which different connection string for each environment. Just needed to indicate the environment for the command to work.
In you Startup.cs class add this code
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
var context = serviceScope.ServiceProvider.GetService<AppDBContext>();
context.Database.Migrate();
}
}

SpecsFor.Mvc Build failed

Attempting to test out SpecsFor.Mvc, unforunitly I'm getting this strange build error when I try to run a test.
Running in both my own project and the SpecsFor latest source I get a "Build failed." ApplicationException from the IISTestRunnerAction class. The following is from the log file but its beyond my understanding.
Using visual studio 2012 pro and IIS Express 8.0
The following is from the log file:
Using "VSMSDeploy" task from assembly "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Microsoft.Web.Publishing.Tasks.dll".
Task "VSMSDeploy"
Package/Publish task Microsoft.Web.Publishing.Tasks.VSMSDeploy load assembly Microsoft.Web.Deployment, Version=9.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
Package/Publish task Microsoft.Web.Publishing.Tasks.VSMSDeploy load assembly Microsoft.Web.Delegation, Version=7.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
Starting Web deployment task from source: manifest(C:\Users\Chris\Desktop\SpecsFor-master\SpecsFor.Mvc.Demo\obj\Test\Package\SpecsFor.Mvc.Demo.SourceManifest.xml) to Destination: package(C:\Users\Chris\Desktop\SpecsFor-master\SpecsFor.Mvc.Demo\obj\Test\Package\SpecsFor.Mvc.Demo.zip).
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Microsoft.Web.Publishing.targets(4007,5): error : Web deployment task failed. (The type initializer for 'Microsoft.Web.Deployment.DeploymentManager' threw an exception.)
Package failed.
Done executing task "VSMSDeploy" -- FAILED.
UPDATE
Here is the AssemblyStartup
[SetUpFixture]
public class AssemblyStartup
{
private SpecsForIntegrationHost _host;
[SetUp]
public void SetupTestRun()
{
var config = new SpecsForMvcConfig();
//SpecsFor.Mvc can spin up an instance of IIS Express to host your app
//while the specs are executing.
config.UseIISExpress()
//To do that, it needs to know the name of the project to test...
.With(Project.Named("SpecsForTesting"))
//And optionally, it can apply Web.config transformations if you want
//it to.
.ApplyWebConfigTransformForConfig("Debug");
//In order to leverage the strongly-typed helpers in SpecsFor.Mvc,
//you need to tell it about your routes. Here we are just calling
//the infrastructure class from our MVC app that builds the RouteTable.
config.BuildRoutesUsing(r => SpecsForTesting.RouteConfig.RegisterRoutes(r));
//SpecsFor.Mvc can use either Internet Explorer or Firefox. Support
//for Chrome is planned for a future release.
config.UseBrowser(BrowserDriver.Chrome);
//Does your application send E-mails? Well, SpecsFor.Mvc can intercept
//those while your specifications are executing, enabling you to write
//tests against the contents of sent messages.
config.InterceptEmailMessagesOnPort(13565);
//The host takes our configuration and performs all the magic. We
//need to keep a reference to it so we can shut it down after all
//the specifications have executed.
_host = new SpecsForIntegrationHost(config);
_host.Start();
}
//The TearDown method will be called once all the specs have executed.
//All we need to do is stop the integration host, and it will take
//care of shutting down the browser, IIS Express, etc.
[TearDown]
public void TearDownTestRun()
{
_host.Shutdown();
}
}
I had this error come up, and it turned out that I had added a new project to my solution. The new project did not include the same configurations i.e. the solution was running of "Test" but my new project only had the default ones of debug and release.
Go into the Configuration Manager and check that all the projects in your solution have the same configurations in place.
If you are looking for the build log, it is outputted to Console by default. Here is how to capture Console output:
var stringWriter = new StringWriter();
try
{
// Build log is sent to console, redirect output to StringWriter
Console.SetOut(stringWriter);
_host.Start();
}
catch (ApplicationException ex)
{
throw new Exception("Build failed. Output: " + stringWriter, ex);
}
It looks like the error is actually from MSDeploy, which SpecsFor.Mvc uses internally through MSBuild to publish your site for testing. Here's the same error directly from MSDeploy: Web deployment task failed. (The type initializer for 'Microsoft.Web.Deployment.DeploymentManager' threw an exception.). Unfortunately there doesn't seem to be a resolution.
Can you try deploying your site manually? This command line should do the trick:
msbuild /p:DeployOnBuild=true;DeployTarget=Package;_PackageTempDir=;AutoParameterizationWebConfigConnectionStrings=false;Platform=AnyCPU
Let me know if that works or if it blows up with a similar error.
I had exactly the same issue trying to get SpecsForMvc working on a Bamboo remote build agent. Matt Honeycutt's answer pointed me in the right direction. I just had to install MS Web Deploy 3.5 on the VM running the agent to fix this error.
I also needed to install IIS Express 8 on the same VM to allow the SpecsForIntegrationHost to spin up a site in.
arni's answer helped me better diagnose the problem, but also caused me some issues later down the line, when I was having trouble with permissions trying to connect to a remote SQL Server from the tested app. These exceptions were not caught by the ApplicationException catch block as they were of class SystemException. They got handled by the global exception handler, bypassing the end of test cleanup which was supposed to shut down the integration host. This left the IIS Express instance for each test running in the background. (As I can't comment on arni's answer, I've added my amended code here)
var stringWriter = new StringWriter();
try
{
// Build log is sent to console, redirect output to StringWriter
Console.SetOut(stringWriter);
_host.Start();
}
catch (Exception ex)
{
_integrationHost.Shutdown();
throw new Exception("Build failed. Output: " + stringWriter, ex);
}

WCF service not working after program update

I have recently added a WCF service reference to my program. When I perform a clean install of this program, everything seems to work as expected. But, when I install the program on a client which already has a previous version (without the new service reference) installed, I get a exception telling me the default endpoint for this particular service could not be found.
It seems that the appname.exe.config is not being updated with the new endpoint settings. Is there any reason for this and how can I force the installer to overwrite the config file? I'm using the default Visual Studio 2008 installer project with RemovePreviousVersions set to True.
Update:
My program encrypts the settings section after the first run with the following code
Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
ConfigurationSection section = config.GetSection(sectionKey);
if (section != null)
{
if (!section.SectionInformation.IsProtected)
{
if (!section.ElementInformation.IsLocked)
{
section.SectionInformation.ProtectSection("DataProtectionConfigurationProvider");
section.SectionInformation.ForceSave = true;
config.Save(ConfigurationSaveMode.Full);
}
}
}
When I do not run the program before installing the new version the app.config gets updated.
You are right that it is the config file that is not updated.
There are several possibilities:
The installer has the old version of the config file
The installer does not have a config file and the program is using the old one on the machine
Try uninstalling the project first, then install and check that the config file has been copied in.