Keep HangFire application awake on IIS in .NET Core environment - hangfire

I am having difficulties with: IApplicationLifetime CancellationTokens:
ApplicationStarted,
ApplicationStopped
in my .NET core application using HangFire batch application.
The delegates passed to ApplicationStarted.Register() and ApplicationStopped.Register() do not get called. I am using .NET Core version 1.0.
Using below workaround in .NET Core (use appropriate Configuration for your app, inject your own "logger") inside the Startup.Configure(..) method. Note RestartURL:URL is a URL to your application - a lightweight hit.
Current workaround to keep batch process alive:
var batchProcessKeepAliveTimer = new Timer();
batchProcessKeepAliveTimer.Elapsed += delegate
{
System.Net.WebRequest req = System.Net.WebRequest.Create(Configuration["Data:RestartURL:URL"]);
System.Net.WebResponse resp = req.GetResponse();
logger.Log("Polling batch restart 60 milli second intervals");
};
batchProcessKeepAliveTimer.Enabled = true;
double result = 60000;//default value in milliseconds
if (Double.TryParse(Configuration["Data:RestartURL:PollingInterval"], out result))
batchProcessKeepAliveTimer.Interval = result;
Draw backs is: When app pool is manually shut down and restarted, no guarantee that Startup will be called right away. I find that I need to make an explicit HTTP request so that the application pool can re-instantiate the Startup objects. In other words, I need to make a single HTTP call to bootstrap this timer.
However, if the Startup has been instantiated like this, timer can go on for a long time, until it is shut down. Would be less maintenance for your application if the Timer interval is less than IdleTime.
Anybody have a better working approach - especially using .NET core cancellation tokens as above?

Related

IHostedService StartAsync called randomely and causes Wrong Timer intervals

I'm using IHostedService back since Asp.net core version 2.1.
I noticed in my logs that StartAsync is called sometimes alot in very messy intervals from 8 mins to one hour and that calls my DoWorkAsync each time.
I have a long process that I don't want it to be recalled on those small intervals It runs in a timer each two hours in normal cases using Timer.
public Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Timed Background Service is starting.");
_timer = new Timer(DoWorkAsync, null, TimeSpan.FromMinutes(2),
TimeSpan.FromHours(2));
return Task.CompletedTask;
}
I'm considering using lock statement but if I made the lock on private object would it be there to lock execution when StartAsync called again.
I'm concerned because the process calls WSI (Webservice) on another server , and afraid it might be recalled before previous call answered and make the other server crashed.
My logging is simply a text file that log times in StartAsync and DoWorkAsync.
I'm running this on Aws windows instance. If the problem might be crashes or self restart how would I see the causes of it. I don't think my simple text file would catch it.
In my opinion the StartAsync() is not supposed to hold a callback object and probably cause your troubles. Pls consider this example: https://learn.microsoft.com/en-us/dotnet/core/extensions/timer-service. It does, what you want to do, but in the right manner.

HttpRequest not aborted (cancelled) on browser abort in ASP.NET Core MVC

I wrote the following MVC Controller to test cancellation functionality:
class MyController : Controller
{
[HttpGet("api/CancelTest")]
async Task<IActionResult> Get()
{
await Task.Delay(1000);
CancellationToken token = HttpContext.RequestAborted;
bool cancelled = token.IsCancellationRequested;
logger.LogDebug(cancelled.ToString());
return Ok();
}
}
Say, I want to cancel the request, so the value 'true' is logged in the controller action above. This is possible server-side if the server implements the IHttpRequestLifetimeFeature. Luckily Kestrel does, and this can be accomplished the following way:
var feature = (IHttpRequestLifetimeFeature) HttpContext.Features[typeof(IHttpRequestLifetimeFeature)];
feature.Abort();
The problem however is that I want to cancel the request on the client side. For example, in the browser. In pre-core versions of ASP.NET MVC/WebApi the cancellation token would automatically be cancelled if the browser aborted a request. Example: refresh the page a couple of times in Chrome. In the Network tab of the chrome dev tools you can now see the previous (unfinished) request be cancelled.
The thing is: in ASP.NET Core running on Kestrel, I can only see the following entry in the log:
Microsoft.AspNetCore.Server.Kestrel.Internal.Networking.UvException:
Error -4081 ECANCELED operation canceled
So the abort request from the browser DOES arrive and is handled by the Kestrel webserver. It does however not affect the RequestAborted property of the HttpContext in the controller, because the value 'false' is still logged by the method.
Question:
Is there a way to abort/cancel my controller's method, so that the HttpContext.RequestAborted property will be marked as cancelled?
Perhaps I can make something that would subscribe to Kestrel's operation cancelled trigger and call the IHttpRequestLifetimeFeature.Abort() method?
Update:
I did some further testing and it seems the HttpRequest IS in fact aborted, but there seems to be some kind of delay before the cancellation actually takes place. The delay is not time-factored, and seems to come straight from libuv (the library where the Kestrel webserver is build on top of). I posted more info on https://github.com/aspnet/KestrelHttpServer/issues/1103
More updates:
Issue has been moved to another one, because the previous one contained multiple problems. https://github.com/aspnet/KestrelHttpServer/issues/1139
Turns out that that simply using HttpContext.RequestAborted is indeed the right way, but due to a bug in Kestrel (the order in which FIN/RST packages were handled), the request was not aborted on a browser abort.
The bug should finally be fixed in Kestrel 2.0.
See the updates in my question for more information.

MVC - what are the benefits of all controller actions being Task<ActionResult>?

I've been using MVC since version 2, and lately I have come across a project where all of the controller actions are 'async', returning Tasks, and I am trying to understand why somebody would do this.
The View Model for each view is built via an async call to an API. I understand that in order to use the await keyword one must use an async method (and return a Task), but surely without the View Model then the view will fail. There is no choice but to wait for the API to build my View Model.
public async Task<ActionResult> MyCar()
{
return View(await MyAPI.BuildMyCarViewModel());
}
For what reason would controller actions need to be asynchronous?
Let's assume that your part of code
MyAPI.BuildMyCarViewModel()
needs for execution 15 seconds. Then let's assume, that you have 10 000 users, which in range of 2 seconds decided to load some model. And then assume that you don't use caching ( for the sake of example ).
IIS by default has pool of threads 5000.
In described case application pool of IIS will be busy with 5000 threads which will translate into awaiting of your 5000 users for 5 seconds, and other 5000 users will wait until code finish executing. But with async/await .Net will generate state machine, and threads will be executed till moment of awaiting, and then threads will be released for making another useful job. And as soon as
MyAPI.BuildMyCarViewModel()
will return results, other threads or the same threads will return you result. And as outcome application pool of IIS will not be exhausted quickly for long running tasks and your users will receive response much faster, then without usage await/async. If to put simply, await/async gives you possibility to avoid thread pool exhausting quickly for long running fragments of code.
I have an MSDN article on the topic of async ASP.NET. In summary, the benefit is that the request does not take up a thread for the duration of the request. This allows your web app to scale if your backend can scale.

Using ThreadPool.QueueUserWorkItem in WCF method

I need to run a background logic that takes around 25-30 sec inside a WCF method that can't take more than 1 sec to complete. I've decided to wrap that logic into a WaitCallback and pass it to ThreadPool.QueueUserWorkItem right before I exit the web method. Initially it worked ok but now I'm having second thoughts because I suspect that sometimes QueueUserWorkItem method doesn't return in a timely manner as a result web method doesn't respond within 1 sec on a regular basis. Are there any issues with using QueueUserWorkItem inside WCF methods?
No not as such, but your question touches upon a more general problem, what to do with long-running service calls? You can either:
Change the configs so that client and server tolerate long service calls, i.e. increase timeouts
Or, design your service calls with a start / get current progress / get final result API, all of which return quickly:
int jobID = serviceProxy.StartJob();
float progress = serviceProxy.GetJobProgress(int jobID);
Result finalResult = serviceProxy.GetJobResult(int jobID);
This is more work, but a better design, and you now also have to maintain a list of running jobs (your async proceessing which could use QueueUserWorkItem or whatever), but all the service calls would return quickly.

30 sec periodic task to poll external web service and cache data

I'm after some advice on polling an external web service every 30 secs from a Domino server side action.
A quick bit of background...
We track the location of cars thru the TomTom api. We now have a requirement to show this in our web app, overlayed onto a map (google, bing, etc.) and mashed up with other lat long data from our application. Think of it as dispatching calls to taxis and we want to assign those calls to the taxis (...it's not taxis\ calls, but it is similar process). We refresh the dispatch controllers screens quite aggressively, so they can see the status of all the objects and assign to the nearest car. If we trigger the pull of data from the refresh of the users screen, we get into some tricky controlling server side, else we will hit the max allowable requests per minute to the TomTom api.
Originally I was going to schedule an agent to poll the web service, write to a cached object in our app, and the refreshing dispatch controllers screen pulls the data from our cache....great, except, user requirement is our cache must be updated every 30secs. I can create a program doc that runs every 1 min, but still not aggressive enough.
So we are currently left with: our .net guy will create a service that polls TomTom every 30secs, and we retrieve from his service, or I figure out a way to do in Domino. It would be nice to do in Domino database, and not some stand alone java app or .net, to keep as much of the logic as possible in one system (Domino).
We use backing beans heavily in our system. I will be testing this later today I hope, but would this seem like a sensible route to go down..?:
Spawning threads in a JSF managed bean for scheduled tasks using a timer
...or are their limitations I am not aware of, has anyone tackled this before in Domino or have any comments?
Thanks in advance,
Nick
Check out DOTS (Domino OSGi Tasklet Service): http://www.openntf.org/internal/home.nsf/project.xsp?action=openDocument&name=OSGI%20Tasklet%20Service%20for%20IBM%20Lotus%20Domino
It allows you to define background Java tasks on a Domino server that have all the advantages of agents (can be scheduled or triggered) with none of the performance or maintenance issues.
If you cache the data in a bean (application or session scoped). Have a date object that contains the last refreshed date. When the data is requested, check last cached date against current time. If it's more than/equal to 30 seconds, refresh data.
A way of doing it would be to write a managed bean which is created in the application scope ( aka there can only be one..). In this managed bean you take care of the 30sec polling of the webservice by good old java webservice implementation and a java thread which you start at the creation of your managed-bean something like
public class ServicePoller{
private static myThread = null;
public ServicePoller(){
if(myThread == null){
myThread = new ServicePollThread();
(new Thread(myThread)).start());
}
}
}
class ServicePollThread implements Runnable(){
private hashMap yourcache = null;
public ServicePollThread(){
}
public void run(){
while(running){
doPoll();
Thread.sleep(4000);
}
}
....
}
This managed bean will then poll every 30 seconds the webservice and save it's findings in a hashmap or some other managed-bean classes. This way you dont need to run an agent or something like that and you achieve when you use the dispatch screen to retrieve data from the cache.
Another option would be to write an servlet ( that would be possible with the extlib but I cant find the information right now ) which does the threading and reading the service for you. Then in your database you should be able to read the cache of the servlet and use it wherever you need.
As Tim said DOTS or as jjtbsomhorst said a thread or an Eclipse job.
I've created a video describing DOTS: http://www.youtube.com/watch?v=CRuGeKkddVI&list=UUtMIOCuOQtR4w5xoTT4-uDw&index=4&feature=plcp
Next Monday I'll publish a sample how to do threads and Eclipse jobs. Here is a preview video: http://www.youtube.com/watch?v=uYgCfp1Bw8Q&list=UUtMIOCuOQtR4w5xoTT4-uDw&index=1&feature=plcp