Using MVC 4.0 Web Api I have a long running DB query which is running asynchronously and, before it completes, the controller completes its "Get" or "Post" operation. This is all as expected/wanted.
However, although it looks like MVC has sent the data back to the client nothing actually get transmitted until the long running query completes.
Is there any way I can force an early "yield" of the data to the client or even to create and transmit a new response?
The point is that I don't need the results from the query - I just
want to fire and forget and it's important to return a response
(saying the query has started) to the client straight away
If it is fire-and-forget and you do not need to send the result to client, simply start the task
Task.Factory.StartNew(() => db.DoThatQueryThatBroughtDownChicago());
and return a string, a JSON result saying "Task started".
Related
I'm building an API that will handle http calls that perform much work immediately.
For example, in one controller, there is an action (Post request) that will take data in the body, and perform some processing that should be done immediately, but will last for about 1 to 2 minutes.
I'm using CQRS, with Mediatr, and inside this post request, I call a command to handle the processing.
Taking this into consideration, I want the post request to launch the command, then return an Ok to the user though the command is still running in the background. The user will be notified via email once everything is done.
Does someone know any best practice to accomplish this?
Though I'm using MediatR, sending the request to the handler, the call is not executed in parallel.
[HttpPost]
public async Task<IActionResult> RequestReports([FromBody] ReportsIntent reportsIntent)
{
await _mediator.Send(new GetEndOfWeekReports.Command(companyId, clientId, reportsIntent));
return Ok();
}
Does someone know any best practice to accomplish this?
Yes (as described on my blog).
You need what I call the "basic distributed architecture". Specifically, you need at least:
A durable queue. "Durable" here means "on disk; not in-memory".
A backend processor.
So the web api will serialize all the necessary data from the request into a queue message and place that on the queue, and then return to the client.
Then a backend processor retrieves work from that queue and does the actual work. In your case, this work concludes with sending an email.
In a way, this is kind of like MediatR but explicitly going out of process with the on-disk queue and a separate process being your handler.
I would handle API calls that takes some time separately from calls that can be completed directly.
when I do API calls that takes time, I queue them up and process them on the backend.
a typical API call can look something like this:
POST http://api.example.com/orders HTTP/1.1
Host: api.example.com
HTTP/1.1 201 Created
Date: Fri, 5 Oct 2012 17:17:11 GMT
Content-Length: 123
Content-Type: application/json
Location: http://poll.example.com/orders/59cc233e-4068-4d4a-931d-cd5eb93f8c52.xml
ETag: "c180de84f951g8"
{ uri: 'http://poll.example.com/orders/59cc233e-4068-4d4a-931d-cd5eb93f8c52.xml'}
the returned URL is a unique url where the client then can poll/query to get an idea about the status of the job.
When the client queries this URL, then it can look something like this:
and when it is later done, the result would be something like:
Where dataurl is a link to the result/report that the client then can download.
I have to create a table, such that it sends Http Request with a parameter (first column value) for each row in the table and shows processing status for that request.
There is same backend process for each row with a column value passed as a parameter.
On completion of the request for the first row, the status column in the table should show 'Success' and then the request for the second record should be sent.
The request should be sent synchronously as there is moderately heavy backend process attached i.e. on completion of the previous request only next request has to be sent.
I have to achieve this using Angular 5 and DotNet Core 2.0.
Please let me know if there is a feature already available for doing such a thing in Angular 5 and .net Core in an optimized way. I have heard of ReactiveX but I am not able to figure out the best way to achieve this as Reactivex is for asynchronous programming.
Any suggestions or similar examples on this would be helpful.
Thanks
I have a subroutine in my Controller
<HttpPost>
Sub Index(Id, varLotsOfData)
'Point B.
'By the time it gets here - all the data has been accepted by server.
What I would like to do it capture the Id of the inbound POST and mark, for example, a database record to say "Id xx is receiving data"
The POST receive can take a long time as there is lots of data.
When execution gets to point B I can mark the record "All data received".
Where can I place this type of "pre-POST completed" code?
I should add - we are receiving the POST data from clients that we do not control - that is, it is most likely a client's server sending the data - not a webbrowser client that we have served up from our webserver.
UPDATE: This is looking more complex than I had imagined.
I'm thinking that a possible solution would be to inspect the worker processes in IIS programatically. Via the IIS Manager you can do this for example - How to use IIS Manager to get Worker Processes (w3wp.exe) details information ?
From your description, you want to display on the client page that the method is executing and you can show also a loading gif, and when the execution completed, you will show a message to the user that the execution is completed.
The answer is simply: use SignalR
here you can find some references
Getting started with signalR 1.x and Mvc4
Creating your first SignalR hub MVC project
Hope this will help you
If I understand your goal correctly, it sounds like HttpRequest.GetBufferlessInputStream might be worth a look. It allows you to begin acting on incoming post data immediately and in "pieces" rather than waiting until the entire post has been received.
An excerpt from Microsoft's documentation:
...provides an alternative to using the InputStream propertywhich waits until the whole request has been received. In contrast, the GetBufferlessInputStream method returns the Stream object immediately. You can use the method to begin processing the entity body before the complete contents of the body have been received and asynchronously read the request entity in chunks. This method can be useful if the request is uploading a large file and you want to begin accessing the file contents before the upload is finished.
So you could grab the beginning of the post, and provided your client-facing page sends the ID towards the beginning of its transmission, you may be able to pull that out. Of course, this would be reading raw byte data which would need to be decoded so you could grab the inbound post's ID. There's also a buffered one that will allow the stream to be read in pieces but will also build a complete request object for processing once it has been completely received.
Create a custom action filter,
Action Filters for executing filtering logic either before or after an action method is called. Action Filters are custom attributes that provide declarative means to add pre-action and post-action behavior to the controller's action methods.
Specifically you'll want to look at the
OnActionExecuted – This method is called after a controller action is executed.
Here are a couple of links:
http://www.infragistics.com/community/blogs/dhananjay_kumar/archive/2016/03/04/how-to-create-a-custom-action-filter-in-asp-net-mvc.aspx
http://www.asp.net/mvc/overview/older-versions-1/controllers-and-routing/understanding-action-filters-vb
Here is a lab, but I think it's C#
http://www.asp.net/mvc/overview/older-versions/hands-on-labs/aspnet-mvc-4-custom-action-filters
I have a Web Api 2 service that will be deployed across 4 production servers. When a request doesn't pass validation a custom response object is generated and returned to the client.
A rudimentary example
if (!ModelState.IsValid)
{
var responseObject = responseGenerator.GetResponseForInvalidModelState(ModelState);
return Ok(responseObject);
}
Currently the responseGenerator is aware of what environment it is in and generates the response accordingly. For example, in development it'll return a lot detail but in production it'll only return a simple failure status.
How can I implement a "switch" that turns details on without requiring a round trip to the database each time?
Due to the nature of our environment using a config file isn't realistic. I've considered using a flag in the database and then caching it at the application layer but environmental constraints make refreshing the cache on all 4 servers very painful.
I ended up going with the parameter suggestion and then implementing a token system on the back end. If a Debug token is present in the request the service validates it against the database. If it's a valid and active token it returns the additional detail.
This allows us to control things from our end while keeping things simple for the vendors and only adds that extra round trip to the database during debugging.
I am trying to write a control panel to
Inform about certain KPIS
Enable the user to init certain requests / jobs by pressing a button that then runs a stored proc on the DB or sets a specific setting etc
So far, so good, except I would like to run some bigger jobs where the length of time that the job is running for is unknown and could run over both the script timeout period AND the time the user is willing to wait for a response.
What I want is a "fire and forget" process so the user hits the button and even if they kill the page or turn off their phone they know the job has been initiated and WILL complete.
I was looking into C# BeginExecuteNonQuery which is an async call to the query so the proc is fired but the control doesn't have to wait for a response from it to carry on. However I don't know what happens when the page/app that fired it is shut.
Also I was thinking of some sort of Ajax command that fires the code in a page behind the scenes so the user doesn't know about it running but then again I believe if the user shuts the page down the script will die and the command will die on the server as well.
The only way for certain I know of is a "queue" table where jobs are inserted into this table then an MS Agent job comes along every minute or two checking for new inserts and then runs the code if there is any. That way it is all on the DB and only a DB crash will destroy it. It won't help with multiple jobs waiting to be run concurrently that both take a long time but it's the only thing I can be sure of that will ensure the code is run at all.
Any ideas?
Any language is okay.
Since web browsers are unconnected, requests from them always take the full amount of time. The governing factor isn't what the browser does, but how long the web site itself will allow an action to continue.
IIS (and in general, web servers) have a timeout period for requests, where if the work being done takes simply too long, the request is terminated. This would involve abruptly stopping whatever is taking so long, such as a database call, running code, and so on.
Simply making your long-running actions asynchronous may seem like a good idea, however I would recommend against that. The reason is that in ASP and ASP.Net, asynchronously-called code still consumes a thread in a way that blocks other legitimate request from getting through (in some cases you can end up consuming two threads!). This could have performance implications in non-obvious ways. It's better to just increase the timeout and allow the synchronously blocking task to complete. There's nothing special you have to do to make such a request complete fully, it will occur even if the sender closes his browser or turns off his phone immediately after (presuming the entire request was received).
If you're still concerned about making certain work finish, no matter what is going on with the web request, then it's probably better to create an out-of-process server/service that does the work and to which such tasks can be handed off. Your web site then invokes a method that, inside the service, starts its own async thread to do the work and then immediately returns. Perhaps it also returns a request ID, so that the web page can check on the status of the requested work later through other methods.
You may use asynchronous method and call the query from this method.
Your simple method can be changed in to a asynch method in the following manner.
Consider that you have a Test method to be called asynchronously -
Class AsynchDemo
{
public string TestMethod(out int threadId)
{
//call your query here
}
//create a asynch handler delegate:
public delegate string AsyncMethodCaller(out int threadId);
}
In your main program /or where you have to call the Test Method:
public static void Main()
{
// The asynchronous method puts the thread id here.
int threadId;
// Create an instance of the test class.
AsyncDemo ad = new AsyncDemo();
// Create the delegate.
AsyncMethodCaller caller = new AsyncMethodCaller(ad.TestMethod);
// Initiate the asychronous call.
IAsyncResult result = caller.BeginInvoke(
out threadId, null, null);
// Call EndInvoke to wait for the asynchronous call to complete,
// and to retrieve the results.
string returnValue = caller.EndInvoke(out threadId, result);
Console.WriteLine("The call executed on thread {0}, with return value \"{1}\".",
threadId, returnValue);
}
From my experience a Classic ASP or ASP.NET page will run until complete, even if the client disconnects, unless you have something in place for checking that the client is still connected and do something if they are not, or a timeout is reached.
However, it would probably be better practice to run these sorts of jobs as scheduled tasks.
On submitting your web page could record in a database that the task needs to be run and then when the scheduled task runs it checks for this and starts the job.
Many web hosts and/or web control panels allow you to create scheduled tasks that call a URL on schedule.
Alternately if you have direct access to the web server you could create a scheduled task on the server to call a URL on schedule.
Or, if ASP.NET, you can put some code in global.asax to run on a schedule. Be aware though, if your website is set to stop after a certain period of inactivity then this will not work unlesss there is frequent continuous activity.