I have two controllers, one provide a stream to download a file, the other allows the file to be deleted.
When a request to delete the file comes in, I'd like to interrupt the stream download (from disk) so that the file can be deleted.
I have tried combining cancellation tokens and storing them in an IMemoryCache. Whilst I can retrieve the value and call the cancel on the CancellationTokenSource, it does not trigger the cancel on the other controller.
Is there some mechanism by which I can stop the streaming of the file in order to delete it?
The use-case is similar to YouTube in the sense that if the content creator wanted to delete their video, YouTube would let you watch what is already buffered, but not allow you to stream any more of the video once the delete request came in.
Edit: Adding code:
Delete:
var cacheEntryOptions = new MemoryCacheEntryOptions().SetPriority(CacheItemPriority.NeverRemove).SetAbsoluteExpiration(TimeSpan.FromMinutes(30));
this.MemoryCache.Set(fileId, true, cacheEntryOptions);
My thought here is that the delete action will add something to the cache which the stream action can check for.
Stream action:
while (!response.HttpContext.RequestAborted.IsCancellationRequested && bytesRemaining > 0)
{
if (this.MemoryCache.TryGetValue(this.FileId, out bool dummyValue))
{
throw new OperationCanceledException("Operation cancelled remotely");
}
}
With RequestAborted I expect if the client makes their own cancellation request the stream will stop (which it does).
Surrounding the while I have a try/catch:
catch (OperationCanceledException cancelled)
Which is there to dispose of the file accessor.
I tried something similar with CancellationTokens and combining them using a TokenSource, but calling cancel seemed to do nothing at all.
The problem with the current approach is that there appears to be a situation in which the access to the file is not disposed when the delete action is called (even if within that call I sleep for 10 minutes). If I make a subsequent call instead after a minute or so (without the 10 minute delay) then the file can be deleted. It's confusing as to what is holding on to a lock to the file and I feel as though cancellation tokens would solve the problem here...
Related
In my Angular app I'm working on notifications and I have an REST API to call for latest user's notifications. I need to call this API on few minutes since it's not really important that user gets notifications in real time (they probably won't even appear that fast). However the idea to refresh notifications on the client side is next:
When user logs in start refreshing notifications - here is the first manual call to start refreshing the API on few minutes
If user leaves app opened or is just navigating through the app then don't change timer and wait for the rest of the time
If user opens subpage where it can perform actions related to notifications and does it, then refresh notifications and reset timer
Refresh notifications until logout
I already have working code for the described procedure, but I'm somehow unsure that it's correct for what I need. Here is the code for performing calls (for manual check there is just a Subject and for stop checking there is a subscription to observable - code below is actually separated, but here is in one place because of readability):
// Subject for manual triggering
this.checkFeed = new Subject<void>();
// Call for refresh in own method
this.checkFeed.next();
// Waiting for manual refresh or triggering it on some interval after it was last triggered
this.feedSub = this.checkFeed.asObservable()
.switchMap(() => Observable.timer(0, this.interval))
.mergeMap(() => this.fetchChanges())
.distinctUntilChanged(this.compareFeed)
.subscribe(res => this.notify(res));
// Unsubscription when logging out
if (this.feedSub) this.feedSub.unsubscribe();
The part which I'm most unsure about is .switchMap(() => Observable.timer(0, this.interval)) since it needs 0 to start right away (which is ok, but still doesn't look correct at all?). So is there any better way to achieve what I described?
I also have another question how to start check for notifications from another observable - which operator should I use. As I mentioned I have call to the Subject's next in own method like this:
refreshFeed(): void {
this.checkFeed.next();
}
So when there is some other observable performing (the action when notifications should be refreshed) I need to call this one. What's the correct way to call void method when other observable has response from API? I was thinking of something like this:
someActionThatCanChangeNotifications(): Observable<any> {
return this.api.get('path/to/endpoint')
.do(() => this.feedService.refreshFeed());
}
Is this ok, or is there also any better way?
Thanks in advance for help!
So basically you have two observable.
One that you call manually:
this.checkFeed
and the interval(let's callit intervalObs):
this.intervalObs = Observable.timer(0, this.interval);
If you see it like this the easyest way is to merge you'r two source stream and then do whatever you want.
var mergedSource = Observable.merge(
this.checkFeed,
this.intervalObs)
subscription = mergedSource.subscribe(this.fetchChanges());
Maybe you need to do some more operation in between but this should give you a more readable alternative.
You can try this working plunker if you want something to play arround https://plnkr.co/edit/n4nNFEMa4YOh2KSjDpSJ?p=preview
From what I can see you've pretty much done it "correctly". As with programming in general, there are many possible (and correct) solutions to a single problem. Personally, I'd do this the same way.
I can give you some commentary on the two points you mentioned too:
.switchMap(() => Observable.timer(0, this.interval))
Observable.timer pretty much an Observable.interval with a custom timeout before the first value. Observable.timer(0, this.interval) is the correct usage.
An alternative could be Observable.just(0).concat(Observable.interval(this.interval)), which returns a value immediately and then starts the interval. I prefer the way you put however; I think it clearly states your intention: "Produce a value after 0 milliseconds, and then an interval of this.interval".
.do(() => this.feedService.refreshFeed())
I'd say this is the totally correct way of doing it. do is meant for side effects, eg. stuff that happening outside the observable.
I can say though, I wouldn't expect someActionThatCanChangeNotifications to kick off a refresh of the feed. When a function returns an observable, I would expect to return an observable that doesn't have any side effects. However, as we live in a non-perfect world, we can't always have what we want.
You can't expect every subscriber to remember to do .do(() => this.feedService.refreshFeed()), instead I'd add a notice in the doc comment for the function: "Note: The returned observable will refresh the feed on every next signal", or something of that kind.
I have some code which imports a csv file and creates database records from the data. Using a standard synchronous approach this works fine, but large csv files can take a while to process. Using async await, I'd like to have the csv processing run asynchronously so the user isn't stuck on the upload form until the processing is complete.
So the user would upload a file and immediately be transferred to a page which would display a message to indicate that the file was importing.
This is what I've tried. This is my controller action that gets the form POST:
public async Task<ActionResult> UploadBulkImportFile(BulkUploadCsvModel model, HttpPostedFileBase file)
{
...
within this I call the functionality that I want to run asynchronously:
await Task.Run(() => testAsyncMethod(csvmodel));
The testAsyncMethod() does all the db crud operations involved in processing the csv.
public void testAsyncMethod(IEnumerable<List<csvmodel>> model)
{
...
This code works but is no different to my previous synchronous code, i.e. the user still has to wait until the file is fully processed, so it's clearly not working properly. Note that the code in testAsyncMethod() is NOT async code, just standard db calls - I'm wondering if that's where I'm going wrong.
Is this even possible in MVC? am I using async await for the wrong reasons?
So the user would upload a file and immediately be transferred to a page which would display a message to indicate that the file was importing.
That's not possible using asynchronous controller actions. This is because (as I describe on my blog) async does not change the HTTP protocol.
In your case, I'd recommend some JavaScript on the client to fire off the POST, change the current page to a "importing..." message, and then update the same page when the POST request completes.
And then remove the async/await/Task.Run from your controller action; it won't provide any benefit.
I am uploading multiple files using javascript.
After I upload the files, I need to run several processing functions.
Because of the processing time that is required, I need a UI on the front telling the user the estimated time left of the entire process.
Basically I have 3 functions:
/upload - this is an endpoint for uploading the files
/generate/metadata - this is the next endpoint that should be triggered after /upload
/process - this is the last endpoint. SHould be triggered after /generate/metadata
This is how I expect the screen to look like basically.
Information such as percentage remaining and time left should be displayed.
However, I am unsure whether to allow server to supply the information or I do a hackish estimate solely using javascript.
I would also need to update the screen like telling the user messages such as
"currently uploading"
if I am at function 1.
"Generating metadata" if I am at function 2.
"Processing ..." if I am at function 3.
Function 2 only occurs after the successful completion of 1.
Function 3 only occurs after the successful completion of 2.
I am already using q.js promises to handle some parts of this, but the code has gotten scarily messy.
I recently come across Backbone and it allows structured ways to handle single page app behavior which is what I wanted.
I have no problems with the server-side returning back json responses for success or failure of the endpoints.
I was wondering what would be a good way to implement this function using Backbone.js
You can use a "progress" file or DB entry which stores the state of the backend process. Have your backend process periodically update this file. For example, write this to the file:
{"status": "Generating metadata", "time": "3 mins left"}
After the user submits the files have the frontend start pinging a backend progress function using a simple ajax call and setTimeout. the progress function will simply open this file, grab the JSON-formatted status info, and then update the frontend progress bar.
You'll probably want the ajax call to be attached to your model(s). Have your frontend view watch for changes to the status and update accordingly (e.g. a progress bar).
Long Polling request:
Polling request for updating Backbone Models/Views
Basically when you upload a File you will assign a "FileModel" to every given file. The FileModel will start a long polling request every N seconds, until get the status "complete".
I am designing an IPhone application. User search something. We grab data from the net. Then we update the table.
THe pseudocode would be
[DoThisAtbackground ^{
LoadData ();
[DoThisAtForeground ^{
UpdateTableAndView();
}];
}];
What about if before the first search is done the user search something else.
What's the industry standard way to solve the issue?
Keep track which thread is still running and only update the table
when ALL threads have finished?
Update the view every time a thread finish?
How exactly we do this?
I suggest you take a look at the iOS Human Interface Guidelines. Apple thinks it's pretty important all application behave in about the same way, so they've written an extensive document about these kind of issues.
In the guidelines there are two things that are relevant to your question:
Make Search Quick and Rewarding: "When possible, also filter remote data while users type. Although filtering users' typing can result in a better search experience, be sure to inform them and give them an opportunity to opt out if the response time is likely to delay the results by more than a second or two."
Feedback: "Feedback acknowledges people’s actions and assures them that processing is occurring. People expect immediate feedback when they operate a control, and they appreciate status updates during lengthy operations."
Although there is of course a lot of nonsense in these guidelines, I think the above points are actually a good idea to follow. As a user, I expect something to happen when searching, and when you update the view every time a thread is finished, the user will see the fastest response. Yes, it might be results the user doesn't want, but something is happening! For example, take the Safari web browser in iOS: Google autocomplete displays results even when you're typing, and not just when you've finished entering your search query.
So I think it's best to go with your second option.
If you're performing the REST request for data to your remote server you can always cancel the request and start the new one without updating the table, which is a way to go. Requests that have the time to finish will update UI and the others won't. For example use ASIHTTPRequest
- (void)serverPerformDataRequestWithQuery:(NSString *)query andDelegate:(__weak id <ServerDelegate)delegate {
[currentRequest setFailedBlock:nil];
[currentRequest cancel];
currentRequest = [[ASIHTTPRequest alloc] initWithURL:kHOST];
[currentRequest startAsynchronous];
}
Let me know if you need an answer for the local SQLite databases too as it is much more complicated.
You could use NSOperationQueue to cancel all pending operations, but it still would not cancel the existing operation. You would still have to implement something to cancel the existing operation... which also works to early-abort the operations in the queue.
I usually prefer straight GCD, unless there are other benefits in my use cases that are a better fit for NSOperationQueue.
Also, if your loading has an external cancel mechanism, you want to cancel any pending I/O operations.
If the operations are independent, consider a concurrent queue, as it will allow the newer request to execute simultaneously as the other(s) are being canceled.
Also, if they are all I/O, consider if you can use dispatch_io instead of blocking a thread. As Monk would say, "You'll thank me later."
Consider something like this:
- (void)userRequestedNewSearch:(SearchInfo*)searchInfo {
// Assign this operation a new token, that uniquely identifies this operation.
uint32_t token = [self nextOperationToken];
// If your "loading" API has an external abort mechanism, you want to keep
// track of the in-flight I/O so any existing I/O operations can be canceled
// before dispatching new work.
dispatch_async(myQueue, ^{
// Try to load your data in small pieces, so you can exit as early as
// possible. If you have to do a monolithic load, that's OK, but this
// block will not exit until that stops.
while (! loadIsComplete) {
if ([self currentToken] != token) return;
// Load some data, set loadIsComplete when loading completes
}
dispatch_async(dispatch_get_main_queue(), ^{
// One last check before updating the UI...
if ([self currentToken] != token) return;
// Do your UI update operations
});
});
}
It will early-abort any operation that is not the last one submitted. If you used NSOperationQueue you could call cancelAllOperations but you would still need a similar mechanism to early-abort the one that is currently executing.
I'm developing an iPad-app and I'm currently struggling with finding the best approach to multithreading. Let me illustrate this with a simplified example:
I have a view with 2 subviews, a directory picker and a gallery with thumbnails of all the images in the selected directory. Since 'downloading' and generating these thumbnails can take quite a while I need multithreading so the interaction and updating of the view doesn't get blocked.
This is what I already tried:
[self performSelectorInBackground:#selector(displayThumbnails:) withObject:currentFolder];
This worked fine because the users interactions didn't get blocked, however it miserably fails when the user taps on another folder while the first folder is still loading. Two threads are trying to access the same view and variables which results in messing up each others proper execution. When the users taps another folder, the displayThumbnails of the currently loading folder should get aborted. I didn't find any way to do this..
NSThreads
I tried this but struggled with almost the same problems as with the first method, I didn't find a (easy) way to cancel the ongoing method. (Yes, I know about [aThread cancel] but didn't find a way to 'resume' the thread). Maybe I should subclass NSThread and implement my own isRunning etc methods? But isn't there any better way or a third (or even fourth and fifth) option I'm overlooking?
I think this is a fairly simple example and I think there is perhaps a better solution without subclassing NSThread. So, what would you do? Your opinions please!
NSOperationQueue should work well for this task.
Another option would be plain GCD, however, if you've never worked with it, NSOperationQueue is probably the better choice since it pretty much automatically guides you to implementing things "the right way", has obvious ways for cancellation, etc.
You want to use Concurrent NSOperations to download and process images in the background. These would be managed by an NSOperationsQueue. Essentially these operations would be configured to fetch one image per operation, process it, save it in the file system, then message back to the main app in the main thread that the image is available.
There are several projects on github that you can look at that show how to do this - just search github using "Concurrent" or "NSOperation".
iOS has a really nice facility for doing background work. Grand Central Dispatch (GCD) and Blocks, but those don't let you have an object using delegate callbacks - thus NSOperation.
So you need to read up on blocks, GCD, and then look at some open source Concurrent NSOperations code. Using Concurrent NSOperations is not as simple as using blocks.
If I had this problem, I would probably go for an approach like this:
a single thread that will load the images, and causes the main thread to display results (I'm not a big fan of having thread mess around with GUI objects)
when a new directory is requested... well, it depends on how you want to manage things. Basically, a standard queue construct (condition variable and array) could be used for the main thread to tell the thread that "this directory will be needed" by passing it the path name; the thread will check the queue even when it's loading images (like after every image or so), and switch to the new directory whenever one shows up
you could make a directory-reader object that keeps all the state, and store this indexed by the path into a dictionary. When a new directory is requested, check that dictionary first, and only create a new object if there's none for this directory. That way, partially loaded directories would stick around until they are needed again, and can continue to load instead of having to start from scratch.
Pseudocode for the thread:
while (forever)
new element = nil
if we have an active directory loader
tell directory loader to load one image
if false then make directory loader inactive
lock queue condition
if queue has elements
new element = retrieve LAST element (we aren't interested in the others)
empty queue
unlock with status "empty"
else
unlock queue
else
lock queue on condition "has elements"
new element = retrieve last element
empty queue
unlock with status "empty"
if new element != nil
if directory loader for new path does not exist
setup new directory loader for new path
store in dictionary
make it the "active" one
else
make the current one the "active"
As for the directory loader, it might look something like this:
read one image:
if there are still images to read:
read, process and store one
return true
else
performSelectorOnMainThread with an "update GUI" method and the image list as parameter
return false;
This is just a quick sketch; there's some code duplication in the thread, and the way I wrote it will only update the GUI after all images have been read, instead of making them appear as we read them. You'll have to copy the current image list, or add synchronization if you want to do that.