I'm relatively new to the async/await pattern, but I'm trying to get up to speed because I'm using a 3rd party API that only exposes async methods. I need to write a recursive method to crawl through a directory structure. Suppose for example the structure looked like this:
/(root)
Folder1/
Folder3/
Folder4/
Folder2/
Folder5/
Folder6
My initial version of the crawl was a "fully" async version, and it looked something like this:
Async Function GetFolderContents(thisPath) As Task(Of Blah())
Dim result as Blah()
thisFolderContents = await _thirdPartyAsyncMethod.GetContents(thisPath)
For Each singleFolder In thisFolderContents
result.Add(singleFolder)
result.AddRange(await GetFolderContents(singleFolder))
Next
Return result
End Function
Whenever I tried to call this method, I could never get any of the results of Folder2 or its children - I could only get it to return Folder1 and its children. Based on my tenuous knowledge of async/await I kind of understand why that would be (though I'm fully prepared to admit I just might be doing it wrong).
I could make it work the way I expected by removing Async from my method definition and using the .Result of the third party async method:
thisFolderContents = _thirdPartyAsyncMethod.GetContents(thisPath).Result
Even though this seems to work my question is, should I be doing it that way? I gather these methods were made async for a reason and this method effectively makes then synchronous. Is there a more proper way to implement recursion when calling async methods that you didn't write, or is this one of the edge cases where using .Result is the only way to do it?
Related
I tried my best but could not find information on calling an API inside the Javascript function when dealing with automation in Karate. Now, I might get suggestions to call the API outside the function and then do operations inside the function. However, my use case is such that I have to call the API inside the function only. Is there a way to do this?
One approach is to create a Java file and then write the code in java. However, I specifically want to know if there is any way to call an API inside a JS function in a FEATURE FILE itself.
First, these kinds of "clever" tests are not recommended, please read this to understand why: https://stackoverflow.com/a/54126724/143475
If you still want to do this, read on.
First - most of the time, this kind of need can be achieved by doing a call to a second feature file:
* if (condition) karate.call('first.feature')
Finally, this is an experimental and un-documented feature in Karate, but there is a JS API to perform HTTP requests:
* eval
"""
var http = karate.http('https://httpbin.org');
http.path('anything');
var response = http.get().body;
karate.log('response:', response);
"""
It is a "fluent API" so you can do everything in one-line:
var body = karate.http('https://httpbin.org/get').get().body;
If you need details, read the source-code of the HttpRequestBuilder and Response classes in the Karate project.
Updated below...
So this is a random question that came about from a discussion over what I consider the over usage of await in projects at work...
Not sure why I never tried it until now, other than the fact that it's weird and random, but the fact it does work just makes me wonder...how?
The .Result usage was only used to get the responseMessage. I know that is blocking and no bueno and it is for demo purposes only...
So this endpoint works fine...nothing is awaited
[HttpGet("kitteh")]
public Task<string> GetCatFact()
{
var client = new HttpClient();
var res = client.GetAsync("https://catfact.ninja/fact").Result;
return res.Content.ReadAsStringAsync();
}
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result...but where...or how?
Is this "less efficient" than awaiting in the endpoint itself as the magic taking place behind the scenes is ultimately just blocking to get the result of the returned task?
Updated
So, I am still skeptical and don't think the answer is as easy as "yes it is blocking" or "not it is not blocking" unless there's some legit proof or something to indicate one way or the other. I tried digging through the code myself, and I still don't have a solid answer but...I do know more than I did before...
I simplified the endpoints I have been testing this with...
[HttpGet("taskstring")]
public Task<string> TaskString()
{
return Task.FromResult("Where does this block?");
}
[HttpGet("asyncstring")]
public async Task<string> AsyncString()
{
return await Task.FromResult("This definitely doesn't block");
}
I stepped through a bunch of code and ultimately landed on ObjectMethodExecutor and AwaitableObjectResultExecutor which sparked some interest.
What appears to happen for both the endpoint versions above is the same or nearly the same, at least the execution of these two endpoints and the code covered below. There's a boatload that goes on during this and even though the rider debugger is great, it's not possible to see a lot of the values while debugging due to "Evaluation is not allowed: The thread is not at a GC-safe point".
When the endpoint is called but prior to the endpoint beginning execution...
ObjectMethodExecutor.ctor is called
This does a lot of inspection of the endpoint to determine a bunch of things
One of the checks is to is determine if the method is "async" and it ultimately calls AwaitableInfo.IsTypeAwaitable to check for the required methods/properties and interfaces to ensure it is
If it is, which in the case of returning Task of string it is, makes sense given the above info
ObjectMethodExecutor.GetExecutorAsync() is called
AwaitableObjectResultExecutor.Execute is then called
this type inherits ActionMethodExecutor which overrides the Execute method
executor.ExecuteAsync(executor type of ObjectMethodExecutorAwaitable) is then called, and awaited
this returns a result of type object, boxing yeah I know
The endpoint then actually executes and returns to AwaitableObjectResultExecutor.Execute
the result is then passed to ConvertToActionResult along with the return type, the T of Task of T and the mapper implementation
the mapped ActionResult is then returned
So...I'm still not certain 100% either way but there is a lot of effort put into inspecting the endpoint that is being called and determining if it's async or not, which would make it seem logical to think this is an attempt to avoid something...maybe blocking?
So...does it block...maybe? Seeing that there is an await from the indirect caller of the endpoint, I'd lean towards, no...but it seems really difficult to say still.
What I do now know is...
It definitely does await the endpoint call, albeit indirectly
It converts the Task of string return type to an IActionResult
An async and task only version seem to follow the exact some flow shown above
A non async version does not
So, based on all of that...#Phil's answer seems to be pretty spot on with what I found...
"The framework supports asynchronous controller actions. In order to do so, it would need to inspect the return value of your methods. If the method returns a Task, it will ultimately await on the result before responding.
Even if your action itself does all the awaiting, it still has to return a Task so the caller will still wait (the alternative being some ugly blocking code)."
Thanks to all for participating in an extremely random question that doesn't hold much value to truly understand
Your assumption here is basically correct...
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result
The framework supports asynchronous controller actions. In order to do so, it would need to inspect the return value of your methods. If the method returns a Task, it will ultimately await on the result before responding.
Even if your action itself does all the awaiting, it still has to return a Task so the caller will still wait (the alternative being some ugly blocking code).
As has been pointed out in some other posts, there are some performance improvements to be had by not awaiting a returned Task so I would write your action as
public async Task<string> GetCatFact()
{
var client = new HttpClient();
var res = await client.GetAsync("https://catfact.ninja/fact");
return res.Content.ReadAsStringAsync(); // no await
}
Your controller handles waiting for the remote response but delegates waiting for the content stream to the caller.
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result...but where...or how?
ASP.NET asynchronously waits for your task to complete, and then it sends the HTTP response based on the result of the task. It's logically similar to await: an asynchronous wait.
Is this "less efficient" than awaiting in the endpoint itself as the magic taking place behind the scenes is ultimately just blocking to get the result of the returned task?
Yes. It is less efficient to block.
ASP.NET doesn't block; it asynchronously waits. Blocking ties up a thread. So when the code calls .Result, it will be using a thread just to wait for that HttpClient call to complete.
The proper solution is to keep async and await:
[HttpGet("kitteh")]
public async Task<string> GetCatFact()
{
var client = new HttpClient();
var res = await client.GetAsync("https://catfact.ninja/fact");
return await res.Content.ReadAsStringAsync();
}
This way, while the GetAsync is in progress, the thread is yielded back to the ASP.NET runtime and is available for handling other requests, instead of being blocked waiting for the GetAsync to complete.
More information: Task<string> is part of the method signature. ASP.NET has special understanding of the Task<T> type and knows to asynchronously wait for it. async is not part of the method signature. ASP.NET knows whether your method returns Task, but it has no idea whether it's async (and doesn't care). So, in some situations, it's OK to elide the keywords (as described on my blog, but only when the method implementation is trivial. If there's any logic in the method, keep the async and await.
I've been asked to help write some server-side scripts that update calendars, contacts, e-mail and other company services from other internal services for compliance reasons. The code needs to access the LDAP server, an SQL database and e-mail server, compile and merge information in a peculiar way and then go through information in the calendars, contacts and update those depending on what's there and in the LDAP/SQL databases. This needs to be done a couple of times a day, so performance isn't particularly important.
I wanted to try to use node.js for this and after spending a few days with it, I'm having second thoughts as to whether node.js is the right tool to do this. I'm an old school programmer, have C, C++, Unix, SQL, LDAP, TCP/IP in my small finger but I've learned JavaScript/Node.js in a few days out of curiosity.
I use LDAP, SQL and CalDav/CardDAV modules from npm. These are based on Promises.
Thanks to promises, the code is super ugly, unreadable and buggy if there's any kind of network problem. Things that are very easy in classic language such as C or Java are suddenly a massive headache, such as the fact that (say) LDAP information will arrive at a later stage, but there's no point in async operations as nothing can be done in parallel while waiting for those. This pattern repeats itself throughout the code - async operations complicating matters incredibly for zero benefit. It's incredibly difficult to calculate any sort of aggregate values and apply them elsewhere when using these async functions.
My question is this: is there a simple way to invoke async functions that will simply return the value instead of invoking callbacks?
in javascript - no. Waiting async call to complete would block the only thread and so would stop all the work, and so such waiting is not implemented.
But there exists async/await mechanism - syntactic sugar over asynchronous calls, which mimics synchronous calls.
First of all, the vast majority of I/O calls in Node.js are asynchronous. So if you're totally uncomfortable with this, it may not be the right fit for what you're doing.
It is a (IMHO) brilliant language, and you'll find it more comfortable to use as you get used to it.
However, you can write code using Promises that looks very much like synchronous code using the async/await syntax. I find this much easier to deal with compared to using lot's of .thens and .catch.
You can also use try and catch in this context and it will work as you would expect.
This makes for much cleaner, more readable code.
Most Node.js modules support Promises, for those that don't you can often convert functions to returning a Promise rather than expecting a callback to be passed.
NB: Don't forget to await calls to async. functions. I find this is one of the most common errors one makes when using this syntax.
For example:
function callApi() {
return new Promise(resolve => setTimeout(() => resolve({ status: 'ok'}), 500))
}
async function testApi() {
try {
console.log("callApi: Calling function...");
let result = await callApi();
console.log("callApi result:", result);
} catch (err) {
console.error("callApi: Error:", err.message);
}
}
testApi();
Then a further example to show catching an error (say the api fails for some reason):
function callApi() {
return new Promise((resolve, reject) => setTimeout(() => reject(new Error("Api error")), 500))
}
async function testApi() {
try {
console.log("callApi: Calling function...");
let result = await callApi();
console.log("callApi result:", result);
} catch (err) {
console.error("callApi: Error:", err.message);
}
}
testApi();
In asp.net core ViewComponent we have to implement logic in an InvokeAsync method that returns an IViewComponentResult. However I do not have any async logic to perform inside the invoke method. So based on SO post here I have removed the async qualifier and just return Task.FromResult
public Task<IViewComponentResult> InvokeAsync(MyBaseModel model)
{
var name = MyFactory.GetViewComponent(model.DocumentTypeID);
return Task.FromResult<IViewComponentResult>(View(name, model));
}
and then in View ( since I don't have async I am not using await here)
#Component.InvokeAsync("MyViewComponent", new { model = Model })
However view renders this:
System.Threading.Tasks.Task1[Microsoft.AspNetCore.Html.IHtmlContent]`
You must await the Component.InvokeAsync. The fact that your method doesn't do anything async doesn't matter. The method itself is async.
However, that's a bit of an oversimplification. Frankly, the ease of the async/await keywords belies how complicated all this actually is. To be accurate, instead of calling these types of methods "async", it's more appropriate to discuss them as "task-returning". A task is essentially a handle for some operation. That operation could be async or sync. It's most closely associated with async, simply because wrapping sync operations in a task would be pretty pointless in most scenarios. However, the point is that just because something must return a task does not also imply that it must be async.
All async does is allow the possibility of a thread switch. In scenarios where there's some operation, typically involving I/O, that would cause the working thread to be idle for some period of time, the thread becomes available to be used for other work, and the original work may complete on a different thread. Notice the use of the passive language here. Async operations can involve no thread switching; the task could complete on the same thread, as if it was sync. The task could even complete immediately, if the underlying operation has already completed.
In your scenario here, you're not doing any async work, which is fine. However, the method definition requires Task<T> as the return, so you must use Task.FromResult to return your actual result. That's all pretty standard stuff, and seems to be understood already by you. What you're missing, I think, is that you're thinking that since you're not actually doing any asynchronous work, it would be wrong to utilize await. There's nothing magical about the await keyword; it basically just means hold here until the task completes. If there's no async work to be done, as is the case here, the sync code will just run as normal and yield back to the calling code when done, However, as a convenience, await also performs one other vital function: it unwraps the task.
That is where your problem lies. Since you're not awaiting, the task itself is being returned into the Razor view processing pipeline. It doesn't know what to do with that, so it does what it does by default and just calls ToString on it, hence the text you're getting back. Unwrapped, you'd just have IViewComponentResult and Razor does know what to do with that.
If your logic performed inside the invoke method is synchronous, i.e., you don't have any await, you have 2 options:
You can define invoke method without async keyword and it should return Task.FromResult<T>
Use public IViewComponentResult Invoke() instead.
I think the async keyword enables the await keyword and that's pretty much about it. Nothing special about async keyword.
On the main thread where your view is getting rendered, since the tag helper method
to invoke a view component Component.InvokeAsync() is awaitable, you do need to put await keyword there to start the task. await examines the view component rendering to see if it has already completed. If it has, then the main thread just keeps going. Otherwise the main thread will tell the ViewComponent to run.
I have a GenServer, which is responsible for contacting an external resource. The result of calling external resource is not important, ever failures from time to time is acceptable, so using handle_cast seems appropriate for other parts of code. I do have an interface-like module for that external resource, and I'm using one GenServer to access the resource. So far so good.
But when I tried to write test for this gen_server, I couldn't figure out how to test the handle_cast. I have interface functions for GenServer, and I tried to test those ones, but they always return :ok, except when GenServer is not running. I could not test that.
I changed the code a bit. I abstracted the code in handle_cast into another function, and created a similar handle_call callback. Then I could test handle_call easily, but that was kind of a hack.
I would like to know how people generally test async code, like that. Was my approach correct, or acceptable? If not, what to do then?
The trick is to remember that a GenServer process handles messages one by one sequentially. This means we can make sure the process received and handled a message, by making sure it handled a message we sent later. This in turn means we can change any async operation into a synchronous one, by following it with a synchronisation message, for example some call.
The test scenario would look like this:
Issue asynchronous request.
Issue synchronous request and wait for the result.
Assert on the effects of the asynchronous request.
If the server doesn't have any suitable function for synchronisation, you can consider using :sys.get_state/2 - a call meant for debugging purposes, that is properly handled by all special processes (including GenServer) and is, what's probably the most important thing, synchronous. I'd consider it perfectly valid to use it for testing.
You can read more about other useful functions from the :sys module in GenServer documentation on debugging.
A cast request is of the form:
Module:handle_cast(Request, State) -> Result
Types:
Request = term()
State = term()
Result = {noreply,NewState} |
{noreply,NewState,Timeout} |
{noreply,NewState,hibernate} |
{stop,Reason,NewState}
NewState = term()
Timeout = int()>=0 | infinity
Reason = term()
so it is quite easy to perform unit test just calling it directly (no need to even start a server), providing a Request and a State, and asserting the returned Result. Of course it may also have some side effects (like writing in an ets table, modifying the process dictionary...) so you will need to initialize those resources before, and check the effect after the assert.
For example:
test_add() ->
{noreply,15} = my_counter:handle_cast({add,5},10).