I want to iterate over a sequence of objects and return the first non-null of an async call.
The point is to perform some kind of async operation that might fail, and I have a series of fallbacks that I want to try in order, one after the other (i.e. lazily / not in parallel).
I've tried to do something similar to what I'd do if it were a sync call:
// ccs: List<CurrencyConverter>
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asSequence()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
IntelliJ complains:
Suspension functions can only be called within coroutine body
Edit: To clarify, this works as expected if mapping on a List, but I want to see how I'd do this on a sequence.
So I guess this is because the map lambda isn't suspended? But I'm not sure how to actually do that. I tried a bunch of different ways but none seemed to work. I couldn't find any examples.
If I re-write this in a more procedural style using a for loop with an async block, I can get it working:
override suspend fun getExchangeRateAsync(from: String, to: String) {
for (cc in ccs) {
var res: BigDecimal? = async {
cc.getExchangeRateAsync(from, to)
}.await()
if (res != null) {
return res
}
}
throw CurrencyConverterException()
}
You are getting an error, because Sequence is lazy by default and it's map isn't an inline function, so it's scope isn't defined
You can avoid using Sequence by creating a list of lazy coroutines
// ccs: List<CurrencyConverter>
suspend fun getExchangeRateAsync(from: String, to: String) =
ccs
.map { async(start = CoroutineStart.LAZY) { it.getExchangeRateAsync(from, to) } }
.firstOrNull { it.await() != null }
?.getCompleted() ?: throw Exception()
This doesn't give any errors and seems to be working. But I'm not sure it's an idiomatic way
I would suggest replacing Sequence with Flow. Flow api and behavior is pretty much same as for Sequence, but with suspending options.
https://kotlinlang.org/docs/reference/coroutines/flow.html
Code:
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asFlow()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
FWIW, I found the suggestion in How to asynchronously map over sequence to be very intuitive. The code at https://github.com/Kotlin/kotlin-coroutines-examples/blob/master/examples/suspendingSequence/suspendingSequence.kt defines SuspendingIterator which allows next() to suspend, then builds SuspendingSequence on top of it. Unfortunately, you need to duplicate extension functions like flatMap(), filter(), etc. since SuspendingSequence can't be related to Sequence, but I did this and am much happier with the result than using a Channel.
Related
I'm using an API that returns a text like this:
BW3511,HGP,ITP,Canceled,32.
I have to continue fetching until I get a response that is not "Canceled".
this code fetches the data:
val flightResponse = async {
println("Started fetching Flight info.")
client.get<String>(FLIGHT_ENDPOINT).also {
println("Finished fetching Flight info.")
}
}
the client.get can only be called within The coroutineScope body, also the flightResponse type is Deferred<String>.
check if it is canceled:
fun isCanceled(
flightResponse: String
) : Boolean {
val (_, _, _, status, _) = flightResponse.split(",")
return status == "Canceled"
}
how can I repeat client.get<String>(FLIGHT_ENDPOINT) until my condition is met using Functional Programming style?
I tried using takeIf but I have to get at least one result and it cannot be a nullable type.
As said in the comment by #Jorn, this looks like an overuse of functional style. It can be implemented by a simple loop and this way it will be probably more clear to the reader:
fun getNextNotCancelled() {
while (true) {
val response = client.get<String>(FLIGHT_ENDPOINT)
if (!isCanceled(response)) return response
}
}
If your real case is more complex, so you have several filters, etc. or for any other reason you really need to do this declaratively, then you need to create some kind of an infinite generator. For classic synchronous code that means sequence and for asynchronous - flow.
Example using a sequence:
generateSequence { client.get<String>(FLIGHT_ENDPOINT) }
.first { !isCanceled(it) }
Flow:
flow {
while (true) {
emit(client.get<String>(FLIGHT_ENDPOINT))
}
}.first { !isCanceled(it) }
As you said you use coroutines, I assume you would like to go for the latter. And as you can see, it is pretty similar to our initial loop-based approach, only more complicated. Of course, we can create a similar generateFlow() utility function and then it would be shorter.
The problem is very simple, but I can't really seem to wrap my head around it. I'm launching a non-blocking thread in the IO scope in order to read from a file. However, I can't get the result in time before I return from the method - it always returns the initial empty value "". What am I missing here?
private fun getFileContents(): String {
var result = ""
val fileName = getFilename()
val job = CoroutineScope(Dispatchers.IO).launch {
kotlin.runCatching {
val file = getFile(fileName)
file.openFileInput().use { inputStream ->
result = String(inputStream.readBytes(), Charsets.UTF_8)
}
}
}
return result
}
Coroutines are launched asynchronously. Your non-suspending function cannot wait for the result without blocking. For more information about why asynchronous code results in your function returning with the default result, read the answers here.
getFileContents() has to be a suspend function to be able to return something without blocking, in which case you don't need to launch a coroutine either. But then whatever calls this function must be in a suspend function or coroutine.
private suspend fun getFileContents(): String = withContext(Dispatchers.IO) {
val fileName = getFilename()
kotlin.runCatching {
val file = getFile(fileName)
file.openFileInput().use { inputStream ->
result = String(inputStream.readBytes(), Charsets.UTF_8)
}
}.getOrDefault("")
}
There are two "worlds" of code: either you are in a suspending/coroutine context or you are not. When you are in a function that is not a suspend function, you can only return results that can be computed immediately, or you can block until the result is ready.
Generally, if you're using coroutines, you launch a coroutine at some high level in your code, and then you are free to use suspend functions everywhere because almost all of your code is initially triggered by a coroutine. By "high level", I mean you launch the coroutine when a UI screen appears or a UI button is pressed, for example.
Basically, your coroutine launches are usually in UI listeners and UI event functions, not in lower-level code like the function in your question. The coroutine calls a suspend function, which can call other suspend functions, so you don't need to launch more coroutines to perform your various sequential tasks.
The alternate solution is to return a Deferred with the result, like this:
private fun getFileContents(): Deferred<String> {
val fileName = getFilename()
return CoroutineScope(Dispatchers.IO).async {
kotlin.runCatching {
val file = getFile(fileName)
file.openFileInput().use { inputStream ->
result = String(inputStream.readBytes(), Charsets.UTF_8)
}
}.getOrDefault("")
}
}
But to unpack the result, you will need to call await() on the Deferred instance inside a coroutine somewhere.
I am learning coroutines and need some help to understand a basic use case.
Implement a non-blocking method that:
Fetches a single item from a (reactive) DB
Determines a range (i.e. the month that the item lives in) based on that item's timestamp
Fetches all items in that month
Returns the items as Flow
Approach
Because it must return a Flow I will not use suspend (like I would when returning a single item). Returning Flow and suspend (which kind of returns a Mono) are most commonly mutually exclusive, right?
So I came up with this signature:
override fun getHistory(beforeUtcMillisExclusive: Long): Flow<Item>
Trying an implementation:
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return emptyFlow()
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end)
}
This gives me on the very first line:
Suspend function 'findOneByTimestampLessThan' should be called only
from a coroutine or another suspend function
I understand the problem that we are not allowed to call a suspend function here and the proposed solution by IntelliJ "adding suspend" does not make sense, when already returning a flow.
So, from this question I got the idea of using a return flow {...}:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return#flow
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return#flow itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start,
range.end)
}
}
The second repository call findByTimestampGreaterThanEqualAndTimestampLessThan returns Flow<Item> and I do not understand why I cannot return it.
This function must return a value of type Unit
Type mismatch.
Required:
Unit
Found:
Flow
return#flow returns from the lambda, not from enclosing function.
You need to reemit items from Flow returned by findByTimestampGreaterThanEqualAndTimestampLessThan call into Flow you're building with flow function:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod != null) {
val range = calcRange(itemInNextPeriod.timestamp)
emitAll(itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end))
}
}
I understand that in Kotlin there is no such thing as "Non-local variables" or "Global Variables" I am looking for a way to modify variables in another "Scope" in Kotlin by using the function below:
class Listres(){
var listsize = 0
fun gatherlistresult(){
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
}
}
}
the value of listsize is always 0 (logging the result from inside of the .addOnSuccessListener scope returns 8) so clearly the listsize variable isn't being modified. I have seen many different posts about this topic on other sites , but none fit my usecase.
I simply want to modify listsize inside of the .addOnSuccessListener callback
This method will always be returned 0 as the addOnSuccessListener() listener will be invoked after the method execution completed. The addOnSuccessListener() is a callback method for asynchronous operation and you will get the value if it gives success only.
You can get the value by changing the code as below:
class Demo {
fun registerListResult() {
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
processResult(listsize)
}
listallinfo.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
fun processResult(listsize: Int) {
print(listResult+"") // you will get the 8 here as you said
}
}
What you're looking for is a way to bridge some asynchronous processing into a synchronous context. If possible it's usually better (in my opinion) to stick to one model (sync or async) throughout your code base.
That being said, sometimes these circumstances are out of our control. One approach I've used in similar situations involves introducing a BlockingQueue as a data pipe to transfer data from the async context to the sync context. In your case, that might look something like this:
class Demo {
var listSize = 0
fun registerListResult() {
val listAll = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
val dataQueue = ArrayBlockingQueue<Int>(1)
listAll.addOnSuccessListener { dataQueue.put(it.items.size) }
listSize = dataQueue.take()
}
}
The key points are:
there is a blocking variant of the Queue interface that will be used to pipe data from the async context (listener) into the sync context (calling code)
data is put() on the queue within the OnSuccessListener
the calling code invokes the queue's take() method, which will cause that thread to block until a value is available
If that doesn't work for you, hopefully it will at least inspire some new thoughts!
So I have parsers and want to use the first that does return a non-null value. How would I do that most elegantly?
return parsers.map { it.parse(content) }.firstOrNull { it != null }
would map all (million?) parsers before picking the first.
return parsers.firstOrNull { it.parse(content) != null }?.parse(content)
would run the (expensive?) parse() once again.
I know I can
for (parser in parsers) {
val result = parser.parse(content)
if (result != null) {
return result
}
}
return null
parsers.forEach { it.parse(content)?.run { return this } }
return null
is the shortest I can get but it's not nice to read.
I'm pretty sure there is a shortcut here that I don't see.
Use a sequence. It makes your computation lazy, so that you will only compute parse as many times as you need.
return parsers.asSequence()
.map { it.parse(content) }
.find { it != null }
As an alternative to the overhead of a Sequence, or mapping lots of values unnecessarily, you could use an extension method such as:
inline fun <T, R> List<T>.firstMappedNotNull(transform: (T) -> R): R? {
for (e in this)
return transform(e) ?: continue
return null
}
This uses the minimum of mapping function calls and temporary objects. It's necessarily written in an imperative way, but it's quite short, and makes your own code short, clear, and functional.
(This version returns null if the list was empty or every mapping returned null. You could of course change the signature and last line to throw an exception instead.)
It's a shame this function isn't already in the standard library. But it's easy to add your own!
Also, you can use the following code:
parsers.asSequence()
.mapNotNull { it.parse(content) }
.first()