I am learning coroutines and need some help to understand a basic use case.
Implement a non-blocking method that:
Fetches a single item from a (reactive) DB
Determines a range (i.e. the month that the item lives in) based on that item's timestamp
Fetches all items in that month
Returns the items as Flow
Approach
Because it must return a Flow I will not use suspend (like I would when returning a single item). Returning Flow and suspend (which kind of returns a Mono) are most commonly mutually exclusive, right?
So I came up with this signature:
override fun getHistory(beforeUtcMillisExclusive: Long): Flow<Item>
Trying an implementation:
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return emptyFlow()
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end)
}
This gives me on the very first line:
Suspend function 'findOneByTimestampLessThan' should be called only
from a coroutine or another suspend function
I understand the problem that we are not allowed to call a suspend function here and the proposed solution by IntelliJ "adding suspend" does not make sense, when already returning a flow.
So, from this question I got the idea of using a return flow {...}:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return#flow
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return#flow itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start,
range.end)
}
}
The second repository call findByTimestampGreaterThanEqualAndTimestampLessThan returns Flow<Item> and I do not understand why I cannot return it.
This function must return a value of type Unit
Type mismatch.
Required:
Unit
Found:
Flow
return#flow returns from the lambda, not from enclosing function.
You need to reemit items from Flow returned by findByTimestampGreaterThanEqualAndTimestampLessThan call into Flow you're building with flow function:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod != null) {
val range = calcRange(itemInNextPeriod.timestamp)
emitAll(itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end))
}
}
Related
I'm using an API that returns a text like this:
BW3511,HGP,ITP,Canceled,32.
I have to continue fetching until I get a response that is not "Canceled".
this code fetches the data:
val flightResponse = async {
println("Started fetching Flight info.")
client.get<String>(FLIGHT_ENDPOINT).also {
println("Finished fetching Flight info.")
}
}
the client.get can only be called within The coroutineScope body, also the flightResponse type is Deferred<String>.
check if it is canceled:
fun isCanceled(
flightResponse: String
) : Boolean {
val (_, _, _, status, _) = flightResponse.split(",")
return status == "Canceled"
}
how can I repeat client.get<String>(FLIGHT_ENDPOINT) until my condition is met using Functional Programming style?
I tried using takeIf but I have to get at least one result and it cannot be a nullable type.
As said in the comment by #Jorn, this looks like an overuse of functional style. It can be implemented by a simple loop and this way it will be probably more clear to the reader:
fun getNextNotCancelled() {
while (true) {
val response = client.get<String>(FLIGHT_ENDPOINT)
if (!isCanceled(response)) return response
}
}
If your real case is more complex, so you have several filters, etc. or for any other reason you really need to do this declaratively, then you need to create some kind of an infinite generator. For classic synchronous code that means sequence and for asynchronous - flow.
Example using a sequence:
generateSequence { client.get<String>(FLIGHT_ENDPOINT) }
.first { !isCanceled(it) }
Flow:
flow {
while (true) {
emit(client.get<String>(FLIGHT_ENDPOINT))
}
}.first { !isCanceled(it) }
As you said you use coroutines, I assume you would like to go for the latter. And as you can see, it is pretty similar to our initial loop-based approach, only more complicated. Of course, we can create a similar generateFlow() utility function and then it would be shorter.
I understand that the kotlin sequence is just a coroutine with a yield suspend function (correct me if I'm wrong).
So the yield function sets the value and suspends the coroutine until the next value is requested.
yield source code
override suspend fun yield(value: T) {
nextValue = value
state = State_Ready
return suspendCoroutineUninterceptedOrReturn { c ->
nextStep = c
COROUTINE_SUSPENDED
}
}
Link to kotlin source code github
I see that it set the nextValue & state and suspends the coroutine by calling suspendCoroutineUninterceptedOrReturn.
Now I would like to know, where does this get resume?
I don't see any resume call inside next() function
override fun next(): T {
when (state) {
State_NotReady, State_ManyNotReady -> return nextNotReady()
State_ManyReady -> {
state = State_ManyNotReady
return nextIterator!!.next()
}
State_Ready -> {
state = State_NotReady
#Suppress("UNCHECKED_CAST")
val result = nextValue as T
nextValue = null
return result
}
else -> throw exceptionalState()
}
}
Just from looking at the source code page you linked:
Calling next() first checks if hasNext() has already been called. If it has, it can immediately return the next value, which was queued in hasNext(). If hasNext() has not yet been called, nextNotReady() is called, which basically calls hasNext() to prepare that next value and then returns it.
So the real work all happens in hasNext(). It checks if the current continuation’s iterator (which exists if yieldAll() is called) has another value, in which case it can queue it without resuming the continuation. But if the current iterator is done or there was no iterator, then it calls resume on the continuation, which makes it run until the next yield() or yieldAll() call.
This description is a big simplification. There are different edge cases that it is handling based on the order of next/hasNext calls.
I am trying to get the size of this firebase collection size of documents, and for some reason in Kotlin, I can't seem to get this to work. I have declared a variable to be zero in an int function and I put it inside a for loop where it increments to the size of the range. Then when I return the value, it is zero. Here is the code I have provided, please help me as to why it is returning zero.
This is just what is being passed to the function
var postSize = 0
That is the global variable, now for below
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
getPostSize(first)
This is the function
private fun getPostSize(first: Query){
first.get().addOnSuccessListener { documents ->
for(document in documents) {
Log.d(TAG, "${document.id} => ${document.data}")
getActualPostSize(postSize++)
}
}
return postSize
}
private fun getActualPostSize(sizeOfPost: Int): Int {
// The number does push to what I am expecting right here if I called a print statement
return sizeOfPost // However here it just returns it to be zero again. Why #tenffour04? Why?
}
It is my understanding, according to the other question that this was linked to, that I was suppose to do something like this.
This question has answers that explain how to approach getting results from asynchronous APIs, like you're trying to do.
Here is a more detailed explanation using your specific example since you were having trouble adapting the answer from there.
Suppose this is your original code you were trying to make work:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
val postSize = getPostSize(first)
// do something with postSize
// Elsewhere in your class:
private fun getPostSize(first: Query): Int {
var postSize = 0
first.get().addOnSuccessListener { documents ->
for(document in documents) {
Log.d(TAG, "${document.id} => ${document.data}")
postSize++
}
}
return postSize
}
The reason this doesn't work is that the code inside your addOnSuccessListener is called some time in the future, after getPostSize() has already returned.
The reason asynchronous code is called in the future is because it takes a long time to do its action, but it's bad to wait for it on the calling thread because it will freeze your UI and make the whole phone unresponsive. So the time-consuming action is done in the background on another thread, which allows the calling code to continue doing what it's doing and finish immediately so it doesn't freeze the UI. When the time-consuming action is finally finished, only then is its callback/lambda code executed.
A simple retrieval from Firebase like this likely takes less than half a second, but this is still too much time to freeze the UI, because it would make the phone seem janky. Half a second in the future is still in the future compared to the code that is called underneath and outside the lambda.
For the sake of simplifying the below examples, let's simplify your original function to avoid using the for loop, since it was unnecessary:
private fun getPostSize(first: Query): Int {
var postSize = 0
first.get().addOnSuccessListener { documents ->
postSize = documents.count()
}
return postSize
}
The following are multiple distinct approaches for working with asynchronous code. You only have to pick one. You don't have to do all of them.
1. Make your function take a callback instead of returning a value.
Change you function into a higher order function. Since the function doesn't directly return the post size, it is a good convention to put "Async" in the function name. What this function does now is call the callback to pass it the value you wanted to retrieve. It will be called in the future when the listener has been called.
private fun getPostSizeAsync(first: Query, callback: (Int) -> Unit) {
first.get().addOnSuccessListener { documents ->
val postSize = documents.count()
callback(postSize)
}
}
Then to use your function in your "calling code", you must use the retrieved value inside the callback, which can be defined using a lambda:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
getPostSizeAsync(first) { postSize ->
// do something with postSize inside the lambda here
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
2. Handle the response directly in the calling code.
You might have noticed in the above solution 1, you are really just creating an intermediate callback step, because you already have to deal with the callback lambda passed to addOnSuccessListener. You could eliminate the getPostSize function completely and just deal with callbacks at once place in your code. I wouldn't normally recommend this because it violates the DRY principle and the principle of avoiding dealing with multiple levels of abstraction in a single function. However, it may be better to start this way until you better grasp the concept of asynchronous code.
It would look like this:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
first.get().addOnSuccessListener { documents ->
val postSize = documents.count()
// do something with postSize inside the lambda here
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
3. Put the result in a LiveData. Observe the LiveData separately.
You can create a LiveData that will update its observers about results when it gets them. This may not be a good fit for certain situations, because it would get really complicated if you had to turn observers on and off for your particular logic flow. I think it is probably a bad solution for your code because you might have different queries you want to pass to this function, so it wouldn't really make sense to have it keep publishing its results to the same LiveData, because the observers wouldn't know which query the latest postSize is related to.
But here is how it could be done.
private val postSizeLiveData = MutableLiveData<Int>()
// Function name changed "get" to "fetch" to reflect it doesn't return
// anything but simply initiates a fetch operation:
private fun fetchPostSize(query: Query) {
first.get().addOnSuccessListener { documents ->
postSize.value = documents.count()
}
}
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
fetchPostSize(first)
postSizeLiveData.observer(this) { postSize ->
// Do something with postSize inside this observer that will
// be called some time in the future.
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
4. Use a suspend function and coroutine.
Coroutines allow you to write synchronous code without blocking the calling thread. After you learn to use coroutines, they lead to simpler code because there's less nesting of asynchronous callback lambdas. If you look at option 1, it will become very complicated if you need to call more than one asynchronous function in a row to get the results you want, for example if you needed to use postSize to decide what to retrieve from Firebase next. You would have to call another callback-based higher-order function inside the lambda of your first higher-order function call, nesting the future code inside other future code. (This is nicknamed "callback hell".) To write a synchronous coroutine, you launch a coroutine from lifecycleScope (or viewLifecycleOwner.lifecycleScope in a Fragment or viewModelScope in a ViewModel). You can convert your getter function into a suspend function to allow it to be used synchronously without a callback when called from a coroutine. Firebase provides an await() suspend function that can be used to wait for the result synchronously if you're in a coroutine. (Note that more properly, you should use try/catch when you call await() because it's possible Firebase fails to retrieve the documents. But I skipped that for simplicity since you weren't bothering to handle the possible failure with an error listener in your original code.)
private suspend fun getPostSize(first: Query): Int {
return first.get().await().count()
}
// In your "calling code" (inside onCreate() or some click listener):
lifecycleScope.launch {
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
val postSize = getPostSize(first)
// do something with postSize
}
// Code under here will run before the coroutine finishes so
// typically, you launch coroutines and do all your work inside them.
Coroutines are the common way to do this in Kotlin, but they are a complex topic to learn for a newcomer. I recommend you start with one of the first two solutions until you are much more comfortable with Kotlin and higher order functions.
I understand that in Kotlin there is no such thing as "Non-local variables" or "Global Variables" I am looking for a way to modify variables in another "Scope" in Kotlin by using the function below:
class Listres(){
var listsize = 0
fun gatherlistresult(){
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
}
}
}
the value of listsize is always 0 (logging the result from inside of the .addOnSuccessListener scope returns 8) so clearly the listsize variable isn't being modified. I have seen many different posts about this topic on other sites , but none fit my usecase.
I simply want to modify listsize inside of the .addOnSuccessListener callback
This method will always be returned 0 as the addOnSuccessListener() listener will be invoked after the method execution completed. The addOnSuccessListener() is a callback method for asynchronous operation and you will get the value if it gives success only.
You can get the value by changing the code as below:
class Demo {
fun registerListResult() {
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
processResult(listsize)
}
listallinfo.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
fun processResult(listsize: Int) {
print(listResult+"") // you will get the 8 here as you said
}
}
What you're looking for is a way to bridge some asynchronous processing into a synchronous context. If possible it's usually better (in my opinion) to stick to one model (sync or async) throughout your code base.
That being said, sometimes these circumstances are out of our control. One approach I've used in similar situations involves introducing a BlockingQueue as a data pipe to transfer data from the async context to the sync context. In your case, that might look something like this:
class Demo {
var listSize = 0
fun registerListResult() {
val listAll = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
val dataQueue = ArrayBlockingQueue<Int>(1)
listAll.addOnSuccessListener { dataQueue.put(it.items.size) }
listSize = dataQueue.take()
}
}
The key points are:
there is a blocking variant of the Queue interface that will be used to pipe data from the async context (listener) into the sync context (calling code)
data is put() on the queue within the OnSuccessListener
the calling code invokes the queue's take() method, which will cause that thread to block until a value is available
If that doesn't work for you, hopefully it will at least inspire some new thoughts!
I want to iterate over a sequence of objects and return the first non-null of an async call.
The point is to perform some kind of async operation that might fail, and I have a series of fallbacks that I want to try in order, one after the other (i.e. lazily / not in parallel).
I've tried to do something similar to what I'd do if it were a sync call:
// ccs: List<CurrencyConverter>
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asSequence()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
IntelliJ complains:
Suspension functions can only be called within coroutine body
Edit: To clarify, this works as expected if mapping on a List, but I want to see how I'd do this on a sequence.
So I guess this is because the map lambda isn't suspended? But I'm not sure how to actually do that. I tried a bunch of different ways but none seemed to work. I couldn't find any examples.
If I re-write this in a more procedural style using a for loop with an async block, I can get it working:
override suspend fun getExchangeRateAsync(from: String, to: String) {
for (cc in ccs) {
var res: BigDecimal? = async {
cc.getExchangeRateAsync(from, to)
}.await()
if (res != null) {
return res
}
}
throw CurrencyConverterException()
}
You are getting an error, because Sequence is lazy by default and it's map isn't an inline function, so it's scope isn't defined
You can avoid using Sequence by creating a list of lazy coroutines
// ccs: List<CurrencyConverter>
suspend fun getExchangeRateAsync(from: String, to: String) =
ccs
.map { async(start = CoroutineStart.LAZY) { it.getExchangeRateAsync(from, to) } }
.firstOrNull { it.await() != null }
?.getCompleted() ?: throw Exception()
This doesn't give any errors and seems to be working. But I'm not sure it's an idiomatic way
I would suggest replacing Sequence with Flow. Flow api and behavior is pretty much same as for Sequence, but with suspending options.
https://kotlinlang.org/docs/reference/coroutines/flow.html
Code:
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asFlow()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
FWIW, I found the suggestion in How to asynchronously map over sequence to be very intuitive. The code at https://github.com/Kotlin/kotlin-coroutines-examples/blob/master/examples/suspendingSequence/suspendingSequence.kt defines SuspendingIterator which allows next() to suspend, then builds SuspendingSequence on top of it. Unfortunately, you need to duplicate extension functions like flatMap(), filter(), etc. since SuspendingSequence can't be related to Sequence, but I did this and am much happier with the result than using a Channel.