I tried to combine several SharedFlow into one StateFlow using stateIn. But my StateFlow seems not updating after I emit new value to it's SharedFlow source. I found out that the problem is from how I used stateIn.
Here is the simplified code I am using (you can run it from kotlin playground).
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.collect
import kotlinx.coroutines.flow.stateIn
import kotlinx.coroutines.launch
fun main() = runBlocking {
val sourceFlow = MutableSharedFlow<Int>()
val stateFlow = sourceFlow.stateIn(GlobalScope, SharingStarted.Lazily, 0)
val job = launch { stateFlow.collect() }
sourceFlow.emit(99)
println(stateFlow.value)
job.cancel()
}
println(stateFlow.value) prints out 0 when it should prints 99. I already follow through this documentation about stateIn but still can not find the issue. Someone knows where I did wrong?
The short answer is that no collection has started when 99 is emitted, and thus the value is simply discarded. This is the expected behaviour of unbuffered shared flow described in the docs of SharedFlow:
A default implementation of a shared flow that is created with MutableSharedFlow() constructor function without parameters has no replay cache nor additional buffer. emit call to such a shared flow suspends until all subscribers receive the emitted value and returns immediately if there are no subscribers.
The reason there are no subscribers on sourceFlow (the shared flow) is because you called stateIn() on this flow with SharingStarted.Lazily:
Sharing is started when the first subscriber appears and never stops
That means the state flow's coroutine only starts collecting the source when the first subscriber starts collecting the state flow. And the reason there is no subscriber on the state flow is because your launch hasn't had time to call collect before you reach the emit.
When you call launch, you're starting an asynchronous operation. The code inside the launch runs concurrently with the code that follows it. Since you're running this launch inside runBlocking's scope, it means it runs on the same thread as the rest of main's code, which means that the body of the launch cannot start running until the main function's body actually suspends. And here it only suspends when reaching emit().
Related
I am using Kotlin together with Arrow-Kt libraries.
I am launching on a specific scope some coroutines that make use of Arrow-kt's Schedule.
At a certain time, I want to be able to cancel all those coroutines that were launched on that scope, but after I cancel the scope basically nothing changes and whatever was running on the Schedule, continues to run, which is not what I wanted.
I already tried to place some yield() calls to force the coroutines to be cancellable, but the behavior didn't change.
Here is the code:
Main function doing the launches:
private val ballFetchingScope= CoroutineScope(CoroutineName("ball-fetching"))
fun fetchBalls(periodicity: Periodicity) {
val balls = stockRepository.getAllBalls() //basically a list of different balls
balls.forEach {
ballFetchingScope.launch(Dispatchers.IO) { ballFetcher.startFetching(it, periodicity) }
}
}
startFetching function, using Arrow-kt's schedule:
suspend fun startFetching(ball: Ball, periodicity: Periodicity) {
Schedule.forever<Unit>()
.and(Schedule.spaced(periodicity))
.repeat {
yield()
//ball fetching logic here
}
}
Expected behavior:
When calling ballFetchingScope.cancel() all coroutines are cancelled and all fetching stops.
Ideally not even needing to wait until it reaches the yield() call, if it is waiting for the next run to happen, I would like for it to cancel and not even start a new run of the repeat block.
What is actually happening:
Fetching continues to happen normally.
It's a bit hard to say here what is going on.
It's very strange that this is not working for you, since Schedule relies on KotlinX kotlin.coroutines.delay to execute the Schedule.spaced. So it should get cancelled while waiting for the next run.
It also checks coroutineContext.ensureActive() before running the function passed to repeat so it also automatically check in the place where you now manually placed yield.
Which version of Arrow are you using? And could you share a fully reproducible example?
I am answering my own post just to make sure you are not making the same dumb mistake I was making.
Everything was working as supposed, the issue was on my side.
I had two different instances of the class containing the ballFetchingScope
This means that I was calling the ballFetchingScope.cancel() on the scope for one of those instances while the Coroutines were running on the scope in the other instance.
Botom-line: Make sure you are not using multiple instances when you think you have only one.
Problem Description
My job is very async IO heavy so a lot of what we do is requesting a value and then listening for the response. Something like
connection.send(GetServices(...)))
connection.receive<GetServicesResponse>()
Now if I did this in Kotlin with suspend functions, I could get incorrect results because the message might have been received to quickly.
connection.send(GetServices(...)))
// Recevied GetServiceResponse Here <---------------
connection.receive<GetServicesResponse>()
// Times out because it never got to see the response
However, if I flip it now I will never send the request lol. So that just straight up won't work.
connection.receive<GetServicesResponse>() // Timeout because we never send the request
connection.send(GetServices(...)))
So then you go "OK I will just launch/async" but no matter which way you do that you still have this problem that you can never be sure the listener is actually listening. There is no way to tell that a suspend function is at the point it should be and is listening.
val response = async { connection.receive<GetServicesResponse>() }
connection.send(GetServices(...)))
response.await()
This can still fail because when we call async it doesn't guarantee the job has ran. So the job could still be scheduled to run by the time we receive the request.
launch { connection.send(GetServices(...))) }
connection.receive<GetServicesResponse>()
This can still fail because when we call launch it could run almost immediately if there are not a lot of jobs and multiple CPUs. Meanwhile, the thread running this code could get suspended by the OS. Kotlin Coroutines are nice but the OS can still stop any thread it feels like.
To fix this, I use UNDISPATCHED so that I guarantee that a launch runs until it suspends.
val response = async(start = CoroutineStart.UNDISPATCHED) {
connection.receive<GetServicesResponse>()
}
connection.send(GetServices(...)))
response.await()
This works but only in simple cases. If an engineer does something that causes a suspend before the listen then I am right back to same problem. The code above is both an example of working and not working code at the same time depending on the implementation of connection.receive. This gets worse when I start trying to use flows to receive data. Flow operations like merge or launchIn end up launching coroutines. So you can have coroutines launching coroutines so something like UNDISPATCHED doesn't appear to be sufficient. The only reason I know that is I actually tested it. Then again my code could be wrong.
Question
So the question is how do I guarantee listening? It seems like I can't with Kotlin Coroutines and flows?
Attempts
It seems like with RxJava I could, because I know when subscribe is called then it went up the entire chain. Once subscribe returns, that Observable is live. However, flows do not work like that in this regard. collect aka subscribe both suspends and eventually the flow starts listening so you have no way to know for sure.
I have thought of literally sending a "START" element on a flow to say it is live. However, you can get into the exact same situation.
flow {
emit("START")
emitAll(realFlow)
}
The OS can suspend my thread between the "START" and the emitAll(realFlow).
chanelFlow {
launch { realFlow.collect { send(it) } }
send("START")
}
Is right back to the same problem above. The job might not have run. So you launch undispatched.
chanelFlow {
launch(start = CoroutineStart.UNDISPATCHED) {
realFlow.collect { send(it) }
}
send("START")
}
But again, this is brittle. For all I know the realFlow has merges of it's own that are going to be scheduled and not executed. This has almost lead me back to using listeners. With a listener, I know I added them to the list of other listeners. No suspension. That seems like a huge step backwards and would make me wonder why I didn't just use RxJava.
If you got to the end. Thank you for reading my problem. I appreciate any attempt to help me.
I have been facing this problem several times as well. If the API is designed this way, there is unfortunately little you can do about it.
Just like you I ended up using UNDISPATCHED in simple cases where I knew one suspension point would be sufficient:
val response = async(start = CoroutineStart.UNDISPATCHED) {
connection.receive<GetServicesResponse>()
}
connection.send(GetServices(...)))
response.await()
When you need to make sure it can go through several suspension points, there is always the option of adding a small delay, but it's dirty, slow and doesn't 100% guarantee anything either:
val response = async {
connection.receive<GetServicesResponse>()
}
delay(50)
connection.send(GetServices(...)))
response.await()
But honestly the best is to have a better API. For instance, in my own STOMP library Krossbow, I have designed the subscribe() as suspend functions that return Flow. It might be unconventional but the API guarantees that after resuming the subscription has been made. The user can then send a request, and collect the Flow afterwards anyway without losing events.
I'm using coroutines in my multiplatform (android + iOS) application. To be able to use multithreaded coroutines, I'm using native-mt builds. So now I need to create a CoroutineDispatcher which runs on a hight priority thread. On android I'm doing it like this
val thread = HandlerThread("myThread", -20).also {
it.start()
}
val highPriorityDispatcher = Handler(thread.looper).asCoroutineDispatcher()
Is there a way to create a similar hight-priority dispatcher on iOS?
First what I tried is to create a new dispatcher using newSingleThreadContext. In k/n the resulting dispatcher exposes its Worker, but I have not found any ways to set worker priority.
Also I tried to raise thread priority with the setThreadPriority method:
val highPriorityDispatcher = newSingleThreadContext("myThread").also {
scope.launch(it){
NSThread.setThreadPriority(1.0)
}
}
But it did not seem to have the proper effect.
I'm thinking about writing a custom dispatcher using dispatch_queue, but it seems to be a non-trivial task. So any help to find an easier way to solve this problem would be appreciated.
Ok, actually it seems I was wrong.
val highPriorityDispatcher = newSingleThreadContext("myThread").also {
scope.launch(it){
NSThread.setThreadPriority(1.0)
}
}
does the job and actually changes the priority, confirmed in the iOS profiler.
At the end I also added
pthread_set_qos_class_self_np(QOS_CLASS_USER_INTERACTIVE, 0)
call to set the QoS. I used the pthread version, because here it says that
NSThread possesses a qualityOfService property, of type
NSQualityOfService. This class will not infer a QoS based on the
context of its execution, so the value of this property may only be
changed before the thread has started. Reading the qualityOfService of
a thread at any time provides its current value.
Is it possible to create a flow operator similar to collect with its own coroutine scope for each item it receives?
Lets say I have an updating list of bicycles, a Flow<List<Bicycle>>. For each bicycle I can request a price, unfortunately this API requires me to request the price for a single bicycle at a time. This is all fine I can spin up some requests using a couple of async builders, no problem so far. The API for requesting prices might be slow at times, potentially slower than the stream of new bicycles that becomes available or taken. This creates the issue of still working or waiting on fetching prices for bikes that are no longer available or taken. Is there any way to create a coroutine scope for each item collected from my Flow<List<Bicycle>> and then cancel it when a new item is collected, so that my current requests are only for the last emitted item (or bicycle) from the flow? If I understand correctly due to the sequential nature of flows, the next item is not requested until the previous is finished/completed?
If you want to make a request only for the latest collected item then you can use collectLatest terminal operator. As soon as it collects a new item, the current request will be cancelled.
The collectLatest official documentation states:
Terminal flow operator that collects the given flow with a provided
action. The crucial difference from collect is that when the original
flow emits a new value, action block for previous value is cancelled.
Here's the example exposed in the official documentation:
flow {
emit(1)
delay(50)
emit(2)
}.collectLatest { value ->
println("Collecting $value")
delay(100) // Emulate work
println("$value collected")
}
And it prints:
Collecting 1
Collecting 2
2 collected
You can find collectLatest official documentation here.
All the queries to the database are written in CouroutineScope. But this line of code in just inside of ViewModel class. And it's not blocking the UI thread. I don't understand how is it executing
private val nights = database.getAllNights()
You can see the whole repo in this link
The Dao function getAllNights() is returning a LiveData:
fun getAllNights(): LiveData<List<SleepNight>>
This return is done immediately upon calling that function, without waiting for the results to load from the database. The LiveData can then be observed to be notified when the data is loaded (on a background thread, asynchronously). It will also notify observers if the result of the query changes later on.
In contrast, if you take one of the calls where it's wrapped inside a launch and a switch to the IO dispatcher, such as getTonight():
fun getTonight(): SleepNight?
A Dao function like this will block the thread until it gets the result of the query, which is why it's important that you take care of going to a background thread before calling it.