Thread-safe access to the same variable from different flows (Kotlin) - kotlin

Is this code thread safe? Do I need a synchronized block or something like that? source1 and source2 endless Kotlin Flow
viewModelScope.launch {
var listAll = mutableListOf<String>()
var list1 = mutableListOf<String>()
var list2 = mutableListOf<String>()
launch {
source1.getNames().collect { list ->
list1 = list
listAll = mutableListOf()
listAll.addAll(list1)
listAll.addAll(list2)
//then consume listAll as StateFlow or return another flow with emit(listAll)
}
}
launch {
source2.getNames().collect { list ->
list2 = list
listAll = mutableListOf()
listAll.addAll(list2)
listAll.addAll(list1)
//then consume listAll as StateFlow or return another flow with emit(listAll)
}
}
}

This code is not thread safe.
However, it is called from viewModelScope.launch which runs on Dispatchers.Main by default. So your inner launch blocks will be called sequentially. This means that after all you will get the result which is produced by second launch block.
To achieve asynchronous behavior, you want to use viewModelScope.launch(Dispatchers.Default).
Your code will probably fire concurrent modification exception in that case.
To synchronize it, you may want to use Java's Collections.synchronizedList which blocks the list while one thread is performing operations with it, so the other thread are not able to perform modifications.
Or perform synchronizing manually using Mutex.
val mutex = Mutex()
viewModelScope.launch(Dispatchers.Default) {
launch {
mutex.withLock {
... // Your code
}
}
launch {
mutex.withLock {
... // Your code
}
}
}
Read official Kotlin guide to shared mutable state
After all, I am struggling to imagine real life example in which you will actually use that code. You probably don't need asynchronous behavior, you will be fine without using two launch blocks. Or you should rethink your design to avoid need of manual synchronization of two coroutines.

Related

Difference between GlobalScope and runBlocking when waiting for multiple async

I have a Kotlin Backend/server API using Ktor, and inside a certain endpoint's service logic I need to concurrently get details for a list of ids and then return it all to the client with the 200 response.
The way I wanted to do it is by using async{} and awaitAll()
However, I can't understand whether I should use runBlocking or GlobalScope.
What is really the difference here?
fun getDetails(): List<Detail> {
val fetched: MutableList<Details> = mutableListOf()
GlobalScope.launch { --> Option 1
runBlocking { ---> Option 2
Dispatchers.IO --> Option 3 (or any other dispatcher ..)
myIds.map { id ->
async {
val providerDetails = getDetails(id)
fetched += providerDetails
}
}.awaitAll()
}
return fetched
}
launch starts a coroutine that runs in parallel with your current code, so fetched would still be empty by the time your getDetails() function returns. The coroutine will continue running and mutating the List that you have passed out of the function while the code that retrieved the list already has the reference back and will be using it, so there's a pretty good chance of triggering a ConcurrentModificationException. Basically, this is not a viable solution at all.
runBlocking runs a coroutine while blocking the thread that called it. The coroutine will be completely finished before the return fetched line, so this will work if you are OK with blocking the calling thread.
Specifying a Dispatcher isn't an alternative to launch or runBlocking. It is an argument that you can add to either to determine the thread pool used for the coroutine and its children. Since you are doing IO and parallel work, you should probably be using runBlocking(Dispatchers.IO).
Your code can be simplified to avoid the extra, unnecessary mutable list:
fun getDetails(): List<Detail> = runBlocking(Dispatchers.IO) {
myIds.map { id ->
async {
getDetails(id)
}
}.awaitAll()
}
Note that this function will rethrow any exceptions thrown by getDetails().
If your project uses coroutines more generally, you probably have higher level coroutines running, in which case this should probably be a suspend function (non-blocking) instead:
suspend fun getDetails(): List<Detail> = withContext(Dispatchers.IO) {
myIds.map { id ->
async {
getDetails(id)
}
}.awaitAll()
}

How to pass Observable emissions to MutableSharedFlow?

well, I have an Observable, I’ve used asFlow() to convert it but doesn’t emit.
I’m trying to migrate from Rx and Channels to Flow, so I have this function
override fun processIntents(intents: Observable<Intent>) {
intents.asFlow().shareTo(intentsFlow).launchIn(this)
}
shareTo() is an extension function which does onEach { receiver.emit(it) }, processIntents exists in a base ViewModel, and intentsFlow is a MutableSharedFlow.
fun <T> Flow<T>.shareTo(receiver: MutableSharedFlow<T>): Flow<T> {
return onEach { receiver.emit(it) }
}
I want to pass emissions coming from the intents Observable to intentsFlow, but it doesn’t work at all and the unit test keeps failing.
#Test(timeout = 4000)
fun `WHEN processIntent() with Rx subject or Observable emissions THEN intentsFlow should receive them`() {
return runBlocking {
val actual = mutableListOf<TestNumbersIntent>()
val intentSubject = PublishSubject.create<TestNumbersIntent>()
val viewModel = FlowViewModel<TestNumbersIntent, TestNumbersViewState>(
dispatcher = Dispatchers.Unconfined,
initialViewState = TestNumbersViewState()
)
viewModel.processIntents(intentSubject)
intentSubject.onNext(OneIntent)
intentSubject.onNext(TwoIntent)
intentSubject.onNext(ThreeIntent)
viewModel.intentsFlow.take(3).toList(actual)
assertEquals(3, actual.size)
assertEquals(OneIntent, actual[0])
assertEquals(TwoIntent, actual[1])
assertEquals(ThreeIntent, actual[2])
}
}
test timed out after 4000 milliseconds
org.junit.runners.model.TestTimedOutException: test timed out after
4000 milliseconds
This works
val ps = PublishSubject.create<Int>()
val mf = MutableSharedFlow<Int>()
val pf = ps.asFlow()
.onEach {
mf.emit(it)
}
launch {
pf.take(3).collect()
}
launch {
mf.take(3).collect {
println("$it") // Prints 1 2 3
}
}
launch {
yield() // Without this we suspend indefinitely
ps.onNext(1)
ps.onNext(2)
ps.onNext(3)
}
We need the take(3)s to make sure our program terminates, because MutableSharedFlow and PublishSubject -> Flow collect indefinitely.
We need the yield because we're working with a single thread and we need to give the other coroutines an opportunity to start working.
Take 2
This is much better. Doesn't use take, and cleans up after itself.
After emitting the last item, calling onComplete on the PublishSubject terminates MutableSharedFlow collection. This is a convenience, so that when this code runs it terminates completely. It is not a requirement. You can arrange your Job termination however you like.
Your code never terminating is not related to the emissions never being collected by the MutableSharedFlow. These are separate concerns. The first is due to the fact that neither a flow created from a PublishSubject, nor a MutableSharedFlow, terminates on its own. The PublishSubject flow will terminate when onComplete is called. The MutableSharedFlow will terminate when the coroutine (specifically, its Job) collecting it terminates.
The Flow constructed by PublishSubject.asFlow() drops any emissions if, at the time of the emission, collection of the Flow hasn't suspended, waiting for emissions. This introduces a race condition between being ready to collect and code that calls PublishSubject.onNext().
This, I believe, is the reason why flow collection isn't picking up the onNext emissions in your code.
It's why a yield is required right after we launch the coroutine that collects from psf.
val ps = PublishSubject.create<Int>()
val msf = MutableSharedFlow<Int>()
val psf = ps.asFlow()
.onEach {
msf.emit(it)
}
val j1 = launch {
psf.collect()
}
yield() // Use this to allow psf.collect to catch up
val j2 = launch {
msf.collect {
println("$it") // Prints 1 2 3 4
}
}
launch {
ps.onNext(1)
ps.onNext(2)
ps.onNext(3)
ps.onNext(4)
ps.onComplete()
}
j1.invokeOnCompletion { j2.cancel() }
j2.join()

Run code in main thread when IO thread dispatch completes?

I'm working with livedata. I want to run some arbitrary code in IO and then once that has completed, run some arbitrary code in the Main thread.
In JavaScript, you can accomplish something like this by chaining promises together. I know Kotlin is different, but that's at least a framework I'm coming from that I understand.
I have a function that will sometimes be called from Main and sometimes from IO, but it requires no special IO features itself. From within class VM: ViewModel():
private val mState = MyState() // data class w/property `a`
val myLiveData<MyState> = MutableLiveData(mState)
fun setVal(a: MyVal) {
mState = mState.copy(a=a)
myLiveData.value = mState
}
fun buttonClickHandler(a: MyVal) {
setVal(a) // Can execute in Main
}
fun getValFromDb() {
viewModelScope.launch(Dispatchers.IO) {
val a: MyVal = fetchFromDb()
setVal(a) // Error! Cannot call setValue from background thread!
}
}
Seems to me the obvious way would be to execute val a = fetchFromDb() from IO and then pull setVal(a) out of that block and into Main.
Is there a way to accomplish this? I don't see a conceptual reason why this feature could not exist. Is there some idea like
doAsyncThatReturnsValue(Dispatchers.IO) { fetchFromDb()}
.then(previousBlockReturnVal, Dispatchers.Main) { doInMain() }
that could be run in a ViewModel?
Please substitute "coroutine" for "thread" wherever appropriate above. :)
Launch is fine. You just have to switch around the dispatchers and use withContext:
fun getValFromDb() {
// run this coroutine on main thread
viewModelScope.launch(Dispatchers.Main) {
// obtain result by running given block on IO thread
// suspends coroutine until it's ready (without blocking the main thread)
val a: MyVal = withContext(Dispatchers.IO){ fetchFromDb() }
// executed on main thread
setVal(a)
}
}

Project Reactor - subscribe on parallel scheduler doesn't work

I'm looking at examples and reading documentation and I've found some problems while trying to subscribe on Flux in a parallel manner.
I have a 3 functions, as below.
private val log = LoggerFactory.getLogger("main")
private val sequence = Flux.just(1, 2)
fun a() {
sequence.subscribeOn(Schedulers.parallel()).subscribe { log.info("*** {}", it) }
sequence.subscribe { log.info(">>> {}", it) }
}
fun b() {
sequence.subscribe { log.info(">>> {}", it) }
}
fun c() {
sequence.subscribeOn(Schedulers.parallel()).subscribe { log.info("*** {}", it) }
}
Now, when I run each method separately I have a proper output from functions a() and b(), but output from c() is empty. Is that to be expected, is it by design? If so, why is that happening?
Flux.just(...) captures value(s) and thus is optimized to execute immediately in the subscribing Thread.
When you use subscribeOn, you change that subscribing Thread from main to something else, making the just truly asynchronous.
In a(), without a subscribeOn that second just would block the main thread just enough that the test doesn't finish before the asynchronous alternative completes.
In c(), there is no such blocking of the main thread. As a consequence, the test terminates before the asynchronous just has had time to emit anything, and that is why you see no output.
To make that more visible, add a Thread.sleep(10) and you'll see some output.

How to inform a Flux that I have an item ready to publish?

I am trying to make a class that would take incoming user events, process them and then pass the result to whoever subscribed to it:
class EventProcessor
{
val flux: Flux<Result>
fun onUserEvent1(e : Event)
{
val result = process(e)
// Notify flux that I have a new result
}
fun onUserEvent2(e : Event)
{
val result = process(e)
// Notify flux that I have a new result
}
fun process(e : Event): Result
{
...
}
}
Then the client code can subscribe to EventProcessor::flux and get notified each time a user event has been successfully processed.
However, I do not know how to do this. I tried to construct the flux with the Flux::generate function like this:
class EventProcessor
{
private var sink: SynchronousSink<Result>? = null
val flux: Flux<Result> = Flux.generate{ sink = it }
fun onUserEvent1(e : Event)
{
val result = process(e)
sink?.next(result)
}
fun onUserEvent2(e : Event)
{
val result = process(e)
sink?.next(result)
}
....
}
But this does not work, since I am supposed to immediately call next on the SynchronousSink<Result> passed to me in Flux::generate. I cannot store the sink as in the example:
reactor.core.Exceptions$ErrorCallbackNotImplemented:
java.lang.IllegalStateException: The generator didn't call any of the
SynchronousSink method
I was also thinking about the Flux::merge and Flux::concat methods, but these are static and they create a new Flux. I just want to push things into the existing flux, such that whoever holds it, gets notified.
Based on my limited understanding of the reactive types, this is supposed to be a common use case. Yet I find it very difficult to actually implement it. This brings me to a suspicion that I am missing something crucial or that I am using the library in an odd way, in which it was not intended to be used. If this is the case, any advice is warmly welcome.