How to wait for a flow to complete emitting the values - kotlin

I have a function "getUser" in my Repository which emits an object representing a user based on the provided id.
flow function
fun getUser(id: String) = callbackFlow {
val collectionReference: CollectionReference =
FirebaseFirestore.getInstance().collection(COLLECTION_USERS)
val query: Query = collectionReference.whereEqualTo(ID, id)
query.get().addOnSuccessListener {
val lst = it.toObjects(User::class.java)
if (lst.isEmpty())
offer(null)
else
offer(it.toObjects(User::class.java)[0])
}
awaitClose()
}
I need these values in another class. I loop over a list of ids and I add the collected user to a new list. How can I wait for the list to be completed when I collect the values, before calling return?
collector function
private fun computeAttendeesList(reminder: Reminder): ArrayList<User> {
val attendeesList = arrayListOf<User>()
for (friend in reminder.usersToShare) {
repoScope.launch {
Repository.getUser(friend).collect {
it?.let { user ->
if (!attendeesList.contains(user))
attendeesList.add(user)
}
}
}
}
return attendeesList
}
I do not want to use live data since this is not a UI-related class.

There are multiple problems to address in this code:
getUser() is meant to return a single User, but it currently returns a Flow<User>
which will never end, and never return more than one user.
the way the list of users is constructed from multiple concurrent query is not thread safe (because multiple launches are executed on the multi-threaded IO dispatcher, and they all update the same unsafe list directly)
the actual use case is to get a list of users from Firebase, but many queries for a single ID are used instead of a single query
Solution to #1
Let's tackle #1 first. Here is a version of getUser() that suspends for a single User instead of returning a Flow:
suspend fun getUser(id: String): User {
val collectionReference = FirebaseFirestore.getInstance().collection(COLLECTION_USERS)
val query = collectionReference.whereEqualTo(ID, id)
return query.get().await().let { it.toObjects(User::class.java) }.firstOrNull()
}
// use the kotlinx-coroutines-play-services library instead
private suspend fun <T> Task<T>.await(): T {
return suspendCancellableCoroutine { cont ->
addOnCompleteListener {
val e = exception
if (e == null) {
#Suppress("UNCHECKED_CAST")
if (isCanceled) cont.cancel() else cont.resume(result as T)
} else {
cont.resumeWithException(e)
}
}
}
}
It turns out that this await() function was already written (in a better way) and it's available in the kotlinx-coroutines-play-services library, so you don't need to actually write it yourself.
Solution to #2
If we could not rewrite the whole thing according to #3, we could deal with problem #2 this way:
private suspend fun computeAttendeesList(reminder: Reminder): List<User> {
return reminder.usersToShare
.map { friendId ->
repoScope.async { Repository.getUser(friendId) }
}
.map { it.await() }
.toList()
}
Solution to #3
Instead, we could directly query Firebase for the whole list:
suspend fun getUsers(ids: List<String>): List<User> {
val collectionReference = FirebaseFirestore.getInstance().collection(COLLECTION_USERS)
val query = collectionReference.whereIn(ID, ids)
return query.get().await().let { it.toObjects(User::class.java) }
}
And then consume it in a very basic way:
private suspend fun computeAttendeesList(reminder: Reminder): List<User> {
return Repository.getUsers(reminder.usersToShare)
}
Alternatively, you could make this function blocking (remove suspend) and wrap your call in runBlocking (if you really need to block the current thread).
Note that this solution didn't enforce any dispatcher, so if you want a particular scope or dispatcher, you can wrap one of the suspend function calls with withContext.

Related

How to efficiently perform concurrent computation with coroutines

I'm trying to improve my knowledge of coroutines and currently working on following problem:
Given a random non empty string with a length of 14 characters, what would be the most efficient way to find a string that contains a specific prefix (let's assume prefix length is 5)?
Most of the solutions I encountered on the internet either a) manually launch async{} 2 or 3 times or b) launch async{} in a loop and then await all of them to complete which won't work for this scenario.
One approach I tried was to launch new coroutines until I get a non null repsonse from the computation function and cancel the scope after, however there's a clear a performance issue that I'm not seeing since this approach can take more than 20s to calculate for a prefix with length 1.
...
private val _flow = MutableSharedFlow<String>()
suspend fun invoke(prefix: String) = withContext(dispatcher) { // dispatcher is Dispatchers.Default
_flow.onEach {
println("String is=$it")
this.cancel()
}.launchIn(this)
repeat(Int.MAX_VALUE) {
launch {
getString(prefix)?.let {
_flow.emit(it)
}
}
}
}
private fun getString(prefix: String): String? { // or any other cpu intensive task
val randomString = generateRandomStringAccordingToSpecs() // implemented elsewhere
if (randomString .startsWith(prefix = "prefix", ignoreCase = true)) {
return randomString
} else {
return null
}
}
I also tried an approach with a while loop and 4 parallel executions, for which I'm getting better performace results, however awaiting after every X calculations doesn't seem like the most efficient solution to me:
suspend fun invoke(prefix: String) = withContext(dispatcher) {
var resultString: String? = getString(prefix)
while (resultString == null) {
val tasks = listOf(
async { getString(prefix) },
async { getString(prefix) },
async { getString(prefix) },
async { getString(prefix) }
)
resultString = tasks.awaitAll().filterNotNull().firstOrNull()
}
println("String is=$resultString")
}
private fun getString(prefix: String): String? { // or any other cpu intensive task
val randomString = generateRandomStringAccordingToSpecs() // implemented elsewhere
if (randomString .startsWith(prefix = "prefix", ignoreCase = true)) {
return randomString
} else {
return null
}
}
In the example above I'm using a find suffix problem, but in general, what is the most efficient way to concurrently perform some CPU intensive calculations with coroutines?
Especially for the calculations where we don't know how many times the task must be executed before we get an answer.
This seems like a job for the select function. Assuming your generateRandomStringAccordingToSpecs() is a computationally blocking function, you want to have all your CPU cores working on the problem simultaneously and you just want the first valid result, you could build an operator like this:
suspend fun <T> getFirstResult(block: suspend CoroutineScope.() -> T): T =
withContext(Dispatchers.Default) {
coroutineScope {
select {
repeat(Runtime.getRuntime().availableProcessors()) {
async { block() }.onAwait {
coroutineContext.cancelChildren()
it
}
}
}
}
}
It starts as many parallel coroutines as there are CPUs, and once any of them returns a result, it cancels the rest and returns that result.
So you can use this with a coroutine block that uses a while loop indefinitely until a result is returned:
suspend fun invoke(prefix: String) = getFirstResult {
while(isActive) {
return#getFirstResult getString(prefix) ?: continue
}
}

How to invoke function based on condition of iterated value of Mono<List<String>> without using subscribe()?

I want to invoke a function that will notify the admin about some information missing, but I do not want to subscribe to this Mono, because I will subscribe to it later. The problem is I have some log which is called inside doOnSuccess() and when I use subscribe() and then build a response where I zip listOfWords value, the same log is logged twice and I do not want a code to behave that way.
Is there any way to retrieve that value in checkCondition() in a way that will not invoke doOnSuccess() or should I use some other function in merge() that can replace doOnSuccess()?
Should I use subscribe() only once on given Mono or is it allowed to use it multiple times?
Thank you in advance!
The functions are called in the presented order.
Code where log is called:
private fun merge(list1: Mono<List<String>>, list2: Mono<List<String>>) =
Flux.merge(
list1.flatMapMany { Flux.fromIterable(it) },
list2.flatMapMany { Flux.fromIterable(it) }
)
.collectList()
.doOnSuccess { LOG.debug("List of words: $it") }
Code where subscribe is called:
private fun checkCondition(
listOfWords: Mono<List<String>>,
) {
listOfWords.subscribe {
it.forEach { word ->
if (someCondition(word)) {
alarmSystem.notify("Something is missing for word {0}")
}
}
}
}
Code where response is built:
private fun buildResponse(
map: Mono<Map<String, String>>,
list1: List<SomeObject>,
listOfWords: Mono<List<String>>
): Mono<List<Answer>> {
val response = Mono.zip(map, Mono.just(list1), listOfWords)
.map { tuple ->
run {
val tupleMap = tuple.t1
val list = tuple.t2
val words = tuple.t3
list
.filter { someCondition(words) }
.map { obj -> NewObject(x,y) }
}
}

await inside await or how to get parent scope? Kotlin

I have a fucntion:
suspend fun getChats() {
val chatList = mutableListOf<Chat>()
getMyChats { chats ->
chats.forEach {
it.getDetail().await()
}
}.await()
}
But compiler show Suspension functions can be called only within coroutine body for await() which inside of forEach loop. How can I avoid this problem or how can I pass parent scope for it?
**getMyChats() receives a callback
According to you, the getMyChats doesn't support taking suspendable block (lambda).
So you can wrap it with a suspendCancellableCoroutine.
suspend fun getMyChatsSuspend(): List<Chat> = suspendCancellableCoroutine { cont ->
getMyChats { cont.resume(it) }
}
Now use your function like this:
suspend fun getChats() {
...
val chats = getMyChatsSuspend()
val chatDetails = chats.map{ chat.getDetail() }
val chatDetailsAwait = awaitAll( *chatDetails.toTypedArray() )
}
Obviously just chain the calls instead of creating multiple variables if you want
If you want everything to be done in single line you can do:
val resolvedDetails = getMyChatsSuspend().map{ chat.getDetail() }.let { awaitAll(*it.toTypedArray()) }
You have to isolate the getMyChats function like #Animesh Sahu said, but that last call to await() looks very suspicious so I'll rewrite it.
I'll also assume that await is not necessarily on a Deferred<T>.
suspend fun getChats() {
val chatList = mutableListOf<Chat>()
val result = CompletableDeferred<List<Chat>>()
getMyChats { result.complete(it) }.await()
val chats = result.await()
chats.forEach {
it.getDetail().await()
}
}
If you provide the function signatures of the functions involved I might be able give you a nicer solution.
Although without looking at anything else, I can tell you that the getMyChats function needs a refactor.

How To await a function call?

So I have some asynchronous operations happening, I can create some lambada, call a function and pass that value to them. But what i want is not to have the result of the operation as a parameter, I want to return them.
As a example, I have a class A with some listeners, if there is a result all listeners are notified. So basically the asyncFunction should return a result if there is one otherwise be suspended.
object A {
val listeners = mutableListOf<(Int) -> Unit>()
fun onResult(value: Int) {
listeners.forEach { it(value) }
}
}
fun asyncFunction(): Deferred<Int> {
return async {
A.listeners.add({ result ->
})
return result
}
}
What I'm thinking right now (maybe I'm completely on the wrong track), is to have something like a Deferred, to which i can send the result and it returns. Is there something like that? Can I implement a Deffered myself?
class A {
private val awaiter: ??? // can this be a Deferred ?
fun onResult(result: Int) {
awaiter.putResult(result)
}
fun awaitResult(): Int {
return awaiter.await()
}
}
val a = A()
launch {
val result = a.awaitResult()
}
launch {
a.onResult(42)
}
So I do know that with callbacks this can be handled but it would be cleaner and easier to have it that way.
I hope there is a nice and clean solution im just missing.
Your asyncFunction should in fact be a suspendable function:
suspend fun suspendFunction(): Int =
suspendCoroutine { cont -> A.listeners.add { cont.resume(it) } }
Note that it returns the Int result and suspends until it's available.
However, this is just a fix for your immediate problem. It will still malfunction in many ways:
the listener's purpose is served as soon as it gets the first result, but it stays in the listener list forever, resulting in a memory leak
if the result arrived before you called suspendFunction, it will miss it and hang.
You can keep improving it manually (it's a good way to learn) or switch to a solid solution provided by the standard library. The library solution is CompletableDeferred:
object A {
val result = CompletableDeferred<Int>()
fun provideResult(r: Int) {
result.complete(r)
}
}
suspend fun suspendFunction(): Int = A.result.await()

Kotlin coroutines and Java Completable future integration

Usually I'm using standard kotlin-jdk8 library to jump from Java *future API world into the Kotlin's suspend heaven.
And it worked great for me, until I encountered Neo4J cursor API, where I can't do .await() on the completion stage, because it immediately starts fetching millions of records into memory.
Kotlin way does not work for me, like this:
suspend fun query() {
driver.session().use { session ->
val cursor: StatementResultCursor = session.readTransactionAsync {
it.runAsync("query ...", params)
}.await() // HERE WE DIE WITH OOM
var record = cursor.nextAsync().await()
while (record != null) {
val node = record.get("node")
mySuspendProcessingFunction(node)
record = cursor.nextAsync().await()
}
}
}
At the same time, Java API works good, we fetch records one by one:
suspend fun query() {
session.readTransactionAsync { transaction ->
transaction.runAsync("query ...", params).thenCompose { cursor ->
cursor.forEachAsync { record ->
runBlocking { // BUT I NEED TO DO RUN BLOCKING HERE :(
val node = record.get("node")
mySuspendProcessingFunction(node)
}
}
}
}.thenCompose {
session.closeAsync()
}.await()
}
The second option works for me, but it is pretty ugly - definitely not Kotlin way, and what is more important, I need to use runBlocking (but these whole block is executed within suspend function)
What am I doing wrong? Is there a better way?
UPD
Tried to do this exercise using new Flow() feature, unfortunately results are the same:
suspend fun query() {
session.readTransactionAsync { transaction ->
transaction.runAsync(query, params).thenApply { cursor ->
cursor.asFlow().onEach { record ->
val node = record.get("node")
mySuspendProcessingFunction(node)
}
}
}.thenCompose {
session.closeAsync()
}.await()
}
fun StatementResultCursor.asFlow() = flow {
do {
val record = nextAsync().await()
if (record != null) emit(record)
} while (record != null)
}