Non-blocking way to lazy-compute a value - kotlin

I feel like I'm trying to reinvent the wheel, but I can't find what I should be using instead. Also, my wheel looks square-ish.
I want to get the result of some long-running operation (download/async API/compute/etc) in a thread-safe, cached way, that is:
if the long-running op has already completed, just return the cached value
if the long-running op is running, just wait for it to complete
otherwise start the long-running op and wait
I'd like all of the above waiting to be done in non-blocking suspend funcs. Am I missing some obvious doodad in the standard toolset?
I'd also like the cached value to be resettable (invalidate the result, so that the next get() starts over).
Just launching a coroutine will create a bunch of parallel long-running ops if called multiple times before it's done (fails case #2).
Both Flow and Deferred can be used to do that, I think, but both need a bunch of logic around them (like deciding when to start the op vs when to just wait).
So far I found this relatively simple way:
class CachedComputation<T>(private val compute: suspend () -> T) {
private var cache = GlobalScope.async(Dispatchers.Unconfined, start = CoroutineStart.LAZY) { compute() }
suspend fun get(): T {
return cache.await()
}
fun reset() {
cache = GlobalScope.async(Dispatchers.Unconfined, start = CoroutineStart.LAZY) { compute() }
}
}
It might need some synchronisation between get() and reset(). But the main problem with that is the usage of GlobalScope. Passing a Scope to get() is ugly and still starts a new coroutine, even though it's already a suspend func. I'd rather have it confined to the context of the caller.
I can solve it with a more complicated class:
class CachedComputation<T>(private val compute: suspend () -> T) {
private var future: CompletableDeferred<T>? = null
/**
* Return the result of computation once it's available.
* If the result is ready, return it immediately.
* If the result is being computed, wait for it to finish and return.
* If the computation is not running, start it and wait for result.
*/
suspend fun get(): T {
val (needToCompute, f) = newOrCurrentFuture()
return if (needToCompute) {
compute().also { f.complete(it) }
} else {
f.await()
}
}
/**
* If the computation hasn't started yet, create, save and return a new deferred result
* and indicate the need to start the computation.
* Otherwise return the current deferred result that can be awaited on, and indicate
* there's no need to start a new computation.
*/
#Synchronized
private fun newOrCurrentFuture(): Pair<Boolean, CompletableDeferred<T>> {
val currentFuture = future
return if (currentFuture != null)
Pair(false, currentFuture)
else
Pair(true, newFuture())
}
/**
* Create a new deferred result and save it in the class field
*/
private fun newFuture(): CompletableDeferred<T> {
return CompletableDeferred<T>()
.also { future = it }
}
#Synchronized
fun reset() {
future = null
}
}
But it seems unnecessarily complicated.

Related

How to implement timeout without interrupting a calculation in Kotlin coroutine?

Let's say a request started a long calculation, but ready to wait no longer than X seconds for the result. But instead of interrupting the calculation I would like it to continue in parallel until completion.
The first condition ("wait no longer than") is satisfied by withTimeoutOrNull function.
What is the standard (idiomatic Kotlin) way to continue the computation and execute some final action when the result is ready?
Example:
fun longCalculation(key: Int): String { /* 2..10 seconds */}
// ----------------------------
cache = Cache<Int, String>()
suspend fun getValue(key: Int) = coroutineScope {
val value: String? = softTimeoutOrNull(
timeout = 5.seconds(),
calculation = { longCalculation() },
finalAction = { v -> cache.put(key, v) }
)
// return calculated value or null
}
This is a niche enough case that I don't think there's a consensus on an idiomatic way to do it.
Since you want the work to continue in the background even if the current coroutine is resuming, you need a separate CoroutineScope to launch that background work, rather than using the coroutineScope builder that launches the coroutine as a child coroutine of the current one. That outer scope will determine the lifetime of the coroutines that it launches (cancelling them if it is cancelled). Typically, if you're using coroutines, you already have a scope on hand that's associated with the lifecycle of the current class.
I think this would do what you're describing. The current coroutine can wrap a Deferred.await() call in withTimeoutOrNull to see if it can wait for the other coroutine (that's not a child coroutine since it was launched directly from an outer CoroutineScope) without interfering with it.
suspend fun getValue(key: Int): String? {
val deferred = someScope.async {
longCalculation(key)
.also { cache.put(key, it) }
}
return withTimeoutOrNull(5000) { deferred.await() }
}
Here's a generalized version:
/**
* Launches a coroutine in the specified [scope] to perform the given [calculation]
* and [finalAction] with that calculation's result. Returns the result of the
* calculation if it is available within [timeout].
*/
suspend fun <T> softTimeoutOrNull(
scope: CoroutineScope,
timeout: Duration,
calculation: suspend () -> T,
finalAction: suspend (T) -> Unit = { }
): T? {
val deferred = scope.async {
calculation().also { finalAction(it) }
}
return withTimeoutOrNull(timeout) { deferred.await() }
}

Why can I cancel a Flow without either invoking yield or determining isActive() identification in Kotlin?

I have read the article.
There are two approaches to making computation code cancellable. The first one is to periodically invoke a suspending function that checks for cancellation. There is a yield function that is a good choice for that purpose. The other one is to explicitly check the cancellation status.
I know Flow is suspending functions.
I run Code B , and get Result B as I expected.
I think I can't making computation Code A cancellable, but in fact I can click "Stop" button to cancel Flow after I click "Start" button to emit Flow, why?
Code A
class HandleMeter: ViewModel() {
var currentInfo by mutableStateOf(2.0)
private var myJob: Job?=null
private fun soundDbFlow() = flow {
while (true) {
val data = (0..1000).random().toDouble()
emit(data)
}
}
fun calCurrentAsynNew() {
myJob?.cancel()
myJob = viewModelScope.launch(Dispatchers.IO) {
soundDbFlow().collect {currentInfo=it }
}
}
fun cancelJob(){
myJob?.cancel()
}
}
#Composable
fun Greeting(handleMeter: HandleMeter) {
var currentInfo = handleMeter.currentInfo
Column(
modifier = Modifier.fillMaxSize(),
) {
Text(text = "Current ${currentInfo}")
Button(
onClick = { handleMeter.calCurrentAsynNew() }
) {
Text("Start")
}
Button(
onClick = { handleMeter.cancelJob() }
) {
Text("Stop")
}
}
}
Code B
import kotlinx.coroutines.*
fun main() = runBlocking {
val job = launch(Dispatchers.IO) {
cal()
}
delay(1300L) // delay a bit
println("main: I'm tired of waiting!")
job.cancelAndJoin()
println("main: Now I can quit.")
}
suspend fun cal() {
val startTime = System.currentTimeMillis()
var nextPrintTime = startTime
var i = 0
while (i < 5) {
if ( System.currentTimeMillis() >= nextPrintTime) {
println("job: I'm sleeping ${i++} ...")
nextPrintTime += 500L
}
}
}
Result B
job: I'm sleeping 0 ...
job: I'm sleeping 1 ...
job: I'm sleeping 2 ...
main: I'm tired of waiting!
job: I'm sleeping 3 ...
job: I'm sleeping 4 ...
main: Now I can quit.
Add Content:
To Tenfour04: Thanks!
If the following content you said is true. I think Code C can be canceled when system finish the operation doBigBlockingCalculation() at one time, right? Why do I need Code D?
Since emit() is a suspend function, your Flow is able to interrupt and end the coroutine the next time the emit() function is called in that while loop.
Code C
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000).doBigBlockingCalculation()
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Code D
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000)
.chunked(100_000)
.flatMap {
it.doBigBlockingCalculation().also { yield() }
}
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
A Flow on its own is cold. Its a wrapper around some suspend functions that will run when collect() or some other terminal suspending function is called on the Flow.
In your Code A, when the Job is cancelled, it is cancelling the coroutine that called collect on the Flow. collect is a suspend function, so that cancellation will propagate down to the function you defined inside soundDbFlow(). Since emit() is a suspend function, your Flow is able to interrupt and end the coroutine the next time the emit() function is called in that while loop.
Here's an example for how you could use this knowledge:
Suppose your function had to do a very long calculation like this:
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000).doBigBlockingCalculation()
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Now if you tried to cancel this flow, it would work, but since the data line is a very slow operation that is not suspending, the Flow will still complete this very long calculation for no reason, eating up resources for longer than necessary.
To resolve this problem, you could break your calculation up into smaller pieces with yield() calls in between. Then the Flow can be cancelled more promptly.
private fun complicatedFlow() = flow {
while (true) {
val data = (0..1_000_000)
.chunked(100_000)
.flatMap {
it.doBigBlockingCalculation().also { yield() }
}
emit(data)
}
}.flowOn(Dispatchers.Default) // since the calculation is blocking
Not a perfect example. It's kind of wasteful to chunk a big IntRange. An IntRange takes barely any memory, but chunked turns it into Lists containing every value in the range.
It has to do with CoroutineScopes and children of coroutines.
When a parent coroutine is canceled, all its children are canceled as well.
More here:
https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html#children-of-a-coroutine

Design pattern to best implement batch api requests that happen transparently to the calling layer

I have a batch processor that I want to refactor to be expressed a 1-to-1 fashion based on input to increase readability, and for further optimization later on. The issue is that there is a service that should be called in batches to reduce HTTP overhead, so mixing the 1-to-1 code with the batch code is a bit tricky, and we may not want to call the service with every input. Results can be sent out eagerly one-by-one, but order must be maintained, so something like a flow doesn't seem to work.
So, ideally the batch processor would look something like this:
class Processor<A, B> {
val service: Service<A, B>
val scope: CoroutineScope
fun processBatch(input: List<A>) {
input.map {
Pair(it, scope.async { service.call(it) })
}.map {
(a, b) ->
runBlocking { b.await().let { /** handle result, do something with a if result is null, etc **/ } }
}
}
}
The desire is to perform all of the service logic in such a way that it is executing in the background, automatically splitting the inputs for the service into batches, executing them asynchronously, and somehow mapping the result of the batch call into the suspended call.
Here is a hacky implementation:
class Service<A, B> {
val inputContainer: MutableList<A>
val outputs: MutableList<B>
val runCalled = AtomicBoolean(false)
val batchSize: Int
suspended fun call(input: A): B? {
// some prefiltering logic that returns a null early
val index = inputContainer.size
inputContainer.add(a) // add to overall list for later batching
return suspend {
run()
outputs[index]
}
}
fun run() {
val batchOutputs = mutableListOf<Deferred<List<B?>>>()
if (!runCalled.getAndSet(true)) {
inputs.chunked(batchSize).forEach {
batchOutputs.add(scope.async { batchCall(it) })
}
runBlocking {
batchOutputs.map {
val res = result.await()
outputs.addAll(res)
}
}
}
}
suspended fun batchCall(input: List<A>): List<B?> {
// batch API call, etc
}
}
Something like this could work but there are several concerns:
All API calls go out at once. Ideally this would be batching and executing in the background while other inputs are being scheduled, but this is not .
Processing of the service result for the first input cannot resume until all results have been returned. Ideally we could process the result if the service call has returned, while other results continue to be performed in the background.
Containers of intermediate results seem hacky and prone to bugs. Cleanup logic is also needed, which introduces more hacky bits into the rest of the code
I can think of several optimizations to the address 1 and 2, but I imagine concerns related to 3 would be worse. This seems like a fairly common call pattern and I would expect there to be a library or much simpler design pattern to accomplish this, but I haven't been able to find anything. Any guidance is appreciated.
You're on the right track by using Deferred. The solution I would use is:
When the caller makes a request, create a CompletableDeferred
Using a channel, pass this CompletableDeferred to the service for later completion
Have the caller suspend until the service completes the CompletableDeferred
It might look something like this:
val requestChannel = Channel<Pair<Request, CompletableDeferred<Result>>()
suspend fun doRequest(request: Request): Result {
val result = CompletableDeferred<Result>()
requestChannel.send(Pair(request, result))
return result.await()
}
fun run() = scope.launch {
while(isActive) {
val (requests, deferreds) = getBatch(batchSize).unzip()
val results = batchCall(requests)
(results zip deferreds).forEach { (result, deferred) ->
deferred.complete(result)
}
}
}
suspend fun getBatch(batchSize: Int) = buildList {
repeat(batchSize) {
add(requestChannel.receive())
}
}

Why does a normal function need to be wrapped with viewModelScope.launch?

The following code is from the project.
1: In my mind,a suspend fun should be launched in another suspend fun or viewModelScope.launch{ }, withContext{ } ... , filterItems() is only a normal function, I don't know why filterItems() need to be wrapped with viewModelScope.launch{ } in the function filterTasks(), could you tell me ?
2: In the function filterTasks(), viewModelScope.launch{ } will launch in coroutines, it's asynchronous, I think return result maybe be launched before I get the result from viewModelScope.launch{}, so the result maybe null, is the code correct?
Code
private fun filterTasks(tasksResult: Result<List<Task>>): LiveData<List<Task>> {
val result = MutableLiveData<List<Task>>()
if (tasksResult is Success) {
isDataLoadingError.value = false
viewModelScope.launch {
result.value = filterItems(tasksResult.data, getSavedFilterType())
//return filterItems(tasksResult.data, getSavedFilterType()) //It will cause error.
}
} else {
result.value = emptyList()
showSnackbarMessage(R.string.loading_tasks_error)
isDataLoadingError.value = true
}
return result //I think it maybe be launched before I get the result from viewModelScope.launch{}
}
private fun filterItems(tasks: List<Task>, filteringType: TasksFilterType): List<Task> {
val tasksToShow = ArrayList<Task>()
// We filter the tasks based on the requestType
for (task in tasks) {
when (filteringType) {
ALL_TASKS -> tasksToShow.add(task)
ACTIVE_TASKS -> if (task.isActive) {
tasksToShow.add(task)
}
COMPLETED_TASKS -> if (task.isCompleted) {
tasksToShow.add(task)
}
}
}
return tasksToShow
}
It doesn't, unless it performs some heavy work and you want to move it to a background thread, which is the case here. Here the author just wanted to disjoint the work so the live data can be updated with an empty list first, and the filtered list later(computationally intensive to get), but forgot to do it out of the main thread.
In this particular case the author may have forgotten to add a background dispatcher as a parameter
viewModelScope.launch(Dispatchers.Default)
hence, in this scenario the intended behavior was not achieved, so you see this "nonsensical" coroutine.
I think you can contribute to the project with a fix :)
yes, you are right. but if you looked up the implementation of the launch {} such in lifecycleScope.launch {} or viewModelScope.launch {} you would find out the "block" which is "the coroutine code which will be invoked in the context of the provided scope" is cast to be suspend, so any block of code between launch {} is suspend code block. so in your example filterItems is cast to suspend under the hood and it's wrapped with viewModelScope.launch{ } to do its heavy task not in main thread.
public fun CoroutineScope.launch(
context: CoroutineContext = EmptyCoroutineContext,
start: CoroutineStart = CoroutineStart.DEFAULT,
// the below line is doing the magic
block: suspend CoroutineScope.() -> Unit
): Job {
val newContext = newCoroutineContext(context)
val coroutine = if (start.isLazy)
LazyStandaloneCoroutine(newContext, block) else
StandaloneCoroutine(newContext, active = true)
coroutine.start(start, coroutine, block)
return coroutine
}
I agree that the code looks suspicious, main reason is that it launches the filterItems coroutine into the Main dispatcher, basically just postponing the moment when filterItems will run on the GUI thread. If filterItems takes long to complete, it will block the GUI; if it doesn't take long, then why would you launch a concurrent coroutine in the first place?
Furthermore, on an architectural level, I don't see a reason why you'd have a function returning LiveData<List<Task>> when you can just have a suspend fun returning List<Task>.

How To await a function call?

So I have some asynchronous operations happening, I can create some lambada, call a function and pass that value to them. But what i want is not to have the result of the operation as a parameter, I want to return them.
As a example, I have a class A with some listeners, if there is a result all listeners are notified. So basically the asyncFunction should return a result if there is one otherwise be suspended.
object A {
val listeners = mutableListOf<(Int) -> Unit>()
fun onResult(value: Int) {
listeners.forEach { it(value) }
}
}
fun asyncFunction(): Deferred<Int> {
return async {
A.listeners.add({ result ->
})
return result
}
}
What I'm thinking right now (maybe I'm completely on the wrong track), is to have something like a Deferred, to which i can send the result and it returns. Is there something like that? Can I implement a Deffered myself?
class A {
private val awaiter: ??? // can this be a Deferred ?
fun onResult(result: Int) {
awaiter.putResult(result)
}
fun awaitResult(): Int {
return awaiter.await()
}
}
val a = A()
launch {
val result = a.awaitResult()
}
launch {
a.onResult(42)
}
So I do know that with callbacks this can be handled but it would be cleaner and easier to have it that way.
I hope there is a nice and clean solution im just missing.
Your asyncFunction should in fact be a suspendable function:
suspend fun suspendFunction(): Int =
suspendCoroutine { cont -> A.listeners.add { cont.resume(it) } }
Note that it returns the Int result and suspends until it's available.
However, this is just a fix for your immediate problem. It will still malfunction in many ways:
the listener's purpose is served as soon as it gets the first result, but it stays in the listener list forever, resulting in a memory leak
if the result arrived before you called suspendFunction, it will miss it and hang.
You can keep improving it manually (it's a good way to learn) or switch to a solid solution provided by the standard library. The library solution is CompletableDeferred:
object A {
val result = CompletableDeferred<Int>()
fun provideResult(r: Int) {
result.complete(r)
}
}
suspend fun suspendFunction(): Int = A.result.await()