If I have a List<A> and a function suspend (A) -> B, how can I apply this function on the list in parallel?
coroutineScope {
list.map {
async {
process(it)
}
} // List<Deferred<B>>
.awaitAll() // List<B>
}
suspend fun process(a: A): B {
...
}
This assumes you are already in a suspend context. Otherwise, you need to launch a new coroutine on the appropriate scope instead of using the coroutineScope scoping function.
You can create an extension function on CoroutineScope, go through each element of the list and launch a coroutine for each element. In this way elements of the list will be processed in parallel. Some code snippet:
fun CoroutineScope.processListInParallel(list: List<A>): List<Deferred<B>> = list.map {
async { // launch a coroutine
processA(it)
}
}
GlobalScope.launch {
val list = listOf(A("name1"), A("name2"), A("name3"))
val deferredList = processListInParallel(list)
val results: List<B> = deferredList.awaitAll() // wait for all items to be processed
}
suspend fun processA(a: A): B {
delay(1000) // emulate suspension
return B("Result ${a.name}")
}
data class A(val name: String) {}
data class B(val name: String) {}
Note: GlobalScope is used here as an example, using it is highly discouraged, application code usually should use an application-defined CoroutineScope.
Related
I have a "high-order" function that have to return some value. Inside that "high-order" function there is an "inside" function which really is a producer of the return value of a "high-order" function.
It is simpler to show it with an example what I mean:
lifecycle.coroutineScope.launch {
val result = doWork()
Log.d("Tag", "some result: ${result.someString}")
}
private val service = SomeService()
suspend fun doWork(): DoWorkResult {
fun onSomeString(someString: String): DoWorkResult {
//some execution
val returnResultForDoWork = DoWorkResult(someString)
//How to return 'returnResultForDoWork' from fun doWork
return returnResultForDoWork
}
service.getString { someString ->
onSomeString(someString)
}
}
class SomeService() {
suspend fun getString(
onResult: (String) -> Unit
) {
delay(1000)
onResult("work is done")
}
}
data class DoWorkResult(val someString: String)
flow execution:
call service.getString
call onSomeString(someString) when a someString is return from service.getString
in onSomeString analyse/handle a someString and return (how?) a DoWorkResult(someString) from doWork
My question is how to return a result of an onSomeString function as a result of a doWork function?
Suspend functions don't need higher order callbacks like that. Really, it's an antipattern, because it restores back "callback hell" that coroutines solve. A proper version of your function would look like:
class SomeService() {
suspend fun getString(): String {
delay(1000)
return "work is done"
}
}
And then your calling function becomes:
suspend fun doWork(): DoWorkResult {
val serviceReturnValue = getString()
//some execution
val returnResultForDoWork = DoWorkResult(serviceReturnValue)
return returnResultForDoWork
}
But let's suppose your service function is not a suspend function, but rather it is asynchronous with a callback, and you don't have control over the source code to make it a suspend function instead.
class SomeService() {
fun getString(
onResult: (String) -> Unit
) {
val handler = Handler(Looper.myLooper())
thread {
Thread.sleep(1000) //simulate some work
handler.post { onResult("work is done") }
}
}
}
Then to be able to return the callback's inner value in a suspend function, you need to convert the asynchronous function into a suspending one. This can be done with suspendCoroutine or suspendCancellableCoroutine. There are many examples you can look up on this site or online, but here's a quick sample. You can write it as an extension function to work like an overloaded version of the asynchronous function.
suspend fun SomeService.getString(): String = suspendCoroutine { continuation ->
getString { continuation.resume(it) }
}
Now you can call this proper suspending version of the function just as in my second code block above.
Honestly, I am not quite sure if I really understand what you try to do but...
is this what you looking for?
private val service = SomeService()
data class DoWorkResult(val someString: String)
suspend fun doWork(): DoWorkResult {
fun onSomeString(someString: String): DoWorkResult {
//some execution
val returnResultForDoWork = DoWorkResult(someString)
//How to return 'returnResultForDoWork' from fun doWork
return returnResultForDoWork
}
return service.getString { someString ->
onSomeString(someString)
}
}
class SomeService {
suspend fun getString(onResult: (String) -> DoWorkResult): DoWorkResult {
delay(1000)
val myStringFromNetworkOrSomething = "work is done"
return onResult(myStringFromNetworkOrSomething)
}
}
The following code are from the project architecture samples at https://github.com/android/architecture-samples
What is the lifetime kotlinx.coroutines.coroutineScope in Kotlin? Will this function saveTask return as soon as the given block and all its children coroutines are completed?
If I pass a ViewModel.viewModelScope to DefaultTasksRepository instead of kotlinx.coroutines.coroutineScope, what are differents ?
BTW, it seems that the Code A don't pass any object of CoroutineScope, why?
Code A
import kotlinx.coroutines.coroutineScope
...
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
...
override suspend fun saveTask(task: Task) {
coroutineScope {
launch { tasksRemoteDataSource.saveTask(task) }
launch { tasksLocalDataSource.saveTask(task) }
}
}
...
}
Code B
object ServiceLocator {
private val lock = Any()
private var database: ToDoDatabase? = null
...
private fun createTasksRepository(context: Context): TasksRepository {
val newRepo = DefaultTasksRepository(FakeTasksRemoteDataSource, createTaskLocalDataSource(context))
tasksRepository = newRepo
return newRepo
}
...
}
Added content
To Animesh Sahu: Thanks!
Are you sure that "A coroutineScope is a factory function that creates a CoroutineScope" , the following code is source code, it seems that the return value is not the object of CoroutineScope.
Source Code
public suspend fun <R> coroutineScope(block: suspend CoroutineScope.() -> R): R =
suspendCoroutineUninterceptedOrReturn { uCont ->
val coroutine = ScopeCoroutine(uCont.context, uCont)
coroutine.startUndispatchedOrReturn(coroutine, block)
}
A coroutineScope is a factory function that creates a CoroutineScope with the same context as it was called with but overriding the Job of that context.
CoroutineScope has lifetime until it is cancelled by calling cancel() on it or calling cancel() on CoroutineScope.coroutineContext or explicitly calling on the attached job coroutineContext[Job].cancel().
a coroutineScope is just a wrapper that creates immediate CoroutineScope that cancels itself up after executing its childrens.
PS: coroutineScope function is used for parallel decomposition of tasks with a new Job instance for control over its children
I have a class which uses an actor to ensure thread safety of shared mutable state. I made a small wrapper around this actor to make it easy to use:
interface Ref<T : Any> {
fun get(): T
fun transform(transformer: (T) -> T): Job
}
Here get uses runBlocking to block until it fetches the actual value of T:
override fun get(): T = runBlocking {
val deferred = CompletableDeferred<T>()
launch {
actor.send(RefOperation.Get(deferred))
}
deferred.await()
}
and transform does something similar without runBlocking and just returns a Job:
override fun transform(transformer: (T) -> T): Job {
val job = Job()
launch {
actor.send(RefOperation.Transform(transformer, job))
}
return job
}
This is fine until a transform call leads to another one:
ref.transform {
...
ref.transform {
}
}
Here I have 2 Jobs but there is no way to combine them to a single Job on which I can call join() if I want to wait for their completion.
The solution for this would be structured concurrency but then I don't know how to create my actor anymore since it is defined as an extension on CoroutineScope.
How can I keep using an actor while retaining the ability to use structured concurrency?
Note that I've created Ref because my project is multiplatform and for targets other than the JVM I use alternative implementations.
actor processes items in the same order as they were added, and does it sequentially in a single coroutine. It means that inner transform will be processed AFTER completion of outer transform, and you can't change it while you are using actor (in actor we can't launch more coroutines, because we confine our state to a single thread, otherwise recurrent processing order would be possible). Trying to join the job of inner transform in body of outer transform (if we mark transform as suspended function) will just cause deadlock.
Are you OK with such behaviour? If not, don't use either actors or nested transforms. If yes, please provide some use cases where creating a nested transform that will be processed after outer transform makes any sense.
As for joining all the jobs, I've got some code. In main we have outer transform that creates an inner transform. Outer one returns 2, inner one returns 8, but inner one starts after completion of outer one, so the result is 8. But as you wanted, transformJob.join() in main waits for inner job too.
private sealed class RefOperation<T>
private class Get<T : Any>(val deferred: CompletableDeferred<T>) : RefOperation<T>()
private class Transform<T : Any>(val transformer: TransformStub<T>.(T) -> T, val stub: TransformStub<T>, val job: CompletableJob) : RefOperation<T>()
interface Ref<T : Any> {
fun get(): T
fun transform(transformer: TransformStub<T>.(T) -> T): Job
}
interface TransformStub<T : Any> {
fun transform(transformer: TransformStub<T>.(T) -> T): Job
}
private class TransformStubImpl<T : Any>(
val actor: SendChannel<RefOperation<T>>,
val scope: CoroutineScope
) : TransformStub<T> {
override fun transform(transformer: TransformStub<T>.(T) -> T): Job {
return scope.launch {
val childJob: CompletableJob = Job()
val childStub = TransformStubImpl(actor, this)
actor.send(Transform(transformer, childStub, childJob))
childJob.join()
}
}
}
class RefImpl<T : Any>(initialValue: T) : Ref<T> {
private val actorJob = Job()
private val actorScope = CoroutineScope(actorJob)
private val actor = actorScope.actor<RefOperation<T>> {
var value: T = initialValue
for (msg in channel) {
when (msg) {
is Get -> {
println("Get! $value")
msg.deferred.complete(value)
}
is Transform -> {
with(msg) {
val newValue = stub.transformer(value)
println("Transform! $value -> $newValue")
value = newValue
job.complete()
}
}
}
}
}
override fun get(): T = runBlocking {
val deferred = CompletableDeferred<T>()
actor.send(Get(deferred))
deferred.await()
}
override fun transform(transformer: TransformStub<T>.(T) -> T): Job {
val stub = TransformStubImpl(actor, GlobalScope)
return stub.transform(transformer)
}
}
fun main() = runBlocking<Unit> {
val ref: Ref<Int> = RefImpl(0)
val transformJob = ref.transform {
transform { 8 }
2
}
transformJob.join()
ref.get()
}
The yield function in Kotlin states the following:
Yields a value to the Iterator being built and suspends until the next
value is requested.
In the following Kotlin code:
val sequence = sequence {
val start = 0
// yielding a single value
yield(start)
// yielding an iterable
yieldAll(1..5 step 2)
// yielding an infinite sequence
yieldAll(generateSequence(8) { it * 3 })
}
The yield statement is inside the sequence function. What I would like to do is have a Pause button that prevents further sequence items from being processed and a Resume button that when pressed allows the processing of the next item in the sequence. However if yield is inside of the sequence function, it isn't clear how a Pause/Resume flag can be used with this.
Sequences are just iterator suppliers, and iterators should be able to provide all elements without pauses:
for (elm in iterator) { }
// all elements must be created at this moment
// no additional elements allowed
Iterator can start sleeping during the next element providing. But, in this case, the caller thread (coroutine) will be blocked (suspended).
I assume you don't want to block threads, so I suggest creating a higher-order function that works like launch but returns Pauseable instead of Job:
fun CoroutineScope.launchPausable(block: suspend PauseableScope.() -> Unit): Pauseable =
PauseableCoroutine(this, block)
interface PauseableScope {
suspend fun pausePoint()
}
interface Pauseable {
fun pause()
fun resume()
val job: Job
}
class PauseableCoroutine(
scope: CoroutineScope,
block: suspend PauseableScope.() -> Unit
) : Pauseable, PauseableScope {
private var continuation: Continuation<Unit>? = null
private var isPaused = false
override val job = scope.launch { block() }
override fun pause() {
isPaused = true
}
override fun resume() {
syncIf({ isPaused }) {
isPaused = false
continuation?.resume(Unit)
continuation = null
}
}
override suspend fun pausePoint() {
suspendCoroutineUninterceptedOrReturn<Unit> { cont ->
syncIf({ isPaused }) {
continuation = cont
COROUTINE_SUSPENDED
} ?: Unit
}
}
}
Where syncIf is simple double-checked locking function:
inline fun <R> Any.syncIf(
crossinline condition: () -> Boolean,
crossinline block: () -> R
): R? {
if (condition())
synchronized(this) {
if (condition()) return block()
}
return null
}
Use case:
val pausable = GlobalScope.launchPausable {
items.forEach { item ->
doSomething(item)
// If it is paused coroutine will get suspended
pausePoint()
}
}
pausable.pause()
pausable.resume()
You can also put pausePoint() inside doSomething(item):
suspend fun PauseableScope.doSomething(item: Item) {
delay(100)
pausePoint()
delay(100)
}
I can't use "by lazy" because the callbacks require suspendCoroutine, which borks in android if it blocks the main thread, so I have to use the following "cache the result" pattern over and over. Is there a way to wrap it in a funButUseCachedResultsIfTheyAlreadyExist pattern to encapsulate the xCached object?
private var cameraDeviceCached: CameraDevice? = null
private suspend fun cameraDevice(): CameraDevice {
cameraDeviceCached?.also { return it }
return suspendCoroutine { cont: Continuation<CameraDevice> ->
... deep callbacks with cont.resume(camera) ...
}.also {
cameraDeviceCached = it
}
}
When what I'd really like to write is
private suspend fun cameraDevice(): CameraDevice = theMagicFunction { cont ->
... deep callbacks with cont.resume(camera) ...
}
You can build a generalized solution by wrapping an async call as follows:
import kotlinx.coroutines.*
import kotlinx.coroutines.CoroutineStart.LAZY
class LazySuspendFun<out T>(
scope: CoroutineScope,
private val block: suspend () -> T
) {
private val deferred = scope.async(Dispatchers.Unconfined, LAZY) { block() }
suspend operator fun invoke() = deferred.await()
}
fun <T> CoroutineScope.lazySuspendFun(block: suspend () -> T) =
LazySuspendFun(this, block)
This is a simple example of how you can use it. Note that we are able to compose them so that we use a lazy-inited value as a dependency to getting another one:
val fetchToken = lazySuspendFun<String> {
suspendCoroutine { continuation ->
Thread {
info { "Fetching token" }
sleep(3000)
info { "Got token" }
continuation.resume("hodda_")
}.start()
}
}
val fetchPosts = lazySuspendFun<List<String>> {
val token = fetchToken()
suspendCoroutine { continuation ->
Thread {
info { "Fetching posts" }
sleep(3000)
info { "Got posts" }
continuation.resume(listOf("${token}post1", "${token}post2"))
}
}
}
On the calling side you must be inside some coroutine context so you can call the suspending functions:
myScope.launch {
val posts = fetchPosts()
...
}
This solution is robust enough that you can concurrently request the value several times and the initializer will run only once.
I'll write this as an answer, since it's not possible to post much code in comments.
What you're looking for is something like this:
private suspend fun cameraDevice() = theMagicFunction {
CameraDevice()
}()
suspend fun theMagicFunction(block: ()->CameraDevice): () -> CameraDevice {
var cameraDeviceCached: CameraDevice? = null
return fun(): CameraDevice {
cameraDeviceCached?.also { return it }
return suspendCoroutine { cont: Continuation<CameraDevice> ->
cont.resume(block())
}.also {
cameraDeviceCached = it
}
}
}
Unfortunately, this will not compile, since closures cannot be suspendable, and neither are local functions.
Best I can suggest, unless I miss a solution there, is to encapsulate this in a class, if this variable bothers you too much.