I'm writing an app using coroutines (code below is greatly simplified). Recently I've watched Coroutines in Practice talk and got a little confused. Turns out I don't know when to use a CoroutineScope's extension function and when to use a suspending function.
I have a mediator (Presenter/ViewModel/Controller/etc) that implements CoroutineScope:
class UiMediator : CoroutineScope {
private val lifecycleJob: Job = Job()
override val coroutineContext = lifecycleJob + CoroutineDispatchersProvider.MAIN
// cancel parent Job somewhere
fun getChannel() {
launch {
val channel = useCase.execute()
view.show(channel)
}
}
}
Business logic (Interactor/UseCase):
class UseCase {
suspend fun execute(): RssChannel = repository.getRssChannel()
}
And a repository:
class Repository {
suspend fun getRssChannel(): RssChannel {
// `getAllChannels` is a suspending fun that uses `withContext(IO)`
val channels = localStore.getAllChannels()
if (channels.isNotEmpty()) {
return channels[0]
}
// `fetchChannel` is a suspending fun that uses `suspendCancellableCoroutine`
// `saveChannel` is a suspending fun that uses `withContext(IO)`
return remoteStore.fetchChannel()
.also { localStore.saveChannel(it) }
}
}
So I have a few questions:
Should I declare Repository#getRssChannel as a CoroutineScope's extension function (because
it spawns new suspending functions: getAllChannels,
fetchChannel, saveChannel)? How can I use it in the UseCase then?
Should I just wrap a Repository#getRssChannel into a
coroutineScope function in order to make all spawned suspending
functions to be children of the latter?
Or maybe it's already fine and I should change nothing. When to
declare a function as a CoroutineScope's extension then?
A suspending function should return once it has completed its task, it executes something, possibly taking some time while not blocking the UI, and when it's done it returns.
A CoroutineScope extension function is for a fire-and-forget scenario, you call it, it spawns a coroutine and returns immediately, while the task continues to execute.
Answer to question 1:
No, you should not declare Repository#getRssChannel as an extension function of CoroutineScope, because you only invoke suspend functions but not start (launch/ async) new jobs. As #Francesc explained extension function of CoroutineScope should only start new jobs, but cannot return immediatly result and should not be declared as suspend by itself.
Answer to question 2:
No, you should not wrap Repository#getRssChannel into a CoroutineScope. Wrapping makes only sense if you start (launch/ async) new coroutines in this method. The new jobs would be children of the current job and the outer method will only return after all parallel jobs are finished. In your case you have sequential invocations of other suspending coroutines and there is no need of a new scope.
Answer to question 3:
Yes, you can stay with your code. If you would need the functionality of UiMediator#getChannel more then once, then this method would be a candidate of an extension function for CoroutineScope.
Related
Suppose I have a blocking function because of some third party library. Something along these lines:
fun useTheLibrary(arg: String): String {
val result = BlockingLibrary.doSomething(arg)
return result
}
Invocations to BlockingLibrary.doSomething should run on a separate ThreadPoolExecutor.
What's the proper way (assuming there is a way) of achieving this with kotlin?
Note: I've read this thread but seems pretty outdated
If the blocking code is blocking because of CPU use, you should use Dispatchers.Default. If it is network- or disk-bound, use Dispatchers.IO. You can make this into a suspending function and wrap the blocking call in withContext to allow this function to properly suspend when called from a coroutine:
suspend fun useTheLibrary(arg: String): String = withContext(Dispatchers.Default) {
BlockingLibrary.doSomething(arg)
}
If you need to use a specific ThreadPoolExecutor because of API requirements, you can use asCoroutineDispatcher().
val myDispatcher = myExecutor.asCoroutineDispatcher()
//...
suspend fun useTheLibrary(arg: String): String = withContext(myDispatcher) {
BlockingLibrary.doSomething(arg)
}
If your library contains a callback-based way to run the blocking code, you can convert it into a suspend function using suspendCoroutine() or suspendCancellableCoroutine(). In this case, you don't need to worry about executors or dispatchers, because it's handled by the library's own thread pool. Here's an example in the Retrofit library, where they convert their own callback-based API into a suspend function.
I am experimenting with coroutines and feel unsure about passing coroutineScope to plain Kotlin UseCase. Can such approach create memory leaks?
Suppose we are initialising our UseCase in VM and will try to pass viewModelScope:
class UploadUseCase(private val imagesPreparingForUploadUseCase: ImagesPreparingForUploadUseCase){
fun execute(coroutineScope: CoroutineScope, bitmap: Bitmap) {
coroutineScope.launch {
val resizedBitmap = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
}
Is it safe code? No difference if I would declare this exact code in VM instead?If no, that means I could pass coroutineScope as constructor argument....Now I initially thought that I should create my execute method in a following way:
fun CoroutineScope.execute(bitmap: Bitmap) {
launch {
val resizedBitmap = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
}
As far as I understand we use extension function in order for method to use parent coroutineScope. That means, I don't need to pass coroutineScope as argument and just change method to use extension function.
However, in my surprise VM cannot see this method available! Why this method is not available from VM to call?
This is marked as red in VM:
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
prepareDataForUploadingUseCase.execute(bitmap)
}
This is not marked red from VM:
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
prepareDataForUploadingUseCase.execute(viewModelScope, bitmap)
}
If my understanding is wrong, why would I use CoroutineScope as extension function instead of passing coroutineScope as function argument?
Passing it as a parameter vs using it as an extension function receiver is effectively the same in the end result. Extension function receivers are basically another parameter that you are passing to the function, just with rearranged syntax for convenience. So you can't use an extension function as a "cheat" to avoid passing a receiver.
But either way, I see it as kind of a clumsy design to have to provide a scope and then hiding the coroutine setup inside the function. This results in spreading coroutine scope manipulation across both sides of the function barrier. The function that calls this function has to be aware that some coroutine is going to get called on the scope it passes, but it doesn't know whether it needs to worry about how to handle cancellation and what it's allowed to do with the scope that it passed.
In my opinion, it would be cleaner to either do this:
suspend fun execute(bitmap: Bitmap) = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
so the calling function can launch the coroutine and handle the entire coroutine in one place. Or pass no coroutine scope, but have the execute function internally generate its own scope (that is dependent on lifecycleScope or viewModelScope if applicable), and handle its own cancellation behavior. Here's an example of creating a child scope of the lifecycle scope and adding it to some collection of jobs that you might want to cancel under certain circumstances.
fun execute(bitmap: Bitmap) {
lifecycleScope.launch {
bitmapScopes += coroutineScope(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
I am answering this specific question: "Why this method is not available from VM to call?"
The method is not available because it takes a receiver (CoroutineScope), but you already have an implicit receiver due to being inside a type declaration: UploadUseCase. Therefore, you cannot just call the second form of the method, because you would somehow have to specify two receivers.
Luckily, Kotlin provides an easy way to do exactly that, the with method.
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
with(prepareDataForUploadingUseCase) {
viewModelScope.execute(bitmap)
}
}
However, I would say that this is quite weird, and agree with #Marko Novakovic that you should remove this responsibility from UseCase.
You can pass CoroutineScope as a function parameter, no problem with that. However I would advise you to remove that responsibility from UseCase. Launch coroutines from ViewModel, Presenter etc.
Extension functions are to be called on the instance of extension type. You don't need to call launch {} and withContext inside same function. Do either. launch(Dispatchers.IO) {}.
Extension functions are not just to access parent scope, you can use them for whatever you need them for, you choose.
I need test() to return a player from my db. I know I can use a callback but how can I make this work with async await?
fun test(): Player {
launch(UI) {
val player = async(CommonPool) { MainActivity.database?.playerDao()!!.loadPlayer() }.await()
return player
}
}
Currently the error is return is not allowed here
In JavaScript for example I would make test async then await it's result from where it's called.
It is impossible to run a coroutine on a raw thread. At the very least you must turn an existing thread into one that spins a top-level event loop. You achieve this with a runBlocking call on the very top of the thread's call stack (i.e., inside its run() method).
On a GUI thread or any other kind of thread that runs an event loop, you need a matching Dispatcher that submits coroutines to this event loop. Kotlin already provides dispatchers for Swing, JavaFX, Android etc. In these cases you need to launch a coroutine from some existing GUI event handler, like this:
myScope.launch {
val player = test()
... use the player ...
}
myScope must be an object that implements CoroutineScope with something like this:
override val coroutineContext = Dispatchers.Main + SupervisorJob()
This will give you a way to cleanly cancel all the coroutines running within the same scope, by calling
coroutineContext[Job]!!.cancel()
My example uses the Main dispatcher, which resolves to the GUI thread when you import the Kotlin coroutines library matching your UI framework.
The test() function must become a suspend fun that temporarily switches the dispatcher to a thread pool for blocking operations. Here's how a basic example could look:
suspend fun test() = withContext(Dispatchers.IO) {
MainActivity.database?.playerDao()!!.loadPlayer()
}
Finally, note I don't mention async at all in this answer. Kotlin's async has a very specific purpose, it is not a general facility like in other languages. Its purpose is strictly parallel decomposition, where you decompose a single task into several concurrent subtasks.
Inside a coroutine I am doing a http-request with OkHttpClient. The request is done from a function that has the suspend keyword:
suspend fun doSomethingFromHttp(someParam:String): Something {
...
val response = HttpReader.get(url)
return unmarshalSomething(response)!!
}
I assume that the function can be suspended on entry since it has the suspend keyword, but will the coroutine also be suspended when doing the http-request? What about other kinds of blocking IO?
There's no automagic going on with Kotlin coroutines. If you call a blocking function like HttpReader.get(), the coroutine won't be suspended and instead the call will block. You can easily assure yourself that a given function won't cause the coroutine to suspend: if it's not a suspend function, it cannot possibly do it, whether or not it's called from a suspend function.
If you want to turn an existing blocking API into non-blocking, suspendable calls, you must submit the blocking calls to a threadpool. The easiest way to achieve it is as follows:
val response = withContext(Dispatchers.IO) { HttpReader.get(url) }
withContext is a suspend fun that will suspend the coroutine, submit the provided block to another coroutine dispatcher (here IO) and resume when that block is done and has come up with its result.
You can also easily instantiate your own ExecutorService and use it as a coroutine dispatcher:
val myPool = Executors.newCachedThreadPool().asCoroutineDispatcher()
Now you can write
val response = withContext(myPool) { HttpReader.get(url) }
This PR has example code for proper OkHttp coroutines support
https://github.com/square/okhttp/pull/4129/files
It uses the thread pools of OkHttp to do the work. The key bit of code is this generic library code.
suspend fun OkHttpClient.execute(request: Request): Response {
val call = this.newCall(request)
return call.await()
}
suspend fun Call.await(): Response {
return suspendCancellableCoroutine { cont ->
cont.invokeOnCancellation {
cancel()
}
enqueue(object : Callback {
override fun onFailure(call: Call, e: IOException) {
if (!cont.isCancelled) {
cont.resumeWithException(e)
}
}
override fun onResponse(call: Call, response: Response) {
if (!cont.isCancelled) {
cont.resume(response)
}
}
})
}
}
There are two types of IO libraries in JAVA world, using IO or NIO.
You can find more documentation at https://dzone.com/articles/high-concurrency-http-clients-on-the-jvm
The ones using NIO, can theoretically provide true nonblocking suspension unlike IO ones which only offload the task to a separate thread.
NIO uses some dispatcher threads in the JVM to handle the input output sockets using multiplexing (Reactor design pattern). The way it works is, we request the NIO/dispatchers to load/unload something and they return us some future reference. This code can be turned into coroutines easily.
For IO based libraries, coroutine implementation is not true non blocking. It actually blocks one of the threads just like in Java, however the general usage pattern is, to use Dispatcher.IO which is a threadpool for such blocking IO tasks.
Instead of using OkHttpClient, I would recommend using https://ktor.io/docs/client.html
I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}