Related
I am trying to understand the coroutineScope() suspend function in Kotlin and I'm having a hard time understanding the exact purpose of this function.
As per the kotlinlang docs,
This function is designed for parallel decomposition of work. When any
child coroutine in this scope fails, this scope fails and all the rest
of the children are cancelled (for a different behavior see
supervisorScope). This function returns as soon as the given block and
all its children coroutines are completed.
But I feel this behavior can be achieved by launching a child coroutine and calling join on it.
So for example
suspend fun other() {
coroutineScope {
launch { // some task }
async { // some task }
}
}
This can be written as (scope is a reference to the scope created by the parent coroutine)
suspend fun other(scope: CoroutineScope) {
scope.launch {
launch { // some task }
async { // some task }
}.join()
}
Is there any difference between these two approaches since it looks
like they will produce same result and also seem to work in the same fashion?
If not, is coroutineScope merely a way to reduce this
boilerplate code of passing scope from parent coroutine and
calling join on child coroutine?
TLDR
Using CoroutineScope as in the example adds boilerplate code, is more confusing, error-prone and may handle cases like errors and cancellations differently. coroutineScope() is generally preferred in such cases.
Full answer
These two patterns are conceptually different and are used in different cases. Coroutines are all about sequential code and structured concurrency. Sequential means we can write a traditional code that waits in-place, it doesn't use callbacks, etc. and at the same time we don't get a performance hit. Structured concurrency means concurrent tasks have their owners, tasks consists of smaller sub-tasks that are explicit to the framework.
By mixing both above together we get a very easy to use and error-proof concurrency model where in most cases we don't have to launch background jobs and then manage them manually, watch for errors, handle cancellations, etc. We simply fork into sub-tasks and then join them in-place - that's all.
In Kotlin this is represented by suspend functions. Suspend functions are always executed within some context, this context is passed everywhere implicitly and the coroutines framework provides utils to use this context easily. One of the most common patterns is to fork and then join and this is exactly what coroutineScope() does. It creates a scope for launching sub-tasks and we can't leave this scope until all children are successful. We don't have to pass the scope manually, we don't have to join, we don't have to pass errors from children to their siblings and to parent, we don't have to pass cancellations from the parent to children - this is all automatic.
Therefore, suspend functions and coroutineScope() should be the default way of writing concurrent code with coroutines. This approach is easy to write, easy to read and it is error-proof. We can't easily leak a background task, because coroutineScope() won't let us go anywhere. We can't mistakenly ignore errors from background tasks. Etc.
Of course, in some cases we can't use this pattern. Sometimes, we actually would like to only launch a long-running task and return immediately. Sometimes, we don't consider the caller to be the owner of the task. For example, we could have some kind of a service that manages its tasks and we only schedule these tasks, but the service itself owns them. For these cases we can use CoroutineScope.
By using the scope explicitly we can launch tasks in the different context than the current one or from outside of coroutine world. We generally have more control, but at the same time we partially opt-out of the code correctness guarantees I mentioned above. For example, if we forget to invoke join() we can easily leak background tasks or perform operations in unexpected order. Also, in your case if the coroutine invoking other() is cancelled, all launched operations will be still running in the background. For these reasons, we should use CoroutineScope explicitly only if needed.
Common patterns
As a result of all that was said above, when working with coroutines we usually use one of these patterns:
Suspend function - it runs within the caller context and it waits for all its subtasks, it doesn't launch anything in the background.
Function receiving CoroutineScope either as a param or receiver - usually, that means the function wants to do something with the context even after returning (because otherwise it could be simply a suspend function). It either launches some background tasks or stores the context somewhere for a later use.
Regular function that uses its own CoroutineScope to launch tasks. Usually, this is some kind of a service that keeps its custom context.
At least to me, function which is suspend and receives CoroutineScope is pretty confusing, it is not entirely clear what to expect from it. Will it execute the operation in the caller context or in the provided one? Will it wait to finish or only schedule the operation in the background and return immediately? Maybe it will do both: first do some initial processing synchronously (therefore suspend), but also schedule additional task in the background (therefore scope: CoroutineScope)? We don't know this, we have to read the documentation or source code to understand its behavior. Your second example is unnecessary complication over a simple suspend function.
To further make my point consider this example:
data class User(
val firstName: String,
val lastName: String,
) {
fun getFullName(user: User) = ...
}
This example is far from perfect, but the main point is that it is confusing why we have to pass user to getFullName() if we call this function on a user already. We don't know whether it returns a full name of the passed user, the user we invoked the function on or maybe some kind of a mix? If that would be a member function not receiving a User or a static utility function receiving a User, everything would be clear. But a member function receiving a User is simply confusing. This is similar to your second example where we pass the context both implicitly and explicitly and we don't know which one is used and how exactly.
Dart makes asynchronous programming extremely easy. All you need to do is surround the asynchronous code in an async method, and within it, use await before every call that is going to take a while.
I'm new to Kotlin, and asynchronous programming doesn't seem that simple here. (Probably because Dart is single-threaded.)
It'd be nice to get a rough outline of the differences both languages provide in their implementation of asynchronous code.
Apologize if I miss-stated any facts. Thanks in advance!
Dart makes asynchronous programming extremely easy. All you need to do is surround the asynchronous code in an async method, and within it, use await before every call that is going to take a while.
Yes (though async+await is not Dart's invention, it dates back to at least C# 5.0 in 2012, which then directly inspired JavaScript, Python, Julia, Kotlin, Swift, Rust, many others, and Dart).
I'm new to Kotlin, and asynchronous programming doesn't seem that simple here.
Kotlin 1.1 has async+await, although await is a postfix method, not an operator unlike in most other languages, but the end-result is the same.
It'd be nice to get a rough outline of the differences both languages provide in their implementation of asynchronous code.
Kotlin and Dart are different languages because they solve different problems, consequently there's simply too much to write about their differences, even when focused entirely on how they handle concurrency and coroutines.
...but in-short, the main difference (as far as you're concerned) is syntactical (which is as far as I can tell: Be aware that I am not a Dart/Flutter nor Kotlin expert, I just know how to read documentation and use Google)
I suggest seeing some simple examples in Kotlin, such as:
First-off, read the announcement where await was introduced to Kotlin 1.1: https://kotlinlang.org/docs/whatsnew11.html#coroutines-experimental
And seeing how it interops with Swift's async + await functions here: https://kotlinlang.org/docs/whatsnew1530.html#experimental-interoperability-with-swift-5-5-async-await (Swift's async features work the same way as Dart's, as far as I know, except without enforced thread isolation)
Kotlin Coroutines Async Await Sequence
This article (which I only skimmed) seems good too: https://www.raywenderlich.com/books/kotlin-coroutines-by-tutorials/v2.0/chapters/5-async-await
I'm new to Kotlin, and asynchronous programming doesn't seem that simple here.
In fact, Kotlin takes it to the next level of simplicity: it's almost invisible. For example:
suspend fun main() {
println("Hello")
delay(1000)
println("Hello again")
}
This code, unbeknownst to you, is actually implemented as asynchronous. But you just see simple, sequential code. The compiled code (in case of the JVM backend) has structure something like this:
public static void main(String[] args) {
System.out.println("Hello");
globalThreadPool.scheduleAfterDelay(() -> {
println("Hello again");
}, 1000, TimeUnit.MILLISECONDS);
}
On top of that, Kotlin makes it super-simple to adapt any async code you may have today so that you can use in the same native way as the above built-in delay function.
Where people trip up mostly is not this basic scenario, but dealing with more advanced topics like structured concurrency, choosing the right thread pool to run your code, error propagation, and so on.
I haven't studied Dart, but from what I know about the async-await pattern in other languages, whenever you call an async function, you have implicitly created a concurrent task, which is very easy to leak out -- all it takes is forgetting to await on it. Kotlin prevents these bad outcomes by design and forces you to address the concurrency you're creating head-on, instead of decyphering out-of-memory logs from production.
The most important difference, beside the syntax, is the multithreading model of these languages.
Check this article:
Dart supports multi-threading using Isolates. Right in the introduction to Isolates, it has been said that
isolates [are] independent workers that are similar to threads but don’t share memory, communicating only via messages.
While Kotlin (on JVM) uses Java threads under the hood, which have access to shared memory.
async/await in both languages is implemented roughly the same, using CPS (glorified callbacks). The important distinction, in Dart you have single threaded event loop dispatching these callbacks, while in Kotlin on JVM you can have multiple event dispatches working together and continuations (callbacks) running truly in parallel on different threads and sharing memory, with all the benefits and issues resulting from that.
Also, note, Kotlin aims to be a multiplatform language, so while on JVM it has multithreaded model, if you compile Kotlin program into JS backend, it would be single-threaded with event-loop, basically same as Dart.
P.S. Watch this video from Roman Elizarov (designer of coroutines in Kotlin), is has a good overview of coroutine usage and internals.
My question is rather theoretical.
I am quite new to kotlin (only passed the tutorial, didn't write any real code).
Reading through the language reference I find myself confused about the fact that "suspend" is a keyword, yet I can't find anything like "launch" in the list of keywords. That makes me think that there is some asymmetry - the "suspend" is a compiler feature, yet "launch" is a library function. Is my understanding correct? If so - wouldn't it have been better to implement both as library features or both as compiler features?
I always thought that you can always write your own standard library using the available language features, but I still can't see if this really applies to this case.
TL;DR: Can I start a coroutine using pure kotlin, without importing any libraries whatsoever (however ugly that would be)?
The suspend marker adds a hidden continuation parameter to the function signature and completely changes the implementation bytecode. Suspension points don't boil down to helper function calls, they turn your linear program code into a state machine, the state being kept in the continuation object. The resulting bytecode isn't even representable as Java program code.
As opposed to that, launch is just regular library code that builds upon the suspend/resume primitive.
#Alexey Soshin's isn't quite correct.
You can use coroutines w/o the library, and it's pretty easy. Here is a about the simplest suspending coroutine example that has 0 dependency on the coroutine library.
import kotlin.coroutines.*
fun main() {
lateinit var context: Continuation<Unit>
suspend {
val extra="extra"
println("before suspend $extra")
suspendCoroutine<Unit> { context = it }
println("after suspend $extra")
}.startCoroutine(
object : Continuation<Unit> {
override val context: CoroutineContext = EmptyCoroutineContext
// called when a coroutine ends. do nothing.
override fun resumeWith(result: Result<Unit>) {
result.onFailure { ex : Throwable -> throw ex }
}
}
)
println("kick it")
context.resume(Unit)
}
This runs fine on the play.kotlinlang.org site.
As you can see from this code, any lambda decorated with suspend has the startCourtine() on it.
In fact, I think the SequenceBuilder() from the standard collection classes uses a simple coroutine like this to generate the sequence, with no dependency on the coroutine library.
The compiler is doing the heavy lifting on the coroutines, splitting the code into different "methods" at each possible suspending point. Look at the java code for this, and you'll see it's "split" into a switch statement. one case before the suspend, and another after.
The library does a ton of nice stuff for you..... and it's likely you'll almost always use it (cuz why not?) but you don't actually need it.
Can I start a coroutine using pure kotlin, without importing any libraries whatsoever (however ugly that would be)?
No. All coroutine generators are inside kotlinx.coroutines library, so you'll need at least that. Now, very theoretically, you could reimplement this functionality yourself. But probably you shouldn't.
How this can be done is a bit too long for a StackOverflow answer, but try invoking method of this Kotlin class from Java:
class AsyncWorks {
suspend fun print() {
println("Hello")
}
}
You'll see that although Kotlin method has no arguments, in Java it requires Continuation<? super Unit>. This is what suspend keyword does. It adds Continuation<T> as the last argument of our function.
wouldn't it have been better to implement both as library features or
both as compiler features?
Ideally, you'd want everything to be a "library feature", since it's easier to evolve. Removing a keyword from a language is very hard. In theory, having suspend as a keyword could be avoided. Quasar, being a framework, uses annotations instead. Go programming language, on the other hand, assumes all functions are suspendable. All those approaches have their advantages and disadvantages.
Kotlin decided to be pragmatic, and add suspend keyword, leaving the decision on the developers. If you're interested in the topic, I highly recommend this talk by Roman Elizarov, author of Kotlin coroutines, that explains their decissions: https://www.youtube.com/watch?v=Mj5P47F6nJg
Answering my own question here.
After a year of Kotlin I tend to think that this IS indeed possible.
The suspend language feature creates an extra class and instantiates it every time your suspend function is called. This class extends ContinuationImpl and stores the progress of your coroutine - to which point it was able to execute so far.
Therefore one will need to write a custom dispatcher that would be able to manage the queue of the continuation objects to decide which one has to run now and a launch function that would take the newly created continuation object and pass it over to the dispatcher.
Now, this is still an asymmetry - the ContinuationImpl lives in kotlin.coroutines.jvm.internal so the compiler assumes this package exists. If one really wants to drop the standard library altogether - he'll need to implement that package to be able use the suspend keyword.
I'm not a kotlin expert though, so I might be wrong.
Because coroutines are valid for use cases that don't support launch. Because suspend requires some specific support from the compiler and launch doesn't if you already have suspend. Because structured concurrency is a library framework on top of the language feature, and launch is a part of that specific framework, that makes specific choices on top of what the language requires.
Starting a coroutine without any libraries can be done with startCoroutine. kotlin.coroutines is part of Kotlin, not a library.
After reading the introduction and the javadoc of CoroutineScope I'm still a little confused what the idea behind a CoroutineScope is.
The first sentence of the doc "Defines a scope for new coroutines." is not clear to me: Why do my coroutines need a scope?
Also, why are standalone coroutine builders deprecated? Why is it better to do this:
fun CoroutineScope.produceSquares(): ReceiveChannel<Int> = produce {
for (x in 1..5) send(x * x)
}
instead of
fun produceSquares(): ReceiveChannel<Int> = produce { //no longer an extension function
for (x in 1..5) send(x * x)
}
You can still use global "standalone" coroutines by spawning them in GlobalScope:
GlobalScope.launch {
println("I'm running unstructured")
}
However, it's not recommended to do this since creating coroutines on a global scope is basically the same we did with good old threads. You create them but somehow need to keep track of a reference to later join/cancel them.
Using structured concurrency, that is nesting coroutines in their scopes, you will have a more maintainable system overall. For example, if you spawn a coroutine inside another one, you inherit the outer scope. This has multiple advantages. If you cancel the outer coroutine, the cancellation will be delegated to its inner coroutines. Also, you can be sure that the outer coroutine will not complete before all its children coroutines have done their work.
There's also a very good example shown in the documentation for CoroutineScope.
CoroutineScope should be implemented on entities with well-defined lifecycle that are responsible for launching children coroutines. Example of such entity on Android is Activity.
After all, the first version of your shown produceSquares methods is better as it is only executable if invoked in a CoroutineScope. That means you can run it inside any other coroutine:
launch {
produceSquares()
}
The coroutine created inside produceSquares inherits the scope of launch. You can be sure that launch does not complete before produceSquares. Also, if you cancelled launch, this would also effect produceSquares.
Furthermore, you can still create a globally running coroutine like this:
GlobalScope.produceSquares()
But, as mentioned, that's not the best option in most cases.
I'd also like to promote an article I wrote. There are some examples demonstrating what scopes mean: https://kotlinexpertise.com/kotlin-coroutines-concurrency/
It is related to the concept of structured concurrency, which defines a structure between coroutines.
On a more philosophical level, you rarely launch coroutines “globally”, like you do with threads. Coroutines are always related to some local scope in your application, which is an entity with a limited life-time, like a UI element. So, with structured concurrency we now require that launch is invoked in a CoroutineScope, which is an interface implemented by your life-time limited objects (like UI elements or their corresponding view models).
As an evident consequence of this concept: by cancelling the context of a scope, all it's subcoroutines will be canceled, too.
Why would I want to use Kotlin's coroutines?
It seems that the RxKotlin library is much more versatile.
Kotlin's coroutines look significantly less powerful and more cumbersome to use in comparison.
I base my opinion on coroutines on this design talk by Andrey Breslav (JetBrains)
Slideshow from the talk is accessible here.
EDIT (thanks to #hotkey):
Better source on the current state of coroutines here.
Disclaimer: Parts of this answer are irrelevant since Coroutines now have the flow API, very similar to Rx one. If you want an up-to-date answer, jump to the last edit.
There is two parts in Rx; the Observable pattern, and a solid set of operators to manipulate, transform and combine them. The Observable pattern, by itself, doesn't do a lot. Same with Coroutines; it's just another paradigm to deal with asynchronism. You can compare the pro/cons of callbacks, Observable and coroutines to solve a given problem, but you can't compare a paradigm with a fully featured library. It's like comparing a language with a framework.
How Kotlin coroutines are better than RxKotlin ? Didn't used coroutines yet, but it's look similar to async/wait in C#. You just write sequential code, everything is as easy as writing synchronous code ... except it execute asynchronously. It's easier to grasp.
Why would I want to use kotlin coroutines ? I will answer for myself. Most of the time I will stick to Rx, because I favor event-driven architecture. But should arise the situation where I am writing sequential code, and I need to call an asynchronous method in the middle, I will happily leverage coroutines to keep it that way and avoiding wrapping everything in Observable.
Edit: Now that I am using coroutines it's time for an update.
RxKotlin is just syntactic sugar to use RxJava in Kotlin, so I will speak about RxJava and not RxKotlin in the following. Coroutines are a lower lever and more general concept than RxJava, they serve others use-cases. That said, there is one use-case where you could compare RxJava and coroutines (channel), it's passing around data asynchronously. Coroutines have a clear advantage over RxJava here:
Coroutines are better to deal with resources
In RxJava you can assign computations to schedulers but subscribeOn() and ObserveOn()are confusing. Every coroutine is given a thread context and return to parent context. For a channel, both side (producer, consumer) execute on his own context. Coroutines are more intuitive on thread or thread pool affectation.
Coroutines give more control on when those computation occur. You can for example pass hand (yield), prioritize (select), parallelize (multiple producer/actor on channel) or lock resource (Mutex) for a given computation. It may not matter on server (where RxJava came first) but on resources limited environment this level of control may be required.
Due to it's reactive nature, backpressure doesn't fit well in RxJava. In the other end send() to channel is a suspensive function that suspend when channel capacity is reached. It's out-of-the-box backpressure given by nature. You could also offer() to channel, in which case the call never suspend but return false in case the channel is full, effectively reproducing onBackpressureDrop() from RxJava. Or you could just write your own custom backpressure logic, which won't be difficult with coroutines, especially compared to do the same with RxJava.
There is another use-case, where coroutines shine and this will answer your second question "Why would I want to use Kotlin coroutines?". Coroutines are the perfect replacement for background threads or AsyncTask (Android). It's as easy as launch { someBlockingFunction() }. Of course you could achieve this with RxJava too, using Schedulers and Completable perhaps. You won't (or little) use the Observer pattern and the operators which are the signature of RxJava, a hint that this work is out of scope for RxJava. RxJava complexity (a useless tax here) will make your code more verbose and less clean than Coroutine's version.
Readability matters. On this regard, RxJava and coroutines approach differ a lot. Coroutines are simpler than RxJava. If you are not at ease with map(), flatmap() and functional reactive programming in general, coroutines manipulations are easier, involving basics instructions: for, if, try/catch ... But I personally find coroutine's code harder to understand for non-trivial tasks. Especially it involves more nesting and indentation whereas operator chaining in RxJava keep everything in line. Functional-style programming make processing more explicit. On top of that RxJava can solve complex transformations with a few standard operators from their rich (OK, way too rich) operator set. RxJava shine when you have complex data flows requiring a lot of combinations and transformations.
I hope those considerations will help you choose the right tool given your needs.
Edit: Coroutine now have flow, an API very, very similar to Rx. One could compare pro/cons of each, but the truth is the differences are minor.
Coroutines as it's core is a concurrency design pattern, with add-on libraries, one of those being a stream API similar to Rx. Obviously, Coroutines having a far broader scope than Rx, there is a lot of things that Coroutines can that Rx can't, and I can't list them all. But usually if I use Coroutines in one of my project it boil down to one reason:
Coroutines are better at removing callback from your code
I avoid using callback wich harm readability too much. Coroutines make asynchronous code simple and easy to write. By leveraging the suspend keyword, your code look like synchronous one.
I have seen Rx used in project mostly for the same purpose of replacing callback, but if you don't plan to modify your architecture to commit to the reactive pattern, Rx will be a burden. Consider this interface:
interface Foo {
fun bar(callback: Callback)
}
The Coroutine equivalent is more explicit, with a return type and the keyword suspend indicating it's an asynchronous operation.
interface Foo {
suspend fun bar: Result
}
But there is a problem with the Rx equivalent:
interface Foo {
fun bar: Single<Result>
}
When you call bar() in the callback or Coroutine version, you trigger the computation; with the Rx version, you get a representation of a computation that you can trigger at will. You need to call bar() then subscribing to the Single. Usually not a big deal, but it's a little confusing for beginner and can lead to subtle problem.
One exemple of such problems, suppose the callback bar function is implemented as such:
fun bar(callback: Callback) {
setCallback(callback)
refreshData()
}
If you don't port it properly, you will end with a Single that can be triggered only once because refreshData() is called in bar() function and not at subscription time. A beginner mistake, granted, but the thing is Rx is way more than a callback replacement and a lot of developers struggle to grasp Rx.
If your objective is to transform an asynchronous task from callback to a nicer paradigm, Coroutines are a perfect fit whereas Rx add some complexity.
I know RxJava very well and I've recently switched to Kotlin Coroutines and Flow.
RxKotlin is basically the same as RxJava, it just adds some syntactic sugar to make it more comfortable / idiomatic writing RxJava code in Kotlin.
A "fair" comparison between RxJava and Kotlin Coroutines should include Flow in the mix and I'm gonna try to explain why here. This is gonna be a bit long but I'll try to keep it as simple as I can with examples.
With RxJava you have different objects (since version 2):
// 0-n events without backpressure management
fun observeEventsA(): Observable<String>
// 0-n events with explicit backpressure management
fun observeEventsB(): Flowable<String>
// exactly 1 event
fun encrypt(original: String): Single<String>
// 0-1 events
fun cached(key: String): Maybe<MyData>
// just completes with no specific results
fun syncPending(): Completable
In kotlin coroutines + flow you do not need many entities cause if you do not have a stream of events you can just use simple coroutines (suspending functions):
// 0-n events, the backpressure is automatically taken care off
fun observeEvents(): Flow<String>
// exactly 1 event
suspend fun encrypt(original: String): String
// 0-1 events
suspend fun cached(key: String): MyData?
// just completes with no specific results
suspend fun syncPending()
Bonus: Kotlin Flow / Coroutines support null values (support removed with RxJava 2)
Suspending functions are what the name hint: they are function that can pause the execution of the code and resume it later when the function is completed; this allow you to write code that feels more natural.
What about the operators?
With RxJava you have so many operators (map, filter, flatMap, switchMap, ...), and for most of them there's a version for each entity type (Single.map(), Observable.map(), ...).
Kotlin Coroutines + Flow do not need that many operators, let's see why with some example on the most common operators
map()
RxJava:
fun getPerson(id: String): Single<Person>
fun observePersons(): Observable<Person>
fun getPersonName(id: String): Single<String> {
return getPerson(id)
.map { it.firstName }
}
fun observePersonsNames(): Observable<String> {
return observePersons()
.map { it.firstName }
}
Kotlin coroutines + Flow
suspend fun getPerson(id: String): Person
fun observePersons(): Flow<Person>
suspend fun getPersonName(id: String): String? {
return getPerson(id).firstName
}
fun observePersonsNames(): Flow<String> {
return observePersons()
.map { it.firstName }
}
You do not need an operator for the "single" case and it is fairly similar for the Flow case.
flatMap()
The flatMap operator and his siblings switchMap, contactMap exists to allow you to combine different RxJava object and thus execute potentially asynchronous code while mapping your events.
Say you need, for each person, to grab from a database (or remote service) it's insurance
RxJava
fun fetchInsurance(insuranceId: String): Single<Insurance>
fun getPersonInsurance(id: String): Single<Insurance> {
return getPerson(id)
.flatMap { person ->
fetchInsurance(person.insuranceId)
}
}
fun observePersonsInsurances(): Observable<Insurance> {
return observePersons()
.flatMap { person ->
fetchInsurance(person.insuranceId) // this is a Single
.toObservable() // flatMap expect an Observable
}
}
Let's see with Kotlin Coroutiens + Flow
suspend fun fetchInsurance(insuranceId: String): Insurance
suspend fun getPersonInsurance(id: String): Insurance {
val person = getPerson(id)
return fetchInsurance(person.insuranceId)
}
fun observePersonsInsurances(): Flow<Insurance> {
return observePersons()
.map { person ->
fetchInsurance(person.insuranceId)
}
}
Like before, with the simple coroutine case we do not need operators, we just write the code like we would if it wasn't async, just using suspending functions.
And with Flow that is NOT a typo, there's no need for a flatMap operator, we can just use map. And the reason is that map lambda is a suspending function! We can execute suspending code in it!!!
We don't need another operator just for that.
I cheated a bit here
Rx flatMap, switchMap and concatMap behave slightly differently.
Rx flatMap generate a new stream for each event and than merge them all together: the order of the new streams events you receive in the output is undetermined, it might not match the order or the events in input
Rx concatMap "fixes" that and guarantee you will get each new stream in the same order of your input events
Rx switchMap will instead dispose of any previously running stream when it gets a new events, only the last input received matter with this operator
So you see, it isn't true that Flow.map is the same, it is actually more similar to Rx concatMap, which is the more natural behavior you expect from a map operator.
But it is true you need less operators, inside map you can do any async operation you want and reproduce the behavior of flatMap because it is a suspendable function. The actual equivalent operator to RxJava flatMap is Flow.flatMapMerge operator.
The equivalent of the RxJava switchMap can be achieved in Flow by using the conflate() operator before the map operator.
For more complex stuff you can use the Flow transform() operator which for every event emit a Flow of your choice.
Every Flow operator accept a suspending function!
In the previous paragraph I told you I cheated. But the key get away of what I meant by Flow do not need as many operators is that most operator's callbacks are suspending function.
So say you need to filter() but your filter need to perform a network call to know if you should keep the value or not, with RxJava you need to combine multiple operators with unreadable code, with Flow you can just use filter()!
fun observePersonsWithValidInsurance(): Flow<Person> {
return observerPersons()
.filter { person ->
val insurance = fetchInsurance(person.insuranceId) // suspending call
insurance.isValid()
}
}
delay(), startWith(), concatWith(), ...
In RxJava you have many operators for applying delay or adding items before and after:
delay()
delaySubscription()
startWith(T)
startWith(Observable)
concatWith(...)
with kotlin Flow you can simply:
grabMyFlow()
.onStart {
// delay by 3 seconds before starting
delay(3000L)
// just emitting an item first
emit("First item!")
emit(cachedItem()) // call another suspending function and emit the result
}
.onEach { value ->
// insert a delay of 1 second after a value only on some condition
if (value.length() > 5) {
delay(1000L)
}
}
.onCompletion {
val endingSequence: Flow<String> = grabEndingSequence()
emitAll(endingSequence)
}
error handling
RxJava have lot of operators to handle errors:
onErrorResumeWith()
onErrorReturn()
onErrorComplete()
with Flow you don't need much more than the operator catch():
grabMyFlow()
.catch { error ->
// emit something from the flow
emit("We got an error: $error.message")
// then if we can recover from this error emit it
if (error is RecoverableError) {
// error.recover() here is supposed to return a Flow<> to recover
emitAll(error.recover())
} else {
// re-throw the error if we can't recover (aka = don't catch it)
throw error
}
}
and with suspending function you can just use try {} catch() {}.
You can achieve ALL the RxJava error operators with a single catch operator because you get a suspending function.
easy to write Flow operators
Due to the coroutines powering Flow under the hood it is way easier to write operators. If you ever checked an RxJava operator you would see how hard it is and how many things you need to learn.
Writing Kotlin Flow operators is easier, you can get an idea just by looking at the source code of the operators that are already part of Flow here. The reason is coroutines makes it easier to write async code and operators just feels more natural to use.
As a bonus, Flow operators are all kotlin Extension Functions, which means either you, or libraries, can easily add operators and they will not feel weird to use (in RxJava observable.lift() or observable.compose() are needed to combine custom operators).
Upstream thread doesn't leak downstream
What does this even mean?
This explain why in RxJava you have subscribeOn() and observeOn() while in Flow you only have flowOn().
Let's take this RxJava example:
urlsToCall()
.switchMap { url ->
if (url.scheme == "local") {
val data = grabFromMemory(url.path)
Flowable.just(data)
} else {
performNetworkCall(url)
.subscribeOn(Subscribers.io())
.toObservable()
}
}
.subscribe {
// in which thread is this call executed?
}
So where is the callback in subscribe executed?
The answer is:
depends...
if it comes from the network it's in an IO thread; if it comes from the other branch it is undefined, depends on which thread is used to send the url.
If you think about it, any code you write: you don't know in which thread it is gonna be executed: always depends on the caller. The issue here is that the Thread doesn't depends on the caller anymore, it depends on what an internal function call does.
Suppose you have this plain, standard code:
fun callUrl(url: Uri) {
val callResult = if (url.scheme == "local") {
grabFromMemory(url.path)
} else {
performNetworkCall(url)
}
return callResult
}
Imagine not having a way of knowing in which thread the line return callResult is executed in without looking inside grabFromMemory() and performNetworkCall().
Think about that for a second: having the thread change based on which function you call and what they do inside.
This happens all the time with callbacks APIs: you have no way of knowing in which thread the callback you provide will be executed unless documented.
This is the concept of "upstream thread leaking downstream".
With Flow and Coroutines this is not the case, unless you explicitly require this behavior (using Dispatchers.Unconfined).
suspend fun myFunction() {
// execute this coroutine body in the main thread
withContext(Dispatchers.Main) {
urlsToCall()
.conflate() // to achieve the effect of switchMap
.transform { url ->
if (url.scheme == "local") {
val data = grabFromMemory(url.path)
emit(data)
} else {
withContext(Dispatchers.IO) {
performNetworkCall(url)
}
}
}
.collect {
// this will always execute in the main thread
// because this is where we collect,
// inside withContext(Dispatchers.Main)
}
}
}
Coroutines code will run in the context that they have been executed into. And only the part with the network call will run on the IO thread, while everything else we see here will run on the main thread.
Well, actually, we don't know where code inside grabFromMemory() will run, but we don't care: we know that it will be called inside the Main thread, inside that suspending function we could have another Dispatcher being used, but we know when it will get back with the result val data this will be again in the main thread.
Which means, looking at a piece of code, it's easier to tell in which thread it will run, if you see an explicit Dispatcher = it's that dispatcher, if you do not see it: in whatever thread dispatcher the suspension call you are looking at is being called.
Structured Concurrency
This is not a concept invented by kotlin, but it is something they embraced more than any other language I know of.
If what i explain here is not enough for you read this article or watch this video.
So what is it?
With RxJava you subscribe to observables, and they give you a Disposable object.
You need to take care of disposing of it when it's not needed anymore. So what you usually do is keep a reference to it (or put it in a CompositeDisposable) to later call dispose() on it when it's not needed anymore. If you don't the linter will give you a warning.
RxJava is somewhat nicer than a traditional thread. When you create a new thread and execute something on it, it's a "fire and forget", you do not even get a way to cancel it: Thread.stop() is deprecated, harmful, and recent implementation actually do nothing. Thread.interrupt() makes your thread fail etc.. Any exceptions goes lost.. you get the picture.
With kotlin coroutines and flow they reverse the "Disposable" concept. You CANNOT create a coroutine without a CoroutineContext.
This context define the scope of your coroutine. Every child coroutine spawned inside that one will share the same scope.
If you subscribe to a flow you have to be inside a coroutine or provide a scope too.
You can still keep reference of the coroutines you start (Job) and cancel them. This will cancel every child of that coroutine automatically.
If you are an Android developer they give you these scopes automatically. Example: viewModelScope and you can launch coroutines inside a viewModel with that scope knowing they will automatically be cancelled when the viewmodel is cleared.
viewModelScope.launch {
// my coroutine here
}
Some scope will terminate if any children fail, some other scope will let each children leave his own lifecycle without stopping other children if one fails (SupervisedJob).
Why is this a good thing?
Let me try to explain it like Roman Elizarov did.
Some old programming language had this concept of goto which basically let you jump from one line of code to another at will.
Very powerful, but if abused you could end up with very hard to understand code, difficult to debug and reason upon.
So new programming languages eventually completely removed it from the language.
When you use if or while or when it is way easier to reason on the code: doesn't matter what happens inside those blocks, you'll eventually come out of them, it's a "context", you don't have weird jumps in and out.
Launching a thread or subscribing to an RxJava observable is similar to the goto: you are executing code which than will keep going until "elsewhere" is stopped.
With coroutines, by demanding you provide a context/scope, you know that when your scope is over everything inside that coroutines will complete when your context completes, doesn't matter if you have a single coroutines or 10 thousands.
You can still "goto" with coroutines by using GlobalScope, which you shouldn't for the same reason you shouldn't use goto in languages that provides it.
Cold vs Hot - ShareFlow and StateFlow
When we work with reactive streams we always have this concept of Cold and Hot streams. Those are concepts on both the Rx world and Kotlin Flows
Cold streams are just like a function in our code: it's there and does nothing until you call it. With a Flow that means it is defined what the stream does but it will do nothing until you start to collect on it. And, like a function, if you collect (call) it twice the stream will runs twice. (ex. a cold stream to perform an http request will execute the request twice if collected twice).
Hot streams do not work like that. When you have multiple collect call on them they all share the same Hot stream under the hood, which means your hot streams runs once and you can have multiple observers.
You can usually turn a Cold stream into an Hot streams with some operator.
On RxJava you can use this concept of Connectable Observable/Flowable.
val coldObservable: Observable<Something> = buildColdObservable()
// create an hot observable from the cold one
val connectableObservable: ConnectableObservable<Something> = coldObservable.publish()
// you can subscribe multiple times to this connectable
val subADisposable: Disposable = connectableObservable.subscribe(subscriberA)
val subBDisposable: Disposable = connectableObservable.subscribe(subscriberB)
// but nothing will be emitted there until you call
val hotDisposable: Disposable = connectableObservable.connect()
// which actually run the cold observable and share the result on bot subscriberA and subscriberB
// while it's active another one can start listening to it
val subCDisposable: Disposable = connectableObservable.subscribe(subscriberC)
You than have other helpful operators like refCount() or autoConnect() which turn back the Connectable into a standard stream and under the hood automatically .connect() when the first subscriber is attached.
buildColdObservable()
.replay(1) // when a new subscriber is attached receive the last data instantly
.autoConnect() // keep the cold observable alive while there's some subscriber
On Flow you have the shareIn() and the stateIn() operators. You can see the API design here. They are less "manual" in handling when you "connect".
buildColdFlow()
.shareIn(
// you need to specify a scope for the cold flow subscription
scope = myScope,
// when to "connect"
started = SharingStarted.WhileSubscribed(),
// how many events already emitted should be sent to new subscribers
replay = 1,
)
scope
The scope is for structured concurrency. On RxJava it's the connect() operation that actually subscribe to the cold observable, it gives you a Disposable you will have to call .dispose() on somewhere. If you use refCount() or autoConnect() it is called on the first subscriber and with refCount() is never disposed while with autoConnect() is disposed when there aren't any more subscribers.
With Flow you need to give a dedicated Scope to collect the cold stream, if you cancel that scope the cold stream will stop emitting and will not be usable anymore.
started
So this one is easy
RxJava refCount() --> Flow SharingStarted.Lazily, starts collecting on the first subscriber
RxJava autoConnect() -> Flow SharingStarted.WhileSubscribed(), starts collecting on the first subscriber and cancel it when there aren't anymore
RxJava call connect() manually before any subscription -> Flow SharingStarted.Eagerly(), starts collecting immediately
The WhileSubscribed() has useful parameters, check them out.
You can also define your own logic for SharingStarted to handle when collecting from the coldFlow.
Behavior and backpressure
When you have an hot observable you always have backpressure issues to deal with. 1 source of data being listened by many means one listener can be slower then others.
Flow .shareIn collect the cold stream in a dedicated coroutine and buffer emission by default. It means if the cold stream emit too fast it will use the buffer. You can change this behavior.
Kotlin SharedFlow also let you access the replay buffer directly to inspect previous emission if you need to.
Cancelling a subscriber will have no effect on the shared flow.
using flowOn() to change the Dispatcher on the subscriber will have no effect on the shared flow (use flowOn() before sharing if you need to run the cold stream in some specific dispatcher)
stateIn
Flow has a "special" version of ShareFlow that is called StateFlow and you can use stateIn() to create one from another stream.
A StateFlow always have 1 value, it cannot be "empty", so you need to provide the initial value when you do stateIn().
A StateFlow can never throw exceptions and can never terminate (in this way is similar to BehaviorRelay in the RxRelay library)
A StateFlow will only emit if the state change (it's like it has a build in distinctUntilChanged().
RxJava Subjects vs Mutable*Flow
A Subject in RxJava is a class that you can use to manually push your data on it while still using it as a stream.
In Flow you can use MutableSharedFlow or MutableStateFlow to achieve a similar effect.
With Kotlin coroutines you can also use Channels but they are considered somewhat a lower level API.
Any Drawback?
Flow is still in development and some features available in RxJava might be marked experimental in Kotlin Coroutines Flow or have some difference here and there.
Some niche operator or operator function might not be yet implemented and you might have to implement it yourself (at least it's easier).
But other than that there aren't any drawbacks I know of.
However there are differences to be aware of that could cause some frictions in switching from RxJava and needs you to learn new things.
Structured concurrency is a step forward, but introduces new concept you need to learn and get used to (scopes, supervisorJob): cancellation is handled completely different.
There's some gotcha to be aware of.
Gotcha: Cancellation Exception
If you cancel() job in a coroutine or throw CancellationException() the exception is propagated to parent coroutines unless you used a Supervisor scope / job.
The parent coroutine also cancel sibling coroutines of the one that got canceled if that happens.
BUT if you catch(e: Exception), even using runCatching {}, you must remember to rethrow CancellationException() otherwise you'll have unexpected results cause the coroutine has been canceled but your code is still trying to execute like it wasn't.
Gotcha: UncaughtExceptionHandler
if you do launch { ... } to create a new coroutine and that coroutine throws, by default, that will terminate the coroutine but will not crash the app and you might completely missed something went wrong.
This code will not crash your app.
launch {
throw RuntimeException()
}
In some cases it might not even print anything in the log.
If it was a cancellation exception it will definitely NOT print anything in the log.
Kotlin coroutines are different from Rx. It is hard to compare them apples-to-apples, because Kotlin coroutines are a thin language feature (with just a couple of basic concepts and a few basic functions to manipulate them), while Rx is a pretty heavy library with quite large variety of ready-to-use operators. Both are designed to address a problem of asynchronous programming, however their approach to solution is very different:
Rx comes with a particular functional style of programming that can be implemented in virtually any programming language without support from the language itself. It works well when the problem at hand easily decomposes into a sequence of standard operators and not so well otherwise.
Kotlin coroutines provide a language feature that let library writers implement various asynchronous programming styles, including, but not limited to functional reactive style (Rx). With Kotlin coroutines you can also write your asynchronous code in imperative style, in promise/futures-based style, in actor-style, etc.
It is more appropriate to compare Rx with some specific libraries that are implemented based on Kotlin coroutines.
Take kotlinx.coroutines library as one example. This library provides a set of primitives like async/await and channels that are typically baked into other programming languages. It also has support for light-weight future-less actors. You can read more in the Guide to kotlinx.coroutines by example.
Channels provided by kotlinx.coroutines can replace or augment Rx in certain use-cases. There is a separate Guide to reactive streams with coroutines that goes deeper into similarities and differences with Rx.
The talk/doc you linked does not talk about channels. Channels are what fill the gap between your current understanding of coroutines and event driven programming.
With coroutines and channels you can do event driven programming as you are probably used to do with rx, but you can do it with synchronous-looking code and without as many "custom" operators.
If you want to understand this better I suggest to look outside of kotlin, where those concepts are more mature and refined (not experimental). Look at core.async from Clojure, Rich Hickey videos, posts and related discussions.
http://discuss.purelyfunctional.tv/t/core-async-channels-vs-rx-observables/519/2
https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/CoreAsync.md
Coroutines are designed to provide a lightweight asynchronous programming framework. Lightweight in terms of resources needed to start the async job. Coroutines don't enforce using an external API and are more natural for the users (programmers). In contrast, RxJava + RxKotlin has an additional data processing package that is not really needed in Kotlin which has a really rich API in the standard library for Sequences and Collections processing.
If you'd like to see more about the practical use of coroutines on Android I can recommend my article:
https://www.netguru.com/codestories/android-coroutines-%EF%B8%8Fin-2020
Coming to RxJava, RxJava streams are prone to leaks, where a stream continues to process items even when you no longer care. Kotlin coroutines use structured concurrency, which makes it much easier to manage the lifecycle of all your concurrent code.RxJava is, as it says on the tin, limited to Java. Coroutines work on any Kotlin-supported platform, so if we ever want to share asynchronous code between Android and iOS we could do it with coroutines.