I'm recently studying and reading a lot about Flow and Kotlin Coroutines. But I still get confused about when I should use Flow and when I should use Channel.
At the beginning it looked more simple. Working with hot streams of data? Channel. Cold ones? Flows. Same goes if you need to listen to streams of data from more than a single place; if that's the case Channel is the choice to go. There are still a lot of examples and questions.
But recently FlowChannels where introduced, together with tons of methods and classes that encourage the use of Flow, which facilities transforming Channels into Flows and so on. With all this new stuff coming on each Kotlin release I am getting more and more confused. So the question is:
When should I use Channel and when should I use Flow?
For many use cases where the best tool so far was Channel, Flow has become the new best tool.
As a specific example, callbackFlow is now the best approach to receiving data from a 3rd-party API's callback. This works especially well in a GUI setting. It couples the callback, a channel, and the associated receiving coroutine all in the same self-contained Flow instance. The callback is registered only while the flow is being collected. Cancellation of the flow automatically propagates into closing the channel and deregistering the callback. You just have to provide the callback-deregistering code once.
You should look at Channel as a lower-level primitive that Flow uses in its implementation. Consider working with it directly only after you realize Flow doesn't fit your requirements.
In my opinion a great explanation is here (Roman Elizarov) Cold flows, hot channels:
Channels are a great fit to model data sources that are intrinsically hot, data sources that exist without application’s requests for them: incoming network connections, event streams, etc.
Channels, just like futures, are synchronization primitives. You shall use a channel when you need to send data from one coroutine to another coroutine in the same or in a different process
But what if we don’t need either concurrency or synchronization, but need just non-blocking streams of data? We did not have a type for that until recently, so welcome Kotlin Flow type...
Unlike channels, flows do not inherently involve any concurrency. They are non-blocking, yet sequential. The goal of flows is to become for asynchronous data streams what suspending functions are for asynchronous operations — convenient, safe, easy to learn and easy to use.
Related
I am working in bluetooth in android kotlin. I found this blessed library and this class BluetoothPeripheral is using ConcurrentLinkedQueue. I don't understand what is the use of
private val commandQueue: Queue<Runnable> = ConcurrentLinkedQueue()
I am looking this enqueue function and I cannot understand the use case in here. what the author trying to achieved in here?
This enqueue function calls in different place i.e. readCharacteristic what is the use case in this function?
Thanks
Building on #broot's comment:
ConcurrentLinkedQueue is part of the java.util.concurrent package which is all about collections that are thread-safe
A Queue is a kind of collection that is designed for efficient adding and removal. Typically they offer First In First Out.
If you have an application that is dealing with a high throughput of tasks, producers put items in a queue and consumers take them. Depending which is faster, you may have more producer threads than consumer threads, or the other way around. You achieve process isolation by using a thread-safe queue, such as ConcurrentLinkedQueue
Some Queue implementations have bounded capacity, but a queue like ConcurrentLinkedQueue is based on a Linked List so typically have have far greater capacity, but mean that some operations, such as search might perform less well.
There is also a Dequeue which is a Queue that you can remove items easily from both ends.
I have no idea what the Bluetooth application is about and why it needs ConcurrentLinkedQueue so I cannot comment on whether it is the "best options to use in bluetooth case"
I'm writing a Kotlin server using Ktor - where my request handlers are written using Kotlin coroutines.
My understanding is each request handler is run on Ktor's thread pool, which contains far fewer threads than the traditional pool size of 1-thread-per-request server frameworks due to the lightweight/suspendable nature of coroutines. Great!
The issue I have is that my application still needs to interact with some blocking resources (JDBC database connection pool), but my understanding is that if I merely call these blocking APIs directly from the request coroutine I will end up with liveness issues - as I can end up blocking all the threads used to handle my requests! Not great.
Since I'm still relatively new to the world of Kotlin and coroutines, I'm wondering if anyone here can give me some tips on the best way to handle this situation.
I've seen Dispatchers.IO referenced a few times elsewhere. Is that considered the best way to manage these blocking calls? Are there any good examples of this?
The API I'm trying to use does allow for some asyncronicity by passing an Executor. Ideally, I could also wrap these calls in a convenient, idiomatic Kotlin API for suspending transactions.
You understand it all correctly. In most cases you should never block the thread when inside a coroutine. One exception is Dispatchers.IO mentioned by you. It is the standard way of handling blocking code and it is very easy to use:
withContext(Dispatchers.IO) {
// blocking code
}
withContext() is a suspend function, so you can think of above as the way to convert blocking to suspend. However, Dispatchers.IO doesn't really perform any magic - it just uses a bigger pool of threads, designated for blocking. I believe by default it creates 64 threads at maximum.
If you need to perform several parallel blocking operations, it is usually better to create your own thread pool to not block other components of the application.
If the IO library provides asynchronous API then generally it is better to use it instead of the blocking API. However, in many cases libraries provide asynchronous API by managing their own internal thread pool for blocking. In that case using asynchronous API and using blocking API with Dispatchers.IO is very similar. Dispatchers.IO could be even better, because it re-uses same IO threads across all IO operations and it can partially share threads with a thread pool designated for CPU computations (Dispatchers.Default).
Yes. the Dispatchers.IO would be the answer. I had a test with quarkus. The vert.x had no 2-seconds-blocking-alarm after I switched JDBC connection to Dispatchers.IO
https://github.com/hmchangm/quarkus-reactive-kotlin/blob/mariadb/src/main/kotlin/tw/idv/brandy/arrow/repo/FruitRepo.kt
I am converting my data service to use the MongoDB reactive driver. With the way that I am querying for information (in several parts, concurrently) it has allowed me to coordinate all of the activities much more efficiently and quickly.
So far, the consumers of this API are not ready to be converted, so I end up calling Mono.zip(...).blockOptional before returning the fully realized object to the rest method to return to the client. But I am wondering if I could benefit from returning a Mono, instead, and still get some benefits, even if the consumers of my data service API are not ready to convert fully to reactive.
Would returning a Mono save on Spring web serialization/deserialization between the two services? That, currently, is the most expensive portion of the entire data flow. Or would it be basically the same cost in time and performance between returning a Mono or the object itself?
Yes, I understand the benefit of making the whole data flow entirely reactive, and I agree that is the best way to go. But, for now, I am trying to learn whether or not I can get the benefit of less serialization before going "full reactive".
There is no difference in serialisation when you return Mono or Foo. There is difference in blocking thread or not.
Simply when you return Mono or flux each IO trip will not block thread but as soon as your data leave your service it will be serialised into json
Also as soon as you call block() on reactive api there is no benefits at all.
The main idea of reactive is not to make api faster it is to make your api to handle more. So in case if you have one service fully reactive and other consumers are not then you still have benefits because your service will be able to handle more consumers
From the official guide and samples from web, I didn't see any mentions of locking or synchronization, or how safe is modifying a shared variable in multiple launch or async calls.
Coroutines bring a concurrent programming model that may result in simultaneously executed code. Just as you know it from thread-based libraries, you have to care about synchronization as noted in the docs:
Coroutines can be executed concurrently using a multi-threaded dispatcher like the Dispatchers.Default. It presents all the usual concurrency problems. The main problem being synchronization of access to shared mutable state. Some solutions to this problem in the land of coroutines are similar to the solutions in the multi-threaded world, but others are unique.
With Kotlin Coroutines you can make use of acquainted strategies like using thread-safe data structures, confining execution to a single thread or using locks (e.g. Mutex).
Besides the common patterns, Kotlin coroutines encourage us to use a "share by communication" style. Concretely, an "actor" can be shared between coroutines. They can be used by coroutines, which may send/take messages to/from it. Also have a look at Channels.
I'm currently making a client-client approach on some simulation with objective-c with two computers (mac1 and mac2).
I have a class Client, and every computer has a instance of the "Client" on it (client1,client2). I expect that both clients will be synchronized: they will both be equal apart from memory locations.
When a user presses a key on mac1, I want both client1 and client2 to receive a given method from class Client (so that they are synchronized, i.e. they are the same apart from it's memory location on each mac).
To this approach, my current idea is to make 2 methods:
- (void) sendSelector:(Client*)toClient,...;
- (void) receiveSelector:(Client*)fromClient,...;
sendSelector: uses NSStringFromSelector() to transform the method to a NSString, and send it over the network (let's not worry about sending strings over net now).
On the other hand, receiveSelector: uses NSSelectorFromString() to transform a NSString back to a selector.
My first question/issue is: to what extent is this approach "standard" on networking with objective-c?
My second question:
And the method's arguments? Is there any way of "packing" a given class instance and send it over the network? I understand the pointer's problem when packing, but every instance on my program as an unique identity, so that should be no problem since both clients will know how to retrieve the object from its identity.
Thanks for your help
Let me address your second question first:
And the method's arguments? Is there any way of "packing" a given
class instance and send it over the network?
Many Cocoa classes implement/adopt the NSCoding #protocol. This means they support some default implementation for serializing to a byte stream, which you could then send over the network. You would be well advised to use the NSCoding approach unless it's fundamentally not suited to your needs for some reason. (i.e. use the highest level of abstraction that gets the job done)
Now for the more philosophical side of your first question; I'll rephrase your question as "is it a good approach to use serialized method invocations as a means of communication between two clients over a network?"
First, you should know that Objective-C has a not-often-used-any-more, but reasonably complete, implementation for handling remote invocations between machines with a high level of abstraction. It was called Distributed Objects. Apple appears to be shoving it under the rug to some degree (with good reason -- keep reading), but I was able to find an old cached copy of the Distributed Objects Programming Topics guide. You may find it informative. AFAIK, all the underpinnings of Distributed Objects still ship in the Objective-C runtime/frameworks, so if you wanted to use it, if only to prototype, you probably could.
I can't speculate as to the exact reasons that you can't seem to find this document on developer.apple.com these days, but I think it's fair to say that, in general, you don't want to be using a remote invocation approach like this in production, or over insecure network channels (for instance: over the Internet.) It's a huge potential attack vector. Just think of it: If I can modify, or spoof, your network messages, I can induce your client application to call arbitrary selectors with arbitrary arguments. It's not hard to see how this could go very wrong.
At a high level, let me recommend coming up with some sort of protocol for your application, with some arbitrary wire format (another person mentioned JSON -- It's got a lot of support these days -- but using NSCoding will probably bootstrap you the quickest), and when your client receives such a message, it should read the message as data and make a decision about what action to take, without actually deriving at runtime what is, in effect, code from the message itself.
From a "getting things done" perspective, I like to share a maxim I learned a while ago: "Make it work; Make it work right; Make it work fast. In that order."
For prototyping, maybe you don't care about security. Maybe when you're just trying to "make it work" you use Distributed Objects, or maybe you roll your own remote invocation protocol, as it appears you've been thinking of doing. Just remember: you really need to "make it work right" before releasing it into the wild, or those decisions you made for prototyping expedience could cost you dearly. The best approach here will be to create a class or group of classes that abstracts away the network protocol and wire format from the rest of your code, so you can swap out networking implementations later without having to touch all your code.
One more suggestion: I read in your initial question a desire to 'keep an object (or perhaps an object graph) in sync across multiple clients.' This is a complex topic, but you may wish to employ a "Command Pattern" (see the Gang of Four book, or any number of other treatments in the wild.) Taking such an approach may also inherently bring structure to your networking protocol. In other words, once you've broken down all your model mutation operations into "commands" maybe your protocol is as simple as serializing those commands using NSCoding and shipping them over the wire to the other client and executing them again there.
Hopefully this helps, or at least gives you some starting points and things to consider.
These days it would seem that the most standard way is to package everything up on JSON.