I can see the following example working in Spring WebFlux handler for a flow builder:
suspend fun getDummyFlow(req: ServerRequest): ServerResponse {
val flow = flow<String> { // flow builder
for (i in 1..3) {
delay(1000) // pretend we are doing something useful here
emit("<p>Hello $i</p>") // emit next value
}
}
return ServerResponse
.ok()
.contentType(MediaType.TEXT_HTML)
.bodyAndAwait(flow)
}
Yet, I need to build a flow with a MutableSharedFlow which is not working in Spring Web Flux. Here it is an example:
suspend fun getDummyFlow(req: ServerRequest): ServerResponse {
return coroutineScope {
val flow = MutableSharedFlow<String>()
launch {
for (i in 1..3) {
delay(1000) // pretend we are doing something useful here
flow.emit("<p>Hello $i</p>") // emit next value
}
}
ServerResponse
.ok()
.contentType(MediaType.TEXT_HTML)
.bodyAndAwait(
flow
.asSharedFlow()
.take(3)
)
}
My implementation is based on the example of SharedFlow documentation.
Yet, any HTTP GET request to this endpoint stays pending and waiting for a response, whereas the former example with flow builder receives the response progressively and fine.
I have already traced my code in debug and I see .bodyAndAwait(..) being called and then emit() in both cases.
Related
emphasized textI am trying to use Kotlin Flow to process some data asynchronously and in parallel, and stream the responses to the client as they occur, as opposed to waiting until all the jobs are complete.
After unsuccessfully trying to just send the flow itself to the response, like this: call.respond(HttpStatusCode.OK, flow.toList())
... I tinkered for hours trying to figure it out, and came up with the following. Is this correct? It seems there should be a more idiomatic way of sending a Flow<MyData> as a response, like one can with a Flux<MyData> in Spring Boot.
Also, it seems that using the below method does not cancel the Flow when the HTTP request is cancelled, so how would one cancel it in Ktor?
data class MyData(val number: Int)
class MyService {
fun updateAllJobs(): Flow<MyData> =
flow {
buildList { repeat(10) { add(MyData(Random.nextInt())) } }
// Docs recommend using `onEach` to "delay" elements.
// However, if I delay here instead of in `map`, all elements are held
// and emitted at once at the very end of the cumulative delay.
// .onEach { delay(500) }
.map {
// I want to emit elements in a "stream" as each is computed.
delay(500)
emit(it)
}
}
}
fun Route.jobRouter() {
val service: MyService by inject() // injected with Koin
put("/jobs") {
val flow = service.updateAllJobs()
// Just using the default Jackson mapper for this example.
val mapper = jsonMapper { }
// `respondOutputStream` seems to be the only way to send a Flow as a stream.
call.respondOutputStream(ContentType.Application.Json, HttpStatusCode.OK) {
flow.collect {
println(it)
// The data does not stream without the newline and `flush()` call.
write((mapper.writeValueAsString(it) + "\n").toByteArray())
flush()
}
}
}
}
The best solution I was able to find (although I don't like it) is to use respondBytesWriter to write data to a response body channel. In the handler, a new job to collect the flow is launched to be able to cancel it if the channel is closed for writing (HTTP request is canceled):
fun Route.jobRouter(service: MyService) {
put("/jobs") {
val flow = service.updateAllJobs()
val mapper = jsonMapper {}
call.respondBytesWriter(contentType = ContentType.Application.Json) {
val job = launch {
flow.collect {
println(it)
try {
writeStringUtf8(mapper.writeValueAsString(it))
flush()
} catch (_: ChannelWriteException) {
cancel()
}
}
}
job.join()
}
}
}
I have a server that relays between two (different) clients. When the User (first client, through websockets) sends a message, the server needs to repeat this message every X milliseconds to the Device (second client) until a new message is received, or the websocket is closed.
I consume the websocket as a flow, and I've created the following operator:
fun <T> flowEvery(value: T, everMilliSeconds: Long): Flow<T> =
flow {
while (true) {
emit(value)
delay(everMilliSeconds)
}
}.cancellable()
#ExperimentalCoroutinesApi
fun <T> Flow<T>.repeatEvery(mSec: Long): Flow<T> =
this.flatMapLatest {
flowEvery(it, mSec)
}
Problem is, once the socket is closed the last message is kept on being sent for ever.
My call site is:
try {
oscConnections.sendTo(
deviceIdentifier,
incoming.consumeAsFlow().repeatEvery(50).mapNotNull { frame ->
when (frame) {
is Frame.Text -> listOf(frame.readText().toFloat())
else -> null
}
})
} finally {
close(CloseReason(CloseReason.Codes.NORMAL, "Ended"))
}
the incoming channel is closed (onCompletion is called) but the stream sent to sendTo is not. sendTo it self consumes the input stream and send a UDP message for every element it consumes.
How can I force the flow to stop?
By using flatMapLatest or transformLatest you replace the last value of the upstream Flow with a never-ending Flow.
You must stop that Flow somehow and CancellationExceptions are used everywhere in coroutines to signal the cancellation of coroutines. You can wrap your never-ending Flow logic in a coroutineScope to precisely cancel only that scope once the upstream flow has completed.
fun <T> Flow<T>.repeatEvery(delay: Long): Flow<T> =
flow<T> {
try {
coroutineScope {
onCompletion { this#coroutineScope.cancel() }
.transformLatest { value ->
while (true) {
emit(value)
delay(delay)
}
}
.collect(::emit)
}
}
catch (e: CancellationException) {
// done
}
}
PS: .cancellable() doesn't do much in your example. As per documentation Flows built using flow builders like flow { … } are automatically cancellable.
I have code that should change SharedPreferences into obsarvable storage with flow so I've code like this
internal val onKeyValueChange: Flow<String> = channelFlow {
val callback = SharedPreferences.OnSharedPreferenceChangeListener { _, key ->
coroutineScope.launch {
//send(key)
offer(key)
}
}
sharedPreferences.registerOnSharedPreferenceChangeListener(callback)
awaitClose {
sharedPreferences.unregisterOnSharedPreferenceChangeListener(callback)
}
}
or this
internal val onKeyValueChange: Flow<String> = callbackFlow {
val callback = SharedPreferences.OnSharedPreferenceChangeListener { _, key ->
coroutineScope.launch {
send(key)
//offer(key)
}
}
sharedPreferences.registerOnSharedPreferenceChangeListener(callback)
awaitClose {
sharedPreferences.unregisterOnSharedPreferenceChangeListener(callback)
}
}
Then I observe this preferences for token, userId, companyId and then log into but there is odd thing as I need to build app three times like changing token not causes tokenFlow to emit anything, then second time new userId not causes userIdFlow to emit anything, then after 3rd login I can logout/login and it works. On logout I am clearing all 3 properties stores in prefs token, userId, companyId.
For callbackFlow:
You cannot use emit() as the simple Flow (because it's a suspend function) inside a callback. Therefore the callbackFlow offers you a synchronized way to do it with the trySend() option.
Example:
fun observeData() = flow {
myAwesomeInterface.addListener{ result ->
emit(result) // NOT ALLOWED
}
}
So, coroutines offer you the option of callbackFlow:
fun observeData() = callbackFlow {
myAwesomeInterface.addListener{ result ->
trySend(result) // ALLOWED
}
awaitClose{ myAwesomeInterface.removeListener() }
}
For channelFlow:
The main difference with it and the basic Flow is described in the documentation:
A channel with the default buffer size is used. Use the buffer
operator on the resulting flow to specify a user-defined value and to
control what happens when data is produced faster than consumed, i.e.
to control the back-pressure behavior.
The trySend() still stands for the same thing. It's just a synchronized way (a non suspending way) for emit() or send()
I suggest you to check Romans Elizarov blog for more detailed information especially this post.
Regarding your code, for callbackFlow you wont' be needing a coroutine launch:
coroutineScope.launch {
send(key)
//trySend(key)
}
Just use trySend()
Another Example, maybe much concrete:
private fun test() {
lifecycleScope.launch {
someFlow().collectLatest {
Log.d("TAG", "Finally we received the result: $it")
// Cancel this listener, so it will not be subscribed anymore to the callbackFlow. awaitClose() will be triggered.
// cancel()
}
}
}
/**
* Define a callbackFlow.
*/
private fun someFlow() = callbackFlow {
// A dummy class which will run some business logic and which will sent result back to listeners through ApiCallback methods.
val service = ServiceTest() // a REST API class for example
// A simple callback interface which will be called from ServiceTest
val callback = object : ApiCallback {
override fun someApiMethod(data: String) {
// Sending method used by callbackFlow. Into a Flow we have emit(...) or for a ChannelFlow we have send(...)
trySend(data)
}
override fun anotherApiMethod(data: String) {
// Sending method used by callbackFlow. Into a Flow we have emit(...) or for a ChannelFlow we have send(...)
trySend(data)
}
}
// Register the ApiCallback for later usage by ServiceTest
service.register(callback)
// Dummy sample usage of callback flow.
service.execute(1)
service.execute(2)
service.execute(3)
service.execute(4)
// When a listener subscribed through .collectLatest {} is calling cancel() the awaitClose will get executed.
awaitClose {
service.unregister()
}
}
interface ApiCallback {
fun someApiMethod(data: String)
fun anotherApiMethod(data: String)
}
class ServiceTest {
private var callback: ApiCallback? = null
fun unregister() {
callback = null
Log.d("TAG", "Unregister the callback in the service class")
}
fun register(callback: ApiCallback) {
Log.d("TAG", "Register the callback in the service class")
this.callback = callback
}
fun execute(value: Int) {
CoroutineScope(Dispatchers.IO).launch {
if (value < 2) {
callback?.someApiMethod("message sent through someApiMethod: $value.")
} else {
callback?.anotherApiMethod("message sent through anotherApiMethod: $value.")
}
}
}
}
been pretty stuck on an issue with Kotlin flows/channels today. Essentially I want to take the values emitted from a flow, and immediately send them in a channel. We then subscribe to that channel as a flow via an exposed method. The use case here is to have a channel subscription that is always live and a flow that can be turned on and off independently.
private val dataChannel = BroadcastChannel<Data>(1)
suspend fun poll() {
poller.start(POLLING_PERIOD_MILLISECONDS)
.collect {
dataChannel.send(it)
}
}
suspend fun stopPoll() {
poller.stop()
}
suspend fun subscribe(): Flow<Data> {
return dataChannel.asFlow()
}
The simple use case I have here is a poller which returns a channelFlow. Ideally I could then emit to the channel in the collect method. This doesn't seem to work though. My rookie coroutine thought is that because collect and send are suspending, the emissions gets suspended in collect and we get stuck.
Is there any built in functions for flow or channel that can handle this or any other way to achieve this behavior?
For your case you can try to use hot stream of data SharedFlow instead of a Channel:
private val dataFlow = MutableSharedFlow<String>(extraBufferCapacity = 1)
suspend fun poll() {
poller.start(POLLING_PERIOD_MILLISECONDS)
.collect {
dataFlow.tryEmit(it)
}
}
suspend fun stopPoll() {
poller.stop()
}
fun subscribe(): Flow<Data> {
return dataFlow
}
tryEmit() - Tries to emit a value to this shared flow without suspending, so calling it will not suspend the collect block.
I wanted to know how can I send/emit items to a Kotlin.Flow, so my use case is:
In the consumer/ViewModel/Presenter I can subscribe with the collect function:
fun observe() {
coroutineScope.launch {
// 1. Send event
reopsitory.observe().collect {
println(it)
}
}
}
But the issue is in the Repository side, with RxJava we could use a Behaviorsubject expose it as an Observable/Flowable and emit new items like this:
behaviourSubject.onNext(true)
But whenever I build a new flow:
flow {
}
I can only collect. How can I send values to a flow?
If you want to get the latest value on subscription/collection you should use a ConflatedBroadcastChannel:
private val channel = ConflatedBroadcastChannel<Boolean>()
This will replicate BehaviourSubject, to expose the channel as a Flow:
// Repository
fun observe() {
return channel.asFlow()
}
Now to send an event/value to that exposed Flow simple send to this channel.
// Repository
fun someLogicalOp() {
channel.send(false) // This gets sent to the ViewModel/Presenter and printed.
}
Console:
false
If you wish to only receive values after you start collecting you should use a BroadcastChannel instead.
To make it clear:
Behaves as an Rx's PublishedSubject
private val channel = BroadcastChannel<Boolean>(1)
fun broadcastChannelTest() {
// 1. Send event
channel.send(true)
// 2. Start collecting
channel
.asFlow()
.collect {
println(it)
}
// 3. Send another event
channel.send(false)
}
false
Only false gets printed as the first event was sent before collect { }.
Behaves as an Rx's BehaviourSubject
private val confChannel = ConflatedBroadcastChannel<Boolean>()
fun conflatedBroadcastChannelTest() {
// 1. Send event
confChannel.send(true)
// 2. Start collecting
confChannel
.asFlow()
.collect {
println(it)
}
// 3. Send another event
confChannel.send(false)
}
true
false
Both events are printed, you always get the latest value (if present).
Also, want to mention Kotlin's team development on DataFlow (name pending):
https://github.com/Kotlin/kotlinx.coroutines/pull/1354
Which seems better suited to this use case (as it will be a cold stream).
Take a look at MutableStateFlow documentation as it is a replacement for ConflatedBroadcastChannel that is going to be deprecated, very soon.
For a better context, look at the whole discussion on the original issue on Kotlin's repository on Github.
UPDATE:
Kotlin Coroutines 1.4.0 is now available with MutableSharedFlow, which replaces the need for Channel. MutableSharedFlow cleanup is also built in so you don't need to manually OPEN & CLOSE it, unlike Channel. Please use MutableSharedFlow if you need a Subject-like api for Flow
ORIGINAL ANSWER
Since your question had the android tag I'll add an Android implementation that allows you to easily create a BehaviorSubject or a PublishSubject that handles its own lifecycle.
This is relevant in Android because you don't want to forget to close the channel and leak memory. This implementation avoids the need to explicitly "dispose" of the reactive stream by tying it to the creation and destruction of the Fragment/Activity. Similar to LiveData
interface EventReceiver<Message> {
val eventFlow: Flow<Message>
}
interface EventSender<Message> {
fun postEvent(message: Message)
val initialMessage: Message?
}
class LifecycleEventSender<Message>(
lifecycle: Lifecycle,
private val coroutineScope: CoroutineScope,
private val channel: BroadcastChannel<Message>,
override val initialMessage: Message?
) : EventSender<Message>, LifecycleObserver {
init {
lifecycle.addObserver(this)
}
override fun postEvent(message: Message) {
if (!channel.isClosedForSend) {
coroutineScope.launch { channel.send(message) }
} else {
Log.e("LifecycleEventSender","Channel is closed. Cannot send message: $message")
}
}
#OnLifecycleEvent(Lifecycle.Event.ON_CREATE)
fun create() {
channel.openSubscription()
initialMessage?.let { postEvent(it) }
}
#OnLifecycleEvent(Lifecycle.Event.ON_DESTROY)
fun destroy() {
channel.close()
}
}
class ChannelEventReceiver<Message>(channel: BroadcastChannel<Message>) :
EventReceiver<Message> {
override val eventFlow: Flow<Message> = channel.asFlow()
}
abstract class EventRelay<Message>(
lifecycle: Lifecycle,
coroutineScope: CoroutineScope,
channel: BroadcastChannel<Message>,
initialMessage: Message? = null
) : EventReceiver<Message> by ChannelEventReceiver<Message>(channel),
EventSender<Message> by LifecycleEventSender<Message>(
lifecycle,
coroutineScope,
channel,
initialMessage
)
By using the Lifecycle library from Android, I can now create a BehaviorSubject that cleans itself up after the activity/fragment has been destroyed
class BehaviorSubject<String>(
lifecycle: Lifecycle,
coroutineScope: CoroutineScope,
initialMessage = "Initial Message"
) : EventRelay<String>(
lifecycle,
coroutineScope,
ConflatedBroadcastChannel(),
initialMessage
)
or I can create a PublishSubject by using a buffered BroadcastChannel
class PublishSubject<String>(
lifecycle: Lifecycle,
coroutineScope: CoroutineScope,
initialMessage = "Initial Message"
) : EventRelay<String>(
lifecycle,
coroutineScope,
BroadcastChannel(Channel.BUFFERED),
initialMessage
)
And now I can do something like this
class MyActivity: Activity() {
val behaviorSubject = BehaviorSubject(
this#MyActivity.lifecycle,
this#MyActivity.lifecycleScope
)
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
if (savedInstanceState == null) {
behaviorSubject.eventFlow
.onEach { stringEvent ->
Log.d("BehaviorSubjectFlow", stringEvent)
// "BehaviorSubjectFlow: Initial Message"
// "BehaviorSubjectFlow: Next Message"
}
.flowOn(Dispatchers.Main)
.launchIn(this#MyActivity.lifecycleScope)
}
}
override fun onResume() {
super.onResume()
behaviorSubject.postEvent("Next Message")
}
}