As far as I know, the consumers are to check the condition variable when they're waiting for the signal.
Consumer:
lock.Lock()
while(condition) {
cond.Wait()
}
lock.Unlock()
Producer:
lock.Lock()
changeCondition()
cond.Signal() or cond.Broadcast()
lock.Unlock()
Why can't it be:
Consumer:
lock.Lock()
cond.Wait()
lock.Unlock()
Producer:
lock.Lock()
changeCondition()
if(condition) {
cond.Signal() or cond.Broadcast()
}
lock.Unlock()
Related
I have two separate message listener classes written in Kotlin to listen to two separate AWS SQS queues (one is main SQS queue and another is Dead Letter Queue (DLQ)).
The problem is that the message listener for DLQ is unable to consume the messages from its DLQ. This is a tricky problem because my listener code of DLQ is absolutely correct and can consume the messages from other SQS queues but not for this DLQ particularly. I tested its working functionality by replacing DLQ name by the main queue name. There are messages on DLQ as well. By the way both queues are standard queues on AWS (not FIFO).
Here is the code snippet where I'm initializing the SQS connection:-
#PostConstruct
fun init() {
try {
if (enableCalmQueue) {
initializeListeners("$podName-${Util.MAIN_QUEUE}", mainQueueListener)
initializeListeners("$podName-${Util.DLQ}", dlqListener)
LOG.info("Initialized SQS Queue Response Handler")
}
} catch (e: Exception) {
LOG.warn("Failed to Initialize response handler due to ${e.message}")
}
}
private fun initializeListeners(queueName: String, messageListener: MessageListener) {
val connection = getSqsConnectionObject()
val session = connection.createSession(false, SQSSession.UNORDERED_ACKNOWLEDGE)
val consumer = session.createConsumer(session.createQueue(queueName))
consumer.messageListener = messageListener
connection.start()
LOG.info("Initialized $messageListener for $queueName")
}
private fun getSqsConnectionObject(): SQSConnection {
val connectionFactory = SQSConnectionFactory(
ProviderConfiguration(),
AmazonSQSClientBuilder.defaultClient()
)
return connectionFactory.createConnection()
}
DLQ Listener:-
#Named("dlqListener")
class DeadLetterQueueListener : MessageListener {
#Value("\${pod_name}")
private val podName: String = "dev"
#Inject
private lateinit var tradeStatusService: TradeStatusService
companion object {
private val LOG = LoggerFactory.getLogger(DeadLetterQueueListener::class.java)
}
override fun onMessage(message: Message?) {
try {
if (message is TextMessage) {
LOG.info("Received message ${message.text} in Dead Letter Queue Listener " +
"with jmsMessageId: ${message.jmsMessageID}")
if(message.getStringProperty(Util.QUEUE_NAME) == "$podName-${Util.MAIN_QUEUE}") {
message.acknowledge()
val jsonMessage = message.text
val objectMapper = ObjectMapper()
val externalTradeInputs = objectMapper.readValue(jsonMessage, ExternalTradeInputs::class.java)
val tradeStatusData = TradeStatusData().apply {
this.tradeBookingId = externalTradeInputs.tradeBookingId
this.statusDateTime = Date()
}
}
}
} catch (e: Exception) {
LOG.error("Error processing in Dead Letter Queue Listener with : ${e.message}")
}
}
}
Like I mentioned, this dlqListener works fine if I pass the main queue name but not DLQ queue name.
I tried to debug a lot but could not find anything concrete. Both main and DLQ exists in AWS. I suspect something some wrong in the way DLQ has been configured but to my best I have verified but nothing found any issues over there.
I would really appreciate if someone can give me some reference or hints about its root cause.
emphasized textI am trying to use Kotlin Flow to process some data asynchronously and in parallel, and stream the responses to the client as they occur, as opposed to waiting until all the jobs are complete.
After unsuccessfully trying to just send the flow itself to the response, like this: call.respond(HttpStatusCode.OK, flow.toList())
... I tinkered for hours trying to figure it out, and came up with the following. Is this correct? It seems there should be a more idiomatic way of sending a Flow<MyData> as a response, like one can with a Flux<MyData> in Spring Boot.
Also, it seems that using the below method does not cancel the Flow when the HTTP request is cancelled, so how would one cancel it in Ktor?
data class MyData(val number: Int)
class MyService {
fun updateAllJobs(): Flow<MyData> =
flow {
buildList { repeat(10) { add(MyData(Random.nextInt())) } }
// Docs recommend using `onEach` to "delay" elements.
// However, if I delay here instead of in `map`, all elements are held
// and emitted at once at the very end of the cumulative delay.
// .onEach { delay(500) }
.map {
// I want to emit elements in a "stream" as each is computed.
delay(500)
emit(it)
}
}
}
fun Route.jobRouter() {
val service: MyService by inject() // injected with Koin
put("/jobs") {
val flow = service.updateAllJobs()
// Just using the default Jackson mapper for this example.
val mapper = jsonMapper { }
// `respondOutputStream` seems to be the only way to send a Flow as a stream.
call.respondOutputStream(ContentType.Application.Json, HttpStatusCode.OK) {
flow.collect {
println(it)
// The data does not stream without the newline and `flush()` call.
write((mapper.writeValueAsString(it) + "\n").toByteArray())
flush()
}
}
}
}
The best solution I was able to find (although I don't like it) is to use respondBytesWriter to write data to a response body channel. In the handler, a new job to collect the flow is launched to be able to cancel it if the channel is closed for writing (HTTP request is canceled):
fun Route.jobRouter(service: MyService) {
put("/jobs") {
val flow = service.updateAllJobs()
val mapper = jsonMapper {}
call.respondBytesWriter(contentType = ContentType.Application.Json) {
val job = launch {
flow.collect {
println(it)
try {
writeStringUtf8(mapper.writeValueAsString(it))
flush()
} catch (_: ChannelWriteException) {
cancel()
}
}
}
job.join()
}
}
}
Can anyone …
explain why the following code keeps retrying indefinitely, once retryWhen is called?
correct the code to stop retrying when Random.nextBoolean()
returns true?
I think there is a 50% chance that retryWhen is called because of Random.nextBoolean(). However, once it is called, it goes into an infinite loop.
import io.reactivex.rxjava3.core.Observable
import kotlin.random.Random
fun main() {
Observable
.just(Unit)
.flatMap {
if (Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}.retryWhen {
it.flatMap {
println("retryWhen")
Observable.just(Unit)
}
}
}
.subscribe()
}
Output:
retryWhen
retryWhen
retryWhen
⋮
(continued)
You've applied the retryWhen to the result of the execution of either branch of the if-statement.
Chain the retryWhen operator to the stream produced by flatMap:
e.g.
Observable
.just(Unit)
.flatMap {
if(Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}
}
.retryWhen {
it.flatMap {
println("retryWhen")
Observable.just(Unit)
}
}
.subscribe()
Please take a look the same question and answer
Copied answer:
retryWhen calls the provided function when an Observer subscribes to it so you have a main sequence accompanied by a sequence that emits the Throwable the main sequence failed with. You should compose a logic onto the Observable you get in this Function so at the end, one Throwable will result in a value on the other end.
I think your expectation is:
Observable
.just(Unit)
.flatMap {
if (Random.nextBoolean()) {
Observable.just(Unit)
} else {
Observable.error(Throwable())
}.retryWhen {
it.takeWhile {
Random.nextBoolean()
}
.doOnNext {
println("retryWhen")
}
}
}
.subscribe()
I have a server that relays between two (different) clients. When the User (first client, through websockets) sends a message, the server needs to repeat this message every X milliseconds to the Device (second client) until a new message is received, or the websocket is closed.
I consume the websocket as a flow, and I've created the following operator:
fun <T> flowEvery(value: T, everMilliSeconds: Long): Flow<T> =
flow {
while (true) {
emit(value)
delay(everMilliSeconds)
}
}.cancellable()
#ExperimentalCoroutinesApi
fun <T> Flow<T>.repeatEvery(mSec: Long): Flow<T> =
this.flatMapLatest {
flowEvery(it, mSec)
}
Problem is, once the socket is closed the last message is kept on being sent for ever.
My call site is:
try {
oscConnections.sendTo(
deviceIdentifier,
incoming.consumeAsFlow().repeatEvery(50).mapNotNull { frame ->
when (frame) {
is Frame.Text -> listOf(frame.readText().toFloat())
else -> null
}
})
} finally {
close(CloseReason(CloseReason.Codes.NORMAL, "Ended"))
}
the incoming channel is closed (onCompletion is called) but the stream sent to sendTo is not. sendTo it self consumes the input stream and send a UDP message for every element it consumes.
How can I force the flow to stop?
By using flatMapLatest or transformLatest you replace the last value of the upstream Flow with a never-ending Flow.
You must stop that Flow somehow and CancellationExceptions are used everywhere in coroutines to signal the cancellation of coroutines. You can wrap your never-ending Flow logic in a coroutineScope to precisely cancel only that scope once the upstream flow has completed.
fun <T> Flow<T>.repeatEvery(delay: Long): Flow<T> =
flow<T> {
try {
coroutineScope {
onCompletion { this#coroutineScope.cancel() }
.transformLatest { value ->
while (true) {
emit(value)
delay(delay)
}
}
.collect(::emit)
}
}
catch (e: CancellationException) {
// done
}
}
PS: .cancellable() doesn't do much in your example. As per documentation Flows built using flow builders like flow { … } are automatically cancellable.
Update Coroutines 1.3.0-RC
Working version:
#FlowPreview
suspend fun streamTest(): Flow<String> = channelFlow {
listener.onSomeResult { result ->
if (!isClosedForSend) {
offer(result)
}
}
awaitClose {
listener.unsubscribe()
}
}
Also checkout this Medium article by Roman Elizarov: Callbacks and Kotlin Flows
Original Question
I have a Flow emitting multiple Strings:
#FlowPreview
suspend fun streamTest(): Flow<String> = flowViaChannel { channel ->
listener.onSomeResult { result ->
if (!channel.isClosedForSend) {
channel.sendBlocking(result)
}
}
}
After some time I want to unsubscribe from the stream. Currently I do the following:
viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
this.cancel()
}
}
If the coroutine gets canceled, the stream is still executed. There is just no subscriber listening to new values. How can I unsubscribe and stop the stream function?
A solution is not to cancel the flow, but the scope it's launched in.
val job = scope.launch { flow.cancellable().collect { } }
job.cancel()
NOTE: You should call cancellable() before collect if you want your collector stop when Job is canceled.
You could use the takeWhile operator on Flow.
flow.takeWhile { it != "someString" }.collect { emittedValue ->
//Do stuff until predicate is false
}
For those willing to unsubscribe from the Flow within the Coroutine scope itself, this approach worked for me :
viewModelScope.launch {
beaconService.streamTest().collect {
//Do something then
this.coroutineContext.job.cancel()
}
}
With the current version of coroutines / Flows (1.2.x) I don't now a good solution. With onCompletion you will get informed when the flow stops, but you are then outside of the streamTest function and it will be hard to stop listening of new events.
beaconService.streamTest().onCompletion {
}.collect {
...
}
With the next version of coroutines (1.3.x) it will be really easy. The function flowViaChannel is deprecated in favor for channelFlow. This function allows you to wait for closing of the flow and do something in this moment, eg. remove listener:
channelFlow<String> {
println("Subscribe to listener")
awaitClose {
println("Unsubscribe from listener")
}
}
When a flow runs in couroutin scope, you can get a job from it to controls stop subscribe.
// Make member variable if you want.
var jobForCancel : Job? = null
// Begin collecting
jobForCancel = viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
// this.cancel() // Don't
}
}
// Call whenever to canceled
jobForCancel?.cancel()
For completeness, there is a newer version of the accepted answer. Instead of explicitly using the launch coroutine builder, we can use the launchIn method directly on the flow:
val job = flow.cancellable().launchIn(scope)
job.cancel()
Based on #Ronald answer this works great for testing when you need to make your Flow emits again.
val flow = MutableStateFlow(initialValue)
flow.take(n).collectIndexed { index, _ ->
if (index == something) {
flow.value = update
}
}
//your assertions
We have to know how many emissions in total we expect n and then we can use the index to know when to update the Flow so we can receive more emissions.
If you want to cancel only the subscription being inside it, you can do it like this:
viewModelScope.launch {
testScope.collect {
return#collect cancel()
}
}
There are two ways to do this that are by design from the Kotlin team:
As #Ronald pointed out in another comment:
Option 1: takeWhile { //predicate }
Cancel collection when the predicate is false. Final value will not be collected.
flow.takeWhile { value ->
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will never hit this
}
Option 2: transformWhile { //predicate }
When predicate is false, collect that value, then cancel
flow.transformWhile { value ->
emit(value)
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will be the last value
}
https://github.com/Kotlin/kotlinx.coroutines/issues/2065