Unit testing Kotlin's ConflatedBroadcastChannel behavior - kotlin

In the new project that I'm currently working on I have no RxJava dependency at all, because until now I didn't need that - coroutines solve threading problem pretty gracefully.
At this point I stumbled upon on a requirement to have a BehaviorSubject-alike behavior, where one can subscribe to a stream of data and receive the latest value upon subscription. As I've learned, Channels provide very similar behavior in Kotlin, so I decided to give them a try.
From this article I've learned, that ConflatedBroadcastChannel is the type of channel that mimics BehaviorSubject, so I declared following:
class ChannelSender {
val channel = ConflatedBroadcastChannel<String>()
fun sendToChannel(someString: String) {
GlobalScope.launch(Dispatchers.Main) { channel.send(someString) }
}
}
For listening to the channel I do this:
class ChannelListener(val channelSender: ChannelSender) {
fun listenToChannel() {
channelSender.channel.consumeEach { someString ->
if (someString == "A") foo.perform()
else bar.perform()
}
}
}
This works as expected, but at this point I'm having difficulties understanding how to unit test ChannelListener.
I've tried to find something related here, but none of example-channel-**.kt classes were helpful.
Any help, suggestion or correction related to my incorrect assumptions is appreciated. Thanks.

With the help of Alexey I could manage to end up having following code, which answers the question:
class ChannelListenerTest {
private val val channelSender: ChannelSender = mock()
private val sut = ChannelListener(channelSender)
private val broadcastChannel = ConflatedBroadcastChannel<String>()
private val timeLimit = 1_000L
private val endMarker = "end"
#Test
fun `some description here`() = runBlocking {
whenever(channelSender.channel).thenReturn(broadcastChannel)
val sender = launch(Dispatchers.Default) {
broadcastChannel.offer("A")
yield()
}
val receiver = launch(Dispatchers.Default) {
while (isActive) {
val i = waitForEvent()
if (i == endMarker) break
yield()
}
}
try {
withTimeout(timeLimit) {
sut.listenToChannel()
sender.join()
broadcastChannel.offer(endMarker) // last event to signal receivers termination
receiver.join()
}
verify(foo).perform()
} catch (e: CancellationException) {
println("Test timed out $e")
}
}
private suspend fun waitForEvent(): String =
with(broadcastChannel.openSubscription()) {
val value = receive()
cancel()
value
}
}

Related

How to express in Kotlin "assign value exactly once on the first call"?

Looking for a natural Kotlin way to let startTime be initialized only in a particular place and exactly once.
The following naive implementation have two problems:
it is not thread safe
it does not express the fact "the variable was or will be assigned exactly once in the lifetime of an Item instance"
class Item {
var startTime: Instant?
fun start(){
if (startTime == null){
startTime = Instant.now()
}
// do stuff
}
}
I believe some kind of a delegate could be applicable here. In other words this code needs something similar to a lazy variable, but without initialization on first read, instead it happens only after explicit call of "touching" method. Maybe the Wrap calls could give an idea of possible implementation.
class Wrap<T>(
supp: () -> T
){
private var value: T? = null
private val lock = ReentrantLock()
fun get(){
return value
}
fun touch(){
lock.lock()
try{
if (value == null){
value = supp()
} else {
throw IllegalStateExecption("Duplicate init")
}
} finally{
lock.unlock()
}
}
}
How about combining AtomicReference.compareAndSet with a custom backing field?
You can use a private setter and make sure that the only place the class sets the value is from the start() method.
class Item(val value: Int) {
private val _startTime = AtomicReference(Instant.EPOCH)
var startTime: Instant?
get() = _startTime.get().takeIf { it != Instant.EPOCH }
private set(value) = check(_startTime.compareAndSet(Instant.EPOCH, value)) { "Duplicate set" }
fun start() {
startTime = Instant.now()
}
override fun toString() = "$value: $startTime"
}
fun main() = runBlocking {
val item1 = Item(1)
val item2 = Item(2)
println(Instant.now())
launch { println(item1); item1.start(); println(item1) }
launch { println(item1) }
delay(1000)
println(item2)
item2.start()
println(item2)
println(item2)
item2.start()
}
Example output:
2021-07-14T08:20:27.546821Z
1: null
1: 2021-07-14T08:20:27.607365Z
1: 2021-07-14T08:20:27.607365Z
2: null
2: 2021-07-14T08:20:28.584114Z
2: 2021-07-14T08:20:28.584114Z
Exception in thread "main" java.lang.IllegalStateException: Duplicate set
I think your Wrap class is a good starting point to implement this. I would definitely make it a property delegate and touch() could be much simplified:
fun touch() {
synchronized(this) {
check(value == null) { "Duplicate init" }
value = supp()
}
}
Then you can remove lock. But generally, this is a good approach.
If you would like to reuse lazy util from stdlib then you can do this by wrapping it with another object which does not read its value until asked:
class ManualLazy<T : Any>(private val lazy: Lazy<T>) {
operator fun getValue(thisRef: Any?, property: KProperty<*>): T? {
return if (lazy.isInitialized()) lazy.value else null
}
fun touch() {
lazy.value
}
}
class Item {
private val _startTime = ManualLazy(lazy { Instant.now() })
val startTime: Instant? by _startTime
fun start(){
_startTime.touch()
}
}
Of course, depending on your needs you can implement it in a much different way, using a similar technique.
This may be considered exploiting or hacking lazy util. I agree and I think Wrap approach is a better one.

CUBA Platform push messages from backend to UI

i was wondering if it is possible to send messages from the backend (for example a running task that receives information from an external system) to the UI. In my case it needs to be a specific session (no broadcast) and only on a specific screen
plan B would be polling the backend frequently but i was hoping to get something more "realtime"
I was trying to work something out like this, but i keep getting a NotSerializableException.
#Push
class StorageAccess : Screen(), MessageListener {
#Inject
private lateinit var stationWSService: StationWebSocketService
#Inject
private lateinit var notifications: Notifications
#Subscribe
private fun onInit(event: InitEvent) {
}
#Subscribe("stationPicker")
private fun onStationPickerValueChange(event: HasValue.ValueChangeEvent<StorageUnit>) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
val prevValue = event.prevValue
if (prevValue != null) {
stationWSService.remove(current.userSession.id)
}
val value = event.value ?: return
stationWSService.listen(current.userSession.id, value, this)
}
override fun messageReceived(message: String) {
val current = AppUI.getCurrent()
current.access {
notifications.create().withCaption(message).show()
}
}
#Subscribe
private fun onAfterDetach(event: AfterDetachEvent) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
stationWSService.remove(current.userSession.id)
}
}
-- The callback interface
interface MessageListener : Serializable {
fun messageReceived(message: String);
}
-- The listen method of my backend service
private val listeners: MutableMap<String, MutableMap<UUID, MessageListener>> = HashMap()
override fun listen(id: UUID, storageUnit: StorageUnit, callback: MessageListener) {
val unitStationIP: String = storageUnit.unitStationIP ?: return
if (!listeners.containsKey(unitStationIP))
listeners[unitStationIP] = HashMap()
listeners[unitStationIP]?.set(id, callback)
}
The Exception i get is NotSerializableException: com.haulmont.cuba.web.sys.WebNotifications which happens during adding the listener to the backend: stationWSService.listen(current.userSession.id, value, this)
as far as i understand this is the place where the UI sends the information to the backend - and with it the entire status of the class StorageAccess, including all its members.
is there an elegant solution to this?
regards
There is an add-on that solves exactly this problem: https://github.com/cuba-platform/global-events-addon

How can I override logRequest/logResponse to log custom message in Ktor client logging?

Currently, the ktor client logging implementation is as below, and it works as intended but not what I wanted to have.
public class Logging(
public val logger: Logger,
public var level: LogLevel,
public var filters: List<(HttpRequestBuilder) -> Boolean> = emptyList()
)
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
if (level.info) {
logger.log("REQUEST: ${Url(request.url)}")
logger.log("METHOD: ${request.method}")
}
val content = request.body as OutgoingContent
if (level.headers) {
logger.log("COMMON HEADERS")
logHeaders(request.headers.entries())
logger.log("CONTENT HEADERS")
logHeaders(content.headers.entries())
}
return if (level.body) {
logRequestBody(content)
} else null
}
Above creates a nightmare while looking at the logs because it's logging in each line. Since I'm a beginner in Kotlin and Ktor, I'd love to know the way to change the behaviour of this. Since in Kotlin, all classes are final unless opened specifically, I don't know how to approach on modifying the logRequest function behaviour. What I ideally wanted to achieve is something like below for an example.
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
...
if (level.body) {
val content = request.body as OutgoingContent
return logger.log(value("url", Url(request.url)),
value("method", request.method),
value("body", content))
}
Any help would be appreciative
No way to actually override a private method in a non-open class, but if you just want your logging to work differently, you're better off with a custom interceptor of the same stage in the pipeline:
val client = HttpClient(CIO) {
install("RequestLogging") {
sendPipeline.intercept(HttpSendPipeline.Monitoring) {
logger.info(
"Request: {} {} {} {}",
context.method,
Url(context.url),
context.headers.entries(),
context.body
)
}
}
}
runBlocking {
client.get<String>("https://google.com")
}
This will produce the logging you want. Of course, to properly log POST you will need to do some extra work.
Maybe this will be useful for someone:
HttpClient() {
install("RequestLogging") {
responsePipeline.intercept(HttpResponsePipeline.After) {
val request = context.request
val response = context.response
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}"
}
GlobalScope.launch(Dispatchers.Unconfined) {
val responseBody =
response.content.tryReadText(response.contentType()?.charset() ?: Charsets.UTF_8)
?: "[response body omitted]"
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}\nBODY START" +
"\n$responseBody" +
"\nBODY END"
}
}
}
}
}
You also need to add a method from the Ktor Logger.kt class to your calss with HttpClient:
internal suspend inline fun ByteReadChannel.tryReadText(charset: Charset): String? = try {
readRemaining().readText(charset = charset)
} catch (cause: Throwable) {
null
}

Kotlin flows as a message queue between coroutines

I'm attempting to use Kotlin's Flow class as a message queue to transfer data from a producer (a camera) to a set of workers (image analyzers) running on separate coroutines.
The producer in my case is a camera, and will run substantially faster than the workers. Back pressure should be handled by dropping data so that the image analyzers are always operating on the latest images from the camera.
When using channels, this solution works, but seems messy and does not provide an easy way for me to translate the data between the camera and the analyzers (like flow.map).
class ImageAnalyzer<Result> {
fun analyze(image: Bitmap): Result {
// perform some work on the image and return a Result. This can take a long time.
}
}
class CameraAdapter {
private val imageChannel = Channel<Bitmap>(capacity = Channel.RENDEZVOUS)
private val imageReceiveMutex = Mutex()
// additional code to make this camera work and listen to lifecycle events of the enclosing activity.
protected fun sendImageToStream(image: CameraOutput) {
// use channel.offer to ensure the latest images are processed
runBlocking { imageChannel.offer(image) }
}
#OnLifecycleEvent(Lifecycle.Event.ON_DESTROY)
fun onDestroy() {
runBlocking { imageChannel.close() }
}
/**
* Get the stream of images from the camera.
*/
fun getImageStream(): ReceiveChannel<Bitmap> = imageChannel
}
class ImageProcessor<Result>(workers: List<ImageAnalyzer<Result>>) {
private val analysisResults = Channel<Result>(capacity = Channel.RENDEZVOUS)
private val cancelMutex = Mutex()
var finished = false // this can be set elsewhere when enough images have been analyzed
fun subscribeTo(channel: ReceiveChannel<Bitmap>, processingCoroutineScope: CoroutineScope) {
// omit some checks to make sure this is not already subscribed
processingCoroutineScope.launch {
val workerScope = this
workers.forEachIndexed { index, worker ->
launch(Dispatchers.Default) {
startWorker(channel, workerScope, index, worker)
}
}
}
}
private suspend fun startWorker(
channel: ReceiveChannel<Bitmap>,
workerScope: CoroutineScope,
workerId: Int,
worker: ImageAnalyzer
) {
for (bitmap in channel) {
analysisResults.send(worker.analyze(bitmap))
cancelMutex.withLock {
if (finished && workerScope.isActive) {
workerScope.cancel()
}
}
}
}
}
class ExampleApplication : CoroutineScope {
private val cameraAdapter: CameraAdapter = ...
private val imageProcessor: ImageProcessor<Result> = ...
fun analyzeCameraStream() {
imageProcessor.subscribeTo(cameraAdapter.getImageStream())
}
}
What's the proper way to do this? I would like to use a ChannelFlow instead of a Channel to pass data between the camera and the ImageProcessor. This would allow me to call flow.map to add metadata to the images before they're sent to the analyzers. However, when doing so, each ImageAnalyzer gets a copy of the same image instead of processing different images in parallel. Is it possible to use a Flow as a message queue rather than a broadcaster?
I got this working with flows! It was important to keep the flows backed by a channel throughout this sequence so that each worker would pick up unique images to operate on. I've confirmed this functionality through unit tests.
Here's my updated code for posterity:
class ImageAnalyzer<Result> {
fun analyze(image: Bitmap): Result {
// perform some work on the image and return a Result. This can take a long time.
}
}
class CameraAdapter {
private val imageStream = Channel<Bitmap>(capacity = Channel.RENDEZVOUS)
private val imageReceiveMutex = Mutex()
// additional code to make this camera work and listen to lifecycle events of the enclosing activity.
protected fun sendImageToStream(image: CameraOutput) {
// use channel.offer to enforce the drop back pressure strategy
runBlocking { imageChannel.offer(image) }
}
#OnLifecycleEvent(Lifecycle.Event.ON_DESTROY)
fun onDestroy() {
runBlocking { imageChannel.close() }
}
/**
* Get the stream of images from the camera.
*/
fun getImageStream(): Flow<Bitmap> = imageChannel.receiveAsFlow()
}
class ImageProcessor<Result>(workers: List<ImageAnalyzer<Result>>) {
private val analysisResults = Channel<Result>(capacity = Channel.RENDEZVOUS)
private val cancelMutex = Mutex()
var finished = false // this can be set elsewhere when enough images have been analyzed
fun subscribeTo(flow: Flow<Bitmap>, processingCoroutineScope: CoroutineScope): Job {
// omit some checks to make sure this is not already subscribed
return processingCoroutineScope.launch {
val workerScope = this
workers.forEachIndexed { index, worker ->
launch(Dispatchers.Default) {
startWorker(flow, workerScope, index, worker)
}
}
}
}
private suspend fun startWorker(
flow: Flow<Bitmap>,
workerScope: CoroutineScope,
workerId: Int,
worker: ImageAnalyzer
) {
while (workerScope.isActive) {
flow.collect { bitmap ->
analysisResults.send(worker.analyze(bitmap))
cancelMutex.withLock {
if (finished && workerScope.isActive) {
workerScope.cancel()
}
}
}
}
}
fun getAnalysisResults(): Flow<Result> = analysisResults.receiveAsFlow()
}
class ExampleApplication : CoroutineScope {
private val cameraAdapter: CameraAdapter = ...
private val imageProcessor: ImageProcessor<Result> = ...
fun analyzeCameraStream() {
imageProcessor.subscribeTo(cameraAdapter.getImageStream())
}
}
It appears that, so long as the flow is backed by a channel, the subscribers will each get a unique image.

How to suspend kotlin coroutine until notified

I would like to suspend a kotlin coroutine until a method is called from outside, just like the old Java object.wait() and object.notify() methods. How do I do that?
Here: Correctly implementing wait and notify in Kotlin is an answer how to implement this with Kotlin threads (blocking). And here: Suspend coroutine until condition is true is an answer how to do this with CompleteableDeferreds but I do not want to have to create a new instance of CompleteableDeferred every time.
I am doing this currently:
var nextIndex = 0
fun handleNext(): Boolean {
if (nextIndex < apps.size) {
//Do the actual work on apps[nextIndex]
nextIndex++
}
//only execute again if nextIndex is a valid index
return nextIndex < apps.size
}
handleNext()
// The returned function will be called multiple times, which I would like to replace with something like notify()
return ::handleNext
From: https://gitlab.com/SuperFreezZ/SuperFreezZ/blob/master/src/superfreeze/tool/android/backend/Freezer.kt#L69
Channels can be used for this (though they are more general):
When capacity is 0 – it creates RendezvousChannel. This channel does not have any buffer at all. An element is transferred from sender to receiver only when send and receive invocations meet in time (rendezvous), so send suspends until another coroutine invokes receive and receive suspends until another coroutine invokes send.
So create
val channel = Channel<Unit>(0)
And use channel.receive() for object.wait(), and channel.offer(Unit) for object.notify() (or send if you want to wait until the other coroutine receives).
For notifyAll, you can use BroadcastChannel instead.
You can of course easily encapsulate it:
inline class Waiter(private val channel: Channel<Unit> = Channel<Unit>(0)) {
suspend fun doWait() { channel.receive() }
fun doNotify() { channel.offer(Unit) }
}
It is possible to use the basic suspendCoroutine{..} function for that, e.g.
class SuspendWait() {
private lateinit var myCont: Continuation<Unit>
suspend fun sleepAndWait() = suspendCoroutine<Unit>{ cont ->
myCont = cont
}
fun resume() {
val cont = myCont
myCont = null
cont.resume(Unit)
}
}
It is clear, the code have issues, e.g. myCont field is not synchonized, it is expected that sleepAndWait is called before the resume and so on, hope the idea is clear now.
There is another solution with the Mutex class from the kotlinx.coroutines library.
class SuspendWait2 {
private val mutex = Mutex(locaked = true)
suspend fun sleepAndWait() = mutex.withLock{}
fun resume() {
mutex.unlock()
}
}
I suggest using a CompletableJob for that.
My use case:
suspend fun onLoad() {
var job1: CompletableJob? = Job()
var job2: CompletableJob? = Job()
lifecycleScope.launch {
someList.collect {
doSomething(it)
job1?.complete()
}
}
lifecycleScope.launch {
otherList.collect {
doSomethingElse(it)
job2?.complete()
}
}
joinAll(job1!!, job2!!) // suspends until both jobs are done
job1 = null
job2 = null
// Do something one time
}