Kotlin Flow: Testing hangs - kotlin

I am trying to test Kotlin implementation using Flows. I use Kotest for testing. This code works:
ViewModel:
val detectedFlow = flow<String> {
emit("123")
delay(10L)
emit("123")
}
Test:
class ScanViewModelTest : StringSpec({
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
launch {
vm.detectedFlow.collect {
it shouldBe detectedString
}
}
}
})
However, in the real ViewModel I need to add values to the flow, so I use ConflatedBroadcastChannel as follows:
private val _detectedValues = ConflatedBroadcastChannel<String>()
val detectedFlow = _detectedValues.asFlow()
suspend fun sendDetectedValue(detectedString: String) {
_detectedValues.send(detectedString)
}
Then in the test I try:
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
runBlocking {
vm.sendDetectedValue(detectedString)
}
runBlocking {
vm.detectedFlow.collect { it shouldBe detectedString }
}
}
The test just hangs and never completes. I tried all kind of things: launch or runBlockingTest instead of runBlocking, putting sending and collecting in the same or separate coroutines, offer instead of send... Nothing seems to fix it. What am I doing wrong?
Update: If I create flow manually it works:
private val _detectedValues = ConflatedBroadcastChannel<String>()
val detectedFlow = flow {
this.emit(_detectedValues.openSubscription().receive())
}
So, is it a bug in asFlow() method?

The problem is that the collect function you used in your test is a suspend function that will suspend the execution until the Flow is finished.
In the first example, your detectedFlow is finite. It will just emit two values and finish. In your question update, you are also creating a finite flow, that will emit a single value and finish. That is why your test works.
However, in the second (real-life) example the flow is created from a ConflatedBroadcastChannel that is never closed. Therefore the collect function suspends the execution forever. To make the test work without blocking the thread forever, you need to make the flow finite too. I usually use the first() operator for this. Another option is to close the ConflatedBroadcastChannel but this usually means modifications to your code just because of the test which is not a good practice.
This is how your test would work with the first() operator
"when the flow contains values they are emitted" {
val detectedString = "123"
val vm = ScanViewModel()
runBlocking {
vm.sendDetectedValue(detectedString)
}
runBlocking {
vm.detectedFlow.first() shouldBe detectedString
}
}

Related

Flows in Kotlin not collecting all values

I'm doing some exercises to learn Flows in Kotlin, and I found some issues which I cannot understand.
When using a MutableStateFlow, in the next example it only prints the number 3. I would expect to print 0 to 3 instead. One could say that maybe is going too fast, or I should put a delay, but this seems to me a patch if such is the case, since if it is true that if sending MutableStateFlow data too fast makes it skip some values, then is something to consider every single time when using it.
val flow = MutableStateFlow<Int>(0)
fun main(): Unit = runBlocking {
launch {
flow.collect {
println(it)
}
}
(0..3).forEach {
flow.emit(it)
}
}
// Expected to print 0, 1, 2, 3
// Printing only 3
Next, I tried to use a MutableSharedFlow instead, but it emits nothing at all, not even 3. Same code as above but replacing the flow with:
val flow = MutableSharedFlow<Int>()
MutableStateFlow cannot be used here because its behavior does not allow to get every value
so I used SharedFlow
Example with SharedFlow:
val flow = MutableSharedFlow<Int>()
fun main(): Unit = runBlocking {
val scope = // scope
flow
.onEach {
println(it)
}
.launchIn(scope)
(0..3).forEach {
flow.emit(it)
}
// delay to wait for println
launch {
delay(10000)
}
}

Flow message not delivered in unit test

I have a consumer that reads messages off MutableSharedFlow (which acts as an EventBus in my application). I am trying to write a unit test to show that passing a message into the Flow triggers my Listener.
This is my Flow definition:
class MessageBus {
private val _messages = MutableSharedFlow<Message>()
val messages = _messages.asSharedFlow()
suspend fun send(message: Message) {
_messages.emit(message)
}
}
Here is the Listener:
class Listener(private val messageBus: MessageBus) {
private val scope = CoroutineScope(Dispatchers.IO + SupervisorJob())
init {
scope.launch {
messageBus.messages.collectLatest { message ->
when (message) {
is CustomMessage -> handleCustomMessage(message)
}
}
}
}
And finally here is my unit test:
class CommandTest {
#Test
fun `should process CustomMessage`(): Unit = runBlocking {
val messageBus = MessageBus()
val listener = Listener(messageBus)
messageBus.send(CustomMessage("test command"))
//argumentCaptor...verify[removed for brevity]
}
}
Unfortunately the above code does not trigger the break point in my Listener (breakpoint on line init is triggered, but a message is never received and no breakpoints triggered in the collectLatest block).
I even tried adding a Thread.sleep(5_000) before the verify statement but the result is the same. Am I missing something obvious with how coroutines work?
Edit: if it matters this is not an Android project. Simply Kotlin + Ktor
I imagine that since the code is in the init block in the Listener once you initialize val listener = Listener(messageBus, this) in the test it reads all messages and at this point you have none then in the next line you emit a message messageBus.send(CustomMessage("test command")) but your launch block should have finished by then. You can emit the message first or place your launch in an loop or in a different method that can be called after you emit the message
First of all I would recomend reading this article about how to test flows in Android.
Secondly in your example the issues arise from having the scope inside the Listener hardcoded. You should pass the scope as a parameter and inject it in the test:
class Listener(private val messageBus: MessageBus, private val scope: CoroutineScope = CoroutineScope(Dispatchers.IO + SupervisorJob()))
class CommandTest {
#Test
fun `should process CustomMessage`(): Unit = runBlockingTest {
val messageBus = MessageBus()
val listener = Listener(messageBus, this)
messageBus.send(CustomMessage("test command"))
//argumentCaptor...verify[removed for brevity]
}
}
I would also recomend using runBlockingTest instead of runBlocking so your tests don't have to actually wait. It will also fail in case any coroutines are left running once the test finishes.
You could use something like this
class Emitter {
private val emitter: MutableSharedFlow<String> = MutableSharedFlow()
suspend fun publish(messages: Flow<String>) = messages.onEach {
emitter.emit(it)
}.collect()
fun stream(): Flow<String> = emitter
}
the collect at the end of your onEach will be used to trigger the collection initially as a terminal operation... I need further understanding on emit because it does not work as I expect in all cases and when used in this way you have initially it does not post anything in your Flow unless you collect first to process
Then in your collector itself
class Collector {
suspend fun collect(emitter: Emitter): Unit = coroutineScope {
println("Starting collection...")
emitter.stream().collect { println("collecting message: $it") }
}
}
then your main (or test)
fun main() = runBlocking {
withContext(Dispatchers.Default + Job()) {
val emitter = Emitter()
val collector = Collector()
launch {
collector.collect(emitter)
}
emitter.publish(listOf("article#1", "article#2", "article#3", "article#4").asFlow())
}
}
output:
Starting collection...
collecting message: article#1
collecting message: article#2
collecting message: article#3
collecting message: article#4

Questions on recursive coroutines in Kotlin

Recently I've been trying to familiarize myself with Kotlin some more, so I decided to write a webscraper utilizing coroutines. What I want to accomplish is pull each page, harvest it for links and contents or posts, then feed the links back to the process, until there is nowhere left to go. As of now it has some obvious shortcomings, such no delay enforced between calls or saving addresses and only visiting new ones. But the questions I have are regarding coroutines, here.
Consider the following class,. I've added some toy classes to simulate how it is intended to work, which I won't detail, but you can imagine how they work.
class Scraper(
private val client: Client = ToyClient(delayMillis = 1000, alwaysFindBody = "Test body"),
private val extraction: Extraction = ToyExtraction(
alwaysFindLinks = listOf("https://google.com"),
alwaysFindPosts = listOf("Test post")
),
private val repository: Repository = ToyRepository()
) {
// I could manage my own coroutine scope's lifecycle, but how would I go about this?
// private val scope = CoroutineScope(Dispatchers.Default + SupervisorJob())
private val seed = "https://google.com"
private val log = KotlinLogging.logger {}
fun start() = runBlocking {
log.info { "Scraping started!" }
scrape(seed).join()
log.info { "Scraping finished!" }
}
private fun CoroutineScope.scrape(address: String): Job = launch(Dispatchers.Default) {
log.info { "A scraping coroutine has started" }
val page = request(address)
val contents = extract(page)
save(contents)
contents.links.forEach { scrape(it) }
// Job would not progress here after submitting new jobs, only after each children have been completed
// log.info { "A scraping coroutine has finished" }
}
private suspend fun request(address: String): Page {
log.info { "Getting page: $address" }
return client.get(address)
}
private suspend fun extract(page: Page): PageContents {
log.info { "Extracting page: ${page.address}" }
return extraction.extract(page)
}
private suspend fun save(contents: PageContents) {
log.info { "Processing contents of: $contents" }
repository.save(contents.posts)
}
}
The main recursive operation is CoroutineScope.scrape() which launches a job, which itself can launch children jobs as well and so on.
My main questions are:
If I were to manage the scope myself as a property, how could I do that and achieve the same behavior? That is, I would wait for all dynamically spawned jobs to complete as well, return when all are finished.
I wrote my webclient's function using a 3rd party library as such:
fun suspend get(address: String): Page { ... }
Am I fine just marking this method as suspend to get all benefits from this in terms of coroutines?
Thanks in advance!
You don't even need a scope for that, launch a top-level job and use job.join() to await until it and all its children are done. If you want to block while waiting for that to happen, then you are already doing it right by using runBlocking.
No, marking a function as suspend doesn't affect its blocking behavior. It only allows the function to suspend itself, which must be explicit either in your code or the code you're calling into.

Why does a normal function need to be wrapped with viewModelScope.launch?

The following code is from the project.
1: In my mind,a suspend fun should be launched in another suspend fun or viewModelScope.launch{ }, withContext{ } ... , filterItems() is only a normal function, I don't know why filterItems() need to be wrapped with viewModelScope.launch{ } in the function filterTasks(), could you tell me ?
2: In the function filterTasks(), viewModelScope.launch{ } will launch in coroutines, it's asynchronous, I think return result maybe be launched before I get the result from viewModelScope.launch{}, so the result maybe null, is the code correct?
Code
private fun filterTasks(tasksResult: Result<List<Task>>): LiveData<List<Task>> {
val result = MutableLiveData<List<Task>>()
if (tasksResult is Success) {
isDataLoadingError.value = false
viewModelScope.launch {
result.value = filterItems(tasksResult.data, getSavedFilterType())
//return filterItems(tasksResult.data, getSavedFilterType()) //It will cause error.
}
} else {
result.value = emptyList()
showSnackbarMessage(R.string.loading_tasks_error)
isDataLoadingError.value = true
}
return result //I think it maybe be launched before I get the result from viewModelScope.launch{}
}
private fun filterItems(tasks: List<Task>, filteringType: TasksFilterType): List<Task> {
val tasksToShow = ArrayList<Task>()
// We filter the tasks based on the requestType
for (task in tasks) {
when (filteringType) {
ALL_TASKS -> tasksToShow.add(task)
ACTIVE_TASKS -> if (task.isActive) {
tasksToShow.add(task)
}
COMPLETED_TASKS -> if (task.isCompleted) {
tasksToShow.add(task)
}
}
}
return tasksToShow
}
It doesn't, unless it performs some heavy work and you want to move it to a background thread, which is the case here. Here the author just wanted to disjoint the work so the live data can be updated with an empty list first, and the filtered list later(computationally intensive to get), but forgot to do it out of the main thread.
In this particular case the author may have forgotten to add a background dispatcher as a parameter
viewModelScope.launch(Dispatchers.Default)
hence, in this scenario the intended behavior was not achieved, so you see this "nonsensical" coroutine.
I think you can contribute to the project with a fix :)
yes, you are right. but if you looked up the implementation of the launch {} such in lifecycleScope.launch {} or viewModelScope.launch {} you would find out the "block" which is "the coroutine code which will be invoked in the context of the provided scope" is cast to be suspend, so any block of code between launch {} is suspend code block. so in your example filterItems is cast to suspend under the hood and it's wrapped with viewModelScope.launch{ } to do its heavy task not in main thread.
public fun CoroutineScope.launch(
context: CoroutineContext = EmptyCoroutineContext,
start: CoroutineStart = CoroutineStart.DEFAULT,
// the below line is doing the magic
block: suspend CoroutineScope.() -> Unit
): Job {
val newContext = newCoroutineContext(context)
val coroutine = if (start.isLazy)
LazyStandaloneCoroutine(newContext, block) else
StandaloneCoroutine(newContext, active = true)
coroutine.start(start, coroutine, block)
return coroutine
}
I agree that the code looks suspicious, main reason is that it launches the filterItems coroutine into the Main dispatcher, basically just postponing the moment when filterItems will run on the GUI thread. If filterItems takes long to complete, it will block the GUI; if it doesn't take long, then why would you launch a concurrent coroutine in the first place?
Furthermore, on an architectural level, I don't see a reason why you'd have a function returning LiveData<List<Task>> when you can just have a suspend fun returning List<Task>.

Kotlin Coroutines - How to block to await/join all jobs?

I am new to Kotlin/Coroutines, so hopefully I am just missing something/don't fully understand how to structure my code for the problem I am trying to solve.
Essentially, I am taking a list of strings, and for each item in the list I want to send it to another method to do work (make a network call and return data based on the response). (Edit:) I want all calls to launch concurrently, and block until all calls are done/the response is acted on, and then return a new list with the info of each response.
I probably don't yet fully understand when to use launch/async, but I've tried to following with both launch (with joinAll), and async (with await).
fun processData(lstInputs: List<String>): List<response> {
val lstOfReturnData = mutableListOf<response>()
runBlocking {
withContext(Dispatchers.IO) {
val jobs = List(lstInputs.size) {
launch {
lstOfReturnData.add(networkCallToGetData(lstInputs[it]))
}
}
jobs.joinAll()
}
}
return lstofReturnData
What I am expecting to happen, is if my lstInputs is a size of 120, when all jobs are joined, my lstOfReturnData should also have a size of 120.
What actually is happening is inconsitent results. I'll run it once, and I get 118 in my final list, run it again, it's 120, run it again, it's 117, etc. In the networkCallToGetData() method, I am handling any exceptions, to at least return something for every request, regardless if the network call fails.
Can anybody help explain why I am getting inconsistent results, and what I need to do to ensure I am blocking appropriately and all jobs are being joined before moving on?
mutableListOf() creates an ArrayList, which is not thread-safe.
Try using ConcurrentLinkedQueue instead.
Also, do you use the stable version of Kotlin/Kotlinx.coroutine (not the old experimental one)? In the stable version, with the introduction of structured concurrency, there is no need to write jobs.joinAll anymore. launch is an extesion function of runBlocking which will launch new coroutines in the scope of the runBlocking and the runBlocking scope will automatically wait for all the launched jobs to finsish. So the code above can be shorten to
val lstOfReturnData = ConcurrentLinkedQueue<response>()
runBlocking {
lstInputs.forEach {
launch(Dispatches.IO) {
lstOfReturnData.add(networkCallToGetData(it))
}
}
}
return lstOfReturnData
runBlocking blocks current thread interruptibly until its completion. I guess it's not what you want. If I think wrong and you want to block the current thread than you can get rid of coroutine and just make network call in the current thread:
val lstOfReturnData = mutableListOf<response>()
lstInputs.forEach {
lstOfReturnData.add(networkCallToGetData(it))
}
But if it is not your intent you can do the following:
class Presenter(private val uiContext: CoroutineContext = Dispatchers.Main)
: CoroutineScope {
// creating local scope for coroutines
private var job: Job = Job()
override val coroutineContext: CoroutineContext
get() = uiContext + job
// call this to cancel job when you don't need it anymore
fun detach() {
job.cancel()
}
fun processData(lstInputs: List<String>) {
launch {
val deferredList = lstInputs.map {
async(Dispatchers.IO) { networkCallToGetData(it) } // runs in parallel in background thread
}
val lstOfReturnData = deferredList.awaitAll() // waiting while all requests are finished without blocking the current thread
// use lstOfReturnData in Main Thread, e.g. update UI
}
}
}
Runblocking should mean you don't have to call join.
Launching a coroutine from inside a runblocking scope should do this for you.
Have you tried just:
fun processData(lstInputs: List<String>): List<response> {
val lstOfReturnData = mutableListOf<response>()
runBlocking {
lstInputs.forEach {
launch(Dispatchers.IO) {
lstOfReturnData.add(networkCallToGetData(it))
}
}
}
return lstofReturnData