I am having some trouble launching coroutines in parallel.
I have some code that makes a network request in a coroutine, this network request returns a collection of intermediate results, which I would like to fetch the complete result for each item in this collection in parallel (i.e. not sequentially).
Here is the relevant API's:
data class IntermediateResult(val id: Long)
data class FinalResult(val id: Long, val name: String)
suspend fun getIntermediateResults() : Collection<IntermediateResult>
suspend fun getFinalResult(id: Long) : FinalResult
And here is the code that is consuming the API's (it's inside a ViewModel):
fun example() {
viewModelScope.launch(Dispatchers.IO) {
val intermediateResults = getIntermediateResults()
intermediateResults.forEach { intermediateResult ->
viewModelScope.launch(Dispatchers.IO) {
val finalResult = getFinalResult(intermediateResult.id)
// Update cache and post updated cache to the UI via live data
}
}
}
}
It was my understanding that launching a new coroutine inside the forEach loop would cause the final result to be fetched for each item in a parallel fashion (i.e. it won't "wait" on the getFinalResult before continuing to the next iteration of the forEach) but, what I am observing is that it's happening sequentially (i.e. the call to getFinalResult in the forEach loop is causing the code to "wait" until the result comes back before starting the next iteration).
In other words, I don't care about the final results coming back in the order of iteration over the intermediate results.
If it's relevant (I'm fairly sure it's not) the implementation of the 2 suspend functions is using suspendCoroutine to wrap up old callback style code into suspend functions.
Related
In my ViewModel I have a function that is called from UI. It can be called many times per second.
Data comes from barcode scanner ViewModel. I'm passing it from one ViewModel to another thru UI for simplicity and to reuse barcode scanner ViewModel.
For simplicity lets assume that I have something like this:
// called from the fragment (that observes some property on the another viewmodel)
public fun processScannedBarcode(barcode : String){
// process data after necessaryData is ready (not null?)
val item = findItemByBarcode(barcode)
}
private var dataFromApi: ArrayList<Item>?
private fun loadNecessaryDataFromTheApi(){
viewModelScope.launch {
canReload = false
dataFromApi = myapi.getDataFromApi() // suspend fun inside
canReload = true
}
}
// called from the fragment button click
public fun reloadNecessaryDataFromTheApi(){
loadNecessaryDataFromTheApi()
}
init {
loadNecessaryDataFromTheApi()
}
My data required to process may be not ready yet because it comes from the API. I have to deferr that processing and resume when data is ready.
I could simply solve this with some Queue and a Thread, but maybe it is possible to do that with Kotlin coroutines?
As your loadNecessaryDataFromTheApi() only sets properties, it is not that easy to observe it or wait for it. You need to use some synchronization utility like locks, channels, etc. In this case it will be probably the easiest to keep a Deferred of the necessary data:
public suspend fun processScannedBarcode(barcode : String){
val data = dataFromApi.await()
val item = findItemByBarcode(barcode)
}
private val dataFromApi = CompletableDeferred<ArrayList<Item>>()
private fun loadNecessaryDataFromTheApi(){
viewModelScope.launch {
canReload = false
dataFromApi.complete(myapi.getDataFromApi())
canReload = true
}
}
Some notes:
We have to make processScannedBarcode suspend if it is going to wait for something.
If you want reloadNecessaryDataFromTheApi() to defer processing barcodes again, simply replace dataFromApi with a new deferred.
I know this question has been asked a couple of times, but I have a funny situation here that I can't figure out.
I have a database with "TASKS" and, apart from the writing/updating methods, I'm calling the get method twice: one for getting the last created task, and another to select an specific task by ID
#Query("SELECT * FROM tasks_history_table ORDER BY taskId DESC LIMIT 1")
suspend fun getCurrentTask(): Task2?
#Query("SELECT * from tasks_history_table WHERE taskId = :key ")
suspend fun get(key: Long): Task2?
Then in the viewModel I'm launching a coroutine for each time I call one of these methods.
This one for the last created Task:
private fun initializeCurrentTask(){
viewModelScope.launch {
_currentTask.value = getCurrentTaskFromDatabase()!!
}
}
suspend fun getCurrentTaskFromDatabase(): Task2? {
var currentTask = database.getCurrentTask()
return currentTask
}
And this one for the specific task
fun initializeSelectedTask(){
viewModelScope.launch {
_currentTask.value = getSelectedTaskFromDatabase(selectedTaskId.value!!)!!
}
}
suspend fun getSelectedTaskFromDatabase(taskId: Long): Task2? {
var currentTask = database.get(selectedTaskId.value!!)!!
return currentTask
}
So they are both pretty much the same, except for the parameter Id passed.
Then, I'm sending that data to the Fragment to update the UI, via LiveData
private val _currentTask = MutableLiveData<Task2>()
val currentTask : LiveData<Task2>
get() = _currentTask
And here the observer:
timerViewModel.currentTask.observe(viewLifecycleOwner) {
updateUIText()
updateCountdownUI()
updateAnimation()
}
Everytime I call the function to get the las saved task, the observers are called and everything works fine. But whenever I call the function to get a specific task by Id, the observers are not called.
I've set Logs all around and I've reached the conclusion that the LiveData is getting updated, but the observer is not triggered.
Here's the repo in case someone can have it a look. Thanks!!
https://github.com/arieldipietro/PomodoroTechnique
In FragmentTimer and FragmentHistory, you have created viewModel instances using their respective fragments as owners, which makes them to observe liveData which are triggered by their instances only. So, now when you trigger a task from FragmentHistory it isn't get observed in FragmentTimer.
You need to use SharedViewModel for passing data between fragments, you have to create object of TimerViewModel using activity as its owner, then you would be able to observe from FragmentTimer. Pass requireActivity() as the owner in ViewModelProvider's constructor.
timerViewModel = ViewModelProvider(requireActivity(), viewModelFactory)[TimerViewModel::class.java]
You can read more about it in this codelab tutorial.
Room executes queries that return LiveData on a background thread automatically. But I want to return a single value that is not wrapped into LiveData (because I don't want live updates).
How do I implement this using coroutines?
How do I return the Task object from this function?
fun getTask(id: Int): Task {
viewModelScope.launch {
repository.getTask(id)
}
}
This function is inside the ViewModel. It forwards the call down to the DAO:
#Query("SELECT * FROM task_table WHERE id = :id")
fun getTask(id: Int): Task
If you don't return a LiveData from Room you won't get updates from the DB. You can however return a LiveData from your viewModel.
val data = liveData {
emit(repository.getTask(id))
}
The liveData extension function runs in a coroutine and then you can use the suspend version of your DAO to handle backgrounding properly.
#Query("SELECT * FROM task_table WHERE id = :id")
suspend fun getTask(id: Int): Task?
A big thing you need to do is make sure it is nullable if you aren't using an aggregate function in your query.
If you are really wanting to call the method in your viewModel to return the task you should run the launch from your activity/fragment (not recommended)
ViewModel
suspend fun getTask(id: Int): Task {
repository.getTask(id)
}
Activity/Fragment
lifecycleScope.launch {
val task = viewModel.getTask(id)
// Do What you want with the task
}
Flow is the new LiveData!
I had a similar problem in two project before, each solved differently. But recently I learnt to use Flow and it appears that it is the cleanest way yet.
Alternative to LiveData
If you don't need to use LiveData you have two option:
Retrieving a Cursor by a #query, suitable for refactoring old projects.
Using Flow, suitable for new projects.
Retrieving only one object/value
LiveDate: You can unsubscribe from the LiveData, remove the observer after the first fetch. Not clean way in my opinion.
Flow: You can retrive just a single object/value if you want and then stop the flow collecting.
Dao:
getTask(): this method return a Flow<Task>:
#Query("SELECT * FROM task_table WHERE id = :id")
fun getTask(id: Int): Flow<Task>
ViewModel:
getTask(): return a Task object (not Flow<Task>), also it is suspend function.
first() The terminal operator that returns the first element emitted by the
flow and then cancels flow’s collection. Throws NoSuchElementException
if the flow was empty.
suspend fun getTask(id: Int): Task {
return dao.getTask(id).first()
}
Fragment/Activity:
Properties:
private var viewModelJob = Job()
private val uiScope = CoroutineScope(Dispatchers.Main + viewModelJob)
don't forget to cancel viewModelJob when fragment/activity not needed aka onClear/onDestory/... so that all coroutines tight to this is canceled.
Usage
Now whenever we want to retrieve our object Task from that suspended function we need to be inside a suspend or coroutine. Therefore using launch builder to create a coroutine is suitable here (since we don't want any return object from that builder we only want to run a suspend function, otherwise async to return a deferred).
onCreate() / onCreateView()
.
..
...
uiScope.launch() {
// Here are the Task & Main-UI
val task = viewModel.getTask(1)
binding.taskTitleTextView.text = task.title
}
If we don't use first() then we need to collect the flow viewModel.getTasks().collect{ it } there are many useful function at kotlinx.coroutines.flow. Flow is the best thing that happen in Coroutine Package, and oh sorry that I pass the repository layer, it is just a duplicated for viewModel in most cases 😅.
Suspend functions in Room are main-safe and run on a custom
dispatcher. Same as LiveData as you mentioned in your question. Below is the example to achieve the same
Inside some function in viewmModel class
viewModelScope.launch {
// suspend and resume make this database request main-safe
// so our ViewModel doesn't need to worry about threading
someLiveData.value =
repository.getSomething()
}
In repository class
suspend fun getSomething(): List<Something> {
return dao.getSomething()
}
In Dao class
#Query("select * from tableName")
suspend fun getSomething(): List<Something>
One of the workaround would be to return Deferred object immediately and then call .await() on the return deferred
fun getTaskAsync(id: Int): Deferred<Task> = viewModelScope.async {
repository.getTask(id)
}
//call-site
getTaskAsync(id).await() // <- this is suspension point
why is it not possible to call Deferred::await in the Sequence::map function like it is possible in List::map?
I made a small example
fun example() = runBlocking {
val list = listOf(1, 2, 3, 4)
list.map { async { doSomething(it) } }
.map { it.await() }
list.asSequence()
.map { async { doSomething(it) } }
.map { it.await() } // Error: Kotlin: Suspension functions can be called only within coroutine body
}
As you can see the last statement does not compile.
It's because list.map is an inline fun, which it can be because it's an eager operation that returns a new list. An inline fun can tolerate a suspend block because it's more like a macro: it gets expanded into the call site. As long as the call site is within a suspend block, this works fine.
sequence.map, on the other hand, is lazy and it just adds another lambda to the pipeline of operations that will be performed eventually, when you pull items from the sequence.
The closest match to a lazy sequence in the suspendable world is Flow, but it has a different model where you get all the data pushed to you in one go, whereas you can pull items from a lazy sequence one by one.
Adding to Marko's answer, which is correct:
Because the sequence is lazy, the await() call might happen after runBlocking is done, essentially. Neither the async call nor the await will happen until you start pulling elements off the list.
And in the case of the code above, nothing pulls the elements out of the sequence, so neither of the mapping operations on the sequence would happen inside the runBlocking block.
I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}