I am trying to make an API call multiple times, passing in a different parameter. When no more data is returned, the rx stream should terminate. After each call, the data is stored in my local repository. Here is what I have:
val startPositions = BehaviorSubject.createDefault(0)
startPositions.flatMap { startPos -> App.context.repository.getConnections(startPos) }
.flatMap { connections -> App.context.repository.storeConnections(connections) }
.doOnNext { startPos -> startPositions.onNext(startPos + 1) }
.subscribe({ startPos -> println("Index $startPos") })
Here is the api method to handle downloading the data:
override fun getConnections(startPos: Int): Observable<List<Connection>> {
return myAPI.getConnections(startPos)
}
And this is the api method for storing the data:
override fun storeConnections(connections: List<Connection>): Completable =
Completable.fromAction {
appDao.storeConnections(connections.map {
mapper.toDb(it)
})
}
The compile error I get is:
Type mismatch: inferred type is (List) -> Completable but ((List) -> ObservableSource!)! was expected
If possible, I don't want to change the return types of my api calls. I'm also not certain about using a flatMap. The data returned from downloading is a list and I want the list to remain as a list in the stream. I don't want to emit individual list items.
In order to fix the type mismatch error, use operator flatMapCompletable instead of flatMap:
.flatMapCompletable { connections -> App.context.repository.storeConnections(connections) }
Then you will have to use doOnComplete instead of doOnNext:
.doOnComplete { startPos -> startPositions.onNext(startPos + 1) }
Related
I have a method which takes a list of object (Widget) -- which contains some properties (header) and nested list(component). I want to flatten the list into a single list and have the below code for same:
#SuppressLint("CheckResult")
fun flatten(fatList: Single<List<Widget>>) {
val flatList: MutableList<IUiData> = mutableListOf()
fatList.map {
Observable.fromIterable(it).map { widget ->
if (widget.header.isNotEmpty()) {
flatList.add(ProductHeaderUi(widget.header))
}
widget.componentList.map { component ->
when (component.type) {
TILE_TEXT -> {
flatList.add(HeaderUi(component))
}
TILE_IMAGE -> {
flatList.add(ImageTileUi(component))
}
TILE_FOOTER -> {
flatList.add(FooterUi(component))
}
UNKNOWN -> {
//Do Nothing
}
}
}
}
}
}
I intend to return a Single of List: Single<MutableList<IUiData>> from this method, this purpose can be served right now, but I am looking for a cleaner way
You're using both Rx's Observable map and Kotlin's Iterable map in an unintended way. They are for converting one type to another, not for iterating something.
You've also nested an unnecessary Observable iterable inside the outer-most map function.
You only need to map the output of the Single. Inside the map function, you iterate (not map) the original List to pull out the data you need for the MutableList.
I'm an Rx novice and didn't check this, so sorry about any syntax errors.
fun flatten(fatList: Single<List<Widget>>): Single<MutableList<IUData>> = fatList.map { widgetList ->
val flatList: MutableList<IUiData> = mutableListOf()
for (widget in widgetList) {
if (widget.header.isNotEmpty()) {
flatList.add(ProductHeaderUi(widget.header))
}
for (component in widget.componentList) {
when (component.type) {
TILE_TEXT -> flatList.add(HeaderUi(component))
TILE_IMAGE -> flatList.add(ImageTileUi(component))
TILE_FOOTER -> flatList.add(FooterUi(component))
// Else do nothing
}
}
}
flatList
}
But in keeping with typical Rx chaining syntax, I would make it an extension function, so I'd have to first line like this. Then you can put it right in the middle of an Rx call chain:
fun Single<List<Widget>>.flatten(): Single<MutableList<IUData>> = map { widgetList ->
You can also do this in a more concise, functional, but less efficient way by using Kotlin's flatMap:
fun Single<List<Widget>>.flatten(): Single<MutableList<IUData>> = map {
it.flatMap { widget ->
listOfNotNull(widget.header.takeIf(Header::isNotEmpty)?.let(::ProductHeaderUi))
+
widget.componentList.mapNotNull { component ->
when (component.type) {
TILE_TEXT -> HeaderUi(component)
TILE_IMAGE -> ImageTileUi(component)
TILE_FOOTER -> FooterUi(component)
else -> null
}
}.toMutableList()
}
...where Header is whatever type widget.header uses.
I have a situation where I need to observe userIds then use those userIds to observe users. Either userIds or users could change at any time and I want to keep the emitted users up to date.
Here is an example of the sources of data I have:
data class User(val name: String)
fun observeBestUserIds(): Flow<List<String>> {
return flow {
emit(listOf("abc", "def"))
delay(500)
emit(listOf("123", "234"))
}
}
fun observeUserForId(userId: String): Flow<User> {
return flow {
emit(User("${userId}_name"))
delay(2000)
emit(User("${userId}_name_updated"))
}
}
In this scenario I want the emissions to be:
[User(abc_name), User(def_name)], then
[User(123_name), User(234_name)], then
[User(123_name_updated), User(234_name_updated)]
I think I can achieve this in RxJava like this:
observeBestUserIds.concatMapSingle { ids ->
Observable.fromIterable(ids)
.concatMap { id ->
observeUserForId(id)
}
.toList()
}
What function would I write to make a flow that emits that?
I believe you're looking for combine, which gives you an array that you can easily call toList() on:
observeBestUserIds().collectLatest { ids ->
combine(
ids.map { id -> observeUserForId(id) }
) {
it.toList()
}.collect {
println(it)
}
}
And here's the inner part with more explicit parameter names since you can't see the IDE's type hinting on Stack Overflow:
combine(
ids.map { id -> observeUserForId(id) }
) { arrayOfUsers: Array<User> ->
arrayOfUsers.toList()
}.collect { listOfUsers: List<User> ->
println(listOfUsers)
}
Output:
[User(name=abc_name), User(name=def_name)]
[User(name=123_name), User(name=234_name)]
[User(name=123_name_updated), User(name=234_name)]
[User(name=123_name_updated), User(name=234_name_updated)]
Live demo (note that in the demo, all the output appears at once, but this is a limitation of the demo site - the lines appear with the timing you'd expect when the code is run locally)
This avoids the (abc_name_updated, def_name_updated) discussed in the original question. However, there's still an intermediate emission with 123_name_updated and 234_name because the 123_name_updated is emitted first and it sends the combined version immediately because they're the latest from each flow.
However, this can be avoided by debouncing the emissions (on my machine, a timeout as small as 1ms works, but I did 20ms to be conservative):
observeBestUserIds().collectLatest { ids ->
combine(
ids.map { id -> observeUserForId(id) }
) {
it.toList()
}.debounce(timeoutMillis = 20).collect {
println(it)
}
}
which gets you the exact output you wanted:
[User(name=abc_name), User(name=def_name)]
[User(name=123_name), User(name=234_name)]
[User(name=123_name_updated), User(name=234_name_updated)]
Live demo
This is unfortunatly non trivial with the current state of kotlin Flow, there seem to be important operators missing. But please notice that you are not looking for rxJavas toList(). If you would try to to do it with toList and concatMap in rxjava you would have to wait till all observabes finish.
This is not what you want.
Unfortunately for you I think there is no way around a custom function.
It would have to aggregate all the results returned by observeUserForId for all the ids which you would pass to it. It would also not be a simple windowing function, since in reality it is conceivable that one observeUserForId already returned twice and another call still didn't finish. So checking whether you already have the same number of users as you passed ids into your aggregating functions isn't enought, you also have to group by user id.
I'll try to add code later today.
Edit: As promised here is my solution I took the liberty of augmenting the requirements slightly. So the flow will emit every time all userIds have values and an underlying user changes. I think this is more likely what you want since users probably don't change properties in lockstep.
Nevertheless if this is not what you want leave a comment.
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.*
import kotlinx.coroutines.runBlocking
data class User(val name: String)
fun observeBestUserIds(): Flow<List<String>> {
return flow {
emit(listOf("abc", "def"))
delay(500)
emit(listOf("123", "234"))
}
}
fun observeUserForId(userId: String): Flow<User> {
return flow {
emit(User("${userId}_name"))
delay(2000)
emit(User("${userId}_name_updated"))
}
}
inline fun <reified K, V> buildMap(keys: Set<K>, crossinline valueFunc: (K) -> Flow<V>): Flow<Map<K, V>> = flow {
val keysSize = keys.size
val valuesMap = HashMap<K, V>(keys.size)
flowOf(*keys.toTypedArray())
.flatMapMerge { key -> valueFunc(key).map {v -> Pair(key, v)} }
.collect { (key, value) ->
valuesMap[key] = value
if (valuesMap.keys.size == keysSize) {
emit(valuesMap.toMap())
}
}
}
fun observeUsersForIds(): Flow<List<User>> {
return observeBestUserIds().flatMapLatest { ids -> buildMap(ids.toSet(), ::observeUserForId as (String) -> Flow<User>) }
.map { m -> m.values.toList() }
}
fun main() = runBlocking {
observeUsersForIds()
.collect { user ->
println(user)
}
}
This will return
[User(name=def_name), User(name=abc_name)]
[User(name=123_name), User(name=234_name)]
[User(name=123_name_updated), User(name=234_name)]
[User(name=123_name_updated), User(name=234_name_updated)]
You can run the code online here
You can use flatMapConcat
val users = observeBestUserIds()
.flatMapConcat { ids ->
flowOf(*ids.toTypedArray())
.map { id ->
observeUserForId(id)
}
}
.flattenConcat()
.toList()
or
observeBestUserIds()
.flatMapConcat { ids ->
flowOf(*ids.toTypedArray())
.map { id ->
observeUserForId(id)
}
}
.flattenConcat()
.collect { user ->
}
I'm building an application which consists of two API calls. I'm still learning RxJava and I'm not sure how to combine properly the two API calls
The first API call is used to retrieve items in form of a list
The second API call is used to retrieve item image using the item name that I got from the first call.
I need to show all the items with their images. Those are my API calls using retrofit
#GET("items/list")
fun getItems(): Observable<ItemResult>
#GET("item/{name}/images")
fun getItemDetails(#Path("name") name: String): Observable<ItemDetails>
This is the code that wrote with RxJava:
fun getItemsData(): Observable<ArrayList<ItemDetails>> {
val data = ArrayList<ItemDetails>()
getItems().flatMap { itemResponse -> Observable.just(itemResponse.message) } //this will give me a list with item names
.flatMapIterable { data -> data }//iterating over the list and for every item...
.map { itemName ->//calling to get the item image
getItemDetails(itemName).map { imageData ->
val itemImage = imageData.message
data.add(ItemData(itemName, itemImage))//from this point on I'm lost, I'm not sure if it's the right thing to add here the data
}.subscribeOn(Schedulers.io())
}.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
return Observable.create(data)// not sure how to create an Observable from the data
}
You don't really need this...
val data = ArrayList<ItemDetails>()
flatMap is not the right operator here:
flatMap { itemResponse -> Observable.just(itemResponse.message) }
you can simplify this by using map instead (the function itemResponse -> itemResponse.message operates on the inner value only).
map { itemResponse -> itemResponse.message }
Next:
.flatMapIterable { data -> data } // here we have Observable<Message>
.flatMap { itemName ->
getItemDetails(itemName).map { imageData ->
ItemData(itemName, imageData)
}
} // Observable<ItemData>
.toList() // Single<List<ItemData>> -> you can use toObservable to get an Observable<List<ItemData>>
You can use toList() instead of manually creating and populating the ArrayList, it simplifies things.
You need to map the flat mapped stream to return the desired type -
getItems()
.flatMap(item -> getItemDetails(item)
.map(itemDetail -> ItemData(item, itemDetail.image))
.subscribe(itemData -> // your desired type containing the original item and image);
You're not subscribing to the inner-stream inside the map. Try this:
val data = ArrayList<ItemDetails>()
getItems().flatMap { itemResponse -> Observable.just(itemResponse.message) } //this will give me a list with item names
.flatMapIterable { data -> data }//iterating over the list and for every item...
.flatmap(itemName -> getItemDetails(itemName).subscribeOn(Schedulers.io())
.map(itemDetail -> {//add to the list)})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
return Observable.fromIterable(data);
If you really wish to return an Observable<ArrayList<ItemDetails>>, may be you could use reduce?
Something like that:
fun getItemsData(): Observable<ArrayList<ItemDetails>> {
return
getItems().flatMap { itemResponse -> Observable.just(itemResponse.message) }
.flatMapIterable { data -> data }
.flatMap { itemName -> getItemDetails(itemName) }.subscribeOn(Schedulers.io())
.map { imageData -> ItemData(itemName, imageData.message) } }
.reduce(ArrayList<ItemDetails>(), (list, item) -> list.add(item))
.toObservable()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
}
I am not sure about the right syntax of reduce in Kotlin...
This question is loosely related to this question, but there were no answers. The answer from Bob Dalgleish is close, but doesn't support the potential error coming from a Single (which I think that OP actually wanted as well).
I'm basically looking for a way to "filter on error" - but don't think this exists when the lookup is RX based. I am trying to take a list of values, run them through a lookup, and skip any result that returns a lookup failure (throwable). I'm having trouble figuring out how to accomplish this in a reactive fashion.
I've tried various forms of error handling operators combined with mapping. Filter only works for raw values - or at least I couldn't figure out how to use it to support what I'd like to do.
In my use case, I iterate a list of IDs, requesting data for each from a remote service. If the service returns 404, then the item doesn't exist anymore. I should remove non-existing items from the local database and continue processing IDs. The stream should return the list of looked up values.
Here is a loose example. How do I write getStream() so that canFilterOnError passes?
import io.reactivex.Single
import io.reactivex.schedulers.Schedulers
import org.junit.Test
class SkipExceptionTest {
private val data: Map<Int, String> = mapOf(
Pair(1, "one"),
Pair(2, "two"),
Pair(4, "four"),
Pair(5, "five")
)
#Test
fun canFilterOnError() {
getStream(listOf(1, 2, 3, 4, 5))
.subscribeOn(Schedulers.trampoline())
.observeOn(Schedulers.trampoline())
.test()
.assertComplete()
.assertNoErrors()
.assertValueCount(1)
.assertValue {
it == listOf(
"one", "two", "four", "five"
)
}
}
fun getStream(list: List<Int>): Single<List<String>> {
// for each item in the list
// get it's value via getValue()
// if a call to getValue() results in a NotFoundException, skip that value and continue
// mutate the results using mutate()
TODO("not implemented")
}
fun getValue(id: Int): Single<String> {
return Single.fromCallable {
val value: String? = data[id]
if (value != null) {
data[id]
} else {
throw NotFoundException("dat with id $id does not exist")
}
}
}
class NotFoundException(message: String) : Exception(message)
}
First .materialize(), then .filter() on non-error events, then .dematerialize():
getStream(/* ... */)
.materialize()
.filter(notification -> { return !notification.isOnError(); })
.dematerialize()
I ended up mapping getValue() to Optional<String>, then calling onErrorResumeNext() on that and either returning Single.error() or Single.just(Optional.empty()). From there, the main stream could filter out the empty Optional.
private fun getStream(list: List<Int>): Single<List<String>> {
return Observable.fromIterable(list)
.flatMapSingle {
getValue(it)
.map {
Optional.of(it)
}
.onErrorResumeNext {
when (it) {
is NotFoundException -> Single.just(Optional.empty())
else -> Single.error(it)
}
}
}
.filter { it.isPresent }
.map { it.get() }
.toList()
}
I'm using Kotlin KBuilders with some protobuffs and have run into a situation that is confusing me.
To start off, I have a function that takes a file name and list of serialized JSON and deserialized that JSON to a protobuff.
fun parseFileData(fileName: String, lines: List<String>): Data.Builder.() -> Unit = when (fileName) {
SOME_FILE_NAME -> deserializeLinesToModel(lines, DataModel::class.java)
.let {
return {
dataMeasurement = buildDataMeasurement {
property1 = it.reduce { acc, n -> acc + n }
measurementMsec = it.map { it.measurementMsec }
}
}
}
else -> throw UnsupportedOperationException()
The first thing I didn't understand was why I needed the return inside the let block. But it worked so I moved on.
I later decided to refactor some stuff to make code elsewhere simpler and ended up with something like this:
fun parseFileData(fileName: String, factory: DataFactory): Sequence<Data.Builder.() -> Unit> = when (fileName) {
SOME_FILE_NAME -> factory.getSomeFileSequence() // returns Sequence<Model>
.batch(1000) // process data in batches of 1000 to reduce memory footprint and payload size
.map { return {
dataMeasurement = buildDataMeasurement {
property1 = it.reduce { acc, n -> acc + n }
measurementMsec = it.map { it.measurementMsec }
}
}
else -> throw UnsupportedOperationException()
So basically, instead of processes each batch as a list, I read the sequence from the factory, batch it into a sequence of lists and try to map each list to a Data.Builder.() -> Unit. However, this time I get return is not allowed here. I've tried multiple variations, with and without return and map and let and whatnot. The closest I've gotten to is a return type of Sequence<() -> Unit> which fails type inference.
Can anyone explain what's going on here? And why this type cannot be inferred?
return in the map lambda is a non-local return. It tries to return from the closest fun function, which happens to be parseFileData.
Non-local returns are only allowed from inline functions, whose lambda parameters are inlined at the call site, and the map extension for Sequence is not an inline function.
If you want to return a value from the lambda itself, use qualified return return#map ..., or omit it completely: then the last expression in the block will be returned as the result.