I have a list of Strings representing serialized data that I want to map to a list of objects. I use the following code:
strings.map { gson.fromJson(it, Model::class.java) }
// .doOtherStuff
However, sometimes there is a parsing error and the stream just stop, I would like to able to recover the list up until the point of failure. For example, if the error occurs at item 7, I would like doOtherStuff to get the 6 items that were successfully processed.
What's the most idiomatic way to do this? I can filter the list to see if the parsing will succeed but that's an expensive operation being doing twice.
You can treat an exception as null and then filter the nulls.
val models = modelsInJson.mapNotNull { json ->
try {
gson.fromJson(json, Model::class.java)
} catch (ex: WhateverException) {
// TODO: logging here?
null
}
}
Replace WhateverException with the correct one for the type of errors you want to handle, other errors can still stop the processing.
What you are looking for seems to be a combination of map and takeWhile. You can always roll your own. e.g. The following is adapted from the sources of mapNotNull and takeWhile:
inline fun <T, R : Any> Iterable<T>.mapWhileNotNull(transform: (T) -> R?): List<R> {
return mapWhileNotNullTo(ArrayList<R>(), transform)
}
inline fun <T, R : Any, C : MutableCollection<in R>> Iterable<T>.mapWhileNotNullTo(destination: C, transform: (T) -> R?): C {
for (element in this) {
val transformed = transform(element)
if (transformed == null) {
break
} else {
destination.add(transformed)
}
}
return destination
}
Now you can map an iterable up to a transform invocation that results in null and, like Jayson Minard's example, you can map whatever exception you need to null:
strings.mapWhileNotNull {
try {
gson.fromJson(it, Model::class.java)
} catch (e: JsonSyntaxException) {
null
}
}
// .doOtherStuff
Related
I currently face the problem of correctly closing resources that never leave their containing Either.
The relevant code looks something like this:
object SomeError
class MyRes : AutoCloseable { [...] }
fun createRes(): Either<SomeError, MyRes> { [...] }
fun extractData(res: MyRes): String { [...] }
fun theProblem(): Either<SomeError, String> {
return createRes()
.map { extractData(it) }
}
What is the most idiomatic way of closing the created MyRes? Closing it before that map prevents extractData from accessing it, and after the map I can't access it anymore via Either's operations. Closing it in extractData severely limits composability.
Currently I have an external List<AutoCloseable> that I iterate over after all the computations, but that can't be the intended way.
I am open to using Arrow Fx (e.g. Resource) if that helps, but I haven't found anything on how to combine Either and Resource in an elegant way.
It's possible to combine the either and Resource safely.
object SomeError
class MyRes : AutoCloseable { [...] }
fun createRes(): Resource<Either<SomeError, MyRes>> { [...] }
fun extractData(res: MyRes): String { [...] }
suspend fun solution(): Either<SomeError, String> = either {
createRes().use { either: Either<SomeError, MyRes> ->
val res = either.bind()
val string = extractData(res)
// call other Either code + `bind()` safely here
[...]
} // <-- MyRes will automatically close here
}
If in this code you encounter Either.Left and you call bind() on it the Resource will first close, because we jump outside of use, and then either will return the encountered Either.Left.
One possible solution I found was wrapping the block passed to map:
fun <B : AutoCloseable, C> andClose(f: (B) -> C): (B) -> C =
{ b: B -> b.use { f(b) } }
fun theProblemSlightlySolved(): Either<SomeError, String> {
return createRes()
.map(andClose { extractData(it) })
}
I'm trying to implement a backoff strategy just using kotlin flow.
I need to fetch data from timeA to timeB
result = dataBetween(timeA - timeB)
if the result is empty then I want to increase the end time window using exponential backoff
result = dataBetween(timeA - timeB + exponentialBackOffInDays)
I was following this article which is explaining how to approach this in rxjava2.
But got stuck at a point where flow does not have takeUntil operator yet.
You can see my implementation below.
fun main() {
runBlocking {
(0..8).asFlow()
.flatMapConcat { input ->
// To simulate a data source which fetches data based on a time-window start-date to end-date
// available with in that time frame.
flow {
println("Input: $input")
if (input < 5) {
emit(emptyList<String>())
} else { // After emitting this once the flow should complete
emit(listOf("Available"))
}
}.retryWhenThrow(DummyException(), predicate = {
it.isNotEmpty()
})
}.collect {
//println(it)
}
}
}
class DummyException : Exception("Collected size is empty")
private inline fun <T> Flow<T>.retryWhenThrow(
throwable: Throwable,
crossinline predicate: suspend (T) -> Boolean
): Flow<T> {
return flow {
collect { value ->
if (!predicate(value)) {
throw throwable // informing the upstream to keep emitting since the condition is met
}
println("Value: $value")
emit(value)
}
}.catch { e ->
if (e::class != throwable::class) throw e
}
}
It's working fine except even after the flow has a successful value the flow continue to collect till 8 from the upstream flow but ideally, it should have stopped when it reaches 5 itself.
Any help on how I should approach this would be helpful.
Maybe this does not match your exact setup but instead of calling collect, you might as well just use first{...} or firstOrNull{...}
This will automatically stop the upstream flows after an element has been found.
For example:
flowOf(0,0,3,10)
.flatMapConcat {
println("creating list with $it elements")
flow {
val listWithElementCount = MutableList(it){ "" } // just a list of n empty strings
emit(listWithElementCount)
}
}.first { it.isNotEmpty() }
On a side note, your problem sounds like a regular suspend function would be a better fit.
Something like
suspend fun getFirstNonEmptyList(initialFrom: Long, initialTo: Long): List<Any> {
var from = initialFrom
var to = initialTo
while (coroutineContext.isActive) {
val elements = getElementsInRange(from, to) // your "dataBetween"
if (elements.isNotEmpty()) return elements
val (newFrom, newTo) = nextBackoff(from, to)
from = newFrom
to = newTo
}
throw CancellationException()
}
How can I transform the following:
List<Try<String>>
to:
Try<List<String>>
Using kotlin and the functional library arrow (0.8.2). I would like to wrap it in a custom exception. It does not matter which one of the 'String' failed.
Update:
As the below answers will suffice, but I find it really hard to read. So, I implemented the following:
Create the following function:
fun getFailedStrings(result: List<Try<String>>): List<Failure> {
return result.fold(
initial = listOf(),
operation = { accumulator, nextUpdate ->
nextUpdate.fold(
ifSuccess = { accumulator },
ifFailure = { accumulator + Failure(it) }
)
})
}
Then use the result of the function:
return if (failedStrings.isNotEmpty()) {
failedStrings.first() // or whatever fits your usecase
} else {
// strings is the initial result of List<Try<String>>
Success(strings.mapNotNull { it.orNull() })
}
If we don't care about keeping the original exceptions we could do something like this with traverse:
val traversedTries = tries.traverse(Try.applicative(), ::identity)
This will return an instance of type Try<ListK<String>> with either all the strings or the first exception it finds.
ListK extends from List but we can optionally cast it by adding .map { it as List<String> } in the end if we need it to be Try<List<String>>
Alternatively, if we want to split the successes and failures we can create the following function:
fun <A> List<Try<A>>.splitSuccessFailure() : Tuple2<List<A>, List<Throwable>> =
fold(emptyList<A>() toT emptyList<Throwable>()) { (successes, failures), it ->
it.fold({ successes toT (failures + it) }, { (successes + it) toT failures })
}
Then, when we want to use it we can do the following:
val (successes, failures) = invalidTries.splitSuccessFailure()
Giving us two lists with the success values and failures respectively.
this seems to work:
fun convert(input: List<Try<String>>): Try<List<String>> =
input.fold(Try.just(emptyList())) { acc, i ->
acc.flatMap { list ->
i.flatMap {
Try.just(list + it)
}
}
}
I often stumble upon this problem but don't see a common implementation: how do I idiomatically (functionally) find an element, stop search after the match, and also return a different type (i.e. map whatever matched to another type)?
I've been able to do a workaround with
fun <F,T> Sequence<F>.mapFirst(block: (F) -> T?): T? =
fold(AtomicReference<T>()) { ref, from ->
if (ref.get() != null) return#fold ref
ref.set(block(from))
ref
}.get()
fun main() {
Files.list(someDir).asSequence().map { it.toFile() }.mapFirst { file ->
file.useLines { lines ->
lines.mapFirst { line ->
if (line == "123") line.toInt() else null
}
}
}?.let { num ->
println("num is $num") // will print 123 as an Int
} ?: println("not a single file had a line eq to '123'")
}
But that doesn't stop on the match (when block() returns non-null) and goes to consume all files and all their lines.
A simple for loop is enough to implement mapFirst:
fun <F,T> Sequence<F>.mapFirst(block: (F) -> T?): T? {
for (e in this) {
block(e)?.let { return it }
}
return null
}
If you need a solution without introducing your own extensions (though there's nothing wrong with it), you can use mapNotNull + firstOrNull combination:
files.asSequence()
.mapNotNull { /* read the first line and return not null if it's ok */ }
.firstOrNull()
I would not map the values you discard then, instead do it like this:
sequenceOf(1, 2, 3)
.firstOrNull() { it == 2 }
?.let { it * 2 } ?: 6
First you find the value that matches your condition, then you transform it too whatever you want. In case you don't find a matching element, you assign a default value (in this case 6).
This question is loosely related to this question, but there were no answers. The answer from Bob Dalgleish is close, but doesn't support the potential error coming from a Single (which I think that OP actually wanted as well).
I'm basically looking for a way to "filter on error" - but don't think this exists when the lookup is RX based. I am trying to take a list of values, run them through a lookup, and skip any result that returns a lookup failure (throwable). I'm having trouble figuring out how to accomplish this in a reactive fashion.
I've tried various forms of error handling operators combined with mapping. Filter only works for raw values - or at least I couldn't figure out how to use it to support what I'd like to do.
In my use case, I iterate a list of IDs, requesting data for each from a remote service. If the service returns 404, then the item doesn't exist anymore. I should remove non-existing items from the local database and continue processing IDs. The stream should return the list of looked up values.
Here is a loose example. How do I write getStream() so that canFilterOnError passes?
import io.reactivex.Single
import io.reactivex.schedulers.Schedulers
import org.junit.Test
class SkipExceptionTest {
private val data: Map<Int, String> = mapOf(
Pair(1, "one"),
Pair(2, "two"),
Pair(4, "four"),
Pair(5, "five")
)
#Test
fun canFilterOnError() {
getStream(listOf(1, 2, 3, 4, 5))
.subscribeOn(Schedulers.trampoline())
.observeOn(Schedulers.trampoline())
.test()
.assertComplete()
.assertNoErrors()
.assertValueCount(1)
.assertValue {
it == listOf(
"one", "two", "four", "five"
)
}
}
fun getStream(list: List<Int>): Single<List<String>> {
// for each item in the list
// get it's value via getValue()
// if a call to getValue() results in a NotFoundException, skip that value and continue
// mutate the results using mutate()
TODO("not implemented")
}
fun getValue(id: Int): Single<String> {
return Single.fromCallable {
val value: String? = data[id]
if (value != null) {
data[id]
} else {
throw NotFoundException("dat with id $id does not exist")
}
}
}
class NotFoundException(message: String) : Exception(message)
}
First .materialize(), then .filter() on non-error events, then .dematerialize():
getStream(/* ... */)
.materialize()
.filter(notification -> { return !notification.isOnError(); })
.dematerialize()
I ended up mapping getValue() to Optional<String>, then calling onErrorResumeNext() on that and either returning Single.error() or Single.just(Optional.empty()). From there, the main stream could filter out the empty Optional.
private fun getStream(list: List<Int>): Single<List<String>> {
return Observable.fromIterable(list)
.flatMapSingle {
getValue(it)
.map {
Optional.of(it)
}
.onErrorResumeNext {
when (it) {
is NotFoundException -> Single.just(Optional.empty())
else -> Single.error(it)
}
}
}
.filter { it.isPresent }
.map { it.get() }
.toList()
}