I'm trying to convert a tail-recursive piece of code that uses Arrow Try, Either and achieve what it does functionally.
The function evaluates the function that was passed in as the argument. If the result is successful, it returns a value of type T. If the evaluation results in a failure, it returns the throwable.
Here is the original function -
private tailrec fun <T> eventuallyAux(
startTime: LocalTime,
timeout: Duration,
retryInterval: Duration,
function: () -> T
): T {
val result = Try { function() }.toEither()
return when (result) {
is Either.Right -> result.b
is Either.Left -> {
throwEventuallyExceptionIfTimedOut(startTime, timeout, result.a)
Thread.sleep(retryInterval.toMillis())
eventuallyAux(startTime, timeout, retryInterval, function)
}
}
}
This is what I have managed to scrape up using Result in kotlin std-lib instead of toEither():
private tailrec fun <T> eventuallyAux(
startTime: LocalTime,
timeout: Duration,
retryInterval: Duration,
function: () -> T
): T {
return runCatching(function)
.onSuccess { it.right() }
.onFailure {
throwEventuallyExceptionIfTimedOut(startTime, timeout, it)
Thread.sleep(retryInterval.toMillis())
eventuallyAux(startTime, timeout, retryInterval, function)
}
}
The compiler complains that a value is being returned with the type Result<T> instead of T. How do I get around this and return T if the evaluation is successful and pass the throwable to the tailrec function if it is a failure?
Thanks
runCatching wraps your return value in a Result, that's why the return type is Result<T>. onSuccess etc just perform an action on the wrapped value, but return the Result itself
That's also why it's not tail-recursive - that requires the recursive call to be the last thing you do in the function, so there's nothing else to do in there, and the current function call can be popped off the stack. No need to backtrack through it when there's nothing to do in there! The result can just be passed to whatever comes before that on the stack. It's like removing the call and starting a new one, like iteration, instead of nesting them.
But because your onFailure code before the Result itself is returned, there's still code to execute in that call, still work to do. So your recursive call isn't a tail call - it's not the last thing, so the current function call can't be removed from the call stack yet. So you end up with regular recursion because it can't be optimised.
Unfortunately you can't do tail calls inside a try/catch either because that's not supported. You need to basically replicate the approach in the original code, which is:
call the function and get the Result
if it's a success, return the unwrapped value
otherwise, return the result of a recursive call
That way, the last thing you do is make the tail call, and the result of that is passed directly back to the original caller. You should be able to do it like this:
private tailrec fun <T> eventuallyAux(
startTime: LocalTime,
timeout: Duration,
retryInterval: Duration,
function: () -> T
): T {
val result = runCatching(function)
return if (result.isSuccess) result.getOrThrow()
else {
throwEventuallyExceptionIfTimedOut(startTime, timeout, it)
Thread.sleep(retryInterval.toMillis())
// last thing that happens, nothing else to do but return the result of this
eventuallyAux(startTime, timeout, retryInterval, function)
}
}
You can't use result.getOrElse because the else block is a lambda - I'm assuming it's because it gets converted to a Function object and calling that isn't handled for tail recursion optimisation, but I haven't looked into it. Doing it this way, with the most basic if/else structure, lets you control exactly what happens when and make sure it gets optimised
Related
I understand that the kotlin sequence is just a coroutine with a yield suspend function (correct me if I'm wrong).
So the yield function sets the value and suspends the coroutine until the next value is requested.
yield source code
override suspend fun yield(value: T) {
nextValue = value
state = State_Ready
return suspendCoroutineUninterceptedOrReturn { c ->
nextStep = c
COROUTINE_SUSPENDED
}
}
Link to kotlin source code github
I see that it set the nextValue & state and suspends the coroutine by calling suspendCoroutineUninterceptedOrReturn.
Now I would like to know, where does this get resume?
I don't see any resume call inside next() function
override fun next(): T {
when (state) {
State_NotReady, State_ManyNotReady -> return nextNotReady()
State_ManyReady -> {
state = State_ManyNotReady
return nextIterator!!.next()
}
State_Ready -> {
state = State_NotReady
#Suppress("UNCHECKED_CAST")
val result = nextValue as T
nextValue = null
return result
}
else -> throw exceptionalState()
}
}
Just from looking at the source code page you linked:
Calling next() first checks if hasNext() has already been called. If it has, it can immediately return the next value, which was queued in hasNext(). If hasNext() has not yet been called, nextNotReady() is called, which basically calls hasNext() to prepare that next value and then returns it.
So the real work all happens in hasNext(). It checks if the current continuation’s iterator (which exists if yieldAll() is called) has another value, in which case it can queue it without resuming the continuation. But if the current iterator is done or there was no iterator, then it calls resume on the continuation, which makes it run until the next yield() or yieldAll() call.
This description is a big simplification. There are different edge cases that it is handling based on the order of next/hasNext calls.
I am trying to get the size of this firebase collection size of documents, and for some reason in Kotlin, I can't seem to get this to work. I have declared a variable to be zero in an int function and I put it inside a for loop where it increments to the size of the range. Then when I return the value, it is zero. Here is the code I have provided, please help me as to why it is returning zero.
This is just what is being passed to the function
var postSize = 0
That is the global variable, now for below
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
getPostSize(first)
This is the function
private fun getPostSize(first: Query){
first.get().addOnSuccessListener { documents ->
for(document in documents) {
Log.d(TAG, "${document.id} => ${document.data}")
getActualPostSize(postSize++)
}
}
return postSize
}
private fun getActualPostSize(sizeOfPost: Int): Int {
// The number does push to what I am expecting right here if I called a print statement
return sizeOfPost // However here it just returns it to be zero again. Why #tenffour04? Why?
}
It is my understanding, according to the other question that this was linked to, that I was suppose to do something like this.
This question has answers that explain how to approach getting results from asynchronous APIs, like you're trying to do.
Here is a more detailed explanation using your specific example since you were having trouble adapting the answer from there.
Suppose this is your original code you were trying to make work:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
val postSize = getPostSize(first)
// do something with postSize
// Elsewhere in your class:
private fun getPostSize(first: Query): Int {
var postSize = 0
first.get().addOnSuccessListener { documents ->
for(document in documents) {
Log.d(TAG, "${document.id} => ${document.data}")
postSize++
}
}
return postSize
}
The reason this doesn't work is that the code inside your addOnSuccessListener is called some time in the future, after getPostSize() has already returned.
The reason asynchronous code is called in the future is because it takes a long time to do its action, but it's bad to wait for it on the calling thread because it will freeze your UI and make the whole phone unresponsive. So the time-consuming action is done in the background on another thread, which allows the calling code to continue doing what it's doing and finish immediately so it doesn't freeze the UI. When the time-consuming action is finally finished, only then is its callback/lambda code executed.
A simple retrieval from Firebase like this likely takes less than half a second, but this is still too much time to freeze the UI, because it would make the phone seem janky. Half a second in the future is still in the future compared to the code that is called underneath and outside the lambda.
For the sake of simplifying the below examples, let's simplify your original function to avoid using the for loop, since it was unnecessary:
private fun getPostSize(first: Query): Int {
var postSize = 0
first.get().addOnSuccessListener { documents ->
postSize = documents.count()
}
return postSize
}
The following are multiple distinct approaches for working with asynchronous code. You only have to pick one. You don't have to do all of them.
1. Make your function take a callback instead of returning a value.
Change you function into a higher order function. Since the function doesn't directly return the post size, it is a good convention to put "Async" in the function name. What this function does now is call the callback to pass it the value you wanted to retrieve. It will be called in the future when the listener has been called.
private fun getPostSizeAsync(first: Query, callback: (Int) -> Unit) {
first.get().addOnSuccessListener { documents ->
val postSize = documents.count()
callback(postSize)
}
}
Then to use your function in your "calling code", you must use the retrieved value inside the callback, which can be defined using a lambda:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
getPostSizeAsync(first) { postSize ->
// do something with postSize inside the lambda here
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
2. Handle the response directly in the calling code.
You might have noticed in the above solution 1, you are really just creating an intermediate callback step, because you already have to deal with the callback lambda passed to addOnSuccessListener. You could eliminate the getPostSize function completely and just deal with callbacks at once place in your code. I wouldn't normally recommend this because it violates the DRY principle and the principle of avoiding dealing with multiple levels of abstraction in a single function. However, it may be better to start this way until you better grasp the concept of asynchronous code.
It would look like this:
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
first.get().addOnSuccessListener { documents ->
val postSize = documents.count()
// do something with postSize inside the lambda here
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
3. Put the result in a LiveData. Observe the LiveData separately.
You can create a LiveData that will update its observers about results when it gets them. This may not be a good fit for certain situations, because it would get really complicated if you had to turn observers on and off for your particular logic flow. I think it is probably a bad solution for your code because you might have different queries you want to pass to this function, so it wouldn't really make sense to have it keep publishing its results to the same LiveData, because the observers wouldn't know which query the latest postSize is related to.
But here is how it could be done.
private val postSizeLiveData = MutableLiveData<Int>()
// Function name changed "get" to "fetch" to reflect it doesn't return
// anything but simply initiates a fetch operation:
private fun fetchPostSize(query: Query) {
first.get().addOnSuccessListener { documents ->
postSize.value = documents.count()
}
}
// In your "calling code" (inside onCreate() or some click listener):
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
fetchPostSize(first)
postSizeLiveData.observer(this) { postSize ->
// Do something with postSize inside this observer that will
// be called some time in the future.
}
// Don't try to do something with postSize after the lambda here. Code under
// here is called before the code inside the lambda because the lambda is called
// some time in the future.
4. Use a suspend function and coroutine.
Coroutines allow you to write synchronous code without blocking the calling thread. After you learn to use coroutines, they lead to simpler code because there's less nesting of asynchronous callback lambdas. If you look at option 1, it will become very complicated if you need to call more than one asynchronous function in a row to get the results you want, for example if you needed to use postSize to decide what to retrieve from Firebase next. You would have to call another callback-based higher-order function inside the lambda of your first higher-order function call, nesting the future code inside other future code. (This is nicknamed "callback hell".) To write a synchronous coroutine, you launch a coroutine from lifecycleScope (or viewLifecycleOwner.lifecycleScope in a Fragment or viewModelScope in a ViewModel). You can convert your getter function into a suspend function to allow it to be used synchronously without a callback when called from a coroutine. Firebase provides an await() suspend function that can be used to wait for the result synchronously if you're in a coroutine. (Note that more properly, you should use try/catch when you call await() because it's possible Firebase fails to retrieve the documents. But I skipped that for simplicity since you weren't bothering to handle the possible failure with an error listener in your original code.)
private suspend fun getPostSize(first: Query): Int {
return first.get().await().count()
}
// In your "calling code" (inside onCreate() or some click listener):
lifecycleScope.launch {
val db = FirebaseFirestore.getInstance()
val first = db.collection("Post").orderBy("timestamp")
val postSize = getPostSize(first)
// do something with postSize
}
// Code under here will run before the coroutine finishes so
// typically, you launch coroutines and do all your work inside them.
Coroutines are the common way to do this in Kotlin, but they are a complex topic to learn for a newcomer. I recommend you start with one of the first two solutions until you are much more comfortable with Kotlin and higher order functions.
I am learning coroutines and need some help to understand a basic use case.
Implement a non-blocking method that:
Fetches a single item from a (reactive) DB
Determines a range (i.e. the month that the item lives in) based on that item's timestamp
Fetches all items in that month
Returns the items as Flow
Approach
Because it must return a Flow I will not use suspend (like I would when returning a single item). Returning Flow and suspend (which kind of returns a Mono) are most commonly mutually exclusive, right?
So I came up with this signature:
override fun getHistory(beforeUtcMillisExclusive: Long): Flow<Item>
Trying an implementation:
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return emptyFlow()
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end)
}
This gives me on the very first line:
Suspend function 'findOneByTimestampLessThan' should be called only
from a coroutine or another suspend function
I understand the problem that we are not allowed to call a suspend function here and the proposed solution by IntelliJ "adding suspend" does not make sense, when already returning a flow.
So, from this question I got the idea of using a return flow {...}:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return#flow
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return#flow itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start,
range.end)
}
}
The second repository call findByTimestampGreaterThanEqualAndTimestampLessThan returns Flow<Item> and I do not understand why I cannot return it.
This function must return a value of type Unit
Type mismatch.
Required:
Unit
Found:
Flow
return#flow returns from the lambda, not from enclosing function.
You need to reemit items from Flow returned by findByTimestampGreaterThanEqualAndTimestampLessThan call into Flow you're building with flow function:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod != null) {
val range = calcRange(itemInNextPeriod.timestamp)
emitAll(itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end))
}
}
I want to iterate over a sequence of objects and return the first non-null of an async call.
The point is to perform some kind of async operation that might fail, and I have a series of fallbacks that I want to try in order, one after the other (i.e. lazily / not in parallel).
I've tried to do something similar to what I'd do if it were a sync call:
// ccs: List<CurrencyConverter>
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asSequence()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
IntelliJ complains:
Suspension functions can only be called within coroutine body
Edit: To clarify, this works as expected if mapping on a List, but I want to see how I'd do this on a sequence.
So I guess this is because the map lambda isn't suspended? But I'm not sure how to actually do that. I tried a bunch of different ways but none seemed to work. I couldn't find any examples.
If I re-write this in a more procedural style using a for loop with an async block, I can get it working:
override suspend fun getExchangeRateAsync(from: String, to: String) {
for (cc in ccs) {
var res: BigDecimal? = async {
cc.getExchangeRateAsync(from, to)
}.await()
if (res != null) {
return res
}
}
throw CurrencyConverterException()
}
You are getting an error, because Sequence is lazy by default and it's map isn't an inline function, so it's scope isn't defined
You can avoid using Sequence by creating a list of lazy coroutines
// ccs: List<CurrencyConverter>
suspend fun getExchangeRateAsync(from: String, to: String) =
ccs
.map { async(start = CoroutineStart.LAZY) { it.getExchangeRateAsync(from, to) } }
.firstOrNull { it.await() != null }
?.getCompleted() ?: throw Exception()
This doesn't give any errors and seems to be working. But I'm not sure it's an idiomatic way
I would suggest replacing Sequence with Flow. Flow api and behavior is pretty much same as for Sequence, but with suspending options.
https://kotlinlang.org/docs/reference/coroutines/flow.html
Code:
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asFlow()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
FWIW, I found the suggestion in How to asynchronously map over sequence to be very intuitive. The code at https://github.com/Kotlin/kotlin-coroutines-examples/blob/master/examples/suspendingSequence/suspendingSequence.kt defines SuspendingIterator which allows next() to suspend, then builds SuspendingSequence on top of it. Unfortunately, you need to duplicate extension functions like flatMap(), filter(), etc. since SuspendingSequence can't be related to Sequence, but I did this and am much happier with the result than using a Channel.
The function send() in the following example calls itself recursively:
internal inner class RouteSender(
val features: List<Feature>,
val exchange: GrpcUniExchange<Point, RouteSummary>
) {
var result: AsyncResult<RouteSummary>? = null // Set in stub for recordRoute.
fun send(numPoints: Int) {
result?.let {
// RPC completed or err'd before sending completed.
// Sending further requests won't error, but they will be thrown away.
return
}
val index = random.nextInt(features.size)
val point = features[index].location
println("Visiting point ${RouteGuideUtil.getLatitude(point)}, " +
"${RouteGuideUtil.getLongitude(point)}")
exchange.write(point)
if (numPoints > 0) {
vertx.setTimer(random.nextInt(1000) + 500L) { _ ->
send(numPoints - 1)
}
} else {
exchange.end()
}
}
}
It can be re-written so that the last operation performed is the recursive call to itself:
...
if (numPoints <= 0) {
exchange.end()
} else {
vertx.setTimer(random.nextInt(1000) + 500L) { _ ->
send(numPoints - 1)
}
}
...
Yet, if I mark it as a tailrec function, I get a warning that the recursive call is not a tail call. This doesn't stop compilation of the successful running of the program. However, why is this not a tail call?
The documentation says:
To be eligible for the tailrec modifier, a function must call itself
as the last operation it performs. You cannot use tail recursion when
there is more code after the recursive call, and you cannot use it
within try/catch/finally blocks.
This is not within a try/catch/finally block and there is no more code after the recursive call. What is it that means this code block is ineligible for tail recursion optimisation?
I'll take a stab at answering my own question, in that it has no return value. Based on this discussion, that's about all I can think of. Thoughts?
Although your method appears to contain a call to itself, it's not actually a recursive method at all.
The call to send appears inside a closure. That means that it's not invoked immediately. It will only be invoked when the closure itself is invoked. In your case, that's done by a timer. It will take place outside of the current call stack, and probably even outside of the current thread.
In any case, the last call is the call to vertx.setTimer.
Kotlin allows inline closure functions which it uses for a number of it's own library functions, such as forEach. The tailrec might work if it's called from an inline closure, after all return from an inline closure returns from the outer function.
However, as noted above, this is a timer callback function, so it can by definition not be an inline call, and not be tail recursive.