I've been running into weird issues with Kotlins JSR-223 implementation for ScriptEngine (kotlin-scripting-jsr223)
Basically, i have a condition, which i parse and compile into a CompiledScript:
fun parse(clazz: KClass<out GenericEvent>, condition: String): String {
val simpleName = clazz.simpleName
return """
import ${clazz.java.name}
val condition: $simpleName.() -> Boolean = { ${trimQuotes(condition)} }
fun check(event: $simpleName) = condition.invoke(event)
""".trimIndent()
}
I then compile the resulting string using (engine as Compilable).compile(code) into a CompiledScript.
Now, whenever i try to eval() these CompiledScripts multiple times in a very short period of time:
listener.compiledScript.eval()
return#Callable (engine as Invocable).invokeFunction("check", event)
It seems to reuse old previously used arguments. First i thought it was a thread-safety issue (Because it heavily looks like one), but using a Mutex or a single threaded executor to compile the script didn't change this behavior.
Full code:
override suspend fun onEvent(e: GenericEvent) {
val events = listeners[e.javaClass] ?: return
events.forEach { listener ->
try {
val event = e.javaClass.cast(e)
if (listener.compiledScript != null) {
val result = executor.submit(Callable { // executor = Executors.newSingleThreadExecutor()
println(Thread.currentThread().id) // confirms it's always the same thread
listener.compiledScript.eval()
return#Callable (engine as Invocable).invokeFunction("check", event)
}).get()
if (result is Boolean && result)
listener.method.callSuspend(listener.instance, event)
} else {
listener.method.callSuspend(listener.instance, event)
}
} catch (throwable: Throwable) {
log.error("An error occurred while evaluating listener condition!", throwable)
}
}
}
At first, this works as expected
Condition: message.contentRaw == "first"
Is actually: first
but it creates weird, but always the same patterns afterwards.
Condition: message.contentRaw == "first"
Is actually: second
Condition: message.contentRaw == "second"
Is actually: second
Condition: message.contentRaw == "first"
Is actually: second
Condition: message.contentRaw == "second"
Is actually: second
Condition: message.contentRaw == "first"
Is actually: second
Condition: message.contentRaw == "second"
Is actually: second
(These messages shouldn't be printed if the condition isn't true)
I did some debugging earlier in which i printed the condition and the actual event in the compiled code itself, and it did reuse the previous event parameter in check(), even though i actually used an entirely different one into it.
Fixed it by creating a new SimpleScriptContext, passing it into the eval() method of CompiledScript, and then calling ScriptEngine#setContext with the newly created context before calling invokeFunction()
Crazy stuff!
Related
I am learning coroutines and need some help to understand a basic use case.
Implement a non-blocking method that:
Fetches a single item from a (reactive) DB
Determines a range (i.e. the month that the item lives in) based on that item's timestamp
Fetches all items in that month
Returns the items as Flow
Approach
Because it must return a Flow I will not use suspend (like I would when returning a single item). Returning Flow and suspend (which kind of returns a Mono) are most commonly mutually exclusive, right?
So I came up with this signature:
override fun getHistory(beforeUtcMillisExclusive: Long): Flow<Item>
Trying an implementation:
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return emptyFlow()
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end)
}
This gives me on the very first line:
Suspend function 'findOneByTimestampLessThan' should be called only
from a coroutine or another suspend function
I understand the problem that we are not allowed to call a suspend function here and the proposed solution by IntelliJ "adding suspend" does not make sense, when already returning a flow.
So, from this question I got the idea of using a return flow {...}:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return#flow
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return#flow itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start,
range.end)
}
}
The second repository call findByTimestampGreaterThanEqualAndTimestampLessThan returns Flow<Item> and I do not understand why I cannot return it.
This function must return a value of type Unit
Type mismatch.
Required:
Unit
Found:
Flow
return#flow returns from the lambda, not from enclosing function.
You need to reemit items from Flow returned by findByTimestampGreaterThanEqualAndTimestampLessThan call into Flow you're building with flow function:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod != null) {
val range = calcRange(itemInNextPeriod.timestamp)
emitAll(itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end))
}
}
I encountered a case where I have a nested Flux. I don't care about the individual results of the inner flux as it returns Unit (in Kotlin / Void in Java), but I want to know if the Flux aborted due to an error or not. I thought I could use the then function, as the doc states: Error signal is replayed in the resulting Mono<V>
My problem can be reduced to the minimum (Kotlin) unit test:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Flux.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So we have 2 elements. In the inner Flux one of the elements will fail on processing and the other won't. I expect the error to propagate through the pipeline where it is catched and discarded in the onErrorContinue.
Therefore I'd expect 1 element in the resulting list, but I get the original 2. I have no clue why.
Now comes the fun part: In this particular test case, I can replace Flux.just(1) with Mono.just(1) (in my real world case this doesn't work ofc because the flux has more than 1 element) and suddenly my test passes:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Mono.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So obviously there is a difference in Mono.then(Mono<T>) and in Flux.then(Mono<T>), but it shouldn't since the Javadoc is the same right?
Side note: Instead of Flux.then(Mono.just(outer)) I also tried Mono.defer but that is not changing anything.
Say I have an API like so:
interface Foo {
val barFlow: Flow<Bar>
}
And I consume it like so:
class FooConsumer(private val foo: Foo) {
init {
CoroutineScope(Dispatchers.IO).launch {
val bar = foo.barFlow.single()
println("Collected bar: $bar)
}
}
}
According to the docs for single a NoSuchElementException can be thrown if the flow is empty. However, this confuses me quite a lot, as a terminal operation on a flow will "await" elements of the flow to be emitted. So how will the call to single know that there were no elements in the flow? Maybe an element just hasn't been emitted yet?
I mean under the hood, the call to single is collecting the source flow before it does the check. Therefore at least 1 item must have been emitted before the check for null is carried out, so that null check should never succeed and a NoSuchElementException should never be thrown (for the case where the flow is of a non nullable type).
So will NoSuchElementException only be a possibility for flows of nullable types?
Here is the source code for single:
/**
* The terminal operator, that awaits for one and only one value to be published.
* Throws [NoSuchElementException] for empty flow and [IllegalStateException] for flow
* that contains more than one element.
*/
public suspend fun <T> Flow<T>.single(): T {
var result: Any? = NULL
collect { value ->
if (result !== NULL) error("Expected only one element")
result = value
}
if (result === NULL) throw NoSuchElementException("Expected at least one element")
#Suppress("UNCHECKED_CAST")
return result as T
}
NoSuchElementException is thrown when the Flow finishes its emission without emitting a single element. One case I can think of right now is when you need to turn a collection into a Flow source. If that collection is empty and you call single on that Flow you will get a NoSuchElementException.
This example may seem absurd but you get the point:
val emptyListFlow = emptyList<Int>().asFlow()
launch {
val data = emptyListFlow.single()
}
In my case, I made a list.first(), where the list was empty
I want to iterate over a sequence of objects and return the first non-null of an async call.
The point is to perform some kind of async operation that might fail, and I have a series of fallbacks that I want to try in order, one after the other (i.e. lazily / not in parallel).
I've tried to do something similar to what I'd do if it were a sync call:
// ccs: List<CurrencyConverter>
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asSequence()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
IntelliJ complains:
Suspension functions can only be called within coroutine body
Edit: To clarify, this works as expected if mapping on a List, but I want to see how I'd do this on a sequence.
So I guess this is because the map lambda isn't suspended? But I'm not sure how to actually do that. I tried a bunch of different ways but none seemed to work. I couldn't find any examples.
If I re-write this in a more procedural style using a for loop with an async block, I can get it working:
override suspend fun getExchangeRateAsync(from: String, to: String) {
for (cc in ccs) {
var res: BigDecimal? = async {
cc.getExchangeRateAsync(from, to)
}.await()
if (res != null) {
return res
}
}
throw CurrencyConverterException()
}
You are getting an error, because Sequence is lazy by default and it's map isn't an inline function, so it's scope isn't defined
You can avoid using Sequence by creating a list of lazy coroutines
// ccs: List<CurrencyConverter>
suspend fun getExchangeRateAsync(from: String, to: String) =
ccs
.map { async(start = CoroutineStart.LAZY) { it.getExchangeRateAsync(from, to) } }
.firstOrNull { it.await() != null }
?.getCompleted() ?: throw Exception()
This doesn't give any errors and seems to be working. But I'm not sure it's an idiomatic way
I would suggest replacing Sequence with Flow. Flow api and behavior is pretty much same as for Sequence, but with suspending options.
https://kotlinlang.org/docs/reference/coroutines/flow.html
Code:
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asFlow()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
FWIW, I found the suggestion in How to asynchronously map over sequence to be very intuitive. The code at https://github.com/Kotlin/kotlin-coroutines-examples/blob/master/examples/suspendingSequence/suspendingSequence.kt defines SuspendingIterator which allows next() to suspend, then builds SuspendingSequence on top of it. Unfortunately, you need to duplicate extension functions like flatMap(), filter(), etc. since SuspendingSequence can't be related to Sequence, but I did this and am much happier with the result than using a Channel.
I have a storm bolt that gets tuples which contain a value of type MyClass, which in turn carries a time field startTime. At the end of the bolt, I'd like to log the time that the processing took from the beginning (marked by startTime). So I could do:
override fun doExecute(input: Tuple): Boolean {
... // my processing logic
input.values.filterIsInstance(MyClass).forEach {
val endToEndTime = (System.currentTimeMillis() - it.startTime).toInt()
stats.recordExecutionTime("mytag", endToEndTime)
}
}
I don't really have a problem with this approach. Just cosmetic wise it looks like I could have more than one instance of MyClass in the tuple (which I do not). So I was thinking if I could just pick the first out of the list like:
override fun doExecute(input: Tuple): Boolean {
... // my processing logic
input.values.filterIsInstance(MyClass).first().let {
val endToEndTime = (System.currentTimeMillis() - it.startTime).toInt()
stats.recordExecutionTime("mytag", endToEndTime)
}
}
This seems to be more readable. However, if the filterIsInstance(MyClass) call returns an empty list, I get an exception by calling first on it, while the first solution goes through fine without any exception. Since this is for pure logging purposes, I'd like it to be exception-free.
Is there a better way to do this?
There is a better way! :)
You can use firstOrNull() and then a safe call:
input.values.filterIsInstance(MyClass).firstOrNull()?.let {
val endToEndTime = (System.currentTimeMillis() - it.startTime).toInt()
stats.recordExecutionTime("mytag", endToEndTime)
}
You could also replace the filterIsInstance call with the version of firstOrNull() that takes a predicate, like so:
input.values.firstOrNull { it is MyClass }?.let {ű
it as MyClass
val endToEndTime = (System.currentTimeMillis() - it.startTime).toInt()
stats.recordExecutionTime("mytag", endToEndTime)
}