Parsing Arithmetic String Operations in Kotlin - kotlin

If I have a string that has an arithmetic expression with paranthesis like:
((4 * 7) / 2) - 7
How do I evaluate it automatically? In particular with Kotlin. I heard that you need to make a parser, so how can I do so in Kotlin and have all the necessary basic operations in the example as such?

fun evaluate(str: String): Double {
data class Data(val rest: List<Char>, val value: Double)
return object : Any() {
fun parse(chars: List<Char>): Double {
return getExpression(chars.filter { it != ' ' })
.also { if (it.rest.isNotEmpty()) throw RuntimeException("Unexpected character: ${it.rest.first()}") }
.value
}
private fun getExpression(chars: List<Char>): Data {
var (rest, carry) = getTerm(chars)
while (true) {
when {
rest.firstOrNull() == '+' -> rest = getTerm(rest.drop(1)).also { carry += it.value }.rest
rest.firstOrNull() == '-' -> rest = getTerm(rest.drop(1)).also { carry -= it.value }.rest
else -> return Data(rest, carry)
}
}
}
fun getTerm(chars: List<Char>): Data {
var (rest, carry) = getFactor(chars)
while (true) {
when {
rest.firstOrNull() == '*' -> rest = getTerm(rest.drop(1)).also { carry *= it.value }.rest
rest.firstOrNull() == '/' -> rest = getTerm(rest.drop(1)).also { carry = it.value / carry }.rest
else -> return Data(rest, carry)
}
}
}
fun getFactor(chars: List<Char>): Data {
return when (val char = chars.firstOrNull()) {
'+' -> getFactor(chars.drop(1)).let { Data(it.rest, +it.value) }
'-' -> getFactor(chars.drop(1)).let { Data(it.rest, -it.value) }
'(' -> getParenthesizedExpression(chars.drop(1))
in '0'..'9', ',' -> getNumber(chars)
else -> throw RuntimeException("Unexpected character: $char")
}
}
fun getParenthesizedExpression(chars: List<Char>): Data {
return getExpression(chars)
.also { if (it.rest.firstOrNull() != ')') throw RuntimeException("Missing closing parenthesis") }
.let { Data(it.rest.drop(1), it.value) }
}
fun getNumber(chars: List<Char>): Data {
val s = chars.takeWhile { it.isDigit() || it == '.' }.joinToString("")
return Data(chars.drop(s.length), s.toDouble())
}
}.parse(str.toList())
}
val expresssion = "((4 * 7) / 2) - 7"
val result = evaluate(expresssion)
println(result) // Output: 7.0

Look up "Math parser" on google. These are various dependencies you can add that allow you to then parse string input into mathematical formulas.

Related

How to simplify when expression in kotlin

I'd like to simplify this expression, especially "isDigit" and "isLetter" case. How to do it?
smoothInput.forEach { char ->
when {
char.isValidOperator() -> {
output.push(char)
}
char.isDigit() -> {
if (output.isNotEmpty() && output.last()!!.isNumeric()) output.addToLast(char)
else output.push(char)
}
char.isLetter() -> {
if (output.isNotEmpty() && output.last()!!.isValidVariableName()) output.addToLast(char)
else output.push(char)
}
else -> {
throw InvalidIdentifierException()
}
}
}
I think, that it isn't important, but it's much better to add code here than in comment
output is InputStack Type:
class InputStack : Stack<String> {
override val storage = mutableListOf<String>()
fun push(e: Char) = push(e.toString())
fun push(e: Operator) = push(e.toString())
fun addToLast(e: Char) {
storage[storage.size - 1] += e.toString()
}
}
Stack Interface:
interface Stack<T> {
val storage: MutableList<T>
fun asString(): String = buildString {
appendLine("----top----")
storage.asReversed().forEach {
appendLine(it)
}
appendLine("-----------")
}
fun push(element: T) = storage.add(element)
fun pop(): T {
if (storage.size == 0) throw EmptyStackException()
return storage.removeAt(storage.size - 1)
}
fun isEmpty(): Boolean = storage.isEmpty()
fun isNotEmpty(): Boolean = !isEmpty()
fun last(): T? = storage.lastOrNull()
fun forEach(action: (T) -> Unit) {
for (element in storage) action(element)
}
}
You can extract some common parts in the following way:
fun addCharToOutputConditionally(char: Char, output: InputStack, conditionOnLast: (String) -> Boolean) {
if (output.isNotEmpty() && conditionOnLast(output.last()!!)) output.addToLast(char)
else output.push(char)
}
smoothInput.forEach { char ->
when {
char.isValidOperator() -> {
output.push(char)
}
char.isDigit() -> {
addCharToOutputConditionally(char, output) {
it.isNumeric()
}
}
char.isLetter() -> {
addCharToOutputConditionally(char, output) {
it.isValidVariableName()
}
}
else -> {
throw InvalidIdentifierException()
}
}
}
However, in cases like this, I don't think it's usually worth spending the time to refactor it this way, considering that there's little to gain by doing so: the resulting code is even longer and arguably harder to read than the original one.
The new when expression:
smoothInput.forEach { char ->
when {
char.isValidOperator() -> output.push(char)
char.isDigit() -> output.addToLastConditionally(char) { it.isNumeric() }
char.isLetter() -> output.addToLastConditionally(char) { it.isValidVariableName() }
else -> throw InvalidIdentifierException()
}
}
I've change the addToLast function in InputStack for addToLastConditionally
fun addToLastConditionally(char: Char, condition: (String) -> Boolean) {
if (isNotEmpty() && condition(last()!!)) storage[storage.size - 1] += char.toString()
else push(char)
}

Kotlin - The method returns how many elements meet the condition

I'm struggling to get this working:
Implement the method countWhere (condition: (T) -> Boolean): Int. The method returns how many elements meet the condition (state).
Here is an example of how it can be used:
NOTE: I can't change the fun main stuff.
fun main () {
val list = LinkedList <String> ()
list . addFirst ("Apple")
list . addFirst ("Banana")
list . addFirst ("Bear")
val fruitStartsWithB = list. countWhere {element ->
element. starts with ("B")
}
println (fruitStartsWithB) // fruitsStartsWithB is 2 because there are two items in the list that go into "B".
}
this is what causing me troubles:
fun countWhere(condition: (T) -> Boolean): Int {
var count: Int = 0
forEach { if (this == condition) count++ }
return count
}
my return is 0. My return has to be 2. Where is my mistake and how do I fix it?
This is all the code I have:
class LinkedList <T>: Iterable<T> {
data class Node <T>( val data : T, var next : Node <T>?)
private var first : Node <T>? = null
var isSorted = true
fun isEmpty() = first == null
fun addFirst(data: T) {
first = Node(data, first)
isSorted = false
}
fun getFirst(): T = get(0)
fun get(n: Int): T {
if (n < 0 ) throw IndexOutOfBoundsException ()
var run = first
var count = 0
while (count < n && run != null) {
run = run.next
count++
}
return run?.data ?: throw IndexOutOfBoundsException ()
}
fun clear () {
first = null // Clear
isSorted = true // Clear
}
// fun size (): Int{
// var run = first
// var count = 0
// while (run != null) {
// count++
// run = run.next
// }
// return count
// }
fun getOrNull(index: Int): T? {
if (index < 0 ) return null
var run = first
var count = 0
while (count < index && run != null) {
run = run.next
count++
}
return run?.data ?: null
}
fun addLast (data: T){
if (isEmpty()) addFirst(data) else {
var runPointer = first
while (runPointer?.next != null) {
runPointer = runPointer.next
}
runPointer?.next = Node(data, null)
}
isSorted = false
}
fun forEach(action: (T) -> Unit) {
for (i in this ) action(i)
}
fun size (): Int{
var count = 0
forEach { count ++ }
return count
}
override fun iterator(): Iterator<T> = object: Iterator <T> {
private var run = first
override fun hasNext(): Boolean = run!= null
override fun next(): T {
val res = run?.data ?: throw NoSuchElementException()
run = run?.next
return res
}
}
fun countWhere(condition: (T) -> Boolean): Int {
var count: Int = 0
forEach { if (condition(it)) count++ }
return count
}
}
you have to invoke your lambda:
fun countWhere(condition: (T) -> Boolean): Int {
var count: Int = 0
forEach { if (condition(it)) count++ }
return count
}

Can't use if/when assignment to return lambda with inferred parameter but can use if/when blocks

I have function returning a lambda based on an input String condition using if statement, which works fine - using this modified example from Head First Kotlin:
fun getConversionLambda(str: String): (Double) -> Double {
if (str == "CelsiusToFahrenheit")
return { it * 1.8 + 32 }
if (str == "KgToPounds")
return { it * 2.204623 }
return { it }
}
But since that's an obvious good place to use when instead, and I use the <function declaration> = <expression> format, including the return type, then at compile or pre-compile time, I get an Unresolved reference: it error:
fun getConversionLambda2(str: String): (Double) -> Double = when(str) {
"CelsiusToFahrenheit" -> { it * 1.8 + 32 }
"KgToPounds" -> { it * 2.204623 }
else -> { it }
}
Even if I specify it as the result of return within the function block, or assign it to a variable first and then return, I still get the Unresolved reference error:
fun getConversionLambda3(str: String): (Double) -> Double {
return when (str) {
"CelsiusToFahrenheit" -> { it * 1.8 + 32 }
"KgToPounds" -> { it * 2.204623 }
else -> { it }
}
}
Only way I could get it to work is by specifying the lambda's input variable-type in the lambda:
// and infers the `(Double) -> Double` return type correctly if removed
fun getConversionLambda4(str: String): (Double) -> Double = when(str) {
"CelsiusToFahrenheit" -> { x: Double -> x * 1.8 + 32 }
"KgToPounds" -> { x: Double -> x * 2.204623 }
else -> { x: Double -> x }
}
(my main:)
fun main(args: Array<String>) {
println("getCL: ${getConversionLambda("KgToPounds")(2.5)}")
// println("getCL2: ${getConversionLambda2("KgToPounds")(2.5)}")
// println("getCL3: ${getConversionLambda3("KgToPounds")(2.5)}")
println("getCL4: ${getConversionLambda4("KgToPounds")(2.5)}")
}
Why does the if version not have a problem with it? It's obviously inferring the lambdas' parameter type and doing the one-param-it based on getConversionLambda's definition's explicit return type. So why not for the when-version 2 & 3? I'm on Kotlin v1.4.32.
Edit: It seems any 'expression assignment' version of if & when prduces this issue unless I explicitly specify the parameter type for the lambda:
// Unresolved reference: it
fun getConversionLambda1A(str: String): (Double) -> Double =
if (str == "KgToPounds") { it * 2.204623 } else { it }
// Unresolved reference: it
fun getConversionLambda1B(str: String): (Double) -> Double {
return if (str == "KgToPounds") { it * 2.204623 } else { it }
}
But these two versions with the lambda parameter specified don't produce the error:
// works
fun getConversionLambda1Aokay(str: String) =
if (str == "KgToPounds") { x: Double -> x * 2.204623 } else { x: Double -> x }
// works
fun getConversionLambda1Bokay(str: String): (Double) -> Double {
return if (str == "KgToPounds") { x: Double -> x * 2.204623 } else { x: Double -> x }
}
The issue is that you're not within the scope of the passed in function when trying to reference "it". Just add braces and you're all set.
fun getConversionLambda1A(str: String): (Double) -> Double =
if (str == "KgToPounds") { { it * 2.204623 } } else { { it } }
fun getConversionLambda2(str: String): (Double) -> Double = when(str) {
"CelsiusToFahrenheit" -> {{ it * 1.8 + 32 }}
"KgToPounds" -> {{ it * 2.204623 }}
else -> {{ it }} }

ojAlgo - Optimization issue with contiguous block logic?

I am using ojAlgo to work through a classroom scheduling problem I'm doing as an exercise. The source code can be found here on GitHub in the kotlin_solution folder:
https://github.com/thomasnield/optimized-scheduling-demo
Everything was going fine until I started to implement contiguous block logic which I've described over on Math Exchange. Bascially, if a class session requires 4 blocks then those 4 blocks need to be together.
For some reason, this modeling logic screeches to a halt when I implement the contiguous logic in this part of the code. It is churning infinitely.
Here is the Kotlin code in it's entirety:
import org.ojalgo.optimisation.ExpressionsBasedModel
import org.ojalgo.optimisation.Variable
import java.time.DayOfWeek
import java.time.LocalDate
import java.time.LocalDateTime
import java.time.LocalTime
import java.util.concurrent.atomic.AtomicInteger
// declare model
val model = ExpressionsBasedModel()
val funcId = AtomicInteger(0)
val variableId = AtomicInteger(0)
fun variable() = Variable(variableId.incrementAndGet().toString().let { "Variable$it" }).apply(model::addVariable)
fun addExpression() = funcId.incrementAndGet().let { "Func$it"}.let { model.addExpression(it) }
// Any Monday through Friday date range will work
val operatingDates = LocalDate.of(2017,10,16)..LocalDate.of(2017,10,20)
val operatingDay = LocalTime.of(8,0)..LocalTime.of(17,0)
val breaks = listOf<ClosedRange<LocalTime>>(
//LocalTime.of(11,30)..LocalTime.of(13,0)
)
// classes
val scheduledClasses = listOf(
ScheduledClass(id=1, name="Psych 101", hoursLength=1.0, repetitions=2),
ScheduledClass(id=2, name="English 101", hoursLength=1.5, repetitions=3),
ScheduledClass(id=3, name="Math 300", hoursLength=1.5, repetitions=2),
ScheduledClass(id=4, name="Psych 300", hoursLength=3.0, repetitions=1),
ScheduledClass(id=5, name="Calculus I", hoursLength=2.0, repetitions=2),
ScheduledClass(id=6, name="Linear Algebra I", hoursLength=2.0, repetitions=3),
ScheduledClass(id=7, name="Sociology 101", hoursLength=1.0, repetitions=2),
ScheduledClass(id=8, name="Biology 101", hoursLength=1.0, repetitions=2)
)
fun main(args: Array<String>) {
println("Job started at ${LocalTime.now()}")
applyConstraints()
println(model.minimise())
Session.all.forEach {
println("${it.name}-${it.repetitionIndex}: ${it.start.dayOfWeek} ${it.start.toLocalTime()}-${it.end.toLocalTime()}")
}
println("Job ended at ${LocalTime.now()}")
}
data class Block(val dateTimeRange: ClosedRange<LocalDateTime>) {
val timeRange = dateTimeRange.let { it.start.toLocalTime()..it.endInclusive.toLocalTime() }
fun addConstraints() {
val f = addExpression().upper(1)
OccupationState.all.filter { it.block == this }.forEach {
f.set(it.occupied, 1)
}
}
companion object {
// Operating blocks
val all by lazy {
generateSequence(operatingDates.start.atTime(operatingDay.start)) {
it.plusMinutes(15).takeIf { it.plusMinutes(15) <= operatingDates.endInclusive.atTime(operatingDay.endInclusive) }
}.filter { it.toLocalTime() in operatingDay }
.map { Block(it..it.plusMinutes(15)) }
.toList()
}
}
}
data class ScheduledClass(val id: Int,
val name: String,
val hoursLength: Double,
val repetitions: Int) {
val sessions by lazy {
Session.all.filter { it.parentClass == this }
}
fun addConstraints() {
//guide 3 repetitions to be fixed on MONDAY, WEDNESDAY, FRIDAY
if (repetitions == 3) {
sessions.forEach { session ->
val f = addExpression().level(session.blocksNeeded)
session.occupationStates.asSequence()
.filter {
it.block.dateTimeRange.start.dayOfWeek ==
when(session.repetitionIndex) {
1 -> DayOfWeek.MONDAY
2 -> DayOfWeek.WEDNESDAY
3 -> DayOfWeek.FRIDAY
else -> throw Exception("Must be 1/2/3")
}
}
.forEach {
f.set(it.occupied,1)
}
}
}
//guide two repetitions to be 48 hours apart (in development)
if (repetitions == 2) {
val first = sessions.find { it.repetitionIndex == 1 }!!
val second = sessions.find { it.repetitionIndex == 2 }!!
}
}
companion object {
val all by lazy { scheduledClasses }
}
}
data class Session(val id: Int,
val name: String,
val hoursLength: Double,
val repetitionIndex: Int,
val parentClass: ScheduledClass) {
val blocksNeeded = (hoursLength * 4).toInt()
val occupationStates by lazy {
OccupationState.all.asSequence().filter { it.session == this }.toList()
}
val start get() = occupationStates.asSequence().filter { it.occupied.value.toInt() == 1 }
.map { it.block.dateTimeRange.start }
.min()!!
val end get() = occupationStates.asSequence().filter { it.occupied.value.toInt() == 1 }
.map { it.block.dateTimeRange.endInclusive }
.max()!!
fun addConstraints() {
val f1 = addExpression().level(0)
//block out exceptions
occupationStates.asSequence()
.filter { os -> breaks.any { os.block.timeRange.start in it } || os.block.timeRange.start !in operatingDay }
.forEach {
// b = 0, where b is occupation state
// this means it should never be occupied
f1.set(it.occupied, 1)
}
//sum of all boolean states for this session must equal the # blocks needed
val f2 = addExpression().level(blocksNeeded)
occupationStates.forEach {
f2.set(it.occupied, 1)
}
//ensure all occupied blocks are consecutive
// PROBLEM, not finding a solution and stalling
/*
b1, b2, b3 .. bn = binary from each group
all binaries must sum to 1, indicating fully consecutive group exists
b1 + b2 + b3 + .. bn = 1
*/
val consecutiveStateConstraint = addExpression().level(1)
(0..occupationStates.size).asSequence().map { i ->
occupationStates.subList(i, (i + blocksNeeded).let { if (it > occupationStates.size) occupationStates.size else it })
}.filter { it.size == blocksNeeded }
.forEach { grp ->
/*
b = 1,0 binary for group
n = blocks needed
x1, x2, x3 .. xn = occupation states in group
x1 + x2 + x3 .. + xn - bn >= 0
*/
val binaryForGroup = variable().binary()
consecutiveStateConstraint.set(binaryForGroup, 1)
addExpression().lower(0).apply {
grp.forEach {
set(it.occupied,1)
}
set(binaryForGroup, -1 * blocksNeeded)
}
}
}
companion object {
val all by lazy {
ScheduledClass.all.asSequence().flatMap { sc ->
(1..sc.repetitions).asSequence()
.map { Session(sc.id, sc.name, sc.hoursLength, it, sc) }
}.toList()
}
}
}
data class OccupationState(val block: Block, val session: Session) {
val occupied = variable().binary()
companion object {
val all by lazy {
Block.all.asSequence().flatMap { b ->
Session.all.asSequence().map { OccupationState(b,it) }
}.toList()
}
}
}
fun applyConstraints() {
Session.all.forEach { it.addConstraints() }
ScheduledClass.all.forEach { it.addConstraints() }
Block.all.forEach { it.addConstraints() }
}
** UPDATE **
I created a self-contained example that simplifies what I'm trying to do above. It seems the contiguous logic is indeed the problem, and the more "slots" the problem has the slower it performs. At 48000 variables, the contiguous logic seems to churn forever.
import org.ojalgo.optimisation.ExpressionsBasedModel
import org.ojalgo.optimisation.Variable
import org.ojalgo.optimisation.integer.IntegerSolver
import java.util.concurrent.ThreadLocalRandom
import java.util.concurrent.atomic.AtomicInteger
// declare ojAlgo Model
val model = ExpressionsBasedModel()
// custom DSL for expression inputs, eliminate naming and adding
val funcId = AtomicInteger(0)
val variableId = AtomicInteger(0)
fun variable() = Variable(variableId.incrementAndGet().toString().let { "Variable$it" }).apply(model::addVariable)
fun addExpression() = funcId.incrementAndGet().let { "Func$it"}.let { model.addExpression(it) }
val letterCount = 9
val numberCount = 480
val minContiguousBlocks = 4
val maxContiguousBlocks = 4
fun main(args: Array<String>) {
Letter.all.forEach { it.addConstraints() }
Number.all.forEach { it.addConstraints() }
model.countVariables().run { println("$this variables") }
model.options.debug(IntegerSolver::class.java)
model.minimise().run(::println)
Letter.all.joinToString(prefix = "\t", separator = "\t").run(::println)
Letter.all.map { it.slotsNeeded }.joinToString(prefix = "\t", separator = "\t").run(::println)
Number.all.forEach { n ->
Letter.all.asSequence().map { l -> l.slots.first { it.number == n }.occupied.value.toInt() }
.joinToString(prefix = "$n ", separator = "\t").run { println(this) }
}
}
class Letter(val value: String, val slotsNeeded: Int = 1) {
val slots by lazy {
Slot.all.filter { it.letter == this }.sortedBy { it.number.value }
}
fun addConstraints() {
// Letter must be assigned once
addExpression().level(1).apply {
slots.forEach { set(it.occupied, 1) }
}
//handle recurrences
if (slotsNeeded > 1) {
slots.rollingBatches(slotsNeeded).forEach { batch ->
val first = batch.first()
addExpression().upper(0).apply {
batch.asSequence().flatMap { it.number.slots.asSequence() }
.forEach {
set(it.occupied, 1)
}
set(first.number.cumulativeState, -1)
}
}
}
//prevent scheduling at end of window
// all slots must sum to 0 in region smaller than slots needed
addExpression().level(0).apply {
slots.takeLast(slotsNeeded - 1)
.forEach {
set(it.occupied, 1)
}
}
}
override fun toString() = value
companion object {
val all = ('A'..'Z').asSequence()
.take(letterCount)
.map { it.toString() }
.map { Letter(it, ThreadLocalRandom.current().nextInt(minContiguousBlocks, maxContiguousBlocks + 1)) }
.toList()
}
}
class Number(val value: Int) {
val slots by lazy {
Slot.all.filter { it.number == this }
}
// b_x
val cumulativeState = variable().lower(0).upper(1)
fun addConstraints() {
// Number can only be assigned once
addExpression().upper(1).apply {
slots.forEach { set(it.occupied, 1) }
}
}
companion object {
val all = (1..numberCount).asSequence()
.map { Number(it) }
.toList()
}
override fun toString() = value.toString().let { if (it.length == 1) "$it " else it }
}
data class Slot(val letter: Letter, val number: Number) {
val occupied = variable().binary()
companion object {
val all = Letter.all.asSequence().flatMap { letter ->
Number.all.asSequence().map { number -> Slot(letter, number) }
}.toList()
}
override fun toString() = "$letter$number: ${occupied?.value?.toInt()}"
}
fun <T> List<T>.rollingBatches(batchSize: Int) = (0..size).asSequence().map { i ->
subList(i, (i + batchSize).let { if (it > size) size else it })
}.filter { it.size == batchSize }
I figured it out. I'll update this answer later with the full mathematical modeling explanation. Essentially for each 15 minute block I queried for slot groups that include that block, and declared the sum of all of them must be no more than one. This ended up being acceptably efficient as it runs in 30-60 seconds.
The code is here on GitHub, as well as below:
https://github.com/thomasnield/optimized-scheduling-demo
import org.ojalgo.optimisation.integer.IntegerSolver
import java.time.LocalDate
import java.time.LocalTime
import org.ojalgo.optimisation.ExpressionsBasedModel
import org.ojalgo.optimisation.Variable
import java.time.DayOfWeek
import java.time.LocalDateTime
import java.util.concurrent.atomic.AtomicInteger
// Any Monday through Friday date range will work
val operatingDates = LocalDate.of(2017,10,16)..LocalDate.of(2017,10,20)
val operatingDay = LocalTime.of(8,0)..LocalTime.of(17,0)
val breaks = listOf<ClosedRange<LocalTime>>(
LocalTime.of(11,30)..LocalTime.of(13,0)
)
// classes
val scheduledClasses = listOf(
ScheduledClass(id=1, name="Psych 101",hoursLength=1.0, repetitions=2),
ScheduledClass(id=2, name="English 101", hoursLength=1.5, repetitions=3),
ScheduledClass(id=3, name="Math 300", hoursLength=1.5, repetitions=2),
ScheduledClass(id=4, name="Psych 300", hoursLength=3.0, repetitions=1),
ScheduledClass(id=5, name="Calculus I", hoursLength=2.0, repetitions=2),
ScheduledClass(id=6, name="Linear Algebra I", hoursLength=2.0, repetitions=3),
ScheduledClass(id=7, name="Sociology 101", hoursLength=1.0, repetitions=2),
ScheduledClass(id=8, name="Biology 101", hoursLength=1.0, repetitions=2)
)
fun main(args: Array<String>) {
println("Job started at ${LocalTime.now()}")
applyConstraints()
model.countVariables().run { println("$this variables") }
model.options.apply {
//debug(IntegerSolver::class.java)
iterations_suffice = 0
}
println(model.minimise())
ScheduledClass.all.forEach {
println("${it.name}- ${it.daysOfWeek.joinToString("/")} ${it.start.toLocalTime()}-${it.end.toLocalTime()}")
}
println("Job ended at ${LocalTime.now()}")
}
// declare model
val model = ExpressionsBasedModel()
val funcId = AtomicInteger(0)
val variableId = AtomicInteger(0)
fun variable() = Variable(variableId.incrementAndGet().toString().let { "Variable$it" }).apply(model::addVariable)
fun addExpression() = funcId.incrementAndGet().let { "Func$it"}.let { model.addExpression(it) }
data class Block(val dateTimeRange: ClosedRange<LocalDateTime>) {
val timeRange = dateTimeRange.let { it.start.toLocalTime()..it.endInclusive.toLocalTime() }
val available get() = (breaks.all { timeRange.start !in it } && timeRange.start in operatingDay)
//val cumulativeState = variable().apply { if (available) lower(0).upper(1) else level(0) }
val slots by lazy {
Slot.all.filter { it.block == this }
}
fun addConstraints() {
if (available) {
addExpression().lower(0).upper(1).apply {
ScheduledClass.all.asSequence().flatMap { it.anchorOverlapFor(this#Block) }
.filter { it.block.available }
.forEach {
set(it.occupied, 1)
}
}
} else {
ScheduledClass.all.asSequence().flatMap { it.anchorOverlapFor(this#Block) }
.forEach {
it.occupied.level(0)
}
}
}
companion object {
// Operating blocks
val all by lazy {
generateSequence(operatingDates.start.atStartOfDay()) {
it.plusMinutes(15).takeIf { it.plusMinutes(15) <= operatingDates.endInclusive.atTime(23,59) }
}.map { Block(it..it.plusMinutes(15)) }
.toList()
}
fun applyConstraints() {
all.forEach { it.addConstraints() }
}
}
}
data class ScheduledClass(val id: Int,
val name: String,
val hoursLength: Double,
val repetitions: Int,
val repetitionGapDays: Int = 2) {
val repetitionGapSlots = repetitionGapDays * 24 * 4
val slotsNeeded = (hoursLength * 4).toInt()
val slots by lazy {
Slot.all.asSequence().filter { it.session == this }.toList()
}
val batches by lazy {
slots.rollingRecurrences(slotsNeeded = slotsNeeded, gapSize = repetitionGapSlots, recurrencesNeeded = repetitions)
}
fun anchorOverlapFor(block: Block) = batches.asSequence()
.filter { it.flatMap { it }.any { it.block == block } }
.map { it.first().first() }
val start get() = slots.asSequence().filter { it.occupied.value.toInt() == 1 }.map { it.block.dateTimeRange.start }.min()!!
val end get() = start.plusMinutes((hoursLength * 60.0).toLong())
val daysOfWeek get() = (0..(repetitions-1)).asSequence().map { start.dayOfWeek.plus(it.toLong() * repetitionGapDays) }.sorted()
fun addConstraints() {
//sum of all boolean states for this session must be 1
addExpression().level(1).apply {
slots.forEach {
set(it.occupied, 1)
}
}
//guide Mon/Wed/Fri for three repetitions
if (repetitions == 3) {
addExpression().level(1).apply {
slots.filter { it.block.dateTimeRange.start.dayOfWeek == DayOfWeek.MONDAY }
.forEach {
set(it.occupied, 1)
}
}
}
//guide two repetitions to start on Mon, Tues, or Wed
if (repetitions == 2) {
addExpression().level(1).apply {
slots.filter { it.block.dateTimeRange.start.dayOfWeek in DayOfWeek.MONDAY..DayOfWeek.WEDNESDAY }.forEach {
set(it.occupied, 1)
}
}
}
}
companion object {
val all by lazy { scheduledClasses }
}
}
data class Slot(val block: Block, val session: ScheduledClass) {
val occupied = variable().apply { if (block.available) binary() else level(0) }
companion object {
val all by lazy {
Block.all.asSequence().flatMap { b ->
ScheduledClass.all.asSequence().map { Slot(b,it) }
}.toList()
}
}
}
fun applyConstraints() {
Block.applyConstraints()
ScheduledClass.all.forEach { it.addConstraints() }
}
fun <T> List<T>.rollingBatches(batchSize: Int) = (0..size).asSequence().map { i ->
subList(i, (i + batchSize).let { if (it > size) size else it })
}.filter { it.size == batchSize }
fun <T> List<T>.rollingRecurrences(slotsNeeded: Int, gapSize: Int, recurrencesNeeded: Int) =
(0..size).asSequence().map { i ->
(1..recurrencesNeeded).asSequence().map { (it - 1) * gapSize }
.filter { it + i < size}
.map { r ->
subList(i + r, (i + r + slotsNeeded).let { if (it > size) size else it })
}.filter { it.size == slotsNeeded }
.toList()
}.filter { it.size == recurrencesNeeded }

RxJava different output between Flowable and Observable with Window and Groupby

I'm using RxJava2 with code that boils down to something like this:
val whitespaceRegex = Regex("\\s+")
val queryRegex = Regex("query=([^&]+)", RegexOption.IGNORE_CASE)
val dateTimeFormatter = DateTimeFormatter.ISO_OFFSET_DATE_TIME
#JvmStatic
fun main(args: Array<String>) {
val cnt = AtomicLong()
val templateStr = "|date| /ignored/ query=|query|"
val random = ThreadLocalRandom.current()
var curDate = ZonedDateTime.of(LocalDate.of(2016, Month.JANUARY, 1), LocalTime.MIDNIGHT, ZoneId.of("UTC"))
val generator = Flowable.generate<String> { emitter ->
// normally these are read from a file, this is for the example
val next = cnt.incrementAndGet()
if (next % 3000 == 0L) {
curDate = curDate.plusDays(1)
}
if (next < 100000) {
val curStr = templateStr
.replace("|date|", dateTimeFormatter.format(curDate))
.replace("|query|", random.nextInt(1, 1000).toString())
emitter.onNext(curStr)
} else {
emitter.onComplete()
}
}
val source = generator
.map { line ->
val cols = line.split(whitespaceRegex)
val queryRaw = queryRegex.find(cols[2])?.groupValues?.get(1) ?: ""
val query = URLDecoder.decode(queryRaw, Charsets.UTF_8.name()).toLowerCase().replace(whitespaceRegex, " ").trim()
val date = dateTimeFormatter.parse(cols[0])
Pair(LocalDate.from(date), query)
}
.share()
source
.window(source.map { it.first }.distinctUntilChanged())
.flatMap { window ->
window
.groupBy { pair -> pair }
.flatMap({ grouping ->
grouping
.count()
.map {
Pair(grouping.key, it)
}.toFlowable()
})
}
.subscribe({ println("Result: $it}") }, { it.printStackTrace() }, { println("Done") })
}
When I use Observable.generate it works fine, but with Flowable.generate there is no output. This is counting how many queries occurred on a given day. The day increase sequentially so I form a window of each day, then count the queries with a groupBy. Do I need to do this differently with Flowable?
As akarnokd mentioned, this was due to flatMap having a default maxConcurrency of 128. I found this issue, https://github.com/ReactiveX/RxJava/issues/5126, which describes the reason in more detail. This fixes the problem:
val cnt = AtomicLong()
val templateStr = "|date| /ignored/ query=|query|"
val random = ThreadLocalRandom.current()
var curDate = ZonedDateTime.of(LocalDate.of(2016, Month.JANUARY, 1), LocalTime.MIDNIGHT, ZoneId.of("UTC"))
val generator = Flowable.generate<String> { emitter ->
val next = cnt.incrementAndGet()
if (next % 3000 == 0L) {
curDate = curDate.plusDays(1)
}
if (next < 1000000) {
val curStr = templateStr
.replace("|date|", dateTimeFormatter.format(curDate))
.replace("|query|", random.nextInt(1, 1000).toString())
emitter.onNext(curStr)
} else {
emitter.onComplete()
}
}
val source = generator
.map { line ->
val cols = line.split(whitespaceRegex)
val queryRaw = queryRegex.find(cols[2])?.groupValues?.get(1) ?: ""
val query = URLDecoder.decode(queryRaw, Charsets.UTF_8.name()).toLowerCase().replace(whitespaceRegex, " ").trim()
val date = dateTimeFormatter.parse(cols[0])
Pair(LocalDate.from(date), query)
}
.share()
source
.window(source.map { it.first }.distinctUntilChanged().doOnEach({println("Win: $it")}))
.flatMap( { window ->
window
.groupBy { pair -> pair }
.flatMap({ grouping ->
grouping
.count()
.map {
Pair(grouping.key, it)
}.toFlowable()
// fix is here
}, Int.MAX_VALUE)
// and here
}, Int.MAX_VALUE)
.subscribe({ println("Result: $it}") }, { it.printStackTrace() }, { println("Done") })