Group by the day of the month - Kotlin Logic Problem - kotlin

I'm stuck with the logic. So here it is, I have one model class Note:
data class Note(
val id: Int,
val title: String,
val description: String,
val date: Long = System.currentTimeMillis()
)
I have a list of multiple notes in my app List<Note>. And I need a way to convert that list into a Map. Where key will be the date: Long, and the value will be List<Note>. So: Map<Long, List<Note>> . I need to group those notes by the day of the month. For example, if multiple notes were created on October 31th, then they should be grouped in a single list of Notes, within a Map.
I'm really not sure how can I achieve that. Always had troubles with those date values. I will appreciate any help. :)

You can add a helper property to get the date in LocalDate format, which would make it easy to sort by day. If you were using this a lot, repeatedly, you might consider adding it as a member property that isn't computed on each retrieval (but not in the constructor because it is computed from another property that participates in equals and hashcode).
val Note.localDate: LocalDate
get() = Instant.ofEpochMilli(date).atZone(ZoneId.systemDefault()).toLocalDate()
Then you can use groupBy to create your Map of dates to lists.
val notesByLocalDate = notes.groupBy(Note::localDate) // or { it.localDate }

This is going to be one of the "just because you can, doesn't mean you should".
.groupBy {
val noteCalendar = Calendar.getInstance()
noteCalendar.timeInMillis = it.date
val day = noteCalendar.get(Calendar.DAY_OF_MONTH)
val month = noteCalendar.get(Calendar.MONTH)
val year = noteCalendar.get(Calendar.YEAR)
val dayCalendar = Calendar.getInstance()
dayCalendar.timeInMillis = 0L
dayCalendar.set(Calendar.DAY_OF_MONTH, day)
dayCalendar.set(Calendar.MONTH, month)
dayCalendar.set(Calendar.YEAR, year)
dayCalendar.set(Calendar.HOUR_OF_DAY, 12)
dayCalendar.timeInMillis
}
Trying to group something by its date of creation in milliseconds will result in no grouping because nothing is created at the exact same time*. So the only way for you to group it is to translate the date range of which those things happen into one value. In this case noon of that day to avoid timezone problems.
...but again I would recommend not grouping this by Long.

Related

filter spark dataframe using udf

I have a student dataframe:
var student = Seq(("h123","078","Ryan"),("h789","078","John"),("h456","ad0","Mike")).toDF("id","div","name")
now I want to filter student on div column based on some logic, for this example assume only 078 value should be present.
For this, I have a udf defined as:
val filterudf = udf((div: String) => div == "078")
currently, I am using following approach to get my work done
val allowedDivs = student.select(col("div")).distinct().filter(filterudf(col("div")))
.collectAsList().asScala.map(row => row.getAs[String](0)).toList
val resultDF = student.filter(col("div").isInCollection(allowedDivs))
The actual table where I have to apply this filter is huge and in order to improve the performance I want to use spark.sql query to get benefit from codgen and other Tungsten optimizations.
This is want I have come to, but this query is not working
filterudf.registerTemplate("filterudf")
val resultDF = spark.sql("select * from student where div in (filterudf(select distinct div from student).div)")
Any help is appreciated.

Joiners with filtering performs very slowly

I have a constraint with some joiners but the performance are very poor. Is it a way to improve it ?
I need to have the count of WorkingDay ( with ::hasPermission ) within the previous four days of the current day analyzed.
Here is my current constraint :
private Constraint fiveConsecutiveWorkingDaysMax(ConstraintFactory constraintFactory) {
return constraintFactory
.from(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(WorkingDay.class,
Joiners.equal(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsBeforeWd1 = wd2.getDayJava().isBefore(wd1.getDayJava());
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return (wd2IsBeforeWd1 && wd2IsAfterFourDaysBeforeWd1);
}))
.groupBy((wd1, wd2) -> wd2, ConstraintCollectors.countBi())
.filter((wd2, count) -> count >= 4)
.penalizeConfigurable(FIVE_CONSECUTIVE_WORKING_DAYS_MAX);
}
Thanx for your help
There is potential for improvement here. First, we pre-filter the right hand side of the join to reduce the size of the cartesian product:
return constraintFactory
.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(constraintFactory.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsBeforeWd1 = wd2.getDayJava().isBefore(wd1.getDayJava());
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return (wd2IsBeforeWd1 && wd2IsAfterFourDaysBeforeWd1);
}))
...
This has the added benefit of simplifying the index as it removes one equals joiner. Next, part of the filter can be replaced by a joiner as well:
return constraintFactory
.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(constraintFactory.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.greaterThan(wd -> wd.getDayJava()),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return wd2IsAfterFourDaysBeforeWd1;
}))
...
Finally, the method does needless boxing of boolean into Boolean, wasting CPU cycles and memory. This is a micro-optimization, but if the filter happens often enough, the benefit will be measurable.
A constraint refactored like this should perform better. That said, large joins are still going to take considerable time and the only way to work around that is to figure out a way to make them smaller.
Also, as Geoffrey said, I'd consider penalizing by the actual count, as what you have here is a textbook example of a score trap.
I don't see why this should be slow. Except maybe because the Cartesian Product explodes for a long time window. How many days is your time window?
Do note that the nurse rostering example has a totally different approach to detecting consecutive working days, using a custom collector. You might want to look at that in optaplanner-examples.

How to filter a date-field with a swift vapor-fluent query

To avoid multiple inserts of the same person in a database, I wrote the following function:
func anzahlDoubletten(_ req: Request, nname: String, vname: String, gebTag: Date)
async throws -> Int {
try await
Teilnehmer.query(on: req.db)
.filter(\.$nname == nname)
.filter(\.$vname == vname)
.filter(\.$gebTag == gebTag)
.count()
}
The function always returns 0, even if there are multiple records with the same surname, prename and birthday in the database.
Here is the resulting sql-query:
[ DEBUG ] SELECT COUNT("teilnehmer"."id") AS "aggregate" FROM "teilnehmer" WHERE "teilnehmer"."nname" = $1 AND "teilnehmer"."vname" = $2 AND "teilnehmer"."geburtstag" = $3 ["neumann", "alfred e.", 1999-09-09 00:00:00 +0000] [database-id: psql, request-id: 1AC70C41-EADE-43C2-A12A-99C19462EDE3] (FluentPostgresDriver/FluentPostgresDatabase.swift:29)
[ INFO ] anzahlDoubletten=0 [request-id: 1AC70C41-EADE-43C2-A12A-99C19462EDE3] (App/Controllers/TeilnehmerController.swift:49)
if I query directly I obtain:
lwm=# select nname, vname, geburtstag from teilnehmer;
nname | vname | geburtstag
---------+-----------+------------
neumann | alfred e. | 1999-09-09
neumann | alfred e. | 1999-09-09
neumann | alfred e. | 1999-09-09
neumann | alfred e. | 1999-09-09
so count() should return 4 not 0:
lwm=# select count(*) from teilnehmer where nname = 'neumann' and vname = 'alfred e.' and geburtstag = '1999-09-09';
count
-------
4
My DateFormatter is defined like so:
let dateFormatter = ISO8601DateFormatter()
dateFormatter.formatOptions = [.withFullDate, .withDashSeparatorInDate]
And finally the attribute "birthday" in my model:
...
#Field(key: "geburtstag")
var gebTag: Date
...
I inserted the 4 alfreds in my database using the model and fluent, passing the birthday "1999-09-09" as a String and fluent inserted all records correctly.
But .filter(\.$gebTag == gebTag) seems to return constantly 'false'.
Is it at all possible to use .filter() with data types other than String?
And if so, what am I doing wrong?
Many thanks for your help
Michael
The problem you've hit is that you're storing only dates whereas you're filtering on dates with times. Unfortunately there's no native way to store just a date. However there are a few options.
The easiest way is to change the date field to a String and then use your date formatter (make sure you remove the time part) to convert the query option to a String.
I am guessing slightly here, but I suspect that your table was not created by a Migration? If it had been, your geburtstag field would include a time component as this is the default and you would have spotted the problem quickly.
In any event, the filter is actually filtering on the time component of gebTag as well as the date. This is why it is returning zero.
I suggest converting the geburtstag to a type that includes the time and ensuring that the time component is set to 0:00:00 when you store it. You can reset the time component to 'midnight' using something like this:
extension Date {
var midnight: Date { return Calendar.current.date(bySettingHour: 0, minute: 0, second: 0, of: self)! }
}
Then change your filter to:
.filter(\.$gebTag == gebTag.midnight)
Alternatively, just use the static method in Calendar:
.filter(\.$gebTag == Calendar.startOfDay(for:gebTag))
I think this is the most straightforward way of doing it.

Optaplanner. School timetabling. Force first lession

I'm trying to add constraints to School timetabling example. For example: "all groups should have the first lesson".
I tried EasyScore and Streaming - no success. EasyScore cant finds a proper solution, shuffles lessons a lot. Streaming gave me an error: Undo for (Lesson(subj...)) does not exist
Code for Streaming:
from(Lesson::class.java)
.filter { it.timeslot != null }
.groupBy({ it.studentGroup }, { it.timeslot!!.day }, ConstraintCollectors.toList())
.filter { group, day, list ->
list.any { it.timeslot!!.number != 1 }
}
.penalize(
"Student must have first lesson",
HardSoftScore.ONE_HARD
) { group, day, list -> list.count { it.timeslot!!.number != 1 } },
Looks like I'm thinking the wrong direction.
https://github.com/Lewik/timetable
Any help will be greatly appreciated.
update: fixed == -> =!
As far as I understand it, I don't think you're enforcing what you intend to enforce. From what I make from your source code, you penalize every studentgroup's first lesson of the day.
What you should do to enforce the intended goal, is to penalize every studentgroup that does NOT have a timeslot with number == 1 but DOES have one (of the same day) where timeslot number != 1.
So something like :
join all Lesson.class instances with all Lesson.class instances where the first lesson's studentGroup equals the second lesson's studentGroup AND the first lesson's timeSlot's day equals the second lesson's timeSlot's day. You obtain a BiConstraintStream<Lesson, Lesson> this way...
from this, filter all Lesson.class instances where the first lesson's timeSlot's number is less than the second lesson's timeSlot number
then penalise the remaining where the first lesson's timeSlot number differs from 1. That equals penalising all of a studentGroup's days where they have some lesson that day without having any lesson that day during the first timeslot.
If I understood you correctly, that's what you wanted ?
I don't know the real source of the problem, but it's about hashCode. The exception was thrown because HashMap with Object key can't find by that Object.
Lesson class:
#Serializable
#NoArg
#PlanningEntity
data class Lesson(
val subject: String,
val teacher: String,
val studentGroup: String,
#PlanningVariable(valueRangeProviderRefs = ["timeslotRange"])
var timeslot: TimeSlot? = null,
#PlanningId
val id: String = UUID.randomUUID().toString(),
)
The implementation above will not work. It could be fixed if I remove data or add override fun hashCode() = Objects.hash(id). #PlanningId does not help here. Kotlin generates hashCode for data classes and seems it not working with optaplanner (or vise versa)
How about using .ifNotExists()?
First, convert student group from a String into a class and add #ProblemFactCollectionProperty List<StudentGroup> on your solution, then do
from(StudentGroup.class)
.ifNotExists(from(Lesson.class).filter(Lesson::isFirstTimeslot),
equals(this -> this, Lesson::getStudentGroup)
.penalize(...);

DDD - Update Value Object in Database with FK dependencie

I am reading about DDD and I have learned that Value Object is immutable, if you want to change it, you will have to create a new one.
I have just read the information on How are Value Objects stored in the database? , it works well for Address class and I also read https://cargotracker.java.net/ and https://gojko.net/2009/09/30/ddd-and-relational-databases-the-value-object-dilemma/. But I want to do something different .
I am working on a billing system , it has 4 tables/classes
TPerson - fields: id_person, name -> <<Entity>>
TMobile - fields: id_mobile, number -> <<Entity>>
TPeriod - fields: id_period, id_person, id_mobile, begin_date, end_date -> <<Value Object>> (I think, because the dates can be change)
TCall - field: id_call, id_period, etc... -> <<Value Object>>
The table TCall has many records, if I change the period record dates (Value Object, table TPeriod) it will create another Object Period then id_period will change(delete, insert a record) , but the foreign key in table TCall will be violated. How Could I implement the period class ? if i implement as a value object , it will be immutable and turns out I will not be able to change anything whatsoever.
Thanks,
Fernando
if it's a value object you don't have a period table/id.
A value object is just a grouping of certain fields. For example a call might have a start time, an end time, and then you could create a Duration Value object with starttime and end time from the call table. In your java code it would be then more convenient to talk about the call duration instead of the start/end time separately.
However, it certainly could make sense to make period an entity, but then period 201601 probally always have the same start/end time and you wouldn't need to make changes to it. And if you did you make changes to the entity directly and keeping the ids in tact.
Thank for your help,
I have this situation:
TPerson - fields: id_person = 1 , name = "John"
TMobile - fields: id_mobile = 100, number "555-0123"
TPeriod - fields: id_period = 1000, id_person = 1 , id_mobile = 1, begin_date = "2016-01-01", end_date = "2049-12-31"
TCall - field: id_call = 1, id_period = 1000
The period is a relation between TPerson and TPeriod, in this example John has a mobile between "2016-01-01" and "2049-12-31". On the table TCall there are John's calls record, but if i replace the period (TPeriod table) end_date to "2016-02-01", from my understanding the end_date will be inconsistent, it turns out i cann't replace because it's a value object, not a entity. I considered to implement like this.
// Create a class DatePeriod
public class DatePeriod {
private final begin_date;
private final end_date;
DatePeriod() {}
public static DatePeriod of(Date begin_date, Date end_date) {
this.begin_date = begin_date;
this.end_date = end_date;
}
// implement equals / hashcode...
}
// Period class
public class Period {
int id;
// others mappings id_person / id_mobile
DatePeriod datePeriod;
}
Still, i will have to update datePeriod attribute
Thank you for your attention to this matter