I’m a data analyst in the insurance industry and we currently have a program in SAS EG that tracks catastrophe development week by week since the start of the event for all of the catastrophic events that are reported.
(I.E week 1 is catastrophe start date + 7 days, week 2 would be end of week 1 + 7 days and so on) then all transaction amounts (dollars) for the specific catastrophes would be grouped into the respective weeks based on the date each transaction was made.
Problem that we’re faced with is we are moving away from SAS EG to GCP big query and the current process of calculating those weeks is a manually read in list which isn’t very efficient and not easily translated to BigQuery.
Curious if anybody has an idea that would allow me to calculate each week number in periods of 7 days since the start of an event in SQL or has an idea specific for BigQuery? There would be different start dates for each event.
It is complex, I know and I’m willing to give more explanation as needed. Open to any ideas for this as I haven’t been able to find anything.
I'm trying to find an efficient way to calculate the booked times for a user(object), given a list of free/available times for the same user\object.
I have an object that will return the "available" times for a given specific day. The duration/end time is fixed to 10 minutes.
Example Starting data:
12/23/2020 8:00 AM
12/23/2020 9:00 AM
12/23/2020 1:00 PM
In this case I need to generate the "unavailable" times and insert them into a database with a fairly simple schema:
start_date | end_date | start_time | end_time
The inserting is fairly trivial, i'm having a hard time determining the best way to calculate the unavailable timespans.
Using the example above i would need to generate the following timespans:
12/23/2020 12:00 AM - 7:59 AM
12/23/2020 08:11 AM - 8:59 AM
12/23/2020 09:11 AM - 12:59 PM
12/23/2020 1:11 PM - 11:59 PM
Any frameworks or libraries that can do the heavy lifting on this for me? Is it possible to solve this problem without looping through the results and calculating all of the offsets?
To anyone asking "why" - hooking together two legacy systems, one system returns the available appointments for a given date this needs to be plumbed into a system that needs the un-available appointments for a given date.
Well, first I written more tour booking systems then I can shake a stick at.
The one Rosetta stone that holds true?
You don't want to generate or have booking slots that are NOT being used in the system PERIOD!!!
Thus you ONLY ever enter into the system a valid booking (starttime, and end time). And that startTime should be a datetime column - this will VAST reduce the complexity of your queries. Given you have date and separate time? Well, then your queries will be more complex - I'll leave that to you.
Given the above? The simple logic to find a booking collision in ALL cases is this:
A collision occurs when:
RequestStartDate <= EndDate
and
RequestEndDate >= StartDate
Now in above, I assume date values, or datetime values.
So if I want a list of any booking for today?
RequestDDTStart = 2020-12-23 9 AM
RequestDTEnd = 2020-12-23 5 PM
And thus any collision can be found with this:
strWhere= dtRequestStartDate <= BookingEndDate" & _
" and dtRequestEndDate >= BookingStartDate"
Now, assumging .net, then above would be something like this as parameters
strWhere= #dtRequestStart <= BookingEndDate" & _
" and #dtRequestEnd >= BookingStartDate"
So, above would return all bookings for today 9 am to 5 pm
A REMARKABLE simple query! Now of course the above query could/would include the exam room, or hotel room or whatever as an additional criteria. But in ALL cases the above simple query returns ANY collision for that 9 am to 5 pm.
And the beauty of this system? As long as you never allow a over-lap into the booking system, then you can book a 10 minute or a 20 minute or a 30 minute session as ONE entry into the database. I would thus not need to create 3x 10 minute slots.
So, this means you NEVER have to create booking slots. The whole system will and can be driver with a simple start + end booking record. And as noted, then you can book 1 hour, or 40 minutes. Your input (UI) can simple limit the time span to increments of 10 minutes - but that's the UI part.
Now I suppose to display things in 10 minute increments on a screen? Well, then you would have to submit 6 requests per hour to "display" the time slots. For a whole day, that suggest for 9 am to 5 pm, you would have to run 8 x 6 = 48 requests to get a list of 10 minute increments. But then again, you COULD just show the existing bookings for a day, and allow new bookings to be added - but don't allow if there is a over lap.
So, as noted, the concept here is you don't really need "slots" in the database. I suppose you could try slots, but it makes the code a HUGE mess to deal with. if you ONLY ever store the start + end? Then I can say move the booking to another day by JUST changing the date. Or I can extend a booking from 10 minutes to say 20 or 40 minutes - and ONLY have to change the end time. As long as no overlap occurs with the above simple "test", then I can simple change the booking to be 40 minutes in length - and ZERO code to update multiple slots is required. And same goes for reducing a booking from 40 minutes to 10 minutes. Again ONLY the end time need be reduced - a ONE row update into the database.
So if at all possible, I would dump the concept of having "slots" in the database. I might consider such a design if a booking was only ever 10 minutes. But if 10 or 20 or 30 is allowed, then you don't need to store ANY un-used slots in the database, but ONLY ever store a valid booked slot. Empty un-used time can thus ALSO be found with the above query. (if the query returns records - then you can't book).
So display of free time in some UI becomes more of a challenge, but showing bookings that span 10 or 20 or whatever minutes is far more easy, and as noted, you can even change a whole booking to a different room by a ONE row update of the room ID. If no collision occurs, then you allow this booking - and you achieve this result by ONLY updating one simple booking record that represents that start + end time.
and this means you also NEVER store the booking totals in the database - you query them!
I also found that if I say store any booking totals in the database? Well, with complex code, we always found that the totals often don't match perfect. So then we wind up writing a routine to go though the data, sum up the totals and write those out.
But, if you never store any booked totals (say people on a bus, or people in a given hotel), then while the query for such display is somewhat more difficult, it becomes dead simple to remove a person from say a tour by simple null out of the tourID.
So, this display shows the above concepts in Action. And the available rooms in the hotel, people booked on bus, and even totals for "group tours" are ALL values NOT stored in the database:
So in above, people booked on bus, booked in rooms, and rooms used? All those values are NOT stored in the database. And no slots exist either. So if we have a bus, then we set the capacity of 46, but we do NOT create 46 slots to book into. So be it a bus, a hotel, a medical exam room? You don't create booking slots ahead of time, but simply insert bookings with a start + end, and then query against that concept.
So, to find a total on a bus (or say in a exam room), I query to find the total for that day. And if I want to move a group booking of 4 people from one bus to another? Then one FK update to the given bus they are on allows the whole system to "cascade" the existing values in the system. And same goes for moving a person from exam room #1 to #5. You only have to update the FK value of the exam room. If no collisions occur, then this again is a one row update. If you have multiple exam rooms, and multiple slots, then what should be a simple one row update in the database becomes a whole hodge podge of now having to update multiple booking slots with whacks of code.
So you book "use" of resources "into" a "day" a "bus" a room, but it is the act of that booking that consumes the time slots - not that you pre-create records or timeslots for each "range". This thus allows you to leverage the relatonal database model, and reduce huge amounts of code - since you not coding against "slots", but only that a exam room is open from 10 am to 4 pm. That available room for that day is thus ONLY ONE record you create in the system, and then you are now free to book into that one day given room range. The bookings into that one room for the day can be 10 minutes, or 40 minutes - but it ONLY one record being added into the database to achieve this goal (booking).
Regardless of the above, that simple collision query works for any collision (including a whole overlap, inside a existing span, or even the end or start overlaps any booking. And that query is dead simple - and it works for all collisions. So I don't have a library to share, but that simple booking collision finder query can thus drive the whole system based on that kind of simple query.
I have a challenge ahead of me. I have looked at this for a couple of days now in a trial and error sense and am getting tired of not getting it… My SQL knowledge is very very basic.
Each quarter I have to report on the questions below (of course the date period changes):
The number of doctors with whom the designated body has a prescribed connection at 31st December.
The number of doctors due to hold an appraisal meeting in the reporting period (from 1st October to 31st December 2017).
The number of those doctors above who held an appraisal meeting in the reporting period.
The number of those doctors above who did not hold an appraisal meeting in the reporting period.
I have three lists:
A list of staff responsible to the designated body. In a linked table- ‘GMC_Main’ Field- GMC Ref No
A list of all appraisals that have ever taken place (historical and ones performed by staff not responsible to us). In a query called-
‘Latest Appraisal’ Fields- ProfNum, MaxOfAppDate
List of emails in a linked table- ‘MARS_Core’ Fields- ProfNum, EmpSurname, EmpFirstName, EmpEmail
Things to consider
ProfNum and GMC Ref No are the Unique identifiers for each member of
staff.
GMC_Main is the list of all staff that need to be considered in the
report. So should have a row regardless of the results from the other
tables.
All appraisals are valid for 365 days. So The date 1 year in the future from that in MaxofAppDate will be needed to calculate expiry in the period.
Due to software limitations I only have available Access 2016.
I need to count all that should have taken place, including ones that
are still overdue from previous quarters.
Count all the ones that actually took place in the period.
Be able to contact all the ones that did not achieve and appraisal.
At year end (31 March) do this for the entire year and not just the
quarter…
Fuff!
Each time I approach this problem I am missing a group of people or feel I am doing it in a very wrong-handed way.
If anyone could help, then that would be amazing. This is a little beyond me.
I have a report that, according to users, started miscalculating dates in one field in November 2015. After some digging around, I found that one of the tables the field referenced seemed to have an end date on 2015-10-31.
The "D" field seems to represent the day of the week, with Sunday being day 1 and Saturday being 7.
Is there a way to extend the calendar so that it ends further into the future, for example 2049-12-31?
Our calendar table, for a variety of reasons, goes the the end of the current year. We have written a query that adds a new year to this table. This query takes care of most of the fields in that table. It does not touch the holiday field. That is updated manually through a web page.
We send ourselves reminders. Starting in March, we send monthly reminders that we should think about adding another year. After ensuring that the database segment has space, and that none of the definitions, such as fiscal periods, have changed, we run the query that adds a year.
Later in the year we start mailing ourselves reminders about the holidays. Then we check to see if HR has declared them, and if so, update the records accordingly.
This meets our business requirements. Yours will be different of course.
I've started thinking about an employee shift management application to handle the shifts (who works when, trading, etc) at my current workplace (that uses pen and paper and hasn't got anyway for us employees to communicate about changes without going through the boss and be on site).
Currently the shifts are modeled loosely as:
There is a recurring 4 week period (from Monday week 1 to Sunday week 4)
There is a template for placing employees in this 4 week period
Every 4 months (ie 3 times a year) the 4 week template is projected over the next 4 month period
The shifts have been the same for a long time and it seems many employees would prefer to have them changed (I can say this by the requests for change that come in every time a new 4 month is set).
What I'm aiming at are the models:
Shift_group_tpl (the 4 week period above)
Shift_tpl (a single shift in the 4 week period, including info on who defaults to work this shift)
Shift_group (a set period of time whit actual shifts)
Shift (a set shift whit a real time period and an employee - and the possibility to be changed both in start_time, end_time and employee)
I've thought of a way to do this with recurring iCalendar events: Creating RRULE's (without an endtime) and then calculate (using temporary start and end times) if that specific Shift_group_tpl could be used within a real Shift_group. (The problem with this approach is that I can't figure out how to trim the Shift_group_tpl's to fit into the start or end of a Shift_group.)
What I'm looking for are some other perspectives or ways of doing it or even just a pat on the shoulder letting me know that I'm on the right track (and then giving advice on the trimming problem).
/iole1
What I'm aiming at are the models:
Shift_group_tpl (the 4 week period above)
Shift_tpl (a single shift in the 4 week period, including info on who defaults to work this shift)
Shift_group (a set period of time whit actual shifts)
Shift (a set shift whit a real time period and an employee - and the possibility to be changed both in start_time, end_time and employee)
You have "sql" as a tag for this post? So im guessing you want these as SQL tables?
By the sounds, the problem is that your considering the data you have, rather than the abstract concepts you need to store that data. Which is what you'd need to do to create an application. (Most likely a "Shifts" table, rather than the four tables above).
There is little information here to help, Consider refining your thoughts and ask another question.