TL;DR Is there a simple way to handle daylight saving timezone for historical data ?
We have a list of timezone along with thier offset to UTC which is changed for certain zones as per DST timings. All the data is stored in UTC but displayed as per the user timezone. The issue is when DST is active or reverted the historical data view also changes . This creates a discrepancy in customer reporting in terms of billing and order counts.
Is there a standard way of handling this ?
Problem overview
I'm working on a simple app for reminding the user of weekly goals. Let's say the goal is to do 30 minutes of exercise on specific days of the week.
Sample goal: do exercise on Mon, Wed, Fri.
The app also needs to track past record, i.e. dates when the user did exercise. It could be just dates, e.g.: 2019-09-02, 2019-09-05, 2019-09-11 means the user did exercise on these days and did not on the others (doesn't need to be on "exercise goal" days of the week).
The goal can change in time. Let's say today is 2019-09-11 and the goal for this week ([2019-09-09, 2019-09-15]) is Mon, Wed, Fri but from 2019-08-05 to 2019-09-08 it was Mon, Thu (repeatedly for all these weeks).
I need to store these week-oriented goals and historic exercise of data and be able to retrieve the following:
The goal days for the current week (or any week, let's say I can compute start and end day for any week given a date).
Exercise history for a larger range of days together with goal days for that range (e.g. to show when the user was supposed to exercise and when they actually did in the last month).
Question
How to best store this data in SQL.
This is a little bit academic because I'm working on a small Android app and the data is just for a single user. So there will be little data and I can successfully use any approach, even a very clumsy one will be efficient enough.
However, I'd like to explore the topic and maybe learn a thing or two.
Possible solutions
Here are two approaches that come to my mind.
In both cases I would store exercise history as a table of dates. If there is an entry for that date it means the user did exercise on that day.
It's the goal storage that is interesting.
Approach 1
Store the goals per-week (it's SQLite so dates are stored as strings - all dates are just 'YEAR-MONTH-DAY'):
CREATE TABLE goals (
start_date TEXT,
exercise_days TEXT);
"start_date" is the first day of the week,
"exercise_days" is a comma-separated list of weekdays (let's say numbers 1-7).
So for the example above we might have two rows:
'2019-08-05', '1,4'
'2019-09-09', '1,3,5'
meaning that since 2019-08-05 the goal is Mon, Thu for all weeks until 2019-09-09, when the goal becomes Mon, Wed, Fri. So there is a gap in the data. I wouldn't want to generate data for weeks starting on 2019-08-12, 2019-08-19, 2019-08-26.
With this approach it is easy to work with the data week-wise. The current goal is the one with MAX('start_date'). The goal for a week for a given date is MAX('start_date') WHERE 'start_date' <= :date.
However it gets cumbersome when I want to get data for the last 3 months and show the user their progress.
Or maybe I want to show the user the percentage of actual exercise days to what they set as their goal in a year.
In this case it seems the best approach is to fetch the data separately and merge it in the application (or maybe write some complex queries), processing week by week. This is ok performance-wise because the amount of data is small and I rarely need more than a handful of weeks.
Approach 2
Store goals in such a way that each goal day is a record:
CREATE TABLE goals (
day TEXT,
);
"day" is a day when the user should exercise. So for the week starting 2019-09-09 (Mon, Wed, Fri) we would have:
'2019-09-09'
'2019-09-11'
'2019-09-13'
and for the week starting 2019-08-05 (Mon, Thu) we would have:
'2019-08-05'
'2019-08-09'
but what for the weeks in-between?
If my app could fill all the weeks in-between then it would be easy to merge this data with the exercise history and display days when the user was supposed to exercise and when they actually did. Extracting the goal for any given week would also be easy.
The problem is: this requires the app to generate data for the "gap" weeks even if the user doesn't tweak the goal. This can be implemented as a transaction that is run each time the app process starts. In some cases it could take noticeable time for occasional users of the app (think progress bar for a second).
Maybe there a smart way to generate the data in-between when making a SELECT query?
I don't like the fact that it requires generating data. I do like the fact that I can just join the tables and then process that (e.g. compute how many exercise days there were supposed to be in August and how many days the user did actually exercise and then show them percentage like "you did 85% of your goal" - in fact I can do this without joining the tables).
Also, it seems this approach gives me more flexibility for analysis in the future.
But is there a third way? Or maybe I am overthinking this? :)
(I am asking mostly for the way of organizing the data, there's no need for exact SQL queries)
Perhaps I'm over-thinking this, but if a goal can have multiple components to it, and can change over time I'd have a goal header record, with the ID, name and other data about the goal as a whole, and then a separate table linked with the components of that goal which are time-boxed, for example:
CREATE TABLE goal_days (goal_day_ID INT,
goal_ID INT,
day_ID INT,
target_minutes INT,
start_date TEXT,
end_date TEXT)
I'd have thought that allows you to easily check against the history to map against each day of the goal - e.g. they got 100% of the Mondays, but kept missing Thursday - however when the goal was changed to Friday instead they got better.
I am looking to save current UK time and problem is they have two timezone GMT and BST accordingly day saving light ?
I am saving time in SQL date time column from mobile app and SQL server. I am insert time using SQL.
I am storing all dates to SQL Server as a UTC date time. I have Time Zone Id for each user stored in user profile as well.
Now when user requests data back, I want to display local time of user for each record using the Time Zone I have stored in profile for the particular user.
What is an easiest and optimized way (as I am processing heaps of records at the same time) to convert all dates and time to particular time zone on the fly while returning data? Either in SQL or in C# would be fine...
Very important question is, let's say there is a record created from Sydney when there was Day Light Saving "ON" and now Day Light Saving is "OFF". As the record was created when Day Light Saving was "ON", will it still convert the same time or will it return conversion as per current time zone status (which is Day Light Saving is "OFF")???
People only see those records which they had created from the particular
let's say there is a record created from Sydney when there was Day Light Saving "ON" and now Day Light Saving is "OFF". As the record was created when Day Light Saving was "ON", will it still convert the same time or will it return conversion as per current time zone status (which is Day Light Saving is "OFF")
The record contains an UTC date and time. This is going to fall into a DST ON or DST OFF period, deterministic. Is irrelevant whether the DST is in effect now. The opposite (storing local time, trying to extract UTC) is undetermined because of the overlap times when the DST changes (a small range of local times cannot be deterministically converted to UTC if they fall into the 60 mins that occurs twice when DST come into effect, assuming a 60 min DST).
As for the question: transform the date in your presentation layer. Use TimeZoneInfo.ConvertTimeFromUtc.
OK, let's say I am writing a calendar application in which a user can create an event. An event can occur in the following situations:
1 time only.
Every Period of time keep reply, until a day.
Every Period of time keep reply, until a day, except some days.
Every some kind of week day.
Let me explain in a detailed example:
1 time only: 24/8/2011 ONLY.
Every Period of time keep reply, until a day: 25/8/2011 Start, and every week end will have this event, until 30/10/2011.
Every Period of time keep reply, until a day, except some days: 25/8/2011 Start, and every week end will have this event, until 30/10/2011, except the 10/09/2011 and 17/09/2011.
Every some kind of week day, .
Every some kind of week day, except some days.
Martin Fowler wrote a nice big article about just this topic a few years ago. Rather than repeat his analysis, I'll just point you at the pdf. It's called Recurring Events For Calendars.