BLUF: Having trouble implementing rrule in web app using UTC datetime variables since byweekday does not translate well between time zones. For example, when using byweekday attributes on rrules where local time zone datetime falls on a day that is not the same as the UTC equivalent of that day (because of UTC offset), the rrule "breaks" or cannot be easily shifted according to the time zone offset (i.e., the shift of day is not handled automatically).
Advice: Given multiple sites in multiple time zones accessed remotely, would it be reasonable to use local time internally according to time zone of the site and not use UTC internally?
TLDR:
I have an application that is deployed to multiple sites each in a different time zone. The application is accessed via a web app that performs user I/O according to the time zone of the site (site's local time). I am currently maintaining all datetime data internally using UTC. My recurrence rules are implemented using rrule.js (https://github.com/jakubroztocil/rrule) on the client side, and dateutil.rrule (http://labix.org/python-dateutil) on the server side.
When I have an rrule that uses a byweekday setting, the UTC offset from local to UTC is calculated correctly, but the byweekday setting is no longer correct (the byweekday day is not shifted). For example, the following rrule is supposed to be run at 11:55PM local time each Sunday (night).
DTSTART:20210411T065500Z
RRULE:FREQ=WEEKLY;UNTIL=20220101T080000Z;INTERVAL=1;WKST=MO;BYDAY=SU
UTC time is correct given a local time zone offset of -07:00 (=23:55 local time, the night before). But, the rrule set is "wrong" as the first occurrence is given as:
Sun, 11 Apr 2021 06:55:00 GMT
So that when the rrule DTSTART is shifted back to the local time zone (-07:00 offset), the rule runs or triggers the day before which is a Saturday. The rule runs at the right time but on the wrong day because the time zone hour shift is correct, but the day shift does not happen. I would rather not have to do the day shifting manually. Seems easier to just implement the internal datetimes in the target time zone. Maybe I've missed a nuance in rrule management?
Related
I’m working with a table that uses date time offset. I have a value that looks like 2020-01-02 13:30:00 -07:00.
Is the time in my time zone 13:30 or do I need to subtract 7 hours from it. I saw people do it differently on YouTube.
In MS SQLServer, the last section of the string representation that you posted of the DateTimeOffset describes the time zone. So, if you are currently located in time zone -7:00 (e.g., Arizona, USA), then the time portion of the string refers to your local time, not UTC. See the Microsoft documentation:
For example, 1999-12-12 12:30:30.12345 -07:00 should be represented [in UTC] as
1999-12-12 19:30:30.12345Z
Someone would subtract the offset from the number only if they want to manually get the UTC value, but that would might produce errors if the data come from a daylight saving time (DST) region, so you would need to enforce DST handling at the time of entry. SQLServer already stores the data in UTC behind the scenes:
The data is stored in the database and processed, compared, sorted,
and indexed in the server as in UTC.
I can't seem to find a question/answer that works for what I'm trying to achieve. Currently, this is how my DB outputs a timestamp:
2015-08-18T19:43:04.738-06:00
However, I would like it to appear as such in the column:
2015-08-18T19:43:04.738 America/Denver
Google has recently changed their formatting options and instead of downloading the output and performing a find/replace, I want an output that doesn't require additional work. I looked on SO and have tried using trim and replace but having no luck.
Thanks for the help in advance!
For whatever reason, the one we've used since February (third from the bottom) no longer works.
2015-08-18T19:43:04.738-06:00 is not quite the right format. Google does not accept milliseconds (which is annoying if they don't just ignore it). You need to send 2015-08-18T19:43:04-06:00. They may have become more strict in what they accept.
Try date_trunc('second', yourtime).
It's not possible to accurately translate an offset like -0600 to a time zone like America/Denver. They say two different things.
-0600 says, with absolute certainty, that this time is 6 hours behind UTC. 12:00:00-06:00 and 18:00:00Z (Z represents UTC) are the same time.
America/Denver means to interpret this timestamp under the rules applicable to the city of Denver, Colorado, USA at that time. To figure out what time it is in UTC you need to look up the offset rules for Denver, Colorado, USA. The offset will change depending on the time of year, usually because of daylight savings time. Because the rules change, it's important to apply the rules as they were at that time.
For example, 2006-03-15 12:00 America/Denver is -0700. But the next year on 2007-03-15 12:00 America/Denver is -0600. Between 2006 and 2007 the daylight savings time rules in the US changed.
Whereas -06:00 avoids all that and simply says the time is offset from UTC by six hours.
You could fake it by simply replacing the offset with America/Denver. So long as you're only sending recent times that should work. You'll be off by at most an hour. But don't do that.
Unless Google Ads specifically needs a time zone there's no point in sending them one. Internally, Postgres is storing your times in UTC anyway and translating them to your server's time zone, America/Denver. Send Google UTC. And, as noted above, chop off the milliseconds.
select date_trunc('second', '2015-08-18T19:43:04.738-06:00'::timestamp with time zone at time zone 'UTC') as datetime;
datetime
---------------------
2015-08-19 01:43:04
I've a problem with storing java.util/sql.Date in PostgreSQL using hibernate: I'm storing dates as timestamp without time zone type. Now, in most cases everything is alright, as I'm mapping it in entities as java.util(sql).Date.
However, I encountered one problem I don't know how to overcome yet:
On March 30th there was a time change (Daylight Saving Time started). In my country it meant switching from local 2am to local 3am.
In my database I have few entries with times between 2 and 3 am, such as
"2014-03-30 02:15:55"
Now, what Java does is it takes this date and displays it as 3:15:55 am, because of the time change. However, I desperately need it to be 02:15:55, exactly how it's stored in the database (basically meaning don't use the +02:00 timezone BEFORE 3am, but use it AFTER). I'm afraid about 26th October as well, that's when the DST ends.
Is this possible using Hibernate and/or Spring? I'd love a global config for such case.
Best,
Marcin
You need to store dates in UTC timezone to ignore DST. See this question How to store date/time and timestamps in UTC time zone with JPA and Hibernate for solutions
I usually have an "interceptor" that right before reading/writing from/to the database does DateTime conversion (from UTC to local time, and from local time to UTC), so I can use DateTime.Now (derivations and comparisions) throughout the system without worrying about time zones.
Regarding serialization and moving data between computers, there is no need to bother, as the datetime is always UTC.
Should I continue storing my dates (SQL 2008 - datetime) in UTC format or should I instead store it using DateTimeOffset (SQL 2008 - datetimeoffset)?
UTC Dates in the database (datetime type) have been working and known for so long, why change it? What are the advantages?
I have already looked into articles like this one, but I'm not 100% convinced though. Any thoughts?
There is one huge difference, where you cannot use UTC alone.
If you have a scenario like this
One server and several clients (all geographically in different timezones)
Clients create some data with datetime information
Clients store it all on central server
Then:
datetimeoffset stores Local time of the client and ALSO offset to the UTC time
all clients know UTC time of all data and also a local time in the place where the information originated
But:
UTC datetime stores just UTC datetime, so you do not have information about local time in the client location where data originated
Other clients do not know the local time of the place, where datetime information came from
Other clients can only calculate their local time from the database (using UTC time) not the local time of the client, where the data originated
Simple example is flight ticket reservation system ... Flight ticket should contain 2 times:
- "take off" time (in timezone of "From" city)
- "landing" time (in timezone of "Destination" city)
You are absolutely correct to use UTC for all historical times (i.e. recording events happened). It is always possible to go from UTC to local time but not always the other way about.
When to use local time? Answer this question:
If the government suddenly decide to change daylight savings, would you like this
data to change with it?
Only store local time if the answer is "yes". Obviously that will only be for future dates, and usually only for dates that affect people in some way.
Why store a time zone/offset?
Firstly, if you want to record what the offset was for the user who carried out the action, you would probably be best just doing that, i.e. at login record the location and timezone for that user.
Secondly if you want to convert for display, you need to have a table of all local time offset transitions for that timezone, simply knowing the current offset is not enough, because if you are showing a date/time from six months ago the offset will be different.
A DATETIMEOFFSET gives you the ability to store local time and UTC time in one field.
This allows for very simple and efficient reporting in local or UTC time without the need to process the data for display in any way.
These are the two most common requirements - local time for local reports and UTC time for group reports.
The local time is stored in the DATETIME portion of the DATETIMEOFFSET and the OFFSET from UTC is stored in the OFFSET portion, thus conversion is simple and, since it requires no knowledge of the timezone the data came from, can all be done at database level.
If you don't require times down to milliseconds, e.g. just to minutes or seconds, you can use DATETIMEOFFSET(0). The DATETIMEOFFSET field will then only require 8 bytes of storage - the same as a DATETIME.
Using a DATETIMEOFFSET rather than a UTC DATETIME therefore gives more flexibility, efficiency and simplicity for reporting.
My location in Sydney, Australia. The dates that I explain will be in UK or Australia date format.
Observe the following:
2010-04-15 04:30:00.000 => 15/04/2010 14:30:00 EST (UK date format - Add 10 hours)
2010-11-05 01:00:00.000 => 05/11/2010 12:00:00 EST (UK date format - Add 11 hours)
Both these times are retrieved from the database in UTC format and then calculated on the Web level whether +10 or +11 hour is applicable.
In Australia, Daylight Savings Time (DST) transition dates vary year by year. The transition dates are usually Early April and Late October.
So how accurate would the Web calculation be? If this year the transition date is a few days later (say 03/04/2010), but the Web calculation bases on a fixed date (say 01/04/2010), wouldn't that mean that the days in between will be off by 1 hour when displayed (due to the fixed calculation nature to a specific day of the month)?
I believe the transition dates is not pre-determined and is actually announced to the public. Is that assumption true?
If not (that means the DST dates are pre-determined), would I be able to do the calculation outside the Web level (on the SQL/Database level)?
The database is SQL Server 2005 and I'm using Report Definition Language (RDL) to display the fields in UTC time. If SQL/database level is not the best way, how do I work out +10 or +11 and format the time accordingly to show the right time?
Thank you.
The database is a bad choice for it: it has less information than c# or .net to work it out. .net uses the registry which is kept upto date periodically by patches. SQL Server would have to have a table with date ranges and offsets.
The transitions are fixed because of scheduling (flights, trains ,whatever). IIRC it only changed once at short notice recently in Australia for some Olympic games and it caused chaos around the world. In 2007 the US changed but this was known in advance.
By fixed, it's the "last Sunday" type fixed even if the date varies.
I would leave it in the web code: the DB does not know where your caller is for example, the web site can work it out.
The problem is whoever wrote this app does not quite understand UTC, its value, and how to use it. The database is the correct location for the dat, but the system is not using UTC as intended.
If you use UTC, then all your date arithmetic should use UTC. In the database. It is currently using a saved UTC and then converting at some (doesn't matter if seciond tier or third) other layer; some other library. Half UTC, and Half something else. Have you considered historic dates, as in what is the DATEDIFF() between 15 Feb 2010 and today ?
This eliminates the concern re DST in Australia or Greenland.; and concern re what date/time the changeover actually happens. Everyone is using Greenwich Mean Time for that particular day.
Do all you date arithmetic in the db, in UTC. And display the result (only) in the local time zone, which as you have it, is the web layer, based on the user.
Many systems have dropped that last step altogether, and display in UTC only, regardless of the user's time zone.
The database can handle DST for you. Use its time zone conversion functions to go from whatever zone you stored the dates in to whatever zone you want to get for the user.
MySQL has CONVERT_TZ(), I don't know what other RDBMS's have.