I’m working with a table that uses date time offset. I have a value that looks like 2020-01-02 13:30:00 -07:00.
Is the time in my time zone 13:30 or do I need to subtract 7 hours from it. I saw people do it differently on YouTube.
In MS SQLServer, the last section of the string representation that you posted of the DateTimeOffset describes the time zone. So, if you are currently located in time zone -7:00 (e.g., Arizona, USA), then the time portion of the string refers to your local time, not UTC. See the Microsoft documentation:
For example, 1999-12-12 12:30:30.12345 -07:00 should be represented [in UTC] as
1999-12-12 19:30:30.12345Z
Someone would subtract the offset from the number only if they want to manually get the UTC value, but that would might produce errors if the data come from a daylight saving time (DST) region, so you would need to enforce DST handling at the time of entry. SQLServer already stores the data in UTC behind the scenes:
The data is stored in the database and processed, compared, sorted,
and indexed in the server as in UTC.
Related
Databases often store a datetime without timezone as a separate type as a datetime with a timezone. As an example, I'll use BigQuery (though most databases store this the same):
DATETIME is a date/time that does not store timezone.
TIMESTAMP is a date/time that does store timezone.
I understand abstractly that "Dec 2 at 2:45 PM" is a different time in Japan as it is in New York, but I'm wondering why that even matters if all the dates are stored in UTC by the application. For example, if the value to be inserted is:
2021-12-02 14:45:00
Wouldn't that value be inserted as 2021-12-02 14:45:00 UTC in both data types? Or, would the 2:45PM be stored as "2:45 PM UTC" in the DATETIME type but would be stored as (if using EST) 2:45 PM EST --> 6:45 PM UTC in the TIMESTAMP type?
And if the value was:
2021-12-02 14:45:00 EST
Wouldn't that value also be inserted as 2021-12-02 18:45:00 UTC in both data types, and stored the same way? It seems to only 'timezone' is on the query-side, and can't that act either as a cursor-variable or some sort of metadata on the field (similar to a NULL check)? I guess I'm not following why the timezone-aware and no-timezone need to be stored as two different types if all date/times get stored as UTC anyways.
The SQL standard defines two types for a date with time-of-day:
TIMESTAMP (Also known more clearly in some databases such as Postgres as TIMESTAMP WITHOUT TIME ZONE)
TIMESTAMP WITH TIME ZONE
The first type is meant to purposely lack any context of time zone or offset-from-UTC. So noon on the 23rd of January next year, 2022-01-23 12:00, means noon anywhere and everywhere. It means noon on Tokyo Japan as well as noon on Toulouse France, and also noon in Toledo Ohio US. These are all obviously different moments, several hours apart. Therefore, this type cannot represent a moment, is not a specific point on the time line.
The second type does represent a moment, is a specific point on ne timeline. When you want to track actual moments, such as when a row was written to the database, or when a shipment arrived in a warehouse, use this type.
Unfortunately, the SQL spec says little about the various date-time types and behaviors. So the various database products vary widely in their support for these types and their interpretations of behavior.
In some databases such as Postgres, a value submitted to a column of the fist type (TIMESTAMP WITHOUT TIME ZONE) containing an indicator of zone or offset will have the date and time recorded as submitted. No adjustment is made. Any zone or offset input is ignored and discarded.
In some databases such as Postgres, a value submitted to a column of the second type (TIMESTAMP WITH TIME ZONE) containing an indicator of zone or offset will have its date and time adjusted to UTC before being written to the database. In such databases, this type is always in UTC, that is, represents a moment as seen with an offset of zero.
What is an offset? Merely a number of hours-minutes-seconds ahead of UTC (+) or behind UTC (-). A time zone, in contrast, is much more. A time zone has a name in Continent/Region format, and contains the history past, present, and future changes to the offset used by the people of a particular region as decided by their politicians.
So the TIMESTAMP WITH TIME ZONE type in databases such as Postgres is a misnomer. No time zone information is stored in the database. Any time zone or offset info submitted with the date and time is used to adjust to UTC. The zone/offset info is then discarded. So if remembering the zone originally submitted is important to you, you will need to store that yourself in a second column. Regarding the misnomer, you can think of the type as being TIMESTAMP WITH REGARD FOR SUBMITTED OFFSET OR TIME ZONE. But be clear, in databases such as Postgres your moment is stored in UTC, always UTC, and is retrieved as UTC, always UTC.
Unfortunately, there is a wrinkle here. All too commonly, tooling and middleware will inject a default time zone, adjusting the retrieved UTC moment into some time zone. While well-intentioned, this anti-feature creates the illusion that the value was stored with that time zone. But the values actually stored in UTC, at least with databases such as Postgres.
You asked:
2021-12-02 14:45:00 Wouldn't that value be inserted as 2021-12-02 14:45:00 UTC in both data types?
No.
In a column of a data type akin to the TIMESTAMP WITHOUT TIME ZONE, that date and time will be stored as submitted, a quarter to 3 PM on the second of December this year.
In a column of a data type akin to TIMESTAMP WITH TIME ZONE, the value stored may depend on the behavior of your particular database and your particular middleware, tooling, and driver. The behavior might simply assume you meant 2021-12-02 14:45:00 as seen in UTC, and store that. Or the behavior may assume you meant 2021-12-02 14:45:00 as seen in a particular time zone. And in databases such as Postgres, an adjustment to UTC would be applied before finally storing. You must study the documentation of your particular database, middleware, tooling, and driver to discover which behavior will be seen in your software. Be sure to conduct experiments to verify your understanding.
You asked:
2021-12-02 14:45:00 … Or, would the 2:45PM be stored as "2:45 PM UTC" in the DATETIME type but would be stored as (if using EST) 2:45 PM EST --> 6:45 PM UTC in the TIMESTAMP type?
”Likely yes”, for the first clause. But no EST involved at all. The date is stored as-is, 2021-12-02, along with the time-of-day as-is, 14:45:00. The EST part is ignored and discarded. (But experiment to verify this behavior in your particular tooling.)
And “maybe” for the second clause. As discussed in last bullet above, the behavior for TIMESTAMP WITH TIME ZONE may vary. Read docs, and conduct experiments.
You said:
though most databases store this the same
No, incorrect. That would be a very big “No”.
Databases vary widely in their support for date-time features, their kinds of date-time types, the names of their types, the technical details of their types, and the behaviors of the database server, middleware, drivers, and tooling.
Some older database systems have legacy data types supplanted by newer types, but all still supported, further complicating the picture.
You said:
I guess I'm not following why the timezone-aware and no-timezone need to be stored as two different types if all date/times get stored as UTC anyways.
You incorrectly assume that the “no-timezone” type stores in UTC. It does not.
That is what the “no-timezone” means: no regard for an offset or zone, no consideration for any offset or zone, no adjustment for any offset or zone, and no concept of offset or zone. The TIMESTAMP WITHOUT TIME ZONE type means simply, literally, a date, a time-of-day, and nothing more. Anything more than that is either (a) a figment of your imagination, or (b) interference by your middleware/tooling/drivers.
I would like to convert times from pacific to UTC. However, I must first convert the times to either PST or PDT depending on the date. Is there a SQL function that can do this, or does anyone have any advice for creating this function?
The link you provided is pretty much useless as a guide to timestamps. If you are going to work with timezones then store your timestamps in a timestamp with time zone field. The timezone will not actually be stored but the timestamp will be stored as a UTC. Whenever a timestamp is entered it is rotated to a UTC value. This makes working with value easier down the road. If you want to take into account DST transitions you will need to use full timezone names e.g. US/Pacific, as they cover the two offsets(PST/PDT) that constitute the standard/daylight savings timezones. As you found using the offset PST(-08) or PDT(-07) gets you a fixed offset regardless of date.
I have a date 2015-03-11T08:43:19.7810000-05:00 the I am inserting into a MSSQL2008 database table datetime field.
I have verified my SQL Database is currently on daylight savings.
The East Coast of the USA is -5 offset.
Immediately upon inserting into SQL the time changes to 9:43am, adding one hour.
This would tell me some UTC or time zone conversion is not working.
I have even had my DBAs look at this and cannot figure it out.
I believe we should be storing datetime with UTC offset, but cannot figure out why this just started happening after the DST change last weekend.
Thanks.
If you want the offset persisted, you'll need to store it into a datetimeoffset field.
datetime fields do not save the offset. Because you provided one, it is applied but then the time is being normalized to the local time zone, which is in UTC-4 for the date in question.
I usually have an "interceptor" that right before reading/writing from/to the database does DateTime conversion (from UTC to local time, and from local time to UTC), so I can use DateTime.Now (derivations and comparisions) throughout the system without worrying about time zones.
Regarding serialization and moving data between computers, there is no need to bother, as the datetime is always UTC.
Should I continue storing my dates (SQL 2008 - datetime) in UTC format or should I instead store it using DateTimeOffset (SQL 2008 - datetimeoffset)?
UTC Dates in the database (datetime type) have been working and known for so long, why change it? What are the advantages?
I have already looked into articles like this one, but I'm not 100% convinced though. Any thoughts?
There is one huge difference, where you cannot use UTC alone.
If you have a scenario like this
One server and several clients (all geographically in different timezones)
Clients create some data with datetime information
Clients store it all on central server
Then:
datetimeoffset stores Local time of the client and ALSO offset to the UTC time
all clients know UTC time of all data and also a local time in the place where the information originated
But:
UTC datetime stores just UTC datetime, so you do not have information about local time in the client location where data originated
Other clients do not know the local time of the place, where datetime information came from
Other clients can only calculate their local time from the database (using UTC time) not the local time of the client, where the data originated
Simple example is flight ticket reservation system ... Flight ticket should contain 2 times:
- "take off" time (in timezone of "From" city)
- "landing" time (in timezone of "Destination" city)
You are absolutely correct to use UTC for all historical times (i.e. recording events happened). It is always possible to go from UTC to local time but not always the other way about.
When to use local time? Answer this question:
If the government suddenly decide to change daylight savings, would you like this
data to change with it?
Only store local time if the answer is "yes". Obviously that will only be for future dates, and usually only for dates that affect people in some way.
Why store a time zone/offset?
Firstly, if you want to record what the offset was for the user who carried out the action, you would probably be best just doing that, i.e. at login record the location and timezone for that user.
Secondly if you want to convert for display, you need to have a table of all local time offset transitions for that timezone, simply knowing the current offset is not enough, because if you are showing a date/time from six months ago the offset will be different.
A DATETIMEOFFSET gives you the ability to store local time and UTC time in one field.
This allows for very simple and efficient reporting in local or UTC time without the need to process the data for display in any way.
These are the two most common requirements - local time for local reports and UTC time for group reports.
The local time is stored in the DATETIME portion of the DATETIMEOFFSET and the OFFSET from UTC is stored in the OFFSET portion, thus conversion is simple and, since it requires no knowledge of the timezone the data came from, can all be done at database level.
If you don't require times down to milliseconds, e.g. just to minutes or seconds, you can use DATETIMEOFFSET(0). The DATETIMEOFFSET field will then only require 8 bytes of storage - the same as a DATETIME.
Using a DATETIMEOFFSET rather than a UTC DATETIME therefore gives more flexibility, efficiency and simplicity for reporting.
My location in Sydney, Australia. The dates that I explain will be in UK or Australia date format.
Observe the following:
2010-04-15 04:30:00.000 => 15/04/2010 14:30:00 EST (UK date format - Add 10 hours)
2010-11-05 01:00:00.000 => 05/11/2010 12:00:00 EST (UK date format - Add 11 hours)
Both these times are retrieved from the database in UTC format and then calculated on the Web level whether +10 or +11 hour is applicable.
In Australia, Daylight Savings Time (DST) transition dates vary year by year. The transition dates are usually Early April and Late October.
So how accurate would the Web calculation be? If this year the transition date is a few days later (say 03/04/2010), but the Web calculation bases on a fixed date (say 01/04/2010), wouldn't that mean that the days in between will be off by 1 hour when displayed (due to the fixed calculation nature to a specific day of the month)?
I believe the transition dates is not pre-determined and is actually announced to the public. Is that assumption true?
If not (that means the DST dates are pre-determined), would I be able to do the calculation outside the Web level (on the SQL/Database level)?
The database is SQL Server 2005 and I'm using Report Definition Language (RDL) to display the fields in UTC time. If SQL/database level is not the best way, how do I work out +10 or +11 and format the time accordingly to show the right time?
Thank you.
The database is a bad choice for it: it has less information than c# or .net to work it out. .net uses the registry which is kept upto date periodically by patches. SQL Server would have to have a table with date ranges and offsets.
The transitions are fixed because of scheduling (flights, trains ,whatever). IIRC it only changed once at short notice recently in Australia for some Olympic games and it caused chaos around the world. In 2007 the US changed but this was known in advance.
By fixed, it's the "last Sunday" type fixed even if the date varies.
I would leave it in the web code: the DB does not know where your caller is for example, the web site can work it out.
The problem is whoever wrote this app does not quite understand UTC, its value, and how to use it. The database is the correct location for the dat, but the system is not using UTC as intended.
If you use UTC, then all your date arithmetic should use UTC. In the database. It is currently using a saved UTC and then converting at some (doesn't matter if seciond tier or third) other layer; some other library. Half UTC, and Half something else. Have you considered historic dates, as in what is the DATEDIFF() between 15 Feb 2010 and today ?
This eliminates the concern re DST in Australia or Greenland.; and concern re what date/time the changeover actually happens. Everyone is using Greenwich Mean Time for that particular day.
Do all you date arithmetic in the db, in UTC. And display the result (only) in the local time zone, which as you have it, is the web layer, based on the user.
Many systems have dropped that last step altogether, and display in UTC only, regardless of the user's time zone.
The database can handle DST for you. Use its time zone conversion functions to go from whatever zone you stored the dates in to whatever zone you want to get for the user.
MySQL has CONVERT_TZ(), I don't know what other RDBMS's have.