I want to add to my query TIMESTAMP column in UTC.
I know BigQuery has CURRENT_TIMESTAMP() function but it doesn't say how to convert it to UTC.
In MYSQL I would have do:
SELECT UTC_TIMESTAMP()
What is the equivalent in BigQuery (Standard SQL only) ?
A TIMESTAMP does not store or use a time zone, as described in the documentation. When parsing it it may have a time zone which is converted from when saving.
A timestamp represents an absolute point in time, independent of any time zone or convention such as Daylight Savings Time.
And more specifically
A timestamp is simply an offset from 1970-01-01 00:00:00 UTC, assuming there are exactly 60 seconds per minute. Leap seconds are not represented as part of a stored timestamp.
When querying for CURRENT_TIMESTAMP() it also shows explicitly it’s in UTC by having zero time zone offset. So it is a timestamp in UTC based on this.
There may be a problem using it since it does not store anything in UTC and it does not care about leap seconds. The specification says this:
Leap seconds are only observable through functions that measure real-world time. In these functions, it is possible for a timestamp second to be skipped or repeated when there is a leap second.
So if these leap seconds are needed it may be required to use another data type in specific time zone to handle leap seconds.
The conversion is also explained:
If your input contains values that use ":60" in the seconds field to represent a leap second, that leap second is not preserved when converting to a timestamp value. Instead that value is interpreted as a timestamp with ":00" in the seconds field of the following minute.
Related
I want to update a date with a timezone (+2 hours) but it ends up as UTC (0 hours)
Date type is 'timestamp-with-timezone'
Query...
update table set date = '2022-05-25 13:28+02:00'
will end up as this in the database.
2022-05-25 11:28:00+00
What's wrong here?
tl;dr
Nothing wrong. Postgres stores values of TIMESTAMP WITH TIME ZONE in UTC, always an offset from UTC of zero. Any submitted offset or zone is used to adjust to UTC.
Details
Date type is 'timestamp-with-timezone'
No such type in standard SQL, nor in Postgres.
I’ll assume you meant TIMESTAMP WITH TIME ZONE.
it ends up as UTC (0 hours)
Read the fine manual. You are seeing documented behavior.
Postgres always stores values in a column of type TIMESTAMP WITH TIME ZONE in UTC, that is, with an offset of zero hours-minutes-seconds.
Any time zone or offset provided with an input is used to adjust into UTC. That provided zone or offset is then discarded.
So the name of the type TIMESTAMP WITH TIME ZONE is a misnomer. First, the authors of the SQL were thinking in terms of offset, not real time zones. Second, any submitted time zone is not stored. A submitted zone is used to adjust and then discarded.
If you need to track the original offset or zone, add an extra column. You’ll have to add code to store the offset amount or the time zone name.
update table set date = '2022-05-25 13:28+02:00' will end up as this in the database. 2022-05-25 11:28:00+00 What's wrong here?
Nothing is wrong. That is a feature, not a bug. Both of those strings represent the very same simultaneous moment.
FYI, database engines vary widely in their behavior handling date-time types and behaviors.
Some do as Postgres does regarding TIMESTAMP WITH TIME ZONE, adjusting to UTC and then discarding any provided time zone or offset. Some others may not.
The SQL standard barely touches on the topic of date-time handling. It declares a few types, and does that poorly with incomplete coverage of all cases. And the standard neglects to define behavior.
So, be very careful when it comes to date-time handling in your database work. Read very carefully the documentation for your particular database engine. Do not make assumptions. Run experiments to validate your understanding. And know that writing portable SQL code for date-time may not be feasible.
I’m working with a table that uses date time offset. I have a value that looks like 2020-01-02 13:30:00 -07:00.
Is the time in my time zone 13:30 or do I need to subtract 7 hours from it. I saw people do it differently on YouTube.
In MS SQLServer, the last section of the string representation that you posted of the DateTimeOffset describes the time zone. So, if you are currently located in time zone -7:00 (e.g., Arizona, USA), then the time portion of the string refers to your local time, not UTC. See the Microsoft documentation:
For example, 1999-12-12 12:30:30.12345 -07:00 should be represented [in UTC] as
1999-12-12 19:30:30.12345Z
Someone would subtract the offset from the number only if they want to manually get the UTC value, but that would might produce errors if the data come from a daylight saving time (DST) region, so you would need to enforce DST handling at the time of entry. SQLServer already stores the data in UTC behind the scenes:
The data is stored in the database and processed, compared, sorted,
and indexed in the server as in UTC.
Date and Timestamps both have time added and interval is used in case of manipulation of dates via addition yearwise, datewise etc.
Still unsure about the exact actual difference though when it comes to dates in oracle especially.
Is there any major difference in terms of efficiency or some other difference on the usage of date, timestamp and interval?
Your question is not clear but this information may help you.
TIMESTAMP supports fractional seconds, unlike DATE which supports only seconds
TIMESTAMP exist in three flavors:
TIMESTAMP does not contain any time zone information.
TIMESTAMP WITH TIME ZONE and TIMESTAMP WITH LOCAL TIME ZONE contain time zone information
Regarding calculation and manipulation there is actually no difference between TIMESTAMP and DATE. There are only a very few functions which support only either of these two types.
DATE is an old data type. TIMESTAMP was introduced later (well "later" means in 9i, i.e. 20 years ago)
INTERVAL YEAR TO MONTH and INTERVAL DAY TO SECOND are interval data types, they do not contain any absolute date information.
Hope this gave some hints. Otherwise please elaborate your question.
date does not store fractional seconds so comparing time with date less than 1 sec wont work!!!!!
SELECT * FROM table_a
WHERE time_1 >= to_timestamp('11/01/2014 10:00 PDT', 'MM/DD/YYYY HH24:MI TZ')
time_1 is in UTC time zone and it is a timestamp with timezone. So, will this give me what I want or I need to do exact UTC time in the function to_timestamp()?
Postgresql has two different timestamp data types and it is confusing which one should be used when. The two types are:
timestamp (also known as timestamp without time zone) It is most likely that this is the type in table_a
timestamp with time zone This is the data type returned by to_timestamp()
You must be sure that you are comparing apples with apples or pairs with pairs and not mix them or you may get undesirable results.
If your table_a.time_1 is a timestamp with time zone then the code you give in your question will work fine.
If your table_a.time_1 is a timestamp then you will need to change your code:
SELECT *
FROM table_a
WHERE time_1 >= to_timestamp('11/01/2014 10:00 PDT', 'MM/DD/YYYY HH24:MI TZ') at time zone 'utc';
The last part of this (at time zone 'utc') will strip the timezone (PDT) off the specified timestamp and translate the timestamp to UTC.
Edit: to help with your comments in this answer...
In order to understand how to translate time zones you need to understand the difference between the two forms of time stamp. It will become clear why you need to understand this below. As I indicate above the difference between the two forms of time stamp is confusing. There is a good manual page but for now just read on.
The main thing to understand is that neither version actually stores a time zone (despite the name). The naming would make much more sense if you added an extra word "translation". Think "timestamp without time zone translation" and "timestamp with time zone translation".
A timestamp with time zone translation doesn't store a time zone at all. It is designed to store time stamps which could come from anywhere in the world and not loose track of their meaning. So when entering one you must provide the time zone it came from or postgresql will assume it came from the time zone of your current session. Postgresql automatically translates it out of the given time zone into an internal time zone for the server. You don't need to know what time zone that is because postgresql will always translate it back from this internal time zone before giving you the value. When you retrieve the value (eg: SELECT my_time FROM foo) postgresql translates the time stamp to the time zone of your current session. Alternatively you can specify the time zone to translate into (eg: SELECT my_time AT TIME ZONE 'PDT' FROM foo).
With that in mind it's easier to understand that a timestamp without time zone translation will never be changed from the time you specify. Postgresql will regard 11:00:00 as happening before 12:00:00 even if you meant 11 in America and 12 in England. It's easy to see why that may not be what you want.
A very common programming error is to think that a timestamp with time zone is at a particular time zone. It isn't. It is at whatever time zone you ask for it to be. And if you don't specify what time zone you want it at then postgresql will assume you want it at your current session time zone.
You've stated that your field is a timestamp with time zone which are all at UTC. This isn't technically correct. Most likely your session time zone is UTC and postgresql is giving you everything in UTC as a result.
So you have a timestamp with time zone and you want to know what these times are in PDT? Easy: SELECT my_time AT TIME ZONE 'PDT' FROM foo.
It's important to understand that the AT TIME ZONE '...' syntax toggles between timestamp and timestamp with time zone.
timestamp AT TIME ZONE 'PDT' converts into a timestamp with time zone and tells postgresql to convert to the PDT time zone.
timestamp with time zone AT TIME ZONE 'PDT' converts into a timestamp telling postgresql to interpret it as coming from 'PDT'.
This symetry means that to reverse AT TIME ZONE 'foo' you just use AT TIME ZONE 'foo'. Put another way SELECT anything AT TIME ZONE 'PDT' AT TIME ZONE 'PDT' will always leave anything unchanged.
In PostgreSQL, the data types timestamp and timestamp with timezone both use 8 bytes.
My questions are:
What format is used to store date & time in a timestamp?
How is the time zone information stored in the timestamp with timezone
type, and how is it parsed later when reading the type?
This is just a misunderstanding stemming from the somewhat misleading type name. The time zone itself is not stored at all. It just acts as offset to compute a UTC timestamp (input), which is actually stored. Or as decorator in the display of a timestamp according to the current or given time zone (output). That's all according to the SQL standard.
Just the point in time is stored, no zone information. That's why 64 bit of information is enough. The timestamp is displayed to the client according to the current time zone setting of the session.
Details:
Ignoring time zones altogether in Rails and PostgreSQL
Also, since Jon mentioned it, time with time zone is defined in the SQL standard and thus implemented in Postgres, but its use is discouraged:
time with time zone is defined by the SQL standard, but the definition
exhibits properties which lead to questionable usefulness.
It's an inherently ambiguous type that cannot deal with DST properly.
Looking at the documentation:
Timestamps are stored either as integers, or (deprecated) floating point numbers
I don't believe timestamp with timezone could be correctly encoded within 8 bytes if it actually stored a time zone. Just the timestamp requires 64 bits, as log2(298989 * 365 * 24 * 60 * 60 * 1000000) is greater than 63. Note that time with time zone requires 12 bytes, with the same precision but a range of a single day.
See Erwin's answer to explain how it actually manages to be stored in 8 bytes - it should be called "timestamp without a time zone, but stored in UTC and converted into the local time zone for display". Ick.