I have a table like this.
create table public.test123
(id int not null primary key, dt timestamp null default clock_timestamp());
I then insert into it from 2 different client sessions,
the first session is from PG Admin, the second one is from DBeaver.
1) In the first session when I run show timezone; it returns UTC.
So this session's timezone is UTC.
So from this session I do an insert as follows.
insert into public.test123(id) values (1);
2) In the second session I have timezone America/New_York and I do this.
insert into public.test123(id) values (2);
I run the two inserts just a few seconds apart from each other.
But I get two very different values in the DB e.g.
id|dt |
--|-------------------|
1|2020-06-05 14:38:18|
2|2020-06-05 10:38:26|
I always thought that in such scenario the clock_timestamp() call is executed on the server, and the timezone of the client session should not matter. And I expected to get two values which are a few seconds apart, and not 4 hours apart.
What am I missing? Could anyone explain in some details please.
And also... is there any way to get a timestamp independent of the client session's timezone?
How can I create a column with default timestamp values
which are really independent on the client session's timezone?
tl;dr: Switch to timestamptz.
I'll refer to the timestamp types by their short names: timestamp without time zone is timestamp and timestamp with time zone is timestamptz.
Both timestamp and timestamptz store an absolute point in time. timestamp with time zone also records the time zone it was entered in.
For example, 2020-06-05 20:22:48Z and 2020-06-05 13:22:48-0700 both represent the same moment in time, but the first is in Universal Coordinated Time and the other is 7 hours offset (Pacific Daylight Time).
timestamptz will store 2020-06-05 13:22:48-0700 as 2020-06-05 20:22:48, the absolute point in time, but also remember that it was entered with 7 hours offset. When you retrieve it you'll get 2020-06-05 13:22:48-0700 which is the same time as 2020-06-05 20:22:48Z.
timestamp will ignore the time zone completely. It stores 2020-06-05 13:22:48-0700 as 2020-06-05 13:22:48, no time zone, no conversion, and that's it.
On to the specific problem: dt timestamp null default clock_timestamp()
clock_timestamp() returns timestamptz, but you're storing it in a timestamp. Postgres will cast one to the other by simply dropping the time zone. It will not convert it. If the time zone is in UTC and it's 2020-06-05 20:22:48Z it will store 2020-06-05 20:22:48. If the time zone is in America/New York and it's 2020-06-05 16:22:48-0400 it will store 2020-06-05 16:22:48.
And also... is there any way to get a timestamp independent of the client session's timezone?
They are all independent of the client's time zone, it's the type conversion that's a problem. Switch to timestamptz.
Time zones are complicated, but here's two strategies to deal with them when moving timestamps in and out of the database.
Use timestamp and always work in UTC.
Use timestamptz and always include a time zone.
The former is simpler, and many frameworks take this approach. But I prefer the latter because, as complex and infuriating as they are, people like the time to be vaguely in sync with the Sun.
Yes, timestamp with time zone, which is an absolute timestamp, would be the correct data type in your case.
If you use timestamp without time zone, the result of clock_timestamp(), which is a timestamp with time zone, is converted (cast implicitly) to a timestamp without time zone according to the current setting of timezone.
Related
I ran the same statements in two different databases: my Local DB and Oracle Live SQL.
CREATE TABLE test(
timestamp TIMESTAMP DEFAULT SYSDATE,
timestamp_tmz TIMESTAMP WITH TIME ZONE DEFAULT SYSDATE,
timestamp_local_tmz TIMESTAMP WITH LOCAL TIME ZONE DEFAULT SYSDATE
);
INSERT INTO test VALUES (DEFAULT, DEFAULT, DEFAULT);
SELECT * FROM test;
(all statements were executed at approximately the same time - 09:35 AM CET)
Results from my Local DB:
TIMESTAMP: 10-JAN-23 09.35.32.000000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 09.35.32.000000000 AM EUROPE/BERLIN
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 09.35.32.000000000 AM
Results from Oracle Live:
TIMESTAMP: 10-JAN-23 08.35.44.000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 08.35.44.000000 AM US/PACIFIC
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 08.35.44.000000 AM
After seeing the results, my questions are:
Why is Oracle Live's TIMESTAMP showing date in a different time zone (8.35 AM instead of 9.35 AM)?
Why does Oracle Live's TIMESTAMP WITH TIME ZONE return US/PACIFIC as time zone?
Is there any difference between TIMESTAMP and TIME STAMP WITH LOCAL TIME ZONE?
The different data types are described in the documentation.
The TIMESTAMP data type is an extension of the DATE data type. It stores year, month, day, hour, minute, and second values. It also stores fractional seconds, which are not stored by the DATE data type.
TIMESTAMP WITH TIME ZONE is a variant of TIMESTAMP that includes a time zone region name or time zone offset in its value.
TIMESTAMP WITH LOCAL TIME ZONE is another variant of TIMESTAMP. It differs from TIMESTAMP WITH TIME ZONE as follows: data stored in the database is normalized to the database time zone, and the time zone offset is not stored as part of the column data. When users retrieve the data, Oracle Database returns it in the users' local session time zone.
You are seeing a difference because you have different timezones, and you are defaulting the values to SYSDATE, which is the system DATE.
In your local database the system time zone (select dbtimezone from dual) seems to be based on CET, while the Live SQL database seems to be based on UTC, as Oracle recommends. As CET is an hour ahead of UTC/GMT, that explains the one-hour difference.
The TIMESTAMP value is just a simple cast, i.e. cast(SYSDATE as TIMESTAMP), so you get the same value you would if you queried SYSDATE directly, with zero fractional seconds added.
For the TIMESTAMP WITH TIME ZONE it has to store a time zone, and it has to get that from somewhere, and by default it uses your session time zone, not the database time zone. In your local DB that also seems to be CET, but Live SQL is defaulting the session time zone to US Pacific time - not unreasonable, given where Oracle is based. So now it's effectively doing from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) for that value, where for you SESSIONTIMEZONE is CET in one database and US/Pacific in the other.
For the TIMESTAMP WITH LOCAL TIME ZONE it is doing the same, but then normalising that back to the database time zone for storage (effectively cast(from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) at time zone DBTIMEZONE as TIMESTAMP) - not actually that internally, but gives you the idea), and converting back from the database time zone to your session time zone again when it is queried.
In both databases, if you alter session set time_zone = ... before inserting, and again to a different value before querying, then you'll see different results - the displayed time portion will stay the same for the first two columns, but the time zone will change for the WITH TIME ZONE, and the time will change for the WITH LOCAL TIME ZONE.
fiddle with different session time zones.
You can read more about all of this behaviour in the documentation I already linked to above.
If you use SYSTIMESTAMP instead of SYSDATE as the default for all of your columns then you will avoid the implicit conversion to your session time zone for the WITH TIME ZONE value, and that will always show the database time zone. The LOCAL column will still display in your session time zone, but they will all represent the same time. You will also still see the one-hour difference between the two databases, because they have different database time zones. You could consider defaulting the plain timestamp to sys_extract_utc(SYSTIMESTAMP), or defaulting them all (or at least the first two) to SYSTIMESTAMP at time zone 'UTC'.
fiddle with UTC-normalised values.
Is the WITH TIME ZONE a postgres-only thing? I'm used to seeing the following three items usually in databases:
Date (used as something like '4th of July')
Time (the local news is on at 5:00pm)
Datetime (A movie comes out at midnight, Feb 17)
Timestamp (a specific moment in time -- such as when I asked this question)
Datetime is never with a time zone, Timestamp is with a time zone, and Time is (usually?) not with a time zone. What then is the use of the WITH[OUT] TIME ZONE in Postgres, as I've never really seen that in any other databases -- is that historical baggage or something else?
I suppose postgres just equates the data types of:
Timestamp = timestamp WITH TIME ZONE
Datetime = timestamp WITHOUT TIME ZONE
Is that correct?
I am having some trouble understanding how to deal in Postgres with time zone semantic:
Consider this table
TABLE MyTable (
MyDate TIMESTAMP NOT NULL,
// other columns
);
And this query
SELECT *
FROM MyTable
WHERE // conditions
AND tstzrange(#{start} ::timestamp with time zone, #{end} ::timestamp with time zone] #> MyDate::timestamp without time zone at time zone 'CET'
I understand that Postgres only stores the epoch value i.e. no time zone info is ever stored, so I understand the need to specify timezone for start and end as they are formatted strings that Postgres needs to calculate the epoch for.
What I don't really understand is:
MyDate::timestamp without time zone at time zone 'CET'
Postgres knows the epoch for MyDate since it's their values are stored, why the need to "convert" to a time zone ?
What are we actually saying here and can this be simplified ?
Postgres never stores timezone information in timestamp or timestamptz. In the timestamptz case the timestamp being stored is rotated to a UTC value using either the timezone information in the presented timestamp or the value of the setting TimeZone and then stored. In the timestamp case that is not done. On output a timestamptz is rotated back from UTC to whatever the TimeZone setting is or what via at time zone <some_tz>. In the timestamp case the retrieved value is assumed to be the setting of TimeZone unless you override with at time zone <some_tz>. Best practices is that you use timestamptz.
I would like to insert time data type in postgresql that includes the timezone and is aware of daylight savings time. This is what I have done:
CREATE TABLE mytable(
...
start_time time(0) with time zone,
end_time time(0) with time zone
)
INSERT INTO mytable(start_time, end_time)
VALUES(TIME '08:00:00 MST7MDT', TIME '18:00:00 MST7MDT')
I get the following error:
invalid input syntax for type time: "08:00:00 MST7MDT"
It works if I use 'MST' instead of 'MST7MDT', but I need it to be aware of DST. I also tried using 'America/Edmonton' as the timezone, but I got the same error.
What is the proper way to insert a time value (not timestamp) with timezone and DST?
EDIT:
I would actually like to use the 'America/Edmonton' syntax
The proper way is not to use time with time zone (note the space between time and zone) at all, since it is broken by design. It is in the SQL standard, so Postgres supports the type - but advises not to use it. More in this related answer:
Accounting for DST in Postgres, when selecting scheduled items
Since you are having problems with DST, timetz (short name) is a particularly bad choice. It is ill-equipped to deal with DST. It's impossible to tell whether 8:00:00 is in winter or summer time.
Use timestamp with time zone (timstamptz) instead. You can always discard the date part. Simply use start_time::time to get the local time from a timestamptz. Or use AT TIME ZONE to transpose to your time zone.
Generally, to take DST into account automatically, use a time zone name instead of a time zone abbreviation. More explanation in this related question & answer:
Time zone names with identical properties yield different result when applied to timestamp
In your particular case, you could probably use America/Los_Angeles (example with timestamptz):
INSERT INTO mytable(start_time, end_time)
VALUES
('1970-01-01 08:00:00 America/Los_Angeles'
, '1970-01-01 18:00:00 America/Los_Angeles')
I found this by checking:
SELECT * FROM pg_timezone_names
WHERE utc_offset = '-07:00'
AND is_dst;
Basics about time zone handling:
Ignoring time zones altogether in Rails and PostgreSQL
How about this?
INSERT INTO mytable(start_time, end_time)
VALUES('08:00:00'::time at time zone 'MST7MDT', '18:00:00'::time at time zone 'MST7MDT')
I have a Postgres table of clock alarms (not really, but this is analogous, and easier to explain). Alarms are set by users with a 1 hour resolution, and users can be from many different timezones. The alarms are repeating daily. I want to reliably fetch the alarms that are supposed to go off at a particular hour of the day, and I am having problems with daylight saving time. How do I do this in the best way?
Example
Alfred and Lotta both live in Stockholm (+1 hour from UTC, but +2h
when it's DST). Sharon lives in Singapore (+8 hours from UTC, no
DST)
During winter, Alfred sets an alarm for 4 AM. The alarm should go off
at 4 AM local time, all year. During summer, Lotta sets an alarm
for 5 AM. Again, it should go off at 5 AM all year round.
Meanwhile, Sharon has set an alarm for 11 AM.
All of these can be stored in the database as 03:00 UTC.
If I query the database in the winter for alarms that should go off at
03:00 UTC, I want Alfred's and Sharon's alarms. Singapore is now +7h
from Sweden, so 11 AM in Singapore is 4 AM in Sweden. Lotta's alarm
should not go off for another hour.
Conversely, if I query the database in the summer for alarms that
should go off at 03:00 UTC, I want Lotta's and Sharon's alarms.
Singapore is +6h from Sweden now, so 11 AM in Singapore is 5 AM in
Sweden now. Sven's alarm went off an hour ago.
How do I store this, and query the database?
I can change the db schema if necessary. At the moment, we don't adjust for DST at all, and in fact just have an "hour" integer field (which seems dumb, a time field would be better).
It seems I need to store both a UTC time and timezone information, but I don't know how to best achieve this in Postgres. I've found that Postgres has some sort of concept of timezones, but no timezone field type as far as I can tell. Also, I guess I need to do some calculations in SQL to determine how to offset the UTC time in the select, based on the timezone data and the creation date. I'm not great with SQL…
I do want to solve this in Postgres, as there can be a lot of "alarms", and I want to avoid the performance issues that come with fetching all of them into Ruby and filter there. (Yes, this is a Rails app.)
Use timestamp with time zone (timestamptz) for calculations.
Times for alarms can be time [without time zone].
But you have to save the time zone explicitly for every row.
Never use time with time zone (timetz) It's a logically broken type, its use is discouraged by PostgreSQL. The manual:
The type time with time zone is defined by the SQL standard, but the
definition exhibits properties which lead to questionable usefulness.
In most cases, a combination of date, time, timestamp without timezone, and timestamp with time zone should provide a complete
range of date/time functionality required by any application.
Demo setup:
CREATE TABLE alarm(name text, t time, tz text);
INSERT INTO alarm VALUES
('Alfred', '04:00', 'Europe/Stockholm') -- Alfred sets an alarm for 4 AM.
, ('Lotta', '05:00', 'Europe/Stockholm') -- Lotta sets an alarm for 5 AM.
, ('Sharon', '11:00', 'Asia/Singapore'); -- Sharon has set an alarm for 11 AM.
It has to be time zone names (not abbreviations) to account for DST. Related:
Time zone names with identical properties yield different result when applied to timestamp
Get matching alarms for "today":
SELECT *
FROM alarm
WHERE (('2012-07-01'::date + t) AT TIME ZONE tz AT TIME ZONE 'UTC')::time
= '03:00'::time
('2012-7-1'::date + t) ... assemble timestamp [without time zone]
Could also just be now()::date + t for "today".
AT WITH TIME ZONE tz ... place timestamp at the saved time zone, resulting in timestamptz.
AT WITH TIME ZONE 'UTC' ... get according UTC timestamp
::time ... simplest way to extract the time component.
Here you can look up time zone names:
SELECT *
FROM pg_timezone_names
WHERE name ~~* '%sing%'
LIMIT 10;
db<>fiddle here - demonstrating summer / winter
Old sqlfiddle
You would do it by using a full time zone name, e.g. America/New_York rather than EDT/EST, and storing the hour in that time zone not UTC. You can then remain blissfully ignorant of the offset changes for daylight savings.
Something like the following should work:
-- CREATE TABLE time_test (
-- user_to_alert CHARACTER VARYING (30),
-- alarm_hour TIME,
-- user_timezone CHARACTER VARYING (30)
-- );
SELECT user_to_alert,
CASE
WHEN EXTRACT(HOUR FROM CURRENT_TIME AT TIME ZONE user_timezone) = EXTRACT(HOUR FROM alarm_hour) THEN TRUE
ELSE FALSE
END AS raise_alarm
FROM time_test;
Or:
SELECT user_to_alert
FROM time_test
WHERE EXTRACT(HOUR FROM CURRENT_TIME AT TIME ZONE user_timezone) = EXTRACT(HOUR FROM alarm_hour);
Given:
SET timezone = 'UTC';
CREATE TABLE tzdemo (
username text not null,
alarm_time_utc time not null,
alarm_tz_abbrev text not null,
alarm_tz text not null
);
INSERT INTO tzdemo (username, alarm_time_utc, alarm_tz_abbrev, alarm_tz) VALUES
('Alfred', TIME '04:00' AT TIME ZONE '+01:00', 'CET', 'Europe/Stockholm'),
('Lotta', TIME '05:00' AT TIME ZONE '+02:00', 'CEST', 'Europe/Stockholm'),
('Sharon', TIME '11:00' AT TIME ZONE '+08:00', 'SGT', 'Singapore');
Try:
SELECT username
FROM tzdemo
WHERE alarm_time_utc AT TIME ZONE alarm_tz_abbrev = TIME '03:00' AT TIME ZONE alarm_tz;
Result:
username
----------
Alfred
Sharon
(2 rows)
Principle:
Store the timezone offset the alarm was created in including whether it was or was not DST at the time
Also store the clock time converted to UTC
When querying, use the fact that full timezone names follow the current UTC rules for times to produce a time that's in the current time zone for the region. Compare to the stored timestamp in what the time zone was when that alarm was created.
This also allows you to cope with cases where the user changes location, and therefore changes timezone.
This approach can be extended by date-qualifying the timestamps when you want to do predictive querying, like "at what local time will alarm sound in location".
I'm not completely confident in this solution and would recommend careful testing.