Is the WITH TIME ZONE a postgres-only thing? I'm used to seeing the following three items usually in databases:
Date (used as something like '4th of July')
Time (the local news is on at 5:00pm)
Datetime (A movie comes out at midnight, Feb 17)
Timestamp (a specific moment in time -- such as when I asked this question)
Datetime is never with a time zone, Timestamp is with a time zone, and Time is (usually?) not with a time zone. What then is the use of the WITH[OUT] TIME ZONE in Postgres, as I've never really seen that in any other databases -- is that historical baggage or something else?
I suppose postgres just equates the data types of:
Timestamp = timestamp WITH TIME ZONE
Datetime = timestamp WITHOUT TIME ZONE
Is that correct?
Related
I ran the same statements in two different databases: my Local DB and Oracle Live SQL.
CREATE TABLE test(
timestamp TIMESTAMP DEFAULT SYSDATE,
timestamp_tmz TIMESTAMP WITH TIME ZONE DEFAULT SYSDATE,
timestamp_local_tmz TIMESTAMP WITH LOCAL TIME ZONE DEFAULT SYSDATE
);
INSERT INTO test VALUES (DEFAULT, DEFAULT, DEFAULT);
SELECT * FROM test;
(all statements were executed at approximately the same time - 09:35 AM CET)
Results from my Local DB:
TIMESTAMP: 10-JAN-23 09.35.32.000000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 09.35.32.000000000 AM EUROPE/BERLIN
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 09.35.32.000000000 AM
Results from Oracle Live:
TIMESTAMP: 10-JAN-23 08.35.44.000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 08.35.44.000000 AM US/PACIFIC
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 08.35.44.000000 AM
After seeing the results, my questions are:
Why is Oracle Live's TIMESTAMP showing date in a different time zone (8.35 AM instead of 9.35 AM)?
Why does Oracle Live's TIMESTAMP WITH TIME ZONE return US/PACIFIC as time zone?
Is there any difference between TIMESTAMP and TIME STAMP WITH LOCAL TIME ZONE?
The different data types are described in the documentation.
The TIMESTAMP data type is an extension of the DATE data type. It stores year, month, day, hour, minute, and second values. It also stores fractional seconds, which are not stored by the DATE data type.
TIMESTAMP WITH TIME ZONE is a variant of TIMESTAMP that includes a time zone region name or time zone offset in its value.
TIMESTAMP WITH LOCAL TIME ZONE is another variant of TIMESTAMP. It differs from TIMESTAMP WITH TIME ZONE as follows: data stored in the database is normalized to the database time zone, and the time zone offset is not stored as part of the column data. When users retrieve the data, Oracle Database returns it in the users' local session time zone.
You are seeing a difference because you have different timezones, and you are defaulting the values to SYSDATE, which is the system DATE.
In your local database the system time zone (select dbtimezone from dual) seems to be based on CET, while the Live SQL database seems to be based on UTC, as Oracle recommends. As CET is an hour ahead of UTC/GMT, that explains the one-hour difference.
The TIMESTAMP value is just a simple cast, i.e. cast(SYSDATE as TIMESTAMP), so you get the same value you would if you queried SYSDATE directly, with zero fractional seconds added.
For the TIMESTAMP WITH TIME ZONE it has to store a time zone, and it has to get that from somewhere, and by default it uses your session time zone, not the database time zone. In your local DB that also seems to be CET, but Live SQL is defaulting the session time zone to US Pacific time - not unreasonable, given where Oracle is based. So now it's effectively doing from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) for that value, where for you SESSIONTIMEZONE is CET in one database and US/Pacific in the other.
For the TIMESTAMP WITH LOCAL TIME ZONE it is doing the same, but then normalising that back to the database time zone for storage (effectively cast(from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) at time zone DBTIMEZONE as TIMESTAMP) - not actually that internally, but gives you the idea), and converting back from the database time zone to your session time zone again when it is queried.
In both databases, if you alter session set time_zone = ... before inserting, and again to a different value before querying, then you'll see different results - the displayed time portion will stay the same for the first two columns, but the time zone will change for the WITH TIME ZONE, and the time will change for the WITH LOCAL TIME ZONE.
fiddle with different session time zones.
You can read more about all of this behaviour in the documentation I already linked to above.
If you use SYSTIMESTAMP instead of SYSDATE as the default for all of your columns then you will avoid the implicit conversion to your session time zone for the WITH TIME ZONE value, and that will always show the database time zone. The LOCAL column will still display in your session time zone, but they will all represent the same time. You will also still see the one-hour difference between the two databases, because they have different database time zones. You could consider defaulting the plain timestamp to sys_extract_utc(SYSTIMESTAMP), or defaulting them all (or at least the first two) to SYSTIMESTAMP at time zone 'UTC'.
fiddle with UTC-normalised values.
I am having some trouble understanding how to deal in Postgres with time zone semantic:
Consider this table
TABLE MyTable (
MyDate TIMESTAMP NOT NULL,
// other columns
);
And this query
SELECT *
FROM MyTable
WHERE // conditions
AND tstzrange(#{start} ::timestamp with time zone, #{end} ::timestamp with time zone] #> MyDate::timestamp without time zone at time zone 'CET'
I understand that Postgres only stores the epoch value i.e. no time zone info is ever stored, so I understand the need to specify timezone for start and end as they are formatted strings that Postgres needs to calculate the epoch for.
What I don't really understand is:
MyDate::timestamp without time zone at time zone 'CET'
Postgres knows the epoch for MyDate since it's their values are stored, why the need to "convert" to a time zone ?
What are we actually saying here and can this be simplified ?
Postgres never stores timezone information in timestamp or timestamptz. In the timestamptz case the timestamp being stored is rotated to a UTC value using either the timezone information in the presented timestamp or the value of the setting TimeZone and then stored. In the timestamp case that is not done. On output a timestamptz is rotated back from UTC to whatever the TimeZone setting is or what via at time zone <some_tz>. In the timestamp case the retrieved value is assumed to be the setting of TimeZone unless you override with at time zone <some_tz>. Best practices is that you use timestamptz.
I am quite new to SQL, and trying to pull a data table from the database (flight) using the following command:
select
flight.FLT_NBR,
flight.LEG_NBR,
flight.LEG_TAIL_NBR,
flight.LEG_IATA_ORIG_CD as FLT_SCHD_ORIG_ARPT_CD,
flight.LEG_IATA_DEST_CD as FLT_SCHD_DEST_ARPT_CD,
flight.SCHD_ARR_TMSTP as Scheduled_Arrival,
flight.ACTL_ARR_TMSTP AS Actual_Arrival,
flight.SCHD_DPRT_TMSTP as Scheduled_Departure,
flight.ACTL_DPRT_TMSTP AS Actual_Departure,
from home/tulips/FT_FLIGHT_LEG flight
Now the problem is there are multiple country origin and destination with different times. How do I incorporate same time zone for all the countries? I tried using the command as time zone 'UTC' such as below but it didn't work... May be I am adding it in a wrong place?
select
flight.FLT_NBR,
flight.LEG_NBR,
flight.LEG_TAIL_NBR,
flight.LEG_IATA_ORIG_CD as FLT_SCHD_ORIG_ARPT_CD,
flight.LEG_IATA_DEST_CD as FLT_SCHD_DEST_ARPT_CD,
flight.SCHD_ARR_TMSTP as Scheduled_Arrival as time zone 'UTC',
flight.ACTL_ARR_TMSTP AS Actual_Arrival as time zone 'UTC',
flight.SCHD_DPRT_TMSTP as Scheduled_Departure as time zone 'UTC',
flight.ACTL_DPRT_TMSTP AS Actual_Departure as time zone 'UTC',
from home/tulips/FT_FLIGHT_LEG flight
Please help me a way to have one time zone for all the Scheduled_Arrival,Actual_Arrival,Scheduled_Departure and Actual_Departure
The expression you want is at time zone, not as time zone.
In order to use it, you need to know what time zone the original datetime value represents. For example, I have a SQL server in Sydney Australia, so getdate() will return my local date and time. However, to convert it to UTC I must first inform SQL of the fact that the value starts off in AUS Eastern Standard Time, and then ask it to convert it to UTC, by chaining at time zone expressions together. Like this:
select getdate() at time zone 'AUS Eastern Standard Time' at time zone 'UTC'
If you don't know the time zone of the original datetime value, there is no way for SQL to know how to change it to a different time zone's value.
I would like to insert time data type in postgresql that includes the timezone and is aware of daylight savings time. This is what I have done:
CREATE TABLE mytable(
...
start_time time(0) with time zone,
end_time time(0) with time zone
)
INSERT INTO mytable(start_time, end_time)
VALUES(TIME '08:00:00 MST7MDT', TIME '18:00:00 MST7MDT')
I get the following error:
invalid input syntax for type time: "08:00:00 MST7MDT"
It works if I use 'MST' instead of 'MST7MDT', but I need it to be aware of DST. I also tried using 'America/Edmonton' as the timezone, but I got the same error.
What is the proper way to insert a time value (not timestamp) with timezone and DST?
EDIT:
I would actually like to use the 'America/Edmonton' syntax
The proper way is not to use time with time zone (note the space between time and zone) at all, since it is broken by design. It is in the SQL standard, so Postgres supports the type - but advises not to use it. More in this related answer:
Accounting for DST in Postgres, when selecting scheduled items
Since you are having problems with DST, timetz (short name) is a particularly bad choice. It is ill-equipped to deal with DST. It's impossible to tell whether 8:00:00 is in winter or summer time.
Use timestamp with time zone (timstamptz) instead. You can always discard the date part. Simply use start_time::time to get the local time from a timestamptz. Or use AT TIME ZONE to transpose to your time zone.
Generally, to take DST into account automatically, use a time zone name instead of a time zone abbreviation. More explanation in this related question & answer:
Time zone names with identical properties yield different result when applied to timestamp
In your particular case, you could probably use America/Los_Angeles (example with timestamptz):
INSERT INTO mytable(start_time, end_time)
VALUES
('1970-01-01 08:00:00 America/Los_Angeles'
, '1970-01-01 18:00:00 America/Los_Angeles')
I found this by checking:
SELECT * FROM pg_timezone_names
WHERE utc_offset = '-07:00'
AND is_dst;
Basics about time zone handling:
Ignoring time zones altogether in Rails and PostgreSQL
How about this?
INSERT INTO mytable(start_time, end_time)
VALUES('08:00:00'::time at time zone 'MST7MDT', '18:00:00'::time at time zone 'MST7MDT')
First off, I realize time with time zone is not recommended. I am going to use it because I'm comparing multiple time with time zone values to my current system time regardless of day. I.e. a user says start everyday at 08:00 and finish at 12:00 with THEIR time zone, not the system time zone. So, I have a time without time zone column in one table, let's call it SCHEDULES.time and I have a UNIX time zone name column in another table, let's call it USERS.tz.
My system time zone is 'America/Regina', which does not use DST and so the offset is always -06.
Given a time of '12:00:00' and a tz of 'America/Vancouver' I would like to select the data into a column of type time with time zone but I DO NOT want to convert the time to my time zone because the user has effectively said begin at when it is 12:00 in Vancouver, not in Regina.
Thus, doing:
SELECT SCHEDULES.time AT TIME ZONE USERS.tz
FROM SCHEDULES JOIN USERS on USERS.ID=SCHEDULES.USERID;
results (at the moment) in:
'10:00:00-08'
but I really want:
'12:00:00-08'
I can't find any documentation relating to applying a time zone to a time, other then AT TIME ZONE. Is there a way to accomplish this without character manipulation or other hacks?
UPDATE:
This can be accomplished by using string concatenation, casting, and the Postgres time zone view as such:
select ('12:00:00'::text || utc_offset::text)::timetz
from pg_timezone_names
where name = 'America/Vancouver';
However, this is fairly slow. There must be a better way, no?
UPDATE 2:
I apologize for the confusion. The SCHEDULES table DOES NOT use time with time zone, I am trying to SELECT a time with time zone by combining values from a time without time zone and a text time zone name.
UPDATE 3:
Thanks to all those involved for their (heated) discussion. :) I have been convinced to abandon my plan to use a time with time zone for my output and instead use a timestamp with time zone as it performs well, is more readable, and solves another problem that I was going to run into, time zones that roll into new dates. IE. '2011-11-21 23:59' in 'America/Vancouver' is '2011-11-22' in 'America/Regina'.
UPDATE 4:
As I said in my last update, I have chosen the answer that #MichaelKrelin-hacker first proposed and #JonSkeet finalized. That is, a timestamp with time zone as my final output is a better solution. I ended up using a query like:
SELECT timezone(USERS.tz, now()::date + SCHEDULES.time)
FROM SCHEDULES
JOIN USERS ON USERS.ID = SCHEDULES.USERID;
The timezone() format was rewritten by Postgres after I entered (current_date + SCHEDULES.time) AT TIME ZONE USERS.tz into my view.
WARNING: PostgreSQL newbie (see comments on the question!). I know a bit about time zones though, so I know what makes sense to ask.
It looks to me like this is basically an unsupported situation (unfortunately) when it comes to AT TIME ZONE. Looking at the AT TIME ZONE documentation it gives a table where the "input" value types are only:
timestamp without time zone
timestamp with time zone
time with time zone
We're missing the one you want: time without time zone. What you're asking is somewhat logical, although it does depend on the date... as different time zones can have different offsets depending on the date. For example, 12:00:00 Europe/London may mean 12:00:00 UTC, or it may mean 11:00:00 UTC, depending on whether it's winter or summer.
On my system, having set the system time zone to America/Regina, the query
SELECT ('2011-11-22T12:00:00'::TIMESTAMP WITHOUT TIME ZONE)
AT TIME ZONE 'America/Vancouver'
gives me 2011-11-22 14:00:00-06 as a result. That's not ideal, but it does at least give the instant point in time (I think). I believe that if you fetched that with a client library - or compared it with another TIMESTAMP WITH TIME ZONE - you'd get the right result. It's just the text conversion that then uses the system time zone for output.
Would that be good enough for you? Can you either change your SCHEDULES.time field to be a TIMESTAMP WITHOUT TIME ZONE field, or (at query time) combine the time from the field with a date to create a timestamp without time zone?
EDIT: If you're happy with the "current date" it looks like you can just change your query to:
SELECT (current_date + SCHEDULES.time) AT TIME ZONE USERS.tz
from SCHEDULES JOIN USERS on USERS.ID=SCHEDULES.USERID
Of course, the current system date may not be the same as the current date in the local time zone. I think this will fix that part...
SELECT ((current_timestamp AT TIME ZONE USERS.tz)::DATE + schedules.time)
AT TIME ZONE USERS.tz
from SCHEDULES JOIN USERS on USERS.ID=SCHEDULES.USERID
In other words:
Take the current instant
Work out the local date/time in the user's time zone
Take the date of that
Add the schedule time to that date to get a TIMESTAMP WITHOUT TIME ZONE
Use AT TIME ZONE to apply the time zone to that local date/time
I'm sure there's a better way, but I think it makes sense.
You should be aware that in some cases this could fail though:
What do you want the result to be for a time of 01:30 on a day when the clock skips from 01:00 to 02:00, so 01:30 doesn't occur at all?
What do you want the result to be for a time of 01:30 on a day when the clock goes back from 02:00 to 01:00, so 01:30 occurs twice?
Here is a demo how to calculate the times without casting to text:
CREATE TEMP TABLE schedule(t time, tz text);
INSERT INTO schedule values
('12:00:00', 'America/Vancouver')
,('12:00:00', 'US/Mountain')
,('12:00:00', 'America/Regina');
SELECT s.t AT TIME ZONE s.tz
- p.utc_offset
+ EXTRACT (timezone from now()) * interval '1s'
FROM schedule s
JOIN pg_timezone_names p ON s.tz = p.name;
Basically you have to subtract the UTC offset and add the offset of your local time zone to arrive at the given time zone.
You can speed up the calculation by hardcoding your local offset. In your case (America/Regina) that should be:
SELECT s.t AT TIME ZONE s.tz
- p.utc_offset
- interval '6h'
FROM schedule s
JOIN pg_timezone_names p ON s.tz = p.name;
As pg_timezone_names is a view and not actually a system table, it is rather slow - just like the demonstrated variant with casting to text representation and back.
I would store the time zone abbreviations and take the double cast via text without joining in pg_timezone_names for optimum performance.
FAST solution
The culprit that's slowing you down is pg_timezone_names. After some testing I found that pg_timezone_abbrevs is far superior. Of course, you have to save correct time zone abbreviations instead of time zone names to achieve this. Time zone names take DST into consideration automatically, time zone abbreviations are basically just codes for a time offset. The documentation:
A time zone abbreviation, for example PST. Such a specification merely
defines a particular offset from UTC, in contrast to full time zone names
which can imply a set of daylight savings transition-date rules as well.
Have a look at these test results or try yourself:
SELECT * FROM pg_timezone_names;
Total runtime: 541.007 ms
SELECT * FROM pg_timezone_abbrevs;
Total runtime: 0.523 ms
Factor 1000. Whether you go with your idea to cast to text and back to timetz or with my method to compute the time is not important. Both methods are very fast. Just don't use pg_timezone_names.
Actually, as soon as you save time zone abbreviations, you can take the casting route without any additional joins. Use the abbreviation instead of the utc_offset. Results are accurate as per your definition.
CREATE TEMP TABLE schedule(t time, abbrev text);
INSERT INTO schedule values
('12:00:00', 'PST') -- 'America/Vancouver'
,('12:00:00', 'MST') -- 'US/Mountain'
,('12:00:00', 'CST'); -- 'America/Regina'
-- calculating
SELECT s.t AT TIME ZONE s.abbrev
- a.utc_offset
+ EXTRACT (timezone from now()) * interval '1s'
FROM schedule s
JOIN pg_timezone_abbrevs a USING (abbrev);
-- casting (even faster!)
SELECT (t::text || abbrev)::timetz
FROM schedule s;