Query datetime as text and include different timezones - sql

Short version:
I have 2 users in DB, each created in a different timezone:
User.find(1).created_at
=> Thu, 04 Aug 2016 11:15:29 IDT +03:00
User.find(33).created_at
=> Sun, 01 Jan 2017 17:50:20 IST +02:00
So my table shows 11:15, and 17:50. so for example I would like to search for 17:50, and later 11:15 as text:
search_param = '17:50'
No problem I'll just convert the date to text, then search it, But user won't be found, since the it's saved as UTC:
User.where("to_char(created_at,'DD-Mon-YYYY HH24:MI:SS') ilike ?", "%#{search_param}%").first
=> nil
To find it I'll just apply the offset to my query (adding time zone UTC+2), and indeed user was found:
User.where("to_char(created_at AT TIME ZONE 'UTC+2','DD-Mon-YYYY HH24:MI:SS') ilike ?", "%#{search_param}%").first
=> User #33 #2017-01-01 17:50:20 +0200
BUT some users are saved as UTC+3 and some as UTC+2.. I can't apply both offsets... So if I change search_param to be 11:15 I won't find user_id_1 because I will also need to change UTC+2 to UTC+3
When I use UTC+2 I will only find users which were created as +2 (like User.find(33)).
When I use UTC+3 I will only find users which were created as +3 (like User.find(1)).
My question: How to do a where query- a text search for both users' created_at hour, as they were both saved in a different timezone offset?
Or in this example a query that for a search_param 17:50 will find user_id_33 and for search_param 11:15 will find user_id_1?
More Details:
I notice in DB they are saved as UTC (I think):
User.select("created_at as created_at_db").find(33).created_at_db
=> "2017-01-01 15:50:20.903289"
User.select("created_at as created_at_db").find(1).created_at_db
=> "2016-08-04 08:15:29.171776"
Time setting:
#application.rb
config.time_zone = 'Jerusalem'
config.active_record.default_timezone = :utc
created_at column info:
User.columns.select{|table_col| table_col.name == 'created_at'}
=> [#<ActiveRecord::ConnectionAdapters::PostgreSQLColumn:0x81f9c48
#coder=nil,
#default=nil,
#limit=nil,
#name="created_at",
#null=false,
#precision=nil,
#primary=false,
#scale=nil,
#sql_type="timestamp without time zone",
#type=:datetime>]

You should be able to query your user entities with:
date_trunc('minute', created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem')::time = '17:50'
Or, with to_char() (but less index-friendly):
to_char(created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem', 'HH24:MI') = '17:50'
The point is that created_at is timestamp (or timestamp without time zone, which is just an alias). First, you can tell PostgreSQL, that interpret its values as they were in UTC, with the <timestamp> AT TIME ZONE <time zone> operator:
created_at AT TIME ZONE 'UTC'
Then, tell PostgreSQL to offset this value as if it were a local time in Asia/Jerusalem with the <timestamp with time zone> AT TIME ZONE <time zone> operator (which is completely different from the operator above with the same name):
created_at AT TIME ZONE 'UTC' AT TIME ZONE 'Asia/Jerusalem'
Now, you just need to truncate this value to extract only the hour & minute part.
Maybe worth mentioning that using these:
created_at AT TIME ZONE 'UTC+2'
created_at AT TIME ZONE 'UTC+3'
worked for you accidentally. These create timestamp with time zone values within the -2 and -3 UTC offsets, respectively. Then, because your config.active_record.default_timezone is set to :utc, it is sent to your client (ruby) within the UTC time zone, which means it added 2 and 3 hours, respectively.
Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.

Related

Difference between TIMESTAMP, TIMESTAMP WITH TIME ZONE and TIMESTAMP WITH LOCAL TIME ZONE

I ran the same statements in two different databases: my Local DB and Oracle Live SQL.
CREATE TABLE test(
timestamp TIMESTAMP DEFAULT SYSDATE,
timestamp_tmz TIMESTAMP WITH TIME ZONE DEFAULT SYSDATE,
timestamp_local_tmz TIMESTAMP WITH LOCAL TIME ZONE DEFAULT SYSDATE
);
INSERT INTO test VALUES (DEFAULT, DEFAULT, DEFAULT);
SELECT * FROM test;
(all statements were executed at approximately the same time - 09:35 AM CET)
Results from my Local DB:
TIMESTAMP: 10-JAN-23 09.35.32.000000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 09.35.32.000000000 AM EUROPE/BERLIN
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 09.35.32.000000000 AM
Results from Oracle Live:
TIMESTAMP: 10-JAN-23 08.35.44.000000 AM
TIMESTAMP WITH TIME ZONE: 10-JAN-23 08.35.44.000000 AM US/PACIFIC
TIMESTAMP WITH LOCAL TIME ZONE: 10-JAN-23 08.35.44.000000 AM
After seeing the results, my questions are:
Why is Oracle Live's TIMESTAMP showing date in a different time zone (8.35 AM instead of 9.35 AM)?
Why does Oracle Live's TIMESTAMP WITH TIME ZONE return US/PACIFIC as time zone?
Is there any difference between TIMESTAMP and TIME STAMP WITH LOCAL TIME ZONE?
The different data types are described in the documentation.
The TIMESTAMP data type is an extension of the DATE data type. It stores year, month, day, hour, minute, and second values. It also stores fractional seconds, which are not stored by the DATE data type.
TIMESTAMP WITH TIME ZONE is a variant of TIMESTAMP that includes a time zone region name or time zone offset in its value.
TIMESTAMP WITH LOCAL TIME ZONE is another variant of TIMESTAMP. It differs from TIMESTAMP WITH TIME ZONE as follows: data stored in the database is normalized to the database time zone, and the time zone offset is not stored as part of the column data. When users retrieve the data, Oracle Database returns it in the users' local session time zone.
You are seeing a difference because you have different timezones, and you are defaulting the values to SYSDATE, which is the system DATE.
In your local database the system time zone (select dbtimezone from dual) seems to be based on CET, while the Live SQL database seems to be based on UTC, as Oracle recommends. As CET is an hour ahead of UTC/GMT, that explains the one-hour difference.
The TIMESTAMP value is just a simple cast, i.e. cast(SYSDATE as TIMESTAMP), so you get the same value you would if you queried SYSDATE directly, with zero fractional seconds added.
For the TIMESTAMP WITH TIME ZONE it has to store a time zone, and it has to get that from somewhere, and by default it uses your session time zone, not the database time zone. In your local DB that also seems to be CET, but Live SQL is defaulting the session time zone to US Pacific time - not unreasonable, given where Oracle is based. So now it's effectively doing from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) for that value, where for you SESSIONTIMEZONE is CET in one database and US/Pacific in the other.
For the TIMESTAMP WITH LOCAL TIME ZONE it is doing the same, but then normalising that back to the database time zone for storage (effectively cast(from_tz(cast(SYSDATE as TIMESTAMP), SESSIONTIMEZONE) at time zone DBTIMEZONE as TIMESTAMP) - not actually that internally, but gives you the idea), and converting back from the database time zone to your session time zone again when it is queried.
In both databases, if you alter session set time_zone = ... before inserting, and again to a different value before querying, then you'll see different results - the displayed time portion will stay the same for the first two columns, but the time zone will change for the WITH TIME ZONE, and the time will change for the WITH LOCAL TIME ZONE.
fiddle with different session time zones.
You can read more about all of this behaviour in the documentation I already linked to above.
If you use SYSTIMESTAMP instead of SYSDATE as the default for all of your columns then you will avoid the implicit conversion to your session time zone for the WITH TIME ZONE value, and that will always show the database time zone. The LOCAL column will still display in your session time zone, but they will all represent the same time. You will also still see the one-hour difference between the two databases, because they have different database time zones. You could consider defaulting the plain timestamp to sys_extract_utc(SYSTIMESTAMP), or defaulting them all (or at least the first two) to SYSTIMESTAMP at time zone 'UTC'.
fiddle with UTC-normalised values.

Combining date and UTC time fields in Postgres

I have 2 separate fields for date and time. The time field is stored in UTC time. How can I combine the 2 into a datetime field into local time
Example:
date: 2021-03-08
time in UTC: 23:00
time zone: GMT+8
I would like to get 2021-03-08 07:00 in local time
or even 2021-03-07 23:00 in UTC
Note: Combining the fields is not an option unfortunately.
Need to know your time zone to convert. To convert utc to america/los_angeles time zone:
select '2021-03-08 23:00'::timestamp at time zone 'UTC' at time zone 'america/los_angeles'
you can check out below codes:
If you have a timestamp without time zone column and you're storing timestamps as UTC, you need to tell PostgreSQL that, and then tell it to convert it to your local time zone.
select created_at at time zone 'utc' at time zone 'america/los_angeles'
from users;
To be more concise, you can also use the abbreviation for the time zone:
select created_at at time zone 'utc' at time zone 'pst'
from users;
To see the list of time zones PostgreSQL supports:
select * from pg_timezone_names;
SInce the time difference is 8 hours, try
SELECT '2021-03-08'::date + ('23:00'::time + '8 hours'::interval);
If you want this to work with arbitrary time zones, the query becomes more complicated:
SELECT '2021-03-08'::date
+ ((current_date + '23:00'::time)
AT TIME ZONE 'UTC'
AT TIME ZONE 'Asia/Ulaanbaatar'
)::time;

How to get all data post midnight of different timezone?

I have a PostgreSQL table named testing with a column named creation_time as timestamp with time zone. The database timezone is UTC
Now I want to get all rows whose time is greater than 00:00 of the current day as per the timezone "America/New_York".
I know how to get all rows after local midnight:
SELECT * FROM testing
WHERE ( creation_time >= now()::date)
ORDER BY id DESC
But how to use this query with a different timezone?
Assuming "the current day" is also defined by NY time, not by the current timezone setting.
SELECT *
FROM testing
WHERE creation_time >= date_trunc('day', now() AT TIME ZONE 'America/New_York') AT TIME ZONE 'America/New_York'
ORDER BY id DESC;
Yes, AT TIME ZONE 'America/New_York' twice. No typo there.
now() AT TIME ZONE 'America/New_York') gets local NY time. date_trunc gets 00:00 of that day. The 2nd AT TIME ZONE 'America/New_York' converts the local time back to timestamptz, which we finally compare to.
If you want NY 00:00 of your local date, it's simpler:
WHERE creation_time >= CURRENT_DATE::timestamp AT TIME ZONE 'America/New_York'
Same time, but can be a different day!
CURRENT_DATE is the local date (date according to the time zone setting of the current session). Effectively the same as now()::date.
Further reading:
Ignoring time zones altogether in Rails and PostgreSQL

Rails/Postgres query all records created after 5 PM in taking into account daylight savings switch?

I am looking to query for all records on a table with a created_at time of after 5 PM, EST WITH taking into account daylight savings.
My DB is Postgres and all timestamps are stored in UTC as a normal Rails app.
So, say I have four records
ID, created_at, time in EST (-5 offset from UTC),
1, "2017-01-01 22:46:21.829333", 5:46 PM
2, "2017-01-01 21:23:27.259393", 4:23 PM
-------- DST SWITCH -----
ID, created_at, time in EDT (-4 offset from UTC),
3, "2017-03-20 21:52:46.135713", 5:52 PM
4, "2017-06-21 20:08:53.034377", 4:08 PM
My query should return records 1 and 3, but should ignore 2 and 4.
I have tried
SELECT "id",
"created_at"
FROM "orders"
WHERE (created_at::TIME WITHOUT TIME ZONE AT TIME ZONE 'utc' AT TIME ZONE 'US/Eastern' > ("created_at"::DATE + TIME '21:00')::TIME AT TIME ZONE 'utc' AT TIME ZONE 'US/Eastern')
ORDER BY "orders"."created_at" ASC
But this query will return record 1,2,3 when it should not return 2.
Sorry about my previous wrong answer. I don't do much with USA time.
all timestamps are stored in UTC
So ("created_at"::DATE + TIME '21:00')::TIME AT TIME ZONE 'utc' AT TIME ZONE 'US/Eastern' actually says it is utc time 21:00, which does not necessarily guarantee it is 5 PM in EST because the timezone offset changes from 5 to 4. So the compare condition is not correct.
You need to get the local time to make the comparison, rather than the utc time.
Try following code:
SELECT
"id",
"created_at"
FROM "orders"
WHERE (created_at AT TIME ZONE 'utc' AT TIME ZONE 'US/Eastern') :: TIME > TIME '5:00 PM'

Postgres SQL Timezone conversion

I could appreciate a second pair of eyes on my Postgres syntax.
Database stores timestamp in UTC. I'm trying to convert from UTC to Eastern Daylight Time EDT, but the output is not accurate.
Here's my syntax:
SELECT
to_char(((timestamp AT TIME ZONE 'UTC') AT TIME ZONE 'EDT'), 'MM/DD/YYYY HH24:MI')
FROM table_name
Record TimeStamp:
09/10/2016 12:00
Query Output:
09/10/2016 16:00
Desired Output:
09/10/2016 08:00
Thanks for your assistance.
Saying AT TIMEZONE twice is redundant since it will just convert from whatever the current timezone (which you suggest is UTC) to UTC then to EDT.
The fact that you feel the need to convert it to UTC tells me you're not storing it as a TIMESTAMP WITH TIMEZONE. Check if this is the case. If it is, that's likely your problem. From the docs:
If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's timezone parameter, and is converted to UTC using the offset for the timezone zone.
Basically, if you don't specify, it assumes it's from your current timezone by default, not UTC. It's possible you entered a UTC timestamp and it assumed it was EDT.