In my query I need to return only those records that are greater than another date, returned by the function. The field I am comparing is a timestamp and the date function returns is a of type DATE.
My query:
SELECT * FROM TABLE1
WHERE MY_TIMESTAMP > MyFunction1('Test Date');
when I am comparing two values where only time differs slightly - in munutes or seconds, the comparison does not work.
For example when
MY_TIMESTAMP=11/27/2018 12:15:42.000000 PM -05:00
and
MyFunction1('Test Date') returns 11/27/2018 12:22:00 PM
no record should be returned from TABLE1 for that ID but the record does get returned. What can I do to make the comparison more granular?
From Datetime and Interval Arithmetic
Oracle Database performs all timestamp arithmetic in UTC time. For
TIMESTAMP WITH LOCAL TIME ZONE data, Oracle Database converts the
datetime value from the database time zone to UTC and converts back to
the database time zone after performing the arithmetic. For TIMESTAMP WITH TIME ZONE data, the datetime value is always in UTC, so no
conversion is necessary.
If you provide a DATE or TIMESTAMP value (i.e. value without any time zone information) then Oracle attaches the SESSIONTIMEZONE to this value and then it does the comparison based on UTC.
Attaching SESSIONTIMEZONE to input value might be correct or not, it depends on your current session and content of MyFunction1
Related
I am having some trouble understanding how to deal in Postgres with time zone semantic:
Consider this table
TABLE MyTable (
MyDate TIMESTAMP NOT NULL,
// other columns
);
And this query
SELECT *
FROM MyTable
WHERE // conditions
AND tstzrange(#{start} ::timestamp with time zone, #{end} ::timestamp with time zone] #> MyDate::timestamp without time zone at time zone 'CET'
I understand that Postgres only stores the epoch value i.e. no time zone info is ever stored, so I understand the need to specify timezone for start and end as they are formatted strings that Postgres needs to calculate the epoch for.
What I don't really understand is:
MyDate::timestamp without time zone at time zone 'CET'
Postgres knows the epoch for MyDate since it's their values are stored, why the need to "convert" to a time zone ?
What are we actually saying here and can this be simplified ?
Postgres never stores timezone information in timestamp or timestamptz. In the timestamptz case the timestamp being stored is rotated to a UTC value using either the timezone information in the presented timestamp or the value of the setting TimeZone and then stored. In the timestamp case that is not done. On output a timestamptz is rotated back from UTC to whatever the TimeZone setting is or what via at time zone <some_tz>. In the timestamp case the retrieved value is assumed to be the setting of TimeZone unless you override with at time zone <some_tz>. Best practices is that you use timestamptz.
I have a String in this format: 2018-11-01T00:00:00-07:00 and I would like to convert it to a TIMESTAMP and insert it into a TIMESTAMP column. However, when I insert it, it drops the -07:00 without first converting it to -00:00. How do I ensure that it is converted and stored in Redshift properly?
Here is an example:
select ORIGINAL_DATE, TO_TIMESTAMP(ORIGINAL_DATE,'YYYY-MM-DD HH24:MI:SS') FROM CDW_LANDING.X where id = XXXXXX;
=> 2018-11-01T00:00:00-07:00 2018-10-31 17:00:00
The TO_TIMESTAMP converts it to 2018-10-31 17:00:00 which is what I want. However, when I insert it, it becomes 2018-11-01 00:00:00 and simply drops the -07:00.
Here is the example:
insert into cdw_stage.X (ORIG_DT)
select TO_TIMESTAMP(ORIGINAL_DATE,'YYYY-MM-DD HH24:MI:SS')
from CDW_LANDING.INVOICE where id = XXXXXX;
But when I query it with select ORIG_DT from cdw_landing.X;, it displays 2018-11-01 00:00:00. What I would like to see is 2018-10-31 17:00:00 which is what the TO_TIMESTAMP function should do.
The ORIG_DT in Redshift is in TIMESTAMP format. The input date is in VARCHAR.
How do I get Redshift to save this correctly? I also added postgres tag because Redshift is based off of postgres. Thank you so much!!!
2018-11-01T00:00:00-07:00 is not a timestamp (timestamp without time zone) literal, strictly speaking. It is a timestamptz (timestamp with time zone) literal. This is the root of all pain in your question. The wrong cast to timestamp ignores the offset. The Postgres manual:
In a literal that has been determined to be timestamp without time zone, PostgreSQL will silently ignore any time zone indication. That
is, the resulting value is derived from the date/time fields in the
input value, and is not adjusted for time zone.
Bold emphasis mine.
The use of TO_TIMESTAMP() can't save you. The Redshift manual:
Formats that include a time zone (TZ, tz, or OF) are not supported as input.
(The same is true in Postgres.)
Solution
Cast to timestamptz (or use a column of that type to begin with), the rest should fall in place:
SELECT cast('2018-11-01T00:00:00-07:00' AS timestamptz);
Or:
SELECT '2018-11-01T00:00:00-07:00'::timestamptz;
The manual about casting in Redshift.
When an actual timestamptz is assigned to a timestamp column it is converted according to the current timezone setting of the session automatically. If you want a different target timezone, use the AT TIME ZONE construct. Details:
Ignoring time zones altogether in Rails and PostgreSQL
The related answer is for Postgres, but timestamp handling in Redshift (while differing in many other aspects!) is the same. The Redshift manual:
When converting DATE or TIMESTAMP to TIMESTAMPTZ, DATE or TIMESTAMP
are assumed to use the current session time zone. The session time
zone is UTC by default. For more information about setting the session
time zone, see timezone.
I need a SQL Server function (or composition) to record the UTC date and time with the timezone offset into one of my columns of type datetimeoffset(7).
I observe that the SYSUTCDATETIME function gets the UTC date and time into a datetime2(7) data type and therefore that doesn't store the offset.
So, my question is: is there a composition of functions that will yield a datetimeoffset(7) value with the UTC date time along with the time offset (0:00)?
I want to create a table in redshift that stores incrementally incoming data from the source. The date field in the mysql source is not stored as UTC. Is it possible to convert and store the new record as UTC upon record creation.
I was thinking doing something like that:
CREATE TABLE test(
my_dt_field datetime without timezone NOT NULL ...)
Any help would be very appreciated!
Redshift provides following options of datatypes available to store dates:
1.DATE
Use the DATE data type to store simple calendar dates without time stamps.
2.TIMESTAMP
TIMESTAMP is an alias of TIMESTAMP WITHOUT TIME ZONE.
Use the TIMESTAMP data type to store complete timestamp values that include the date and the time of day.
TIMESTAMP columns store values with up to a maximum of 6 digits of precision for fractional seconds.
If you insert a date into a TIMESTAMP column, or a date with a partial time stamp value, the value is implicitly converted into a full time stamp value with default values (00) for missing hours, minutes, and seconds. Time zone values in input strings are ignored.
By default, TIMESTAMP values are Coordinated Universal Time (UTC) in both user tables and Amazon Redshift system tables.
3.TIMESTAMPTZ
TIMESTAMPTZ is an alias of TIMESTAMP WITH TIME ZONE.
Use the TIMESTAMPTZ data type to input complete time stamp values that include the date, the time of day, and a time zone. When an input value includes a time zone, Amazon Redshift uses the time zone to convert the value to Coordinated Universal Time (UTC) and stores the UTC value.
To view a list of supported time zone names, execute the following command.
select pg_timezone_names();
To answer your question declare your column datatype as TIMESTAMP, by default it stores in UTC
You can also refer AWS document here: https://docs.aws.amazon.com/redshift/latest/dg/r_Datetime_types.html
I have a SQL Builder library that direcltly uses ADO.NET. I have a means of creating a select query with a greater-than-or-equal operator, like:
select *
from book
where book.date_created >= {some date}
My issue is that {some date} is going to always be in the UTC time zone, but it's being compared to the book.date_created column which is a TIMESTAMP(6) WITH TIME ZONE column, which will not be in the UTC timezone.
I can execute the query, but my results are off becuaes of timezone comparisons. My query is for all books where the date_created >= x, but some of the results returned are not greater than x because after subtracting 5 hours for the time zone, they are now less than x. The IDataRecord DateTime fields returned are converted to UTC using DateTime.SpecifyKind()
Can I form my query such that it interprets book.date_created in the UTC timezone?
Note: While I'd love to change my Oracle DB columns to not specify timezones, changing table structures is not something I can do.
Edit:
Currently, {some date} is a SQL Parameter. It's backing datatype is a DateTime with UTC as the timezone. As a parameter, it is a TimestampWithTZ. The Value of the parameter is a DateTime with the kind specified as UTC as well.
Update:
The issue seems to be related to my results set from the IDataRecord. When I pull DateTimes off, I use DateTime.SpecifyKind() to put them in UTC mode. The problem is, the date times come out as DateTimeKind.Unspecified. When converting from Unspecified to UTC, it just drops the timezone and declares it is UTC without changing the underlying value. I'm not sure how to have the IDataRecord pull in the TimeZone value.
You need to use the FROM_TZ function that transforms a TIMESTAMP into a TIMESTAMP WITH TIME ZONE. For example, if you know that your variable is in UTC time (+0:00):
SELECT *
FROM book
WHERE date_created >= from_tz(<timestamp>, '+0:00');
Here's a sample script that shows the behaviour you describe (your local time zone should be set to +1:00):
CREATE TABLE t (tz TIMESTAMP(6) WITH TIME ZONE);
INSERT INTO t VALUES
(to_timestamp_tz('20000101 00:00:00 +1:00','yyyymmdd hh24:mi:ss tzh:tzm'));
INSERT INTO t VALUES
(to_timestamp_tz('20000101 00:00:00 -1:00','yyyymmdd hh24:mi:ss tzh:tzm'));
-- This will return two values instead of one
SELECT *
FROM t
WHERE tz >= to_timestamp('20000101 00:00:00', 'yyyymmdd hh24:mi:ss');
-- This query will return only one row
SELECT *
FROM t
WHERE tz >= from_tz (to_timestamp('20000101 00:00:00',
'yyyymmdd hh24:mi:ss'), '+0:00');
below links will help you.
Datetime Datatypes and Time Zone Support
TIMESTAMP WITH TIME ZONE Data Type
Write Time Zone Aware Code in Oracle
ORACLE timezone summary
Oracle Date and Time data types