The temporal definition in MonetDB says that:
"TIMESTAMP [ '(' posint ')' ] date concatenated with unique time, precision"
That this mean that if two rows are created with a DEFAULT TIMESTAMP that these timestamps are guaranteed to be unique, even if they are created during the same millisecond?
A MonetDB timestamp is (quoting from mtime.mal):
# #+ Timestamp
# An absolute point of time, as formed by the combination of a date a daytime in GMT
# e.g. 1999-01-31##23:59:59:000
so, your answer is no, a timestamp is not guaranteed to be unique.
caveat: I haven't looked at the part of the MonetDB sources which handles DEFAULT TIMESTAMP, so I'm only 90% sure about my answer.
Related
I want to update a date with a timezone (+2 hours) but it ends up as UTC (0 hours)
Date type is 'timestamp-with-timezone'
Query...
update table set date = '2022-05-25 13:28+02:00'
will end up as this in the database.
2022-05-25 11:28:00+00
What's wrong here?
tl;dr
Nothing wrong. Postgres stores values of TIMESTAMP WITH TIME ZONE in UTC, always an offset from UTC of zero. Any submitted offset or zone is used to adjust to UTC.
Details
Date type is 'timestamp-with-timezone'
No such type in standard SQL, nor in Postgres.
I’ll assume you meant TIMESTAMP WITH TIME ZONE.
it ends up as UTC (0 hours)
Read the fine manual. You are seeing documented behavior.
Postgres always stores values in a column of type TIMESTAMP WITH TIME ZONE in UTC, that is, with an offset of zero hours-minutes-seconds.
Any time zone or offset provided with an input is used to adjust into UTC. That provided zone or offset is then discarded.
So the name of the type TIMESTAMP WITH TIME ZONE is a misnomer. First, the authors of the SQL were thinking in terms of offset, not real time zones. Second, any submitted time zone is not stored. A submitted zone is used to adjust and then discarded.
If you need to track the original offset or zone, add an extra column. You’ll have to add code to store the offset amount or the time zone name.
update table set date = '2022-05-25 13:28+02:00' will end up as this in the database. 2022-05-25 11:28:00+00 What's wrong here?
Nothing is wrong. That is a feature, not a bug. Both of those strings represent the very same simultaneous moment.
FYI, database engines vary widely in their behavior handling date-time types and behaviors.
Some do as Postgres does regarding TIMESTAMP WITH TIME ZONE, adjusting to UTC and then discarding any provided time zone or offset. Some others may not.
The SQL standard barely touches on the topic of date-time handling. It declares a few types, and does that poorly with incomplete coverage of all cases. And the standard neglects to define behavior.
So, be very careful when it comes to date-time handling in your database work. Read very carefully the documentation for your particular database engine. Do not make assumptions. Run experiments to validate your understanding. And know that writing portable SQL code for date-time may not be feasible.
Unfortunately the postgresql timestamp type only can store timestamps with microsec precision but i need the nanosec also.
PostgreSQL - 8.5. Date/Time Types:
Timestamp, and interval accept an optional precision value p which specifies the number of fractional digits retained in the seconds field. By default, there is no explicit bound on precision. The allowed range of p is from 0 to 6 for the timestamp and interval types.
And i need 7:
0,000 000 001 [ billionth ] nanosecond [ ns ]
0,000 001 [ millionth ] microsecond [ µs ]
0,001 [ thousandth ] millisecond [ ms ]
0.01 [ hundredth ] centisecond [ cs ]
1.0 second [ s ]
Is there any elegant and efficient way to handle this problem?
EDIT:
Maybe store the timestamp in bigint?
Use numeric as a base type of nano timestamps. The function converts a numeric value to its textual timestamp representation:
create or replace function nanotimestamp_as_text(numeric)
returns text language sql immutable as $$
select concat(to_timestamp(trunc($1))::timestamp::text, ltrim(($1- trunc($1))::text, '0'))
$$;
You can also easily convert numeric values to regular timestamps in cases where the super precision is not necessary, example:
with my_data(nano_timestamp) as (
select 1508327235.388551234::numeric
)
select
to_timestamp(nano_timestamp)::timestamp,
nanotimestamp_as_text(nano_timestamp)
from my_data;
to_timestamp | nanotimestamp_as_text
----------------------------+-------------------------------
2017-10-18 13:47:15.388551 | 2017-10-18 13:47:15.388551234
(1 row)
As others have pointed out, Postgres doesn't provide such type out of the box. However, it's relatively simple to create an extension that supports nanosecond resolution due to the open-source nature of Postgres. I faced similar issues a while ago and created this timestamp9 extension for Postgres.
It internally stores the timestamp as a bigint and defines it as the number of nanoseconds since the UNIX epoch. It provides some convenience functions around it that make it easy to view and manipulate the timestamps. If you can live with the limited time range that these timestamps can have, between the year 1970 and the year 2262, then this is a good solution.
Klin's answer is a perfect solution if you don't want to have an extension installed on your system. However, it cooperates less nicely with existing timestamp/interval types and it's also less efficient, because it uses a numeric type as storage. Having an extension gives you greater flexibility.
Disclaimer: I'm the author of the extension
Bigint will work. If you are going to save all the timestamps with nanosecond precision, I would recommend to define a new cast:
CREATE CAST (timestamp AS bigint)
WITHOUT FUNCTION;
My table has a category 'timestamp' where the timestamps are formatted 2015-06-22 18:59:59
However, using DBVisualizer Free 9.2.8 and Vertica, when I try to pull up rows by timestamp with a
SELECT * FROM table WHERE timestamp = '2015-06-22 18:59:59';
(directly copy-pasting the stamp), nothing comes up. Why is this happening and is there a way around it?
FYI, saying "the timestamps are formatted 2015-06-22 18:59:59" is incorrect if you are indeed using a TIMESTAMP type. Such types have their own internal representation of a date-time value, almost always a count since epoch. In your case with Vertica, 8 bytes are used for such storage. The formatting of the date-time value happens when a string representation is generated. Never confuse the string representation with the date-time value. Conflating the two may well be related to your problem/confusion.
A few different thoughts about possible problems…
String Literals
Are you sure Vertica takes strings as timestamp literals? That format you used is common SQL format. But given that Vertica seems to be a specialized database, I would double-check that.
If strings are not allowed, you may need to call some kind of function to transform the string into a date-time values.
Fractional Second
As the comment by Martin Smith points out, the doc for Timestamp-related data types in Vertica 7.1 says those types can have a fractional second to resolution of microseconds. That means up to 6 decimal places of a fraction.
So if you are searching for "2015-06-22 18:59:59" but the stored value is "2015-06-22 18:59:59.012345", no match on the query.
Half-Open
The fractional seconds issue described above is often the cause of problems people have when handling a span of time. If you naïvely try to pinpoint the ending time, you are likely to have problems. Seeing the "59:59" in your example string makes me think this applies to you.
The better approach to spans of time is "Half-Open" (or Half-Closed, whatever) where the beginning is inclusive while the ending is exclusive. Common notation for this is [). In comparison logic this means: value >= start AND value < stop. Notice the lack of EQUALS SIGN in the stop comparison. In English we would say "look for an hour's worth of invoices starting at 2:00 PM and going up to, but not including, 3:00 PM".
Half-Open for a week means Monday-Monday, for a month the first of one month to the first of the next month, and for a year the January 1 of one year to January 1 of the following year.
Half-Open means not using BETWEEN in SQL. SQL's BETWEEN has often be criticized. Instead do something like the following to look for an hour's worth of invoices. Notice the Z on the end of string literal which means "UTC time zone" ("Z" for "Zulu"). (But verify, as my SQL syntax may need fixing.)
SELECT *
FROM some_table_
WHERE invoice_received_ >= '2015-06-22 18:00:00Z'
AND invoice_received_ < '2015-06-22 19:00:00Z'
;
This query will catch any values such as '2015-06-22 18:59:59.654321" which seems to be eluding you.
Reserved Word
I hope you have not really named your table 'table' and your column 'timestamp'. Such use of keywords and reserved words can cause explicit errors or more subtle weird problems.
Tip: The easy way to avoid any of the over a thousand reserved words in various databases is to append a trailing underscore. The SQL standard explicitly promises to never using a trailing underscore in its reserved words. So use "timestamp_" rather than "timestamp". Another example: "invoice_" table and "received_" column. I recommend doing that as a habit on everything your name in SQL: columns, tables, constraints, indexes, and so on.
Time Zone
You are using the TIMESTAMP which is short for TIMESTAMP WITHOUT TIME ZONE. Or so I presume; the Vertica doc is vague but that is the common usage as seen in the Postgres doc, and may even be standard SQL.
Anyways, TIMESTAMP WITHOUT TIME ZONE is usually the wrong type for most business purposes. The WITH time zone is misnamed and often misunderstood as a consequence: It means "with respect for time zone" where data inputs that include an offset or other time zone information from UTC are adjusted to UTC during the INSERT/UPDATE operations. The WITHOUT type simply ignores any such offset or time zone information.
The WITHOUT type should only be used for the concept of a date-time generally without being tied to any one locality. For example, saying "Christmas this year starts at beginning of December 25, 2015". That means in any time zone rather than a specific time zone. Obviously Christmas starts earlier in Paris, for example, than in Montréal.
If you are timestamping legal documents such as invoices, or booking appointments with people across time zones, or scheduling shipments in various localities, you should be using WITH time zone type.
So back to your possible problem: Test how Vertica or your client app or your database driver is handling your input string. It may be adjusting time zones as part of the parsing of the string using your client machine’s current default time zone. When sent to the database, that value will not match the stored value if during storage no adjustment to UTC was made.
Tip: Generally best practice is to do all your storage and business logic in UTC, adjusting to local time zones only where expected by user.
In PostgreSQL, the data types timestamp and timestamp with timezone both use 8 bytes.
My questions are:
What format is used to store date & time in a timestamp?
How is the time zone information stored in the timestamp with timezone
type, and how is it parsed later when reading the type?
This is just a misunderstanding stemming from the somewhat misleading type name. The time zone itself is not stored at all. It just acts as offset to compute a UTC timestamp (input), which is actually stored. Or as decorator in the display of a timestamp according to the current or given time zone (output). That's all according to the SQL standard.
Just the point in time is stored, no zone information. That's why 64 bit of information is enough. The timestamp is displayed to the client according to the current time zone setting of the session.
Details:
Ignoring time zones altogether in Rails and PostgreSQL
Also, since Jon mentioned it, time with time zone is defined in the SQL standard and thus implemented in Postgres, but its use is discouraged:
time with time zone is defined by the SQL standard, but the definition
exhibits properties which lead to questionable usefulness.
It's an inherently ambiguous type that cannot deal with DST properly.
Looking at the documentation:
Timestamps are stored either as integers, or (deprecated) floating point numbers
I don't believe timestamp with timezone could be correctly encoded within 8 bytes if it actually stored a time zone. Just the timestamp requires 64 bits, as log2(298989 * 365 * 24 * 60 * 60 * 1000000) is greater than 63. Note that time with time zone requires 12 bytes, with the same precision but a range of a single day.
See Erwin's answer to explain how it actually manages to be stored in 8 bytes - it should be called "timestamp without a time zone, but stored in UTC and converted into the local time zone for display". Ick.
Q1: I want to convert a unix timestamp (INT) to monetdb timestamp ('YYYY-MM-DD HH:MM:SS') format
but it is giving me the GMT time not my actual time.
When I do
select (epoch(cast(current_timestamp as timestamp))-epoch(timestamp '2013-04-25 11:49:00'))
where 2013-04-25 11:49:00 is my systems current time it gives the same difference
I tried using
set time zone interval '05:30' HOUR TO MINUTE;
but it did not change the result
How can I solve this problem??
Example Problem:
I wanted to convert unix timestamp 1366869289 which should be around "2013-04-25 11:25:00" but monetdb gives "2013-04-25 05:55:00"
Knowing nothing about MonetDB, but a lot about timezones, I decided to look in their documentation to see what kind of datatypes are supported and how conversions are handled.
I found this page on Temporal data types. Based on that, I can conclude that a timestamp in MonetDB is always intended to reference UTC/GMT time - which is consistent with other systems.
In order to get a value that is for a particular time zone, they offer the following example:
SET TIME ZONE INTERVAL '1' HOUR TO MINUTE
I assume this means to set the database to offset all times by 1 hour, effectively placing the values all in UTC+01:00, such as is the offset for British Summer Time.
The page also goes on to point out the problems that can arise with using just and offset to adjust time values (see TimeZone != Offset in the TimeZone tag wiki). It also offers a list of various named time zones. But it does not show how to set a time zone to one of the named values. Also, their list appears to be proprietary, and incomplete. While at first glance they appear to have similarities to the IANA/Olson time zone database - the identifiers they specify are not valid TZDB names.
There are some other functions listed on this page, without much explanation. One that looks promising for your needs is LOCALTIMESTAMP. Perhaps this will take the local time zone into account, which appears to be what you were looking for.
I could not find any additional details specific to MonetDB date/time/timezone handling. The documentation appears to be fairly incomplete. You might want to reach out to their mailing list.