Unfortunately the postgresql timestamp type only can store timestamps with microsec precision but i need the nanosec also.
PostgreSQL - 8.5. Date/Time Types:
Timestamp, and interval accept an optional precision value p which specifies the number of fractional digits retained in the seconds field. By default, there is no explicit bound on precision. The allowed range of p is from 0 to 6 for the timestamp and interval types.
And i need 7:
0,000 000 001 [ billionth ] nanosecond [ ns ]
0,000 001 [ millionth ] microsecond [ µs ]
0,001 [ thousandth ] millisecond [ ms ]
0.01 [ hundredth ] centisecond [ cs ]
1.0 second [ s ]
Is there any elegant and efficient way to handle this problem?
EDIT:
Maybe store the timestamp in bigint?
Use numeric as a base type of nano timestamps. The function converts a numeric value to its textual timestamp representation:
create or replace function nanotimestamp_as_text(numeric)
returns text language sql immutable as $$
select concat(to_timestamp(trunc($1))::timestamp::text, ltrim(($1- trunc($1))::text, '0'))
$$;
You can also easily convert numeric values to regular timestamps in cases where the super precision is not necessary, example:
with my_data(nano_timestamp) as (
select 1508327235.388551234::numeric
)
select
to_timestamp(nano_timestamp)::timestamp,
nanotimestamp_as_text(nano_timestamp)
from my_data;
to_timestamp | nanotimestamp_as_text
----------------------------+-------------------------------
2017-10-18 13:47:15.388551 | 2017-10-18 13:47:15.388551234
(1 row)
As others have pointed out, Postgres doesn't provide such type out of the box. However, it's relatively simple to create an extension that supports nanosecond resolution due to the open-source nature of Postgres. I faced similar issues a while ago and created this timestamp9 extension for Postgres.
It internally stores the timestamp as a bigint and defines it as the number of nanoseconds since the UNIX epoch. It provides some convenience functions around it that make it easy to view and manipulate the timestamps. If you can live with the limited time range that these timestamps can have, between the year 1970 and the year 2262, then this is a good solution.
Klin's answer is a perfect solution if you don't want to have an extension installed on your system. However, it cooperates less nicely with existing timestamp/interval types and it's also less efficient, because it uses a numeric type as storage. Having an extension gives you greater flexibility.
Disclaimer: I'm the author of the extension
Bigint will work. If you are going to save all the timestamps with nanosecond precision, I would recommend to define a new cast:
CREATE CAST (timestamp AS bigint)
WITHOUT FUNCTION;
Related
I have some SQLite database in which one of the columns has data type as realdate and the column has value as 2453137.5
can anyone please comment on this?
any help is appreciated :)
From SQLlite Docs
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
In your example you are using REAL datatype to store Dates. It will give the output which is not human readable.
For eg., If i'm storing current date and time
CREATE TABLE
IF NOT EXISTS DATEREAL (d1 real);
INSERT INTO DATEREAL (d1)
VALUES(julianday('now'));
SELECT * from DATEREAL;
Output : 2458792.7882345
You can read this using built-in date() and time() as shown below
SELECT
date(d1),
time(d1)
FROM
datereal;
Output :
date(d1) time(d1)
2019-11-05 06:55:03
Check demo here
One of the powerful features of SQLite is allowing you to choose the storage type.
Real number has 2 advantages:
High precision regarding fraction seconds
Longest time range
I got this answer from a user named Zso.
Here's the link to the original post How do DATETIME values work in SQLite?.
Hope this might help you to understand better.
My solution is the following
SELECT TIMESTAMP_ADD('1970-01-01', INTERVAL 1551692341 SECOND) AS ts
Is there any other, more readable, way to convert a unix timestamp to a datetime ?
Yes there is.
TIMESTAMP_SECONDS(int64_expression). Description. Interprets
int64_expression as the number of seconds since 1970-01-01 00:00:00
UTC
Example:
SELECT timestamp_seconds(1551692341)
returns
2019-03-04 09:39:01 UTC
I am curious, why would you want anything simpler than what you already have.
Alternatively, BQ has support for UDF (user defined function) wherein you can create TEMP functions embedded with your JS snippet to parse the epoch value and convert into a required date value in any format deemed fit for data.
In most cases, it is good to have all such formatting and value transformations at the app layer and not create bespoke utilities at DB layer.
In any case, you may want to have a quick read at here
The temporal definition in MonetDB says that:
"TIMESTAMP [ '(' posint ')' ] date concatenated with unique time, precision"
That this mean that if two rows are created with a DEFAULT TIMESTAMP that these timestamps are guaranteed to be unique, even if they are created during the same millisecond?
A MonetDB timestamp is (quoting from mtime.mal):
# #+ Timestamp
# An absolute point of time, as formed by the combination of a date a daytime in GMT
# e.g. 1999-01-31##23:59:59:000
so, your answer is no, a timestamp is not guaranteed to be unique.
caveat: I haven't looked at the part of the MonetDB sources which handles DEFAULT TIMESTAMP, so I'm only 90% sure about my answer.
A previous solution regarding obtaining an answer in milliseconds for differences between two timestamps does not work in Hive 1.0 on Amazon EMR. Hive returns a blank column when casting a timestamp as double in my testing today. No errors are thrown when doing the CAST. Being able to calculate a time difference in fractions of a second between two columns of type "timestamp" are critical to our analysis. Any ideas?
You should try to convert into unix_timestamp using unix_timestamp(timestamp) but I think you will still be losing milliseconds.
select (unix_timestamp(DATE1)-unix_timestamp(DATE2)) TIMEDIFF from TABLE;
In PostgreSQL, the data types timestamp and timestamp with timezone both use 8 bytes.
My questions are:
What format is used to store date & time in a timestamp?
How is the time zone information stored in the timestamp with timezone
type, and how is it parsed later when reading the type?
This is just a misunderstanding stemming from the somewhat misleading type name. The time zone itself is not stored at all. It just acts as offset to compute a UTC timestamp (input), which is actually stored. Or as decorator in the display of a timestamp according to the current or given time zone (output). That's all according to the SQL standard.
Just the point in time is stored, no zone information. That's why 64 bit of information is enough. The timestamp is displayed to the client according to the current time zone setting of the session.
Details:
Ignoring time zones altogether in Rails and PostgreSQL
Also, since Jon mentioned it, time with time zone is defined in the SQL standard and thus implemented in Postgres, but its use is discouraged:
time with time zone is defined by the SQL standard, but the definition
exhibits properties which lead to questionable usefulness.
It's an inherently ambiguous type that cannot deal with DST properly.
Looking at the documentation:
Timestamps are stored either as integers, or (deprecated) floating point numbers
I don't believe timestamp with timezone could be correctly encoded within 8 bytes if it actually stored a time zone. Just the timestamp requires 64 bits, as log2(298989 * 365 * 24 * 60 * 60 * 1000000) is greater than 63. Note that time with time zone requires 12 bytes, with the same precision but a range of a single day.
See Erwin's answer to explain how it actually manages to be stored in 8 bytes - it should be called "timestamp without a time zone, but stored in UTC and converted into the local time zone for display". Ick.