I have a column named 'CreatedAt' in postgres (DBeaver client) that is an int8 datatype and holds a unix timestamp value. Example: 1659347651689
I am writing a query that I'd like to input an ISO datet ime in the where clause and have it automatically convert to find the applicable records.
For example:
Normally, I'd write:
select * from table1 where CreatedAt = '2022-08-01 09:54:11.000'
I can't do that because the CreatedAt column value is 1659347651689. Is there a way to have it automatically convert and locate the record with that datetime?
I tried this:
`select * from table1 where CreatedAt = date("CreatedAt",strtotime('2022-08-01 09:53:27.000'))`
but strtotime doesn't exist (guessing because it's a Python command. I tried date, dateadd, but no luck)
Your data appears to be in milliseconds, so:
select to_timestamp(1659347651689/1000);
Yes thank you Jeroen Mostert and a_horse_with_no_name (great userid). After reading the links here, I got it.
If anyone else is looking, the answer is:
select * from table1 pfs
where timestamp 'epoch' + pfs."CreatedAt" /1000 * interval '1 second' = '2022-08-01 09:53:13.000'
Related
I have a timestamp with time zone column within which I'd like to run a query returning all matching dates. eg. I want all rows which have a timestamp with date 2019-09-30. I'm trying something like this but haven't been able to figure it out:
SELECT * FROM table WHERE
x='1277' AND
date='2019-09-30 21:40:01.316240 +00:00'::DATE;
There are two options:
range search:
WHERE timestampcol >= TIMESTAMPTZ '2019-09-30'
AND timestampcol < (TIMESTAMPTZ '2019-09-30' + INTERVAL '1 day')
The proper index to make this fast is
CREATE INDEX ON atable (timestampcol);
conversion to date:
WHERE CAST(timestampcol AS date) = '2019-09-30'
The proper index to make this fast is
CREATE INDEX ON atable ((CAST(timestampcol AS date)));
Both methods work equally well. The second method has a shorter WHERE clause, but a specialized index that maybe no other query can benefit from.
You can use such a collation among your date converted column value and fixed date value :
with tab( x, date ) as
(
select 1277, '2019-09-30 21:40:01.316240 +00:00'::timestamp with time zone union all
select 1278, '2019-09-29 21:40:01.316240 +00:00'::timestamp with time zone
)
select *
from tab
where date::date = date'2019-09-30';
Demo
I am not familiar with postgresql, but the following page https://www.postgresql.org/docs/9.1/datatype-datetime.html indicates that there is an optional precision value p that is used to specify the number of fractional digits retained in the seconds field. How was the data type configured for your date field? Is the timestamp column being stored as an eight byte integer or a floating-point number? If the later, the effective limit of precision might be less than 6. All of this info is on the above linked page.
You have several options, as they have been mentioned, and may vary depending on the data type of your field named "date".
For example,
Turn your field to date, returning format 'yyyy-mm-dd':
SELECT * FROM table WHERE x='1277' AND date::DATE='2019-09-30';
"date" just after field name.
Convert it to char and retrieve 10 characters:
SELECT * FROM table WHERE x='1277' AND LEFT(date::varchar,10)='2019-09-30';
Like the previous:
SELECT * FROM table WHERE x='1277' AND to_char(date,'yyyymmdd')='20190930';
And there are many others. For more specific info, you have to check PostgreSQL documentation to check which one is best for you or post more information in order we can guess more about your problem.
https://www.postgresql.org/docs/9.4/functions-datetime.html
I have an integer date column "date_created" storing values like...
20180527, 20191205, 20200208
And am wondering what the best way to parse as a date is so I could do something like this in a query...
select * from table where formatted(date_created) > formatted(date_created) - 90
(to return everything within the last 90 days)
I've found some similar examples that convert from date ints representing seconds or milliseconds, but none where the columns are essentially date strings stored as integers.
Appreciate any thoughts on the best way to achieve this
And am wondering what the best way to parse as a date is so I could do something like this in a query...
You can convert "date as a number" (eg. 20180527 for May 27, 2018) using the following:
cast to varchar
parse_datetime with appropriate format
cast to date (since parse_datetime returns a timestamp)
Example:
presto> SELECT CAST(parse_datetime(CAST(20180527 AS varchar), 'yyyyMMdd') AS date);
_col0
------------
2018-05-27
However, this is not necessarily the best way to query your data. By adapting your search conditions to the format of your data (and not vice versa), you can potentially benefit from predicate push down and partition pruning. See #GordonLinoff answer for information how to do this.
You can do the comparison in the world of integers or of dates. You might as well convert the current date minus 90 days to a number:
select t.*
from t
where date_created >= cast(date_format(current_date - interval '90 day',
'%Y%m%d'
) as int
);
the below query is index friendly for any database since it does not use function on indexed column
select * from table where date_created > timestamp (formatted(date) - 90)
In addition, suppose we have date in format 20211011_1234 and we want one month older date and want back the original format, we can use the following formatting to convert date to int and vice versa.
select cast(date_format(
CAST(parse_datetime(cast(
split_part('20211011_1234', '_', 1) as varchar), 'yyyyMMdd')
AS date) - interval '30' day ,'%Y%m%d') as int) as column_name
I can't seem to get reliable results from the query against a sqlite database using a datetime string as a comparison as so:
select *
from table_1
where mydate >= '1/1/2009' and mydate <= '5/5/2009'
how should I handle datetime comparisons to sqlite?
update:
field mydate is a DateTime datatype
To solve this problem, I store dates as YYYYMMDD. Thus,
where mydate >= '20090101' and mydate <= '20050505'
It just plain WORKS all the time. You may only need to write a parser to handle how users might enter their dates so you can convert them to YYYYMMDD.
SQLite doesn't have dedicated datetime types, but does have a few datetime functions. Follow the string representation formats (actually only formats 1-10) understood by those functions (storing the value as a string) and then you can use them, plus lexicographical comparison on the strings will match datetime comparison (as long as you don't try to compare dates to times or datetimes to times, which doesn't make a whole lot of sense anyway).
Depending on which language you use, you can even get automatic conversion. (Which doesn't apply to comparisons in SQL statements like the example, but will make your life easier.)
I had the same issue recently, and I solved it like this:
SELECT * FROM table WHERE
strftime('%s', date) BETWEEN strftime('%s', start_date) AND strftime('%s', end_date)
The following is working fine for me using SQLite:
SELECT *
FROM ingresosgastos
WHERE fecharegistro BETWEEN "2010-01-01" AND "2013-01-01"
Following worked for me.
SELECT *
FROM table_log
WHERE DATE(start_time) <= '2017-01-09' AND DATE(start_time) >= '2016-12-21'
Sqlite can not compare on dates. we need to convert into seconds and cast it as integer.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
I have a situation where I want data from up to two days ago and up until the end of today.
I arrived at the following.
WHERE dateTimeRecorded between date('now', 'start of day','-2 days')
and date('now', 'start of day', '+1 day')
Ok, technically I also pull in midnight on tomorrow like the original poster, if there was any data, but my data is all historical.
The key thing to remember, the initial poster excluded all data after 2009-11-15 00:00:00. So, any data that was recorded at midnight on the 15th was included but any data after midnight on the 15th was not.
If their query was,
select *
from table_1
where mydate between Datetime('2009-11-13 00:00:00')
and Datetime('2009-11-15 23:59:59')
Use of the between clause for clarity.
It would have been slightly better. It still does not take into account leap seconds in which an hour can actually have more than 60 seconds, but good enough for discussions here :)
I had to store the time with the time-zone information in it, and was able to get queries working with the following format:
"SELECT * FROM events WHERE datetime(date_added) BETWEEN
datetime('2015-03-06 20:11:00 -04:00') AND datetime('2015-03-06 20:13:00 -04:00')"
The time is stored in the database as regular TEXT in the following format:
2015-03-06 20:12:15 -04:00
Right now i am developing using System.Data.SQlite NuGet package (version 1.0.109.2). Which using SQLite version 3.24.0.
And this works for me.
SELECT * FROM tables WHERE datetime
BETWEEN '2018-10-01 00:00:00' AND '2018-10-10 23:59:59';
I don't need to use the datetime() function. Perhaps they already updated the SQL query on that SQLite version.
Below are the methods to compare the dates but before that we need to identify the format of date stored in DB
I have dates stored in MM/DD/YYYY HH:MM format so it has to be compared in that format
Below query compares the convert the date into MM/DD/YYY format and get data from last five days till today. BETWEEN operator will help and you can simply specify start date AND end date.
select * from myTable where myColumn BETWEEN strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day') AND strftime('%m/%d/%Y %H:%M',datetime('now','localtime'));
Below query will use greater than operator (>).
select * from myTable where myColumn > strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day');
All the computation I have done is using current time, you can change the format and date as per your need.
Hope this will help you
Summved
You could also write up your own user functions to handle dates in the format you choose. SQLite has a fairly simple method for writing your own user functions. For example, I wrote a few to add time durations together.
My query I did as follows:
SELECT COUNT(carSold)
FROM cars_sales_tbl
WHERE date
BETWEEN '2015-04-01' AND '2015-04-30'
AND carType = "Hybrid"
I got the hint by #ifredy's answer. The all I did is, I wanted this query to be run in iOS, using Objective-C. And it works!
Hope someone who does iOS Development, will get use out of this answer too!
Here is a working example in C# in three ways:
string tableName = "TestTable";
var startDate = DateTime.Today.ToString("yyyy-MM-dd 00:00:00"); \\From today midnight
var endDate = date.AddDays(1).ToString("yyyy-MM-dd HH:mm:ss"); \\ Whole day
string way1 /*long way*/ = $"SELECT * FROM {tableName} WHERE strftime(\'%s\', DateTime)
BETWEEN strftime('%s', \'{startDate}\') AND strftime('%s', \'{endDate}\')";
string way2= $"SELECT * FROM {tableName} WHERE DateTime BETWEEN \'{startDate}\' AND \'{endDate}\'";
string way3= $"SELECT * FROM {tableName} WHERE DateTime >= \'{startDate}\' AND DateTime <=\'{endDate}\'";
select *
from table_1
where date(mydate) >= '1/1/2009' and date(mydate) <= '5/5/2009'
This work for me
I am using SQLite, now how can I Select a query if I have a timestamp column that has this value for example 07:00 06/03/13 but I want only to select where timestamp 06/03/13.
This is my example of my query..
select
timestamp, pname
from
attendance
where
timestamp between `06/01/13` and `06/10/13`
Use date
select timestamp, pname
from attendance
where date(timestamp) = '2013-03-06'
When filtering on a timestamp column, you want this logic.
where YourField >= TheStartOfYourRange
and YourField < TheDayAfterTheEndOfYourRange
This is equivalent to the two answers that use the date() function, but will generally perform faster. Using functions in the where clause usually slows down production, especially on indexed fields.
SQLite support function date, which can be used to get the date part of a timestamp. For example:
SELECT * FROM mytable WHERE date(timestamp_col) = '2013-05-20'
See SQLite date and time functions documentation for more info.
I'm working with Hive and I have a table structured as follows:
CREATE TABLE t1 (
id INT,
created TIMESTAMP,
some_value BIGINT
);
I need to find every row in t1 that is less than 180 days old. The following query yields no rows even though there is data present in the table that matches the search predicate.
select *
from t1
where created > date_sub(from_unixtime(unix_timestamp()), 180);
What is the appropriate way to perform a date comparison in Hive?
How about:
where unix_timestamp() - created < 180 * 24 * 60 * 60
Date math is usually simplest if you can just do it with the actual timestamp values.
Or do you want it to only cut off on whole days? Then I think the problem is with how you are converting back and forth between ints and strings. Try:
where created > unix_timestamp(date_sub(from_unixtime(unix_timestamp(),'yyyy-MM-dd'),180),'yyyy-MM-dd')
Walking through each UDF:
unix_timestamp() returns an int: current time in seconds since epoch
from_unixtime(,'yyyy-MM-dd') converts to a string of the given format, e.g. '2012-12-28'
date_sub(,180) subtracts 180 days from that string, and returns a new string in the same format.
unix_timestamp(,'yyyy-MM-dd') converts that string back to an int
If that's all getting too hairy, you can always write a UDF to do it yourself.
Alternatively you may also use datediff. Then the where clause would be
in case of String timestamp (jdbc format) :
datediff(from_unixtime(unix_timestamp()), created) < 180;
in case of Unix epoch time:
datediff(from_unixtime(unix_timestamp()), from_unixtime(created)) < 180;
I think maybe it's a Hive bug dealing with the timestamp type. I've been trying to use it recently and getting incorrect results.
If I change your schema to use a string instead of timestamp, and supply values in the
yyyy-MM-dd HH:mm:ss
format, then the select query worked for me.
According to the documentation, Hive should be able to convert a BIGINT representing epoch seconds to a timestamp, and that all existing datetime UDFs work with the timestamp data type.
with this simple query:
select from_unixtime(unix_timestamp()), cast(unix_timestamp() as
timestamp) from test_tt limit 1;
I would expect both fields to be the same, but I get:
2012-12-29 00:47:43 1970-01-16 16:52:22.063
I'm seeing other weirdness as well.
TIMESTAMP is milliseconds
unix_timestamp is in seconds
You need to multiply the RHS by 1000.
where created > 1000 * date_sub(from_unixtime(unix_timestamp()), 180);
After reviewing this and referring to Date Difference less than 15 minutes in Hive I came up with a solution. While I'm not sure why Hive doesn't perform the comparison effectively on dates as strings (they should sort and compare lexicographically), the following solution works:
FROM (
SELECT id, value,
unix_timestamp(created) c_ts,
unix_timestamp(date_sub(from_unixtime(unix_timestamp()), 180), 'yyyy-MM-dd') c180_ts
FROM t1
) x
JOIN t1 t ON x.id = t.id
SELECT to_date(t.Created),
x.id, AVG(COALESCE(x.HighestPrice, 0)), AVG(COALESCE(x.LowestPrice, 0))
WHERE unix_timestamp(t.Created) > x.c180_ts
GROUP BY to_date(t.Created), x.id ;