I have a timestamp with time zone column within which I'd like to run a query returning all matching dates. eg. I want all rows which have a timestamp with date 2019-09-30. I'm trying something like this but haven't been able to figure it out:
SELECT * FROM table WHERE
x='1277' AND
date='2019-09-30 21:40:01.316240 +00:00'::DATE;
There are two options:
range search:
WHERE timestampcol >= TIMESTAMPTZ '2019-09-30'
AND timestampcol < (TIMESTAMPTZ '2019-09-30' + INTERVAL '1 day')
The proper index to make this fast is
CREATE INDEX ON atable (timestampcol);
conversion to date:
WHERE CAST(timestampcol AS date) = '2019-09-30'
The proper index to make this fast is
CREATE INDEX ON atable ((CAST(timestampcol AS date)));
Both methods work equally well. The second method has a shorter WHERE clause, but a specialized index that maybe no other query can benefit from.
You can use such a collation among your date converted column value and fixed date value :
with tab( x, date ) as
(
select 1277, '2019-09-30 21:40:01.316240 +00:00'::timestamp with time zone union all
select 1278, '2019-09-29 21:40:01.316240 +00:00'::timestamp with time zone
)
select *
from tab
where date::date = date'2019-09-30';
Demo
I am not familiar with postgresql, but the following page https://www.postgresql.org/docs/9.1/datatype-datetime.html indicates that there is an optional precision value p that is used to specify the number of fractional digits retained in the seconds field. How was the data type configured for your date field? Is the timestamp column being stored as an eight byte integer or a floating-point number? If the later, the effective limit of precision might be less than 6. All of this info is on the above linked page.
You have several options, as they have been mentioned, and may vary depending on the data type of your field named "date".
For example,
Turn your field to date, returning format 'yyyy-mm-dd':
SELECT * FROM table WHERE x='1277' AND date::DATE='2019-09-30';
"date" just after field name.
Convert it to char and retrieve 10 characters:
SELECT * FROM table WHERE x='1277' AND LEFT(date::varchar,10)='2019-09-30';
Like the previous:
SELECT * FROM table WHERE x='1277' AND to_char(date,'yyyymmdd')='20190930';
And there are many others. For more specific info, you have to check PostgreSQL documentation to check which one is best for you or post more information in order we can guess more about your problem.
https://www.postgresql.org/docs/9.4/functions-datetime.html
Related
I have an integer date column "date_created" storing values like...
20180527, 20191205, 20200208
And am wondering what the best way to parse as a date is so I could do something like this in a query...
select * from table where formatted(date_created) > formatted(date_created) - 90
(to return everything within the last 90 days)
I've found some similar examples that convert from date ints representing seconds or milliseconds, but none where the columns are essentially date strings stored as integers.
Appreciate any thoughts on the best way to achieve this
And am wondering what the best way to parse as a date is so I could do something like this in a query...
You can convert "date as a number" (eg. 20180527 for May 27, 2018) using the following:
cast to varchar
parse_datetime with appropriate format
cast to date (since parse_datetime returns a timestamp)
Example:
presto> SELECT CAST(parse_datetime(CAST(20180527 AS varchar), 'yyyyMMdd') AS date);
_col0
------------
2018-05-27
However, this is not necessarily the best way to query your data. By adapting your search conditions to the format of your data (and not vice versa), you can potentially benefit from predicate push down and partition pruning. See #GordonLinoff answer for information how to do this.
You can do the comparison in the world of integers or of dates. You might as well convert the current date minus 90 days to a number:
select t.*
from t
where date_created >= cast(date_format(current_date - interval '90 day',
'%Y%m%d'
) as int
);
the below query is index friendly for any database since it does not use function on indexed column
select * from table where date_created > timestamp (formatted(date) - 90)
In addition, suppose we have date in format 20211011_1234 and we want one month older date and want back the original format, we can use the following formatting to convert date to int and vice versa.
select cast(date_format(
CAST(parse_datetime(cast(
split_part('20211011_1234', '_', 1) as varchar), 'yyyyMMdd')
AS date) - interval '30' day ,'%Y%m%d') as int) as column_name
I have a table.
as you can see created_date column is timestamp field. now on selection i want to consider only date value. for e.g if i want to make selection of rows from today i want to do something like:-
select * from audit_logs where created_date = '2018-11-28';
the above query returns null. is it possible to make selection this way ?
You can use date() function
select * from audit_logs where date(created_date) = '2018-11-28'
An alternative function is date_trunc(text, timestamp), if you want more flexibility (truncate to hour, for example). Check out the postgres documentation (which is great) for all the date/time functions: https://www.postgresql.org/docs/current/functions-datetime.html
You do get a timestamp back instead of date, mind.
I have the same problem as this. The only difference is that I use Oracle. I want to select the rows which has insertion_date='20.11.2018'. So my query was
select * from table where insertion_date='20.11.2018'
In that question they suggested datediff, so I looked at its equivalent in oracle and I learned that I can do date arithmetic. So I tried somethings like these:
select * from table where insertion_date -'20.11.2018'=0;
It gave ora-00932 inconsistent datatypes expected date got number.
So, then I tried;
select * from table where insertion_date - to_date('20.11.2018', 'dd.MM.YYYY') = 0;
It does not give error but also does not display the results which I know there must be. What am I doing wrong here? Thanks.
Update: Sorry I forgot to mention that insertion_date is type date. But it also has time(hour, minutes, seconds) info in it.
What is INSERTION_DATE's datatype?
If it is DATE, then comparing it to another date (note: this is date literal; value you used is a string!)
select * from table where insertion_date = date '2018-11-20'
might work, unless INSERTION_DATE contains time component (hours and minutes). Then, the simplest option is to truncate its value (so that you get date itself, at midnight):
select * from table where trunc(insertion_date) = date '2018-11-20'
but it'll ruin index you have on that column (unless it is a function-based one). For small tables, it won't make any difference. For large amount of data, it would so convert it to
select * from table where insertion_date >= date '2018-11-20'
and insertion_date < date '2018-11-21'
If, on the other hand, INSERTION_DATE is a string (VARCHAR2 or CHAR) datatype (which is a really bad idea; consider switching to DATE datatype), then you have to know its format, convert it to DATE first and then compare to another date. For example, if it was a string that contains date values in format dd.mm.yyyy, then
select * from table where to_date(insertion_date, 'dd.mm.yyyy') = date '2018-11-20'
This will certainly fail if any string in that column doesn't match such a format or contains invalid values (such as "date" 53.67.Bx48).
I want to select a varchar field as a date field
For example a field has this value "30.12.2011 21:15:03"
and when i select this
select DATE from TABLE where DATE = '30.12.2011'
i get no result.
You ask about getting the date part of a timestamp field, but what your question is actually about is filtering on the date of a timestamp field. There is a much simpler method of accomplishing that: you can use the knowledge that all the possible timestamps on a specific date won't have any timestamps for different dates between them.
select DATE
from TABLE
where DATE >= '30.12.2011' and DATE < '31.12.2011'
Your edit explains that you haven't got a timestamp field at all. Nevertheless, a similar approach may still work:
select DATE
from TABLE
where DATE LIKE '30.12.2011 %'
Or the Firebird-specific
select DATE
from TABLE
where DATE starting with '30.12.2011 '
Assuming the field is a date field, use the DATE introducer combined with yyyy-mm-dd (or TIMESTAMP with time as well).
So use:
select datefield from sometable where datefield = DATE '2011-12-30'
Technically you can leave off the introducer, but it is 'correcter' in the light of the SQL standard.
Assuming a TIMESTAMP field, you won't get results unless the timestamp is (always) at 00:00:00.0000 (in which case it should have been a DATE instead).
For the comparison to work, you need to use either BETWEEN, eg:
select timestampfield from sometable
where timestampfield BETWEEN '2011-12-30 00:00:00.0000' AND '2011-12-30 23:59:59.9999'
or truncate the timestamp to a date (this may adversely effect performance if the timestamp is indexed, because then the index can no longer be used), eg:
select timestampfield from sometable
where CAST(timestampfield AS DATE) = '2011-12-30'
If the date is stored in a VARCHAR field (which in itself is a bad idea), there are several solutions, first is to handle it as date manipulation:
select varcharfield from sometable
where CAST(CAST(varcharfield AS TIMESTAMP) AS DATE) = '2011-12-30'
The double cast is required if you have a time-component in VARCHARFIELD as well. This assumes dates in the supported format listed below. If you use BETWEEN as above, you can use a single cast to timestamp)
The other solution (as suggested by hvd) is to treat it purely as string manipulation, for example:
select varcharfield from sometable
where varcharfield STARTING WITH '30.12.2011'
This has its own set of problems if you want to select ranges. Bottomline: use a real TIMESTAMP field!
Note that Firebird supports multiple formats:
yyyy-mm-dd, eg 2014-05-25 (ISO-8601 format, probably best to use as it reduces confusion)
dd.mm.yyyy, eg 25.05.2014
mm/dd/yyyy, eg 05/25/2014
mm-dd-yyyy, eg 05-25-2014
dd mmm yyyy, eg 25 MAY 2014 (+ variations with a -, . or / as separator)
mmm dd yyyy, eg MAY 25 2014 (+ variations with a -, . or / as separator)
select DATE from TABLE where cast(DATE as date) = '30.12.2011'
Date field is a timestamp
Here is the answere to my question:
CAST
(
SUBSTRING
(field FROM 1 FOR 2)
||'.'||
SUBSTRING
(field FROM 4 FOR 2)
||'.'||
SUBSTRING
(field FFROM 7 FOR 4)
AS DATE)
This took me 5 hours to find this out, maybe there should be a "-" instead of "." but it works.
I'm working with Hive and I have a table structured as follows:
CREATE TABLE t1 (
id INT,
created TIMESTAMP,
some_value BIGINT
);
I need to find every row in t1 that is less than 180 days old. The following query yields no rows even though there is data present in the table that matches the search predicate.
select *
from t1
where created > date_sub(from_unixtime(unix_timestamp()), 180);
What is the appropriate way to perform a date comparison in Hive?
How about:
where unix_timestamp() - created < 180 * 24 * 60 * 60
Date math is usually simplest if you can just do it with the actual timestamp values.
Or do you want it to only cut off on whole days? Then I think the problem is with how you are converting back and forth between ints and strings. Try:
where created > unix_timestamp(date_sub(from_unixtime(unix_timestamp(),'yyyy-MM-dd'),180),'yyyy-MM-dd')
Walking through each UDF:
unix_timestamp() returns an int: current time in seconds since epoch
from_unixtime(,'yyyy-MM-dd') converts to a string of the given format, e.g. '2012-12-28'
date_sub(,180) subtracts 180 days from that string, and returns a new string in the same format.
unix_timestamp(,'yyyy-MM-dd') converts that string back to an int
If that's all getting too hairy, you can always write a UDF to do it yourself.
Alternatively you may also use datediff. Then the where clause would be
in case of String timestamp (jdbc format) :
datediff(from_unixtime(unix_timestamp()), created) < 180;
in case of Unix epoch time:
datediff(from_unixtime(unix_timestamp()), from_unixtime(created)) < 180;
I think maybe it's a Hive bug dealing with the timestamp type. I've been trying to use it recently and getting incorrect results.
If I change your schema to use a string instead of timestamp, and supply values in the
yyyy-MM-dd HH:mm:ss
format, then the select query worked for me.
According to the documentation, Hive should be able to convert a BIGINT representing epoch seconds to a timestamp, and that all existing datetime UDFs work with the timestamp data type.
with this simple query:
select from_unixtime(unix_timestamp()), cast(unix_timestamp() as
timestamp) from test_tt limit 1;
I would expect both fields to be the same, but I get:
2012-12-29 00:47:43 1970-01-16 16:52:22.063
I'm seeing other weirdness as well.
TIMESTAMP is milliseconds
unix_timestamp is in seconds
You need to multiply the RHS by 1000.
where created > 1000 * date_sub(from_unixtime(unix_timestamp()), 180);
After reviewing this and referring to Date Difference less than 15 minutes in Hive I came up with a solution. While I'm not sure why Hive doesn't perform the comparison effectively on dates as strings (they should sort and compare lexicographically), the following solution works:
FROM (
SELECT id, value,
unix_timestamp(created) c_ts,
unix_timestamp(date_sub(from_unixtime(unix_timestamp()), 180), 'yyyy-MM-dd') c180_ts
FROM t1
) x
JOIN t1 t ON x.id = t.id
SELECT to_date(t.Created),
x.id, AVG(COALESCE(x.HighestPrice, 0)), AVG(COALESCE(x.LowestPrice, 0))
WHERE unix_timestamp(t.Created) > x.c180_ts
GROUP BY to_date(t.Created), x.id ;