Oracle: efficiently where clause to filter timestamp column to get all records of a specific day - sql

I have a monthly partitioned table in Oracle by a timestamp column. This table contains > 1 billion rows from a 2019 history data. Now I want to filter this table to get all results for a specific day, regarding the HH24:MI:SS part.
The Problem (#1) I'm facing is that when using TO_CHAR(TIMESTAMPCOLUMN, 'YYYY-MM-DD'), the running time of my queries increases for more recent months. Example:
SELECT * FROM BIG_PART_TABLE WHERE TO_CHAR(TIMESTAMPCOLUMN, 'YYYY-MM-DD') = '2019-01-01' -- 3 sec
SELECT * FROM BIG_PART_TABLE WHERE TO_CHAR(TIMESTAMPCOLUMN, 'YYYY-MM-DD') = '2019-02-01' -- 6 sec
SELECT * FROM BIG_PART_TABLE WHERE TO_CHAR(TIMESTAMPCOLUMN, 'YYYY-MM-DD') = '2019-12-01' -- 36 sec
So I got rid of the TO_CHAR and started to filter like this:
SELECT * FROM BIG_PART_TABLE WHERE TIMESTAMPCOLUMN BETWEEN DATE '2019-01-01' AND DATE '2019-01-02' -- 0.032 sec
SELECT * FROM BIG_PART_TABLE WHERE TIMESTAMPCOLUMN BETWEEN DATE '2019-12-01' AND DATE '2019-12-02' -- 0.031 sec
The Problem (#2) is that I'm to lazy to write a BETWEEN clause, apart from the fact that it increases the chance of errors.
Finally, what I really want is a efficient single where clause to filter my table, like:
SELECT * FROM BIG_PART_TABLE WHERE TIMESTAMPCOLUMN = DATE '2019-01-01'
Thanks for all.

The correct approach is not to use date functions on the date column - using functions like this makes the query non SARGable, meaning that it cannot take advantage of an index on the date column.
There is no syntactical sugar that would make the expression shorter to write.
I would also suggest using half-open intervals instead of between:
WHERE
TIMESTAMPCOLUMN >= DATE '2019-01-01'
AND TIMESTAMPCOLUMN < DATE '2019-01-02'
BETWEEN is inclusive on both ends, so your expression implies that timestamps on 2019-01-02 00:00:00 would be filtered in, while this is most likely not what you want.

Use the partition_extension_clause syntax:
SELECT *
FROM BIG_PART_TABLE PARTITION FOR (DATE '2019-12-01')
WHERE TRUNC(TIMESTAMPCOLUMN) = DATE '2019-12-01' ;
This code is still a bit messy. But at least this syntax allows you to use the same date literal instead of having to create a brand new date expression. And although the code has duplication, the duplication is a bit self-documenting: the first expression is to use partition-pruning to find the nearest segment, the second expression is to get the exact rows.

In order to use the partitioning, Oracle has to recognize the partitioning key. If it is using the full timestamp, then you might have a problem.
There is a reasonable chance it is using trunc(TIMESTAMPCOLUMN) or trunc(TIMESTAMPCOLUMN, 'DD'). If so, then you can use that
WHERE TRUNC(TIMESTAMPCOLUMN) = DATE '2019-01-01'
Once you figure it out, you can add a computed column to the table, so you have:
alter table big_part_table add column timestampcolumn_date as trunc(timestampcolumn);
Then you can use timestampcolumn_date in the WHERE clause.

In the fastest way to in Orace access data is to use the partition-name.
Like in this example:
select * from BIG_PART_TABLE partition(ParititonName);

Related

Date comparison between same data type in sql query

I have a column in the format -
MAX_DATE PROCESS_DATE
2022-10-14T16:09:09.272+00:00 2022-11-08T15:43:29.027+00:00
WHEN I AM WRITING THE CONDITION
SELECT * FROM ANC
WHERE MAX_DATE > PROCESS_DATE
Even then a lot of rows are coming although they do not fulfill the condition. The date format of both is the same, i am not sure how to change both of them in the same date format, i tried the below but still I am getting all rows so the above condition is not working
SELECT * FROM ANC
WHERE TO_DATE(to_char(MAX_DATE,'DD-MM-YYY'),'DD-MM-YYY') > TO_DATE(to_char(PROCESS_DATE,'DD-MM-YYY'),'DD-MM-YYY')
Also this is giving me the output like
MAX_DATE PROCESS_DATE
2022-10-14T00:00:00.000+00:00 2022-11-08T00:00:00.000+00:00
How to compare both the dates ?
If your columns have a TIMESTAMP WITH TIME ZONE data type then just compare the columns directly:
SELECT *
FROM ANC
WHERE MAX_DATE > PROCESS_DATE
If your columns have the VARCHAR2 data type then, firstly, that is bad practice and you should alter the table and change the data type to TIMESTAMP WITH TIME ZONE, and secondly, if you are going to keep them as strings then you need to convert them to timestamps using the TO_TIMESTAMP_TZ function before comparing them:
SELECT *
FROM ANC
WHERE TO_TIMESTAMP_TZ(MAX_DATE, 'YYYY-MM-DD"T"HH24:MI:SS.FFTZH:TZM')
> TO_TIMESTAMP_TZ(PROCESS_DATE, 'YYYY-MM-DD"T"HH24:MI:SS.FFTZH:TZM')
please try with below query
where
convert(datetime, MAX_DATE) >= convert(datetime,PROCESS_DATE)

SELECT matching dates from a timestamp with time zone column

I have a timestamp with time zone column within which I'd like to run a query returning all matching dates. eg. I want all rows which have a timestamp with date 2019-09-30. I'm trying something like this but haven't been able to figure it out:
SELECT * FROM table WHERE
x='1277' AND
date='2019-09-30 21:40:01.316240 +00:00'::DATE;
There are two options:
range search:
WHERE timestampcol >= TIMESTAMPTZ '2019-09-30'
AND timestampcol < (TIMESTAMPTZ '2019-09-30' + INTERVAL '1 day')
The proper index to make this fast is
CREATE INDEX ON atable (timestampcol);
conversion to date:
WHERE CAST(timestampcol AS date) = '2019-09-30'
The proper index to make this fast is
CREATE INDEX ON atable ((CAST(timestampcol AS date)));
Both methods work equally well. The second method has a shorter WHERE clause, but a specialized index that maybe no other query can benefit from.
You can use such a collation among your date converted column value and fixed date value :
with tab( x, date ) as
(
select 1277, '2019-09-30 21:40:01.316240 +00:00'::timestamp with time zone union all
select 1278, '2019-09-29 21:40:01.316240 +00:00'::timestamp with time zone
)
select *
from tab
where date::date = date'2019-09-30';
Demo
I am not familiar with postgresql, but the following page https://www.postgresql.org/docs/9.1/datatype-datetime.html indicates that there is an optional precision value p that is used to specify the number of fractional digits retained in the seconds field. How was the data type configured for your date field? Is the timestamp column being stored as an eight byte integer or a floating-point number? If the later, the effective limit of precision might be less than 6. All of this info is on the above linked page.
You have several options, as they have been mentioned, and may vary depending on the data type of your field named "date".
For example,
Turn your field to date, returning format 'yyyy-mm-dd':
SELECT * FROM table WHERE x='1277' AND date::DATE='2019-09-30';
"date" just after field name.
Convert it to char and retrieve 10 characters:
SELECT * FROM table WHERE x='1277' AND LEFT(date::varchar,10)='2019-09-30';
Like the previous:
SELECT * FROM table WHERE x='1277' AND to_char(date,'yyyymmdd')='20190930';
And there are many others. For more specific info, you have to check PostgreSQL documentation to check which one is best for you or post more information in order we can guess more about your problem.
https://www.postgresql.org/docs/9.4/functions-datetime.html

Date difference = 0 in where clause Oracle?

I have the same problem as this. The only difference is that I use Oracle. I want to select the rows which has insertion_date='20.11.2018'. So my query was
select * from table where insertion_date='20.11.2018'
In that question they suggested datediff, so I looked at its equivalent in oracle and I learned that I can do date arithmetic. So I tried somethings like these:
select * from table where insertion_date -'20.11.2018'=0;
It gave ora-00932 inconsistent datatypes expected date got number.
So, then I tried;
select * from table where insertion_date - to_date('20.11.2018', 'dd.MM.YYYY') = 0;
It does not give error but also does not display the results which I know there must be. What am I doing wrong here? Thanks.
Update: Sorry I forgot to mention that insertion_date is type date. But it also has time(hour, minutes, seconds) info in it.
What is INSERTION_DATE's datatype?
If it is DATE, then comparing it to another date (note: this is date literal; value you used is a string!)
select * from table where insertion_date = date '2018-11-20'
might work, unless INSERTION_DATE contains time component (hours and minutes). Then, the simplest option is to truncate its value (so that you get date itself, at midnight):
select * from table where trunc(insertion_date) = date '2018-11-20'
but it'll ruin index you have on that column (unless it is a function-based one). For small tables, it won't make any difference. For large amount of data, it would so convert it to
select * from table where insertion_date >= date '2018-11-20'
and insertion_date < date '2018-11-21'
If, on the other hand, INSERTION_DATE is a string (VARCHAR2 or CHAR) datatype (which is a really bad idea; consider switching to DATE datatype), then you have to know its format, convert it to DATE first and then compare to another date. For example, if it was a string that contains date values in format dd.mm.yyyy, then
select * from table where to_date(insertion_date, 'dd.mm.yyyy') = date '2018-11-20'
This will certainly fail if any string in that column doesn't match such a format or contains invalid values (such as "date" 53.67.Bx48).

Select where date without time

I am using SQLite, now how can I Select a query if I have a timestamp column that has this value for example 07:00 06/03/13 but I want only to select where timestamp 06/03/13.
This is my example of my query..
select
timestamp, pname
from
attendance
where
timestamp between `06/01/13` and `06/10/13`
Use date
select timestamp, pname
from attendance
where date(timestamp) = '2013-03-06'
When filtering on a timestamp column, you want this logic.
where YourField >= TheStartOfYourRange
and YourField < TheDayAfterTheEndOfYourRange
This is equivalent to the two answers that use the date() function, but will generally perform faster. Using functions in the where clause usually slows down production, especially on indexed fields.
SQLite support function date, which can be used to get the date part of a timestamp. For example:
SELECT * FROM mytable WHERE date(timestamp_col) = '2013-05-20'
See SQLite date and time functions documentation for more info.

Issue with Fetching Date using SYSDATE?

I have a table structure having a EMP_DATE column as below
ID EMP_DATE
---- -----------
5400 14-FEB-2012
and i have inserted records into the table as below
INSERT INTO TEST_DATE VALUES(5400,SYSDATE);
After inserting records while i am trying to fetch the records of those who has EMP_DATE
as SYSDATE its giving no rows selected.
For time being let the SYSDATE be '01-JUL-2012`
SELECT * FROM TEST_DATE WHERE EMP_DATE = SYSDATE;
(OR)
SELECT * FROM TESt_DATE WHERE EMP_DATE = '01-JUL-2012';
i was not able figure out any solution .
Any suggestions would be helpful .
The main problem is that a date includes hours, minutes and seconds, which you're not allowing for. If you want everything for a single day you can use the trunc function in order to get this:
SELECT * FROM TEST_DATE WHERE trunc(EMP_DATE) = trunc(SYSDATE);
By default trunc removes the time portion of a date, when operating on a date column. I would normally recommend a functional index on trunc(emp_date) in order to optimize this query. Something like:
create index i_test_date on test_date(trunc(emp_date));
I've built a little SQL Fiddle to demonstrate this.
There is an additional problem; though Oracle does support ANSI date literals your second query is wrong. Always, explicitly convert to a string to a date using the to_date function.
SELECT * FROM TEST_DATE WHERE EMP_DATE = to_date('01-07-2012','dd-mm-yyyy');
I've used the mm datetime format model instead of mon as there's no guarantee that JUL will always mean July; it depends on your NLS parameters, what "date language" your particular database or session is using. A datetime format model is how you tell the database the format of whatever string you're passing it to be converted into a date.
If you're interested the ANSI syntax for your second query would be:
SELECT * FROM TESt_DATE WHERE trunc(EMP_DATE) = DATE '2012-07-01'
It must be in this format (YYYY-MM-DD) to work.