Returning results for the whole of yesterday and not the last 24 hours - sql

I have this query that needs data for yesterday. What i have below returns result for the last 24 hours which is different from yesterday 00.00 - 23.59.
Here is what i have but doesn't solve the problem.
Select * from message where now() - arrival_timestamp <= interval '24 hour'

You could cast the timestamp to date with the syntax expression::type (more info on The Type Casts section of The PostgreSQL Documentation). Sufficient tools for making the comparison between dates can be found from the section 9.9. Date/Time Functions and Operators:
SELECT * FROM message WHERE arrival_timestamp::date = current_date - 1;
If you have an index on arrival_timestamp the cast to date would render the index unusable in the query. In that case use other comparison operators:
SELECT * FROM message WHERE arrival_timestamp >= current_date - 1 AND arrival_timestamp < current_date;

You might try this:
SELECT * FROM message
WHERE date_trunc('day', arrival_timestamp) = current_date - interval '1 day'

When working with time or any other data type, you want to avoid a filter criteria in the form
function( field ) comparison other_field_or_constant
This is non-sargable. What that means is that the optimizer can't take advantage if field is indexed. Since it cannot predict what value will result from the function call, it must perform a complete table scan and submit every field value to the function to make the comparison.
So you want every expression in the where clause to be in the form
field comparison other_field_or_constant
Note that function( constant ) is still a constant as all such results are cached and re-used for each comparison rather than the function being called for each comparison.
In your case, here, in pseudo-code, is what you want to do. You want yesterday. Fine, every DBMS I know has a system function that returns Now. Yesterday is from midnight the day before Now right up to but not including midnight of Now.
where arrival_timestamp >= trunc( now ) - 1 day
and arrival_timestamp < trunc( now );
Translating the expression above to English is: "where the value is on or after midnight yesterday morning and any time up to but not including midnight last nite." In other words, "yesterday." Performing the comparison this way, you don't have to worry about how close to midnight the system clock can generate a timestamp. As soon as the time clicks to midnight last nite, no matter how large or extremely small that click may be, the comparison yields a valid result.
Notice, no manipulation is performed on arrival_timestamp, not even to convert it to a more convenient data type. The expression trunc( now ) (including the one where a day is subtracted) is a constant. The functions are called once and the result used over and over for every row. This is why all such functions must be deterministic. Even functions that are NOT in fact deterministic, like Now, are deemed to be deterministic for the life of the query.
For that reason you can do pretty much whatever you need to do on the right side of the comparison. For example, because SQL Server doesn't have a trunc function, the first line written above would look like this:
where arrival_timestamp >= DateAdd( day, -1, DateAdd( day, DateDiff( day, 0, GetDate()), 0))
It doesn't matter. The calculation is performed once and the result used for the entire duration of the query.
So the where clause can now be seen as
where field >= constant1
and field < constant2;
This means if field is indexed, that index can now be used.

you can use the DATE_PART function
select * from message where DATE_PART('day', now() - arrival_timestamp) <= 1

Related

Query with CASE WHEN / LAST_DAY is not giving any output

Running below query and not getting the output. Can someone please tell whats wrong in it?
Select distinct (table.datex)
from table
where table.datex =
(
CASE when extract( day from sysdate) >=19
then last_day(add_months(sysdate, -1))
else last_day(add_months(sysdate, -2))
END
)
Sample data
Datex
ID
30-JUN-21
A
31-MAY-21
B
29-JUN-21
C
Expected result
Datex
30-JUN-21
When I am passing the value hard-coded(calculated by the case) to where clause it's working fine, but when I apply the case it's not working. No error. No output is coming.
Date or datetime?
Oracle's LAST_DAY doesn't do what the name suggests, and the docs (https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/LAST_DAY.html#GUID-296C7C02-7FB9-4AAC-8927-6A79320CE0C6) fail to explain that, too.
Unlike several other DBMS Oracle doesn't have a date type. It only has a datetime type and they even call that inappropriately DATE. This means that a "date" in Oracle always has a time part. A date with its time part set to 00:00:00 can be considered a day's midnight (i.e. the very beginning of the day) or the whole day.
The function SYSDATE gives us a date in the sense of the DATE datatype, not in the sense of a real day, i.e. it gives us the datetime of "now", e.g. 2021-07-20 14:38:00. ADD_MONTHS changes the month in that datetime (and sometimes the year and sometimes even the day), i.e. leaves the time part untouched. LAST_DAY, too, changes the date part to get to the last day of the month, but leaves the time part untouched.
Your CASE expression hence results in something like TIMESTAMP '2021-07-20 14:38:00' and not in DATE '2021-07-20' as one might expect.
You say that you tried your query with the date you computed wth your case expression, and it worked. Did you compute the resulting day in your head or with a query? If the latter: The tool you are using may be set to only display a datetime's date part and omit the time part. This would explain why you only saw 30-JUN-21 when checking the CASE expression.
Solution
Truncate the datetime down to a whole day
Select distinct datex
from mytable
where (extract(day from sysdate) >=19 and datex = trunc(last_day(add_months(sysdate, -1))))
or (extract(day from sysdate) < 19 and datex = trunc(last_day(add_months(sysdate, -2))))
It doesn't matter whether you apply TRUNC late as in my example or right away on SYSDATE (with TRUNC(SYSDATE)) by the way. The only aim is to get rid of the time part at some point in the expression.
Don't use case in where clauses. Boolean logic can handle that.
And take a look if it is really the condition you want
Select distinct datex
from your_table
where
(
extract(day from sysdate) >=19
and datex = last_day(add_months(sysdate,-1))
)
or
(
extract(day from sysdate) < 19
and datex = last_day(add_months(sysdate,-2))
)

SQL - how to change the format of a current_timestamp to have 'mm ss' as zeros?

I want to check if a metric is still missing 4 hours later and return a single record if it exists. I wrote a query that checks if there were metrics in the last 4 hours. But I need to check if there is a metric for a certain hour that was expected to load 4 hours before.
-- Returns records that appeared within the last 4 hours
select * from main.basic_metrics
where metric_name = 'common_metric'
and transaction_time > current_timestamp - interval 4 hours
The problem is that transaction_timeis in the following format 2019-10-30T12:00:00.000+0000 where mm ss are always zeros. So when I check it like transaction_time = current_timestamp - interval 4 hours it returns nothing since current_timestamp contains mm ss data.
How should I format timestamp to the format similar to transaction_time - 2019-10-30T12:00:00.000+0000 ?
UPD: There was a typo, mentioned in the comments below. fixed it
That should be very simple: cast the string to timestamp with time zone:
WHERE CAST(transaction_time AS timestamp with time zone)
> current_timestamp - INTERVAL '4 hours'
Try the following:
select * from main.basic_metrics
where metric_name = 'common_metric'
and transaction_time = date_trunc('hour',current_timestamp) - interval 4 hours
This is not necessarily the best query for what you're doing, but it does solve the problem you're having. My guess is that some version of "between" or > and < would solve it, however without knowing exactly how the "transaction time" is populated, I'm could only venture guesses.
The trick in my example is to "truncate" everything after the "hours" off of the current_timestamp using date_trunc()
Note: It helps a lot to realize that timestamps are NOT formatted. Timestamps are a single long integer field that happens to get formatted on your screen so you can make sense of it. Text comparisons are nearly always the wrong way to do things, and datetime aware functions are the preferred method of doing any comparison.

google bigquery select from a timestamp column between now and n days ago

I have a dataset in bigquery with a TIMESTAMP column "register_date" (sample value "2017-11-19 22:45:05.000 UTC" ).
I need to filter records based on x days or weeks before today criteria.
Example query
select all records which are 2 weeks old.
Currently I have this query (which I feel like a kind of hack) that works and returns the correct results
SELECT * FROM `my-pj.my_dataset.sample_table`
WHERE
(SELECT
CAST(DATE(register_date) AS DATE)) BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -150 DAY)
AND CURRENT_DATE()
LIMIT 10
My question is do I have to use all that CASTing stuff on a TIMESTAMP column (which seems like over complicating the otherwise simple query)?
If I remove the CASting part, my query doesn't run and returns error.
Here is my simplified query
SELECT
*
FROM
`my-pj.my_dataset.sample_table`
WHERE
register_date BETWEEN DATE_ADD(CURRENT_DATE(), INTERVAL -150 DAY)
AND CURRENT_DATE()
LIMIT
10
that results into an error
Query Failed
Error: No matching signature for operator BETWEEN for argument types: TIMESTAMP, DATE, DATE. Supported signature: (ANY) BETWEEN (ANY) AND (ANY) at [6:17]
any insight is highly appreciated.
Use timestamp functions:
SELECT t.*
FROM `my-pj.my_dataset.sample_table` t
WHERE register_date BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -150 DAY) AND CURRENT_TIMESTAMP()
LIMIT 10;
BigQuery has three data types for date/time values: date, datetime, and timestamp. These are not mutually interchangeable. The basic idea is:
Dates have no time component and no timezone.
Datetimes have a time component and no timezone.
Timestamp has both a time component and a timezone. In fact, it represents the value in UTC.
INTERVAL values are defined in gcp documentation
Conversion between the different values is not automatic. Your error message suggests that register_date is really stored as a Timestamp.
One caveat (from personal experience): the definition of day is based on UTC. This is not much of an issue if you are in London. It can be a bigger issue if you are in another time zone and you want the definition of "day" to be based on the local time zone. If that is an issue for you, ask another question.

Date not valid for month specified

I have a problem when running this Oracle SQL statement:
SELECT *
FROM tbl_content
WHERE last_updated >= (systimestamp - INTERVAL '1' month(1))
ORDER BY last_updated desc
And this error:
java.sql.SQLException: ORA-01839: date not valid for month specified
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:111)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:330)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:287)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:742)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:212)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:951)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1053)
at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:835)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1123)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3284)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3328)
at com.vtdd.sms.content.xskt.XsktService.getKQXSFollowArea(XsktService.java:4044)
at com.vtdd.sms.content.xskt.XsktService.getMessages(XsktService.java:421)
at com.vht.sms.content.ContentAbstract.getSubmitMessages(ContentAbstract.java:47)
at com.vht.sms.content.ContentFactory.getSubmitMessages(ContentFactory.java:335)
at com.vht.sms.content.ContentFactory.run(ContentFactory.java:62)
at java.lang.Thread.run(Thread.java:662)
Could you tell me what is wrong?
Firstly, why are you using systimestamp? If you want this to the month then surely sysdate is exact enough? Secondly, I like - i.e. it's personal preference - to make it extremely clear what's happening. Oracle has an add_months function, which will do what you want. So your query could easily be:
SELECT *
FROM tbl_content
WHERE last_updated >= add_months(sysdate, -1)
ORDER BY last_updated desc
What is actually wrong is that interval arithmetic doesn't adjust days - see the 6th bullet in the link:
When interval calculations return a datetime value, the result must be an actual datetime value or the database returns an error.
ADD_MONTHS does; as that link says:
If date is the last day of the month or if the resulting month has fewer days than the day component of date, then the result is the last day of the resulting month.
So, ADD_MONTHS(DATE '2011-12-31', -1) gives you 2011-11-30, while DATE '2011-12-31' - INTERVAL '1' MONTH tries to give you 2011-11-31, which as the message says, isn't a valid date.
(It's debatable if this behaviour is actually wrong; it's unexpected, but I believe it's conforming to ANSI. There may be times you want it to work this way, though I can't think of any...)

Timestamp Difference In Hours for PostgreSQL

Is there a TIMESTAMPDIFF() equivalent for PostgreSQL?
I know I can subtract two timestamps to get a postgresql INTERVAL. I just want the difference between the two timestamps in in hours represented by an INT.
I can do this in MySQL like this:
TIMESTAMPDIFF(HOUR, links.created, NOW())
I just need the difference between two timestamps in hours represented as an integer.
Solution works for me:
SELECT "links_link"."created",
"links_link"."title",
(EXTRACT(EPOCH FROM current_timestamp - "links_link"."created")/3600)::Integer AS "age"
FROM "links_link"
The first things popping up
EXTRACT(EPOCH FROM current_timestamp-somedate)/3600
May not be pretty, but unblocks the road. Could be prettier if division of interval by interval was defined.
Edit: if you want it greater than zero either use abs or greatest(...,0). Whichever suits your intention.
Edit++: the reason why I didn't use age is that age with a single argument, to quote the documentation: Subtract from current_date (at midnight). Meaning you don't get an accurate "age" unless running at midnight. Right now it's almost 1am here:
select age(current_timestamp);
age
------------------
-00:52:40.826309
(1 row)
Get fields where a timestamp is greater than date in postgresql:
SELECT * from yourtable
WHERE your_timestamp_field > to_date('05 Dec 2000', 'DD Mon YYYY');
Subtract minutes from timestamp in postgresql:
SELECT * from yourtable
WHERE your_timestamp_field > current_timestamp - interval '5 minutes'
Subtract hours from timestamp in postgresql:
SELECT * from yourtable
WHERE your_timestamp_field > current_timestamp - interval '5 hours'
Michael Krelin's answer is close is not entirely safe, since it can be wrong in rare situations. The problem is that intervals in PostgreSQL do not have context with regards to things like daylight savings. Intervals store things internally as months, days, and seconds. Months aren't an issue in this case since subtracting two timestamps just use days and seconds but 'days' can be a problem.
If your subtraction involves daylight savings change-overs, a particular day might be considered 23 or 25 hours respectively. The interval will take that into account, which is useful for knowing the amount of days that passed in the symbolic sense but it would give an incorrect number of the actual hours that passed. Epoch on the interval will just multiply all days by 24 hours.
For example, if a full 'short' day passes and an additional hour of the next day, the interval will be recorded as one day and one hour. Which converted to epoch/3600 is 25 hours. But in reality 23 hours + 1 hour should be a total of 24 hours.
So the safer method is:
(EXTRACT(EPOCH FROM current_timestamp) - EXTRACT(EPOCH FROM somedate))/3600
As Michael mentioned in his follow-up comment, you'll also probably want to use floor() or round() to get the result as an integer value.
You can use the "extract" or "date_part" functions on intervals as well as timestamps, but I don't think that does what you want. For example, it gives 3 for an interval of '2 days, 3 hours'. However, you can convert an interval to a number of seconds by specifying 'epoch' as the time element you want: extract(epoch from '2 days, 3 hours'::interval) returns 183600 (which you then divide by 3600 to convert seconds to hours).
So, putting this all together, you get basically Michael's answer: extract(epoch from timestamp1 - timestamp2)/3600. Since you don't seem to care about which timestamp precedes which, you probably want to wrap that in abs:
SELECT abs(extract(epoch from timestamp1 - timestamp2)/3600)
postgresql get seconds difference between timestamps
SELECT (
(extract (epoch from (
'2012-01-01 18:25:00'::timestamp - '2012-01-01 18:25:02'::timestamp
)
)
)
)::integer
which prints:
-2
Because the timestamps are two seconds apart. Take the number and divide by 60 to get minutes, divide by 60 again to get hours.
extract(hour from age(now(),links.created)) gives you a floor-rounded count of the hour difference.
To avoid the epoch conversion you could extract the days multiply them by 24 and add the extraction of hours to it.
select current_timestamp, (current_timestamp - interval '500' hour), (extract(day from (current_timestamp - (current_timestamp - interval '500' hour)) * 24) + extract(hour from (current_timestamp - (current_timestamp - interval '500' hour))));
For MySQL timestampdiff I don't know, but for MSSQL datediff(hour, start, end) the best equivalent in PostgreSQL is floor(extract(epoch from end - start)/3600), because in MSSQL select datediff(hour,'2021-10-31 18:00:00.000', '2021-10-31 18:59:59.999') return 0
This might sound crazy to a lot of developers who like to take advantage of database functions,
But after exhaustive problems thinking, creating and bugfixing applications for mysql and postgrsql with php comparing date functions, I've come to the conclusion (for myself), that the easiest way, that is the simplest with less SQL headaches is not to take advantage of any of them.
Why? because if you are developing in a middleware language like PHP, PHP has all of these functions, and they are easier to implement in the application ode as comparing integers. PostgreSQL timestamp is NOT == UNIX TIMESTAMP and MySQL's UNIX TIMESTAMP is NOT PostgresQL's or Oracles timestamp.. it gets harder to port if you use database timestamps..
so just use an integer, not a timestamp,
as the number of seconds since january 1st 1970 midnight. and never mind database timestamps.
, and use gmdate() and store everything as gmt time to avoid timezone issues.
if you need to search, sort or compare the day from other data, or the month or the year or the day of the week, or anything, in your application,
and INTEGER datatype for time_day, time_hour, time_seconds.. or whatever you wnat to index to be searched will make for smoother and more portable databases.
you can just use one field, in most instances: INTEGER time_created NOT NULL
(more fields in your database row is the only drawback to this solution that i have found, and that doesnt cause as many headaches, or cups of coffee :)
php's date functions are outstanding to compare dates,
but in mysql or postgresql, comparing dates ? nah.. use integer sql comparisons
i realize it may SEEM easier to use CURRENT_TIMESTAMP on an insert function. HA!
don't be fooled.
You cant do DELETE FROM SESSION_TABLE WHERE time-initialized < '2 days'
if time-intitialized is a postgresql timestamp.
but you CAN do:
DELETE FROM SESSION_TABLE WHERE time_initialized < '$yesterday'
As long as you set $yesterday in php as the integer of seconds since 1970 that yesterday was.
This is easier housekeeping of session records than comparing timestamps in postgresql select statements.
SELECT age(), SELECT extract(), and asbtime are headaches in an of themselves. this is just my opinion.
you can do addition, substraction, <, >, all with php date objects
_peter_sysko
U4EA Networks, Inc.