I am trying to subtract departure time from arrival time to get the duration of trips.
The arrival/departure is in the format of HH:MM.
I'm using postgresql, the time columns just say 'string' for how they are defined. A friend helped me cast them as integers, but I'm still stick at this point:
SELECT x, cast (table1.arrival as integer) - cast (table.1departure as integer), sum(y), count(*)
FROM table1 INNER JOIN table2
ON .....
GROUP BY .....
ORDER BY....
A friend told me this but I'm not totally sure what he means: "No you need to create a new variable for just minutes. 1:30 = 90 minutes. Cast that as integer then compute." How do I do this?
If the column is indeed a varchar (or text) then you need to first cast the column to a proper time.
SELECT x, table1.arrival::time - table.1departure::time, sum(y), count(*)
FROM table1 INNER JOIN table2
ON .....
GROUP BY .....
ORDER BY.
The above will fail if the time stored in the columns is not a properly formatted time in 24 hour format (e.g. 22:40)
The result of time - time is an interval. If you want to get the result in minutes, you need to extract them from the interval:
extract(epoch from table1.arrival::time - table.1departure::time) / 60
extract(epoch ..) returns the seconds in the interval that's why you need to divide it by 60 to get minutes.
This will however not work properly if the arrival and departure time are not on the same day (e.g. departure is at 23:25 and arrival is the next day at 07:23)
Unrelated but: you should never store dates, times or timestamps in varchar or text columns. Always use the proper data type.
If your RDBMS is Mysql you can use the MySQL DATEDIFF() Function
For example :
SELECT DATEDIFF(arrivale,departure) AS DiffDate
I assume here that the arrivale and the departure are in the same day.
And that arrivale and departue are TimeDate formated like HH:MM.
Related
I am trying to calculate the difference between two dates in an oracle database using a JDBC connection. I followed the advice from this question using a query like this:
SELECT CREATE_DATE - CLOSED
FROM TRANSACTIONS;
and I get the following error:
Incompatible value type specified for
column:CREATE_DATE-CLOSED. Column Type = 11 and Value Type =
8.[10176] Error Code: 10176
What should I change so I can successfully calculate the difference between the dates?
note: CREATE_DATE and CLOSED both have TIMESTAMP type
The answer you found is related to date datatypes, but you are dealing with timestamps. While substracting two Oracle dates returns a number, substracting timestamps produces an interval datatype. This is probably not what you want, and, apparently, your driver does not properly handle this datatype.
For this use case one solution is to cast the timestamps to dates before substracting them:
select cast(create_date as date) - cast(closed as date) from transactions;
As it was mentioned, it seems that JDBC cannot work with the INTERVAL datatype. What about casting it with the EXTRACT function to the expected output as number? If you want number of seconds between those two timestamps, it would be:
SELECT EXTRACT(SECOND FROM (CREATE_DATE - CLOSED)) FROM TRANSACTIONS;
Here are list of options which might be used instead of SECOND:
https://docs.oracle.com/database/121/SQLRF/functions067.htm#SQLRF00639
When we subtract one date from another Oracle gives us the difference as a number: it's straightforward arithmetic. But when we subtract one timestamp from another - which is what you're doing - the result is an INTERVAL. Older versions of JDBC don't like the INTERVAL datatype (docs) .
Here are a couple of workarounds, depending on what you want to do with the result. The first is to calculate the number of seconds from the interval result. extract second from ... only gives us the numbers of seconds in the interval. This will be fine providing none of your intervals are more than fifty-nine seconds long. Longer intervals require us to extract minute, hour and even days. So that solution would be:
select t.*
, extract (day from (t.closed - t.create_date)) * 84600
+ extract (hour from (t.closed - t.create_date)) * 3600
+ extract (minute from (t.closed - t.create_date)) * 60
+ extract (second from (t.closed - t.create_date)) as no_of_secs
from transactions t
A second solution is to follow the advice in the JDBC mapping guide and turn the interval into a string:
select t.*
, cast ((t.closed - t.create_date) as varchar2(128 char)) as intrvl_str
from transactions t
The format of a string interval is verbose:INTERVAL'+000000001 04:40:59.710000'DAY(9)TO SECOND. This may not be useful in the Java side of the application. But with regex we can turn it into a string which can be converted into a Java 8 Duration object (docs) : PnDTnHnMn.nS.
select t.id
, regexp_replace(cast ((t.closed - t.create_date) as varchar2(128 char))
, 'INTERVAL''\+([0-9]+) ([0-9]{2}):([0-9]{2}):([0-9]{2})\.([0-9]+)''DAY\(9\)TO SECOND'
, 'P\1DT\2H\3M\4.\5S')
as duration
from transactions t
There is a demo on db<>fiddle
I am trying to write a query that will let me get records entered between now and a some number of minutes in the past. I have the number of minutes as milliseconds. So, I'm trying to write a SQL query that will let me get those records.
I see a DateAdd function. However, I don't see a function to Subtract some time from a date. From a pseduo-code perspective, I'm trying to do this:
SELECT
*
FROM
MyTable
WHERE
CreatedAt > (GetUtcDate() - milliseconds)
How do I get records from MyTable that have happened within some past window of time?
We can try using DATEADD in millisecond mode:
SELECT *
FROM MyTable
WHERE CreatedAt > DATEADD(ms, -60000, GETDATE()); -- 60000 ms = 1 minute
This would return all records which were created within one minute of the current date and time.
The function you want is DATEADD (Transact-SQL).
SELECT {YourColumns}
FROM MyTable MT
WHERE CreatedAt > DATEADD(millisecond,{value},GETUTCDATE());
Despite its name, the DATEADD function is also the "DATESUBTRACT" function that you're looking for. (Intentionally double-quoted as quasi-code.)
The first parameter defines the sorts of units you're adding or subtracting, i.e. HOUR, DAY, WEEK, etc., or, in your case, MILLISECOND. In the second parameter, if you want to add time, you enter a positive number. If you want to subtract time, as you do here, enter a negative number of units.
The best way to interact with date and time in SQL is to use specific date and time functions rather than adding dates as you would add numbers.
In this case the function you are looking for is DATEADD, using it your cord should change like this:
SELECT *
FROM MyTable
WHERE CreatedAt > DATEADD(ms, -milliseconds, GetUtcDate())
I have a table with two timestamp columns, startTime and stopTime, and I would like to calculate the average difference of these timestamps in my table. I have a solution that works in Postgres and in HSQLDB (which I use for local unit testing) but not both, and I'm having trouble trying to figure out a common solution.
Postgres:
select EXTRACT(EPOCH FROM(avg(m.stopTime - m.startTime))) from Measures m
HSQL:
select avg(FUNC('UNIX_TIMESTAMP', m.stopTime) - FUNC('UNIX_TIMESTAMP', m.startTime) from Measures m
Is there a way to use the same query for both databases? All of the functions I've found seem to only be supported in one database or the other.
I think my main problem is that there isn't a common function to convert a timestamp to seconds in order to perform the calculation. EPOCH is only compatible with Postgres and UNIX_TIMESTAMP is only compatible with HSQL.
The crux of your problem is converting the dates and timestamps down to a number of seconds. Instead of using epoch, I'll use a julian date for the date. I'll convert the julian date to seconds, then add the number of seconds since minight for each timestamp being compared. The following query does not calculate the difference, it simply converts the date to a number that's similar on both platforms .. you'll have to do this once for each date being compared. note: replace "current"timestamp" with m.startTime and m.stopTime respectively.
select
(to_number(to_char(current_timestamp,'J'),'99999999999999999999')*86400/*convert from julian days to julian seconds*/)
+ (to_number(to_char(current_timestamp,'HH'),'99') * 3600) /*Hours to seconds */
+ (to_number(to_char(current_timestamp,'MM'),'99') * 60) /*Minutes to seconds */
+ (to_number(to_char(current_timestamp,'SS'),'99') /*add in the seconds*/
Ugly as sin, I know-- perhaps you can rewrite it easier as function, but as I don't know hsqls full feature set, I'll leave it in this form rather than using a CTE or function.
I'm working with Hive and I have a table structured as follows:
CREATE TABLE t1 (
id INT,
created TIMESTAMP,
some_value BIGINT
);
I need to find every row in t1 that is less than 180 days old. The following query yields no rows even though there is data present in the table that matches the search predicate.
select *
from t1
where created > date_sub(from_unixtime(unix_timestamp()), 180);
What is the appropriate way to perform a date comparison in Hive?
How about:
where unix_timestamp() - created < 180 * 24 * 60 * 60
Date math is usually simplest if you can just do it with the actual timestamp values.
Or do you want it to only cut off on whole days? Then I think the problem is with how you are converting back and forth between ints and strings. Try:
where created > unix_timestamp(date_sub(from_unixtime(unix_timestamp(),'yyyy-MM-dd'),180),'yyyy-MM-dd')
Walking through each UDF:
unix_timestamp() returns an int: current time in seconds since epoch
from_unixtime(,'yyyy-MM-dd') converts to a string of the given format, e.g. '2012-12-28'
date_sub(,180) subtracts 180 days from that string, and returns a new string in the same format.
unix_timestamp(,'yyyy-MM-dd') converts that string back to an int
If that's all getting too hairy, you can always write a UDF to do it yourself.
Alternatively you may also use datediff. Then the where clause would be
in case of String timestamp (jdbc format) :
datediff(from_unixtime(unix_timestamp()), created) < 180;
in case of Unix epoch time:
datediff(from_unixtime(unix_timestamp()), from_unixtime(created)) < 180;
I think maybe it's a Hive bug dealing with the timestamp type. I've been trying to use it recently and getting incorrect results.
If I change your schema to use a string instead of timestamp, and supply values in the
yyyy-MM-dd HH:mm:ss
format, then the select query worked for me.
According to the documentation, Hive should be able to convert a BIGINT representing epoch seconds to a timestamp, and that all existing datetime UDFs work with the timestamp data type.
with this simple query:
select from_unixtime(unix_timestamp()), cast(unix_timestamp() as
timestamp) from test_tt limit 1;
I would expect both fields to be the same, but I get:
2012-12-29 00:47:43 1970-01-16 16:52:22.063
I'm seeing other weirdness as well.
TIMESTAMP is milliseconds
unix_timestamp is in seconds
You need to multiply the RHS by 1000.
where created > 1000 * date_sub(from_unixtime(unix_timestamp()), 180);
After reviewing this and referring to Date Difference less than 15 minutes in Hive I came up with a solution. While I'm not sure why Hive doesn't perform the comparison effectively on dates as strings (they should sort and compare lexicographically), the following solution works:
FROM (
SELECT id, value,
unix_timestamp(created) c_ts,
unix_timestamp(date_sub(from_unixtime(unix_timestamp()), 180), 'yyyy-MM-dd') c180_ts
FROM t1
) x
JOIN t1 t ON x.id = t.id
SELECT to_date(t.Created),
x.id, AVG(COALESCE(x.HighestPrice, 0)), AVG(COALESCE(x.LowestPrice, 0))
WHERE unix_timestamp(t.Created) > x.c180_ts
GROUP BY to_date(t.Created), x.id ;
I'm trying to get the amount of time spent on something. I have the DateTime for the start and end of the thing in two columns. I can do:
cnext.CreatedDate - c.CreatedDate
which results in something like this:
1900-01-01 00:00:19.190
which means they spent 19.19 seconds on the thing. That's great for each row but i would like to group by the specific thing and sum up the time spent on each thing:
SUM(cnext.CreatedDate - c.CreatedDate)
but I get a:
Operand data type datetime is invalid for sum operator.
How should i go about getting the total time is I can't sum up DateTime types?
You can't SUM datetime values. This is like taking the square root of a string.
Try:
SUM(DATEDIFF(SECOND, c.CreatedDate, cnext.CreatedDate))
Or if you want greater precision:
SUM(DATEDIFF(MILLISECOND, c.CreatedDate, cnext.CreatedDate))
If the date potion is 1/1/1900, convert the dates to floats, subtract them, and convert the result back to datetime.