I didn't find any simple answer to this while I was looking around, so I thought I'd put it up here in case anyone was having the same problem as me with what should have been a trivial issue.
I was using ReDash analytics with Google's BigQuery and had turned on Standard SQL in the datasource settings. For the purposes of my query, I needed to convert a timestamp - unix time in milliseconds, as a string - to a Date format so that I could use the DATE_DIFF method.
As an example... "1494865480000" to "2017-05-15"
The difficulty was that casting and conversion was excessively strict and there seemed no adequate way to make it parse. See my answer down below!
(Though let me know if some SQL sensei knows a more eloquent way!)
In Standard SQL use TIMESTAMP_MICROS function together with EXTRACT(DATE FROM <timestamp>):
SELECT EXTRACT(DATE FROM TIMESTAMP_MILLIS(1494865480000))
A simpler way with TIMESTAMP_MILLIS():
#standardSQL
SELECT DATE(TIMESTAMP_MILLIS(CAST("1494865480000" AS INT64)))
2017-05-15
After much trial and error, this was my solution:
DATE_ADD( DATE'1970-01-01', INTERVAL CAST( ( CAST( epochTimestamp AS INT64 ) / 86400000 ) AS INT64 ) DAY ) AS convertedDate
That is, I took the string, cast it to an integer, divided it by the number of milliseconds in a day, then used a DATE_ADD method, and added the result to the start of Epoch time, and calculated the resulting day.
I hope this saves another junior some time!
Use UTC_USEC_TO_TIMESTAMP():
select UTC_USEC_TO_TIMESTAMP(postedon * 1000)
You can then extract the date using date():
select DATE(UTC_USEC_TO_TIMESTAMP(postedon * 1000))
This doesn't require knowing the internal format of Unix timestamps.
Related
I am trying to calculate the difference between two dates in an oracle database using a JDBC connection. I followed the advice from this question using a query like this:
SELECT CREATE_DATE - CLOSED
FROM TRANSACTIONS;
and I get the following error:
Incompatible value type specified for
column:CREATE_DATE-CLOSED. Column Type = 11 and Value Type =
8.[10176] Error Code: 10176
What should I change so I can successfully calculate the difference between the dates?
note: CREATE_DATE and CLOSED both have TIMESTAMP type
The answer you found is related to date datatypes, but you are dealing with timestamps. While substracting two Oracle dates returns a number, substracting timestamps produces an interval datatype. This is probably not what you want, and, apparently, your driver does not properly handle this datatype.
For this use case one solution is to cast the timestamps to dates before substracting them:
select cast(create_date as date) - cast(closed as date) from transactions;
As it was mentioned, it seems that JDBC cannot work with the INTERVAL datatype. What about casting it with the EXTRACT function to the expected output as number? If you want number of seconds between those two timestamps, it would be:
SELECT EXTRACT(SECOND FROM (CREATE_DATE - CLOSED)) FROM TRANSACTIONS;
Here are list of options which might be used instead of SECOND:
https://docs.oracle.com/database/121/SQLRF/functions067.htm#SQLRF00639
When we subtract one date from another Oracle gives us the difference as a number: it's straightforward arithmetic. But when we subtract one timestamp from another - which is what you're doing - the result is an INTERVAL. Older versions of JDBC don't like the INTERVAL datatype (docs) .
Here are a couple of workarounds, depending on what you want to do with the result. The first is to calculate the number of seconds from the interval result. extract second from ... only gives us the numbers of seconds in the interval. This will be fine providing none of your intervals are more than fifty-nine seconds long. Longer intervals require us to extract minute, hour and even days. So that solution would be:
select t.*
, extract (day from (t.closed - t.create_date)) * 84600
+ extract (hour from (t.closed - t.create_date)) * 3600
+ extract (minute from (t.closed - t.create_date)) * 60
+ extract (second from (t.closed - t.create_date)) as no_of_secs
from transactions t
A second solution is to follow the advice in the JDBC mapping guide and turn the interval into a string:
select t.*
, cast ((t.closed - t.create_date) as varchar2(128 char)) as intrvl_str
from transactions t
The format of a string interval is verbose:INTERVAL'+000000001 04:40:59.710000'DAY(9)TO SECOND. This may not be useful in the Java side of the application. But with regex we can turn it into a string which can be converted into a Java 8 Duration object (docs) : PnDTnHnMn.nS.
select t.id
, regexp_replace(cast ((t.closed - t.create_date) as varchar2(128 char))
, 'INTERVAL''\+([0-9]+) ([0-9]{2}):([0-9]{2}):([0-9]{2})\.([0-9]+)''DAY\(9\)TO SECOND'
, 'P\1DT\2H\3M\4.\5S')
as duration
from transactions t
There is a demo on db<>fiddle
I can't understand how to convert this type of real - 42389.520752314813 to timestamp.
I got this data from one source, but I need to convert it to normal timestamp format.
I think you have received wrong input data.
this type of timestamp is only occurring when Destination tool is excel and which has CELL as "Number" type, and during Copy-Paste of timestamp, destination field has Calculated it as Mathematical function. Please re-verify your source. I am sure about this mathematical calculation. please check the below sample of such data. So practically you cannot perform its reverse operation.
Hard to do without knowing what timestamp should be returned.
If the date is sourced from a SQL Server it might be '2016-01-22 12:29:53':
cast( date '1900-01-01' + myCol as timestamp(3))
+ (cast(86400 * (myCol mod 1) as dec(12,6)) * interval '00:00:01.000000' hour to second)
If it's from Excel it's two days earlier and you must start at '1899-12-30'
I have a table with two timestamp columns, startTime and stopTime, and I would like to calculate the average difference of these timestamps in my table. I have a solution that works in Postgres and in HSQLDB (which I use for local unit testing) but not both, and I'm having trouble trying to figure out a common solution.
Postgres:
select EXTRACT(EPOCH FROM(avg(m.stopTime - m.startTime))) from Measures m
HSQL:
select avg(FUNC('UNIX_TIMESTAMP', m.stopTime) - FUNC('UNIX_TIMESTAMP', m.startTime) from Measures m
Is there a way to use the same query for both databases? All of the functions I've found seem to only be supported in one database or the other.
I think my main problem is that there isn't a common function to convert a timestamp to seconds in order to perform the calculation. EPOCH is only compatible with Postgres and UNIX_TIMESTAMP is only compatible with HSQL.
The crux of your problem is converting the dates and timestamps down to a number of seconds. Instead of using epoch, I'll use a julian date for the date. I'll convert the julian date to seconds, then add the number of seconds since minight for each timestamp being compared. The following query does not calculate the difference, it simply converts the date to a number that's similar on both platforms .. you'll have to do this once for each date being compared. note: replace "current"timestamp" with m.startTime and m.stopTime respectively.
select
(to_number(to_char(current_timestamp,'J'),'99999999999999999999')*86400/*convert from julian days to julian seconds*/)
+ (to_number(to_char(current_timestamp,'HH'),'99') * 3600) /*Hours to seconds */
+ (to_number(to_char(current_timestamp,'MM'),'99') * 60) /*Minutes to seconds */
+ (to_number(to_char(current_timestamp,'SS'),'99') /*add in the seconds*/
Ugly as sin, I know-- perhaps you can rewrite it easier as function, but as I don't know hsqls full feature set, I'll leave it in this form rather than using a CTE or function.
I can't seem to get reliable results from the query against a sqlite database using a datetime string as a comparison as so:
select *
from table_1
where mydate >= '1/1/2009' and mydate <= '5/5/2009'
how should I handle datetime comparisons to sqlite?
update:
field mydate is a DateTime datatype
To solve this problem, I store dates as YYYYMMDD. Thus,
where mydate >= '20090101' and mydate <= '20050505'
It just plain WORKS all the time. You may only need to write a parser to handle how users might enter their dates so you can convert them to YYYYMMDD.
SQLite doesn't have dedicated datetime types, but does have a few datetime functions. Follow the string representation formats (actually only formats 1-10) understood by those functions (storing the value as a string) and then you can use them, plus lexicographical comparison on the strings will match datetime comparison (as long as you don't try to compare dates to times or datetimes to times, which doesn't make a whole lot of sense anyway).
Depending on which language you use, you can even get automatic conversion. (Which doesn't apply to comparisons in SQL statements like the example, but will make your life easier.)
I had the same issue recently, and I solved it like this:
SELECT * FROM table WHERE
strftime('%s', date) BETWEEN strftime('%s', start_date) AND strftime('%s', end_date)
The following is working fine for me using SQLite:
SELECT *
FROM ingresosgastos
WHERE fecharegistro BETWEEN "2010-01-01" AND "2013-01-01"
Following worked for me.
SELECT *
FROM table_log
WHERE DATE(start_time) <= '2017-01-09' AND DATE(start_time) >= '2016-12-21'
Sqlite can not compare on dates. we need to convert into seconds and cast it as integer.
Example
SELECT * FROM Table
WHERE
CAST(strftime('%s', date_field) AS integer) <=CAST(strftime('%s', '2015-01-01') AS integer) ;
I have a situation where I want data from up to two days ago and up until the end of today.
I arrived at the following.
WHERE dateTimeRecorded between date('now', 'start of day','-2 days')
and date('now', 'start of day', '+1 day')
Ok, technically I also pull in midnight on tomorrow like the original poster, if there was any data, but my data is all historical.
The key thing to remember, the initial poster excluded all data after 2009-11-15 00:00:00. So, any data that was recorded at midnight on the 15th was included but any data after midnight on the 15th was not.
If their query was,
select *
from table_1
where mydate between Datetime('2009-11-13 00:00:00')
and Datetime('2009-11-15 23:59:59')
Use of the between clause for clarity.
It would have been slightly better. It still does not take into account leap seconds in which an hour can actually have more than 60 seconds, but good enough for discussions here :)
I had to store the time with the time-zone information in it, and was able to get queries working with the following format:
"SELECT * FROM events WHERE datetime(date_added) BETWEEN
datetime('2015-03-06 20:11:00 -04:00') AND datetime('2015-03-06 20:13:00 -04:00')"
The time is stored in the database as regular TEXT in the following format:
2015-03-06 20:12:15 -04:00
Right now i am developing using System.Data.SQlite NuGet package (version 1.0.109.2). Which using SQLite version 3.24.0.
And this works for me.
SELECT * FROM tables WHERE datetime
BETWEEN '2018-10-01 00:00:00' AND '2018-10-10 23:59:59';
I don't need to use the datetime() function. Perhaps they already updated the SQL query on that SQLite version.
Below are the methods to compare the dates but before that we need to identify the format of date stored in DB
I have dates stored in MM/DD/YYYY HH:MM format so it has to be compared in that format
Below query compares the convert the date into MM/DD/YYY format and get data from last five days till today. BETWEEN operator will help and you can simply specify start date AND end date.
select * from myTable where myColumn BETWEEN strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day') AND strftime('%m/%d/%Y %H:%M',datetime('now','localtime'));
Below query will use greater than operator (>).
select * from myTable where myColumn > strftime('%m/%d/%Y %H:%M', datetime('now','localtime'), '-5 day');
All the computation I have done is using current time, you can change the format and date as per your need.
Hope this will help you
Summved
You could also write up your own user functions to handle dates in the format you choose. SQLite has a fairly simple method for writing your own user functions. For example, I wrote a few to add time durations together.
My query I did as follows:
SELECT COUNT(carSold)
FROM cars_sales_tbl
WHERE date
BETWEEN '2015-04-01' AND '2015-04-30'
AND carType = "Hybrid"
I got the hint by #ifredy's answer. The all I did is, I wanted this query to be run in iOS, using Objective-C. And it works!
Hope someone who does iOS Development, will get use out of this answer too!
Here is a working example in C# in three ways:
string tableName = "TestTable";
var startDate = DateTime.Today.ToString("yyyy-MM-dd 00:00:00"); \\From today midnight
var endDate = date.AddDays(1).ToString("yyyy-MM-dd HH:mm:ss"); \\ Whole day
string way1 /*long way*/ = $"SELECT * FROM {tableName} WHERE strftime(\'%s\', DateTime)
BETWEEN strftime('%s', \'{startDate}\') AND strftime('%s', \'{endDate}\')";
string way2= $"SELECT * FROM {tableName} WHERE DateTime BETWEEN \'{startDate}\' AND \'{endDate}\'";
string way3= $"SELECT * FROM {tableName} WHERE DateTime >= \'{startDate}\' AND DateTime <=\'{endDate}\'";
select *
from table_1
where date(mydate) >= '1/1/2009' and date(mydate) <= '5/5/2009'
This work for me
I have this query, that's been giving me some issues, it looks like this:
UPDATE servicecontracts
SET planned_duration = (to_char(due_date) - to_char(start_date) + 1)
,actual_duration =''
,progress = NULL
WHERE servicecontractsid = '263'
After some research, I managed to figure out what this query is trying to do, it' s just trying to find the planned duration, by subtracting the due date and the start date. Why, this is trying to do that by subtracting strings, I do not know. Also, the to_char function requires a second parameter.
So, anyway, now I need to find the planned_duration, but how do I do that. According to the Postgresql docs, the to_char function doesn't have an option to return an integer, if you set it to return text and then if you try to convert the string into an integer using explicit casts, like ::integer, you get an error because an integer can't have colons in there.
So, is there a way for to_char to return an integer that somehow represents the date, and then subtract the two?
If not, what should I do to carry this out?
I quote from the fine manual here
SELECT EXTRACT(EPOCH FROM TIMESTAMP WITH TIME ZONE '2001-02-16 20:38:40.12-08');
Result: 982384720.12
But for computing intervals, there is simpler way:
SELECT (due_date - start_date)
Just subtracting two date types from each other should return their difference in days.
SET planned_duration = (due_date - start_date)
Not sure why to_char is being used, unless I'm missing something here.