DolphinDB: concatenate date and time - sql

The following two columns are of DATETIME and SECOND data type, respectively.
ActionDay minute
2023.01.31T18:47:20 00:00:05
2023.01.31T18:47:20 00:00:05
2023.01.31T18:47:20 00:00:01
2023.01.31T18:47:20 00:00:01
2023.01.31T18:47:20 00:00:06
Now I want to create a column of TIMESTAMP type based on these two columns. How can I do with SQL statement?

I'm not sure what you actually want because DATETIME type already has second part. It's an example for adding-up DATETIME and SECOND:
select timestamp(ActionDay + minute) from t

Solution 1:
Convert column “minute“ (of SECOND type) to INT type, and add to column “ActionDay“ (of DATETIME type).
select ActionDay + minute.int() from T
Solution 2:
Create a new column by adding column “ActionDay“ and “minute“, and then convert it to DATETIME type. Both datetime() and cast(, datetime)` can be used:
select datetime(ActionDay+minute) from T
select cast(ActionDay+minute, DATETIME) as dt from T
Solution 3:
Use the concatDateTime() function to combine the date of olumn “ActionDay“ with “minute“:
select concatDateTime(date(ActionDay),minute) from T
I test the above three methods on an in-memory table with 10 million records.
n=10000000
T = table(take(2023.01.10T00:00:00 2023.01.10T00:00:00,n) as `ActionDay, take(15:00:00 15:19:00 ,n) as `minute);
timer select cast(ActionDay+minute, DATETIME) as dt from T
timer select concatDateTime(date(ActionDay),minute) from T
timer select ActionDay + minute.int() from T
timer select datetime(ActionDay+minute) from T
The results are as follows:
Time elapsed: 19.946 ms
Time elapsed: 16.956 ms
Time elapsed: 10.971 ms
Time elapsed: 9.973 ms
It can be seen from the result that the solution 1 (9.973 ms) and solution 2 using datetime() (10.971 ms) are relatively quicker.

Related

Converting a 5 digit int into time

In a legacy system (SQL Server 2005) I have a column that stores a 5 digit integer (ie 86340) as time. The legacy application shows 86340 as 23:59:00. I am unsure how how to translate that 5 digit integer into a date-time data type using SQL.
SQL Server 2012+ has TIMEFROMPARTS function:
TIMEFROMPARTS ( hour, minute, seconds, fractions, precision )
Returns a time value for the specified time and with the specified precision.
Which is similiar to Excel's TIME:
TIME(hour, minute, second)
Excel version could handle values over 0-60 range:
Minute Required. A number from 0 to 32767 representing the minute. Any value greater than 59 will be converted to hours and minutes.
And SQL counterpart cannot do that.
It looks like that value is simply number of seconds so you could use:
DECLARE #A1 INT = 86340;
SELECT DATEADD(second, #A1,CAST('00:00:00' AS TIME));
DBFiddle Demo
EDIT:
As SQL Server 2005 does not support TIME data type, you could use DATETIME instead and skip date part in application.
DECLARE #A1 INT = 86340;
SELECT DATEADD(second, #A1,CAST('00:00:00' AS DATETIME));
DBFiddle Demo2
Since the value you have is an integer representation of the seconds since midnight, you have a couple of choices in SQL Server 2005. You can either render the value as a VARCHAR, which is readable, you can render it as DATETIME, which appends the date information, or you can maybe pull in a date from another field to get an meaningful DATETIME for your value.
SELECT CONVERT(VARCHAR(12), DATEADD(SECOND, 86340, 0), 114) AS [InVarchar];
+--------------+
| InVarchar |
+--------------+
| 23:59:00:000 |
+--------------+
SELECT DATEADD(SECOND, 86340, 0) AS [InDatetime];
+-------------------------+
| InDatetime |
+-------------------------+
| 1900-01-01 23:59:00.000 |
+-------------------------+
SELECT DATEADD(SECOND, 86340, CAST('2018-09-05' AS DATETIME)) AS [InDatetimeWithDate];
+-------------------------+
| InDatetimeWithDate |
+-------------------------+
| 2018-09-05 23:59:00.000 |
+-------------------------+
USE below query:
SELECT CONVERT(DATETIME, #Column_Name)
FROM Table;

Date is string between hyphens in SQL Server

I have date formats that are as follows:
Date_str
19-12-2007
31-7-2009
3-1-2010
31-11-2009
etc.
I can't do the following:
CONCAT(RIGHT(Date_str,4),SUBSTRING(Date_str,3,3),LEFT(2))
because as you can see above, the dates are not the same length. Is there a way in SQL Server to extract the date as datetime/date?
I also tried
Convert(datetime, Date_str)
but it just threw an error:
The conversion of a varchar data type to a datetime data type resulted
in an out-of-range value.
If 2012+, I would use Try_Convert(). This will return bogus dates as NULL.
Example
Declare #YourTable Table ([Date_str] varchar(50))
Insert Into #YourTable Values
('19-12-2007')
,('31-7-2009')
,('3-1-2010')
,('31-11-2009')
Select *
,try_convert(date,Date_Str,105)
from #YourTable
Returns
Date_str (No column name)
19-12-2007 2007-12-19
31-7-2009 2009-07-31
3-1-2010 2010-01-03
31-11-2009 NULL -- Notice 11/31 is NOT a date
See https://learn.microsoft.com/en-us/sql/t-sql/functions/cast-and-convert-transact-sql for date formats
You probably need
CONVERT(DateTime, Date_str, 105)
As I mentioned in the comments, the only realistic solution is to convert that string into a proper date-typed column. The current format doesn't allow date sorting, or search for a range of dates, eg to find entries in the last week, or between one date and the other.
Parsing with CONVERT or TRY_PARSE means that no indexes can be used to speed up queries. Each time :
WHERE CONVERT(Date, Date_str, 105) > '20170101'
is used, the server will have to scan the entire table to convert the data, then filter the rows.
If you can't change the type of the field itself, you can create a persisted computed column that returns the value as a date and add indexes to it. You'll be able to use that column for indexed querying:
alter table SomeTable add date2 as TRY_convert(Actual_Date,date_str,105) PERSISTED
create index IX_SomeTable_ActualDate on SomeTable (Actual_Date)
This will allow you to perform sorting without tricks:
SELECT *
FROM SomeTable
ORDER BY Actual_Date
Or run range queries that take advantage of the IX_SomeTable_ActualDate index:
SELECT *
FROM SomeTable
Where Actual_Date Between DATEADD(d,-7,GETDATE()) AND GETDATE()
If you have 1000 rows, you could get 1000 times better performance.
Existing applications won't even notice the change. Newer queries and applications will be able to take advantage of indexing and sorting
I had a similar problem: my column (<my_date_field>) had date values in the form of
2021-01
2021-02
2021-10
2021-12
and so on, with data type of nvarchar(4000), and I always ran into the Conversion failed when converting date and/or time from character string. error (when trying e.g. CAST(<my_date_field> AS DATE) or CAST(CAST(<my_date_field> AS VARCHAR(7)) AS DATE) etc.)
I was able to convert them to date with the following code:
SELECT
CONVERT(date, <my_date_field> + '-01') AS first_day_of_month
FROM my_table
which resulted in
2021-08-01
2021-07-01
2021-06-01
2021-05-01

How to filter SQL by time (greater and less than)?

I want to query a subset from a dataset. Each row has a time stamp of the following format:
2014-04-25T17:25:14
2014-04-25T18:40:16
2014-04-25T18:44:57
2014-04-25T19:10:32
2014-04-25T20:22:12
...
Currently, I use the following query to select a time-based subset:
time LIKE '%2014-04-25T18%' OR time LIKE '%2014-04-25T19%'
This becomes quite complicated when you start to filter by mintutes or seconds.
Is there a way to run a query that such as ...
time > '%2014-04-25T18%' AND time < '%2014-04-25T19%'
A regular expression would be okay, too.
The database is a SpatiaLite database. The time column is of type VARCHAR.
If the date is being treated as a string and based on the example above:
time LIKE '%2014-04-25T18%' AND time <> '%2014-04-25T18:00:00:000'
Otherwise, you could convert the date to seconds since midnight and add 60 minutes to that to create the range part of the filter
DECLARE #test DATETIME = '2014-04-25T17:25:14'
SELECT #test
, CONVERT(DATE,#test) AS JustDate
, DATEDIFF(s,CONVERT(DATETIME,(CONVERT(DATE,#test))), #test) AS SecondsSinceMidnight
-- 60 seconds * 60 minutes * 24 hours = 86400
Thanks to your posts and this answer I came up with this solution:
SELECT * FROM data
WHERE DATETIME(
substr(time,1,4)||'-'||
substr(time,6,2)||'-'||
substr(time,9,2)||' '||
substr(time,12,8)
)
BETWEEN DATETIME('2014-04-25 18:00:00') AND DATETIME('2014-04-25 19:00:00');

How do I match an entire day to a datetime field?

I have a table for matches. The table has a column named matchdate, which is a datetime field.
If I have 3 matches on 2011-12-01:
2011-12-01 12:00:00
2011-12-01 13:25:00
2011-12-01 16:00:00
How do I query that? How do I query all matches on 1 single date?
I have looked at date_trunc(), to_char(), etc.
Isn't there some "select * where datetime in date" function?
Cast your timestamp value to date if you want simple syntax. Like this:
SELECT *
FROM tbl
WHERE timestamp_col::date = '2011-12-01'; -- date literal
However, with big tables this will be faster:
SELECT *
FROM tbl
WHERE timestamp_col >= '2011-12-01 0:0' -- timestamp literal
AND timestamp_col < '2011-12-02 0:0';
Reason: the second query does not have to transform every single value in the table and can utilize a simple index on the timestamp column. The expression is sargable.
Note excluded the upper bound (< instead of <=) for a correct selection.
You can make up for that by creating an index on an expression like this:
CREATE INDEX tbl_ts_date_idx ON tbl (cast(timestamp_col AS date));
Then the first version of the query will be as fast as it gets.
not sure if i am missing something obvious here, but i think you can just
select * from table where date_trunc('day', ts) = '2011-12-01';
Just use the SQL BETWEEN function like so:
SELECT * FROM table WHERE date BETWEEN '2011-12-01' AND '2011-12-02'
You may need to include times in the date literals, but this should include the lover limit and exclude the upper.
From rails I believe you can do:
.where(:between => '2011-12-01'..'2011-12-02')

select statement using Between with datetime type does not retrieve all fields?

I'm facing a strange query result and I want to ask you why I'm facing this issue.
I store some datetime data into TestTable as following :
creation_time
-----------------------
2010-07-10 00:01:43.000
2010-07-11 00:01:43.000
2010-07-12 00:01:43.000
This table is created and filled as following :
create table TestTable(creation_time datetime);
Insert into TestTable values('2010-07-10 00:01:43.000');
Insert into TestTable values('2010-07-11 00:01:43.000');
Insert into TestTable values('2010-07-12 00:01:43.000');
when I execute this query , I get two rows only instead of three as I expected:
SELECT * FROM TestTable
WHERE creation_time BETWEEN CONVERT(VARCHAR(10),'2010-07-10',111) -- remove time part
and CONVERT(VARCHAR(10),'2010-07-12',111) -- remove time part
Or if I execute this query , the same issue ..
SELECT * FROM TestTable
WHERE CONVERT(datetime,creation_time,111) BETWEEN CONVERT(VARCHAR(10),'2010-07-10',111) -- remove time part
and CONVERT(VARCHAR(10),'2010-07-12',111) -- remove time part
My Question :
Why the last row ('2010-07-12 00:01:43.000') does not appear in
the result even if I set the date range to cover all the day from 2010-07-10 to 2010-07-12?
I use Sql server 2005 express edition with windows xp 32-bits.
I'm trying to don't use a workaround solution such as increasing the date range to cover additional day to get the days I want.
Thanks .
You need to remove the time part from creation_time as well. Just use the same CONVERT if it works.
Currently you're asking if 2010-07-12 00:01:43.000 is less than 2010-07-12 00:00:00.000, which is not true.
it does not show the date because you have removed the time part, which would make the date equivalent to '2010-07-12 00:00:00.000' and since the last row is greater than this, so it is not displaying in the query results.
Your script should look like this:
SELECT *
FROM TestTable
WHERE creation_time BETWEEN
convert(datetime, convert(char, '2010-07-10', 106))-- remove time part
and **DATEADD**(day, 1, convert(datetime, convert(char, '2010-07-**11**', 106))) -- remove time part and add 1 day
This script will return all between 2010-07-10 00:00:00 and 2010-07-12 00:00:00. Basically this means all items created in 2 days: 2010-07-10 and 2010-07-11.
Converting columns in your table for comparison can be costly and cause indexes to not be used. If you have a million rows in your table and you have an index on creation_time, you will be doing an index scan and converting all million values to a string for comparison.
I find it better to use >= the start date and < (end date + 1 day):
SELECT *
FROM TestTable
WHERE creation_time >= '2010-07-10'
AND creation_time < dateadd(day, 1, '2010-07-12')
And the reason your second one may not work is because format 111 uses slashes ("2010/07/10"), format 120 uses dashes ("2010-07-10"). Your converts aren't doing anything to your start and end date because you are converting a string to varchar, not a date. If you did this, it might work, but I would still recommend not doing the conversion:
SELECT * FROM TestTable
WHERE CONVERT(datetime, creation_time, 111) BETWEEN
CONVERT(VARCHAR(10), CONVERT(datetime, '2010-07-10'), 111) -- remove time part
and CONVERT(VARCHAR(10), CONVERT(datetime, '2010-07-12'), 111) -- remove time part
Date/time inclusive between 7/10/2010 and 7/12/2010:
SELECT * FROM TestTable
WHERE creation_time BETWEEN
CONVERT(VARCHAR,'2010-07-10',101) -- remove time part
and CONVERT(VARCHAR,'2010-07-13',101) -- remove time part