How to compare string data to time - sql

I have data type in string and the time is like 06:00A, 09:00P, etc. I would like to query data from 6am to 12pm, how do I convert the string data to time format and query it in linq to sql?

Use DateTime.ParseExact or DateTime.TryParseExact to convert the string to a date. If you can't guarantee that the string version of your time is always going to be correct, stick to the TryParseExact version.
Once you have it converted to date, query as normal.
Example at: https://dotnetfiddle.net/MDnERt
Edited after response:
If you are using the code as written against EntityFramework then no, this will not work. (Please also note that there is a big difference between Linq To SQL and Entity Framework, but the same concepts apply, to some degree)
ORMs that support LINQ are actually converting your where clauses into an Expression which is then translated by the ORM into SQL. You will get a NotSupported exception, or something similar.
Is there some reason why the table in question is using that time format? Why would you not just use a datetime in the table? There is also the option of using the time datatype in sql server (assuming you are targetting sql server) which is mapped to the TimeSpan type in .net.
You would define your table in Sql server like:
create table log ( data varchar(20), logtime time )
and the LINQ expression would look something like:
from x in Logs
where x.Logtime >= new TimeSpan(6,0,0) && x.Logtime <= new TimeSpan(12,0,0)
select x
Now we are getting into actual design questions, though, which is off topic. :)

I'd suggest writing own parser and represent times as TimeSpan:
TimeSpan? ToTimeSpan(string str)
{
// get A or P at the end
var amPm = str.Last();
int hrs, mins;
try
{
hrs = int.Parse(str.Substring(0, 2));
mins = int.Parse(str.Substring(3, 2));
}
catch
{
return null;
}
switch (amPm)
{
case 'P': hrs += 12; break;
case 'A': break;
default: return null;
}
return new TimeSpan(hrs, mins, 0);
}

Related

GORM not returning results, but when I run the SQL query in my database client, records come back

My GORM query looks a little like this:
selectCallSQL = "SELECT * from callautomation_schedule WHERE id = ?"
testSelect = "SELECT * FROM callautomation_schedule WHERE next_planned_call > date_trunc('minute', now())"
func SelectCall(id int) *CallSchedule{
var result CallSchedule
connection.Raw(selectCallSQL, id).Scan(&result)
return &result
}
func SelectCall2() *CallSchedule{
var result CallSchedule
connection.Raw(testSelect).Scan(&result)
return &result
}
The first function returns a result as expected, however, the second function does not.
If I run the testSelect SQL in my database client, I do get a result. Why is this happening?
The issue I discovered was with my connection string and column setup. I was using the type TIMESTAMP NO TIMEZONE in my table schema, but in my connection string, I was connecting via the Asia Timezone.
Annoying bug, but fixed now!

How to filter SQL Server DateTime using .NET Core WebAPI with OData v4

I am running a simple .NET Core WebApi application with OData Query v4 and SQL Server 2012.
This works, but it's extremely slow:
GET /api-endpoint?$filter=date(MyDateTimeField) ge 2018-01-01&$top=100
SQL Query generated by the URL above:
SELECT TOP 100 * FROM MyTable WHERE ((((DATEPART(year, [MyDateTimeField]) * 10000) + (DATEPART(month, [MyDateTimeField]) * 100)) + DATEPART(day, [MyDateTimeField])) >= (((2018 * 10000) + (1 * 100)) + 1))
When I try to do this:
GET /api-endpoint?$filter=MyDateTimeField ge 2018-01-01T00:00:00.00Z&$top=100
It generates the following SQL query:
SELECT TOP 100 * FROM MyTable WHERE [MyDateTimeField] > '2018-01-01T00:00:00.0000000'
Which returns this error:
Conversion failed when converting date and/or time from character
string.
What would the OData Query syntax be to generate a SQL query similiar to this?
SELECT TOP 100 * FROM MyTable WHERE [MyDateTimeField] > '2018-01-01'
Assuming that the field of MyDateTimeField is datetime instead of datatime2, decorate the MyDateTimeField with a column annotation [Column(TypeName="datetime")] firstly :
public class MyTable
{
// ... other props
[Column(TypeName="datetime")]
public DateTime MyDateTimeField {get;set;}
}
To query with datetime, cast it into DateTimeOffset:
?$filter=MyDateTimeField ge cast(2018-01-01T00:00:00.00Z,Edm.DateTimeOffset)
The generated sql is something like :
SELECT ..., [$it].[MyDateTimeField],
FROM [MyTable] AS [$it]
WHERE [$it].[MyDateTimeField] >= '2018-01-01T08:00:00.000'
Note the datetime above is 2018-01-01T08:00:00.000 instead of 2018-01-01T00:00:00.0000000.
A screenshot of Demo:
After drowning incredulous in my own frustration I finally found a solution which would not force my api consumers to cast the DateTime string in an, ugly, verbose, disturbing expression.
I also want my model to transparently look like using DateTimeOffset instead of DateTime, this will allow me in the future to refactor the database and finally use DateTimeOffset even if so far I don't need to handle time zones.
My limitation is that I cannot update my legacy database and so here is the solution.
Is this solution for you?
This solution is useful only if:
You cannot (or want to) update your db column type (if you can you should and solve the problem)
You use or can change you entities to use DateTimeOffset instead of DateTime (If it is a new api consider doing this to comply with odata standard and allowing you to refactor in the future the underlay database)
You don't need to handle different time zones (this can be done but you need to work on the solution to do it)
You use EF Core >= 2.1
Solution
Update your entity using DateTimeOffset
public class MyEntity {
[...]
public DateTimeOffset CreatedDateTime { get; set; }
public DateTimeOffset ModifiedDateTime { get; set; }
[...]
}
Create a ValueConverter
public static class ValueConvertes
{
public static ValueConverter<DateTimeOffset, DateTime> DateTimeToDateTimeOffset =
new ValueConverter<DateTimeOffset, DateTime>(
model => model.DateTime,
store => DateTime.SpecifyKind(store, DateTimeKind.UTC));
}
Update your model mapping
public void Configure(EntityTypeBuilder<QuestionQML> builder)
{
builder.ToTable("MyEntityTable");
builder.Property(e => e.CreatedDateTime)
.HasColumnName("CreatedDateTime") // On DB datetime Type
.HasConversion(ValueConvertes.DateTimeToDateTimeOffset);
builder.Property(e => e.ModifiedDateTime)
.HasColumnName("ModifiedDateTime") // On DB datetime Type
.HasConversion(ValueConvertes.DateTimeToDateTimeOffset);
[...]
}
This allow you to filter in the following ways:
?$filter=CreatedDateTime gt 2010-01-25T02:13:40Z
?$filter=CreatedDateTime gt 2010-01-25T02:13:40.01234Z
?$filter=CreatedDateTime gt 2010-01-25
Special thanks to chris-clark
EDIT:
Corrected code DateTimeKind.UTC can be used when the datetime stored in the database is in UTC, if you store in a different timezone you need to set the Kind to the timezone you use, but this changes the way your datetimes will be shown in the results, showing for UK Timezone, for example, Z(GMT time) or +01:00 (BST time.

Bad performance when filtering Azure logs - WCF Data Services filters

Azure Diagnostics is pushing Windows Events into a storage table "WADWindowsEventLogsTable".
I would like to query this storage Table using VisualStudio(2015) and CloudExplorer.
As this table has an huge content, I'm indefinitely waiting for the results..
Here is a query sample:
EventId eq 4096 and Timestamp gt datetime'2016-06-24T08:20:00' and Timestamp lt datetime'2016-06-24T10:00:00'
I suppose that this query is correct ?
Does exist a way to improve performance ?
filter result columns ?
return only TOP X results ?
another usefull tips ?
I know that a better way would be to script that; for example using Python, but I would like to use the UI as much as possible..
(Edit) following Gaurav Mantri answer I used this little C# program to build my query. The answer is so quick and that solve my initial performance issue:
static void Main(string[] args)
{
string startDate = "24 June 2016 8:20:00 AM";
string endDate = "24 June 2016 10:00:00 AM";
string startPKey = convertDateToPKey(startDate);
string endPKey = convertDateToPKey(endDate);
Debug.WriteLine("(PartitionKey gt '" + startPKey + "'"
+ " and PartitionKey le '" + endPKey +"')"
+ " and (EventId eq 4096)"
);
}
private static string convertDateToPKey(string myDate)
{
System.DateTime dt = System.Convert.ToDateTime(myDate);
long dt2ticks = dt.Ticks;
string ticks = System.Convert.ToString(dt2ticks);
return "0" + ticks;
}
NB: for those, like me, who are searching so far away how to export results to a CSV file, you should know that this icon is your answer (and it's not a 'undo' ;) ):
In your query, you're filtering on Timestamp attribute which is not indexed (Only PartitionKey and RowKey attributes are indexed). Thus your query is making a full table scan (i.e. going from the 1st record till the time it finds a matching record) and hence not optimized.
In order to avoid full table scan, please use PartitionKey in your query. In case of WADWindowsEventLogsTable, the PartitionKey essentially represents the date/time value in ticks. What you would need to do is convert the date/time range for which you want to get the data into ticks, prepend a 0 in front of it and then use it in the query.
So your query would be something like:
(PartitionKey gt 'from date/time value in ticks prepended with 0' and PartitionKey le 'to date/time value in ticks prepended with 0') and (EventId eq 4096)
I wrote a blog post about it some time ago that you may find useful: http://gauravmantri.com/2012/02/17/effective-way-of-fetching-diagnostics-data-from-windows-azure-diagnostics-table-hint-use-partitionkey/

Setting a Clob value in a native query

Oracle DB.
Spring JPA using Hibernate.
I am having difficulty inserting a Clob value into a native sql query.
The code calling the query is as follows:
#SuppressWarnings("unchecked")
public List<Object[]> findQueryColumnsByNativeQuery(String queryString, Map<String, Object> namedParameters)
{
List<Object[]> result = null;
final Query query = em.createNativeQuery(queryString);
if (namedParameters != null)
{
Set<String> keys = namedParameters.keySet();
for (String key : keys)
{
final Object value = namedParameters.get(key);
query.setParameter(key, value);
}
}
query.setHint(QueryHints.HINT_READONLY, Boolean.TRUE);
result = query.getResultList();
return result;
}
The query string is of the format
SELECT COUNT ( DISTINCT ( <column> ) ) FROM <Table> c where (exact ( <column> , (:clobValue), null ) = 1 )
where "(exact ( , (:clobValue), null ) = 1 )" is a function and "clobValue" is a Clob.
I can adjust the query to work as follows:
SELECT COUNT ( DISTINCT ( <column> ) ) FROM <Table> c where (exact ( <column> , to_clob((:stringValue)), null ) = 1 )
where "stringValue" is a String but obviously this only works up to the max sql string size (4000) and I need to pass in much more than that.
I have tried to pass the Clob value as a java.sql.Clob using the method
final Clob clobValue = org.hibernate.engine.jdbc.ClobProxy.generateProxy(stringValue);
This results in a java.io.NotSerializableException: org.hibernate.engine.jdbc.ClobProxy
I have tried to Serialize the Clob using
final Clob clob = org.hibernate.engine.jdbc.ClobProxy.generateProxy(stringValue);
final Clob clobValue = SerializableClobProxy.generateProxy(clob);
But this appears to provide the wrong type of argument to the "exact" function resulting in (org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144) - SQL Error: 29900, SQLState: 99999
(org.hibernate.engine.jdbc.spi.SqlExceptionHelper:146) - ORA-29900: operator binding does not exist
ORA-06553: PLS-306: wrong number or types of arguments in call to 'EXACT'
After reading some post about using Clobs with entities I have tried passing in a byte[] but this also provides the wrong argument type (org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144) - SQL Error: 29900, SQLState: 99999
(org.hibernate.engine.jdbc.spi.SqlExceptionHelper:146) - ORA-29900: operator binding does not exist
ORA-06553: PLS-306: wrong number or types of arguments in call to 'EXACT'
I can also just pass in the value as a String as long as it doesn't break the max string value
I have seen a post (Using function in where clause with clob parameter) which seems to suggest that the only way is to use "plain old JDBC". This is not an option.
I am up against a hard deadline so any help is very welcome.
I'm afraid your assumptions about CLOBs in Oracle are wrong. In Oracle CLOB locator is something like a file handle. And such handle can be created by the database only. So you can not simply pass CLOB as bind variable. CLOB must be somehow related to database storage, because this it can occupy up to 176TB and something like that can not be held in Java Heap.
So the usual approach is to call either DB functions empty_clob() or dbms_lob.create_temporary (in some form). Then you get a clob from database even if you think it is "IN" parameter. Then you can write as many data as you want into that locator (handle, CLOB) and then you can use this CLOB as a parameter for a query.
If you do not follow this pattern, your code will not work. It does not matter whether you use JPA, SpringBatch or plan JDBC. This constrain is given by the database.
It seems that it's required to set type of parameter explicitly for Hibernate in such cases. The following code worked for me:
Clob clob = entityManager
.unwrap(Session.class)
.getLobHelper()
.createClob(reader, length);
int inserted = entityManager
.unwrap(org.hibernate.Session.class)
.createSQLQuery("INSERT INTO EXAMPLE ( UUID, TYPE, DATA) VALUES (:uuid, :type, :data)")
.setParameter("uuid", java.util.Uuid.randomUUID(), org.hibernate.type.UUIDBinaryType.INSTANCE)
.setParameter("type", java.util.Uuid.randomUUID(), org.hibernate.type.StringType.INSTANCE)
.setParameter("data", clob, org.hibernate.type.ClobType.INSTANCE)
.executeUpdate();
Similar workaround is available for Blob.
THE ANSWER: Thank you both for your answers. I should have updated this when i solved the issue some time ago. In the end I used JDBC and the problem disappeared in a puff of smoke!

NHibernate Like with integer

I have a NHibernate search function where I receive integers and want to return results where at least the beginning coincides with the integers, e.g.
received integer: 729
returns: 729445, 7291 etc.
The database column is of type int, as is the property "Id" of Foo.
But
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(NHibernate.Criterion.Expression.InsensitiveLike("Id", id.ToString() + "%"));
return criteria.List<Foo>();
does result in an error (Could not convert parameter string to int32). Is there something wrong in the code, a work around, or other solution?
How about this:
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(Expression.Like(Projections.Cast(NHibernateUtil.String, Projections.Property("Id")), id.ToString(), MatchMode.Anywhere));
return criteria.List<Foo>();
Have you tried something like this:
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(NHibernate.Criterion.Expression.Like(Projections.SqlFunction("to_char", NHibernate.NHibernateUtil.String, Projections.Property("Id")), id.ToString() + "%"));
return criteria.List<Foo>();
The idea is convert the column before using a to_char function. Some databases do this automatically.
AFAIK, you'll need to store your integer as a string in the database if you want to use the built in NHibernate functionality for this (I would recommend this approach even without NHibernate - the minute you start doing 'like' searches you are dealing with a string, not a number - think US Zip Codes, etc...).
You could also do it mathematically in a database-specific function (or convert to a string as described in Thiago Azevedo's answer), but I imagine these options would be significantly slower, and also have potential to tie you to a specific database.