I'd like to be able to implement a search method that can take any arbitrary properties of my POCO class as arguments. This works well:
public static IEnumerable<iUser> Search(DataContext context, Func<iUser, bool> predicate)
{
return from i in context.GetTable<iUser>().Where(predicate) select i;
}
but in this case the filtering appears to take place after collecting all the rows in the table.
Is it possible to use Linq to generate an arbitrary query like this without filtering after the sql call? What approaches would you recommend?
Thanks!
LINQ to DB is an Object-Relational Mapper (ORM) that is capable of translating LINQ expressions into SQL. The word "expression" is important here. A Func is not an expression but a delegate, you have to use Expression<Func<>> in LINQ methods for LINQ to DB to be able to translate them. Otherwise the data will be pulled from the database first after which the Func filters them in memory.
So your function should look like:
public static IEnumerable<iUser> Search(DataContext context,
Expression<Func<iUser, bool>> predicate)
{
return context.GetTable<iUser>().Where(predicate);
}
The return type depends on the what you want the caller of this function to be capable of. If you return IQueryable<iUser> the caller will be able to extend the expression by their own expressions. That is, Search(context, somePredicate).Where(...) will be translated into SQL as a whole. Returning IEnumerable will apply any subsequent predicates (either as Func or as Expression) in memory.
Side note, in order to line up with common naming conventions, if iUser is an interface (I have no idea if LINQ to DB supports interfaces) then you should rename it into IUser, otherwise name it User.
Related
Looking to achieve the below but it is failing as the locations.Any() is being treated as an IEnumerable instead of an IQueryable and scalar functions invoked via EF require IQueryable. I need this filter to happen at the database level (not materialize the list first).
How can I get the locations.Any() to be treated as an IQueryable here? I understand the list doesn't exist in the database but is there a way for Entity Framework to understand this any and build and AND statement with nested OR in SQL?
public Address GetAddresses(List<Loctions> locations)
{
_context.Addresses.
Where(a => locations.Any(l => MyContext.CustomFunction(l.PropA,l.PropB, a.PropA, a.ProbB) > 1 ))
}
[DbFunction("fn_DistanceBetweenCoordinates", "dbo")]
public static decimal CustomFunction(decimal SourceLatitude, decimal SourceLongitude, decimal TargetLatitude, decimal TargetLongitude) {
throw new NotImplementedException();
}
You could achieve this by moving CustomFunction into the database and use that server side function when querying from EF.
Please read User defined function mapping and try to adapt the sample according to your use case.
We haven't seen the body of the CustomFunction, so it's impossible to tell if it's viable to do the transfer from client based UDF to server based.
We also don't know how the Locations list is populated. Depending on how that is done, the adaptation of the example code might become more cumbersome.
Just about to start playing with jOOQ for a proof of concept. jOOQ looks really simple, expressive and makes SQL maintenance a lot more easier.
We're a Java 8 shop. The usecase here is to write the data layer for a reporting app that dynamically queries tables, columns, filters and functions based on the user selection on the screen.
Although I really like the idea of writing type-safe queries (using the jOOQ codegen), I suppose for my usecase, that won't be a best fit. Because tables, columns etc etc are completely unknown, I suppose I just need the jOOQ SQL builder. That means I have to give up type safety. Is my assessment correct? Or are there any patterns I could use for building "dynamic" SQLs without compromising the type safety? Any pointers would be much appreciated.
You don't have to use jOOQ's code generator to take advantage of most features in jOOQ. The manual's introduction section states that jOOQ can be easily used as a SQL builder without the extra type static safety provided by the code generator:
https://www.jooq.org/doc/latest/manual/getting-started/use-cases/jooq-as-a-standalone-sql-builder
The type safety provided by code generation
The code generator essentially provides two type safety elements:
Names of objects are hard-wired into class names (tables, schemas, sequences, data types, procedures, etc.) and attribute names (columns, type attributes, procedure parameters).
Types of attributes (columns, attributes, parameters) are hard-wired into the generic attribute definitions.
These things certainly helps develop your application
The type safety provided by the jOOQ API
... but beware that the code generator simply reverse engineers a static snapshot of your schema. It is type safe because the entire jOOQ API allows for this kind of type safety. For instance, a Field<T> type has a generic type <T>, which can be used without the code generator as well, e.g. by using the plain SQL APIs:
Field<String> firstName = field(name("USER", "FIRST_NAME"), SQLDataType.VARCHAR(50));
The above API usage (DSL.field(Name, DataType)) does roughly the same as what the code generator would do anyway. It creates a column reference with column type information attached to it. You can use it like the columns generated by the code generator:
DSL.using(configuration)
.select(firstName)
.from(name("USER"))
.where(firstName.like("A%")) // Compiles
.and(firstName.eq(1)) // Doesn't compile: firstName must be compared to String
.join(name("ADDRESS")) // Doesn't compile: the SQL syntax is wrong
.fetch();
As you can see, the only thing that changed compared to using the code generator is the table / column references.
Dynamic SQL
But this means, that jOOQ is even more powerful for you without the code generator. You can still create dynamic SQL statements very easily. For instance:
// Construct your SQL query elements dynamically, and type safely
Condition condition = hasFirstNameFilter()
? firstName.like("A%")
: DSL.trueCondition();
DSL.using(configuration)
.select(firstName)
.from(name("USER"))
.where(condition) // Use dynamically constructed element here
.fetch();
You could also do this in a "functional way":
DSL.using(configuration)
.select(firstName)
.from(name("USER"))
.where(condition()) // Call a function to create the condition here
.fetch();
Or even better
public static Select<Record1<String>> firstNames(
Function<? super Field<String>, ? extends Condition> condition
) {
return
DSL.using(configuration)
.select(firstName)
.from(name("USER"))
.where(condition.apply(firstName)); // Lazy evaluate the predicate here
}
// Use it like this:
firstNames(col -> col.like("A%")).fetch();
Or even better, make the above a higher order function:
public static Function<
? super Function<? super Field<String>, ? extends Condition>,
? extends Select<Record1<String>>
> firstNames() {
// Lazy construct query here
return f -> DSL.using(configuration)
.select(firstName)
.from(name("USER"))
.where(f.apply(firstName)); // Lazy evaluate the predicate here
}
// Use it like this:
firstNames().apply(col -> col.like("A%")).fetch();
More details here:
https://www.jooq.org/doc/latest/manual/sql-building/dynamic-sql
Conclusion:
As you can see, while the code generator does add a lot of value for static schemas, there's nothing really static in the jOOQ API. jOOQ is an API for dynamic SQL query construction, that just happens to work well for static queries as well.
I'm working with an MS-SQL database with tables that use a customized date/time format stored as an integer. The format maintains time order, but is not one-to-one with ticks. Simple conversions are possible from the custom format to hours / days / months / etc. - for example, I could derive the month with the SQL statement:
SELECT ((CustomDateInt / 60 / 60 / 24) % 13) AS Month FROM HistoryData
From these tables, I need to generate reports, and I'd like to do this using LINQ-to-SQL. I'd like to have the ability to choose from a variety of grouping methods based on these dates (by month / by year / etc.).
I'd prefer to use the group command in LINQ that targets one of these grouping methods. For performance, I would like the grouping to be performed in the database, rather than pulling all my data into POCO objects first and then custom-grouping them afterwords. For example:
var results = from row in myHistoryDataContext.HistoryData
group row by CustomDate.GetMonth(row.CustomDateInt) into grouping
select new int?[] { grouping.Key , grouping.Count() }
How do I implement my grouping functions (like CustomDate.GetMonth) so that they will be transformed into SQL commands automatically and performed in the database? Do I need to provide them as Func<int, int> objects or Expression<> objects, or by some other means?
You can't write a method and expect L2S to automatically know how to take your method and translate it to SQL. L2S knows about some of the more common methods provided as part of the .NET framework for primitive types. Anything beyond that and it will not know how to perform the translation.
If you have to keep your db model as is:
You can define methods for interacting with the custom format and use them in queries. However, you'll have to help L2S with the translation. To do this, you would look for calls to your methods in the expression tree generated for your query and replace them with an implementation L2S can translate. One way to do this is to provide a proxy IQueryProvider implementation that inspects the expression tree for a given query and performs the replacement before passing it off to the L2S IQueryProvider for translation and execution. The expression tree L2S will see can be translated to SQL because it only contains the simple arithmetic operations used in the definitions of your methods.
If you have the option to change your db model:
You might be better off using a standard DateTime column type for your data. Then your could model the column as System.DateTime and use its methods (which L2S understands). You could achieve this by modifying the table itself or providing a view that performs the conversion and having L2S interact with the view.
Update:
Since you need to keep your current model, you'll want to translate your methods for L2S. Our objective is to replace calls to some specific methods in a L2S query with a lambda L2S can translate. All other calls to these methods will of course execute normally. Here's an example of one way you could do that...
static class DateUtils
{
public static readonly Expression<Func<int, int>> GetMonthExpression = t => (t / 60 / 60 / 24) % 13;
static readonly Func<int, int> GetMonthFunction;
static DateUtils()
{
GetMonthFunction = GetMonthExpression.Compile();
}
public static int GetMonth(int t)
{
return GetMonthFunction(t);
}
}
Here we have a class that defines a lambda expression for getting the month from an integer time. To avoid defining the math twice, you could compile the expression and then invoke it from your GetMonth method as shown here. Alternatively, you could take the body of the lambda and copy it into the body of the GetMonth method. That would skip the runtime compilation of the expression and likely execute faster -- up to you which you prefer.
Notice that the signature of the GetMonthExpression lambda matches the GetMonth method exactly. Next we'll inspect the query expression using System.Linq.Expressions.ExpressionVisitor, find calls to GetMonth, and replace them with our lambda, having substituted t with the value of the first argument to GetMonth.
class DateUtilMethodCallExpander : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
LambdaExpression Substitution = null;
//check if the method call is one we should replace
if(node.Method.DeclaringType == typeof(DateUtils))
{
switch(node.Method.Name)
{
case "GetMonth": Substitution = DateUtils.GetMonthExpression;
}
}
if(Substitution != null)
{
//we'd like to replace the method call; we'll need to wire up the method call arguments to the parameters of the lambda
var Replacement = new LambdaParameterSubstitution(Substitution.Parameters, node.Arguments).Visit(Substitution.Body);
return Replacement;
}
return base.VisitMethodCall(node);
}
}
class LambdaParameterSubstitution : ExpressionVisitor
{
ParameterExpression[] Parameters;
Expression[] Replacements;
public LambdaParameterExpressionVisitor(ParameterExpression[] parameters, Expression[] replacements)
{
Parameters = parameters;
Replacements = replacements;
}
protected override Expression VisitParameter(ParameterExpression node)
{
//see if the parameter is one we should replace
int p = Array.IndexOf(Parameters, node);
if(p >= 0)
{
return Replacements[p];
}
return base.VisitParameter(node);
}
}
The first class here will visit the query expression tree and find references to GetMonth (or any other method requiring substitution) and replace the method call. The replacement is provided in part by the second class, which inspects a given lambda expression and replaces references to its parameters.
Having transformed the query expression, L2S will never see calls to your methods, and it can now execute the query as expected.
In order to intercept the query before it hits L2S in a convenient way, you can create your own IQueryable provider that is used as a proxy in front of L2S. You would perform the above replacements in your implementation of Execute and then pass the new query expression to the L2S provider.
I think you can register your custom function in the DataContext and use it in the linq query. In this post is very well explained: http://msdn.microsoft.com/en-us/library/bb399416.aspx
Hope it helps.
Found a reference to some existing code which implements an IQueryable provider as Michael suggests.
http://tomasp.net/blog/linq-expand.aspx
I think assuming that code works, the other lingering issue is that you would have to have an Expression property for each type which contained the date.
The resulting code for avoiding doing that appears to be a bit cumbersome (though it would avoid the sort of errors you're trying to avoid by putting the calculation in a method):
Group Expression:
group row by CustomDate.GetMonth(row, x => x.customdate).Compile().Invoke(row)
Method to Return Group Expression:
public class CustomDate
{
public static Expression<Func<TEntity, int>> GetMonth<TEntity>(TEntity entity, Func<TEntity, int> func)
{
return x => ((func.Invoke(entity)/60/60/24)%13);
}
}
I'm not entirely sure whether that nested .Invoke would cause problems with the Expandable expression or whether the concept would have to be tweaked a bit more, but that code seems to supply an alternative to building a custom IQueryProvider for simple mathematical expressions.
There doesn't appear to be any way to instruct LINQ-to-SQL to call your SQL UDF. However, I believe you can encapsulate a reusable C# implementation in System.Linq.Expressions.Expression trees...
public class CustomDate {
public static readonly Expression<Func<int, int>> GetMonth =
customDateInt => (customDateInt / 60 / 60 / 24) % 13;
}
var results = from row in myHistoryDataContext.HistoryData
group row by CustomDate.GetMonth(row.CustomDateInt) into grouping
select new int?[] { grouping.Key , grouping.Count() }
Is it possible to convert:
public IList Get()
{
return Session.CreateCriteria(typeof(T)).List();
}
to return IQueryable?
What is the difference between IList and IQueryable?
One of the most important aspects when thinking of SQL and Linq is that returning IList means that the query has been executed. Returning IQueryable keeps open the option of deferring the sql execution later (so you could potentially build on the query outside of this method).
However, this would only be the case with NHibernate when using the Linq to NHibernate provider. The Criteria API is not Linqified for the SQL statements. So realistically in this instance returning IList or IQueryable has no significant difference.
What is possible is to return an IEnumerable like this:
public IEnumerable<T> Get()
{
return Session.CreateCriteria(typeof(T)).Future<T>();
}
This way you can do deferred execution as you do with Linq to SQL.
For more info about the Future method see:
http://ayende.com/Blog/archive/2009/04/27/nhibernate-futures.aspx
You can also simulate the IQueryable by returning the ICriteria interface instead of an IList:
public ICriteria<T> Get()
{
return Session.CreateCriteria(typeof(T));
}
This way you can start building the query outside of your method and finally execute it whenever you want.
This must be a simple question. Given a criteria, how one deletes the entities satisfying the criteria?
The rationale:
HQL and NH criteria are NHibernate specific constructs and as such they are server side DAL implementation details. I do not want them to "leak" to the client side. So, our client side provides LINQ expressions for the server to process. Up until now the requests where select requests and LINQ to NHibernate dealed with them just fine.
However, there is a need to implement batch delete operation now. As usual, the client side provides a LINQ expression and the server is to delete entities satisfying the expression.
Unfortunately, LINQ to NHibernate is of no help here. The most it can do is translate the given LINQ expression to NHibernate criteria.
Anyway, this is the story. I wish to stress that the client side is unaware of NHibernate at all and I like it to stay this way.
P.S.
I am using NH 2.1
You may use the criteria to select the IDs of your elements, join them in a string and use HQL to delete them?
Something like:
public void Delete(ICriteria criteria, string keyName, string tableName)
{
criteria.setProjection(Projections.Attribute(keyName));
IList<int> itemIds = criteria.List<int>();
string collection = string.Join(",", Array.ConvertAll<int, string>(itemIds, Convert.ToString));
Session.HQL(string.Format("delete from {0} where {1} in ({2})", tableName, keyName, collection);
}
This code was not tested or compiled (in particular I'm not sure of the HQL section), but I think that you got the idea: we don't fetch the whole objects thanks to the projection, but only the indices.
Simply put, up until 2.1.2 you cannot.
However, if you can translate the LINQ expression to HQL (or the ICriteria to HQL) then you can use the overloaded ISession.Delete() method which uses a passed HQL string.
In your repository/dao/persistencemanager/whatever class:
public IEnumerable<T> FindAll(DetachedCriteria criteria)
{
return criteria.GetExecutableCriteria(Session).List<T>();
}
and then
public void Delete(DetachedCriteria criteria)
{
foreach (T entity in FindAll(criteria))
{
Delete(entity);
}
}
See Davy Brion's post Data Access with NHibernate.
Edit:
As far as I know, if you want to use Criteria you need to load the objects and iterate over them to delete them. Alternatively use HQL or pass in the SQL to the session.
I know this is an old question but for argument sake; if one uses repository pattern you can declare a delete method which does the following:
public void Delete(System.Linq.Expressions.Expression<System.Func<TEntity, bool>> predicate)
{
var entities = _session.Query<TEntity>().Where(predicate);
foreach (var entity in entities)
_session.Delete(entity);
}
Note the code is using expressions in order for repository interface to be generic enough so you can also implement a for example Entity Framework repository.