I have an EF Code First Db context that I'm using to query the database. I noticed some performance issues when passing in queries as Func<Product, bool>s from my Aggregate Repository and on investigating further it turned out that the queries were not being translated into SQL Queries.
After a little more digging I discovered the following.
var results = _context.Products
.Where(p => p.ProductCode.Contains("AAA"))
.Where(p => p.CategoryId == 1)
.ToList();
This works exactly as expected. It generates some parametrized SQL with a Where Clause.
==================================================================
var results2 = _context.Products
.Where(p => p.ProductCode.Contains("AAA") && p.CategoryId == 1)
.ToList();
This also works as expected. It generates the same sql as above
==================================================================
Func<Product, bool> pred = (p => p.ProductCode.Contains("AAA") && p.CategoryId == 1);
var results3 = _context.Products.Where(pred).ToList();
This is broken. It doesn't generate the where clause in the SQL, it returns everything and then filters it in code.
Because in order to translate into SQL, it has to be an Expression<...>, not a Func<...>.
This is done automatically for you by the compiler, and since the overloads on the Linq-to-SQL classes takes expressions, not delegates, the compiler will automagically translate your code (which looks like a lambda or an anonymous method) into an expression object and pass that.
However, if you take care of building the function yourself, the compiler cannot do this, and Linq-to-SQL does not take anonymous methods, it only takes expressions.
What you can do is to execute the parts of your query that you can, and then filter the results through your function, but I would look into just changing the type of your value into an expression instead.
No sooner than I posted this ReSharper helped answer my question by showing me the overload method signature for the Where() extension method.
It takes both Func<T, bool> and Expression<Func<T, bool>>. If your declaring your predicates externally, you must use the Expression variation as the former is not translated into sql.
Here's why the query reads the whole table.
When a Func is used instead of an Expression, the compiler chooses methods on System.Linq.Enumerable - instead of System.Linq.Queryable. The Enumerable methods iterate the source collection (sometimes lazily) while the Queryable methods build up the expression tree.
Since the call to Where isn't part of the expression tree, the sql generator doesn't see it during the query translation.
Related
We have a search routine that uses criteria to build SQL query (because its restrictions added dynamically).
In a particular case (a very complicated case) we need to search over a table-valued function.(our model object is mapped to the function).
The result would be something like this :
SELECT count(*) FROM dbo.GetSubStaffsLetterInstances(#staffId) WHERE LetterNumber="1234";
The problem is I don't know how to pass #staffId to my criteria(I tried adding an Eq restrictions without success since restrictions are working on properties)
I know I can add a parameter to an IQuery but I don't know how I can do it using an ICriteria object.
If I understand your question completely, you can resort back to standard SQL:-
var sql = "SELECT count(*) FROM dbo.GetSubStaffsLetterInstances(:staffId)
WHERE LetterNumber=:letterNum";
var count = session.CreateSqlQuery(sql)
.setInt32("staffId", 12345)
.setString("letternum", "A1")
.UniqueResult<int>();
or try .UniqueResult<long>(); as I can't remember which one HQL returns
I've a simple class like this:
Public Class CalculationParameter{
public Long TariffId{get;set;}
}
In a workflow activity, I've an Assign like this:
(From tariffDetail In db.Context.TariffDetails
Where tariffDetial.TariffId = calculationParameter.TariffId).FirstOrDefault()
Dto is passed to Activity as an Input Argument.
It raise following error and I'm wondering how to assign Id. Any Idea?
LINQ to Entities does not recognize the method 'Int64
GetValue[Int64](System.Activities.LocationReference)' method, and this
method cannot be translated into a store expression.
How can I assign the calculationParameter.TariffId to tariffDetial.TariffId?!
UPDATE:
Screen shot attached shows that how I'm trying to assign calculationParameter.TariffId to tariffDetail.TariffId (car.Id = Dto.Id) and the query result should assign to CurrentTrafficDetail object.
Here's your problem. I don't know if there is a solution to it.
As you said in a (now deleted, unfortunately necessitating that I answer) comment, the exception you're getting is
LINQ to Entities does not recognize the method Int64 GetValue[Int64](System.Activities.LocationReference) method, and this method cannot be translated into a store expression.
in your Linq query, calculationParameter is a Variable defined on the workflow. That Variable is actually an instance that extends the type System.Activities.LocationReference and NOT CalculationParameter.
Normally, when the workflow executes, the LocationReference holds all the information it needs to find the value which is assigned to it. That value isn't retrieved until the last possible moment. At runtime, the process of retrieval (getting the executing context, getting the value, converting it to the expected type) is managed by the workflow.
However, when you introduce Linq into the mix, we have the issue you are experiencing. As you may or may not know, your expression gets compiled into the extension method version of the same.
(From tariffDetail In db.Context.TariffDetails
Where tariffDetial.TariffId = calculationParameter.TariffId)
.FirstOrDefault()
is compiled to
db.Context.TariffDetails
.Where(x => x.TariffId = calculationParameter.TariffId)
.FirstOrDefault();
When this executes, L2E doesn't actually execute this code. It gets interpreted and converted into a SQL query which is executed against the database.
As the interpreter isn't omniscient, there are a well defined set of limitations on what methods you can use in a L2S query.
Unfortunately for you, getting the current value of a LocationReference is not one of them.
TL:DR You cannot do this.
As for workarounds, the only thing I think you can do is create a series of extension methods on your data context type or add methods to your CalculationParameter class that you can call from within the Expression Editor. You can create your Linq to Entities queries within these methods, as all types will already have been dereferenced by the workflow runtime, which means you won't have to worry about the L2E interpreter choking on LocationReferences.
*Edit: A workaround can be found here (thanks to Slauma who mentioned this in a comment on the question)
This is probably fairly straightforward but i can't seem to find a reasonable explanation in any documentation.
I'm trying to use an NHibernate.ISQLQuery and using SetResultTransformer() to return a custom set of results from a custom SQL query. Like so:
public virtual IList<T> GetSQLObject<T>(string sql, IDbParameter[] parameters = null)
{
ISQLQuery qry = _sess.CreateSQLQuery(sql);
qry.SetResultTransformer(Transformers.AliasToBean(typeof(T)));
if (parameters != null) {
foreach (IDbParameter parameter in parameters) {
qry.SetParameter(parameter.Name, parameter.Value);
}
}
return qry.List<T>();
}
From looking at the examples, it seems that in the sql query I have to use parameters in the format :param1 instead of #param1 as I would in a standard SQL query. If i use the latter syntax in the query, it throws an error at qry.SetParameter().
Is there a reason why ISQLQuery/NHibernate requires them in this format and won't work with the normal syntax?
SQL Server uses #param, but not every other database does. For example, MySQL uses ?param
NHibernate allows you to swap out 1 database implementation for another with little to no reworking of your DAL. It sets the parameters based on the database you configured when you setup the NH Configuration.
Edit: Also I think :param came about from Hibernate being targeted at Oracle when it was initially developed, since Oracle uses :param
Phil has answered the "why"; so perhaps I can recommend a "how"; why not just add a new extension method to the IDbParameter type (something like .GetNHibernateName() ) that will return the parameter name with the "#" replaced with a ":"; that should be trivial to implement.
Is there a way to convert a MethodBody (or other Reflection technique) into a System.Linq.Expressions.Expression tree?
It is indeed possible, see DelegateDecompiler:
https://github.com/hazzik/DelegateDecompiler
NOTE: I am not affiliated with this project
Edit
Here is the basic approach that the project takes:
Get MethodInfo for the method you want to convert
Use methodInfo.GetMethodBody to get a MethodBody object. This contains,
among other things, the MSIL and info on arguments and locals
Go through the instructions, examine the opcodes, and build the appropriate Expressions
Tie it all together and return an optimized Expression
Here is a code snippet from the project that decompiles a method body:
public class MethodBodyDecompiler
{
readonly IList<Address> args;
readonly VariableInfo[] locals;
readonly MethodInfo method;
public MethodBodyDecompiler(MethodInfo method)
{
this.method = method;
var parameters = method.GetParameters();
if (method.IsStatic)
args = parameters
.Select(p => (Address) Expression.Parameter(p.ParameterType, p.Name))
.ToList();
else
args = new[] {(Address) Expression.Parameter(method.DeclaringType, "this")}
.Union(parameters.Select(p => (Address) Expression.Parameter(p.ParameterType, p.Name)))
.ToList();
var body = method.GetMethodBody();
var addresses = new VariableInfo[body.LocalVariables.Count];
for (int i = 0; i < addresses.Length; i++)
{
addresses[i] = new VariableInfo(body.LocalVariables[i].LocalType);
}
locals = addresses.ToArray();
}
public LambdaExpression Decompile()
{
var instructions = method.GetInstructions();
var ex = Processor.Process(locals, args, instructions.First(), method.ReturnType);
return Expression.Lambda(new OptimizeExpressionVisitor().Visit(ex), args.Select(x => (ParameterExpression) x.Expression));
}
}
No, there isn't.
You're basically asking for a somewhat simpler version of Reflector.
Yes, it is possible... but it hasn't been done yet, as far as I know.
If anyone does know of a library that de-compiles methods to expression trees, please let me know, or edit the above statement.
The most difficult part of what you would have to do is write a CIL de-compiler. That is, you would need to translate the fairly low-level CIL instructions (which conceptually target a stack machine) into much higher-level expressions.
Tools such as Redgate's Reflector or Telerik's JustDecompile do just that, but instead of building expression trees, they display source code; you could say they go one step further, since expression trees are basically still language-agnostic.
Some notable cases where this would get especially tricky:
You would have to deal with cases of CIL instructions for which no pre-defined Expression tree node exists; let's say, tail.call, or cpblk (I'm guessing a little here). That is, you'd have to create custom expression tree node types; having them compiled back into an executable method when you .Compile() the expression tree might be an issue, because the expression tree compiler tries to break down custom nodes into standard nodes. If that is not possible, then you cannot compile the expression tree any more, you could only inspect it.
Would you try to recognise certain high-level constructs, such as a C# using block, and try to build a (custom) expression tree node for it? Remember that C# using breaks down to the equivalent of try…finally { someObj.Dispose(); } during compilation, so that is what you might see instead of using if you reflected over the method body's CIL instructions and exception handling clauses.
Thus, in general, expect that you need to be able to "recognise" certain code patterns and summarise them into a higher-level concept.
Consider the following line of code:
private void DoThis() {
int i = 5;
var repo = new ReportsRepository<RptCriteriaHint>();
// This does NOT work
var query1 = repo.Find(x => x.CriteriaTypeID == i).ToList<RptCriteriaHint>();
// This DOES work
var query1 = repo.Find(x => x.CriteriaTypeID == 5).ToList<RptCriteriaHint>();
}
So when I hardwire an actual number into the lambda function, it works fine. When I use a captured variable into the expression it comes back with the following error:
No mapping exists from object type
ReportBuilder.Reporter+<>c__DisplayClass0
to a known managed provider native
type.
Why? How can I fix it?
Technically, the correct way to fix this is for the framework that is accepting the expression tree from your lambda to evaluate the i reference; in other words, it's a LINQ framework limitation for some specific framework. What it is currently trying to do is interpret the i as a member access on some type known to it (the provider) from the database. Because of the way lambda variable capture works, the i local variable is actually a field on a hidden class, the one with the funny name, that the provider doesn't recognize.
So, it's a framework problem.
If you really must get by, you could construct the expression manually, like this:
ParameterExpression x = Expression.Parameter(typeof(RptCriteriaHint), "x");
var query = repo.Find(
Expression.Lambda<Func<RptCriteriaHint,bool>>(
Expression.Equal(
Expression.MakeMemberAccess(
x,
typeof(RptCriteriaHint).GetProperty("CriteriaTypeID")),
Expression.Constant(i)),
x)).ToList();
... but that's just masochism.
Your comment on this entry prompts me to explain further.
Lambdas are convertible into one of two types: a delegate with the correct signature, or an Expression<TDelegate> of the correct signature. LINQ to external databases (as opposed to any kind of in-memory query) works using the second kind of conversion.
The compiler converts lambda expressions into expression trees, roughly speaking, by:
The syntax tree is parsed by the compiler - this happens for all code.
The syntax tree is rewritten after taking into account variable capture. Capturing variables is just like in a normal delegate or lambda - so display classes get created, and captured locals get moved into them (this is the same behaviour as variable capture in C# 2.0 anonymous delegates).
The new syntax tree is converted into a series of calls to the Expression class so that, at runtime, an object tree is created that faithfully represents the parsed text.
LINQ to external data sources is supposed to take this expression tree and interpret it for its semantic content, and interpret symbolic expressions inside the tree as either referring to things specific to its context (e.g. columns in the DB), or immediate values to convert. Usually, System.Reflection is used to look for framework-specific attributes to guide this conversion.
However, it looks like SubSonic is not properly treating symbolic references that it cannot find domain-specific correspondences for; rather than evaluating the symbolic references, it's just punting. Thus, it's a SubSonic problem.