A few weeks ago I decided to switch from using Linq to SQL to use NHibernate instead. (Reasons include: other Java-based projects use Hibernate; target DB not yet decided; multiple DBs may have to be targeted)
Anyway, I wanted to continue using LINQ and saw that NHibernate has LINQ support. I want to have a small nummber of object access methods and be able to pass a LINQ expression which filters the query but it has not worked as expected.
Here is an example based on the PredicateBuilder from http://www.albahari.com/nutshell/predicatebuilder.aspx
public static Expression<Func<Product, bool>> ContainsInDescription(params string[] keys)
{
var predicate = PredicateBuilder.False<Product>();
foreach (string keyword in keys)
{
string temp = keyword.ToLower();
predicate = predicate.Or(p => p.Description.Contains(temp));
}
return predicate;
}
The Predicate.False statment causes the following error message:
Atf.NUnit.Model.TestLinq.TestProductCID():
System.Exception : Could not determine member type from Constant, False, System.Linq.Expressions.ConstantExpression
The p.Description.Contains statement causes this error message:
Atf.NUnit.Model.TestLinq.TestProductCID():
System.Exception : Could not determine member type from Invoke, Invoke(p => p.Description.Contains(value(Atf.Model.Linq.ProductLinq+<>c__DisplayClass2).temp), f), System.Linq.Expressions.InvocationExpression
I get similar errors when using string.Equals and other such methods.
Am I doing something wrong here? Should I use a different approach?
Related
I have an OData Controller which looks pretty standard.
[HttpGet]
[ODataRoute("GridData")]
[EnableQuery]
public async Task<IQueryable<GridData>> GetGridData(ODataQueryOptions<GridData> odataOptions)
{
var query = odataOptions.ApplyTo(_service.GetGridDataQueryable()) as IQueryable<GridData>
return query;
}
Projection looks like this :
.Select(async x =>
{
//Pretty resource heavy
x.Ownership = await _ownershipService.ComputeAsync(_currentUser));
return x;
})
.Select(t => t.Result)
.ToList();
Now the problem is that I need to actually return a GridDataDTO object from this call. There is some processing that cannot be done at the database level. The processing is pretty heavy so I would not like to add it inside the GetGridDataQueryable().Also the processing is async, and need a materialized result set to be able to apply it.
I also need to return the IQueryable in the controller to be able to benefit from $count, $select, etc .
This hooks up to a pretty complex grid with a lot of options for filtering/sorting so I would not like to remove the OData functionality.
Is there a simple way to add postprocessing here ? After the result is materialized, project it to my GridDataDTO ?
There is no need for insert/update/delete support, as this will be only used for read operations.
There is no requirement for your controller method to only pass through a query from the database, in fact your method does not need to return an IQueryable<T> result at all!
You can still benefit from OData $select, $expand and $filter operators on result sets that are not IQueryable<T>, but you lose most of the performance benefits of doing so and you have to prepare you data so that the operators can be processed, and you will have to explicitly decorate your endpoint with the [EnableQuery] attribute.
In the following example you current query is materialized into memory, after applying the query options, then we can iterate over the set and manipulate it as we need to.
In the end the same recordset, with the modified records is returned, cast as queryable to match the method signature, however the method would still function the same if the result was IEnumerable<T>
There is a strong argument that says you should return IEnumerable<T> because it conveys the correct information that the recordset has been materialized and is not deferred.
[HttpGet]
[ODataRoute("GridData")]
public async Task<IQueryable<GridDataDTO>> GetGridData(ODataQueryOptions<GridData> odataOptions)
{
// NOTE: GridDataDTO : GridData
// apply $filter, $top and $skip to the DB query
IQueryable<GridData> query = odataOptions.ApplyTo(_service.GetGridDataQueryable());
// materialize
var list = query.ToList();
// project into DTO
List<GridDataDTO> output = list.Select(async x =>
{
var o = new GridDataDTO(x);
o.Ownership = await _ownershipService.ComputeAsync(_currentUser));
}).ToList();
// return, as Queryable
return output.AsQueryable();
}
UPDATE:
When the manipulations involve projection into a new type, then to properly support OData query options the type defined in your ODataQueryOptions<> needs to be assignable from the output element type. You can do this through inheritance or with implicit cast definitions.
If an explicit cast is required (or no cast is available at all) then you will have to manually validate the ApplyTo logic, the ODataQueryOptions must be a valid type reference to match the output.
I'm using Remote.Linq to serialise / deserialise my Expressions as I want to create the ability to send dynamic expressions from a client application to our web services. Standard .NET expressions cannot be serialised so I'm using Remote.Linq instead.
However, I cannot see how to execute the Expression. Normally I would invoke the Compile() and Invoke() methods to execute the Expression against the data. But Remote.Linq expressions don't support such methods.
The following unit test may explain more clearly what I'm trying to achieve.
[TestMethod]
public void SerializeLinqExpressionsTests()
{
var testdata = GetTestdata();
Expression<Func<ModuleEntityAdmins, ModuleEntityAdmin>> expr1 = m => m.Modules.Find(q => q.Id == 1);
var remoteExpression1 = expr1.ToRemoteLinqExpression();
string strexpr1 = SerialiseExpression(remoteExpression1);
try
{
var deserexpr1 = DeserialiseExpression<Remote.Linq.Expressions.LambdaExpression>(strexpr1.NormalizeJsonString());
//what is the equivalent of doing this with a Remote.Linq Expression?
var compiled1 = expr1.Compile();
var result = compiled1.Invoke(testdata);
Assert.IsNotNull(result);
Assert.IsTrue(result.Id == 1);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Assert.Fail("Error deserialising LINQ expression tree");
}
}
How do you invoke a Remote.Linq expression?
A remote linq expression may be converted back into a system linq expression and be compiled and executed as such.
However, this is not what you actually want when sending expressions to a server to query data. On server side you want to use the Execute extension methods to execute your expression against a data source. Make sure to add a using for namespace Remote.Linq.Expressions.
Here's the sample code from Remote.Linq source repo:
using Remote.Linq.Expressions;
public interface IQueryService
{
IEnumerable<DynamicObject> ExecuteQuery(Expression queryExpression);
}
public class QueryService : IQueryService, IDisposable
{
// any linq provider e.g. entity framework, nhibernate, ...
private IDataProvider _datastore = new ObjectRelationalMapper();
// you need to be able to retrieve an IQueryable by type
private Func<Type, IQueryable> _queryableProvider = type => _datastore.GetQueryableByType(type);
public IEnumerable<DynamicObject> ExecuteQuery(Expression queryExpression)
{
// `Execute` is an extension method provided by Remote.Linq
// it applies an expression to a data source and returns the result
return queryExpression.Execute(queryableProvider: _queryableProvider);
}
public void Dispose() => _datastore.Dispose();
}
Also, there are additional nuget packages for expression execution with EF and EF Core, so you can simply provide a DbContext to the Execute method.
In addition you may want to check out the demos/samples found in the project's github repo.
I am using Apache Ignite as the back-end data store in a SpringBoot Application.
I have a requirement where I need to get all the entities whose name matches one of the names from a set of names.
Hence i am trying to get it implemented using a #Query configuration and a method named findAllByName(Iterable<String> names)as below:
Here on the Query, I am trying to use the 'IN' clause and want to pass an array of names as an input to the 'IN' clause.
#RepositoryConfig(cacheName = "Category")
public interface CategoryRepository extends IgniteRepository<Category, Long>
{
List<Category> findByName(String name);
#Query("SELECT * FROM Category WHERE name IN ( ? )")
Iterable<Category> findAllByName(Iterable<String> names); // this method always returns empty list .
}
In this the method findAllByName always returns empty list, even when ignite has Categories for which the name field matches the data passed in the query.
I am unable to figure out if there is a problem with the Syntax or the query of the method signature or the parameters.
Please try using String[] names instead for supplying parameters.
UPDATE: I have just checked the source, and we don't have tests for such scenario. It means that you're on uncharted territory even if it is somehow possible to get to work.
Otherwise looks unsupported currently.
I know your question is more specific to Spring Data Ignite feature. However, as an alternate, you can achieve it using the SqlQuery abstraction of Ignite.
You will form your query like this. I have pasted the sample below with custom sql function inSet that you will write. Also, the below tells how this is used in your sql.
IgniteCache<String, MyRecord> cache = this.ignite
.cache(this.environment.getProperty(Cache.CACHE_NAME));
String sql = "from “my-ignite-cache”.MyRecord WHERE
MyRecord.city=? AND inSet(?, MyRecord.flight)"
SqlQuery<String, MyRecord> sqlQuery = new SqlQuery<>(MyRecord.class,
sql);
sqlQuery.setArgs(MyCity, [Flight1, Flight2 ] );
QueryCursor<Entry<String, MyRecord>> resultCursor = cache.query(sqlQuery);
You can iterate the result cursor to do something meaningful from the extracted data.
resultCursor.forEach(e -> {
MyRecord record = e.getValue();
// do something with result
});
Below is the Ignite Custom Sql function which is used in the above Query - this will help in replicating the IN clause feature.
#QuerySqlFunction
public static boolean inSet(List<String> filterParamArgIds, String id) {
return filterParamArgIds.contains(id);
}
And finally, as a reference MyRecord referred above can be defined something like this.
public class MyRecord implements Serializable {
#QuerySqlField(name = "city", index = true)
private String city;
#QuerySqlField(name = "flight", index = true)
private String flight;
}
I'm getting the following compilation error when I'm using QueryOver with a list of sub queries:
"The type arguments for method 'xxxx' cannot be inferred from the usage. Try specifying the type arguments explicitly."
Here is the code but not sure how I can correct it:
List<QueryOver> subQueries = new List<QueryOver>();
subQueries.Add(QueryOver.Of<Customer>().Where(...));
subQueries.Add(QueryOver.Of<Address>().Where(...));
subQueries.Add(QueryOver.Of<Account>().Where(...));
var query = session.QueryOver<Customer>();
foreach (QueryOver subQuery in subQueries)
{
query.WithSubquery.WhereProperty(c => c.CustomerID)
.In(subQuery); // this is throwing the compilation error
}
var result = query.Select(Projections.RowCount())
.FutureValue<int>()
.Value;
I need to do this programatically as I am generating the subQueries dynamically and don't know how many sub queries there will be. Do I need to use dynamic types or something?
I think you can get around this by rewriting things slightly:
foreach (QueryOver subQuery in subQueries)
{
query.Where(
Restrictions.EqProperty(
Projections.Property<Customer>(c => c.CustomerID),
Projections.SubQuery(subQuery.DetachedCriteria)));
}
I don't know enough about your subqueries to say, but you might be able to use a List<QueryOver<Customer>> instead. The issue is that .WithSubquery...In requires a QueryOver<U> where your list is the non-generic base type (QueryOver).
I've managed to solve this with a little refactoring. Andrew's response gave me the idea to use .Where() (instead of .WithSubquery) and then use a Conjunction for the sub queries. The refactored code looks like this:
Conjunction conj = new Conjunction();
conj.Add(Subqueries.WhereProperty<Customer>(x => x.CustomerID).In(QueryOver.Of<Customer>()...));
conj.Add(Subqueries.WhereProperty<Customer>(x => x.CustomerID).In(QueryOver.Of<AttributeValue>()...));
conj.Add(Subqueries.WhereProperty<Customer>(x => x.CustomerID).In(QueryOver.Of<CustomerListEntry>()...));
ISession session = sf.OpenSession();
using (var tran = session.BeginTransaction())
{
var query = session.QueryOver<Customer>()
.Where(conj)
.Select(Projections.RowCount())
.FutureValue<int>()
.Value;
tran.Commit();
}
I can now programatically build up and selectively apply the subqueries.
I was wondering if there is any way to intercept and modify the sql generated from linq to Sql before the query is sent off?
Basically, we have a record security layer, that given a query like 'select * from records' it will modify the query to be something like 'select * from records WHERE [somesecurityfilter]'
I am trying to find the best way to intercept and modify the sql before its executed by the linq to sql provider.
Ok, first to directly answer your question (but read on for words of caution ;)), there is a way, albeit a finicky one, to do what you want.
// IQueryable<Customer> L2S query definition, db is DataContext (AdventureWorks)
var cs = from c in db.Customers
select c;
// extract command and append your stuff
DbCommand dbc = db.GetCommand(cs);
dbc.CommandText += " WHERE MiddleName = 'M.'";
// modify command and execute letting data context map it to IEnumerable<T>
var result = db.ExecuteQuery<Customer>(dbc.CommandText, new object[] { });
Now, the caveats.
You have to know which query is generated so you would know how to modify it, this prolongs development.
It falls out of L2S framework and thus creates a possible gaping hole for sustainable development, if anyone modifies a Linq it will hurt.
If your Linq causes parameters (has a where or other extension causing a WHERE section to appear with constants) it complicates things, you'll have to extract and pass those parameters to ExecuteQuery
All in all, possible but very troublesome. That being said you should consider using .Where() extension as Yaakov suggested. If you want to centrally controll security on object level using this approach you can create an extension to handle it for you
static class MySecurityExtensions
{
public static IQueryable<Customer> ApplySecurity(this IQueryable<Customer> source)
{
return source.Where(x => x.MiddleName == "M.");
}
}
//...
// now apply it to any Customer query
var cs = (from c in db.Customers select c).ApplySecurity();
so if you modify ApplySecurity it will automatically be applied to all linq queries on Customer object.
If you want to intercept the SQL generated by L2S and fiddle with that, your best option is to create a wrapper classes for SqlConnection, SqlCommand, DbProviderFactory etc. Give a wrapped instance of SqlConnection to the L2S datacontext constructor overload that takes a db connection. In the wrapped connection you can replace the DbProviderFactory with your own custom DbProviderFactory-derived class that returns wrapped versions of SqlCommand etc.
E.g.:
//sample wrapped SqlConnection:
public class MySqlConnectionWrapper : SqlConnection
{
private SqlConnecction _sqlConn = null;
public MySqlConnectionWrapper(string connectString)
{
_sqlConn = new SqlConnection(connectString);
}
public override void Open()
{
_sqlConn.Open();
}
//TODO: override everything else and pass on to _sqlConn...
protected override DbProviderFactory DbProviderFactory
{
//todo: return wrapped provider factory...
}
}
When using:
using (SomeDataContext dc = new SomeDataContext(new MySqlConnectionWrapper("connect strng"))
{
var q = from x in dc.SomeTable select x;
//...etc...
}
That said, do you really want to go down that road? You'll need to be able to parse the SQL statements and queries generated by L2S in order to modify them properly. If you can instead modify the linq queries to append whatever you want to add to them, that is probably a better alternative.
Remember that Linq queries are composable, so you can add 'extras' in a separate method if you have something that you want to add to many queries.
first thing come to my mind is to modify the query and return the result in Non-LINQ format
//Get linq-query as datatable-schema
public DataTable ToDataTable(System.Data.Linq.DataContext ctx, object query)
{
if (query == null)
{
throw new ArgumentNullException("query");
}
IDbCommand cmd = ctx.GetCommand((IQueryable)query);
System.Data.SqlClient.SqlDataAdapter adapter = new System.Data.SqlClient.SqlDataAdapter();
adapter.SelectCommand = (System.Data.SqlClient.SqlCommand)cmd;
DataTable dt = new DataTable("sd");
try
{
cmd.Connection.Open();
adapter.FillSchema(dt, SchemaType.Source);
adapter.Fill(dt);
}
finally
{
cmd.Connection.Close();
}
return dt;
}
try to add your condition to the selectCommand and see if it helps.
Try setting up a view in the DB that applies the security filter to the records as needed, and then when retrieving records through L2S. This will ensure that the records that you need will not be returned.
Alternatively, add a .Where() to the query before it is submitted that will apply the security filter. This will allow you to apply the filter programmatically (in case it needs to change based on the scenario).