NHibernate Linq Query with Projection and Count error - nhibernate

I have the following query:
var list = repositoy.Query<MyClass>.Select(domain => new MyDto()
{
Id = domain.Id,
StringComma = string.Join(",", domain.MyList.Select(y => y.Name))
});
That works great:
list.ToList();
But if I try to get the Count I got an exception:
list.Count();
Exception
NHibernate.Hql.Ast.ANTLR.QuerySyntaxException
A recognition error occurred. [.Count[MyDto](.Select[MyClass,MyDto](NHibernate.Linq.NhQueryable`1[MyClass], Quote((domain, ) => (new MyDto()domain.Iddomain.Name.Join(p1, .Select[MyListClass,System.String](domain.MyList, (y, ) => (y.Name), ), ))), ), )]
Any idea how to fix that without using ToList ?

The point is, that we should NOT call Count() over projection. So this will work
var query = repositoy.Query<MyClass>;
var list = query.Select(domain => new MyDto()
{
Id = domain.Id,
StringComma = string.Join(",", domain.MyList.Select(y => y.Name))
});
var count = query.Count();
When we use ICriteria query, the proper syntax would be
var criteria = ... // criteria, with WHERE, SELECT, ORDER BY...
// HERE cleaned up, just to contain WHERE clause
var totalCountCriteria = CriteriaTransformer.TransformToRowCount(criteria);
So, for Count - use the most simple query, i.e. containing the same JOINs and WHERE part

If you really don't need the results, but only the count, then you shouldn't even bother writing the .Select() clause. Radim's answer as posted is a good way to both get the results and the count, but if your database supports it, use future queries to execute both in the same roundtrip to the database:
var query = repository.Query<MyClass>;
var list = query.Select(domain => new MyDto()
{
Id = domain.Id,
StringComma = string.Join(",", domain.MyList.Select(y => y.Name))
}).ToFuture();
var countFuture = query.Count().ToFutureValue();
int actualCount = countFuture.Value; //queries are actually executed here
Note that there in NH prior to 3.3.3, this would still execute two round-trips (see https://nhibernate.jira.com/browse/NH-3184), but it would work, and if you ever upgrade NH, you get a (minor) performance boost.

Related

CrossDB queries with EFCore = no / instantiating context entites as async lists = works - What is the true approach should I be taking?

I have a system with two database servers I am working with:
One of them is database first - a database managed by a legacy enterprise application and I don't have full control over changing the database structure.
The second is code first and I have full control in the code first database to make changes.
Security policies prevent me from making a view that joins tables from the two database servers in the code first DB which might be a way to make this better according to what i've seen on SO posts.
I have one context for each database since they are separate.
The data and structure in the code first tables is designed to be able to join to the non-code first database as if they were all in one database.
I CAN get what I need working using this set of queries:
// Set up EF tables
var person = await _context1.Person.ToListAsync();
var labor = await _context1.Labor.ToListAsync();
var laborCraftRate = await context1.LaborCraftRate.ToListAsync();
var webUsers = await context2.WebUsers.ToListAsync();
var workOrders = await _context1.Workorder
.Where(r => r.Status == "LAPPR" || r.Status == "APPR" || r.Status == "REC")
.ToListAsync();
var specialRequests = await _context1.SwSpecialRequest
.Where(r => r.Requestdate > DateTime.Now)
.ToListAsync();
var distributionListQuery = (
from l in labor
from p in person.Where(p => p.Personid == l.Laborcode).DefaultIfEmpty()
from wu in webUsers.Where(wu => wu.Laborcode == l.Laborcode).DefaultIfEmpty()
from lcr in laborCraftRate.Where(lcr => lcr.Laborcode == l.Laborcode).DefaultIfEmpty()
select new
{
Laborcode = l.Laborcode,
Displayname = p.Displayname,
Craft = lcr.Craft,
Crew = l.Crewid,
Active = wu.Active,
Admin = wu.FrIsAdmin,
FrDistLocation = wu.FrDistLocation,
}).Where(r => r.Active == "Y" && (r.FrDistLocation == "IPC" || r.FrDistLocation == "IPC2" || r.FrDistLocation == "both"))
.OrderBy(r => r.Craft)
.ThenBy(r => r.Displayname);
// Build a subquery for the next query to use
var ptoSubQuery =
from webUser in webUsers
join workOrder in workOrders on webUser.Laborcode equals workOrder.Wolablnk
join specialRequest in specialRequests on workOrder.Wonum equals specialRequest.Wonum
select new
{
workOrder.Wonum,
Laborcode = workOrder.Wolablnk,
specialRequest.Requestdate
};
// Build the PTO query to join with the distribution list
var ptoQuery =
from a in ptoSubQuery
group a by a.Wonum into g
select new
{
Wonum = g.Key,
StartDate = g.Min(x => x.Requestdate),
EndDate = g.Max(x => x.Requestdate),
Laborcode = g.Min(x => x.Laborcode)
};
// Join the distribution list and the object list to return
// list items with PTO information
var joinedQuery = from dl in distributionListQuery
join fl in ptoQuery on dl.Laborcode equals fl.Laborcode
select new
{
dl.Laborcode,
dl.Displayname,
dl.Craft,
dl.Crew,
dl.Active,
dl.Admin,
dl.FrDistLocation,
fl.StartDate,
fl.EndDate
};
// There are multiple records that result from the join,
// strip out all but the first instance of PTO for all users
var distributionList = joinedQuery.GroupBy(r => r.Laborcode)
.Select(r => r.FirstOrDefault()).OrderByDescending(r => r.Laborcode).ToList();
Again, this works and gets my data back in a reasonable but clearly not optimal timeframe that I can work with in my UI that needs this by preloading the data before it is needed. Not the best, but works.
If I change the variable declarations to not be async which I was told I should do in another SO post, this turns into a cross db query and netcore says no:
// Set up EF tables
var person = _context1.Person;
var labor = _context1.Labor;
var laborCraftRate = context1.LaborCraftRate;
var webUsers = context2.WebUsers;
var workOrders = _context1.Workorder
.Where(r => r.Status == "LAPPR" || r.Status == "APPR" || r.Status == "REC");
var specialRequests = _context1.SwSpecialRequest
.Where(r => r.Requestdate > DateTime.Now);
Adding ToListAsync() is what allows the join functionality I need to work.
Q - Can anyone elaborate on possible downsides and problems with what I am doing?
Thank you for helping me understand!
It's not that calling ToList() "doesn't work." The problem is that it materializes (I think that's the right word) the query and returns a potentially larger than intended amount of data to the client. Any further LINQ operations are done on the client side. This can increase the load on the database and network. In your case, it works because you're bringing all that data to the client side. At that point, it no longer matters that it was a cross-database query.
This was a frequent concern during the transition from .NET Core 2.x to 3.x. If an operation could not be performed server side, .NET Core 2.x would silently insert something like ToList(). (Well, not completely silently. I think it was logged somewhere. But many developers weren't aware of it.) 3.x stopped doing that and would give you an error. When people tried to upgrade to 3.x, they often found it difficult to convert the queries into something that could run server side. And people resisted throwing in an explicit ToList() because muh performance. But remember, that's what it was always doing. If there wasn't a performance issue before, there isn't one now. And at least now you're aware of what it's actually doing, and can fix it if you really need to.

Breeze - Writing complex query with multiple parameters from client

I am trying to write a Breeze query for the Northwind database to show orders that contain all three products that a user specifies. So if a user selects ProductID 41, 51, and 65 from drop downs, the query would return order id 10250.
This is just a sample scenario that I am looking to base another query on in a project I am working on, but I thought using Northwind to explain it would be easier than describing the project. I can easily do it in T-SQL using derived tables, but I need to get the parameters from the client. Any thoughts? Thanks in advance!
If you're still interested in doing this on the client, you can try the following Breeze query.
var listOfProductIds = [41, 51, 65];
var preds = listOfProductIds.map(function (id) {
return Predicate.create("OrderDetails", "any", "ProductID", "==", id);
});
var whereClause = Predicate.and(preds);
var query = EntityQuery.from('Orders').where(whereClause);
This will retrieve all Orders that have at least all 3 of the Products specified.
To further filter this so you have all Orders that have only all 3 of the Products specified, you can write,
entityManager.executeQuery(query)
.then(filterOrders);
//once you get results on the client
function filterOrders(data) {
var allOrders = data.results;
var filteredOrders = allOrders.filter(function (o) {
return o.OrderDetails.length == listOfProductIds.length;
});
}
You can only filter on the client since OData doesn't yet support the Aggregate Count function like Linq to Entities does. This is probably not ideal but it's an option if you decide not to do it on the server.
Sorry in my comment I said look at the and() method but in actuality for your use you needed to look at the or() method -
var p1 = breeze.Predicate("Product.ProductID", "==", 41);
var p2 = breeze.Predicate("Product.ProductID", "==", 51);
var p3 = breeze.Predicate("Product.ProductID", "==", 65);
var newPred = breeze.Predicate.or(p1, p2, p3);
var query = EntityQuery.from('Order_Details').select('Order.OrderID').where(newPred);
The only issue with these queries on the client is that depending on how you are building them client-side and how many predicates you are adding they can get very long in some situations, such as select all 200 records except 1 id type queries which can bite you on IE8.
You can use the EntityQuery.withParameters method to call a server side query that has your linq query. i.e. something like this.
// On the client
var query = EntityQuery.from("GetOrdersWithProductIds")
.withParameters({ productIds: [41, 51, 65] });
// On the server
[HttpGet]
public IQueryable<Order> GetOrdersWithProductIds([FromUri] int[] productIds) {
return ContextProvider.Context.Orders.Where(... your linq query here ...);
}
or you might try using the support for 'any'/'all' query operators. i.e. something like this.
var predicate = breeze.Predicate("ProductID", "==", 41)
.or("ProductID", "==", 51)
.or("ProductID", "==", 65);
var query = EntityQuery.from("Orders")
.where("OrderDetails", "all", predicate)

RavenDB : Use "Search" only if given within one query

I have a situation where my user is presented with a grid, and it, by default, will just get the first 15 results. However they may type in a name and search for an item across all pages.
Alone, either of these works fine, but I am trying to figure out how to make them work as a single query. This is basically what it looks like...
// find a filter if the user is searching
var filters = request.Filters.ToFilters();
// determine the name to search by
var name = filters.Count > 0 ? filters[0] : null;
// we need to be able to catch some query statistics to make sure that the
// grid view is complete and accurate
RavenQueryStatistics statistics;
// try to query the items listing as quickly as we can, getting only the
// page we want out of it
var items = RavenSession
.Query<Models.Items.Item>()
.Statistics(out statistics) // output our query statistics
.Search(n => n.Name, name)
.Skip((request.Page - 1) * request.PageSize)
.Take(request.PageSize)
.ToArray();
// we need to store the total results so that we can keep the grid up to date
var totalResults = statistics.TotalResults;
return Json(new { data = items, total = totalResults }, JsonRequestBehavior.AllowGet);
The problem is that if no name is given, it does not return anything; Which is not the desired result. (Searching by 'null' doesn't work, obviously.)
Do something like this:
var q= RavenSession
.Query<Models.Items.Item>()
.Statistics(out statistics); // output our query statistics
if(string.IsNullOrEmpty(name) == false)
q = q.Search(n => n.Name, name);
var items = q.Skip((request.Page - 1) * request.PageSize)
.Take(request.PageSize)
.ToArray();

NHibernate - Linq query using COUNT(DISTINCT)

I'm trying to get a paged query to work properly using LINQ and NHibernate. On a single table, this works perfect, but when more than one table is joined, it's causing me havoc. Here is what I have so far.
public virtual PagedList<Provider> GetPagedProviders(int startIndex, int count, System.Linq.Expressions.Expression<Func<Record, bool>> predicate) {
var firstResult = startIndex == 1 ? 0 : (startIndex - 1) * count;
var query = (from p in session.Query<Record>()
.Where(predicate)
select p.Provider);
var rowCount = query.Select(x => x.Id).Distinct().Count();
var pageOfItems = query.Distinct().Skip(firstResult).Take(count).ToList<Provider>();
return new PagedList<Provider>(pageOfItems, startIndex, count, rowCount);
}
There are a couple issues I'm having with this. First off, the rowCount variable is pulling back a COUNT(*) on a join that has one to many. So my count is way off of what is should be, I'm expecting 129, but getting back over 1K.
The second issue I'm happening is with the results. If I'm on the first page (firstResult = 0), then I'm getting correct results. However, if firstResult is something other than 0, it produces completely different SQL. Below is the SQL generated by both scenarios, I've trimmed out some of the fat to make it a little more readable.
--firstResult = 0
select distinct TOP (30) provider1_.Id as Id12_, provider1_.TaxID as TaxID12_,
provider1_.Facility_Name as Facility5_12_, provider1_.Last_name as Last6_12_,
provider1_.First_name as First7_12_, provider1_.MI as MI12_
from PPORecords record0_ left outer join PPOProviders provider1_ on record0_.Provider_id=provider1_.Id,
PPOProviders provider2_ where record0_.Provider_id=provider2_.Id
and provider2_.TaxID='000000000'
--firstResult = 30
SELECT TOP (30) Id12_, TaxID12_, Facility5_12_, Last6_12_, First7_12_, MI12_,
FROM (select distinct provider1_.Id as Id12_, provider1_.TaxID as TaxID12_,
provider1_.Facility_Name as Facility5_12_,
provider1_.Last_name as Last6_12_,
provider1_.First_name as First7_12_,
provider1_.MI as MI12_,
ROW_NUMBER() OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
from PPORecords record0_ left outer join PPOProviders provider1_ on record0_.Provider_id=provider1_.Id,
PPOProviders provider2_
where record0_.Provider_id=provider2_.Id and provider2_.TaxID='000000000') as query
WHERE query.__hibernate_sort_row > 30
ORDER BY query.__hibernate_sort_row
The problem with the second query is the "distinct" keyword is not present on the outer query, only the inner query. Any ideas how to correct this?
Thanks for any suggestions!
[UPDATE]
This is the code that is building the the Linq query predicate.
private Expression<Func<Record, bool>> ParseQueryExpression(string Query) {
Expression<Func<Record, bool>> mExpression = x => true;
string[] splitQuery = Query.Split('|');
foreach (string query in splitQuery) {
if (string.IsNullOrEmpty(query))
continue;
int valStartIndex = query.IndexOf('(');
string variable = query.Substring(0, valStartIndex);
string value = query.Substring(valStartIndex + 1, query.IndexOf(')') - valStartIndex - 1);
switch (variable) {
case "tax":
mExpression = x => x.Provider.TaxID == value;
break;
case "net":
mExpression = Combine<Record>(mExpression, x => x.Network.Id == int.Parse(value));
break;
case "con":
mExpression = Combine<Record>(mExpression, x => x.Contract.Id == int.Parse(value));
break;
case "eff":
mExpression = Combine<Record>(mExpression, x => x.Effective_Date >= DateTime.Parse(value));
break;
case "trm":
mExpression = Combine<Record>(mExpression, x => x.Term_Date <= DateTime.Parse(value));
break;
case "pid":
mExpression = Combine<Record>(mExpression, x => x.Provider.Id == long.Parse(value));
break;
case "rid":
mExpression = Combine<Record>(mExpression, x => x.Rate.Id == int.Parse(value));
break;
}
}
return mExpression;
}
It seems as if the Linq provider has some "unexpected" behaviour there. I could reproduce the issue with the rowCount that you are describing and also encountered some issues when trying to fix it with a subquery.
If I understand your intentions correctly you basically want to have a paged list of Providers whose Records match certain criteria.
In that case I would suggest using a subquery. However, I tried implementing the subquery with the Query() method but it did not work. So I tried the QueryOver() method which worked flawlessly. In your case the desired queries would look like this:
Provider pAlias = null;
var query = session.QueryOver<Provider>(() => pAlias).WithSubquery
.WhereExists(QueryOver.Of<Record>()
.Where(predicate)
.And(r => r.Provider.Id == pAlias.Id)
.Select(r => r.Provider));
var rowCount = query.RowCount();
var pageOfItems = query.Skip(firstResult).Take(count).List<Provider>();
That way you do not have to struggle with the Distinct(). And as much as I personally like and use NHibernate.Linq, I suppose, if it does not work in a given situation, one should use something else that works.
Edit: Splitting the query into smaller units.
// the method ParseQueryExpression() does not need to be modified, the Expression should work like that
System.Linq.Expressions.Expression<Func<Record, bool>> predicate = x => x.Provider.TaxID == "000000000";
// Query to evaluate predicate
IQueryable<Record> queryId = session.Query<Record>().Where(predicate).Select(r => r.Provider);
// extracting the IDs to use in a subquery
List<int> idList = queryId.Select(p => p.Id).Distinct().ToList();
// total count of distinct Providers
int rowCount = idList.Count;
// new Query on Provider, using the distinct Ids
var query = session.Query<Provider>().Where(p => idList.Contains(p.Id));
// the List<Provider> to display on the page
var pageOfItems = query.Skip(firstResult).Take(count).ToList();
The problem here is that you have multiple queries and therefore multiple roundtrips to the db. Another problem apperas when you examine the generated sql. The subquery idList.Contains(p.Id) will generate an in-clause with a lot of parameters.
As you can see, these are NHibernate.Linq queries. That is because the new QueryOver feature is not a true Linq provider and does not work well with dynamic Linq expressions.
You could get around that limitation by using Detached QueryOvers. The drawback here is that you would have to modify your ParseQueryExpression() somehow, e.g. like that:
private QueryOver<Record> ParseQueryExpression(string Query)
{
Record rAlias = null;
var detachedQueryOver = QueryOver.Of<Record>(() => rAlias)
.JoinQueryOver(r => r.Provider)
.Where(() => rAlias.Provider.TaxID == "000000000")
.Select(x => x.Id);
// modify your method to match the return value
return detachedQueryOver;
}
// in your main method
var detachedQueryOver = QueryOver.Of<Record>()
.WithSubquery
.WhereProperty(r => r.Id
.In(this.ParseQueryExpression(...));
var queryOverList = session.QueryOver<Provider>()
.WithSubquery
.WhereProperty(x => x.Id)
.In(detachedQueryOver.Select(r => r.Provider.Id));
int rowCount = queryOverList.RowCount();
var pageOfItems = queryOverList.Skip(firstResult).Take(count).List();
Well, I hope you are not too confused by that.

LINQ: Count number of true booleans in multiple columns

I'm using LINQ to SQL to speed up delivery of a project, which it's really helping with. However I'm struggling with a few things I'm used to doing with manual SQL.
I have a LINQ collection containing three columns, each containing a boolean value representing whether an e-mail, mobile or address is availble.
I want to write a LINQ query to give me an count of trues for each column, so how many rows in the e-mail column are set to true (and the same for the other two columns)
If you need a single object containing the results:
var result = new {
HasEmailCount = list.Count(x => x.HasEmail),
HasMobileCount = list.Count(x => x.HasMobile),
HasAddressCount = list.Count(x => x.HasAddress)
};
Or using the aggregate function:
class Result
{
public int HasEmail;
public int HasAddress;
public int HasMobile;
}
var x = data.Aggregate(
new Result(),
(res, next) => {
res.HasEmail += (next.HasEmail ? 0 : 1);
res.HasAddress += (next.HasAddress ? 0 : 1);
res.HasMobile += (next.HasMobile ? 0 : 1);
return res;
}
);
x is of Type Result and contains the aggregated information. This can also be used for more compelx aggregations.
var mobileCount = myTable.Count(user => user.MobileAvailable);
And so on for the other counts.
You can do it like so:
var emailCount = yourDataContext.YourTable.Count(r => r.HasEmail);
etc.