How can I recreate this complex SQL Query using NHibernate QueryOver? - nhibernate

Imagine the following (simplified) database layout:
We have many "holiday" records that relate to going to a particular Accommodation on a certain date etc.
I would like to pull from the database the "best" holiday going to each accommodation (i.e. lowest price), given a set of search criteria (e.g. duration, departure airport etc).
There will be multiple records with the same price, so then we need to choose by offer saving (descending), then by departure date ascending.
I can write SQL to do this that looks like this (I'm not saying this is necessarily the most optimal way):
SELECT *
FROM Holiday h1 INNER JOIN (
SELECT h2.HolidayID,
h2.AccommodationID,
ROW_NUMBER() OVER (
PARTITION BY h2.AccommodationID
ORDER BY OfferSaving DESC
) AS RowNum
FROM Holiday h2 INNER JOIN (
SELECT AccommodationID,
MIN(price) as MinPrice
FROM Holiday
WHERE TradeNameID = 58001
/*** Other Criteria Here ***/
GROUP BY AccommodationID
) mp
ON mp.AccommodationID = h2.AccommodationID
AND mp.MinPrice = h2.price
WHERE TradeNameID = 58001
/*** Other Criteria Here ***/
) x on h1.HolidayID = x.HolidayID and x.RowNum = 1
As you can see, this uses a subquery within another subquery.
However, for several reasons my preference would be to achieve this same result in NHibernate.
Ideally, this would be done with QueryOver - the reason being that I build up the search criteria dynamically and this is much easier with QueryOver's fluent interface. (I had started out hoping to use NHibernate Linq, but unfortunately it's not mature enough).
After a lot of effort (being a relative newbie to NHibernate) I was able to re-create the very inner query that fetches all accommodations and their min price.
public IEnumerable<HolidaySearchDataDto> CriteriaFindAccommodationFromPricesForOffers(IEnumerable<IHolidayFilter<PackageHoliday>> filters, int skip, int take, out bool hasMore)
{
IQueryOver<PackageHoliday, PackageHoliday> queryable = NHibernateSession.CurrentFor(NHibernateSession.DefaultFactoryKey).QueryOver<PackageHoliday>();
queryable = queryable.Where(h => h.TradeNameId == website.TradeNameID);
var accommodation = Null<Accommodation>();
var accommodationUnit = Null<AccommodationUnit>();
var dto = Null<HolidaySearchDataDto>();
// Apply search criteria
foreach (var filter in filters)
queryable = filter.ApplyFilter(queryable, accommodationUnit, accommodation);
var query1 = queryable
.JoinQueryOver(h => h.AccommodationUnit, () => accommodationUnit)
.JoinQueryOver(h => h.Accommodation, () => accommodation)
.SelectList(hols => hols
.SelectGroup(() => accommodation.Id).WithAlias(() => dto.AccommodationId)
.SelectMin(h => h.Price).WithAlias(() => dto.Price)
);
var list = query1.OrderByAlias(() => dto.Price).Asc
.Skip(skip).Take(take+1)
.Cacheable().CacheMode(CacheMode.Normal).List<object[]>();
// Cacheing doesn't work this way...
/*.TransformUsing(Transformers.AliasToBean<HolidaySearchDataDto>())
.Cacheable().CacheMode(CacheMode.Normal).List<HolidaySearchDataDto>();*/
hasMore = list.Count() == take;
var dtos = list.Take(take).Select(h => new HolidaySearchDataDto
{
AccommodationId = (string)h[0],
Price = (decimal)h[1],
});
return dtos;
}
So my question is...
Any ideas on how to achieve what I want using QueryOver, or if necessary Criteria API?
I'd prefer not to use HQL but if it is necessary than I'm willing to see how it can be done with that too (it makes it harder (or more messy) to build up the search criteria though).
If this just isn't doable using NHibernate, then I could use a SQL query. In which case, my question is can the SQL be improved/optimised?

I have manage to achieve such dynamic search criterion by using Criteria API's. Problem I ran into was duplicates with inner and outer joins and especially related to sorting and pagination, and I had to resort to using 2 queries, 1st query for restriction and using the result of 1st query as 'in' clause in 2nd creteria.

Related

Entity Framework problem with reducing projection

I've been working on improving performance for our .NET core API with EF 5.0.11 by reducing the projection of our queries, but I'm currently stuck with the following scenario:
I improved the projection of the queries like this:
var employeeEmailQuery = context.Employee
.Where(e => e.Active == true)
.Select(e => new EmployeeEmailView
{
Name = e.FullName,
Email = e.Email
});
This reduces the select query to just the two columns I need instead of a SELECT * on 80+ columns in the database.
In my database, I also have columns with translated descriptions. It looks like this:
What I would like to do is select the relevant translated description, based on the current culture, so I added the following code:
var culture = CultureInfo.DefaultThreadCurrentUICulture;
var employeeEmailQuery = context.Employee
.Where(e => e.Active == true)
.Select(e => new EmployeeEmailView
{
Name = e.FullName,
Email = e.Email,
this.SetDescription(e, culture);
});
The SetDescription method checks the culture and picks the correct column to set a Description property in the EmployeeEmailView. However, by adding this code, the query is now once again doing a SELECT *, which I don't want.
Does anybody have an idea on how to dynamically include a select column using EF without rewriting everything into raw SQL?
Thanks in advance.
I think the only way is to use an Interceptor to modify the query, or dynamically generate the EF IQueryable with Expressions.

Has anyone noticed that EF Core 1.0 2015.1 has made the queries very inefficient

After upgrading to Asp.Net Core 2015.1 I have noticed that a lot of EF queries have become a lot slower to run.
I have done some investigation and found that a lot of the queries with where filters now get evaluated in Code, rather than passing the filters to SQL as part of a where clause to run with the query.
We ended up having to re-write a number of our queries as stored procedures to get back performance. Note these used to be efficient prior to the 2015.1 release. Something obviously was changed, and a lot of queries are doing select all queries on a table and then filtering the data in code. This approach is terrible for performance, e.g. reading a table with lots of rows, to filter everything but maybe 2 rows.
I have to ask what changed, and whether anyone else is seeing the same thing?
For example: I have a ForeignExchange table along with a ForeignExchangeRate table which are linked via ForeignExchangeid = ForeignExchangeRate.ForeignExchangeId
await _context.ForeignExchanges
.Include(x => x.ForeignExchangeRates)
.Select(x => new ForeignExchangeViewModel
{
Id = x.Id,
Code = x.Code,
Name = x.Name,
Symbol = x.Symbol,
CountryId = x.CountryId,
CurrentExchangeRate = x.ForeignExchangeRates
.FirstOrDefault(y => (DateTime.Today >= y.ValidFrom)
&& (y.ValidTo == null || y.ValidTo >= DateTime.Today)).ExchangeRate.ToFxRate(),
HistoricalExchangeRates = x.ForeignExchangeRates
.OrderByDescending(y => y.ValidFrom)
.Select(y => new FxRate
{
ValidFrom = y.ValidFrom,
ValidTo = y.ValidTo,
ExchangeRate = y.ExchangeRate.ToFxRate(),
}).ToList()
})
.FirstOrDefaultAsync(x => x.Id == id);
And I use this to get the data for editing a foreign exchange rate
So the SQL generated is not as expected. It generates the following 2 SQL statements to get the data
SELECT TOP(1) [x].[ForeignExchangeId], [x].[ForeignCurrencyCode], [x].[CurrencyName], [x].[CurrencySymbol], [x].[CountryId], (
SELECT TOP(1) [y].[ExchangeRate]
FROM [ForeignExchangeRate] AS [y]
WHERE ((#__Today_0 >= [y].[ValidFrom]) AND ([y].[ValidTo] IS NULL OR ([y]. [ValidTo] >= #__Today_1))) AND ([x].[ForeignExchangeId] = [y].[ForeignExchangeId])
)FROM [ForeignExchange] AS [x]
WHERE [x].[ForeignExchangeId] = #__id_2
and
SELECT [y0].[ForeignExchangeId], [y0].[ValidFrom], [y0].[ValidTo], [y0].[ExchangeRate]
FROM [ForeignExchangeRate] AS [y0]
ORDER BY [y0].[ValidFrom] DESC
The second query is the one that causes the slowness. If the table has many rows, then it essentially gets the whole table and filters the data in code
This has changed in the latest release as this used to work in the RC versions of EF
One other query I used to have was the following
return await _context.CatchPlans
.Where(x => x.FishReceiverId == fishReceiverId
&& x.FisherId == fisherId
&& x.StockId == stockId
&& x.SeasonStartDate == seasonStartDate
&& x.EffectiveDate >= asAtDate
&& x.BudgetType < BudgetType.NonQuotaed)
.OrderBy(x => x.Priority)
.ThenBy(x => x.BudgetType)
.ToListAsync();
and this query ended up doing a Table read (the entire table which was in the tens of thousands of rows) to get a filter subset of between 2 and 10 records. Very inefficient. This was one query I had to replace with a stored procedure. Reduced from approx 1.5-3.0 seconds down to milliseconds. And note this used to run efficiently before the upgrade
This is a known issue on EF core 1.0.The solution right now is to convert all your critical queries to sync one.The problem is on Async queries right now.They'll sort out this issue on EF core 1.1.0 version.But it has not been released yet.
Here is the Test done by the EF core dev team member :
You can find out more info here :EF Core 1.0 RC2: async queries so much slower than sync
Another suggestion I would like to do.That is try your queries with .AsNoTracking().That too will improve the query performance.
.AsNoTracking()
Sometimes you may want to get entities back from a query but not have
those entities be tracked by the context. This may result in better
performance when querying for large numbers of entities in read-only
scenarios.

nhibernate querying over other query results

Hello i'm trying to get all users which have had payments at least 6 months over the given period (which must be a year). I've written SQL which works fine, but i have difficulties trying to convert it to nhibernate.
SQL:
SELECT COUNT(UserId) AS paidMonthsCount, UserId FROM (
SELECT DISTINCT UserId,
YEAR(PayDate) as _year,
MONTH(PayDate) as _month
FROM Payments
WHERE PayDate >= '2014-04-02T00:00:00' AND PayDate < '2015-04-02T23:59:00'
)result GROUP BY result.UserId
i have converted inner SQL:
var subQuery = Session.QueryOver(() => paymentAlias)
.SelectList(list => list
.Select(Projections.Distinct(Projections.Property<VelferdPayment>(p => p.Client.Id)).WithAlias(() => userWithHelp.Id))
.Select(p => p.AssignmentYear).WithAlias(() => userWithHelp.AssignmentDate)
)
.WhereRestrictionOn(p => p.AssignmentDate)
.IsBetween(parameters.FromDate)
.And(parameters.ToDate);
which selects the distinct part and i have the other part which is selecting from result:
var query = Session.QueryOver(() => userWithHelp).
SelectList(list => list
.SelectCount(p=> p.Id).WithAlias(()=> userWithHelpCount.Count)
.SelectGroup(p => p.Id).WithAlias(() => userWithHelpCount.Id)
)
.TransformUsing(Transformers.AliasToBean<UserWithHelpCount>())
.List<UserWithHelpCount>();
How can i queryover the subQuery results or is it possible to write single request to SQL. Working for a long time please help.
In general, with NHibernate we can only (or mainly) query Entities, not TABLES. Other words, we firstly map tables or views or even some <subselect>s into entities. The below mapping of the User (C# object User)
<class name="user" table="[dbo].[user_table]" ...
Will allow us to create query over C# User.
session.QueryOver<User>()...
Behind the scene it will generate FROM clause, which will contain the content of table attribute, i.e. FROM [dbo].[user_table]
That's it. There is no other way how to set the generated FROM clause. Just by mapping.
But there is a way which allow us to use existing ADO.NET connection to create custom query and even convert its result to some entity, or DTO. It is CreateSQLQuery() API:
17.1.5. Returning non-managed entities
It is possible to apply an IResultTransformer to native sql queries. Allowing it to e.g. return non-managed entities.
sess.CreateSQLQuery("SELECT NAME, BIRTHDATE FROM CATS")
.SetResultTransformer(Transformers.AliasToBean(typeof(CatDTO)))
This query specified:
the SQL query string
a result transformer
The above query will return a list of CatDTO which has been instantiated and injected the values of NAME and BIRTHNAME into its corresponding properties or fields.
So, we can use native SQL SELECT statements to get any results. We can even create some custom DTO and let NHibernate to transform result into them...

Using a subquery for a column with QueryOver

I'm trying to get something similar to the SQL below via QueryOver:
SELECT
docs.*,
(SELECT TOP 1 eventDate from events WHERE id=docs.id
AND type=4 ORDER BY eventDate DESC) as eventDate
FROM documents as docs
WHERE doc.accountId = ...
I've got close with a projection, however I'm not sure how to get the entire documents table back. Documents has a one-to-many relationship with Events, I don't want to outer join as it will bring multiple results, and an inner join may not bring back a row:
var query = QueryOver<Document>
.Where(d => d.Account == account)
.SelectList(list => list
.Select(d => d)
.Select(d => d.Events.OrderByDescending(e => e.EventDate).FirstOrDefault(e => e.Type == 4))
)
.List<object[]>()
.Select(d => return new DocumentSummary(d[0],d[1]) etc.);
Is there an easier way of performing subqueries for columns? I'm reluctant to replace this with the property performing a query in its get.
After some research it looks like HQL (which QueryOver is converted into) does not support TOP inside subqueries.
My solution: create a view which includes the computed properties, and then mark these properties in the mappings files as insert="false" and update="false"

NHibernate Nested query in FROM clause

I’m having a problem with translating T-SQL query into Nhibernate query – or writing query that will return same results but will be properly written in NH query language (HQL, Criteria, QueryOver, LINQ – I don’t truly care).
I would like to execute similar query from NHibernate:
SELECT
lic.RegNo,
lic.ReviewStatus
FROM
Licence lic
WHERE
lic.RegNo IN
(
SELECT
grouped.RegNo
FROM
(
SELECT
g.[Type],
g.Number,
MAX(g.Iteration) AS [Iteration],
MAX(g.RegNo) AS [RegNo]
FROM
Licence g
GROUP BY
g.[Type],
g.Number
) as grouped
)
ORDER BY
lic.RegNo desc
It returns the top most licenses and fetch their review status if exists. RegNo is created from Type, Number and Iteration (pattern: {0}{1:0000}-{2:00}). Each license can have multiply iterations and some of them can contains ReviewStatus, for instance:
W0004-01 NULL
W0001-03 1
P0004-02 3
P0001-02 4
If iteration part is greater than 1 it means that there are multiply iterations (n) for specific licence.
I’ve manage to create NH query by going twice to database:
LicenceInfoViewModel c = null;
var grouped = session.QueryOver<Licence>()
.SelectList(l => l
.SelectGroup(x => x.Type)
.SelectGroup(x => x.Number)
.SelectMax(x => x.Iteration)
.SelectMax(x => x.RegNo).WithAlias(() => c.RegNo)
).TransformUsing(Transformers.AliasToBean<LicenceInfoViewModel>())
.Future<LicenceInfoViewModel>();
var proper = session.QueryOver<Licence>()
.Select(x => x.RegNo, x => x.ReviewStatus)
.WhereRestrictionOn(x => x.RegNo)
.IsIn(grouped.Select(x => x.RegNo).ToArray())
.TransformUsing(Transformers.AliasToBean<LicenceInfoViewModel>())
.List<LicenceInfoViewModel>();
// ...
public class LicenceInfoViewModel
{
public string RegNo { get; set; }
public LicReviewStatus? ReviewStatus { get; set; }
}
public enum LicReviewStatus
{
InProgress,
Submitted,
Validated,
RequestForInformation,
DecissionIssued
}
However this solution is not good as it require to download all grouped licences from database, and there could be a thousands of them.
Is there a better way to write this query or is there a way to translate provided above T-SQL query into NHibernate?
adding nhibernate and hibernate tags as IMO if this can be done in hibernate it should be easily translated into nh
I don't think that SQL does what you think it does. Iteration is not used for anything.
In any case, it seems unnecessary. You can change the WHERE to the following, and you'll have both valid SQL and HQL:
lic.RegNo IN
(
SELECT
MAX(g.RegNo)
FROM
Licence g
GROUP BY
g.Type,
g.Number
)