Performance issue with SQL server due to sql query - sql

Need some help to solve this error:
The query processor ran out of internal resources and could not
produce a query plan. This is a rare event and only expected for
extremely complex queries or queries that reference a very large
number of tables or partitions. Please simplify the query. If you
believe you have received this message in error, contact Customer
Support Services for more information.
SQL query: (I am just putting one of the query. I got 9 such queries running)
var query1 = from article in _db.Articles
from scl in article.Scls
where article.publishDate < DateTime.Now
&& article.removalDate > DateTime.Now
&& article.finished == true
&& article.flagged== true
&& listOfScl.Contains(scl.id)
select article;
var query2 = from article in _db.Articles
from com in article.Coms
where article.publishDate < DateTime.Now
&& article.removalDate > DateTime.Now
&& article.finished == true
&& article.flagged== true
&& listOfCom.Contains(com.id)
select article;
query = (query1.Union(query2)).Distinct();
Don't know what is wrong with query its working fine but giving some performance issue on SQL Server.
I need some help to avoid such error and re-write this SQL query without using UNION.
Thanks in advance.

Also note that by default LINQ does UNION, which will eliminate duplicates, therefore there is no need for Distinct, which will likely improve performance.
"re-write this SQL query without using UNION.". How is this?
var query1 = from article in _db.Articles
from scl in article.Scls
where article.publishDate < DateTime.Now
&& article.removalDate > DateTime.Now
&& article.finished == true
&& article.flagged== true
&& (
listOfScl.Contains(scl.id)
|| listOfCom.Contains(com.id)
)
select article;
You could try the performance of this, so that you don't need a distinct. It will involve a subquery though, so it's hard to say what will perform better without testing:
var query1 = _db.Articles.Where(article => article.publishDate < DateTime.Now
&& article.removalDate > DateTime.Now
&& article.finished == true
&& article.flagged== true
&& ( article.Scls.Any(s=> listOfScl.Contains(s.id))
|| article.Coms.Any(c=> listOfCom.Contains(c.id))
)
);

Related

Different performance from SQL Server query from Management Studio vs EF Core 5

I wrote a simple EF Core query that makes a select on a table using some where clause to filter data: start date and finish date between the actual date and a field (DescrizioneCommessa) containing a value.
var query = _ctx.Commessas
.Where(x => (x.DataInizioCommessa.HasValue && x.DataInizioCommessa <= DateTime.Now) || !x.DataInizioCommessa.HasValue)
.Where(x => (x.DataFineCommessa.HasValue && x.DataFineCommessa >= DateTime.Now) || !x.DataFineCommessa.HasValue)
.Where(x => x.DescrizioneCommessa.Contains(pattern))
.OrderBy(x => x.DescrizioneCommessa);
To get the raw SQL I just execute the statement:
var sql = facis.ToQueryString();
And the resultant query is:
DECLARE #__pattern_0 nvarchar(50) = N'COMUNE';
SELECT *
FROM [Commessa] AS [c]
WHERE (([c].[DataInizioCommessa] IS NOT NULL AND [c].[DataInizioCommessa] <= GETDATE()) OR [c].[DataInizioCommessa] IS NULL)
AND (([c].[DataFineCommessa] IS NOT NULL AND ([c].[DataFineCommessa] >= GETDATE())) OR [c].[DataFineCommessa] IS NULL)
AND ((#__pattern_0 LIKE N'') OR (CHARINDEX(#__pattern_0, [c].[DescrizioneCommessa]) > 0))
ORDER BY [c].[DescrizioneCommessa]
I notice that it takes very long to perform the query comparing to its hand-written version:
SELECT *
FROM Commessa
WHERE (DescrizioneCommessa LIKE '%COMUNE%')
AND (DataInizioCommessa <= GETDATE() OR DataInizioCommessa IS NULL)
AND (DataFineCommessa >= GETDATE() OR DataFineCommessa IS NULL);
EF Query takes even more than one minute to elaborate, while the normal one is immediate.
I verified that the problem is this part of where clause:
AND ((#__pattern_0 LIKE N'') OR (CHARINDEX(#__pattern_0, [c].[DescrizioneCommessa]) > 0))
If I substitute the above line with:
AND (DescrizioneCommessa LIKE '%COMUNE%')
the problem is resolved, the performance is optimal.
Why this line
.Where(x => x.DescrizioneCommessa.Contains(pattern))
creates this issue?
This is documented behaviour in EF.Core as discussed on SO: Entity framework EF.Functions.Like vs string.Contains
As a general proposition, LIKE expressions do NOT work well with query optimisers. In larger datasets this becomes a serious problem, even though they work just fine in much smaller unoptimized sets. The optimisation is heavily dependent on the pattern being matched and if it is a ranged lookup or not. In your case the pattern cannot make use indexes, EF is simply trying to convert it into an expression that might be indexable, in which case after running the expression enough the rest of the database insights engines would advise you to implement an appropriate index.
Read about some other discussions about parsing String.Contains() to SQL in git hub: https://github.com/dotnet/efcore/issues/474
When you explicitly want to use SQL LIKE, EF Core added EF.Functions.Like():
Like(DbFunctions, String, String)
Like-operator in Entity Framework Core 2.0
var query = _ctx.Commessas
.Where(x => (x.DataInizioCommessa.HasValue && x.DataInizioCommessa <= DateTime.Now) || !x.DataInizioCommessa.HasValue)
.Where(x => (x.DataFineCommessa.HasValue && x.DataFineCommessa >= DateTime.Now) || !x.DataFineCommessa.HasValue)
.Where(x => EF.Functions.Like(x.DescrizioneCommessa, pattern)
.OrderBy(x => x.DescrizioneCommessa);

How to get the latest/last record with a group by clause with NHibernate Linq provider

I have used too much time (days) on this and I really hope someone can help me out.
I found a good article on describing my problem in a generic way so let's stick to it.
I am trying to build this query but NHibernate fails to build the correct sql and returns a sql query exception.
Column vSagsAendring.Id is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause. It could not execute the following query:
select
viewsagsae0_.Id as Id155_,
viewsagsae0_.SagId as SagId155_,
viewsagsae0_.JournalNr as JournalNr155_,
viewsagsae0_.LbfNr as LbfNr155_,
viewsagsae0_.OrgNr as OrgNr155_,
viewsagsae0_.OrgNavn as OrgNavn155_,
viewsagsae0_.AfdNavn as AfdNavn155_,
viewsagsae0_.SagsType as SagsType155_,
viewsagsae0_.Status as Status155_,
viewsagsae0_.SagsbehandlerInit as Sagsbeh10_155_,
viewsagsae0_.Dato as Dato155_,
viewsagsae0_.JournalAktionType as Journal12_155_,
viewsagsae0_.Beskrivelse as Beskriv13_155_,
viewsagsae0_.Ekstern as Ekstern155_
from vSagsAendring viewsagsae0_
group by viewsagsae0_.SagId
var query = from p in _session.Query<ViewSagsAendring>()
group p by p.SagId
into grp
select grp.OrderByDescending(g => g.Dato).First();
This is another version also took from the article:
var query = from p in _session.Query<ViewSagsAendring>()
group p by p.SagId
into grp
let maxDato = grp.Max(g => g.Dato)
from p in grp
where p.Dato == maxDato
select p;
It's have been a long journey, but now it's over. I hope that I can help someone else in the same situation by answering my own question.
var aendring = from sagsAendring in _session.Query<ViewSagsAendring>()
where sagsAendring.Dato ==
(
from innersagsAendring in _session.Query<ViewSagsAendring>()
where innersagsAendring.SagId == sagsAendring.SagId
select innersagsAendring.Dato
).Max()
select sagsAendring;
var result = aendring.ToList();
And because you can chain linq statements you can build a linq filter like this
if(Filters.VisInterneAendringer == false)
query = query.Where(x => x.Ekstern == true);
if (Filters.VisKunNyesteAendringer)
{
query = query.Where(sagsAendring => sagsAendring.Dato ==
(
from innerSagsAendring in Session.Query<ViewSagsAendring>() where innerSagsAendring.SagId == sagsAendring.SagId
select innerSagsAendring.Dato
).Max());
}
return query;
Your queries seem legit for LINQ in EntityFramework.
I'm not sure about hibernate, you might try to use QueryOver API instead of Query
http://nhibernate.info/blog/2009/12/17/queryover-in-nh-3-0.html

Slow running EF query

I have this query and it is running VERY slowly. Why would this be? I have added all the non NVARCHAR(max) columns into a non-clustered index but still have this problem.
return (from ir in IndemnityRatings
where date >= ir.ValidFrom &&
date <= ir.ValidTo &&
ir.CoverLevel.CoverLevelId == coverLevelId &&
ir.Age == age &&
ir.OwnIndemnity == ownIndemnity &&
ir.CustomerIndemnity == customerIndemnity &&
ir.FullTimePartTimeReference == fullOrPartTime &&
ir.TurnoverReference == turnover
select ir).SingleOrDefault();
Incidently, when executing the same query (the equivalent thereof) directly on the database I get results back immediately. Furthermore I get a single row when calling the database dirctly (which is correct) and null when calling this piece of code.

Linq to sql constantly returns null

I have the following SQL query:
SELECT DISTINCT
Participant.BackgroundTrainingID,
Location.TrainingSite
FROM Registration, ProgramLocation, Participant, Program, Location
WHERE ProgramLocation.LocationID = Location.LocationID
AND ProgramLocation.ProgramID=Registration.ProgramID
AND Registration.ParticipantID=Participant.ParticipantId
I wrote the following LINQ to SQL to match the query above:
var trainingsiteinfo = (from c in db.ProgramLocations
from n in db.Registrations
from l in db.Participants
from h in db.Locations
where c.LocationID == h.LocationID
&& c.ProgramID == n.ProgramID
&& n.ParticipantID == l.ParticipantId
select new {
h.TrainingSite,
l.BackgroundTrainingID }).Distinct();
The SQL query works fine but LINQ constantly returns null.
I use Linqer http://www.sqltolinq.com/ when i get out of ideas :-) Aswell it speeds up conversion jobs.
Make sure your db context isn't going out of scope before you have bound your results. You can test this by adding .ToList() after your .Distinct() to force eager loading of the results.
Your linq code has an unclosed } bracket in the select statement

Having problems converting conditional where clause in LINQ back over to SQL

I've got myself in a bit of a pickle!
I've done a snazzy LINQ statement that does the job in my web app, but now I'd like to use this in a stored procedure:
var r = (from p in getautocompleteweightsproducts.tblWeights
where p.MemberId == memberid &&
p.LocationId == locationid
select p);
if (level != "0")
r = r.Where(p => p.MaterialLevel == level);
if (column == "UnitUserField1")
r = r.Where(p => p.UnitUserField1 == acitem);
if (column == "UnitUserField2")
r = r.Where(p => p.UnitUserField2 == acitem);
return r.OrderBy(p => p.LevelNo).ToList();
However, I can't for the life of me get the conditional where clause to work!!
If someone can point me in the right direction, I'd be most grateful.
Kind regards
Maybe something like this?
SELECT *
FROM dbo.weights
WHERE member_id = #memberid
AND location_id = #locationid
AND material_level = CASE WHEN #level = '0' THEN material_level
ELSE #level END
AND #acitem = CASE #column WHEN 'UnitUserField1' THEN unit_user_field_1
WHEN 'UnitUserField2' THEN unit_user_field_2
ELSE #acitem END
ORDER BY level_no
Have you tried LinqPAD, I'm pretty sure last time I played with that you could enter "LINQ to SQL" code and see the resulting SQL that produced. Failing that, place a SQL trace/profiler on your code running the LinqTOSQL and find the query being executed in the trace.
LukeH's answer will give you the correct rows, but there is something lost when you try to replace a query-generating-machine with a single query. There are parts of that query that are opaque to the optimizer.
If you need the original queries as-would-have-been-generated-by-linq, there are two options.
Generate every possible query and control which one runs by IF ELSE.
Use Dynamic sql to construct each query (although this trades away many of the benefits of using a stored procedure).
If you do decide to use dynamic sql, you should be aware of the curse and blessings of it.