I have this SQL query:
select
sum(h.nbHeures)
from
Heures h
join
HeuresProjets hp on h.HpGuid=hp.HPId
join
ActivityCodes ac on h.Code=ac.ActivityId
join
MainDoeuvre mdo on ac.ActivityId=mdo.CodeGuid
where
hp.ExtraGuid = '61E931C8-3268-4C9C-9FF5-ED0213D348D0'
and mdo.NoType = 1
It runs in less than a second, which is good. My project uses LINQ to entities to get data. This (very similar to the sql) query is terribly slow, taking more than a minute.
var result = (from hp in this.HeuresProjets
join h in ctx.Heures on hp.HPId equals h.HpGuid
join ac in ctx.ActivityCodes on h.Code equals ac.ActivityId
join mdo in ctx.MainDoeuvre on ac.ActivityId equals mdo.CodeGuid
where hp.ExtraGuid == this.EntityGuid && mdo.NoType == (int)spType
select h.NbHeures).Sum();
total = result;
I tried using nested loops instead. It's faster but still slow (~15 seconds).
foreach (HeuresProjets item in this.HeuresProjets)
{
foreach (Heures h in ctx.Heures.Where(x => x.HpGuid == item.HPId))
{
if (h.ActivityCodes != null && h.ActivityCodes.MainDoeuvre.FirstOrDefault() != null && h.ActivityCodes.MainDoeuvre.First().NoType == (int)type)
{
total += h.NbHeures;
}
}
}
Am I doing something obviously wrong? If there's no way to optimize this I'll just call a stored procedure but I would really like the keep the logic in the code.
EDIT
I modified my query according to IronMan84's advice.
decimal total = 0;
var result = (from hp in ctx.HeuresProjets
join h in ctx.Heures on hp.HPId equals h.HpGuid
join ac in ctx.ActivityCodes on h.Code equals ac.ActivityId
join mdo in ctx.MainDoeuvre on ac.ActivityId equals mdo.CodeGuid
where hp.ExtraGuid == this.EntityGuid && mdo.NoType == (int)spType
select h);
if(result.Any())
total = result.Sum(x=>x.NbHeures);
This almost works. It runs fast and gives back a decimal but:
1. It's not the right value
2. The result is clearly cached because it returns the exact same value with different parameters.
From looking at your code I'm thinking that your query is grabbing every single record from those tables that you're joining on (hence the long amount of time). I'm seeing you using this.HeuresProjets, which I'm assuming is a collection of database objects that you already had grabbed from the database (and that's why you're not using ctx.HeuresProjets). That collection, then, has probably already been hydrated by the time you get to your join query. In which case it becomes a LINQ-To-Objects query, necessitating that EF go and grab all of the other tables' records in order to complete the join.
Assuming I'm correct in my assumption (and let me know if I'm wrong), you might want to try this out:
var result = (from hp in ctx.HeuresProjets
join h in ctx.Heures on hp.HPId equals h.HpGuid
join ac in ctx.ActivityCodes on h.Code equals ac.ActivityId
join mdo in ctx.MainDoeuvre on ac.ActivityId equals mdo.CodeGuid
where hp.ExtraGuid == this.EntityGuid && mdo.NoType == (int)spType
select h).Sum(h => h.NbHeures);
total = result;
Also, if this.HeuresProjets is a filtered list of only specific objects, you can then just add to the where clause of the query to make sure that the IDs are in this.HeuresProjets.Select(hp => hp.HPId)
Related
I want to construct an SQL along this but try not to use sqlu.
select el.oid, el.name, el.res_cat from el
left join bk on (el.cat = bk.cat and bk.oid=100)
where not exists (select 1 from dates bd where
el.oid=bd.lots_oid and bd.bk_oid = bk.oid) and el.e_oid=bk.e_oid
Are there Slick functions for SQL exists or not exists? Thanks
Update 1
I realized my mistakes when I revisited my Slick code again. I want to apologize for the false alarm I set. This is not an answer and hopefully someone can help me to rectify my mistakes. For the time being, I am using Slick's plain SQL to continue my work.
The Slick query I constructed didn't work. It was close to the SQL I wanted. What I did was,
val elQuery = elTable.joinLeft(bkTable)
.on((el, bk) => el.cat === bk.cat && bk.oid === 100)
val query = for {
a <- elQuery if bdTable.filterNot(bd => a._2.map(_.oid === bd.bkOid).isDefined && a._1.oid === bd.elOid).exists
} yield a
finalQuery.result.statements.foreach(x => Logger.debug(s"xx => $x"))
I notice filterNot does not generate a SQL not exists. This is the other portion that lost me.
I don't have enough reputation to make comment yet. But I assume that you want to get all rows that doesn't exit in dates table. I would rewrite your query like below:
SELECT
el.oid, el.name, el.res_cat.cat
FROM
el
LEFT JOIN bk ON bk.cat = el.cat
AND bk.e_oid = el.e_oid
AND bk.oid = 100
LEFT JOIN dates bd ON bd.lots_oid = el.oid
AND bd.bk_oid = bk.oid
WHERE
bd.lots_oid IS NULL
Explanation:
Instead of taking NOT EXISTS, you can achieve the same thing by LEFT JOIN dates and specify on WHERE condition that the primary key (PK) for dates IS NULL. I don't know the PK for dates table, so I just add the column I know. You should adjust it to the PK of dates table.
LEFT JOINing and WHERE PK IS NULL ensures you that the row doesn't exist on the left joined table.
I would like to know how can i change the following sql statement into Linq or Lambda Extension in C#
SELECT m.mdes as AgeGroup,COUNT(DISTINCT(mbcd))as "No.of Member" FROM mageg m
LEFT JOIN (select distinct(mbcd) ,mage
FROMtevtl
JOIN mvipm
ON tevtl.mbcd = mvipm.mvip
WHERE datm >= '2014-04-01'
AND datm <= '2014-04-30'
)vip
ON m.tage >= vip.mage AND m.fage <= vip.mage
GROUP BY m.mdes
I manage to do the first half of the LINQ statement. Not sure If it is correct
here is the first half. I do not know how to connect with the left join.
(from mem in mvipms
from log in tevtls
from grp in magegs
where mem.mage >=grp.fage && mem.mage <=grp.tage && mem.mvip.Equals(log.mbcd)
&& log.datm >= DateTime.Parse("2014-04-01") && log.datm <= DateTime.Parse("2014-04-30")
select new {mem.mvip,grp.mdes}).Distinct()
Pls advice. I am using the MSSQL 2008 and VS2010.
Thanks a million.
I am no expert on LINQ, but since no-one else has answered, here goes!
Firstly you cannot (AFAIK) do a LINQ join on anything other than equals so the structure of your LEFT JOIN has to change. Partly for debugging purposes, but also for readability, I prefer to layout my LINQ in bite-size chunks, so what I would do in your case is something like this (assuming I have understood your data structure correctly):
var vip = (from t in tevtl
join v in mvipl
on t.mbcd equals v.mvip
where t.datm >= new DateTime(2014, 4, 1) && t.datm <= new DateTime(2014, 4, 30)
select new { t.mbcd, v.mage }).Distinct();
var mv = from m in magegl
from v in vip
where m.tage >= v.mage && m.fage <= v.mage
select new
{
m.mdes,
v.mbcd
};
var gmv = from m in mv
group m by new { m.mdes } into grp
select new
{
mdes = grp.Key.mdes,
CountMBCD = grp.Count()
};
var lj = from m in magegl
join v in gmv
on m.mdes equals v.mdes into lhs
from x in lhs.DefaultIfEmpty()
select new
{
AgeGroup = m.mdes,
CountMBCD = x != null ? x.CountMBCD : 0
};
By way of explanation. mv is the equivalent structure for your left join in that it has the relevant where clause, but obviously it does not contain any nulls - it is equivalent to an INNER JOIN. gmv is a group on mv, so is still effectively an INNER JOIN. Then to pick up the missing mdes (if any) I re-join on magel with the added syntax DefaultIfEmpty() - this converts the join into the equivalent of a LEFT OUTER JOIN.
Without any sample data, I haven't been able to test this, but I hope it gives you enough to point you in the right direction!
When this gets called two times, the second invocation doesn't re-run the query on the database, the query caching works.
var query = from p in session.Query<Product>()
where p.YearIntroduced >= 0
select p;
query = query.Cacheable();
var t = query.ToList();
However, when I put some join on the query, the query cache is not working anymore, hence when this is invoked two times, the query is invoked on database two times too:
var query = from p in session.Query<Product>()
join l in session.Query<ProductLanguage>()
on p.ProductId equals l.ProductId
where p.YearIntroduced >= 0
select new { p, l };
query = query.Cacheable();
var t = query.ToList();
Might be a dumb question, is query caching can work on one table only, hence when adding a join, the query is not cacheable anymore?
What's the solution to make a query cacheable even it has a join?
Another oddity, the query caching with join will work if I remove the where clause. When this gets called two times, the second invocation doesn't re-run the query on database, the query caching works
var query = from p in session.Query<Product>()
join l in session.Query<ProductLanguage>()
on p.ProductId equals l.ProductId
select new { p, l };
query = query.Cacheable();
var t = query.ToList();
But what's the use of a query when you can't put a where clause on it?
Is this an NHibernate bug, or am I just using query caching the wrong way?
Found the solution, put the Where clause on the final part of the query:
var query = from p in session.Query<Product>()
join l in session.Query<ProductLanguage>()
on p.ProductId equals l.ProductId
select new { p, l };
query = query.Where(x => x.p.ProductId && x.l.LanguageCode == "en").Cacheable();
var t = query.ToList();
I have an Entity Framework model with the following classes (I have simplified the classes for simpler viewing):
PuzzleItem
-PuzzleId (int, primary key)
-Title
PuzzleProgress
-ProgressId (int, primary key)
-PuzzleId (FK)
-UserId (FK)
In PuzzleItem I have a number of levels. The PuzzleProgress keeps track of which level the user is at by inserting a record of that level when the user completes a previous level. For a start a new user will have one entry in PuzzleProgress with PuzzleId = 1.
In my code, I am using the following statement to perform a left outer join, so that I will get a list of all puzzles, and indicating to me which puzzle has not been solved. I made reference to this post from StackOverflow.
Here is my code:
var result = from pzs in e.PuzzleItems
join prg in e.PuzzleProgresses on pzs equals prg.PuzzleItem
into pzs_prg_tbl
from pzs_prg in pzs_prg_tbl.DefaultIfEmpty()
where pzs_prg.UserId == userId
select new SimplePuzzleItem()
{
PuzzleId = pzs_prg.PuzzleId,
PuzzleName = pzs_prg.PuzzleItem.Title,
IsUnlocked = (pzs_prg == null?false:true)
};
After running the code, only the first level of this new user is returned (while the PuzzleItem table has 3 records).
I've tried playing around with the code but the one I pasted above is the nearest I can get to, can anybody point me in the correct direction? Thanks!
It's a bit hard to say exactly without seeing more of the code, but where pzs_prg.UserId == userId is probably negating the left outer join.
What I mean is, if you're intending PuzzleItems LEFT JOIN PuzzleProgress then you want all PuzzleItems even when there is no PuzzleProgress. But the where pzs_prg.UserId == userId means PuzzleProgress cannot be null, because it must have a UserId (of value userId). So, you effectively have an inner join.
Personally, I don't like the "correct" way of doing joins (left or inner) in linq, so this is how I would correct the linq statement:
var result = from pz in db.PuzzleItems
from pg in db.PuzzleProgresses
.Where(pg => pg.PuzzleId == pz.PuzzleId)
.Where(pg => pg.UserId == userId)
.DefaultIfEmpty()
select new
{
PuzzleId = pz.PuzzleId,
PuzzleName = pz.Title,
IsUnlocked = (pg != null)
};
This reads a lot more like SQL joins, which I learned a long time ago so it fits my thinking.
If you want to re-factor the join type syntax, look at this 'LINQ Joining in c# with multiple conditions'
I think the where clause is filtering out the records. You need to include the where clause in your left join. Like so:
var result = from pzs in e.PuzzleItems
join prg in e.PuzzleProgresses on new { pzs.PuzzleId, UserId = userId } equals new { prg.PuzzleId, prg.UserId }
into pzs_prg_tbl
from pzs_prg in pzs_prg_tbl.DefaultIfEmpty()
select new SimplePuzzleItem()
{
PuzzleId = pzs_prg.PuzzleId,
PuzzleName = pzs_prg.PuzzleItem.Title,
IsUnlocked = (pzs_prg == null?false:true)
};
My LINQ isnt the best, neither is my SQL but im trying to do something like this in LINQ (its kind of in pseudo-code)
select * from CarePlan c
-- only newest Referral based on r.Date (if more than one exists)
left outer join Referral r on r.CarePlanId = c.CarePlanId
where c.PatientId = 'E4A1DA8B-F74D-4417-8AC7-B466E3B3FFD0'
The data looks like this:
A Patient can have a bunch of careplans
each careplan can have 0 to n referrals (I want the newest referral per careplan - if any exist)
Would like to return a list of careplans for each patient (whether or not they have a referral or not, if it has more than one referral - grab the newest one)
Thanks for any help guys
In LINQ you use the DefaultIfEmpty to achieve a left outer join - examples at http://msdn.microsoft.com/en-us/library/bb397895.aspx
Assuming that the referrals are not a (potentially empty) collection on the care plans so you're joining two collections together ...
Your query it would be something like:
Get the latest referral per Care Plan:
var latestReferrals = from r in referrals
group r by r.CarePlanId into lr
select new { CarePlanId = lr.Key, LatestReferral = lr.OrderByDescending(lrd => lrd.Date).First()};
Find the combined details:
var q = from c in CarePlan
where c.PatientId = 'E4A1DA8B-F74D-4417-8AC7-B466E3B3FFD0'
join lr in latestReferrals on c.CarePlanId equals lr.CarePlanId into gj
from subReferral in gj.DefaultIfEmpty()
select new { CarePlanId = c.CarePlanId, LatestReferral = (subReferral == null) ? null : subReferral.LatestReferral };
Depending on whether you want many referral properties or just a few you may or may not want the whole Referral object in the second part or just extract the relevant properties.
You may be able to combine these into a single query but for readability it may be easier to keep them separate. If there is a combined solution you should also compare performance of the two approaches.
EDIT: see comment for joining patients/other tables from care plans
If Patient is joined from Referral (as per comment) then its more complex because you're doing several left outer joins. So switching to the slightly more concise syntax:
var combined = from c in carePlans
where c.PatientId = 'E4A1DA8B-F74D-4417-8AC7-B466E3B3FFD0'
from lr in latestReferral.Where(r => r.CarePlanId == c.CarePlanId).DefaultIfEmpty()
from p in patients.Where(patient => patient.PatientId == ((lr != null) ? lr.LatestReferral.PatientId : -1)).DefaultIfEmpty()
select new { c.CarePlanId, PatientName = (p == null) ? "no patient" : p.PatientName, LatestReferral = (lr == null) ? null : lr.LatestReferral };