How can I make a nested projection in a Linq query when using the group by clause? - sql

Question

I'm trying to work with grouped data coming back from SQL.
The method I'm writing is to provide the data for a "Case Status Overview" screen.
It must produce a nested XML document.
Now, I could do it the easy way, but I'm trying to learn whether it's possible to use the linq "group by" statement and then to project the data already nested. (the easy way would be just to pull back the data in a tabular fashion from the database and then for-loop through it forming the Xml document for output)
Here is the data hierarchy:
Every Case has a DebtType and every DebtType has a Client.
Here is the SQL that retrieves the data:
SELECT ClientNames.ClientID ,
ClientNames.ClientCode ,
ClientNames.ClientName ,
DebtTypes.DebtTypeID ,
DebtTypes.DebtTypeShortDesc ,
DebtTypes.DebtTypeLongDesc ,
Cases.CurrentStateCode ,
SUM(1 - CAST(Cases.CaseClosed AS INT)) AS OpenCaseCount ,
SUM(CAST(Cases.CaseClosed AS INT)) AS ClosedCaseCount ,
SUM(CAST(Cases.CaseOnHold AS INT)) AS OnHoldCaseCount ,
SUM(CAST(Cases.CaseReferred AS INT)) AS ReferredCaseCount ,
COUNT(Cases.CaseID) AS TotalCaseCount ,
SUM(Cases.CaseTotalPaid) AS TotalAmountPaid ,
SUM(Cases.CaseCurrentOutstandingAmount) AS TotalAmountOutstanding,
SUM(Cases.CaseTotalDebtWrittenOff) AS TotalAmountWrittenOff ,
SUM(Cases.CaseTotalDebtCancelled) AS TotalAmountCancelled
FROM ClientNames
INNER JOIN ClientDebtTypes
ON ClientNames.ClientID = ClientDebtTypes.ClientID
INNER JOIN DebtTypes
ON ClientDebtTypes.DebtTypeID = DebtTypes.DebtTypeID
INNER JOIN Cases
ON ClientDebtTypes.ClientDebtTypeID = Cases.CaseClientDebtTypeID
GROUP BY ClientNames.ClientID ,
ClientNames.ClientCode ,
ClientNames.ClientName ,
DebtTypes.DebtTypeID ,
DebtTypes.DebtTypeShortDesc,
DebtTypes.DebtTypeLongDesc ,
Cases.CurrentStateCode
ORDER BY ClientNames.ClientID,
DebtTypes.DebtTypeID,
CurrentStateCode
Using Linqer it converts it to:
from clientnames in db.ClientNames
join clientdebttypes in db.ClientDebtTypes on clientnames.ClientID equals clientdebttypes.ClientID
join debttypes in db.DebtTypes on clientdebttypes.DebtTypeID equals debttypes.DebtTypeID
join cases in db.Cases on new { ClientDebtTypeID = clientdebttypes.ClientDebtTypeID } equals new { ClientDebtTypeID = cases.CaseClientDebtTypeID }
group new {clientnames, debttypes, cases} by new {
clientnames.ClientID,
clientnames.ClientCode,
clientnames.ClientName1,
debttypes.DebtTypeID,
debttypes.DebtTypeShortDesc,
debttypes.DebtTypeLongDesc,
cases.CurrentStateCode
} into g
orderby
g.Key.ClientID,
g.Key.DebtTypeID,
g.Key.CurrentStateCode
select new {
ClientID = (System.Int32?)g.Key.ClientID,
g.Key.ClientCode,
g.Key.ClientName1,
DebtTypeID = (System.Int32?)g.Key.DebtTypeID,
g.Key.DebtTypeShortDesc,
g.Key.DebtTypeLongDesc,
g.Key.CurrentStateCode,
OpenCaseCount = (System.Int64?)g.Sum(p => 1 - Convert.ToInt32(p.cases.CaseClosed)),
ClosedCaseCount = (Int32?)g.Sum(p => Convert.ToInt32(p.cases.CaseClosed)),
OnHoldCaseCount = (Int32?)g.Sum(p => Convert.ToInt32(p.cases.CaseOnHold)),
ReferredCaseCount = (Int32?)g.Sum(p => Convert.ToInt32(p.cases.CaseReferred)),
TotalCaseCount = (Int64?)g.Count(p => p.cases.CaseID != null),
TotalAmountPaid = (System.Decimal?)g.Sum(p => p.cases.CaseTotalPaid),
TotalAmountOutstanding = (System.Decimal?)g.Sum(p => p.cases.CaseCurrentOutstandingAmount),
TotalAmountWrittenOff = (System.Decimal?)g.Sum(p => p.cases.CaseTotalDebtWrittenOff),
TotalAmountCancelled = (System.Decimal?)g.Sum(p => p.cases.CaseTotalDebtCancelled)
}
Now as I mentioned, I could stop there and right a for loop to create the Xml data.
But I'm trying to create a nested group (IGrouping<ClientName,IGrouping<DebtType,SummaryClass>>)
and then project the data in a nested format.
Now we're using LinqToXsd to create strong type wrappers for out Xml documents, but essentially all this means is that out output type is:
private class ClientSummary
{
public string ClientName { get; set; }
public IList<DebtTypeSummary> DebtTypes { get; set; }
}
private class DebtTypeSummary
{
public string DebtType { get; set; }
public IList<StateCodeSummary> StateCodes { get; set; }
}
private class StateCodeSummary
{
public string StateCode { get; set; }
public int TotalCount { get; set; }
public decimal TotalAmountPaid { get; set; }
//etc
//etc
//etc
}
Now I got as far as writing the following Linq:
var grouping = from cases in db.Cases
join clientdebttypes in db.ClientDebtTypes on cases.CaseClientDebtTypeID equals clientdebttypes.ClientID
join debttypes in db.DebtTypes on clientdebttypes.DebtTypeID equals debttypes.DebtTypeID
group cases by new ClientDebtTypePair() { ClientDebtType = clientdebttypes, DebtType = debttypes } into casesByClientDebtTypes
join clientnames in db.ClientNames on casesByClientDebtTypes.Key.ClientDebtType.ClientName equals clientnames
group casesByClientDebtTypes by clientnames;
var projected = from casesByClientDebtTypes in grouping
let client = casesByClientDebtTypes.Key
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = new Client()
{
ClientID = client.ClientID,
DisplayName = client.ClientName1,
},
DebtTypes = from cases in casesByClientDebtTypes
let debttype = cases.Key.DebtType
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType.DebtTypesLocalType()
{
DebtType = new DebtType()
{
DebtTypeID = debttype.DebtTypeID,
Description = debttype.DebtTypeLongDesc,
DisplayName = debttype.DebtTypeShortDesc,
},
StatesCodes = from cases2 in cases
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType.DebtTypesLocalType.StatesCodesLocalType()
{
ClosedCasesCount = cases2.Sum(p => Convert.ToInt32(p.cases.CaseClosed))
which joins and groups the database tables and then tries to project the result a ClientSummary (the class names are different but that's because the above is a simplified view of the output classes). I fail completely when I've drilled all the way down to the Cases table and I find that I don't really understand how to do agregate functions. They appear to only be available on IGrouping<K, T>s and it seems I've just got confused.
I need to also ensure that the summaries are calculated server side, pulling back millions of cases would be bad.
Can anybody help me with this one? Is this even possible?
Regards,
James.
-------### UPDATE 1 ###-------
OK, been working on this again today.
I decided to use Linq2SQL to pull pack 2D data and then reformat it using Linq2Objects.
Here is what I started with:
var sql = from clientnames in db.ClientNames
join clientdebttypes in db.ClientDebtTypes on clientnames.ClientID equals clientdebttypes.ClientID
join debttypes in db.DebtTypes on clientdebttypes.DebtTypeID equals debttypes.DebtTypeID
join cases in db.Cases on new { ClientDebtTypeID = clientdebttypes.ClientDebtTypeID } equals new { ClientDebtTypeID = cases.CaseClientDebtTypeID }
group new { clientnames, debttypes, cases } by new
{
clientnames.ClientID,
clientnames.ClientCode,
clientnames.ClientName1,
debttypes.DebtTypeID,
debttypes.DebtTypeShortDesc,
debttypes.DebtTypeLongDesc,
cases.CurrentStateCode
} into g
orderby
g.Key.ClientID,
g.Key.DebtTypeID,
g.Key.CurrentStateCode
select new
{
Client = new Client{ ClientID = g.Key.ClientID, DisplayName = g.Key.ClientName1 },
DebtType = new DebtType{ DebtTypeID = g.Key.DebtTypeID, DisplayName = g.Key.DebtTypeShortDesc, Description = g.Key.DebtTypeLongDesc },
StateSummary = new LoadCaseStatusOverviewScreenOutput.ClientsLocalType.DebtTypesLocalType.StatesCodesLocalType()
{
StateCode = g.Key.CurrentStateCode,
OpenCasesCount = g.Sum(p => 1 - Convert.ToInt32(p.cases.CaseClosed)),
ClosedCasesCount = g.Sum(p => Convert.ToInt32(p.cases.CaseClosed)),
OnHoldCasesCount = g.Sum(p => Convert.ToInt32(p.cases.CaseOnHold)),
ReferredCasesCount = g.Sum(p => Convert.ToInt32(p.cases.CaseReferred)),
TotalCasesCount = g.Count(p => p.cases.CaseID != null),
TotalAmountPaid = g.Sum(p => p.cases.CaseTotalPaid),
TotalAmountOutstanding = g.Sum(p => p.cases.CaseCurrentOutstandingAmount),
TotalAmountWrittenOff = g.Sum(p => p.cases.CaseTotalDebtWrittenOff),
TotalAmountCancelled = g.Sum(p => p.cases.CaseTotalDebtCancelled),
}
};
var res = sql.ToList();
output.Clients = (from results in res
group results by results.Client into resultsByClient
from resultsByDebtType in
(from results in resultsByClient
group results by results.DebtType)
group resultsByDebtType by resultsByClient.Key into resultsByDebtTypeByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = resultsByDebtTypeByClient.Key,
DebtTypes = (from resultsByDebtType in resultsByDebtTypeByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType.DebtTypesLocalType()
{
DebtType = resultsByDebtType.Key,
StatesCodes = (from results in resultsByDebtType
let summary = results.StateSummary
select results.StateSummary).ToList()
}).ToList()
}).ToList();
That runs, but produces one Client/DebtType/Summary set for every result. So even though there is only one client in this case, I end up with 1300 clients, all identical.
I simplified it to the following:
output.Clients = (from results in res
group results by results.Client into resultsByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = resultsByClient.Key,
DebtTypes = null,
}).ToList();
That produces 1300 clients. Next I tried this:
output.Clients = (from results in res
group results by results.Client.ClientID into resultsByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = new Client { ClientID = resultsByClient.Key },
DebtTypes = null,
}).ToList();
And THAT produces ONE client (hurray!). Except I loose all the client information (boo!)
Guessing that as it's comparing client by refernce instead of by content I wrote the following:
public partial class Client
{
public static bool operator ==(Client left, Client right)
{
return left.ClientID == right.ClientID;
}
public static bool operator !=(Client left, Client right)
{
return left.ClientID != right.ClientID;
}
public override int GetHashCode()
{
return ClientID;
}
}
That did nothing. It repeatedly calls GetHashCode(), which I fudged to force it to return the same hash code for any matching ClientID, but it still created 1300 Client groups.
Regards,
James.
-------### UPDATE 2 ###-------
OK, I thought I would have a go at making the Linq2Sql output only simple values for grouping by:
g.Key.ClientID,
g.Key.ClientName1,
g.Key.DebtTypeID,
g.Key.DebtTypeShortDesc,
g.Key.DebtTypeLongDesc,
And then changed the test Linq2Objects to:
output.Clients = (from results in res
group results by new { ClientID = results.ClientID, DisplayName = results.ClientName1 } into resultsByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = new Client { ClientID = resultsByClient.Key.ClientID, DisplayName = resultsByClient.Key.DisplayName },
DebtTypes = null,
}).ToList();
That works. So anonymous types compare in the way I want them to, by content not reference (apparently)
This does not:
output.Clients = (from results in res
group results by new SiDemClient { ClientID = results.ClientID, DisplayName = results.ClientName1 } into resultsByClient
select new LoadCaseStatusOverviewScreenOutput.ClientsLocalType()
{
Client = resultsByClient.Key,//new Client { ClientID = resultsByClient.Key.ClientID, DisplayName = resultsByClient.Key.DisplayName },
DebtTypes = null,
}).ToList();
That still creates 1300 groups.
So, anonymous types compare in a magical way that I don't understand. How can I make my Client class compare like an anonymous type?
Regards,
James.
-------### SOLUTION FOUND ###-------
-------### MANY THANKS TO Enigmativity ###-------
I needed to override the Equals() method instead of implementing the == operator.
Now the grouping works and I have a wonderful Xml document to reutrn!
public partial class SiDemClient
{
public override bool Equals(object obj)
{
if (obj is SiDemClient)
{
return this.ClientID.Equals(((SiDemClient)obj).ClientID);
}
return false;
}
public override int GetHashCode()
{
return ClientID;
}
}
Many Thanks,
James.

Answer 1

When you override GetHashCode you must also override Equals. The == & != operators are irrelevant.
Try with this:
public partial class Client
{
public override bool Equals(object obj)
{
if (obj is Client)
{
return this.ClientID.Equals(((Client)obj).ClientID);
}
return false;
}
public override int GetHashCode()
{
return this.ClientID.GetHashCode();
}
}
See if that helps.