Perform aggregate count() on ADO.net Entity data model - sql

I am trying to perform an aggregate count() just like a SQL query would against my database, but instead of SQL I want to use LINQ.
I am trying to use LINQ to query my Entity Framework Data Model and perform an aggregate sum(). Specifically I want to do a count of the column(TimeWorked) in the TimeEntry table grouping by Project Name and Phase between 2 specific dates and then do a natural join on the Project table. I am then going to take that query result and load it into an observable collection and display it in a ListView.
My desired result is [ProjectName],[Phase],[Count(TimeWorked)],[Date]
I want to filter the counting to only Projects with a TimeEntry that is between two dates that for the sake of this example will just be called Date1 and Date2.
I am not familiar with LINQ 100% yet, I am still learning so please excuse my lack of LINQ terminology.
Here is a picture of my Relationship and tables.
DB Schema
Here are the data types
Project Table data types,
TimeEntry table
I have been looking everywhere and can't seem to find any good resources or examples. Could someone please point me in the right direction.

So you have projects and timeentries. There is a one-to-many relation between a Project and a TimeEntry: every Project has zero or more TimeEntries, every TimeEntry belongs to exactly one Project.
If you'd followed the entity framework code first conventions, you would have created classes like this:
class Project
{
public int Id {get; set;}
// every Project has zero or more TimeEntries:
public virtual ICollection<TimeEntry> TimeEntries {get; set;}
... // other properties
}
class TimeEntry
{
public int Id {get; set;}
// every TimeEntry belongs to exactly one Project using foreign key:
public int ProjectId {get; set;}
public virtual Project Project {get; set;}
... // other properties
}
class MyDbContext : DbContext
{
public DbSet<Project> Projects {get; set;}
public DbSet<TimeEntry> TimeEntries {get; set;}
}
Because you followed the conventions, this would be enough to inform entity framework that you planned a one-to-many. Entity Framework would be able to detect the primary and foreign keys and the relation between Projects and TimeEntries (possible problem: pluralization of time entry).
If you want different table names or column names, you'll need attributes or fluent API. But the structure remains similar.
So now you have your Project and TimeEntries. You want for every Project the Number of TimeEntries where TimeWorked is in a given time interval (are you sure? you want the Count, not the sum of time worked?)
I'd go for this:
var projectWithCountTimeWorked = dbContext.Projects
.Select(project => new
{
ProjectName = project.ProjectName,
...
// the Count of TimeEntries of this project in this period:
CountTimeWorked = project.TimeEntries
.Where(timeEntry => minDate <= timeEntry.TimeWorked
&& timeEntry.TimeWorked <= maxDate)
.Count(),
});
Because I used the ICollections, entity framework will internally do the proper joins to calculate the result.
If you want to specify the joins yourself I'd go for this:
var result = dbContext.Project // GroupJoin Projects
.GroupJoin(dbContext.TimeEntries // and TimeEntries
project => project.Id, // from every Project take the Id
timeEntry => timeEntry.ProjectId, // from every timeEntry take the ProjectId
(project, timeEntries) => new // for every Project and his matching
{ // timeEntries make a new object
Name = project.Name,
...
CountTimeWorked = timeEntries // count all timeEntries during the period
.Where(timeEntry => minDate <= timeEntry.TimeWorked
&& timeEntry.TimeWorked <= maxDate)
.Count(),
If you are unfamiliar with entity-framework code first basics. This web site helped me a lot to get me on track
This article was a good summary for me to have a look at most used linq methods

Related

How to use LINQs Include in SQL

In LINQ I have written a simple query where I am searching for an animal using the ID property. However, I am also including the Farm the animal belongs using the Include property.
I want to write the same LINQ query in SQL where I can include Farm. How can I include Farm using SQL. I have an incomplete SQL syntax below. Can anyone help me out.
LINQ
await _dbContext.Animals.Where(x => x.id == 1)
.Include(x => x.Farm)
.ToListAsync();
SQL
select * from Animals where id = 1;
Apparently your database has a table with Animals and a table with Farms. There seems to be a one-to-many relation between Animals and Farms: on every Farm live zero or more Animals; every Animal lives on exactly one Farm, namely the Farm that the foreign key refers to.
I think you will have classes similar to the following:
class Farm
{
public int Id {get; set;}
public string Name {get; set;}
... // etc
// Every Farm has zero or more Animals (one-to-many)
public virtual ICollection<Animal> {get; set;}
}
class Animal
{
public int Id {get; set;}
public string Name {get; set;}
... // etc
// Every Animal lives on exactly one Farm, using foreign key
public int FarmId {get; set;}
public virtual Farm Farm {get; set;}
}
I want to write the same LINQ query in SQL where I can include Farm.
A small trick: if you want to know the SQL code generated by Entity Framework, use property DbContext.Database.Log.
using (var dbContext = new DbContext())
{
// Log generated SQL to debug window:
dbContext.Database.Log = System.Diagnostics.Debug.Write;
// execute your LINQ:
var fetchedAnimals = _dbContext.Animals.Where(x => x.id == 1)
.Include(x => x.Farm)
.ToList();
}
Write your own SQL
You'll have to join Animals with Farms, and keep only the Animal with ID = 1:
See SQL Join
// Select only the properties of Animals and Farms that you actually plan to use
SELECT Animals.Id, Animals.Name, ...,
Farms.Id, Farms.Name, ...
FROM Animals INNER JOIN Farms
ON Animals.FarmId = Farm.Id
WHERE Animals.Id = 1
You should not use "" to fetch everything. If Farm [10] Has 5000 Chickens, then every Chicken will have a foreign key with a value 10. If you use "" you will transfer this value 10 more than 5000 times, while you already know the value of the foreign key.
There's room for improvement
When using entity framework to fetch data, always use Select, and select only the properties that you plan to use, even if you Select all properties. Only omit Select and / or use Include if you plan to change / update the fetched data.
The reason is, that fetching data without using Select is not very efficient.
If you fetch data without using Select, entity framework will put the fetched item in the DbContext.ChangeTracker, together with a copy of the fetched item. You get a reference to the copy. Whenever you change properties of the fetched item, you change the copy in the ChangeTracker. When you call DbContext.SaveChanges, the original is compared with the copy, property per property to see which properties are changed, and thus need to be updated in the database.
So if you don't plan to change the fetched data, it would be a waste of processing power to put this data AND a copy in the ChangeTracker. Hence: always use Select, unless you plan to update the fetched data.

SQL Join tables on different column types

I have two tables:
dbo.Dashboards
Id (int PK) Title(nvarchar) WidgetIds(nvarchar)
1 Test [1,2]
dbo.Widgets
Id (int PK) Details(nvarchar)
1 {'text': 'some data'}
2 {'text': 'test'}
Expected output:
Dashboard.Id Dashboard.Title Widget.Id Widget.Details
1 Test 1 {'text': 'some data'}
1 Test 2 {'text': 'test'}
I would like to get dashboards with assigned widgets by using Entity Framework.
My first solution is to get dbo.Dashboards and then dbo.Widgets. After that I can merge it in a backend, but it is not the best practice.
Is there any option to get Dashboards with assigned Widget list?
Function Include() is not working because there isn't FK relationship between tables.
It seems to me that you have a many-to-many relationship between Dashboards and Widgets: Every Dashboard has zero or more Widgets and every Widget is used by zero or more Dashboards.
In a proper database you would have a separate junction table. Apparently you chose not to use this pattern, but create a string that contains a textual representation of the widgets that a 'Dashboard` has.
If you plan to create a serious application I strongly advise you to
use the standard pattern in many-to-many relationships
If you don't, all your queries will be more difficult. Imagine the problems you'll experience if you want to delete a Widget. You'd have to check the textual representation of every Dashboard to check if the widget that you want to remove is used somewhere and change it.
If you want to configure your many-to-many relations ship according to the Entity Framework Code-First Conventions, you will have something like this:
class Dashboard
{
public int Id {get; set;}
public string Title {get; set;}
// every Dashboard has zero or more Widgets
public virtual ICollection<Widget> Widgets {get; set;}
... // other properties
}
class Widget
{
public int Id {get; set;}
// every Widget is used in zero or more Dashboards
public virtual ICollection<Dashboard> Dashboards{get; set;}
... // other widget properties
}
class MyDbContext : DbContext
{
public DbSet<Dashboard> Dashboards {get; set;}
public DbSet<Widget> Widgets {get; set;}
}
Because you stuck to the conventions, this is all that entity framework needs to know to understand that you want to configure a many-to-many relationship between Dashboards and Widgets. Entity Framework will create the junction table for you. It will automatically update this table whenever you add a Widget to a Dashboard. It will also create the proper joins whenever you want to fetch Dashboards with their Widgets, or Widgets with the Dasheboards that use them.
Your query will be fairly simple:
var DashBoardsWithTheirWidgets = myDbcontext.Dashboards
// I only want to see the super dashboards
.Where(dashboard => dashboard.Type = DashboardType.Super)
.Select(dashboard => new
{
// Select only the properties you plan to use:
Id = dashboard.Id,
Title = dashboard.Title,
// select only the Widgets you plan to use:
Widgets = dashboard.Widgets
.Where(widget => widget.Price > 100.00)
.Select(widget => new
{
// again select only the properties you plan to use
Name = widget.Name,
Price = widget.Price,
})
.ToList();
});
See how easy it is if you stick to the conventions?
If you really want your obscure method of using foreign keys, you need a function to remove the square brackets and the commas from the widgetIds, split the string into sub-strings, Parse them to numbers, and do a join.
But before you plan to continue on this path, experiment on how to add a Widget and a Dashboard. How to add a Widget to a Dashboard, how to remove a Widget. I think the time needed to reform your database into proper format is much less than the time you'll need to implement those functions
Solution 1:
You need to restructure the dbo.dashboards table. Change the column layout of dbo.dashboards to
Auto_Generated_ID, Unique_Identifier(PK), Title, WidgetIds
I know the above column restructuring is done in a bad way. But still this will work in your case.
After redesigning it you can use join between dbo.dashboards and dbo.widgets to retrieve it in an efficient way.
Solution 2:
The below-normalized tables will work in your case
dbo.dashboard
id, title (columns)
dbo.dashboard_widget
id, dashboard_id, widget_id (columns)
dbo.widgets
id, details (columns)
Query:
select d.id, d.title, dw.widget_ids, w. details from dbo.dashboard d INNER JOIN dbo.dashboard_widget dw ON d.id = dw.dashboard_id INNER JOIN dbo.widgets w ON dw.widget_id = w.id where d.id = << id number >>

Mapping an extension table that might not have a row

I'm working with Fluent nHibernate on a legacy database and have a main Person table and several extension tables containing additional information about the person. These extension tables are one-to-one, meaning that a person will only have one row on the extension table and the extension table should always map back to one person.
Table: Person
Columns: PersonID, FirstName, LastName, etc.
Table: PersonLogin
Columns: PersonID (FK, unique), UserName, Password, etc.
I have my mappings defined as this (with the irrelevant properties omitted):
public PersonMap()
{
Table("Person");
Id(x => x.Id, "PersonID").Not.Nullable();
References(x => x.Login, "PersonID").LazyLoad();
}
public LoginMap()
{
Table("PersonLogin");
Id(x => x.Id, "PersonID").GeneratedBy.Foreign("Person");
References(x => x.Person, "PersonID").LazyLoad();
}
This works when I have data on both tables, but I recently learned that some of the extension tables don't have data for all Person rows. This caused me to get errors during the query. So, I added .NotFound.Ignore() to my PersonMap making it look like this:
References(x => x.Login, "PersonID").LazyLoad().NotFound.Ignore();
That caused me to get unnecessary selects from the Login table due to https://nhibernate.jira.com/browse/NH-1001 when my business layer doesn't need to project any of the extension table values. It is causing the performance to be terrible in some of my search queries.
I've scoured a lot of posts, but haven't found a rock solid answer about how to address this scenario. Below are the options I've tried:
Option One:
Create rows on the extension table to ensure there is no Person without a row on the extension table and then remove the .NotFound.Ignore().
The issue with this option is that it's a legacy database and I'm not sure where I'd need to update to ensure that a PersonLogin is inserted when a Person is inserted.
Option Two:
Remove the PersonLogin reference from my PersonMap and custom load it inside my Person class. Like this:
public class Person
{
/// <summary> Gets or sets the PersonID </summary>
public virtual int Id { get; set; }
private bool loadedLogin;
private PersonLogin login;
public virtual PersonLogin Login
{
get
{
if (!loadedLogin)
{
login = SessionManager.Session().Get<PersonLogin>(Id);
loadedLogin = true;
}
return login;
}
set
{
login = value;
loadedLogin = true;
}
}
}
The issue I'm having with it is that I can't eagerly fetch the data when performing a query to pull back a large number of Person objects and their Logins.
Option Three:
I just started playing to see if I could write a custom IEntityNotFoundDelegate to not throw the exception for these objects.
private class CustomEntityNotFoundDelegate : IEntityNotFoundDelegate
{
public void HandleEntityNotFound(string entityName, object id)
{
if (entityName == "my.namespace.PersonLogin")
{
return;
}
else
{
throw new ObjectNotFoundException(id, entityName);
}
}
}
And I added this to the config
cfg.EntityNotFoundDelegate = new CustomEntityNotFoundDelegate();
It catches my scenario and returns back now instead of throwing the error, but now when I try to project those PersonLogin properties onto my business objects, it's attempting to use the Proxy object and throws this error that I'm trying to figure out if I can handle cleanly (possibly in a IPostLoadEventListener).
System.Reflection.TargetException occurred
Message = Non-static method requires a target
I think I've got this working now by keeping the .NotFound.Ignore().
I originally stated:
That caused me to get unnecessary selects from the Login table due to https://nhibernate.jira.com/browse/NH-1001 when my business layer doesn't need to project any of the extension table values. It is causing the performance to be terrible in some of my search queries.
I was able to tweak my LINQ queries to use the IQueryOver in some instances and to improve my use of LINQ in other scenarios to project only the necessary values. This appears to have resolved the queries from pulling back the extension tables since their values were not needed in the projections.
I thought that my queries weren't projecting these extension tables, but figured out that I had a method ToKeyValuePair that I was using in the projection to concatenate the ID and a Name field together of some related properties. That method was causing the objects to load completely since LINQ wasn't able to determine that the needed fields were present without joining to the extension table.

Statistical query in SQL - is this possible with NHibernate LINQ?

I have an application that uses a few data warehousing principles such as dimensional modeling to do reporting on a fairly simple database.
An example (simplified) entity named Call looks like this:
public virtual long Id { get; set; }
public virtual string OriginatorNumber { get; set; }
public virtual string DestinationNumber { get; set; }
public virtual DateDimension DateDimension { get; set; }
A few of the properties of the real model have been removed as they are irrelevant. The simplified DateDimension looks like this:
public virtual long Id { get; set; }
public virtual DateTime Date { get; set; }
public virtual int DayOfMonth { get; set; }
public virtual int Weekday { get; set; }
There are a LOT more columns like this - they are prepopulated for the current decade by application setup. So each date in the entire decade has a row in this table, and each Call has a link to the date that it occured. This is all mapped in Fluent NHibernate and working fine.
If I want to do some reporting, I can do this easily with the improved NHibernate LINQ provider in 3.0. We would like to use LINQ for the improved maintainability it gives us, but if we really MUST, we'll consider HQL, ICriteria or even plain SQL.
So say I want to build a report that shows the number of calls from a certain number, divided by the day of the week they occur. I can do that easily this way:
var query = Calls
.Where(c => c.OriginatorNumber == "402")
.GroupBy(c => c.DateDimension.Weekday)
.Select(g => new { Day = g.Key, Calls = g.Count() } );
In this example, "Calls" is basically an IQueryable returned from NHibernates LINQ provider (Query) through a repository interface. The query above gives me the correct results, NHibernate Profiler shows me that the SQL is pretty optimal, all is well.
However, if I want to do something slightly more advanced, I get stuck. Say I want the average number of calls per weekday. Not too far from the above, right? I just need to figure out the number of unique dates each weekday has in the result set, divide the total number of calls by it, and we're all set - right? Well, no, this is where I start to hit the limitations of the NHibernate LINQ provider. With LINQ to objects I could construct a query to do it - something along the lines of
.Select(g => g.Count() / g.GroupBy(c => c.DateDimension.Date).Count());
However, this does not convert into the correct query when using it in NHibernate. Rather, it turns both .Count() calls in the above to the same count(*) of call records, so the result is always 1.
I COULD of course just query for each call, weekday and date as a new anonymous object, then do the math on the application side, but according to conventional wisdom, That's Just Wrong (tm). I could end up doing it in desperation, tho, even tho it means pain when the table grows to a million++ calls.
The below is an SQL query that gives me the result I am looking for.
select ss.Weekday, AVG(cast(ss.Count as decimal))
from
(
select dd.Weekday, dd.Date, COUNT(*) as Count
from Call c
left outer join DateDimension dd
on c.DateDimension_id = dd.Id
where c.OriginatorNumber = '402'
group by dd.Weekday, dd.Date
) ss
group by ss.Weekday
order by ss.Weekday
Is it possible to do this with the NHibernate LINQ provider? Or, if that is not possible, how close can I get before I have to let the application fetch the intermediary result and do the rest?
There are a lot of things you can't do with the LINQ provider. Using HQL or CreateCriteria is just something you'll have to accept with NHibernate.
I haven't tried it, but it looks like you should be able to do what you want to do using HQL or CreateCriteria (with DetatchedCriteria).
If you are desperate you can also fall back to plain SQL using CreateSqlQuery.

Joins in fluent nhibernate

I am using fluent nhibernate.
example:
i have 3 tables i.e.
CUSTOMER
CustomerId pk
CustomerName
PRODUCT
ProductId pk
ProductName
Cust_Product
cust_prodId pk
ProductId fk
CustomerId fk
Now, I want to show customername, productnae
so, how do i write mapping class for the same.
i want to use
session.CreateCriteria(typeof("className")).List()
like this. how do i do this..?
If you're looking for a full tutorial on how to do this, I recommend the FNH wiki or one of the many blog postings which can be found through Google.
However, you're trying to implement a many-to-many relationship here, and that seems to throw a lot of people off. Here's a rough guide:
On your Customer class, you'll need a collection like:
IList<Product> Products { get; private set; }
And similarly, on your Product class:
IList<Customers> Customers { get; private set; }
You start off a many-to-many map with the HasManyToMany function:
public class CustomerMap : ClassMap<Customer>
{
public CustomerMap()
{
// other mappings
HasManyToMany<Product>(x => x.Products)
.WithTableName("Cust_Product") // Specifies the join table name
.WithParentKeyColumn("CustomerId") // Specifies the key joining back to this table (defaults to [class]_id, Customer_id in this case)
.WithChildKeyColumn("ProductId")
.FetchType.Join(); // Instructs NHibernate to use a join instead of sequential select
}
}
Then repeat the process for the other side of the relationship (the Customers property on the Product class).