Linq takes more than 20 seconds to query a table with less than 100 records - sql

Unfortunately I haven't found a good answer for this problem yet. The answers and questions I have seen so far in here are about big tables with a lot of records.
I'm trying to query a table called Tickets with the following code:
var Status = ticketStatusService.GetByName("New");
string StatusID = Status.Id;
var tickets = db.Tickets.Where(e =>
!e.Deleted &&
e.Project == null &&
e.Status != null &&
e.Status.Id == StatusID);
var list = tickets.ToList();
The table currently has less than 100 records, this query takes an average of 22 seconds to execute.
The code first model for it is as follows:
public class Ticket : Base
{
[Key]
[Required]
public Guid Id { get; set; }
[Display(Name = "Date")]
public DateTime RowDate { get; set; } = DateTime.Now;
public bool Deleted { get; set; } = false;
[Index(IsUnique = true)]
public int? Number { get; set; }
[Display(Name = "Ticket Subject")]
public string Subject { get; set; }
[Display(Name = "Notes (Employees Only)")]
public string Notes { get; set; }
[Display(Name = "E-Mail")]
public string From { get; set; }
[Display(Name = "Phone Number")]
public string Phone { get; set; }
[Display(Name = "Secondary Phone Number")]
public string PhoneAlt { get; set; }
[Display(Name = "Client Name")]
public string Name { get; set; }
[Display(Name = "Message")]
public string Messages { get; set; }
[DataType(DataType.DateTime)]
public DateTime? OpenDate { get; set; }
[DataType(DataType.DateTime)]
public DateTime? CloseDate { get; set; }
[DataType(DataType.DateTime)]
public DateTime? AssignedDate { get; set; }
public bool? Origin { get; set; }
public virtual User AssignedUser { get; set; }
public virtual List<TicketFile> TicketFiles { get; set; }
public virtual List<Task> Tasks { get; set; }
public virtual Project Project { get; set; }
public virtual TicketStatus Status { get; set; }
public virtual TicketClosingCategory TicketClosingCategory { get; set; }
public virtual TicketGroup TicketGroup { get; set; }
public virtual TicketPriority TicketPriority { get; set; }
}
Any insight into this issue would be appreciated. Thank you very much!
Edit: Running the same query directly on SQL Server Management Studio also takes very long, about 9 to 11 seconds. So there might be an issue with the table itself.

I see several possible improvements.
For some reason you chose to deviate from the entity framework code fist conventions. One of them is the use of a List instead of an ICollection, another it that you omit to mention the foreign keys.
Use ICollection istead of List
Are you sure that Ticket.TicketFiles[4] has a defined meaning? And what would Ticket.TicketFiles.Insert(4, new TicketFile()) mean?
Better stick to an interface that prohibits usage of functions that have no defined meaning. Use ICollection<TicketFile>. This way you'll have only functions that have a proper meaning in the context of a database. Besides it gives entity framework the freedom to chose the most efficient collection type to execute its queries.
Let your classes represent the tables
Let your classes just be POCOs. Don't add any functionality that is not in your tables.
In entity framework the columns of a table are represented by non-virtual properties. The virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
Let entity framework decide what's the most efficient to initialize the data in your sequences. Don't use a constructor where you create a List, which will be immediately thrown away by entity framework to replace it with its own ICollection. Don't automatically initialize property Deleted, if entity framework immediately replaces it with its own value.
You will probably have only one procedure where you will add a Ticket to the database. Use this function to properly initialize the field of any "newly added Ticket"
Don't forget the foreign keys
You defined several relations between your tables (one-to-many, or many-to-many?) but you forgot to define the foreign keys. Because of your use of virtual entity framework can understand that it needs foreign keys and will add them, but in your query you need to write e.Status != null && e.Status.Id == statusId, while obviously you could just use the foreign key e.StatusId == statusId. For this you don't have to join with the Statuses table
Another reason to specify the foreign keys: they are real columns in your tables. If you define that these classes represent your tables, they should be in these classes!
Only select the properties you actually plan to use
One of the slower parts of a database query is the transport of the selected data from the database management system to your local process. Hence it is wise to select only the data you actually plan to use.
Example. There seems to be a one-to-many between a User and a Ticket: every User has zero or more Tickets, every Ticket belongs to exactly one User. Suppose User 4 has 20 Tickets. Every Ticket will have a UserId with a value 4. If you fetch these 20 Tickets without a proper Select you will fetch all properties of the same User 4 once per Ticket, and you will transport the data of this same User 20 times (with all his properties, and maybe all his relations). What a waste of processing power!
Always use Select to query your data and Select only the properties you actually plan to use. Only use Include if you plan to updated the Included data.
var tickets = dbContext.Tickets.Where(ticket => !ticket.Deleted
// improvement: use foreign keys
&& ticket.ProjectId == 0 (or == null, if ProjectId nullable)
&& ticket.StatusId == statusId) // no Join with Statuses needed
.Select(ticket => new
{
...
}

Related

MVC editing model relationships from database first

Building web app with MVC5 using database first modeling approach. Unfortunately, the existing database that I'm building from has a table that's missing foreign key constraints, has extra primary keys and is not generally well formed. Because of this, when I build the MVC model, it can't automatically generate all of the necessary many to many relationships between the tables. I can't change the original table's definition.
I discovered the ability to create new associations through the MVC model diagram and the Model Browser, but it only seems to allow me to associate the tables and not specify the fields that join them. When I update the model, I get errors about the associations I created not mapping, without any details.
All the examples I can find for database first modeling start with well-formed and well-named tables, which is not always the case in real life.
Can someone explain or point me to a link about how to start with a database first model but then fix the problems or define the missing pieces so that the MVC model can build correctly?
Or maybe for this one table, I should just model the class directly? The only part of that I don't understand is where I define the details of the navigation properties so the model knows what tables and fields to join.
Directed by the comment above, I eventually determined a much better approach to creating an MVC app for an existing database that may not be designed the way you want to lay out your model.
I am using a CodeFirst approach of defining the Model classes directly instead of generating them from the existing database. Then, using Entity Framework mapping properties, I map each class and its individual properties to the existing table and fields. This lets me define the class however I wish, give it the property names I want and define the foreign keys that may not be defined in the database.
Here's some examples of classes that I created that map to existing database tables.
[Table("uofs00137_contacts")]
public partial class Contact
{
[Key]
[Column("id")]
public int ID { get; set; }
[Column("first_name")]
[StringLength(30)]
[Display(Name = "First Name")]
public string FirstName { get; set; }
[Column("last_name")]
[StringLength(30)]
[Display(Name = "Last Name")]
public string LastName { get; set; }
[Column("organisation")]
[StringLength(30)]
[Display(Name = "Organization")]
public string Organization { get; set; }
[Column("title")]
[StringLength(30)]
[Display(Name = "Job Title")]
public string JobTitle { get; set; }
[Column("email")]
[StringLength(40)]
[Display(Name = "Email")]
public string Email { get; set; }
[Column("status_code")]
[StringLength(1)]
[Required]
[Display(Name = "Contact Status Code")]
public string StatusCode { get; set; }
[Display(Name = "Full Name")]
public string FullName
{
get
{
return FirstName + " " + LastName;
}
}
public virtual ICollection<DepartmentContact> DepartmentContacts { get; set; }
}
[Table("uofs00137_dept_contacts")]
public partial class DepartmentContact
{
[Key]
[Column("id")]
public int ID { get; set; }
[Column("department_code")]
[StringLength(7)]
[ForeignKey("VirtualDepartment")]
[Display(Name = "Department Code")]
public string DepartmentCode { get; set; }
[Column("contact_id")]
[ForeignKey("Contact")]
[Display(Name = "Contact ID")]
public int ContactID { get; set; }
[Column("status_code")]
[StringLength(1)]
[Required]
[Display(Name = "Department Contact Status Code")]
public string StatusCode { get; set; }
[Display(Name = "Department Contact Status")]
public string StatusDesc
{
get
{
if (StatusCode == "I")
return "Inactive";
else if (StatusCode == "A")
return "Active";
else
return "Unknown Status";
}
}
public virtual VirtualDepartment VirtualDepartment { get; set; }
public virtual Contact Contact { get; set; }
}
The connection to the database is made via a DbContext class:
public partial class AccountingAdminEntities : DbContext
{
public AccountingAdminEntities() : base("name=AccountingAdminEntities")
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
// This is necessary so that as you add new items to the context
// you don't get the following exception:
// The model backing the AccountingAdminEntities context has
// changed since the database was created. Consider using Code
// First Migrations to update the database.
Database.SetInitializer<AccountingAdminEntities>(null);
base.OnModelCreating(modelBuilder);
}
public virtual DbSet<Contact> Contacts { get; set; }
public virtual DbSet<DepartmentContact> DepartmentContacts { get; set; }
}
Entity Framework follows some conventions to "assume" certain properties, such as a field named id will be a key, but I chose to explicitly define all the mapping properties for clarity.
This approach allowed me to accomplish my goal of generating an MVC app for an existing database and follow our naming conventions without needing to change the database or add foreign key constraints to it.
Other reference links here:
https://blogs.msdn.microsoft.com/efdesign/2010/06/01/conventions-for-code-first/
and here:
https://msdn.microsoft.com/en-us/data/jj591617.aspx#1.1

Querying data from a child table in .Net Mobile service

I have two simple models in .net backend based Azure Mobile Service Project, as shown below & I am not able to query the child table (querying parent table, UserItem, works just fine)
(The Id is nvarchar(128) & is autogenerated as newId by DB)
public class AnswerItem: EntityData
{
public string Content { get; set; }
public UserItem By { get; set; }
public QuestionItem ForQuestion { get; set; }
public double Rating { get; set; }
public string Comment { get; set; }
}
& a child to this UserItem Table as shown below
public class QuestionItem: EntityData
{
public string Content { get; set; }
public bool IsAnswered { get; set; }
public int NumberOfAnswers {get; set;}
public UserItem By { get; set; }
public string ById { get; set; }
public string AtLocation { get; set; }
}
As you notice, the QuestionItem has a FK relationship to UserItem table on ById field (Referencing Id field in UserItem Table)
The issue is I am getting a Bad Request error when I try to query the data from child table
Following are some queries that I tried
private IMobileServiceTable<QuestionItem> questionTable = App.MobileService.GetTable<QuestionItem>();
questions = await questionTable.Where(x=>x.IsAnswered==true).ToCollectionAsync(); (Does not Work)
questions = await questionTable.Where(x=>x.ById="UserIdGoesHere").ToCollectionAsync(); (Does Not Work)
questions = await questionTable.Where(x=>x.Content.StartsWith("q")).ToCollectionAsync(); (This Works)
questions = await questionTable.ToCollectionAsync(); (This Works as well)
If I fire a TSQL query in Sql Server Object explorer they all return correct values.
I am at my wits end on what could be wrong with my approach.
Any help is really appreciated.
Thanks
Supreet
Investigating further the Request it was generating was like this
192.168.2.4:50002/tables/QuestionItem?$filter=(byid eq 'myUniqueGuId')
analyzing fiddler output shows this error
"The query specified in the URI is not valid. Could not find a property named 'byid' on type 'x2Service.DataObjects.QuestionItem'"
Off course there is no fields in the table by the name of 'byid' the one I have is called 'ById' Its the JsonProperty adorner that changed it [JsonProperty(PropertyName = "byid")] In my client class.
Removed the Json Property & it worked just fine

ASP.NET MVC: Need help organizing data structure classes

I come from a PHP/MySQL background, so maybe the problems I'm having with .NET stem from that. After a week of posting how to create an inventory tracker for our computer hardware and software, someone finally told me that he thought I was doing things wrong. The issue was not my code, but my design. That's certainly possible. I'm trying to do this in EF Code First and the idea of generating a database with code is foreign to me. However, I got the database working and everything was pointing to the right thing. But I can't pull what I need from the database.
What I want to do is create a dashboard page that would have categories for the types of hardware. So there would be a list of PCs, a list of monitors, a list of printers, etc. Initially, what I did was based on my knowledge of MySQL. I created a Hardware table (class) and a HardwareTypes table (class). In MySQL, what I would have done is put the ID for the HardwareType in the Hardware table, so I could do joins. Then I can get all of my PCs with a simple inner join.
.NET seems like it's different. It wants to create an intermediate table -- HardwareHardwareTypes, and then connect the two other tables. That seems strange, but OK. But when I go to get all of my PCs, I can't seem to get the help I need to write the query. So please take a look at my query and my classes, and let me know your thoughts.
Query, which returns Hardware, not HardwareTypes -- how do I get HardwareTypes?):
var pcs = db.Hardware.Where(h => h.HardwareType.Any(hwt => hwt.HType == "PC"));
ViewBag.Pcs = pcs.ToList();
Hardware class:
public class Hardware
{
public int Id { get; set; }
public virtual ICollection<DeviceType> Type { get; set; }
public string AssetTagId { get; set; }
public virtual ICollection<Manufacturer> Manufacturer { get; set; }
[Required]
[StringLength(50)]
public string ServiceTagId { get; set; }
[Required]
[StringLength(50)]
public string SerialNumber { get; set; }
[Required]
[StringLength(75)]
public string ProductNumber { get; set; }
// [Required]
[StringLength(20)]
public string PurchaseDate { get; set; }
[StringLength(20)]
public string WarrantyExpiration { get; set; }
[Required]
[StringLength(20)]
public string WarrantyType { get; set; }
public virtual ICollection<Location> Location { get; set; }
public virtual ICollection<HardwareType> HardwareType { get; set; }
[Required]
[StringLength(2000)]
public string Notes { get; set; }
public string POATag { get; set; }
}
HardwareTypes class:
public class HardwareType
{
public int Id { get; set; }
[Required]
[StringLength(128)]
public string HType { get; set; }
public virtual ICollection<Hardware> Hardware { get; set; }
}
Again, if what I need is more of a high-level design change, please let me know. If I need a different query, let me know that. The third (intermediate) table is dynamically generated and it's hard to know how to post that. I'd appreciate any and all help with this. What I need in the end is a list of PCs. Here is some sample seed data:
... new Hardware { AssetTagId = "2134",
Type = device.Where(h => h.DType == "Network Device").ToArray(),
Manufacturer = manuf.Where(h => h.ManufacturerName == "SonicWall").ToArray(),
ServiceTagId = "5243",
SerialNumber = "3456",
ProductNumber = "2345",
PurchaseDate = "2012-10-23",
WarrantyExpiration = "2012-11-12",
WarrantyType = "NBD",
Location = loc.Where(h => h.LocationName == "Paradise Lane").ToArray(),
Notes = "Scrapped",
HardwareType = htype.Where(h => h.HType == "PC").ToArray()}, ...
var htype = new List<HardwareType> {
new HardwareType { HType = "PC" },
new HardwareType { HType = "Monitor" },
new HardwareType { HType = "Printer" },
new HardwareType { HType = "Miscellaneous" }
};
If my seed data is structured wrong, please let me know that. Thanks.
Let me tell you first of all that the same database design which works in PHP/MySQL can work here also.
The easiest approach I will suggest to you is to create a view in the database which joins table Hardware and HardwareType, add nwly created view to you database model and fetch the desired data from the view instead of tables.

MVC4 / EF5 / Auto Increment

I'm not sure how to exactly word my question which is probably why I cannot find an example of this anywhere. I'm playing around with MVC4 & EF5 (Web API too) but I'm not sure how to proceed with the Model as I've never really had to do much with them before. I'm doing something around the Periodic Tablet of Elements and I want to make it so that I have a list built for an element with it's electron configuration. However, I'd like to have it just auto number based on the input order. How can I tell EF to auto-increment a field? Basically like a primary key field without that limitation behind it. Here's what I have so far - I'm just not sure how to proceed:
public class Elements
{
public int ElementID { get; set; }
public string Name { get; set; }
public int AtomicNumber { get; set; }
public string Symbol { get; set; }
public virtual Categories Category { get; set; }
public virtual States State { get; set; }
public virtual Occurences Occurence { get; set; }
public virtual ICollection<Configurations> Configuration { get; set; }
}
public class Categories
{
public int CategoryID { get; set; }
public string CategoryName { get; set; }
}
public class States
{
public int StateID { get; set; }
public string StateName { get; set; }
}
public class Occurences
{
public int OccurenceID { get; set; }
public string OccurenceName { get; set; }
}
public class Configurations
{
public int ConfigurationID { get; set; }
public int Order { get; set; }
public int Value { get; set; }
}
Looking above what I'd like is for anytime a value is added to Configurations.Order the value starts at 1 and increases with each new 'row' but only for that specific ElementID.
Does that make sense? I was looking at using Data Annotations but I couldn't find anything that matched other than a Key Field but that'd make each Order a unique number - which I don't want. I feel like I'm not expressing this correctly because of all the stuff I've been looking at to figure it out, so here's a picture! yay!
This very well could be something that is better off from a programmatic standpoint. Even though this data changes once in a blue moon, I wanted to try and do it through EF if possible just so I know how.
Thanks a ton in advance. Also, if you see any other glaring errors, by all means let me know :) I rarely get to work with this side of web dev so I'm sure there's ways to do things better.
How can I tell EF to auto-increment a field?
You can't. Not even for a simple auto-incrementing primary key. Let alone for a field that should increment in relation to other values.
The HasDatabaseGeneratedOption mapping method is not a way to tell EF how to generate key values. It tells EF if and how the database generates values for properties, so EF knows how to respond to that.
So you either have to generate the order numbers in code, or let the database do it (by a trigger, or by mapping CUD actions on Configurations to stored procedures) and tell EF that the database computes the values by HasDatabaseGeneratedOption(DatabaseGeneratedOption.Computed) in the configuration of the Order property.

RavenDB document design, patching, and index creation

I am revisiting RavenDB after a brief experiment quite a while ago. At the moment I'm considering document design which is nested 3 levels deep, i.e.
public class UserEvent
{
public UserEvent()
{
Shows = new List<Show>();
}
public readonly string IdPrefix = "Events/";
public string Id { get; set; }
public string Title { get; set; }
public List<Show> Shows { get; set; }
}
public class Show
{
public Show()
{
Entries = new List<ShowEntry>();
}
public readonly string IdPrefix = "Shows/";
public string Id { get; set; }
public string EventId { get; set; }
public string Name { get; set; }
public DateTime Date { get; set; }
public List<ShowEntry> Entries { get; set; }
}
public class ShowEntry
{
public readonly string IdPrefix = "ShowEntries/";
public string Id { get; set; }
public string DogId { get; set; }
public string OwnerName { get; set; }
public EntryClass Class { get; set; }
}
First of all, is this a sensible design? A UserEvent generally has a few (less than 6) Show, but a Show can have between tens to hundreds of ShowEntry. I have included DogId in ShowEntry but maybe later I will change it to a property of Dog type. A Dog is of a particular Breed, and a Breed belongs to a Group. The Dog side of the story will have to be another question but for now I'm interested in the UserEvent side.
If my documents are designed this way can I use the Patching API to add items into the Entries collection within a Show? I would like to have an index which will summarise Entries based on Dog properties. Will indexes get processed if an a document is patched?
Your design certainly looks sensible from an outside perspective. The big question you need to ask yourself is, "What do you plan on querying a majority of the time?"
For instance, Show seems to be a fairly common object that would benefit from being an Aggregate Root (from Domain Driven Design). I find that when organizing my documents, the most important question is, "how often do you plan on querying the object."
To answer your last question, Patching should definitely causing re-indexing.