I have been learning Entity Framework Core for about two weeks. I kind of understand how to use it. But when it comes to querying. The query that'is generated from LINQ looks so crazy. It has so many unnecessary duplicated left joins. I cant post my query here and have people help me to optimise it. But I want to know are there a documentation or some article that I can read to understand how queries are generated from LINQ. And all the ways to optimize one query at LINQ level not talking about indexing and stuff related to database.
Related
I cannot find the difference between to two in my search on SO. Can someone explain the difference to me? I feel pretty confident on my understanding of an ORM. But I do not know what SQL Mapping is, and therefore how it is different from an ORM.
I am using Java if it matters.
I control all of the database.
Is there an example of a small understandable SQL Mapper I could look at?
EDIT: I really do not want to embed SQL in my code. That's the devil.
I'm interested in writing a SQL-like query syntax for a CMS I work with. The idea would be that a CMS query could be written in a SQL-ish syntax, and I would convert that to execute through the CMS API.
There would be no field or table selection, so I need some way to get from this:
SELECT WHERE Something = 'something' AND (SomethingElse != 'something' OR AnotherThing == 'something')
Essentially then, I need some way to get the WHERE clauses grouped correctly based on their parentheticals and AND/ORs.
Is there some framework for doing this? Some example of when it's been done? I don't want to re-invent the wheel here, and I know someone else has to have done this in the past.
The answer is yes, there are many frameworks that work in an analog of SQL and convert to SQL. Linq and various Linq translators are a prime example. Knowing exactly which CMS you're working with, and thus which language and platform you're developing in, would be helpful. Some .NET ORMs that support code queries are:
NHibernate - allows use of a SQL-ish language called HQL in strings, or more code-based query construction using expression lists and Linq.
Linq2SQL - On its way out, but for your simpler applications it should be fine. The framework generates DAO classes that map between tables and your domain objects, and you can use coded Linq queries to work with the classes very much like the real tables.
And of course you can use good ol' vanilla ADO.NET with a string SQL query. This has numerous drawbacks, but if you want to have queries in your code, why not make them real SQL? If you wanted to hide your table structure, you could translate table names before submitting queries, so the SQL contained at the web layer (shudder) won't run against your DB.
I'm started looking at Subsonic just yesterday and having trouble figuring out how to do even the most basic tasks. I've watched the demos for ActiveRecord and SimpleRepository, but they don't fit what we want so I'm trying to use the Linq Templates.
The getting started guide for Linq walks through enough to do a query, but how do I do other things like insert a record and get it's auto-increment ID back?
Is there a reasonably comprehensive guide to using Subsonic Linq somewhere?
Well there is this:
http://subsonicproject.com/docs/Using_AdvancedTemplates
Which I can see is a bit sparse :). It works just like Linq to SQL in most cases in that you need to create "DB". That DB allows you to Insert, Delete, etc for all the objects. You can also do aggregates and so on.
using(var db=new NorthwindDB()){
db.Insert.Into("Name").Values("New Name").Execute();
}
The tools used to interact with the DB follow along with our Simple Query tool:
http://subsonicproject.com/docs/Simple_Query_Tool
If you want more things done for you (like getting the new id back, etc) you should stick with ActiveRecord.
Out of curiosity - what doesn't fit?
Say if I have a large dataset, the table has well over a million records and the database is normalized so theres foreign keys and stuff. Ive set up the relations properly and i get a list of the first object applications = EntityLoad("entityName") but because of the relations and stuff the page takes like 24 seconds to load, even when i limit the number of records to show to like 5 it takes an awful long time to load.
My solution to this was create another object that just gets the list, and then when the user wants to , use the object with all the relations and show it to the user. Is this the right way to approach it, or am i missing a big ORM concept?
Are you counting just the time to get the data, or are you perhaps doing a CFDUMP on it or something else visually that could be slow. In other words, have you wrapped the EntityLoad by itself in a cftimer tag to be sure that it is the culprit?
The first thing I would do is enable SQL logging in your Application.cfc. Add logSQL=true to This.ormSettings.
That should allow you to grab the SQL that ORM generates. Run it in an analyzer. See if the ORM SQL is doing somethign crazy. See if it is an index that you missed or something.
Also are you doing paging as Ray talks about here: http://www.coldfusionjedi.com/index.cfm/2009/8/14/Simple-ColdFusion-9-ORM-Paging-Demo?
If not have you tried using ORMExecuteQuery and HQL to enable paging.
Those are my thoughts.
When defining complex domain models with Hibernate - you will sometimes need to tweak the mapping to improve performance. This is especially true if you are dealing with inheritance (not sure how much inheritance is in your model). The ultimate goal is to have your query pulling from as few tables as possible while still preserving your domain model. This might require using the advanced inheritance mappings (more on that in a sec).
LOGGING SQL
As Terry mentioned, you will want to be sure you can log the actual SQL that is being passed to your database (yeah, you don't totally get away from SQL with ORM). Here is a great article on setting up logging for Hibernate in CF9 from Rupesh:
http://www.rupeshk.org/blog/index.php/2009/07/coldfusion-orm-how-to-log-sql/
HIBERNATE MAPPING FILES
Anytime you want to do something beyond the basic, you want to be sure that you are looking at the actual Hibernate mapping files that are generated for your CFC's. Be sure to set the following with all of your hibernate options in Application.cfc:
savemapping = true
While the cfproperty properties allow you to define many aspects of the mapping, there are actually some things that can only be done in the Hibernate mapping files (and there are tons of community resources on this.
INHERITANCE MAPPING
As I mentioned earlier, Hibernate provides different inheritance strategies for mapping. They are Table per Hierarchy, Table per subclass, Table per concrete class, and implicit polymorphism. You can read more about these types in the CF9 docs under Advanced Mapping > Inheritance Mapping or in the Hibernate documentation (as it would take forever to explain each of these).
Knowing how your tables are mapped is very important with inheritance (and it is also where Hibernate can generate some HUGE queries if you don't tweak your setup).
Those are the things I can think of - if you can give some additional information about your domain model - we can look to see what other things might be done to tweak it.
There is a good chance Hibernate is doing it's caching thing. A fair comparison in my mind (everyone please feel free to add) is doing an:
EntityLoad("entity_name") is the same as doing a select * from TABLE
So, in this case, what Hibernate might be doing in instantiating the memory, and caching it a certain way, your database server might do this similarly when you sent such a broad SQL instruction.
I have been extremely interested in ORM the past few weeks and it looks to be a very rewarding undertaking.
For this reason, is there a tiem you would ever load all 500,000 records as a result? I assume not.
I have one large logging table that I will be attacking, I am finding that the SQL good stuff must be there. For example, mark the fields that are indexes as such, this will speed it up incredibly when searching. I am sure the ORM can handle this.
Beyond this:
Find some excellent Hibernate forums, resources, and tutorials so you can learn Hibernate. This isn't really as much a Coldfusion --> ORM issue as what Hibernate might do on it's own. I have ordered a few Hibernate books that I'm waiting on to see how they are.
Likewise there seems to be an incredible amount of Hibernate resources out there where you can bring the Performance enhancement solutions of Hibernate into the Coldfusion sphere. I might be making it too simple, but I see the CF-ORM implementation as a wrapper with some code generation to save us time.
Take a look at implementing filters to cut down your data in the EntityLoad() call.
As recommended in other threads, turn on sql logging and see what sql is being generated. Chances are it might not be what you need. Check out HQL to see if you can form a better statement.
Most importantly, share what you find. I'll volunteer to do the same on this as you've tempted me to go try this out in my spare time a bit sooner than planned.
Faisal, we ran into this with Linq (c# orm).
Our solution was to create simple objects not holding the relational data. For instance, along with Users we had a SimpleUsers object which held little or no relation to any other object and had a limited set of columns.
There could be other ways of handling this but this approach helped tremendously with the query speed.
Struggling between choosing linq2sql and nhibernate.
Let me give you some insight in the application in point form:
this is a asp.net mvc application
it will have lots of tables, maybe 50-60 (sql server 2008)
i would want all the basic crud logic done for me (which I think nhiberate + repository pattern can give me)
i don't have too complicated mappings, my tables will look something like:
User(userID, username)
UserProfile(userID, ...)
Content(contentID, title, body, date)
Content_User(contentID, userID)
So in general, I will have a PK table, then lots of other tables that reference that PK (i.e. FK tables).
I will also have lots of mapping tables, that will contain PK, FK pairs.
Entity wise, I want User.cs, UserProfile.cs and then a way to load each object.
I am not looking for a User class that has a UserProfile property, and a Content Collection property (there will be maybe 10-20 tables that will related to the user, I just like to keep things linear if that makes sense).
The one thing that makes me learn towards nhibernate is: cross db potential, and the repository pattern that will give me basic db operations accross all my main tables almost instantly!
Since you seem to have a quite straight forward mapping from class to table in mind Linq to SQL should do the trick, without any difficulties. That would let you get started very quickly, without the initial work of mapping the domain manually to the database.
An alternative could be using NHibernate + Fluent NHibernate and its AutoMapping feature, but keep in mind that the Fluent NHibernate AutoMapping is still quite young.
I'm not quite sure I understand what you want your entities to look like, but with Linq to SQL you will get a big generated mess, which you then could extend by using partial classes. NHibernate lets you design you classes however you want and doesn't generate anything for you out of the box. You could kind of use POCO classes with Linq to SQL but that would take away all the benefits of using Linq to SQL rather than NHibernate.
Concerning the repository pattern, and the use of a generic repository, that can be done quite nicely with Linq to SQL as well, and not only with NHibernate. In my opinion that is one of the nice things about Linq to SQL.
If you probably will need support for other databases than SQL Server, NHibernate is the only choice. However, if it probably won't be an issue I would recommend not using that as the primary factor when choosing. The other factors will probably influence your project more.
Conclusion:
All in all, I would recomment Linq to SQL, in this case, since it would let you get started quickly and is sufficient for your needs. The precondition for that is that you don't have a problem with the thought of having generated, messy code in your domain, and that you are quite sure there will not be any need to support other databases in the future. Otherwise I would recommend NHibernate, since it is truly an awesome ORM.
linq2sql really wants you to work with 1 table per class mapping. So if you have a UserMaster and a UserDetail table, you are looking at two objects when using default linq object generation. You can get around that by mapping linq entities to business entities (see Rob Conery's storefront screencasts), but then you are back to writing object mapping code or using something like Automapper.
If you want to be able to split your classes across multiple tables, then I'd say go with NHibernate. If not, then linq has a lower learning curve.
The only way I'd ever use nHibernate in through Castle Project's ActiveRecord library. Otherwise, nHibernate becomes its own little infrastructure project. Check out some questions in the nHibernate tag to see what I'm talking about.
The only thing I might change about AR is to return results of SELECT operations as List instead of T[]. Of course, with the source code in C# I can do that if I want.
With ActiveRecord, the mapping information is saved in attributes you decorate your classes with. It's genius and I am a huge proponent of the pattern and this particular library.