This is in reference to a question I asked earlier. Aside from viewing the SQL generated by NHibernate on calls to the database, what is the best way to find bottlenecks between NHibernate and the DB? In particular, I have queries that return very quickly when run directly against the database, but very slow (over 3-4x) return times when running the code in unit tests and on the web page. I am relatively sure this has something to do with the way I have mapped my tables and the primary keys. How can I dig in further to see where my slow areas are occurring? Are there other tools available? I know this is an extremely broad question, but I have not had the need to explore these problems yet. Any help would be greatly appreciated.
AFAIK there is no single tool to profile NHibernate, yet. This is about to change with Ayende's NHIbernate Profiler. In the meantime, you can use a combination of code profilers (e.g. dotTrace), SQL Server Profiler, the NHibernate logger, and static analysis, i.e. if you know about the SELECT N+1 problem, most of the time you can spot it just by looking at the code.
EDIT: NHProf is now available!
I use JProbe for analyzing my code. It can show the amount of time spent in any particular location in my code, and where the bottlenecks are. I'm sure there are other tools available though, some of which may be cheaper.
for now use Sql profiler. Later on you can BUY Ayende's Nhibernate Profiler. Also you can log the sql and check.
Related
I am making an application that will serve as a demonstration of the data analysis capabilities our team can provide. I am very new to django and a the auto-generated APIs seem pretty cool, but I worry about scalability and having the quickness that only a carefully constructed and queried database can provide. Has anyone been in this situation and regretted/been satisfied with there choice of raw queries vs django APIs?
Django's ORM is great. It is one of the most complete and easy to use ORM I have ever encounter, but as every ORM, it has its limitations.
If your application requires full control of the database and very efficient queries, you might consider other approaches and compare them to Django and see which one fits better. You'll have to do some research.
Django is great for developing very fast complicated database applications, but I'm sure --if the application grows long enough-- sooner or later you'll have to start working directly with your database engine for optimization reasons. ORMs are generic tools, so database engine specific functions will not be available.
There is no rule to decide wheter or not django is gonna work for you but, one thing I can tell you is that its ORM helps you get your app started very quick and if you find some specific circumstance where you need to customize your SQL, then you can do it in Django as well. If not, just create a Python module which handle the database as you like in those specific circumstances and use it from your Django code. That probably will be the best way if you need to show your very efficient data analysis capabilities.
I hope this bring some light, it is a very wide question. One thing I'm sure is that you won't regret until your app grows big enough and when it does, you'll have the resources to find great programmers that could twist Python to handle every specific situation that behaves odd with Django's database accessing tools.
Found this link which may be helpful.
Good luck!
I'd consider this a case of premature optimization.
Django ORM is good enough in a general case, provided that your database is reasonably designed, has appropriate indexes, etc.
In > 90% cases this will be adequate, and often optimal.
When you will have identified specific complicated and slow queries, have reviewed the ORM-generated SQL and came up with a better query, then you may add a special case for it.
Maybe you will have more than one such special case. I still think that ORM will save you a lot of legwork in the 90% of database access cases where it is adequate.
Besides querying, an ORM allows you to describe the DB schema, its constraints, ways to recreate it and migrate it between versions, etc. Even if the ORM would not let you query the DB, these management capabilities would be enough reason to use an ORM.
Hi
I need to optimize an application which is already there for a long time. Optimization will include move inline queries from php pages to "stored procedures", get rid of sub queries and convert them to "joins" etc etc.
I guess the best way is to use benchmarking tools for this purpose, but is there any GUI based tool available which I could use with Windows 7? Please help!
Also moving the inline queries to stored procedures and getting rid of sub queries, will that help in a major performance boost? Please feel free to express your opinion.
The major focus is on finding a suitable tool for benchmarking purposes however. Just a quick question will "Mysql workbench" help in this scenario? Pls advise.
Many thanks for your time in advance. Any kind of help is much appreciated.
I do not know about the moving the inline queries to stored procedures. It really depends if you are going to use that query a lot or not. Switching from usign many queries to JOIN coudl be a major improvement in most cases, depends how much extra queries you were running initially.
As for benchmarking, well you can always use even phpMyAdmin to see how much time are takign certain queries and/or use a build in application benchmakring/profiling tool (even created from you PHP code) to bench the performance. This things usually do not have a straight and simple answer :(
Link with some suggestions http://beerpla.net/2008/04/16/mysql-conference-liveblogging-benchmarking-tools-wednesday-425pm/ maybe try WAST http://west-wind.com/presentations/webstress/webstress.htm
If you have a DB abstraction layer use it to log the performance of the queries and see if there are any that repeat too often or take too much time and also in which script they where called.
I'm working on a small Windows Form application that will be run on a Netbook computer. I will control the hardware/environment, meaning I provide the hardware and software to the end user. It will have a single database on the local drive that only this one app will access. It will have a couple tables and a few hundred (or maybe a couple thousands) rows in one of the tables. No foreign keys, etc. Really simple. I just need a place to store this data and perform simple queries and map to objects (ORM).
I understand the basics of Nhibernate and EF4 and have experimented a little with both. I'd use EF4 with POCOs if I decided to use EF.
I don't think performance is an issue because its a small amount of data. But, Netbooks are not real powerful so I'm wondering which of these two products would offer me a more lightweight solution.
We're a Microsoft shop and not using EF4 yet, but I think we may be going that way as our data engine of the future, so this may influence my decision. But this app is kind of an island of its own so I could potentially use nhibernate without too much political fallout. :) My general impression of EF4 and its wizards and generators and magic is that its bloated. I may be wrong, but thats the feeling I get. I'd hate to select EF4 and find out its bogging down my Netbook's performance.
Any comments are welcome. I know this is a wide open subject. ;)
I don't think the difference between the two is even measurable with a small amount of data. The sql query itself will take much longer than the work done by the orm.
Yes, this is a wide open subject. You will only know the difference for the exact case you use when you measure it.
Personally, I wouldn't use an orm at all for just a couple of tables.
I also wouldn't think about performance before I have a performance problem.
I like NHibernate, and still wait EF to impress me, but for your not so complex application I would not use neither of them.
Instead I'll advice to use Linq2Sql, the most easiest solution and enough powerful.
I think NHibernate and EF is for more complex applications
Say if I have a large dataset, the table has well over a million records and the database is normalized so theres foreign keys and stuff. Ive set up the relations properly and i get a list of the first object applications = EntityLoad("entityName") but because of the relations and stuff the page takes like 24 seconds to load, even when i limit the number of records to show to like 5 it takes an awful long time to load.
My solution to this was create another object that just gets the list, and then when the user wants to , use the object with all the relations and show it to the user. Is this the right way to approach it, or am i missing a big ORM concept?
Are you counting just the time to get the data, or are you perhaps doing a CFDUMP on it or something else visually that could be slow. In other words, have you wrapped the EntityLoad by itself in a cftimer tag to be sure that it is the culprit?
The first thing I would do is enable SQL logging in your Application.cfc. Add logSQL=true to This.ormSettings.
That should allow you to grab the SQL that ORM generates. Run it in an analyzer. See if the ORM SQL is doing somethign crazy. See if it is an index that you missed or something.
Also are you doing paging as Ray talks about here: http://www.coldfusionjedi.com/index.cfm/2009/8/14/Simple-ColdFusion-9-ORM-Paging-Demo?
If not have you tried using ORMExecuteQuery and HQL to enable paging.
Those are my thoughts.
When defining complex domain models with Hibernate - you will sometimes need to tweak the mapping to improve performance. This is especially true if you are dealing with inheritance (not sure how much inheritance is in your model). The ultimate goal is to have your query pulling from as few tables as possible while still preserving your domain model. This might require using the advanced inheritance mappings (more on that in a sec).
LOGGING SQL
As Terry mentioned, you will want to be sure you can log the actual SQL that is being passed to your database (yeah, you don't totally get away from SQL with ORM). Here is a great article on setting up logging for Hibernate in CF9 from Rupesh:
http://www.rupeshk.org/blog/index.php/2009/07/coldfusion-orm-how-to-log-sql/
HIBERNATE MAPPING FILES
Anytime you want to do something beyond the basic, you want to be sure that you are looking at the actual Hibernate mapping files that are generated for your CFC's. Be sure to set the following with all of your hibernate options in Application.cfc:
savemapping = true
While the cfproperty properties allow you to define many aspects of the mapping, there are actually some things that can only be done in the Hibernate mapping files (and there are tons of community resources on this.
INHERITANCE MAPPING
As I mentioned earlier, Hibernate provides different inheritance strategies for mapping. They are Table per Hierarchy, Table per subclass, Table per concrete class, and implicit polymorphism. You can read more about these types in the CF9 docs under Advanced Mapping > Inheritance Mapping or in the Hibernate documentation (as it would take forever to explain each of these).
Knowing how your tables are mapped is very important with inheritance (and it is also where Hibernate can generate some HUGE queries if you don't tweak your setup).
Those are the things I can think of - if you can give some additional information about your domain model - we can look to see what other things might be done to tweak it.
There is a good chance Hibernate is doing it's caching thing. A fair comparison in my mind (everyone please feel free to add) is doing an:
EntityLoad("entity_name") is the same as doing a select * from TABLE
So, in this case, what Hibernate might be doing in instantiating the memory, and caching it a certain way, your database server might do this similarly when you sent such a broad SQL instruction.
I have been extremely interested in ORM the past few weeks and it looks to be a very rewarding undertaking.
For this reason, is there a tiem you would ever load all 500,000 records as a result? I assume not.
I have one large logging table that I will be attacking, I am finding that the SQL good stuff must be there. For example, mark the fields that are indexes as such, this will speed it up incredibly when searching. I am sure the ORM can handle this.
Beyond this:
Find some excellent Hibernate forums, resources, and tutorials so you can learn Hibernate. This isn't really as much a Coldfusion --> ORM issue as what Hibernate might do on it's own. I have ordered a few Hibernate books that I'm waiting on to see how they are.
Likewise there seems to be an incredible amount of Hibernate resources out there where you can bring the Performance enhancement solutions of Hibernate into the Coldfusion sphere. I might be making it too simple, but I see the CF-ORM implementation as a wrapper with some code generation to save us time.
Take a look at implementing filters to cut down your data in the EntityLoad() call.
As recommended in other threads, turn on sql logging and see what sql is being generated. Chances are it might not be what you need. Check out HQL to see if you can form a better statement.
Most importantly, share what you find. I'll volunteer to do the same on this as you've tempted me to go try this out in my spare time a bit sooner than planned.
Faisal, we ran into this with Linq (c# orm).
Our solution was to create simple objects not holding the relational data. For instance, along with Users we had a SimpleUsers object which held little or no relation to any other object and had a limited set of columns.
There could be other ways of handling this but this approach helped tremendously with the query speed.
I am looking for some up to date information comparing NHibernate and iBATIS.NET. I found some information searching Google, but a good bit of it applies either to the Java versions of these products or is dated.
Some specific things I am interested in:
Which is better if you control both the data model and the application?
iBATIS is repeatedly called simpler to learn - does this have long-term maintenance consequences (i.e. easy to start, hard to maintain)?
Do both make it easy to switch the underlying database vendor?
How skilled do your developers need to be with SQL?
Any major feature that one has that the other lacks?
Is either product more suitable for a particular type of application?
Real world examples of observed benefits and drawbacks are appreciated!
EDIT: Thanks for the information. I am doing my own evaluation as well. One thing I am wondering about still, does iBATIS help you to save/update complex object graphs? It seems like NHibernate is nice in that I can pass it a root object and it figures out the details of what, if anything, needs to be updated in the database.
I made some research a while ago.
One specific question from me, might give you some additional information:
Would you use NHibernate for a project with a legacy database, which is partly out of your control?
Some of your points of interest I can answer:
Which is better if you control both the data model and the application?
I can answer it the other way around: If you don't have control over the data model and thus facing some legacy database, iBatis is the better choice.
iBATIS is repeatedly called simpler to learn - does this have long-term maintenance consequences (i.e. easy to start, hard to maintain)?
It depends what you want to do with it. If you have a domain driven development approach then iBatis might get painful by time. If you just do simple data manipulation and don't have a full blown domain model then nHibernate might be a overkill by the time.
Do both make it easy to switch the underlying database vendor?
Both have mechanisms to shield you off from a specific database vendor, but I admit that have not done intense research in this direction.
How skilled do your developers need to be with SQL?
When you use iBatis, you need more SQL skills than NHibernate. Using iBatis you always need to code some SQL. NHibernate doesn't require you to code SQL statements -- it even can do the DDLs for you. Powerful features will require you to go to old good SQL, which will be inevitable.
Some other points:
I personally find that iBatis much more lightweighter. You can get things done very quickly. NHibernate is more powerful, but has much more features, which you can use in wrong way.
It is possible to combine the use of NHibernate and iBatis! You can use NHibernate for your business logic. For reporting purposes, where you just read data out of tables, fallback to iBatis.
If your application has a longer life cycle and a lot of business logic, consider NHibernate. It has a lot of feature aiding you in handle business objects.
The community around NHibernate is very active and come up with useful tools.
In a sense it's comparing apples to oranges.
Which is better if you control both the data model and the application?
They both work with normalized databases well, so they are more-or-less equal if you can shape the db. iBatis is better at mapping to legacy databases since it doesn't actually care about the database structure at all. It only cares about the shape of the result set.
.iBATIS is repeatedly called simpler to learn - does this have long-term maintenance consequences (i.e. easy to start, hard to maintain)?
It is much simpler, but that is because it has a much smaller featureset. I don't think it has any ticking timebomb long term maintenance issues.
Do both make it easy to switch the underlying database vendor?
Yes
How skilled do your developers need to be with SQL?
Both require a good knowledge of SQL. With iBatis, you still have to write the sql queries/procs. With NHibernate you have to know how to write NHibernate queries to get effective SQL. Neither are a replacement for SQL knowledge.
Any major feature that one has that the other lacks?
iBatis is a datamapper (a term used on the iBatis site). NHibernate is a full-blown Object Relational Mapper. iBatis is a great way to go if you primarily want something that takes the monotony out of mapping objects to result sets. However, it doesn't go all the way in trying to solve the object/relational mismatch. NHibernate has many more features such as dirty tracking, caching based on identity /identity map, flexible querying, dynamic sql, batching etc... NHibernate is much more dynamic in that it can do many things in one trip to the DB that could take iBatis several trips.
We recently posted an article comparing these two tools, and I think many of your questions are addressed. The article is here on our wiki site.