And, if I can, does that mean I lose my advantage of treating the results as objects? I find complex queries confusing in many ORMs, not just Django's. But, it is probably because I have never really used an ORM. Does anyone use straight up SQL anymore?
edit: Am I defeating the purpose of having a framework if I bypass the ORM completely? They all have a "nifty" ORM, but when it comes to queries with lots of subqueries, derived tables, it doesn't look pretty.
Using Django's QuerySet API you have different possibilities:
You can use extra() which will return a queryset which evaluates to model objects. Therefore it is, as the name says, somehow limited, because for returning model instances it is necessary to eg. query the model's table. But you have the possibility to add additional SQL eg. the WHERE or ORDER clause. Querysets that use extra() can still use the features of the ORM - like chaining multiple filter() for example.
raw() returns a RawQueryset which also can be iterated over to get model instances, but you loose a lot of features that the ORM would normally provide.
And of course you can execute SQL directly, using a low level connection cursor API (no model instances of course).
Study the documentation on raw queries, there's also a lot of information on eg. how to map a model's fields on the data coming from a raw query and documeting a few gotchas when passing parameters into the query.
To also answer your edited question: I wouldn't use raw SQL when you can do it with the ORM, but of course the ORM is limited and if you need to do some more complex stuff you will always have to switch to SQL (but sometimes using extra() is enough-so you can still use the advantages of the ORM). Don't forget that the ORM works with every DB backend, while the custom SQL might not work with every database.
You can use raw SQL to either return objects; or if you want you can bypass the ORM completely.
Related
In a JPA project I need to display a table whose data comes from 5 related entities.
Without JPA I could write a sql query which joins the 5 database tables together and filters according to some criteria.
Suppose that the fields involved in the filtering criteria are only those of the first entity.
Using JPA I can load filtered instances of the first entity and navigate through the properties till the final entity.
My concern is that way the number of queries to the database can explode if I cannot use or do a mistake with the fecttype=eager annotation.
Which is the best approach in such cases ?
I would like to have a strict control over the sql queries that will be executed, so I can optimize them, but if I write the sql query with the joins by hand do I have to use the 'old' resultset to retrive the data ?
You can use JPA's built-in query language, the JPQL, can't you? (It does have a JOIN operator for sure.) Be aware though that this is not standard SQL, only something similar, so read the JPQL docs thoroughly. Yes, this is still plain text queries embedded in Java code, which is a shame, but hey, that's how far Java can go supporting the development process.
The main advantage here is that you get entity objects as the result of your queries - although you still need to cast them from Object. You can also use the objects (records) and their member variables (attributes) directly in the query string, so this is a step up from good old JDBC.
Alternatively you could also choose the Criteria API, but frankly, my experiences were not very good with it. The syntax is quite horrible and you basically end up building the low-level query yourself. This is clearly Java at its worst... but at least Strings containing queries can be eliminated from the code. I'm not sure it's worth it though.
Check this page for more information and examples:
http://download.oracle.com/javaee/6/tutorial/doc/gjise.html
although I am pretty decent at PHP I am new to frameworks.
started with CI last week and found myself looking at Kohana this week.
I have few questions to that regard:
why ORM vs traditional SQL or active queries?
if the model must fetch data from DB , how come in ORM most of the action happens in the controller ( or so it seems ) ie ( $data=$q->where('category', '=', 'articles')->find_all();}
how would I do a conditional query in ORM? ( something like if (isset($_GET['category']))...etc ) if the condition is passed to the model? or should the controller do all the conditions
FYI my queries tend to have numerous joins and my limited knowledge tells me that I should have a query controller that passes queries parameters to a query model which does the query and returns results.
Please let me know if my understanding is correct
thank you very much
ORM is some kind of wrapper over the DB layer. So, you just call $user->find($id) instead of $db->query('select * from users where id='.$id) or DB::select()->from('users')->where('id', '=', $id)->limit(1)->execute(). You declare model params (table name, relations etc) and use only model methods to work with its data. You can easily change DB structure or DB engine without modifying a lot of controller code.
Agree with Ikke, controller should avoid model specific data like query conditions. For example, create method get_by_category($category).
See #2. All args you want should be passed into model method (this can be done using chaining, like $object->set_category($category)->set_time_limit(time())->limit(10)).
ORM is just another way to get at your data. The idea is that there are many common kind of operations, and that could be automated. And because the relations between tables can easily be translated to objects referencing eachother, ORM was created.
It's up to you if you want to use the supplied ORM module. There are others which are also commonly used (like sprig, jelly and auto-modeler).
My personal opinion is to limit that kind of operations to a minimum. Very simple operations can be done this way, because it barely produces any advantages in placing them in the model, but the best way is to try to put the business logic as much in the models as possible.
Another point is that it should be the view that gets the data from the models. That way, when you want to reuse a view, very little code has to be duplicated. But to prevent too much logic getting in your views, it's recommended to use so-called viewclasses which contain the logic for your views, and is the interface for your views to talk to.
There is a Validation library to make sure that all the data for your model is correct. Your models shouldn't know about $_GET and $_POST, but the data from those arrays can be passed to your models.
My gut tells me that advanced NHibernate users would be against it and I have been looking for actual analysis on this and have found nothing, I'd like for the answer to address these questions:
What are the pros/cons of using it?
Are there any performance implications, both good or bad (e.g. use it to call stored procedures?)
In which scenarios should we use/avoid it?
Who should use/avoid it?
basically, what are the reasons to use/avoid it and why?
CreateSQLQuery exists for a reason, which is executing queries that are either:
Not supported
Hard to write
using any of the other methods.
Of course it's usually the last choice, because:
It's not object oriented (i.e. you're back to thinking of tables and columns instead of entities, properties and relationships)
It ties you to the physical model
It ties you to a specific RDBMS
It usually forces you to do more work in order to retrieve entities
It doesn't automatically support features like paging
But if you think it's needed for a particular query, go ahead. Make sure to learn all the other methods first (HQL, Linq, QueryOver, Criteria and Get) to avoid doing unnecessary work.
One of the main reasons to avoid SQL and use HQL is to avoid making the code base dependent on the RDBMS type (e.g. MySQL, Oracle). Another reason is that you have to make your code dependent on the table and column names rather than the entity names and properties.
If you are comparing raw SQL to using the NHibernate LINQ provider there are other compelling reasons to go for LINQ queries (when it works), such as type safety and being able to use VS reference search to determine in what queries a certain table or column is referenced.
My opinion is that CreateSQLQuery() is a "last way out" option. It is there because there are things you cannot do with the other NHibernate APIs but it should be avoided since it more or less goes against the whole idea of using NHibernate in the first place.
A colleague of mine is currently designing SQL queries like the one below to produce reports, which are displayed in excel files through an external data query.
At present, only reporting processes on the DB are required (no CRUD operations).
I am trying to convince him that it would be better to use a ruby ORM in order to be able to display the data in a rails/sinatra app.
Despite the obvious advantages in displaying the data, what advantages are there for him in learning to use an ORM like Sequel or Datamapper?
The SQL queries he is writing are clearly quite complex, and being relatively new to SQL, he often complains that it is very time-consuming and confusing.
Is it possible to write extremely complex queries with an ORM? and if so, which is the most suitable(I have heard Sequel is good for legacy dbs)? and what are the advantages of learning ruby and using an ORM versus sticking with plain SQL, in making complex database queries?
I'm the DataMapper maintainer, and I think for complex reporting you should use SQL.
While I do think someday we'll have a DSL that provides the power and conciseness of SQL, everything I've seen so far requires you to write more Ruby code than SQL for complex queries. I would much rather maintain a 5 line SQL query than 10-15 lines of Ruby code to describe the same complex operation.
Please note I say complex.. if you have something simple, use the ORM's build-in finders. However, I do believe there is a line you can cross where SQL becomes simpler. Now, most apps aren't just reporting. You may have alot of CRUD type operations, for which an ORM is perfectly suited and far better than doing those things by hand.
One thing that an ORM will usually provide is some sort of organization to your application logic. You can group code based around each model in the same file. It's usually there that I'll put the complex SQL query, rather than embedding it in the controller, eg:
class User
include DataMapper::Resource
property :id, Serial
property :name, String, :length => 1..100, :required => true
property :age, Integer, :min => 1, :max => 130
def self.some_complex_query
repository.adapter.select <<-SQL
SELECT ...
FROM ...
WHERE ...
... more complex stuff here ...
SQL
end
end
Then I can just generate the report using User.some_complex_query. You could also push the SQL query into a view if you wanted to further cleanup this code.
EDIT: By "view" in the above sentence I meant RDBMS view, rather than view in the MVC context. Just wanted to clear up any potential confusion.
If you are writing your queries by hand you have the chance to optimize them. When I look at that query I see some potential for optimizations (E.ICGROUPNAME LIKE '%san-fransisco%' or E.ICGROUPNAME LIKE '%bordeaux%' wont use an index = Table Scan).
When using an OR Mapper (the native Objects/Tables) for reporting you have no or little control over the resulting SQL Query.
But: You could put that query in an View or Stored Procedure and map that View/Proc with an OR Mapper. You can optimize your queries and you can use all features of your Application Framework.
Unless you're dealing with objects, an ORM is not necessary. It sounds like your friend simply needs to generate reports, in which case pure SQL is just fine so long as he knows what he's doing (e.g. avoiding SQL injection issues).
ORM stands for "Object-Relational Mapping". If you don't have the "O" (objects), then it's probably not a good fit for your app. Where ORMs really shine is in persisting objects to the database and loading them from a database.
ORM stands for Object Relational Mapping - but looking at the query your friend seems to be wanting a pretty specific table of sums and other items... I've not used Ruby's Sequel, but I've used Hibernate, and Python's SQLAlchemy (for Django/Turbogears) and while you can do these sorts of queries, I don't believe that is their strength.
The power of ORM comes from being able to finding Foo->Bar object relationships, say you want all the Bar objects for Foo's field greater then X... That sort of thing. Therefore I would not classify an ORM as a "good" solution, though moving to a real programming language like Ruby and doing the SQL through it instead of Excel... that in itself is a win.
Just my 2 cents.
In a situation like that, I'd probably write them by hand or use a View (if the DB you're using supports views)
ORM's are used when you have Objects (Business Objects). I am therefore assuming that you have an application with which you creating and Managing the Business Objects that are ultimately saved into the database. If you have then you have almost definitely got some representation of the relationships and probably many of the calculations you are going to use in reports. The problem with using SQL to directly access your database for reports is simply maintainability.
You typically put a lot of effort into ensuring that your Business Objects hide any details of their database. You implement business rules and do common calculations in your Business Objects. Build a common language for all members of the team etc etc. You then use an ORM to map to the database and use Habanero or NHibernate or something like that to do this. This is all great. We do this all in the name of Maintainability and is great. You can migrate your application change your design etc etc.
You now go and write SQL to run reports over time you have hundreds of report. Firstly they often duplicate logic you already have in your BusinessObjects (Usually without any tests) and even worse Bham Damb sorry maintainability is now stuffed forget about moving a that field from one table to another forget about splitting that table into two changing that relationship etc you have a number of reports that are going to break unexpectedly.
The problem with quering through your Domain Objects/Business Objects is simply one of performance.
In summary if you are using Domain Driven Design or Business Object concepts try to use these for reports. (You will probably run directly from DB using SQL or stored procs for performance reasons but try limit these use your Business Objects first and then use SQL).
The other option of course is using a separate reporting database (Like some of the BI concepts) The mapping from your transactional DB to your reporting DB is therefore in one place and easily changeable in cases where you want to change your design.
Domain Objects (Business Objects) and ORMs have all the knowledge to allow you to start building high performing queries that run directly on the Database while using the Domain Terminology. Lets hope that these continue to evolve to a point where this is a reality.
Until then if you are using Business Objects in your application try use them for Reporting when performance is an issue resort to SQL.
what's the purpose of entity sql, i mean if you have linq to entities why would you need to write queries in string, are there any performance reasons or something ?
LINQ to Entities does not allow you access to every feature of your database. Being able to "reach into" the database is sometimes necessary for advanced queries, either to pull them off in the first place or to improve the sometimes horrible choices that the LINQ to Entities system will make about your query.
That said, I believe that LINQ to Entities should be the first tool reached for. If the performance becomes a problem, or you have something more complex I would then encapsulate that problem piece in a stored procedure and call that. There is no reason for strings being used as the basis of queries these days.
ESQL does allow you to choose a collation on a where clause, something which isn't supported in LINQ-to-Anything. This can be genuinely useful. ESQL also allows you to specify the precise type you want returned when types inherit from each other (as opposed to LINQ's OfType, which returns instances of a certain type and any subtype). Beyond that, I can't think of a great reason to use it. It's occasionally nice to be able to build queries in strings, but DynamicQuery/Dynamic LINQ is generally good enough in the very rare cases where this is necessary.
I think (perhaps cynically) that the "real" purpose of ESQL is "it predates LINQ."
Regarding Godeke's point of fixing non-optimal queries, I have yet to see one I couldn't fix by changing the LINQ expression. Both ESQL and L2E end up as CCTs, so the SQL generation pipeline is the same.