With beta4 and latest beta5 the DB-Feature-Implementation appears to have pretty much finished. There's a couple of tutorials out there how to handle a single Database using the TableGateway Pattern, but it appears there is none for handling M-N-Relationships.
In ZF1 we had findDependantRowset() on the TableGateway which was kind of dirty, as this simply was a second Query to the databse which pretty much isn't always neccessary.
In ZF2 i expected there to be a way to have good Joins mapping to specified models, but i can't find anything within the code. Maybe i'm blind, maybe there really isn't anything like this.
Has anyone of you managed to handle joins and models all together in ZF2? If so, please be so kind to instruct me how to do it, hint me to specific points of the documentation or link me some blogpost to one who has done it.
Thanks in advance guys!
The obvious solution if you need a generic solution is to use Doctrine ORM or Propel.
If you want to use Zend\Db, then within your concrete table gateway classes, you should write a specific method that retrieves the correct rows from the linked table. This way you can ensure that the SQL is optimised for the query that you need.
Related
I am locking for a read-write lock in Django, using PostgreSQL.
I know select_for_update
On top, I need SELECT FOR SHARE
I founds this super old Django-ticket
I couldn't find any third party library implementing this for Django
Now I am wondering:
Is there a good reason this was not implemented in the Django-ORM?
Does anyone know any third party library for this?
Or any easy work around?
Can't you wrap the logic in pl/pgsql function that uses select for share and you then call the function from django?
Is there a good reason this was not implemented in the Django-ORM?
The ticket you've posted probably provides the reason: no one is motivated enough to write a patch.
Does anyone know any third party library for this?
Not me, sorry.
And if by any chance you are thinking about ditching Django for some other ORM then you must ask yourself: "There is a feature I need that's missing in Django... what features will I miss in this other ORM?"
Or any easy work around?
Probably not. But here are some options:
Every ORM I know has an escape hatch to raw SQL. You probably know that, but reluctant to use it... But, as you also lack the motivation to make a pull request, that probably means that you do not have hundreds of requests that require SELECT FOR SHARE functionality. So you should consider it. Performing raw SQL queries
Stored procedures as steve mentioned
https://docs.djangoproject.com/en/3.0/topics/db/sql/#calling-stored-procedures
The last comment on the ticked you've posted is from a man (David Schwärzle) who claims that he has a solution (not for PostgreSQL specifically but a solution nevertheless)... maybe you should try and contact him.
Haven't tried it, but probably the way you can add the desired functionality is by Writing your own Query Expressions
You can easily implement with a raw query.
from django.db import connection
query = f"""SELECT * FROM "appname_modelname" WHERE id={obj_id} FOR SHARE"""
with connection.cursor() as cursor:
cursor.execute(query, None)
obj_id is python variable.
appname_modelname is name of table created in your db by django. Django by default combines lowercased app name and model name with underscore.
While developing with Fluent nHibernate, I notice that on relationships I can specify a Fetch property, with possible options of Select(), Join(), and Subselect().
I did some searches for these and yielded very little information. I did find them in the nHibernate documentation and the fluent nHibernate documentation, but it does little other than give their signatures, which doesn't help me too much.
I was wondering if there is any real explanation for what these are, and what they really do. I've been rather perplexed myself. From my own evaluation they seem to change the way that referenced entities are pulled into the object graph, but I've yet to entirely discern how they change it, and which one is optimal for what situation...
I did find this blog post (http://www.mkyong.com/hibernate/hibernate-fetching-strategies-examples/) that has a little bit of detail but I'm still pretty perplexed about the entire situation. I've also seen other examples that state using Select() is more optimal, but the reasoning behind it. Additionally I found a post at (http://community.jboss.org/wiki/AShortPrimerOnFetchingStrategies) that is geared towards the original Java Hibernate platform, but I presume the concept is the same. In this one, my theory seems to be blown a bit as it focuses more on the lazy loading aspect of what they do, but I've still not seen any really flat examples.
Join fetching - NHibernate retrieves the associated instance or collection in the same SELECT, using an OUTER JOIN.
Select fetching - a second SELECT is used to retrieve the associated entity or collection. Unless you explicitly disable lazy fetching by specifying lazy="false", this second select will only be executed when you actually access the association.
Subselect fetching - a second SELECT is used to retrieve the associated collections for all entities retrieved in a previous query or fetch. Unless you explicitly disable lazy fetching by specifying lazy="false", this second select will only be executed when you actually access the association.
Check out the fetching strategy document # The Nhibernate Documentation
I'm not really familiar with nHibernate (I work with Hibernate and Java), but based on analogy, this enables you to specify association/collection property which you want to load eagerly, with the given entity. This is useful when you don't have full control over (n)Hibernate sessions (i.e if some other framework like Spring in Java is taking care of sessions/transactions).
So your assumption is basically correct.
Select, Join, and Subselect are the ways to obtain the related property, and determine what kind of query will be performed in database. Which one is optimal, really depends on the situation you have.
Hope this helps a little,
Cheers.
Has anyone used the EmpireDB from Apache? I am planning to use it and would like to know if there are any known issues with this new concept?
I have used ORM like Hibernate and JPA. But EmpireDB sounds like it easy.
Anyone knows its limitations then please let me know.
Thanks.
EmpireDB places impositions on your model classes; you can't just persist your designed model. You add a dependence to their codebase (which could change later). You have to write a lot more code to use it (using its Tutorial) than you would using JDO or JPA. How that is somehow "easy" i can't think.
If you need full control over the sql, you don't want to use entities for everything and hate sql/jpql strings it is the perfect tool.
It depends largely on what you plan to do with it.
I have been working on sql server and front end coding and have usually faced problem formulating queries.
I do understand most of the concepts of sql that are needed in formulating queries but whenever some new functionality comes into the picture that can be dont using sql query, i do usually fails resolving them.
I am very comfortable with select queries using joins and all such things but when it comes to DML operation i usually fails
For every query that i never done before I usually finds uncomfortable with that while creating them. Whenever I goes for an interview I usually faces this problem.
Is it their some concept behind approaching on formulating sql queries.
Eg.
I need to create an sql query such that
A table contain single column having duplicate record. I need to remove duplicate records.
I know i can find the solution to this query very easily on Googling, but I want to know how everyone comes to the desired result.
Is it something like Practice Makes Man Perfect i.e. once you did it, next time you will be able to formulate or their is some logic or concept behind.
I could have get my answer of solving above problem simply by posting it on stackoverflow and i would have been with an answer within 5 to 10 minutes but I want to know the reason. How do you work on any new kind of query. Is it a major contribution of experience or some an implementation of concepts.
Whenever I learns some new thing in coding section I tries to utilize it wherever I can use it. But here scenario seems to be changed because might be i am lagging in some concepts.
EDIT
How could I test my knowledge and
concepts in Sql and related sql
queries ?
Typically, the first time you need to open a child proof bottle of pills, you have a hard time, but after that you are prepared for what it might/will entail.
So it is with programming (me thinks).
You find problems, research best practices, and beat your head against a couple of rocks, but in the process you will come to have a handy set of tools.
Also, reading what others tried/did, is a good way to avoid major obsticles.
All in all, with a lot of practice/coding, you will see patterns quicker, and learn to notice where to make use of what tool.
I have a somewhat methodical method of constructing queries in general, and it is something I use elsewhere with any problem solving I need to do.
The first step is ALWAYS listing out any bits of information I have in a request. Information is essentially anything that tells me something about something.
A table contain single column having
duplicate record. I need to remove
duplicate
I have a table (I'll call it table1)
I have a
column on table table1 (I'll call it col1)
I have
duplicates in col1 on table table1
I need to remove
duplicates.
The next step of my query construction is identifying the action I'll take from the information I have.
I'll look for certain keywords (e.g. remove, create, edit, show, etc...) along with the standard insert, update, delete to determine the action.
In the example this would be DELETE because of remove.
The next step is isolation.
Asnwer the question "the action determined above should only be valid for ______..?" This part is almost always the most difficult part of constructing any query because it's usually abstract.
In the above example you're listing "duplicate records" as a piece of information, but that's really an abstract concept of something (anything where a specific value is not unique in usage).
Isolation is also where I test my action using a SELECT statement.
Every new query I run gets thrown through a select first!
The next step is execution, or essentially the "how do I get this done" part of a request.
A lot of times you'll figure the how out during the isolation step, but in some instances (yours included) how you isolate something, and how you fix it is not the same thing.
Showing duplicated values is different than removing a specific duplicate.
The last step is implementation. This is just where I take everything and make the query...
Summing it all up... for me to construct a query I'll pick out all information that I have in the request. Using the information I'll figure out what I need to do (the action), and what I need to do it on (isolation). Once I know what I need to do with what I figure out the execution.
Every single time I'm starting a new "query" I'll run it through these general steps to get an idea for what I'm going to do at an abstract level.
For specific implementations of an actual request you'll have to have some knowledge (or access to google) to go further than this.
Kris
I think in the same way I cook dinner. I have some ingredients (tables, columns etc.), some cooking methods (SELECT, UPDATE, INSERT, GROUP BY etc.) then I put them together in the way I know how.
Sometimes I will do something weird and find it tastes horrible, or that it is amazing.
Occasionally I will pick up new recipes from the internet or friends, then use parts of these in my own.
I also save my recipes in handy repositories, broken down into reusable chunks.
On the "Delete a duplicate" example, I'd come to the result by googling it. This scenario is so rare if the DB is designed properly that I wouldn't bother keeping this information in my head. Why bother, when there is a good resource is available for me to look it up when I need it?
For other queries, it really is practice makes perfect.
Over time, you get to remember frequently used patterns just because they ARE frequently used. Rare cases should be kept in a reference material. I've simply got too much other stuff to remember.
Find a good documentation to your software. I am using Mysql a lot and Mysql has excellent documentation site with decent search function so you get many answers just by reading docs. If you do NOT get your answer at least you are learning something.
Than I set up an example database (or use the one I am working on) and gradually build my SQL. I tend to separate the problem into small pieces and solve it step by step - this is very successful if you are building queries including many JOINS - it is best to start with some particular case and "polute" your SQL with many conditions like WHEN id = "123" which you are taking out as you are working towards your solution.
The best and fastest way to learn good SQL is to work with someone else, preferably someone who knows more than you, but it is not necessarry condition. It can be replaced by studying mature code written by others.
Your example is a test of how well you understand the DISTINCT keyword and the GROUP BY clause, which are SQL's ways of dealing with duplicate data.
Examples and experience. You look at other peoples examples and you create your own code and once it groks, you don't need to think about it again.
I would have a look at the Mere Mortals book - I think it's the one by Hernandez. I remember that when I first started seriously with SQL Server 6.5, moving from manual ISAM databases and Access database systems using VB4, that it was difficult to understand the syntax, the joins and the declarative style. And the SQL queries, while powerful, were very intimidating to understand - because typically, I was looking at generated code in Microsoft Access.
However, once I had developed a relatively systematic approach to building queries in a consistent and straightforward fashion, my skills and confidence quickly moved forward.
From seeing your responses you have two options.
Have a copy of the specification for whatever your working on (SQL spec and the documentation for the SQL implementation (SQLite, SQL Server etc..)
Use Google, SO, Books, etc.. as a resource to find answers.
You can't formulate an answer to a problem without doing one of the above. The first option is to become well versed into the capabilities of whatever you are working on.
The second option allows you to find answers that you may not even fully know how to ask. You example is fairly simplistic, so if you read the spec/implementation documentaion you would know the answer right away. But there are times, where even if you read the spec/documentation you don't know the answer. You only know that it IS possible, just not how to do it.
Remember that as far as jobs and supervisors go, being able to resolve a problem is important, but the faster you can do it the better which can often be done with option 2.
Say if I have a large dataset, the table has well over a million records and the database is normalized so theres foreign keys and stuff. Ive set up the relations properly and i get a list of the first object applications = EntityLoad("entityName") but because of the relations and stuff the page takes like 24 seconds to load, even when i limit the number of records to show to like 5 it takes an awful long time to load.
My solution to this was create another object that just gets the list, and then when the user wants to , use the object with all the relations and show it to the user. Is this the right way to approach it, or am i missing a big ORM concept?
Are you counting just the time to get the data, or are you perhaps doing a CFDUMP on it or something else visually that could be slow. In other words, have you wrapped the EntityLoad by itself in a cftimer tag to be sure that it is the culprit?
The first thing I would do is enable SQL logging in your Application.cfc. Add logSQL=true to This.ormSettings.
That should allow you to grab the SQL that ORM generates. Run it in an analyzer. See if the ORM SQL is doing somethign crazy. See if it is an index that you missed or something.
Also are you doing paging as Ray talks about here: http://www.coldfusionjedi.com/index.cfm/2009/8/14/Simple-ColdFusion-9-ORM-Paging-Demo?
If not have you tried using ORMExecuteQuery and HQL to enable paging.
Those are my thoughts.
When defining complex domain models with Hibernate - you will sometimes need to tweak the mapping to improve performance. This is especially true if you are dealing with inheritance (not sure how much inheritance is in your model). The ultimate goal is to have your query pulling from as few tables as possible while still preserving your domain model. This might require using the advanced inheritance mappings (more on that in a sec).
LOGGING SQL
As Terry mentioned, you will want to be sure you can log the actual SQL that is being passed to your database (yeah, you don't totally get away from SQL with ORM). Here is a great article on setting up logging for Hibernate in CF9 from Rupesh:
http://www.rupeshk.org/blog/index.php/2009/07/coldfusion-orm-how-to-log-sql/
HIBERNATE MAPPING FILES
Anytime you want to do something beyond the basic, you want to be sure that you are looking at the actual Hibernate mapping files that are generated for your CFC's. Be sure to set the following with all of your hibernate options in Application.cfc:
savemapping = true
While the cfproperty properties allow you to define many aspects of the mapping, there are actually some things that can only be done in the Hibernate mapping files (and there are tons of community resources on this.
INHERITANCE MAPPING
As I mentioned earlier, Hibernate provides different inheritance strategies for mapping. They are Table per Hierarchy, Table per subclass, Table per concrete class, and implicit polymorphism. You can read more about these types in the CF9 docs under Advanced Mapping > Inheritance Mapping or in the Hibernate documentation (as it would take forever to explain each of these).
Knowing how your tables are mapped is very important with inheritance (and it is also where Hibernate can generate some HUGE queries if you don't tweak your setup).
Those are the things I can think of - if you can give some additional information about your domain model - we can look to see what other things might be done to tweak it.
There is a good chance Hibernate is doing it's caching thing. A fair comparison in my mind (everyone please feel free to add) is doing an:
EntityLoad("entity_name") is the same as doing a select * from TABLE
So, in this case, what Hibernate might be doing in instantiating the memory, and caching it a certain way, your database server might do this similarly when you sent such a broad SQL instruction.
I have been extremely interested in ORM the past few weeks and it looks to be a very rewarding undertaking.
For this reason, is there a tiem you would ever load all 500,000 records as a result? I assume not.
I have one large logging table that I will be attacking, I am finding that the SQL good stuff must be there. For example, mark the fields that are indexes as such, this will speed it up incredibly when searching. I am sure the ORM can handle this.
Beyond this:
Find some excellent Hibernate forums, resources, and tutorials so you can learn Hibernate. This isn't really as much a Coldfusion --> ORM issue as what Hibernate might do on it's own. I have ordered a few Hibernate books that I'm waiting on to see how they are.
Likewise there seems to be an incredible amount of Hibernate resources out there where you can bring the Performance enhancement solutions of Hibernate into the Coldfusion sphere. I might be making it too simple, but I see the CF-ORM implementation as a wrapper with some code generation to save us time.
Take a look at implementing filters to cut down your data in the EntityLoad() call.
As recommended in other threads, turn on sql logging and see what sql is being generated. Chances are it might not be what you need. Check out HQL to see if you can form a better statement.
Most importantly, share what you find. I'll volunteer to do the same on this as you've tempted me to go try this out in my spare time a bit sooner than planned.
Faisal, we ran into this with Linq (c# orm).
Our solution was to create simple objects not holding the relational data. For instance, along with Users we had a SimpleUsers object which held little or no relation to any other object and had a limited set of columns.
There could be other ways of handling this but this approach helped tremendously with the query speed.