I have tested and done quite some research online, but still no luck. Did anyone ever encounter this problem ?
Say, I have a doctrine query set up like:
$q = Doctrine_Query::create()
->update('PckFolder')
->set('id_path', "CONCAT(?, RIGHT(id_path, LENGTH(id_path)-?))", array($newPath, $lenOld))
->where("id_path like '$oldPath%'");
// and I print the query out
$qstr = $q->getSqlQuery(array($newPath, $lenOld));
Instead of giving me:
UPDATE pck_folder SET id_path = CONCAT(?, RIGHT(id_path, LENGTH(id_path)-?)) WHERE (id_path like '1/2//%')
Doctrine gave me:
UPDATE pck_folder SET id_path = CONCAT(?, RIGHT(id_path, LENGTH(id_path-?))) WHERE (id_path like '1/2//%')
please note this part RIGHT(id_path, LENGTH(id_path)-?)
(Note: I'm assuming you're using Doctrine 1.2. I haven't used Doctrine 2.0 yet.)
I had not encountered that specific bug before, but I have found numerous problems with the implementation of update() in Doctrine_Query. Essentially anything but the most very straightforward update queries will cause the parser to generate wrong or invalid queries. For example, it can't handle sub-selects within an update.
Try writing a Raw SQL query, or else use a less efficient but fully-functional workaround: Select the records you want to update using Doctrine_Query, then iterate over them and set the field in PHP, then call save() on each one.
By the way, there's a big GOTCHA inherent with use of UPDATE queries and Doctrine that sort of forces you to use that workaround in many cases anyway. That is, if you or your plugins have made use of the nifty Doctrine hook methods within your models but you execute a SQL-level update that affects those records, the hooks will get silently circumvented. Depending on your application, that may wreck your business logic processing.
Related
I have word model and phrase model
class Word(models.Model):
checked = models.BooleanField(default=False)
class Phrase(models.Model):
words = models.ManyToManyField(Word, null=True,related_name = "phrases")
Word model has attribute checked and many to many connection to phrase
I have a query, something like this:
Phrase.objects.exclude(words__checked=True).filter(words__group__pk__in = groups_ids)
But it works really really slow on big datasets, and the problem is in exclude section, cause without it - it works fast enough
So I found a suggestion that I should use raw sql here,
Performance issue with django exclude
So, how can rewrite this sentence with raw sql ?
(I need this query to both work postgres and mysql due to requirements, or I will need two queries if one query can't achieve this, postgres query has more priority)
So far I 've tried to use .extra syntax,but it didn't work, so asking it here.
I've been working off of the tutorial pages but seem to have a fundamental disconnect in my thinking transitioning off of RDBMS systems. I'm using MarkLogic and handling this database interaction through the Java API focusing on the search access via POJO method outlines in the tutorial documentation.
My reference up to this point has come from here principally: http://developer.marklogic.com/learn/java/processing-search-results
My scenario is this:
I have a series of documents. We'll call them 'books' for simplicity. I'm writing these books into my DB like this:
jsonDocMgr.write("/" + book.getID() + "/",
new StringHandle(
"{name: \""+book.getID()+"\","+
"chaps: "+ book.getNumChaps()+","+
"pages: "+ book.getNumPages()+","+
"}"));
What I want is to execute the following type of operation:
-Query all documents with the name "book*" (as ID is represented by book0, book1, book2, etc)
where chaps > 3. For these documents only, I want to modify the number of pages by reducing by half.
In an RDBMS, I'd use something like jdbcTemplate and get a result set for me to iterate through. For each iteration I'd know I was working with a single record (aka a book), parse the field values from the result set, make a note of the ID, then update the DB accordingly.
With MarkLogic, I'm awash in a sea of different handlers and managers...none of which seems to follow the pattern of the ResultSet with a cursor abstraction. Ultimately I want to do a two-step operation of check the chapter count then update the page field for that specific URI.
What's the most common approach to this? It seems like the most basic of operations...
Try the high-level Java API and see if it works for you. Create a multi-statement transaction with a query by example, then use document operations.
At a lower level, the closest match to a ResultSet is the ResultSequence class. The examples at http://docs.marklogic.com/javadoc/xcc/overview-summary.html are pretty good. For updates the interaction model between Java and MarkLogic is a bit different from JDBC and SQL. There is no SELECT... FOR UPDATE syntax.
The most efficient low-level technique is to select and update in one XQuery transaction, something like a stored procedure. However this requires good knowledge of XQuery. The other low-level approach is to use an XCC multi-statement transaction, which requires a little less knowledge of XQuery.
A minor issue in your code ... you definately do NOT want to end your JSON docuement URIs with "/" as you do in your sample code. You should end them with the ".json" or some other extension or no extension but definately not "/" as that is treated specially in the server.
I recently came across a program which is developed using sql statements in a table with a code for each statement. rather than having specific sql statements in the program itself.
So, rather than having code like this:
string query = "SELECT id, name from [Users]";
cmd.ExecuteQuery(query);
They use code like this: (simplified)
string firstQuery = "SELECT queryText from [Queries] where queryCode = 'SELECT_ALL_USERS'";
string userQuery = cmd.ExecuteQuery(firstQuery);//pretend this directly returns the result of the first query
cmd.ExecuteQuery(userQuery);
The logic behind this as far as I've heard is that it makes the program easier to maintain as the developer is free to change the "user sql" without having to actually change the program.
However, this struck me as maybe a little counterproductive. Would this kind of code be considered a good idea?
EDIT: I'm not looking for suggestions like "use an ORM". Assume that sql queries are the only option.
In my opinion, this approach is ridiculous. There is value (maintainability, modularity) in separating as much SQL from the middle tier as possible, but to accomplish this end, I would recommend using stored procedures.
No i really dont think its a good idea to proceed further with design.
As a test or learning activity is a differetn part, but going foward with such implementations is definately not advisable.
pros:
1. We get complete modularity. The Real Business Schema can change at any time, and we do not need to modify the Running application to get the results from Different schema (Considering result Format dont change).
Cons.
1. With this implementation we are firing 2 SQLs to Database each time when we want to execute 1. I/O call including DB calls are always performnace hit, and with this implementation we are doubling the performance which is definately not advisable.
I am using Groovy's Sql object to perform queries on a postgres db. The queries are being executed as follows:
List<Map> results = sql.rows("select * from my_table")
List<Map> result2= sql.rows("select * from my_second_table")
I have a groovy method that performs two queries and then does some processing to loop through the data to make a different dataset, however, on some occasions I recieve a postgres exception "This ResultSet is closed" error.
having searched, I originally thought it might be to do with the issue here: SQLException: This ResultSet is closed (running multiple queries and trying to access the data from the resultsets after the fact) - however, we only seem to get the exception on quite high load - which suggests that it isnt as simple as the first dataset is closed on executing the second query as if this was the case I would expect it to happen consistently.
Can anyone shed any light on how Groovy's Sql object handles these situations or suggest what might be going wrong?
Groovy SQL is kind of a weird cat. Easy to use for simple stuff. If you have more complex scenarios you probably are better off using something else. IMHO
I first suggest doing one query, storing the results into a collection, do the second query and store the results in a collection and then do your operations between two collections rather than result sets. If you data is too large for that, find some way to store the data locally before you start doing your aggregation or whatever.
If you don't like that, you might need to checkout the GDK source code to get a better idea what is done with the Sql.getInstance() related to result sets etc. Then you can sidestep whatever land mine you are inadvertently stepping on.
Perhaps
List<Map> results = sql.rows("select * from my_table")
List<Map> result2= sql.rows("select * from my_second_table")
will not work even in plain Java (as already said in the answer you provided when second call is made on statement all resources dedicated during the previous call have to be released). As mentioned by #Todd W Crone Groovy can optimize resources, e.g. release them dynamically or don't release them depending on certain run.
Actually I've tried with only one query. E.g. I've tried to get ResultSet and then iterate through it, like this (don't mind the names of table and field, query is rather simple; and result is one row that contains one column due to LIMIT 1 clause):
def resultSet = sql.executeQuery("SELECT age FROM person WHERE id = 12345 LIMIT 1")
resultSet.next()
and got This ResultSet is closed error. Seems that Groovy optimizes resources and closes ResultSet immediately. I didn't look into the source code of Groovy SDK. I found that eachRow and other methods with closure-style handlers work fine and don't throw This ResultSet is closed error.
Perhaps, methods with closure-style handlers can help you. For example, look at except from the article where rows() method with closure is used:
String query = 'select id as identifier, name as langName from languages'
def rows = db.rows(query, { meta ->
assert meta.tableName == 'languages'
assert meta.columnCount == 2
// ...
})
Is there a good explanation out there on what exactly lazy="extra" is capable of?
All the posts I've seen all just repeat the fact that it turns references to MyObject.ItsCollection.Count into select count(*) queries (assuming they're not loaded already).
I'd like to know if it's capable of more robust things, like turning MyObject.ItsCollection.Any(o => o.Whatever == 5) into a SELECT ...EXISTS query.
Section 18.1 of the docs only touches on it. I'm not an NH developer, so I can't really experiment with it and watch SQL Profiler without doing a bit of work getting everything set up; I'm just looking for some sort of reference describing what this feature is capable of.
Thank you!
for version 2.x it is only used to translate a collection.Count() into a select count and as far as i can see in the source, it will also allow the construct collection[5] to fetch that particular entity (with index 5) instead of hydrating the whole collection.
For version 3.x i didn't see anything related in the release notes
Just tried calling Any() on a Collection Customer.Orders mapped with lazy="extra"
customer.Orders.Any()
and the resulting SQL statement looked something like this (simplified):
SELECT *
FROM Order
WHERE CustomerId = 120
Whereas when calling
customer.Orders.Count > 0
the resulting SQL looked like this:
SELECT count(*)
FROM Order
WHERE CustomerId = 120
The lazy = extra allow to count the element of a collection without needing of fetching it, since the lazy entity is decorated with a proxy, when the client code ask for the .Count on the collection, a proper "select count" query is issued to the database. Without lazy=extra the collection is read from the database.