When I call EntityLoad does it load ALL the data in the table or just sets up a reference or something, because say if i do this:
<cfset test = EntityLoad("Table") />
and then do:
ArrayLen(test)
I get the number of rows in the database. If its doing that then it should be loading all the data, and that is really inefficent, say I have a table with 20x10^5 rows or something, it'll be horrible to load all data just to get the length of the table.
Without seeing your code, I don't know exactly what you're talking about, but here's the run-down:
EntityLoad w/ 1 argument (the Entity name) is going to load all the Entities with that name from the DB. It does however take some additional (optional) arguments to adjust it's behavior. Instead of describing all of them here I'll point you to the beta documentation and a site giving some further exmaples:
http://help.adobe.com/en_US/ColdFusion/9.0/Developing/WS02BE9BC5-8206-434b-8486-95CD09CDB985.html#WSf01dbd23413dda0e54af3c7912012f78097-7ff8
http://www.coldfusion-ria.com/Blog/index.cfm/2009/7/22/ColdFusion-9s-ORM-Entity-Load-Functions
To do what you are trying to do efficiently without incurring a select * on your table, use HQL.
<cfset hqlQuery = "select count(*) From person" />
<cfset results = ORMExecuteQuery(hqlQuery)[1] />
Related
When you use the cachedwithin attribute in a cfquery how does it store the query in memory. Does it store it by only the name you assign to the query? For example, if on my index page I cache a query for an hour and name it getPeople will a query with the same name on a different page (or the same page for that matter) use the cached results or does it use some better logic to decide if it is the same query?
Also, if there is a variable in your query does the cache take into account the value of the variable?
It's not only the name -- it's the exact query you're running.
<cfquery name="getPeople" cachedwithin=".5" ...>
select name from employee order by name
</cfquery>
If you invoke this same query anywhere else in your app, you'll get the cached version if it's within half a day of the first query. But these will hit the database for fresh data:
<!--- Different name, same SQL: A new cached query --->
<cfquery name="getEmployees" cachedwithin=".5" ...>
select name from employee order by name
</cfquery>
<!--- Different SQL, same name: Redefining the cached query --->
<!--- Note: As pointed out in comments, it's not really overwriting the old query
of the same name, but making a new one in the cache. The first one by the
same name is still in the cache, waiting for eviction. --->
<cfquery name="getPeople" cachedwithin=".5" ...>
select name from employee order by name desc
</cfquery>
And yes, it does take a variable into account. If you use cfqueryparam -- which you should be doing -- your database will cache the query plan, but even using cachedwithin, each query with a changed parameter will be treated as different from a query caching perspective. Note that this means if you use cachedwithin on a query that runs many times with different parameters, you'll be flooding your query cache with queries that have low cache hit rates.
From http://help.adobe.com/en_US/ColdFusion/9.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7fae.html
To use cached data, current query
must use same SQL statement, data
source, query name, user name,
password.
So those are the 'keys' that "decide if it is the same query"
variable? yes, as long as you use <cfqueryparam>
I'm passing three parameter to URL: &p1=eventID, &p2=firstItem, &p3=numberOfItems. first parameter is a column of table. second parameter is the first item that I'm looking for. Third parameter says pick how many items after firstItem.
for example in first query user send &p1=1, &p2=0, &p3=20. Therefore, I need first 20 items of list.
Next time if user sends &p1=1, &p2=21, &p3=20, then I should pass second 20 items (from item 21 to item 41).
PHP file is able to get parameters. currently, I'm using following string to query into database:
public function getPlaceList($eventId, $firstItem, $numberOfItems) {
$records = array();
$query = "select * from {$this->TB_ACT_PLACES} where eventid={$eventId}";
...
}
Result is a long list of items. If I add LIMIT 20 at the end of string, then since I'm not using Token then result always is same.
how to change the query in a way that meets my requirement?
any suggestion would be appreciated. Thanks
=> update:
I can get the whole list of items and then select from my first item to last item via for(;;) loop. But I want to know is it possible to do similar thing via sql? I guess this way is more efficient way.
I would construct the final query to be like this:
SELECT <column list> /* never use "select *" in production! */
FROM Table
WHERE p1ColumnName >= p2FirstItem
ORDER BY p1ColumnName
LIMIT p3NumberOfItems
The ORDER BY is important; according to my reading of the documentation, PostGreSql won't guarantee which rows you get without it. I know Sql Server works much the same way.
Beyond this, I'm not sure how to build this query safely. You'll need to be very careful that your method for constructing that sql statement does not leave you vulnerable to sql injection attacks. At first glance, it looks like I could very easily put a malicious value into your column name url parameter.
If you are using mysql, then use the LIMIT keyword.
SELECT *
FROM Table
LIMIT $firstItem, $numberOfItems
See the documentation here (search for LIMIT in the page).
I found my answer here: stackoverflow.com/a/8242764/513413
For other's reference, I changed my query based on what Joel and above link said:
$query = "select places.title, places.summary, places.description, places.year, places.date from places where places.eventid = $eventId order by places.date limit $numberOfItems offset $firstItem";
I am using Groovy's Sql object to perform queries on a postgres db. The queries are being executed as follows:
List<Map> results = sql.rows("select * from my_table")
List<Map> result2= sql.rows("select * from my_second_table")
I have a groovy method that performs two queries and then does some processing to loop through the data to make a different dataset, however, on some occasions I recieve a postgres exception "This ResultSet is closed" error.
having searched, I originally thought it might be to do with the issue here: SQLException: This ResultSet is closed (running multiple queries and trying to access the data from the resultsets after the fact) - however, we only seem to get the exception on quite high load - which suggests that it isnt as simple as the first dataset is closed on executing the second query as if this was the case I would expect it to happen consistently.
Can anyone shed any light on how Groovy's Sql object handles these situations or suggest what might be going wrong?
Groovy SQL is kind of a weird cat. Easy to use for simple stuff. If you have more complex scenarios you probably are better off using something else. IMHO
I first suggest doing one query, storing the results into a collection, do the second query and store the results in a collection and then do your operations between two collections rather than result sets. If you data is too large for that, find some way to store the data locally before you start doing your aggregation or whatever.
If you don't like that, you might need to checkout the GDK source code to get a better idea what is done with the Sql.getInstance() related to result sets etc. Then you can sidestep whatever land mine you are inadvertently stepping on.
Perhaps
List<Map> results = sql.rows("select * from my_table")
List<Map> result2= sql.rows("select * from my_second_table")
will not work even in plain Java (as already said in the answer you provided when second call is made on statement all resources dedicated during the previous call have to be released). As mentioned by #Todd W Crone Groovy can optimize resources, e.g. release them dynamically or don't release them depending on certain run.
Actually I've tried with only one query. E.g. I've tried to get ResultSet and then iterate through it, like this (don't mind the names of table and field, query is rather simple; and result is one row that contains one column due to LIMIT 1 clause):
def resultSet = sql.executeQuery("SELECT age FROM person WHERE id = 12345 LIMIT 1")
resultSet.next()
and got This ResultSet is closed error. Seems that Groovy optimizes resources and closes ResultSet immediately. I didn't look into the source code of Groovy SDK. I found that eachRow and other methods with closure-style handlers work fine and don't throw This ResultSet is closed error.
Perhaps, methods with closure-style handlers can help you. For example, look at except from the article where rows() method with closure is used:
String query = 'select id as identifier, name as langName from languages'
def rows = db.rows(query, { meta ->
assert meta.tableName == 'languages'
assert meta.columnCount == 2
// ...
})
I saw a query run in a log file on an application. and it contained a query like:
SELECT ID FROM CUST_ATTR49 WHERE 1=0
what is the use of such a query that is bound to return nothing?
A query like this can be used to ping the database. The clause:
WHERE 1=0
Ensures that non data is sent back, so no CPU charge, no Network traffic or other resource consumption.
A query like that can test for:
server availability
CUST_ATTR49 table existence
ID column existence
Keeping a connection alive
Cause a trigger to fire without changing any rows (with the where clause, but not in a select query)
manage many OR conditions in dynamic queries (e.g WHERE 1=0 OR <condition>)
This may be also used to extract the table schema from a table without extracting any data inside that table. As Andrea Colleoni said those will be the other benefits of using this.
A usecase I can think of: you have a filter form where you don't want to have any search results. If you specify some filter, they get added to the where clause.
Or it's usually used if you have to create a sql query by hand. E.g. you don't want to check whether the where clause is empty or not..and you can just add stuff like this:
where := "WHERE 0=1"
if X then where := where + " OR ... "
if Y then where := where + " OR ... "
(if you connect the clauses with OR you need 0=1, if you have AND you have 1=1)
As an answer - but also as further clarification to what #AndreaColleoni already mentioned:
manage many OR conditions in dynamic queries (e.g WHERE 1=0 OR <condition>)
Purpose as an on/off switch
I am using this as a switch (on/off) statement for portions of my Query.
If I were to use
WHERE 1=1
AND (0=? OR first_name = ?)
AND (0=? OR last_name = ?)
Then I can use the first bind variable (?) to turn on or off the first_name search criterium. , and the third bind variable (?) to turn on or off the last_name criterium.
I have also added a literal 1=1 just for esthetics so the text of the query aligns nicely.
For just those two criteria, it does not appear that helpful, as one might thing it is just easier to do the same by dynamically building your WHERE condition by either putting only first_name or last_name, or both, or none. So your code will have to dynamically build 4 versions of the same query. Imagine what would happen if you have 10 different criteria to consider, then how many combinations of the same query will you have to manage then?
Compile Time Optimization
I also might add that adding in the 0=? as a bind variable switch will not work very well if all your criteria are indexed. The run time optimizer that will select appropriate indexes and execution plans, might just not see the cost benefit of using the index in those slightly more complex predicates. Hence I usally advice, to inject the 0 / 1 explicitly into your query (string concatenating it in in your sql, or doing some search/replace). Doing so will give the compiler the chance to optimize out redundant statements, and give the Runtime Executer a much simpler query to look at.
(0=1 OR cond = ?) --> (cond = ?)
(0=0 OR cond = ?) --> Always True (ignore predicate)
In the second statement above the compiler knows that it never has to even consider the second part of the condition (cond = ?), and it will simply remove the entire predicate. If it were a bind variable, the compiler could never have accomplished this.
Because you are simply, and forcedly, injecting a 0/1, there is zero chance of SQL injections.
In my SQL's, as one approach, I typically place my sql injection points as ${literal_name}, and I then simply search/replace using a regex any ${...} occurrence with the appropriate literal, before I even let the compiler have a stab at it. This basically leads to a query stored as follows:
WHERE 1=1
AND (0=${cond1_enabled} OR cond1 = ?)
AND (0=${cond2_enabled} OR cond2 = ?)
Looks good, easily understood, the compiler handles it well, and the Runtime Cost Based Optimizer understands it better and will have a higher likelihood of selecting the right index.
I take special care in what I inject. Prime way for passing variables is and remains bind variables for all the obvious reasons.
This is very good in metadata fetching and makes thing generic.
Many DBs have optimizer so they will not actually execute it but its still a valid SQL statement and should execute on all DBs.
This will not fetch any result, but you know column names are valid, data types etc. If it does not execute you know something is wrong with DB(not up etc.)
So many generic programs execute this dummy statement for testing and fetching metadata.
Some systems use scripts and can dynamically set selected records to be hidden from a full list; so a false condition needs to be passed to the SQL. For example, three records out of 500 may be marked as Privacy for medical reasons and should not be visible to everyone. A dynamic query will control the 500 records are visible to those in HR, while 497 are visible to managers. A value would be passed to the SQL clause that is conditionally set, i.e. ' WHERE 1=1 ' or ' WHERE 1=0 ', depending who is logged into the system.
quoted from Greg
If the list of conditions is not known at compile time and is instead
built at run time, you don't have to worry about whether you have one
or more than one condition. You can generate them all like:
and
and concatenate them all together. With the 1=1 at the start, the
initial and has something to associate with.
I've never seen this used for any kind of injection protection, as you
say it doesn't seem like it would help much. I have seen it used as an
implementation convenience. The SQL query engine will end up ignoring
the 1=1 so it should have no performance impact.
Why would someone use WHERE 1=1 AND <conditions> in a SQL clause?
If the user intends to only append records, then the fastest method is open the recordset without returning any existing records.
It can be useful when only table metadata is desired in an application. For example, if you are writing a JDBC application and want to get the column display size of columns in the table.
Pasting a code snippet here
String query = "SELECT * from <Table_name> where 1=0";
PreparedStatement stmt = connection.prepareStatement(query);
ResultSet rs = stmt.executeQuery();
ResultSetMetaData rsMD = rs.getMetaData();
int columnCount = rsMD.getColumnCount();
for(int i=0;i<columnCount;i++) {
System.out.println("Column display size is: " + rsMD.getColumnDisplaySize(i+1));
}
Here having a query like "select * from table" can cause performance issues if you are dealing with huge data because it will try to fetch all the records from the table. Instead if you provide a query like "select * from table where 1=0" then it will fetch only table metadata and not the records so it will be efficient.
Per user milso in another thread, another purpose for "WHERE 1=0":
CREATE TABLE New_table_name as select * FROM Old_table_name WHERE 1 =
2;
this will create a new table with same schema as old table. (Very
handy if you want to load some data for compares)
An example of using a where condition of 1=0 is found in the Northwind 2007 database. On the main page the New Customer Order and New Purchase Order command buttons use embedded macros with the Where Condition set to 1=0. This opens the form with a filter that forces the sub-form to display only records related to the parent form. This can be verified by opening either of those forms from the tree without using the macro. When opened this way all records are displayed by the sub-form.
In ActiveRecord ORM, part of RubyOnRails:
Post.where(category_id: []).to_sql
# => SELECT * FROM posts WHERE 1=0
This is presumably because the following is invalid (at least in Postgres):
select id FROM bookings WHERE office_id IN ()
It seems like, that someone is trying to hack your database. It looks like someone tried mysql injection. You can read more about it here: Mysql Injection
Is there a good explanation out there on what exactly lazy="extra" is capable of?
All the posts I've seen all just repeat the fact that it turns references to MyObject.ItsCollection.Count into select count(*) queries (assuming they're not loaded already).
I'd like to know if it's capable of more robust things, like turning MyObject.ItsCollection.Any(o => o.Whatever == 5) into a SELECT ...EXISTS query.
Section 18.1 of the docs only touches on it. I'm not an NH developer, so I can't really experiment with it and watch SQL Profiler without doing a bit of work getting everything set up; I'm just looking for some sort of reference describing what this feature is capable of.
Thank you!
for version 2.x it is only used to translate a collection.Count() into a select count and as far as i can see in the source, it will also allow the construct collection[5] to fetch that particular entity (with index 5) instead of hydrating the whole collection.
For version 3.x i didn't see anything related in the release notes
Just tried calling Any() on a Collection Customer.Orders mapped with lazy="extra"
customer.Orders.Any()
and the resulting SQL statement looked something like this (simplified):
SELECT *
FROM Order
WHERE CustomerId = 120
Whereas when calling
customer.Orders.Count > 0
the resulting SQL looked like this:
SELECT count(*)
FROM Order
WHERE CustomerId = 120
The lazy = extra allow to count the element of a collection without needing of fetching it, since the lazy entity is decorated with a proxy, when the client code ask for the .Count on the collection, a proper "select count" query is issued to the database. Without lazy=extra the collection is read from the database.