.Where(x => !x.Rated)
This creates sql that looks like:
not (cdrcalltmp0_.Rated=1)
Our dba says I have to remove the not for some filtered index to work.
.Where(x => x.Rated == false)
This creates sql that looks like:
cdrcalltmp0_.Rated=#p2 order by cdrcalltmp0_.Created asc'
This doesn't work because of the parameter.
He would like this sql:
cdrcalltmp0_.Rated=0 order by cdrcalltmp0_.Created asc'
Is it possible to make nhibernate not use parameters?
So that a filtered index works.
Preface:
The following answer is assuming you are using SQL Server 2008. If you are not, then it is quite possible that the database technology in question does not support indexes when using the NOT operator. So, if your using SQL Server 2008...
Your DBA doesn't know what he's talking about.
The following syntax
NOT ( SomeTableAlias.SomeTableColumn = 1 )
will absolutely be understood by the SQL Server Query Analyzer. I've got queries from NHibernate that look exactly like the above syntax and they are indeed using the proper indexes.
And to answer your question, no. NHibernate always uses parameters when it creates the SQL for you. Parameterized queries are extremely common place, even when using traditional ADO.NET yourself.
The only way to get NHibernate to not use parameters is if you supply the SQL it needs to execute yourself using the session.CreateSQLQuery() method.
At any rate, the above line you posted:
Where(x => x.Rated == false) This creates sql that looks like: cdrcalltmp0_.Rated=#p2 order by cdrcalltmp0_.Created asc'
is completely valid. When SQL Server receives the parameterized query, it will use whatever index is on your Rated column.
If your DBA still doubts you, tell him to run the query in Sql Server Management Studio with the "Display Estimated Execution Plan" feature on. That will prove that the query is using the index.
Related
We use Flask+SQLAlchemy in our web app. We found some slow sql queries in DB monitor. Is there any good way to find/locate the original SQLAlchemy queries according to generated sql? i.e. code mapping between ORM code and SQL code.
e.g. Add any unique token in generated sql comment, so we can find original ORM code by the token.
You can add sql comment with suffix_with method
query = Table.query
query = query.suffix_with('-- %s' % query_token)
I'm not particularly familiar with Ruby on Rails, but I'm troubleshooting an issue we're experiencing with a rake job that is supposed to be cleaning database tables. The tables grow very large very quickly, and the query generated by ActiveRecord doesn't seem to be efficient enough to handle it.
The Ruby calls looks like this:
Source.where("id not IN (#{Log.select('DISTINCT source_id').to_sql})").delete_all
and this:
Log.joins(:report).where(:report_id => Report.where(cond)).delete_all
I'm trying to get at the SQL, so we can have our DBA's attempt to optimize it better. I've noticed if I drop the ".delete_all" I can add a ".to_sql" which gives me the SELECT statement of the query, prior to the call to ".delete_all". I'd like to see what SQL is being generated by that delete_all method though.
Is there a way to do that?
Another option is to use raw Arel syntax, similar to a simplified version of what ActiveRecord::Relation#delete_all does.
relation = Model.where(...)
arel = relation.arel
stmt = Arel::DeleteManager.new
stmt.from(arel.join_sources.empty? ? Model.arel_table : arel.source)
stmt.wheres = arel.constraints
sql = Model.connection.to_sql(stmt, relation.bound_attributes)
print sql
This will give you the generated delete sql. Here's an example using postgres as the sql adapter
relation = User.where('email ilike ?', '%#gmail.com')
arel = relation.arel
stmt = Arel::DeleteManager.new
stmt.from(arel.join_sources.empty? ? User.arel_table : arel.source)
stmt.wheres = arel.constraints
sql = User.connection.to_sql(stmt, relation.bound_attributes)
=> DELETE FROM "users" WHERE (email ilike '%#gmail.com')
From the fine manual:
delete_all(conditions = nil)
Deletes the records matching conditions without instantiating the records first, and hence not calling the destroy method nor invoking callbacks. This is a single SQL DELETE statement that goes straight to the database, much more efficient than destroy_all.
So a Model.delete_all(conditions) ends up as
delete from models where conditions
When you say Model.where(...).delete_all, the conditions for the delete_all come from the where calls so these are the same:
Model.delete_all(conditions)
Model.where(conditions).delete_all
Applying that to your case:
Source.where("id not IN (#{Log.select('DISTINCT source_id').to_sql})").delete_all
you should see that you're running:
delete from sources
where id not in (
select distinct source_id
from logs
)
If you run your code in a development console you should see the SQL in the console or the Rails logs but it will be as above.
As far as optimization goes, my first step would be to drop the DISTINCT. DISTINCT usually isn't cheap and IN doesn't care about duplicates anyway so not in (select distinct ...) is probably pointless busy work. Then maybe an index on source_id would help, the query optimizer might be able to slurp the source_id list straight out of the index without having to do a table scan to find them. Of course, query optimization is a bit of a dark art so these simple steps may or may not work.
ActiveRecord::Base.logger = Logger.new(STDOUT) should show you all the SQL generated by rails on your console.
I am a SQL Server DBA for a company that sells an ASP.NET MVC3 application that uses LINQ and Entity Framework 4 for all database access. When I find an inefficient query in my SQL Server's plan cache that was generated by LINQ, I would like to be able to find that LINQ statement in the source code so that I can optimize it. What is the best way to find the LINQ that generated a given SQL statement?
For example, is there any way to put an entry in a config file or decorate the code somehow so that the class and method name or the LINQ statement itself are included as comments in the generated SQL?
The commercial tools ORM Profiler, Entity Framework Profiler or Hugati Query Profiler will both give you a stack trace for the methods which generated the SQL. That makes it fairly easy to find the LINQ in code, though it isn't displayed directly.
These tools also have the advantage that they make it easy to find inefficient queries amongst the many other SQL statements executed by the app.
Although it is not a free tool, this may provide the information you need:
http://efprof.com/
There is also a less expensive tool described here, which I have not used, but it looks very promising:
http://huagati.blogspot.com/2010/06/entity-framework-support-in-huagati.html
http://www.huagati.com/L2SProfiler/
I bet Entity Framework Profiler (http://efprof.com/) would help you out. The workflow is very different from what you asked for (which would be pretty cool BTW). It is a good tool, and is worth a look even if it's not your final solution.
Good luck!
If you have access to the ASP.NET code where the LINQ code is you can more or less know which query you are looking for, copy it into a freeware tool called LINQPad and run it directly there to get the generated SQL statements. http://www.linqpad.net/
You need first get the LINQ queries on your .net code, create a connection to your datasource, paste the Linq code in new queries and run them. You will get the SQL Query generated from the LINQ code.
For example:
from e in ETUSERs
where e.LoginName.Contains("a")
orderby e.LoginName
select e
SQL Results Tab:
-- Region Parameters
DECLARE #p0 VarChar(1000) = '%a%'
-- EndRegion
SELECT [t0].[UserID], [t0].[UsrFirstName], [t0].[UsrLastName], [t0].[LoginName], [t0].[Location], [t0].[Password], [t0].[UsrEmail], ...
FROM [ETUSER] AS [t0]
WHERE [t0].[LoginName] LIKE #p0
ORDER BY [t0].[LoginName]
This is probably not exactly what you are looking for, but it is worth knowing about this tool since it is very helpful to quickly test LINQ queries. There you can quickly edit and run to improve the code without recompiling the whole stuff.
I don't think you can modify the generated SQL easily but what you can do is to get the generated SQL before sending the query to the database.
So you can log every query in a separate textfile with timestamp and source code context information. But that means to modify each place in your source where LINQ queries are sent to the database. Maybe there is an extension point somewhere in the DataContext class for simplifying this.
Anyway here is the code to get the corresponding sql query for a LINQ query:
YourDataContext dc = new YourDataContext();
IQueryable<YourEntityClass> query =
from e in dc.YourEntities
where ...
select e;
string command = dc.GetCommand(query).CommandText;
I'm a bit of a NHibernate newbie and Im taking on some code written by another developer. I want to find out how NHibernate converts lambda based criteria into SQL.
I know in Linq to SQL using Lambda expressions on queries means that the whole thing is turned into an expression tree and then into SQL (where possible) by the Linq to SQL provider. This can be seen by doing DataContext.Log = Console.Out.
But what about an NHibernate criteria expression where Linq to NHibernate isnt being used?
The following namespaces are imported...
using NHibernate;
using NHibernate.Criterion;
using NHibernate.LambdaExtensions;
.. and the criteria code looks like this...
return Session.CreateCriteria<MyObjectType>()
.Add<MyObjectType>(x => x.Id == id)
.UniqueResult<MyObjectType>();
Will this be turned into an SQL statement e.g.
Select distinct * from table where id = [param]
... or will the whole dataset be pulled into memory giving a List and then have the lambda expressions applied against the objects. e.g.
return List<MyObject>.Where(x => x.id = id) [or something similar].
I', not sure if my importing NHibernate.LambdaExtensions provides a sort of translation into SQL.
It is turned to an HQL statement first (enable logging and look at the console for the statements) and then to an SQL and sent to the database.
It does not select the whole table to memory and filters there.
I'd like to "dry-run" Hibernate HQL queries. That is I'd like to know what actual SQL queries Hibernate will execute from given HQL query without actually executing the HQL query against real database.
I have access to hibernate mapping for tables, the HQL query string, the dialect for my database. I have also access to database if that is needed.
Now, how can I find out all the SQL queries Hibernate can generate from my HQL without actually executing the query against any database? Are there any tools for this?
Note, that many SQL queries can be generated from one HQL query and the set of generated SQL queries may differ based on the contents of database.
I am not asking how to log SQL queries while HQL query is executing.
Edit: I don't mind connecting to database to fetch some metadata, I just don't want to execute queries.
Edit: I also know what limits and offsets are applied to query. I also have the actual parameters that will be bind to query.
The short answer is "you can't". The long answer is below.
There are two approaches you can take:
A) Look into HQLQueryPlan class, particularly its getSqlStrings() method. It will not get you the exact SQL because further preprocessing is involved before query is actually executed (parameters are bound, limit / offset are applied, etc...) but it may be close enough to what you want.
The thing to keep in mind here is that you'll need an actual SessionFactory instance in order to construct HQLQueryPlan, which means you won't be able to do so without "connecting to any database". You can, however, use in-memory database (SqlLite and the likes) and have Hibernate auto-create necessary schema for it.
B) Start with ASTQueryTranslatorFactory and descend into AST / ANTLR madness. In theory you may be able to hack together a parser that would work without relying on metadata but I have a hardest time imagining what is it you're trying to do for this to be worth it. Perhaps you can clarify? There has to be a better approach.
Update: for an offline, dry-run of some HQL, using HQLQueryPlan directly is a good approach. If you want to intercept every query in the app, while it's running, and record the SQL, you'll have to use proxies and reflection as described below.
Take a look at this answer for Criteria Queries.
For HQL, it's the same concept - you have to cast to Hibernate implementation classes and/or access private members, so it's not a supported method, but it will work with a the 3.2-3.3 versions of Hibernate. Here is the code to access the query from HQL (query is the object returned by session.createQuery(hql_string):
Field f = AbstractQueryImpl.class.getDeclaredField("session");
f.setAccessible(true);
SessionImpl sessionImpl = (SessionImpl) f.get(query);
Method m = AbstractSessionImpl.class.getDeclaredMethod("getHQLQueryPlan", new Class[] { String.class, boolean.class });
m.setAccessible(true);
HQLQueryPlan plan = (HQLQueryPlan) m.invoke(sessionImpl, new Object[] { query.getQueryString(), Boolean.FALSE });
for (int i = 0; i < plan.getSqlStrings().length; ++i) {
sql += plan.getSqlStrings()[i];
}
I would wrap all of that in a try/catch so you can go on with the query if the logging doesn't work.
It's possible to proxy your session and then proxy your queries so that you can log the sql and the parameters of every query (hql, sql, criteria) before it runs, without the code that builds the query having to do anything (as long as the initial session is retrieved from code you control).