Currently I use a block of code like this, to fetch a set of DB objects with matching IDs.
List<subjects> getSubjectsById(List<long> subjectIDs){
return ctx.tagSubjects.Where(t => subjectIDs.Contains(t.id)).ToList();
}
But this is really inefficient, because it requires the entire table to be read from the database and then filtered inside of C#.
What I would rather do would be something the equivelent of:
SELECT * FROM subjects WHERE subjects.id IN (1,2,3,4,5,...);
The big difference is that in the first example the filtering is happening inside the C# code, and in the second the filtering is done on the SQL server (where the data is).
Is there a [better] way to do this with LINQ?
Where did you find out that it downloads the entire table from SQL Server?
I'm sure it does what you want. It translates the query to a parameterized IN clause like:
... IN (#p1, #p2, #p3)
and passes the contents of the list as values to those parameters. You can confirm this with tools such as SQL Profiler and LINQ to SQL debugger visualizer or set the DataContext.Log property to console (before executing the query) and read the generated SQL:
dataContext.Log = Console.Out;
Related
My project is in Visual Foxpro and I use MS SQL server 2008. When I fire sql queries in batch, some of the queries don't execute. However, no error is thrown. I haven't used BEGIN TRAN and ROLLBACK yet. What should be done ??
that all depends... You don't have any sample of your queries posted to give us an indication of possible failure. However, one thing I've had good response with from VFP to SQL is to build into a string (I prefer using TEXT/ENDTEXT for readabilty), then send that entire value to SQL. If there are any "parameter" based values that are from VFP locally, you can use "?" to indicate it will come from a variable to SQL. Then you can batch all in a single vs multiple individual queries...
vfpField = 28
vfpString = 'Smith'
text to lcSqlCmd noshow
select
YT.blah,
YT.blah2
into
#tempSqlResult
from
yourTable YT
where
YT.SomeKey = ?vfpField
select
ost.Xblah,
t.blah,
t.blah2
from
OtherSQLTable ost
join #tempSqlResult t
on ost.Xblah = t.blahKey;
drop table #tempSqlResult;
endtext
nHandle = sqlconnect( "your connection string" )
nAns = sqlexec( nHandle, lcSqlCmd, "LocalVFPCursorName" )
No I don't have error trapping in here, just to show principle and readability. I know the sample query could have easily been done via a join, but if you are working with some pre-aggregations and want to put them into temp work areas like Localized VFP cursors from a query to be used as your next step, this would work via #tempSqlResult as "#" indicates temporary table on SQL for whatever the current connection handle is.
If you want to return MULTIPLE RESULT SETs from a single SQL call, you can do that too, just add another query that doesn't have an "into #tmpSQLblah" context. Then, all instances of those result cursors will be brought back down to VFP based on the "LocalVFPCursorName" prefix. If you are returning 3 result sets, then VFP will have 3 cursors open called
LocalVFPCursorName
LocalVFPCursorName1
LocalVFPCursorName2
and will be based on the sequence of the queries in the SqlExec() call. But if you can provide more on what you ARE trying to do and their samples, we can offer more specific help too.
While this seems like a basic problem, I've been ripping my hair out FOR DAYS trying to get an efficient solution to this.
I have a lookup table of values on a server that I read from and assemble into a string using a C# Script task. I write this string into a variable that I want to pass in as my WHERE parameters inside a large SQL query on a ADO.NET data source (from a different server which I only have read access to) in my data flow. For example, this string would just be something like
('Frank', 'John', 'Markus', 'Tom')
and I would append that as my WHERE clause.
I can't read from a variable directly for an ADO.NET data source AND I can't use the 'Expression' property to set my SQL either as my SQL query is over 4000 characters. I could use an Execute SQL Task to run my query, load the results into a recordset and I assume, then loop through the recordset but that's extremely inefficient.
What would be the best way to do this? My end goal is to put these results inside a table on the first server.
You could try to set up Script Component as source - variables and strings inside scripts can be longer than 4000 characters so you can fit your query inside.
Setup your component similar to this article: http://beyondrelational.com/modules/2/blogs/106/posts/11119/script-componentsource-part1.aspx
In this one you have example how to fetch data using ExecuteReader and put it to output of script component: http://beyondrelational.com/modules/2/blogs/106/posts/11124/ssis-script-component-as-source-adonet.aspx In this one you have instructions how to aquire connection properly: http://www.toadworld.com/platforms/sql-server/b/weblog/archive/2011/05/30/use-connections-properly-in-an-ssis-script-task
By joining this pieces of information you should be able to write your source Script Component which can fetch data using any length dynamically constructed query.
Good luck :)
You can do a simple select statement to return a list of values that will include ('Frank', 'John', 'Markus', 'Tom'). So your select would return :
Name
----------
Frank
John
Markus
Tom
Then, in SSIS, use a Merge Join Component (that will act as a INNER JOIN) instead of a where clause in your main query.
This is the cleaniest way to achieve what you want.
I am a SQL Server DBA for a company that sells an ASP.NET MVC3 application that uses LINQ and Entity Framework 4 for all database access. When I find an inefficient query in my SQL Server's plan cache that was generated by LINQ, I would like to be able to find that LINQ statement in the source code so that I can optimize it. What is the best way to find the LINQ that generated a given SQL statement?
For example, is there any way to put an entry in a config file or decorate the code somehow so that the class and method name or the LINQ statement itself are included as comments in the generated SQL?
The commercial tools ORM Profiler, Entity Framework Profiler or Hugati Query Profiler will both give you a stack trace for the methods which generated the SQL. That makes it fairly easy to find the LINQ in code, though it isn't displayed directly.
These tools also have the advantage that they make it easy to find inefficient queries amongst the many other SQL statements executed by the app.
Although it is not a free tool, this may provide the information you need:
http://efprof.com/
There is also a less expensive tool described here, which I have not used, but it looks very promising:
http://huagati.blogspot.com/2010/06/entity-framework-support-in-huagati.html
http://www.huagati.com/L2SProfiler/
I bet Entity Framework Profiler (http://efprof.com/) would help you out. The workflow is very different from what you asked for (which would be pretty cool BTW). It is a good tool, and is worth a look even if it's not your final solution.
Good luck!
If you have access to the ASP.NET code where the LINQ code is you can more or less know which query you are looking for, copy it into a freeware tool called LINQPad and run it directly there to get the generated SQL statements. http://www.linqpad.net/
You need first get the LINQ queries on your .net code, create a connection to your datasource, paste the Linq code in new queries and run them. You will get the SQL Query generated from the LINQ code.
For example:
from e in ETUSERs
where e.LoginName.Contains("a")
orderby e.LoginName
select e
SQL Results Tab:
-- Region Parameters
DECLARE #p0 VarChar(1000) = '%a%'
-- EndRegion
SELECT [t0].[UserID], [t0].[UsrFirstName], [t0].[UsrLastName], [t0].[LoginName], [t0].[Location], [t0].[Password], [t0].[UsrEmail], ...
FROM [ETUSER] AS [t0]
WHERE [t0].[LoginName] LIKE #p0
ORDER BY [t0].[LoginName]
This is probably not exactly what you are looking for, but it is worth knowing about this tool since it is very helpful to quickly test LINQ queries. There you can quickly edit and run to improve the code without recompiling the whole stuff.
I don't think you can modify the generated SQL easily but what you can do is to get the generated SQL before sending the query to the database.
So you can log every query in a separate textfile with timestamp and source code context information. But that means to modify each place in your source where LINQ queries are sent to the database. Maybe there is an extension point somewhere in the DataContext class for simplifying this.
Anyway here is the code to get the corresponding sql query for a LINQ query:
YourDataContext dc = new YourDataContext();
IQueryable<YourEntityClass> query =
from e in dc.YourEntities
where ...
select e;
string command = dc.GetCommand(query).CommandText;
I'm a bit of a NHibernate newbie and Im taking on some code written by another developer. I want to find out how NHibernate converts lambda based criteria into SQL.
I know in Linq to SQL using Lambda expressions on queries means that the whole thing is turned into an expression tree and then into SQL (where possible) by the Linq to SQL provider. This can be seen by doing DataContext.Log = Console.Out.
But what about an NHibernate criteria expression where Linq to NHibernate isnt being used?
The following namespaces are imported...
using NHibernate;
using NHibernate.Criterion;
using NHibernate.LambdaExtensions;
.. and the criteria code looks like this...
return Session.CreateCriteria<MyObjectType>()
.Add<MyObjectType>(x => x.Id == id)
.UniqueResult<MyObjectType>();
Will this be turned into an SQL statement e.g.
Select distinct * from table where id = [param]
... or will the whole dataset be pulled into memory giving a List and then have the lambda expressions applied against the objects. e.g.
return List<MyObject>.Where(x => x.id = id) [or something similar].
I', not sure if my importing NHibernate.LambdaExtensions provides a sort of translation into SQL.
It is turned to an HQL statement first (enable logging and look at the console for the statements) and then to an SQL and sent to the database.
It does not select the whole table to memory and filters there.
I'd like to "dry-run" Hibernate HQL queries. That is I'd like to know what actual SQL queries Hibernate will execute from given HQL query without actually executing the HQL query against real database.
I have access to hibernate mapping for tables, the HQL query string, the dialect for my database. I have also access to database if that is needed.
Now, how can I find out all the SQL queries Hibernate can generate from my HQL without actually executing the query against any database? Are there any tools for this?
Note, that many SQL queries can be generated from one HQL query and the set of generated SQL queries may differ based on the contents of database.
I am not asking how to log SQL queries while HQL query is executing.
Edit: I don't mind connecting to database to fetch some metadata, I just don't want to execute queries.
Edit: I also know what limits and offsets are applied to query. I also have the actual parameters that will be bind to query.
The short answer is "you can't". The long answer is below.
There are two approaches you can take:
A) Look into HQLQueryPlan class, particularly its getSqlStrings() method. It will not get you the exact SQL because further preprocessing is involved before query is actually executed (parameters are bound, limit / offset are applied, etc...) but it may be close enough to what you want.
The thing to keep in mind here is that you'll need an actual SessionFactory instance in order to construct HQLQueryPlan, which means you won't be able to do so without "connecting to any database". You can, however, use in-memory database (SqlLite and the likes) and have Hibernate auto-create necessary schema for it.
B) Start with ASTQueryTranslatorFactory and descend into AST / ANTLR madness. In theory you may be able to hack together a parser that would work without relying on metadata but I have a hardest time imagining what is it you're trying to do for this to be worth it. Perhaps you can clarify? There has to be a better approach.
Update: for an offline, dry-run of some HQL, using HQLQueryPlan directly is a good approach. If you want to intercept every query in the app, while it's running, and record the SQL, you'll have to use proxies and reflection as described below.
Take a look at this answer for Criteria Queries.
For HQL, it's the same concept - you have to cast to Hibernate implementation classes and/or access private members, so it's not a supported method, but it will work with a the 3.2-3.3 versions of Hibernate. Here is the code to access the query from HQL (query is the object returned by session.createQuery(hql_string):
Field f = AbstractQueryImpl.class.getDeclaredField("session");
f.setAccessible(true);
SessionImpl sessionImpl = (SessionImpl) f.get(query);
Method m = AbstractSessionImpl.class.getDeclaredMethod("getHQLQueryPlan", new Class[] { String.class, boolean.class });
m.setAccessible(true);
HQLQueryPlan plan = (HQLQueryPlan) m.invoke(sessionImpl, new Object[] { query.getQueryString(), Boolean.FALSE });
for (int i = 0; i < plan.getSqlStrings().length; ++i) {
sql += plan.getSqlStrings()[i];
}
I would wrap all of that in a try/catch so you can go on with the query if the logging doesn't work.
It's possible to proxy your session and then proxy your queries so that you can log the sql and the parameters of every query (hql, sql, criteria) before it runs, without the code that builds the query having to do anything (as long as the initial session is retrieved from code you control).