I've recently upgraded my Linq provider to the new AST one. (NH3 on NuGet)
With the previous provider I was using linq to do "inline projections
to my DTO"
e.g.
from o in Session.Query<MyObject>()
select new MyObjectDTO {
Name = o.Name,
SubName = o.OtherObject.Name,
Sub2NAme = o.OtherObject2.Name
}
and this would generate a
SELECT o.Name, sn1.Name, sn2.Name FROM .....
JOIN.... JOIN....
statement.
Once I upgraded my provider I found a lot of select statements being
fired off. (My projected object is more complex than above).
I have come accross Fetch/FetchMany, which might help with the number
of queries, but as far as I can tell it means the full object will
come back for each flattened field I require.
Is there a way I can get the smallest possible number of columns required for the projection to be selected, rather than loading the full object graph to the project with?
Thanks,
Chris
It must be something with your usage of the result (like iterating many times the IQueryable), something odd with the mappings, or some complexity that was removed from the example.
I just tried that exact query, and only one SQL statement was generated.
Related
I have joined a new job where I am required to use FileMaker (and gradually transition systems to other databases). I have been a DB Admin of a MS SQL Server database for ~2 years, and I am very well versed in PL/SQL and T-SQL. I am trying to pan my SQL knowledge to FMP using the ExecuteSQL functionaloty, and I'm kinda running into a lot of small pains :)
I have 2 tables: Movies and Genres. The relevant columns are:
Movies(MovieId, MovieName, GenreId, Rating)
Genres(GenreId, GenreName)
I'm trying to find the movie with the highest rating in each genre. The SQL query for this would be:
SELECT M.MovieName
FROM Movies M INNER JOIN Genres G ON M.GenreId=G.GenreId
WHERE M.Rating=
(
SELECT MAX(Rating) FROM Movies WHERE GenreId = M.GenreId
)
I translated this as best as I could to an ExecuteSQL query:
ExecuteSQL ("
SELECT M::MovieName FROM Movies M INNER JOIN Genres G ON M::GenreId=G::GenreId
WHERE M::Rating =
(SELECT MAX(M2::Rating) FROM Movies M2 WHERE M2::GenreId = M::GenreId)
"; "" ; "")
I set the field type to Text and also ensured values are not stored. But all I see are '?' marks.
What am I doing incorrectly here? I'm sorry if it's something really stupid, but I'm new to FMP and any suggestions would be appreciated.
Thank you!
--
Ram
UPDATE: Solution and the thought process it took to get there:
Thanks to everyone that helped me solve the problem. You guys made me realize that traditional SQL thought process does not exactly pan to FMP, and when I probed around, what I realized is that to best use SQL knowledge in FMP, I should be considering each column independently and not think of the entire result set when I write a query. This would mean that for my current functionality, the JOIN is no longer necessary. The JOIN was to bring in the GenreName, which is a different column that FMP automatically maps. I just needed to remove the JOIN, and it works perfectly.
TL;DR: The thought process context should be the current column, not the entire expected result set.
Once again, thank you #MissJack, #Chuck (how did you even get that username?), #pft221 and #michael.hor257k
I've found that FileMaker is very particular in its formatting of queries using the ExecuteSQL function. In many cases, standard SQL syntax will work fine, but in some cases you have to make some slight (but important) tweaks.
I can see two things here that might be causing the problem...
ExecuteSQL ("
SELECT M::MovieName FROM Movies M INNER JOIN Genres G ON
M::GenreId=G::GenreId
WHERE M::Rating =
(SELECT MAX(M2::Rating) FROM Movies M2 WHERE M2::GenreId = M::GenreId)
"; "" ; "")
You can't use the standard FMP table::field format inside the query.
Within the quotes inside the ExecuteSQL function, you should follow the SQL format of table.column. So M::MovieName should be M.MovieName.
I don't see an AS anywhere in your code.
In order to create an alias, you must state it explicitly. For example, in your FROM, it should be Movies AS M.
I think if you fix those two things, it should probably work. However, I've had some trouble with JOINs myself, as my primary experience is with FMP, and I'm only just now becoming more familiar with SQL syntax.
Because it's incredibly hard to debug SQL in FMP, the best advice I can give you here is to start small. Begin with a very basic query, and once you're sure that's working, gradually add more complicated elements one at a time until you encounter the dreaded ?.
There's a number of great posts on FileMaker Hacks all about ExecuteSQL:
Since you're already familiar with SQL, I'd start with this one: The Missing FM 12 ExecuteSQL Reference. There's a link to a PDF of the entire article if you scroll down to the bottom of the post.
I was going to recommend a few more specific articles (like the series on Robust Coding, or Dynamic Parameters), but since I'm new here and I can't include more than 2 links, just go to FileMaker Hacks and search for "ExecuteSQL". You'll find a number of useful posts.
NB If you're using FMP Advanced, the Data Viewer is a great tool for testing SQL. But beware: complex queries on large databases can sometimes send it into fits and freeze the program.
The first thing to keep in mind when working with FileMaker and ExecuteSQL() is the difference between tables and table occurrences. This is a concept that's somewhat unique to FileMaker. Succinctly, tables store the data, but table occurrences define the context of that data. Table occurrences are what you're seeing in FileMaker's relationship graph, and the ExecuteSQL() function needs to reference the table occurrences in its query.
I agree with MissJack regarding the need to start small in building the SQL statement and use the Data Viewer in FileMaker Pro Advanced, but there's one more recommendation I can offer, which is to use SeedCode's SQL Explorer. It does require the adding of table occurrences and fields to duplicate the naming in your existing solution, but this is pretty easy to do and the file they offer includes a wizard for building the SQL query.
I have tried to use FLINQ but it is rather out of date with F# 3.0 beta.
Can someone give me some pointers on how to create dynamic SQL queries in F#?
We have recently developed a library, FSharpComposableQuery, aimed at supporting more flexible composition of query expressions in F# 3.0 and above. It's intended as a drop-in replacement overloading the standard query builder.
Tomas's example can be modified as follows:
open FSharpComposableQuery
// Initial query that simply selects products
let q1 =
<# query { for p in ctx.Products do
select p } #>
// Create a new query that specifies only expensive products
let q2 =
query { for p in %q1 do
where (p.UnitPrice.Value > 100.0M) }
This simply quotes the query expression and splices it into the second query. However, this results in a quoted query expression that the default QueryBuilder may not be able to turn into a single query, because q2 evaluates to the (equivalent) expression
query { for p in (query { for p in ctx.Products do
select p }) do
where (p.UnitPrice.Value > 100.0M) }
which (as in Tomas's original code) will likely be evaluated by loading all of the products into memory, and doing the selection in memory, whereas what we really want is something like:
query { for p in ctx.Products do
where (p.UnitPrice.Value > 100.0M) }
which will turn into an SQL selection query. FSharpComposableQuery overrides the QueryBuilder to perform this, among other, transformations. So, queries can be composed using quotation and antiquotation more freely.
The project home page is here: http://fsprojects.github.io/FSharp.Linq.ComposableQuery/
and there is some more discussion in an answer I just provided to another (old) question about dynamic queries: How do you compose query expressions in F#?
Comments or questions (especially if something breaks or something that you think should work doesn't) are very welcome.
[EDIT: Updated the links to the project pages, which have just been changed to remove the word "Experimental".]
In F# 3.0, the query is quoted automatically and so you cannot use quotation splicing (the <# foo %bar #> syntax) that makes composing queries possible. Most of the things that you could write by composing queries using splicing can still be done in the "usual LINQ way" by creating a new query from the previous source and adding i.e. filtering:
// Initial query that simply selects products
let q1 =
query { for p in ctx.Products do
select p }
// Create a new query that specifies only expensive products
let q2 =
query { for p in q1 do
where (p.UnitPrice.Value > 100.0M) }
This way, you can dynamically add conditions, dynamically specify projection (using select) and do a couple of other query compositions. However, you don't get the full flexibility of composing queries as with explicit quotations. I guess this is the price that F# 3.0 has to pay for a simpler syntax similar to what exists in C#.
In principle, you should be able to write query explicitly using the query.Select (etc.) operators. This would be written using explicit quotations and so you should be able to use splicing. However, I don't exactly know how the translation works, so I can't give you a working sample. Something like this should work (but the syntax is very ugly, so it is probably better to just use strings or some other techniques):
<# query.Select(Linq.QuerySource<_, _>(ctx.Products), fun prod ->
// You could use splicing here, for example, if 'projection' is
// a quotation that specifies the projection, you could write:
// %projection
prod.ProductName) #>
|> query.Run
The queries in F# 3.0 are based on IQueryable, so it might be possible to use the same trick as the one that I implemented for C#. However, I guess that some details would be different, so I wouldn't expect that to work straight away. The best implementation of that idea is in LINQKit, but I think it won't directly work in F#.
So, in general, I think the only case that works well is the first example - where you just apply additional query operators to the query by writing multiple queries.
My company has just started using LINQ and I still am having a little trouble with the abstractness (if thats a word) of the LINQ command and the SQL, my question is
Dim query = (From o In data.Addresses _
Select o.Name).Count
In the above in my mind, the SQL is returning all rows and the does a count on the number rows in the IQueryable result, so I would be better with
Dim lstring = Aggregate o In data.Addresses _
Into Count()
Or am I over thinking the way LINQ works ? Using VB Express at home so I can't see the actual SQL that is being sent to the database (I think) as I don't have access to the SQL profiler
As mentioned, these are functionally equivalent, one just uses query syntax.
As mentioned in my comment, if you evaluate the following as a VB Statement(s) in LINQPad:
Dim lstring = Aggregate o In Test _
Into Count()
You get this in the generated SQL output window:
SELECT COUNT(*) AS [value]
FROM [Test] AS [t0]
Which is the same as the following VB LINQ expression as evaluated:
(From o In Test_
Select o.Symbol).Count
You get the exact same result.
I'm not familiar with Visual Basic, but based on
http://msdn.microsoft.com/en-us/library/bb546138.aspx
Those two approaches are the same. One uses method syntax and the other uses query syntax.
You can find out for sure by using SQL Profiler as the queries run.
PS - The "point" of LINQ is you can easily do query operations without leaving code/VB-land.
An important thing here, is that the code you give will work with a wide variety of data sources. It will hopefully do so in a very efficient way, though that can't be fully guaranteed. It certainly will be done in an efficient way with a SQL source (being converted into a SELECT COUNT(*) SQL query. It will be done efficiently if the source was an in-memory collection (it gets converted to calling the Count property). It isn't done very efficiently if the source is an enumerable that is not a collection (in this case it does read everything and count as it goes), but in that case there really isn't a more efficient way of doing this.
In each case it has done the same conceptual operation, in the most efficient manner possible, without you having to worry about the details. No big deal with counting, but a bigger deal in more complex cases.
To a certain extent, you are right when you say "in my mind, the SQL is returning all rows and the does a count on the number rows". Conceptually that is what is happening in that query, but the implementation may differ. Compare with how the real query in SQL may not match the literal interpretation of the SQL command, to allow the most efficient approach to be picked.
I think you are missing the point as Linq with SQL has late binding the search is done when you need it so when you say I need the count number then a Query is created.
Before that Linq for SQL creates Expression trees that will be "translated" in to SQL when you need it....
http://weblogs.asp.net/scottgu/archive/2007/05/19/using-linq-to-sql-part-1.aspx
http://msdn.microsoft.com/en-us/netframework/aa904594.aspx
How to debug see Scott
http://weblogs.asp.net/scottgu/archive/2007/07/31/linq-to-sql-debug-visualizer.aspx
(source: scottgu.com)
I'm trying to reproduce the HqlQuery style 'select new ObjectToProjectOut' functionality. i.e. take a list of columns returned from a query and return as a list of ObjectToProjectOut types that are instantiated using a Constructor with as many parameters as the columns in the query.
This is in effect what 'select new ObjectToProjectOut' achieves in Hql.... but clearly that's not available in SqlQuery. I think I need to set a result transform and use either PassThroughResultTransformer, DistinctRootEntityResultTransformer etc to get it to work.
Anyone know what I should use ?
ok.... after looking at the NHibernate code it seems that I was looking for AliasToBeanConstructorResultTransformer.... of course!
However I may have found an nHibernate bug. If you have the same column name returned twice from two different tables (market.name and account.name, say) then by the time nHibernate returns the array from the db to the transformer, the first occurance of 'Name' will be used for both. Nasty.
Work around is to uniquely alias. With Hql, the sql generated is heavily aliased, so this is only a bug with SqlQuery.
Grrrr. Today must be my day, also found another nHibernate bug/issue I've posted to StackOverflow for comment.
You could use the AddEntity method to fill entities from a SQL query.
Here are two examples from the NHibernate docs:
sess.CreateSQLQuery("SELECT * FROM CATS")
.AddEntity(typeof(Cat));
sess.CreateSQLQuery("SELECT ID, NAME, BIRTHDATE FROM CATS")
.AddEntity(typeof(Cat));
I am fairly new to nHibernate and DDD, so please bear with me.
I have a requirement to create a new report from my SQL table. The report is read-only and will be bound to a GridView control in an ASP.NET application.
The report contains the following fields Style, Color, Size, LAQty, MTLQty, Status.
I have the entities for Style, Color and Size, which I use in other asp.net pages. I use them via repositories. I am not sure If should use the same entities for my report or not. If I use them, where I am supposed to map the Qty and Status fields?
If I should not use the same entities, should I create a new class for the report?
As said I am new to this and just trying to learn and code properly.
Thank you
For reports its usually easier to use plain values or special DTO's. Of course you can query for the entity that references all the information, but to put it into the list (eg. using databinding) it's handier to have a single class that contains all the values plain.
To get more specific solutions as the few bellow you need to tell us a little about your domain model. How does the class model look like?
generally, you have at least three options to get "plain" values form the database using NHibernate.
Write HQL that returns an array of values
For instance:
select e1.Style, e1.Color, e1.Size, e2.LAQty, e2.MTLQty
from entity1 inner join entity2
where (some condition)
the result will be a list of object[]. Every item in the list is a row, every item in the object[] is a column. This is quite like sql, but on a higher level (you describe the query on entity level) and is database independent.
Or you create a DTO (data transfer object) only to hold one row of the result:
select new ReportDto(e1.Style, e1.Color, e1.Size, e2.LAQty, e2.MTLQty)
from entity1 inner join entity2
where (some condition)
ReportDto need to implement a constructor that has all this arguments. The result is a list of ReportDto.
Or you use CriteriaAPI (recommended)
session.CreateCriteria(typeof(Entity1), "e1")
.CreateCriteria(typeof(Entity2), "e2")
.Add( /* some condition */ )
.Add(Projections.Property("e1.Style", "Style"))
.Add(Projections.Property("e1.Color", "Color"))
.Add(Projections.Property("e1.Size", "Size"))
.Add(Projections.Property("e2.LAQty", "LAQty"))
.Add(Projections.Property("e2.MTLQty", "MTLQty"))
.SetResultTransformer(AliasToBean(typeof(ReportDto)))
.List<ReportDto>();
The ReportDto needs to have a proeprty with the name of each alias "Style", "Color" etc. The output is a list of ReportDto.
I'm not schooled in DDD exactly, but I've always modeled my nouns as types and I'm surprised the report itself is an entity. DDD or not, I wouldn't do that, rather I'd have my reports reflect the results of a query, in which quantity is presumably count(*) or sum(lineItem.quantity) and status is also calculated (perhaps, in the page).
You haven't described your domain, but there is a clue on those column headings that you may be doing a pivot over the data to create LAQty, MTLQty which you'll find hard to do in nHibernate as its designed for OLTP and does not even do UNION last I checked. That said, there is nothing wrong with abusing HQL (Hibernate Query Language) for doing lightweight reporting, as long as you understand you are abusing it.
I see Stefan has done a grand job of describing the syntax for that, so I'll stop there :-)