I am trying to paginate a simple query using HQL, and retrieve the total row count as part of the same query.
My query is simple enough...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Individual is mapped as a many-to-one on the ListMember class. This works great. The pagination works as expected and generates the following Sql...
SELECT TOP ( 10 /* #p0 */ ) DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM tblMembers listmember0_
inner join tblIndividuals listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
I read this article posted by Ayende called Paged data + Count(*) with NHibernate: The really easy way!, so I tried to implement it in my query.
I followed the steps in the article to add the custom HQL function called rowcount(), and changed my query to this...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Sql that is generated is almost correct, however it includes one of the columns twice resulting in this error...
System.Data.SqlClient.SqlException:
The column '...' was specified
multiple times for 'query'.
The Sql it generates looks like this...
SELECT TOP ( 10 /* #p0 */ )
col_0_0_,
col_1_0_,
Director1_0_1_,
DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT
listmember0_.DirPeerGrpMemberID as col_0_0_,
count(*) over() as col_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM RCMUser.dbo.tblDirPeerGrpMembers listmember0_
inner join RCMAlpha.dbo.tblDirectorProfileDetails listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
For some reason it includes the Director1_0_1_ column twice in the projection, which causes this error. This Sql is frustratingly close to what I would like, and I’m hoping an NHibernate expert out there can help explain why this would happen.
Suggestions Tried
Thanks to the suggestion from #Jason . I tried it with the non-generic version of .List() method to execute the query but this unfortunately also produced the same Sql with the duplicate column...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex * pageSize)
.SetMaxResults(pageSize)
.List()
.Cast<Tuple<ListMember, int>>()
.Select(x => x.First);
Update
It doesn't look like this is going to be possible without getting into the NH source code. My solution requirements have changed and I am no longer going to pursue the answer.
In summary, the solution would be to either...
Use Futures or MultiQuery to execute two statements in a single command - one to retrieve the page of data and one the total row count.
Modify your pagination solution to do without a total result count - Continuous scrolling for example.
Hmm, one issue is that you're using a ListMember-typed List method. In the example at the page you linked, he uses List() which returns a list of tuples. The first item of your tuple would be a ListMember and the second would be the row count. That List<> might affect your query and would probably throw an exception even if it did return.
Try using:
var tuples = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List();
var members = tuples.Select<Tuple<ListMember, int>, ListMember>(x => x.Item1);
but I kinda agree with #dotjoe. A MultiQuery might be easier. It's what I use. Here's a a good link about it from the same author you linked to before (Ayende).
Related
In Hibernate 3, is there a way to do the equivalent of the following MySQL limit in HQL?
select * from a_table order by a_table_column desc limit 0, 20;
I don't want to use setMaxResults if possible. This definitely was possible in the older version of Hibernate/HQL, but it seems to have disappeared.
This was posted on the Hibernate forum a few years back when asked about why this worked in Hibernate 2 but not in Hibernate 3:
Limit was never a supported clause
in HQL. You are meant to use
setMaxResults().
So if it worked in Hibernate 2, it seems that was by coincidence, rather than by design. I think this was because the Hibernate 2 HQL parser would replace the bits of the query that it recognised as HQL, and leave the rest as it was, so you could sneak in some native SQL. Hibernate 3, however, has a proper AST HQL Parser, and it's a lot less forgiving.
I think Query.setMaxResults() really is your only option.
// SQL: SELECT * FROM table LIMIT start, maxRows;
Query q = session.createQuery("FROM table");
q.setFirstResult(start);
q.setMaxResults(maxRows);
If you don't want to use setMaxResults() on the Query object then you could always revert back to using normal SQL.
The setFirstResult and setMaxResults Query methods
For a JPA and Hibernate Query, the setFirstResult method is the equivalent of OFFSET, and the setMaxResults method is the equivalent of LIMIT:
List<Post> posts = entityManager.createQuery("""
select p
from Post p
order by p.createdOn
""")
.setFirstResult(10)
.setMaxResults(10)
.getResultList();
The LimitHandler abstraction
The Hibernate LimitHandler defines the database-specific pagination logic, and as illustrated by the following diagram, Hibernate supports many database-specific pagination options:
Now, depending on the underlying relational database system you are using, the above JPQL query will use the proper pagination syntax.
MySQL
SELECT p.id AS id1_0_,
p.created_on AS created_2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
LIMIT ?, ?
PostgreSQL
SELECT p.id AS id1_0_,
p.created_on AS created_2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
LIMIT ?
OFFSET ?
SQL Server
SELECT p.id AS id1_0_,
p.created_on AS created_on2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
OFFSET ? ROWS
FETCH NEXT ? ROWS ONLY
Oracle
SELECT *
FROM (
SELECT
row_.*, rownum rownum_
FROM (
SELECT
p.id AS id1_0_,
p.created_on AS created_on2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
) row_
WHERE rownum <= ?
)
WHERE rownum_ > ?
The advantage of using setFirstResult and setMaxResults is that Hibernate can generate the database-specific pagination syntax for any supported relational databases.
And, you are not limited to JPQL queries only. You can use the setFirstResult and setMaxResults method seven for native SQL queries.
Native SQL queries
You don't have to hardcode the database-specific pagination when using native SQL queries. Hibernate can add that to your queries.
So, if you're executing this SQL query on PostgreSQL:
List<Tuple> posts = entityManager.createNativeQuery(
SELECT
p.id AS id,
p.title AS title
from post p
ORDER BY p.created_on
""", Tuple.class)
.setFirstResult(10)
.setMaxResults(10)
.getResultList();
Hibernate will transform it as follows:
SELECT p.id AS id,
p.title AS title
FROM post p
ORDER BY p.created_on
LIMIT ?
OFFSET ?
Cool, right?
Beyond SQL-based pagination
Pagination is good when you can index the filtering and sorting criteria. If your pagination requirements imply dynamic filtering, it's a much better approach to use an inverted-index solution, like ElasticSearch.
If you don't want to use setMaxResults, you can also use Query.scroll instead of list, and fetch the rows you desire. Useful for paging for instance.
You can easily use pagination for this.
#QueryHints({ #QueryHint(name = "org.hibernate.cacheable", value = "true") })
#Query("select * from a_table order by a_table_column desc")
List<String> getStringValue(Pageable pageable);
you have to pass new PageRequest(0, 1)to fetch records and from the list fetch the first record.
You need to write a native query, refer this.
#Query(value =
"SELECT * FROM user_metric UM WHERE UM.user_id = :userId AND UM.metric_id = :metricId LIMIT :limit", nativeQuery = true)
List<UserMetricValue> findTopNByUserIdAndMetricId(
#Param("userId") String userId, #Param("metricId") Long metricId,
#Param("limit") int limit);
String hql = "select userName from AccountInfo order by points desc 5";
This worked for me without using setmaxResults();
Just provide the max value in the last (in this case 5) without using the keyword limit.
:P
My observation is that even you have limit in the HQL (hibernate 3.x), it will be either causing parsing error or just ignored. (if you have order by + desc/asc before limit, it will be ignored, if you don't have desc/asc before limit, it will cause parsing error)
If can manage a limit in this mode
public List<ExampleModel> listExampleModel() {
return listExampleModel(null, null);
}
public List<ExampleModel> listExampleModel(Integer first, Integer count) {
Query tmp = getSession().createQuery("from ExampleModel");
if (first != null)
tmp.setFirstResult(first);
if (count != null)
tmp.setMaxResults(count);
return (List<ExampleModel>)tmp.list();
}
This is a really simple code to handle a limit or a list.
Criteria criteria=curdSession.createCriteria(DTOCLASS.class).addOrder(Order.desc("feild_name"));
criteria.setMaxResults(3);
List<DTOCLASS> users = (List<DTOCLASS>) criteria.list();
for (DTOCLASS user : users) {
System.out.println(user.getStart());
}
Below snippet is used to perform limit query using HQL.
Query query = session.createQuery("....");
query.setFirstResult(startPosition);
query.setMaxResults(maxRows);
You can get demo application at this link.
You can use below query
NativeQuery<Object[]> query = session.createNativeQuery("select * from employee limit ?");
query.setparameter(1,1);
#Query(nativeQuery = true,
value = "select from otp u where u.email =:email order by u.dateTime desc limit 1")
public List<otp> findOtp(#Param("email") String email);
I'm having a problem with a slight ordering anomaly in a legacy web application, and figured I'd start with the back-end SQL query generated by Hibernate with DB2Dialect:
FROM (SELECT inner2_.*,
ROWNUMBER()
OVER(
ORDER BY ORDER OF inner2_) AS rownumber_
FROM (SELECT this_.sohn AS SOHN1_15_11_,
this_.aslc AS ASLC2_15_11_,
this_.cc AS CC3_15_11_,
bb1_.sbn AS SBN1_2_0_,
bb1_.abc AS ABC3_4_5_,
mh2_.smhn AS SMHN1_9_1_,
mh2_.sabc AS SABC3_4_6_,
og8_.sogn AS SOGN1_11_2_,
og8_.sogo AS SOGO3_4_7_,
oc9_.socn AS SOCN_1_13_3_,
oc9_.soco AS SOCO_3_4_8_
FROM ott.oh this_
INNER JOIN ott.bb1_
ON this_.sbn = bb1_.sbn
INNER JOIN ott.mh2_
ON this_.smhn = mh2_.smhn
LEFT OUTER JOIN ott.og og8_
ON this_.sogn = og8_.sogn
LEFT OUTER JOIN ott.oc oc9_
ON this_.socn = oc9_.socn
WHERE ( 1 = 1 )
AND bb1_.sbn = ?
AND mh2_.smhn = ?
FETCH first 200 ROWS only) AS inner2_) AS inner1_
WHERE rownumber_ > 190
ORDER BY rownumber_
What does this query do? I am especially curious about OVER(), which isn't coming up when I google for such a SQL function (but it is an MDX function?).
This query functions in the application to grab the last page of a paginated list that is ordered by a field that doesn't even appear in the query. The query to populate the first page on initial load is different, and its generated SQL does ORDER BY the desired field.
So to get through this I need to understand how the query functions. Takers?
OVER() is part of so called OLAP functions - a good desrciption can be found in the DB2 SQL Cookbook - i.e. available here:
http://www.ids-system.de/images/Downloads/DB2V97CK.PDF
It is a group of really useful functions.
Also good additional stuff
http://www.ibm.com/developerworks/data/library/techarticle/dm-0401kuznetsov/
I'd like to select order items from all orders with a specific item. In SQL I'd do it like this:
SELECT DISTINCT i.id, i.name, order.name
FROM items i
JOIN orders o ON i.order_id=o.id
WHERE o.id IN (
SELECT o2.id FROM orders o2
JOIN items i2 ON i2.order_id=o2.id AND i2.id=5
)
AND i.id != 5
ORDER BY o.orderdate DESC
LIMIT 10
How would I do this query with the query builder?
This is how I would try it:
/** #var Doctrine\ORM\EntityManager $em */
$expr = $em->getExpressionBuilder();
$em->createQueryBuilder()
->select(array('DISTINCT i.id', 'i.name', 'o.name'))
->from('Item', 'i')
->join('i.order', 'o')
->where(
$expr->in(
'o.id',
$em->createQueryBuilder()
->select('o2.id')
->from('Order', 'o2')
->join('Item',
'i2',
\Doctrine\ORM\Query\Expr\Join::WITH,
$expr->andX(
$expr->eq('i2.order', 'o2'),
$expr->eq('i2.id', '?1')
)
)
->getDQL()
)
)
->andWhere($expr->neq('i.id', '?2'))
->orderBy('o.orderdate', 'DESC')
->setParameter(1, 5)
->setParameter(2, 5)
;
I didn't test this of course, and made some assumptions about your models. Possible problems:
Limit: this has been somewhat of a problem in Doctrine 2, it seems query builder is not very good at accepting limits. Do take a look here, here and here.
The IN clause is usually used with an array, but I think it will work with a subquery.
You probably can use the same parameter ?1, instead of two parameters (because they're the same value), but I'm not sure.
Concluding, this may not work first time, but will surely put you on the right track. Do tell us the final 100% correct answer afterwards.
Just to avoid confusion of the last comment posted by clang1234.
The DQL query example really works. It's true that the The expr->in() will cast the second parameter to an array, in this case the DQL string. What it does, it just create an array with the DQL query string as the first element. That's exactly what the the Expr\Func is waiting for, an array. It's a little deeper in the Doctrine 2 code that the dql query string array element will be managed correctly. (see DBAL/Platforms/AbstractPlatform.php method getInExpression for more details, the array get imploded into IN())
Anybody know how to write a LINQ to SQL statement to return every nth row from a table? I'm needing to get the title of the item at the top of each page in a paged data grid back for fast user scanning. So if i wanted the first record, then every 3rd one after that, from the following names:
Amy, Eric, Jason, Joe, John, Josh, Maribel, Paul, Steve, Tom
I'd get Amy, Joe, Maribel, and Tom.
I suspect this can be done... LINQ to SQL statements already invoke the ROW_NUMBER() SQL function in conjunction with sorting and paging. I just don't know how to get back every nth item. The SQL Statement would be something like WHERE ROW_NUMBER MOD 3 = 0, but I don't know the LINQ statement to use to get the right SQL.
Sometimes, TSQL is the way to go. I would use ExecuteQuery<T> here:
var data = db.ExecuteQuery<SomeObjectType>(#"
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % 25) = 1");
You could also swap out the n:
var data = db.ExecuteQuery<SomeObjectType>(#"
DECLARE #n int = 2
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % #n) = 1", n);
Once upon a time, there was no such thing as Row_Number, and yet such queries were possible. Behold!
var query =
from c in db.Customers
let i = (
from c2 in db.Customers
where c2.ID < c.ID
select c2).Count()
where i%3 == 0
select c;
This generates the following Sql
SELECT [t2].[ID], [t2]. --(more fields)
FROM (
SELECT [t0].[ID], [t0]. --(more fields)
(
SELECT COUNT(*)
FROM [dbo].[Customer] AS [t1]
WHERE [t1].[ID] < [t0].[ID]
) AS [value]
FROM [dbo].[Customer] AS [t0]
) AS [t2]
WHERE ([t2].[value] % #p0) = #p1
Here's an option that works, but it might be worth checking that it doesn't have any performance issues in practice:
var nth = 3;
var ids = Table
.Select(x => x.Id)
.ToArray()
.Where((x, n) => n % nth == 0)
.ToArray();
var nthRecords = Table
.Where(x => ids.Contains(x.Id));
Just googling around a bit I haven't found (or experienced) an option for Linq to SQL to directly support this.
The only option I can offer is that you write a stored procedure with the appropriate SQL query written out and then calling the sproc via Linq to SQL. Not the best solution, especially if you have any kind of complex filtering going on.
There really doesn't seem to be an easy way to do this:
How do I add ROW_NUMBER to a LINQ query or Entity?
How to find the ROW_NUMBER() of a row with Linq to SQL
But there's always:
peopleToFilter.AsEnumerable().Where((x,i) => i % AmountToSkipBy == 0)
NOTE: This still doesn't execute on the database side of things!
This will do the trick, but it isn't the most efficient query in the world:
var count = query.Count();
var pageSize = 10;
var pageTops = query.Take(1);
for(int i = pageSize; i < count; i += pageSize)
{
pageTops = pageTops.Concat(query.Skip(i - (i % pageSize)).Take(1));
}
return pageTops;
It dynamically constructs a query to pull the (nth, 2*nth, 3*nth, etc) value from the given query. If you use this technique, you'll probably want to create a limit of maybe ten or twenty names, similar to how Google results page (1-10, and Next), in order to avoid getting an expression so large the database refuses to attempt to parse it.
If you need better performance, you'll probably have to use a stored procedure or a view to represent your query, and include the row number as part of the stored proc results or the view's fields.
I am currently working on a project using NHiberate as the DAL with .NET 2.0 and NHibernate 2.2.
Today I came to a point where I had to join a bunch of entities/collections to get what I want. That is fine.
What got me was that I do not want the query to return a list of objects of a certain entity type but rather the result would include various properties from different entities.
The following query is not what I am doing but it is kind of query that I am talking about here.
select order.id, sum(price.amount), count(item)
from Order as order
join order.lineItems as item
join item.product as product,
Catalog as catalog
join catalog.prices as price
where order.paid = false
and order.customer = :customer
and price.product = product
and catalog.effectiveDate < sysdate
and catalog.effectiveDate >= all (
select cat.effectiveDate
from Catalog as cat
where cat.effectiveDate < sysdate
)
group by order
having sum(price.amount) > :minAmount
order by sum(price.amount) desc
My question is, in this case what type result is supposed to be returned? It is certainly not of type Order, neither is of type LineItems.
Thanks for your help!
John
you can always use List of object[] for returning data and it will work fine.
This is called a projection, and it happens any time you specify an explicit select clause that contains rows from various tables (or even aggregate / summary data from a single table).
Using LINQ you can create anonymous objects to store these rows of data, like this:
var crunchies = (from foo in bar
where foo.baz == quux
select new { foo.corge, foo.grault }).ToList();
Then you can do crunchies[0].corge for example to pull out the rows & columns.
If you are using NHibernate.Linq this will "just work".
If you're using HQL or Criteria API, then what Fahad mentioned will work. You'll get a List<object[]> as a result, and the index of the array references the order of the columns that you returned in your select clause.