Why Spring Data JPA is not support "limit" statement? [duplicate] - sql

In Hibernate 3, is there a way to do the equivalent of the following MySQL limit in HQL?
select * from a_table order by a_table_column desc limit 0, 20;
I don't want to use setMaxResults if possible. This definitely was possible in the older version of Hibernate/HQL, but it seems to have disappeared.

This was posted on the Hibernate forum a few years back when asked about why this worked in Hibernate 2 but not in Hibernate 3:
Limit was never a supported clause
in HQL. You are meant to use
setMaxResults().
So if it worked in Hibernate 2, it seems that was by coincidence, rather than by design. I think this was because the Hibernate 2 HQL parser would replace the bits of the query that it recognised as HQL, and leave the rest as it was, so you could sneak in some native SQL. Hibernate 3, however, has a proper AST HQL Parser, and it's a lot less forgiving.
I think Query.setMaxResults() really is your only option.

// SQL: SELECT * FROM table LIMIT start, maxRows;
Query q = session.createQuery("FROM table");
q.setFirstResult(start);
q.setMaxResults(maxRows);

If you don't want to use setMaxResults() on the Query object then you could always revert back to using normal SQL.

The setFirstResult and setMaxResults Query methods
For a JPA and Hibernate Query, the setFirstResult method is the equivalent of OFFSET, and the setMaxResults method is the equivalent of LIMIT:
List<Post> posts = entityManager.createQuery("""
select p
from Post p
order by p.createdOn
""")
.setFirstResult(10)
.setMaxResults(10)
.getResultList();
The LimitHandler abstraction
The Hibernate LimitHandler defines the database-specific pagination logic, and as illustrated by the following diagram, Hibernate supports many database-specific pagination options:
Now, depending on the underlying relational database system you are using, the above JPQL query will use the proper pagination syntax.
MySQL
SELECT p.id AS id1_0_,
p.created_on AS created_2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
LIMIT ?, ?
PostgreSQL
SELECT p.id AS id1_0_,
p.created_on AS created_2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
LIMIT ?
OFFSET ?
SQL Server
SELECT p.id AS id1_0_,
p.created_on AS created_on2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
OFFSET ? ROWS
FETCH NEXT ? ROWS ONLY
Oracle
SELECT *
FROM (
SELECT
row_.*, rownum rownum_
FROM (
SELECT
p.id AS id1_0_,
p.created_on AS created_on2_0_,
p.title AS title3_0_
FROM post p
ORDER BY p.created_on
) row_
WHERE rownum <= ?
)
WHERE rownum_ > ?
The advantage of using setFirstResult and setMaxResults is that Hibernate can generate the database-specific pagination syntax for any supported relational databases.
And, you are not limited to JPQL queries only. You can use the setFirstResult and setMaxResults method seven for native SQL queries.
Native SQL queries
You don't have to hardcode the database-specific pagination when using native SQL queries. Hibernate can add that to your queries.
So, if you're executing this SQL query on PostgreSQL:
List<Tuple> posts = entityManager.createNativeQuery(
SELECT
p.id AS id,
p.title AS title
from post p
ORDER BY p.created_on
""", Tuple.class)
.setFirstResult(10)
.setMaxResults(10)
.getResultList();
Hibernate will transform it as follows:
SELECT p.id AS id,
p.title AS title
FROM post p
ORDER BY p.created_on
LIMIT ?
OFFSET ?
Cool, right?
Beyond SQL-based pagination
Pagination is good when you can index the filtering and sorting criteria. If your pagination requirements imply dynamic filtering, it's a much better approach to use an inverted-index solution, like ElasticSearch.

If you don't want to use setMaxResults, you can also use Query.scroll instead of list, and fetch the rows you desire. Useful for paging for instance.

You can easily use pagination for this.
#QueryHints({ #QueryHint(name = "org.hibernate.cacheable", value = "true") })
#Query("select * from a_table order by a_table_column desc")
List<String> getStringValue(Pageable pageable);
you have to pass new PageRequest(0, 1)to fetch records and from the list fetch the first record.

You need to write a native query, refer this.
#Query(value =
"SELECT * FROM user_metric UM WHERE UM.user_id = :userId AND UM.metric_id = :metricId LIMIT :limit", nativeQuery = true)
List<UserMetricValue> findTopNByUserIdAndMetricId(
#Param("userId") String userId, #Param("metricId") Long metricId,
#Param("limit") int limit);

String hql = "select userName from AccountInfo order by points desc 5";
This worked for me without using setmaxResults();
Just provide the max value in the last (in this case 5) without using the keyword limit.
:P

My observation is that even you have limit in the HQL (hibernate 3.x), it will be either causing parsing error or just ignored. (if you have order by + desc/asc before limit, it will be ignored, if you don't have desc/asc before limit, it will cause parsing error)

If can manage a limit in this mode
public List<ExampleModel> listExampleModel() {
return listExampleModel(null, null);
}
public List<ExampleModel> listExampleModel(Integer first, Integer count) {
Query tmp = getSession().createQuery("from ExampleModel");
if (first != null)
tmp.setFirstResult(first);
if (count != null)
tmp.setMaxResults(count);
return (List<ExampleModel>)tmp.list();
}
This is a really simple code to handle a limit or a list.

Criteria criteria=curdSession.createCriteria(DTOCLASS.class).addOrder(Order.desc("feild_name"));
criteria.setMaxResults(3);
List<DTOCLASS> users = (List<DTOCLASS>) criteria.list();
for (DTOCLASS user : users) {
System.out.println(user.getStart());
}

Below snippet is used to perform limit query using HQL.
Query query = session.createQuery("....");
query.setFirstResult(startPosition);
query.setMaxResults(maxRows);
You can get demo application at this link.

You can use below query
NativeQuery<Object[]> query = session.createNativeQuery("select * from employee limit ?");
query.setparameter(1,1);

#Query(nativeQuery = true,
value = "select from otp u where u.email =:email order by u.dateTime desc limit 1")
public List<otp> findOtp(#Param("email") String email);

Related

How to call postgres position() function in jpa query

I'm want to retreive posts from database ordered by ids given in IN operator
I'm using below spring data JPA query to get posts from database, but postgres is not respecting order of rows specified in given postIds list in result and returning result ordered by postId descending (or ascending, doesn't matter)
#Query("SELECT p from Post p where p.postId in (:postIds))
fun findPostByIdIn(postIds: List<Long>): List<Post>
To maintain order of result, i could use this query
SELECT p from posts p where p.post_id in (10000442, 10000436, 10000440) ORDER BY POSITION(post_id::text IN '10000442, 10000436, 10000440');
My question is, how can i use position() function (which is standard postgresql function) from jpa query with given params? Do i must need to use native query? If so, how can i use native query in spring boot ?
I've tried below query but both of them are not working
#Query(
"SELECT p from posts p where p.post_id in (:postIds) ORDER BY POSITION(p.post_id::text IN ':postIds')",
nativeQuery = true
)
#Query(
"SELECT p from posts p where p.post_id in (:postIds) ORDER BY POSITION(p.post_id\\:\\:text IN ':postIds')",
nativeQuery = true
)
It says ` The column name post_id was not found in this ResultSet.`
Try to make List<Object[]> as return type instead of List then check result content to analyze the issue.

nhibernate group by and join query

I need nhiberante query (not HQL) equivalent following SQL:
SELECT ur.*
FROM (SELECT MAX(requestTime) rt, macAddress ma
FROM UpdateRequests
GROUP BY macAddress) mur
JOIN dbo.UpdateRequests ur
ON mur.ma = ur.macAddress AND mur.rt = ur.requestTime
I had no luck with other similar examples on stackoverflow.
Having UpdateRequest mapping, it seems that is not possible with Query API, how about QueryOver?
Finally one Guru suggested me to change SQL query without changing execution plan:
SELECT ur.*
FROM [dbo].[UpdateRequests] AS ur
WHERE ur.[RequestTime] = (SELECT MAX(mur.[RequestTime])
FROM [dbo].[UpdateRequests] mur
WHERE mur.[MacAddress] = ur.[MacAddress])
So in code it transforms into:
session
.Query<UpdateRequest>()
.Where(ur => ur.RequestTime == session.Query<UpdateRequest>()
.Where(mur => mur.MacAddress == ur.MacAddress)
.Max(mur => mur.RequestTime))
.ToList();
And this is exactly what i need.

Doing a WHERE .. IN subquery in Doctrine 2

I'd like to select order items from all orders with a specific item. In SQL I'd do it like this:
SELECT DISTINCT i.id, i.name, order.name
FROM items i
JOIN orders o ON i.order_id=o.id
WHERE o.id IN (
SELECT o2.id FROM orders o2
JOIN items i2 ON i2.order_id=o2.id AND i2.id=5
)
AND i.id != 5
ORDER BY o.orderdate DESC
LIMIT 10
How would I do this query with the query builder?
This is how I would try it:
/** #var Doctrine\ORM\EntityManager $em */
$expr = $em->getExpressionBuilder();
$em->createQueryBuilder()
->select(array('DISTINCT i.id', 'i.name', 'o.name'))
->from('Item', 'i')
->join('i.order', 'o')
->where(
$expr->in(
'o.id',
$em->createQueryBuilder()
->select('o2.id')
->from('Order', 'o2')
->join('Item',
'i2',
\Doctrine\ORM\Query\Expr\Join::WITH,
$expr->andX(
$expr->eq('i2.order', 'o2'),
$expr->eq('i2.id', '?1')
)
)
->getDQL()
)
)
->andWhere($expr->neq('i.id', '?2'))
->orderBy('o.orderdate', 'DESC')
->setParameter(1, 5)
->setParameter(2, 5)
;
I didn't test this of course, and made some assumptions about your models. Possible problems:
Limit: this has been somewhat of a problem in Doctrine 2, it seems query builder is not very good at accepting limits. Do take a look here, here and here.
The IN clause is usually used with an array, but I think it will work with a subquery.
You probably can use the same parameter ?1, instead of two parameters (because they're the same value), but I'm not sure.
Concluding, this may not work first time, but will surely put you on the right track. Do tell us the final 100% correct answer afterwards.
Just to avoid confusion of the last comment posted by clang1234.
The DQL query example really works. It's true that the The expr->in() will cast the second parameter to an array, in this case the DQL string. What it does, it just create an array with the DQL query string as the first element. That's exactly what the the Expr\Func is waiting for, an array. It's a little deeper in the Doctrine 2 code that the dql query string array element will be managed correctly. (see DBAL/Platforms/AbstractPlatform.php method getInExpression for more details, the array get imploded into IN())

Pagination with total row count in NHibernate

I am trying to paginate a simple query using HQL, and retrieve the total row count as part of the same query.
My query is simple enough...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Individual is mapped as a many-to-one on the ListMember class. This works great. The pagination works as expected and generates the following Sql...
SELECT TOP ( 10 /* #p0 */ ) DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM tblMembers listmember0_
inner join tblIndividuals listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
I read this article posted by Ayende called Paged data + Count(*) with NHibernate: The really easy way!, so I tried to implement it in my query.
I followed the steps in the article to add the custom HQL function called rowcount(), and changed my query to this...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Sql that is generated is almost correct, however it includes one of the columns twice resulting in this error...
System.Data.SqlClient.SqlException:
The column '...' was specified
multiple times for 'query'.
The Sql it generates looks like this...
SELECT TOP ( 10 /* #p0 */ )
col_0_0_,
col_1_0_,
Director1_0_1_,
DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT
listmember0_.DirPeerGrpMemberID as col_0_0_,
count(*) over() as col_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM RCMUser.dbo.tblDirPeerGrpMembers listmember0_
inner join RCMAlpha.dbo.tblDirectorProfileDetails listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
For some reason it includes the Director1_0_1_ column twice in the projection, which causes this error. This Sql is frustratingly close to what I would like, and I’m hoping an NHibernate expert out there can help explain why this would happen.
Suggestions Tried
Thanks to the suggestion from #Jason . I tried it with the non-generic version of .List() method to execute the query but this unfortunately also produced the same Sql with the duplicate column...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex * pageSize)
.SetMaxResults(pageSize)
.List()
.Cast<Tuple<ListMember, int>>()
.Select(x => x.First);
Update
It doesn't look like this is going to be possible without getting into the NH source code. My solution requirements have changed and I am no longer going to pursue the answer.
In summary, the solution would be to either...
Use Futures or MultiQuery to execute two statements in a single command - one to retrieve the page of data and one the total row count.
Modify your pagination solution to do without a total result count - Continuous scrolling for example.
Hmm, one issue is that you're using a ListMember-typed List method. In the example at the page you linked, he uses List() which returns a list of tuples. The first item of your tuple would be a ListMember and the second would be the row count. That List<> might affect your query and would probably throw an exception even if it did return.
Try using:
var tuples = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List();
var members = tuples.Select<Tuple<ListMember, int>, ListMember>(x => x.Item1);
but I kinda agree with #dotjoe. A MultiQuery might be easier. It's what I use. Here's a a good link about it from the same author you linked to before (Ayende).

LINQ to SQL Every Nth Row From Table

Anybody know how to write a LINQ to SQL statement to return every nth row from a table? I'm needing to get the title of the item at the top of each page in a paged data grid back for fast user scanning. So if i wanted the first record, then every 3rd one after that, from the following names:
Amy, Eric, Jason, Joe, John, Josh, Maribel, Paul, Steve, Tom
I'd get Amy, Joe, Maribel, and Tom.
I suspect this can be done... LINQ to SQL statements already invoke the ROW_NUMBER() SQL function in conjunction with sorting and paging. I just don't know how to get back every nth item. The SQL Statement would be something like WHERE ROW_NUMBER MOD 3 = 0, but I don't know the LINQ statement to use to get the right SQL.
Sometimes, TSQL is the way to go. I would use ExecuteQuery<T> here:
var data = db.ExecuteQuery<SomeObjectType>(#"
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % 25) = 1");
You could also swap out the n:
var data = db.ExecuteQuery<SomeObjectType>(#"
DECLARE #n int = 2
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % #n) = 1", n);
Once upon a time, there was no such thing as Row_Number, and yet such queries were possible. Behold!
var query =
from c in db.Customers
let i = (
from c2 in db.Customers
where c2.ID < c.ID
select c2).Count()
where i%3 == 0
select c;
This generates the following Sql
SELECT [t2].[ID], [t2]. --(more fields)
FROM (
SELECT [t0].[ID], [t0]. --(more fields)
(
SELECT COUNT(*)
FROM [dbo].[Customer] AS [t1]
WHERE [t1].[ID] < [t0].[ID]
) AS [value]
FROM [dbo].[Customer] AS [t0]
) AS [t2]
WHERE ([t2].[value] % #p0) = #p1
Here's an option that works, but it might be worth checking that it doesn't have any performance issues in practice:
var nth = 3;
var ids = Table
.Select(x => x.Id)
.ToArray()
.Where((x, n) => n % nth == 0)
.ToArray();
var nthRecords = Table
.Where(x => ids.Contains(x.Id));
Just googling around a bit I haven't found (or experienced) an option for Linq to SQL to directly support this.
The only option I can offer is that you write a stored procedure with the appropriate SQL query written out and then calling the sproc via Linq to SQL. Not the best solution, especially if you have any kind of complex filtering going on.
There really doesn't seem to be an easy way to do this:
How do I add ROW_NUMBER to a LINQ query or Entity?
How to find the ROW_NUMBER() of a row with Linq to SQL
But there's always:
peopleToFilter.AsEnumerable().Where((x,i) => i % AmountToSkipBy == 0)
NOTE: This still doesn't execute on the database side of things!
This will do the trick, but it isn't the most efficient query in the world:
var count = query.Count();
var pageSize = 10;
var pageTops = query.Take(1);
for(int i = pageSize; i < count; i += pageSize)
{
pageTops = pageTops.Concat(query.Skip(i - (i % pageSize)).Take(1));
}
return pageTops;
It dynamically constructs a query to pull the (nth, 2*nth, 3*nth, etc) value from the given query. If you use this technique, you'll probably want to create a limit of maybe ten or twenty names, similar to how Google results page (1-10, and Next), in order to avoid getting an expression so large the database refuses to attempt to parse it.
If you need better performance, you'll probably have to use a stored procedure or a view to represent your query, and include the row number as part of the stored proc results or the view's fields.