LINQ to SQL Every Nth Row From Table - sql

Anybody know how to write a LINQ to SQL statement to return every nth row from a table? I'm needing to get the title of the item at the top of each page in a paged data grid back for fast user scanning. So if i wanted the first record, then every 3rd one after that, from the following names:
Amy, Eric, Jason, Joe, John, Josh, Maribel, Paul, Steve, Tom
I'd get Amy, Joe, Maribel, and Tom.
I suspect this can be done... LINQ to SQL statements already invoke the ROW_NUMBER() SQL function in conjunction with sorting and paging. I just don't know how to get back every nth item. The SQL Statement would be something like WHERE ROW_NUMBER MOD 3 = 0, but I don't know the LINQ statement to use to get the right SQL.

Sometimes, TSQL is the way to go. I would use ExecuteQuery<T> here:
var data = db.ExecuteQuery<SomeObjectType>(#"
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % 25) = 1");
You could also swap out the n:
var data = db.ExecuteQuery<SomeObjectType>(#"
DECLARE #n int = 2
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (ORDER BY id) AS [__row]
FROM [YourTable]) x WHERE (x.__row % #n) = 1", n);

Once upon a time, there was no such thing as Row_Number, and yet such queries were possible. Behold!
var query =
from c in db.Customers
let i = (
from c2 in db.Customers
where c2.ID < c.ID
select c2).Count()
where i%3 == 0
select c;
This generates the following Sql
SELECT [t2].[ID], [t2]. --(more fields)
FROM (
SELECT [t0].[ID], [t0]. --(more fields)
(
SELECT COUNT(*)
FROM [dbo].[Customer] AS [t1]
WHERE [t1].[ID] < [t0].[ID]
) AS [value]
FROM [dbo].[Customer] AS [t0]
) AS [t2]
WHERE ([t2].[value] % #p0) = #p1

Here's an option that works, but it might be worth checking that it doesn't have any performance issues in practice:
var nth = 3;
var ids = Table
.Select(x => x.Id)
.ToArray()
.Where((x, n) => n % nth == 0)
.ToArray();
var nthRecords = Table
.Where(x => ids.Contains(x.Id));

Just googling around a bit I haven't found (or experienced) an option for Linq to SQL to directly support this.
The only option I can offer is that you write a stored procedure with the appropriate SQL query written out and then calling the sproc via Linq to SQL. Not the best solution, especially if you have any kind of complex filtering going on.

There really doesn't seem to be an easy way to do this:
How do I add ROW_NUMBER to a LINQ query or Entity?
How to find the ROW_NUMBER() of a row with Linq to SQL
But there's always:
peopleToFilter.AsEnumerable().Where((x,i) => i % AmountToSkipBy == 0)
NOTE: This still doesn't execute on the database side of things!

This will do the trick, but it isn't the most efficient query in the world:
var count = query.Count();
var pageSize = 10;
var pageTops = query.Take(1);
for(int i = pageSize; i < count; i += pageSize)
{
pageTops = pageTops.Concat(query.Skip(i - (i % pageSize)).Take(1));
}
return pageTops;
It dynamically constructs a query to pull the (nth, 2*nth, 3*nth, etc) value from the given query. If you use this technique, you'll probably want to create a limit of maybe ten or twenty names, similar to how Google results page (1-10, and Next), in order to avoid getting an expression so large the database refuses to attempt to parse it.
If you need better performance, you'll probably have to use a stored procedure or a view to represent your query, and include the row number as part of the stored proc results or the view's fields.

Related

Rank over multiple columns executed by the server?

How can I implement the RANK() function taking into account two columns for the ranking? The main column does not have unique values. This is the query:
select *, RANK() over (order by score, posteddate desc) as rank from Post
I need to implement pagination without the offset limit pattern and I thought a kind of ranking function would be ok. I have a partial implementation which only works with uniques, using the '>' or '<' operands on the key used for pagination.
Any idea? I cannot find a solution online.
Cheers.
Edit upon request: I am using c#.
We do paging with .Skip(pageIndex * pageSize) and .Take(pageSize). You just need to make sure you have ordered the data first. This will all execute on the server.
For example, if you want the content for page 4 and each page should contain 20 results:
var pageIndex = 4;
var pageSize = 20;
var pageContent = db.Post
.OrderBy(p => p.score)
.ThenByDescending(p => p.posteddate)
.Skip(pageIndex * pageSize)
.Take(pageSize);
The generated SQL here automatically includes the addition of ROW_NUMBER() and TOP. It works pretty well up to a couple hundred thousand records.
Not exactly an answer, but I found a better approach as described in http://use-the-index-luke.com/sql/partial-results/fetch-next-page
SELECT *
FROM ( SELECT *
FROM sales
WHERE sale_date <= ?
AND NOT (sale_date = ? AND sale_id >= ?)
ORDER BY sale_date DESC, sale_id DESC
)
WHERE rownum <= 10

Create a LINQ for SQL query

I'm learning Linq and using MVC. I have written a SQL query which I need to convert to a LINQ query.
select TokenID,TokenAsset,packet from TokenTable where id = 6 and packet = ''
and TokenID not in (select TokenID from TokenTable where id=6 and packet <> '')
group by TokenID,TokenAsset,Packet
I kindly ask help to convert the above query to a LINQ query. I know that the SQL query isn't efficient. It would better if you can help me to fix it.
Try this one:
var result = Tokens.Where(x=>x.Id==6 &&
x.Packet=="" &&
!Tokens.Exists(y=>y.TokenID==x.TokenID &&
y.Id==6 &&
y.Packet!="")
)
.GroupBy(x=>x.ID)
.ThenGroupBy(x=>x.TokenAsset)
.ThenGroupBy(x=>x.Packet);
Note I suppose that collection Tokens holds all the tokens you have.
Firstly your SQL query can just be
select distinct TokenID, TokenAsset, packet
from TokenTable
where id = 6 and packet = ''
the group by is not that useful since there are no aggregated columns. All selected columns are in the group by clause. Use distinct to achieve the same.
the secondary AND condition for tokenid is also redundant. It is exclusive to the first condition and hence doesn't change the result.
use this LINQ query:
var results = dbcontext.TokenTables
.Where(t => t.id == 6 && t.Packet == "")
.Select(t => new { t.TokenId, t.TokenAsset, t.Packet }).Distinct();
project only columns you need for performant calls by avoiding extra data transfer.

How to use SQL Count() in QueryOver

I'm trying to perform some simple SQL (using sql server 2005) using QueryOver in NHibernate. The query is to count how many times a serial number is repeatedly used in a list of items and then to select the serial numbers with only 1 use. Note that I don't want distinct serial numbers since I don't want to use serial numbers that have >1 use.
The SQL query to do this is:
SELECT SERNUM, expr1
FROM (SELECT SERNUM, COUNT(SERNUM) AS expr1
FROM ITEM
GROUP BY SERNUM) AS derivedtbl_1
WHERE (expr1 = 1)
What I have done so far using QueryOver is:
var query = session.QueryOver<Item>()
.Select(Projections.ProjectionList()
.Add(Projections.Count<Item>(x => x.Sernum))
.Add(Projections.GroupProperty("Sernum"))).List();
This code generates SQL (though it does not like putting the Count() column to a List). I'm not sure how to access the column of data generated by Count() to say only return where it is 1. The SQL that it does generates is:
SELECT count(this_.SERNUM) as y0_,
this_.SERNUM as y1_
FROM ITEM this_
GROUP BY this_.SERNUM
I'm not sure if I'm approaching this the right way but hopefully someone can point me to it.
var serials = session.QueryOver<Item>()
.Where(Restrictions.Eq(Projections.Count<Item>(i => i.Id), 1));
.Select(Projections.GroupProperty<Item>(i => i.Sernum))
.List();
should generate something like
SELECT SERNUM FROM ITEM GROUP BY SERNUM HAVING COUNT(*) = 1
to get the items, do something like
var subquery = QueryOver.Of<Item>()
.Where(Restrictions.Eq(Projections.Count<Item>(i => i.Id), 1));
.Select(Projections.GroupProperty<Item>(i => i.Sernum));
var items = QueryOver.Of<Item>()
.WithSubquery.Where(i => i.Sernum).In(subquery);
.List();

Pagination with total row count in NHibernate

I am trying to paginate a simple query using HQL, and retrieve the total row count as part of the same query.
My query is simple enough...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Individual is mapped as a many-to-one on the ListMember class. This works great. The pagination works as expected and generates the following Sql...
SELECT TOP ( 10 /* #p0 */ ) DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM tblMembers listmember0_
inner join tblIndividuals listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
I read this article posted by Ayende called Paged data + Count(*) with NHibernate: The really easy way!, so I tried to implement it in my query.
I followed the steps in the article to add the custom HQL function called rowcount(), and changed my query to this...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List<ListMember>();
The Sql that is generated is almost correct, however it includes one of the columns twice resulting in this error...
System.Data.SqlClient.SqlException:
The column '...' was specified
multiple times for 'query'.
The Sql it generates looks like this...
SELECT TOP ( 10 /* #p0 */ )
col_0_0_,
col_1_0_,
Director1_0_1_,
DirPeerG1_1_0_,
Director1_0_1_,
Director2_1_0_,
Forename2_0_1_,
Surname0_1_
FROM (SELECT
listmember0_.DirPeerGrpMemberID as col_0_0_,
count(*) over() as col_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirPeerGrpMemberID as DirPeerG1_1_0_,
listmember1_.DirectorKeyID as Director1_0_1_,
listmember0_.DirectorKeyId as Director2_1_0_,
listmember1_.Forename1 as Forename2_0_1_,
listmember1_.Surname as Surname0_1_,
ROW_NUMBER()
OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
FROM RCMUser.dbo.tblDirPeerGrpMembers listmember0_
inner join RCMAlpha.dbo.tblDirectorProfileDetails listmember1_
on listmember0_.DirectorKeyId = listmember1_.DirectorKeyID) as query
WHERE query.__hibernate_sort_row > 10 /* #p1 */
ORDER BY query.__hibernate_sort_row
For some reason it includes the Director1_0_1_ column twice in the projection, which causes this error. This Sql is frustratingly close to what I would like, and I’m hoping an NHibernate expert out there can help explain why this would happen.
Suggestions Tried
Thanks to the suggestion from #Jason . I tried it with the non-generic version of .List() method to execute the query but this unfortunately also produced the same Sql with the duplicate column...
var members = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex * pageSize)
.SetMaxResults(pageSize)
.List()
.Cast<Tuple<ListMember, int>>()
.Select(x => x.First);
Update
It doesn't look like this is going to be possible without getting into the NH source code. My solution requirements have changed and I am no longer going to pursue the answer.
In summary, the solution would be to either...
Use Futures or MultiQuery to execute two statements in a single command - one to retrieve the page of data and one the total row count.
Modify your pagination solution to do without a total result count - Continuous scrolling for example.
Hmm, one issue is that you're using a ListMember-typed List method. In the example at the page you linked, he uses List() which returns a list of tuples. The first item of your tuple would be a ListMember and the second would be the row count. That List<> might affect your query and would probably throw an exception even if it did return.
Try using:
var tuples = UnitOfWork.CurrentSession.CreateQuery(#"
select m, rowcount()
from ListMember as m
join fetch m.Individual as i")
.SetFirstResult(pageIndex*pageSize)
.SetMaxResults(pageSize)
.List();
var members = tuples.Select<Tuple<ListMember, int>, ListMember>(x => x.Item1);
but I kinda agree with #dotjoe. A MultiQuery might be easier. It's what I use. Here's a a good link about it from the same author you linked to before (Ayende).

getting count(*) using createSQLQuery in hibernate?

I have several sql queries that I simply want to fire at the database.
I am using hibernate throughout the whole application, so i would prefer to use hibernate to call this sql queries.
In the example below i want to get count + name, but cant figure out how to get that info when i use createSQLQuery().
I have seen workarounds where people only need to get out a single "count()" from the result, but in this case I am using count() + a column as ouput
SELECT count(*), a.name as count FROM user a
WHERE a.user_id IN (SELECT b.user_id FROM user b)
GROUP BY a.name
HAVING COUNT(*) BETWEEN 2 AND 5;
fyi, the above query would deliver a result like this if i call it directly on the database:
1, John
2, Donald
1, Ralph
...
Alternatively, you can use
SQLQuery query = session.createSQLQuery("SELECT count(*) as num, a.name as name FROM user a WHERE a.user_id IN (SELECT b.user_id FROM user b) GROUP BY a.name HAVING COUNT(*) BETWEEN 2 AND 5;";
query.addScalar("num", Hibernate.INTEGER).addScalar("name", Hibernate.STRING);
// you might need to use org.hibernate.type.StandardBasicTypes.INTEGER / STRING
// for Hibernate v3.6+,
// see https://hibernate.onjira.com/browse/HHH-5138
List<Object> result = query.list();
// result.get(2*i + 0) -> i-th row num
// result.get(2*i + 1) -> i-th row name
I'm using this in case of time-pressure, imo much faster to code then creating your own beans & transformers.
Cheers!
Jakub
cheers for the info Thomas, worked wonderful for generating objects
the problem i had with my initial query was that "count" was a reserved word :P
when i changed the name to something else it worked.
If your SQL statement looks like this SELECT count(*) as count, a.name as name... you could use setResultTransformer(new AliasToBeanResultTransformer(YourSimpleBean.class)) on your Query.
Where YourSimpleBean has the fields Integer count and String name respectively the setters setCount and setName.
On execution of the query with query.list() hibernate will return a List of YourSimpleBeans.