I'm wondering what's the best approach for getting back the total number of records for a search, and at the same time, get back the Nth 128 record block segment of data which seems to be the cap imposed by RavenDb run-time.
For instance given this query, I also need to know the total number of records.
var bookmarks = session.Query<Bookmark>()
.OrderByDescending(i => i.DateCreated)
.Skip(pageCount * (pageNumber – 1))
.Take(pageCount)
.ToList();
Thank you,
Stephen
RavenQueryStatistics stats;
var bookmarks = session.Query<Bookmark>()
.OrderByDescending(i => i.DateCreated)
.Skip(pageCount * (pageNumber – 1))
.Take(pageCount)
.Statistics(out stats)
.ToList();
int bookmarksFound = stats.TotalResults;
Related
At a high level, I need a query that can pull a subset of records based on the sum of a column, just like Linq: How to query items from a collection until the sum reaches a certain value.
However, the key difference is that he's already got his records in an object, and I don't and can't. My table can have millions of records. If I build my query the way he did, I get this error:
"A lambda expression with a statement body
cannot be converted to an expression tree"
Which makes sense after researching it, LINQ can't turn the answer in the above referenced question into valid SQL.
I'm going to make a hypothetical table that represents my situation.
Order Id | Cookie Name | Qty
1 Sugar 5
2 Snickerdoodle 4
3 Chocolate chip 8
4 Snickerdoodle 10
5 Snickerdoodle 5
Given this sample, I need to write a query that grabs the first X orders of Snickerdoodle until the summed Qty exceedes an input from the parameter (i.e. If the user chooses 13, it would return records 2 & 4 ).
I'm using Nhibernate.Linq, because I'm more comfortable in LINQ. I'm completely open to ICreate if the need arises.
As a side note, I'm interested in this as a concept as well as a direct problem. Even though I need a Sum, there has to be a way to do something akin to a takewhile that executes until a condition is met.
pragmatic approach
int needed = ...;
int actual = 0;
int page = 0;
const int pagesize = 20; // set to some sensible value, eg. the pagesize of the grid shown to the user
var results = new List<CookieOrder>();
while (actual < needed)
{
var partialResults = session.Query<CookieOrder>()
.Where(c => c.Name == "Snickerdoodle")
.OrderBy(c => c.Id)
.Skip(page * pagesize)
.Take(pagesize)
.ToList();
for(int i = 0; i < partialResults.Length && actual < needed; i++)
{
results.Add(partialResults[i]);
actual = partialResults[i].Quantity;
}
page++;
}
return results;
I need some help creating an LINQ select, i have a table with some columns in it, but only 2 of interest to this problem.
userid, type
Now this table have many thousands entries, and I only want the top, let’s say 50. So far so good, but the hard part is that there a lot of rows in success that should only be counted as 1.
Example
Type UserId
============
Add 1
Add 1
Add 1
Add 2
I would like this to only be counted as 2 in the limit of rows I am taking out, but I would like all the rows to be outputted still.
Is this possible with a single SQL request, or should I find another way to do this?
Edit: I can add columns to the table, with values if this would solve the problem.
Edit2: Sotred procedures are also an solution
Example 2: This should be counted as 3 rows
Type UserId
============
Add 1
Add 1
Add 2
Add 1
Are you stuck on LINQ?
Add a PK identity.
Order by PK.
Use a DataReader and just count the changes.
Then just stop when the changes count is at your max.
If you are not in a .NET environment then same thing with a cursor.
Since LINQ is deferred you might be able to just order in LINQ and then on a ForEach just exit.
I'm not close to a computer right now so I'm not sure is 100% correct syntax wise, but I believe you're looking for something like this:
data.Select(x => new {x.Type, x.UserId})
.GroupBy(x => x.UserId)
.Take(50);
You could do it with Linq, but it may be a LOT slower than a traditional for loop. One way would be:
data.Where((s, i) => i == 0 ||
!(s.Type == data[i-1].Type && s.UserId == data[i-1].UserId))
That would skip any "duplicate" items that have the same Type and UserID as the "previous" item.
However this ONLY works if data has an indexer (an array or something that implements IList). An IEnumerable or IQueryable would not work. Also, it is almost certainly not translatable to SQL so you'd have to pull ALL of the results and filter in-memory.
If you want to do it in SQL I would try either scanning a cursor and filling a temp table if one of the values change or using a common table expression that included a ROW_NUMBER column, then doing a look-back sub-query similar to the Linq method above:
WITH base AS
(
SELECT
Type,
UserId,
ROW_NUMBER() OVER (ORDER BY ??? ) AS RowNum
FROM Table
)
SELECT b1.Type, b1.UserId
FROM base b1
LEFT JOIN base b2 ON b1.RowNum = b2.RowNum - 1
WHERE (b1.Type <> b2.Type OR b1.UserId <> b2.UserId)
ORDER BY b1.RowNum
You can do this with LINQ, but I think it might be easier to go the "for(each) loop" route...
data.Select((x, i) => new { x.Type, x.UserId, i })
.GroupBy(x => x.Type)
.Select(g => new
{
Type = g.Key,
Items = g
.Select((x, j) => new { x.UserId, i = x.i - j })
})
.SelectMany(g => g.Select(x => new { g.Type, x.UserId, x.i }))
.GroupBy(x => new { x.Type, x.i })
.Take(50);
.SelectMany(g => g.Select(x => new { x.Type, x.UserId }));
I am trying to figure out how to do a Take and FetchMany Together.
var c = session.Query<Table1>().Take(5).FetchMany(x => x.Table2).ToList();
I expect this to return 5 rows from Table1 and unbound result from Table2. Instead I get one row back from Table1 and 5 rows from Table2.
var c = session.Query<Table1>().Take(1000).FetchMany(x => x.Table2).ToList();
I get 309 records from Table1(it has 6200 total records).
I don't get how nhibernate decided to return that many records.
well I am not sire but how this has a bearing on your code, but fetchmany will bring the whole result set to memory and then Take the specified subset.
use joinqueryover or joinalias to achieve the same..
var c = session.Query<Table1>().Take(5).FetchMany(x => x.Table2).ToList();
this becomes
Table2 table2Alias=null;
var c = session.QueryOver<Table1>().JoinAlias(x => x.Table2,()=>table2Alias).Take(5).ToList();
let me know if it works out for you.. or you can write an HQL query instead
I have a need to get a Count of Documents in a particular collection :
There is an existing index Raven/DocumentCollections that stores the Count and Name of the collection paired with the actual documents belonging to the collection. I'd like to pick up the count from this index if possible.
Here is the Map-Reduce of the Raven/DocumentCollections index :
from doc in docs
let Name = doc["#metadata"]["Raven-Entity-Name"]
where Name != null
select new { Name , Count = 1}
from result in results
group result by result.Name into g
select new { Name = g.Key, Count = g.Sum(x=>x.Count) }
On a side note, var Count = DocumentSession.Query<Post>().Count(); always returns 0 as the result for me, even though clearly there are 500 odd documents in my DB atleast 50 of them have in their metadata "Raven-Entity-Name" as "Posts". I have absolutely no idea why this Count query keeps returning 0 as the answer - Raven logs show this when Count is done
Request # 106: GET - 0 ms - TestStore - 200 - /indexes/dynamic/Posts?query=&start=0&pageSize=1&aggregation=None
For anyone still looking for the answer (this question was posted in 2011), the appropriate way to do this now is:
var numPosts = session.Query<Post>().Count();
To get the results from the index, you can use:
session.Query<Collection>("Raven/DocumentCollections")
.Where(x=>x.Name == "Posts")
.FirstOrDefault();
That will give you the result you want.
Imagine the following (simplified) database layout:
We have many "holiday" records that relate to going to a particular Accommodation on a certain date etc.
I would like to pull from the database the "best" holiday going to each accommodation (i.e. lowest price), given a set of search criteria (e.g. duration, departure airport etc).
There will be multiple records with the same price, so then we need to choose by offer saving (descending), then by departure date ascending.
I can write SQL to do this that looks like this (I'm not saying this is necessarily the most optimal way):
SELECT *
FROM Holiday h1 INNER JOIN (
SELECT h2.HolidayID,
h2.AccommodationID,
ROW_NUMBER() OVER (
PARTITION BY h2.AccommodationID
ORDER BY OfferSaving DESC
) AS RowNum
FROM Holiday h2 INNER JOIN (
SELECT AccommodationID,
MIN(price) as MinPrice
FROM Holiday
WHERE TradeNameID = 58001
/*** Other Criteria Here ***/
GROUP BY AccommodationID
) mp
ON mp.AccommodationID = h2.AccommodationID
AND mp.MinPrice = h2.price
WHERE TradeNameID = 58001
/*** Other Criteria Here ***/
) x on h1.HolidayID = x.HolidayID and x.RowNum = 1
As you can see, this uses a subquery within another subquery.
However, for several reasons my preference would be to achieve this same result in NHibernate.
Ideally, this would be done with QueryOver - the reason being that I build up the search criteria dynamically and this is much easier with QueryOver's fluent interface. (I had started out hoping to use NHibernate Linq, but unfortunately it's not mature enough).
After a lot of effort (being a relative newbie to NHibernate) I was able to re-create the very inner query that fetches all accommodations and their min price.
public IEnumerable<HolidaySearchDataDto> CriteriaFindAccommodationFromPricesForOffers(IEnumerable<IHolidayFilter<PackageHoliday>> filters, int skip, int take, out bool hasMore)
{
IQueryOver<PackageHoliday, PackageHoliday> queryable = NHibernateSession.CurrentFor(NHibernateSession.DefaultFactoryKey).QueryOver<PackageHoliday>();
queryable = queryable.Where(h => h.TradeNameId == website.TradeNameID);
var accommodation = Null<Accommodation>();
var accommodationUnit = Null<AccommodationUnit>();
var dto = Null<HolidaySearchDataDto>();
// Apply search criteria
foreach (var filter in filters)
queryable = filter.ApplyFilter(queryable, accommodationUnit, accommodation);
var query1 = queryable
.JoinQueryOver(h => h.AccommodationUnit, () => accommodationUnit)
.JoinQueryOver(h => h.Accommodation, () => accommodation)
.SelectList(hols => hols
.SelectGroup(() => accommodation.Id).WithAlias(() => dto.AccommodationId)
.SelectMin(h => h.Price).WithAlias(() => dto.Price)
);
var list = query1.OrderByAlias(() => dto.Price).Asc
.Skip(skip).Take(take+1)
.Cacheable().CacheMode(CacheMode.Normal).List<object[]>();
// Cacheing doesn't work this way...
/*.TransformUsing(Transformers.AliasToBean<HolidaySearchDataDto>())
.Cacheable().CacheMode(CacheMode.Normal).List<HolidaySearchDataDto>();*/
hasMore = list.Count() == take;
var dtos = list.Take(take).Select(h => new HolidaySearchDataDto
{
AccommodationId = (string)h[0],
Price = (decimal)h[1],
});
return dtos;
}
So my question is...
Any ideas on how to achieve what I want using QueryOver, or if necessary Criteria API?
I'd prefer not to use HQL but if it is necessary than I'm willing to see how it can be done with that too (it makes it harder (or more messy) to build up the search criteria though).
If this just isn't doable using NHibernate, then I could use a SQL query. In which case, my question is can the SQL be improved/optimised?
I have manage to achieve such dynamic search criterion by using Criteria API's. Problem I ran into was duplicates with inner and outer joins and especially related to sorting and pagination, and I had to resort to using 2 queries, 1st query for restriction and using the result of 1st query as 'in' clause in 2nd creteria.