Sitecore Lucene search - hit count differs from result count - lucene

I have the following method to return search results based on a supplied query
private List<Item> GetResults(QueryBase qBase)
{
using (IndexSearchContext sc = SearchManager.GetIndex("story").CreateSearchContext())
{
var hits = sc.Search(qBase, int.MaxValue);
var h1 = hits.FetchResults(0, 25);
var h2 = h1.Select(r => r.GetObject<Item>());
var h3 = h2.Where(item => item != null);
return h3.ToList();
}
}
The index being searched indexes web and master content. If I pass in a query that I know matches a single published item and break at the line beginning var h2 = then I see the variable hits has 2 items. This I expect because actually the items are both the same item, one from web and one from master.
However, the variable h1 only has a single result. The result from web has been omitted.
This is the case whether I'm debugging in the context of web or master. Can anyone explain?

When fetching the items using FetchResults method, Sitecore groups the items from lucene by the id of the item. First of the items becomes a SearchResult in the resulting SearchResultCollection object, and other items become Subresults for this results.
For example if you have a home item with id {110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9} with one published version and 4 versions in different languages for the home item, what you'll get from lucene is a single result and 4 subresults for this result:
using (IndexSearchContext sc = SearchManager.GetIndex("story").CreateSearchContext())
{
var hits = sc.Search(qBase, int.MaxValue);
var h1 = hits.FetchResults(0, 25);
foreach (SearchResult result in h1)
{
var url = result.Url;
foreach (SearchResult subresult in result.Subresults)
{
var subUrl = subresult.Url; // other versions of this item
}
}
}
The urls for results and subresults in my case would be:
sitecore://web/{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}?lang=en&ver=1
sitecore://master/{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}?lang=en&ver=1 (subresult)
sitecore://master/{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}?lang=ja-JP&ver=1 (subresult)
sitecore://master/{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}?lang=de-DE&ver=1 (subresult)
sitecore://master/{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}?lang=da&ver=1 (subresult)
so for retrieving the all the items with their versions you can use this code:
private List<Item> GetResults(QueryBase qBase)
{
using (IndexSearchContext sc = SearchManager.GetIndex("story").CreateSearchContext())
{
var hits = sc.Search(qBase, int.MaxValue);
var h1 = hits.FetchResults(0, 25);
var h2 = h1.Select(r => r.GetObject<Item>()).ToList();
// add other versions of item to the resulting list
foreach (IEnumerable<SearchResult> subresults in h1.Select(sr => sr.Subresults))
{
h2.AddRange(subresults.Select(r => r.GetObject<Item>()));
}
var h3 = h2.Where(item => item != null);
return h3.ToList();
}
}
You can not assume with item will be returned as the first one from the lucene and which items will be returned as subresults. If you want to get any specific item you need to pass version number, language and database to the query.

Related

Get total results for revisions (RavenDB Client API)

I am trying to display document revisions in a paginated list. This will only work when I know the total results count so I can calculate the page count.
For common document queries this is possible with session.Query(...).Statistics(out var stats).
My current solution is the following:
// get paged revisions
List<T> items = await session.Advanced.Revisions.GetForAsync<T>(id, (pageNumber - 1) * pageSize, pageSize);
double totalResults = items.Count;
// get total results in case items count equals page size
if (pageSize <= items.Count || pageNumber > 1)
{
GetRevisionsCommand command = new GetRevisionsCommand(id, 0, 1, true);
session.Advanced.RequestExecutor.Execute(command, session.Advanced.Context);
command.Result.Results.Parent.TryGet("TotalResults", out totalResults);
}
Problem with this approach is that I need an additional request just to get the TotalResults property which indeed has already been returned by the first request. I just don't know how to access it.
You are right the TotalResults is returned from server but not parsed on the client side.
I opened an issue to fix that: https://issues.hibernatingrhinos.com/issue/RavenDB-15552
You can also get the total revisions count for document by using the /databases/{dbName}/revisions/count?id={docId} endpoint, client code for example:
var docId = "company/1";
var dbName = store.Database;
var json = await store.GetRequestExecutor().HttpClient
.GetAsync(store.Urls.First() + $"/databases/{dbName}/revisions/count?id={docId}").Result.Content.ReadAsStringAsync();
using var ctx = JsonOperationContext.ShortTermSingleUse();
using var obj = ctx.ReadForMemory(json, "revision-count");
obj.TryGet("RevisionsCount", out long revisionsCount);
Another way could be getting all the revisions:
var revisions = await session.Advanced.Revisions.GetForAsync<Company>(docId, 0, int.MaxValue);
Then using the revisions.Count as the total count.

Why aren't my Sharepoint List contents being saved to the list?

I'm successfully creating a Sharepoint List (named "XMLToPDFTestList"), which I can see via Site Actions > View All Site Content, but my attempts to add columns to the list has so far proven fruitless.
Here is how I'm trying to do it:
private void ProvisionallyCreateList()
{
SPWeb mySite = SPContext.Current.Web;
// Check to see if list already exists; if so, exit
if (mySite.Lists.TryGetList(listTitle) != null) return;
SPListCollection lists = mySite.Lists;
SPListTemplateType listTemplateType = new SPListTemplateType();
listTemplateType = SPListTemplateType.GenericList;
string listDescription = "This list is to hold inputted vals";
lists.Add(listTitle, listDescription, listTemplateType);
// Now add a couple of columns
SPList list = lists["XMLToPDFTestList"];
string faveNum = list.Fields.Add("favoriteNumber", SPFieldType.Text, false);
list.Fields[faveNum].Description = "favorite number";
list.Fields[faveNum].Update();
string faveCol = list.Fields.Add("favoriteColor", SPFieldType.Text, false);
list.Fields[faveCol].Description = "favorite color";
list.Fields[faveCol].Update();
}
This is all I see when I click "XMLToPDFTestList":
My "gut feeling" is that this line:
SPList list = lists["XMLToPDFTestList"];
...is not right/not specific enough. Instead of "XMLToPDFTestList" it should be something else/prepend something, or so. But what, exactly?
It was, as is so often the case, "my bad" (YMMV?).
The problem is in my list item creation code which, because I was not assigning anything to the default/inherited "Title" field, made it appear to me (as in the scream shot above) that no item was being added.
Once I fixed the code, by changing this:
private void SaveInputToList()
{
using (SPSite site = new SPSite(siteUrl))
{
using (SPWeb web = site.RootWeb)
{
SPList list = web.Lists[listTitle];
SPListItem SPListItemFaveNum = list.Items.Add();
SPListItemFaveNum["favoriteNumber"] = "7"; //inputtedNumber; TODO: Once 7 and teal are being saved and retrieved successfully, assign the var vals - will need to declare the controls created in CreateChildControls() globally
SPListItemFaveNum.Update();
SPListItem SPListItemFaveHue = list.Items.Add();
SPListItemFaveHue["favoriteColor"] = "teal";
SPListItemFaveHue.Update();
}
}
}
...to this:
private void SaveInputToList()
{
using (SPSite site = new SPSite(siteUrl))
{
using (SPWeb web = site.RootWeb)
{
SPList list = web.Lists[listTitle];
SPListItem spli = list.Items.Add();
spli["Title"] = "Write the Title";
spli["favoriteNumber"] = "7";
//SPListItemFaveNum.Update();
//SPListItem SPListItemFaveHue = list.Items.Add();
spli["favoriteColor"] = "teal";
//SPListItemFaveHue.Update();
spli.Update();
}
}
}
...it works fine: an item is added with all three values (Title, favoriteNumber, and favoriteColor).
I was assuming the item wasn't being created because "Title" was blank, and I was calling update on each SPListItem, whereas all I really need to do is call Update once, and on one SPListItem, not multiple.

Why can I not use Continuation when using a proxy class to access MS CRM 2013?

So I have a standard service reference proxy calss for MS CRM 2013 (i.e. right-click add reference etc...) I then found the limitation that CRM data calls limit to 50 results and I wanted to get the full list of results. I found two methods, one looks more correct, but doesn't seem to work. I was wondering why it didn't and/or if there was something I'm doing incorrectly.
Basic setup and process
crmService = new CrmServiceReference.MyContext(new Uri(crmWebServicesUrl));
crmService.Credentials = System.Net.CredentialCache.DefaultCredentials;
var accountAnnotations = crmService.AccountSet.Where(a => a.AccountNumber = accountNumber).Select(a => a.Account_Annotation).FirstOrDefault();
Using Continuation (something I want to work, but looks like it doesn't)
while (accountAnnotations.Continuation != null)
{
accountAnnotations.Load(crmService.Execute<Annotation>(accountAnnotations.Continuation.NextLinkUri));
}
using that method .Continuation is always null and accountAnnotations.Count is always 50 (but there are more than 50 records)
After struggling with .Continutation for a while I've come up with the following alternative method (but it seems "not good")
var accountAnnotationData = accountAnnotations.ToList();
var accountAnnotationFinal = accountAnnotations.ToList();
var index = 1;
while (accountAnnotationData.Count == 50)
{
accountAnnotationData = (from a in crmService.AnnotationSet
where a.ObjectId.Id == accountAnnotationData.First().ObjectId.Id
select a).Skip(50 * index).ToList();
accountAnnotationFinal = accountAnnotationFinal.Union(accountAnnotationData).ToList();
index++;
}
So the second method seems to work, but for any number of reasons it doesn't seem like the best. Is there a reason .Continuation is always null? Is there some setup step I'm missing or some nice way to do this?
The way to get the records from CRM is to use paging here is an example with a query expression but you can also use fetchXML if you want
// Query using the paging cookie.
// Define the paging attributes.
// The number of records per page to retrieve.
int fetchCount = 3;
// Initialize the page number.
int pageNumber = 1;
// Initialize the number of records.
int recordCount = 0;
// Define the condition expression for retrieving records.
ConditionExpression pagecondition = new ConditionExpression();
pagecondition.AttributeName = "address1_stateorprovince";
pagecondition.Operator = ConditionOperator.Equal;
pagecondition.Values.Add("WA");
// Define the order expression to retrieve the records.
OrderExpression order = new OrderExpression();
order.AttributeName = "name";
order.OrderType = OrderType.Ascending;
// Create the query expression and add condition.
QueryExpression pagequery = new QueryExpression();
pagequery.EntityName = "account";
pagequery.Criteria.AddCondition(pagecondition);
pagequery.Orders.Add(order);
pagequery.ColumnSet.AddColumns("name", "address1_stateorprovince", "emailaddress1", "accountid");
// Assign the pageinfo properties to the query expression.
pagequery.PageInfo = new PagingInfo();
pagequery.PageInfo.Count = fetchCount;
pagequery.PageInfo.PageNumber = pageNumber;
// The current paging cookie. When retrieving the first page,
// pagingCookie should be null.
pagequery.PageInfo.PagingCookie = null;
Console.WriteLine("#\tAccount Name\t\t\tEmail Address");while (true)
{
// Retrieve the page.
EntityCollection results = _serviceProxy.RetrieveMultiple(pagequery);
if (results.Entities != null)
{
// Retrieve all records from the result set.
foreach (Account acct in results.Entities)
{
Console.WriteLine("{0}.\t{1}\t\t{2}",
++recordCount,
acct.EMailAddress1,
acct.Name);
}
}
// Check for more records, if it returns true.
if (results.MoreRecords)
{
// Increment the page number to retrieve the next page.
pagequery.PageInfo.PageNumber++;
// Set the paging cookie to the paging cookie returned from current results.
pagequery.PageInfo.PagingCookie = results.PagingCookie;
}
else
{
// If no more records are in the result nodes, exit the loop.
break;
}
}

RavenDB : Use "Search" only if given within one query

I have a situation where my user is presented with a grid, and it, by default, will just get the first 15 results. However they may type in a name and search for an item across all pages.
Alone, either of these works fine, but I am trying to figure out how to make them work as a single query. This is basically what it looks like...
// find a filter if the user is searching
var filters = request.Filters.ToFilters();
// determine the name to search by
var name = filters.Count > 0 ? filters[0] : null;
// we need to be able to catch some query statistics to make sure that the
// grid view is complete and accurate
RavenQueryStatistics statistics;
// try to query the items listing as quickly as we can, getting only the
// page we want out of it
var items = RavenSession
.Query<Models.Items.Item>()
.Statistics(out statistics) // output our query statistics
.Search(n => n.Name, name)
.Skip((request.Page - 1) * request.PageSize)
.Take(request.PageSize)
.ToArray();
// we need to store the total results so that we can keep the grid up to date
var totalResults = statistics.TotalResults;
return Json(new { data = items, total = totalResults }, JsonRequestBehavior.AllowGet);
The problem is that if no name is given, it does not return anything; Which is not the desired result. (Searching by 'null' doesn't work, obviously.)
Do something like this:
var q= RavenSession
.Query<Models.Items.Item>()
.Statistics(out statistics); // output our query statistics
if(string.IsNullOrEmpty(name) == false)
q = q.Search(n => n.Name, name);
var items = q.Skip((request.Page - 1) * request.PageSize)
.Take(request.PageSize)
.ToArray();

How do I iterate through a list of items in a Ext.dataview.List

I'm currently trying to find out how to set items to be selected on a list in ST2.
I've found the following:
l.select(0, true);
l.select(1, true);
which would select the first 2 items on my list. But the data coming from the server is in csv format string with the ids of the items in the list to be selected.
e.g. "4, 10, 15"
So I currently have this code at the moment.
doSetSelectedValues = function(values, scope) {
var l = scope.getComponent("mylist");
var toSet = values.split(",");
// loop through items in list
// if item in list has 'id' property matching whatever is in the toSet array then select it.
}
The problem is I can't seem to find a way of iterating over the items in the list and then inspect the "id" property of the item to see if it matches with the item in the array.
l.getItems()
Doesn't seem to return an array of items. The list is populated via a store with the "id" & "itemdesc" properties. I just want to be able to select those items from a csv string. I've scoured the Api on this and I can't seem to find a way of iterating over the items in the list and being able to inspect its backing data.
the Ext.List's items are not the items you are looking for. The items under the Ext.List object are those:
Ext.create('Ext.List', {
fullscreen: true,
itemTpl: '{title}',
store: theStore,
**items: [item1, item2]**
});
Granted, usually an Ext.List doesn't have items like these. What you are looking for are the Ext.Store items. The Ext.Store items are the exact same items in the same order as presented in the Ext.List.
To iterate over those, and select the corresponding items in the list, do the following:
var s = l.getStore();
var itemIndicesToSelect = [];
for (var i = 0 ; i < s.data.items.length ; i++){
if (arrayContainsValue(toSet, s.data.items[i].data.id)){
itemIndicesToSelect.push(i);
}
}
for (var i = 0 ; i < itemIndicesToSelect.length ; i++){
l.selectRange(itemIndicesToSelect[i], itemIndicesToSelect[i], true);
}
You would have to implement the function arrayContainsValue (one possible solution).
doSetSelectedValues = function(values, scope) {
var l = scope.getComponent("mylist"),
store = l.getStore(),
toSet = values.split(",");
Ext.each(toSet, function(id){
l.select(store.getById(id), true);
});
}