Sitecore 7 Index treelist lucene - lucene

I have Sitecore items with a treelist property referring to other items (with different a template).
My goal is to find item A that contains item B in the treelist property using the ContentSearch api (lucene).
I've added the treelist property to my index:
<indexConfigurations>
<defaultLuceneIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">
<fieldMap type="Sitecore.ContentSearch.FieldMap, Sitecore.ContentSearch">
<fieldNames hint="raw:AddFieldByFieldName">
<field patch:before="field[0]" fieldName="TreelistProperty" storageType="YES" indexType="UNTOKENIZED" vectorType="NO" boost="1f" type="System.String"
settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>
</fieldNames>
</fieldMap>
</defaultLuceneIndexConfiguration>
</indexConfigurations>
I would expect that lucene stores the treelist property as a concatenation of guids.
Assuming this is correct and my index is populated my query looks like this:
master = Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");
using (var context = master.CreateSearchContext())
{
var results = context.GetQueryable<SearchResultItem>()
.Where(x => x["TreelistProperty"].Contains("{456-41414-my-guid-here-1516}"))
.GetResults();
var hits = results.Hits.ToArray();
}
This returns nothing. Where did it go wrong?

You should normalize your guid, like this:
var master = Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");
using (var context = master.CreateSearchContext())
{
Sitecore.Data.ID myId = ID.Parse("{456-41414-my-guid-here-1516}");
string normalizedID = Sitecore.ContentSearch.Utilities.IdHelper.NormalizeGuid(myId );
var results = context.GetQueryable<SearchResultItem>()
.Where(x => x["TreelistProperty"].Contains(normalizedID))
.GetResults();
var hits = results.Hits.ToArray();
}

I think there is a typo in your index configuration, can you try
indexType="UN_TOKENIZED"
You can also investigate what values are in your lucene indexes using luke
http://www.sitecore.net/en-gb/learn/blogs/technical-blogs/getting-to-know-sitecore/posts/2013/06/using-luke-to-understand-sitecore-7-search.aspx

I think the guid values are stored without the braces and dashes by default. Try converting to ToShortId() before the comparison.

Related

Dynamics CRM- update record of contact entity

I have created contact record using ASP.NET. Now I need to Check if the contact record exists. If exists, update the same record. Through advance find have downloaded FetchXML and added to my FetchXML variable. Please suggest the logic. Below is my code.
// Establish a connection to crm and get the connection proxy
string connectionString = "xyz; Username= xyz ;Password=xyz";
CrmConnection connect = CrmConnection.Parse(connectionString);
OrganizationService service;
using (service = new OrganizationService(connect))
{
WhoAmIRequest request = new WhoAmIRequest();
Guid userId = ((WhoAmIResponse)service.Execute(request)).UserId;
ContactDetails contact = new ContactDetails();
//Check if the contact record exists . If exists , update the same record.
//Fecthxml query
string fetchXml = #" <fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
<entity name='contact'>
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute='fullname' descending='false' />
<filter type='and'>
<condition attribute= 'mobilephone' operator='not-null' />
</filter>
</entity>
</fetch>" ;
FetchExpression query = new FetchExpression(fetchXml);
EntityCollection results = service.RetrieveMultiple(query);
if (results.Entities.Count > 0)
{
Entity contactRecord = results[0];
contactRecord["firstname"] = contactInfo.FirstName;
contactRecord["lastname"] = contactInfo.LastName;
contactRecord["emailaddress1"] = contactInfo.EmailId;
contactRecord["mobilephone"] = contactInfo.MobilePhone;
contactRecord["address1_line1"] = contactInfo.Street1;
contactRecord["address1_line2"] = contactInfo.Street2;
contactRecord["address1_line3"] = contactInfo.Street3;
contactRecord["address1_city"] = contactInfo.City;
service.Update(contactRecord);
}
//Else, Create the contact record
else
{
Entity entity = new Entity();
entity.LogicalName = "contact";
entity["firstname"] = contactInfo.FirstName;
entity["lastname"] = contactInfo.LastName;
entity["emailaddress1"] = contactInfo.EmailId;
entity["mobilephone"] = contactInfo.MobilePhone;
entity["address1_line1"] = contactInfo.Street1;
entity["address1_line2"] = contactInfo.Street2;
entity["address1_line3"] = contactInfo.Street3;
entity["address1_city"] = contactInfo.City;
entity["address1_stateorprovince"] = contactInfo.State;
entity["address1_country"] = contactInfo.Country;
entity["spousesname"] = contactInfo.SpouseName;
entity["birthdate"] = contactInfo.Birthday;
entity["anniversary"] = contactInfo.Anniversary;
//Create entity gender with option value
if (contactInfo.Gender == "Male")
{
entity["gendercode"] = new OptionSetValue(1);
}
else
{
entity["gendercode"] = new OptionSetValue(2);
}
//entity["familystatuscode"] = contactInfo.MaritalStatus;
if (contactInfo.MaritalStatus == "Single")
{
entity["familystatuscode"] = new OptionSetValue(1);
}
else
{
entity["familystatuscode"] = new OptionSetValue(2);
}
service.Create(entity);
}
}
// Create the entity
your logic seems ok, with the exception of the FectchXML query. The way you have your code, you will always end up updating the first record retrieved that has its mobilephone field filled. That does not seem a good way to check if a contact already exists.
I suggest you to change the filter of your fetch. In your filter condition you have to use an attribute that represents uniqueness for all contacts.
Apart of this your code looks ok.
Like nunoalmeieda says, you need to have a better way to ascertain if the Contact already exists. A common way of identifying if the Contact already exists would be to check if the email address already exists as it is very unlikely that two people will have the same email address.
I have updated your basic code to show how it is done with FetchXML.
<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
<entity name="contact">
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute="fullname" descending="false" />
<filter type="and">
<condition attribute="emailaddress1" operator="eq" value=contactInfo.EmailId />
</filter>
</entity>
</fetch>
The logic here is that I am checking if the value of emailaddress1 (field in the contact entity of CRM) is equal to the value of your contactInfo.EmailId. I am assuming that contactInfo is the record that you get from ASP.NET.
The rest of your code is fine (I have formatted it a bit to make the question more readable).

Sorting on date field with Sitecore 7 ContentSearch

I'm trying to add a field sort to a date field in a ContentSearch query. I'm able to filter on the index field properly so I'm assuming the field is getting populated with values properly, however, results are not being sorted properly. Any thoughts? Here's the code I'm using to do query:
public static IEnumerable<Episode> GetPastEpisodes(Show show, bool includeMostRecent = false, int count = 0)
{
IEnumerable<Episode> pastEpisodes;
using (var context = _index.CreateSearchContext())
{
// querying against lucene index
pastEpisodes = context.GetQueryable<Episode>().Where(GetPastAirDatePredicate(show));
if(!includeMostRecent)
{
pastEpisodes = pastEpisodes.Where(item => item.Id != GetMostRecentEpisode(show).Id);
}
pastEpisodes = pastEpisodes.OrderByDescending(ep => ep.Latest_Air_Date);
if (count > 0)
{
pastEpisodes = pastEpisodes.Take(count);
}
pastEpisodes = pastEpisodes.ToList();
// map the lucene documents to Sitecore items using the database
foreach (var episode in pastEpisodes)
{
_database.Map(episode);
}
}
return pastEpisodes;
}
private static Expression<Func<Episode,bool>> GetPastAirDatePredicate(Show show)
{
var templatePredicate = PredicateBuilder.Create<Episode>(item => item.TemplateId == IEpisodeConstants.TemplateId);
var showPathPredicate = PredicateBuilder.Create<Episode>(item => item.FullPath.StartsWith(show.FullPath));
var airDatePredicate = PredicateBuilder.Create<Episode>(item => item.Latest_Air_Date < DateTime.Now.Date.AddDays(1));
var fullPredicate = PredicateBuilder.Create<Episode>(templatePredicate).And(showPathPredicate).And(airDatePredicate);
return fullPredicate;
}
The field is stored and untokenized and using the LowerCaseKeywordAnalyzer.
<field fieldName="latest_air_date" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO" boost="1f" type="System.DateTime" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider"/>
</field>
Episode class has IndexField attribute set:
[IndexField("latest_air_date")]
public virtual DateTime Latest_Air_Date {get; set; }
Kyle,
As far as I can tell everything looks correct with your configuration and code. I mocked out something very similar in a vanilla Sitecore 7.2 instance and Dates sorted without issue.
One thing to note though, and this might be what is giving you some issues, is that Sitecore's FieldReader for DateTime only store the date portion of the DateTime. If you are expecting to be able to sort by true DateTime you will need to add a custom FieldReader or some computed fields.
See this blog that discusses the issue and explains the process to swap out a custom field reader: http://reasoncodeexample.com/2014/01/30/indexing-datetime-fields-sitecore-7-content-search/
I would also suggest looking at the index with Luke to verify what data is actually in the index. https://code.google.com/p/luke/
And finally you may want to enable Search debugging in Sitecore to see exactly how Sitecore is executing the query in Lucene.
Sitecore.ContentSearch.config:
<setting name="ContentSearch.EnableSearchDebug" value="true" />

split multi valued dynamic fields in Solr Data import handler

Is it possible to split multivalued dynamic fields?
Schema:
<dynamicField name="*_s" type="string" indexed="true" stored="true" multiValued="true"/>
DIH-config:
<field column="*_s" splitBy="\|" />
It doesn't seem to work. Any help is appriciated!
UPDATE inspired on the comment
<dataConfig>
<dataSource driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://${dataimporter.request.sqlserver};databaseName=${dataimporter.request.sqlcatelog};responseBuffering=adaptive;"
user="${dataimporter.request.sqluser}"
password="${dataimporter.request.sqlpassword}"
readOnly="true" batchSize="500"/>
<script><![CDATA[
function SplitDynamicColumn(row, context) {
var fields = context.getAllEntityFields();
// find dynamic columns with 'splitBy' rule
for (var f = 0; f < fields.size(); f++) {
var field = fields.get(f);
var columnMask = field.get('column');
if (columnMask.contains('*') && field.containsKey('splitBy')) {
var columnNameRegex = columnMask.replace('*', '\\w+');
var columns = row.keySet().toArray();
// find columns that match mask
for (var c = 0; c < columns.length; c++) {
var columnName = columns[c];
if (columnName.matches(columnNameRegex)) {
// split column value
var value = row.get(columnName);
if (value !== null) {
var arr = new java.util.ArrayList();
var sp = value.split(field.get('splitBy'));
for (var i = 0; i < sp.length; i++) {
arr.add(sp[i]);
}
row.put(columnName, arr);
}
}
}
}
}
return row;
}
]]></script>
<document name="pages">
<entity name="pages" transformer="RegexTransformer,script:SplitDynamicColumn" query="EXEC A_STORED_PROCEDURE">
<field column="*_s" splitBy="\|" />
</entity>
</document>
</dataConfig>
splitBy is a flag for the RegexTransformer. Make sure you have the transformers sets on that entity properly.
However, more importantly, I do not believe DIH will support the wildcard the way you define it. DIH does support wildcard mapping by trying to match field name to schema name, but that does not allow you to define any transformations.
You may just not be able to combine those two features. One way to get around it is by writing your own custom transformer that does not require an definition with the attribute to run.

CAML Query using "Membership Type='CurrentUserGroups'" returns no results

I have the following CAML query to programmatically filter a list:
<Query>
<Where>
<And>
<Geq><FieldRef Name='notificationExpires' /><Value IncludeTimeValue='TRUE' Type='DateTime'>2011-03-30T00:00:00Z</Value></Geq>
<Or>
<Membership Type='CurrentUserGroups'><FieldRef Name='notificationTargetRoles'/></Membership>
<Contains><FieldRef Name='notificationTargetRoles'/><Value Type='User'>MyDomain\administrator</Value></Contains>
</Or>
</And>
</Where>
</Query>
When I execute this exact same query (running under the same account) in the U2U CAML Query Builder I get filtered users, groups and dates.
When apply exact the same filter in code, the Groups do not appear in the results.
What can be the reason that the group filter (CurrentGroups) does not return any results?
The code for applying the filter is:
SPListItemCollection items = null;
SPSecurity.RunWithElevatedPrivileges(delegate()
{
using (SPSite elevatedSite = new SPSite(theSiteName))
{
using (SPWeb elevatedWeb = elevatedSite.OpenWeb())
{
SPList alertList = elevatedWeb.Lists[theListName];
SPQuery query = new SPQuery();
query.Query = "<Where><And><Geq><FieldRef Name='notificationExpires' /><Value IncludeTimeValue='TRUE' Type='DateTime'>2011-03-30T00:00:00Z</Value></Geq> <Or><Membership Type='CurrentUserGroups'><FieldRef Name='notificationTargetRoles'/></Membership><Contains><FieldRef Name='notificationTargetRoles'/><Value Type='User'>BSFLMK\administrator</Value></Contains></Or></And></Where>";
items = alertList.GetItems(query);
}
}
});
It turns out that removing the "SPSecurity.RunWithElevatedPrivileges(delegate()" part corrected the problem

SQL IN equivalent in CAML

Is there a "nice" way to create a CAML query for SharePoint that does something like this?
SELECT *
FROM table
WHERE Id IN (3, 12, ...)
Or am I stuck with a nightmare of nested <Or> nodes?
EDIT: This was my solution to generate the <Or> nodes.
/// Simulates a SQL 'Where In' clause in CAML
/// </summary>
/// <param name="columnType">Specifies the data type for the value contained by the field.</param>
/// <returns>Nested 'Or' elements portion of CAML query</returns>
public static string CamlIn<T>(string internalFieldName, string columnType, T[] values)
{
XDocument doc = new XDocument();
XElement prev = null;
int index = 0;
while (index < values.Length)
{
XElement element =
new XElement("Or",
new XElement("Eq",
new XElement("FieldRef",
new XAttribute("Name", internalFieldName)),
new XElement("Value",
new XAttribute("Type", columnType),
values[index++].ToString())));
if (index == values.Length - 1)
{
element.AddFirst(
new XElement("Eq",
new XElement("FieldRef",
new XAttribute("Name", internalFieldName)),
new XElement("Value",
new XAttribute("Type", columnType),
values[index++].ToString())));
}
if (prev != null)
prev.AddFirst(element);
else
doc.Add(element);
prev = element;
}
return doc.ToString(SaveOptions.DisableFormatting);
}
Usage:
int[] ids = new int[] { 1, 2, 4, 5 };
string query = string.Format("<Where>{0}</Where>", CamlIn("SomeColumn", "Number", ids));
Output:
<Where>
<Or>
<Or>
<Or>
<Eq>
<FieldRef Name=\"SomeColumn\" />
<Value Type=\"Number\">5</Value>
</Eq>
<Eq>
<FieldRef Name=\"SomeColumn\" />
<Value Type=\"Number\">4</Value>
</Eq>
</Or>
<Eq>
<FieldRef Name=\"SomeColumn\" />
<Value Type=\"Number\">2</Value>
</Eq>
</Or>
<Eq>
<FieldRef Name=\"SomeColumn\" />
<Value Type=\"Number\">1</Value>
</Eq>
</Or>
</Where>
Also made this overload for working with Lookup Fields a bit easier
/// <summary>
/// Simulates a SQL 'Where In' clause in CAML
/// </summary>
/// <param name="lookupId">Specify whether to use the Lookup column's Id or Value.</param>
/// <returns>Nested 'Or' elements portion of CAML query</returns>
public static string CamlIn<T>(string internalFieldName, bool lookupId, T[] values)
{
XDocument doc = new XDocument();
XElement prev = null;
int index = 0;
while (index < values.Length)
{
XElement element =
new XElement("Or",
new XElement("Eq",
new XElement("FieldRef",
new XAttribute("Name", internalFieldName),
lookupId ? new XAttribute("LookupId", "TRUE") : null),
new XElement("Value",
new XAttribute("Type", "Lookup"),
values[index++].ToString())));
if (index == values.Length - 1)
{
element.AddFirst(
new XElement("Eq",
new XElement("FieldRef",
new XAttribute("Name", internalFieldName),
lookupId ? new XAttribute("LookupId", "TRUE") : null),
new XElement("Value",
new XAttribute("Type", "Lookup"),
values[index++].ToString())));
}
if (prev != null)
prev.AddFirst(element);
else
doc.Add(element);
prev = element;
}
if (values.Length == 1)
{
XElement newRoot = doc.Descendants("Eq").Single();
doc.RemoveNodes();
doc.Add(newRoot);
}
return doc.ToString(SaveOptions.DisableFormatting);
}
For those using Sharepoint 2010, there is an IN element available:
http://msdn.microsoft.com/en-us/library/ff625761.aspx
Here's a working example:
SPQuery locationsQuery = new SPQuery();
locationsQuery.Query = string.Concat("<Where>",
"<In>",
"<FieldRef Name='ID' />",
"<Values>",
"<Value Type='Number'>6</Value>",
"<Value Type='Number'>7</Value>",
"<Value Type='Number'>8</Value>",
"</Values>",
"</In>",
"</Where>");
NO, you'll need to deal with nested OR tags; these are supported query instructions on CAML
Maybe CAML.NET can help you in your quest.
FullTextSqlQuery
It is possible to search MOSS using SQL statements, using the FullTextSqlQuery class. I have no experience of using this class personally. These articles may be of use:
http://blogit.create.pt/blogs/ricardocosta/archive/2007/06/15/How-to-use-FullTextSqlQuery-to-search-in-WSS.aspx
http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/03/06/how-to-use-the-moss-enterprise-search-fulltextsqlquery-class.aspx
YACAMLQT
Alternatively, there is also YACAMLQT (Yet Another CAML Query Tool) which allows you to create SharePoint CAML queries using a T-SQL syntax.
LINQ to SharePoint
If you are up to speed with LINQ, then the LINQ to SharePoint project provides a tool to query SharePoint lists using the LINQ syntax. Please note, this tool is still in the alpha testing phase, so it may not be production ready.
U2U CAML Query Builder
If you are working with CAML queries, I would recommend using the U2U CAML Query Builder for SharePoint (2003 and 2007) tool to build up your CAML queries. The tool allows you to build up your query string, and to execute it against the target list, using a point-and-click interface, as shown below.
(source: u2u.net)
Of the above four methods, I can recommend the U2U CAML Query Builder, having used it almost daily over the last 6 months. It also appears to be the most widely used CAML tool in the SharePoint community .
Note, if you are building the CAML queries in code, then I recommend that you take a look at the CAML.NET project on CodePlex, which provides "a set of .NET language-based tools for creating dynamic, reusable CAML query components".
I faced a similar thing and ultimately had to create a recursive algorithm to generate the nested OR structure. Here is my algorithm
var DynamicQuery = '<Query><Where>{{DYNAMICSTRING}}</Where></Query>';
var OneOR = '<Or><Eq><FieldRef Name="IMEI" /><Value Type="Text">{{SearchValue}}</Value></Eq>{{DYNAMICSTRING}}</Or>';
var TwoOr = '<Or><Eq><FieldRef Name="IMEI" /><Value Type="Text">{{SearchValue}}</Value></Eq><Eq><FieldRef Name="IMEI" /><Value Type="Text">{{SearchValue}}</Value></Eq></Or>';
var OnlyEq = '<Eq><FieldRef Name="IMEI" /><Value Type="Text">{{SearchValue}}</Value></Eq>';
function generateAdvancedInQuery(x){
if(x.length == 1)
return OnlyEq.replace('{{SearchValue}}',x[0]);
else if(x.length == 2)
return TwoOr.replace('{{SearchValue}}',x[0]).replace('{{SearchValue}}',x[1]);
else
return OneOR.replace('{{SearchValue}}',x[x.length-1]).replace('{{DYNAMICSTRING}}',generateAdvancedInQuery(x.splice(0,x.length-1) ) );
}
x = ['438753234098792','438753234098793','438753234098794','438753234098795','438753234098796','438753234098797','438753234098798'];
var Caml = DynamicQuery.replace('{{DYNAMICSTRING}}',generateAdvancedInQuery(x) )
This generates the XML caml query as
<Query>
<Where>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098798</Value>
</Eq>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098797</Value>
</Eq>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098796</Value>
</Eq>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098795</Value>
</Eq>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098794</Value>
</Eq>
<Or>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098792</Value>
</Eq>
<Eq>
<FieldRef Name="IMEI" />
<Value Type="Text">438753234098793</Value>
</Eq>
</Or>
</Or>
</Or>
</Or>
</Or>
</Or>
</Where>
</Query>