How to link Sitecore Content Editor to a custom Solr search index configured for a certain Data Template? - optimization

I have created a custom Solr search index for a specific Data Template as I expect to maintain tons of items created based on that template. As you would guess I organised my items into Sitecore Item Buckets for better performance and also I could configure my Sitecore front-end to utilise that custom Solr search index. Now I would like to optimise the search across my Item Buckets in Content Editor, but it seems that Sitecore takes the master index over any custom indexes. How can I configure Sitecore to use my custom index?

Once you have created your custom Solr index you should also have created a corresponding config file as per one of the example files provided by Sitecore. For instance, for Master DB you can use Sitecore.ContentSearch.Solr.Index.Master.config.example as a config template and simply add a patch:before="*[1]" attribute to your custom index definition as follows:
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
<sitecore>
<contentSearch>
<configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
<indexes hint="list:AddIndex">
<index patch:before="*[1]" id="my_sitecore_custom_master_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider">
<param desc="name">$(id)</param>
<param desc="core">$(id)</param>
<param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
<configuration ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration" />
<strategies hint="list:AddStrategy">
<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster" />
</strategies>
<locations hint="list:AddCrawler">
<crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
<Database>master</Database>
<Root>/sitecore</Root>
</crawler>
</locations>
<enableItemLanguageFallback>false</enableItemLanguageFallback>
<enableFieldLanguageFallback>false</enableFieldLanguageFallback>
</index>
</indexes>
</configuration>
</contentSearch>
</sitecore>
</configuration>
Now Sitecore will load my_sitecore_custom_master_index index configuration as first priority and utilise it while processing any search queries for your custom items instead of the default Sitecore one.

Related

Sulu: how to query custom entity type?

I have defined custom entity type "matches" and it is back-end editable, everything works as expected.
Now I need a way to query content in that type. Can smart content field type do that? I expected that xml like this would do the trick:
<property name="matches" type="smart_content">
<!-- #see https://docs.sulu.io/en/2.2/reference/content-types/smart_content.html -->
<meta>
<title lang="en">Matches</title>
<title lang="de">Streichhölzer</title>
</meta>
<params>
<param name="provider" value="lists"/>
<param name="types" value="matchevents"/>
<param name="max_per_page" value="5"/>
<param name="page_parameter" value="m"/>
</params>
</property>
But that didn't help. Can it be done with "smart_content" field type? I yes - how? If no - what would be the best way to achieve that?
you have to create a custom DataProvider for your entity and then use your provider in the xml-definition.
So you have to create a repository which implements the DataProviderRepositoryInterface. This repository is used to query the entity.
Furthermore you also have to create the actual dataprovider service. This class should extend form the BaseDataProvider and finally you have to make the service definition with the tag sulu.smart_content.data_provider and an alias.
The alias is used in the xml-file e.g.
<property name="matches" type="smart_content">
<meta>
<title lang="en">Matches</title>
<title lang="de">Streichhölzer</title>
</meta>
<params>
<param name="provider" value="myCustomDataProviderAlias"/>
</params>
</property>
Check out the documentation for a step by step tutorial on how to create a custom data provider. https://docs.sulu.io/en/2.2/cookbook/smart-content-data-provider.html#how-to-create-a-custom-dataprovider

Infinispan configuration

I am bit confused reading Infinispan guide.
I want to have two clustered caches, propably must have separate jgroups files to have different multicast addresses, but should there be only one cache containter?
<?xml version="1.0" encoding="UTF-8"?>
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:8.0 http://www.infinispan.org/schemas/infinispan-config-8.0.xsd"
xmlns="urn:infinispan:config:8.0">
<jgroups>
<stack-file name="file1" path="jgroups1.xml" />
<stack-file name="file2" path="jgroups2.xml" />
</jgroups>
<cache-container default-cache="cache1">
<transport stack="file1" node-name="${nodeName}" />
<invalidation-cache name="cache1" configuration="invalidation-template" />
<invalidation-cache name="cache2" configuration="invalidation-template" />
<invalidation-cache-configuration name="invalidation-template" mode="SYNC" >
<locking isolation="READ_COMMITTED" striping="true"/>
<transaction locking="OPTIMISTIC"/>
<eviction max-entries="20500" strategy="LRU"/>
<expiration interval="10500" />
</invalidation-cache-configuration>
</cache-container>
</infinispan>
You don't need to have a separate jgroups file for each clustered cache. Simply configure the transport element of the cache-container, and then you can define any number of caches in that cache-container.
Different multicast addresses are only needed if you want cache containers to form different clusters. I think the confusing thing here is that you can define multiple stack-file elements in JGroups but you can only really specify a single cache-container element. The XSD is not precise enough, but the parser within the code assumes a single global configuration builder instance, hence a single cache container. So, if you want to create two separate cache containers, these should be currently defined in separate XML files.

Sitecore web indexing issue for more version items in master database?

When a new version of an item is created in master database, if we edit the item and save and publish it.The item gets published and web database contains the latest updated version item. But while browsing item is not visible on the website
some $name is visible for the item.
Sometimes the item is not visible and sometimes it is visible with $name.
Also when browsing the item directly through url item is visible with latest updated content,so the item is published.Its seems some problem related with indexing.
But when an item is directly edited without the creating a new version,then the issue doesn't exist.So the issue is with the indexing as well as versions as I guess there will be more than one latest versions for the web index which is creating the problem.
How to fix this issue? As a workaround I have to delete the item from web database and republish and rebuild the index again to solve the issue.
Is there a need to customize the existing indexing and crawler strategy for more than one versions of items?If so which files needs to customize or override and change?"
Here is the code snippet for web indexing for the data items to be displayed?
<!-- sitecore_web_content_mag_index -->
<indexes hint="list:AddIndex">
<index id="sitecore_web_content_mag_index" type="Sitecore.ContentSearch.LuceneProvider.SwitchOnRebuildLuceneIndex, Sitecore.ContentSearch.LuceneProvider">
<param desc="name">$(id)</param>
<param desc="folder">$(id)</param>
<param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
<configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration" />
<strategies hint="list:AddStrategy">
<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
</strategies>
<commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
<policies hint="list:AddcommitPolicy">
<policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
</policies>
</commitPolicyExecutor>
<locations hint="list:AddCrawler">
<crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
<Database>web</Database>
<Root>/sitecore/content/Site/Home</Root>
</crawler>
</locations>
</index>
</indexes>
When using the ContentSearch API, you can filter the results to return only the latest version by doing something like this:
searchContext.GetQueryable<SearchResultItem>()
.Where(result => result["_latestversion"].Equals("1");
Or, if you have created your own search result model, you can add a property to it to make the query syntax more consise.
Search Result Model:
public class CustomSearchResultItem : SearchResultItem
{
[IndexField("_latestversion")]
public bool IsLatestVersion { get; set; }
// other properties
}
Query
searchContext.GetQueryable<CustomSearchResultItem>()
.Where(result => result.IsLatestVersion);
Another approach is to use extension methods as described in this post: http://laubplusco.net/generic-extension-methods-sitecore-contentsearch/

Upgrading sitecore 6.6 index configuration to sitecore 7 (using ComputedFields)

Sitecore CMS+DMS 6.6.0 rev.130404 => 7.0 rev.130424
In our project we have been using AdvancedDatabaseCrawler (ADC) for our indexes (specially because of it's dynamic fields feature). Here's a sample index configuration:
<index id="GeoIndex" type="Sitecore.Search.Index, Sitecore.Kernel">
<param desc="name">$(id)</param>
<param desc="folder">$(id)</param>
<analyzer ref="search/analyzer" />
<locations hint="list:AddCrawler">
<web type="scSearchContrib.Crawler.Crawlers.AdvancedDatabaseCrawler, scSearchContrib.Crawler">
<database>web</database>
<root>/sitecore/content/Globals/Locations</root>
<IndexAllFields>true</IndexAllFields>
<include hint="list:IncludeTemplate">
<!--Suburb Template-->
<suburb>{FF0D64AA-DCB4-467A-A310-FF905F9393C0}</suburb>
</include>
<dynamicFields hint="raw:AddDynamicFields">
<dynamicField type="OurApp.CustomSearchFields.SearchTextField,OurApp" name="search text" storageType="NO" indexType="TOKENIZED" vectorType="NO" />
<dynamicField type="OurApp.CustomSearchFields.LongNameField,OurApp" name="display name" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO" />
</dynamicFields>
</web>
</locations>
</index>
As you can see, we use scSearchContrib.Crawler.Crawlers.AdvancedDatabaseCrawler as the crawler and it uses the fields defined inside <dynamicFields hint="raw:AddDynamicFields"> section to inject custom fields into the index.
Now we are upgrading our project to sitecore 7. In Sitecore 7, they have ported the DynamicFields functionality from ADC into sitecore. I found out some articles on this and converted our custom search field classes to implement sitecore 7 IComputedIndexField interface instead of inheriting from BaseDynamicField class in ADC. Now my problem is how to change the index configuration to match with new sitecore 7 APIs. There were bits and pieces on the web but couldn't find all the examples I needed to convert my configuration. Can anybody help me on this?
While I'm doing this I'm under the impression that we won't have to rebuild our indexes since it still uses Lucene internally. I don't want to change the index structure. Just want to upgrade the code and configuration from AdvancedDatabaseCrawler to Sitecore 7. Should I be worried about breaking our existing indexes? Please shed some light on this as well.
Thanks
A few quick clarifications :)
We have not merged ADC into Sitecore 7, the ContentSearch layer is a complete rewrite of the old search layer for Sitecore. We have taken some of the core concepts from ADC, such as dynamic fields, and put them in the new implementation (as ComputedFields). They are not 1:1 compatible and you will have to do some work on your indexes.
The version of Lucene has also been upgraded from 2.* to 3.0.3 so all indexes will need to be re-created anyway as they are a very different version of Lucene.
There are two options here, the old Lucene search (Sitecore.Search namespace) (which ADC was built upon) has not been touched and will still work in the same way, although I am not sure about ADC compatibility with SItecore 7 as in theory this has now been superseded.
The next option is to update your index to take advantage of the new search features of Sitecore 7. The configuration you have will not be directly compatible but, while you will need to rework your index into the new configuration, the basic concepts should be familiar to you. The dynamic fields you have should map logically to ComputedFields (fields that are calculated when an item is indexed) and everything else is straightforward.
While it looks like a lot of extra config for ContentSearch a lot of it is default config that you will not need to touch, you will just need to override the configuration parts for the computed fields you want to add and the template you want to include.
An example of creating your own configuration override can be found here : http://www.mikkelhm.dk/post/2013/10/12/Defining-a-custom-index-in-Sitecore-7-and-utilizing-it.aspx
I would also recommend making sure you upgrade to 7.0 rev. 131127 (7.0 Update-3) as this fixes a bug in the IncludeTemplates logic in the version you currently have.
I managed to convert the index configuration for sitecore ContentSearch API. Looking at Sitecore default index configurations was a great help for this.
Note: As also mentioned by Stephen, <include hint="list:IncludeTemplate"> does not work in Sitecore 7.0 initial release. It's fixed in Sitecore 7.0 rev. 131127 (7.0 Update-3) and I'm planning to upgrade to it.
Here's a good article on sitecore 7 index update strategies by John West. It'll help you in configuration your indexes the way you want.
Converted configuration:
<sitecore>
<contentSearch>
<configuration type="Sitecore.ContentSearch.LuceneProvider.LuceneSearchConfiguration, Sitecore.ContentSearch.LuceneProvider">
<DefaultIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">
<IndexAllFields>true</IndexAllFields>
<include hint="list:IncludeTemplate">
<!--Suburb Template-->
<suburb>{FF0D64AA-DCB4-467A-A310-FF905F9393C0}</suburb>
</include>
<fields hint="raw:AddComputedIndexField">
<field fieldName="search text" storageType="NO" indexType="TOKENIZED" vectorType="NO">OurApp.CustomSearchFields.SearchTextField,OurApp</field>
<field fieldName="display name" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO">OurApp.CustomSearchFields.LongNameField,OurApp</field>
</fields>
</DefaultIndexConfiguration>
<indexes hint="list:AddIndex">
<index id="GeoIndex" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
<param desc="name">$(id)</param>
<param desc="folder">$(id)</param>
<!-- This initializes index property store. Id has to be set to the index id -->
<param desc="propertyStore" ref="contentSearch/databasePropertyStore" param1="$(id)" />
<strategies hint="list:AddStrategy">
<!-- NOTE: order of these is controls the execution order -->
<strategy ref="contentSearch/indexUpdateStrategies/onPublishEndAsync" />
</strategies>
<commitPolicy hint="raw:SetCommitPolicy">
<policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
</commitPolicy>
<commitPolicyExecutor hint="raw:SetCommitPolicyExecutor">
<policyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch" />
</commitPolicyExecutor>
<locations hint="list:AddCrawler">
<crawler type="Sitecore.ContentSearch.LuceneProvider.Crawlers.DefaultCrawler, Sitecore.ContentSearch.LuceneProvider">
<Database>web</Database>
<Root>/sitecore/content/Globals/Countries</Root>
</crawler>
</locations>
</index>
</indexes>
</configuration>
</contentSearch>
</sitecore>

Sitecore Lucene Index not including all fields

I created a new index which uses default database crawler. I can't get it to index all the fields on 5 templates that I specified.
I am using the IndexViewer module to check for the above fields. In the available fields, it lists all the fields that I want indexed but is only indexing the following fields - _url, _group, _name, and _tags.
I also wrote some code to test against the index fields and I am getting the desired results. I just need my index to include all the fields on the specified templates. Below is my configuration for the index.
<index id="Articles" type="Sitecore.Search.Index, Sitecore.Kernel">
<param des="name">$(id)</param>
<param des="folder">__articles</param>
<Analyzer ref="search/analyzer"/>
<locations hint="list:AddCrawler">
<customindex type="Sitecore.Search.Crawlers.DatabaseCrawler, Sitecore.Kernel">
<Database>web</Database>
<Root>/sitecore/content/[websitehome]</Root>
<Tags>articles</Tags>
<IndexAllFields>true</IndexAllFields>
<include hint="list:IncludeTemplate">
<template1>{C4663677-909E-4C4D-AB3E-78AADBB36CF7}</template1>
<template2>{444D1797-1EA9-46F2-988D-2211CF926501}</template2>
<template3>{1A859C38-FFFA-4102-BF7F-9E670495C3AF}</template3>
<template4>{6EA89465-C6C4-4643-9589-188FBB180883}</template4>
<template5>{52F0AB89-E9C3-4D10-9242-ACB669841C41}</template5>
</include>
</customindex>
</locations>
Try using the Lukeall tool for observing index - IndexViewer may not show unstored fields. To use Lukeall just select the C:\inetpub\wwwroot\Sitecore\Data\indexes__articles folder, check "read-only" and "force unlock" and click OK.
To make Lucene store index value - set storageType="YES" on field definition.
<fieldMap type="Sitecore.ContentSearch.FieldMap, Sitecore.ContentSearch">
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="_uniqueid" storageType="YES"
I figured this out. The index was including fields just not their values. Not sure if this is the desired functionality of the index but when I query against it, I get results back.