I am using Lucene search with Sitecore 7.2 and using predicate builder to search for data. I have included a computed field in the index which is a string. When I search on that field using .Contains(mystring), it fails when there is 'and' present in mystring. If there is no 'and' in the mystring it works.
Can you please suggest me anything?
Lucene by default, when the field and query is processed, will strip out what are called "stop words" such as and and the etc.
If you dont want this behaviour you can add an entry into the fieldMap section of your configuration to tell Sitecore how to process the field ...
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="YOURFIELDNAME" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>
...
</fieldNames>
.. this example tells Sitecore, for that field, to not tokenize and also to put everything into lowercase. You can change to different analyzers to get the results you want.
You can try setting the indexType to TOKENIZED but still using the LowerCaseKeywordAnalyzer as another combination. UN_TOKENIZED will mean that your string will be processed as a single token which may not be what you want.
I have solved it, taking a hint from #Stephen Pope 's reply. In order to make your computed field untokenized you have to add it to both raw:AddFieldByFieldName and AddComputedIndexField.
See link below
http://www.sitecore.net/Community/Technical-Blogs/Martina-Welander-Sitecore-Blog/Posts/2013/09/Sitecore-7-Search-Tips-Computed-Fields.aspx
Related
How do I get empty fields in SOLR indexed? I am using solr 7.2.0
I am using schemaless SOLR to try to index everything as string, but for files with empty fields, those fields do not get indexed. Is there a way to get them to show up?
col1,col2,col3
a,,1
d,e,
g,h,3
for example column 1 shows up as
{
"col1":"a",
"col3":"1",
}
I'm trying to also get col2 to show up.
in my solrconfig.xml i have this
<dynamicField name="*" type="text_general" indexed="true" stored="true" required="true" default="" />
and I have any traces of the remove-blank processor removed from my config. I've reloaded and deleted/recreated by collection multiple times. Is there a solution for this?
The CSV import module has its own option to keep empty fields - f.<field name>.keepEmpty=true.
If you don't give that option, the CSV handler will never give the empty field value to the next step in your indexing process.
Giving f.col2.keepEmpty=True as an URL argument should at least give you a better starting point.
maybe preprocess your csv file like this:
s/,,/, ,/g
That is, add an space between both commas (you will have to specially deal with the last value differntly though, there is a regex for that).
And then try again. Right now solr is reading the value as non existant, making it a space has more chances to make it through, and would not change search results (if you don't have some crazy analysis chains)
I am facing a problem, after publishing the web in sitecore then building the sitecore_web_index the items are indexed but with document filed value of null!
I am working on Sitecore 8.1 on windows 10
using luke to see whats going on
any suggestions how to solve this problem?
Check the storage type on the index for those fields. By default a lot of the fields in the Lucene indexes are set to storageType="NO" - this will index the field content but not store the data in the index, so the fields will always appear empty in the results.
Example config from Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config:
<fieldTypes hint="raw:AddFieldByFieldTypeName">
<fieldType fieldTypeName="attachment" storageType="NO" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider" />
<!-- omitted for brevity -->
</fieldTypes>
If you need to see the field contents in the results, set storageType="YES" for the require field types in the config. Note tho that this will increase the size of your index.
I am using solr 4.10. I have to change relevance of documents based on a field boost and document score. For that, I have come to know that I should use function query. Following is the syntax of boost field in schema
<field name="boost" type="float" stored="true" indexed="false" default="1.0"/>
My first question is that can function queries be used on stored fields only?
When I try using above schema, like following query
http://localhost:8983/solr/select?q=bank&df=keywords&fl=id&sort=pow(score,%20boost)%20asc
There was some error saying like
sort param could not be parsed as a query, and is not a field that exists in the index:
then I changed the schema like
<field name="boost" type="float" stored="true" indexed="true" default="1.0"/>
Then above problem was gone but a new error appeared for query
http://localhost:8983/solr/select?q=bank&df=keywords&fl=id,pow(score,%20boost)
Following error appeared
<lst name="error">
<str name="msg">undefined field: "score"</str>
<int name="code">400</int>
</lst>
Where I am wrong?
Am I correct to change attributes of boost field?
I would recommend to use a boost function and sort just by score (default = no order param needed).
bf=linear(boost,100,0)
You may use other functions. That depends on your usecase.
Just check out the solr docs for function queries.
I created a new index which uses default database crawler. I can't get it to index all the fields on 5 templates that I specified.
I am using the IndexViewer module to check for the above fields. In the available fields, it lists all the fields that I want indexed but is only indexing the following fields - _url, _group, _name, and _tags.
I also wrote some code to test against the index fields and I am getting the desired results. I just need my index to include all the fields on the specified templates. Below is my configuration for the index.
<index id="Articles" type="Sitecore.Search.Index, Sitecore.Kernel">
<param des="name">$(id)</param>
<param des="folder">__articles</param>
<Analyzer ref="search/analyzer"/>
<locations hint="list:AddCrawler">
<customindex type="Sitecore.Search.Crawlers.DatabaseCrawler, Sitecore.Kernel">
<Database>web</Database>
<Root>/sitecore/content/[websitehome]</Root>
<Tags>articles</Tags>
<IndexAllFields>true</IndexAllFields>
<include hint="list:IncludeTemplate">
<template1>{C4663677-909E-4C4D-AB3E-78AADBB36CF7}</template1>
<template2>{444D1797-1EA9-46F2-988D-2211CF926501}</template2>
<template3>{1A859C38-FFFA-4102-BF7F-9E670495C3AF}</template3>
<template4>{6EA89465-C6C4-4643-9589-188FBB180883}</template4>
<template5>{52F0AB89-E9C3-4D10-9242-ACB669841C41}</template5>
</include>
</customindex>
</locations>
Try using the Lukeall tool for observing index - IndexViewer may not show unstored fields. To use Lukeall just select the C:\inetpub\wwwroot\Sitecore\Data\indexes__articles folder, check "read-only" and "force unlock" and click OK.
To make Lucene store index value - set storageType="YES" on field definition.
<fieldMap type="Sitecore.ContentSearch.FieldMap, Sitecore.ContentSearch">
<fieldNames hint="raw:AddFieldByFieldName">
<field fieldName="_uniqueid" storageType="YES"
I figured this out. The index was including fields just not their values. Not sure if this is the desired functionality of the index but when I query against it, I get results back.
I added a multivalue field to schema.xml as follows:
<field name="fieldsharedsite" type="string" indexed="true" stored="false" multiValued="true" />
<field name="fieldsharedchannelnew" type="string" indexed="true" stored="false" multiValued="true" />
When I search for a document contents, I get the following result:
<fieldsharedsite><item key="0">33</item></fieldsharedsite>
<fieldsharedchannelnew><item key="0">52</item></fieldsharedchannelnew>
so I am sure fieldsharedchannelnew is in the results
When I do the following search:
q=fieldsharedsite:33
I do get the document
but when I do
q=fieldsharedchannelnew:52
I don't get any results.
fieldsharedsite has been here for a while and I'm trying to add fieldsharedchannelnew.
I did reindex all the content but did not help the search.
If I look at the schema browser, I have for fieldsharedsite:
Field Type: string
Properties: Indexed, Multivalued, Omit Norms, Sort Missing Last
Schema: Indexed, Multivalued, Omit Norms, Sort Missing Last
Index: (unstored field)
Index Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
Query Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
Docs: 902
and for fieldsharedchannnelnew I have:
Field Type: string
Properties: Indexed, Multivalued, Omit Norms, Sort Missing Last
Index Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
Query Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
What step did I miss in adding the fieldsharedchannelnew index? Why its not returning any results when I search for it?
The schema browser result for the field fieldsharedchannnelnew does not indicate it is populated in the documents.
the Docs information is missing, as for fieldsharedsite which shows it exists in 902 docs.
Field Type: string
Properties: Indexed, Multivalued, Omit Norms, Sort Missing Last
Index Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
Query Analyzer: org.apache.solr.schema.FieldType$DefaultAnalyzer
When I search for a document contents, I get the following result:
<fieldsharedsite><item key="0">33</item></fieldsharedsite>
<fieldsharedchannelnew><item key="0">52</item></fieldsharedchannelnew>
As the fields are not stored, the fields would not be returned with the results.
Is this the data you are feeding Solr ? how do they appear in results ?
Are you using value as is or want to use copyfield ?
You may mark the fields as stored and reindex the contents and check if they are returned with the results and the schema browser shows the Docs information.
If so, you should be able to search it as well.