I am new to SOLR & successfully managed to made full-import from mongoDB to SOLR in windows but when i tried to perform delta-import it was not done and there is no log information also found can any suggest me to complete delta import from mongoDB to SOLR 6 in windows
My data-config.xml as follows
<entity processor="MongoEntityProcessor"
query="{Name:'Erik'}"
deltaImportQuery="{'id':{$eq:'${dih.request.id}'}}"
deltaQuery="{'UpdateDate':{$gt:'${dih.last_index_time}'}}"
collection="sample1"
datasource="MyMongo"
transformer="MongoMapperTransformer" name="sample_entity10">
<field column="_id" name="id" hashObjectId="true"/>
<field column="UpdateDate" name="UpdateDate" mongoField="UpdateDate"/>
<field column="EmpNumber" name="EmployeeNumber" mongoField="EmpNumber" stored="true"/>
</entity>
and my document in mongodb loks like
{
"UpdateDate":"2016-06-06T11:56:26.709Z",
"EmployeeNumber":854665,
"id":"575564ea2f91dd2db616972f",
"Name":"Erik",
"_version_":1536393867196104704},
}
Related
I managed to make the system send a message as a notification when some expiration date is close
BUT
I created a server action and an automatic action both in the GUI
I'd like to use xml files to create such actions
Otherwise I have to remember to create them every time I create a new db
I found something but for Odoo 15, I'm on Odoo 14
The creation of server actions and automated actions from XML are same in Odoo 14.0 or 15.0 and you can check the Odoo 14.0 Documentation
Example for server action:
<record id="act_hr_employee_holiday_request" model="ir.actions.server">
<field name="name">Time off Analysis</field>
<field name="model_id" ref="hr_holidays.model_hr_leave_report"/>
<field name="binding_model_id" ref="hr.model_hr_employee"/>
<field name="state">code</field>
<field name="groups_id" eval="[(4, ref('base.group_user'))]"/>
<field name="code">
action = model.action_time_off_analysis()
</field>
</record>
Examples for automated action.
I am migrating from SOLR 4.10.2 to 8.1.1. For some reason, in the 8.1.1 core, a pdate index named IDX_ExpirationDate is appearing as a field in the search results documents.
I have several other indexes that are defined and (correctly) do not appear in the results. But the index I am having trouble with is the only one based on a pdate.
Here is a sample 8.1.1 response that demonstrates the issue:
"response":{"numFound":58871,"start":0,"docs":[
{
"id":"11111",
"ExpirationDate":"2018-01-26T00:00:00Z",
"_version_":1641033044033798170,
"IDX_ExpirationDate":["2018-01-26T00:00:00Z"]},
{
"id":"22222",
"ExpirationDate":"2018-02-20T00:00:00Z",
"_version_":1641032965380112384,
"IDX_ExpirationDate":["2018-02-20T00:00:00Z"]},
ExpirationDate is supposed to be there, but IDX_ExpirationDate should not. I know that I can probably keep using date, but it is deprecated, and part of the reason for upgrading to 8.1.1 is to use the latest non-deprecated stuff ;-)
I have an index named IDX_ExpirationDate based on a field called ExpirationDate that was a date field in 4.10.2:
<fieldType name="date" class="solr.TrieDateField" precisionStep="0" positionIncrementGap="0"/>
<field name="IDX_ExpirationDate" type="date" indexed="true" stored="false" multiValued="true" />
<field name="ExpirationDate" type = "date" indexed = "true" stored = "true" />
<copyField source="ExpirationDate" dest="IDX_ExpirationDate"/>
In the 8.1.1 core, I have this configured as a pdate:
<fieldType name="pdate" class="solr.DatePointField" docValues="true"/>
<field name="IDX_ExpirationDate" type="pdate" indexed="true" stored="false" multiValued="true" />
<field name="ExpirationDate" type = "pdate" indexed = "true" stored = "true" />
<copyField source="ExpirationDate" dest="IDX_ExpirationDate"/>
Fixed.
According to Shawn Heisey on the solruser mailing list, the pdate type defaults to docValues=true and useDocValuesAsStored="true", which makes it appear in results.
So I changed the IDX_ExpirationDate by adding useDocValuesAsStored="false", reloaded the index, and it no longer appears in the results:
<field name="IDX_ExpirationDate" type="pdate" indexed="true" stored="false" multiValued="true" useDocValuesAsStored="false"/>
While indexing data from a local folder on my system, i am using given below configuration.However the lastmodified attribute is getting indexed in the format "Wed 23 May 09:48:08 UTC" , which is not the standard format used by solr for filter queries .
So, I am trying to format the lastmodified attribute in the data-config.xml as given below .
<dataConfig>
<dataSource name="bin" type="BinFileDataSource" />
<document>
<entity name="f" dataSource="null" rootEntity="false"
processor="FileListEntityProcessor"
baseDir="D://FileBank"
fileName=".*\.(DOC)|(PDF)|(pdf)|(doc)|(docx)|(ppt)" onError="skip"
recursive="true" transformer="DateFormatTransformer">
<field column="fileAbsolutePath" name="path" />
<field column="fileSize" name="size" />
<field column="fileLastModified" name="lastmodified" dateTimeFormat="YYYY-MM-DDTHH:MM:SS.000Z" locale="en"/>
<entity name="tika-test" dataSource="bin" processor="TikaEntityProcessor"
url="${f.fileAbsolutePath}" format="text" onError="skip">
<field column="Author" name="author" meta="true"/>
<field column="title" name="title" meta="true"/>
<!--<field column="text" />-->
</entity>
</entity>
</document>
</dataConfig>
But there is no effect of transformer, and same format is indexed again . Has anyone got success with this ? Is the above configuration right , or am i missing something ?
Your dateTimeFormat provided does not seem to be correct. The upper and lower case characters have different meaning. Also the format you showed does not match the date text you are trying to parse. So, it probably keeps it as unmatched.
Also, if you have several different date formats, you could parse them after DIH runs by creating a custom UpdateRequestProcessor chain. You can see schemaless example where there is several date formats as part of auto-mapping, but you could also do the same thing for a specific field explicitly.
I want to index my document from MySql to Solr via DIH. I have a table structure like this
Table User
id
1
2
3
name
Jay
Chakra
Rabbit
Address
id
1
2
3
number
1111111111
2222222222
3333333333
email
test#email.com
test123#test.co
unique#email.com
and other associations.
I want to index this in a nested document structure but unable to find any resource via which it can be done using DIH.
Resources refered:
http://yonik.com/solr-nested-objects/
https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers
Please suggest a way to index it through DIH
This feature has been implemented by SOLR-5147 and should be available for Solr 5.1+
Here is a sample configuration taken from the original Jira ticket.
<dataConfig>
<dataSource type="JdbcDataSource" />
<document>
<entity name="PARENT" query="select * from PARENT">
<field column="id" />
<field column="desc" />
<field column="type_s" />
<entity child="true" name="CHILD" query="select * from CHILD where parent_id='${PARENT.id}'">
<field column="id" />
<field column="desc" />
<field column="type_s" />
</entity>
</entity>
</document>
</dataConfig>
note the child="true" is required for child entities.
I'm stuck on this one issue. What i want to do is to query on a Multivalued and see if a value comes up at least try. For example the field must be "FREE","FREE" and not just "FREE" or "FREE","IN_USE".
Field
<field name="point_statusses" type="string" indexed="true" stored="true" multiValued="true" />
Type
<fieldType name="string" class="solr.StrField" sortMissingLast="true" />
SQL
GROUP_CONCAT(cp.status) as point_statusses
Clarification:
I have an object that has multiple plugs and those all have a status of FREE, IN_USE or ERROR. What i want to do is filter on ones that have two plugs with status FREE and I can't change the structure of the schema.xml. How do i query to for this?
Unfortunately, it cannot be done without applying any changes to schema, because solr.StrField does not preserve term frequency information.
Quote from schema.xml:
...
1.2: omitTermFreqAndPositions attribute introduced, true by default
except for text fields.
...
However, if you can apply some changes, then the following will work (tested on the Solr 4.5.1):
1) Make one of the following changes to schema:
Change field to text_general (or any solr.TextField field);
<field name="point_statusses" type="text_general" indexed="true" stored="true" multiValued="true" />
OR add omitTermFreqAndPositions="false" to point_statusses definition:
<field name="point_statusses" type="string" indexed="true" stored="true" multiValued="true" omitTermFreqAndPositions="false"/>
2) Filter by term frequency.
Examples:
Search documents having exactly 2 'FREE' point_statusses:
{!frange l=2 u=2}termfreq(point_statusses,'FREE')
Or from 2 to 3 'FREE' point_statusses:
{!frange l=2 u=3}termfreq(point_statusses,'FREE')
The final solr query may look like this:
http://localhost:8983/solr/stack20746538/select?q=*:*&fq={!frange l=2 u=3}termfreq(point_statusses,'FREE')