split multi valued dynamic fields in Solr Data import handler - dynamic

Is it possible to split multivalued dynamic fields?
Schema:
<dynamicField name="*_s" type="string" indexed="true" stored="true" multiValued="true"/>
DIH-config:
<field column="*_s" splitBy="\|" />
It doesn't seem to work. Any help is appriciated!
UPDATE inspired on the comment
<dataConfig>
<dataSource driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://${dataimporter.request.sqlserver};databaseName=${dataimporter.request.sqlcatelog};responseBuffering=adaptive;"
user="${dataimporter.request.sqluser}"
password="${dataimporter.request.sqlpassword}"
readOnly="true" batchSize="500"/>
<script><![CDATA[
function SplitDynamicColumn(row, context) {
var fields = context.getAllEntityFields();
// find dynamic columns with 'splitBy' rule
for (var f = 0; f < fields.size(); f++) {
var field = fields.get(f);
var columnMask = field.get('column');
if (columnMask.contains('*') && field.containsKey('splitBy')) {
var columnNameRegex = columnMask.replace('*', '\\w+');
var columns = row.keySet().toArray();
// find columns that match mask
for (var c = 0; c < columns.length; c++) {
var columnName = columns[c];
if (columnName.matches(columnNameRegex)) {
// split column value
var value = row.get(columnName);
if (value !== null) {
var arr = new java.util.ArrayList();
var sp = value.split(field.get('splitBy'));
for (var i = 0; i < sp.length; i++) {
arr.add(sp[i]);
}
row.put(columnName, arr);
}
}
}
}
}
return row;
}
]]></script>
<document name="pages">
<entity name="pages" transformer="RegexTransformer,script:SplitDynamicColumn" query="EXEC A_STORED_PROCEDURE">
<field column="*_s" splitBy="\|" />
</entity>
</document>
</dataConfig>

splitBy is a flag for the RegexTransformer. Make sure you have the transformers sets on that entity properly.
However, more importantly, I do not believe DIH will support the wildcard the way you define it. DIH does support wildcard mapping by trying to match field name to schema name, but that does not allow you to define any transformations.
You may just not be able to combine those two features. One way to get around it is by writing your own custom transformer that does not require an definition with the attribute to run.

Related

Sitecore 7 Index treelist lucene

I have Sitecore items with a treelist property referring to other items (with different a template).
My goal is to find item A that contains item B in the treelist property using the ContentSearch api (lucene).
I've added the treelist property to my index:
<indexConfigurations>
<defaultLuceneIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">
<fieldMap type="Sitecore.ContentSearch.FieldMap, Sitecore.ContentSearch">
<fieldNames hint="raw:AddFieldByFieldName">
<field patch:before="field[0]" fieldName="TreelistProperty" storageType="YES" indexType="UNTOKENIZED" vectorType="NO" boost="1f" type="System.String"
settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>
</fieldNames>
</fieldMap>
</defaultLuceneIndexConfiguration>
</indexConfigurations>
I would expect that lucene stores the treelist property as a concatenation of guids.
Assuming this is correct and my index is populated my query looks like this:
master = Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");
using (var context = master.CreateSearchContext())
{
var results = context.GetQueryable<SearchResultItem>()
.Where(x => x["TreelistProperty"].Contains("{456-41414-my-guid-here-1516}"))
.GetResults();
var hits = results.Hits.ToArray();
}
This returns nothing. Where did it go wrong?
You should normalize your guid, like this:
var master = Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");
using (var context = master.CreateSearchContext())
{
Sitecore.Data.ID myId = ID.Parse("{456-41414-my-guid-here-1516}");
string normalizedID = Sitecore.ContentSearch.Utilities.IdHelper.NormalizeGuid(myId );
var results = context.GetQueryable<SearchResultItem>()
.Where(x => x["TreelistProperty"].Contains(normalizedID))
.GetResults();
var hits = results.Hits.ToArray();
}
I think there is a typo in your index configuration, can you try
indexType="UN_TOKENIZED"
You can also investigate what values are in your lucene indexes using luke
http://www.sitecore.net/en-gb/learn/blogs/technical-blogs/getting-to-know-sitecore/posts/2013/06/using-luke-to-understand-sitecore-7-search.aspx
I think the guid values are stored without the braces and dashes by default. Try converting to ToShortId() before the comparison.

Dynamics CRM- update record of contact entity

I have created contact record using ASP.NET. Now I need to Check if the contact record exists. If exists, update the same record. Through advance find have downloaded FetchXML and added to my FetchXML variable. Please suggest the logic. Below is my code.
// Establish a connection to crm and get the connection proxy
string connectionString = "xyz; Username= xyz ;Password=xyz";
CrmConnection connect = CrmConnection.Parse(connectionString);
OrganizationService service;
using (service = new OrganizationService(connect))
{
WhoAmIRequest request = new WhoAmIRequest();
Guid userId = ((WhoAmIResponse)service.Execute(request)).UserId;
ContactDetails contact = new ContactDetails();
//Check if the contact record exists . If exists , update the same record.
//Fecthxml query
string fetchXml = #" <fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
<entity name='contact'>
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute='fullname' descending='false' />
<filter type='and'>
<condition attribute= 'mobilephone' operator='not-null' />
</filter>
</entity>
</fetch>" ;
FetchExpression query = new FetchExpression(fetchXml);
EntityCollection results = service.RetrieveMultiple(query);
if (results.Entities.Count > 0)
{
Entity contactRecord = results[0];
contactRecord["firstname"] = contactInfo.FirstName;
contactRecord["lastname"] = contactInfo.LastName;
contactRecord["emailaddress1"] = contactInfo.EmailId;
contactRecord["mobilephone"] = contactInfo.MobilePhone;
contactRecord["address1_line1"] = contactInfo.Street1;
contactRecord["address1_line2"] = contactInfo.Street2;
contactRecord["address1_line3"] = contactInfo.Street3;
contactRecord["address1_city"] = contactInfo.City;
service.Update(contactRecord);
}
//Else, Create the contact record
else
{
Entity entity = new Entity();
entity.LogicalName = "contact";
entity["firstname"] = contactInfo.FirstName;
entity["lastname"] = contactInfo.LastName;
entity["emailaddress1"] = contactInfo.EmailId;
entity["mobilephone"] = contactInfo.MobilePhone;
entity["address1_line1"] = contactInfo.Street1;
entity["address1_line2"] = contactInfo.Street2;
entity["address1_line3"] = contactInfo.Street3;
entity["address1_city"] = contactInfo.City;
entity["address1_stateorprovince"] = contactInfo.State;
entity["address1_country"] = contactInfo.Country;
entity["spousesname"] = contactInfo.SpouseName;
entity["birthdate"] = contactInfo.Birthday;
entity["anniversary"] = contactInfo.Anniversary;
//Create entity gender with option value
if (contactInfo.Gender == "Male")
{
entity["gendercode"] = new OptionSetValue(1);
}
else
{
entity["gendercode"] = new OptionSetValue(2);
}
//entity["familystatuscode"] = contactInfo.MaritalStatus;
if (contactInfo.MaritalStatus == "Single")
{
entity["familystatuscode"] = new OptionSetValue(1);
}
else
{
entity["familystatuscode"] = new OptionSetValue(2);
}
service.Create(entity);
}
}
// Create the entity
your logic seems ok, with the exception of the FectchXML query. The way you have your code, you will always end up updating the first record retrieved that has its mobilephone field filled. That does not seem a good way to check if a contact already exists.
I suggest you to change the filter of your fetch. In your filter condition you have to use an attribute that represents uniqueness for all contacts.
Apart of this your code looks ok.
Like nunoalmeieda says, you need to have a better way to ascertain if the Contact already exists. A common way of identifying if the Contact already exists would be to check if the email address already exists as it is very unlikely that two people will have the same email address.
I have updated your basic code to show how it is done with FetchXML.
<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
<entity name="contact">
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute="fullname" descending="false" />
<filter type="and">
<condition attribute="emailaddress1" operator="eq" value=contactInfo.EmailId />
</filter>
</entity>
</fetch>
The logic here is that I am checking if the value of emailaddress1 (field in the contact entity of CRM) is equal to the value of your contactInfo.EmailId. I am assuming that contactInfo is the record that you get from ASP.NET.
The rest of your code is fine (I have formatted it a bit to make the question more readable).

Sorting on date field with Sitecore 7 ContentSearch

I'm trying to add a field sort to a date field in a ContentSearch query. I'm able to filter on the index field properly so I'm assuming the field is getting populated with values properly, however, results are not being sorted properly. Any thoughts? Here's the code I'm using to do query:
public static IEnumerable<Episode> GetPastEpisodes(Show show, bool includeMostRecent = false, int count = 0)
{
IEnumerable<Episode> pastEpisodes;
using (var context = _index.CreateSearchContext())
{
// querying against lucene index
pastEpisodes = context.GetQueryable<Episode>().Where(GetPastAirDatePredicate(show));
if(!includeMostRecent)
{
pastEpisodes = pastEpisodes.Where(item => item.Id != GetMostRecentEpisode(show).Id);
}
pastEpisodes = pastEpisodes.OrderByDescending(ep => ep.Latest_Air_Date);
if (count > 0)
{
pastEpisodes = pastEpisodes.Take(count);
}
pastEpisodes = pastEpisodes.ToList();
// map the lucene documents to Sitecore items using the database
foreach (var episode in pastEpisodes)
{
_database.Map(episode);
}
}
return pastEpisodes;
}
private static Expression<Func<Episode,bool>> GetPastAirDatePredicate(Show show)
{
var templatePredicate = PredicateBuilder.Create<Episode>(item => item.TemplateId == IEpisodeConstants.TemplateId);
var showPathPredicate = PredicateBuilder.Create<Episode>(item => item.FullPath.StartsWith(show.FullPath));
var airDatePredicate = PredicateBuilder.Create<Episode>(item => item.Latest_Air_Date < DateTime.Now.Date.AddDays(1));
var fullPredicate = PredicateBuilder.Create<Episode>(templatePredicate).And(showPathPredicate).And(airDatePredicate);
return fullPredicate;
}
The field is stored and untokenized and using the LowerCaseKeywordAnalyzer.
<field fieldName="latest_air_date" storageType="YES" indexType="UN_TOKENIZED" vectorType="NO" boost="1f" type="System.DateTime" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
<analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider"/>
</field>
Episode class has IndexField attribute set:
[IndexField("latest_air_date")]
public virtual DateTime Latest_Air_Date {get; set; }
Kyle,
As far as I can tell everything looks correct with your configuration and code. I mocked out something very similar in a vanilla Sitecore 7.2 instance and Dates sorted without issue.
One thing to note though, and this might be what is giving you some issues, is that Sitecore's FieldReader for DateTime only store the date portion of the DateTime. If you are expecting to be able to sort by true DateTime you will need to add a custom FieldReader or some computed fields.
See this blog that discusses the issue and explains the process to swap out a custom field reader: http://reasoncodeexample.com/2014/01/30/indexing-datetime-fields-sitecore-7-content-search/
I would also suggest looking at the index with Luke to verify what data is actually in the index. https://code.google.com/p/luke/
And finally you may want to enable Search debugging in Sitecore to see exactly how Sitecore is executing the query in Lucene.
Sitecore.ContentSearch.config:
<setting name="ContentSearch.EnableSearchDebug" value="true" />

Get simple product sku and qty using salesOrderInfo of SOAP API in magento

I have added following code in the
/app/code/core/Mage/Sales/Model/Order/Api.php
File.
public function info($orderIncrementId)
{
------
-------
$order = Mage::getModel('sales/order')->loadByIncrementId($orderIncrementId);
// get order total value
$orderValue = number_format ($order->getGrandTotal(), 2, '.' , $thousands_sep = '');
// get order item collection
$orderItems = $order->getItemsCollection();
$skuQtyArray = array();
foreach ($orderItems as $item)
{
$product_id = $item->product_id;
$product_sku = $item->sku;
$product_name = $item->getName();
$product_qty = $item->getQtyOrdered();
$_product = Mage::getModel('catalog/product')->load($product_id);
$cats = $_product->getCategoryIds();
$category_id = $cats[0]; // just grab the first id
$category = Mage::getModel('catalog/category')->load($category_id);
$category_name = $category->getName();
$product = Mage::getModel('catalog/product')->loadByAttribute('sku', $product_sku);
$productType=$product->getTypeID();
if($productType=='simple')
{
$skuQtVal = $product_sku."=".$product_qty;
$skuQtyArray[] = $skuQtVal;
}
}
$result['simple_product_skus'] = $skuQtyArray;
Mage::log($skuQtyArray,null,"logTest.txt",true);
return $result;
}
But when I run following code in the application root
<?php
$client = new SoapClient('localhost/magento/index.php/api/v2_soap/index?wsdl=1');
$session = $client->login('testuser', 'testuser');
$result = $client ->salesOrderInfo($session, '100000026');
print_r($result);
?>
I am not getting the changes which I did.
Please suggest some solution.
edited:
My directory structure to override the core code is as following.
I my Overridden Api.php, I am using like this.
class Sigma_Sales_Model_Order_Api extends Mage_Sales_Model_Order_Api
Got it:
I need to override like this
class Sigma_Sales_Model_Order_Api_V2 extends Mage_Sales_Model_Order_Api_V2
Because:- Mage_Sales_Model_Order_Api_V2 extends Mage_Sales_Model_Order_Api
Muk, you have to go to app\code\core\Mage\Sales\etc and modify wsdl.xml and wsi.xml and add the element for sku or whatever you want as per your requirement.
<element name="sku" type="xsd:string" minOccurs="0" /> //in wsdl.xml
<xsd:element name="sku" type="xsd:string" minOccurs="0" /> //wsi.xml
if you don't want to modify core file than you have to override it.
First correction is on
/app/code/core/Mage/Sales/Model/Order/Api.php
from:
$orderValue = number_format ($order->getGrandTotal(), 2, '.' , $thousands_sep = '');
try to remove the space after number_format
to:
$orderValue = number_format($order->getGrandTotal(), 2, '.' , $thousands_sep = '');
Second is check the Order Increment ID, is it exist?
Third is check on var/log/logTest.txt, what's the value $skuQtyArray?
if you are only add that function, it must be correct now.
My suggestion is do not overwrite core, because Magento is often to upgrade the version, you'll be confused if you want to upgrade it.

Hibernate HQL Query : How to set a Collection as a named parameter of a Query with composite key?

Given the following HQL Query:
from Foo foo where foo.id in (:fooIds)
but here i have composite key in the Id ex we have two PK1 and pk2 as Id's.
How can we implement this query..
how can i pass both paramets in setparameters function of query
My question is similar this question
HBM file containing composite key is present below
<?xml version="1.0"?>
<composite-id>
<key-property name="foo1" column="FOO1" type="java.lang.String" length="36"/>
<key-property name="foo2" column="FOO2" type="java.lang.Short" />
</composite-id>
<property name="EffectiveDt" type="java.sql.Date" column="EFFECTIVE_DT" />
<property name="effectiveTypeCd" type="java.lang.String" column="CERT_EFF_TYPE_CD" />
<property name="statusCd" type="java.lang.String" column="CERT_STATUS_CD" />
</class>
Are you using a composite id? Do you have a separate class representing the composite-id or do you have 2 fields in Foo and you want to search using them in your query?
Posting you Foo class would help!
I'm not 100% sure you can use in in this case. One thing you can do is to build the query manually with something like
String hqlQuery = "from Foo foo where "
boolean first = true;
for( ID id : fooids ) {
if( first ) {
hqlQuery += "foo.id = ?";
first = false;
} else {
hqlQuery += " OR foo.id = ?";
}
}
Query q = em.createQuery(hqlQuery);
int position = 0;
for( ID id : fooids ) {
q.setParameter(position, id);
position++;
}
You might want to double check the code, as I'm writing it here, so there's a big chance there's a typo or two.