Plugging in Custom Serializer in Apache Ignite
I tried to add Kyro Serializer in the Binary Configuration bean but at runtime it gave me a class type conversion error.
My Code is
<property name="binaryConfiguration">
<bean class="org.apache.ignite.configuration.BinaryConfiguration">
<property name="typeConfigurations">
<list>
<bean class="org.apache.ignite.binary.BinaryTypeConfiguration">
<property name="typeName" value="testPojo" />
<property name="serializer">
<bean class="com.esotericsoftware.kryo.serializers.DefaultSerializers" />
</property>
</bean>
</list>
</property>
</bean>
</property>
Error Log is
Caused by: java.lang.IllegalStateException: Cannot convert value of type [com.esotericsoftware.kryo.serializers.DefaultSerializers] to required type [org.apache.ignite.binary.BinarySerializer] for property 'serializer': no matching editors or conversion strategy found
at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:302)
at org.springframework.beans.AbstractNestablePropertyAccessor.convertIfNecessary(AbstractNestablePropertyAccessor.java:576)
... 104 more
On digging in the Apache Ignite provided BinarySerializer, came to a conclusion that one has to write a custom implementation for the serializer as Other plugin Serializers to implement it.
How is the Optimized Marshaller beneficial ?
BinarySerializer is an interface that can be implemented to customize (de)serialization logic for a particular type. This is an analogy to Externalizable for BinaryMarshaller, which is the default marshaller since Ignite 1.5. Refer to this page for details: https://apacheignite.readme.io/docs/binary-marshaller
OptimizedMarshaller implements legacy serialization protocol, which was used before binary format was introduced. It's still available, but binary format is recommended.
You can also implement your own marshaller (e.g., based on Kryo). To do this, implement the Marshaller interface and provide this implementation in configuration via IgniteConfiguration.setMarshaller() property.
Related
I'm using Ignite 2.5 and have deployed a couple of servers like this:
One computer acts as DB server with persistence enabled.
Three other computers are compute servers with same cache as on DB server but without persistence.
I have classes like this:
public class Address implements Serializable
{
String streetName;
String houseNumber;
String cityName;
String countryName;
}
public class Person implements Serializable
{
#QuerySqlField
String firstName;
#QuerySqlField
String lastName;
#QuerySqlField
Address homeAddress;
}
The cache is configured on all servers with this XML:
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="Persons" />
<property name="cacheMode" value="PARTITIONED" />
<property name="backups" value="0" />
<property name="storeKeepBinary" value="true" />
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="writeSynchronizationMode" value="FULL_SYNC"/>
<property name="indexedTypes">
<list>
<value>java.lang.String</value>
<value>Person</value>
</list>
</property>
</bean>
On the DB server in addition there is persistence enabled like this:
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="storagePath" value="/data/Storage" />
<property name="walPath" value="/data/Wal" />
<property name="walArchivePath" value="/data/WalArchive" />
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="initialSize" value="536870912" />
<property name="maxSize" value="1073741824" />
<property name="persistenceEnabled" value="true" />
</bean>
</property>
</bean>
</property>
<property name="binaryConfiguration">
<bean class="org.apache.ignite.configuration.BinaryConfiguration">
<property name="compactFooter" value="false" />
</bean>
</property>
The cache is used with put/get but also with SqlQuery and SqlFieldsQuery.
From time to time I have to update the class definitions, i.e. add another field or so. I'm fine to shut down the whole cluster for updating the classes as it requires an application update anyway.
I believe the above configuration is generally OK to use for Ignite?
Do I understand this other question (Apache Ignite persistent store recommended way for class versions) correctly that on the DB server I shall not have the Person classes in the classpath? Wouldn't then the XML config fail because it's missing the index classes?
On compute servers I shall also not use the Person classes but instead read from cache into BinaryObject? Is the idea to manually fill my Person class from the BinaryObject?
Currently when I update a field in the Person class I get strange errors like:
Unknown pair [platformId=0, typeId=1968448811]
Sorry if there are multiple questions here, I somehow am lost with the "Unknown pair" issues and am now questioning if my complete setup is right.
Thanks for any advise.
I believe the above configuration is generally OK to use for Ignite?
No, you can't configure persistence only for one node only. So in your case, all nodes will store data, but only one node will persist its data, so only part of data will be persisted and this can lead to unpredictable consequences. If you want only one node to store data you need to configure node filter.
With the node filter, the cache will be located only on one node and this node will store data, however in this case your compute nodes would have to do network IO to read from cache.
Do I understand this other question (Apache Ignite persistent store
recommended way for class versions) correctly that on the DB server I
shall not have the Person classes in the classpath? Wouldn't then the
XML config fail because it's missing the index classes?
You don't need classes of your model to be in the classpath, but please, make sure that you work with BinaryObjects only on the server side, so all compute tasks should use BinaryObjects. Also as you mentioned, this configuration won't work, you need to use Query Entity instead for index configuration.
On compute servers I shall also not use the Person classes but instead read from cache into BinaryObject? Is the idea to manually fill my Person class from the BinaryObject?
Well, if you don't have the Person class on the server side you just can't create Person class, you need to use BinaryObject in your compute jobs.
Currently when I update a field in the Person class I get strange errors like: Unknown pair [platformId=0, typeId=1968448811]
Could you please provide the full stacktrace and say on what operation you get this error?
I'm trying to convert a Spring 3 app to Spring 4 (4.3.2.RELEASE) but I can't get org.springframework.jms.listener.adapter.MessageListenerAdapter to lazy load anymore.
To isolate the problem I removed all references to the following bean def but it still tries to load and throws a java.lang.NoClassDefFoundError: javax/jms/MessageListener, which is a runtime dependency that I am not providing on purpose in this setup:
<bean id="messageListener" class="org.springframework.jms.listener.adapter.MessageListenerAdapter" lazy-init="true" scope="prototype">
<property name="defaultListenerMethod" value="handleRequest"/>
<property name="defaultResponseDestination" ref="defaultResponseDestination" />
<property name="delegate" ref="stringRequestToStreamRequestHandler" />
</bean>
Is this expected behaviour or a bug?
In Fabric8, the preferred way to obtain an ActiveMQ connection is via the mq-fabric profile, which provides an ActitveMQConnection object via Declarative Services. An example of this is given on GitHub, which works just fine.
However, I've yet to find a way for Declarative Services and Blueprint Services to collaborate in Fabric8 (or any OSGI-environment, really), thus, my OSGI application must either use DS or blueprint. Mixing both doesn't seem to be an option.
If you want to use blueprint (which I do), you must first create a broker through the web UI, then go back to the console and type cluster-list, finding the port that Fabric8 assigned to the broker and then configure a connection in blueprint like so:
<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://mydomain:33056" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
While this does work, it's not exactly deployment-friendly, as it involves a few manual steps that I'd like to avoid if possible. The main issue is that I don't know what that port is going to be. I've combed through the config files and couldn't find it anywhere.
Is there a cleaner, more automated way to obtain an ActiveMQ connection in Fabric8 via blueprint, or must we use Declarative Services?
Stumbled across a solution to this issue in the fabric-camel-demo, which illustrates how to instantiate an ActiveMQConnectionFactory bean in Fabric8 via Blueprint.
<!-- use the fabric protocol in the brokerURL to connect to the ActiveMQ broker registered as default name -->
<!-- notice we could have used amq as the component name in Camel, and avoid any configuration at all,
as the amq component is provided out of the box when running in fabric -->
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="discovery:(fabric:default)"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
Hope this helps!
I'm switching to VelocityTools 2.0 but the new tools.xml doesn't load. I replaced toolbox.xml and now I use tools.xml but when I run the server I get:
"XMLToolboxManager:100: XMLToolboxManager has been deprecated. Please
use org.apache.velocity.tools.ToolboxFactory instead.
ServletToolboxManager:131: ServletToolboxManager has been deprecated.
Please use org.apache.velocity.tools.ToolboxFactory instead."
How can I use the ToolboxFactory? XMLToolboxManager or ServletToolboxManager are not referenced anywhere in my code so I can't simply replace the classes.
When I use toolbox.xml with VelocityTools 2.0 it works just fine, but I need to get the new xml syntax work in tools.xml
My bean definition in velocity.xml looks like this:
<bean id="viewResolver" class="org.springframework.web.servlet.view.velocity.VelocityLayoutViewResolver">
<property name="viewClass">
<value>org.springframework.web.servlet.view.velocity.VelocityLayoutView</value>
</property>
<property name="contentType">
<value>text/html;charset=UTF-8</value>
</property>
<property name="cache"><value>true</value></property>
<property name="prefix"><value></value></property>
<property name="suffix"><value>.vm</value></property>
<property name="toolboxConfigLocation"><value>/WEB-INF/tools.xml</value></property>
</bean>
I tried removing the property for "toolboxConfigLocation" from the bean and hoped that the default tools.xml will be available, but when I tried to use i.e $math didn't work. What am I doing wrong?
Can someone help please?
I've this configuration in the hibernate.cfg.xml:
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2">
<session-factory>
<property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
<property name="connection.driver_class">NHibernate.Driver.SqlClientDriver</property>
<property name="dialect">NHibernate.Dialect.MsSql2005Dialect</property>
<property name="connection.connection_string">Data Source=.\SQLEXPRESS;Initial Catalog=MyDB;Integrated Security=SSPI;</property>
<property name="show_sql">true</property>
</session-factory>
</hibernate-configuration>
I've just created a Class Library and I've created an integration test using MbUnit. It fails. A part of the report(the one which I think is enough) goes here:
** NO TESTS WERE RUN (No tests found) **
TestCase 'M:IntegrationTests.RepositoryTests.ListAllPostsReturnsAListOfPost'
failed: The ProxyFactoryFactory was not configured.
Initialize 'proxyfactory.factory_class' property of the session-factory configuration section with one of the available NHibernate.ByteCode providers.
I have read many tutorials and haven't seen this proxy factory configuration. Is specifying it really necessary? If so, how can I do that? Do I've to reference some other library?
If you're using the latest of NHibernate(2.1), you'll notice that mainline for NH doesn't have a dependency on castle for proxy generation anymore, so all those tutorials you've been looking at are probably out of date.
Basically, you now have a few choices of how you want your dynamic proxies created, so you'll need to explicitly configure which proxy generator you want to use. Examples can be found in this how-to post on forge. A full list of the options is referenced here.
P.S. if you want to keep things simple, just use Castle as the older versions of NHibernate all used it by default.