Am trying to enable Ignite Native Persistence in Ignite Yarn Deployment.
Purpose of this is to have data written to disc when RAM overflows.
But when I try to add large number of records to Ignite Grid, the node is getting disconnected and getting below exception.
Error :class org.apache.ignite.internal.NodeStoppingException: Operation has been cancelled (node is stopping).
javax.cache.CacheException: class org.apache.ignite.internal.NodeStoppingException: Operation has been cancelled (node is stopping).
ERROR com.project$$anonfun$startWritingToGrid$1: org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1287)
ERROR com.project$$anonfun$startWritingToGrid$1: org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1648)
ERROR com.project$$anonfun$startWritingToGrid$1: org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.putAll(IgniteCacheProxyImpl.java:1071)
ERROR com.project$$anonfun$startWritingToGrid$1: org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.putAll(GatewayProtectedCacheProxy.java:928)
Please find below the details.
Ignite Version : 2.3.0
Cluster details for Yarn Deployment:
IGNITE_NODE_COUNT=10
IGNITE_RUN_CPU_PER_NODE=5
IGNITE_MEMORY_PER_NODE=10096
IGNITE_VERSION=2.3.0
IGNITE_PATH=/tmp/ignite/2.3.0/apache-ignite-fabric-2.3.0-bin.zip
IGNITE_RELEASES_DIR=/tmp/ignite/2.3.0/releases
IGNITE_WORKING_DIR=/tmp/ignite/2.3.0/work
IGNITE_XML_CONFIG=/tmp/ignite/2.3.0/config/ignite-config.xml
IGNITE_USERS_LIBS=/tmp/ignite/2.3.0/libs
IGNITE_LOCAL_WORK_DIR=/local/home/ignite/2.3.0
Ignite Configuration for Yarn deployment:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util-2.0.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="clientMode" value="false"/>
<property name="dataStorageConfiguration">
<bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="true"/>
</bean>
</property>
</bean>
</property>
<property name="peerClassLoadingEnabled" value="true"/>
<property name="networkTimeout" value="10000000"/>
<property name="networkSendRetryCount" value="50"/>
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value><hosts>:47500</value>
</list>
</property>
</bean>
</property>
<property name="networkTimeout"
value="10000000"/>
<property name="joinTimeout"
value="10000000"/>
<property name="maxAckTimeout"
value="10000000"/>
<property name="reconnectCount" value="50"/>
<property name="socketTimeout" value="10000000"/>
</bean>
</property>
</bean>
</beans>
Code to Add Data to grid :
var cacheConf: CacheConfiguration[Long, Data] = new
CacheConfiguration[Long, Data]("DataCache")
cacheConf.setCacheMode(CacheMode.PARTITIONED)
cacheConf.setIndexedTypes(classOf[Long], classOf[Data])
val cache = ignite.getOrCreateCache(cacheConf)
var dataMap = getDataMap()
cache.putAll(dataMap)
Code to Count records:
val sql1 = "select * from DataCache"
val count = cache.query(new SqlFieldsQuery(sql1)).getAll.size()
Related
I have the following configuration file
<bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="peerClassLoadingEnabled" value="true"/>
<property name="includeEventTypes">
<list>
<!--Task execution events-->
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
<util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
</list>
</property>
<property name="metricsUpdateFrequency" value="10000"/>
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>127.0.0.1:47500..47509</value>
<value>127.0.0.1:48500..48509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<!-- Enabling the required Failover SPI. -->
<property name="failoverSpi">
<bean class="org.apache.ignite.spi.failover.jobstealing.JobStealingFailoverSpi"/>
</property>
<property name="collisionSpi">
<bean class="org.apache.ignite.spi.collision.jobstealing.JobStealingCollisionSpi">
<property name="activeJobsThreshold" value="50"/>
<property name="waitJobsThreshold" value="0"/>
<property name="messageExpireTime" value="1000"/>
<property name="maximumStealingAttempts" value="10"/>
<property name="stealingEnabled" value="true"/>
</bean>
</property>
</bean>
The closure gets executed over the server nodes in the grid as expected.
When we add a new node by executing the below command to the grid during the execution of closure
The existing nodes acknowledge the addition of the new node in the grid but the closure is not distributed to the newly added node.
Below is my closure implementation
#Override
public AccruedSimpleInterest apply(SimpleInterestParameter simpleInterestParameter) {
BigDecimal si = simpleInterestParameter.getPrincipal()
.multiply(new BigDecimal(simpleInterestParameter.getYears()))
.multiply(new BigDecimal(simpleInterestParameter.getRate())).divide(SimpleInterestClosure.HUNDRED);
System.out.println("Calculated SI for id=" + simpleInterestParameter.getId() + " SI=" + si.toPlainString());
return new AccruedSimpleInterest(si, simpleInterestParameter);
}
Below is the main class
public static void main(String... args) throws IgniteException, IOException {
Factory<SimpleInterestClosure> siClosureFactory = FactoryBuilder.factoryOf(new SimpleInterestClosure());
ClassPathResource ress = new ClassPathResource("example-ignite-poc.xml");
File file = new File(ress.getPath());
try (Ignite ignite = Ignition.start(file.getPath())) {
System.out.println("Started Ignite Cluster");
IgniteFuture<Collection<AccruedSimpleInterest>> igniteFuture = ignite.compute()
.applyAsync(siClosureFactory.create(), createParamCollection());
Collection<AccruedSimpleInterest> res = igniteFuture.get();
System.out.println(res.size());
}nter code here
As far as my understanding goes, Job Stealing SPI requires you to implement some additional APIs in order to work.
Please see this discussion on user list:
Some remarks about job stealing SPI:
1)You have some nodes that can proceed the tasks of some compute job.
2)Tasks will be executed in public thread pool by default:
https://apacheignite.readme.io/docs/thread-pools#section-public-pool
3)If some node thread pool is busy then some task of compute job can be
executed on other node.
In next cases it will not work:
1)In case if you choose specific node for your compute task
2)In case if you do affinity call (the same as above but node will be
choose by affinity mapping)
I've been working on authenticating against an active directory server with jasper 6.4.0 for a while now, and have been getting the following error.
2017-10-16 13:39:35,145 WARN JSLdapAuthenticationProvider,http-apr-8080-exec-9:62 - [
LDAP: error code 49 - 80090308: LdapErr: DSID-0C09042F, comment: AcceptSecurityContext error, data 52e, v2580 ];
nested exception is javax.naming.AuthenticationException:
[LDAP: error code 49 - 80090308: LdapErr: DSID-0C09042F, comment: AcceptSecurityContext error, data 52e, v2580 ]
From what I've gathered, data 52e indicates invalid credentials. I tried changing my service account password in my configuration to bob to see if I would get the same error (showing that it's an error when binding to the ldaps server rather than the test account I was logging in with), and I did.
Here is the configuration for the service account in jasper.
<bean id="ldapContextSource" class="com.jaspersoft.jasperserver.api.security.externalAuth.ldap.JSLdapContextSource">
<constructor-arg value="ldaps://MyLDAPSServer:636/"/>
<!-- manager user name and password (may not be needed) -->
<property name="userDn" value="dc=mydomain,dc=com,uid=MyServiceAccount"/>
<property name="password" value="SomePasswordWithSpecialCharacters"/>
</bean>
I'm confident that the username and password are correct. I'm able to authenticate against the same ldaps server using the following Python code.
from ldap3 import Server, \
Connection, \
AUTO_BIND_NO_TLS, \
SUBTREE, \
ALL_ATTRIBUTES
def get_ldap_info(u):
with Connection(Server('MyLDAPSServer', port=636, use_ssl=True),
auto_bind=AUTO_BIND_NO_TLS,
read_only=True,
check_names=True,
user='MyServiceAccount', password='SomePasswordWithSpecialCharacters') as c:
c.search(search_base='DC=mydomain,DC=com',
search_filter='(&(samAccountName=' + u + '))',
search_scope=SUBTREE,
attributes=ALL_ATTRIBUTES,
get_operational_attributes=True)
print(c.response_to_json())
print(c.result)
get_ldap_info('test.user')
The one thing I've been able to think of, is maybe jasper doesn't like having special characters in the password?
Here is the remainder of the configuration for the jasper server in case I'm missing something.
<!--
~ Copyright (C) 2005 - 2014 TIBCO Software Inc. All rights reserved.
~ http://www.jaspersoft.com.
~ Licensed under commercial Jaspersoft Subscription License Agreement
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">
<!-- ############ LDAP authentication ############
- Sample configuration of external authentication via an external LDAP server.
-->
<bean id="proxyAuthenticationProcessingFilter" class="com.jaspersoft.jasperserver.api.security.EncryptionAuthenticationProcessingFilter"
parent="mtAuthenticationProcessingFilter">
<property name="authenticationManager">
<ref local="ldapAuthenticationManager"/>
</property>
<property name="authenticationSuccessHandler" ref="externalAuthSuccessHandler" />
</bean>
<bean id="proxyAuthenticationSoapProcessingFilter"
class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.MTDefaultAuthenticationSoapProcessingFilter">
<property name="authenticationManager" ref="ldapAuthenticationManager"/>
<property name="authenticationSuccessHandler" ref="externalAuthSuccessHandler" />
<property name="filterProcessesUrl" value="/services"/>
</bean>
<bean id="proxyAuthenticationRestProcessingFilter" class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.MTDefaultAuthenticationRestProcessingFilter">
<property name="authenticationManager">
<ref local="ldapAuthenticationManager"/>
</property>
<property name="authenticationSuccessHandler" ref="externalAuthSuccessHandler" />
<property name="filterProcessesUrl" value="/rest/login"/>
</bean>
<bean id="proxyRequestParameterAuthenticationFilter"
class="com.jaspersoft.jasperserver.war.util.ExternalRequestParameterAuthenticationFilter" parent="requestParameterAuthenticationFilter">
<property name="authenticationManager">
<ref local="ldapAuthenticationManager"/>
</property>
<property name="externalDataSynchronizer" ref="externalDataSynchronizer"/>
</bean>
<bean id="externalAuthSuccessHandler"
class="com.jaspersoft.jasperserver.api.security.externalAuth.JrsExternalAuthenticationSuccessHandler" parent="successHandler">
<property name="externalDataSynchronizer">
<ref local="externalDataSynchronizer"/>
</property>
</bean>
<bean id="proxyBasicProcessingFilter"
class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.MTExternalAuthBasicProcessingFilter" parent="mtBasicProcessingFilter">
<property name="authenticationManager" ref="ldapAuthenticationManager"/>
<property name="externalDataSynchronizer" ref="externalDataSynchronizer"/>
</bean>
<bean id="ldapAuthenticationManager" class="com.jaspersoft.jasperserver.api.security.externalAuth.wrappers.spring.JSProviderManager">
<property name="providers">
<list>
<ref local="ldapAuthenticationProvider"/>
<ref bean="${bean.daoAuthenticationProvider}"/>
<!--anonymousAuthenticationProvider only needed if filterInvocationInterceptor.alwaysReauthenticate is set to true
<ref bean="anonymousAuthenticationProvider"/>-->
</list>
</property>
</bean>
<bean id="ldapAuthenticationProvider" class="com.jaspersoft.jasperserver.api.security.externalAuth.wrappers.spring.ldap.JSLdapAuthenticationProvider">
<constructor-arg>
<bean class="com.jaspersoft.jasperserver.api.security.externalAuth.wrappers.spring.ldap.JSBindAuthenticator">
<constructor-arg><ref local="ldapContextSource"/></constructor-arg>
<property name="userSearch" ref="userSearch"/>
</bean>
</constructor-arg>
<constructor-arg>
<bean class="com.jaspersoft.jasperserver.api.security.externalAuth.wrappers.spring.ldap.JSDefaultLdapAuthoritiesPopulator">
<constructor-arg index="0"><ref local="ldapContextSource"/></constructor-arg>
<constructor-arg index="1"><value></value></constructor-arg>
<property name="groupRoleAttribute" value="title"/>
<property name="groupSearchFilter" value="(uid={1})"/>
<property name="searchSubtree" value="true"/>
<!-- Can setup additional external default roles here <property name="defaultRole" value="LDAP"/> -->
</bean>
</constructor-arg>
</bean>
<bean id="userSearch"
class="com.jaspersoft.jasperserver.api.security.externalAuth.wrappers.spring.ldap.JSFilterBasedLdapUserSearch">
<constructor-arg index="0">
<value>cn=Users</value>
</constructor-arg>
<constructor-arg index="1">
<!--<value>(uid={0})</value>-->
<!--<value>(uid=test.user)</value>-->
<value>(sAMAccountName={0})</value>
</constructor-arg>
<constructor-arg index="2">
<ref local="ldapContextSource" />
</constructor-arg>
<property name="searchSubtree">
<value>true</value>
</property>
</bean>
<bean id="ldapContextSource" class="com.jaspersoft.jasperserver.api.security.externalAuth.ldap.JSLdapContextSource">
<constructor-arg value="ldaps://MyLDAPSServer:636/"/>
<!-- manager user name and password (may not be needed) -->
<property name="userDn" value="dc=mydomain,dc=com,uid=MyServiceAccount"/>
<property name="password" value="SomePasswordWithSpecialCharacters"/>
</bean>
<!-- ############ LDAP authentication ############ -->
<!-- ############ JRS Synchronizer ############ -->
<bean id="externalDataSynchronizer"
class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.MTExternalDataSynchronizerImpl">
<property name="externalUserProcessors">
<list>
<ref local="ldapExternalTenantProcessor"/>
<ref local="mtExternalUserSetupProcessor"/>
<!-- Example processor for creating user folder-->
<!--<ref local="externalUserFolderProcessor"/>-->
</list>
</property>
</bean>
<bean id="abstractExternalProcessor" class="com.jaspersoft.jasperserver.api.security.externalAuth.processors.AbstractExternalUserProcessor" abstract="true">
<property name="repositoryService" ref="${bean.repositoryService}"/>
<property name="userAuthorityService" ref="${bean.userAuthorityService}"/>
<property name="tenantService" ref="${bean.tenantService}"/>
<property name="profileAttributeService" ref="profileAttributeService"/>
<property name="objectPermissionService" ref="objectPermissionService"/>
</bean>
<!--
Multi-tenant configuration. For a JRS deployment with multiple
organizations, modify this bean to set up your organizations. For
single-organization deployments, comment this out and uncomment the version
below.
-->
<bean id="ldapExternalTenantProcessor" class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.processors.ldap.LdapExternalTenantProcessor" parent="abstractExternalProcessor">
<property name="ldapContextSource" ref="ldapContextSource"/>
<property name="multiTenancyService"><ref bean="internalMultiTenancyService"/></property>
<property name="excludeRootDn" value="false"/>
<!--only following LDAP attributes will be used in creation of organization hierarchy.
Eg. cn=Smith,ou=Developement,o=Jaspersoft will produce tanant Development as child of
tenant Jaspersoft (if excludeRootDn=false) as child of default tenant organization_1-->
<property name="organizationRDNs">
<list>
<value>dc</value>
<value>c</value>
<value>o</value>
<value>ou</value>
<value>st</value>
</list>
</property>
<property name="rootOrganizationId" value="organization_1"/>
<property name="tenantIdNotSupportedSymbols" value="#{configurationBean.tenantIdNotSupportedSymbols}"/>
<!-- User credentials are setup in js.externalAuth.properties-->
<property name="externalTenantSetupUsers">
<list>
<bean class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.processors.MTAbstractExternalProcessor.ExternalTenantSetupUser">
<property name="username" value="${new.tenant.user.name.1}"/>
<property name="fullName" value="${new.tenant.user.fullname.1}"/>
<property name="password" value="${new.tenant.user.password.1}"/>
<property name="emailAddress" value="${new.tenant.user.email.1}"/>
<property name="roleSet">
<set>
<value>ROLE_ADMINISTRATOR</value>
<value>ROLE_USER</value>
</set>
</property>
</bean>
</list>
</property>
</bean>
<!--
Single tenant configuration. For a JRS deployment with a single
organization, uncomment this bean and configure it to set up your organization.
Comment out the multi-tenant version of ldapExternalTenantProcessor above
-->
<!--<bean id="ldapExternalTenantProcessor" class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.processors.ldap.LdapExternalTenantProcessor" parent="abstractExternalProcessor">
<property name="ldapContextSource" ref="ldapContextSource"/>
<property name="multiTenancyService"><ref bean="internalMultiTenancyService"/></property>
<property name="excludeRootDn" value="true"/>
<property name="defaultOrganization" value="organization_1"/>
</bean>-->
<bean id="mtExternalUserSetupProcessor" class="com.jaspersoft.jasperserver.multipleTenancy.security.externalAuth.processors.MTExternalUserSetupProcessor" parent="abstractExternalProcessor">
<!--Default permitted role characters; others are removed. Change regular expression to allow other chars.
<property name="permittedExternalRoleNameRegex" value="[A-Za-z0-9_]+"/>-->
<property name="userAuthorityService">
<ref bean="${bean.internalUserAuthorityService}"/>
</property>
<property name="defaultInternalRoles">
<list>
<value>ROLE_USER</value>
</list>
</property>
<property name="organizationRoleMap">
<map>
<!-- Example of mapping customer roles to JRS roles -->
<entry>
<key>
<value>ROLE_ADMIN_EXTERNAL_ORGANIZATION</value>
</key>
<!-- JRS role that the <key> external role is mapped to-->
<value>ROLE_ADMINISTRATOR</value>
</entry>
</map>
</property>
</bean>
<!-- EXAMPLE Processor
<bean id="externalUserFolderProcessor"
class="com.jaspersoft.jasperserver.api.security.externalAuth.processors.ExternalUserFolderProcessor"
parent="abstractExternalProcessor">
<property name="repositoryService" ref="${bean.unsecureRepositoryService}"/>
</bean>
-->
<!-- ############ JRS Synchronizer ############ -->
</beans>
For anyone in the future who's struggling with this like I was, the problem was the userDn in my ldapContextSource.
The correct userDn was
<property name="userDn" value="CN=myserviceaccount,OU=my ou,DC=mydomain,DC=com"/>
I found that by running this python script and looking at the output:
from ldap3 import Server, \
Connection, \
AUTO_BIND_NO_TLS, \
SUBTREE, \
ALL_ATTRIBUTES
def get_ldap_info(u):
with Connection(Server('MyLDAPSServer', port=636, use_ssl=True),
auto_bind=AUTO_BIND_NO_TLS,
read_only=True,
check_names=True,
user='MyServiceAccount', password='SomePasswordWithSpecialCharacters') as c:
c.search(search_base='DC=mydomain,DC=com',
search_filter='(&(samAccountName=' + u + '))',
search_scope=SUBTREE,
attributes=ALL_ATTRIBUTES,
get_operational_attributes=True)
print(c.response_to_json())
print(c.result)
get_ldap_info('myserviceaccount')
Im trying to retrieve the cached value for every element in the JavaPairRDD. Im using the LOCAL cache mode as i want to minimize data shuffling of cached data. The ignite nodes are started in embedded mode within a spark job. The following code works fine if i run it on a single node. However, when i run it on a cluster of 5 machines, i get zero results.
The first attempt i had was using the IgniteRDD sql method:
dataRDD.sql("SELECT v.id,v.sub,v.obj FROM VPRow v JOIN table(id bigint = ?) i ON v.id = i.id",new Object[] {objKeyEntries.toArray()});
where objKeyEntries is a collected set of entries in an RDD. The second attempt was using AffinityRun:
JavaPairRDD<Long, VPRow> objEntries = objKeyEntries.mapPartitionsToPair(new PairFlatMapFunction<Iterator<Tuple2<Long, Boolean>>, Long, VPRow>() {
#Override
public Iterator<Tuple2<Long, VPRow>> call(Iterator<Tuple2<Long, Boolean>> tuple2Iterator) throws Exception {
ApplicationContext ctx = new ClassPathXmlApplicationContext("ignite-rdd.xml");
IgniteConfiguration igniteConfiguration = (IgniteConfiguration) ctx.getBean("ignite.cfg");
Ignite ignite = Ignition.getOrStart(igniteConfiguration);
IgniteCache<Long, VPRow> cache = ignite.getOrCreateCache("dataRDD");
ArrayList<Tuple2<Long,VPRow>> lst = new ArrayList<>();
while(tuple2Iterator.hasNext()) {
Tuple2<Long, Boolean> val = tuple2Iterator.next();
ignite.compute().affinityRun("dataRDD", val._1(),()->{
lst.add(new Tuple2<>(val._1(),cache.get(val._1())));
});
}
return lst.iterator();
}
});
The following is the ignite-rdd.xml configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="memoryConfiguration">
<bean class="org.apache.ignite.configuration.MemoryConfiguration">
<property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
<property name="defaultMemoryPolicyName" value="default_mem_plc"/>
<property name="memoryPolicies">
<list>
<bean class="org.apache.ignite.configuration.MemoryPolicyConfiguration">
<property name="name" value="default_mem_plc"/>
<property name="initialSize" value="#{5 * 1024 * 1024 * 1024}"/>
</bean>
</list>
</property>
</bean>
</property>
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<!-- Set a cache name. -->
<property name="name" value="dataRDD"/>
<!-- Set a cache mode. -->
<property name="cacheMode" value="LOCAL"/>
<!-- Index Integer pairs used in the example. -->
<property name="indexedTypes">
<list>
<value>java.lang.Long</value>
<value>edu.code.VPRow</value>
</list>
</property>
<property name="affinity">
<bean class="org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction">
<property name="partitions" value="50"/>
</bean>
</property>
</bean>
</list>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<value>[IP5]</value>
<value>[IP4]</value>
<value>[IP3]</value>
<value>[IP2]</value>
<value>[IP1]</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
Are you sure that you need to use LOCAL cache mode?
Most likely you filled cache only on one node and local caches on other nodes still empty.
affinityRun doesn't work because you have LOCAL cache, not PARTITIONED, so, it's not possible to determine owner node for key with AffinityFunction.
I'm working with spring 3, hibernate 4. I'm trying to follow this tutorial http://www.mkyong.com/hibernate/hibernate-display-generated-sql-to-console-show_sql-format_sql-and-use_sql_comments/, but my hibernate configuration is different:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jee="http://www.springframework.org/schema/jee"
xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:beans="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.2.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd
http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd">
<!-- JDBC Data Source. It is assumed you have MySQL running on localhost port 3306 with
username root and blank password. Change below if it's not the case -->
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost:3306/project"/>
<property name="username" value="root"/>
<property name="password" value="1234"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="show_sql" value="true" />
<property name="format_sql" value="true" />
<property name="use_sql_comments" value="true" />
</bean>
</beans>
And the properties show_sql, format_sql and use_sql_comments are not working this way. I get this exception:
Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'show_sql' of bean class [org.apache.commons.dbcp.BasicDataSource]: Bean property 'show_sql' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
Is there anyway to achieve the tutorial with the definition of the bean??
show_sql not a property of org.apache.commons.dbcp.BasicDataSource . You have to define it in session factory configuration.
like this
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="data" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.H2Dialect</prop>
<prop key="hibernate.current_session_context_class">thread</prop>
<prop key="hibernate.transaction.factory_class">org.hibernate.transaction.JDBCTransactionFactory</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.hbm2ddl.auto">update</prop>
</props>
</property>
</bean>
The simplest approach probably is to set following logger to DEBUG:
org.hibernate.SQL
If you use log4j, find / create a log4j.properties file on your classpath root and add
log4j.logger.org.hibernate.SQL=DEBUG
See here for more info about log4j properties: http://logging.apache.org/log4j/1.2/manual.html
As I'm using JEE 8 and JBoss EAP, I managed to got the SQL after adding this line:
-Dorg.jboss.as.logging.per-deployment=false
on the end of "VM arguments" (Server tab -> JBoss Properties -> Open lauch configuration).
I'm trying to setup spring's DefaultMessageListenerContainer class to redeliver messages after an exception is thrown or session.rollback() is called. I am also trying to get this running on glassfish 3.1.2 web profile.
When calling session.rollback() in the onMessage() method of my SessionAwareMessageListener, I get an exception with the message saying: MessageDispatcher - [C4024]: The session is not transacted. I don't see this problem with ActiveMQ, but of course that configuration is different because I'm not using it in an application server.
Has anyone here gotten this working? My configuration follows:
<bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
<prop key="java.naming.factory.initial">com.sun.enterprise.naming.SerialInitContextFactory</prop>
<prop key="java.naming.provider.url">${jms.jndicontext.url}</prop>
<prop key="java.naming.factory.state">com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl</prop>
<prop key="java.naming.factory.url.pkgs">com.sun.enterprise.naming</prop>
</props>
</property>
</bean>
<bean id="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jndiTemplate" />
<property name="jndiName" value="${jms.connection.factory}" />
</bean>
<bean id="jmsTemplate"
class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="defaultDestination" ref="jmsServiceQueue"/>
</bean>
<bean id="jmsServiceProducer"
class="net.exchangesolutions.services.messaging.service.jms.JmsMessageServiceProducerImpl">
<property name="serviceTemplate" ref="jmsTemplate"/>
<property name="serviceDestination" ref="jmsServiceQueue"/>
</bean>
<bean id="myMessageListener"
class="com.myorg.jms.MessageDispatcher"/>
<bean id="jmsServiceContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="destination" ref="jmsServiceQueue"/>
<property name="messageListener" ref="myMessageListener"/>
<property name="errorHandler" ref="jmsErrorHandler" />
<property name="receiveTimeout" value="180000"/>
<property name="concurrentConsumers" value="1"/>
<property name="cacheLevelName" value="CACHE_NONE"/>
<property name="pubSubNoLocal" value="true"/>
<property name="sessionTransacted" value="true"/>
<property name="sessionAcknowledgeMode" value="2" />
<property name="transactionManager" ref="jmsTransactionManager"/>
</bean>
<bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>
Setting the acknowledge="auto", the message is acknowledged before listener execution, so the message is deleted from queue.
I have also achieved the DLQ scenario in Spring Application by doing the following changes to your code.
First, we set the acknowledge="transacted" (Since we want guaranteed redelivery in case of exception thrown and Trans acknowledgment for successful listener execution)
<jms:listener-container container-type="default" connection-factory="connectionFactory" acknowledge=" transacted">
Next, since we want to throw the JMSException, we are implementing SessionAwareMessageListener.
public class MyMessageQueueListener implements SessionAwareMessageListener {
public void onMessage( Message message , Session session ) throws JMSException {
//DO something
if(success){
//Do nothing – so the transaction acknowledged
} else {
//throw exception - So it redelivers
throw new JMSException("..exception");
}
}
}
I have tested this. This seems working fine.