How to restore cache after ignite server reconnected - ignite

Question

Really appreciate if someone can help me out.
I have an ignite server written in Java, and have a client written in C#, the client can be connected to the server, and can get server's cache correctly.
once the server restarted, the client received the EVT_CLIENT_NODE_RECONNECTED event from server. But the cache cannot be used any more.
Server code:
CacheConfiguration cacheConfiguration = new CacheConfiguration();
cacheConfiguration.setName("Sample");
cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cacheConfiguration.setBackups(0);
cacheConfiguration.setCopyOnRead(true);
cacheConfiguration.setStoreKeepBinary(false);
cacheConfiguration.setReadThrough(false);
cacheConfiguration.setWriteThrough(true);
cacheConfiguration.setWriteBehindEnabled(true);
cacheConfiguration.setWriteBehindFlushFrequency(2000);
cacheConfiguration.setWriteBehindFlushThreadCount(2);
DriverManagerDataSource theDataSource = new DriverManagerDataSource();
theDataSource.setDriverClassName("org.postgresql.Driver");
theDataSource.setUrl("jdbc:postgresql://192.168.224.128:5432/sample");
theDataSource.setUsername("postgres");
theDataSource.setPassword("password");
CacheJdbcPojoStoreFactory jdbcPojoStoreFactory = new CacheJdbcPojoStoreFactory<Long, SampleModel>()
.setParallelLoadCacheMinimumThreshold(0)
.setMaximumPoolSize(1)
.setDataSource(theDataSource);
cacheConfiguration.setCacheStoreFactory(jdbcPojoStoreFactory);
Collection<JdbcType> jdbcTypes = new ArrayList<JdbcType>();
JdbcType jdbcType = new JdbcType();
jdbcType.setCacheName("Sample");
jdbcType.setDatabaseSchema("public");
jdbcType.setKeyType("java.lang.Long");
Collection<JdbcTypeField> keys = new ArrayList<JdbcTypeField>();
keys.add(new JdbcTypeField(Types.BIGINT, "id", long.class, "id"));
jdbcType.setKeyFields(keys.toArray(new JdbcTypeField[keys.size()]));
Collection<JdbcTypeField> vals = new ArrayList<JdbcTypeField>();
jdbcType.setDatabaseTable("sample");
jdbcType.setValueType("com.nmf.SampleModel");
vals.add(new JdbcTypeField(Types.BIGINT, "id", long.class, "id"));
vals.add(new JdbcTypeField(Types.VARCHAR, "name", String.class, "name"));
jdbcType.setValueFields(vals.toArray(new JdbcTypeField[vals.size()]));
jdbcTypes.add(jdbcType);
((CacheJdbcPojoStoreFactory)cacheConfiguration.getCacheStoreFactory()).setTypes(jdbcTypes.toArray(new JdbcType[jdbcTypes.size()]));
IgniteConfiguration icfg = new IgniteConfiguration();
icfg.setCacheConfiguration(cacheConfiguration);
Ignite ignite = Ignition.start(icfg);
SampleModel:
public class SampleModel implements Serializable {
private long id;
private String Name;
public long getId() {
return id;
}
public void setId(long id) {
id = id;
}
public String getName() {
return Name;
}
public void setName(String name) {
Name = name;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof SampleModel)) return false;
SampleModel that = (SampleModel) o;
return id == that.id;
}
#Override
public int hashCode() {
return (int) (id ^ (id >>> 32));
}
}
Client Code:
ExecutorService executor = Executors.newSingleThreadExecutor(r -> new Thread(r, "worker"));
CacheConfiguration cacheConfiguration = new CacheConfiguration();
cacheConfiguration.setName("Sample");
cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cacheConfiguration.setBackups(0);
cacheConfiguration.setCopyOnRead(true);
cacheConfiguration.setStoreKeepBinary(false);
IgniteConfiguration icfg = new IgniteConfiguration();
icfg.setCacheConfiguration(cacheConfiguration);
icfg.setClientMode(true);
final Ignite ignite = Ignition.start(icfg);
ignite.events().localListen(new IgnitePredicate<Event>() {
public boolean apply(Event event) {
if (event.type() == EVT_CLIENT_NODE_RECONNECTED) {
System.out.println("Reconnected");
executor.submit(()-> {
IgniteCache<Long, SampleModel> cache = ignite.getOrCreateCache("Sample");
System.out.println("Got the cache");
SampleModel model = cache.get(1L);
System.out.println(model.getName());
});
}
return true;
}
}, EVT_CLIENT_NODE_RECONNECTED);
IgniteCache<Long, SampleModel> cache = ignite.getOrCreateCache("Sample");
SampleModel model = cache.get(1L);
System.out.println(model.getName());
Error log on Client:
SEVERE: Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], nodeId=dea5f59b, evt=DISCOVERY_CUSTOM_EVT]
class org.apache.ignite.IgniteCheckedException: Failed to start component: class org.apache.ignite.IgniteException: Failed to initialize cache store (data source is not provided).
at org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8726)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1486)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1931)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1833)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:379)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:688)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:529)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1806)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteException: Failed to initialize cache store (data source is not provided).
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.start(CacheAbstractJdbcStore.java:298)
at org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8722)
... 9 more
July 25, 2017 12:58:38 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=dea5f59b-bdda-47a1-b31d-1ecb08fc746f, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.224.1, 192.168.6.15, 192.168.80.1, 2001:0:9d38:90d7:c83:fac:98d7:5fc1], sockAddrs=[/0:0:0:0:0:0:0:1:0, /127.0.0.1:0, Ares-W11/169.254.194.93:0, /2001:0:9d38:90d7:c83:fac:98d7:5fc1:0, /192.168.6.15:0, windows10.microdone.cn/192.168.224.1:0, /192.168.80.1:0], discPort=0, order=2, intOrder=0, lastExchangeTime=1500958697559, loc=true, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true], topVer=2, nodeId8=dea5f59b, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1500958718133]], crd=TcpDiscoveryNode [id=247d2926-010d-429b-aef2-97a18fbb3b5d, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.224.1, 192.168.6.15, 192.168.80.1, 2001:0:9d38:90d7:c83:fac:98d7:5fc1], sockAddrs=[/192.168.6.15:47500, /2001:0:9d38:90d7:c83:fac:98d7:5fc1:47500, windows10.microdone.cn/192.168.224.1:47500, /192.168.80.1:47500, Ares-W11/169.254.194.93:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1500958718083, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], nodeId=dea5f59b, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=842035444], init=false, lastVer=null, partReleaseFut=null, affChangeMsg=null, skipPreload=true, clientOnlyExchange=false, initTs=1500958718133, centralizedAff=false, changeGlobalStateE=null, exchangeOnChangeGlobalState=false, forcedRebFut=null, evtLatch=0, remaining=[247d2926-010d-429b-aef2-97a18fbb3b5d], srvNodes=[TcpDiscoveryNode [id=247d2926-010d-429b-aef2-97a18fbb3b5d, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.224.1, 192.168.6.15, 192.168.80.1, 2001:0:9d38:90d7:c83:fac:98d7:5fc1], sockAddrs=[/192.168.6.15:47500, /2001:0:9d38:90d7:c83:fac:98d7:5fc1:47500, windows10.microdone.cn/192.168.224.1:47500, /192.168.80.1:47500, Ares-W11/169.254.194.93:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1500958718083, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false]], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Failed to start component: class o.a.i.IgniteException: Failed to initialize cache store (data source is not provided)., hash=1281081640]]
class org.apache.ignite.IgniteCheckedException: Failed to start component: class org.apache.ignite.IgniteException: Failed to initialize cache store (data source is not provided).
at org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8726)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1486)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1931)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1833)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:379)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:688)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:529)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1806)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteException: Failed to initialize cache store (data source is not provided).
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.start(CacheAbstractJdbcStore.java:298)
at org.apache.ignite.internal.util.IgniteUtils.startLifecycleAware(IgniteUtils.java:8722)
... 9 more

Answer 1

Only server nodes store caches(except of Local caches), so, when you restarted server node, this cache was stopped. The problem here is that client node was reconnected to the cluster, but not joined it as new node. That's why cache was not created again.
I think it's wrong behavior and cache should be recreated at client reconnecting.
I've created an issue for that.
As a workaround, you can use Ignite.GetOrCreateCache("Sample") method instead of Ignite.GetCache("Sample")

Answer 2

Are you still having issues where Ignite.GetOrCreateCache("Sample") hangs? Make sure you aren't making that call from a thread in the System Pool. I was listening for the EVT_CLIENT_NODE_RECONNECTED event and calling Ignite.GetOrCreateCache("Sample") when I ran into a similar issue. For more information, see the answer to this question: Closures stuck in 2.0 when try to add an element into the queue