Load cache with predicate - ignite

Question

I have a cache with many types in it. I am running pure Java nodes. I load the cache at start up before deploying services. The cache is backed by CacheJdbcPojoStore. I am using Ignite 1.8.
When I load with null predicate, everything works fine. All types and their instances are loaded in to memory. When I load with a predicate however, I get the following error. The curious thing is that on windows, the error does not occur and the predicate works fine. On Linux though, it fails. What could possibly be going wrong here?
The cache load code. ServiceStatus and ServiceMetric are just two of my pojo types backed by my database.
log.info("Local load cache started.");
cache.localLoadCache(new IgniteBiPredicate() {
#Override
public boolean apply(Object key, Object value) {
// include by default, exclude explicitly
// no service status before today
if (value instanceof ServiceStatus) {
if (((ServiceStatus)value).getLastUpdated().before(Timestamp.valueOf(LocalDate.now().atStartOfDay()))){
return false;
}
}
// no service metrics before today
if (value instanceof ServiceMetric) {
if (((ServiceMetric)value).getLastUpdated().before(Timestamp.valueOf(LocalDate.now().atStartOfDay()))){
return false;
}
}
return true;
}
}, null);
log.info("Local load cache finished.");
The error.
[ERROR] 2017-01-30 13:01:28.913 [main] ServiceGrid - Failed with exception.
javax.cache.integration.CacheLoaderException: Failed to load cache: D9Cache
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.loadCache(CacheAbstractJdbcStore.java:847) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadCache(GridCacheStoreManagerAdapter.java:512) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.localLoadCache(GridDhtCacheAdapter.java:497) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.localLoadCache(IgniteCacheProxy.java:413) ~[ignite-core-1.8.0.jar:1.8.0]
at com.nmf.model.persistence.CacheUtil.loadCache(CacheUtil.java:87) ~[shared-data-model-1.0-SNAPSHOT.jar:?]
at com.nmf.grid.ServiceGrid.main(ServiceGrid.java:75) [dragon9-service-grid-1.0-SNAPSHOT.jar:?]
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed resolve class for ID: 292145121
at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:696) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1491) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1450) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:637) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.loadEntry(GridDhtCacheAdapter.java:528) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.access$300(GridDhtCacheAdapter.java:94) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$4.apply(GridDhtCacheAdapter.java:501) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$4.apply(GridDhtCacheAdapter.java:497) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter$3.apply(GridCacheStoreManagerAdapter.java:528) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore$1.call(CacheAbstractJdbcStore.java:462) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore$1.call(CacheAbstractJdbcStore.java:429) ~[ignite-core-1.8.0.jar:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_111]
Caused by: org.apache.ignite.IgniteCheckedException: Class definition was not found at marshaller cache and local file. [id=292145121, file=/tmp/ignite/work/marshaller/292145121.classname]
at org.apache.ignite.internal.MarshallerContextImpl.className(MarshallerContextImpl.java:218) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:174) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:680) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1491) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1450) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:637) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.loadEntry(GridDhtCacheAdapter.java:528) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.access$300(GridDhtCacheAdapter.java:94) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$4.apply(GridDhtCacheAdapter.java:501) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter$4.apply(GridDhtCacheAdapter.java:497) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter$3.apply(GridCacheStoreManagerAdapter.java:528) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore$1.call(CacheAbstractJdbcStore.java:462) ~[ignite-core-1.8.0.jar:1.8.0]
at org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore$1.call(CacheAbstractJdbcStore.java:429) ~[ignite-core-1.8.0.jar:1.8.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_111]

Answer 1

Without additional information, it's hard to advise but you can check if it is reproducible with cache.withKeepBinary(). Also, try to bounce the whole cluster. It's strange that the case stands only for Linux.

Answer 2

I have also seen this issue which (in the interests of time) led me away from using predicates at all. On loading the cache when using predicates I would sometimes receive the exception you have noted above. I observed this on both Linux and windows, however it was not consistent. Sometimes the cache would load, sometimes not. As I said i was pushed for time so went with the assumption it was a race condition somewhere in the Ignite code that's running the predicate and worked around the issue.
If any Ignite experts can shed further light on a potential source of the issue I would be happy to help where possible.
--UPDATE --
I've worked around the issue by "warming" the binary marshaller before attempting to load date from the backing store (which was resulting in the file not found error)
So what I do is along the following lines:
IgniteCache cache = ignite.getOrCreateCache("my-cache");
warmBinaryMarshaller(cache);
cache.localLoadCache(cache, new MyPredicate(), (Object[])null);
...
private static void warmBinaryMarshaller(IgniteCache cache) throws Exception
{
final IgniteCache transientCache = cache.withSkipStore();
try{
CacheKey key = new CacheKey();
CacheEntity entity = CacheEntity();
key.setId(0);
entity.setId(0);
transientCache.put(key, entity);
transientCache.remove(key);
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
...
I can then load the object from my database successfully