We switched from 2.4.0 to 2.7.0. What might cause this? I can't rule out a problem in our code but we are not in the stack.
22:39:36.270 [mgmt-#66] ERROR o.a.i.i.p.task.GridTaskWorker - Failed
to obtain remote job result policy for result from
ComputeTask.result(..) method (will fail the whole task):
GridJobResultImpl [job=C2 [c=LoadCacheJobV2 [keepBinary=false]],
sib=GridJobSiblingImpl
[sesId=6461c7cd961-d4d7605d-33c7-4941-84bc-f6eca074593f,
jobId=7461c7cd961-d4d7605d-33c7-4941-84bc-f6eca074593f,
nodeId=503c524c-c53a-4f98-aadd-4c95ac2b168b, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=7461c7cd961-d4d7605d-33c7-4941-84bc-f6eca074593f,
timeoutObj=null, attrs={}], node=TcpDiscoveryNode
[id=503c524c-c53a-4f98-aadd-4c95ac2b168b, addrs=[127.0.0.1,
172.17.0.2], sockAddrs=[/127.0.0.1:47500, /
172.17.0.2:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1554182314332, loc=false,
ver=2.4.0#20180305-sha1:aa342270, isClient=false], ex=class
o.a.i.IgniteException: null, hasRes=true, isCancelled=false,
isOccupied=true] org.apache.ignite.IgniteException: Remote job threw
user exception (override or implement ComputeTask.result(..) method if
you would like to have automatic failover for this exception).
at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047)
at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6655)
at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040)
at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858)
at org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1077)
at org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1312)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.ignite.IgniteException: null
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1858)
at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6623)
at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1123)
at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1921)
... 7 common frames omitted
Caused by: org.apache.ignite.IgniteException: null
at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:980)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJob.localExecute(GridCacheAdapter.java:5525)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJobV2.localExecute(GridCacheAdapter.java:5569)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$TopologyVersionAwareJob.execute(GridCacheAdapter.java:6184)
at org.apache.ignite.compute.ComputeJobAdapter.call(ComputeJobAdapter.java:132)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855)
... 14 common frames omitted
Caused by: org.apache.ignite.IgniteCheckedException: null
at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7244)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.sessionEnd0(GridCacheStoreManagerAdapter.java:943)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadCache(GridCacheStoreManagerAdapter.java:549)
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.localLoadCache(GridDhtCacheAdapter.java:608)
at org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.localLoadCache(GridCacheProxyImpl.java:217)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJob.localExecute(GridCacheAdapter.java:5520)
...18 common frames omitted Caused by: java.lang.NullPointerException: null
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter$SessionData.access$900(GridCacheStoreManagerAdapter.java:964)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.sessionEnd0(GridCacheStoreManagerAdapter.java:937)
...22 common frames omitted
22:39:36.293 [pool-3-thread-1]
ERROR c.b.a.c.c.i.cache.CeresCacheManager - There was an issue in loading data for cache ceres-daily-2019-02-27. Issue: ()
java.lang.NullPointerException: null
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter$SessionData.access$900(GridCacheStoreManagerAdapter.java:964)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.sessionEnd0(GridCacheStoreManagerAdapter.java:937)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadCache(GridCacheStoreManagerAdapter.java:549)
at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.localLoadCache(GridDhtCacheAdapter.java:608)
at org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.localLoadCache(GridCacheProxyImpl.java:217)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJob.localExecute(GridCacheAdapter.java:5520)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$LoadCacheJobV2.localExecute(GridCacheAdapter.java:5569)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$TopologyVersionAwareJob.execute(GridCacheAdapter.java:6184)
at org.apache.ignite.compute.ComputeJobAdapter.call(ComputeJobAdapter.java:132)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855)
at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6623)
at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1123)
at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1921)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
public class XYZLoader
extends CacheLoadOnlyStoreAdapter<XYZKey, byte[], XYZRecord>
implements ComputeJob, Serializable {
public XYZLoader(...) {
reconfigureCacheLoadOnlyStoreAdapter(gridNodePhysicalThreadCount, externalParallelism);
}
private void reconfigureCacheLoadOnlyStoreAdapter(int gridNodePhysicalThreadCount, int externalParallelism) {
int physicalThreadCount = ...
this.setBatchSize(...);
this.setBatchQueueSize(...);
this.setThreadsCount(...);
}
int[] getLocalPartitions(Ignite ignite) {
//...
return ...;
}
#Override
protected Iterator<XYZRecord> inputIterator(#Nullable Object... args) throws CacheLoaderException {
int[] parts = getLocalPartitions(ignite);
IgniteCluster cluster = ignite.cluster();
ClusterNode localNode = cluster.localNode();
int partitionCount = ignite.affinity(dataloadDescriptor.cacheName).partitions();
Iterator<XYZRecord> reader = new XYZReader(parts, ...);
iteratorFinishedFlag = false;
return new Iterator<XYZRecord>() {
// wrap inner iterator reader here..
};
}
protected IgniteBiTuple<XYZKey, byte[]> parse(XYZRecord rec, #Nullable Object... args) {
// parsing foo...
return record;
}
#IgniteInstanceResource
public void setIgnite(Ignite ignite) {
this.ignite = ignite;
}
#Override // ComputeJob
public void cancel() {
throw new RuntimeException("Not implemented");
}
#Override // ComputeJob
public Object execute() throws IgniteException {
throw new RuntimeException("Not implemented");
}
#TaskSessionResource
public void setTaskSession(ComputeTaskSession taskSes) {
ComputeTaskSession ses = taskSes;
logger.info(... ses);
}
}
My bet that you forgot to call super.method() somewhere in your CacheStore implementation, leading to partial initialization of underlying data structures. In loadCache() or loadFromStore() perhaps.
Related
I have a usecase where my Ignite cache key/value cannot have corresponding Java class, I am using BinaryObject as key and value type. Below code works fine (without any readthrough).
CacheConfiguration cfg = new CacheConfiguration("cache1");
IgniteCache<BinaryObject, BinaryObject> cache1 = ignite.getOrCreateCache(cfg).withKeepBinary();
cache1.put(createBOKey("mykey", "k1"), createBOVal());
cache1.put(createBOKey("mykey", "k2"), createBOVal());
cache1.put(createBOKey("mykey", "k3"), createBOVal());
Object v1 = cache1.get(createBOKey("mykey", "k1"));
System.out.println("**** cache1 key " + v1); // shows value
Object v2 = cache1.get(createBOKey("mykey", "k10"));
System.out.println("**** cache1 key " + v2); // show null
BinaryObject createBOKey(String k, String v) {
BinaryObjectBuilder builder = ignite.binary().builder("key-type1");
builder.setField(k, v);
return builder.build();
}
BinaryObject createBOVal() {
BinaryObjectBuilder builder = ignite.binary().builder("value-type1");
builder.setField(fieldName("cache1", "f1"), "hello");
builder.setField(fieldName("cache1", "f2"), 10.75);
return builder.build();
}
Now as soon as I enable readThrough for this cache it starts failing for cache miss cases.
cfg.setReadThrough(true);
cfg.setCacheLoaderFactory(new Factory<CacheLoader<BinaryObject, BinaryObject>>() {
#Override
public CacheLoader<BinaryObject, BinaryObject> create() {
return new CacheLoader<BinaryObject, BinaryObject>() {
#Override
public BinaryObject load(BinaryObject obj) throws CacheLoaderException {
System.out.println("**** Loading from backingstore " + obj);
return null;
}
#Override
public Map<BinaryObject, BinaryObject> loadAll(Iterable<? extends BinaryObject> arg0) throws CacheLoaderException {
return null;
}
};
}
});
On cache miss it throw below exceptions and not even reaches cacheloader.load().
Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: key-type1
at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2066)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:1093)
at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:676)
at com.basit.bo.btree.TestBinaryTreeMapKey.main(TestBinaryTreeMapKey.java:74)
Caused by: class org.apache.ignite.IgniteCheckedException: key-type1
at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7507)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:975)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: key-type1
at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:762)
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:792)
at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:176)
at org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:67)
at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:136)
at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1808)
at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1796)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadFromStore(GridCacheStoreManagerAdapter.java:314)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.load(GridCacheStoreManagerAdapter.java:293)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadAllFromStore(GridCacheStoreManagerAdapter.java:434)
at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadAll(GridCacheStoreManagerAdapter.java:400)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2225)
at org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2223)
at org.apache.ignite.internal.processors.cache.GridCacheContext$3.call(GridCacheContext.java:1479)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7005)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
... 4 more
Caused by: java.lang.ClassNotFoundException: key-type1
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8828)
at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:324)
at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:753)
... 22 more
You should also set storeKeepBinary to true.
Can anyone Help?
I want to dynamically display my Database content into TableView in FXML file.
The Code works when I don't have any null Value in any of the column database.
I expected the code should display empty space in TableView of FXML when my database column has a null value.
dateOfIssue.setCellValueFactory(new Callback<CellDataFeatures<ObservableList, String>, ObservableValue<String>>() {
public ObservableValue<String> call(CellDataFeatures<ObservableList, String> param) {
return new SimpleStringProperty(param.getValue().get(2).toString());
}
});
studentID.setCellValueFactory(new Callback<CellDataFeatures<ObservableList, String>, ObservableValue<String>>() {
public ObservableValue<String> call(CellDataFeatures<ObservableList, String> param) {
return new SimpleStringProperty(param.getValue().get(3).toString());
}
});
primary.setCellValueFactory(new Callback<CellDataFeatures<ObservableList, String>, ObservableValue<String>>() {
public ObservableValue<String> call(CellDataFeatures<ObservableList, String> param) {
return new SimpleStringProperty(param.getValue().get(0).toString());
}
});
secondary.setCellValueFactory(new Callback<CellDataFeatures<ObservableList, String>, ObservableValue<String>>() {
public ObservableValue<String> call(CellDataFeatures<ObservableList, String> param) {
return new SimpleStringProperty(param.getValue().get(1).toString());
}
});
I have my secondary column as null at few rows.here is my error.
Exception in thread "JavaFX Application Thread" java.lang.NullPointerException
at library.DatabaseController$4.call(DatabaseController.java:105)
at library.DatabaseController$4.call(DatabaseController.java:103)
at javafx.scene.control.TableColumn.getCellObservableValue(TableColumn.java:578)
at javafx.scene.control.TableColumn.getCellObservableValue(TableColumn.java:563)
at javafx.scene.control.TableCell.updateItem(TableCell.java:644)
at javafx.scene.control.TableCell.indexChanged(TableCell.java:468)
at javafx.scene.control.IndexedCell.updateIndex(IndexedCell.java:116)
at com.sun.javafx.scene.control.skin.TableRowSkinBase.updateCells(TableRowSkinBase.java:533)
at com.sun.javafx.scene.control.skin.TableRowSkinBase.init(TableRowSkinBase.java:147)
at com.sun.javafx.scene.control.skin.TableRowSkin.<init>(TableRowSkin.java:64)
at javafx.scene.control.TableRow.createDefaultSkin(TableRow.java:212)
at javafx.scene.control.Control.impl_processCSS(Control.java:872)
at javafx.scene.Node.processCSS(Node.java:9058)
at javafx.scene.Node.applyCss(Node.java:9155)
at com.sun.javafx.scene.control.skin.VirtualFlow.setCellIndex(VirtualFlow.java:1964)
at com.sun.javafx.scene.control.skin.VirtualFlow.addTrailingCells(VirtualFlow.java:1344)
at com.sun.javafx.scene.control.skin.VirtualFlow.layoutChildren(VirtualFlow.java:1197)
at javafx.scene.Parent.layout(Parent.java:1087)
at javafx.scene.Parent.layout(Parent.java:1093)
at javafx.scene.Parent.layout(Parent.java:1093)
at javafx.scene.Parent.layout(Parent.java:1093)
at javafx.scene.Parent.layout(Parent.java:1093)
at javafx.scene.Scene.doLayoutPass(Scene.java:552)
at javafx.scene.Scene$ScenePulseListener.pulse(Scene.java:2397)
at com.sun.javafx.tk.Toolkit.lambda$runPulse$29(Toolkit.java:398)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.tk.Toolkit.runPulse(Toolkit.java:397)
at com.sun.javafx.tk.Toolkit.firePulse(Toolkit.java:424)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:518)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:498)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(QuantumToolkit.java:491)
at com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$403(QuantumToolkit.java:319)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "JavaFX Application Thread" java.lang.NullPointerException
at library.DatabaseController$4.call(DatabaseController.java:105)
at library.DatabaseController$4.call(DatabaseController.java:103)
at javafx.scene.control.TableColumn.getCellObservableValue(TableColumn.java:578)
at javafx.scene.control.TableColumn.getCellObservableValue(TableColumn.java:563)
at javafx.scene.control.TableCell.updateItem(TableCell.java:644)
at javafx.scene.control.TableCell.indexChanged(TableCell.java:468)
at javafx.scene.control.IndexedCell.updateIndex(IndexedCell.java:116)
at com.sun.javafx.scene.control.skin.TableRowSkinBase.updateCells(TableRowSkinBase.java:533)
at com.sun.javafx.scene.control.skin.TableRowSkinBase.init(TableRowSkinBase.java:147)
at com.sun.javafx.scene.control.skin.TableRowSkin.<init>(TableRowSkin.java:64)
at javafx.scene.control.TableRow.createDefaultSkin(TableRow.java:212)
at javafx.scene.control.Control.impl_processCSS(Control.java:872)
at javafx.scene.Parent.impl_processCSS(Parent.java:1280)
at javafx.scene.Parent.impl_processCSS(Parent.java:1280)
at javafx.scene.Parent.impl_processCSS(Parent.java:1280)
at javafx.scene.Parent.impl_processCSS(Parent.java:1280)
at javafx.scene.control.Control.impl_processCSS(Control.java:868)
at javafx.scene.Node.processCSS(Node.java:9058)
at javafx.scene.Node.processCSS(Node.java:9051)
at javafx.scene.Node.processCSS(Node.java:9051)
at javafx.scene.Node.processCSS(Node.java:9051)
at javafx.scene.Scene.doCSSPass(Scene.java:545)
at javafx.scene.Scene.access$3600(Scene.java:159)
at javafx.scene.Scene$ScenePulseListener.pulse(Scene.java:2392)
at com.sun.javafx.tk.Toolkit.lambda$runPulse$29(Toolkit.java:398)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.tk.Toolkit.runPulse(Toolkit.java:397)
at com.sun.javafx.tk.Toolkit.firePulse(Toolkit.java:424)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:518)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:498)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(QuantumToolkit.java:491)
at com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$403(QuantumToolkit.java:319)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
at java.lang.Thread.run(Thread.java:748)
I assume your issue is at below point. If your value is null, obviously you will get NPE when accessing toString(). Try to do some null checks before calling toString().
param.getValue().get(1).toString()
If one of the ObservableLists contains null, trying to invoke toString for such an object results in the NullPointerException. I recommend using different code to convert to a string, e.g. Objects.toString:
secondary.setCellValueFactory(new Callback<CellDataFeatures<ObservableList, String>, ObservableValue<String>>() {
#Override
public ObservableValue<String> call(CellDataFeatures<ObservableList, String> param) {
return new SimpleStringProperty(Objects.toString(param.getValue().get(1), ""));
}
});
I was playing around with the custom shadow variables and custom variable listeners. I found that the "Exam timetabling" example uses custom shadow variables to update "Following exams" when the "Leading Exam" changes.
So what i did to assert asking in this question was true Custom VariableListener updating more than one shadow variables i added a second shadow variable in the Following Exam class which references the variable "period" , and then in the variable listener of the LeadingExam planning entity(PeriodUpdatingVariableListener) i update both the variables.
Here is the code for the "FollowingExam" class
#PlanningEntity
#XStreamAlias("FollowingExam")
public class FollowingExam extends Exam {
protected LeadingExam leadingExam;
// Shadow variables
protected Period period;
protected Integer var;
#CustomShadowVariable(variableListenerClass = PeriodUpdatingVariableListener.class, sources = {
#CustomShadowVariable.Source(entityClass = LeadingExam.class, variableName = "period") })
public Period getPeriod() {
return period;
}
public void setPeriod(Period period) {
this.period = period;
}
#CustomShadowVariable(variableListenerRef = #PlanningVariableReference(variableName = "period"))
public Integer getVar() {
return var;
}
public void setVar(Integer var) {
this.var = var;
}
public LeadingExam getLeadingExam() {
return leadingExam;
}
public void setLeadingExam(LeadingExam leadingExam) {
this.leadingExam = leadingExam;
}
}
And in the "PeriodUpdatingVariableListener" i have the next code
public class PeriodUpdatingVariableListener implements VariableListener<LeadingExam> {
public void beforeEntityAdded(ScoreDirector scoreDirector, LeadingExam leadingExam) {
// Do nothing
}
public void afterEntityAdded(ScoreDirector scoreDirector, LeadingExam leadingExam) {
updatePeriod(scoreDirector, leadingExam);
}
public void beforeVariableChanged(ScoreDirector scoreDirector, LeadingExam leadingExam) {
// Do nothing
}
public void afterVariableChanged(ScoreDirector scoreDirector, LeadingExam leadingExam) {
updatePeriod(scoreDirector, leadingExam);
}
public void beforeEntityRemoved(ScoreDirector scoreDirector, LeadingExam leadingExam) {
// Do nothing
}
public void afterEntityRemoved(ScoreDirector scoreDirector, LeadingExam leadingExam) {
// Do nothing
}
protected void updatePeriod(ScoreDirector scoreDirector, LeadingExam leadingExam) {
Period period = leadingExam.getPeriod();
for (FollowingExam followingExam : leadingExam.getFollowingExamList()) {
scoreDirector.beforeVariableChanged(followingExam, "period");
followingExam.setPeriod(period);
scoreDirector.afterVariableChanged(followingExam, "period");
//additional lines of code to update the "var" variable
Integer var = followingExam.getVar();
if(var == null){
var = new Integer(1);
}
else var++;
scoreDirector.beforeVariableChanged(followingExam, "var");
followingExam.setVar(var);
scoreDirector.afterVariableChanged(followingExam, "var");
}
}
}
It also seem like that there is no problem for OptaPlanner registering that this is another shadow variable because i get this message when i run the application
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Model annotations parsed for Solution Examination:
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Entity Exam:
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Variable room (genuine)
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Entity LeadingExam:
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Variable period (genuine)
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Entity FollowingExam:
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Variable period (shadow)
2016-04-11 15:26:15,309 [AWT-EventQueue-0] TRACE Variable var (shadow)
The error that i get is the next error , this error only appears when i change the environment mode to FULL_ASSERT ,but when it is left to the default REPRODUCIBLE it runs without any errors.
Caused by: java.lang.IllegalStateException: VariableListener corruption: the entity (426)'s shadow variable (FollowingExam.var)'s corrupted value (null) changed to uncorrupted value (1) after all VariableListeners were triggered without changes to the genuine variables.
Probably the VariableListener class for that shadow variable (FollowingExam.var) forgot to update it when one of its sources changed after completedAction (Initial score calculated).
at org.optaplanner.core.impl.score.director.AbstractScoreDirector.assertShadowVariablesAreNotStale(AbstractScoreDirector.java:349)
at org.optaplanner.core.impl.solver.recaller.BestSolutionRecaller.solvingStarted(BestSolutionRecaller.java:84)
at org.optaplanner.core.impl.solver.DefaultSolver.solvingStarted(DefaultSolver.java:196)
at org.optaplanner.core.impl.solver.DefaultSolver.solve(DefaultSolver.java:175)
at org.optaplanner.examples.common.business.SolutionBusiness.solve(SolutionBusiness.java:307)
at org.optaplanner.examples.common.swingui.SolverAndPersistenceFrame$SolveWorker.doInBackground(SolverAndPersistenceFrame.java:287)
at org.optaplanner.examples.common.swingui.SolverAndPersistenceFrame$SolveWorker.doInBackground(SolverAndPersistenceFrame.java:1)
at javax.swing.SwingWorker$1.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at javax.swing.SwingWorker.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
This code in a variable listener is bad:
Integer var = followingExam.getVar();
...
var++;
...
followingExam.setVar(var);
...
If its called multiple times, the var changes, even if the genuine var hasn't changed. If that var is used in score rules, the solution's score isn't always the same for the same solution.
A shadow variable is the caching of the result of a formula based on at least 1 genuine variable (directly or indirectly). For example shadow C = genuine A + problem property B. So if B is 10 and ranges A from 1 to 5, then C will be 11 if A is 1, 12 if A is 2, etc.
I have a web application where the server returns any results in JSON format. For creating the JSON, I use the codehaus Jackson ObjectWriter, version 1.9.8.
The problem I'm having that sometimes there is an error, in the mapping, and from then on all server calls result in an error. I haven't been able to determine what causes the error, I did discover the origin.
When the exception occurs, the server returns "(was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: com.onior.modm.restlet.helpers.ServiceResult["success"])", which means that the exception thrown in 'toJSON' was catched and mapped to JSON by 'toRepresentation', otherwise it would have returned empty.
I just don't know why it is sometimes failing. I will be able to use the application all morning without problems, and then suddenly I will get this error. It happens on different calls, but these calls will succeed at other times. From my point of view it seems quite random, but maybe someone can help me see the light? :)
The server result that is being mapped:
public class ServiceResult<T> {
private boolean success;
private T results = null;
private ModmServiceStatus message = null;
public ServiceResult() {
}
public ServiceResult(T results) {
this.success = true;
this.results = results;
}
public ServiceResult(boolean success, ModmServiceStatus message) {
this.success = success;
this.message = message;
}
public ServiceResult(ServiceError error) {
this(false, new ModmServiceStatus(error));
}
public static ServiceResult<ModmServiceStatus> serviceException(
ServiceError error) {
return new ServiceResult<ModmServiceStatus>(false,
new ModmServiceStatus(error.getCode(), error.getDescription()));
}
public static ServiceResult<ModmServiceStatus> dbError() {
return ServiceResult
.serviceException(ServiceError.GENERIC_DATABASE_ERROR);
}
public static ServiceResult<ModmServiceStatus> invalidJson() {
return ServiceResult
.serviceException(ServiceError.GENERIC_INVALID_JSON);
}
public static ServiceResult<ModmServiceStatus> missingEntity() {
return ServiceResult .serviceException(ServiceError.GENERIC_MISSING_OR_INCOMPLETE_ENTITY);
}
public static ServiceResult<ModmServiceStatus> entityNotFound() {
return ServiceResult
.serviceException(ServiceError.GENERIC_ENTITY_NOT_FOUND);
}
public static ServiceResult<ModmServiceStatus> entityDeleted(String entity) {
return new ServiceResult<ModmServiceStatus>(true,
new ModmServiceStatus(0, entity + " deleted."));
}
}
The mapping:
public class RestUtils {
private static final boolean PRETTY_PRINT = true;
public static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
public static final ObjectWriter OBJECT_WRITER = (PRETTY_PRINT ? OBJECT_MAPPER
.writerWithDefaultPrettyPrinter() : OBJECT_MAPPER.writer());
#SuppressWarnings("unchecked")
public static <T> JacksonRepresentation<T> toJSON(T t) throws IOException {
JsonRepresentation jsonRepresentation = null;
JacksonRepresentation<T> jacksonRepresentation = null;
jsonRepresentation = new JsonRepresentation(
OBJECT_WRITER.writeValueAsString(t)); // Origin of incidental
// server error
jacksonRepresentation = new JacksonRepresentation<T>(
jsonRepresentation, (Class<T>) t.getClass());
return jacksonRepresentation;
}
public static <T> Representation toRepresentation(ServiceResult<T> ss) {
Representation representation = null;
try {
representation = RestUtils.toJSON(ss);
} catch (IOException jsonException) {
jsonException.printStackTrace();
try {
jsonException.printStackTrace();
representation = RestUtils.toJSON(jsonException.getMessage());
} catch (Exception e) {
e.printStackTrace();
}
}
return representation;
}
}
The call:
RestUtils.toRepresentation(new ServiceResult<List<Group>>(groups));
The exception with stacktrace:
org.codehaus.jackson.map.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: com.onior.modm.restlet.helpers.ServiceResult["success"])
at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:218)
at org.codehaus.jackson.map.JsonMappingException.wrapWithPath(JsonMappingException.java:183)
at org.codehaus.jackson.map.ser.std.SerializerBase.wrapAndThrow(SerializerBase.java:140)
at org.codehaus.jackson.map.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:158)
at org.codehaus.jackson.map.ser.BeanSerializer.serialize(BeanSerializer.java:112)
at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:610)
at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256)
at org.codehaus.jackson.map.ObjectWriter._configAndWriteValue(ObjectWriter.java:456)
at org.codehaus.jackson.map.ObjectWriter.writeValueAsString(ObjectWriter.java:393)
at com.onior.modm.restlet.helpers.RestUtils.toJSON(RestUtils.java:52)
at com.onior.modm.restlet.helpers.RestUtils.toRepresentation(RestUtils.java:71)
at com.onior.modm.restlet.resources.GroupCollectionResource.toJsonRead(GroupCollectionResource.java:191)
at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.restlet.resource.ServerResource.doHandle(ServerResource.java:506)
at org.restlet.resource.ServerResource.get(ServerResource.java:707)
at org.restlet.resource.ServerResource.doHandle(ServerResource.java:589)
at org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:649)
at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:348)
at org.restlet.resource.ServerResource.handle(ServerResource.java:952)
at org.restlet.resource.Finder.handle(Finder.java:246)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Router.doHandle(Router.java:431)
at org.restlet.routing.Router.handle(Router.java:648)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Router.doHandle(Router.java:431)
at org.restlet.routing.Router.handle(Router.java:648)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:155)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.routing.Filter.doHandle(Filter.java:159)
at org.restlet.routing.Filter.handle(Filter.java:206)
at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:211)
at org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:84)
at org.restlet.Application.handle(Application.java:381)
at org.restlet.ext.servlet.ServletAdapter.service(ServletAdapter.java:206)
at org.restlet.ext.spring.RestletFrameworkServlet.doService(RestletFrameworkServlet.java:124)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException
at org.codehaus.jackson.impl.WriterBasedGenerator.writeRaw(WriterBasedGenerator.java:577)
at org.codehaus.jackson.util.DefaultPrettyPrinter$Lf2SpacesIndenter.writeIndentation(DefaultPrettyPrinter.java:279)
at org.codehaus.jackson.util.DefaultPrettyPrinter.beforeObjectEntries(DefaultPrettyPrinter.java:98)
at org.codehaus.jackson.impl.WriterBasedGenerator._writePPFieldName(WriterBasedGenerator.java:410)
at org.codehaus.jackson.impl.WriterBasedGenerator._writeFieldName(WriterBasedGenerator.java:340)
at org.codehaus.jackson.impl.WriterBasedGenerator.writeFieldName(WriterBasedGenerator.java:217)
at org.codehaus.jackson.map.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:444)
at org.codehaus.jackson.map.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:150)
... 58 more
Got the exact same ArrayIndexOutOfBoundsException with Jackson 1.9.9
After investigating, our root cause was a shared PrettyPrinter used by multiple threads.
If you look in the Jackson code for the DefaultPrettyPrinter, you will see:
/**
* Number of open levels of nesting. Used to determine amount of
* indentation to use.
*/
protected int _nesting = 0;
_nesting ends up being used to access arrays. This variable is incremented and decremented when processing object. If multiple threads decrements it, it may end up negative causing the ArrayIndexOutOfBoundsException. Once negative, it will not be ever increased again because the exception will be generated before reaching a piece of code that would increment it.
In your code, I see you have a static object writer. You probably end up with a single instance of the DefaultPrettyPrinter. If your application can concurrently produce json object, given enough time, you will get the exception.
Stéphan
I'm having a tough time figuring out the issue with Serialization in hadoop. Here's my class -
public class CrawlerTweet implements Serializable, Writable {
private static final long serialVersionUID = 1L;
private String keywords;
private List<TweetStatus> tweets;
private long queryTime = 0;
public CrawlerTweet() {}
public CrawlerTweet(String keys, List<TweetStatus> tweets, long queryTime){
this.keywords = keys;
this.tweets = tweets;
this.queryTime = queryTime;
}
public static CrawlerTweet read(DataInput in) throws IOException {
CrawlerTweet ts= new CrawlerTweet();
ts.readFields(in);
return ts;
}
#Override
public void readFields(DataInput din) throws IOException {
queryTime = din.readLong();
keywords = din.readLine();
tweets.clear();
IntWritable size = new IntWritable();
size.readFields(din);
int n = size.get();
while(n -- > 0) {
TweetStatus ts = new TweetStatus();
ts.readFields(din);
tweets.add(ts);
}
}
#Override
public void write(DataOutput dout) throws IOException {
dout.writeChars(keywords);
dout.writeLong(queryTime);
IntWritable size = new IntWritable(tweets.size());
size.write(dout);
for (TweetStatus ts : tweets)
ts.write(dout);
}
public String getKeywords(){
return keywords;
}
public List<TweetStatus> getTweets(){
return tweets;
}
public long getQueryTime(){
return queryTime;
}
}
If I implment both Serizable n WRitable interfaces, I get following exception,
java.lang.ClassNotFoundException: mydat.twitter.dto.CrawlerTweet
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:601)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1572)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1493)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1729)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1326)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)
at focusedCrawler.util.storage.socket.ServerConnectionHandler.buildRequestObject(ServerConnectionHandler.java:136)
at focusedCrawler.util.storage.socket.ServerConnectionHandler.run(ServerConnectionHandler.java:340)
And, if I implment Writable only, I get NotSerializableException -
Erro de comunicacao: bigdat.twitter.dto.CrawlerTweet
Dormindo 5 mls
`[21/JUN/2013:11:23:39] [SocketAdapterFactory] [produce] [hadoop22:3190]
Erro de comunicacao: bigdat.twitter.dto.CrawlerTweet
java.io.NotSerializableException: bigdat.twitter.dto.CrawlerTweet
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at focusedCrawler.util.storage.socket.StorageRemoteAdapter.serializeParamObject(StorageRemoteAdapter.java:113)
at focusedCrawler.util.storage.socket.StorageRemoteAdapter.defaultMethod(StorageRemoteAdapter.java:205)
at focusedCrawler.util.storage.socket.StorageRemoteAdapter.insert(StorageRemoteAdapter.java:289)
at focusedCrawler.util.storage.distribution.StorageRemoteAdapterReconnect.insert(StorageRemoteAdapterReconnect.java:213)
at bigdat.twitter.crawler.CrawlTwitter.download(Unknown Source)
at bigdat.twitter.crawler.CrawlTwitter.run(Unknown Source)
Further information extracted from comments:
CrawlerTweet is packaged in BDAnalytics16.jar file.
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/rgupta/bdAnalytics/lib/*
hadoop jar $jarpath/BDAnalytics16.jar bigdat.twitter.crawler.CrawlTwitter \
$crwlInputFile > $logsFldr/crawler_$1.log 2>&1 &
Help will be much appreciated!
Thx!