HSQLDB batch delete failed but in Oracle it works - sql

I have a setup with : Spring+Hibernate+JPA+Junit to test my code.
I have Oracle as database. When I test my code every test passed. This code is even in production.I wanted to use HSQLDB in memory for the junit instead of Oracle.Most of the tests still passed with HSQLDB, but one particular test always fail with HQSQLDB. (Works fin in Oracle).
here the line that cause trouble. (I used log4jdbc to output the sql statements).
creating batch for two statements:
1: delete from IMAGE_TAGS where IMAGE_TAGS_KEY=1
2: delete from IMAGE_TAGS where IMAGE_TAGS_KEY=2
in JPA, I'm looking on entity to delete them. Works fine on the first iteration, but when I arrive to the second item, it fail with this exception :
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:85)
at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:70)
at org.hibernate.jdbc.BatchingBatcher.checkRowCounts(BatchingBatcher.java:90)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:114)
at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:109)
at org.hibernate.jdbc.AbstractBatcher.prepareBatchStatement(AbstractBatcher.java:244)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2666)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:2911)
at org.hibernate.action.EntityDeleteAction.execute(EntityDeleteAction.java:97)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:189)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1185)
at org.hibernate.impl.SessionImpl.executeUpdate(SessionImpl.java:1283)
at org.hibernate.impl.QueryImpl.executeUpdate(QueryImpl.java:117)
at com.videotron.imagemanager.dao.ImageContentGroupDAOImpl.removeImageContentGroup(ImageContentGroupDAOImpl.java:143)
at com.videotron.imagemanager.dao.ImageContentGroupDAOImpl$$FastClassByCGLIB$$8ef3e206.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:627)
at com.videotron.imagemanager.dao.ImageContentGroupDAOImpl$$EnhancerByCGLIB$$a153b522.removeImageContentGroup(<generated>)
at com.videotron.imagemanager.service.ImageManagerServiceImpl.cleanContent(ImageManagerServiceImpl.java:270)
at com.videotron.imagemanager.service.ImageManagerServiceImpl$$FastClassByCGLIB$$82a03251.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:627)
at com.videotron.imagemanager.service.ImageManagerServiceImpl$$EnhancerByCGLIB$$1a5a3f0d.cleanContent(<generated>)
at com.videotron.imagemanager.image.impl.v1.ImageManagerImpl.cleanContent(ImageManagerImpl.java:1454)
at com.videotron.imagemanager.image.impl.v1.ImageManagerImpl$$FastClassByCGLIB$$4ded3863.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)
at com.videotron.imagemanager.image.impl.v1.ImageManagerImpl$$EnhancerByCGLIB$$c6ce4df.cleanContent(<generated>)
at com.videotron.imagemanager.service.ImageManagerServiceFacade.cleanContent(ImageManagerServiceFacade.java:350)
at com.videotron.imagemanager.service.ImageManagerServiceFacade$$FastClassByCGLIB$$558f388b.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:627)
at com.videotron.imagemanager.service.ImageManagerServiceFacade$$EnhancerByCGLIB$$fb734807.cleanContent(<generated>)
at com.videotron.imagemanager.service.ImageManagerServiceFacadeTest.testCleanContent(ImageManagerServiceFacadeTest.java:671)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:88)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

Cause of exception org.hibernate.StaleStateException is mainly because of
"Hibernate caches objects from the session. If the object was
modified, and Hibernate doesn’t know about it, it will throw this
exception".
Or
Flushing the data before committing the object may lead to clear all
object pending for persist.
I will suggest set log level of spring and log4jdbc to debug and enable show_sql of hibernate this willl gives you clear idea.
Update
As on first look i Think that:
On First iteration when you say Delete IMAGE_TAGS_KEY 1 and 2 hibernate fires this query so that you are getting this query logged by log4jdbc but still transaction is not commited so the changes is still not written in DB and on second iteration again you are deleting the same keys which is pending to get reflected.

Related

Why do I get "File could only be replicated to 0 nodes" when writing to a partitioned table?

I create an external table in Hive with partitions and then try to populate it from the existing table, however, I get the following exceptions:
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /apps/hive/warehouse/pavel.db/browserdatapart/.hive-staging_hive_2018-12-28_13-22-45_751_6056004898772238481-1/_task_tmp.-ext-10000/cityid=1/_tmp.000001_3 could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1719)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3372)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3296)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:850)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:504)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:814)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:133)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:170)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:555)
... 18 more
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /apps/hive/warehouse/pavel.db/browserdatapart/.hive-staging_hive_2018-12-28_13-22-45_751_6056004898772238481-1/_task_tmp.-ext-10000/cityid=1/_tmp.000001_3 could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1719)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3372)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3296)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:850)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:504)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554)
at org.apache.hadoop.ipc.Client.call(Client.java:1498)
at org.apache.hadoop.ipc.Client.call(Client.java:1398)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:459)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:290)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:202)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:184)
at com.sun.proxy.$Proxy12.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1580)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1375)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:
According to the internet these exceptions occur when datanode can't communicate with namenode or when you are running low on memory, but in my case everything is fine. I have already tried formatting my namenode and datanode as well. What else could be the issue?
https://wiki.apache.org/hadoop/CouldOnlyBeReplicatedTo I've also read this. And it didn't help me.
I am running on tez. this works:
insert into table browserdatapart partition(cityid) select UserAgent,cityid from browserdata limit 100;
And this fails with the exception I provided:
insert into table browserdatapart partition(cityid) select UserAgent,cityid from browserdata;
SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000;
Setting the above parameters solved it for me. I guess hive was not able to replicate data to those partitions that show up in the exception, because there were more than the maximum (which is 224 in the case of my dataset).

Spark 2.2 toPandas() is returning Py4JError

Currently refactoring some PySpark code and this specific snippet is causing me issues whereas it has ran fine before:
zip_table=spark.read.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").schema(schemas.zip_schema).load(path_to_file,header=True)
zip_pandas = zip_table.toPandas()
running into:
Py4JJavaError: An error occurred while calling o167.get.
: java.util.NoSuchElementException: spark.sql.execution.pandas.respectSessionTimeZone
at org.apache.spark.sql.internal.SQLConf$$anonfun$getConfString$2.apply(SQLConf.scala:1175)
at org.apache.spark.sql.internal.SQLConf$$anonfun$getConfString$2.apply(SQLConf.scala:1175)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.internal.SQLConf.getConfString(SQLConf.scala:1175)
at org.apache.spark.sql.RuntimeConfig.get(RuntimeConfig.scala:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
Other Spark Dataframes are successfully able to call toPandas() while others such as this return the same error.
The file in question is not particularly big and should be able to fit in the driver no problem. Also, I know it may be better to just load it into Pandas directly and not convert from Spark, but there is further logic in the code that requires both Spark/Pandas dataframes (and a later re-architecture of this work will address this issue).

EclipseLink static weaved entities breaks remove of an entity operation

I successfully performed static weaving in EclipseLink-2.4.0 with eclipselink-staticweave-maven-plugin 1.0.3 , but when I also run integration tests. going through all layers in the system, they breaks. In startUp() method I create it's created an User entity which it's committed in the test DB. But in tearDown() method where I clear the DB state that entity (already weaved) could not be removed by same EntityManager.
The test goes without errors when entities are not weaved. What do you think the problem could be here ?
Here the exception status also:
java.lang.AbstractMethodError: nl.innovity.youbank.core.entity.user.User._persistence_setCacheKey(Lorg/eclipse/persistence/internal/identitymaps/CacheKey;)V
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.updateCachedAttributes(ObjectBuilder.java:3790)
at org.eclipse.persistence.internal.sessions.MergeManager.mergeChangesOfWorkingCopyIntoOriginal(MergeManager.java:871)
at org.eclipse.persistence.internal.sessions.MergeManager.mergeChangesOfWorkingCopyIntoOriginal(MergeManager.java:687)
at org.eclipse.persistence.internal.sessions.MergeManager.mergeChanges(MergeManager.java:307)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.mergeChangesIntoParent(UnitOfWorkImpl.java:3260)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.mergeChangesIntoParent(RepeatableWriteUnitOfWork.java:369)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.commitRootUnitOfWork(RepeatableWriteUnitOfWork.java:283)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitAndResume(UnitOfWorkImpl.java:1147)
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commitInternal(EntityTransactionImpl.java:84)
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:63)
at nl.innovity.youbank.core.dao.DBManager.reinitDB(DBManager.java:510)
at nl.innovity.youbank.core.integration.IntegrationTest.tearDown(IntegrationTest.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:35)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:146)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at $Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:145)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:87)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
Thanks in advance,
Simeon
It looks like a problem with the configuratoin of the maven static weaving - it is using a different EclipseLink version than 2.4 version you are running against. Looking at the doc you linked, it is set by default to use EclipseLink 2.3.2: I believe EclipseLink 2.4 changed to add the _persistence_setCacheKey method to the PersistenceEntity used in weaving. This method will be missing if you weaving using EclipseLink 2.3.2.
You will need to change the dependencies list to point to EclipseLink 2.4 as the doc you linked to described (where it refers to using 2.3.X versions other than 2.3.2).

No X11 DISPLAY variable was set, but this program performed an operation which requires it

I have some tests that recently failed with the following reason: No X11 DISPLAY variable was set, but this program performed an operation which requires it.
Here's the complete stack trace:
testGetDialog(simple.marauroa.application.core.IAddApplicationDialogProviderTest) Time elapsed: 112 sec <<< ERROR!
java.awt.HeadlessException:
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
at java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:159)
at java.awt.Window.<init>(Window.java:431)
at java.awt.Frame.<init>(Frame.java:403)
at java.awt.Frame.<init>(Frame.java:368)
at javax.swing.SwingUtilities$SharedOwnerFrame.<init>(SwingUtilities.java:1733)
at javax.swing.SwingUtilities.getSharedOwnerFrame(SwingUtilities.java:1810)
at javax.swing.JDialog.<init>(JDialog.java:253)
at javax.swing.JDialog.<init>(JDialog.java:187)
at javax.swing.JDialog.<init>(JDialog.java:135)
at simple.marauroa.application.core.IAddApplicationDialogProviderTest$IAddApplicationDialogProviderImpl.getDialog(IAddApplicationDialogProviderTest.java:97)
at simple.marauroa.application.core.IAddApplicationDialogProviderTest.testGetDialog(IAddApplicationDialogProviderTest.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
In the mentioned line of code is just this:
return new JDialog();
I guess is related to the Jenkins environment not having something enabled. How can this get fixed?
I can disable that test, but why should I?
You need to run your build inside a virtual (headless) graphical environment, using XVNC (or XVFB). See wiki.cloudbees.com/bin/view/DEV/Testing+GUI+applications.
Edit 20/01/2016:
The link above no longer takes you to a page with relevant information. This link shows the page as it was at the time: https://web.archive.org/web/20120717015714/http://wiki.cloudbees.com/bin/view/DEV/Testing+GUI+applications
You might also consider refactoring your code a bit to use a mock service so that tests do not need to load AWT. This will likely make tests faster and more reliable, as well as minimizing the chance that they will fail in novel environments such as a CI server.

Deploying Liferay on Cloudbees

I'd like to deploy Liferay on Cloudbees. I found this blog '15 minutes to liferay on cloudbees paas' with instructions on how to do it. I followed all those instructions, and the actual deployment seemed to go fine. There are no error messages in the logs either. But when I click the link that should lead to my site (with Liferay on it), I get the message that the resource isn't available. Does anyone have any clues what I might have done wrong?
My own guess is that I don't have the right values in portal-ext.properties, so here it is:
jdbc.default.driverClassName=com.cloudbees.jdbc.Driver
jdbc.default.url=jdbc:cloudbees://liferay_db/liferay_db?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
jdbc.default.username=MyUserName
jdbc.default.password=MyPassword
#lucene.store.type=ram
#lucene.store.jdbc.auto.clean.up.enabled=true
[update]
I have now tried the Liferay Clickstart. Now I get a '502 Bad gateway' when I access the site. But at least this time there are error messages in the logs:
19:17:03,413 ERROR [main][JDBCExceptionReporter:76] Table 'liferay-po-12.portalpreferences' doesn't exist
19:17:03,565 ERROR [main][FileChecker:123]
com.liferay.portal.kernel.exception.SystemException:
com.liferay.portal.kernel.dao.orm.ORMException:
org.hibernate.exception.SQLGrammarException: could not execute query
com.liferay.portal.kernel.exception.SystemException:
com.liferay.portal.kernel.dao.orm.ORMException:
org.hibernate.exception.SQLGrammarException: could not execute query
at
com.liferay.portal.service.persistence.impl.BasePersistenceImpl.processException(BasePersistenceImpl.java:193)
at com.liferay.portal.service.persistence.PortalPreferencesPersistenceImpl.fetchByO_O(PortalPreferencesPersistenceImpl.java:600)
at com.liferay.portal.service.persistence.PortalPreferencesPersistenceImpl.fetchByO_O(PortalPreferencesPersistenceImpl.java:519)
at com.liferay.portal.service.impl.PortalPreferencesLocalServiceImpl.doGetPreferences(PortalPreferencesLocalServiceImpl.java:170)
at com.liferay.portal.service.impl.PortalPreferencesLocalServiceImpl.getPreferences(PortalPreferencesLocalServiceImpl.java:101)
at com.liferay.portal.service.impl.PortalPreferencesLocalServiceImpl.getPreferences(PortalPreferencesLocalServiceImpl.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:122)
at com.liferay.portal.spring.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:71)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:57)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:57)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:57)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ChainableMethodAdvice.invoke(ChainableMethodAdvice.java:57)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.security.pacl.PACLAdvice.invoke(PACLAdvice.java:51)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:211)
at $Proxy67.getPreferences(Unknown Source)
at com.liferay.portal.service.PortalPreferencesLocalServiceUtil.getPreferences(PortalPreferencesLocalServiceUtil.java:282)
at com.liferay.portal.util.PrefsPropsUtil.getPreferences(PrefsPropsUtil.java:250)
at com.liferay.portal.util.PrefsPropsUtil.getPreferences(PrefsPropsUtil.java:241)
at com.liferay.portal.util.PrefsPropsUtil.getString(PrefsPropsUtil.java:448)
at com.liferay.portal.deploy.DeployUtil.getAutoDeployDestDir(DeployUtil.java:64)
at com.liferay.portal.deploy.DeployManagerImpl.getInstalledDir(DeployManagerImpl.java:60)
at com.liferay.portal.kernel.deploy.DeployManagerUtil.getInstalledDir(DeployManagerUtil.java:48)
at com.liferay.portal.security.pacl.checker.FileChecker.afterPropertiesSet(FileChecker.java:119)
at com.liferay.portal.security.pacl.BasePACLPolicy.initChecker(BasePACLPolicy.java:111)
at com.liferay.portal.security.pacl.BasePACLPolicy.initCheckers(BasePACLPolicy.java:146)
at com.liferay.portal.security.pacl.BasePACLPolicy.<init>(BasePACLPolicy.java:48)
at com.liferay.portal.security.pacl.InactivePACLPolicy.<init>(InactivePACLPolicy.java:32)
at com.liferay.portal.security.pacl.PACLPolicyManager.<clinit>(PACLPolicyManager.java:200)
at com.liferay.portal.security.pacl.PACLAdvice.invoke(PACLAdvice.java:47)
at com.liferay.portal.spring.aop.ServiceBeanMethodInvocation.proceed(ServiceBeanMethodInvocation.java:118)
at com.liferay.portal.spring.aop.ServiceBeanAopProxy.invoke(ServiceBeanAopProxy.java:211)
at $Proxy48.clear(Unknown Source)
at com.liferay.portal.service.LockLocalServiceUtil.clear(LockLocalServiceUtil.java:267)
at com.liferay.portal.events.StartupAction.doRun(StartupAction.java:75)
at com.liferay.portal.events.StartupAction.run(StartupAction.java:52)
at com.liferay.portal.servlet.MainServlet.processStartupEvents(MainServlet.java:1306)
at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:214)
at javax.servlet.GenericServlet.init(GenericServlet.java:212)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1206)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1026)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4421)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4734)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.startup.Embedded.start(Embedded.java:825)
at com.staxnet.appserver.TomcatServerBase.startContainer(TomcatServerBase.java:120)
at com.staxnet.appserver.TomcatServerBase.start(TomcatServerBase.java:190)
at com.staxnet.appserver.StaxAppServer.main(StaxAppServer.java:89)
at com.staxnet.appserver.SnazAppServer.main(SnazAppServer.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at net.stax.appserver.bootstrap.Bootstrap.invokeAppServerMain(Bootstrap.java:41)
at net.stax.appserver.bootstrap.Bootstrap.main(Bootstrap.java:30)
Caused by: com.liferay.portal.kernel.dao.orm.ORMException:
org.hibernate.exception.SQLGrammarException: could not execute query
at com.liferay.portal.dao.orm.hibernate.ExceptionTranslator.translate(ExceptionTranslator.java:30)
at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:98)
at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:75)
at com.liferay.portal.service.persistence.PortalPreferencesPersistenceImpl.fetchByO_O(PortalPreferencesPersistenceImpl.java:575)
... 65 more
Caused by: org.hibernate.exception.SQLGrammarException: could not execute query
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:92)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.loader.Loader.doList(Loader.java:2545)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
at org.hibernate.loader.Loader.list(Loader.java:2271)
at org.hibernate.hql.classic.QueryTranslatorImpl.list(QueryTranslatorImpl.java:940)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1268)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:86)
... 67 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'liferay-po-12.portalpreferences' doesn't exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:407)
at com.mysql.jdbc.Util.getInstance(Util.java:382)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3603)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3535)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1989)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2150)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2626)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2281)
at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1953)
at org.hibernate.loader.Loader.doQuery(Loader.java:802)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
at org.hibernate.loader.Loader.doList(Loader.java:2542)
... 74 more
19:17:04,035 ERROR [main][JDBCExceptionReporter:76] Table 'liferay-po-12.lock_' doesn't exist
There was also a message about a log file being non-existing:
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /tmp/liferay/logs/liferay.2012-09-17.log (Permission denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.rolling.RollingFileAppender.activateOptions(RollingFileAppender.java:179)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:191)
at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:523)
at org.apache.log4j.xml.DOMConfigurator.parseRoot(DOMConfigurator.java:492)
at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:1001)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:867)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:815)
at com.liferay.util.log4j.Log4JUtil.configureLog4J(Log4JUtil.java:79)
at com.liferay.util.log4j.Log4JUtil.configureLog4J(Log4JUtil.java:57)
at com.liferay.portal.util.InitUtil.init(InitUtil.java:99)
at com.liferay.portal.spring.context.PortalContextLoaderListener.contextInitialized(PortalContextLoaderListener.java:149)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4705)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.startup.Embedded.start(Embedded.java:825)
at com.staxnet.appserver.TomcatServerBase.startContainer(TomcatServerBase.java:120)
at com.staxnet.appserver.TomcatServerBase.start(TomcatServerBase.java:190)
at com.staxnet.appserver.StaxAppServer.main(StaxAppServer.java:89)
at com.staxnet.appserver.SnazAppServer.main(SnazAppServer.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at net.stax.appserver.bootstrap.Bootstrap.invokeAppServerMain(Bootstrap.java:41)
at net.stax.appserver.bootstrap.Bootstrap.main(Bootstrap.java:30)
[/update]
I recommend using the newly created ClickStart for Liferay, which you can find here:
https://github.com/CloudBees-community/liferay-clickstart
You would have to click the "Deploy instantly on CloudBees" button, and there are two modifications to be done before it can run properly (And maybe those were the reason why it didn't work originally for you, as Tomcat will return "resource not available" if it runs out of PermGen space)
Set a bigger PermGen space with:
bees app:update ACCOUNT/APP_NAME jvmPermSize=256
Change the size of your application with the CloudBees web management console to the recommended 1024MB for Liferay.