Arquillian Does Not Execute More Than 4 Tests - jboss-arquillian

Question

This one is a bit of a headscratcher: I have an Arquillian test class with 8 tests. If I execute 4 of them at once, all of them work, but if I add even one other test, the last one fails. If I execute all 8 of them, the last 4 fail. If I change the the order, still the last 4 fail. There is no rhyme nor reason to which ones get executed and which one fails besides that.
(The same holds true if I put half of the tests in another class. Then both classes execute fine on their own, but if they are started as part of the same run configuration, the second one will fail all its tests.)
The error occurs inside the #Before method, so here it is:
#PersistenceContext
private EntityManager em;
#Inject
private UserTransaction transaction;
#Before
public void setUp() throws Exception {
Assert.assertNotNull("EntityManager cannot be null!", this.em);
this.transaction.begin();
this.em.createNativeQuery(MY_SCRIPT_1).executeUpdate();
this.em.createNativeQuery(MY_SCRIPT_2).executeUpdate();
this.transaction.commit();
}
Where MY_SCRIPT_X is a bunch of code, but the relevant portion might be:
-- MY_SCRIPT_1
DROP SCHEMA IF EXISTS scheme1 CASCADE;
DROP SCHEMA IF EXISTS scheme2 CASCADE;
DROP SCHEMA IF EXISTS public CASCADE;
CREATE SCHEMA public;
-- MY_SCRIPT_2
CREATE TABLE scheme1.dbconfig (
propkey character varying NOT NULL,
propvalue character varying NOT NULL,
CONSTRAINT dbconfig_pkey PRIMARY KEY (propkey)
);
COMMENT ON TABLE scheme1.dbconfig IS 'global database configuration properties';
INSERT INTO scheme1.dbconfig VALUES
('initialized', true),
('version', 2);
The error does not occur if I use only one big script [em.createNativeQuery(MY_SCRIPT_1 + MY_SCRIPT_2)], even though the table scheme1.dbconfig is only referenced in one of the files.
All I get as an error message is:
09:20:18,243 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (default task-24) ERROR: relation "scheme1.dbconfig" does not exist
Position: 169
I have no idea where position 169 is (ignoring all comments, the 169th position should be between the R and the O of propkey, which seems odd). But I checked the database after executing the entire class and getting the above error, and the table scheme1.dbconfig does indeed exist.
I know, it's an oddly specific problem, but does anybody what the problem might be (or even how to debug it further)?
Some more infos:
database: PostgreSQL
Hibernate: 4.3.5
container: Wildfly 8.1.0.Final
And all it needs for my to reproduce the bug is (the beans.xml is an default empty file, and the persistence.xml has only the jta-data-source):
#RunWith(Arquillian.class)
public class BugTest {
private static final String CLEANUP_SCRIPT = "DROP SCHEMA IF EXISTS scheme1 CASCADE; "
+ "DROP SCHEMA IF EXISTS public CASCADE; "
+ "CREATE SCHEMA public AUTHORIZATION testdba;";
private static final String CREATE_SCRIPT = "CREATE SCHEMA scheme1; "
+ "CREATE TABLE scheme1.dbconfig ( propkey character varying NOT NULL, propvalue character varying NOT NULL, CONSTRAINT dbconfig_pkey PRIMARY KEY (propkey) ); "
+ "INSERT INTO scheme1.dbconfig VALUES ('initialized', true), ('version', 2);";
#Deployment
public static Archive<?> createDeployment() {
return ShrinkWrap.create(JavaArchive.class).addAsManifestResource("META-INF/beans.xml", "beans.xml")
.addAsManifestResource("META-INF/persistence.xml", "persistence.xml");
}
#PersistenceContext
private EntityManager em;
#Inject
private UserTransaction transaction;
#Before
public void setUp() throws Exception {
this.transaction.begin();
this.em.createNativeQuery(CLEANUP_SCRIPT).executeUpdate();
this.em.createNativeQuery(CREATE_SCRIPT).executeUpdate();
this.transaction.commit();
}
// since an exception gets thrown in the setUp() method, these are not called
// they are only needed because the exception gets thrown for more than
// four tests methods
#Test public void test1() { }
#Test public void test2() { }
#Test public void test3() { }
#Test public void test4() { }
#Test public void test5() { }
#Test public void test6() { }
#Test public void test7() { }
#Test public void test8() { }
}

Answer 1

I ran over that problem again and again and again in the previous two years. The only thing that reliable works are the following:
using one script instead of two; in this case, I'll concatenate the two files before executing
not initializing every time; I'll test if one of the schemata is present and hope and pray that the previous tests left the database in an usable condition (good luck debugging using this one)

Answer 2

No, Arquillian will run as many tests as you need in a single file, or at least until JUnit (or TestNG) stop running. Here are some examples from Apache DeltaSpike with more than four tests running:
https://github.com/apache/deltaspike/blob/master/deltaspike/core/impl/src/test/java/org/apache/deltaspike/test/core/impl/exception/control/handler/CallingHandlersTest.java
https://github.com/apache/deltaspike/blob/master/deltaspike/modules/data/impl/src/test/java/org/apache/deltaspike/data/impl/handler/EntityRepositoryHandlerTest.java
https://github.com/apache/deltaspike/blob/master/deltaspike/core/impl/src/test/java/org/apache/deltaspike/test/core/api/message/MessageContextTest.java
You must have something in your setup that is causing the problem.