Configure ActiveMQ Artemis in WildFly 12 to use a PostgreSQL datastore - postgresql-9.5

I have configured my WildFly 12 instance with a PostgreSQL datastore to use as the journal datastore for ActiveMQ Artemis. However, when WildFly starts up I get the following error:
AMQ222010: Critical IO Error, shutting down the server. file=org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFile#35d3d246, message=Error attempting to open JDBC file.: org.postgresql.util.PSQLException: ERROR: column "data" is of type oid but expression is of type bytea
Hint: You will need to rewrite or cast the expression.
Any idea what I might be doing wrong? My datasource definition is:
<datasource jta="true" jndi-name="java:jboss/datasources/ActiveMQ-DS" pool-name="ActiveMQ-DS" enabled="true" use-ccm="true" spy="false">
<connection-url>jdbc:postgresql://localhost/activemq</connection-url>
<driver-class>org.postgresql.Driver</driver-class>
<driver>postgresql</driver>
<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
<pool>
<min-pool-size>1</min-pool-size>
<max-pool-size>16</max-pool-size>
</pool>
<security>
<user-name>amq</user-name>
<password>amq</password>
</security>
<validation>
<check-valid-connection-sql>select 1</check-valid-connection-sql>
<validate-on-match>false</validate-on-match>
<background-validation>false</background-validation>
<background-validation-millis>300000</background-validation-millis>
</validation>
<statement>
<prepared-statement-cache-size>0</prepared-statement-cache-size>
<share-prepared-statements>false</share-prepared-statements>
</statement>
<timeout>
<idle-timeout-minutes>5</idle-timeout-minutes>
</timeout>
</datasource>
Here is my messaging-activemq subsystem configuration:
<subsystem xmlns="urn:jboss:domain:messaging-activemq:3.0">
<server name="default">
<journal datasource="ActiveMQ-DS" />
<security-setting name="#">
<role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
</security-setting>
<address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10"/>
<http-connector name="http-connector" socket-binding="jms-virtual" endpoint="http-acceptor"/>
<http-connector name="http-connector-throughput" socket-binding="jms-virtual" endpoint="http-acceptor-throughput">
<param name="batch-delay" value="50"/>
</http-connector>
<in-vm-connector name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-connector>
<http-acceptor name="http-acceptor" http-listener="default"/>
<http-acceptor name="http-acceptor-throughput" http-listener="default">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</http-acceptor>
<in-vm-acceptor name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-acceptor>
<jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
<jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
<jms-topic name="VideoVaultSync" entries="jms/topic/VideoVaultSync"/>
<jms-topic name="VICADS-DeviceAlarm" entries="jms/topic/VICADS-DeviceAlarm"/>
<jms-topic name="VICADS-VideoAlarm" entries="jms/topic/VICADS-VideoAlarm java:jboss/exported/jms/topic/VICADS-VideoAlarm"/>
<jms-topic name="VICADS-RadarTarget" entries="jms/topic/VICADS-RadarTarget java:jboss/exported/jms/topic/VICADS-RadarTarget"/>
<jms-topic name="VICADS-AnnunciatorAlarm" entries="jms/topic/VICADS-AnnunciatorAlarm"/>
<jms-topic name="VICADS-Admin" entries="jms/topic/VICADS-Admin java:jboss/exported/jms/topic/VICADS-Admin"/>
<jms-topic name="VICADS-VideoSvc" entries="jms/topic/VICADS-VideoSvc java:jboss/exported/jms/topic/VICADS-VideoSvc"/>
<jms-topic name="VideoVault-SceneAuthentication" entries="jms/topic/VideoVault-SceneAuthentication java:jboss/exported/jms/topic/VideoVault-SceneAuthentication"/>
<jms-topic name="VICADS-Geometry" entries="jms/topic/VICADS-Geometry java:jboss/exported/jms/topic/VICADS-Geometry"/>
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>
<pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
</server>
</subsystem>

Related

Apache Ignite Structured Logging

I am looking to enable structured logging for Ignite.
Ignite runs inside a docker container.
I enabled the log4j2 module and added a log4j2 configuration file that tries to use <JsonTemplateLayout.../> as described here but in the logs i get the message:
Console contains an invalid element or attribute "JsonTemplateLayout"
Which is probably caused by not having the log4j-layout-template-json dependency available inside ignite. Is there a way how to add the dependency to Ignite or is there another option on how to get structured logging working?
Ignite configuration:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
...
<property name="gridLogger">
<bean class="org.apache.ignite.logger.log4j2.Log4J2Logger">
<constructor-arg type="java.lang.String" value="config/ignite-log4j2-custom.xml"/>
</bean>
</property>
</bean>
</beans>
log4j2 configuration:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration monitorInterval="60" status="debug">
<Appenders>
<Console name="CONSOLE" target="SYSTEM_OUT">
<!-- <PatternLayout pattern="[%d{ISO8601}][%-5p][%t][%c{1}]%notEmpty{[%markerSimpleName]} %m%n"/> -->
<ThresholdFilter level="ERROR" onMatch="DENY" onMismatch="ACCEPT"/>
<JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>
</Console>
<Console name="CONSOLE_ERR" target="SYSTEM_ERR">
<!-- <PatternLayout pattern="[%d{ISO8601}][%-5p][%t][%c{1}]%notEmpty{[%markerSimpleName]} %m%n"/> -->
<JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>
</Console>
<File name="CONSISTENCY" fileName="${sys:IGNITE_HOME}/work/log/consistency.log">
<PatternLayout>
<Pattern>"[%d{ISO8601}][%-5p][%t][%c{1}] %m%n"</Pattern>
</PatternLayout>
</File>
<Routing name="FILE">
<Routes pattern="$${sys:nodeId}">
<Route>
<RollingFile name="Rolling-${sys:nodeId}" fileName="${sys:IGNITE_HOME}/work/log/${sys:appId}-${sys:nodeId}.log"
filePattern="${sys:IGNITE_HOME}/work/log/${sys:appId}-${sys:nodeId}-%i-%d{yyyy-MM-dd}.log.gz">
<PatternLayout pattern="[%d{ISO8601}][%-5p][%t][%c{1}]%notEmpty{[%markerSimpleName]} %m%n"/>
<Policies>
<TimeBasedTriggeringPolicy interval="6" modulate="true" />
<SizeBasedTriggeringPolicy size="10 MB" />
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
</Appenders>
<Loggers>
<!-- <Logger name="org.apache.ignite" level="INFO"/> -->
<!--
Uncomment to disable courtesy notices, such as SPI configuration
consistency warnings.
-->
<!--
<Logger name="org.apache.ignite.CourtesyConfigNotice" level=OFF/>
-->
<Logger name="org.springframework" level="WARN"/>
<Logger name="org.eclipse.jetty" level="WARN"/>
<Logger name="org.apache.ignite.internal.visor.consistency" additivity="false" level="INFO">
<AppenderRef ref="CONSISTENCY"/>
</Logger>
<!--
Avoid warnings about failed bind attempt when multiple nodes running on the same host.
-->
<Logger name="org.eclipse.jetty.util.log" level="ERROR"/>
<Logger name="org.eclipse.jetty.util.component" level="ERROR"/>
<Logger name="com.amazonaws" level="WARN"/>
<Root level="INFO">
<!-- Uncomment to enable logging to console. -->
<AppenderRef ref="CONSOLE" level="INFO"/>
<AppenderRef ref="CONSOLE_ERR" level="ERROR"/>
<AppenderRef ref="FILE" level="DEBUG"/>
</Root>
</Loggers>
</Configuration>
When adding the JAR to libs (as suggested by Stanislav below) i get a step further but also get an error (not a java person so any hint is highly appreciated):
main ERROR An exception occurred processing Appender CONSOLE org.apache.logging.log4j.core.appender.AppenderLoggingException: java.lang.IllegalAccessError: class org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder tried to access method 'void org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(java.nio.charset.CharsetEncoder, java.nio.CharBuffer, java.nio.ByteBuffer, java.lang.StringBuilder, org.apache.logging.log4j.core.layout.ByteBufferDestination)' (org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder and org.apache.logging.log4j.core.layout.TextEncoderHelper are in unnamed module of loader 'app')
at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:165)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2017)
at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1983)
at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1275)
at org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:472)
at org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:464)
at org.apache.ignite.internal.GridLoggerProxy.info(GridLoggerProxy.java:137)
at org.apache.ignite.internal.plugin.IgniteLogInfoProviderImpl.ackConfiguration(IgniteLogInfoProviderImpl.java:222)
at org.apache.ignite.internal.plugin.IgniteLogInfoProviderImpl.ackKernalInited(IgniteLogInfoProviderImpl.java:98)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:902)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1799)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1721)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1160)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1054)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:940)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:839)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:709)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)
at org.apache.ignite.Ignition.start(Ignition.java:353)
at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
Caused by: java.lang.IllegalAccessError: class org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder tried to access method 'void org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(java.nio.charset.CharsetEncoder, java.nio.CharBuffer, java.nio.ByteBuffer, java.lang.StringBuilder, org.apache.logging.log4j.core.layout.ByteBufferDestination)' (org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder and org.apache.logging.log4j.core.layout.TextEncoderHelper are in unnamed module of loader 'app')
at org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder.encode(JsonTemplateLayout.java:241)
at org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder.encode(JsonTemplateLayout.java:216)
at org.apache.logging.log4j.layout.template.json.JsonTemplateLayout.encode(JsonTemplateLayout.java:304)
at org.apache.logging.log4j.layout.template.json.JsonTemplateLayout.encode(JsonTemplateLayout.java:58)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:197)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:190)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:181)
at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
... 31 more
Solution
As Stanislav Lukyanov (see accepted answer) suggested the solution was to just download the JAR and place it below $IGNITE_HOME/libs. The error mentioned above was caused by a version mismatch. Having the following JARs with correct version made it work:
log4j-api-2.17.1.jar (default provided by ignite distribution)
log4j-core-2.17.1.jar (default provided by ignite distribution)
ignite-log4j2-2.13.0.jar (default provided by ignite distribution)
log4j-layout-template-json-2.17.1.jar (added, did not work with version 2.18.x)
If you run Ignite using Maven, you'll need to add the required dependency to your application POM, as described in the documentation:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-layout-template-json</artifactId>
<version>2.18.0</version>
</dependency>
If you run Ignite using a ZIP distribution, you'll need to download the dependency as a JAR, e.g. from here and add it to the $IGNITE_HOME/libs.

Issue in 2 node cluster using infinispan 5.3 | ActionStatus.ABORT_ONLY > is not in a valid state to be invoking cache operations on

I have a setup of 2 node cluster using Infinispan 5.3. I am testing the failover scenario. When I killed one node, i'm getting the below exception (I'm using the sync cache). The cluster is not getting. So I need to restart the application, which is not practically possible in production environment
2020-05-06 18:50:28,082 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] ISPN000136: Execution error
java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: -3f57f478:dd0a:5eb2b455:2d461 status: ActionStatus.ABORT_ONLY > is not in a valid state to be invoking cache operations on.
at org.infinispan.interceptors.TxInterceptor.enlist(TxInterceptor.java:275)
at org.infinispan.interceptors.TxInterceptor.enlistIfNeeded(TxInterceptor.java:239)
at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:233)
at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:229)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:96)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:216)
at org.infinispan.statetransfer.StateTransferInterceptor.handleDefault(StateTransferInterceptor.java:200)
at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:96)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:113)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:96)
at org.infinispan.interceptors.IsMarshallableInterceptor.visitGetKeyValueCommand(IsMarshallableInterceptor.java:97)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92)
at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:96)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:62)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343)
at org.infinispan.CacheImpl.containsKey(CacheImpl.java:372)
at org.infinispan.DecoratedCache.containsKey(DecoratedCache.java:410)
at com.abcr.ServiceContext.existsInSyncCache(ServiceContext.java:1740)
at com.abcr.ServiceContext.getObjectForUpdateInSyncCache(ServiceContext.java:1778)
at com.abcr.core.cache.ClusterServiceNodeListCacheManager.getObjectForUpdate(ClusterServiceNodeListCacheManager.java:90)
at com.suntecgroup.tbms.tpe.core.server.ServerManager.callBackOnMembersModified(ServerManager.java:3385)
at com.abcr.core.ServiceContainerCommandDespatcher.run(ServiceContainerCommandDespatcher.java:64)
2020-05-06 18:50:28,086 ERROR [com.abcr.core.ServiceContainer] Invocation of callback APIs on leaving coordinator role failed for service 'ABC'.
com.suntecgroup.tbms.container.services.ContainerPlatformServicesException: Failed to retrieve object[SERVER/SERVICE_NODES/28000] for update.
This my infinspan and jgroups configuration
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.3 http://www.infinispan.org/schemas/infinispan-config-5.3.xsd"
xmlns="urn:infinispan:config:5.3">
<global>
<!-- Note that if these are left blank, defaults are used. See the user guide for what these defaults are -->
<asyncListenerExecutor factory="org.infinispan.executors.DefaultExecutorFactory">
<properties>
<property name="maxThreads" value="5" />
<property name="threadNamePrefix" value="AsyncListenerThread" />
</properties>
</asyncListenerExecutor>
<asyncTransportExecutor factory="org.infinispan.executors.DefaultExecutorFactory">
<properties>
<property name="maxThreads" value="25" />
<property name="threadNamePrefix" value="AsyncSerializationThread" />
</properties>
</asyncTransportExecutor>
<evictionScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
<properties>
<property name="threadNamePrefix" value="EvictionThread" />
</properties>
</evictionScheduledExecutor>
<replicationQueueScheduledExecutor factory="org.infinispan.executors.DefaultScheduledExecutorFactory">
<properties>
<property name="threadNamePrefix" value="ReplicationQueueThread" />
</properties>
</replicationQueueScheduledExecutor>
<globalJmxStatistics enabled="false" jmxDomain="infinispan_1" />
<!--
If the transport is omitted, there is no way to create distributed or clustered caches.
There is no added cost to defining a transport but not creating a cache that uses one, since the transport
is created and initialized lazily.
-->
<transport clusterName="PC_SITE_1" distributedSyncTimeout="50000" transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport">
<properties>
<property name="configurationFile" value="./tmp/_clusterconfig/pc_jgroups_main_sync.xml" />
</properties>
</transport>
<!-- Note that the JGroups transport uses sensible defaults if no configuration property is defined. -->
<!-- See the JGroupsTransport javadocs for more flags -->
<!-- Again, sensible defaults are used here if this is omitted. -->
<serialization marshallerClass="org.infinispan.marshall.VersionAwareMarshaller" version="1.0" />
<!--
Used to register JVM shutdown hooks.
hookBehavior: DEFAULT, REGISTER, DONT_REGISTER
-->
<shutdown hookBehavior="DEFAULT" />
</global>
<!-- *************************** -->
<!-- Default "template" settings -->
<!-- *************************** -->
<!-- this is used as a "template" configuration for all caches in the system. -->
<default>
<!--
isolation levels supported: READ_COMMITTED and REPEATABLE_READ
-->
<locking isolationLevel="READ_COMMITTED" lockAcquisitionTimeout="60000" writeSkewCheck="false" concurrencyLevel="5000" useLockStriping="false" />
<!--
Used to register a transaction manager and participate in ongoing transactions.
-->
<!-- ECPCacheTxManagerLookup -->
<!--
Used to register JMX statistics in any available MBean server
-->
<jmxStatistics enabled="false" />
<!--
Used to enable invocation batching and allow the use of Cache.startBatch()/endBatch() methods.
-->
<clustering mode="replication">
<sync replTimeout="600000" />
<stateTransfer timeout="480000" fetchInMemoryState="true" />
</clustering>
<storeAsBinary enabled="true" />
</default>
<namedCache name="GLOBAL_SYNC_CACHE">
<transaction transactionMode="TRANSACTIONAL" transactionManagerLookupClass="com.suntecgroup.tbms.container.services.cluster.ContainerCacheTxManagerLookup" syncRollbackPhase="false" syncCommitPhase="true" useEagerLocking="true" lockingMode="PESSIMISTIC" />
</namedCache>
<namedCache name="GLOBAL_NONTX_SYNC_CACHE">
<transaction transactionMode="NON_TRANSACTIONAL" />
</namedCache>
</infinispan>
JGROUPS configuration..
<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="urn:org:jgroups" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:org:jgroups file:schema/JGroups-2.8.xsd">
<TCP bind_port="7800" loopback="true" recv_buf_size="20M" send_buf_size="640K" max_bundle_size="64000" max_bundle_timeout="30" enable_bundling="false" use_send_queues="true" sock_conn_timeout="300" tcp_nodelay="true" thread_pool.enabled="true" thread_pool.min_threads="1" thread_pool.max_threads="25" thread_pool.keep_alive_time="5000" thread_pool.queue_enabled="false" thread_pool.queue_max_size="100" thread_pool.rejection_policy="run" oob_thread_pool.enabled="true" oob_thread_pool.min_threads="1" oob_thread_pool.max_threads="8" oob_thread_pool.keep_alive_time="5000" oob_thread_pool.queue_enabled="false" oob_thread_pool.queue_max_size="100" oob_thread_pool.rejection_policy="run" enable_diagnostics="false" />
<!--MPING mcast_addr="232.1.2.13"
mcast_port="7500"
num_initial_members="2"
timeout="2000" /-->
<TCPPING timeout="3000" initial_hosts="${jgroups.tcpping.initial_hosts:localhost[7800],localhost[7801]}" port_range="0" num_initial_members="3" />
<MERGE2 max_interval="100000" min_interval="20000" />
<FD_SOCK />
<FD timeout="60000" max_tries="5" />
<VERIFY_SUSPECT timeout="30000" />
<BARRIER />
<pbcast.NAKACK use_mcast_xmit="false" exponential_backoff="500" discard_delivered_msgs="true" />
<UNICAST timeout="300,600,1200" />
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000" />
<pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true" />
FRAG2 frag_size="60000"
pbcast.STATE_TRANSFER
</config>
Current transaction was aborted (probably due to a timeout, but maybe as a consequence of delivery failure). You need to rollback current transaction and start new.
However let me note that 5.3 was released 2013/06/26 - you're using almost 7 years old version. If there is a bug, no-one will even try to check it out.

Not honoring Bundle-RequiredExecutionEnvironment in manifest.mf

I am running Photon on Java 10.
I am testing a small plugin used to help me to learn how to write plugins. When I run the plugin I get the following popup validation message:
Missing Constraint: Bundle-RequiredExecutionEnvironment: JavaSE-10
My MANIFEST.MF file is as follows:
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Application
Bundle-SymbolicName: com.packtpub.e4.application;singleton:=true
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: com.packtpub.e4.application.Activator
Bundle-Vendor: PACKTPUB
Require-Bundle: javax.inject;bundle-version="0.0.0",
org.eclipse.swt;bundle-version="0.0.0",
org.eclipse.e4.ui.model.workbench;bundle-version="0.0.0",
org.eclipse.jface;bundle-version="0.0.0",
org.eclipse.e4.ui.services;bundle-version="0.0.0",
org.eclipse.e4.ui.workbench;bundle-version="0.0.0",
org.eclipse.e4.core.di;bundle-version="0.0.0",
org.eclipse.e4.ui.di;bundle-version="0.0.0",
org.eclipse.e4.core.contexts;bundle-version="0.0.0",
org.eclipse.core.runtime;bundle-version="0.0.0",
org.eclipse.e4.core.di.extensions;bundle-version="0.15.100",
org.eclipse.e4.core.services;bundle-version="2.1.100",
org.eclipse.osgi.services;bundle-version="3.7.0"
Bundle-RequiredExecutionEnvironment: JavaSE-10
Automatic-Module-Name: com.packtpub.e4.application
Import-Package: javax.annotation;version="0.0.0"
Bundle-ActivationPolicy: lazy
Service-Component: OSGI-INF/*.xml
My launch configuration is here:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.pde.ui.RuntimeWorkbench">
<booleanAttribute key="append.args" value="true"/>
<stringAttribute key="application" value="org.eclipse.e4.ui.workbench.swt.E4Application"/>
<booleanAttribute key="askclear" value="true"/>
<booleanAttribute key="automaticAdd" value="false"/>
<booleanAttribute key="automaticValidate" value="true"/>
<stringAttribute key="bootstrap" value=""/>
<stringAttribute key="checked" value="[NONE]"/>
<booleanAttribute key="clearConfig" value="false"/>
<booleanAttribute key="clearws" value="true"/>
<booleanAttribute key="clearwslog" value="false"/>
<stringAttribute key="configLocation" value="${workspace_loc}/.metadata/.plugins/org.eclipse.pde.core/com.packtpub.e4.application.product"/>
<booleanAttribute key="default" value="false"/>
<booleanAttribute key="includeOptional" value="true"/>
<stringAttribute key="location" value="${workspace_loc}/runtime-com.packtpub.e4.application.product"/>
<stringAttribute key="org.eclipse.jdt.launching.JRE_CONTAINER" **value="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-10"/>**
<stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-os ${target.os} -ws ${target.ws} -arch ${target.arch} -nl ${target.nl} -consoleLog -clearPersistedState"/>
<stringAttribute key="org.eclipse.jdt.launching.SOURCE_PATH_PROVIDER" value="org.eclipse.pde.ui.workbenchClasspathProvider"/>
<stringAttribute key="pde.version" value="3.3"/>
<stringAttribute key="product" value="com.packtpub.e4.application.product"/>
<stringAttribute key="productFile" value="\com.packtpub.e4.application\com.packtpub.e4.application.product"/>
<stringAttribute key="selected_target_plugins" value="com.ibm.icu#default:default,javax.annotation#default:default,javax.inject#default:default,org.apache.batik.constants#default:default,org.apache.batik.css#default:default,org.apache.batik.i18n#default:default,org.apache.batik.util#default:default,org.apache.commons.io#default:default,org.apache.commons.jxpath#default:default,org.apache.commons.logging#default:default,org.apache.felix.scr#1:true,org.apache.xmlgraphics#default:default,org.eclipse.core.commands#default:default,org.eclipse.core.contenttype#default:default,org.eclipse.core.databinding.beans#default:default,org.eclipse.core.databinding.observable#default:default,org.eclipse.core.databinding.property#default:default,org.eclipse.core.databinding#default:default,org.eclipse.core.expressions#default:default,org.eclipse.core.jobs#default:default,org.eclipse.core.runtime#default:true,org.eclipse.e4.core.commands#default:default,org.eclipse.e4.core.contexts#default:default,org.eclipse.e4.core.di.annotations#default:default,org.eclipse.e4.core.di.extensions.supplier#default:default,org.eclipse.e4.core.di.extensions#default:default,org.eclipse.e4.core.di#default:default,org.eclipse.e4.core.services#default:default,org.eclipse.e4.emf.xpath#default:default,org.eclipse.e4.ui.bindings#default:default,org.eclipse.e4.ui.css.core#default:default,org.eclipse.e4.ui.css.swt.theme#default:default,org.eclipse.e4.ui.css.swt#default:default,org.eclipse.e4.ui.di#default:default,org.eclipse.e4.ui.model.workbench#default:default,org.eclipse.e4.ui.services#default:default,org.eclipse.e4.ui.widgets#default:default,org.eclipse.e4.ui.workbench.renderers.swt#default:default,org.eclipse.e4.ui.workbench.swt#default:default,org.eclipse.e4.ui.workbench3#default:default,org.eclipse.e4.ui.workbench#default:default,org.eclipse.emf.common#default:default,org.eclipse.emf.databinding#default:default,org.eclipse.emf.ecore.change#default:default,org.eclipse.emf.ecore.xmi#default:default,org.eclipse.emf.ecore#default:default,org.eclipse.equinox.app#default:default,org.eclipse.equinox.common#2:true,org.eclipse.equinox.concurrent#default:default,org.eclipse.equinox.ds#1:true,org.eclipse.equinox.event#default:default,org.eclipse.equinox.preferences#default:default,org.eclipse.equinox.registry#default:default,org.eclipse.jface.databinding#default:default,org.eclipse.jface#default:default,org.eclipse.osgi.compatibility.state#default:false,org.eclipse.osgi.services#default:default,org.eclipse.osgi.util#default:default,org.eclipse.osgi#-1:true,org.eclipse.swt.win32.win32.x86_64#default:false,org.eclipse.swt#default:default,org.w3c.css.sac#default:default,org.w3c.dom.events#default:default,org.w3c.dom.smil#default:default,org.w3c.dom.svg#default:default"/>
<stringAttribute key="selected_workspace_plugins" value="com.packtpub.e4.application#default:default"/>
<booleanAttribute key="show_selected_only" value="false"/>
<booleanAttribute key="tracing" value="false"/>
<booleanAttribute key="useCustomFeatures" value="false"/>
<booleanAttribute key="useDefaultConfig" value="true"/>
<booleanAttribute key="useDefaultConfigArea" value="true"/>
<booleanAttribute key="useProduct" value="true"/>
<booleanAttribute key="usefeatures" value="false"/>
</launchConfiguration>
As you can see, the launch configuration is requesting JaveSE 10- I am at a loss to see why I am getting this error as the plugin loads and runs without other problems.
This seems to be an Eclipse bug. JavaSE-10 just doesn't work in Bundle-RequiredExecutionEnvironment.
A workaround is to just use JavaSE-9 in the Bundle-RequiredExecutionEnvironment. You may get a warning that the JRE container is not a perfect match, this can be ignored.

IHS not redirecting the request to WAS

So, I have installed WAS, and IHS.
I'm able to access the application directly from WAS. But IHS is trying to reply the request and its not using the plugin to reply the request.
I'm able to access the default IHS page, but not to the test application.
To create the IHS, I have done the standard stuff:
1- Install IHS and Plugin
2- Configure the plugin within the IHS
3- Create an unmanaged node on the DMGR
4- Create the webserver instance on the DMGR
I'm just trying to call a sample application called hello (for helloworld)
I see the request on the webservers logs, but they are not being redirected to the WAS.
This is the mention of the plugin on the httpd.conf:
LoadModule was_ap22_module /opt/ibm/plugin-ihs-85/test_webserver_instance1/bin/64bits/mod_was_ap22_http.so
WebSpherePluginConfig /opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-cfg.xml
This is the plugin-cfg.xml:
<?xml version="1.0" encoding="ISO-8859-1"?><!--HTTP server plugin config file for the webserver test_Cell.test_webserver_01_unmanagedNode_1.test_webserver_instance1 generated on 20
17.06.25 at 07:37:51 PM CDT-->
<Config ASDisableNagle="false" AcceptAllContent="true" AppServerPortPreference="HostHeader" ChunkedResponse="false" FIPSEnable="false" FailoverToNext="false" HTTPMaxHeaders="
300" IISDisableFlushFlag="false" IISDisableNagle="false" IISPluginPriority="High" IgnoreDNSFailures="false" KillWebServerStartUpOnParseErr="false" MarkBusyDown="false" OS400C
onvertQueryStringToJobCCSID="false" RefreshInterval="60" ResponseChunkSize="64" SSLConsolidate="true" StrictSecurity="false" TrustedProxyEnable="false" VHostMatchingCompat="f
alse">
<Log LogLevel="Error" Name="/opt/ibm/plugin-ihs-85/test_webserver_instance1/logs/test_webserver_instance1/http_plugin.log"/>
<Property Name="ESIEnable" Value="true"/>
<Property Name="ESIMaxCacheSize" Value="1024"/>
<Property Name="ESIInvalidationMonitor" Value="false"/>
<Property Name="ESIEnableToPassCookies" Value="false"/>
<Property Name="ESICacheidFull" Value="false"/>
<Property Name="PostSizeLimit" Value="-1"/>
<Property Name="PostBufferSize" Value="0"/>
<Property Name="PluginInstallRoot" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/"/>
<Property Name="Keyfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.kdb"/>
<Property Name="Stashfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.sth"/>
<VirtualHostGroup Name="default_host">
<VirtualHost Name="*:9080"/>
<VirtualHost Name="*:80"/>
<VirtualHost Name="*:9443"/>
<VirtualHost Name="*:5060"/>
<VirtualHost Name="*:5061"/>
<VirtualHost Name="*:443"/>
<VirtualHost Name="*:9061"/>
<VirtualHost Name="*:9044"/>
<VirtualHost Name="*:9062"/>
<VirtualHost Name="*:9081"/>
<VirtualHost Name="*:9444"/>
<VirtualHost Name="*:9045"/>
</VirtualHostGroup>
<ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="false" LoadBalance="Round Robin" Name="test" PostBufferSize="0" PostSizeLimit="-1"
RemoveSpecialHeaders="true" RetryInterval="60" ServerIOTimeoutRetry="-1">
<Server CloneID="1bjgscrk8" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="test_app_01_node_1_test1" ServerIOTimeout="900" W
aitForContinue="false">
<Transport Hostname="test_app_01" Port="9080" Protocol="http"/>
<Transport Hostname="test_app_01" Port="9443" Protocol="https">
<Property Name="keyring" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.sth"/>
</Transport>
</Server>
<Server CloneID="1bjgscsei" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="test_app_01_node_1_test2" ServerIOTimeout="900" W
aitForContinue="false">
<Transport Hostname="test_app_01" Port="9081" Protocol="http"/>
<Transport Hostname="test_app_01" Port="9444" Protocol="https">
<Property Name="keyring" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.sth"/>
</Transport>
</Server>
<Server CloneID="1bjgscstv" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="test_app_02_node_1_test3" ServerIOTimeout="900" W
aitForContinue="false">
<Transport Hostname="test_app_02" Port="9080" Protocol="http"/>
<Transport Hostname="test_app_02" Port="9443" Protocol="https">
<Property Name="keyring" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.sth"/>
</Transport>
</Server>
<Server CloneID="1bjgsctbv" ConnectTimeout="5" ExtendedHandshake="false" LoadBalanceWeight="2" MaxConnections="-1" Name="test_app_02_node_1_test4" ServerIOTimeout="900" W
aitForContinue="false">
<Transport Hostname="test_app_02" Port="9081" Protocol="http"/>
<Transport Hostname="test_app_02" Port="9444" Protocol="https">
<Property Name="keyring" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.kdb"/>
<Property Name="stashfile" Value="/opt/ibm/plugin-ihs-85/test_webserver_instance1/config/test_webserver_instance1/plugin-key.sth"/>
</Transport>
</Server>
<PrimaryServers>
<Server Name="test_app_01_node_1_test1"/>
<Server Name="test_app_01_node_1_test2"/>
<Server Name="test_app_02_node_1_test3"/>
<Server Name="test_app_02_node_1_test4"/>
</PrimaryServers>
</ServerCluster>
<UriGroup Name="default_host_test_URIs">
<Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/hello/*"/>
</UriGroup>
<Route ServerCluster="test" UriGroup="default_host_test_URIs" VirtualHostGroup="default_host"/>
<RequestMetrics armEnabled="false" loggingEnabled="false" rmEnabled="false" traceLevel="HOPS">
<filters enable="false" type="URI">
<filterValues enable="false" value="/snoop"/>
<filterValues enable="false" value="/hitcount"/>
</filters>
<filters enable="false" type="SOURCE_IP">
<filterValues enable="false" value="255.255.255.255"/>
<filterValues enable="false" value="254.254.254.254"/>
</filters>
<filters enable="false" type="JMS">
<filterValues enable="false" value="destination=aaa"/>
</filters>
<filters enable="false" type="WEB_SERVICES">
<filterValues enable="false" value="wsdlPort=aaa:op=bbb:nameSpace=ccc"/>
</filters>
</RequestMetrics>
</Config>
Normally IHS is going to use port 80 to receive communication and rely to the application server.
The port 80 is in the default virtual host. So I added the listening port I'm using (1080) to the default virtual host.

Jboss application-policy ignored in mutual/client-cert auth with PBESecurityDomain

With the following, mutual client cert, SSL (TLS) handshake works for a rest endpoint (yay!) - validated via testing and debugging: javax.net logging & wireshark. But...
1st observation: HTTPServletRequest and JAX-RS annotated SecurityContext has null Principal info
2nd observation: Tampering with the login-config.xml, containing application-policy elements, has no effect
In short, TLS works but the transfer of the cert DN to the HTTPServletRequest object in the request thread does not preventing the application from picking up on the caller's ID. Does anyone have any advice?
On JBoss 6:
deploy/jbossweb.sar/server.xml:
<Connector protocol="HTTP/1.1" debug="10"
SSLEnabled="true"
...
secure="true"
clientAuth="true"
sslProtocol = "TLS"
securityDomain="java:/jaas/mydomain"
SSLImplementation="org.jboss.net.ssl.JBossImplementation" />
deploy/jbossweb.sar/META-INF/jboss-beans.xml:
<depends>jboss.security:service=PBESecurityDomain</depends>
deploy/security-service.xml:
<?xml version="1.0" encoding="UTF-8"?>
<server>
<mbean code="org.jboss.security.plugins.JaasSecurityDomain"
name="jboss.security:service=PBESecurityDomain">
<constructor> <arg type="java.lang.String" value="mydomain"/>
</constructor>
<attribute name="KeyStoreURL">${jboss.server.home.dir}/mykeystore.jks</attribute>
<attribute name="KeyStorePass">{CLASS}org.jboss.security.plugins.FilePassword:${jboss.server.home.dir}/mykeystorepass.pbe</attribute>
<attribute name="TrustStoreURL">${jboss.server.home.dir}/mytruststore.jks</attribute>
<attribute name="TrustStorePass">password</attribute>
<attribute name="Salt">abunchofrandomchars</attribute>
<attribute name="IterationCount">13</attribute>
<depends>jboss.security:service=JaasSecurityManager</depends>
</mbean>
</server>
deploy/security/security-jboss-beans.xml:
<bean name="XMLLoginConfig" class="org.jboss.security.auth.login.XMLLoginConfig">
<property name="configResource">login-config.xml</property>
</bean>
<bean name="SecurityConfig" class="org.jboss.security.plugins.SecurityConfig">
<property name="mbeanServer"><inject bean="JMXKernel" property="mbeanServer"/></property>
<property name="defaultLoginConfig"><inject bean="XMLLoginConfig"/></property>
</bean>
conf/login-config.xml:
<application-policy name="mydomain">
<authentication>
<login-module code="org.jboss.security.auth.spi.BaseCertLoginModule"
flag = "required">
<module-option name="password-stacking">useFirstPass</module-option>
<module-option name="securityDomain">java:/jaas/mydomain</module-option>
<module-option name="verifier">org.jboss.security.auth.certs.AnyCertVerifier</module-option>
<module-option name="principalClass">org.jboss.security.auth.certs.SubjectDNMapping</module-option>
</login-module>
<login-module code="org.jboss.security.auth.spi.UserRolesLoginModu"
flag = "required">
<module-option name="password-stacking">useFirstPass</module-option>
<module-option name="usersProperties">users.properties</module-option>
<module-option name="rolesProperties">roles.properties</module-option>
</login-module>
</authentication>
</application-policy>
war/WEB-INF/jboss-web.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jboss-web PUBLIC
"-//JBoss//DTD Web Application 2.4//EN"
"http://www.jboss.org/j2ee/dtd/jboss-web_4_0.dtd">
<jboss-web>
<security-domain>java:/jaas/mydomain</security-domain>
<context-root>/myapp</context-root>
</jboss-web>
Add the special ClientLoginModule to login-context.xml to fix the null principal issue.
<login-module code="org.jboss.security.ClientLoginModule" flag="required"></login-module>