Worklight Application Runtime error - ibm-mobilefirst

Question

We are facing an issue on our PROD environment where it shows the following error message:
The home page shows that the runtime contains the project files:
There error on the WAS is:
[5/1/15 12:38:29:901 EDT] 0000009f ApplicationMa E
com.worklight.mgmt.impl.ApplicationManagementImpl getApplicationRuntimeInformation FWLSE0023E: application 'MobileBanking' not found, please redeploy. [project MobileBanking]
[5/1/15 12:38:29:947 EDT] 0000009f DMAdapter I com.ibm.ws.ffdc.impl.DMAdapter getAnalysisEngine FFDC1009I: Analysis Engine using data base: /wl62/profiles/node1/properties/logbr/ffdc/adv/ffdcdb.xml
[5/1/15 12:38:30:069 EDT] 0000009f FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /wl62/profiles/node1/logs/ffdc/server11_wl_3e94673b_15.05.01_12.38.29.9336586937544274552029.txt com.ibm.ws.management.AdminServiceImpl.invoke 693
[5/1/15 12:38:30:573 EDT] 00000100 ApplicationSe E FWLSE3000E: A server error was detected.
java.lang.IllegalArgumentException: FWLSE0023E: application 'MobileBanking' not found, please redeploy. [project MobileBanking]
at com.worklight.mgmt.impl.ApplicationManagementImpl.getApplicationRuntimeInformation(ApplicationManagementImpl.java:165)
at com.worklight.core.jmx.ProjectManagementMXBeanImpl.getApplicationRuntimeInformation(ProjectManagementMXBeanImpl.java:673)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:76)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:607)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:607)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:193)
at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:175)
at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:117)
at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:54)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1335)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1228)
at com.ibm.ws.management.connector.AdminServiceDelegator.invoke(AdminServiceDelegator.java:181)
at com.ibm.ws.management.connector.ipc.CallRouter.route(CallRouter.java:247)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink.doWork(IPCConnectorInboundLink.java:360)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink$IPCConnectorReadCallback.complete(IPCConnectorInboundLink.java:602)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1784)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)
We have reinstalled the runtime more than once, and it says successful, we also recycled the dmgr, node, cluser, server and the runtimes and all shows green arrows with successful start up message.
Same WAR file is used on our Dev, QAT and UAT and they all running up with no issues.
Worklight Version 6.2.0.1
WAS version 8.5.5.1
Any help to resolve it, thanks.

Answer 1

We were able to resolve both the application issues and the Worklight service issue. There were multiple pieces that needed to be fixed. For the most part, these issues were a result of having the complexity of the how Worklight implements security. All the environment is up and running now.
Steps we followed to resolve the issue:
Make sure the exact same configuration are running on both Nodes.
Make sure that the JVM configuration are matching.
Review the security configuration on each node (SSL certificate - SLL alias and SSL keystore).