MobileFirst 7.0 Server stuck "Deploying" when trying to deploy a .wlapp file to Tomcat - ibm-mobilefirst

While after finishing installing and configuring successfully the MobileFirst 7.0 Server, I'm getting the issue that the MF server is stucking in "Deploying" when I try to Deploy Application => Selecting a MF studio /bin folder .wlapp file.
And here are some log messages from Tomcat log files:
10614 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
10646 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
10682 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.ApplicationDeploymentTransaction beforeAll
SEVERE: Failed to generate download links to transfer data during the deployment of applications. See the next exception in the log. Switch to direct data transfer via JMX. This is not fatal but it requires larger memory than the data transfer with download links.
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.BaseTransaction internalRun
SEVERE: BaseTransaction.internalRun() Exception
<openjpa-1.2.2-r422266:898935 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Object "com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1" is not managed by this context.
FailedObject: com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1
at org.apache.openjpa.persistence.EntityManagerImpl.processArgument(EntityManagerImpl.java:1216)
at org.apache.openjpa.kernel.BrokerImpl.processArgument(BrokerImpl.java:2149)
at org.apache.openjpa.kernel.BrokerImpl.gatherCascadeRefresh(BrokerImpl.java:2816)
at org.apache.openjpa.kernel.BrokerImpl.refresh(BrokerImpl.java:2793)
at org.apache.openjpa.kernel.DelegatingBroker.refresh(DelegatingBroker.java:1078)
at org.apache.openjpa.persistence.EntityManagerImpl.refresh(EntityManagerImpl.java:688)
at com.ibm.worklight.admin.actions.BaseTransaction.setNewProjectTransactionID(BaseTransaction.java:643)
at com.ibm.worklight.admin.actions.BaseTransaction.internalRun(BaseTransaction.java:336)
at com.ibm.worklight.admin.actions.BaseTransaction$1.run(BaseTransaction.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.BaseTransaction$1 run
SEVERE: internalRun() Exception
<openjpa-1.2.2-r422266:898935 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Object "com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1" is not managed by this context.
FailedObject: com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1
at org.apache.openjpa.persistence.EntityManagerImpl.processArgument(EntityManagerImpl.java:1216)
at org.apache.openjpa.kernel.BrokerImpl.processArgument(BrokerImpl.java:2149)
at org.apache.openjpa.kernel.BrokerImpl.gatherCascadeRefresh(BrokerImpl.java:2816)
at org.apache.openjpa.kernel.BrokerImpl.refresh(BrokerImpl.java:2793)
at org.apache.openjpa.kernel.DelegatingBroker.refresh(DelegatingBroker.java:1078)
at org.apache.openjpa.persistence.EntityManagerImpl.refresh(EntityManagerImpl.java:688)
at com.ibm.worklight.admin.actions.BaseTransaction.setNewProjectTransactionID(BaseTransaction.java:643)
at com.ibm.worklight.admin.actions.BaseTransaction.internalRun(BaseTransaction.java:336)
at com.ibm.worklight.admin.actions.BaseTransaction$1.run(BaseTransaction.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 03, 2015 2:04:10 PM com.worklight.core.ilmt.WorklightILMTLogger dumpLicense
INFO: FWLSE0277I: Creating an ILMT record in the file '/var/ibm/common/slm/e1178f0cf1cfd7b4017f37827cae010b.slmtag'.

Finally this issue was solved by setting
max_allowed_packet = 30M
and
binlog_format = mixed
in mysql's my.cnf file.

Related

JDBC client to Hive - No data or no sasl data in the stream Exception

We have a Kerberised cluster and I'm trying to run a Java action in Oozie where I make a JDBC connection to Hive. This JDBC connections works fine on the Sandbox without Kerberos.
The connection string is as simple as the following, where I'm providing username and password in it:
Connection con = DriverManager.getConnection("jdbc:hive2://W12345:10000/control;principal=hive/W12345.companynet.net#COMPANYNET.NET","user123","passw123");
The Oozie action (strangely) completes succesfully, and the Java action log does not present any error:
1742 [main] INFO org.apache.hive.jdbc.Utils - Supplied authorities: W12345:10000
1742 [main] INFO org.apache.hive.jdbc.Utils - Resolved authority: W12345:10000
1766 [main] INFO org.apache.hive.jdbc.HiveConnection - Will try to open client transport with JDBC Uri: jdbc:hive2://W12345:10000/control;principal=hive/W12345.companynet.net#COMPANYNET.NET
<<< Invocation of Main class completed <<<
Oozie Launcher ends
1785 [main] INFO org.apache.hadoop.mapred.Task - Task:attempt_1464245290012_0129_m_000000_0 is done. And is in the process of committing
1847 [main] INFO org.apache.hadoop.mapred.Task - Task attempt_1464245290012_0129_m_000000_0 is allowed to commit now
1854 [main] INFO org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt_1464245290012_0129_m_000000_0' to hdfs://danskehadoop/user/user123/oozie-oozi/0000013-160527101253015-oozie-oozi-W/JavaAction--java/output/_temporary/1/task_1464245290012_0129_m_000000
1909 [main] INFO org.apache.hadoop.mapred.Task - Task 'attempt_1464245290012_0129_m_000000_0' done.
But in reality the Java main does not complete correctly the execution (and does not execute the needed queries) because the JDBC connection fails with an exception that I can see only in the Hive log:
ERROR [HiveServer2-Handler-Pool: Thread-78363]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:739)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:736)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:736)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 10 more
I'm actually connected to the cluster, and already done further kinit on my username.
Does anybody know what could the cause of this exception be?
Thanks in advance for the help!
Antonio
This happened to me on MapR hadoop distribution platform.
In my case it was Keepalived checking Hive port every 5 seconds and producing such error. I simply used "nc" command to check if Hive port is in use and did not use any authentication method. Later I switched to "maprcli" command which uses SASL authentication and the error was gone.

Weblogic Admin Server Start issue - DataSource passwords got exparied

We are trying to use one of the existing weblogic 12c domain and It's DataSource passwords been expired.
Since AdminServer is not responding correctly I tried to re-start AdminServer as well, Now I have changed the DB passwords and wanted to set the new passwords starting the AdminServer, but I cant start AdminServer it's failing complaining passwords are expired. (I could have get a way with this issue if i keep the admin server running and set the new passwords)
I can see DataSources are targeted to Admin Server and I thought if I untarget DS from AdminServer I could start AdminServer correctly. hence I removed the AdminServer as a target from config.xml and tried o start the Admin but it's still failing complaining passwords are expired. Is Config cached anywhere , looks like I am Admin is still using the old config file ? by the way I have tried removing the tmp folder as well.
Also, I tried encrypting the new password and placing on JDBC config files, Probably the way I encrypted was wrong. These are the steps I used to encrypt
1. Connect to WLST offline ( because Admin is not up)
2. Read domain
3. Call encrypt function for new password
4. Print the encrypt password
Anything wrong ? Appreciate any suggestion to resolve this issue.
Error is starting like this ,
Jun 22, 2015 4:38:04 PM oracle.security.jps.JpsStartup start
INFO: Jps initializing.
Jun 22, 2015 4:38:07 PM org.hibernate.validator.util.Version <clinit>
INFO: Hibernate Validator 12.1.3.0.0
Jun 22, 2015 4:38:07 PM org.hibernate.validator.engine.resolver.DefaultTraversableResolver detectJPA
INFO: Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.
[EL Severe]: ejb: 2015-06-22 16:38:11.173--ServerSession(143991231)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
Error Code: 1017
Jun 22, 2015 4:38:11 PM oracle.security.jps.internal.common.config.AbstractSecurityStore getSecurityStoreVersion
WARNING: Unable to get the Version from Store returning the default oracle.security.jps.service.policystore.PolicyStoreException: javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
Error Code: 1017
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.processJPAException(JpsDBDataManager.java:2180)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.init(JpsDBDataManager.java:1028)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.jpsObjectBaseQuery(JpsDBDataManager.java:3089)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.queryBaseObjects(JpsDBDataManager.java:5761)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersion(AbstractSecurityStore.java:211)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersion(AbstractSecurityStore.java:195)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.<init>(AbstractSecurityStore.java:99)
at oracle.security.jps.internal.credstore.AbstractCredentialStore.<init>(AbstractCredentialStore.java:104)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.<init>(LdapCredentialStore.java:130)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStoreProvider.getInstance(LdapCredentialStoreProvider.java:235)
at oracle.security.jps.internal.credstore.rdbms.DbmsCredentialStoreProvider.getInstance(DbmsCredentialStoreProvider.java:101)
at oracle.security.opss.internal.runtime.ServiceContextManagerImpl.createContextInternal(ServiceContextManagerImpl.java:432)
Thanks.
First take backup of complete config folder inside domain.It looks like you are using rdbms policy store inside domain.So check for security-realm tag in config.xml there you will able to find encrypted password change the same using newly encrypted password and your admin server should start.

Weblogic nodemanager errors/timeout

Running a Windows domain on a NAS, we infrequently see this error. We're worried it may be our configuration causing a problem, possibly some nodemanager.properties setting, a WLST call, etc.
Eventually we are unable to command the NodeManager via JMX and WLST. Cycling the NM works but is not a good solution. Is there something else we can try via WLST?
<Jan 22, 2015 6:09:28 PM> <SEVERE> <Domain1> <ms1> <Unexpected error while monitoring server>
java.lang.Error: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.create(FileKey.java:27)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.<init>(FileChannelImpl.java:1037)
at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:806)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:823)
at java.nio.channels.FileChannel.lock(FileChannel.java:860)
at weblogic.nodemanager.util.ConcurrentLockedFile.read(ConcurrentLockedFile.java:54)
at weblogic.nodemanager.common.StateInfo.load(StateInfo.java:130)
at weblogic.nodemanager.server.AbstractServerMonitor.loadStateInfo(AbstractServerMonitor.java:497)
at weblogic.nodemanager.server.AbstractServerMonitor.runMonitor(AbstractServerMonitor.java:273)
at weblogic.nodemanager.server.AbstractServerMonitor.run(AbstractServerMonitor.java:246)
at weblogic.nodemanager.server.ServerMonitor.run(ServerMonitor.java:25)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.init(Native Method)
at sun.nio.ch.FileKey.create(FileKey.java:25)
... 11 more

Tomcat stopped accepting input and is crashed

I have deployed SOAP Service using tomcat. I pushed a request to the SOAP service and it worked. But when i submitted bulk request., tomcat crashed.
I can see below error in log files:-
SEVERE: Endpoint ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=7106] ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:408)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
WARNING: Reinitializing ServerSocket
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint null ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.ServerSocket.createImpl(ServerSocket.java:255)
at java.net.ServerSocket.getImpl(ServerSocket.java:205)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:141)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:50)
at org.apache.tomcat.util.net.PoolTcpEndpoint.initEndpoint(PoolTcpEndpoint.java:293)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:469)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
WARNING: Restarting endpoint
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint null shutdown due to exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.ServerSocket.createImpl(ServerSocket.java:255)
at java.net.ServerSocket.getImpl(ServerSocket.java:205)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:141)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:50)
at org.apache.tomcat.util.net.PoolTcpEndpoint.initEndpoint(PoolTcpEndpoint.java:293)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:481)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Caught exception (java.lang.ThreadDeath) executing org.apache.tomcat.util.net.LeaderFollowerWorkerThread#4d8d50, terminating thread
Can some body tell me cause and solution to the problem.

ERROR: Initialization failure: Cannot create configuration

i'm trying to get CouldBees working. Heres the error i get when im running:
C:\cloudbees-sdk-1.5.0>bees init --proxyHost=localhost --proxyPort=8008 (or 8080)
You have not created a CloudBees configuration profile, let's create
one now... Enter your default CloudBees API end point [us | eu]: eu
Enter your CloudBees account email address: abs#abs.com
Enter your CloudBees account password: Jul 18, 2013 1:32:09 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:09 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request Jul 18, 2013 1:32:10 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:10 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request Jul 18, 2013 1:32:11 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:11 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request
ERROR: Initialization failure: Cannot create configuration
Can anyone read out what's causing this error?
It looks like the SDK can't establish Internet connections to the CloudBees website. If you are running behind a proxy, you will need to use proxy flags to connect.
bees init --proxyHost=YOUR_PROXY_HOST --proxyPort=YOUR_PROXY_PORT
This is covered in the CloudBees SDK docs: Running behind a proxy
It helps, to set the system time exactly to six hours ago (US-time).
Also cloudbees documentation sais, that You should create on your file system (under Windows7) c:\Users\Your_User.bees\bees.config file, which contains following line (if you want to call cloudbbees eu server):
bees.api.url=https\://api-eu.cloudbees.com/api>
but actually it didn't help in my case (maybe outdated version)