Tomcat stopped accepting input and is crashed - apache

I have deployed SOAP Service using tomcat. I pushed a request to the SOAP service and it worked. But when i submitted bulk request., tomcat crashed.
I can see below error in log files:-
SEVERE: Endpoint ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=7106] ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:408)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
WARNING: Reinitializing ServerSocket
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint null ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.ServerSocket.createImpl(ServerSocket.java:255)
at java.net.ServerSocket.getImpl(ServerSocket.java:205)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:141)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:50)
at org.apache.tomcat.util.net.PoolTcpEndpoint.initEndpoint(PoolTcpEndpoint.java:293)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:469)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
WARNING: Restarting endpoint
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint null shutdown due to exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
at java.net.ServerSocket.createImpl(ServerSocket.java:255)
at java.net.ServerSocket.getImpl(ServerSocket.java:205)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:141)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:50)
at org.apache.tomcat.util.net.PoolTcpEndpoint.initEndpoint(PoolTcpEndpoint.java:293)
at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:481)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
Jun 5, 2014 6:43:08 PM org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Caught exception (java.lang.ThreadDeath) executing org.apache.tomcat.util.net.LeaderFollowerWorkerThread#4d8d50, terminating thread
Can some body tell me cause and solution to the problem.

Related

Infinispan 8 start up error in domain mode

When ever I start the infinispan server 8 in domain mode, I am getting the below exception .I am not sure what is going wrong.
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.datagrid-infinispan-endpoint.memcached.memcached-connector: org.jboss.msc.service.StartException in service jboss.datagrid-infinispan-endpoint.memcached.memcached-connector: DGENDPT10004: Failed to start MemcachedServer
Caused by: java.lang.IllegalStateException: failed to create a child event loop
... 5 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
Caused by: java.io.IOException: Unable to establish loopback connection
... 24 more
Caused by: java.io.IOException: An existing connection was forcibly closed y the remote host
... 32 more
2016-05-11 20:01:55,600 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-8) MSC000001: Failed to start service jboss.datagrid-infinispan-endpoint.websocket.websocket-connector: org.jboss.msc.service.StartException in service jboss.datagrid-infinispan-endpoint.websocket.websocket-connector: GENDPT10004: Failed to start WebSocketServer
caused by: java.lang.IllegalStateException: failed to create a child event loop ... 5 more
Your logs contain Unable to establish loopback connection which suggests that there is something wrong with your OS configuration.
The similar issue can be found here: Failed to initialize monitor Thread: Unable to establish loopback connection

MobileFirst 7.0 Server stuck "Deploying" when trying to deploy a .wlapp file to Tomcat

While after finishing installing and configuring successfully the MobileFirst 7.0 Server, I'm getting the issue that the MF server is stucking in "Deploying" when I try to Deploy Application => Selecting a MF studio /bin folder .wlapp file.
And here are some log messages from Tomcat log files:
10614 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
10646 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
10682 WorklightManagementPU-mysql INFO [pool-2-thread-1] openjpa.Runtime - Though you are using optimistic transactions, OpenJPA is now beginning a datastore transaction because you have requested a lock on some data.
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.ApplicationDeploymentTransaction beforeAll
SEVERE: Failed to generate download links to transfer data during the deployment of applications. See the next exception in the log. Switch to direct data transfer via JMX. This is not fatal but it requires larger memory than the data transfer with download links.
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.BaseTransaction internalRun
SEVERE: BaseTransaction.internalRun() Exception
<openjpa-1.2.2-r422266:898935 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Object "com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1" is not managed by this context.
FailedObject: com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1
at org.apache.openjpa.persistence.EntityManagerImpl.processArgument(EntityManagerImpl.java:1216)
at org.apache.openjpa.kernel.BrokerImpl.processArgument(BrokerImpl.java:2149)
at org.apache.openjpa.kernel.BrokerImpl.gatherCascadeRefresh(BrokerImpl.java:2816)
at org.apache.openjpa.kernel.BrokerImpl.refresh(BrokerImpl.java:2793)
at org.apache.openjpa.kernel.DelegatingBroker.refresh(DelegatingBroker.java:1078)
at org.apache.openjpa.persistence.EntityManagerImpl.refresh(EntityManagerImpl.java:688)
at com.ibm.worklight.admin.actions.BaseTransaction.setNewProjectTransactionID(BaseTransaction.java:643)
at com.ibm.worklight.admin.actions.BaseTransaction.internalRun(BaseTransaction.java:336)
at com.ibm.worklight.admin.actions.BaseTransaction$1.run(BaseTransaction.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 03, 2015 2:03:32 PM com.ibm.worklight.admin.actions.BaseTransaction$1 run
SEVERE: internalRun() Exception
<openjpa-1.2.2-r422266:898935 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Object "com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1" is not managed by this context.
FailedObject: com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-com.ibm.worklight.dataaccess.jpa.entity.ProjectEntity-1
at org.apache.openjpa.persistence.EntityManagerImpl.processArgument(EntityManagerImpl.java:1216)
at org.apache.openjpa.kernel.BrokerImpl.processArgument(BrokerImpl.java:2149)
at org.apache.openjpa.kernel.BrokerImpl.gatherCascadeRefresh(BrokerImpl.java:2816)
at org.apache.openjpa.kernel.BrokerImpl.refresh(BrokerImpl.java:2793)
at org.apache.openjpa.kernel.DelegatingBroker.refresh(DelegatingBroker.java:1078)
at org.apache.openjpa.persistence.EntityManagerImpl.refresh(EntityManagerImpl.java:688)
at com.ibm.worklight.admin.actions.BaseTransaction.setNewProjectTransactionID(BaseTransaction.java:643)
at com.ibm.worklight.admin.actions.BaseTransaction.internalRun(BaseTransaction.java:336)
at com.ibm.worklight.admin.actions.BaseTransaction$1.run(BaseTransaction.java:235)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 03, 2015 2:04:10 PM com.worklight.core.ilmt.WorklightILMTLogger dumpLicense
INFO: FWLSE0277I: Creating an ILMT record in the file '/var/ibm/common/slm/e1178f0cf1cfd7b4017f37827cae010b.slmtag'.
Finally this issue was solved by setting
max_allowed_packet = 30M
and
binlog_format = mixed
in mysql's my.cnf file.

Configuration of CloudAMQP Connection

I'm having difficulty configuring my connection to CloudAMQP in my deployed grails application. I can run the application locally against a locally installed RabbitMQ instance but can't figure out how to correctly define my application to run on CloudBees using the CloudAMQP service.
In my Config.groovy, I'm defining my connection info and a queue:
rabbitmq {
connectionfactory {
username = 'USERNAME'
password = 'PASSWORD'
hostname = 'lemur.cloudamqp.com'
}
queues = {
testQueue autoDelete: false, durable: false, exclusive: false
}
}
When the application starts and tries to connect, I see the following log messages:
2013-08-23 21:29:59,195 [main] DEBUG listener.SimpleMessageListenerContainer - Starting Rabbit listener container.
2013-08-23 21:29:59,205 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,405 [SimpleAsyncTaskExecutor-1] WARN listener.SimpleMessageListenerContainer - Consumer raised exception, processing can restart if the connection factory supports it
org.springframework.amqp.AmqpIOException: java.io.IOException
at org.springframework.amqp.rabbit.connection.RabbitUtils.convertRabbitAccessException(RabbitUtils.java:112)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:163)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:228)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:119)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:163)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:109)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:199)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:524)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:106)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:102)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:124)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:381)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:516)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:545)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.net.SocketException: Connection reset
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:67)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:33)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:343)
at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:216)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)
... 3 more
Caused by: java.net.SocketException: Connection reset
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:508)
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] INFO listener.SimpleMessageListenerContainer - Restarting Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Closing Rabbit Channel: null
2013-08-23 21:30:08,407 [SimpleAsyncTaskExecutor-2] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
Aug 23, 2013 9:30:11 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'grails'
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8634
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8634
According to https://developer.cloudbees.com/bin/view/RUN/CloudAMQP
when you bind your CloudAMQP service to your app - some config params are provided in the pattern of CLOUDAMQP_URL_ - this is the type of thing you would need to put in your config files so they can be wired in when the app launches.
Make sure to specify the virtualHost for CloudAMQP connections. That worked for me.

ERROR: Initialization failure: Cannot create configuration

i'm trying to get CouldBees working. Heres the error i get when im running:
C:\cloudbees-sdk-1.5.0>bees init --proxyHost=localhost --proxyPort=8008 (or 8080)
You have not created a CloudBees configuration profile, let's create
one now... Enter your default CloudBees API end point [us | eu]: eu
Enter your CloudBees account email address: abs#abs.com
Enter your CloudBees account password: Jul 18, 2013 1:32:09 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:09 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request Jul 18, 2013 1:32:10 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:10 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request Jul 18, 2013 1:32:11 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing
request: Connection refused: connect Jul 18, 2013 1:32:11 PM
org.apache.commons.httpclient.HttpMethodDirector execute WithRetry
INFO: Retrying request
ERROR: Initialization failure: Cannot create configuration
Can anyone read out what's causing this error?
It looks like the SDK can't establish Internet connections to the CloudBees website. If you are running behind a proxy, you will need to use proxy flags to connect.
bees init --proxyHost=YOUR_PROXY_HOST --proxyPort=YOUR_PROXY_PORT
This is covered in the CloudBees SDK docs: Running behind a proxy
It helps, to set the system time exactly to six hours ago (US-time).
Also cloudbees documentation sais, that You should create on your file system (under Windows7) c:\Users\Your_User.bees\bees.config file, which contains following line (if you want to call cloudbbees eu server):
bees.api.url=https\://api-eu.cloudbees.com/api>
but actually it didn't help in my case (maybe outdated version)

bees SDK is attempting to communicate with localhost

The cloudbees SDK 1.1 is attempting to connect to localhost:8080 when I run commands.
Any idea what I need to do to fix it?
Example
bees app:info -v
# CloudBees SDK version: 1.1
Enter application ID (ex: account/appname) : account/app
API call: http://localhost:8080/api?timestamp=1344846702&v=1.0&api_key=KEY&action=application.info&app_id=account%2Fapp&format=xml&sig_version=1&sig=SIGN
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused
Aug 13, 2012 6:31:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
ERROR: Connection refused
Thank you
Jono
Add bees.api.url=https\://api.cloudbees.com/api to ~/.bees/bees.config
After backing up ~/.bees and then running bees init and recreating my bees configuration directory I noticed the extra entry in bees.config.