timeout error on weblogic cluster - weblogic

I want to run a web application on a cluster.There are 2 servers(10.0.0.32 and 10.0.0.192). When I try to run this application on 10.0.0.192 I am getting below error.
By the way I am using adminserver on 10.0.0.32. I can control server 10.0.0.192 without any problem from 10.0.0.32 adminserver.
####<Dec 30, 2013 3:02:36 PM EET> <Error> <HTTP> <dell-PC> <10-0-0-192> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1388408556025> <BEA-101083> <Connection failure.
java.io.IOException: A complete message could not be read on socket: 'weblogic.servlet.internal.MuxableSocketHTTP#4bd7f3dd:Socket[addr=/10.0.0.32,port=55179,localport=7002] - idle timeout: '30000' ms, socket timeout: '5000' ms', in the configured timeout period of '60' secs
at weblogic.socket.SocketMuxer$TimerListenerImpl.timerExpired(SocketMuxer.java:1076)
at weblogic.timers.internal.TimerImpl.run(TimerImpl.java:293)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

Cause:
The above error indicates that the default configured timeout period of '60' secs is not sufficinet to transfer the complete message to the server.
Resolution:
Increase the Complete Message Time Out to 480 seconds in weblogic console:
Server --> Protocols --> General -> Advanced Attributes -> Complete Message Timeout
You must reboot the server.
A complete message could not be read on socket: 'weblogic.rjvm.t3.T3JVMConnection'" on WebLogic :
weblogic.rjvm.PeerGoneException: ; nested exception is: java.io.IOException: A complete message could not be read on socket: 'weblogic.rjvm.t3.T3JVMConnection#12c22dd4', in the configured timeout period of '60' secs
at weblogic.rjvm.BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:108)
at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:284)
for this we can increase timeout period by setting the parameters
-Dweblogic.CompleteT3MessageTimeout=
-Dweblogic.CompleteMessageTimeout=

Related

Client timeout in Gemfire logs

​
I am currently in the process of trouble shooting Gemfire logs for client timeout:
​
Logs:​
​[warning 2018/06/11 23:45:05.685 CDT xxxxsrv01_instance_42404_cacheserver <ClientHealthMonitor Thread> tid=0x97] Server connection from [identity(xx.xx.xx.xx(5338:loner):33228:4cae4ef2,connection=2; port=33261] is terminated because its client timeout of 10,000 has expired.
Even increasing the ​"read-timeout" value in client pool config to 200000 gives "client time out of 200000 has expired".
Also noticed below logs:
xxxxx_myinstance_42404_cacheserver <Handshaker /0:0:0:0:0:0:0:0:42404 Thread 229> tid=0x2c29cc] Bridge server: failed accepting client connection {0}
​Can you please help me on possible causes of the issue.
Regards,​
Rahul N.​

Weblogic 12c managed server not able to start from console but able to stop from console

Hi My current environment is Weblogic 12c Std
i have 2 servers which is running one Admin server 2 managed servers
Node -1 contains admin & 1 managed servers
Node -2 contains Managed servers
My issues is i am able to stop node-2 managed server node 2 from Admin console but not able to start from Admin console
While start i am getting below Error on Console
Message icon - Warning For server mrm_node2, the Node Manager associated with machine mrm_machine2 is not reachable.
Message icon - Warning All of the servers selected are currently in a state which is incompatible with this operation or are not associated with a running Node Manager or you are not authorized to perform the action requested. No action will be performed.
In Adminserver log i am getting below warning
<[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1505972472314>
<[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1505972472314>
<[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1505972472314>
<[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1505972472315>
Note:-- I enrolled NM Control on Node-2 also
To be able to start a Managed Server via the Admin Server, the Node Manager needs to be running and obviously be reachable.
You can start the Node Manager with the startNodeManager.sh script in <weblogic-home>/server/bin.
If the Managed Server is already running you can shut it down through the Admin Server even without the Node Manager, because then the Managed Server is already in communication with the Admin Server.

Weblogic nodemanager errors/timeout

Running a Windows domain on a NAS, we infrequently see this error. We're worried it may be our configuration causing a problem, possibly some nodemanager.properties setting, a WLST call, etc.
Eventually we are unable to command the NodeManager via JMX and WLST. Cycling the NM works but is not a good solution. Is there something else we can try via WLST?
<Jan 22, 2015 6:09:28 PM> <SEVERE> <Domain1> <ms1> <Unexpected error while monitoring server>
java.lang.Error: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.create(FileKey.java:27)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.<init>(FileChannelImpl.java:1037)
at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:806)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:823)
at java.nio.channels.FileChannel.lock(FileChannel.java:860)
at weblogic.nodemanager.util.ConcurrentLockedFile.read(ConcurrentLockedFile.java:54)
at weblogic.nodemanager.common.StateInfo.load(StateInfo.java:130)
at weblogic.nodemanager.server.AbstractServerMonitor.loadStateInfo(AbstractServerMonitor.java:497)
at weblogic.nodemanager.server.AbstractServerMonitor.runMonitor(AbstractServerMonitor.java:273)
at weblogic.nodemanager.server.AbstractServerMonitor.run(AbstractServerMonitor.java:246)
at weblogic.nodemanager.server.ServerMonitor.run(ServerMonitor.java:25)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.init(Native Method)
at sun.nio.ch.FileKey.create(FileKey.java:25)
... 11 more

Deploying ear to Weblogic 12c

iI have an ear file which successfully deployed on JBoss7 & WebSphere 8.5
When trying on WebLogic 12c I'm getting this error at the AdminServer.log:
####<Apr 10, 2013 5:29:36 PM IDT> <Error> <J2EE> <wbl12t2.autowblciddmeronserver.autop> <AdminServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <> <1365604176997> <BEA-160228> <AppMerge failed to merge your application. Merge again with the -verbose option for more details. See the error message(s) below.>
####<Apr 10, 2013 5:30:09 PM IDT> <Debug> <Socket> <wbl12t2.autowblciddmeronserver.autop> <AdminServer> <ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'> <<WLS Kernel>> <> <> <1365604209078> <BEA-000450> <Socket 420 internal data record unavailable (probable closure due idle timeout), event received -32>
and this in the web console:
Error Unable to access the selected application.
Error Exception in AppMerge flows' progression
Error Exception in AppMerge flows' progression
Error com.demo.tasks.api.RTExecutor
Error com.demo.tasks.api.RTExecutor
The ear use jax-ws and jax-rs (using Jersey dependency).
the class com.demo.tasks.api.RTExecutor is interface with #WebService annotation.
Does anyone know what the problem is?
Is there anyway to get more details about the problems (more log files I miss, etc..)?
I was receiving the same error code and, after a bit of digging, I found out that Weblogic wants the extension of you EAR to be all lower case, otherwise it considered it invalid.
So your application must be called YourApp.ear and cannot be called YourApp.EAR for example.

Stuck weblogic thread when starting managed server using node manager

We are receiving the following error -- a thread gets stuck -- when we try to start any managed server in our cluster via the admin console in Weblogic 10.3.2. The managed servers used to start up fine last week. The only difference between this week and last week is that our VM (virtual machine) was powered down for a week. We started it back up and tried to bring up the managed servers and we get this error. My inkling is that something changed in the physical host or VM configuration (external to our operating system), but I have been told that none of that had been changed.
We are running WebLogic on Windows Server 2008.
Any ideas?
<Jan 3, 2011 6:10:45 PM GMT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy fo
r "228" seconds working on the request "weblogic.kernel.WorkManagerWrapper$1#291d575", which is more than the configured time (StuckThreadMaxTime) of "120" second
s. Stack trace:
jrockit.net.SocketNativeIO.readBytesPinned(Native Method)
jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:32)
java.net.SocketInputStream.socketRead0(SocketInputStream.java)
java.net.SocketInputStream.read(SocketInputStream.java:129)
weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:159)
java.io.InputStream.read(InputStream.java:85)
com.certicom.tls.record.ReadHandler.readFragment(Unknown Source)
com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
com.certicom.tls.record.ReadHandler.read(Unknown Source)
com.certicom.io.InputSSLIOStreamWrapper.read(Unknown Source)
sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
java.io.InputStreamReader.read(InputStreamReader.java:167)
java.io.BufferedReader.fill(BufferedReader.java:136)
java.io.BufferedReader.readLine(BufferedReader.java:299)
java.io.BufferedReader.readLine(BufferedReader.java:362)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:287)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:312)
weblogic.nodemanager.client.NMServerClient.start(NMServerClient.java:101)
weblogic.nodemanager.mbean.StartRequest.start(StartRequest.java:75)
weblogic.nodemanager.mbean.StartRequest.execute(StartRequest.java:47)
weblogic.kernel.WorkManagerWrapper$1.run(WorkManagerWrapper.java:63)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
For us, we received this error because our Weblogic instances, which are configured with an Active Directory provider could not communicate with Active Directory. Once we fixed the communication issue, this problem went away.