We are receiving the following error -- a thread gets stuck -- when we try to start any managed server in our cluster via the admin console in Weblogic 10.3.2. The managed servers used to start up fine last week. The only difference between this week and last week is that our VM (virtual machine) was powered down for a week. We started it back up and tried to bring up the managed servers and we get this error. My inkling is that something changed in the physical host or VM configuration (external to our operating system), but I have been told that none of that had been changed.
We are running WebLogic on Windows Server 2008.
Any ideas?
<Jan 3, 2011 6:10:45 PM GMT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy fo
r "228" seconds working on the request "weblogic.kernel.WorkManagerWrapper$1#291d575", which is more than the configured time (StuckThreadMaxTime) of "120" second
s. Stack trace:
jrockit.net.SocketNativeIO.readBytesPinned(Native Method)
jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:32)
java.net.SocketInputStream.socketRead0(SocketInputStream.java)
java.net.SocketInputStream.read(SocketInputStream.java:129)
weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:159)
java.io.InputStream.read(InputStream.java:85)
com.certicom.tls.record.ReadHandler.readFragment(Unknown Source)
com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
com.certicom.tls.record.ReadHandler.read(Unknown Source)
com.certicom.io.InputSSLIOStreamWrapper.read(Unknown Source)
sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
java.io.InputStreamReader.read(InputStreamReader.java:167)
java.io.BufferedReader.fill(BufferedReader.java:136)
java.io.BufferedReader.readLine(BufferedReader.java:299)
java.io.BufferedReader.readLine(BufferedReader.java:362)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:287)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:312)
weblogic.nodemanager.client.NMServerClient.start(NMServerClient.java:101)
weblogic.nodemanager.mbean.StartRequest.start(StartRequest.java:75)
weblogic.nodemanager.mbean.StartRequest.execute(StartRequest.java:47)
weblogic.kernel.WorkManagerWrapper$1.run(WorkManagerWrapper.java:63)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
For us, we received this error because our Weblogic instances, which are configured with an Active Directory provider could not communicate with Active Directory. Once we fixed the communication issue, this problem went away.
Related
I try to run a *.sh file in linux system ,this *.sh file is used to start a Java Application.this Application use gemfire as its distributed cache system. it seems that i can not build a new tcp connection of gemfire. is there anyone knows how to solve this problem?
here is the exception:
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:437)
at sun.nio.ch.Net.bind(Net.java:429)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:378)
at com.gemstone.gemfire.internal.cache.BridgeServerImpl.start(BridgeServerImpl.java:297)
at spark.cache.CacheServicePoint.enableServer(CacheServicePoint.java:197)
at orion.di.service.profile.ProfileService.initialize(ProfileService.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1544)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1485)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1417)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
Address already in use is that already your application instance is running/listening on the same port. If you are an ubuntu user
netstat -tulpn | grep YOURAPPNAME
This will give you the ProcessId which is running the instance in your system. Find the process Id number and kill the instance and start the application once again.
To kill
kill -9 PROCESSID
Are you starting two processes of your Java application on the same box? GemFire server by default opens up a port on 40404 to listen to client connections, so when you start more than two servers on the same box, the second one gets an Address already in use exception. Look the script used to start your application. You will need to provide a different port for each GemFire server you are trying to start. Using GemFireShell i.e. gfsh this could be done like so:
gfsh>start server --name=server1 --server-port=4045
Or, if there are no clients (i.e. peer-to-peer deployment of GemFire), you can disable listening to clients like so:
gfsh>start server --name=server1 --disable-default-server
I am using the following on Linux
MFP 6.3
WAS Libery 8.5.5.6 (core trial)
Tried with JDK1.7 and JDK1.6 but nothing worked out
MySQL
I could not see any other error/exception in messages.log except this and I am not sure where to change the 'timeout' value in WAS Liberty profile.
http://pastebin.com/7uuVtjHL (server.xml)
http://pastebin.com/2ScrUQLa (messages.log)
Exception thrown by application class
'com.worklight.core.auth.impl.AuthenticationFilter.isWaitingForSynchronization:598'
javax.servlet.ServletException: java.lang.RuntimeException: Timeout
while waiting for the management service to start up.120 secs. at
com.worklight.core.auth.impl.AuthenticationFilter.isWaitingForSynchronization(AuthenticationFilter.java:598)
at
com.worklight.core.auth.impl.AuthenticationFilter.doFilter(AuthenticationFilter.java:141)
at
com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes] Caused by: java.lang.RuntimeException: Timeout
while waiting for the management service to start up.120 secs. at
com.worklight.core.init.WorklightServletInitializer$1.run(WorklightServletInitializer.java:121)
at java.lang.Thread.run(Thread.java:798)
Right now it does seem like you are experiencing the same issue as mentioned here: How to solve management service not starting up in Worklight 6.2
You are currently using IBM Java 1.7 per the messages.log file:
java.home = /usr/lib/jvm/java-1.7.0-ibm-1.7.0.9.0.x86_64/jre
Download Oracle Java 1.7 and make sure your java.home points to it. Start the server and see if there are any differences.
Instead, or in addition, you can try this: https://developer.ibm.com/answers/questions/184195/no-runtime-can-be-found-and-failed-to-obtain-jmx-c.html
In server.xml find the following:
<jndiEntry jndiName="ibm.worklight.admin.jmx.host" value="localhost"/>
Replace "localhost" with the Public IP address of the host machine and start the server.
Running a Windows domain on a NAS, we infrequently see this error. We're worried it may be our configuration causing a problem, possibly some nodemanager.properties setting, a WLST call, etc.
Eventually we are unable to command the NodeManager via JMX and WLST. Cycling the NM works but is not a good solution. Is there something else we can try via WLST?
<Jan 22, 2015 6:09:28 PM> <SEVERE> <Domain1> <ms1> <Unexpected error while monitoring server>
java.lang.Error: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.create(FileKey.java:27)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.<init>(FileChannelImpl.java:1037)
at sun.nio.ch.FileChannelImpl.fileLockTable(FileChannelImpl.java:806)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:823)
at java.nio.channels.FileChannel.lock(FileChannel.java:860)
at weblogic.nodemanager.util.ConcurrentLockedFile.read(ConcurrentLockedFile.java:54)
at weblogic.nodemanager.common.StateInfo.load(StateInfo.java:130)
at weblogic.nodemanager.server.AbstractServerMonitor.loadStateInfo(AbstractServerMonitor.java:497)
at weblogic.nodemanager.server.AbstractServerMonitor.runMonitor(AbstractServerMonitor.java:273)
at weblogic.nodemanager.server.AbstractServerMonitor.run(AbstractServerMonitor.java:246)
at weblogic.nodemanager.server.ServerMonitor.run(ServerMonitor.java:25)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: The semaphore timeout period has expired
at sun.nio.ch.FileKey.init(Native Method)
at sun.nio.ch.FileKey.create(FileKey.java:25)
... 11 more
iI have an ear file which successfully deployed on JBoss7 & WebSphere 8.5
When trying on WebLogic 12c I'm getting this error at the AdminServer.log:
####<Apr 10, 2013 5:29:36 PM IDT> <Error> <J2EE> <wbl12t2.autowblciddmeronserver.autop> <AdminServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <> <1365604176997> <BEA-160228> <AppMerge failed to merge your application. Merge again with the -verbose option for more details. See the error message(s) below.>
####<Apr 10, 2013 5:30:09 PM IDT> <Debug> <Socket> <wbl12t2.autowblciddmeronserver.autop> <AdminServer> <ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'> <<WLS Kernel>> <> <> <1365604209078> <BEA-000450> <Socket 420 internal data record unavailable (probable closure due idle timeout), event received -32>
and this in the web console:
Error Unable to access the selected application.
Error Exception in AppMerge flows' progression
Error Exception in AppMerge flows' progression
Error com.demo.tasks.api.RTExecutor
Error com.demo.tasks.api.RTExecutor
The ear use jax-ws and jax-rs (using Jersey dependency).
the class com.demo.tasks.api.RTExecutor is interface with #WebService annotation.
Does anyone know what the problem is?
Is there anyway to get more details about the problems (more log files I miss, etc..)?
I was receiving the same error code and, after a bit of digging, I found out that Weblogic wants the extension of you EAR to be all lower case, otherwise it considered it invalid.
So your application must be called YourApp.ear and cannot be called YourApp.EAR for example.
may i know if someone has encountered this before? Found this in WebLogic
managed server logs.
####<Jul 15, 2011 9:59:34 AM EST> <Error> <WTC> <mi009.aiu.com> <ilpmServer3> <ExecuteThread: '58' for queue: 'weblogic.kernel.Default'> <ICO_WS1> <> <BEA-180029> <Caught this Ferror exception: 5 (FBADFLD). Additional exception info found: Unknown fldid32: 168877871>
The exception explanation at WebLogic does not provide much information about the error.
It seems that WLS is calling a TUXEDO server via WTC Tuxedo server is calling a WLS services exported via WTC.
In any of both cases the FIELDID 168877871 is not mapped to the same variable by Tuxedo server OR WLS code.
Please verify with the developers of the app if the FIELDID 168877871 correspond to the same variable for WLS and Tuxedo Server.
In Tuxedo server please check the files of)
env | egrep "FIELDTBLS|FLDTBLDIR"
In Java verify the source.