I'm trying to generate a full heapdump from Weblogic 10.3.6 due to an OutOfMemoryError generated by a Web Application deployed on the Server.
I've setted the following start script:
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/heapdump
When the OutOfMemoryError occurs, Weblogic generates an empty hprof file (0 bytes size) in /path/to/heapdump folder, and nothing happens: the Server remains in RUNNING mode, even if is not reachable anymore.
The java process is still alive, but with 0% of processor.
Even the server.out log seems completely frozen, without any trace of the OutOfMemoryError.
What's wrong with the configuration?
Probably you can use Java Flight Recorder to save events and check which objects are generating OOM.
(any profiler should work as well).
Been there :( . I remember at the time that we've found it was somewhat logical since there was not enough memory for normal operation, the JVM could not automagically find enough memory to create a heapdump either. If memory serves me well, at that time we did 2 things to debug the memory leak. First we were "lucky" enough that the problem was happening fairly regularly so a close manual monitoring was possible (monitoring of the gc.log looking for repeated FullGC and monitoring of the performance tab in the console). Knowing when the onset of the problem was starting we were doing some kill -3 to get the dump manually. We also used jstack {PID} (JDK 1.6 on Linux) with some luck. With those, at the time, the devs were able to identify the memory leak. Hope that helps.
Okay, your configuration looks alright.. you might want to check if the weblogic process user has the rights to edit the heap dump file.
You can take heap dump by Java tools :
JAVA_HOME/bin/jmap -dump:format=b,file=path_of_the_file
OR
%JROCKIT_HOME%\bin\jrcmd hprofdump filename=path_of_the_file
Related
I have a program in Apache + Delphi XE7 with Datasnap. My httpd.exe of Apache consume memory until give messsage "Out of Memory" then I need to restart it.
I developed a simple program just testing this problem. In this program I ve configured Server Module (FD components, Mysql) and WebModule appropriately
In client I have a DBGrid that shows the content of a query.
I open TaskManager and monitor the memory.
The problems:
if I connect and disconnect TSQLConnection the memory increase.
If I open and Close TClientDataSet the memory increase.
What I have already did:
I used Pascal Analyzer searching memory leaks but it´s all fine
I couldn't use ReportMemoryLeaksOnShutdown in the server, because is a DLL, and I dont know how to do this :(
I tried to turn off keep alive of Apache but there is no difference
I ve tried all types of LifeCycle in WebModule (Session, Server and Invocation) no difference too.
Anyone has a problem like this? work with this softwares? And could help?
Tks
I have packstack-allinone setup on my RHEL7.1 trial for Juno release.
I am facing problem while launching VM(for ex: cirros) with a disk size mentioned in flavor. If there is 0gb disk size then VM are getting launched but not for higher flavor sizes.
I also observe that when I do this, openstack-nova-compute service goes down which I observed when I checked using nova-manage service list with nova-compute being XXX making me restart the service everytime I try this scenario. The compute logs doesn't throw any error, it just gets stuck at "Creating image".
Is there any Filesystem issue which i missing to be configured? I am new to this, so please help.
PS: I run all commands with "root" user.
The problem was with esxi. Esxi needs to be 5.5v to support RHEL7x Since mine was 5.1v it only supported RHEL6x.
After upgrading esxi5.1 to 5.5v it worked fine.
I recently was modifying some of my server properties in Rational Application Developer to try and increase the memory of my JVM on startup. I forgot to take a backup before doing this, and by adding in an incorrect JVM variable, it seems I have broke my server in an unworking state. Whenever I try and startup my server to do any configuration changes, the JVM refuses to start with invalid params being passed in.
Is there a way to reset any JVM changes for WebSpehere Application Server v7.0 through the filesystem, or a way to do it without needing the server running already? I have been looking around in the wasProfile hoping to stumble onto a file where my settings ultimately live, but have had no luck.
It should be possible to write a wsadmin script to view/adjust the JVM options, but if you're on a non-z/OS platform, the fastest way to get back to working is probably to edit PROFILE_HOME/config/cells/CELL/nodes/NODE/servers/SERVER/server.xml; the JVM settings are typically written at the very end.
I have deployed a system integrated with weblogic, but until now I faced a problem is the weblogic increasing the stdout.out size heavily(by GB per week), it caused the system to load slowly and slowly.
Any way to prevent it increase the size heavily or redirect into .log?
Thanks alot
As David Herget says above, using the WebLogic Scripting Tool (WLST) to redirect StdOut and StdErr did not actually work for me either; I had to also do so through the web console (even though they appear to be set on the console) and restart the relevant jvms.
I can't reply to David's comment above due to being a newbie. [Edited since for clarity]
Not totally sure to understand fully your question.
Are you talking about the {server_name}.out file located in the {Domain_Path}/servers/{server_name}/logs ?
If so, I've never found anyway to rotate those logs automatically so I run a script each day to rotate it (basically copying it to another name, zip it and echoing a NULL in the orginal file...erasing the older one after).
If you are talking about redirecting StdOut to the logs though, that can be done within the console for each server in the logging tab by checking "Redirect stdout logging enabled". Configuration to rotate those logs can also be done within that tab.
On that, StdErr can also be redirected, but not from the console (in WL9). You have to put "RedirectStderrToServerLogEnabled" at true in the MBean tree by wlst (it's located at /Servers/{server_name}/Log/{server_name}
I know the question was ask long time ago but hoping it would help nonetheless
Weblogic provides features of log files rotation based on the size and time interval.
You can try rotating the log files based on the size. You would need to configure the log rotation policy from the admin console. Please refer the below link for further details.
http://docs.oracle.com/cd/E12840_01/wls/docs103/ConsoleHelp/taskhelp/logging/RotateLogFiles.html
If you want to rotate the log files on demand, you can use the below WSLT script.
C:\>java weblogic.WLST
#connect WLST to an Administration Server
wls:/offline> connect('username','password')
#navigate to the ServerRuntime MBean hierarchy
wls:/mydomain/serverConfig> serverRuntime()
wls:/mydomain/serverRuntime>ls()
#navigate to the server LogRuntimeMBean
wls:/mydomain/serverRuntime> cd('LogRuntime/myserver')
wls:/mydomain/serverRuntime/LogRuntime/myserver> ls()
-r-- Name myserver
-r-- Type LogRuntime
-r-x forceLogRotation java.lang.Void :
#force the immediate rotation of the server log file
wls:/mydomain/serverRuntime/LogRuntime/myserver> cmo.forceLogRotation()
wls:/mydomain/serverRuntime/LogRuntime/myserver>
http://docs.oracle.com/cd/E12840_01/wls/docs103/logging/config_logs.html#wp1001654
I have created a web application in Apache Cocoon.This website is running properly but after every 3-4 days, it stops responding. It doesn't run until and unless, we restart the tomcat service. In the catalina.2011-05-09.log file, it shows following error:-
"May 9, 2011 3:17:34 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/webresources] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation."
I am not been able to understand the cause of this problem. Can someone suggest me how to resolve this issue?
You are using a library that is starting one or more threads and is not properly shutting them down or releasing other resources captured by the thread. This often happens with things like Apache HTTP components (I get this error with Http Components) and anything that uses separate threads internally. What libraries are you using in your Cocoon application?
It is telling you the issue:
[...] is still processing a request that has yet to finish
You need to find out what that request is/is going to. One easy way is to have something like PsiProbe installed.
Also, it's not a bad idea to restart Tomcat every night. It can help alleviate these kinds of issues until you find the root cause.