when I deploy a Mule API and after deployment, I do some changes and save it then my API gets redeploy.
during redeployment, I am getting the below error in the console that stops the deployment.
java.lang.OutOfMemoryError: Metaspace
Dumping heap to java_pid19656.hprof ...
Heap dump file created [197920637 bytes in 0.811 secs]
#
# java.lang.OutOfMemoryError: Metaspace
# -XX:OnOutOfMemoryError="taskkill /F /PID %p"
# Executing "taskkill /F /PID 19656"...
JVM exited unexpectedly.
Automatic JVM Restarts disabled. Shutting down.
<-- Wrapper Stopped
anyone can help me to fix this issue?
Thanks
Probably you are having a known issue related to redeployments that cause the metaspace are to get exhausted. It is recommended to use the latest version of Mule and the latest version of every connector used in your applications to mitigate that issue.
Also ensure that the MetaspaceSize is half of MaxMetaspaceSize. You can increase MaxMetaspaceSize if you feel you are deploying lots of classes or applications, but keep the proportion mentioned.
wrapper.java.additional.7=-XX:MetaspaceSize=128m
wrapper.java.additional.8=-XX:MaxMetaspaceSize=256m
Related
I had a question regarding karate.robot do we have any method or function to shut down or close or quit Karate.robot session like driver. quit or close?
it seems some threads are occupied getting following error many times:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000065a03e06, pid=11236,
tid=8836
#
# JRE version: Java(TM) SE Runtime Environment (8.0_25-b18) (build 1.8.0_25-b18)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.25-b02 mixed mode windows-amd64
compressed oops)
# Problematic frame:
# V [jvm.dll+0x4c3e06]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions
of Windows
#
[thread 6836 also had an error]
[.error occurred during error reporting , id 0xc0000005]
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
Dll Process Attached
Loading jawt.dll
Dll Process Detach
Process finished with exit code 1
No, we don't see a need. You typically start the Robot instance and it stays up until the end of your entire suite. Maybe you should try install the JDK 64-bit or 32-bit.
You are welcome to contribute code to improve anything if required. So far no one has reported any problems like this. Maybe you are trying to do things in parallel threads which is not supported. Provide a way to replicate if you can: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
EDIT - one area you can help us investigate is if we need to do more to release JNA resources after a Scenario in this method.
Also see this answer: Java JNA: JRE crashes after application completes
I'm using GraphDb Free 8.6.1 in research project, I'm running it with default configuration on linux server having 4GB memory in total.
Currently, we execute quite many CRUD operations in tripplestore.
GraphDB throwed exception in console:
java.lang.OutOfMemoryError: Java heap space
-XX:OnOutOfMemoryError="kill -9 %p"
Executing /bin/sh -c "kill -9 1411"...
Looking into process, GraphDB runs with parameter XX:MaxDirectMemorySize=128G
I was not able to changed, even with ./graph -Xmx3g, process is still running with XX:MaxDirectMemorySize=128G.
I've tried to configure ./grapdh parameter, setting the GDB_HEAP_SIZE=3072m, now process runs with additional -Xms3072m -Xmx3072m parameters, but remains XX:MaxDirectMemorySize=128G.
After update to GDB_HEAP_SIZE=3072m, repository went down again without .hprof file, no exception, nothing suspicious in logs. The following line was flushed into console: Java HotSpot(TM) 64-Bit Server VM warning:
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f5b4b6d0000, 65536, 1) failed; error='Cannot allocate memory' (errno=12)
Please, can you help me to properly configure GraphDB tripplestore to get rid of the Heap Space exceptions?
Thank you.
By default, the value of the -XX:MaxDirectMemorySize (off heap memory) parameter in the JVM is equal to the -XMx (on heap memory). For very large repositories the size of the off heap memory may become insufficient so the GraphDB developers made this parameter 128GB or unlimited.
I suspect that your actual issue is actually allocating too much on heap memory, which leaves no space for the off heap in the RAM. When the database tries to allocate off heap RAM you hit this low OS-level error 'Cannot allocate memory'.
You have two options in solving this problem:
Increase the RAM of the server to 8GB and keep the same configuration - this would allow the 8 GB RAM to be distributed: 2GB (OS) + 3GB (on heap) + 3GB (off heap)
Decrease the -Xmx value to 2GB so the 4GB RAM will be distributed: 1GB (OS) + 2GB (on heap) + 1GB (off heap)
To get a good approximation how much RAM GraphDB needs please check the hardware sizing page:
http://graphdb.ontotext.com/documentation/8.6/free/requirements.html
I have a 2 node OpenStack Mitaka environment consisting of a controller/compute node and a compute node.
I've followed the setup guide to enable instance live migration using LVM block storage. I.e.: There's no shared storage backend, just local LVM block storage.
Using OpenStack Horizon to perform the live migration a success message is displayed, however, the migration is far from successful. This worked pretty much out-of-the-box with our Juno installation. I've exhausted Google and cannot find any other instances of people facing the same problem. I thought it might have been a time synchronisation problem so have set both nodes to UTC. Still the problems persists.
Source machine /var/log/nova/nova-compute.log
2016-08-12 15:56:42.120 2230 ERROR nova.virt.libvirt.driver [req-b71ea7b0-5fa8-4b57-92d2-4edec62135c2 b017d86d1143461a92a267d4b912c104 88c686f09e1b427fb750f5c00716f84e - - -] [instance: 5763b6b6-370c-448c-8e8f-8b71eafaa8f1] Migration operation has aborted
2016-08-12 15:56:42.470 2230 ERROR nova.virt.libvirt.driver [req-b71ea7b0-5fa8-4b57-92d2-4edec62135c2 b017d86d1143461a92a267d4b912c104 88c686f09e1b427fb750f5c00716f84e - - -] [instance: 5763b6b6-370c-448c-8e8f-8b71eafaa8f1] Live Migration failure: internal error: unable to execute QEMU command 'migrate': Migration disabled: failed to allocate shared memory
Target node /var/log/libvirt/libvirtd.log
2016-08-12 15:56:41.864+0000: 2170: error : qemuMonitorJSONGetMigrationStatsReply:2443 : internal error: info migration reply was missing return status
2016-08-12 15:56:41.864+0000: 2170: error : virNetClientProgramDispatchError:177 : Cannot open log file: '/var/log/libvirt/qemu/instance-0000006a.log': Device or resource busy
There are no other events captured in the source or target nova or libvirt logs.
I should also note that I am trying to use qemu+tcp (libvirt listening enabled, default tcp port, no auth) rather than qemu+ssh in order to keep things simple while testing. In fact, I intend to only use qemu+tcp anyway.
Which version of ubuntu did you deploy?
I had the same error with ubuntu 14.04 and mitaka version.
And I figured out that default kernel (3.13) makes this problem.
I upgraded the kernel from 3.13 to 4.40 and this problem is gone now.
I hope my experience help you solve this problem out.
Thanks
Currently we are encountering an problem, our java application can run smoothly with JRE6, but for JRE7, it will report "Could not create the Java virtual machine" sometimes and failed to start program. we have "-Xmx1024m" option in the launch command line.
after searching in google, people suggest check the max heap size that can be requested in PC. then I tried below ones.
for JRE6,
C:\Users\joey>java -Xmx1214m -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) Client VM (build 20.45-b01, mixed mode)
C:\Users\joey>java -Xmx1215m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
for JRE7,
D:\Program Files\Java\jre7\bin>java.exe -Xmx930m -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) Client VM (build 24.51-b03, mixed mode)
D:\Program Files\Java\jre7\bin>java.exe -Xmx931m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
And when i close most processes in PC and free memory to 2.5 GB, for JRE7, it will succeed with 1094m, like below.
D:\Program Files\Java\jre7\bin>java.exe -Xmx1094m -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) Client VM (build 24.51-b03, mixed mode)
D:\Program Files\Java\jre7\bin>java.exe -Xmx1095m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
It seems that the max heap size that JVM can request is related with system available memory size, does someone know the exact relationship?
Is there some way to enable tracing logs of JRE when creating JVM?
Thank you!
Regards,
Joey
I faced the same issue on various Windows OS flavous (Win7, Win2008R2) while having enough free RAM (3-5G), when switching from jre6_20 to jre7_51 (both 32 bit VMs)
Using vm args -Xmx1300m and -XX:MaxPermSize=256m on java 6 the VM is able to start. However, with java7 I had to lower down startup max memory (xmx with 100m or permgen with 100m) in order to be able to start VM.
The interesting part is this: if I restarted my box, the jvm 7 started with original vm args (-Xmx1300m and -XX:MaxPermSize=256m) so I suspect the root cause might be the OS with JVM combination.
I know that on Windows OSes - 32 bit VMs expect to have a single large chunk of memory, but cannot use the maximum process memory size will range from 1.4G to 1.6G (because of additional constraints)
Any troubleshooting pointers? Oracle does not provide any information in java7 release notes (or I could not spot those)
I have a similar problem Windows 7 64bit, no JAVA_HOME set nor in PATH
This fails
"C:\Program Files (x86)\Java\jdk1.7.0_51\bin\java.exe" -Xms32m -Xmx1024m -XX:MaxPermSize=256m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
This works
"C:\Program Files (x86)\Java\jdk1.7.0_51\bin\java.exe" -Xms32m -Xmx512m -XX:MaxPermSize=256m -version
This also works (I can recompile or not, doesn't make a difference
"C:\Program Files (x86)\Java\jdk1.6.0_45\bin\java.exe" -Xms32m -Xmx1024m -XX:MaxPermSize=256m -version
I tried turning off windows DEP, didnt make a difference
http://windows.microsoft.com/en-us/windows-vista/data-execution-prevention-frequently-asked-questions
I twiddled with the Virtual Memory settings , changed from allowing Windows to Manage
http://windows.microsoft.com/en-us/windows/change-virtual-memory-size#1TC=windows-7
I upped the Virual Memory in windows and it allowed it to work, sometimes...(with xmx1024)
Nothing else is consuming a lot of resources (memory, etc, I did have oracle database running locally, but shut it down)
"C:\Program Files (x86)\Java\jdk1.7.0_51\bin\java.exe" -Xms32m -Xmx1024m -XX:MaxPermSize=256m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
This works
"C:\Program Files (x86)\Java\jdk1.7.0_51\bin\java.exe" -Xms32m -Xmx900m -XX:MaxPermSize=256m -version
This fails
U:>"C:\Program Files (x86)\Java\jdk1.7.0_51\bin\java.exe" -Xms32m -Xmx2024m -XX
:MaxPermSize=256m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
New version , works even with letting Windows Managage the Virtual Memory and DEP turned on
"C:\Program Files (x86)\Java\jdk1.7.0_65\bin\java.exe" -Xms32m -Xmx1024m -XX:MaxPermSize=256m -version
I am setting up a GlassFish cluster following the guide at http://javadude.wordpress.com/2011/04/25/glassfish-3-1-clustering-tutorial/. I started from fresh installs of GlassFish 3.1.1. I also have the same architecture as in the guide: two nodes with one instance each. The DAS is on node1.
I've tried starting from scratch several times and am able to create the cluster, nodes and instances without issue. I also have the DAS communicating with node2 via SSH. However, each time when I get to the point where I attempt to start instance2 it fails:
$ ./asadmin start-local-instance --node node1 --sync normal instance2
Previous synchronization failed at Feb 23, 2012 2:41:53 PM
Will perform full synchronization.
Removing all cached state for instance instance2.
CLI802 Synchronization failed for directory config, caused by:
javax.xml.bind.JAXBException
- with linked exception:
[java.lang.ClassNotFoundException: com.sun.xml.bind.v2.ContextFactory]
Command start-local-instance failed.
I spent the day Googling and searching GlassFish's Jira, but couldn't find a solution to this issue. I'd very much appreciate any ideas you have on how to solve this problem.
My operating system is CentOS 5.7 and my Java version is 1.6.0_20
Unfortunately my instance directory is empty, I'm assuming because it never started. So there is no log file. I set AS_DEBUG=true but it gives no stack trace. The last debug lines before the error are
Removing all cached state for instance instance2.
Removing: /usr/local/glassfish3_1_1/glassfish/nodes/blade-50/instance2/config
Removing: /usr/local/glassfish3_1_1/glassfish/nodes/blade-50/instance2/applications
Removing: /usr/local/glassfish3_1_1/glassfish/nodes/blade-50/instance2/generated
Removing: /usr/local/glassfish3_1_1/glassfish/nodes/blade-50/instance2/lib
Removing: /usr/local/glassfish3_1_1/glassfish/nodes/blade-50/instance2/docroot
Got exception: javax.xml.bind.JAXBException
Acting on a tip from a user in the Glassfish forum, I learned that Java 1.6.0_20 is an older release of Java that is not supported by Glassfish 3.1.1. I worked with a sysadmin to get Java 1.6.0_31 installed on both nodes of the cluster and that did the trick--both instances start up without errors.