I got a message that if we set min and max of heap size and young generation size to equal separately from [Java Performance], jvm will disable AdaptiveSizePolicy function automatically.
But this work failed in my test bed of jdk1.6 and 1.7.
Only add option -XX:-UseAdaptiveSizePolicy will disable JVM auto scale the young and heap size.
Set -XX:-UseAdaptiveSizePolicy
Related
I wrote a program using JMX API to calculate JVM Usage percentage. Code makes use of 2 attributes of java.lang:type=Memory MBean (used and max attribute).
When I use the formula (used/max * 100) to calculate JVM Used Memory Percent, it gives me very different value than what is displayed in JMC Software. For example:
In JMC I see the Percentage as 45.3%. But used = 708MB and max = 6GB, which in turn in my code gives very less percentile.
From Task Manager > Process Tab > Memory Column, I looked for the corresponding TomEE memory usage which is closed to the usage attribute in JMC.
Need some guidance here on what is the right way to calculate or look for the JVM Usage Percentage. Why there is a difference of percentage in JMC attribute (HeapMemoryUsagePercent) and the percentage calculation in my code?
I'm using GraphDb Free 8.6.1 in research project, I'm running it with default configuration on linux server having 4GB memory.
However, it has started to throw exceptions pointing to insufficient memory:
Caused by: org.eclipse.rdf4j.repository.RepositoryException: Query evaluation error: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb (HTTP status 500)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.execute(SPARQLProtocolSession.java:1143)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.executeOK(SPARQLProtocolSession.java:1066)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQueryViaHttp(SPARQLProtocolSession.java:834)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.getTupleQueryResult(SPARQLProtocolSession.java:763)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQuery(SPARQLProtocolSession.java:391)
at org.eclipse.rdf4j.repository.http.HTTPTupleQuery.evaluate(HTTPTupleQuery.java:69)
Please, can you help me to identify the problem?
How can I properly configure GraphDB?
The behavior you observe is part of memory optimization of distinct/group by operations. The error message itself is related to the default threshold of 250 mb and it's there to let you know you need to adjust your memory. When the free heap memory became less than the threshold, a QueryEvaluationException is thrown so to avoid running out of memory due to hungry distinct/group by operation. You can adjust the threshold to minimize those errors, by reducing it with with passing the following argument when starting GraphDB "-Ddefaut.min.distinct.threshold=XXX" (which could be set to the amount of memory in bytes for the threshold).
Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
238Mb = free heap space reported by the JVM
250Mb = the default threshold below which the protection should raise an exception to prevent OME
0Mb = the current buffer used for distinct and group by
I suspect another operation takes most of your RAM and once you run any DISTINCT/GROUP BY query it immediately stop because of the OME protection.
This answer would have helped me.
Example, if your machine has 32 GB RAM:
/opt/graphdb-free/app/graphdb-free.cfg (cutout)
[JVMOptions]
-Xms20G
-Xmx20G
-XX:PermSize=4G
-XX:MaxPermSize=4G
-Dfile.encoding=UTF-8
-Djava.net.preferIPv4Stack=true
--add-exports
jdk.management.agent/jdk.internal.agent=ALL-UNNAMED
--add-opens
java.base/java.lang=ALL-UNNAMED
Via GUI and Settings:
graphdb.page.cache.size 10G
Is there anyway to get information about how many Garbage collection been performed for different generations from a dump file. When I try to run some psscor4 commands I get following.
0:003> !GCUsage
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to
displaying, finding or traversing objects as well as gc heap segments may not
work properly. !dumpheap and !verifyheap may incorrectly complain of heap
consistency errors.
Error: Requesting GC Heap data
0:003> !CLRUsage
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to
displaying, finding or traversing objects as well as gc heap segments may not
work properly. !dumpheap and !verifyheap may incorrectly complain of heap
consistency errors.
Error: Requesting GC Heap data
I can get output from eehpeap though, but it does not give me what I am looking for.
0:003> !EEHeap -gc
Number of GC Heaps: 1
generation 0 starts at 0x0000000002c81030
generation 1 starts at 0x0000000002c81018
generation 2 starts at 0x0000000002c81000
ephemeral segment allocation context: none
segment begin allocated size
0000000002c80000 0000000002c81000 0000000002c87fe8 0x6fe8(28648)
Large object heap starts at 0x0000000012c81000
segment begin allocated size
0000000012c80000 0000000012c81000 0000000012c9e358 0x1d358(119640)
Total Size: Size: 0x24340 (148288) bytes.
------------------------------
GC Heap Size: Size: 0x24340 (148288) bytes.
Dumps
You can see the number of garbage collections in performance monitor. However, the way performance counters work makes me believe that this information is not available in a dump file and probably even not available during live debugging.
Think of Debug.WriteLine(): once the text was written to the debug output, it is gone. If you didn't have DebugView running at the time, the information is lost. And that's good, otherwise it would look like a memory leak.
Performance counters (as I understand them) work in a similar fashion. Various "pings" are sent out for someone else (the performance monitor) to be recorded. If noone does, the ping with all its information is gone.
Live debugging
As already mentioned, you can try performance monitor. If you prefer WinDbg, you can use sxe clrn to see garbage collections happen.
PSSCOR
The commands you mentioned, do not show information about garbage collection count:
0:016> !gcusage
Number of GC Heaps: 1
------------------------------
GC Heap Size 0x36d498(3,593,368)
Total Commit Size 0000000000384000 (3 MB)
Total Reserved Size 0000000017c7c000 (380 MB)
0:016> !clrusage
Number of GC Heaps: 1
------------------------------
GC Heap Size 0x36d498(3,593,368)
Total Commit Size 0000000000384000 (3 MB)
Total Reserved Size 0000000017c7c000 (380 MB)
Note: I'm using PSSCOR2 here, since I have the same .NET 4.5 issue on this machine. But I expect the output of PSSCOR4 to be similar.
I am running this code on AIX 6.1
while(true)
{
int a = rand(); //generate a random integer value
void* test = malloc(a*a); //allocate large chunk of memory block
usleep(3000000); //sleep for 3 sec
free(test); // release memory block
}
using MALLOCTYPE=buckets
My observation is
Resident set size(real memory) and data section size for process is continuously increasing. This is check by command ps v PID
pg sp value shown in topas for process is slowly increasing.
Can someone justify this behavior.
On free, memory is not released to AIX os, but it is reserved for reuse. With MALLOCOPTIONS=disclaim, free releases memory back to AIX os and their is not increase in memory utilization. But with MALLOCOPTIONS=disclaim, CPU utilization is almost 2-3 times greater.
We are creating a Real-Time Process in VxWorks 6.x, and we would like to limit the amount of memory which can be allocated to the heap. How do we do this?
When creating a RTP via rtpSpawn(), you can specify an environment variable which controls how the heap behaves.
There are 3 environment variables:
HEAP_INITIAL_SIZE - How much heap to allocate initially (defaults to 64K)
HEAP_MAX_SIZE - Maximum heap to allocate (defaults to no limit)
HEAP_INCR_SIZE - memory increment when adding to RTP heap (defaults to 1 virtual page)
The following code shows how to use the environment variables:
char * envp[] = {"HEAP_INITIAL_SIZE=0x20000", "HEAP_MAX_SIZE=0x100000", NULL);
rtpSpawn ("myrtp.vxe", NULL, envp, 100, 0x10000, 0, 0);
This can be done through the use of the HEAP_MAX_SIZE environment variable. If it is set, it limits the ability of the heap to grow beyond that size. It does not, however, limit the initial heap size.
See page 31