I'm using GraphDb Free 8.6.1 in research project, I'm running it with default configuration on linux server having 4GB memory.
However, it has started to throw exceptions pointing to insufficient memory:
Caused by: org.eclipse.rdf4j.repository.RepositoryException: Query evaluation error: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb (HTTP status 500)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.execute(SPARQLProtocolSession.java:1143)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.executeOK(SPARQLProtocolSession.java:1066)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQueryViaHttp(SPARQLProtocolSession.java:834)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.getTupleQueryResult(SPARQLProtocolSession.java:763)
at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQuery(SPARQLProtocolSession.java:391)
at org.eclipse.rdf4j.repository.http.HTTPTupleQuery.evaluate(HTTPTupleQuery.java:69)
Please, can you help me to identify the problem?
How can I properly configure GraphDB?
The behavior you observe is part of memory optimization of distinct/group by operations. The error message itself is related to the default threshold of 250 mb and it's there to let you know you need to adjust your memory. When the free heap memory became less than the threshold, a QueryEvaluationException is thrown so to avoid running out of memory due to hungry distinct/group by operation. You can adjust the threshold to minimize those errors, by reducing it with with passing the following argument when starting GraphDB "-Ddefaut.min.distinct.threshold=XXX" (which could be set to the amount of memory in bytes for the threshold).
Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
238Mb = free heap space reported by the JVM
250Mb = the default threshold below which the protection should raise an exception to prevent OME
0Mb = the current buffer used for distinct and group by
I suspect another operation takes most of your RAM and once you run any DISTINCT/GROUP BY query it immediately stop because of the OME protection.
This answer would have helped me.
Example, if your machine has 32 GB RAM:
/opt/graphdb-free/app/graphdb-free.cfg (cutout)
[JVMOptions]
-Xms20G
-Xmx20G
-XX:PermSize=4G
-XX:MaxPermSize=4G
-Dfile.encoding=UTF-8
-Djava.net.preferIPv4Stack=true
--add-exports
jdk.management.agent/jdk.internal.agent=ALL-UNNAMED
--add-opens
java.base/java.lang=ALL-UNNAMED
Via GUI and Settings:
graphdb.page.cache.size 10G
Related
I'm getting OutOfMemory error in Apache Lucene.
Here is the problem code:
DirectoryReader oldReader = directoryReader;
DirectoryReader newReader = DirectoryReader.openIfChanged(directoryReader);
if ((newReader != null) & (oldReader != newReader)) {
directoryReader = newReader;
}
Here is the log:
Caused by: java.lang.OutOfMemoryError: Java Heap Space
at java.lang.Class.getMethodImpl(Native Method)
at java.lang.Class.getMethod(Class.java:917)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput$1.run(MMapDirectory.java:244)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput$1.run(MMapDirectory.java:241)
at java.security.AccessController.doPrivileged(AccessController.java:327)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.freeBuffer(MMapDirectory.java:241)
at org.apache.lucene.store.ByteBufferIndexInput.close(ByteBufferIndexInput.java:295)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:788)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at org.apache.lucene.index.StandardDirectoryReader.isCurrent(StandardDirectoryReader.java:349)
at org.apache.lucene.index.StandardDirectoryReader.doOpenNoWriter(StandardDirectoryReader.java:303)
at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:266)
at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:254)
at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
Any idea, what can be the problem?
Since you're using MMapDirectory, you should be aware, that:
NOTE: memory mapping uses up a portion of the virtual memory address
space in your process equal to the size of the file being mapped.
Before using this class, be sure your have plenty of virtual address
space, e.g. by using a 64 bit JRE, or a 32 bit JRE with indexes that
are guaranteed to fit within the address space. On 32 bit platforms
also consult MMapDirectory(Path, LockFactory, int) if you have
problems with mmap failing because of fragmented address space. If you
get an OutOfMemoryException, it is recommended to reduce the chunk
size, until it works.
MMapDirectory uses memory-mapped IO when reading. This is a good
choice if you have plenty of virtual memory relative to your index
size, eg if you are running on a 64 bit JRE, or you are running on a
32 bit JRE but your index sizes are small enough to fit into the
virtual memory space
So, consider tuning your chunk size or choose other implementation of the Directory, like NIOFSDirectory or SimpleFSDirectory
when I try to export data, it runs out of memory, regardless of table size (even empty tables.)
Out of memory! ] 162926/498508267 rows (0.0%) on total estimated data (14 sec., avg: 11637 recs/sec)
Issuing rollback() due to DESTROY without explicit disconnect() of DBD::Oracle::db handle (DESCRIPTION=(ADDRESS=(PORT=1521)(HOST=192.168.0.42)
(PROTOCOL=tcp))(CONNECT_DATA=(SID=orcl))) at
/usr/local/lib/perl/5.18.2/DBD/Oracle.pm line 348.
Asking the author, I got this response:
You don't have enough memory. If you can't increase the memory size than reduce the value of DATA_LIMIT in ora2pg.conf. Try with 5000 and if it doesn't works use 2500.
Opened ./config/ora2pg.conf and modfied set DATA_LIMIT 5000 solved the issue.
I originally tried to add more RAM, but only doubled it from 2GB to 4GB, it did not help. Reducing the DATA_LIMIT was the solution.
I got a message that if we set min and max of heap size and young generation size to equal separately from [Java Performance], jvm will disable AdaptiveSizePolicy function automatically.
But this work failed in my test bed of jdk1.6 and 1.7.
Only add option -XX:-UseAdaptiveSizePolicy will disable JVM auto scale the young and heap size.
Set -XX:-UseAdaptiveSizePolicy
Is there anyway to get information about how many Garbage collection been performed for different generations from a dump file. When I try to run some psscor4 commands I get following.
0:003> !GCUsage
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to
displaying, finding or traversing objects as well as gc heap segments may not
work properly. !dumpheap and !verifyheap may incorrectly complain of heap
consistency errors.
Error: Requesting GC Heap data
0:003> !CLRUsage
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to
displaying, finding or traversing objects as well as gc heap segments may not
work properly. !dumpheap and !verifyheap may incorrectly complain of heap
consistency errors.
Error: Requesting GC Heap data
I can get output from eehpeap though, but it does not give me what I am looking for.
0:003> !EEHeap -gc
Number of GC Heaps: 1
generation 0 starts at 0x0000000002c81030
generation 1 starts at 0x0000000002c81018
generation 2 starts at 0x0000000002c81000
ephemeral segment allocation context: none
segment begin allocated size
0000000002c80000 0000000002c81000 0000000002c87fe8 0x6fe8(28648)
Large object heap starts at 0x0000000012c81000
segment begin allocated size
0000000012c80000 0000000012c81000 0000000012c9e358 0x1d358(119640)
Total Size: Size: 0x24340 (148288) bytes.
------------------------------
GC Heap Size: Size: 0x24340 (148288) bytes.
Dumps
You can see the number of garbage collections in performance monitor. However, the way performance counters work makes me believe that this information is not available in a dump file and probably even not available during live debugging.
Think of Debug.WriteLine(): once the text was written to the debug output, it is gone. If you didn't have DebugView running at the time, the information is lost. And that's good, otherwise it would look like a memory leak.
Performance counters (as I understand them) work in a similar fashion. Various "pings" are sent out for someone else (the performance monitor) to be recorded. If noone does, the ping with all its information is gone.
Live debugging
As already mentioned, you can try performance monitor. If you prefer WinDbg, you can use sxe clrn to see garbage collections happen.
PSSCOR
The commands you mentioned, do not show information about garbage collection count:
0:016> !gcusage
Number of GC Heaps: 1
------------------------------
GC Heap Size 0x36d498(3,593,368)
Total Commit Size 0000000000384000 (3 MB)
Total Reserved Size 0000000017c7c000 (380 MB)
0:016> !clrusage
Number of GC Heaps: 1
------------------------------
GC Heap Size 0x36d498(3,593,368)
Total Commit Size 0000000000384000 (3 MB)
Total Reserved Size 0000000017c7c000 (380 MB)
Note: I'm using PSSCOR2 here, since I have the same .NET 4.5 issue on this machine. But I expect the output of PSSCOR4 to be similar.
We are creating a Real-Time Process in VxWorks 6.x, and we would like to limit the amount of memory which can be allocated to the heap. How do we do this?
When creating a RTP via rtpSpawn(), you can specify an environment variable which controls how the heap behaves.
There are 3 environment variables:
HEAP_INITIAL_SIZE - How much heap to allocate initially (defaults to 64K)
HEAP_MAX_SIZE - Maximum heap to allocate (defaults to no limit)
HEAP_INCR_SIZE - memory increment when adding to RTP heap (defaults to 1 virtual page)
The following code shows how to use the environment variables:
char * envp[] = {"HEAP_INITIAL_SIZE=0x20000", "HEAP_MAX_SIZE=0x100000", NULL);
rtpSpawn ("myrtp.vxe", NULL, envp, 100, 0x10000, 0, 0);
This can be done through the use of the HEAP_MAX_SIZE environment variable. If it is set, it limits the ability of the heap to grow beyond that size. It does not, however, limit the initial heap size.
See page 31