Ontotext Parameter defaut.min.distinct.threshold not honored - graphdb

Question

Ontotext database throws
NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory ... , threshold:250Mb
although parameter defaut.min.distinct.threshold=1074000000
set.
It seems that setting the parameter is not taken into account by the Ontotext engine und thus we are still getting the error.
We are using Ontotext 8.10.1-Free in Docker.
Running the container we provide as much memory to the container as is set for the Xmx parameter
We tried different memory settings Xms, Xmx and the aforementioned threshold parameter. But in the logs it states that the threshold's default value of 250m is being used.
This is how our process info looks at runtime:
> ps auxw|grep java
root 1 85.4 47.1 5729308 964184 ? Ssl 09:24 0:46 /docker-java-home/bin/java -XX:MaxDirectMemorySize=128G -Xms2g -Xmx2g -Ddefaut.min.distinct.threshold=1074000000
-Djava.net.preferIPv4Stack=true -XX:+UseParallelGC -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/graphdb/dist/heapdump.hprof -XX:OnOutOfMemoryError=kill -9 %p
-Djava.awt.headless=true -Dfile.encoding=UTF-8 -Dgraphdb.foreground=yes -Dgraphdb.dist=/opt/graphdb/dist -cp /opt/graphdb/dist/lib/* -Dgraphdb.home=/opt/graphdb/home com.ontotext.graphdb.server.GraphDBWorkbench
However, when we look into the logs (main-... .log) we see statements like these (mind the line where I added 2 stars at the beginning):
[INFO ] 2019-08-08 09:33:22,793 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'FSIPC' to default value '2'
[INFO ] 2019-08-08 09:33:23,138 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,244 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,299 [repositories/kg-test-graph-import | c.o.f.s.RepositoryController] GET query -1149340244
[INFO ] 2019-08-08 09:33:23,334 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'order.construct' to default value 'false'
**[INFO ] 2019-08-08 09:33:23,345 [repositories/kg-test-graph-import | c.o.c.AbstractParameter] Configured parameter 'default.min.distinct.threshold' to default value '250.0M'
[INFO ] 2019-08-08 09:33:23,357 [repositories/kg-test-graph-import | c.o.g.s.GraphQueryResultView] Request for query -1149340244 is finished
[INFO ] 2019-08-08 09:33:23,367 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,415 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,487 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
[INFO ] 2019-08-08 09:33:23,704 [repositories/kg-test-graph-import | c.o.f.s.RepositoryController] GET query 1759975767
Moreover the ontotext logs show the exceptions that can also be witnessed in the client running our tests and Sparql request against the DB:
[WARN ] 2019-08-08 09:41:10,442 [repositories/kg-test-graph-import | c.o.t.u.DistinctMemoryMonitor] Insufficient free Heap Memory 238Mb for group by and distinct, threshold 250Mb, reached 0Mb
[ERROR] 2019-08-08 09:41:10,469 [repositories/kg-test-graph-import | c.o.t.SailConnectionImpl] Memory limit for GroupBy/Distinct reached
com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
at com.ontotext.trree.util.DistinctMemoryMonitor.updateGroupByMemory(DistinctMemoryMonitor.java:73)
at com.ontotext.trree.util.LongKeyBase.<init>(LongKeyBase.java:38)
at com.ontotext.trree.util.LongKeySet.<init>(LongKeySet.java:26)
at com.ontotext.trree.query.OwlimDistinctIteration.<init>(OwlimDistinctIteration.java:20)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:1086)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:639)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at org.eclipse.rdf4j.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:42)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:824)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:237)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:796)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:230)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:650)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1786)
at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnection(SailConnectionImpl.java:1679)
at com.ontotext.trree.SailConnectionImpl.evaluateInternal(SailConnectionImpl.java:249)
at com.ontotext.trree.SailConnectionImpl.evaluate(SailConnectionImpl.java:2316)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.evaluateWhereClause(SailUpdateExecutor.java:487)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeModify(SailUpdateExecutor.java:450)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeUpdate(SailUpdateExecutor.java:121)
at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:64)
at com.ontotext.trree.monitorRepository.MonitorRepositoryConnection$1.execute(MonitorRepositoryConnection.java:382)
at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:254)
at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.handleRequestInternal(StatementsController.java:114)
at com.ontotext.graphdb.sesame.StatementsController.handleRequestInternal(StatementsController.java:41)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:52)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ontotext.forest.core.request.RequestFilter.doFilterInternal(RequestFilter.java:45)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:263)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ontotext.forest.security.AdminDelegatingFilterProxy.doFilter(AdminDelegatingFilterProxy.java:38)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
[ERROR] 2019-08-08 09:41:10,549 [repositories/kg-test-graph-import | c.o.f.s.GraphDBProtocolExceptionResolver] Error while handling request (500)
org.eclipse.rdf4j.http.server.ServerHTTPException: org.eclipse.rdf4j.query.UpdateExecutionException: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:266)
at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.handleRequestInternal(StatementsController.java:114)
at com.ontotext.graphdb.sesame.StatementsController.handleRequestInternal(StatementsController.java:41)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:52)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:908)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ontotext.forest.core.request.RequestFilter.doFilterInternal(RequestFilter.java:45)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:263)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ontotext.forest.security.AdminDelegatingFilterProxy.doFilter(AdminDelegatingFilterProxy.java:38)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.rdf4j.query.UpdateExecutionException: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:69)
at com.ontotext.trree.monitorRepository.MonitorRepositoryConnection$1.execute(MonitorRepositoryConnection.java:382)
at org.eclipse.rdf4j.http.server.repository.statements.StatementsController.getSparqlUpdateResult(StatementsController.java:254)
... 45 common frames omitted
Caused by: org.eclipse.rdf4j.sail.SailException: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1815)
at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnection(SailConnectionImpl.java:1679)
at com.ontotext.trree.SailConnectionImpl.evaluateInternal(SailConnectionImpl.java:249)
at com.ontotext.trree.SailConnectionImpl.evaluate(SailConnectionImpl.java:2316)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.evaluateWhereClause(SailUpdateExecutor.java:487)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeModify(SailUpdateExecutor.java:450)
at org.eclipse.rdf4j.repository.sail.helpers.SailUpdateExecutor.executeUpdate(SailUpdateExecutor.java:121)
at org.eclipse.rdf4j.repository.sail.SailUpdate.execute(SailUpdate.java:64)
... 47 common frames omitted
Caused by: com.ontotext.trree.util.NotEnoughMemoryForDistinctGroupBy: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
at com.ontotext.trree.util.DistinctMemoryMonitor.updateGroupByMemory(DistinctMemoryMonitor.java:73)
at com.ontotext.trree.util.LongKeyBase.<init>(LongKeyBase.java:38)
at com.ontotext.trree.util.LongKeySet.<init>(LongKeySet.java:26)
at com.ontotext.trree.query.OwlimDistinctIteration.<init>(OwlimDistinctIteration.java:20)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:1086)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:639)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at org.eclipse.rdf4j.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:42)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:824)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:237)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:796)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:230)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:650)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.StrictEvaluationStrategy.evaluate(StrictEvaluationStrategy.java:228)
at org.eclipse.rdf4j.query.algebra.evaluation.impl.TupleFunctionEvaluationStrategy.evaluate(TupleFunctionEvaluationStrategy.java:65)
at com.ontotext.trree.query.OwlimEvaluationStrategyImpl.evaluate(OwlimEvaluationStrategyImpl.java:135)
at com.ontotext.trree.SailConnectionImpl.evaluateWithRepoConnectionInternal(SailConnectionImpl.java:1786)
... 54 common frames omitted
[INFO ] 2019-08-08 09:41:10,666 [repositories/kg-test-graph-import | c.o.g.s.StatementsController] POST SPARQL update request to repository
What are our options to mitigate the situation?
Currently I am not able to proceed with our software to get it up and running again due to the failing tests.
Backround here is that we are evaluating to switch to Ontotext from another triple store.
Lots of thanks and
best regards
Tom

Answer 1

Got it working for now.
It seems to be a Docker Mac specific problem. By default, only 2GB of memory gets allocated to the docker engine. Raising that and giving the onto process enough memory (Xmx) solved the problem for me now.
See below screenshot for the settings.
Got there by DockerIcon -> Preferences -> Advanced
Docker memory preferences