When we do select count(*) from table , whole ignite server becomes unresponsive for execution of query. Query execution time is also very high and become higher if number of records are higher.
Even if query takes long time, whole server should not become unresponsive (unable to even ssh), all other queries timeout.
Apache ignite version 2.7.5
Apache Ignite version : 2.7.5
Ignite persistence is enabled (true)
2 node cluster in partitioned mode
RAM - 150 GB per node
JVM xms and xmx 20G
Number of records - 160 million
JVM options -
/usr/java/jdk1.8.0_144/bin/java -XX:+AggressiveOpts -server -Xms20g -Xmx20g -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/etappdata/ignite/logs/PROD/etail-prod-ignite76-163/logs -XX:+ExitOnOutOfMemoryError -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Xloggc:/etappdata/ignite/logs/PROD/etail-prod-ignite76-163/gc.log -XX:+PrintAdaptiveSizePolicy -XX:+UseTLAB -verbose:gc -XX:+ParallelRefProcEnabled -XX:+UseLargePages -XX:+AggressiveOpts -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true -Djava.net.preferIPv6Stack=false -Djava.net.preferIPv6Addresses=false -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8996 -Dcom.sun.management.jmxremote.rmi.port=8996 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.local.only=false -Djava.rmi.server.hostname=etail-prod-ignite76-163 -XX:MaxDirectMemorySize=4g -javaagent:/tmp/apminsight-javaagent-prod/apminsight-javaagent.jar -Dfile.encoding=UTF-8 -XX:+UseG1GC -DIGNITE_QUIET=false -DIGNITE_SUCCESS_FILE=/ignite/apache-ignite-2.7.5-bin/work/ignite_success_7d9ec20d-9728-475a-aa80-4355eb8eaf02 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49112 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -DIGNITE_HOME=/ignite/apache-ignite-2.7.5-bin -DIGNITE_PROG_NAME=./bin/ignite.sh -cp /ignite/apache-ignite-2.7.5-bin/libs/:/ignite/apache-ignite-2.7.5-bin/libs/ignite-indexing/:/ignite/apache-ignite-2.7.5-bin/libs/ignite-spring/:/ignite/apache-ignite-2.7.5-bin/libs/licenses/ org.apache.ignite.startup.cmdline.CommandLineStartup config/config-cache.xml
For queries such as SELECT * FROM table (without WHERE) it is recommended to enable lazy mode so that Ignite will not try to store the entire result set on heap at once.
You can enable it via JDBC/ODBC driver properties/connection string or on SqlFieldsQuery.
Related
I m using IntelliJ 2016, by default the vmoptions contains :
-server
-Xms128m
-Xmx512m
-XX:ReservedCodeCacheSize=240m
-XX:+UseConcMarkSweepGC
-XX:SoftRefLRUPolicyMSPerMB=50
-ea
-Dsun.io.useCanonCaches=false
-Djava.net.preferIPv4Stack=true
-XX:+HeapDumpOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
I would like to customize IntelliJ to make it faster.
Here the properties for my system.
I m using : Java as language, Apache Camel as Framework
Is there an optimal configuration ?
Thxs
I use this contains;
I think you like use this.
-ea
-server
-Xms2G
-Xmx4096M
-Xss2m
-XX:MaxMetaspaceSize=2G
-XX:ReservedCodeCacheSize=1G
-XX:MetaspaceSize=512m
-XX:+UseConcMarkSweepGC
-XX:+DoEscapeAnalysis
-XX:SoftRefLRUPolicyMSPerMB=50
-XX:+UnlockExperimentalVMOptions
-Djava.net.preferIPv4Stack=true
-Dsun.io.useCanonCaches=false
-XX:LargePageSizeInBytes=256m
-XX:+UseCodeCacheFlushing
-XX:ParallelGCThreads=8
-XX:+DisableExplicitGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+PrintGCDetails
-XX:+PrintFlagsFinal
-XX:+AggressiveOpts
-XX:+CMSClassUnloadingEnabled
-XX:CMSInitiatingOccupancyFraction=60
-XX:+CMSClassUnloadingEnabled
-XX:+CMSParallelRemarkEnabled
-XX:+UseAdaptiveGCBoundary
-XX:+UseSplitVerifier
-XX:CompileThreshold=10000
-XX:+OptimizeStringConcat
-XX:+UseStringCache
-XX:+UseFastAccessorMethods
-XX:+UnlockDiagnosticVMOptions
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseCompressedOops
-XX:-OmitStackTraceInFastThrow
-Dawt.useSystemAAFontSettings=lcd
-Dfile.encoding=UTF-8
-Xverify:none
-XX:ErrorFile=$USER_HOME/java_error_in_idea_%p.log
-XX:HeapDumpPath=$USER_HOME/java_error_in_idea.hprof
-Xbootclasspath/a:../lib/boot.jar
It depends on the size of your projects and the technologies/frameworks/languages they're using. The best solution to dealing with slowness is to report it: https://intellij-support.jetbrains.com/hc/en-us/articles/207241235-Reporting-performance-problems
That said, if you have a lot of free physical memory (and you likely do), you could increase the Xmx parameter to e.g. 2g, that might speed things up.
First,
If you have a 64 bits as OS, it s better to launch idea64.exe version.
idea64 (Version 2016) must run with a JDK 8 (64 bits)
SET IDEA_JDK_64 = ../path of jdk8
Edit idea64.exe.vmoptions "-Xmx1024m"
I am using jboss 7.1.1 Final with jdk version 1.6.0_45 and while starting jboss i just configured 5gb for heap and 1gb for non-heap. My Linux complete RAM size is around 60GB. After starting jboss some time i can see from linux top command its occupied ram is 50GB. Moreover from jconsole & jvisualvm tool i can see my jboss ram utilization keep up & down reaching max 90% ( approx. using between 4-5 gb)
top - 11:52:35 up 1 day, 17:40, 4 users, load average: 0.89, 1.20, 1.27
Tasks: 174 total, 1 running, 173 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.1%us, 1.8%sy, 0.0%ni, 85.8%id, 0.0%wa, 0.1%hi, 0.2%si, 0.0%st
Mem: 62.948G total, 49.872G used, 13.076G free, 347.309M buffers
Swap: 8197.219M total, 0.000k used, 8197.219M free, 2053.590M cached
And Jboss parameters are like this :
-D[Standalone] -server -Xms5120m -Xmx5120m -XX:MaxPermSize=1024m -XX:PermSize=1024m -Djava.net.preferIPv4Stack=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dorg.jboss.resolver.warning=true -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+UseParNewGC
Please help out why more Linux RAM is consumed ????
Regards
Veera
I'm trying to create a minecraft server and the following error:
E:\Documentos\Minecraft>java -Xms1G -Xmx2G -jar server.jar -o false
Error occurred during initialization of VM Could not reserve enough
space for 2097152KB object heap E:\Documentos\Minecraft>PAUSE
Pressione qualquer tecla para continuar. . .
Can someone help me?
Your error message states
"Error occurred during initialization of VM Could not reserve enough space for 2097152KB object heap"
You don't have enough memory to run your server. Try upgrading your storage capacity.
The computer you are running this on does not have enough memory to allocate 1 gigabyte to a process. You will need to get a computer with more RAM to run a server.
You are using more RAM to the server that you have, in run.bat change:
java -Xms1G -Xmx2G -jar server.jar -o false
to
java -Xms1G -Xmx1G -jar server.jar
In recent versions "-o" don't works more.
If I add google play services library to my project, then gc overhead limit exceeded is showing!
This is eclipse.ini content:
-startup
plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
1024M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
1024m
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.6
-Xms512m
-Xmx1024m
I was also facing the same problem. Then I set the values to 2048 for xms and xmx. However high CPU uses was observed for intial few seconds, but it worked.
-Xms2048m
-Xmx2048m
I have an Apache tomcat 7.0.5 Server on a Windows R2 Server and I've deployed a webApp ( java JSF) on it, everything was working fine, but it suddenly stopped and won't run again. when I try to run it, it tells me "Error 1067: The Process terminated unexpectedly" and In the logs I get these files:
tomcat7-stdout.2012-08-08.log
2012-08-08 18:00:06 Commons Daemon procrun stdout initialized
commons-daemon.2012-08-08.log
[2012-08-08 19:02:30] [info] Commons Daemon procrun finished
[2012-08-08 19:13:02] [info] Commons Daemon procrun (1.0.4.0 64-bit) started
[2012-08-08 19:13:02] [info] Running 'Tomcat7' Service...
[2012-08-08 19:13:02] [info] Starting service...
[2012-08-08 19:13:02] [error] CreateJavaVM Failed
[2012-08-08 19:13:03] [info] Service started in 1000 ms.
[2012-08-08 19:13:03] [info] Run service finished.
[2012-08-08 19:13:03] [info] Commons Daemon procrun finished
tomcat7-stderr.2012-08-08.log
2012-08-08 18:00:06 Commons Daemon procrun stderr initialized
Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future
Unrecognized VM option '+HeapDumpOnOutOfMemoryError '
since I was having some 'perm gen' memory error I added some stuff to my apache Tomcat properties, folowing this link how to handle Perm Gen
so my Java Options look like this:
-Dcatalina.home=C:\Program Files\Apache Software Foundation\Tomcat 7.0
-Dcatalina.base=C:\Program Files\Apache Software Foundation\Tomcat 7.0
-Djava.endorsed.dirs=C:\Program Files\Apache Software Foundation\Tomcat 7.0\endorsed
-Djava.io.tmpdir=C:\Program Files\Apache Software Foundation\Tomcat 7.0\temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=C:\Program Files\Apache Software Foundation\Tomcat 7.0\conf\logging.properties
-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled
-XX:PermSize=256m
-XX:MaxPermSize=256m
-XX:+HeapDumpOnOutOfMemoryError
Any ideas as to why the server won't start anymore? THANKS!
~Myy
After spending hours trying to figure this out, I finally found it. I had an extra space at the end of VM command.
'+HeapDumpOnOutOfMemoryError '
which was giving me the Unrecognized VM option.....
Thanks for reading, hopefully when you get this error, you can take extra precautions to verify the syntax.
to manage the memory you need to set some Xms and Xmx parameter
Xms manages the initial heap size
Xmx manages the maximum heap size
Thus try to add :
-Xms=256m -Xmx=256m
regards
The error is on option "HeapDumpOnOutOfMemoryError" which is not even listed in your post. Can you post a screen print of the "Java" tab from the "tomcat" service screen (similar to the link you have provided) ? This will tell us the actual options you are using and also the memory you have set.
copy the file msvcr71.dll from the bin dir of your java installation, to the bin dir of the tomcat installation.