We are using Spring data gemfire, we are planning to migrate to Apache geode latest version. In the VMWare gemfire version we had to explicitly set the path of the gemfire installable for the pulse to work properly. If we are using Apache geode jar, will we able to get the pulse up and running without specifying the installable location.
We are not using gfsh in our project, we want to ensure that we have minimal dependency on the installable version when we upgrade gemfire.
You don't need to set the GEODE_HOME environment variable when using spring-boot-data-geode, you just need to make sure the correct dependencies are within the classpath of your application (see here for more details).
I've written a very basic example showing how to start a Locator with the Pulse application embedded, you can find it here
As a side note, and regarding the following:
We are using Spring data gemfire, we are planning to migrate to Apache geode latest version.
In order to avoid weird and hard to fix runtime issues, please always make sure to use a combination of versions fully supported in the Spring Boot for Apache Geode and VMware Tanzu GemFire Version Compatibility Matrix
After going through various answers and documentations I was able to start pulse by the help of following article.
Start Gemfire Pulse
Related
The Problem
I have an Amazon Neptune cluster with an instance running in db.t3.medium DB instnance class. I do not see a choice to move this to the Serverless instance.
How can I migrate this instance?
Root Cause
You can only migrate an instance running Neptune Engine version 1.2 or later.
How to Fix
You need to migrate your Neptune Engine version first to 1.2. Once that is done, you will get the migration option to Serverless.
The engine version is controlled not in the cluster instance but at the cluster level and if you are running an older version of the engine, you may need to incrementally upgrade to from the highest version in the major version group, then move up to the next higher version. If you are running 1.0.x, you will first need to go to 1.1.0 R7 then move onto 1.2.
As with any major version upgrade, you could incur some downtime during migration.
To change the engine version, "Modify" the cluster (not instance) settings (the top right button on the console page) and select the latest possible DB engine version. You can keep the rest of the settings, and you can apply the change to take effect immediately if you can afford to initiate a downtime shortly after. Continue to upgrade to the next higher level until you reach 1.2. Each upgrade can take a while.
Our company uses Geode services for some of our applications, we are making use of Geode Member group configurations as well for maintaining different regions.
We have been undergoing an effort of migrating our applications from Geode version 1.6 to the latest version 1.12.
We have seen dramatic performance decrease after the upgrade, if we use the older parameters for the server and locator startup scripts, and things work fine when we remove those parameters.
We are now planning to take the route of understanding the parameters (earlier used) and available, to determine the most optimal configurations for the server and locator to get the best out of the new Geode version.
I was wondering if someone has any best practices or recommendations to follow for this task.
Below are the configurations for the Geode locator and server startup scripts for old and new versions.
Locator startup command
---Old configurations ( works great with Geode 1.6 version but not with any version after Geode 1.8)
gfsh start locator --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --initial-heap=2G --max-heap=2G --dir=/opt/compnayname/geode/locator --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-locator.xml --J=-DCLUSTER=${ECS_CLUSTER} --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-locator.xml' --J=-Dgemfire.distributed-system-id=${DISTRIBUTED_SYSTEM_ID} --J=-Dgemfire.member-timeout=30000 --J=-Dgemfire.max-num-reconnect-tries=0 --J=-Dgemfire.jmx-manager=true --J=-Dgemfire.jmx-manager-start=true --J=-Dgemfire.jmx-manager-port=1099 --J=-Dgemfire.http-service-port=0 --J=-Dgemfire.log-level=info --J=-Dgemfire.log-file-size-limit=10 --J=-Dgemfire.log-disk-space-limit=10 --J=-Dgemfire.disable-auto-reconnect=true
---New configuration (works great with all versions)
gfsh start locator --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --J=-Xmx2048m --dir=/opt/compnayname/geode/locator --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-locator.xml --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-locator.xml'
Server Startup command
---Old configurations ( works great with Geode 1.6 version but not with any version after Geode 1.8)
gfsh start server --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --initial-heap=${GEODE_INIT_HEAP} --max-heap=${GEODE_MAX_HEAP} --group=${SERVER_GROUP} --dir=/opt/compnayname/geode/server --classpath=/opt/compnayname/geode/services-geode.jar --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-server.xml --J=-DCLUSTER=${ECS_CLUSTER} --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-server.xml' --J=-Dgemfire.distributed-system-id=${DISTRIBUTED_SYSTEM_ID} --J=-Dgemfire.member-timeout=30000 --J=-Dgemfire.max-num-reconnect-tries=0 --J=-Dgemfire.socket-buffer-size=16777215 --J=-Dgemfire.off-heap-memory-size=${GEODE_OFF_HEAP} --J=-XX:+UseParNewGC --J=-XX:+UseConcMarkSweepGC --J=-XX:CMSInitiatingOccupancyFraction=60 --eviction-heap-percentage=70 --critical-heap-percentage=90 --J=-Dgemfire.http-service-port=0 --J=-Dgemfire.log-level=info --J=-Dgemfire.log-file-size-limit=10 --J=-Dgemfire.log-disk-space-limit=10 --J=-Dgemfire.disable-auto-reconnect=true ${ADDTL_GEODE_SERVER_OPTS}
---New configuration (works great with all versions)
gfsh start server --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --J=-Xmx${GEODE_MAX_HEAP} --group=${SERVER_GROUP} --dir=/opt/compnayname/geode/server --classpath=/opt/compnayname/geode/services-geode.jar --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-server.xml --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-server.xml'
Test Environment Details
We are using the exact same environment (read AWS) for testing the old and new configurations and performing the same test to measure the response time. We are using 3 Geode locators and 3 Geode servers for the different member groups.
The only difference is the Geode version
We are actually doing a count operation (we have written a count function to execute on Geode regions to count the records existing in the downloaded data which is actually data sketches (https://datasketches.apache.org/)). This count operation on the same data in the same testing environment is giving a drastically slow response with the old configuration using any Geode version beyond 1.8
Another surprising thing is that if I use the old configurations in my local laptop (my laptop serves as locator and server both) with any Geode version greater than 1.8 (including the latest version of Geode), then I am not seeing this issue. Somehow these extra configurations are causing slowness in the AWS environment in the distributed infrastructure.
Please let me know if more information is required and I will be glad to provide more details.
Any information on this will be appreciated.
The main difference I see is the inclusion of --J='-javaagent:/opt/xyz/geode/jmxtrans-agent-1.2.6.jar=/opt/xyz/geode/jmxtrans-agent-server.xml' in the startup parameters. This seems to be a third party Java Agent to expose several JVM metrics through JMX.
Do you know if the agent itself is modifying the byte code?, I've seen negative effects of that approach for Geode applications in the past (not performance related, though). Have you tried upgrading the agent to the latest available version (1.2.10)?. As a side note, Geode already exposes a lot of metrics and information through JMX out of the box, is there any reason why you're relying on yet another external tool for this?.
We have seen dramatic performance decrease after the upgrade
How are you measuring performance?, where do you see the degradation?, are you executing exactly the same workload on exactly the same machines, where the only difference is the Geode version?. There are several actors in play here.
That said, diagnosing and troubleshooting performance degradations can be a long and though process, so my suggestion would be to open a Geode JIRA Ticket with all the relevant information and artefacts.
Cheers.
Can Gemfire be used like Big Memory Go as a L2 cache provider with hibernate? Using hibernate XML files , not annotations. If an applications uses lots of redundant hibernate calls, trying to see if Gemfire could integrate as a L2 cache to use as a off heap solution for caching.
Prior to Pivotal GemFire 9.0.x (e.g. Pivotal GemFire 8.2.x and earlier) GemFire had support for Hibernate L2 Cache; see here.
However, this was pulled in Pivotal GemFire 9 due to a lack of support on maintaining the feature and keeping it up-to-date with the latest versions of Hibernate.
SIDE NOTE:
I am not sure if you are aware of this... but Pivotal GemFire was released to the Apache Software Foundation (ASF) as the Apache Geode open source project (April, 2015) and became a TLP last year (~October 2016). Therefore, Apache Geode is the open source core for Pivotal GemFire, especially as of Pivotal GemFire 9.0.
I mention this because the work/code is not lost, it is mostly a WIP. See...
https://issues.apache.org/jira/browse/GEODE-1972
I see that the feature branch (i.e. feature/GEODE-1972) does NOT exist yet.
There was discussion about this on the Geode dev list...
http://apache.markmail.org/thread/uvuzoohkfplkg46u
So, it probably just needs some "interests", maybe even some help/contributions from the community to move this along. A good opportunity to get involved and have an impact.
Cheers,
John
is it possible to run Solr 6.4.1 on tomcat?
I read that Solr does not support tomcat anymore, is that true, if yes is there any other option without tomcat?
Yes, any version of Solr from 5 and onwards does not support Tomcat as an alternative officially.
The reasoning for this has been documented on the wiki:
Solr is intended to be a server not a Java web application, similar to mysql or the Apache web server. When Solr was first created, designing it as a web application was a convenient choice, to avoid writing a lot of tricky code to build a network layer. These days, this design decision has become a limiting factor.
When you download Solr and install it onto your machine, it should be Solr that gets started. It should not be necessary to install Solr into a third-party application (servlet container) before it will work.
At this time, Solr is still a webapp, but this is an internal implementation detail, not an immutable property. The intention is to make Solr into a completely standalone application. Startup scripts that start the included container are the first step towards that goal. Jetty might still be the technology used once Solr is a standalone application, but if that happens, it will be internally embedded.
At the moment you can still attempt to run Solr in a different container, as the current version bundles jetty and loads Solr inside jetty, but you can run into unexplainable issues where you'll always suspect the container to be the issue .. and if you have a problem, reporting bugs while running under Tomcat won't do any good.
From one of the comments on the old tomcat page on the community wiki:
If you want to go against recommendations and run 5.3 or later in Tomcat, you can likely still do so, but you will need to inform tomcat about an exploded web application (found in server/solr-webapp) instead of the .war file.
The server/solr_webapp/webapp folder is an exploded web application. Tomcat documentation should be able to tell you how to add such an application.
.. but if you're deploying Solr now, you really shouldn't. Use the bundled version of jetty (which might change to a stand alone version later) and the solr command / script.
They have stopped the support for the same.
Other option could be, you can check out the code and see if you can build the solr. I had tried it for earlier version (3.3).
I am not sure of the current version. But that could be the option for you.
I have posted instructions on how to get solr 6.2 running on Tomcat here. However, these instructions no longer work on Solr 6.3 or 6.4.
I couldn't find an answer to this in any other questions and I wanted to see if anyone knew. I'm using JBoss AS 7.1 on Kepler eclipse, and I was wondering if there is a way to change your standalone.xml while the server is running and have Jboss push the change. Would just cleaning the server do this?
You can't edit the raw XML files and see runtime changes. In fact there is a good chance any changes will be overwritten by the server.
The best way to make runtime changes is either via the web console or the CLI environment. I don't know if JBoss Tools has any kind of CLI type of client that can be used.