Can Gemfire be used like Big Memory Go as a L2 cache provider with hibernate? Using hibernate XML files , not annotations. If an applications uses lots of redundant hibernate calls, trying to see if Gemfire could integrate as a L2 cache to use as a off heap solution for caching.
Prior to Pivotal GemFire 9.0.x (e.g. Pivotal GemFire 8.2.x and earlier) GemFire had support for Hibernate L2 Cache; see here.
However, this was pulled in Pivotal GemFire 9 due to a lack of support on maintaining the feature and keeping it up-to-date with the latest versions of Hibernate.
SIDE NOTE:
I am not sure if you are aware of this... but Pivotal GemFire was released to the Apache Software Foundation (ASF) as the Apache Geode open source project (April, 2015) and became a TLP last year (~October 2016). Therefore, Apache Geode is the open source core for Pivotal GemFire, especially as of Pivotal GemFire 9.0.
I mention this because the work/code is not lost, it is mostly a WIP. See...
https://issues.apache.org/jira/browse/GEODE-1972
I see that the feature branch (i.e. feature/GEODE-1972) does NOT exist yet.
There was discussion about this on the Geode dev list...
http://apache.markmail.org/thread/uvuzoohkfplkg46u
So, it probably just needs some "interests", maybe even some help/contributions from the community to move this along. A good opportunity to get involved and have an impact.
Cheers,
John
Related
We are using Spring data gemfire, we are planning to migrate to Apache geode latest version. In the VMWare gemfire version we had to explicitly set the path of the gemfire installable for the pulse to work properly. If we are using Apache geode jar, will we able to get the pulse up and running without specifying the installable location.
We are not using gfsh in our project, we want to ensure that we have minimal dependency on the installable version when we upgrade gemfire.
You don't need to set the GEODE_HOME environment variable when using spring-boot-data-geode, you just need to make sure the correct dependencies are within the classpath of your application (see here for more details).
I've written a very basic example showing how to start a Locator with the Pulse application embedded, you can find it here
As a side note, and regarding the following:
We are using Spring data gemfire, we are planning to migrate to Apache geode latest version.
In order to avoid weird and hard to fix runtime issues, please always make sure to use a combination of versions fully supported in the Spring Boot for Apache Geode and VMware Tanzu GemFire Version Compatibility Matrix
After going through various answers and documentations I was able to start pulse by the help of following article.
Start Gemfire Pulse
Our company uses Geode services for some of our applications, we are making use of Geode Member group configurations as well for maintaining different regions.
We have been undergoing an effort of migrating our applications from Geode version 1.6 to the latest version 1.12.
We have seen dramatic performance decrease after the upgrade, if we use the older parameters for the server and locator startup scripts, and things work fine when we remove those parameters.
We are now planning to take the route of understanding the parameters (earlier used) and available, to determine the most optimal configurations for the server and locator to get the best out of the new Geode version.
I was wondering if someone has any best practices or recommendations to follow for this task.
Below are the configurations for the Geode locator and server startup scripts for old and new versions.
Locator startup command
---Old configurations ( works great with Geode 1.6 version but not with any version after Geode 1.8)
gfsh start locator --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --initial-heap=2G --max-heap=2G --dir=/opt/compnayname/geode/locator --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-locator.xml --J=-DCLUSTER=${ECS_CLUSTER} --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-locator.xml' --J=-Dgemfire.distributed-system-id=${DISTRIBUTED_SYSTEM_ID} --J=-Dgemfire.member-timeout=30000 --J=-Dgemfire.max-num-reconnect-tries=0 --J=-Dgemfire.jmx-manager=true --J=-Dgemfire.jmx-manager-start=true --J=-Dgemfire.jmx-manager-port=1099 --J=-Dgemfire.http-service-port=0 --J=-Dgemfire.log-level=info --J=-Dgemfire.log-file-size-limit=10 --J=-Dgemfire.log-disk-space-limit=10 --J=-Dgemfire.disable-auto-reconnect=true
---New configuration (works great with all versions)
gfsh start locator --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --J=-Xmx2048m --dir=/opt/compnayname/geode/locator --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-locator.xml --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-locator.xml'
Server Startup command
---Old configurations ( works great with Geode 1.6 version but not with any version after Geode 1.8)
gfsh start server --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --initial-heap=${GEODE_INIT_HEAP} --max-heap=${GEODE_MAX_HEAP} --group=${SERVER_GROUP} --dir=/opt/compnayname/geode/server --classpath=/opt/compnayname/geode/services-geode.jar --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-server.xml --J=-DCLUSTER=${ECS_CLUSTER} --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-server.xml' --J=-Dgemfire.distributed-system-id=${DISTRIBUTED_SYSTEM_ID} --J=-Dgemfire.member-timeout=30000 --J=-Dgemfire.max-num-reconnect-tries=0 --J=-Dgemfire.socket-buffer-size=16777215 --J=-Dgemfire.off-heap-memory-size=${GEODE_OFF_HEAP} --J=-XX:+UseParNewGC --J=-XX:+UseConcMarkSweepGC --J=-XX:CMSInitiatingOccupancyFraction=60 --eviction-heap-percentage=70 --critical-heap-percentage=90 --J=-Dgemfire.http-service-port=0 --J=-Dgemfire.log-level=info --J=-Dgemfire.log-file-size-limit=10 --J=-Dgemfire.log-disk-space-limit=10 --J=-Dgemfire.disable-auto-reconnect=true ${ADDTL_GEODE_SERVER_OPTS}
---New configuration (works great with all versions)
gfsh start server --locators=$locators_str --name=${EC2_HOSTNAME}.aws.compnaynamedigital.net --J=-Xmx${GEODE_MAX_HEAP} --group=${SERVER_GROUP} --dir=/opt/compnayname/geode/server --classpath=/opt/compnayname/geode/services-geode.jar --J=-Dlog4j.configurationFile=/opt/compnayname/geode/log4j2-server.xml --J='-javaagent:/opt/compnayname/geode/jmxtrans-agent-1.2.6.jar=/opt/compnayname/geode/jmxtrans-agent-server.xml'
Test Environment Details
We are using the exact same environment (read AWS) for testing the old and new configurations and performing the same test to measure the response time. We are using 3 Geode locators and 3 Geode servers for the different member groups.
The only difference is the Geode version
We are actually doing a count operation (we have written a count function to execute on Geode regions to count the records existing in the downloaded data which is actually data sketches (https://datasketches.apache.org/)). This count operation on the same data in the same testing environment is giving a drastically slow response with the old configuration using any Geode version beyond 1.8
Another surprising thing is that if I use the old configurations in my local laptop (my laptop serves as locator and server both) with any Geode version greater than 1.8 (including the latest version of Geode), then I am not seeing this issue. Somehow these extra configurations are causing slowness in the AWS environment in the distributed infrastructure.
Please let me know if more information is required and I will be glad to provide more details.
Any information on this will be appreciated.
The main difference I see is the inclusion of --J='-javaagent:/opt/xyz/geode/jmxtrans-agent-1.2.6.jar=/opt/xyz/geode/jmxtrans-agent-server.xml' in the startup parameters. This seems to be a third party Java Agent to expose several JVM metrics through JMX.
Do you know if the agent itself is modifying the byte code?, I've seen negative effects of that approach for Geode applications in the past (not performance related, though). Have you tried upgrading the agent to the latest available version (1.2.10)?. As a side note, Geode already exposes a lot of metrics and information through JMX out of the box, is there any reason why you're relying on yet another external tool for this?.
We have seen dramatic performance decrease after the upgrade
How are you measuring performance?, where do you see the degradation?, are you executing exactly the same workload on exactly the same machines, where the only difference is the Geode version?. There are several actors in play here.
That said, diagnosing and troubleshooting performance degradations can be a long and though process, so my suggestion would be to open a Geode JIRA Ticket with all the relevant information and artefacts.
Cheers.
Planning to Migrate the Websphere from 7.0 to 9 and 8.5 to 9.
Can anyone help me getting the detailed Process
Migration here is "In place". (Migration will be done on the same servers, where the old Installation are in)
if at all any migration tools need to be used, please provide the clear info on them.
any documental references, or any video references for the questioner is appreciated.
OS used : RHEL
CUrrent version: WAS 7x and 8.5
Migrating to : WAS 9.0
It sounds like you're in the very beginning stages of doing this migration. Therefore, I highly recommend you take some time to plan this out, especially to figure out the exact steps you'll be taking and how you'll handle something going wrong. For WebSphere, there is a collection of documents from IBM that discuss planning and executing the upgrade. Look around there for documentation on the tools and step by step guides for different kinds of topologies. The step by step guide for an in place migration of a cell is here.
You should make sure to take good backups before you start the process so you can restore back to before the migration if you need to.
In addition to doing the upgrade, an important part is to also make sure your applications are going to work on the new version if you haven't already. IBM provides this tool to scan applications and identify potential issues that developers will have to fix. There is documentation for the tool at that link as well.
If you are in the planning phase, I'd strongly suggest you to consider migrating to WebSphere Liberty instead of traditional WAS v9. All these migration tools (toolkit for binaries, Eclipse migration toolkit) support both migration scenarios.
Choosing Liberty might be a bit more work at the beginning, but you will gain more deployment flexibility and speed up future development. Liberty is also much better fitted for any cloud/containers environments, as it is much more lightweight, so in the future, if you would like to move to containers, it would be much easier.
Check this tutorial Migrate traditional WebSphere apps to WebSphere Liberty on IBM Cloud Private by using Kubernetes, although it shows the steps to migrate to Liberty on ICP, beginning is the same - analyzing of the application whether they are fit for Liberty and migrating. If you don't have access to IBM Cloud or ICP, you can use stand alone version of the Transformation Advisor that was released recently - IBM Cloud Transformation Advisor.
Having said all that, some apps include old or proprietary traditional WebSphere APIs and in that case it may be easier and cheaper to temporary migrate them to WAS v9, and modernize in the future.
Our application is using Hibernate Search for indexing some of its data. The application is running on two JBoss EAP 6.2 application servers for load distribution and failover. We need changes made on one machine to be immediately visible on the other. The index is a central part of the application and needs to be consistent with the database data. Completely rebuilding it takes a long time so it is important that it remains intact even in the case of a server crash. Also, the index is expected to grow too large to keep all of it in memory.
Our current solution is to use the standard filesystem directory with a shared filesystem (NFS) and the JGroups backend to ensure that only one server writes to a given index at any time. This works more or less, but sometimes we have problems with index updates taking very long (up to 20 seconds) or failing completely. Due to some other reasons we need to migrate away from the currently used file system, so we are evaluating alternatives for the current setup.
One thing we tried is the Infinispan directory with a file cache store for persistence, but we had some problems there regarding OutOfMemoryErrors (see also my post in the Infinispan forums https://developer.jboss.org/thread/253732). Also, performance was still not acceptable in our first tests (about 3 seconds for an index update with two clustered servers set up on my developer machine), though that may be due to configuration issues.
I think this is not such an uncommon requirement, but I couldn't find much information on best practices to implement it.
Who has experiences with similar setups? Does the Infinispan directory work for you? Can anybody suggest a working configuration or how to proceed to arrive at one? What alternatives have you tried and which work?
You need to be careful about which versions are being used. The Infinispan version which is bundled within JBoss EAP is not intended (i.e. tested as extensively as for other purposes) for storing the Lucene index.
When JBoss EAP 6.2 was released, the bundled Infinispan was considered good to go for the internal needs of the application server, but as you might have discovered, the feature of index storage was having at least some performance issues.
In recent developments of Infinispan we applied many improvements to the index storage feature, fixing some bugs and getting very significant performance improvements out of it. I would hope you could be willing to try Infinispan 7.2.0.Beta1 ?
All of these improvements are also being backported to JBoss Data Grid, version 6.5 will make them available as a supported product. Note this feature of storing an Hibernate Search index wasn't supported before - it is going to be a new feature of JDG 6.5.
Modules from JDG 6.5 will be compatible with JBoss EAP, you'll just have to make sure you'll use the Infinispan build provided by JDG and not the one meant for internal usage of EAP.
Performance improvements are still being worked on. It's much better already - especially compared to that older version - but we won't stop working on that yet so if you could try latest bleeding edge versions of Infinispan 7.2.x (another release is scheduled for tomorrow), I'd highly appreciate your feedback to keep pushing it.
The Coherence functionality seems to based on clustering concept. Does this mean that, if I want to install WLS on my system for development use, I do not need the Coherence component that comes with WLS ?
Correct, it is not required to use WLS.
This may help to answer the question:
Coherence is now on the same "release train" as WebLogic, starting with version WebLogic version 12.1.2 (which is the second point release of the WebLogic "12c" major release version). So from now on, Coherence and WebLogic versions are in sync, and the two products are tested and certified together.
Coherence is "physically" included as part of some WebLogic installers. Coherence can be purchased as a separate product (three editions: Standard, Enterprise, Grid), or it can be purchased as part of the WebLogic Suite product (which comes with Enterprise Edition, but can be upgraded to Grid Edition).
WebLogic now has a free developer license available, and that free developer license covers Coherence development as well. (Previously, customers had to pay for a developer license, mainly AFAICT because that is how the database was licensed. We are glad to have fixed this!)
WebLogic can be used without Coherence. Coherence can be used without WebLogic. And both products can also be used together, using Coherence*Web for session management, WebLogic Active Cache for simple caching use cases, or Coherence Containers for full-blown, deployable, life-cycle managed, multi-tier data grid applications that are integrated with Java EE applications on WebLogic.
I hope this helps to clarify the possible options that you have.
For the sake of full disclosure, I work at Oracle. The opinions and views expressed in this post are my own, and do not necessarily reflect the opinions or views of my employer.