I have an apache geode setup. I have connected geode with external datasource which is postgres. There is a scenario where i have few data present in apache geode and rest all is in postgres. I am using query service to fetch data and when i fire query then i want that apache geode should provide me data present in its server and from postgres as well(for data not present in apache geode ). How can i proceed with that?
I have my cache loaded which loads data if it is not present in Apache Geode.
'''
#Service
public class QuoteLoader implements CacheLoader<String, Employee> {
private static Logger logger = LoggerFactory.getLogger(QuoteLoader.class);
#Autowired
EmployeeImpl employeeImpl;
#Autowired
EmployeeRepository employeeRepository;
public QuoteLoader() {
}
#Override
public void init(Properties props) {
}
#Override
public void close() {
}
#Override
public Employee load(LoaderHelper<String, Employee> helper) throws CacheLoaderException {
Employee value = null;
try {
value = employeeImpl.getEmployeeById(Long.valueOf(helper.getKey()));
} catch (Exception e) {
e.printStackTrace();
logger.error("Error Occured", e);
}
return value;
}
}
'''
The query engine simply executes the OQL and returns the result, no CacheLoader is invoked whatsoever during the query execution. The CacheLoader is only invoked if there's a cache miss while executing a Region.get(key) operation.
This, if you think about it, makes total sense... a query is an arbitrary string that can return any results (including none), so the query engine can't know whether an entry is expected to be part of the result or not if the actual entry is not already present in the Geode region.
Related
Is there a way to add a specific behavior that should only be executed on (and follow) the Ignite Coordinator ServerNode?
Is there a add-on hook to add some customer specific behavior?
Thanks in advance.
Greg
AFAIK, there is not any built-in hook to achieve this.
However, it can be achieved through Node Singleton Ignite Service and using an api on TcpDiscoverySpi.isLocalNodeCoordinator().
If different discovery mechanism other than TcpDiscovery, is used, above mentioned way by #Alexandr can be used to get a co-ordinator node.
Define an Ignite Service as follows. It will schedule a task that will periodically execute on every node of the cluster and execute certain logic, if it's a coordinator node.
public class IgniteServiceImpl implements Service {
#IgniteInstanceResource
Ignite ignite;
#Override
public void cancel(ServiceContext ctx) {
}
#Override
public void init(ServiceContext ctx) throws Exception {
System.out.println("Starting a service");
}
#Override
public void execute(ServiceContext ctx) throws Exception {
Timer timer = new Timer();
timer.schedule(new TimerTask() {
#Override
public void run() {
System.out.println("Inside a service");
if (ignite != null) {
DiscoverySpi discoverySpi = ignite.configuration().getDiscoverySpi();
if (discoverySpi instanceof TcpDiscoverySpi) {
TcpDiscoverySpi tcpDiscoverySpi = (TcpDiscoverySpi) discoverySpi;
if(tcpDiscoverySpi.isLocalNodeCoordinator())
doSomething();
} else {
ClusterNode coordinatorNode = ((IgniteEx) ignite).context().discovery().discoCache().oldestAliveServerNode();
UUID localNodeId = ((IgniteEx) ignite).context().localNodeId();
if(localNodeId.equals(coordinatorNode.id()))
doSomething();
}
} else {
System.out.println("Ignite is null");
}
}
}, 5, (30 * 1000L));
}
private void doSomething(){
System.out.println("Hi I am Co-ordinator Node");
}
}
Start above service as node singleton using Ignite.services() as follows
IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
igniteConfiguration.setIgniteInstanceName("ignite-node");
Ignite ignite = Ignition.start(igniteConfiguration);
IgniteServices services = ignite.services();
services.deployNodeSingleton("test-service",new IgniteServiceImpl());
In case of a low-level logic, you can extend Ignite with custom plugins.
I'm not sure if there is an easy way to check if a node is indeed the coordinator, but you might check for the oldest one:
private boolean isCoordinator() {
ClusterNode node = ((IgniteEx)ignite()).context().discovery().discoCache().oldestAliveServerNode();
return node != null && node.isLocal();
}
Otherwise, just run a custom initialization logic/compute task, once a node is started.
Can I use Spring Data Redis #Repositories, and a CachingConfigurer to implement error handling? For instance -
#Override
public CacheErrorHandler errorHandler() {
return new RedisCacheError();
}
Using the CacheConfigurer above along with the repository as defined below, note the use of CrudRepository operations.
#Repository
public interface ObjectRepository extends CrudRepository<object, String>
I notice that CrudRepository has operations such as save, wondering if that is doing a redis.add() in the background? And basically whether or not the ErrorHandler see below, will actually catch errors on the save operation.
public class RedisCacheError implements CacheErrorHandler {
#Override
public void handleCacheGetError(RuntimeException exception,
Cache cache, Object key) {
log.info("caught error in here: " + exception );
}
#Override
public void handleCachePutError(RuntimeException exception, Cache cache,
Object key, Object value) {
log.info("caught error in here: " + exception );
}
#Override
public void handleCacheEvictError(RuntimeException exception, Cache cache,
Object key) {
//Do something on error while Evict cache
}
#Override
public void handleCacheClearError(RuntimeException exception,Cache cache){
//Do something on error while clearing cache
}
}
we are currently trying to upgrade from Camel 2.19.0 to 2.20.0.
We have one timed routed which schedule autostart = false
#ConfigurationProperties
#Component
public class AdaptionService extends SpringRouteBuilder {
#Value("${control.timer.cron}")
private String timerCron;
#Override
public void configure() throws Exception {
from("quartz2://adaptionServices/UserUpdateTimer?cron=" + timerCron)
.routeId("scheduler").autoStartup(false)
.to("direct:route-userUpdate");
from("direct:route-userUpdate")
.routeId("adaption_service")
.log("Executing Adaption Service (timed)");
}
No in the CamelContextConfiguration I try to star the timer route (I know I could simply achieve this by autoStartup(true). It is just an example to abstract from more complex use case we have.
#Configuration
public class CamelApplicationContextConfiguration implements CamelContextConfiguration {
#Override
public void beforeApplicationStart(CamelContext camelContext) {
camelContext.setUseMDCLogging(true);
}
#Override
public void afterApplicationStart(CamelContext camelContext) {
try {
camelContext.startRoute("scheduler");
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
That does not work. I checked startRoute(..) and appearently the routeService inside the camelContext is completely empty. With 2.19.0 this works without a flaw.
Any suggestions?
Regards
Holger
The Camel 2.20 release has changed the startup logic when using Spring / Spring-Boot with Camel. You can see that in the release notes.
However we should make CamelContextConfiguration call the afterApplicationStart at a later stage. And also document this API a bit better when its called:
I have logged a ticket about this: https://issues.apache.org/jira/browse/CAMEL-11945
I confess I am a total newbie at the Java way of doing things and I am totally lost trying to get a simple unit test running.
I am building a data access library and want to unit test it. I am using Spring Data Neo4j 4.0.0.BUILD-SNAPSHOT because I need to connect to a remote Neo4j server in the real world.
After battling errors all day I am at the point where I have a test class:
#RunWith(SpringJUnit4ClassRunner.class)
#ComponentScan(basePackages = {"org.mystuff.data"})
#ContextConfiguration(classes={Neo4jTestConfiguration.class})
public class PersonRepositoryTest {
#Autowired
PersonRepository personRepository;
protected GraphDatabaseService graphDb;
#Before
public void setUp() throws Exception {
graphDb = new TestGraphDatabaseFactory().newImpermanentDatabase();
}
#After
public void tearDown() {
graphDb.shutdown();
}
#Test
public void testCreatePerson() throws Exception {
assertNotNull(personRepository);
Person p = new Person("Test", "User");
personRepository.save(p);
}
}
Neo4jTestConfiguration.java
#Configuration
#EnableNeo4jRepositories(basePackages = "org.mystuff.data")
#EnableTransactionManagement
public class Neo4jTestConfiguration extends Neo4jConfiguration {
#Bean
public SessionFactory getSessionFactory() {
return new SessionFactory("org.mystuff.data");
}
#Bean
public Neo4jServer neo4jServer() {
// What to return here? I want in-memory database
return null;
}
#Bean
#Scope(value = "session", proxyMode = ScopedProxyMode.TARGET_CLASS)
public Session getSession() throws Exception {
return super.getSession();
}
}
When running tests the personRepository.save() throws and exception 'No Scope registered for scope "session"'
I don't know if I need the configuration class but my test class won't work without it because Spring needs #ContextConfiguration and I want all the DI niceness Spring provides (amongst other things).
How can I get my tests to work with Spring?
You can use an InProcessServer which is an in-memory database:
#Bean
public Neo4jServer neo4jServer() {
return new InProcessServer();
}
Omit the session scope as your test isn't running in a web container. An example: https://github.com/neo4j-examples/sdn4-cineasts/blob/4.0-RC1/src/test/java/org/neo4j/cineasts/PersistenceContext.java
This will require dependencies as described in this question: Spring Data Neo4j 4.0.0.M1 Test Configuration
In your test class PersonRepositoryTest, you don't need to construct an instance of the database, your tests will run against the same InProcessServer.
Here's an example: https://github.com/neo4j-examples/sdn4-cineasts/blob/4.0-RC1/src/test/java/org/neo4j/cineasts/domain/DomainTest.java
I'm using Castle ActiveRecord for persistence, and I'm trying to write a base class for my persistence tests which will do the following:
Open a transaction for each test case and roll it back at the end of the test case, so that I get a clean DB for each test case without me having to rebuild the schema for each test case.
Provide the ability to flush my NHibernate session and get a new one in the middle of a test, so that I know that my persistence operations have really hit the DB rather than just the NHibernate session.
In order to prove that my base class (ARTestBase) is working, I've come up with the following sample tests.
[TestFixture]
public class ARTestBaseTest : ARTestBase
{
[Test]
public void object_created_in_this_test_should_not_get_committed_to_db()
{
ActiveRecordMediator<Entity>.Save(new Entity {Name = "test"});
Assert.That(ActiveRecordMediator<Entity>.Count(), Is.EqualTo(1));
}
[Test]
public void object_created_in_previous_test_should_not_have_been_committed_to_db()
{
ActiveRecordMediator<Entity>.Save(new Entity {Name = "test"});
Assert.That(ActiveRecordMediator<Entity>.Count(), Is.EqualTo(1));
}
[Test]
public void calling_flush_should_make_nhibernate_retrieve_fresh_objects()
{
var savedEntity = new Entity {Name = "test"};
ActiveRecordMediator<Entity>.Save(savedEntity);
Flush();
// Could use FindOne, but then this test would fail if the transactions aren't being rolled back
foreach (var entity in ActiveRecordMediator<Entity>.FindAll())
{
Assert.That(entity, Is.Not.SameAs(savedEntity));
}
}
}
Here is my best effort at the base class. It correctly implements Flush(), so the third test case passes. However it does not rollback the transactions, so the second test fails.
public class ARTestBase
{
private SessionScope sessionScope;
private TransactionScope transactionScope;
[TestFixtureSetUp]
public void InitialiseAR()
{
ActiveRecordStarter.ResetInitializationFlag();
ActiveRecordStarter.Initialize(typeof (Entity).Assembly, ActiveRecordSectionHandler.Instance);
ActiveRecordStarter.CreateSchema();
}
[SetUp]
public virtual void SetUp()
{
transactionScope = new TransactionScope(OnDispose.Rollback);
sessionScope = new SessionScope();
}
[TearDown]
public virtual void TearDown()
{
sessionScope.Dispose();
transactionScope.Dispose();
}
protected void Flush()
{
sessionScope.Dispose();
sessionScope = new SessionScope();
}
[TestFixtureTearDown]
public virtual void TestFixtureTearDown()
{
SQLiteProvider.ExplicitlyDestroyConnection();
}
}
Note that I'm using a custom SQLite provider with an in-memory database. My custom provider, taken from this blog post, keeps the connection open at all times to maintain the schema. Removing this and using a regular SQL Server database doesn't change the behaviour.
Is there a way to acheive the required behaviour?
Not too sure about ActiveRecord, but in NHibernate a transaction belongs to a session, not the otherway round.
If you've used ADO.Net a lot, this will make more sense, as to create an IDbTransaction you need to use the connection. ActiveRecord's TransactionScope (and NHibnerate's ITransaction) essentially wrap an IDbTransaction, so you need to create the SessionScope before the TransactionScope.
What you might also find (depending on if you're using NHibernate 1.2 GA or NHibernate 2.*, and what FlushMode your SessionScope has) is that your call to FindAll() may cause the session to flush anyway, as NHibernate will realise that it can't retrieve the correct data without actioning the last call to Save.
This all said and done, have you tried using SessionScope.Flush() instead of creating a new SessionScope?
Using SessionScope.Flush() makes my third test fail. As I understand it, Flush() executes the SQL to push my records into the DB, but does not evict objects from the session. That fits with what you say about FindAll() causing a flush.
What I really want is SessionScope.Flush() (to synchronise state of DB with session) plus SessionScope.EvictAll() (to ensure I get fresh objects in subsequent queries). My new SessionScope() was an attempt to simulate EvictAll().
Your comments about the session enclosing the transaction rather than the other way round did give me an idea. I'm not sure how kosher it is to create a new SessionScope inside a TransactionScope inside a flushed SessionScope, and expect it to participate in the transaction, but it seems to work:
public abstract class ARTestBase
{
private SessionScope sessionScope;
private TransactionScope transactionScope;
private bool reverse;
private IList<SessionScope> undisposedScopes;
[TestFixtureSetUp]
public void InitialiseAR()
{
ActiveRecordStarter.ResetInitializationFlag();
ActiveRecordStarter.Initialize(typeof (Entity).Assembly, ActiveRecordSectionHandler.Instance);
ActiveRecordStarter.CreateSchema();
InitialiseIoC();
undisposedScopes = new List<SessionScope>();
}
[SetUp]
public virtual void SetUp()
{
sessionScope = new SessionScope();
transactionScope = new TransactionScope(OnDispose.Rollback);
transactionScope.VoteRollBack();
base.CreateInstanceUnderTest();
reverse = false;
}
[TearDown]
public virtual void TearDown()
{
if (reverse)
{
sessionScope.Dispose();
transactionScope.Dispose();
}
else
{
transactionScope.Dispose();
sessionScope.Dispose();
}
}
[TestFixtureTearDown]
public virtual void TestFixtureTearDown()
{
foreach (var scope in undisposedScopes)
{
scope.Dispose();
}
SQLiteProvider.ExplicitlyDestroyConnection();
}
protected void Flush()
{
reverse = true;
sessionScope.Flush();
undisposedScopes.Add(sessionScope);
sessionScope = new SessionScope();
}
}
On further thought, this won't allow you to flush more than once in each test case. I think I can handle that by tracking the scopes more carefully. I might look into it later.