I have apache geode as inline cache where it is connected to postgres as datasource. Since for getting all keys at once when getall method id invoked from region It calls CacheLoader Sequentially. Is there a way so that i can get all keys and call my datasource at once such as calling in query from Cacheloader.
I don't there's a way of accomplishing this out of the box using a CacheLoader since, as you already verified, the callback is invoked sequentially on every key not found within the Region. You might be able to pre-populate the Region with all those keys you know must be there, though, but keys not found while executing Region.getAll() will still be retrieved sequentially by invoking the configured CacheLoader.
Related
I would like to maintain comma separated lists of entries of the following form <ip>:<app> indexed by a an account ID. There would be one such list for each user indexed by their account ID with the number of users in the millions. This is mainly to track which server in a cluster a user using a certain application is connected to.
Since all servers are written in Java, with Redisson I'm currently doing:
RSet<String> set = client.getSet(accountKey);
and then I can modify the set using some typical Java container APIs supported by Redisson. I basically need three types of updates to these comma separated lists:
Client connects to a new application = append
Client reconnects with existing application to new endpoint = modify
Client disconnects = remove
A new connection would require a change to a field like:
1.1.1.1:foo,2.2.2.2:bar -> 1.1.1.1:foo,2.2.2.2:bar,3.3.3.3:baz
A reconnect would require an update like:
1.1.1.1:foo,2.2.2.2:bar -> 3.3.3.3:foo,2.2.2.2:bar
A disconnect would require an update like:
1.1.1.1:foo,2.2.2.2:bar -> 2.2.2.2:bar
As mentioned the fields would be keyed by the account ID of the user.
My question is the following: Without using Redisson how can I implement this "directly" on top of Redis commands? The goal is to allow rewriting certain components in a language different than Java. The cluster handles close to a million requests per second.
I'm actually quite curious how Redisson implements an RSet under the hood and I haven't had time to dig into it. I guess one option would be to use Lua, but I've never used it with Redis. Any ideas how to efficiently implement these operations on top of Redis on a manner that is easily supported by multiple languages, i.e. not relying on a specific library?
Having actually thought about the problem properly, it can be solved directly with a HSET. Where <app> is the field name and the value are the IPs. Keys being user accounts.
I have an application using Springboot, Gemfire and MySQL. The Springboot application serves as a rest api. I want to "lock" the cache entry so that only one request sent to rest api can access certain entry in GemFire at a time. Others cannot do CRUD on that entry until the entry owner release the possession. I have two approaches as of now.
Approach 1 - Create a GemFire function, which performs a lock/unlock on the entry when invoked by rest api(at different time) using org.apache.geode.cache.Region.getDistributedLock.
Approach 2 - Create a region(eg. Lock) where an entry is created when an entry of target region(eg. Customer) is accessed for the fist time. When the 2nd request wants to access the same entry, the rest api checks the region Lock first. Rest api retrieves and returns the entry from region Customer if the key does not exist in region Lock. Otherwise, no entry will be returned. Once the first requester finishes, rest api removes the entry in region Lock.
I am wondering if there are any alternatives besides these two options.
If you want a more space efficient solution, you could add a boolean field to the value to indicate if it was locked. You can then use region.replace(K,V,V) to efficiently set the "lock" on the entry as well. Although, this will leak your locking concerns into your business objects.
For the function:
invokeAll()
It use Map/Set which contains the entry will be processed, I want process the each entry in a custom order, i.e as the same of the key order
in document:
The order that the entries for the keys are processed is undefined. Implementations may choose to process the entries in any order, including concurrently. Furthermore there is no guarantee implementations will use the same EntryProcessor instance to process each entry, as the case may be in a non-local cache topology.
For this line:
Implementations may choose to process the entries in any order, including concurrently
I don't know how to do this, is there any example?
If I use a TreeMap/TreeSet to save the key with order, does the entry will be handled same as its key order in the TreeMap/TreeSet?
By the way, as invoke has a internal lock, does invokeAll will also hold the lock for all the keys in map / set, until the entryprocessor finished?
The documentation you're referring to is, in fact, inherited from javax.cache.Cache::invokeAll. "Implementation" here means not an EntryProcessor but an implementation of the JSR 166 (AKA JCache, AKA javax.cache package) - and Ignite implements it in IgniteCache.
What this documentation means is that specification of the javax.cache.Cache interface allows its implementations to invoke EntryProcessors in any order. Ignite chooses not give any additional details to it, and there is not way to influence the order here.
Also, remember that Ignite is distributed, so the processing of entries in invokeAll is inherently concurrent. If you need strict order, it's probably better to iterate over the keys and use invoke on each key.
Using Geode 1.2 and 9.1 Pivotal native client the following code:
IRegion<string, IPdxInstance> r = cache.GetRegion<string, IPdxInstance>("myRegion");
return r[key];
then triggers an AfterCreate event for myRegion. Why does that happen when no data is created, only read?
Same here, never used Native Client. I agreed with what #Urizen suspected - you are calling r[key] from an instance of Geode that doesn't have the entry, so it pulls the data from other instance, which "create" the entry locally.
You have a few options here:
Performing an interest registration for the instance you are initiating the call using registerAllKeys() (doc here). There is a catch here: (might not be applicable for native client), in Java API, you have an option to register interest with an InterestResultPolicy. If you use KEYS_VALUES, you will load all data to local from remote on startup WITHOUT triggering afterCreate callback. If you choose KEYS only or NONE, you will likely have similar problem.
You can check for boolean flag remoteOrigin in EntryEvent. If it is false, it is purely local. In a non-WAN setup, this should be enough to distinguish your local operation from remotely initiated operation (be it a cache syncing or a genuine creation initiated by other cache). Vaguely remembering WAN works a bit different here.
I've never used the Native Client but, at a first glance, it should be expected for the afterCreate event to be invoked on the client side as the entry is actually being created on the local cache. What I mean is that the entry might exists on the server but, internally, the client needs to retrieve it from the server, and then create it locally (thus invoking the afterCreate for the locally installed CacheListener). Makes sense?.
I was wondering if there is a way to share an NSManagedObject between two or more NSManagedObjectContext objects running in the same thread.
I have the following problem: I have one main context shared through all my code in the application and several different contexts that are created for each remote fetch request that I issue. (I created a custom class that fetches remotely and inserts all the objects found in the server in his own NSManagedObjectContext). Those fetch requests may run simultaneously since they use NSURLConnection objects that may end at different times. If the same remote object gets fetched by different connections, I will end up with duplicates at the moment of saving and merging the context with the main one. (That is, objects that have the same remote ID but a different objectID).
One possible solution would be to save (and so persist) every object as soon as it is created but I can't do that because it may have some relationships that may still have not been filled and won't validate during the save operation.
I'm really looking forward to a method that allows you to share the same non-persistent instance of an object between context. If anybody has encountered this issue and came up with a solution, I would be pleased to know!
Context cannot communicate between each other save through their stores. However, you can insert a managed object with a nil managed object context and it will be independent (albeit without relationships) of any context. You could pass that independent managed object around however you wished and insert it into a context when you needed to persist it. This is dangerous but possible.
However, if you're not running each connection on a separate thread then you don't gain anything by having multiple context. Each connection object will activate its delegate in sequence on the main thread. In this case, your easiest solution would be to use the same delegate for all the connections and let the delegate handle the insertions into a single context. To prevent duplication, just do a fetch on the remoteID and see if you get back an extant object before inserting a new object for that remoteID.
I don't think what you want to do is possible. I mean if you want to share changes between different contexts, you got to use notifications and merge it whenever did save or did change occur. But in your case, I'd say just use 1 context and save in the end. Or a less elegant way: save all the remote ids temporary in your app and check before inserting new ones. In this case, you can continue use multiple contexts and save after each didfinishloading.