I am using Jedis for Accessing Redis via Spring Data.
<bean
id="jedisPoolConfig"
class="redis.clients.jedis.JedisPoolConfig"
p:maxTotal="100"
p:maxIdle="10"
p:maxWaitMillis="5000"
/>
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
p:host-name="${redis.server.host}" p:port="${redis.server.port}"
p:password="${redis.server.password}"
p:use-pool="${redis.server.usepool}"
p:pool-config-ref="jedisPoolConfig"/>
<bean id="genericJackson2JsonRedisSerializer" class="org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer"/>
<bean id="stringSerializer" class="org.springframework.data.redis.serializer.StringRedisSerializer"/>
<!-- redis template definition -->
<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate" p:connection-factory-ref="jedisConnectionFactory"> <property name="keySerializer" ref="stringSerializer"/>
<property name="hashKeySerializer" ref="stringSerializer"/>
<property name="hashValueSerializer" ref="genericJackson2JsonRedisSerializer"/>
</bean>
<bean id="stringRedisTemplate" class="org.springframework.data.redis.core.StringRedisTemplate" p:connection-factory-ref="jedisConnectionFactory" />
Would like to perform BRPOPLPUSH, I dont see this option under
OPTION_1 -> getRedisTemplate().boundListOps("Key").*
OPTION_2 -> getRedisTemplate().opsForList().rightPopAndLeftPush("sourceKey", "destinationKey") ---> No blocking ?
Instead it is available at,
OPTION_3 -> getRedisTemplate().getConnectionFactory().getConnection().bRPopLPush(timeout, srcKey, dstKey)
Question_0: Is this the only option? Why it is not available in above two api's?
Answer_0: No, whenever timeunit is mentioned with left/rightPop(timeout, unit) in boundListOps, opsForList it is blocking call. timeunit with 0 seconds for blocking for ever.
Question_1: Why there are so many variants ..? or May be when to use boundListOps, opsForList and getConnection?
Partial_Answer_1: Looks like if multiple operation to be acted upon a single key then boundListOps, because it is bound to that single key, and no need to mention that in the repeated actions, where as in opsForList, every action key has to be mentioned.
From doc of boundListOps:
Returns the operations performed on list values bound to the given
key.
Just note that, boundListOps still uses opsForList inherently by passing the registered key to it.
Ok, still when should i use directly getConnectionFactory().getConnection() ?
Question_2: If i use getConnectionFactory().getConnection(), Do i need to close() this connection under finally(){}? (Since this is the one currently having blocking RPopLPush support).
From the code boundListOps is inherently using opsForList, and its all action are executed with finally like below with release connection, hence asked.
RedisTemplate.java
public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
RedisConnectionFactory factory = getConnectionFactory();
RedisConnection conn = null;
try {
conn = RedisConnectionUtils.getConnection(factory);
...
...
// TODO: any other connection processing?
return postProcessResult(result, connToUse, existingConnection);
} finally {
RedisConnectionUtils.releaseConnection(conn, factory);
}
Question_3: Why getRedisTemplate().boundListOps("key").rightPush(new HashMap()); is accepting Map instead of value like String/Object though "key" is already mentioned...?
Answer_3: It is my problem, I have declared that way.
public RedisTemplate<String, Map<String, Object>> getRedisTemplate() {
return redisTemplate;
}
And the same RedisTemplate object is passed as this while fetching DefaultBoundListOperations where V is referred from RedisTemplate.
public BoundListOperations<K, V> boundListOps(K key) {
return new DefaultBoundListOperations<K, V>(key, this);
}
class DefaultBoundListOperations<K, V> extends DefaultBoundKeyOperations<K> implements BoundListOperations<K, V> {
public DefaultBoundListOperations(K key, RedisOperations<K, V> operations) {
super(key, operations); //RedisOperations<K, V> is converted to BoundListOperations<K, V> here
this.ops = operations.opsForList();
}
}
Question_4: Why getConnection().bRPopLPush is accepting srcKey, dstKey as byte[] instead of String ..?
Sorry for many questions, it is all raised because i don't find proper java doc in spring data OR tutorials to explain the usage.
Regarding question 4: First of all, it's important to note that Redis keys and values can be any binary. Spring-data templates abstract this away and let developers deal with Java objects rather than byte arrays. See the following quote from the spring-data reference:
...The template offers a high-level abstraction for Redis interactions. While RedisConnection offers low level methods that accept and return binary values (byte arrays), the template takes care of serialization and connection management, freeing the user from dealing with such details.
It seems that when working with RedisConnection, Spring doesn't know what type your key or value is, and it makes no effort to convert an object (e.g., String) to the binary Redis understands, hence, you are required to pass the raw binary to Redis.
First, call RedisTemplate.opsForList() to get ListOperations.
Then, use ListOperations.rightPopAndLeftPush(K sourceKey, K destinationKey, long timeout, TimeUnit unit) to do BRPOPLPUSH.
Related
I defined a data provider help class to populate my local DB during testing and I'm using a ReactorCrudRepository. Te defined saveAll method is:
<S extends T> Flux<S> saveAll(Publisher<S> entityStream);
And my PostgresDataProvider.insertData is returning a Mono because I'm not interested in receive the inserted entities. How can pipeline my current method from Flux to cast into Mono?
public Mono<Void> insertData(Flux<Entity> entities) {
return repository.saveAll(entities);
}
I finally used the operator
then() → will just replay the source terminal signal, resulting in a Mono to indicate that this never signals any onNext.
That allow me to produce a Mono on the onComplete flux's signal.
public Mono<Void> insertData(Flux<RateEntity> rateEntities) {
return repository.saveAll(rateEntities).then();
}
I am working in a Kotlin and Spring Boot project and I am trying to use Caffeine for caching. I have a service with a suspending function that makes an http call. Here is my config:
#Bean
open fun caffeineConfig(): #NonNull Caffeine<Any, Any> {
return Caffeine.newBuilder().expireAfterWrite(60, TimeUnit.SECONDS)
}
#Bean
open fun cacheManager(caffeine: Caffeine<Any, Any>): CacheManager {
val caffeineCacheManager = CaffeineCacheManager()
caffeineCacheManager.getCache("test")
caffeineCacheManager.setCaffeine(caffeine)
return caffeineCacheManager
}
And here is the function that I want to cache:
#Cacheable(value = ["test"])
open suspend fun getString(id: String): String {
return client.getString(id)
}
But it seems that the caching is not working since I can see from logs that the client gets called every time the service-function gets called. Does #Cacheable not work for suspending functions? Or am I missing something else?
The documentation of #Cacheable says:
Each time an advised method is invoked, caching behavior will be applied, checking whether the method has been already invoked for the given arguments. A sensible default simply uses the method parameters to compute the key, but a SpEL expression can be provided via the key() attribute, or a custom KeyGenerator implementation can replace the default one (see keyGenerator()).
The suspend modifier inserts an Continuation<String> parameter in the generated code which accepts input from the caller. This presumably means each invocation gets its own continuation and the cache detects this as a unique call.
However since the return value also gets changed depending on the continuation you cannot have the cache ignore the continuation parameter. A better approach is to not use suspend functions and instead returning a Deferred which consumers can share:
#Cacheable(value = ["test"])
open fun getString(id: String): Deferred<String> {
return someScope.async {
client.getString(id)
}
}
// Consumer side
getString(id).await()
This should work with the standard caching mechanism since Deferred is a normal object and no special parameters are required.
I want to serialize FAIL object via Jackson:
interface OptionalResult<out ResultType : Any> {
val data: ResultType?
object FAIL : OptionalResult<Nothing> {
override val data: Nothing? = null
}
}
What I get is {} but I expect to receive {"data": null}.
How can I fix my object?
By the way, the following object is serialized properly:
object FAIL : OptionalResult<Int> {
override val data: Int? = null
}
Technical problem is that Jackson determines that indicator that would normally indicate existence of a property (public or annotated setter) will be filtered out, as getter is seen as public void getData() that returns nothing.
Filtering is done at low level processing, along with removal of static methods, methods that are neither annotated nor follow naming convention and so on.
It might be possible to improve upon this detection since there is actual difference between void and Void (similar to primitive/Wrapper difference).
But this is the first time such usage has been reported.
One thing that you could try which may (or might not) help: add #JsonProperty for val data. It could help if filtering is only done for non-annotated accessors.
I have something like:
interface Options {
fun load(conf: JsonObject)
}
object BasicOptions : Options { }
object PersonOptions : Options { }
object CarOptions : Options { }
then I would like to get all Objects that implements Options interface and call load forEach.
fun main(args: Array) {
configFuture.whenComplete { config ->
options.forEach { it.load(config) }
}
}
This is not supported by the language, this is why there are Dependency Injection, Registry, or Service Lookup solutions.
If all objects are registered in a Dependency Injection framework, you may be able to iterate all objects registered to the framework and find the instances of your Interface.
This is likely to be rather inefficient - so cache the results to avoid the extra overhead.
Another approach is to use custom class loader and add this custom functionality. The simplest way is probably using the Reflections library, to scan and load these classes. E.g.:
val reflections = Reflections("my.project.prefix")
val allClasses = reflections.getSubTypesOf(Options::class.java)
The simplest, and most commonly used solution would be simply maintain your own "registry" e.g. a static object holding the list of all instances. It will require manually adding any new class implementing the interface - but it will be simple, performant, and robust solution.
Of course, you can!
Using the Reflections library it's really easy:
First, you need to find all the java classes that implement your interface "Objects". Then you convert the java classes to kotlin classes (KClass).
KClass has the property objectInstance that is null if the class does not have an object instance... otherwise IS the singleton object instance.
So:
val reflections = Reflections(ConfigurationBuilder())
val jList = reflections.getSubTypesOf(Options::class.java)
val kList = jList.map { it.kotlin }
val oList = kList.map { it.objectInstance }.filterNotNull()
;-)
You can always use the Factory pattern for creating new instances of Options, that way you can have your own OptionsFactory and your own caching mechanism.
That OptionsFactory after instantiating an object keeps it in cache (could be in-memory cache or DB..etc), then anytime you can ask the factory for it's cached instances when needed.
This is fairly better when your Options objects are created in run-time/upon demand.
I'm new to Golang and have been taking a TDD approach while learning the language. I've been getting along okay yet I find testing third party packages quite clunky, which leads me to believe that I've been taking the wrong approach.
The specific case I'm having trouble with is mocking a Redis client for error handling. The approach I've taken is to create my own interface, and the implementation wraps the clients methods that I want to use.
type Redis interface {
Get(key string) (string, error)
}
type RedisClient struct {
client *redis.Client
}
func (redisClient *RedisClient) New(client *redis.Client) *RedisClient {
redisClient.client = client
return redisClient
}
func (redisClient *RedisClient) Get(key string) (string, error) {
return redisClient.client.Get(key).Result()
}
I can then create a mock which implements that same interface to return whichever values I specify, particularly for testing error handling.
I've hit a roadblock where a specific method on the client to perform transactions (MULTI) returns another interface belonging to that package. What would I do in this scenario? Implementing that interface myself seems out of the question.
Similarly, as usage of this client grows, my own implementation can grow to the point that it implements the whole interface to Redis - this seems to go against the whole idea of delegating this out to an external dependency.
Is there a better way to test third-party packages like this, for things such as error handling?
One approach would be to create a type that focuses on what you want to accomplish instead on what methods of the client you are using.
Let's say all you want is a storage to save and fetch users, you could imagine an interface like this:
type UserStore interface {
SaveUser(*User) error
GetUserByID(id string) (*User, error)
SearchUsers(query string) ([]User, error)
}
You could then implement a Redis version of this storage and call whatever client methods you want inside, it doesn't matter. You can even implement one in PostgreSQL or whatever.
Also, mocking is way easier with this approach since you all you need to do is to implement this interface instead of the Redis one.
Here is an example of a mock version of this interface:
type UserStoreMock struct {
SaveUserFn func (*User) error
SaveUserInvoked bool
...
}
func (m *UserStoreMock) SaveUser(u *User) error {
m.SaveUserInvoked = true
if m.SaveUserFn != nil {
return m.SaveUserFn(u)
}
return nil
}
...
You can then use this mock in tests like this:
var m UserStoreMock
m.SaveUserFn = func(u *User) error {
if u.ID != "123" {
t.Fail("Bad ID")
}
return ErrDuplicateError
}
...