The last time I checked the logstash doc said that the official redis input plugin uses "blpop" command to get messages from redis. (Correct me if I'm wrong.)
However, our redis cluster does not allow command "blpop". So I'm wondering is there any redis input plugin that uses "lpop" command? Or do I have to write my own.
Thx!
There wasn't any so I wrote my own version of logstash redis input plugin that using "lpop" instead of "blpop" (in the official logstash redis input plugin) here
Related
I have set up a Celery task that is using RabbitMQ as the broker and Redis as the backend. After running I noticed that my Redis server was still using a lot of memory. Upon inspection I found that there were still keys for each task that was created.
Is there a way to get Celery to clean up these keys only after the response has been received? I know some MessageBrokers use acks, is there an equivalent for a redis backend in Celery?
Yes, use result_expires. Please note that celery beat should run as well, as written in the documentation:
A built-in periodic task will delete the results after this time (celery.backend_cleanup), assuming that celery beat is enabled. The task runs daily at 4am.
Unfortunately Celery doesn't have acks for its backend, so the best solution for my project was to call forget on my responses after I was done with them.
I was wondering why do we need Redis server for running CKAN.
If needed, why? And How do I configure it with CKAN?
p.s
I am running my ckan instance in RHEL7.
Update: Redis has been a requirement since CKAN 2.7, when a new system for asynchronous background jobs was introduced which relies on Redis. You can configure the Redis connection using the ckan.redis.url option.
Redis is not required for the current version of CKAN (2.6.2 at the time of this writing), it's not even mentioned in the CKAN 2.6.2 documentation.
However, the upcoming 2.7 release will require Redis for its new system of asynchronous background jobs. Redis will be configured using the new ckan.redis.url option.
I use monitor redis command for debugging some flakiness in our integration tests. Does order of commands in output of monitor equal to real order of commands processed by server?
I use redis 3.0.7 from official redis:3.0 docker image.
Yes, you can. In terms of implementation, before executing any command, Redis sends it to all MONITORing clients, so the order of commands in the the monitor stream matches the order of commands' execution.
Is there a way for a client to get notified about failover events in the Redis cluster? If so, which client library would support this? I am currently using Jedis but have the flexibility to switch to any other Java client.
There are two ways that I can think of to check this, one of them is to grep for master nodes on the cluster, keeping in mind their IDs, if the ports changed for any of them then a failover happened.
$ redis-cli -p {PORT} cluster nodes | grep master
Another way, but it is not as robust of a solution is using the consistency checker ruby script, that will start showing errors in writes as an output, which you can monitor and send notifications depending on it, since that happens when the read server is trying to take its master's role.
Sentinel (http://redis.io/topics/sentinel) has the ability to monitor the cluster member, and send a publish/subscribe notification upon failure. The link contains a more in-depth explanation and tutorial.
I just started evaluating Redis. I am using Redis 2.8.19 which the most latest stable release. Redis 2.9 is still unstable and Redis 3.0 is just available for developer's preview (not recommended for production). I was tryin to setus a cluster of Redis and when I changed my redis.conf and appended
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
and started my Redis server by
src/redis-server ./redis.conf
it gave me an error as follows
* FATAL CONFIG FILE ERROR *
Reading the configuration file, at line 2
'cluster-enabled yes'
Bad directive or wrong number of arguments
I googled the error and got to know that my version (2.8.19) does not support cluster. I was still unable to fine any such specification in Redis Docs. My question is simple. Does Redis 2.8.19 supports redis cluster configuration? Or I have to upgrade to Redis 2.9 or Redis 3.0. I am evaluating Redis because I need to deploy it in production. Please guide.
Redis Cluster support is only for versions >= 3.0.0. Redis 3.0.0 will be released as a stable version in a matter of days, it's a good idea to use it if you want to use Cluster. The cluster support is considered to be stable, however for it to be considered mature we want to see adoption. Btw there is already at least a very large site using it in production. Currently the most sane thing to do if you need Redis Cluster is to test it for your use case, and if it looks great, use it.
Redis cluster is supported only in Redis 3.0+ (which is now stable). I have written a simple API called "Simple Redis Cluster Client" which can be used in redis's sub 3.0 versions for running in a cluster like mode (Not precisely a cluster, it just distributes keys among redis nodes based on the key's hashcode, You can have a look # https://github.com/prash-mi/simple-redis-cluster-client
Cluster support for Redis is only from v3 - v2.8.19 doesn't do clustering.