I use redis-py to operate on redis, and our environment use twemproxy as redis proxy. But looks clinet pipeline doesn't work when connect to twemproxy.
import redis
client = redis.StrictRedis(host=host, port=port, db=0)
pipe = client.pipeline()
pipe.smembers('key')
print pipe.execute()
it throws exception when do execute method
redis.exceptions.ConnectionError: Socket closed on remote end
In twemproxy environment, client pipeline doesn't work or it is an issue of redis-py ?
Becouse of twemproxy supports not all redis commands.
Here is actual supported commands list https://github.com/twitter/twemproxy/blob/master/src/proto/nc_redis.c
redis-py pipeline will use transaction by default, try this:
pipe = r.pipeline(transaction=False)
Related
I installed and set up Airflow 2.0.1 with Celery, Rabbitmq and Postgresql9.6 on RHEL7, using the constraints https://raw.githubusercontent.com/apache/airflow/constraints-2.0.1/constraints-3.7.txt.
So I am not using Docker container, and in fact am building a cluster with 3 nodes.
I created DB and username for PSQL, and created user and set permission for Rabbitmq and am able to access its WebUI in 15672 port.
I am able to run airflow webserver and scheduler and access airflow WebUI with no problem.
The issue arises when I try to start airflow worker (whether from master node or worker nodes). Even though airflow.cfg is set to point out to rabbitmq, I get the error that says:
ImportError: Missing redis library (pip install redis)
Because it is trying to access redis instead of rabbitmq, but I have no idea why.
I checked airflow.cfg line by line and there is not a single line with redis, so am I missing a configuration or what?
My airflow.cfg configuration:
sql_alchemy_conn = postgresql+psycopg2://airflow_user:airflow_pw#10.200.159.59:5432/airflow
broker_url= amqp://rabbitmq_user:rabbitmq_pw#10.200.159.59:5672/airflow_virtual_host
celery_result_backend = db+postgresql://airflow_user:airflow_pw#10.200.159.59:5432/airflow
dags_are_paused_at_creation = True
load_examples = False
Why does my airflow worker try to reach redis when it is configured for rabbitmq?
I found the problem after spending many hours on such a simple, silly issue.
Airflow still tried to connect to redis, which isthe default Airflow config despite my rabbitmq configuration in airflow.cfg because I had written all of the configs under [core] section, wheras lines must be written to related parts in airflow.cfg.
I moved broker_url and result_backend to under [celery] and issue was resolved.
How to get all connected clients of a redis cluster?
I am using AWS elasticCache redis with non cluster mode and redission as my redis client.
My Use Case:
I need to run specific code from only 1 connected redis client.
Thanks
redis has command about client information like CLIENT LIST, check out this page .
you could checkout this page for the command redisson has not supported yet.
I am a novice to Celery, Redis, and RabbitMQ.
Currently, I'm using RabbitMQ as a message broker, and nothings are set in configuration. (with Django, MySQL)
I am wondering if it's possible to use Redis as a result store in backend, at the same time, RabbitMQ as a message broker.
The thing I know is only adding some settings, CELERY_RESULT_BACKEND = "redis"
Yes, it's possible. Just set:
CELERY_RESULT_BACKEND = "redis://:<password>#<hostname>:<port>/<db_number>"
replacing <password>, <hostname>, <port> and <db_number>.
I am getting this exception. Any clue?
StackExchange.Redis.RedisConnectionException:
It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail.
Possible reasons:
Is Redis up and running? Try to connect to it using redis-cli.
Is your connection settings to Redis right in your code (i.e. you're connecting to the right host and port...)?
The exception is quite expressive: ConnectionMultiplexer couldn't reach target Redis server.
I'm having problems figuring out how to use the Jedis library to connect to a redis socket connection.
I know how to connect through a network port:
Jedis jedis = new Jedis("localhost");
//Jedis jedis = new Jedis(unix_socket_path="/tmp/redis.sock");
But the socket connection(second in the list) doesn't work. The commands looked simlair to redis-py(python client) but when I tried the same syntax it didn't work. I also looked through the jedis sourcecode on github but couldn't see anything. Any ideas?
I don't think Jedis support unix domain sockets.
The constructor with a single parameter only accepts a hostname (using default TCP port).
Java is portable. It is supposed to provide the same API on different platforms. Unix domain sockets are specific to Unix/Linux. So the Java standard API does not support unix domain sockets. There are separate Java packages for this, but AFAIK, Jedis do not use them.