AWS ElastiCache servers with redis come in everything from very small to very large multi cpu boxes. But redis is single threaded. Anyone know what Amazon is doing to make it use all the cores? I'm assuming that they do, otherwise it's kind of strange that they would be offering it.
The response from AWS was that redis is indeed single threaded. But it's a good suggestion to have more than one CPU to handle OS and network chores, so that Redis gets the resources to run. This makes sense.
Related
AWS Redis Elasticache is advertised as being sub-millisecond. But in any meaningful use case there will be a network latency calling the elasticache service, so at best in the 10s of milliseconds…is this correct ? Redis Elasticache always has to be called over the network, not running on the same server as the application server correct ?
Thats correct. Elasticache provides sub millisecond response times, but you will always have network delays. If your application is hosted on EC2 in the same region then latency will be sub millisecond at the application end. However, the further away you place your application from the elasticache cluster the greater latency you will receive, you cannot defy the laws of physics.
Here is a good guide to reducing latencies for Elasticache for Redis:
https://aws.amazon.com/premiumsupport/knowledge-center/elasticache-redis-correct-high-latency/
I need an HA redis solution instead of a single instance. Should I use cluster or Sentinel? I have tried to find out the difference between them, there is no official document about this, thanks a lot.
Well, for a HA redis solution , it depends upon the number of nodes you want to configure.
According to offical Redis documentation on Redis-cluster and Redis-sentinel both provides HA Solution but.....
Redis Sentinel provides high availability for Redis. In practical terms this means that using Sentinel you can create a Redis deployment that resists without human intervention to certain kind of failures.
Redis Cluster provides a way to run a Redis installation where data is automatically sharded across multiple Redis nodes.
Redis Cluster also provides some degree of availability during partitions, that is in practical terms the ability to continue the operations when some nodes fail or are not able to communicate. However the cluster stops to operate in the event of larger failures (for example when the majority of masters are unavailable).
For more information please refer the official docs :)
Cheers
I am currently setting up an infrastructure for an App in AWS. App is written in Django and is using Redis for some transactions. High availability is key for this application and I am having a hard time trying to get my head around how to configure Redis for High availability.
Application level changes are not an option.
Ideally I would like to have a redis setup, to which I can write and read and replicate and scale when required.
Current Setup is a Redis Fail-over scenario with HAProxy --> Redis Master --> Replica Slave.
Could someone guide me understand various options ? and how to scale redis for high availability !
Use AWS ElastiCache Redis Cluster with Multi-AZ. They provides automatic fail-over. It provides endpoint to access master node.
If master goes down AWS route your endpoint to another node. everything happens automatically, you don't have to do anything.
Just make sure that if you are doing DNS to IP caching in your application, its set to 60 seconds or so instead of default.
http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoFailover.html
Thanks,
KS
I see there are multiple modes of operation for Redis (cluster, sentinel, master-slave, etc?). I don't fully understand the implications of each, but my question is this:
If I have a web application that requires distributed session persistence, which configuration of Redis makes the most sense? The main reason I'm using redis is to achieve some level of fault tolerance. If one of my frontend servers fails, I want the sessions to be available for other nodes to pickup the workload. If a redis node goes down, I don't want this to affect the user experiences, and I don't want to have to wake up a developer at midnight to correct the matter.
From everything I've read, Redis Sentinel is the way to go for fault tolerance.
I'm considering Redis for a section of the architecture of a new project. It will consist of a lot of clients (node.js connections) SUBSCRIBING to particular keys with one process PUBLISHING to those keys as needed.
I'm curious about the limits of the PUBLISH/SUBSCRIBE commands and how to mitigate those. An obvious limit is the amount of file descriptors open on the machine with Redis so at some point I'll need to implement Master-Slave or Consistent Hashing to multiple Redis instances.
Does anyone have any solutions about how to scale this architecture with Redis' PubSub?
Redis PubSub scales really easily since the Master/Slave replication automatically publishes to all slaves.
The easiest way is to load balance the connections to node.js with for instance HAProxy, run a Redis slave on each webserver that syncs with a single master that publishes the messages.
I can't give you exact numbers since that greatly depends on the underlying system, but this should scale extremely well. And you don't need to manage the clients and which server they connect to manually. You obviously need some way to handle session state, so you might need to do that anyway, but that's a lot easier to do in the load balancer than in your application.