AWS Redis Elasticache is advertised as being sub-millisecond. But in any meaningful use case there will be a network latency calling the elasticache service, so at best in the 10s of milliseconds…is this correct ? Redis Elasticache always has to be called over the network, not running on the same server as the application server correct ?
Thats correct. Elasticache provides sub millisecond response times, but you will always have network delays. If your application is hosted on EC2 in the same region then latency will be sub millisecond at the application end. However, the further away you place your application from the elasticache cluster the greater latency you will receive, you cannot defy the laws of physics.
Here is a good guide to reducing latencies for Elasticache for Redis:
https://aws.amazon.com/premiumsupport/knowledge-center/elasticache-redis-correct-high-latency/
Related
We are using ElastiCache for redis with cluster-mode on. Recently we are seeing random replica-to-primary connection issue which leads our client connection timeout at mget or xread. Usually this lasts for about 20 minutes and will recover itself but it still brought us customer experience impact. My question here is:
if redis fail to read from one replica, would it re-route the same request to another replica in the same shard?
what's the recommended way to mitigate the impact?
thanks in advance!
I have a Redis Elasticache non-clustered instance with one primary node and two read replicas. Using the Stack Exchange Redis client I provide the reader endpoint and make a get request. Based on the documentation I would expect:
A reader endpoint will split incoming connections to the endpoint
between all read replicas in a Redis cluster.
However, 100% of the requests go to one of the read replicas. First question, why? Second question, how do I get Redis to distribute the load across all of the read replicas without having to manage the read instances at an application level?
You should use the "Reader Endpoint" connection string. (the connection string with "-ro")
This will split the connection between your replicas in case you have more than one connection to the Redis Cache server. Also to achieve this you need to have significant CPU usage to the first redis-replica server.
As EKS SLA, a 99.9% avialbility is committed by Amazon.
How I can increase that to 99.99% (or even 99.999%)? Would it help if I add master/slave nodes?
Thanks.
I don't think there is a way to do this and still call your setup an AWS EKS Cluster. It is defined in the EKS SLA that an EKS Cluster means that the control plane would be run by AWS. Three masters in different AZs provide pretty much good HA.
A workaround may be introducing a queue between the control plane (i.e. the k8s api) and your requests. The queue can retain the requests which were not successful due to availability issues and can again send the request based on some priority or time-based logic. This won't increase the HA for real-time tasks, but wouldn't let requests made from asynchronous use cases go to waste.
I am currently setting up an infrastructure for an App in AWS. App is written in Django and is using Redis for some transactions. High availability is key for this application and I am having a hard time trying to get my head around how to configure Redis for High availability.
Application level changes are not an option.
Ideally I would like to have a redis setup, to which I can write and read and replicate and scale when required.
Current Setup is a Redis Fail-over scenario with HAProxy --> Redis Master --> Replica Slave.
Could someone guide me understand various options ? and how to scale redis for high availability !
Use AWS ElastiCache Redis Cluster with Multi-AZ. They provides automatic fail-over. It provides endpoint to access master node.
If master goes down AWS route your endpoint to another node. everything happens automatically, you don't have to do anything.
Just make sure that if you are doing DNS to IP caching in your application, its set to 60 seconds or so instead of default.
http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoFailover.html
Thanks,
KS
AWS ElastiCache servers with redis come in everything from very small to very large multi cpu boxes. But redis is single threaded. Anyone know what Amazon is doing to make it use all the cores? I'm assuming that they do, otherwise it's kind of strange that they would be offering it.
The response from AWS was that redis is indeed single threaded. But it's a good suggestion to have more than one CPU to handle OS and network chores, so that Redis gets the resources to run. This makes sense.