Trying to create a NiFi Flow to query all keys in Redis DB.
There is no available processors that can do it.
Can anyone assist?
Related
With apache flink is it possible to write to a hive cluster such that the cluster is able to distribute the data among his nodes?
Example as described here seems to indicate data is intended to a HDFS on the apache flink node itself. But what options exist if you intend to have the HDFS on a separate cluster and not on the flink worker nodes?
Please bear with me, I am totally new to this topic and I could get something conceptually completely wrong.
Yes, you can read from and write to Hive using Flink. There's an overview available at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/hive/hive_read_write/
I've read https://github.com/redisson/redisson
And I found out that there are several
Redis Replicated setup (including support of AWS ElastiCache and Azure Redis Cache)
Redis Cluster setup (including support of AWS ElastiCache Cluster and Azure Redis Cache)
Redis Sentinel setup
Redis with Master with Slave only
I am not a big expert in clusters and I don't understand the difference between these setups.
Could you beiefly explain the differences ?
Disclaimer I am an AWS employee.
I do not know how Redis Replicated Setup is different from Redis in Master-Slave mode. Maybe they mean cross-region replication?
In any case, I can try and explain setups I know about:
Redis with Master with Slave only - is a single shard setup where you create a primary replica together with one or more secondary (slave) replicas (let's hope PC police won't arrest me). This setup is used to improve the durability of your in-memory store. It's not advised to use your secondaries for reads because such setup has eventual consistency guarantees and your replica reads may be stale (depending on the replication lag).
Redis Cluster setup - the setup supported by cloud provides such as AWS Elasticache. In this setup your workload can be spread horizontally across multiple shards and each shard may have its own secondary replicas. Your client library must support this setup since it requires maintaining multiple connections to several nodes at a client level. Moreover, there are some locality rules you need to follow in order to use cluster mode efficiently:
Keys with foo{<shard>}bar notation will be routed to their shard according to what is stored inside curly brackets.
You can not use mset, mget and other multi-key commands across shards. You can still use these commands if their keys contain the same {shard} part.
There are additional cluster mode admin commands that are exposed by Redis but they are usually hijacked and hidden from users by cloud providers since cloud provides use them in order to manage redis cluster themselves.
Redis cluster have an ability to migrate part of your workload between shards. However, it still obliged to preserve correctness with respect to {shard} notation. Since your client library is responsible to fetch data from specific shard it must handle "moved" response when a shard might redirect it to another node.
Redis Sentinel setup - using an additional server that provides service discovery functionality for Redis clusters. Not strictly required and I believe is less popular across users. It serves as a single source of truth regarding each node's health and state. It provides monitoring, management, and service discovery functions for managing your Redis cluster. Many Redis client libraries provide the option of connecting to Redis sentinel nodes in order to achieve automatic service discovery and seamless failover flow. One of the reasons why this setup is less popular is because cloud companies like AWS Elasticache provide this service out of the box.
Redis by default have 16 databases that can be selected by using SELECT command.
But as per the docs, https://redis.io/commands/select,
When using Redis Cluster, the SELECT command cannot be used, since
Redis Cluster only supports database zero. In the case of a Redis
Cluster, having multiple databases would be useless and an unnecessary
source of complexity.
As per redis docs, cluster/HA is achieved by
cluster - https://redis.io/topics/cluster-tutorial
sentinel -
https://redis.io/topics/sentinel.
I am clear that Redis Cluster only supports database zero, but I couldn't get the info anywhere to check the support for multiple databases in Sentinel setup with multiple nodes?
Any reference to this would be helpful. Thanks!
With sentinel setup, you can have multiple databases.
Redis Sentinel is only used to provide HA for Redis, it doesn't change Redis in any way. And you can use this Redis as the single instance Redis without sentinel.
I have deployed my application on SAP Cloud Foundary which creates connection with Redis Multinode instance running on the cloud.
The application can push and fetch data from Redis successfully.
Now I want to check high availability of Redis Nodes by crashing nodes of redis one by one (cloud has the instance of Redis with 3 nodes, 1 being master and other 2 as its slave).
I have been trying to crash anyone of the nodes by using redis-cli, ssh connection but could not get success in that.
Can anyone please help me to know how to crash any single node of Redis cluster in SAP cloud environment?
Any help would be appreciated.
Thanks :)
I try to use redis for http session data replication. My Use case is as follows:
We have 2 indepentent datacenters(RZ and RR) each of them has 4 tomcat servers.
I have installed redis cluster with sentinel (One Master, 2 slaves and 3 sentinels)on one dedicated server on each datacenter. Each cluster is working as expected.
Now I want to synchrinze data between the 2 Masters nodes (cross-datacenter replication), so if our loadbalancer decide to shwitch from DC RZ(Primary) to DC RR (secondary) session data is available and no session is lost.
I tried to install dynomite framework formy purpose but failling to install it. So My question can redis handle such senario without third party tools such dynomite?
Any Help to achieve the replication between datacenter is very willcome.
Sorry for my bad english.
Thank you in Advance.
No, Redis does not offer master-master solution. See redis replication and cluster tutorial to understand how redis replication works
https://redis.io/topics/replication
https://redis.io/topics/cluster-tutorial
Dynomite is brilliant active-active solution, we have been using it for quite sometime.
No, there is no normal way to do this.