For a startup-company project, we are renting three linux servers from the same datacenter in France (from OVH).
We are using three VPS at the moment. We will later switch to dedicated servers in case of commercial success.
We want to install a replicated distributed database on these 3 VPS, using a replication factor of 2 to allow a minimum of fault-tolerance.
If possible, we'd like to use Aerospike, as we prefer it over MongoDB and CouchDB.
So my question is : is it possible to use Aerospike Community Edition to replicate the database records across these 3 VPS, without XDR ? And how can we achieve that ?
Sure, XDR is only needed for replication across datacenters. To replicate within a cluster in a datacenter, configure your namespace's replication-factor to the desired value.
If you want your data replicated on two separate but identical configuraton Aerospike clusters (clusterA with 3 VSPS, clusterB with 3 VSPs) (Is what you are asking for?) on CE without using XDR, you can instantiate two client objects in your application, use one clientA object to write to clusterA, use the other clientB object to repeat the operation to the other clusterB. You will have a performance hit but may work for you.
If you just have one cluster of 3 VSPs, setting replication factor of two in your namespace configuration automatically keeps one master record and one replica on the same cluster, record level data evenly distributed across the cluster, with master and replica of any record always on different nodes.
Related
I have 3 redis instance with redis. One is the master and the other two, are the slaves. I have connected to master node and get info by redis-cli with INFO command. I can see the parameter cluster_enabled:0 and
#Replication
role:master
connected_slaves:2
slave0:ip=xxxxx,port=6379,state=online,offset=15924636776,lag=1
slave1:ip=xxxxx,port=6379,state=online,offset=15924636776,lag=0
And the keyspace, each node has different dbs. I need to migrate all data to a single memorystore in GCP but I don't know how. Anyone can help me?
Since the two nodes are slaves and clustering is not enabled, you only need to replicate the master node. RIOT is a great tool for migrating data in and out of Redis.
However, if you say DB by node do you mean redis DB that you access by select? In that case you'll need to prefix keys as there may be overlap between the keysets of the DBs.
I think setting up another Redis cluster in a single node configuration is the least of your worries.
The real challenge for you would be migrating all your records over to the new setup. This is not a simple question to answer and would depend heavily on multiple factors:
The total size of your data being migrated
Is this is a live Database in production
Do you want to keep the two DB schemas in your new configuration separate?
Ok, I believe currently your Redis Instances are hosted on Google Compute Engine.
And you are looking to migrate to Memorystore for Redis.
As mentioned here, you can leverage Redis snapshots for this. It provides you step-wise instructions on how to achieve this, leveraging GCS buckets as transient storage.
import data into Cloud Memorystore instances using RDB (Redis Database Backup) snapshots, as well as back up data from existing Redis instances.
Design a Database Architecture for a Multi-tenant Application 🏘️ with the following Criteria:
Multiple Tenant Types ( different schema for each Tenant with around 20 tables each)
starting with 2 should scale to 50+ Tenant Types 📈
Each Tenant Type starting with 20 should scale to 1000 Tenants with 50 to 500 users each 🚀
Data isolation is not crucial
Cost of ownership at least, in the beginning, should be super low 😬
What would be an ideal choice based on these criteria?
My choice :
Citus PostgreSQL with a Different schema/Database per Tenant type and sharding tenants, running the server on a VM. Now Citus has 2 deployment types ie..Single machine cluster and Multi-machine cluster
Can we start with a Single machine cluster initially thereby saving some bucks, and them convert to Multi-machine cluster later on with no or minimal downtime? so from there we can scale out horizontally adding new physical worker node instances.
Switching from single machine to multi machine should be possible, details will vary depending on your exact environment. At worst you should be able to replicate the worker databases from the single machine to new machines & then switch to using those new machines
Let us say I've two server nodes in one data center DC1 and two more server
nodes in another data center DC2. Two data centers have some network delay.
Now I'm using SQL select statements on caches which are replicated. Now
those caches' write synchronization mode is FULL_SYNC.
Now at a time we have working clients nodes only in one DC but not both.
Let's say we have two clients in DC1.
So total nodes is 6 (2 client nodes and 2 server nodes in DC1 and 2
server nodes in DC2).
Our use case is such a way that..
2 clients should query only 2 server nodes in DC1 and not the other 2
servers in DC2.
All the cache queries should be in FULL_SYNC with 2 server nodes in DC1
and DC1-DC2 should be done in ASYNC mode.
A doubt I got which is, if in client's node discoveryspi, if I (X,Y) ip
list as server nodes ips, would the queries always reach X,Y even though the
entire topology contains X,Y,Z as server nodes?
Please someone provide us the solution for this.
Note: I saw one GridGain's capability for cluster-cluster replication but that comes under paid version. I am looking for a solution in the community edition.
A doubt I got which is, if in client's node discoveryspi, if I (X,Y) ip list as server nodes ips, would the queries always reach X,Y
even though the entire topology contains X,Y,Z as server nodes?
No, DiscoverySPI is used only for the connecting to the cluster, after that, client node will be working with all nodes in the cluster.
All the cache queries should be in FULL_SYNC with 2 server nodes in
DC1 and DC1-DC2 should be done in ASYNC mode.
It's not possible to do this, only one synchronization mode can be used for one cache in the cluster.
2 clients should query only 2 server nodes in DC1 and not the other 2 servers in DC2.
It's not possible to do this for cache operations, but you can do this for computing operations - you can send a job to a certain node with a primary or backup copy in DC1 and it will take the local partition. But compute creates some overhead compared to the plain cache operations if it used only for getting the entries.
So, as you mentioned, the best way here is the DataCenter Replication, which is available as a part of GridGain, because, based on your requirements, you need 2 separate clusters here.
I have 2 nodes, in which im trying to run 4 ignite servers, 2 on each node and 16 ignite clients, 8 on each node. I am using replicated cache mode. I could see the load on cluster is not distributed eventually to all servers.
My intension of having 2 servers per node is to split the load of 8 local clients to local servers and server can work in write behind to replicate the data across all servers.
But I could notice that only one server is taking the load, which is running at 200% cpu and other 3 servers are running at very less usage of around 20%cpu. How can I setup the cluster to eventually distribute the client loads across all servers. Thanks in advance.
I'm generating load by inserting same value 1Million times and trying to get the value using the same key
Here is your problem. Same key is always stored on the same Ignite node, according to Affinity Function (see https://apacheignite.readme.io/docs/data-grid), so only one node takes read and write load.
You should use a wide range of keys instead.
I'm a newbie to Redis and I was wondering if someone could help me to understand if it can be the right tool.
This is my scenario:
I have many different nodes, everyone behaving like a master and accepting clients connections to read and write a few geographical data data and the timestamp of the incoming record.
Each master node could be hosted onto a drone that only randomly get in touch and can comunicate with others, accordind to network conditions; when this happens they should synchronize their data according to their age (only the ones more recent than a specified time).
Is there any way to achieve this by Redis or do I have to implement this feature at application level?
I tried master/slaves configuration without success and I was wondering if Redis Cluster can somewhat meet my neeeds.
I googled around, but what I found had not an answer good for me
https://serverfault.com/questions/717406/redis-multi-master-replication
Using Redis Replication on different machines (multi master)
Teo, as a matter of fact, redis don't have a multi master replication.
And the cluster shard it's data through different instances. Say you have only two redis instances. Instance1 will accept store and retrieve instance1 and instance2 data. But he will ask for, and store in, instance2 every key that does not belong to his shard.
This is not, I think, really what you want. You could give a try to PostgreSQL+BDR as PostgreSQL supports nosql store and BDR provides a real master master replication (https://wiki.postgresql.org/wiki/BDR_Project) if that's really what you need.
I work with both today (and also MongoDB). Each one with a different goal. Redis would provide a smaller overhead and memory use, fast connection and fast replication. But it won't provide multi master (if you really need it).