I'm migrating my RavenDb database to RavenHq and trying to setup replication for the first time. I'm considering to have a Master-Master replication with Failover equal to AllowReadsFromSecondariesAndWritesToSecondaries. What kind of behaviour will change in this case in comparison to single server scenario.
As I understand RavenDb will write to Master and read from Master (so edit-list scenario should not be a problem) until failover happens. So my only concern should be resolve failover-related conflicts (that are pretty unlikely to happen). Or should I take anything else into consideration?
Yes, that is correct. Note that conflicts can be handled automatically by the server by selecting the appropriate strategy
Related
I'm reading about OpenLDAP replication and I don't understand why you would use refreshOnly mode vs. refreshAndPersist mode.
I've tried to do some searching online, but wasn't able to find any discussion on the benefits (if any) of refreshOnly.
This page says "In this configuration, assuming that a refreshAndPersist type of synchronization is used (it is not clear why you would even want to think about using refreshOnly but it is possible), then a write (modify) to any master will be immediately propagated to all the other masters (providers) acting in their slave (consumer) role."
But's it's referring to Multi-Master replication and doesn't say anything similar for Provider/Consumer replication.
Is there any reason why I should ever consider refreshOnly for OpenLDAP replication?
Thanks!
You would use refreshOnly if you wanted to completely control when replication took place, e.g. every midnight. You would use refreshAndPersist when you want replication to be continuous and as instantaneous as possible.
I have an application that, for performance reasons, will have completely independent standalone instances in several Azure data centers. The stack of Azure IaaS and PaaS components at each data center will be exactly the same. Primarily, there will be a front end application and a database.
So let's say I have the application hosted in 4 data centers. I would like to have the data coming into each Azure SQL database replicate it's data asynchronously to all of the other 3 databases, in an eventually consistent manner. Each of these databases needs to be updatable.
Does anyone know if Active Geo-Replication can handle this scenario? I know I can do this using a VM and IaaS, but would prefer to use SQL Azure.
Thanks...
Peer-to-peer tranasaction replication supports what you're asking for, to some extent - I'm assuming that's what you're referring to when you mention setting it up in IaaS, but it seems like it would be self defeating if you're looking to it for a boost in write performance (and against their recommendations):
From https://msdn.microsoft.com/en-us/library/ms151196.aspx
Although peer-to-peer replication enables scaling out of read operations, write performance for the topology is like that for a single node. This is because ultimately all inserts, updates, and deletes are propagated to all nodes. Replication recognizes when a change has been applied to a given node and prevents changes from cycling through the nodes more than one time. We strongly recommend that write operations for each row be performed at only node, for the following reasons:
If a row is modified at more than one node, it can cause a conflict or even a lost update when the row is propagated to other nodes.
There is always some latency involved when changes are replicated. For applications that require the latest change to be seen immediately, dynamically load balancing the application across multiple nodes can be problematic.
This makes me think that you'd be better off using Active Geo Replication - you get the benefit of PaaS and not having to manage your own VMs, not having to manage TR, which gets messy, and if the application is built to deal with "eventual consistency" in the UI, you might be able to get away with slight delays in the secondaries being up to date.
I am looking for a (SQL/RDB) database setup that works something like this:
I will have 3+ databases in an active/active/active configuration
prior to doing any insert, the database will communicate with atleast a majority of the others, such that they all either insert at the same time or rollback (transaction)
this way I can write and read from any of the databases, and always get the same results (as long as the field wasn't updated very recently)
note: this is for a use case that will be very read-heavy and have few writes (and delay on the writes is an OK situation)
does anything like this exist? I see all sorts of solutions with database HA configurations, but most of them suggest writing to a primary node or having a passive backup
alternatively I could setup a custom application, and have each application talk to exactly 1 database, and achieve a similar result, but I was hoping something similar would already exist
So my questions is: does something like this exist? if not, are there any technical/architectural reasons why not?
P.S. - I will NOT be using a SAN where all databases can store/access the same data
edit: more clarifications as far as what I am looking for:
1. I have no database picked out yet, but I am more familiar with MySQL / SQL Server / Oracle, so I would have a minor inclination towards on of those
2. If a majority of the nodes are down (or a single node can't communicate with the collective), then I expect all writes from that node to fail, and accept that it may provide old/outdated information
failure / recover scenario expectations:
1. A node goes down: it will query and get updates from the other nodes when it comes back up
2. A node loses connection with the collective: it will provide potentially old data to read request, and refuse any writes
3. A node is in disagreement with the data stores in others: majority rule
4. 4. majority rule does not work: go with whomever has the latest data (although this really shouldn't happen)
5. The entries are insert/update/read only, i.e. there will be no deletes (except manually ofc), so I shouldn't need to worry about an update after a delete, however in that case I would choose to delete the record and ignore the update
6. Any other scenarios I missed?
update: I the closest I see to what I am looking for seems to be using a quorum + 2 DBs, however I would prefer if I could have 3 DBs instead, such that I can query any of them at any time (to further distribute the reads, and also to keep another copy of the data)
You need to define "very recently". In most environments with replication for inserts, all the databases will have the same data within a few seconds of an insert (and a few seconds seems pessimistic).
An alternative approach is a "read-from-one/write-to-all" approach. In this case, reads are spread through the system. Writes are then sent to all nodes by the application (or a common layer that the application uses).
Remember, though, that the issue with database replication is not how it works when it works. The issue is how it recovers when it fails and even how failures are identified. You need to decide what happens when nodes go down, how they recover lost transactions, how you decide that nodes are really synchronized. I would suggest that you peruse the documentation of the database that you are actually using and understand the replication mechanisms provided by that platform.
We are using Replication and seem to be having endless problems with it. It seems to shut down for unknown reasons. It needs to be shut down to remove a column and only starts back up half the time. Does anyone have any advice on how to properly use replication or some alternatives to it.
Edit:
We are using Sql Server 2005, We cannot use database mirroring as we used the other database for reporting. As far as I am aware you cannot query from a mirrored database.
If you need just couple of tables from your DB for reports, replication is more useful, but you also can set up log shipping with secondary server in STAND BY mode (especially if you need significant part of your data for reports), then you can run reports on secondary server. You just have to remember that log shipping will interfere with transaction log backups, so you have to use the same folder with log backup files for both processes.
I would think the combination of database mirroring and database snapshots will solve your issues.
First, database mirroring is very easy to setup and I have never had any problems with it (using it for the past 4+ years).
Second, creating a database snapshot on your failover server will allow you to run reports. You can setup a sql agent job to drop and re-create the snapshot on whatever acceptable interval you like.
Of course this is all dependent on if you need your reports to run on real-time data or if they can be delayed somewhat.
Here are a list of the problems that I have had to resolve to get replication working:
1) The replication sometimes lies to me and tells me this, even when its working fine.
"The server 'Bob' is not a Subscriber. (.Net SqlClient Data Provider)" I have tried to re-initialise it thinking that it was broken and it never was...
2) It can take a little while to restart itself, especially if your remote DB is on the other side of the planet, which it is in my case. If you are on a slow network connection, or it is not 100% reliable, then you can have problems. Also, the jobs which restart the process can sometimes take a while to run, which also delays things further.
3) Some changes require full re-initalisation which involves sending a new snapshot out. If you don't have your permissions quite right, and you can re-initialise manually, but it doesn't happen automatically, then this can be a another reason for problems.
We have a SQL transactional replication which runs perfectly happily. You seem to say that it is when you are making schema changes to the publisher that you get problems. Each time we do a schema change we drop the publication, subscription and the subscription database. Do the change, then re-build it all. We can do this becuase we can tolerate the time it takes to re-apply the snapshot. There are ways to apply schema changes to the publication and have them propogate to the subscriber. Take a look at sp_register_custom_scripting. We have made this work once, so I can give some more information about it if you need.
As #Jason says, you can report from a mirrored database by using a snapshot. Beware that the snapshot will take up space, and cause more work for the mirror server. Although how much space will depend on how much data is changing and how big your original database is. We do use a snapshot on a mirrored database for occasional reports because our entire database is not replicated.
log shipping http://msdn.microsoft.com/en-us/library/ms187103.aspx
What version of SQL Server are you using?
We're using replication now for a particular solution, and it seems to just work, day in, day out.
I would examine your event log's, and SQL Server logs to see if you can determine why it is shutting down, and why it doesn't start up.
Are you possibly patching the servers, or are you having network errors?
The alternatives to replication are log shipping, or database mirroring.
I personally prefer Database Mirroring, but it really depends what you're trying to do, as some of these aren't appropriate for certain situations.
We also have used SQL transactional replication. We had the same pains with updating schema, which requires dropping the publication on all servers, performing the updates, and then reinitializing replication, and hoping for the best. Sometimes it would not initialize, or a node would fall behind and we'd get little warning for it. A few times we even lost all the stored procedure execute permissions causing pretty much total failure on the websites.
We have a rather large database so reinitialization could take quite some time, meaning all updates had to be done at 2am on Sunday - not exactly when we're awake and alert and able to use all our faculties to deal with a problem that might arise.
We are ditching replication in favor of failover clustering on SQL 2008, but it can still be done all the way back to SQL 2000.
http://technet.microsoft.com/en-us/library/cc917693.aspx
I am researching the possibility of using Firebird for a project.
However, one potential problem is replication and failover, or rather, lack of a (subjective) "good" solution. There are several potential solutions listed in the Firebird FAQ but they are either 1) Windows-centric; 2) horribly outdated; 3) commerical; or 4) not full-featured.
The only potential option I see is FIBRE and that looks 1) immature; 2) potentially dead; and 3) not full-featured.
I've learned about DRBD and Heartbeat and these solutions look promising. I am looking for your feedback should you already have 1) setup a replicated Firebird configuration; and/or 2) used DRBD with Firebird.
Any "gotchas", recommendations, tips, etc.?
Thanks!
There is one session about replication in Firebird Conference 2009
Holger Klemt
* Firebird Replicated Part 1
* Firebird Replicated Part 2
o In this two sessions you will see how easy it is to implement
your own replication system in a
Firebird database. Based on triggers
and simple scripts, your can create a
live backup system. The architecture
allows master-master, master-slave,
multi-master, online and offline
replication. The replicated Firebird
cluster can be used by any client
without interuption, also in the case
of partial hardware failures, planned
hardware and software maintenance
operations, for example the switch to
a new Firebird version.
We have been using DRBD/Heartbeat/Pacemaker Solution for the last 2 years for exactly the same problem. To keep Firebird databases up and running and failover. The setup is actually quite easy and I have a few suggestions that I will give you to get a head start. So these are just suggestions ...
create a drbd partition, format it and mount it to /data (with pacemaker of course)
put your aliases.conf to the drbd partion, so you won't have to change the aliases.conf twice everytime you make a change to it. Copy the aliases.conf file to /data and link it to /etc/firebird/2.1/aliases.conf on both nodes
The downside of using Drbd/Pacemaker in a Primary/Secondary setup is that the clients will loose the connection as soon as the primary node dies and until the secondary node is up. The will have to reconnect. I haven't really found another way arround that, although the firebird client should allow a connection timeout it never really worked with our applications (maybe the applications or the libraries we use don't really use the firebird connection timeout).
As for database replication, I am afraid you have to go the way as Hugues Van Landeghem decribed or quoted it. We developed such an application, that works with triggers. So a new line is added to a table, a trigger copies the key of the entry to another table which is constantly read by an application which grabs that entry and inserts it to another database. Quite ugly but it works just fine! I personally think Firebird should invest some time in having their own datbase replications system...they are really far behind...
Hope my information helped you a little bit. I have further questions feel free to contact me or visit my site # gefoo.org