I have a physical Prod DB Server (SQL05) and now a VM DB Server. The idea is if the physical machine goes down, we repoint our router (via NAT) to the VM machine. I am thinking of using Log Shipping to keep the VM DB basically current.
Is this the correct way to do it?
Should I be looking at another way, mirroring perhaps?
We would like the VM DB to be in an usable state at all times (so I think this precludes mirroring)
Any (good) suggestions requested! :)
Why wouldn't you use mirroring?
Related
I am trying to update a legacy system's sql solution to use the cloud.
The solution today involves a customer Windows SQL server installed onsite, then various machines are configured to connect to that IP Address / Port / Server Name. When they do connect the machines will set up any tables that are missing and regularly send their data. Data rates are low for an individual machine. Roughly one write request ever 10 seconds (it varies a lot), no more than 2-3k of information on each write request.
Moving this to the cloud is tricky mostly because each of the machines do not have a unique identifier. The good news is that we have the legacy machines connected to a IOT Gateway (Just think RPI) that knows a unique machineId. Furthermore the IOTG is a full fledge computer but not too powerful of one, and its Disk is an SD card.
New and Old Network Layout
So far I have had a few things fall on their face.
1) Setting the Machine to think the DB's IP/Port is that of the IOT Gateway. Setting up an Express server on the IOTG, listening, then injecting the unique id into the queries that I'd proxy up to the cloud. I may have had a bug, but for some reason I couldn't even see the requests coming in on the port. Even if I could I'd still have to figure out how to decode them. Shouldn't I at least be able to see these requests coming in?
2) Started looking into SQLite. The idea being to have SQLite listen on the port as an actual DB then have a process in the IOTG query data out of SQLite, append a unique ID, and then send it to the cloud. Unfortunately SQLite does not listen on a port.
I am starting to looking at just installing a whole SQL server on the device, but I'd really like to avoid that. I'm pretty sure its fairly large and writing to disk is not advisable for a small embedded system like I'm running.
Generally my questions boil down to:
1) Should I be able to see SQL Queries in an express server?
2) Should I be using a different tech? I failed to find a different more sql specific proxy.
3) Am I correct to think that the SQLite path is dead? Even if I could find a way to attach it to a port there is still not going to be any sort of response from SQLite when the clients try to make a connection.
4) Am I wrong to fear the local server? Diving into some documentation for making express work with DBs gets me to here: https://www.microsoft.com/en-us/sql-server/developer-get-started/node/ubuntu/ which suggests 4GB of memory, we're working on 0.5GB.
Any other thoughts on how to approach this would be great.
Are there any differences i.e. advantages/disadvantages to saving the machine state of a VirtualBox appliance while it is paused?
I used to save the machine state of appliances without them being paused (Close -> Save the Machine State), but have just found out that if I do this after pausing the appliance that it seems to have the same effect. In fact, when I start the appliance again, it is automatically unpaused, which is interesting.
Does anyone know if pausing an appliance affects saving the machine state in any way?
Thanks in advance for your reply.
actually that's some of the perks of working with Virtual Machines, not to mention the security and consistence you get if you work with them. Saving the state machine and other ways does not affect the performance of them, you even can do things like:
You can move the copy to another installation and resume your
execution.
Do snapshots of the Virtual Machines. Which is something like a "photo" of that moment of the Virtual Machine.
Cloning it to run another copy on the host.
Live migration and this does not affect the processes of the user.
If you want to learn more, I recommend you this book: http://iips.icci.edu.iq/images/exam/Abraham-Silberschatz-Operating-System-Concepts---9th2012.12.pdf
I am using redis version 2.8.3. I want to build a redis cluster. But in this cluster there should be multiple master. This means I need multiple nodes that has write access and applying ability to all other nodes.
I could build a cluster with a master and multiple slaves. I just configured slaves redis.conf files and added that ;
slaveof myMasterIp myMasterPort
Thats all. Than I try to write something into db via master. It is replicated to all slaves and I really like it.
But when I try to write via a slave, it told me that slaves have no right to write. After that I just set read-only status of slave in redis.conf file to false. Hence, I could write something into db.
But I realize that, it is not replicated to my master replication so it is not replicated to all other slave neigther.
This means I could'not build an active-active cluster.
I tried to find something whether redis has active-active cluster capability. But I could not find exact answer about it.
Is it available to build active-active cluster with redis?
If it is, How can I do it ?
Thank you!
Redis v2.8.3 does not support multi-master setups. The real question, however, is why do you want to set one up? Put differently, what challenge/problem are you trying to solve?
It looks like the challenge you're trying to solve is how to reduce the network load (more on that below) by eliminating over-the-net reads. Since Redis isn't multi-master (yet), the only way to do it is by setting up each app server with a master and a slave (to the other master) - i.e. grand total of 4 Redis instances (and twice the RAM).
The simple scenario is when each app updates only a mutually-exclusive subset of the database's keys. In that scenario this kind of setup may actually be beneficial (at least in the short term). If, however, both apps can touch all keys or if even just one key is "shared" for writes between the apps, then you'll need to bake locking/conflict resolution/etc... logic into your apps to consolidate local master and slave differences (and that may be a bit of an overkill). In either case, however, you'll end up with too many (i.e. more than 1) Redises, which means more admin effort at the very least.
Also note that by colocating app and database on the same server you're setting yourself for near-certain scalability failure. What will happen when you need more compute resources for your apps or Redis? How will you add yet another app server to the mix?
Which brings me back to the actual problem you are trying to solve - network load. Why exactly is that an issue? Are your apps so throughput-heavy or is the network so thin that you are willing to go to such lengths? Or maybe latency is the issue that you want to resolve? Be the case as it may be, I recommended that you consider a time-proven design instead, namely separating Redis from the apps and putting it on its own resources. True, network will hit you in the face and you'll have to work around/with it (which is what everybody else does). On the other hand, you'll have more flexibility and control over your much simpler setup and that, in my book, is a huge gain.
Redis Enterprise has had this feature for quite a while, but if you are looking for an open source solution KeyDB is a fork with Active Active support (called Active Replica).
Setting it up is just a little more work than standard replication:
Both servers must have "active-replica yes" in their respective configuration files
On server B execute the command "replicaof [A address] [A port]"
Server B will drop its database and load server A's dataset
On server A execute the command "replicaof [B address] [B port]"
Server A will drop its database and load server B's dataset (including the data it just transferred in the prior step)
Both servers will now propagate writes to each other. You can test this by writing to a key on Server A and ensuring it is visible on B and vice versa.
https://github.com/JohnSully/KeyDB/wiki/KeyDB-(Redis-Fork):-Active-Replica-Support
I'm building a whiteboard web app with self-contained "rooms" of clients that runs off Amazon EC2 instances (a single one for now). Commands are sent via websockets to a PHP server, which stores all commands in a SQL database.
Up until now I was using Google Cloud SQL. My plan was to learn how to scale with EC2 and have all instances use the same remote database. I've learned this won't work due to the 200 ms write latency of a remote SQL server vs. the 0.5 ms write latency of a local SQL server. The server makes a write every time a command arrives.
I'm new to scalability and distributed systems. My intuition tells me I either need to use Amazon RDS and hope for millisecond latencies if my EC2 and RDS instances are in the same region, or work with SQL locally on EC2 instances. I'm leaning toward the latter. Here's my issue: EC2 is elastic. What happens when I need to get rid of an instance?
All I can think of right now is somehow replicating the SQL data from each EC2 instance to a master instance (maybe even Google Cloud SQL!). In other words, all reads/writes for each "room" happen locally, and are eventually replicated to the master server for long-term storage. If a "room" is re-opened a week later, a different EC2 instance can grab data from the master server, work with it locally, and replicate changes back before being destroyed.
Does my approach sound correct--is replication the right concept here? If so, how much support for what I'm trying to do already exists? That is, do I need to set up a master server that manages EC2 instances and distributes/collects the SQL data manually (100% custom implementation), or is there are there existing libraries/mechanisms for SQL and maybe even EC2 instance replication/management? And if my approach is wrong, what are some better approaches? This is one of those times where I don't know what to research on my own. Thanks!
I'd agree with user02525 perhaps look at using Elasticache redis, it sounds more in line with what you're doing.
I am trying to figure out something and I've been searching for a while with no results.
What happens if a Redis server loses power or gets shut down or something that would wipe the RAM? Does it keep a backup somewhere?
I am wanting to use Redis for a SaaS style app so if I go to app.com/usernamesapp it would use redis to verify usernamesapp exists and get the ID... At which point it would use MySQL for all the rest of the stuff... Reasons being I want to begin showing the page ASAP and most of the stuff is javascript so all the MySQL would happen after the fact.
Thanks
Redis can be configured to write to disk at regular intervals so if the server fails you wont lose your data.
http://redis.io/topics/persistence
From the Redis FAQ
Redis is an in-memory but persistent on disk database
So a critical failure should not result in data loss. Read more at http://redis.io/topics/faq