I've been doing some research for the past hour or so and I've been hearing some conflicting information regarding the replication of Amazon RDS databases. My database is pretty big, 15 tables with a total size of 4 GB. So, basically, is it possible for me to create a local replica of a remote RDS InnoDB or does Amazon not allow it?
you can create replicas of an RDS but only as another RDS. You can't do a replica on an EC2 or a local machine.
Related
In Airflow I know that you can use SQLToS3Operator to copy data from an SQL database to an S3 bucket, but I need it to go the other way; copying data from an S3 bucket into an SQL database. This would specifically be copying keys into a table, one key per table, into a locally hosted MariaDB SQL database just on my computer through Docker. Any ideas?
You can use S3ToMySqlOperator which works with mariadb too.
As the title of this topic is quite self-explanatory, The issue I am facing is that I am using "Ongoing Replication" as a replication type and everything is going well. Except for migrating the MySQL DB users, AWS DMS is unable to do that. Could you please guide us to a most effective solution with minimum downtime, and it would be great if there is no downtime. I have so far explored few options like creating an aurora replica from RDS MySQL DB (https://aws.amazon.com/blogs/aws/new-create-an-amazon-aurora-read-replica-from-a-mysql-db-instance/). Or Exporting the DB users from snapshot stored on the S3 bucket (https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Migrating.ExtMySQL.html#AuroraMySQL.Migrating.ExtMySQL.S3).
Thank you so much in advance.
create an Aurora replica for RDS MySQL is the AWS suggested way for RDS to Aurora migration with minimal downtime: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Migrating.RDSMySQL.Replica.html
You don't need to use AWS DMS at all. The high level steps are:
Create a Aurora RR (MySQL 5.6) from RDS MySQL 5.6
Upgrade the Aurora RR to 5.7
Ensure Replica lag is zero from master to new Aurora 5.7 replica
Promote the new Aurora MySQL 5.7
Hi I'm playing around with HDInsight. I'm putting log files into Azure storage and then using Hive external tables to map onto them. I believe Microsoft recommend Azure storage to HDFS so you can delete and recreate the clusters without losing data. What is the scalability vs HDFS. My understanding of HDFS is that it is spread over multiple nodes to allow parallel processesing how does this compare to Azure storage.
On HDInsight, HDFS storage is based on disks that run in the physical hosts of the VMs (PaaS VMs called worker roles in Windows Azure).
Windows Azure storage has its own scalability mechanisms. The scalability targets are documented here: http://msdn.microsoft.com/en-us/library/windowsazure/dn249410.aspx
To give you an idea, Windows Azure storage is where an OS disk lives for Windows Azure IaaS VMs.
I have a large database in an AWS instance running SQL Server 2008 on a Windows Server 2008 R2.
The database is constantly changing and writing information, and its size is about ~100GB
I wish to migrate from our Amazon services to Microsoft Azure.
But I cannot afford any lost of information more them for more than 20-30 minutes
I don't mind using the Azure SQL or running a SQL Server under a VM in Azure Cloud, but I must keep the databases live and updated, there are few main tables that information is being added to them constantly
What would be the best way to do so ?
if you are using an AWS instance and not RDS and you are going to an Azure instance and not "Azure SQL Database" you can use log-shipping or something similar to get the downtime down to a few seconds: http://msdn.microsoft.com/en-us/library/ms187103.aspx
The steps you need to take:
Take full backup on AWS
restore full backup without recovery on Azure
take log backup on AWS
restore log backup without recovery on Azure
repeat 3 and 4 until the time it takes is short enough (you probably want to script this out)
take app offline
take another log backup on AWS
restore that log backup WITH recovery on Azure
repoint App to Azure
bring App online again.
3, 4 and 5 is what log-shipping would automate, but you could just write a powershell script too.
Does Amazon RDS (sql-server) support sql transaction logs? if so
How can i enable it through RDS interface?
How can i access it?
If you have automated snapshots turned on, then you can see it through the AWS RDS interface under DB Snapshots Automated. You can choose to do automated backups when setting up the DB. http://aws.amazon.com/rds/faqs/ under Automated Backups and Database Snapshots it talks more about this procedure.
What I don't know is how to enable automated backups after creating the db.