Does Horizontal scaling(scale out) option available in AZURE SQL Managed Instance ?
Yes, Azure SQL managed instance support scale out.
You you reference the document #Perter Bons have provided in comment:
Document here:
Scale up/down: Dynamically scale database resources with minimal downtime
Azure SQL Database and SQL Managed Instance enable you to dynamically
add more resources to your database with minimal downtime; however,
there is a switch over period where connectivity is lost to the
database for a short amount of time, which can be mitigated using
retry logic.
Scale out: Use read-only replicas to offload read-only query workloads
As part of High Availability architecture, each single database,
elastic pool database, and managed instance in the Premium and
Business Critical service tier is automatically provisioned with a
primary read-write replica and several secondary read-only replicas.
The secondary replicas are provisioned with the same compute size as
the primary replica. The read scale-out feature allows you to offload
read-only workloads using the compute capacity of one of the
read-only replicas, instead of running them on the read-write
replica.
HTH.
Yes scale out option is available in Business Critical(BC) tier. The BC utilizes three nodes. One is primary and two are secondary. They use Always on on the backend. If you need to utilize for reporting, just ApplicationIntent=Readonly in the connection string and your application will be routed one of the secondary nodes.
Related
Backdrop
I develop a forecasting engine (time series) for different purposes. Processing, modeling and forecasting modules are written in Python, and data is currently stored in an Azure SQL database. Currently the database is General Purpose (vCore-based) service tier, Provisioned compute tier and Gen5 (12 vCores) hw config. I'm approaching the limit of maximum storage (approx 3 TB), but since I read almost the entire database daily (cold start models only), I do not see many other options than increasing the storage size. Truncating parts of historical data is out of the question.
Problem
At 12 vCores, maximum storage is approx 3 TB, and increasing vCores to enable the approx 4 TB maximum storage size is not feasible in a $-perspective (especially since it is storage, not compute, that is the bottleneck). I have read a bit about the alternative services / tiers on the Azure platform, and see that Hyperscale could possibly solve my problem: I can keep vCores untouched and have up to 100 TB storage. A config with zero secondary replicas (other things equal) will end up in the same $-range as before (see "Backdrop"). I get the impression that secondary replicas (read only nodes) are central to the Hyperscale architecture, so I'm not sure if such an outlined setup with zero secondary replicas is abuse / misuse. E.g. would it give the same performance, or could I expect a performance hit (even with the same vCore config)? Will the primary read / write node basically resemble a non-Hyperscale node? Other aspects I should think about? Adding a secondary replica (or several) might be relevant in the future (e.g. in combination with decreasing vCores), but is $-wise not an option atm.
Microsoft states that "The capability to change from Hyperscale to another service tier is not supported" (really?), so I would like clarify this to avoid doing a semi manual data migration (and delta migration) and having two instances side-by-side if the shait hits the fan. Given the scope of such a reconfig and the forecasting system as a whole, I feel it is not feasible to do small / full scale testing in advance to get representtive benchmarks. If there is anything else I should think about (related or semi related), feel free to point me in the right direction.
Since Hyperscale service tier is the newly added service tier in Azure SQL Database, it would be difficult to get the solid answer for your question.
It's true that it provides upto 100 TB database size, but the beauty is it will only charge for the capacity you use.
The Hyperscale service tier removes many of the practical limits
traditionally seen in cloud databases. Where most other databases are
limited by the resources available in a single node, databases in the
Hyperscale service tier have no such limits. With its flexible storage
architecture, storage grows as needed. In fact, Hyperscale databases
aren't created with a defined max size. A Hyperscale database grows as
needed - and you're billed only for the capacity you use. For
read-intensive workloads, the Hyperscale service tier provides rapid
scale-out by provisioning additional replicas as needed for offloading
read workloads.
You can have primary and secondary replica in Hyperscale service tier.
Primary replica serves read and write operations
Secondary replica provides read scale-out, high availability, and geo-replication
Secondary replicas are always read-only, and can be of three different types:
High Availability replica (recommended)
Named replica (in Preview, no guaranteed SLA)
Geo-replica (in Preview, no guaranteed SLA)
You should consider Hyperscale service tier because:
you need more size than 4 TB
require fast vertical and horizontal compute scaling, high performance, instant backup, and fast database restore
Note: Users may adjust the total number of high-availability replicas from 0-4, depending on the need.
You can check the Hyperscale pricing model here.
Considering above points, Hyperscale is the good, if not the best, solution for your requirement.
These two links will definitely help you to take your decision. Hyperscale service tier, Hyperscale secondary replicas
I'm one of the PM on the Azure SQL DB team. I see that UtkarshPal-MT already gave you extensive answer, so I'm chimining in on to complete the picture. Azure SQL DB Hyperscale offers different type of secondary replicas. The replicas that can help to get an higher SLA are named High-Availability replica. You can use 0 replicas without any issue. What will happen is that if the primary replicas for any reason is not available, we need to spin up a new (compute) replica from scratch (as there are no HA replica available) so that can take some time (minutes, usually) which means that your service will not be available that amount time. Having an HA replica, drastically diminish the time in which the database is not available.
You can read all the details here:
https://learn.microsoft.com/en-us/azure/azure-sql/database/service-tier-hyperscale-replicas?tabs=tsql
The SLA are defined here:
https://www.azure.cn/en-us/support/sla/sql-data/
Regarding the performances: unless you are specifically using secondary replicas also to offload read-only workload, you'll not have performance hit by not having an HA replica
What I'd like to achive is to be able to scale out Azure SQL Database.
Business Critical tier has this feature to enable several read-only replicas. This is a great feature that would let me offload some traffic over to those replicas
The problem for me is that I don't understand how to manage those replicas and I don't understand how load balancing works there. Basically, I should be able to manage how many replicas there are, I probably need to have around 10 of replicas and have traffic equality balanced across them
Is this something that I could do?
If you look at the note here, it says
In Premium and Business Critical service tiers, only one of the read-only replicas is accessible at any given time. Hyperscale supports multiple read-only replicas.
This means Premium and Business critical service tiers may have multiple replicas (3-4) but only 1 of them is accessible as read only. There is no control as to which one and there is no load balancing capabilities. It is only good for use if there is a separate application which require read access only (example analytical workloads).
For Hyperscale you can refer to this.
Hyperscale allows for 1-4 secondaries(1 by default). The link states
If more than one secondary replica is present, the workload is distributed across all available secondaries.
There is no additional information and it seems the the control to load balance is abstracted away from us.
You can definitely not achieve your requirement of 10 read replicas from any of these configurations.
Can i use azure cosmos db instead of redis cache for server side caching , i feel that cosmos Db also provides key value storage, has geo replication , read write access and lower latency than redis cache
If you're still reading this 2 years later note the following. The answer is yes but the real story is that they work better together. Azure Cache for Redis now has an Enterprise Tier through the same Marketplace tile. This gives you the ability to deploy Redis in an Active-Active model across multiple regions where all instances are readable and writeable with conflict resolution built into the different datatypes that Redis supports. Couple that with higher performance through the redis enterprise proxy and up to 5 9's of availability gives you additional options to choose from. Azure Cache for Redis Enterprise (ACRE) in front of Cosmos is a real option as ACRE has sub-millisecond latency capabilities. Note, I work for Redis Labs and have seen this work and deployed it myself.
Redis is an in-memory datastore hence it's primary use-case is in-memory caching. Since it is a Key-value store, it has generally limited query ability, only allowing queries by primary key.
While, CosmosDB is Globally distributed, horizontally scalable, multi-model database service. It becomes handy in scenarios where you need the ability to query over heterogeneous data.
Those two are totally for different purposes, even Microsoft has redis cache as a service apart from CosmosDB only to serve this purpose.
Cosmos is probably going to be more expensive, from a cost perspective, than using Redis - depending on your throughput.
The one big benefit you can achieve with Cosmos is multi-read regions so your availability could increase and also the latency to your users if they're reading from a Cosmos region closer to them.
I have started using ApplicationIntent=ReadOnly; for Premium Databases which works great.
However if I use the Fail-over group connection then the connection is always READ_WRITE. Can you not use ApplicationIntent=ReadOnly with the main FOG connection string:
Server=tcp:xxxx-fog.database.windows.net,1433;Persist Security Info=False;Connection Timeout=300;TrustServerCertificate=True;Database=xxx;ApplicationIntent=ReadOnly;
I can use the geo-secondary fine for this - but we already have a lot of load on the geo-secondary, so I wanted to use some of the primary replicas for reading too.
ApplicationIntent allows routing to a local HA replica of Premium or Business Critical databases. When using failover groups, to load balance read-only traffic you should use Server=tcp:.secondary.database.windows.net. This will work for any database in the FOG (premium or not) and will ensure that the queries do not fail on failover. We recommend also using ApplicationIntent as well. In case you upgrade the database to Premium or Business Critical, this will allow the read-only clients to take advantage of the local HA replica without changing the connection string.
Just to update - Azure have confirmed this is a bug and they are fixing it in the next few weeks. Where if we turn on Read Scale-Out feature on a database after we add it to a failover group, the connections through failover group endpoint are not honoring read scale out redirections.
Current workaround before the issue being fixed, is to turn on read scale feature on your databases before adding them to a failover group.
We are considering to move our application and SQL servers to AWS EC2 instances. Currently, we have one SQL Server Standard and we have failover clustering for high availability.
On AWS, We are planning to have EC2 instance(s) with EBS volumes as needed for SQL server. So, to sql server implement high availability only option available if Synchronous DB Mirroring unless we go for some storage level replication and implement cluster on top of it.
This application does not need cross availability zone (multi site) high availability.
My basic question is why we need to take care of SQL High availability? Asking this question as
Keeping EC2 up is the amazon's responsibility
EBS volumes gets automatically replicated internally in availability zone.
Only think I can think of that we need separate high availability is EC2 instance becomes irresponsive or something on OS / Driver side gets corrupted. Do you see any other reason then these ones?
Thanks for taking time to read this question.
If you'd like a managed, highly available SQL Server, Amazon RDS would be a better option, as you would have HA (Multi-AZ), Backup, Patching and other administrative tasks automatically. Using RDS, you do not have to manage the underlying EC2 instances and EBS volumes yourself.
Using Amazon EC2, you need to take care of application HA (SQL Server in this case) because this is not (and can not be) managed by Amazon. For example, if you are hosting your personal blog you could use one instance in a single Availability Zone, for a database with primary-secondary (HA) replication you may use two instances in separate AZs and for a multi-master distributed database, you may use several instances in several regions. The design and cost of High Availability is very context dependent.
You can try and learn RDS for free with this self-paced lab: https://run.qwiklab.com/focuses/preview/1013?locale=en