We are having significant performance problems on azure. Various factors have made this difficult to examine precisely on azure itself. If the problems are in the performance of the code or of the database I would like to examine them by running locally. However it appears that the default configuration of our database on azure is different than it is locally, e.g. apparently an azure created database defaults to run with different configuration than my local database, e.g. the default on azure includes read committed snapshot as I understand, but that is not the default for a database I create in sql server. That means that performance issues are different for the two.
My question is how can I find all such discrepancies between the setup of the two and correct them so that when I find speed issues locally I will know they represent speed issues on azure. I am a sql server novice. I recognize that I cannot recreate "time to database" and "network time" issues that way, but I don't think those are what are killing us.
You might find my answer to this post useful.
We had great advantages in implementing telemetry to gather information and use it later for analysis, to finally find out where and how you are spending your time interacting with SQL and therefore how to improve the query plans. Here is a link to the CAT blog post: http://blogs.msdn.com/b/windowsazure/archive/2013/06/28/telemetry-basics-and-troubleshooting.aspx
Related
When I want to move data between two databases, what better choice.
A) Linked Servers
database local-> Linked Servers -> database azure
b) ETL - SSIS
database local create procedure xml -> integration service -> serialize xml to object c#> call wcf service async(queue-servicebus) -> persist database azure
The following link addresses the pros and cons of Linked Servers vs. SSIS, with a recommendation that Linked Servers are best applied in moderation for queries.
https://dba.stackexchange.com/questions/5712/whats-the-difference-between-linked-server-solution-and-ssis-solution
It really boils down to how much data your are looking at moving from one database to another and for what purpose. That is, are you dealing with real-time data that must be acquired for an interface? It must be considered on a case-by-case basis. In my development environment, real-time is not required when pulling information from other sources into the database. In this case, SSIS works best and it provides a great log of the package applications throughout the day.
Additional observations:
SSIS is typically faster using BULK INSERTS and has better security benefits.
Linked Servers can create disaster recovery issues and can pose a problem when moving code between environments where one or more servers may not be available.
Lastly, I recommend that you speak with your DBA about applying Linked Servers. The DBA's I've worked with in the past have mostly been apprehensive with the responsibility of maintaining their application. This is one of those "could" vs. "should" issues in development where you must focus on the impact to the system as whole.
When we use Linked Servers, there are also options to use BULK INSERT. In this case, SSIS won't be faster (in many cases it's even slower).
SSIS has some limitations in certain implementations:
- cross domains issues when the domains are not trusted (when we call the packages, SSIS does not work with SQL authentication)
- not easy to automate when the schema changes
- if transformations are required, TSQL is generally faster.
- SSIS with integrated CDC Data Sources works incorrectly and slow in certain scenarios. Confirmed by Microsoft, the issues are not yet fixed (SQL 2014/2016)
As mentioned above, it should be "must be considered on a case-by-case basis". There is no 'YES' or 'NO' here.
We are finding Azure SQL Database very slow. Its probably about 10x slower than the same spend on a SQL on an Azure VM.
However a VM based solution requires maintenance and backups and im concerned I'll loose the VM and data if something horrible goes wrong.
Thus the SQL Azure solution seems safer to me. So I have two specific questions.
Are you seeing this speed difference and if so is there a solution
Is there any nice solution for ensure the SQL on VM is backed up automatically and offsite.
Azure SQL database is slightly slower than SQL Server on Azure Virtual Machine. However, we didn't find it as slow as 10X. May be you should try Premium tier, which delivers more powerful and predictable performance, in case your database is in some other tier.
Regarding SQL Server on Azure Virtual Machine, there is support available for automated backup and patching. Please visit the below link for more details.
http://azure.microsoft.com/blog/2015/01/29/automated-everything-with-sql-server-on-iaas-vms/
While performance analysis, it is worth checking for ASYNC_NETWORK_IO wait type. Make Sure your client application is deployed close to your database in Azure.
I've been a SQL Azure Database user for some time (over a year). I have a mostly readonly 5GB database that fuels my website. Queries hit the database about once or twice a second, and response times are generally sub 100ms.
There have been a few times when performance for all queries goes down the toilet. Today for example, I awoke to alarms that the database was performing poorly. Simple queries that normally take 30ms are taking over 3 minutes! My load on the server is no greater than usual, so I attribute this decline in performance to my DB sharing an instance with one or more DBs from other Azure users.
To solve this problem, I copy the database to a new instance (CREATE DATABASE NEW_DB AS COPY OF OLD_DB), and point the website to the new instance. All is well until this happens the next time. In about a year's time, this has happened 4 or 5 times.
My question: does anyone have some advice on how to mitigate this? If this is just life under Azure, it's pretty unacceptable.
EDIT: just realized that this question is from 2014. If you're still having issues, the questions and suggestions below may guide you in the right direction. If you've resolved the performance issues, feel free to share how any actions you may have taken to improve performance.
What tier are you on right now?
Reference: http://searchsqlserver.techtarget.com/tip/SQL-Azure-database-recommendations-and-best-practices
Are your users coming from different geographical regions? If so, are you using endpoint monitoring for the web app that accesses your SQL Azure db?
Reference: https://azure.microsoft.com/en-us/documentation/articles/web-sites-monitor/#webendpointstatus
Have you tried reading through the official performance guide?
Reference: https://msdn.microsoft.com/en-us/library/azure/dn369873.aspx
Here's a 3rd-party writeup that mentions "the differences in connectivity behavior or that SQL Azure resources get throttled when you overload the database require you to take such things into account and code your application to handle issues you may not have using traditional a SQL Server application."
Reference: http://searchsqlserver.techtarget.com/tip/SQL-Azure-database-recommendations-and-best-practices
This article requires (free) email signup before reading the full article, but it may help you with some recommendations and best practices.
Hope that helps!
Please advise what suits my problem better. I have a highload web app hosted on the same server where SQL server is hosted. I also have SQL Service reporting running on the same server, generating user reports.
So my server basically works on top of disk read/write speed. I'm going to get another server and install there another SQL server in order to host SSRS there. So my criteria is to get as fresh data as it possible.
I've looked a couple of solution, currently I do make backup via jobs, copy it to second server and restore it there, also via jobs. But that's not the best solution.
All replication mechanism(transaction, merge, snapshot) affect publisher database by locking it's table, what is unacceptable for me.
So I wonder is there any possibility to create a replica with read only access, that would be synced periodically not affecting main db? I would put all report load to that replica and make my primary db be used only by web app.
What solution might suit my problem? As I'm not a DBA, I'd start investigating that direction. Thanks.
Transactional Replication is typically used to off-load reporting to another server/instance and can be near real-time in a best case scenario. The benefit of Transactional Replication is you can place different indexes on the subscriber(s) to optimize reporting. You can also choose to replicate only a portion of the data if only a subset is needed for reporting.
The only time locking occurs with Transactional Replication is when you generate a snapshot. With concurrent snapshot processing, which is the default for Transactional Replication, the shared locks are only held for a short period of time, so users are able to continue working uninterrupted. Either way, this shouldn't be an issue since you'll likely be generating the snapshot during a period of low user activity anyway.
I am working on a project which uses a relational database (SQL Server 2008). The local (on-premises) application both reads and writes to the database. I am working on a different front end for Azure (MVC2 Web Role), which will use the same data, but in a read only fashion. If I was deploying a traditional web app, I would use SQL Express to act as the local database, and deploy changes with updates to the application (the data changes very slowly) or via some sync system.
With Azure, the picture is a little cloudy (sorry, I had to). I can't seem to find any information to indicate if SQL Express will work inside of Web Roles, and if so, how to do it. Does anyone know if using SQL Express in an Azure web role is possible?
Other options I could do if forced: SQL CE or use SQL Azure. Both have a number of downsides, and are definitely less than perfect.
Thanks,
Erick
Edit
I think my scenario may not have been clear enough.
This data won't change between deployments, and is only accessed from within the Web Role; it is basically a static cache. The on-premises part is kind of a red herring, as it doesn't impact the data on the web role (aside from being its source). Basically, what I want to do is have a local data store/cache that I use existing T-SQL/DAL code with.
While I could use SQL Azure, it doesn't add anything, and if anything only adds additional overhead and failure points. I could also use a VM Role, but that is way too costly/complex.
In a perfect world, I would package the MDF into the cspkg (so it gets deployed with the app) and then use it locally from within the role. If there is no way to do this, then that is ok and I need to figure out the pros and cons of other solutions. We don't live in a perfect world. :)
You might be able to run SQL Express using a custom VHD but you won't be able to rely on any data every being present on that VHD. The VMs are completely reset when they reboot - there is no physical persistence across reboots.
If you wanted to, you might be able to locate your entire SQL Server installation in Azure blob storage.
However, in doing all of this, you'll only be able to have one worker/web role that can use that database. Remember: a SQL Server database can only be attached to one SQL Server at a time. If you want to scale out, you'll have to create new SQL Server instances for every web/worker role.
Outside of cost concerns, I can't think of anything that is in SQL Express that should be a show stopper for 99.9% of applications out there.
Adding to Jeremiah's answer: SQL Azure should give you nearly everything SQL Express does today, and you can use the Sync service to synchronize on-premise SQL Server with SQL Azure.
If you installed SQL Express into a VM role, you'd be consuming around $90 monthly just for that instance, plus blob storage (you'd want a Cloud Drive for durability). By definition, a VM Role (or any role) must support scale-out; if you were to scale to 2 instances for whatever reason, both instances would need their own copy of the database, so you'd need to create a blob snapshot for each instance.
Keep in mind, though, if you choose to install SQL Express in a VM: once you're at 2 instances, along with, say, 20GB per instance of blob storage, you're nearing $200 monthly and you're maintaining your VM's OS patches, SQL Express configuration and updates, failure recovery procedures, etc. In contrast, SQL Azure at 20GB, while costing the same $200, will offer better performance and works with the sync service, while completely removing any OS or database server management tasks from you.
To add to the already existing answers and for anyone wondering if its a good idea to run SQL Express in the cloud:
it does makes sense as a temporary storage area. Consider this architectural approach:
say you're spinning up nodes to run jobs. Storing a gazillion of calculation results might be a good idea inside a local SQL Express for each node, and provide the aggregated responses immediately when the job finishes on the node. Transfer of the no longer hot results to off-prem SQL server for future reporting/etc can be done afterwords. SQL Azure may not be optimal from the volume/latency/cost perspective to store gazillion of results and ATS will not always fit the bill, especially when relational data, performance or existing code are involved.
To expand on what David mentioned you can register for SQL Azure Data Sync CTP2 that would allow sync from SQL Server to SQL Azure here: http://www.microsoft.com/en-us/SQLAzure/datasync.aspx
Make sure to use CTP2 though since CTP1 did not support SQL Server.
If it's a read only local cache - SQL CE 4 or SQLite.
Both have Entity Framework providers.
If you're writing to it - SQL Azure