I'm building a website that will surface data from a third party system. The third party will provide a copy of all the data I need as a SQL restore file (*.bak) inside a rar file on their sftp server. The data changes every day, so my application will need to connect to the sftp site, get the file, unzip it, then restore it into my database server every night. I'm fairly comfortable scripting this in a standard windows environment, but the customer would prefer the application to be built on the MS Azure cloud, which doesn't seem to support a common solution to the problem. It's possible we could abandon Azure, but I'd like to know what the best strategy would be for implementing in Azure if it's possible.
This depends on whether you are trying to use Azure PaaS (cloud service and SQL Azure) or IaaS (VMs). If you are using VMs on Windows Azure, there is going to be no difference between Windows Azure and your familiar Windows environment - so yes, you can do this on Windows Azure.
This can't really be done in Azure cloud services and SQL Azure (SQL Azure cannot restore a .bak file). But your application doesn't seem to be the kind that would run as a cloud service anyway.
Stick to doing it on VMs and it will work as you are familiar with.
Related
Requirement: I wanted to copy data from a specific table/view residing on a on-premise SQL Server to Azure SQL DB.
Infrastructure: As depicted in below picture. Essentially, the Azure network is directly connected with corporate network over Express Route. Thus it's a pure private network connection; as good as the corporate network itself.
Issue/Question: I know there are multiple approaches present to get this operation done and I am not restricted to use ADF copy Data tool only. BUT, for all of these I see some cavets or extra steps needed to be done as below:
ADF Copy Data Tool: Needs a SH-IR and a small MSI package needs to be installed on on-premise machine which hosts the SQL server for registration purpose.
Logic Apps: Needs a Virtual Gateway (OR) ASE
App Service: If the operation is wrapped in a C# application and I choose to deploy to a Azure Web Apps. Then in-order to connect to on-premise SQL Server we need to setup hybrid connection manager and as in #1 we need to install something in on-premise machine.
For my case, none of these extra steps can be done. essentially, the on-premise SQL Server comes under a different BU and thus I don't have any permission there; except they have given grant to a table/view. Thus, none of these extra shitty steps can be done.
Moreover, as mentioned above; since it's connected over express route as direct connection, As can be seen in above picture, both the on-premise and azure SQL are essentially inside the same corporate network. THUS, I should be able to access them directly without configuring any of these extra steps as mentioned above.
Please confirm on these and provide a suggestion.
Thank You.
You can still go with the ADF scenario without a SHIR by creating ADF in a Managed VNET using Private Endpoint. As you already have an ER circuit and have the flexibility to configure the Azure side, can you do this with Azure IR: Access on-premises SQL Server from Data Factory Managed VNet using Private Endpoint - Azure Data Factory | Microsoft Docs
There are 2 solutions which could work for your scenario but even for them to work ,you would need access to on prem SQL server machine access to some extent atleast for one time config and Azure SQL db should be accessible via SSMS installed on on-prem machine.
Using linked server
You can create a linked server ( process explained here https://www.sqlshack.com/create-linked-server-azure-sql-database/ ) on on-prem server and create a agent server job to insert data to azure SQL db table.
Via Python Script
This would need Python installation on on-prem machine. Once installed you can write script to transfer data between on-prem SQL server and Azure SQL db. You can schedule this script again by using an agent server job.
We are supporting a legacy system for our organisation. In the current scenario, we receive a SQL Server backup (.bak files) from the application vendor on an FTP location. For every weekend on Sunday it is a Full backup and for every other day its the differential one.
On our side, we have a SQL server instance running which has custom stored procedures written and scheduled to check the location every morning and then restore the backups every day. These restored backups are then used by the organisation for internal reporting purposes. There are 100s of other stored procedures written for different reports in different DBs on the same instance.
Since SQL Server 2008 is now out of support and for cost-saving purposes of running on-premise system, my team has been given a task to look into migrating this whole system to Azure SQL database.
My question is what is the most effective way in which we can move this workflow to the cloud? I have an azure trial account set up for me to try but haven't been successful in restoring the .bak files on Azure SQL instance.
Thanks.
You essentially have two options for Azure, either perform a fairly linear Lift and Shift to SQL Server on an Azure VM or go with a more advanced Azure PaaS offering in Azure SQL Database Managed Instance. The specific deployment Azure SQL Database (Single Instance) will not support your current solution requires with regard to the .bak file support, and I have detailed that below. For further details between the difference between Azure SQL Database Single Instance versus Managed Instance, please see: Features comparison: Azure SQL Database and Azure SQL Managed Instance
The second option, is to leverage the Azure Enterprise Ready Analytics Architecture (AERAA) (link) of Azure (PaaS) Analytics services. With Azure SQL Database (PaaS) services, as opposed to on-premise SQL Server or SQL Server on an Azure VM, there is no Integration Runtime or Analysis Services as a bundled service component. These services are separate PaaS offerings and with the help of the linked AERAA blog, you can gain a better understanding of the Azure Analytics services.
The .bak versus .bacpac support dilemma:
Since the main requirement for your solution is support of .bak files, you need to understand where .bak and where .bacpac files are supported. The term Azure SQL Database applies to both a specific deployment option for an Azure SQL database (PaaS) service and as a general term for Azure SQL cloud databases. As for the specific deployment option, Azure SQL Database (Single Instance nor Elastic Pools) will support your scenario with .bak files. This deployment option will support export/import functionality via .bacpac file format. It will not support full/partial restore functionality. The backup/restore functionality although configurable, is only in scope for the specific database hosted by an Azure SQL (logical) Server instance. Basically, you can not restore an external file. You can import, which is always a full copy. So, for that reason, for an Azure PaaS database service you will need Azure SQL Database Managed Instance for .bak file support or deploy an SQL Server VM image to an Azure VM, and migrate your objects via Azure Database Migration Service.
Regards,
Mike
Our new system handle HIPPA data and has security equirements.
Azure handles secure connections, but we're looking at TDE as well.
One of our consultants said that TDE is possible by creating a virtual server in Azure and loading SQL Server 2012 Enterprise edition directly on to the server. This would be outside of the Azure provisioning. Would this work?
I know that a recent release that's currently in preview that would accomodate TDE. Does any one know were I can get the preview version?
Thank you for your help
From your description, I'm assuming you're referring to running SQL Server in an Azure VM. If that's correct, TDE has nothing to do with Azure provisioning.
You can pick any of the SQL Server VM templates available that supports TDE. After Azure completes provision of the VM, you can login to your SQL Server and enable TDE. You can certainly still upload your own VM image but unless you have some corporate standards, pre-installed software, etc... a regular provision is probably the easiest and fastest.
I had a engineer design our .net application back in 2009, my guess is that it was coded using visual studio, and all I have is the installer application. We have been using it on our 1 or 2 local client machines very well for the past few years, but now I want to move this front end to the cloud. Instead of installing it as an application on our windows 7 machines.
It is a very simple application used in our small warehouse that keeps track of cargo/shipments etc. It uses Sql Server 2008 Express as a backend which is stored locally.
I know how to get the database in the cloud, their are many options for that, using Amazon or Azure, but how do i get the local client application to the cloud?
I dont have access to the visual studio code, i just have the runtime executable file..
I am sure there is no way to do this, and many of SO users will say i need to re-write the front end.
I have tried to contact the developer and they hav since closed down. Is their anyway i can run this in the cloud?
I welcome all options and solutions!
Thanks.
I believe you have two options for hosting this application:
If you are able to configure the database connection string, you could host the database in the cloud, and distribute the application to your end users. However, you've already stated that you know how to move the database, so I assume this isn't an option.
The only alternative is to run the entire application on a cloud server, and send the user interface to a client using terminal services. This makes it appear as if the application is running locally on the user's computer, while it is actually running on the server.
For an off-the-shelf solution to achieve this, you could consider using Microsoft's RemoteApp Azure service. I'm sure there are other similar offerings available.
In our organization we are using Hyper-V VMs. We are using Progress Database and apps in the workstations.
For us to migrate into the Microsoft Azure cloud, do we have to migrate our existing Progress database to SQL and rewrite our apps ?
No. You haven't given us much detail about your applications or architecture, but if I make the assumption that you are using the embedded database product by Progress software, then I see no reason that can't run on an Azure VM.