I have a vb.net application connected with an sql server. This applications handles files.
Recently, this application connected with an sql server, which is in a VM of Azure.
My question is, how i can hanndle the files?
I want my application to upload(over internet) the files somewhere and then server side to haddle where these files will be saved. And the opposite.
Can you tell me what options i have? I don't want OneDrive.
Depending on the kinds of files you store and the way your application handles them, you have multiple options with Azure. These are Azure Blob Storage ( with blob types: Block, Append, and Page), Azure Files, or Azure Data Lake Store.
Azure Blob Storage:
The following blob types are great of your data is unstructured.
Block Blobs: for use of binary data or text. You store in blocks that can be manged.
Page Blobs: to store random access files, good for storing VHDs that are backing up VMs.
Append Blobs: similar to block blobs but are append-only and optimized for append-only workloads. Good for storing log files storage.
If you handle files using native file systems APIs and want to "lift and shift" your application as is, Azure Files might be your best option which uses the SMB protocol.
Another option you might want to try, which is in preview (not generally available yet ) is Azure Data Lake Store Gen 2 which allows you to interact with Azure Blob storage through a file system interface.
From the way you describe your application, I doubt you want to use Azure Disks service. Here is a comparison table to help you decide: https://learn.microsoft.com/en-us/azure/storage/common/storage-decide-blobs-files-disks?toc=%2fazure%2fstorage%2fblobs%2ftoc.json
Related
I am currently in the process of building a SQL database in Microsoft Azure for handling pictures, documents, etc. What is the most efficient/best way of storing data? Uploading the files directly to the DB, or by sourcing the files from something like Azure BLOB? I have read numerous posts about people uploading it directly to the DB, but I am concerned about its efficiency.
Thank you in advance for any replies.
You can store in something like Azure SQL DB for example but I would not recommend it, you should definitely store in Azure Storage (BLOB) and then for reference store in a DB. Azure has multiple relational and NoSQL data stores which are offered as platform services.
I would do two things, use a NoSQL platform data store like Cosmos DB using SQL Core API to store the metadata for the images, here you can use the filename as the partition ID to do a point read (this is very fast read and it would be a very cheap option with blazing fast performance) and secondly I would use Azure CDN to make sure images are accessed via CDN so that they are faster.
Azure CDN has three options; Akamai, Verizon and Microsoft. You can test which CDN is faster from where you are from here: https://cloudharmony.com/speedtest-for-azure
Using the above URL you can also use to test which Azure region is closer to you so to use that region, or test for your end-users and choose the region closer tot them.
I would say storing in Azure BLOBs is a better idea. Imagine you have 100 GB files stored in DB.
It will slow down your query if your table is not designed properly.
Backup & Restore DB will be very slow.
Azure DB is more expensive than Azure BLOB for the same size.
If your total file size is small enough, it doesn't make much difference.
We're storing about 4 million files (4 TB or so) of miscellaneous files, mainly Word and PDF, in Azure BLOB storage. I'm looking to replicate this data in a different cloud for disaster recovery and peace of mind, and Amazon S3 seems as good a candidate as any.
Trouble is, I don't have a local server large enough to hold a local copy of these files. Ideally, I'd want to sync right from Azure Blob to S3. We're adding new files continually, so the sync would need to be frequent as well (multiple times per day).
I see lots of options for download from Azure to local => upload from local to S3, but very little for direct Azure => S3 sync. What are some good options here?
We can migrate the azure storage data to amazon s3 by node.js package.
You can see the full description provided here.
You can also use azure data factory to replicate as it provides a copy tool which can be modified according to your needs and settings for transferring data .
You can refer to this document on Azure data factory and copy tool.
I have a vb.net based application which references an Azure SQL Database, I have set up a storage account to which I would like to store files to from the application. I am not sure how to create that link between the DB and the Storage account?
Going through the "SQL Server Data Files in Windows Azure Storage service" Tutorial I cannot create a URI for the sotrage blob. Using Azure Storage Explorer I select my container go into security and generate a signature which all works fine. When I test the URI with the "Test in Browser" button I get this error:
<Error>
<Code>AuthenticationFailed</Code>
<Message>
Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:22ab2830-0001-001d-67a0-5460bb000000 Time:2014-10-17T14:06:11.9864269Z
</Message>
<AuthenticationErrorDetail>
Signature did not match. String to sign used was r 2014-10-17T06:00:00Z 2014-10-25T06:00:00Z /macrocitrus/$root 2014-02-14
</AuthenticationErrorDetail>
</Error>
to what this means I have no idea. I am a completely new user with Windows Azure so I am not even sure that I am on the right track?
Is there any documentation that actually explains the steps or what one would require to allow storage access from an SQL DB to an Azure Storage account?
I would not recommend saving the binary content in SQL Database. Instead I would recommend that you save them in blob storage. Here are my reasons for doing so:
Blob storage is designed for that purpose.
Storing data in blob storage is much-much cheaper than storing the data in SQL Database.
By storing binary data with other data, you're unnecessarily making your data access layer bulkier as all the data will be streamed through your database.
General approach in these kinds of scenarios is to keep binary data in blob storage as blobs (think of blobs as files in the cloud). Since each blob gets a unique URL, you can just store the URL in your SQL Database table. So if we go with this approach, what you will be doing is first uploading the blob in blob storage, get its URL and then update the database.
If you search for uploading files in blob storage, I am sure you will find a lot of examples with source code (so I will not bother providing it here :); I hope its all right).
Now coming to the error you're getting. Basically the link you created using Azure Storage Explorer is known as Shared Access Signature (SAS) URL which basically grants a time-limited/permission bound access to your storage account. Now Azure Storage Explorer gave you a SAS URL for the container. There are two ways you can use that URL (assuming you granted Read & List permissions when creating the SAS URL:
To list blobs in that container, just append restype=container&comp=list to your URL and then paste it in the browser and you will see an XML listing of all blobs.
To download a blob, you would need to insert the name of the blob in the URL. So if your URL is like https://[youraccount].blob.core.windows.net/[yourcontainer]?[somestuffhere] and your blob name is myawesomepicture.png, your SAS URL for viewing the file in the browser would be https://[youraccount].blob.core.windows.net/[yourcontainer]/myawesomepicture.png?[somestuffhere]
I wrote a blog post on using Shared Access Signatures which you may find useful: http://gauravmantri.com/2013/02/13/revisiting-windows-azure-shared-access-signature/.
I've seen examples of web apps that has user accounts and stores information about the users, but does not use a database. I've been searching for a while and cannot figure out how this would be done without a database. Can someone point me in a direction?
Clearly the data has to be stored somewhere. It could stored on the client, using cookies or HTML5 Web Storage, but this would make it nearly impossible for the data to be shared across devices. (Technically, the data could be shared between devices provided that another device maintains an active connection to the web server at the same time; the web server would merely transfer the data between devices without storing it. However, this would be extremely impractical.) If the user data is stored in any centralized location, there would effectively be a database, even if specialized database software, such as MySQL, is not used, i.e. just using the filesystem of the server directly.
Some used database such as SQLite to store data, but for other application they simply saved it to a file somewhere in the directory (may be cookies, temporary file, settings/configuration, ....etc).
There are 2 main ways of storing data based on the requirement:
Storing data On client
1.HTML5 Webstorage is now available, which store data locally and it can support data upto 5MB and it is more secured and faster.
2.Cookies
Storing data On Server
Through flat files that is text file or through XML files.XML is standard in many companies as a way to store data. This is really quick.
Having said that, there are downsides to it as well.
I'm extremely confused, so I've created an SQL Database in Windows Azure, created a "video table" with a "video_file" column as "varbinary(max)" because I want to upload a video file into that field, however Azure offers no "Upload" option like say, PHPMyAdmin does where you can hit "browse" and upload a video directly into the field. Can anyone guide me as to how to actually upload a file into a Windows Azure SQL Database so it can be read as a varbinary type? Can it be done within the Azure management portal? Or does it require some sort of external program/service?
To answer your question, the functionality to upload files directly into SQL Azure Database does not exist. This is something you have to do on your own.
Can anyone guide me as to how to actually upload a file into a Windows
Azure SQL Database so it can be read as a varbinary type?
Do a search for uploading files in SQL Server and you will find plenty of examples on how to do that. Take a look at this link for example: http://www.codeproject.com/Articles/225446/Uploading-and-downloading-files-to-from-a-SQL-Serv
Can it be done within the Azure management portal? Or does it require some sort of external program/service?
No. This functionality does not exist in Azure Management Portal. As mentioned above, you would need to write some code to do so.
A little bit off-topic comment:
May I suggest that instead of saving the image files in the database you save them in Blob Storage and store the URL of the blob in your table. There're some advantages I could see in this:
Compared to SQL Database, Azure Blob Storage is much cheaper. If you store video files (or in other words large files) in the database, you will end up with large database and thus end up paying more money.
You will be choking the database when reading this large data from the database which will impact the performance of your application.