how big (binary(xy)) should I make my table column in SQL database if I want to store there pictures taken by camera - that means variable size up to.. I don't know.. 7MB? But if I should rather limit the size up to 2MB or something, I would. Whats your opinion?
EDIT
Or where else should I store them? I am building a web gallery using asp.net mvc.
What you're talking about is a varbinary column. Of course, if you make varbinary greater than 8000, it immediately converts it into a varbinary(max) column, meaning it can store up to 2GB. This has to do with how SQL Server stores rows (8k per page).
Therefore, each row stores the column as a pointer to the bits, anyway. So, what I would do, if I were me, would be to store the images on the file system, and store the location of those files inside the database.
If you want to store images in SQL Server then use the varbinary(max) column type. It permits up to 2Gb (if I recall).
Also, as you are using SQL Server 2008 (I don't know about the express edition tho') you could use the new filestream data type.
Of course the big advantage of storing this in the database is that you only have one thing to back up and you don't have issues with the file system and database getting out of sync. The new filestream type is an interesting development because it can help alleviate these problems.
The disadvantage of storing this data in the database is that you put additional strain on the database, especially if the bandwidth between your database and webserver is already strained.
As others have already stated in comments (which BTW you guys should have been answers despite the pendantic police) you really have to have some killer reason to store images in the database. Otherwise just place them in the file system.
Especially this is true in the case where the images are delivered from a web server. The web server is way more effecient at delivering images from the file system than your code will be extracting them from a database.
Related
There is a situation in my company where we are developing a light weight .net web application with least dependencies. Application will be used hosted on client server. However there will not be any internet connection and they will use application locally.
We do not want any type of database installation on client machine. We want to keep it as simple as possible on client side. for this purpose we want to save/access data from file, as data on client side will not exceed more than 100 000 rows. We are also concerned about the speed for accessing data.
Here I want to ask how the data should be saved in file so that it can be accessed fast? What file format should be?
Whether I can use any db file which does not require any database installation on client side.
You could save all data to a json file, this will become increasingly slow and prone to corruption.
Also, have a look at SqlLite.
You can try Sql Compact Edition or SqlLite. Both are file based solution and fit as per your need.
Advantage of using these two would be that you can perform almost all the database queries on it and the data retrieval will be very fast. Also the you can think of optimizing the data storage and create tables etc.
You can use SQLite which is heavily uses in such scenarios (among others used by Chrome and Firefox). It is even public domain, so no license costs etc.
I need to get large data (~100MB) from sql server into app's sqlite once a day wirelessly.
App has json/restful webservice for other things, but figured this isn't possible as 100MB loaded into memory via json object would cause memory crash when I try to write json to sqlite.
I am now considering retrieving a file from url and saving it locally. That way data isn't loaded into memory.
The part I get fuzzy on is best way to get data, ie download a .sqlite compressed file or download a text file with prepared insert statements to an existing sqlite. Pretty sure the prior is best choice, but not DB savy enough to know what's possible on the sql server. Is it possible for a sql server to select data subsets and create a sqlite file? maybe it just needs to be scripted.
One thing to consider is the structrue of data on the sql server. I need subsets of data from several tables, not the entire tables. Example: SQL server houses 100 physical sites data, app is at site X today, just load site X data.
Am I on the right track or did a I miss an obvious solution?
We are using several text files as Templates to create the results of a WCF Data Services - Service Operation call.
The text files are each less than 3000 Bites Max.
What are the pros and cons of storing my template files on the file system with the WCF Data Services files vs storing them in a SQL Server 2008 R2 server?
Prior to SQL Server 2008, I recommended strongly against storing large objects like text files in the database. It tended to slow down access and made them generally harder to work with. Instead, I generally recommended storing links to the files in question.
Of course, this meant that the database would not protect the files in the event someone deleted something they shouldn't and the files needed to be backed up and transferred separately from the database.
With SQL Server 2008, I think many of the former problems have been overcome using the filestream functions and I think that storing files using filestream can be quite useful at times. It continues to store the actual data outside the database, which avoids many of the former complications. But it still binds the two together and permits the database to protect the files rather than just relying on the links in the database to be correct.
There's a lot of pros and cons for either storage method. Nowadays (my opinion has changed, and may change again some day), I'd focus on security and managability.
If it is sensitive data, you might get a bit more security by storing them in the database. If nothing else, it might be more difficult to hack a database than a file system. If security is not so important, it can be easier/simpler to store it on the OS.
For managing, if the data gets updated (and how frequently does that happen), how easy is it to update? One instance in a database is simpler to update (or corrupt...) than an instance on each of however many servers are in your web farm. (1 server, no problem, 20 servers, possible headache.)
I think it's better to store the data directly in the database. This makes it even faster to access because a database is more efficient in reading and generally handling data. You could always store movies in databases - that's no problem. Then it's also possible to stream large data.
For security reasons existing more as enough options to configure your database. And if you cluster your database - this is even more scalable.
I am about to create an ASP.NET MVC app which will have over 2000 products and each products will approx 20 photos. The app will be asp.net mvc app and
I am using sql server 2008 r2 to manage my data. which way is the better approcah here;
Uploading pictures to a path and
storing their file names to database
in order to be able to make a
relation to each other.
Storing pictures inside the database
as byte as well and retreive them
from there when needded.
definitely in the filesystem (store path) is better, i have done both in the past.
Against SQL server to store images
A) betting data in and out can be more difficult as have to used blob type objects and some ORMs don't really cater for this
B) your data base is much bigger so effects your backup/restore policy. The more frequently you backup the better but space will be increased. Storing in file, yep you still need to backup but backing up filesystem is easy.
C) when you run out of storage space you just add another NAS drive / server and start storing images there, so scales horizontally
The common perception is not as good as data stored in two places but for me its better as the type of data in stored in the best storage medium for the data types ..
Definitely storing as a path rather than the byte array. This means you can easily change the actual image itself without having to alter any code or muck around in SQL (as long s the new file has the same name as the old one).
Hope this helps.
In the database using FILESTREAM which combines the 2 ideas (file and database)
FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.
This changes the file vs database arguments
If you want to store paths only, then you'll have to accept the fact that images and database will get out of synch over time.
After having checked extensively for an answer to this question, I still cannot find a satisfying solution. So here goes.
I need to store possibly large amount of data in a column of an SQL Server 2005 table. I absolutely need to work in a streaming fashion, so that :
When writing, the data is sent in chunks to the database. Is there some builtin way to achieve this from C# client-code using the classes in System.Data.SqlClient? Or do I need to resort to using ADODB.Net Stream object ? Not sure how to mix the two concepts (especially with regards to participating in the current SqlTransaction.
Alternatively, is there a way for a T-SQL stored procedure to append data to an existing column. With this approach, the stored procedure would be called multiple times from the client code and this would achieve the streaming requirement.
When reading, the data should be read one chunk at a time, in a streaming fashion.
Alternatively, is there a way for a T-SQL stored procedure to provide sequential or even random access to the contents of an image field?
Actually, there is a way, it just hurts a bit.
SQL Server 2005 supports updating part of a column:
http://technet.microsoft.com/en-us/library/ms177523(SQL.90).aspx
And you can do a read of part of a column with substring (yes, even binary - it'll cheerfully return an array of bytes).
The caveats here are that it's only an update - so you'd have to create a value in the varbinary(max) field first, then update it. Other than it, it's absolutely possible to deal with as though you're streaming data to/from SQL Server. I wrapped the retrieval / update functionality with a stream class to make my life easier.
Hope this helps.
Well, answering my own question.
The truth is... there is actually no way to do what I want. Either from pure client code or from server side stored procedures in T-SQL. Until we switch to SQL Server 2008, we will have to find another soluton.
However, there is actually a way to simulate this behavior, so that the streaming requirement is achieved. The solution lies in a collaboration between client and server code.
The server database should, for instance, expose the entire contents to be streamed as a set of records in a fragments table. Each record, representing a chunk of the entire contents to be streamed. Whereas in the client, the stream is read in sequence, and each chunk is then sent to the database to fill one record.
With suitable bookkeeping, the reading of the stored data can also be done in a streaming fashion.
Incidentally, that's what Microsoft BizTalk Server is doing, and that's how I find out.