There is a situation in my company where we are developing a light weight .net web application with least dependencies. Application will be used hosted on client server. However there will not be any internet connection and they will use application locally.
We do not want any type of database installation on client machine. We want to keep it as simple as possible on client side. for this purpose we want to save/access data from file, as data on client side will not exceed more than 100 000 rows. We are also concerned about the speed for accessing data.
Here I want to ask how the data should be saved in file so that it can be accessed fast? What file format should be?
Whether I can use any db file which does not require any database installation on client side.
You could save all data to a json file, this will become increasingly slow and prone to corruption.
Also, have a look at SqlLite.
You can try Sql Compact Edition or SqlLite. Both are file based solution and fit as per your need.
Advantage of using these two would be that you can perform almost all the database queries on it and the data retrieval will be very fast. Also the you can think of optimizing the data storage and create tables etc.
You can use SQLite which is heavily uses in such scenarios (among others used by Chrome and Firefox). It is even public domain, so no license costs etc.
Related
I've seen examples of web apps that has user accounts and stores information about the users, but does not use a database. I've been searching for a while and cannot figure out how this would be done without a database. Can someone point me in a direction?
Clearly the data has to be stored somewhere. It could stored on the client, using cookies or HTML5 Web Storage, but this would make it nearly impossible for the data to be shared across devices. (Technically, the data could be shared between devices provided that another device maintains an active connection to the web server at the same time; the web server would merely transfer the data between devices without storing it. However, this would be extremely impractical.) If the user data is stored in any centralized location, there would effectively be a database, even if specialized database software, such as MySQL, is not used, i.e. just using the filesystem of the server directly.
Some used database such as SQLite to store data, but for other application they simply saved it to a file somewhere in the directory (may be cookies, temporary file, settings/configuration, ....etc).
There are 2 main ways of storing data based on the requirement:
Storing data On client
1.HTML5 Webstorage is now available, which store data locally and it can support data upto 5MB and it is more secured and faster.
2.Cookies
Storing data On Server
Through flat files that is text file or through XML files.XML is standard in many companies as a way to store data. This is really quick.
Having said that, there are downsides to it as well.
I have a vb.net windows form application with a database on SQL Server 2008 on the ./SQLEXPRESS instance.
I have created a setup of my project using the link below..
http://msdn.microsoft.com/en-US/library/49b92ztk(v=vs.80).aspx
When a user installs my application, the database will be available for him, and user can just export the SQL Server database.
How can I secure my database so that user shouldn't have a easily available copy of my database?
I thought of creating a new password protected server (as I have created the database in above walkthrough)... while installation of my application on user's pc, other than ./sqlexpress. And a complete copy of database used by my application will not be simply available for user to just export and get a copy of my database.
So could anyone please guide me...
The question is; how far do you want to go to protect your data?
Better protection of your data usually comes at the cost of more development time and likely less user friendliness, for example due to lower performance (encryption is not free). More complex code usually results in more support requests too.
Where the best balance is depends on your business model (if any) and on your user requirements.
Keep in mind that anything you deploy to an end-users machine is in the end vulnerable. If something is valuable enough there will be people trying to steal it.
So, you could argue that the best protection is not to deploy the data at all. You could back your end-user application with a web service and keep the data on your own server, for example in the cloud.
I've found however that you sometimes just need to trust your users. If you build a good product that makes them happy, they have no reason to steal from you. In fact, they are probably glad to pay you.
If you decide that you need to deploy the data and that you need to encrypt it, you should think about why you chose SQL Server.
What database features do you need exactly? Do you need a fullblown database server for that?
Any local admin can gain control over any SQL Server database in seconds so the built-in SQL server authentication will not bring you a lot of benefits.
You could switch to SQLServer CE and keep the database within your application. That would make the database a lot harder to access for a regular user.
If all you're doing is looking up words, you may be better off with a different storage engine like Lucene.
Lucene is actually a search engine, so it's highly optimized for matching words or parts of words.
You can run Lucene inside your .NET application so you don't even need the end-user to install SQL Server. There is a .NET version of Lucene here.
Lucene however doesn't protect your data. There's tooling available that will allow anybody to view and extract the data from the stored index files.
Since Lucene is open source though, you could extend it to support encrypted data storage (see this related question).
We are using several text files as Templates to create the results of a WCF Data Services - Service Operation call.
The text files are each less than 3000 Bites Max.
What are the pros and cons of storing my template files on the file system with the WCF Data Services files vs storing them in a SQL Server 2008 R2 server?
Prior to SQL Server 2008, I recommended strongly against storing large objects like text files in the database. It tended to slow down access and made them generally harder to work with. Instead, I generally recommended storing links to the files in question.
Of course, this meant that the database would not protect the files in the event someone deleted something they shouldn't and the files needed to be backed up and transferred separately from the database.
With SQL Server 2008, I think many of the former problems have been overcome using the filestream functions and I think that storing files using filestream can be quite useful at times. It continues to store the actual data outside the database, which avoids many of the former complications. But it still binds the two together and permits the database to protect the files rather than just relying on the links in the database to be correct.
There's a lot of pros and cons for either storage method. Nowadays (my opinion has changed, and may change again some day), I'd focus on security and managability.
If it is sensitive data, you might get a bit more security by storing them in the database. If nothing else, it might be more difficult to hack a database than a file system. If security is not so important, it can be easier/simpler to store it on the OS.
For managing, if the data gets updated (and how frequently does that happen), how easy is it to update? One instance in a database is simpler to update (or corrupt...) than an instance on each of however many servers are in your web farm. (1 server, no problem, 20 servers, possible headache.)
I think it's better to store the data directly in the database. This makes it even faster to access because a database is more efficient in reading and generally handling data. You could always store movies in databases - that's no problem. Then it's also possible to stream large data.
For security reasons existing more as enough options to configure your database. And if you cluster your database - this is even more scalable.
I often find myself writing one off queries to either answer someone's question or trouble shoot something and I would like to be able to quickly expose the on demand refreshable results of the query graphically so that I can share these results to others without having to go through the process of creating an SSRS report and publishing it to a reporting services server.
I have thought about using excel to do this or maybe running a local SSRS server but both of these options are still labor intensive and I cannot justify the time it would take to do these since no one has officially requested that I turn this data into a report.
The way I see it the business I work for has invested money in me creating these queries that often return potentially useful data that other people in the organization might want but since it isn't exposed in any way and I don't know that this data is something they want and they may not even realize they want this data, the potential value of the query is not realized. I want to increase the company's return on investment on all these one off queries that I and other developers write by exposing their results graphically so that they can be browsed by others and then potentially turned into more formalized SSRS reports if they provide enough value to justify the development of the report.
What is the fastest way for me to take a query and turn it into a refreshable graph of the results set?
Why dont you simply use what you may already have. Excel...you can import data via an ODBC / Oracle / SQL Connection. Get Data..and bam you can run the query and format it right in the spreadsheet and provide sorting etc. All you need to supply is the database name and user name and password to connect to the db.
JonH is right regarding Excel's built in ODBC support, but I have had tons of trouble with this. In my case, the ODBC connection required the client software to be installed so that it could use the encryption methods, etc. Also, even if that were not the case, the user (I believe) would still have to manually install and set up an ODBC connection.
Now if you just want something on your machine to do the queries and refresh them, JohH's solution is great and my caveats are probably irrelavent. But if you want other users to have access, you should consider having a middle-man app (basically a PHP script, assuming a web server is an option for you), that does a query, transforms the results into XML, and outputs it as "report-xyz.xml". You can then point anybody running a newer version of Excel to that address and they can very easily import the data into Excel with no overhead. (basically a kind of web service).
Keep in mind, I don't think you should have a web script that will allow users to make queries to your Database server! You would have some admin page where you make pass the query in and a new xml file with the results gets made. So my idea is also based on the idea that you want to run the same queries over and over without any specifics passed in. (if that were the case, I'd look into just finding a pre-built web services bridge for your database that already has security features built in. Then you could have users make the limited changes allowed.)
My database must be updated at arbitrary intervals, manually, with new info to put on standart tables that don't require structural modifications. one app will update the database.
Another app, the one I will distribute (to friends and family only, but doesn't require any security feature to it, like encrypting the database) will read the database and use its data on labels, listviews, etc.
The problem is, I'm the perfect definition of full-fledged n00b at programming of any sort, and still don't know what database to use.
I thought that I should use a SQL CE (*.sdf) file, and store that database thing on an FTP. then, I could download it and get data from it everytime the "client" app runs, and certain button ("connect") is clicked.
After some hard-core googling, I found out how to connect to the sdf thing, using this connection string:
Provider=Microsoft.SQLSERVER.CE.OLEDB.3.5;Data Source=D:\Documents and Settings\Camilo\JCTM.sdf
So it connects, or at least didn't show any error.
I don't know if it's a good idea to use sdf SQL CE files as databases, if it's too hard maybe I should go for XML? what do you guys suggest, what is the easiest way to implement very simple databases in VB.NET?
By simple databases I mean:
- no search needed
- no advanced features except storing strings on tables with columns and rows
- easy to access, read, edit, etc. by different VB.NET apps
Is sdf a good idea?
I would recommend Sql Server Express Its free and can be redistributed with .net applications as part of the install process.
The challenge will be syncing the changes between the different clients. If you have access to a FTP server, you may have the ability to host a website in IIS. If you can do that you can just use webservices and read against one database instead of copying one local.
Luckily for you, you can abstract away the need to be concerned with which back-end database you use to store your data.
Technologies such as ODBC (Open Database Connectivity) and OLEDB (Object Linking and Embedding, Database) allow you to limit your concern for the backend datastore to the task of crafting your connection string. In your example, the connection string says, "I'm going to connect to a SQL Server CE database via its OLEDB provider, and it's physically located over on D:/...
Within the code, you use standard OLEDB mechanisms to access and manage the database. Because of this abstraction, you can use OLEDB providers for SQL Server, Oracle, XML, Access or comma delimited text files as your backing store if you wish, and the only change you need to make to your code is the connection string. Your choice then should be to pick the database that you have the tools and know-how to set up and manage initially.
I'd start with Microsoft Access because it has its own UI, and can play well with .NET.
You can also try the ADO.Net implementation for SQLite, which I've also found very useful.