I am putting some heavy though into re-writing the data access layer in my software(If you could even call it that). This was really my first project that uses, and things were done in an improper manner.
In my project all of the data that is being pulled is being stored in an arraylist. some of the data is converted from the arraylist into an typed object, before being put backinto an arraylist.
Also, there is no central set of queries in the application. This means that some queries are copy and pasted, which I want to eliminate as well.This application has some custom objects that are very standard to the application, and some queries that are very standard to those objects.
I am really just not sure if I should create a layer between my objects and the class that reads and writes to the database. This layer would take the data that comes from the database, type it as the proper object, and if there is a case of multiple objects being returned, return a list of those object. Is this a good approach?
Also, if this is a good way of doing things, how should I return the data from the database? I am currently using SqlDataReader.read, and filling an array list. I am sure that this is not the best method to use here, i am just not real clear on how to improve this.
The Reason for all of this, is I want to centralize all of the database operations into a few classes, rather than have them spread out amongst all of the classes in the project
You should use an ORM. "Not doing so is stealing from your customers" - Ayende
One thing comes to mind right off the bat. Is there a reason you use ArrayLists instead of generics? If you're using .NET 1.1 I could understand, but it seems that one area where you could gain performance is to remove ArrayLists from the picture and stop converting and casting between types.
Another thing you might think about which can help a lot when designing data access layers is an ORM. NHibernate and LINQ to SQL do this very well. In general, the N-tier approach works well for what it seems like you're trying to accomplish. For example, performing data access in a class library with specific methods that can be reused is far better than "copy-pasting" the same queries all over the place.
I hope this helps.
It really depends on what you are doing. If it is a growing application with user interfaces and the like, you're right, there are better ways.
I am currently developing in ASP.NET MVC, and I find Linq to SQL really comfortable. Linq to SQL uses code generation to create a collection of code classes that model your data.
ScottGu has a really nice introduction to Linq to SQL on his blog:
http://weblogs.asp.net/scottgu/archive/2007/05/19/using-linq-to-sql-part-1.aspx
I have over the past few projects used a base class which does all my ADO.NET work and that all other data access classes inherit. So my UserDB class will inherit the DataAccessBase class. I have it at the moment that my UserDB class actualy takes the data returned from the database and populates a User object which is then returned to the calling Business Object. If multiple objects are returned then these are then a Generic list ie List<Users> is returned.
There is a good article by Daemon Armstrong (search Google for Daemon Armstrong which demonstrates on how this can be achived.
""http://www.simple-talk.com/dotnet/.net-framework/.net-application-architecture-the-data-access-layer/""
However I have now started to move all of this over to use the entitty framework as its performs much better and saves on all those manual CRUD operations. Was going to use LINQ to SQL but as it seems to be going to be dead in the water very soon thought it would be best to invest my time in the next ORM.
"I am really just not sure if I should create a layer between my objects and the class that reads and writes to the database. This layer would take the data that comes from the database, type it as the proper object, and if there is a case of multiple objects being returned, return a list of those object. Is this a good approach?"
I'm a Java developer, but I believe that the language-agnostic answer is "yes".
Have a look at Martin Fowler's "Patterns Of Enterprise Application Architecture". I believe that technologies like LINQ were born for this.
Related
In the past, any database administrator worth his salt would have told you never to query against a table directly. In fact, they would have prevented it by putting all tables in one schema & cutting-off direct access to that schema...thereby forcing you to query or CRUD from views & procedures etc. And further, protecting the data with security-layers like this made sense from a security perspective.
Now enters Entity Framework...
We use Entity Framework now where I work. The IQueryable is King! Everything is back into the DBO schema. And, people are going directly to tables left-and-right because Repository patterns and online examples seem to encourage this very practice.
Does Entity Framework do something under the hood that makes this practice okay now?
If so, what does it do?
Why is this okay now?
Help me understand.
Does Entity Framework do something under the hood that makes this
practice okay now?
EF doesn't really change the dynamic. It is a convenience of data access and rapid development, not a better way to get data.
If so, what does it do?
It does not. It does, I think, at least avoid constructing SELECT * type queries.
Why is this okay now?
It remains a matter of opinion and culture. In general a "strict" DBA will want you to hit only exposed objects layered on top of the tables for CRUD. It is much easier to tune such queries and maintain control of performance if the application is using the expose custom objects rather than using an ORM or hand-coding direct queries.
IMO, ORM are great for rapid protyping and basic stuff, but anything with more complex logic or substantial performance should be moved to custom built objects.
Where/when that lines is will vary substantially based on any number of things including:
Size/load of database
Availability of database professionals vs app
developers
Maturity of company
There are different types of Entity Framework (Code First, Database First, Model First). It appears you have an issue with Code First, which creates the database based on your classes and limits what you can do on the database side of things.
However with EF Database First, you can still do everything you did before. You can restrict access to your tables and expose Stored Procedures/Views for your CRUD operations. You would still benefit from EF, because of the strongly typed classes that are generated from your Views, and strongly typed methods generated from your Stored Procedures.
Now everyone can be happy - you get to cut off access to the schema and IQueryable is still king
For a hobby project I am building an application to keep track of my money. Register everything that comes in and goes out. I am using sqlite as a database backend.
I have two data access models in mind.
Creating one master object as a sort of database connector, which contains methods which execute the queries and provide the required sets of data as a list of objects
Have objects who need data execute the queries themselves
Which one of these is 'the best' and why? Or are there different, better models out there?
The latter option is better. In the first option, you would end up having to touch your universal data access object for just about any update to the code that wasn't purely a change in display logic. If you have different data access objects, then you will have much more testable, maintainable code.
I suggest you read up a bit on the model-view-controller paradigm. The wikipedia article on it is a good start: http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller.
Also, you didn't say which language/platform you were coding in, but most platforms have numerous options for auto-generating a starting point your data access classes from your database. You may find something like that useful.
Much of a muchness really, the thing to avoid is having the "same" sql sprinkled all over your code base.
The key point is. You've just added a new column to Table1. When you do Find In Files "Table1", how many hits are you going to get and where.
If you use one class and there's a lot of db operations, it's going to get very messy very quickly, but if you have one interface (say IModel) with one implementation, you can swap backends very easily.
So how many db operations, and how likely is it you will move away from SqlLite.
would redbeanphp's bean can server be a useful orm for knockout (using the mapping plugin?). i have (or will have) a mysql database with many one to many, many to many, and one to one relationships. i would like to edit a record and all its related data as an object in a single form based interface.
as far as the ui is concerned, i would be working with a nested json object, viewing it in html, and editing it using form element templates, adding them to the dom as needed.
beancan server (or simply the export/import functions) would convert beans into json objects (and vice versa), knockout would handle the modifications to the object. beancan server would then manage the crud functions with the mysql database. i know, i should probably be using a schemaless database like couchdb or somesuch, but in this case it isn't an option.
is this outrageous? or possible workable? i can't seem to locate any round trip examples with any complexity for redbean, so i don't know if this makes sense or not. i've had a lot of success with frameworks -- not being a programmer for the most part, but able to grasp a concept if given a concrete example to work from. any help would be greatly appreciated.
I've never used redbean before, but as an avid KnockoutJS user, I can tell you this sounds reasonable.
You're converting your model objects to JSON, manipulating them in the UI via KnockoutJS, then sending them back to the server for saving.
That's perfectly reasonable and is typically how we do things, no matter the ORM. Really, the ORM should not affect the UI tech you use. And in this case, as long as your objects can be converted to/from JSON, you should be just fine.
Really newbie question coming up. Is there a standard (or good) way to deal with not needing all of the information that a database table contains loaded into every associated object. I'm thinking in the context of web pages where you're only going to use the objects to build a single page rather than an application with longer lived objects.
For example, lets say you have an Article table containing id, title, author, date, summary and fullContents fields. You don't need the fullContents to be loaded into the associated objects if you're just showing a page containing a list of articles with their summaries. On the other hand if you're displaying a specific article you might want every field loaded for that one article and maybe just the titles for the other articles (e.g. for display in a recent articles sidebar).
Some techniques I can think of:
Don't worry about it, just load everything from the database every time.
Have several different, possibly inherited, classes for each table and create the appropriate one for the situation (e.g. SummaryArticle, FullArticle).
Use one class but set unused properties to null at creation if that field is not needed and be careful.
Give the objects access to the database so they can load some fields on demand.
Something else?
All of the above seem to have fairly major disadvantages.
I'm fairly new to programming, very new to OOP and totally new to databases so I might be completely missing the obvious answer here. :)
(1) Loading the whole object is, unfortunately what ORMs do, by default. That is why hand tuned SQL performs better. But most objects don't need this optimization, and you can always delay optimization until later. Don't optimize prematurely (but do write good SQL/HQL and use good DB design with indexes). But by and large, the ORM projects I've seen resultin a lot of lazy approaches, pulling or updating way more data than needed.
2) Different Models (Entities), depending on operation. I prefer this one. May add more classes to the object domain, but to me, is cleanest and results in better performance and security (especially if you are serializing to AJAX). I sometimes use one model for serializing an object to a client, and another for internal operations. If you use inheritance, you can do this well. For example CustomerBase -> Customer. CustomerBase might have an ID, name and address. Customer can extend it to add other info, even stuff like passwords. For list operations (list all customers) you can return CustomerBase with a custom query but for individual CRUD operations (Create/Retrieve/Update/Delete), use the full Customer object. Even then, be careful about what you serialize. Most frameworks have whitelists of attributes they will and won't serialize. Use them.
3) Dangerous, special cases will cause bugs in your system.
4) Bad for performance. Hit the database once, not for each field (Except for BLOBs).
You have a number of methods to solve your issue.
Use Stored Procedures in your database to remove the rows or columns you don't want. This can work great but takes up some space.
Use an ORM of some kind. For .NET you can use Entity Framework, NHibernate, or Subsonic. There are many other ORM tools for .NET. Ruby has it built in with Rails. Java uses Hibernate.
Write embedded queries in your website. Don't forget to parametrize them or you will open yourself up to hackers. This option is usually frowned upon because of the mingling of SQL and code. Also, it is the easiest to break.
From you list, options 1, 2 and 4 are probably the most commonly used ones.
1. Don't worry about it, just load everything from the database every time: Well, unless your application is under heavy load or you have some extremely heavy fields in your tables, use this option and save yourself the hassle of figuring out something better.
2. Have several different, possibly inherited, classes for each table and create the appropriate one for the situation (e.g. SummaryArticle, FullArticle): Such classes would often be called "view models" or something similar, and depending on your data access strategy, you might be able to get hold of such objects without actually declaring any new class. Eg, using Linq-2-Sql the expression data.Articles.Select(a => new { a .Title, a.Author }) will give you a collection of anonymously typed objects with the properties Title and Author. The generated SQL will be similar to select Title, Author from Article.
4. Give the objects access to the database so they can load some fields on demand: The objects you describe here would usaly be called "proxy objects" and/or their properties reffered to as being "lazy loaded". Again, depending on your data access strategy, creating proxies might be hard or easy. Eg. with NHibernate, you can have lazy properties, by simply throwing in lazy=true in your mapping, and proxies are automatically created.
Your question does not mention how you are actually mapping data from your database to objects now, but if you are not using any ORM framework at the moment, do have a look at NHibernate and Entity Framework - they are both pretty solid solutions.
I am involved in development of a tiered application that uses LINQ2SQL separated from the web server with a NET.TCP Binding on WCF.
My questions are:
What sort of measures should I take
to achieve the best performance?
Since the entity objects returned by
the LINQ need to be converted to a
IEnumerable list to be serialized
everytime, is there anyway to remove
this dependency?
1) Concentrate on a properly normalized database design. I would say that when you are forced to make design tradeoffs in your code vs. database design, if performance is your goal, make tradeoffs in your object design instead of your database design. Understand that you aren't going to be able to do a proper supertype/subtype database design which will work with Linq to SQL (I'm told you need to use the EF instead).
2) Depends what you mean here. If you're asking how you would serialize anonymous classes across the wire, the easy answer is: "you can't, so don't try". If you want to put lists of objects across the wire, just use the ToArray() extension method on your IEnumerable collections to ship arrays of your business objects over the wire.
Linq to SQL is very slow unless you compile queries. Otherwise your application will be CPU bound as most of the time will be spend converting Expression trees into SQL.
We are talking about 10x performance gain if you use compiled queries. Try it :)