I am looking for a good design pattern for rewriting the middle tier of a system incrementally while still maintaining its original database until enough of the system has been replaced that the old database can be swapped out in favour of the new.
So for example imagine you have a legacy system with Table A, and Table A has 100 coloumns thats maps to Object A that has 100 properties. You come along and decide that actually Object A can be split logically into 5 objects which in an ideal world would back onto 5 tables, but for the mean while has to persist into the original 100 column table. I'm looking for a way that a section of the middle tier can be swapped out and your new 5 objects swapped in but in the background it persists to the same database.
In reality we'll not be swapping out the old middle tier but rather writing a new one that will slowly take hold, but for a period of time need to persist to the original db.
Any ideas?
Cheers.
Typically, you'd use Views to mask table design changes to maintain the database "API" without client code changes
Note: objects and tables are not the same things...
The adapter and facade patterns sound like good matches for what you are doing.
I've decided to use NHibernate to take care of the mappings until we can start to replace the underlying database. So the object model will be properly normalised and svelte and using the mapping ability of NHibernate will map to the existing tables first before then swapping that mapping out for a new table mapping.
Related
For a hobby project I am building an application to keep track of my money. Register everything that comes in and goes out. I am using sqlite as a database backend.
I have two data access models in mind.
Creating one master object as a sort of database connector, which contains methods which execute the queries and provide the required sets of data as a list of objects
Have objects who need data execute the queries themselves
Which one of these is 'the best' and why? Or are there different, better models out there?
The latter option is better. In the first option, you would end up having to touch your universal data access object for just about any update to the code that wasn't purely a change in display logic. If you have different data access objects, then you will have much more testable, maintainable code.
I suggest you read up a bit on the model-view-controller paradigm. The wikipedia article on it is a good start: http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller.
Also, you didn't say which language/platform you were coding in, but most platforms have numerous options for auto-generating a starting point your data access classes from your database. You may find something like that useful.
Much of a muchness really, the thing to avoid is having the "same" sql sprinkled all over your code base.
The key point is. You've just added a new column to Table1. When you do Find In Files "Table1", how many hits are you going to get and where.
If you use one class and there's a lot of db operations, it's going to get very messy very quickly, but if you have one interface (say IModel) with one implementation, you can swap backends very easily.
So how many db operations, and how likely is it you will move away from SqlLite.
I'm a relative newbie at NHibernate, so I'll beg forgiveness in advance if this a stupid question. I've googled it and searched the documentation, and am getting all wrapped around the axle.
I'm maintaining/enhancing an existing application that uses NHibernate for a relatively straightforward table. The table has about 10-12 fields, and no foreign key relations. The table contains somewhere around a dozen or so rows, give or take.
Two of the fields are huge blobs (multi-megabytes). As a result, the table is taking an excessive amount of time (4 minutes) to load when working with a remote DB.
The thing is that those two fields are not needed until a user selects one of the rows and begins to work on it, and then they are only needed for the one row that he selects.
This seems like exactly what lazy loading was meant for. I just can't quite figure out how to apply it unless I break up the existing DB schema and put those columns in their own table with one-to-one mapping, which I don't want to.
If it matters, the program is using NHiberate.Mapping.Attributes rather than hbm files, so I need to be able to make alterations in the attributes of the Domain objects that will propagate to the hbm.
Thanks for any help.
You need lazy properties IN NHibernate 3 to accomplish this. I assume, but don't know, that you can set that using attributes.
I'm designing a C# application
Presentation ( web site + flex apps )
Business Logical Layer (might be WCF to enable multi client platforms)
Data Access Layer ( With NHibernate )
We're going to integrate our solution in many preexistant client's database environnements and we would like to use NHibernate in the DAL.. My colleague pointed out that generating classes from client's DB (like User or Image) with NHibernate would cause the BLL to blow up in our face at each DB changes !
So the question is how do we prevent that from happening ?
We're thinking about creating business objects and map NHibernate objects to these BO (hum, does that make them DTOs ?) with AutoMapper and prevent dal changes from affecting BLL.. Is this the way to go ?
Thanks !
EDIT :
To give a better understanding of what we're trying to achieve, you might need context :
We're building a photo storing/sharing app in Flex for the front-end and C# on the back-end mainly for our company, so we handle every aspects of the code and DB.
But : that product can also be bought by tiers, which eventually have already a database with User table or Image table. I'm thinking here about a new prospect who have an Image table with a few hundred millions of rows and adding columns for our business logic isn't going to be happening because of a too long ALTERing of the table.
Even though it would be possible (User table for example can be modified because of lesser rows), we're asking ourselves how to handle table structure changes without impacting all of our solution each time we have to integrate in a tier database, from BLL to client app in Flex !
in my experience, your business objects (AKA Domain Objects) should be modelled in OO to represent your real life business entities and your tables in 3rd normal form (this may change depending on what design you are after speed vs file size)
NHibernate should map between your BO's and Tables, using its mapping files.
now you have legitimate cases:
You need to add/remove a column, we decided to remove addressline4, this will echo a change in your Address Object, thats fine.
You move a column to a better place, our Client object contains notes, which is currently stored in the Contract_Extra table, which is going to be moved into the Client table. moving the column into a better place will only effect the Mapping file, in this case
I doubt there is a blanket reasoning, however I hope the examples make you think about this
I have not tried NH accross multiple Db's, also should each database have its own service on top?
here are some links
Multi table entites
PoEAA <- look at the Single Table inheritance, Class table inheritance and the other one
Hope this helps
It sounds like you want to design your domain model to be database agnostic. I too am interested in the best approach to having a central domain model that can map over to multiple different database models.
The way you are proposing, to create DTO's from each database using code generators, could be an option. Another would be to create custom NHibernate mappings for each preexisting database. You still may need to use some DTOs to make some of the mappings less difficult but it may give you more control.
These are just some thoughts. More experienced users with NHibernate probably will have better insight to your situation.
Overview: I'm designing a restaurant management application, and I have an entity called Order which has Items. Since a restaurant can operate for many years, with many thousands of completed 'orders', and in the interest of making the networking side of my application easier and keeping the database fast, I want to introduce the concept of a ClosedOrder, which is an Order has been paid for, esentially.
I have a few options for doing this: I could add an isClosed attribute to my Order entity, and perform all fetch requests for 'open' orders with a predicate, but that keeps the problem of there being lots of records for the DB to go through every time a fetch is needed, which is often with the Order entity because of the workflow of my app. If I understand correctly, creating a 'ClosedOrder' subentity would also have the same problem, as Core Data stores all subentities in the same table in the database.
Is creating a completely separate entity foolish in this case? Or is it exactly what I need to do? I apologize for my overall lack of knowledge about database performance, Core Data is beautiful in that it abstracts needing to learn about that away, but at the same time it can't actually make it not important, especially in a case like this where performance could severely degrade if it's pushed far by my users.
If you're using SQLite store type and the "isClosed" attribute is "Indexed" (a setting in your entity editor panel), you could have hundreds of thousands of orders and still have a nice, quick fetch time when filtering only for "isClosed == YES".
Creating a separate entity won't really buy you much performance but it will make it more of a pain to maintain when things change (two migration steps for the price of one, for example). You're still storing all those items and SQLite is a competent DB library when it's setup correctly. Use an indexed attribute here, then generate some test data, then measure the performance. I'm sure you'll be satisfied.
I've shown up at a new job and discovered database which is in dire need of some help. There are many many things wrong with it, including
No foreign keys...anywhere. They're faked by using ints and managing the relationship in code.
Practically every field can be NULL, which isn't really true
Naming conventions for tables and columns are practically non-existent
Varchars which are storing concatenated strings of relational information
Folks can argue, "It works", which it is. But moving forward, it's a total pain to manage all of this with code and opens us up to bugs IMO. Basically, the DB is being used as a flat file since it's not doing a whole lot of work.
I want to fix this. The issues I see now are:
We have a lot of data (migration, possibly tricky)
All of the DB logic is in code (with migration comes big code changes)
I'm also tempted to do something "radical" like moving to a schema-free DB.
What are some good strategies when faced with an existing DB built upon a poorly designed schema?
Enforce Foreign Keys: If a relationship exists in the domain, then it should have a Foreign Key.
Renaming existing tables/columns is fraught with danger, especially if there are many systems accessing the Database directly. Gotchas include tasks that run only periodically; these are often missed.
Of Interest: Scott Ambler's article: Introduction To Database Refactoring
and Catalog of Database Refactorings
Views are commonly used to transition between changing data models because of the encapsulation. A view looks like a table, but does not exist as a finite object in the database - you can change what column is being returned for a given column alias as desired. This allows you to setup your codebase to use a view, so you can move from the old table structure to the new one without the application needing to be updated. But it means the view has to return the data in the existing format. For example - your current data model has:
SELECT t.column --a list of concatenated strings, assuming comma separated
FROM TABLE t
...so the first version of the view would be the query above, but once you created the new table that uses 3NF, the query for the view would use:
SELECT GROUP_CONCAT(t.column SEPARATOR ',')
FROM NEW_TABLE t
...and the application code would never know that anything changed.
The problem with MySQL is that the view support is limited - you can't use variables within it, nor can they have subqueries.
The reality to the changes you wish to make is effectively rewriting the application from the ground up. Moving logic from the codebase into the data model will drastically change how the application gets the data. Model-View-Controller (MVC) is ideal to implement with changes like these, to minimize the cost of future changes like these.
I'd say leave it alone until you really understand it. Then make sure you don't start with one of the Things You Should Never Do.
Read Scott Ambler's book on Refactoring Databases. It covers a good many techniques for how to go about improving a database - including the transitional measures needed to allow both old and new programs to work with the changing design.
Create a completely new schema and make sure that it is fully normalized and contains any unique, check and not null constraints etc that are required and that appropriate data types are used.
Prepopulate each table that fills the parent role in a foreign key relationship with a single 'Unknown' record.
Create an ETL (Extract Transform Load) process (I can recommend SSIS (SQL Server Integration Services) but there are plenty of others) that you can use to refill the new schema from the existing one on a regular basis. Use the 'Unknown' record as the parent of any orphaned records - there will be plenty ;). You will need to put some thought into how you will consolidate duplicate records - this will probably need to be on a case by case basis.
Use as many iterations as are necessary to refine your new schema (ensure that the ETL Process is maintained and run regularly).
Create views over the new schema that match the existing schema as closely as possible.
Incrementally modify any clients to use the new schema making temporary use of the views where necessary. You should be able to gradually turn off parts of the ETL process and eventually disable it completely.
First see how bad the code is related to the DB if it is all mixed in no DAO layer you shouldn't think about a rewrite but if there is a DAO layer then it would be time to rewrite that layer and DB along with it. If possible make the migration tool based on using the two DAOs.
But my guess is there is no DAO so you need to find what areas of the code you are going to be changing and what parts of the DB that relates to hopefully you can cut it up into smaller parts that can be updated as you maintain. Biggest deal is to get FKs in there and start checking for proper indexes there is a good chance they aren't being done correctly.
I wouldn't worry too much about naming until the rest of the db is under control. As for the NULLs if the program chokes on a value being NULL don't let it be NULL but if the program can handle it I wouldn't worry about it at this point in the future if it is doing a default value move that to the DB but that is way down the line from the sound of things.
Do something about the Varchars sooner rather then later. If anything make that the first pure background fix to the program.
The other thing to do is estimate the effort of each areas change and then add that price to the cost of new development on that section of code. That way you can fix the parts as you add new features.