Does anyone have any good sources of information of using NHibernate with Sql Azure with the implications of sharding (because of the 10gb cap)? I know there are posts on the internet that reference a sharding project for NH but they are from 3rd quarter 09 and I haven't found any much more relevant on google.
Related does anyone have information about manually implementing sharding if the sharding project isn't viable to use yet? Would it just be as simple as creating a session factory for each shard and keep a collection of factories? That seems like it would be problematic reproducing the ISession calls through each factory however I suppose it could be achieved by passing operations as Funcs that get invoked on the ISession from each factory but seems more like the wrong path to be going down.
I wrote a proof of concept about a month ago using NHibernate on SQLAzure/Sharding. As you've pointed out, there are aspects that just do not feel right about it. Until the NH support has evolved, you may have to try a few things to find out what works best for you. I can tell you a general flow of how it worked for us.
We implemented a simple sharding strategy factory that provides strategies that decide which shard to place you in based on our needs. Your needs may vary here. The key is creating strategies that process, merge and order your query results. From there, session creation and usage is all the same as any other session usage, which is highly desirable.
EDIT: I know this post by Ayende is a few months old, but it's exactly how we implemented it and it works. The rumor is better support in nHibernate will be coming.
Related
I am working on a project that has over 2000 integration tests that full circles the database. I want to speed up the process so I thought why not fake the database.
We are using Fluent NHibernate as our ORM which is probably why I am having such trouble. We have already implemented this concept in a program that does not use NHibernate but rather basic CRUD operations.
Basically on any CRUD operation I would like to save the object in memory to say a dictionary list. This will speed up the process of our tests which will hopefully lessen our build times. Not to mention the cool factor.
I have looked into having two separate sessions and having the session factory use one or the other but I would have to implement many methods/properties that I really don't care about.
I also considering using a go between in class that does the translating myself but it would probably include me changing A LOT of existing code. I am trying to limit the impact on the rest of the project as much as possible so regression testing will not be a HUGE factor.
Please let me know of any other suggestions anyone has!
Thanks!
Use a SQLite in-memory database. Here's the blog post that originally showed me how...
http://jasondentler.com/blog/2009/08/nhibernate-unit-testing-with-sqlite-in-memory-db/
The last 6 months learning curve have been challenging with CQRS and DDD the main culprits.
It has been fun and we are 1/2 way through our project and the area I have not had time to delve into is a messaging framework.
Currently I don't use DTC so there is a very good likely hood that if my read model is not updated then I will have inconsistency between the read and write databases. Also my read and write database will be on the same machine. I doubt we will ever put them on separate machines.
I don't have a large volume of messages in my system so my concern is more to do with consistency and reliability of the system.
So, do I have to put in a messaging framework like NServiceBus (even though both read and write databases are on the same machine) or do I have other options? Yes there is learning curve but I suppose there would be a hell of a lot to learn if I don't use it.
Also, I don't want to put in a layer if it is not necessary
Thoughts?
Currently I don't use DTC so there is a very good likely hood that if
my read model is not updated then I will have inconsistency between
the read and write databases.
Personally, I dislike the DTC and try to avoid it. Instead, it is often possible to implement a compensation mechanism, especially for something like a read model where eventual consistency is already acceptable and updates are idempotent. For example, you could implement a version on entities and have a background task which ensures versions are in-sync. Having a DTC will provide transactional retry functionality, but it still won't solve cases where failure occurs after retries - you still have to watch the error log and have procedures in place to deal with errors.
So, do I have to put in a messaging framework like NServiceBus (even
though both read and write databases are on the same machine) or do I
have other options?
It depends on a few things. What you often encounter in a CQRS system is need for pub/sub where several sub-systems publish events to which the query/caching system subscribes to. If you see a need for pub/sub beyond basic point-to-point messaging, then go with something like NServiceBus. Also, I wouldn't immediately shy away from using NServiceBus even if you don't need it for scalability purposes because I think the logical partitioning is beneficial on its own. On the other hand, as you point out, adding layers of complexity is costly, therefore first try to see if the simplest possible thing will work.
Another question to ask is whether you need a separate query store at all. If all you have is a single machine, why bother? You could use something simpler like the read-model pattern and still reap a lot of the benefits of CQRS.
Does a CQRS project need a messaging framework like NServiceBus?
The short answer: no.
It is the first time I hear about the 'read-model pattern' mentioned by eulerfx. It is a nice enough name but there is a bit more to it:
The general idea behind the 'query' part is to query a denormalized view of your data. In the 'read-model pattern' link you will notice that the query used to populate the read-model is doing some lifting. In the mentioned example the required data manipulation is not that complex but what if it does become more complex? This is where the denomalization comes in. When you perform your 'command' part the next action is to denormalize the data and store the results for easy reading. All the heavy lifting should be done by your domain.
This is why you are asking about the messaging. There are several techniques here:
denormalized data in same database, same table, different columns
denormalized data in same database, different table
denormalized data in different database
That's the storage. How about the consistency?:
immediately consistent
eventually consistent
The simplest solution (quick win) is to denormalize your data in your domain and then after saving your domain objects through the repository you immediately save the denomarlized data to the same data store, same table(s), different columns. 100% consistent and you can start reading the denormalized data immediately.
If you really want to you can create a separate bunch of objects to transport that data but it is simpler to just write a simple query layer that returns some data carrying object provided by your data-access framework (in the case of .Net that would be a DataRow/DataTable). Absolutely no reason to get fancy. There will always be exceptions but then you can go ahead and write a data container.
For eventual consistency you will need some form of queuing and related processing. You can roll your own solution or your may opt for a service bus. That is up to you and your time / technical constraints :)
BTW: I have a free open-source service bus here:
Shuttle.Esb
documentation
Any feedback would be welcomed. But any old service bus will do (MassTransit / NServiceBus / etc.).
Hope that helps.
I'm experimenting with the new Umbraco 5 hive, and I'm kinda a bit confused.
I'm plugging in an existing Linq to SQL services layer, which I developed for a webforms site.
I don't know much about the repository pattern, my services handle all connections with the data context, and work very well.
I have made a few repositories that plug in to the hive, and handle conversion of my entities to the Umbraco TypedEntity type.
These repositiories reference my existing services layer, to retrieve, add, update and delete. The services also handle other entity specific functions, which will not be used by the hive.
Now, it's nice to plug in these services, and just reference them in the hive repositories, but it seems I may be doing things the wrong way round, according to the offical repository pattern as I have read about.
I know there's no hard fast rules, but I would appreciate comments on what I'm doing to achieve this functionality.
I've asked this here instead of the Umbraco forum, as I want a wider perspective.
Cheers.
I personally feel that the Hive is overkill. With the ability to use your own classes directly within razor macros, I think the best approach is to forego the hive altogether and simply use your classes. Why would you trade all of the power of your existing service just to make it fit into the hive interface?
If you're writing a library for other Umbraco developers, you may need to do this, but it's my personal opinion that the hive is over-engineered at worst and a layer of abstraction aimed at newish developers at best.
So, if I were to advise you, I would say to consider the more general principles: "Keep It Simple" and "You Aren't Gonna Need It". If the interface they give you offers a tangible benefit, implement it. If not, consider what you really gain for all of that work.
We are using NHibernate Search in an application which is going to be clustered.
I have been reading up on the approaches for maintaining separate collections, in particular the master/slave configuration and I was wondering how to go about implementing it using MSMQ if indeed there is an implementation for this at this time. The JMS implementation (as described in NHibernate Search in Action) seems a little daunting to me, especially as we are using a .NET environment.
Alternatively, I'm open to suggestions with regard to instantiating local RAMDirectories for the lucene collections. I know that Lucene can build a RAMDir from a FSDir and I know how to initialise an NHibernate app with a blank RAMDir but I'm getting a little lost when it comes to initialising an app with a RAMDir from an existing (network shared) FSDir.
Or indeed any other approaches.
Cheers,
Steve
I have actually recently came across this very problem. Primarily because we shared index over several webapps, for realtime updates to the indexes. However, we suffered from index corruption and couldn't really figure out why, aswell as the fact that it wouldn't work in a clustered enviroment.
My approach was this: using a service that schematically indexed new entities at very frequent intervals, aswell as reindexing everything at certain timeintervals. I also run an optimize automatically, since NHSearch doesn't seem to support auto-optimize yet.
At application start, I index everything into a RAMDirectoryProvider.
The choice you make depends highly on the data that you want to index and how sensitive you are to delays in that data, aswell as how frequent it changes. In my case, it was to allow textsearches amongst product-data for a website, so any delay was fine with me.
I did some brief research about Master-Slave providers, however I think I felt that NHSearch is rather imature as opposed to the original Java-implementation.
For me, an optimal solution would be to have a Master-Master provider, that would cross-apply all updates to the index on all nodes. I haven't research how much work it would be to write a DirectoryProvider myself, but that would be an option but much effort aswell.
What are the opinions of the software developers who have used Spring.NET and NHibernate in their projects? In couple of our projects in the company where I work, we have used these third party tools very extensively and we have gained tremendous productivity. A little caveat is that it is a little challenging for junior .Net Developer to grasp the concept.
Imo, Spring.Net and NHibernate go hand in hand: If you master the learning curve you will be very pleased when Spring is used for (almost transparent) transaction handling for NHibernate, Dependency Injection as well as Db:Provider handling.
NHibernate (or similar ORM's) can be extremely helpful with "greenfield" development, where the domain object model can be designed properly, and then the database schema and NHibernate mappings can be code-generated to match the model. On the other hand, NHibernate can be very tough to apply to a legacy or poorly normalized database. Although it has support for handling some unusual data structures, the learning curve is much higher for those techniques.
I can't say much about Spring.Net except that in 2007, our team tried to implement it into an existing, complex solution to handle DB transactions and entity validation, but we scrapped the idea after a week of work. Our resident Spring/Java expert concluded that Spring.Net was a couple versions behind Spring and wasn't mature enough to handle the use cases in our project. Disclaimer: not sure what version of Spring.Net that was, so things may have improved recently.
There has lot of benefits using spring.net/nHibernate.
For data access you are able to take full advantage of nHibernate ORM with lazy loading, database independent programming, easy save load update delete your entities. No need to write any query to do that. That is why after designing a database and after development if any change is required at any state of development it is very much easy to change.
Dependency Injection(DI) helps to separate the UI and business logic. That is why any time you are able to change your graphical user interface without breaking any business logic. Same way if you need any change at business logic then need only build business layer. And it is also configurable.
Aspect Oriented Programming (AOP) is another feature of Spring.Net. It helps logging application, dynamic method interception allow more control over method call. It makes easy to authorize application. Declarative transaction management is a nice feature to manage transactions. You do not need to write code for start stop database connection. Just write at configuration file at which method you want to maintain transaction. So at large scale project it will save lot of time and will give you more flexibility over transaction management.
Spring webservice is very easy to write and consume from a client. Not need to add any custom attribute, no WSDL. You can intercept every method call, you are able to add advice like logging advice, authentication/authorization advice.
Overall it save approximately 33% of development time, make application more maintainable, highly configurable.