I have this administration site that is using (Fluent) NHibernate for data access. As has been stated elsewhere the initialization of the SessionFactory is slow due to creation of the Configuration object. The suggested solution is to serialize the Configuration object to disk and then reuse this untill the entity assembly or fluent initialization assembly changes.
In theory these sound very elegant and appealing. In practice even with a 1-entity model, deserializing the Configuration takes about 500ms and creating the SessionFactory another 500ms. (This is comparable to the startup time with no serialization, but I suspect the gain will be apparent once I put more entities in there.)
After IIS recycles my Application Pool, it doesn't start it up again untill a new request comes in. So with default recycle management of IIS (every 29h), and a probable scenario of an administrator logging into my administration site once a day, that 1s delay will happen almost every time this admin logs in, giving the impression of a slow starting site.
So, I wonder if there is any way to tell IIS/WAS to actually start the replacement Application Pool as part of the recycling instead of waiting untill the next request comes in? I realize this goes against the idea of WAS trying to keep as few concurrent application pools running at any given time, but it would solve my problem (as I assume my ServiceHostFactory+ServiceHost would be created as soon as the Application Pool starts up).
Will IIS 7 autostart feature help you? Also, you can implement System.Web.Hosting.IProcessHostPreloadClient interface to make sure the application is up when the pool is up (see this blog post for an example).
Related
In the context of a windows web service that's meant to run jobs, we try to reuse the NHibernate DAL we developed for the web application.
For session management we have two options, each one having its advantages and drawbacks:
Stateful session
Going to grow a lot as it keeps track of everything (L1/session cache)
Needs to be carefully closed, session disposal doesn't seem to be enough to clear L1 cache (what I noticed using memory profiler)
Stateless Session
Currently fails to reuse mappings. All bags declared with "lazy=true" ends up with the following exception (even though the session has not be closed):
Initializing [...] failed to lazily initialize a collection of role:
[...], no session or session was closed
Obviously, we cannot update the mappings (they are shared with the web app) with lazy="false", it's gonna be a huge drawback for performances
Cannot interact with L2 cache: when shared L2 cache will be deployed, the service will be unable to invalidate L2 cache data in order for web application to have fresh up-to-date data
NHibernate has proven to be good until now, we have successfully used stateful session and NHibernate LINQ it in a web context, with structuremap for dependency injection.
My questions are:
Are there any good solutions to use NHibernate in a long running thread?
I'd prefer to use stateful session, but how to avoid memory leak?
Problem solved! There were actually a couple of problems.
First one was about instances' scope, and multi-threading:
Create a new session for each thread.
As soon as the thread finishes its work, clean all the instances attached to the thread. With StructureMap, within the thread, use new HybridLifecycle().FindCache().DisposeAndClear();. It will cause the session attached to the thread to close and dispose.
When the lifecycle is thread scoped, StructureMap uses a ThreadStatic variable to keep a reference to the object cache. So the trick is to call StructureMap's ObjectFactory within the thread. Initially, in our application, a main thread was responsible for creating new threads, and call the ObjectFactory. That's the major mistake we did, and were indeed unable to clean the threads once their job was done.
Session type:
No need to use a StateLessSession, as soon as the StateFul sessions instantiated are carefully disposed. In our case, StatelessSession have too many drawbacks (cache management is the main)
Important remark: be careful to instantiate NHibernate NHibernate Session Factory only once!
When NHibernate instances are managed carefully, there is no memory leak.
It's never a good idea to keep a stateful session open in a long running process.
My suggestion is to redesign your process to separate database related code from non-database related code so any database related operation can be kept within a short-span session.
Just started reading up on Lucene.net and I would like some of my REST based web services to use the powerful searching facilities of Lucene.net
However I came across a link which said that I should create a windows service (with WCF) to do all the lucene searches/indexes etc as IIS recycles the application pool which will cause all sorts of locking issues.
My question is, is this correct? If so, is there another way of resolving this problem without creating a windows service (with WCF)? Also since I have REST based services, would I make a call from these services to the Windows WCF service which would make things slower?
Indexing
During your reading you would have picked up that indexing is done using the IndexWriter class. Lucene will only allow 1 IndexWriter instance open at a time. When using the default locking it creates a lock file in the index directory and prevents any other IndexWriter instances from being created. For this reason it may be better to implement indexing in a process that you have more control over.
If your indexing process is terminated with extreme prejudice and your IndexWriter class does not get closed, the lock on your index folder is maintained and no other instances will be allowed. Because of this Lucene allows you to lift a lock from an Indexed folder (using IndexWriter.unlock)- a dangerous method because if there are two IndexWriters open on the same index it will corrupt the index. If you have a windows service that is performing the indexing, and it's the only process in your solution that does the indexing (and any updates), you can confidently unlock the indexing folder on startup of the service. In a web service based environment where you are performing indexing from a web method - controlling and recovering from locking issues becomes problematic.
Searching
The IndexSearcher class is used for the searches. This in readonly mode can be done from your service based code. I don't think it's necessary to create a separate set of WCF methods for this purpose.
Optimization
The index may required to be optimized for performance periodically depending on the volumes. Once again having the indexing in a separate process you can schedule the optimization nightly, weekly or what ever is required. Optimization is done by a call to one method.
Indexing new data
How and when to get the indexing process to index new data.... I don't know what data you're indexing so it's hard to tell. In my scenario I have WCF methods that are responsible for input data - high volume. I require the data that has been received to be available for searching as soon as possible. So,
my Model layer has a notification layer that when new records of the required type have been successfully committed, a simple notification message is inserted into a local queue in MSMQ.
The reason for MSMQ is that the queue is persisted and transactional and that any messages in there are available even after a crash of system reboot - allowing me to never (cough!) lose any messages.
The indexing service takes the notification, build the Lucene Document and indexes the data.
The indexing service can also be triggered to do a full re-index by deleting the existing index an crawling the Db.
EDIT:
Example architecture:
WCF Service Methods taking on data commiting it to the Model layer. The Model layer notifies a listening client that an CRUD operation occurred successfully on items. The listening client posts the notification in a queue.
Windows Service handles Indexing of data, watching the queue for indexing requests.
ASP.Net app provides user interface with search features.
You can simply disable application pool recycling and host your application/service in IIS.
To disable recycling on config changes, use the disallowRotationOnConfigChange parameter.
You can also split your application in two parts: Index updates and searches.
Handle index updates from a windows service, and have your IIS portion handles searches (readonly). You would do this by having a mechanism that detects index updates, and refresh the IndexSearchers. This way, if the performance penalty of using services is a concern for you, it wont impact search time which is the important aspect for the users. With this configuration you can even have a master index update node, and distribute searches across different web servers in a farm. The only downside is you dont have the near real time searching functionality thats built in the IndexWriter class.
http://wiki.apache.org/lucene-java/NearRealtimeSearch
That being said, I've never had performance issues with setups that have the Lucene functions exposed over a WCF service, especially if your running either on the same machine with NetNamedPipe or on a local LAN with NetTcp.
I have created an Odata service that i use for my IPad application to talk to an SQL Server database. The problem is the first time the odata server is accessed every morning (with no-one using it over night) it takes along time to connect. Once the first connection is connected than all connections are instant after that.
Does anyone know what i need to do to stop this from happening? I dont mind extending the timeout of an app pool if needed.
Thanks
Two options
1) Keep app alive simply by making occasional calls (Every 5 mins or so). I generally just write a quick vbs or js file to make this call and schedule a task
or (probably cleaner, but haven't done this yet)
2) See ScottGu's 4.0 approach here
I am just wondering why is restarting IIS 7 application pool consider a good practice? I know that it'll clean up orphan resources like threads or session state, but how does other web server deal with this kind of problem? Do java developer/admin have to restart weblogic/websphere/apache/tomcat every so often to clean up junk in memory?
Typically it's used to compensate for bad programming. Sometimes yours, sometimes third parties depending on the libraries you are using.
And, as with all recommendations, it's best to look at your specific environment to see if it's really necessary before doing it.
Start with profiling the app. A simple way is to just let it run for a while under load. Look for things like does memory utilization settle down or does it continually grow?
If it keeps growing, you will need to set IIS to recycle every so often and go fix your app. If it grows to a point then settles down then you should be just fine. Make sure you test even the edge cases of your app.
With IIS it is possible to have non-managed code - the best example is a COM object - that don't behave well. As a result you can set your IIS to recycle processes periodically, to allow for the memory leakage that might occur with such code.
It's not necessary to do it though. You can eliminate the restart thresholds, if you prefer not to restart the processes.
If this idea doesn't exist in other servers, then it may be because other servers allow only managed code.
We are experiencing this issue approximately once a month. It is very hard to pinpoint the cause so any help would be appreciated. This causes the App pool to stop and brings the site down. We have gone through all log files and have concluded nothing. We are using the 2.0.3 version on IIS 6.
I've noticed IIS defaults web apps on a 29-hour recycle schedule, which can be troublesome since it may recycle at times your users do not expect it to.
For example: web app starts at 12 am, which means the next day it recycles at 5am, the day after that at 10am, the day after that at 3pm, etc. (this is assuming there is enough request activity against your app to keep it alive so it does not shutdown due to inactivity)
If your web app relies heavily on in-memory session state this is especially bad because the recycle will kill sessions and possibly force users to re-authenticate and lose any unsaved work. (if you don't design your app to work seamlessly with recycling)
Check the recycle schedule and make sure it recycles at a time that you expect. See this for screenshots: http://remy.supertext.ch/2010/08/iis7-worker-process-reached-its-allowed-processing-time-limit/
Not sure about the infinite loop suggestion... sounds like you just have a recycling configuration issue to resolve.
This likely indicates an infinite loop in your application code.
Basically, every time a request comes into the web server, IIS hands the request off to a worker process. You can configure in IIS how many of those workers there are, and what the timeout value is. The timeout is to keep things moving in case the application code hangs -- it gets killed so the thread can go back in the pool to keep servicing new requests.
So look through your code for likely infinite loops. Or alternatively, it could be an extremely long-running database query that could have eventually finished but exceeded the timeout value. Perhaps your web application offers the end user an opportunity to make too broad of a query that returns too much data or requires too much DB processing time.
It's hard to give a specific cause for you, of course, but try to think along these lines.
If you're experiencing a crash as a result (sounds like you are) then you might want to grab a copy of Debugging Tools for Windows and spend some time reading Tess Ferrandez' blog--she offers great advice on performing post mortem crash analysis and makes WinDbg a whole lot more approachable.