I have a self-hosted WCF application where the service objects (per call, single concurrency) use embedded COM objects (referenced from the class throughout the call) built on a platform called Clarion. The COM instances are created and destroyed together with the service objects. However, underneath they access a singleton repository which takes long time to initialise (very similar to this: Startup Code for Loading COM Object for WCF Service). So a first call is very long. Hence, I must call it on startup.
If I don't instantiate the COM object on startup, everything is fine (except for the long first call). But if I do, after the garbage collection the next request will crash with access violation exception when trying to access the COM.
The service objects implement IDisposable where all the COM objects are nicely released using Marshal.ReleaseComObject.
The startup call also releases the COM object.
My guess is that the startup COM gets somehow reused or recycled. I don't want it to! Can I make sure it dies forever? Or if it's not possible, can I mark it not for garbage collection? Obviously, GC.KeepAlive is irrelevant here, because these are different threads in different methods.
More details: the COM objects were made MTA-capable recently. They are thread-safe and fully concurrent, but before, when they were STA only, there was no such issue. Also, nothing bad happens outside of WCF when these guys run in separate threads.
OK. Looks like I'm on it.
It's the bloody hidden singleton objects or, rather, a runtime library of the platform the COM is written on (SoftVelocity Clarion). It got deallocated for some reason when the startup COM got killed, probably because the reference count went down and it was time to unload the DLL itself. Although when I tweaked DllCanUnloadNow, it did not help, but I'll figure out where it's coming from.
EDIT: Clarion support of COM objects is not straightforward. The code, generated by templates, performs allocation and deallocation of the database dictionary (DctInit and DctKill respectively) and some specific classes in the main program, which ends when the main thread ends. However, in MTA COM objects end of the main thread does not mean end of the program. Hence, the easiest solution is to embed code to prevent execution of DctKill.
Also, do not forget to call AttachThreadToClarion(TRUE) in .Destruct methods, since the garbage collection thread will be different.
This issue may surface in older generation IDEs where a runtime or global objects are used extensively. Beware.
Related
I have a static object that needs to initialize an imaging API. The allocated resources of this imaging API need to be released by the same thread.
So I'm starting a thread in my static object that initializes everything and then waits for a counter to reach zero. When this happens the thread cleans all up and finishes.
This is an unmanaged class inside a managed library, so I can't use System::Threading::Thread (needs a managed static member function) or std::thread (compiler error, not supported with /clr).
So I have to start my thread like:
CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)&Initialize, this, 0, 0);
All works fine, the init is done and the API functions work. But when I close the application I see that the usage counter of my static object reaches zero but the clean up function is never called by the thread, as if the thread was killed. Is there a way to make sure the thread will continue to exist and execute until its end?
After turning this around in all possible ways and adding events etc I guess this is not possible so I'll have to change the structure of my code and encapsulate the non managed class inside a managed class, and add the thread to the managed class.
I think you could proceed in one of two ways:
Wrap the resources in RAII-style classes, and refactor to have the objects' lifetimes be on the stack of your created thread, ensuring their destructors get called when the thread loop exits without having to call any additional cleanup. If there is no issue with the thread returning correctly when your counter reaches 0, this should be the simplest and cleanest way of addressing this.
I'm thinking you could intercept the WM_CLOSE message using window procedures, process necessary cleanup and then pass the message on, effectively "stalling" it until you are ready to close. Note that even though you are in a DLL you can still set up a window procedure and message pump system, you don't need a GUI to do that. I am however not 100% sure on whether you'll receive the WM_CLOSE message that concerns the application that "owns" your DLL, it's not something I've tried out yet.
You will have to implement some form of messaging through events within your thread's loop however, as the WindowProc will be called on a different thread, so you know when to call the cleanup procedure.
I also am not very familiar with CLR, so there might be a simpler way of interacting with those APIs than with raw C++ calls and handles.
I have VS2010 and added a reference to a COM library to my project and VS embedded a primary interop inside the project.
If I reference objects from the COM library and I want to dispose of them quickly without waiting for the GC, is it needed to call ReleaseComObject ?
Marshal.ReleaseComObject provides a way to immediately drop references to this COM object from everywhere it is being consumed within managed code (because it releases the underlying IUnknown from the RCW that is holding it).
As Hans notes, the generally right course is to simply null your object allow the CLR and GC to do the COM object destruction at the appropriate time.
However, in situations where you need immediate action (COM object holds expensive/scarce resources, or perhaps during shutdown where the sequencing is very complex), calling ReleaseComObject can be the right thing to do.
Martyn
There are two types of approaches in terms of cleaning memory.
Reference Counting Algorithms
Garbage Collection
COM components use reference counting algorithm to reclaim memory. Whenever you do not need a reference of a com component you can call it. But Usually my approach is creating a virtual stack and deleting references like the source code below in C#. .NET guarantees as long as RCW is alive COM components is alive. When RCW is garbage collected release method is invoked. It is not necessary to call release in your code. But it does not effect gc cycles.
void DoSth
{
{
RunTimeCallableWrapper area for the code
ReleaseComObject
}
}
Having trouble with memory management of a third party library. I have the source but it's very complex (COM stuff), full of macros and those annoying Microsoft annotations, etc, and interacts with another library the source of which I don't have to boot. Now some quick debug runtimes have shown that it's leaking memory and in a pretty big way. I make extensive use of self-releasing pointers like unique_ptr and know that I released everything that I created. Is it my only option to try and clean up (and understand) the source?
In addition, is it safe to allocate COM objects with operator new, or do they have to go in the COM heap?
COM is quite agnostic about how you allocate your own COM objects. They get created by the class factory and your IUnknown::AddRef and Release methods keep the reference count. Using operator new in the class factory and delete this in Release is fine.
You do have to be careful about any pointers you return in your interface methods. Typical Automation objects like BSTR and SAFEARRAY do indeed need to be allocated in the COM heap so that the client code can release them. Pretty hard to mess that up, the API does the work.
The client code can be responsible for the leak, fumbling the reference count is a pretty standard COM bug. Usually easy to diagnose when you have access to the AddRef/Release implementations. Debugging this in Vista or Win7 is also strongly recommended, they have a much better heap manager that doesn't silently ignore attempts to free memory from the wrong heap.
If you're pretty sure it is the COM server that leaks then isolate the problem with the <crtdbg.h> header and unit tests to exercise the interface methods.
I was experimenting with ways to get rid of some memory leaks within my application the other day when I realized that I know virtually nothing about cleaning up my resources. I did some research, and hoped that just calling the .dispose() would solve all of my problems. We have a table in our database that contains about 65,000 records. Obviously when I fill my dataset from the data adapter, the memory usage can get pretty high. When I called the dispose method on the dataset, I was surprised to find out that NONE of the memory got released. Why did this happen? Clearing the dataset doesn't help either.
IDisposable and thus Dispose is not used to reduced memory pressure, although in some cases it might, but instead used for deterministic cleanup.
Consider this, you construct an object that maintains an active and open connection to your database server. This connection uses resources, both on your machine, and the server.
You could of course just leave the object be when you're done with it, and eventually it'll get picked up by the garbage collector, but suppose you want to make sure at least the resources gets freed, and thus the connection closed, when you're done with it. This is where IDisposable.Dispose comes into play.
It is used to clean up resources managed by the object.
It will, however, not free the managed memory allocated to the object. This is still left to the garbage collector, that will kick in at some later time to do that.
Do you actually have a memory problem, or do you just look at the memory usage in Task Manager or similar and go "that's a bit high."?
If the latter, then you should just leave it be for now. .NET will run garbage collection more often if you have less memory available, so unless you're in a situation where you get, or might suspect you will get soon, a memory overflow condition, you're probably not going to have any problems.
Let me explain what I mean by "run less often".
If you have 8GB of memory in your machine, and only have Windows and Notepad running, most of that memory will be available. When you now run your program, even if it loads minor data blocks into memory, you can keep doing that for a long time, and memory usage will steadily grow. Exactly when the GC will kick in and try to reduce your memory footprint I don't know, but I can almost guarantee you that you will wonder why it gets so high.
Let's just for the sake of the argument say that your program will eventually use 2GB of memory.
Now, if you run your program on a machine that has less memory available, GC will occur more often, and will kick in on a lower limit, which might keep the memory usage below 500MB or possibly even less.
The important part to note here is that in order for you to get an accurate picture of how much memory application actually requires, then you can't rely on Task Manager or similar ways to measure it, you need something more targetted.
Calling Dispose() will only release unmanaged resources, such as file handles, database connections, unmanaged memory, etc. It will not release garbage collected memory.
Garbage collected memory will only get released at the next collection. Usually when the application domain memory is deamed full.
I'm going to point out something here that hasn't been explicitly mentioned: calling Dispose() will only clean up (free) unmanaged resources if the developer of the component has coded it.
What I mean is this: if you suspect you have a memory leak, calling Dispose() is not going to fix it if the original developer has done a lousy job and not correctly freed up unmanaged resources. For a bit more info, check this blog post. Take note of the statement The behaviour of Dispose is defined by the developer.
Some objects will ask one or more other entities to do something on its behalf until further notice, to the detriment of other entities. If an object which did so were to disappear without informing the former entities that their services were no longer needed, those entities would continue to uselessly act on behalf of an object that no longer needed them, to the continuing detriment of other entities that would want to use them.
In many cases, for an object "George" to tell an outside entity "Joe" that its services were no longer needed, George would have to know that its services were no longer needed. There are two normal means via which that can happen in .NET, finalization and IDIsposable.
If an object overrides a method called Finalize, then when the object is created the .NET garbage collector will add it to a list of objects with registered finalizers. If the GC discovers that there exists no rooted reference to the object other than that list, the GC will remove the object from that list and add it to a strongly-rooted queue of objects which should have their Finalize method called as soon as possible. Such an object can then use its Finalize method to inform other entities that their services are no longer required.
Although finalization-based cleanup can sometimes work, there's no guarantee of timeliness. At one point during the design of .net Microsoft may have intended that finalization would be the primary cleanup method, but for a variety of reasons it cannot safely be relied upon.
The other cleanup approach, which should be the focus of one's efforts, is IDisposable. Basically, the idea behind IDisposable is simple: for every object that implements IDisposable, there should be one entity (generally either an object or a nested execution scope) which is responsible for ensuring that that object's IDisposable.Dispose method will get called sometime within the lifetime of the universe (which would imply sometime while a reference to the object still exists), and preferably as soon as code can tell that the object's services will no longer be required.
Note that IDisposable.Dispose generally promises that any outside entities which had been asked to do something on an object's behalf will be told that they no longer need to do so, but such a promise does not imply that the number of entities is non-zero. If an object hasn't asked any outside entities to do anything on its behalf, then delivering a message "all" such entities doesn't require doing anything at all. On the other hand, the fact that a Dispose method may do nothing in some cases doesn't mean that it's guaranteed never to do anything in any case, nor that failure to call it in those cases where it would do something won't have detrimental effects.
A lot of what I have learned about VB I learned from using Static Code Analysis (Particularly Aivosto's Project Analyzer). And one one of things it checks for is whether or not you cleared all objects and arrays. I used to just do this blindly because PA said so. But now that I know a little bit more about the way VB releases resources, it seems to me that these things should be happening automatically. Is this a legacy feature from pre VB6, or is there a reason why you should explicitly set objects back to nothing and use Erase on arrays?
Matt Curland, author of Advanced Visual Basic 6, who knows more about Visual Basic than most of us ever will, thinks it is wasted effort. Consider this quote (p110) about DAO, the COM data access library that primarily targets the Access Database Engine:
another example of poor teardown code.
DAO has Close methods that must be
called in the correct order, and the
objects must be released in the
correct order as well (Recordset
before Database, for example). This
single poor object model behavior has
led to the misconception that VB leaks
memory unless you explicitly set all
the local variables to nothing at the
end of a function. This is a
completely false notion in a
well-designed object model. VB can
clear the variables faster at the End
Sub line than you can from code, and
it checks the variables even if you
explicitly release your references.
Any effort you make is duplicated.
The problem, as I understand it, has to do with the fact that VB6 (and its predecessors) has its roots in COM, and its reference-counting garbage collection system.
Imagine, for instance, that you declare a refernece to an object from a 3rd party library. That object has a COM reference count that is used both to keep it alive and to determine when it should be destroyed. It isn't destroyed when you set it to Nothing, but when the object's reference count reaches zero.
Now, not all COM components were written in Visual Basic. Some were written in C or C++. Structured exception handling didn't exist across all languages. So if an error occurred, the reference count on the object was not guaranteed to be properly reduced, and COM objects were known to hang around longer than they were intended to. This wasn't a problem with Visual Basic, per se. It was a COM problem. (And that, you might note, is why .NET doesn't use reference counting.)
That's why Visual Basic developers became obsessive about releasing object references prior to exiting routines. You simply don't know what a component you're allocating is creating under the hood. But when you release your reference to it, you're at least releasing your reference count to it. It became almost a religious mantra. Declare, use, release. It was the COM way of doing things.
Sure, Visual Basic might be better or faster at dereferencing variables I declared on the stack. But dammit, I want it to be OBVIOUS that those objects were released. A little assurance goes a long way when you're trying to track down a memory leak.
Have you read this Aivosto web page (from the creators of Project Analyzer)?
If you are using static variables,
it's important to reclaim the memory
they occupied when you don't need the
variables any more. With dynamic
variables memory isn't so much of a
problem, because they are destroyed
when the procedure ends.
In other words, you don't need to worry about clearing ordinary, non-static, local variables.
I always do it for good practice, you never know what an exception might do if you fall in one and your objects are not deallocated. You should relase them in finally statements and ensure they are not using any memory otherwise you may run into a memory leak.
I had an issue inside of a simple time off tracker system where the server kept on crashing randomly, it took weeks to determine it was a memory leak of an object that was supposed to self destruct on its own. My code was being thrown into an exception and never cleaned up after itself causing the server (the actual web site not the entire server) to go down.
Yes, set all objects to Nothing and clean up as much as you can. VB6 is notorious for having memory leaks when not cleaning up your stuff. Garbage collection was sub-par in VB6/VBA.