I am trying to understand how memory is managed when python is embedded in C# using python.net. Python is executed in C# using the py.GIL (global interpreter lock), essentially C# calls the python interpreter and executes the python code. What I don't understand is how is memory allocated and deallocated for python objects.
I couldn't find anything online and any type of help will be appreciated.
C# PyObject is holding a Python object handle (which are pointers into C heap). Internally Python objects count references to themselves.
In C# you can either force releasing the handle by calling PyObject.Dispose, in which case Python object refcount is decreased immediately, and if at this point it reaches 0, Python object will be destroyed and deallocated (from C heap).
Alternatively, wait until PyObject instance is collected by .NET garbage collector. When that happens, the Python object handle is added to an internal queue, which is periodically emptied when you create new PyObjects (e.g. PyObject constructor will go check that queue, and release all handles in it).
This applies to Python.NET 2.5 and the upcoming 3.0.
Related
I have a self-hosted WCF application where the service objects (per call, single concurrency) use embedded COM objects (referenced from the class throughout the call) built on a platform called Clarion. The COM instances are created and destroyed together with the service objects. However, underneath they access a singleton repository which takes long time to initialise (very similar to this: Startup Code for Loading COM Object for WCF Service). So a first call is very long. Hence, I must call it on startup.
If I don't instantiate the COM object on startup, everything is fine (except for the long first call). But if I do, after the garbage collection the next request will crash with access violation exception when trying to access the COM.
The service objects implement IDisposable where all the COM objects are nicely released using Marshal.ReleaseComObject.
The startup call also releases the COM object.
My guess is that the startup COM gets somehow reused or recycled. I don't want it to! Can I make sure it dies forever? Or if it's not possible, can I mark it not for garbage collection? Obviously, GC.KeepAlive is irrelevant here, because these are different threads in different methods.
More details: the COM objects were made MTA-capable recently. They are thread-safe and fully concurrent, but before, when they were STA only, there was no such issue. Also, nothing bad happens outside of WCF when these guys run in separate threads.
OK. Looks like I'm on it.
It's the bloody hidden singleton objects or, rather, a runtime library of the platform the COM is written on (SoftVelocity Clarion). It got deallocated for some reason when the startup COM got killed, probably because the reference count went down and it was time to unload the DLL itself. Although when I tweaked DllCanUnloadNow, it did not help, but I'll figure out where it's coming from.
EDIT: Clarion support of COM objects is not straightforward. The code, generated by templates, performs allocation and deallocation of the database dictionary (DctInit and DctKill respectively) and some specific classes in the main program, which ends when the main thread ends. However, in MTA COM objects end of the main thread does not mean end of the program. Hence, the easiest solution is to embed code to prevent execution of DctKill.
Also, do not forget to call AttachThreadToClarion(TRUE) in .Destruct methods, since the garbage collection thread will be different.
This issue may surface in older generation IDEs where a runtime or global objects are used extensively. Beware.
I can't found any reference about Blocks and Garbage Collection; even the "Apple Bocks Reference" doesn't mention anithing about it (just few notes).
I never developed using blocks in a GC enviroment and I would like to know how it works, what is supported and how much "automatic" is the whole process.
Thanks in adv.
As an implementation optimisation blocks are initially stack allocated; this means you can pass them down the stack without issue.
However should you wish to pass them up the stack (as return values) or store them they must first be copied to the heap. Once on the heap normal memory management rules apply, whether you are using MRC, GC or ARC - so they work fine under GC.
To copy them onto the heap you must use block_copy() (or [ copy]) if you are using GC or MRC; under ARC provided you don't cast away the type (e.g. by assigning to a variable of type id) then the block_copy is automatic when storing into a strong variable or returning from a function - but calling it causes no harm so if in doubt you can call it.
There's a reason Garbage Collection isn't discussed in newer documentation, and it isn't that Garbage Collection is too new.
Watch WWDC 2012 session 101, from 1:13:00—1:14:10. This session is under NDA, but you will not use Garbage Collection after watching 70 seconds of that session. (And I'll update this answer once the details go public, which they should when Mountain Lion ships.)
Also, see Chris Lattner's post here. Chris Lattner works for Apple, and posted this before WWDC.
Bottom line:
Don't use garbage collection.
If you're using garbage collection, you'll want to migrate once Xcode 4.4 comes out.
I have a Mac OS X App that I need to store objective-c objects in stl containers (as pointers). Everything works fine until I compile the program with garbage collection "supported" due to a custom framework that I need to use.
Problem is all the objects that get stored in the stl containers get released. Is there a way to tell the compiler to not collect those away and I'll release them when I'm done with them?
Is this possible? Is there a way to have a framework that uses garbage collection and the main program doesn't? That would also solve my problem. I can recompile the framework, but it heavily relies on GC.
You can disable garbage collection for individual objects using NSGarbageCollector's disableCollectorForPointer: method. It will then hang around until you call -enableCollectorForPointer: on it.
I believe (but I'm not totally sure) you can also use CFRetain/CFRelease on NSObject subclasses for more or less the same effect.
I have VS2010 and added a reference to a COM library to my project and VS embedded a primary interop inside the project.
If I reference objects from the COM library and I want to dispose of them quickly without waiting for the GC, is it needed to call ReleaseComObject ?
Marshal.ReleaseComObject provides a way to immediately drop references to this COM object from everywhere it is being consumed within managed code (because it releases the underlying IUnknown from the RCW that is holding it).
As Hans notes, the generally right course is to simply null your object allow the CLR and GC to do the COM object destruction at the appropriate time.
However, in situations where you need immediate action (COM object holds expensive/scarce resources, or perhaps during shutdown where the sequencing is very complex), calling ReleaseComObject can be the right thing to do.
Martyn
There are two types of approaches in terms of cleaning memory.
Reference Counting Algorithms
Garbage Collection
COM components use reference counting algorithm to reclaim memory. Whenever you do not need a reference of a com component you can call it. But Usually my approach is creating a virtual stack and deleting references like the source code below in C#. .NET guarantees as long as RCW is alive COM components is alive. When RCW is garbage collected release method is invoked. It is not necessary to call release in your code. But it does not effect gc cycles.
void DoSth
{
{
RunTimeCallableWrapper area for the code
ReleaseComObject
}
}
Consider an case that I have to call C++ code from my Java Program. The C++ code creates thousands of Objects. Where are these dynamic objects stored ? I suspect in the JVM heap because the native code will be a part of the same process as the JVM.
If yes, do the rules of Java Garbage collector thread apply on Objects of the C++ code ?
For the first question, C++ will allocate resources using its own runtime which has nothing to do with the JVM - the JVM is not aware of any activity in this memory allocator.
For the second question, the Java garbage collector will not GC the memory allocated by C++. You will have to make sure that your Java wrapper initiates the memory release. Before an object is GC'd by java, the runtime calls the finalize() method. The default one is inherited from java.lang.Object and basically does nothing. You can override this and use it as a hook to initiate deallocating your manually managed memory.