int atClass1::read_file
(String^ file_path, /* Path tofile */
HdfCallVars % ret_vals)
This is my function. Within it I have lots of native c++ code. I ran into a serious problem though
/* Iterate through the links, filling in needed data as discovered. */
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)& ret_vals);
will not compile! Says ret_vals is managed and I can't do pointerey ampersandey stuff to it. Am I in trouble? Or is there a way out of my dilemma? The H5 function is a call into the HDF5 library.
thanks,
saroj
In .Net there is no guarantee that an object will remain on it current memory position as the garbage collector will "compact" the heap space when it wants to.
To get a native pointer to a managed object you should "pin" the object:
pin_ptr<HdfCallVars> pinned = &ret_vals;
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)pinned);
Note that the pointer will be unpinned after the variable pinned goes out of scope, if H5Literate stores the pointer for future use, you should pin the value using System::Runtime::InteropServices::GCHandle, like this:
GCHandle ^handle = GCHandle::Alloc(ret_vals);
io_err = H5Literate (group_id, H5_INDEX_NAME, H5_ITER_NATIVE,
&i, get_sonar_data, (void*)handle->AddrOfPinnedObject());
When you don't need the pointer anymore, you should free it:
handle->Free();
Related
Until now, I've always used pixDestroy to clean up PIX objects in my JavaCPP/Leptonica application. However, I recently noticed a weird memory leak issue that I tracked down to a Leptonica function internally returning a pixClone result. I managed to reproduce the issue by using the following simple test:
#Test
public void test() throws InterruptedException {
String pathImg = "...";
for (int i = 0; i < 100; i++) {
PIX img = pixRead(pathImg);
PIX clone = pixClone(img);
pixDestroy(clone);
pixDestroy(img);
}
Thread.sleep(10000);
}
When the Thread.sleep is reached, the RAM memory usage in Windows task manager (not the heap size) has increased to about 1GB and is not released until the sleep ends and the test finishes.
Looking at the docs of pixClone, we see it actually creates a handle to the existing PIX:
Notes:
A "clone" is simply a handle (ptr) to an existing pix. It is implemented because (a) images can be large and hence expensive to
copy, and (b) extra handles to a data structure need to be made with a
simple policy to avoid both double frees and memory leaks. Pix are
reference counted. The side effect of pixClone() is an increase by 1
in the ref count.
The protocol to be used is: (a) Whenever you want a new handle to an existing image, call pixClone(), which just bumps a ref count. (b)
Always call pixDestroy() on all handles. This decrements the ref
count, nulls the handle, and only destroys the pix when pixDestroy()
has been called on all handles.
If I understand this correctly, I am indeed calling pixDestroy on all handles, so the ref count should reach zero and thus the PIX should have been destroyed. Clearly, this is not the case though. Can someone tell me what I'm doing wrong? Thanks in advance!
As an optimization for the common case when a function returns a pointer it receives as argument, JavaCPP also returns the same object to the JVM. This is what is happening with pixClone(). It simply returns the pointer that the user passes as argument, and thus both img and clone end up referencing the same object in Java.
Now, when pixDestroy() gets called on the first reference img, Leptonica helpfully resets its address to 0, but we've now lost the address, and the second call to pixDestroy() receives that null pointer, resulting in a noop, and a memory leak.
One easy way to avoid this issue is by creating explicitly a new PIX reference after each call to pixClone(), for example, in this case:
PIX clone = new PIX(pixClone(img));
I'm working with the NativeCall interface.
The library is going to call my callback function a bunch of times.
That works fine. I can just declare my callback with the right
signature, pass it in as &callback and the library calls the sub just
fine.
It also has the capability to set a payload void *pointer to anything
I want, and it will include that in the call to my callback function.
Can I hide a Perl Str, for example, in the payload and successfully round trip it?
sub set_userdata(Pointer) returns int32 is native { ... }
sub set_callback(&callback(Pointer $userdata --> int32)) returns int32 is native { ... }
sub callback(Pointer $userdata) returns int32 {
my Str $mystring = ???
...
}
my Str $my-userdata-string;
set_userdata(???);
set_callback(&callback);
It seems like it could work with some incantation of binding, "is rw", nativecast() and/or .deref.
You can only use a native representation in such a case (such as CStruct, CArray, and CPointer), or alternatively a Blob. You are also responsible for ensuring that you keep a reference to the thing you pass as userdata alive from Perl 6's perspective also, so the GC doesn't reclaim the memory that was passed to the C function.
Memory management is the reason you can't pass any old Perl 6 object off to a C function: there's no way for the GC to know whether the object is still reachable through some C data structure it can't introspect. In a VM like MoarVM objects are moved around in memory over time as part of the garbage collection process also, meaning that the C code could end up with an out-dated pointer.
An alternative strategy is not not pass a pointer at all, but instead pass an integer and use that to index into an array of objects. (That's how the libuv binding inside of MoarVM tracks down the VM-level callbacks, fwiw.)
I got around this by just ignoring the userdata and making a new closure referencing the Perl object directly for every callback function. Since there is a new closure created every time I set the callback, I think this will leak memory over time.
I have a C++/CLI method, ManagedMethod, with one output argument that will be modified by a native method as such:
// file: test.cpp
#pragma unmanaged
void NativeMethod(int& n)
{
n = 123;
}
#pragma managed
void ManagedMethod([System::Runtime::InteropServices::Out] int% n)
{
pin_ptr<int> pinned = &n;
NativeMethod(*pinned);
}
void main()
{
int n = 0;
ManagedMethod(n);
// n is now modified
}
Once ManagedMethod returns, the value of n has been modified as I would expect. So far, the only way I've been able to get this to compile is to use a pin_ptr inside ManagedMethod, so is pinning in fact the correct/only way to do this? Or is there a more elegant way of passing n to NativeMethod?
Yes, this is the correct way to do it. Very highly optimized inside the CLR, the variable gets the [pinned] attribute so the CLR knows that it stores an interior pointer to an object that should not be moved. Distinct from GCHandle::Alloc(), pin_ptr<> can do it without creating another handle. It is reported in the table that the jitter generates when it compiles the method, the GC uses that table to know where to look for object roots.
Which only ever matters when a garbage collection occurs at the exact same time that NativeMethod() is running. Doesn't happen very often in practice, you'd have to use threads in the program. YMMV.
There is another way to do it, doesn't require pinning but requires a wee bit more machine code:
void ManagedMethod(int% n)
{
int copy = n;
NativeMethod(copy);
n = copy;
}
Which works because local variables have stack storage and thus won't be moved by the garbage collector. Does not win any elegance points for style but what I normally use myself, estimating the side-effects of pinning is not that easy. But, really, don't fear pin_ptr<>.
I'm confused about COM string assignments. Which of the following string assignment is correct. Why?
CComBSTR str;
.
.
Obj->str = L"" //Option1
OR should it be
Obj->str = CComBSTR(L"") //Option2
What is the reason
A real BSTR is:
temporarily allocated from the COM heap (via SysAllocString() and family)
a data structure in which the string data is preceded by its length, stored in a 32-bit value.
passed as a pointer to the fifth byte of that data structure, where the string data resides.
See the documentation:
MSDN: BSTR
Most functions which accept a BSTR will not crash when passed a BSTR created the simple assignment. This leads to confusion as people observe what seems to be working code from which they infer that a BSTR can be initialized just like any WCHAR *. That inference is incorrect.
Only real BSTRs can be passed to OLE Automation interfaces.
By using the CComBSTR() constructor, which calls SysAllocString(), your code will create a real BSTR. The CComBSTR() destructor will take care of returning the allocated storage to the system via SysFreeString().
If you pass the CComBSTR() to an API which takes ownership, be sure to call the .Detach() method to ensure the BSTR is not freed. BSTRs are not reference counted (unlike COM objects, which are), and therefore an attempt to free a BSTR more than once will crash.
If you use str = CComBSTR(L"") you use the constructor:
CComBSTR( LPCSTR pSrc );
If you use str = L"" you use the assignment operator:
CComBSTR& operator =(LPCSTR pSrc);
They both would initialize the CComBSTR object correctly.
Personally, I'd prefer option 1, because that doesn't require constructing a new CComBSTR object. (Whether their code does so behind the scenes is a different story, of course.)
Option 1 is preferred because it only does one allocation for the string where as option 2 does 2 (not withstanding the creation of a new temporary object for no particular reason). Unlike the bstr_t type in VC++ the ATL one does not do referenced counted strings so it will copy the entire string across.
I have a
public ref class Test
inside this class, I have:
int frameWidth;
int frameHeight;
int frameStride;
When I try to compile this, I get the error:
error C2664: 'GetImageSize' : cannot convert parameter 1 from 'cli::interior_ptr<Type>' to 'int *'
GetImageSize is a native function and it works only if I move the declaration of the 3 ints above to outside the class or inside the block that calls GetImageSize.
How can I solve this?
Those 3 ints needs to be accessible by more than one function within the class, right now I made it work because I moved them to outside the class, but it's not the right thing to do I believe since they become global.
According to this post, the reason you are seeing this is because the ints are inside a ref class which can be moved around the heap by the garbage collector at will, the address of the ints could change and you wouldn't be told.
To overcome this, you need to tell the GC not to move the objects while you are using them. To do this you need to use
pin_ptr<int*> pinnedFrameWidth = &frameWidth;
then pass pinnedFrameWidth into GetImageSize. The pin_ptr will be automatically cast to int* when passed into the method.
You need to be careful when using pin_ptr. Because the GC can't move the instance of Test class around during a collection, the managed heap can become fragmented and, eventually, performance can suffer. Ideally pin as few objects for the least amount of time possible.
There is a brief discussion of pin pointers in this .Net Rocks show.