(C++/CLI) How to get callbacks from Native Code to Managed Code in C++ CLI? - c++-cli

RANT-BEGIN
Before jumping right into already answered band wagon, please read this paper about SE outdated answers https://ieeexplore.ieee.org/document/8669958
Things changes after a time, and I am afraid Computer science is one of the most if not the most field out there where APIs and Interfaces change radically very very fast. Needless to say that a solution that might worked last month might not after latest feature added to a platform/framework. I humbly request you to not mark this question as answered with decade old post when many mainstream things did not even existed. If you dont know latest solution dont bother about it and leave question for someone else who might.
For a community representative of Computer Science where innovations is everyday thing, it is very toxic, new comer unfriendly and conservative.
END-RANT
This question has already been answered by me and will be accepted tomorrow (SE policy). Thank you for your interest.
Many times you have function pointers in unmanaged context which are called by some kind of events, We will see how it can be achieved with Top-Level Functions and also with member functions of a managed class.
Again, Please dont mark it as answered by linking to a decade old posts.
PS:
So many edits due to unstable internet in third world country, yeah bite me!

unmanaged.cpp
#pragma unmanaged
// Declare an unmanaged function type that takes one int arguments and callbacks
// our function after incrementing it by 1
// Note the use of __stdcall for compatibility with managed code
// if your unmanaged callback uses any other calling convention you can
// UnmanagedFunctionPointerAttribute (check msdn for more info) on your delegate
typedef int(__stdcall* ANSWERCB)(int);//Signature of native callback
int TakesCallback(ANSWERCB fp, int a) {
if (fp) {
return fp(a+1);//Native Callback
}
// This code will be executed when passed without fp
return 0;
}
#pragma managed
managed.cpp
using namespace System;
using namespace System::Runtime::InteropServices;
namespace Callbacks {
// Following delegate is for unmanaged code and must match its signature
public delegate void MyNativeDelegate(int i);
// This delegate is for managed/derived code and ideally should have only managed parameters
public delegate void MyManagedDelegate(int i);
public ref class TestCallback {// Our demo Managed class
private:
GCHandle gch;// kept reference so that it can be freed once we are done with it
void NativeCallbackListener(int i);//unmanaged code will call this function
public:
void TriggerCallback(int i); // Its here for demo purposes, usually unmanaged code will call automatically
event MyManagedDelegate^ SomethingHappened;//plain old event
~TestCallback();//free gch in destructor as its managed.
};
};
void Callbacks::TestCallback::NativeCallbackListener(int i) {
// Callback from Native code,
// If you need to transform your arguments do it here, like transforming void* to somekind of native structure.
// and then pass SomethingHappened::raise with Managed Class/Struct
return SomethingHappened::raise(i); // similar to SomethingHappened.Invoke() in c#
}
void Callbacks::TestCallback::TriggerCallback(int i)
{
MyNativeDelegate^ fp = gcnew MyNativeDelegate(this, &TestCallback::NativeCallbackListener);
// use this if your nativecallback function is not a member function MyNativeDelegate^ fp = gcnew MyNativeDelegate(&NativeCallbackListener);
gch = GCHandle::Alloc(fp);
IntPtr ip = Marshal::GetFunctionPointerForDelegate(fp);
ANSWERCB cb = static_cast<ANSWERCB>(ip.ToPointer());// (ANSWERCB)ip.ToPointer(); works aswell
// Simulating native call, it should callback to our function ptr NativeCallbackListener with 2+1;
// Ideally Native code keeps function pointer and calls back without pointer being provided every time.
// Most likely with a dedicated function for that.
TakesCallback(cb, i);
}
void Callbacks::TestCallback::~TestCallBack() {
gch.Free();//Free GCHandle so GC can collect
}
implementation.cpp
using namespace System;
void OnSomethingHappened(int i);
int main(array<System::String^>^ args)
{
auto cb = gcnew Callbacks::TestCallback();
cb->SomethingHappened += gcnew Callbacks::MyManagedDelegate(&OnSomethingHappened);
cb->TriggerCallback(1);
return 0;
}
void OnSomethingHappened(int i)
{
Console::WriteLine("Got call back with " + i);
}

Related

C++/CLI pin_ptr's usage in proper way

I am newbie of C++/CLI.
I already know that the pin_ptr's functionality is making GC not to learn to specified object.
now let me show you msdn's example.
https://msdn.microsoft.com/en-us//library/1dz8byfh.aspx
// pin_ptr_1.cpp
// compile with: /clr
using namespace System;
#define SIZE 10
#pragma unmanaged
// native function that initializes an array
void native_function(int* p) {
for(int i = 0 ; i < 10 ; i++)
p[i] = i;
}
#pragma managed
public ref class A {
private:
array<int>^ arr; // CLR integer array
public:
A() {
arr = gcnew array<int>(SIZE);
}
void load() {
pin_ptr<int> p = &arr[0]; // pin pointer to first element in arr
int* np = p; // pointer to the first element in arr
native_function(np); // pass pointer to native function
}
int sum() {
int total = 0;
for (int i = 0 ; i < SIZE ; i++)
total += arr[i];
return total;
}
};
int main() {
A^ a = gcnew A;
a->load(); // initialize managed array using the native function
Console::WriteLine(a->sum());
}
hear is the question.
Isn't it okay, the passed object(arr) not pinned ?
because the unmanaged code(native_function) is sync operation and finished before the C++/CLI code (load)
is there any chance the gc destory arr, even though the main logic is running?
(I think A is main's stack variable and arr is A's member variable, so while running main, it should visible)
if so, how can we guarantee that the A is there before invoking load?
(only while not running in native-code?)
int main() {
A^ a = gcnew A;
// I Think A or arr can be destroyed in here, if it is able to be destroyed in native_function.
a->load();
...
}
Thanks, in advance.
The problem that is solved by pinning a pointer is not a normal concurrency issue. It might be true that no other thread will preempt the execution of your native function. However, you have to count in the garbage collector, which might kick in whenever the .NET runtime sees fit. For instance, the system might run low on memory, so the runtime decides to collect disposed objects. This might happen while your native function executes, and the garbage collector might relocate the array it is using, so the pointer you passed in isn't valid anymore.
The golden rule of thumb is to pin ALL array pointers and ALL string pointers before passing them to a native function. ALWAYS. Don't think about it, just do it as a rule. Your code might work fine for a long time without pinning, but one day bad luck will strike you just when it's most annoying.

Pass callbacks with managed parameters and return types between managed and unmanaged C++

I am writing a native C++ project with a managed C++ wrapper that can receive and invoke callbacks from C#. The managed side should be able to retrieve back that callback and trigger on the managed environment as well.
The signature of the said callback is:
// Native C++
typedef EvaluateResult (*NativeFunction) (std::vector<EvaluateResult> args);
// Managed C++ wrapper
delegate EvaluateResultWrapper^ ManagedFunction (List<EvaluateResultWrapper^> args);
The EvaluateResultWrapper is the managed wrapper class for the native class EvaluateResult. The conversion between the EvaluateResult is:
EvaluateResult result;
EvaluateResultWrapper^ wrapper = gcnew EvaluateResultWrapper (result);
result = EvaluateResult (*wrapper.original);
I want to implement the constructor EvaluateResultWrapper::EvaluateResultWrapper (ManagedFunction^ func) that can roughly do the following:
// NOTE: Pseudo code
void EvaluateResultWrapper::EvaluateResultWrapper (ManagedFunction^ func) {
this->func = func; // Store the func as a member to avoid GC
// original is the pointer to the EvaluateResult that this object is wrapping around
this->original = new EvaluateResult ([&func](std::vector<EvaluateResult> args) -> EvaluateResult {
List<EvaluateResultWrapper^>^ argsList; // Convert args from vector to List. Assuming it is done under the hood
EvaluateResultWrapper^ wrapper = func->Invoke (argsList); // Invoke the managed callback
return EvaluateResult (wrapper.GetOriginal ()); // Convert the managed result to the native counterpart
});
}
I know the above code will not work, but the idea I should be able to wrap the managed callback with codes that able to do conversion of both the callback arguments and return types, so that it is native friendly.
Ideally, I can also do the other way around (not important)
// NOTE: Pseudo code
ManagedFunction^ EvaluateResultWrapper::GetFunction (ManagedFunction^ func) {
// if the callback is set by the managed side, return the same callback back
if (this->func != nullptr) return this->func;
// Otherwise, the callback is a native one
NativeFunction nativeFunc = this->original->GetFunction ();
return gcnew ManagedFunction ([&nativeFunc] (List<EvaluateResultWrapper^>^ args) -> EvaluaResultWrapper {
std::vector argsList; // Convert the args from List back to vector. Assuming it is done under the hood
EvaluateResult result = nativeFunc (argsList); // Invoke the native function
return gcnew EvaluateResultWrapper (result); // Convert the native result into the managed one
});
}
I wonder whether this can be done?
A little bit of context: I am writing an external scripting system on native C++ for our games (similar to Lua scripting). The EvaluateResult is a class representing an evaluation result of any statement. It is basically a value coupled with the type. The type can be either number, boolean, string, array, object, or in this case: function callback.
The function callback can be either set within the native C++ (when the interpreter pass the user-defined function in the scripts) or a function set by the host (managed side).
The idea is the host (C# side) should be able to define and set functions into the memory (defining print() function to print into the host console for example). The callback is wrapped as an EvaluateResult class before storing into the the scripting memory.
For inspection purpose, C# side must be able to get the function callback. Therefor, the ability to get the function is nice to have (but not important, since I can always instruct the native side to execute the function for me)

Is it possible to cast a managed bytes-array to native struct without pin_ptr, so not to bug the GC too much?

It is possible to cast a managed array<Byte>^ to some non-managed struct only using pin_ptr, AFAIK, like:
void Example(array<Byte>^ bfr) {
pin_ptr<Byte> ptr = &bfr[0];
auto data = reinterpret_cast<NonManagedStruct*>(ptr);
data->Header = 7;
data->Length = sizeof(data);
data->CRC = CalculateCRC(data);
}
However, is with interior_ptr in any way?
I'd rather work on managed data the low-level-way (using unions, struct-bit-fields, and so on), without pinning data - I could be holding this data for quite a long time and don't want to harass the GC.
Clarification:
I do not want to copy managed-data to native and back (so the Marshaling way is not an option here...)
You likely won't harass the GC with pin_ptr - it's pretty lightweight unlike GCHandle.
GCHandle::Alloc(someObject, GCHandleType::Pinned) will actually register the object as being pinned in the GC. This lets you pin an object for extended periods of time and across function calls, but the GC has to track that object.
On the other hand, pin_ptr gets translated to a pinned local in IL code. The GC isn't notified about it, but it will get to see that the object is pinned only during a collection. That is, it will notice it's pinned status when looking for object references on the stack.
If you really want to, you can access stack memory in the following way:
[StructLayout(LayoutKind::Explicit, Size = 256)]
public value struct ManagedStruct
{
};
struct NativeStruct
{
char data[256];
};
static void DoSomething()
{
ManagedStruct managed;
auto nativePtr = reinterpret_cast<NativeStruct*>(&managed);
nativePtr->data[42] = 42;
}
There's no pinning at all here, but this is only due to the fact that the managed struct is stored on the stack, and therefore is not relocatable in the first place.
It's a convoluted example, because you could just write:
static void DoSomething()
{
NativeStruct native;
native.data[42] = 42;
}
...and the compiler would perform a similar trick under the covers for you.

Accessing a C/C++ structure of callbacks through a DLL's exported function using JNA

I have a vendor supplied .DLL and an online API that I am using to interact with a piece of radio hardware; I am using JNA to access the exported functions through Java (because I don't know C/C++). I can call basic methods and use some API structures successfully, but I am having trouble with the callback structure. I've followed the TutorTutor guide here and also tried Mr. Wall's authoritative guide here, but I haven't been able to formulate the Java side syntax for callbacks set in a structure correctly.
I need to use this exported function:
BOOL __stdcall SetCallbacks(INT32 hDevice,
CONST G39DDC_CALLBACKS *Callbacks, DWORD_PTR UserData);
This function references the C/C++ Structure:
typedef struct{
G39DDC_IF_CALLBACK IFCallback;
//more omitted
} G39DDC_CALLBACKS;
...which according to the API has these Members (Note this is not an exported function):
VOID __stdcall IFCallback(CONST SHORT *Buffer, UINT32 NumberOfSamples,
UINT32 CenterFrequency, WORD Amplitude,
UINT32 ADCSampleRate, DWORD_PTR UserData);
//more omitted
I have a G39DDCAPI.java where I have loaded the DLL library and reproduced the API exported functions in Java, with the help of JNA. Simple calls to that work well.
I also have a G39DDC_CALLBACKS.java where I have implemented the above C/C++ structure in a format works for other API structures. This callback structure is where I am unsure of the syntax:
import java.util.Arrays;
import java.util.List;
import java.nio.ShortBuffer;
import com.sun.jna.Structure;
import com.sun.jna.platform.win32.BaseTSD.DWORD_PTR;
import com.sun.jna.win32.StdCallLibrary.StdCallCallback;
public class G39DDC_CALLBACKS extends Structure {
public G39DDC_IF_CALLBACK IFCallback;
//more omitted
protected List getFieldOrder() {
return Arrays.asList(new String[] {
"IFCallback","DDC1StreamCallback" //more omitted
});
}
public static interface G39DDC_IF_CALLBACK extends StdCallCallback{
public void invoke(ShortBuffer _Buffer,int NumberOfSamples,
int CenterFrequency, short Amplitude,
int ADCSampleRate, DWORD_PTR UserData);
}
}
Edit: I made my arguments more type safe as Technomage suggested. I am still getting a null pointer exception with several attempts to call the callback. Since I'm not sure of my syntax regarding the callback structure above, I can't pinpoint my problem in the main below. Right now the relevant section looks like this:
int NumberOfSamples=65536;//This is usually 65536.
ShortBuffer _Buffer = ShortBuffer.allocate(NumberOfSamples);
int CenterFrequency=10000000;//Specifies center frequency (in Hz) of the useful band
//in received 50 MHz wide snapshot.
short Amplitude=0;//The possible value is 0 to 32767.
int ADCSampleRate=100;//Specifies sample rate of the ADC in Hz.
DWORD_PTR UserData = null;
G39DDC_CALLBACKS callbackStruct= new G39DDC_CALLBACKS();
lib.SetCallbacks(hDevice,callbackStruct,UserData);
//hDevice is a handle for the hardware device used-- works in other uses
//lib is a reference to the library in G39DDCAPI.java-- works in other uses
//The UserData is a big unknown-- I don't know what to do with this variable
//as a DWORD_PTR
callbackStruct.IFCallback.invoke(_Buffer, NumberOfSamples, CenterFrequency,
Amplitude, ADCSampleRate, UserData);
EDIT NO 2:
I have one callback working somewhat, but I don't have control over the buffers. More frustratingly, a single call to invoke the method will result in several runs of the custom callback, usually with multiple output files (results vary drastically from run to run). I don't know if it is because I am not allocating memory correctly on the Java side, because I cannot free the memory on the C/C++ side, or because I have no cue on which to tell Java to access the buffer, etc. Relevant code looks like:
//before this, main method sets library, starts DDCs, initializes some variables...
//API call to start IF
System.out.print("Starting IF... "+lib.StartIF(hDevice, Period)+"\n")
G39DDC_CALLBACKS callbackStructure = new G39DDC_CALLBACKS();
callbackStructure.IFCallback = new G39DDC_IF_CALLBACK(){
#Override
public void invoke(Pointer _Buffer, int NumberOfSamples, int CenterFrequency,
short Amplitude, int ADCSampleRate, DWORD_PTR UserData ) {
//notification
System.out.println("Invoked IFCallback!!");
try {
//ready file and writers
File filePath = new File("/users/user/G39DDC_Scans/");
if (!filePath.exists()){
System.out.println("Making new directory...");
filePath.mkdir();
}
String filename="Scan_"+System.currentTimeMillis();
File fille= new File("/users/user/G39DDC_Scans/"+filename+".txt");
if (!fille.exists()) {
System.out.println("Making new file...");
fille.createNewFile();
}
FileWriter fw = new FileWriter(fille.getAbsoluteFile());
//callback body
short[] deBuff=new short[NumberOfSamples];
int offset=0;
int arraySize=NumberOfSamples;
deBuff=_Buffer.getShortArray(offset,arraySize);
for (int i=0; i<NumberOfSamples; i++){
String str=deBuff[i]+",";
fw.write(str);
}
fw.close();
} catch (IOException e1) {
System.out.println("IOException: "+e1);
}
}
};
lib.SetCallbacks(hDevice, callbackStructure,UserData);
System.out.println("Main, before callback invocation");
callbackStructure.IFCallback.invoke(s_Pointer, NumberOfSamples, CenterFrequency, Amplitude, ADCSampleRate, UserData);
System.out.println("Main, after callback invocation");
//suddenly having trouble stopping DDCs or powering off device; assume it has to do with dll using the functions above
//System.out.println("StopIF: " + lib.StopIF(hDevice));//API function returns boolean value
//System.out.println("StopDDC2: " + lib.StopDDC2( hDevice, Channel));
//System.out.println("StopDDC1: " + lib.StopDDC1( hDevice, Channel ));
//System.out.println("test_finishDevice: " + test_finishDevice( hDevice, lib));
System.out.println("Program Exit");
//END MAIN METHOD
You need to extend StdCallCallback, for one, otherwise you'll likely crash when the native code tries to call the Java code.
Any place you see a Windows type with _PTR, you should use a PointerType - the platform package with JNA includes definitions for DWORD_PTR and friends.
Finally, you can't have a primitive array argument in your G39DDC_IF_CALLBACK. You'll need to use Pointer or an NIO buffer; Pointer.getShortArray() may then be used to extract the short[] by providing the desired length of the array.
EDIT
Yes, you need to initialize your callback field in the callbacks structure before passing it into your native function, otherwise you're just passing a NULL pointer, which will cause complaints on the Java or native side or both.
This is what it takes to create a callback, using an anonymous instance of the declared callback function interface:
myStruct.callbackField = new MyCallback() {
public void invoke(int arg) {
// do your stuff here
}
};

Addref on COM RCW

Is it possible to increase the RCW reference count on an unknown interface?
(i.e. not the reference count on the underlying COM object)
I have some old COM server code
int Method1(object comobject) {
try {
// do something with comobject
return 0;
}
finally {
Marshal.ReleaseComObject(comobject);
}
}
This code works fine but now I need to call it from another method.
int Method2(object comobject) {
int result = Method1(comobject);
// Do something with combject
}
The type of comobject will vary (that is why it is object)
There is a way, the RCW count counts how many times the object has been marshaled you can increase this number by performing an additional marshal.
public static T AddRcwRef<T>(T t)
{
IntPtr ptr = Marshal.GetIUnknownForObject(t);
try {
return (T)Marshal.GetObjectForIUnknown(ptr);
}
finally {
Marshal.Release(ptr); // done with the IntPtr
}
}
I'm not sure I would recommend using this method, it's probably better to try and get rid of your ReleaseComObject calls.
For further reading, see this blog post on the subject I wrote.
There's the Marshal.AddRef() method, wrong reference count change though. I'm pretty sure incrementing the RCW count directly is not possible. Dig yourself out of the deep hole you're in and fix the old code.