How to pass a reference to a void* from C++/CLI to a native C function - c++-cli

I'm trying to call a native Windows API from managed C++/CLI. One of the arguments is a void**. The idea is that the function will allocate a memory structure and return a void pointer to the caller, which should be passed back to the API on the next call. So I need to allocate storage for a pointer on the managed side and pass a reference to the C API. I can't figure out how to do this.
I've tried declaring a void * in the caller and passing a reference via various operators: &, internal_ptr<>, pin_ptr<>. I did the same with an IntPtr. I get errors saying the compiler can't convert this to a void**.
Here's one attempt using IntPtr and pin_ptr. I get the following compile error on line 28 (the line that declares the pin_ptr):
E0144 a value of type "interior_ptr<System::IntPtr>" cannot be used to initialize an entity of type "cli::pin_ptr<void *>"
#include <msclr\marshal.h>
using namespace msclr::interop;
using namespace System;
namespace CLRStorage
{
public ref class CompoundFile
{
private:
String ^ pathname;
IntPtr pRootStorage;
public:
CompoundFile CompoundFile::Create(String^ path)
{
STGOPTIONS stgOptions;
stgOptions.usVersion = 1;
stgOptions.reserved = 0;
stgOptions.ulSectorSize = 4096;
stgOptions.pwcsTemplateFile = NULL;
auto cf = gcnew CompoundFile();
cf->pathname = path;
marshal_context^ context = gcnew marshal_context();
pin_ptr<void*> ppRootStorage = &cf->pRootStorage;
StgCreateStorageEx(
context->marshal_as<WCHAR*>(path),
STGM_READWRITE & STGM_CREATE,
STGFMT_DOCFILE,
0,
&stgOptions,
NULL,
IID_IStorage,
ppRootStorage);
}
};
}

IntPtr can be converted to and from void*, but it isn't the same type.
Since the parameter is out-only, the simple solution is just to use a temporary:
void* pRootStorage;
StgCreateStorageEx(
context->marshal_as<WCHAR*>(path),
STGM_READWRITE & STGM_CREATE,
STGFMT_DOCFILE,
0,
&stgOptions,
NULL,
IID_IStorage,
&pRootStorage);
cf->pRootStorage = IntPtr(pRootStorage);
This will actually be a tiny bit faster as well, because no pinning is needed.
You also have a separate problem with bad member function syntax. You want
static CompoundFile^ Create(String^ path)
instead of
CompoundFile CompoundFile::Create(String^ path)
and don't forget to
return cf;
Then, marshal_context is not a ref class, so this line is wrong:
marshal_context^ context = gcnew marshal_context();
Instead use
marshal_context context;
and since it is not a pointer,
context.marshal_as<WCHAR*>(path)

Related

c# ReadOnlyMemory from pointer

Hi there I have a c native library that is returning me json as char*. What I would like to do in c# is to use this pointer and write it straight to the
this.ControllerContext.HttpContext.Response.BodyWriter;
I'm able to create ReadOnlySpan from the ptr but as far as I can tell PipeWriter only accepts ReadOnlyMemory<byte> which does not have a constructor from IntPtr. Is there a way to create ReadOnlyMemory<byte> from IntPtr or some other way to writer my string from native library withou copying it one extra time?
This answer provides a solution that does not need to copy the entire buffer:
Marshalling pointer to array P/Invoke
TL;DR: Take UnmanagedMemoryManager from Pipelines.Sockets.Unofficial by Marc Gravell.
int* ptr = ...
int len = ...
var memory = new UnmanagedMemoryManager<int>(ptr, len).Memory;
Unfortunately, you still need to allocate the MemoryManager (it must be a class, not a struct).
Thank you for your answers but none of them was without extra copy. I was finally figure it out so in case somebody struggle with it, here is the solution.
So the only way I as able to achieve this is like.
await Response.StartAsync(HttpContext.RequestAborted);
var dest = Response.BodyWriter.GetMemory((int)jsonLen).Pin();
unsafe { memcpy(dest.Pointer), srcPtr, srcLen); }
Response.BodyWriter.Advance(srcLen);
await Response.BodyWriter.FlushAsync(HttpContext.RequestAborted);
Maybe use something like this?
public class Utility
{
public System.ReadOnlyMemory<T> ConvertToReadOnlyMemory(System.ReadOnlySpan<T> input) {
var tmp = new System.Memory<T>();
input.CopyTo(tmp.Span);
return (System.ReadOnlyMemory<T>)tmp;
}
}
However, I think this will involve completely copying the stream into heap storage, which is probably not what you want...
I glad if this could speed up and match to what you wants.
namespace Helper
{
using System;
using System.Runtime.InteropServices;
public static class CStringMapper
{
// convert unmanaged c string to managed c# string
public string toCSharpString(char* unmanaged_c_string)
{
return Marshal.PtrToStringAnsi((IntPtr)unmanaged_c_string);
}
// Free unmanaged c pointer
public void free(char* unmanaged_c_string)
{
Marshal.FreeHGlobal((IntPtr)unmanaged_c_string);
}
}
}
Usage:
using Helper;
/* generate your unmanaged c string here */
try
{
// eg. char* OO7c = cLibFunc();
string cSharpString = CStringMapper.toCSharpString(OO7c);
}
finally
{
// Make sure to freeing the pointer
CStringMapper.free(OO7c);
}

Accessing a C/C++ structure of callbacks through a DLL's exported function using JNA

I have a vendor supplied .DLL and an online API that I am using to interact with a piece of radio hardware; I am using JNA to access the exported functions through Java (because I don't know C/C++). I can call basic methods and use some API structures successfully, but I am having trouble with the callback structure. I've followed the TutorTutor guide here and also tried Mr. Wall's authoritative guide here, but I haven't been able to formulate the Java side syntax for callbacks set in a structure correctly.
I need to use this exported function:
BOOL __stdcall SetCallbacks(INT32 hDevice,
CONST G39DDC_CALLBACKS *Callbacks, DWORD_PTR UserData);
This function references the C/C++ Structure:
typedef struct{
G39DDC_IF_CALLBACK IFCallback;
//more omitted
} G39DDC_CALLBACKS;
...which according to the API has these Members (Note this is not an exported function):
VOID __stdcall IFCallback(CONST SHORT *Buffer, UINT32 NumberOfSamples,
UINT32 CenterFrequency, WORD Amplitude,
UINT32 ADCSampleRate, DWORD_PTR UserData);
//more omitted
I have a G39DDCAPI.java where I have loaded the DLL library and reproduced the API exported functions in Java, with the help of JNA. Simple calls to that work well.
I also have a G39DDC_CALLBACKS.java where I have implemented the above C/C++ structure in a format works for other API structures. This callback structure is where I am unsure of the syntax:
import java.util.Arrays;
import java.util.List;
import java.nio.ShortBuffer;
import com.sun.jna.Structure;
import com.sun.jna.platform.win32.BaseTSD.DWORD_PTR;
import com.sun.jna.win32.StdCallLibrary.StdCallCallback;
public class G39DDC_CALLBACKS extends Structure {
public G39DDC_IF_CALLBACK IFCallback;
//more omitted
protected List getFieldOrder() {
return Arrays.asList(new String[] {
"IFCallback","DDC1StreamCallback" //more omitted
});
}
public static interface G39DDC_IF_CALLBACK extends StdCallCallback{
public void invoke(ShortBuffer _Buffer,int NumberOfSamples,
int CenterFrequency, short Amplitude,
int ADCSampleRate, DWORD_PTR UserData);
}
}
Edit: I made my arguments more type safe as Technomage suggested. I am still getting a null pointer exception with several attempts to call the callback. Since I'm not sure of my syntax regarding the callback structure above, I can't pinpoint my problem in the main below. Right now the relevant section looks like this:
int NumberOfSamples=65536;//This is usually 65536.
ShortBuffer _Buffer = ShortBuffer.allocate(NumberOfSamples);
int CenterFrequency=10000000;//Specifies center frequency (in Hz) of the useful band
//in received 50 MHz wide snapshot.
short Amplitude=0;//The possible value is 0 to 32767.
int ADCSampleRate=100;//Specifies sample rate of the ADC in Hz.
DWORD_PTR UserData = null;
G39DDC_CALLBACKS callbackStruct= new G39DDC_CALLBACKS();
lib.SetCallbacks(hDevice,callbackStruct,UserData);
//hDevice is a handle for the hardware device used-- works in other uses
//lib is a reference to the library in G39DDCAPI.java-- works in other uses
//The UserData is a big unknown-- I don't know what to do with this variable
//as a DWORD_PTR
callbackStruct.IFCallback.invoke(_Buffer, NumberOfSamples, CenterFrequency,
Amplitude, ADCSampleRate, UserData);
EDIT NO 2:
I have one callback working somewhat, but I don't have control over the buffers. More frustratingly, a single call to invoke the method will result in several runs of the custom callback, usually with multiple output files (results vary drastically from run to run). I don't know if it is because I am not allocating memory correctly on the Java side, because I cannot free the memory on the C/C++ side, or because I have no cue on which to tell Java to access the buffer, etc. Relevant code looks like:
//before this, main method sets library, starts DDCs, initializes some variables...
//API call to start IF
System.out.print("Starting IF... "+lib.StartIF(hDevice, Period)+"\n")
G39DDC_CALLBACKS callbackStructure = new G39DDC_CALLBACKS();
callbackStructure.IFCallback = new G39DDC_IF_CALLBACK(){
#Override
public void invoke(Pointer _Buffer, int NumberOfSamples, int CenterFrequency,
short Amplitude, int ADCSampleRate, DWORD_PTR UserData ) {
//notification
System.out.println("Invoked IFCallback!!");
try {
//ready file and writers
File filePath = new File("/users/user/G39DDC_Scans/");
if (!filePath.exists()){
System.out.println("Making new directory...");
filePath.mkdir();
}
String filename="Scan_"+System.currentTimeMillis();
File fille= new File("/users/user/G39DDC_Scans/"+filename+".txt");
if (!fille.exists()) {
System.out.println("Making new file...");
fille.createNewFile();
}
FileWriter fw = new FileWriter(fille.getAbsoluteFile());
//callback body
short[] deBuff=new short[NumberOfSamples];
int offset=0;
int arraySize=NumberOfSamples;
deBuff=_Buffer.getShortArray(offset,arraySize);
for (int i=0; i<NumberOfSamples; i++){
String str=deBuff[i]+",";
fw.write(str);
}
fw.close();
} catch (IOException e1) {
System.out.println("IOException: "+e1);
}
}
};
lib.SetCallbacks(hDevice, callbackStructure,UserData);
System.out.println("Main, before callback invocation");
callbackStructure.IFCallback.invoke(s_Pointer, NumberOfSamples, CenterFrequency, Amplitude, ADCSampleRate, UserData);
System.out.println("Main, after callback invocation");
//suddenly having trouble stopping DDCs or powering off device; assume it has to do with dll using the functions above
//System.out.println("StopIF: " + lib.StopIF(hDevice));//API function returns boolean value
//System.out.println("StopDDC2: " + lib.StopDDC2( hDevice, Channel));
//System.out.println("StopDDC1: " + lib.StopDDC1( hDevice, Channel ));
//System.out.println("test_finishDevice: " + test_finishDevice( hDevice, lib));
System.out.println("Program Exit");
//END MAIN METHOD
You need to extend StdCallCallback, for one, otherwise you'll likely crash when the native code tries to call the Java code.
Any place you see a Windows type with _PTR, you should use a PointerType - the platform package with JNA includes definitions for DWORD_PTR and friends.
Finally, you can't have a primitive array argument in your G39DDC_IF_CALLBACK. You'll need to use Pointer or an NIO buffer; Pointer.getShortArray() may then be used to extract the short[] by providing the desired length of the array.
EDIT
Yes, you need to initialize your callback field in the callbacks structure before passing it into your native function, otherwise you're just passing a NULL pointer, which will cause complaints on the Java or native side or both.
This is what it takes to create a callback, using an anonymous instance of the declared callback function interface:
myStruct.callbackField = new MyCallback() {
public void invoke(int arg) {
// do your stuff here
}
};

Trouble using PInvoke against ssdeep's fuzzy.dll (unbalanced stack)

I'm attempting to call a method on the ssdeep fuzzy.dll
The .h file is here and a friendly reference is here
Specifically, I'm trying to call this method....
int fuzzy_hash_filename (
const char * filename,
char * result
)
I've got the following...
<DllImport("C:\SSDeep\Fuzzy.dll", EntryPoint:="fuzzy_hash_filename")>
Private Shared Function fuzzy_hash_filename(
<InAttribute(),
MarshalAsAttribute(UnmanagedType.LPStr)>
ByVal Filename As String, ByVal Result As StringBuilder) As Integer
End Function
Public Shared Function FuzzyHash(Filename As String) As String
Dim Ret As New StringBuilder
Ret.Capacity = NativeConstants.FUZZY_MAX_RESULT
Dim Success = fuzzy_hash_filename(Filename, Ret)
If Success <> 0 Then
Throw New Exception("SSDeep fuzzy hashing failed")
End If
Return Ret.ToString
End Function
If I run this code, VS gives me a modal dialogue
A call to PInvoke function '(Blah)::fuzzy_hash_filename' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.
(FWIW The call seems to succeed if I ignore the warning so I must be close)
What change do I need to make to my definition to get this going?
I found someone that had the same issue on MSDN forums:
Concerning the PInvokeStackImbalance.
1.1 This is usually due to mismatch of the calling convention used by the API and that declared for the API in the C# code.
1.2 By default, if the CallingConvention argument for the DllImportAttribute is not set, then StdCall is used by default.
1.3 If the DoSomething() API is to use __cdecl (as is the default in C++ projects), then you should use the following declaration for
DoSomething() in the C# code : [DllImport(#"dll.dll",
CallingConvention=CallingConvention.Cdecl)]
1.4 Also, I suggest that you declare the API as extern "C" otherwise it will be subject to name mangling by the C++ compiler.
The accepted answer appears to have solved the original asker's problem, but the equivalent code in c# did not work for me. After trying increasingly complex annotations, going back to basics eventually did work. For everyone's reference, I include the declaration for three of the interface functions and working code (built against ssdeep version 2.9).
//Note: StringBuilder here is the standard way to do it, but is a perf hit because unicode stringbuilder can't be pinned when martialling char*.
//See http://msdn.microsoft.com/en-us/magazine/cc164193.aspx#S4
//int fuzzy_hash_buf(const unsigned char *buf, uint32_t buf_len, char *result)
[DllImport("fuzzy.dll")]
public static extern int fuzzy_hash_buf(StringBuilder buf, int buf_len, StringBuilder result);
//int fuzzy_hash_filename(const char* filename, char* result)
[DllImport("fuzzy.dll")]
static extern int fuzzy_hash_filename(string filename, StringBuilder result);
//int fuzzy_compare (const char *sig1, const char *sig2)
[DllImport("fuzzy.dll")]
static extern int fuzzy_compare(string sig1, string sig2);
static void Main(string[] args)
{
StringBuilder buf = new StringBuilder("test");
StringBuilder result0 = new StringBuilder(150);
fuzzy_hash_buf(buf, 4, result0);
Console.WriteLine(result0);
string filename = "test.txt";
StringBuilder result1 = new StringBuilder(150);
fuzzy_hash_filename(filename, result1);
Console.WriteLine(result1);
int matchScore = fuzzy_compare(result0.ToString(), result1.ToString());
Console.WriteLine("MatchScore: " + matchScore);
}
Output:
ssdeeptest.exe
3:Hn:Hn
24:gRnIM7stweRp+fEWU1XRk+/M98D6Dv3JrEeEnD/MGQbnEWqv3JW:gRIMwtrMU1Bk2I3Jrg53JW
MatchScore: 0

How to use interlocked operations against memory-mapped files in .Net

Is there any way to use the Interlocked.CompareExchange(); and Interlocked.Increment(); methods against values stored in a memory-mapped file?
I'd like to implement a multi-threaded service that will store its data in a memory-mapped file, but since it's multi-threaded I need to prevent conflicting writes, therefore I wonder about the Interlocked operations rather than using explicit locks.
I know it's possible with native code, but can it be done in managed code on .NET 4.0?
OK, this is how you do it! We had to figure this out, and I figured we could give some back to stackoverflow!
class Program
{
internal static class Win32Stuff
{
[DllImport("kernel32.dll", SetLastError = true)]
unsafe public static extern int InterlockedIncrement(int* lpAddend);
}
private static MemoryMappedFile _mmf;
private static MemoryMappedViewStream _mmvs;
unsafe static void Main(string[] args)
{
const int INT_OFFSET = 8;
_mmf = MemoryMappedFile.CreateOrOpen("SomeName", 1024);
// start at offset 8 (just for example)
_mmvs = _mmf.CreateViewStream(INT_OFFSET, 4);
// Gets the pointer to the MMF - we dont have to worry about it moving because its in shared memory
var ptr = _mmvs.SafeMemoryMappedViewHandle.DangerousGetHandle();
// Its important to add the increment, because even though the view says it starts at an offset of 8, we found its actually the entire memory mapped file
var result = Win32Stuff.InterlockedIncrement((int*)(ptr + INT_OFFSET));
}
}
This does work, and works across multiple processes! Always enjoy a good challenge!
TravisWhidden, actually you can use Interlocked.Increment Static method as dan-gph said, you just have to be careful with pointer casting and operator priority, plus parenthesis usage, in facts...
You'll cast a memory pointer (plus the desired offset), into a pointer to an int variable, then you'll use that pointer as a variable. Then you'll have to use it as a variable reference.
Below you'll find the corresponding snippet of yours using .net library instead of external static import.
P&L
class Program
{
private static MemoryMappedFile _mmf;
private static MemoryMappedViewStream _mmvs;
static void Main(string[] args)
{
const int INT_OFFSET = 8;
_mmf = MemoryMappedFile.CreateOrOpen("SomeName", 1024);
_mmvs = _mmf.CreateViewStream(INT_OFFSET, 4);
unsafe
{
IntPtr ptr = _mmvs.SafeMemoryMappedViewHandle.DangerousGetHandle();
Interlocked.Increment(ref (*((int*)(ptr + INT_OFFSET)))
}
}
}

array of pin_ptr<Type>

I need to marshal an array of String^ to call a unmanaged function that expects an array of BSTRs.
On MSDN I found the article
How to: Marshal COM Strings Using C++ Interop
with this code sample:
// MarshalBSTR1.cpp
// compile with: /clr
#define WINVER 0x0502
#define _AFXDLL
#include <afxwin.h>
#include <iostream>
using namespace std;
using namespace System;
using namespace System::Runtime::InteropServices;
#pragma unmanaged
void NativeTakesAString(BSTR bstr) {
printf_s("%S", bstr);
}
#pragma managed
int main() {
String^ s = "test string";
IntPtr ip = Marshal::StringToBSTR(s);
BSTR bs = static_cast<BSTR>(ip.ToPointer());
pin_ptr<BSTR> b = &bs;
NativeTakesAString( bs );
Marshal::FreeBSTR(ip);
}
So I created a new BSTRs' array and called the Marshal::StringToBSTR() for every String of the array.
Then I created a managed pin_ptr array.
array<pin_ptr<BSTR> >^ gcDummyParameters = gcnew array<pin_ptr<BSTR> >(asParameters->Length);
but I receved the error:
Error 2 error C2691: 'cli::pin_ptr<Type>' : a managed array cannot have this element type
I tried also with a native array:
pin_ptr<BSTR> dummyParameters[100000];
but even in this case I got an error:
Error 1 error C2728: 'cli::pin_ptr<Type>' : a native array cannot contain this managed type
What else can I do?
Microsoft sample looks strange: there is no need to pin BSTR type because it is unmanaged. Just create BSTR array and fill every member using Marshal::StringToBSTR. Don't use pin_ptr.
pin_ptr should be removed from this sample. bs is a local variable and will not be moved by the garbage collector, also it is passed to the native function by value so there would be no problem if it did move.
The BSTR content to which it points is natively allocated by the system's BSTR allocator, it also will not be moved by the garbage collector.