I have a very strange bug which happens only on certain Android phones happens on all tested phones, the techonology is Monogame(C#), the OpenAL library is in C++.
A TL/DR version: float and double arguments are corrupted when passed by value from C# to C++, but passing them be reference(pointer) works totaly ok. Corrupt float and double arguments also shift all other arguments by one or two places in the argument list.
Compilers: android-ndk-r10e, Visual Studio 2015.
Link to Android build file configs:
Android Build Files
I have a C++ function like this inside the OpenAL C++ lib:
void foo(int i, float x, float y, float z){...}
This function is called from C#. The first int param 'i' and the first float param 'x' are passed correctly. But the last two float params, 'y' and 'z', have one byte of the four bytes corrupted, it's always the same byte (least significant or most significant, can't remember 100% now).
I tried all possible calling conventions.
The funny thing is if the C++ function has only int params, eg
void foo(int,int,int,int)
it works completely alright.
I also tried functions with these signatures:
void foo(float)
void foo(float,float)
void foo(float,float,float)
void foo(float,float,float,float)
And in all cases the last float was the one that was not corrupted.
I then try this signature (and a few similar):
void foo(int,int,float,int,int,int,float,int,int)
and it seems as if each float params shifts the int params that are after it by 4 bytes (aka "shifts by one entire function param" since they are all 4 bytes).
Also something funny that happened in the above signature is that float argument order seemed inverted (the last float argument was replaced by the first one, while integer argument order remained the same).
I also tried using doubles and all possible float/double combinations between C# and C++ calls but nothing worked. Doubles shifted the argument twice as much.
Passing doubles by reference (pointer) works perfectly fine.
Any ideas what could be causing it or clues as what to try next?
Also: is there any way I can debug the interop layer between C# and C++ to see what happens?
Dll imports in C#:
[DllImport(AL.Lib, EntryPoint = "ropoFII", ExactSpelling = true, CallingConvention = AL.Style), SuppressUnmanagedCodeSecurity()]
public static extern void ropoFII(float x, int a, int b);
[DllImport(AL.Lib, EntryPoint = "ropoFIF", ExactSpelling = true, CallingConvention = AL.Style), SuppressUnmanagedCodeSecurity()]
public static extern void ropoFIF(float x, int a, float b);
[DllImport(AL.Lib, EntryPoint = "ropoFIFI", ExactSpelling = true, CallingConvention = AL.Style), SuppressUnmanagedCodeSecurity()]
public static extern void ropoFIFI(float x, int a, float b, int c);
[DllImport(AL.Lib, EntryPoint = "ropoIFFF", ExactSpelling = true, CallingConvention = AL.Style), SuppressUnmanagedCodeSecurity()]
public static extern void ropoIFFF(int a, float b, int c, float d);
[DllImport(AL.Lib, EntryPoint = "ropoIIFII", ExactSpelling = true, CallingConvention = AL.Style), SuppressUnmanagedCodeSecurity()]
public static extern void ropoIIFII(int i1, int i2, float f, int i3, int i4);
Related
The following C function populate a C struct in Visual Works (is working ok):
<C:int ssh_pki_import_pubkey_file (const char * filename, VOID * pkey)>
Now a second function is defined as:
int ssh_userauth_try_publickey (ssh_session session, const char * username, const ssh_key pubkey)
In Smalltalk:
<C:int ssh_userauth_try_publickey (VOID session, const char * username, const VOID pubkey)>
If i call the second function (ssh_userauth_try_publickey) with the populated argument (with no transformation) of the first function (ssh_pki_import_pubkey_file) it fail.
So VOID * pkey has to match const VOID pubkey.
In GemStone/S this is done with #'&ptr' and #'ptr', so #'&ptr' will get the value of the pointer (the CPointer’s value will be passed and updated on return).
Reading DLL & C Connect User’s Guide does not yield result yet.
Short answer
use void** in your first function and void* in your second function
Long answer
In C, void means "nothing" and if you have a return type of void that means you don't return anything.
But void* means pointer to nothing (...that i know about)... basically you get a pointer to something where you don't know what it is. But it's still a pointer, which is not nothing.
If you have a function that produces a value not via return but via parameter, you need to pass a pointer so that the function can set its value. You can do that via void* but that's unintentional. Consider the following wrong C code:
void var;
myFunc(&var);
myFunc would take a void* as parameter in order to fill its value, but a variable of type void is wrong because what would be its value. In correct C you would do it like that:
void* var = NULL;
myFunc(&var);
Here the type of var is clearly a pointer and its value is even initialised. For myFunc there's no real difference here, except that it'll now have a void** as parameter.
So if you modify ssh_pki_import_pubkey_file's declaration to have a void** parameter and change ssh_userauth_try_publickey's declaration to accept a void* parameter, you should be fine.
I am in the process of porting an application from (Objective-)C to Swift but have to use a third-party framework written in C. There are a couple of incompatibilities like typedefs that are interpreted as Int but have to be passed to the framework's functions as UInts or the like. So to avoid constant casting operations throughout the entire Swift application I decided to transfer the C header files to Swift, having all types as I I need them to be in one place.
I was able to transfer nearly everything and have overcome a lot of hurdles, but this one:
The C header defines a struct which contains a uint64_t variable among others. This struct is used to transfer data to a callback function as a pointer. The callback function takes a void pointer as argument and I have to cast it with the UnsafeMutablePointer operation to the type of the struct (or another struct of the header if appropriate). All the casting and memory-accessing works fine as long as I use the original struct from the C header that was automatically transformed by Swift on import.
Replicating the struct manually in Swift does not "byte-fit" however.
Let me show you a reduced example of this situation:
Inside the CApiHeader.h file there is something like
typedef struct{
uint32_t var01;
uint64_t var02;
uint8_t arr[2];
}MyStruct, *MyStructPtr;
From my understanding this here should be the Swift equivalent
struct MyStruct{
var01: UInt32
var02: UInt64
arr: (UInt8, UInt8)
}
Or what should also work is this tuple notation
typealias MyStruct = (
var01: UInt32,
var02: UInt64,
arr: (UInt8, UInt8)
)
This works normally, but not as soon as there is an UInt64 type.
Okay, so what happens?
Casting the pointer to one of my own Swift MyStruct implementations the hole data is shifted by 2 bytes, starting at the UInt64 field. So in this example the both arr fields are not at the correct position, but inside the UInt64 bits, that should be 64 in number. So it seams that the UInt64 field has only 48 bits.
This accords to my observation that if I replace the UIn64 variable with this alternative
struct MyStruct{
var01: UInt32
reserved: UInt16
var02: UInt32
arr: (UInt8, UInt8)
}
or this one
struct MyStruct{
var01: UInt32
var02: (UInt32, UInt32)
arr: (UInt8, UInt8)
}
(or the equivalent tuple notation) it aligns the arr fields correctly.
But as you can easily guess var02 contains not directly usable data, because it is split over multiple address ranges. It is even worse with the first alternative, because it seams that Swift fills up the gap between the reserved field and the var02 field with 16 bits - the missing / shifted 2 bytes I mentioned above - but these are not easily accessible.
So I haven't figured out any equivalent transformation of the C struct in Swift.
What happens here exactly and how does Swift transforms the struct from the C header actually?
Do you guys have a hint or an explanation or even a solution for me, please?
Update
The C framework has an API function with this signature:
int16_t setHandlers(MessageHandlerProc messageHandler);
MessageHandlerProc is procedure type:
typedef void (*messageHandlerProc)(unsigned int id, unsigned int messageType, void *messageArgument);
So setHandlers is a C procedure inside the framework that gets a pointer to a callback function. This callback function has to provide an argument of a void Pointer, that gets casted to e.g.
typedef struct {
uint16_t revision;
uint16_t client;
uint16_t cmd;
int16_t parameter;
int32_t value;
uint64_t time;
uint8_t stats[8];
uint16_t compoundValueOld;
int16_t axis[6];
uint16_t address;
uint32_t compoundValueNew;
} DeviceState, *DeviceStatePtr;
Swift is smart enough to import the messageHandlerProc with the convention(c) syntax, so the procedure type is directly available. On the other hand it is not possible use the standard func syntax and bitcast my messageHandler callback function to this type. So I used the closure syntax to define the callback function:
let myMessageHandler : MessageHandlerProc = { (deviceID : UInt32, msgType : UInt32, var msgArgPtr : UnsafeMutablePointer<Void>) -> Void in
...
}
I converted the above mentioned structure into the different structures of my original post.
And No! Defining stats as Swift Array does not work. An Array in Swift in not equivalent to an Array in C, because Swift's Array is a extended type. Writing to and reading from it with a pointer causes an exception
Only Tuples are natively implemented in Swift and you can run back and forth with pointers over it.
Okay... this works all fine and my callback function gets called whenever data is available.
So inside myMessageHandler I want to use the stored Data inside msgArgPtr which is a void pointer and thus has to be cast into DeviceState.
let state = (UnsafeMutablePointer<MyDeviceState>(msgArgPtr)).memory
Accessing state it like:
...
print(state.time)
print(state.stats.0)
...
Whenever I use the automatically generated Swift pendant of DeviceState it all works nicely. The time variable has the Unix Time Stamp and the following stats (accessible with tuple syntax!!!) are all where they belong.
Using my manually implemented struct however results in a completely senseless time stamp value and the stats fields are shifted to the left (towards the time field - that's probably why the time stamp value is useless, because it contains bits from the stats "array"). So in the last two fields of stats I get values from compoundValueOld and the first axis field - with all the overflowing of course.
As long as I am willing to sacrifice the time value and change the UInt64 variable by either a tuple of two UInt32 types or by changing it to a UInt32 type and adding a auxiliary variable of the type UInt16 right before time, I receive a stats "array" with correct alignment.
Have a nice day! :-)
Martin
This is an update to my earlier answer after reading your updated question and experimenting some more. I believe the problem is an alignment discrepancy between the imported C structure and the one you manually implemented in Swift. The problem can be solved by using a C helper function to get an instance of the C struct from void pointer as was suggested yesterday, which can then be converted to the manually implemented Swift struct.
I've been able to reproduce the problem after creating an abbreviated mock-up of your DeviceState structure that looks like
typedef struct
{
uint16_t revision;
uint16_t client;
uint16_t cmd;
int16_t parameter;
int32_t value;
uint64_t time;
uint8_t stats[8];
uint16_t compoundValueOld;
} APIStruct;
The corresponding hand-crafted Swift native structure is:
struct MyStruct
{
init( _apis : APIStruct)
{
revision = _apis.revision
client = _apis.client
cmd = _apis.cmd
parameter = _apis.parameter
value = _apis.value
time = _apis.time
stats = _apis.stats
compoundValueOld = _apis.compoundValueOld
}
var revision : UInt16
var client : UInt16
var cmd : UInt16
var parameter : Int16
var value : Int32
var time : UInt64
var stats : (UInt8, UInt8, UInt8, UInt8, UInt8, UInt8, UInt8, UInt8);
var compoundValueOld : UInt16
}
The C framework you are working with could have been compiled using a different struct packing, resulting in a non-matching alignment. I used
#pragma pack(2)
in my C code to break the bit-matching between the Swift's native and imported C struct.
If I do something like
func swiftCallBackVoid( p: UnsafeMutablePointer<Void> )
{
...
let _locMS:MyStruct = (UnsafeMutablePointer<MyStruct>(p)).memory
...
}
the data in _locMS is different from what was placed there by C code. This problem only occurs if I change struct packing using a pragma in my C code; the above unsafe conversion works fine if the default alignment is used. One can solve this problem as follows:
let _locMS:MyStruct = MyStruct(_apis: (UnsafeMutablePointer<APIStruct>(p)).memory)
BTW, the way Swift imports the C struct, the array members become tuples; this can be seen from the fact that tuple notation has to be used to access them in Swift.
I have a sample Xcode project illustrating all this that I've placed on github:
https://github.com/omniprog/xcode-samples
Obviously, the approach of using a helper C function to get APIStruct from a void pointer and then converting the APIStruct to MyStruct may or may not be an option, depending on how the structures are used, how large they are, and on the performance requirements of the application. As you can tell, this approach involves some copying of the structure. Other approaches, I think, include writing a C-layer between Swift code and the 3rd party C framework, studying the memory layout of the C structure and accessing it in creative ways (may break easily), using the imported C struct more extensively in your Swift code, etc...
Here is a way to share data between C and Swift code without unnecessary copying and with changes made in Swift visible to C code. With the following approach, however, it's imperative to be aware of object lifetime and other memory management issues. One can create a class as follows:
// This typealias isn't really necessary, just a convenience
typealias APIStructPtr = UnsafeMutablePointer<APIStruct>
struct MyStructUnsafe
{
init( _p : APIStructPtr )
{
pAPIStruct = _p
}
var time: UInt64 {
get {
return pAPIStruct.memory.time
}
set( newVal ) {
pAPIStruct.memory.time = newVal
}
}
var pAPIStruct: APIStructPtr
}
Then we can use this structure as follows:
func swiftCallBackVoid( p: UnsafeMutablePointer<Void> )
{
...
var _myUnsafe : MyStructUnsafe = MyStructUnsafe(_p: APIStructPtr(p))
...
_myUnsafe.time = 9876543210 // this change is visible in C code!
...
}
Your two definitions are not equivalent. An array is not the same as a tuple. Your C struct gives 24 bytes (see this question as to why). The size in Swift differs depend on how you implement it:
struct MyStruct1 {
var var01: UInt32
var var02: UInt64
var arr: (UInt8, UInt8)
}
typealias MyStruct2 = (
var01: UInt32,
var02: UInt64,
arr: (UInt8, UInt8)
)
struct MyStruct3 {
var var01: UInt32
var var02: UInt64
var arr: [UInt8] = [0,0]
}
print(sizeof(MyStruct1)) // 18
print(sizeof(MyStruct2)) // 18
print(sizeof(MyStruct3)) // 24, match C's
I am trying to create a CLI value class c_Location with overloaded operators, but I think I have an issue with boxing. I have implemented the operator overloading as seen in many manuals, so I'm sure this must be right.
This is my code:
value class c_Location
{
public:
double x, y, z;
c_Location (double i_x, double i_y, double i_z) : x(i_x), y(i_y), z(i_z) {}
c_Location& operator+= (const c_Location& i_locValue)
{
x += i_locValue.x;
y += i_locValue.y;
z += i_locValue.z;
return *this;
}
c_Location operator+ (const c_Location& i_locValue)
{
c_Location locValue(x, y, z);
return locValue += i_locValue;
}
};
int main()
{
array<c_Location,1>^ alocData = gcnew array<c_Location,1>(2);
c_Location locValue, locValue1, locValue2;
locValue = locValue1 + locValue2;
locValue = alocData[0] + alocData[1]; // Error C2679 Binary '+': no operator found which takes a right-hand operand of type 'c_Location'
}
After searching for a longer time, I found that the error comes from the operand being a reference type, as it is an array element of a value type, and the function accepting only value types as it takes an unmanaged reference. I now have 2 possibiblities:
adding a unboxing cast to c_Location and so changing the faulty line in main() to
locValue = alocData[0] + (c_Location)alocData[1];
modifying the operator+ overloading so that it takes the parameter by value instead of by reference:
c_Location operator+ (const c_Location i_locValue)
both options work, but as far as I can see, they both have disadvantages:
opt 1 means that I have to explicitly cast wherever needed.
opt 2 means that the function will create a copy of the parameter on its call and therefore waste performance (not much though).
My questions: Is my failure analysis correct at all or does the failure have another reason?
Is there a better third alternative?
If not: which option, 1 or 2, is the better one? I currently prefer #2.
The rules are rather different from native C++:
the CLI demands that operator overloads are static members of the class
you can use the const keyword in C++/CLI but you get no mileage from it, the CLI does not support enforcing const-ness and there are next to no other .NET languages that support it either.
passing values of a value type ought to be done by value, that's the point of having value types in .NET in the first place. Using a & reference is very troublesome, that's a native pointer at runtime which the garbage collector cannot adjust. You'll get a compile error if you try to use your operator overload on a c_Location that's embedded in a managed class. If you want to avoid value copy semantics then you should declare a ref class instead. The hat^ in your code.
any interop type you create in C++/CLI should be declared public so it is usable from other assemblies and .NET languages. It isn't entirely clear if that's your intention, it is normally the reason you write C++/CLI code.
You could make your value class look like this instead:
public value class c_Location
{
public:
double x, y, z;
c_Location (double i_x, double i_y, double i_z) : x(i_x), y(i_y), z(i_z) {}
static c_Location operator+= (c_Location me, c_Location rhs)
{
me.x += rhs.x;
me.y += rhs.y;
me.z += rhs.z;
return me;
}
static c_Location operator+ (c_Location me, c_Location rhs)
{
return c_Location(me.x + rhs.x, me.y + rhs.y, me.z + rhs.z);
}
};
Untested, ought to be close. You'll now see that your code in main() compiles without trouble.
TL;DR version:
For managed code, use % for a pass by reference parameter, not &
You diagnosis is not completely correct. Boxing has nothing to do with your problem. But reference types do, in a way.
You were really close when you said that "I found that the error comes from the operand being a reference type". Well, the operand is a value type not a reference type. But the error occurs when the operand is stored inside a reference type, because then it's inside the garbage-collected heap (where all instances of reference types are placed). This goes for arrays as well as your own objects which contain a member of value type.
The danger is that when the garbage collector runs, it can move items around on the gc heap. And this breaks native pointers (*) and references (&), because they store the address and expect it to stay the same forever. To handle this problem, C++/CLI provides tracking pointers (^) and tracking references (%) which work together with the garbage collector to do two things:
make sure the enclosing object isn't freed while you're using it
find the new address if the garbage collector moves the enclosing object
For use from C++/CLI, you can make operator+ a non-member, just like normal C++.
value class c_Location
{
public:
double x, y, z;
c_Location (double i_x, double i_y, double i_z) : x(i_x), y(i_y), z(i_z) {}
c_Location% operator+= (const c_Location% i_locValue)
{
x += i_locValue.x;
y += i_locValue.y;
z += i_locValue.z;
return *this;
}
};
c_Location operator+ (c_Location left, const c_Location% right)
{
return left += right;
}
The drawback is that C# won't use non-members, for compatibility with C#, write it like a non-member operator (with two explicit operands) but make it a public static member.
value class c_Location
{
public:
double x, y, z;
c_Location (double i_x, double i_y, double i_z) : x(i_x), y(i_y), z(i_z) {}
c_Location% operator+= (const c_Location% i_locValue)
{
x += i_locValue.x;
y += i_locValue.y;
z += i_locValue.z;
return *this;
}
static c_Location operator+ (c_Location left, const c_Location% right)
{
return left += right;
}
};
There's no reason to worry about this for operator+= since C# doesn't recognize that anyway, it will use operator+ and assign the result back to the original object.
For primitive types like double or int, you may find that you need to use % also, but only if you need a reference to an instance of that primitive type is stored inside a managed object:
double d;
array<double>^ a = gcnew darray<double>(5);
double& native_ref = d; // ok, d is stored on stack and cannot move
double& native_ref2 = a[0]; // error, a[0] is in the managed heap, you MUST coordinate with the garbage collector
double% tracking_ref = d; // ok, tracking references with with variables that don't move, too
double% tracking_ref2 = a[0]; // ok, now you and the garbage collector are working together
I have created new Win32 project in my VS and have selected Dynamic Library ( *.dll ) for this aim.
I have defined some exporting function in the main file:
__declspec(dllexport)
int TestCall(void)
{
int value = 4 / 2;
std::cout << typeid(value).name() << std::endl;
return value;
}
__declspec(dllexport)
void SwapMe(int *first, int *second)
{
int tmp = *first;
*first = *second;
*second = tmp;
}
When I've looked at the dumpin /exports, I've got:
ordinal hint RVA name
1 0 00001010 ?SwapMe##YAXPEAH0#Z
2 1 00001270 ?TestCall##YAHXZ
I'm calling in the C# version like this:
[DllImport(#"lib1.dll", EntryPoint = "?TestCall##YAHXZ",
CallingConvention = CallingConvention.Cdecl)]
static extern int TestCall();
It's not the correct form of using exported methods. Where did I fail with generating such names for exporting-methods in the C++ dll project?
This is normal, the C++ compiler applies name decoration to functions. The C++ language supports function overloading, much like C# does. So you could write a Foo(int) and a Foo(double) function. Clearly they cannot both be exported as a function named "Foo", the client code wouldn't know which one to call. So the extra characters encode the name so that it is unique for the overload.
You can turn that off by declaring the function extern "C", the C language doesn't support overloading so doesn't require the same kind of decoration.
But it is actually better if you don't. Because it is also an excellent way to catch mistakes. Like changing the function declaration in your C++ code but forgetting to modify the pinvoke declaration in your C# code. You will now get an easy to diagnose "Entrypoint not found" exception instead of a non-descriptive and very hard to diagnose AccessViolationException. Which doesn't necessarily have to be raised in the C++ code, the stack imbalance can also crash your C# code. Looking up the decorated name is a bit painful however, improve that by asking the linker to create a map file (/MAP option).
use extern "C" to specify the linkage to avoid the name mangling:
extern "C" __declspec(dllexport) int TestCall(void);
extern "C" __declspec(dllexport) void SwapMe(int *first, int *second);
Help!
I'm totally exhausted/frustrated with what seems to be a reasonably easy task. I’m not sure what I'm doing wrong; let alone if I'm doing it correct. I'm "required" to use an existing library (a C static library – over 100,000 lines of straight C code) in developing a WPF application (VS 2010, C# 4.0). Oh, and I can't touch the existing C code - use it as is!
I've read so many postings (advanced topics, how-to, etc), yet I'm so new to C++/CLI that it's just not making sense. From what I've read the best approach is to wrap the C static library as follows:
Unmanaged C static library <---> C++/CLI managed wrapper DLL <--->
managed WPF application
This is the stripped down C header file:
/* Call this function to execute a command. */
int issue_command(int command, long param1, long param2);
/* Completion call back function; you must supply a definition. */
extern int command_completed(int command, long param1, long param2);
struct struct_command_str
{
char command_str[10];
char param1_st[2];
char param2_st[2];
char success;
};
/* You must supply definitions to the following extern items. */
extern int command_status;
extern struct struct_command_str command_str;
The problem(s):
What I can’t seem to do correctly is provide a C++/CLI implementation for the call back functions, and the two extern items (command_status and struct command_str).
Can someone provide a sample C++/CLI implementation for the above missing call back functions and externs?
Thanks in advance for your assistance.
in your C++/CLI managed wrapper project, add 2 files :
a .c file :
extern void doSomething();
int command_status = 0;
struct_command_str command_str = { "command1", "p1", "p2", 't' };
int command_completed(int command, long param1, long param2) {
...
command_status = 1;
...
doSomething();
...
command_status = 2;
...
return 3;
}
a cpp file
void doSomethingManagedWrapper() {
...
call managed code
...
}
void doSomething() {
doSomethingManagedWrapper();
}
when you implement these in your c++/cli module, use the same signature shown in the c header file,but prefixed with extern "C".
also put an extern "C" block around the #include of the C header file.