How to write a Raku declaration for a C function returning a whole struct? - raku

I have this C code:
typedef struct {
double dat[2];
} gsl_complex;
gsl_complex gsl_poly_complex_eval(const double c[], const int len, const gsl_complex z);
The C function returns a whole struct, not just a pointer, so I cannot write the Raku declaration as:
sub gsl_poly_complex_eval(CArray[num64] $c, int32 $len, gsl_complex $z --> gsl_complex)
is native(LIB) is export { * }
Any suggestion?

For that you need a CStruct. The P5localtime module contains a more elaborate example.

The problem
Some C APIs work with structs using a three-phase approach, passing around structs by reference, like this:
struct mystruct *init_mystruct(arguments, ...);
double compute(struct mystruct *);
void clean_mystruct(struct mystruct *);
This way the implementation hides the data structure, but this comes with a price: the final users have to keep track of their pointers and remember to clean up after themselves, or the program will leak memory.
Another approach is the one that the library I was interfacing used: return the data on the stack, so it can be assigned to an auto variable and automatically discarded when it goes out of scope.
In this case the API is modeled as a two-phase operation:
struct mystruct init_mystruct(arguments, ...);
double compute(struct mystruct);
The structure is passed on the stack, by value and there's no need to clean up afterwards.
But Raku's NativeCall interface is only able to use C structs passing them by reference, hence the problem.
The solution
This is not a clean solution, because it steps back into the first approach described, the three-phase one, but it's the only one I have been able to devise so far.
Here I consider two C functions from the library's API: the first creates a complex number as a struct, the second adds up two numbers.
First I wrote a tiny C code interface, the file complex.c:
#include <gsl/gsl_complex.h>
#include <gsl/gsl_complex_math.h>
#include <stdlib.h>
gsl_complex *alloc_gsl_complex(void)
{
gsl_complex *c = malloc(sizeof(gsl_complex));
return c;
}
void free_gsl_complex(gsl_complex *c)
{
free(c);
}
void mgsl_complex_rect(double x, double y, gsl_complex *res)
{
gsl_complex ret = gsl_complex_rect(x, y);
*res = ret;
}
void mgsl_complex_add(gsl_complex *a, gsl_complex *b, gsl_complex *res)
{
*res = gsl_complex_add(*a, *b);
}
I compiled it this way:
gcc -c -fPIC complex.c
gcc -shared -o libcomplex.so complex.o -lgsl
Note the final -lgsl used to link the libgsl C library I am interfacing to.
Then I wrote the Raku low-level interface:
#!/usr/bin/env raku
use NativeCall;
constant LIB = ('/mydir/libcomplex.so');
class gsl_complex is repr('CStruct') {
HAS num64 #.dat[2] is CArray;
}
sub mgsl_complex_rect(num64 $x, num64 $y, gsl_complex $c) is native(LIB) { * }
sub mgsl_complex_add(gsl_complex $a, gsl_complex $b, gsl_complex $res) is native(LIB) { * }
sub alloc_gsl_complex(--> gsl_complex) is native(LIB) { * }
sub free_gsl_complex(gsl_complex $c) is native(LIB) { * }
my gsl_complex $c1 = alloc_gsl_complex;
mgsl_complex_rect(1e0, 2e0, $c1);
say "{$c1.dat[0], $c1.dat[1]}"; # output: 1 2
my gsl_complex $c2 = alloc_gsl_complex;
mgsl_complex_rect(1e0, 2e0, $c2);
say "{$c2.dat[0], $c2.dat[1]}"; # output: 1 2
my gsl_complex $res = alloc_gsl_complex;
mgsl_complex_add($c1, $c2, $res);
say "{$res.dat[0], $res.dat[1]}"; # output: 2 4
free_gsl_complex($c1);
free_gsl_complex($c2);
free_gsl_complex($res);
Note that I had to free explicitly the three data structures I created, spoiling the original C API careful design.

Related

How to use CStruct

I'm trying to return struct from shared library written in C. This is simple code, for testing of returning structure and simple int32, libstruct.c, compiled by gcc -shared -Wl,-soname,libstruct.so.1 -o libstruct.so.1 libstruct.c:
#include <stdint.h>
int32_t newint(int32_t arg) {
return arg;
}
struct MyStruct {
int32_t member;
};
struct MyStruct newstruct(int32_t arg) {
struct MyStruct myStruct;
myStruct.member = arg;
return(myStruct);
}
I can use this library with simple C program, usestruct.c, compiled by gcc -o usestruct usestruct.c ./libstruct.so.1:
#include <stdio.h>
#include <stdint.h>
struct MyStruct {
int32_t member;
};
extern struct MyStruct newstruct(int32_t);
extern int32_t newint(int32_t);
int main() {
printf("%d\n", newint(42));
struct MyStruct myStruct;
myStruct = newstruct(42);
printf("%d\n", myStruct.member);
return 0;
}
I can launch it with LD_LIBRARY_PATH=./ ./usestruct, and it works correctly, prints two values. Now, let's to write analogous program in raku, usestruct.raku:
#!/bin/env raku
use NativeCall;
sub newint(int32) returns int32 is native('./libstruct.so.1') { * }
say newint(42);
class MyStruct is repr('CStruct') {
has int32 $.member;
}
sub newstruct(int32) returns MyStruct is native('./libstruct.so.1') { * }
say newstruct(42).member;
This prints first 42, but then terminates with segmentation fault.
In C this example works, but I'm not expert in C, maybe I forgot something, some compile options? Or is this a bug of rakudo?
NativeCall interface requires that transaction of C structs be made with pointers:
CStruct objects are passed to native functions by reference and native functions must also return CStruct objects by reference.
Your C function, however, returns a new struct by value. Then, i guess, this is tried to be interpreted as a memory address as it expects a pointer, and tries to read/write from wild memory areas, hence the segfault.
You can pointerize your function as:
struct MyStruct* newstruct(int32_t val) {
/* dynamically allocating now */
struct MyStruct *stru = malloc(sizeof *stru);
stru->member = val;
return stru;
}
with #include <stdlib.h> at the very top for malloc. Raku program is essentially the same modulo some aesthetics:
# prog.raku
use NativeCall;
my constant LIB = "./libstruct.so";
class MyStruct is repr("CStruct") {
has int32 $.member;
}
# C bridge
sub newint(int32) returns int32 is native(LIB) { * }
sub newstruct(int32) returns MyStruct is native(LIB) { * }
say newint(42);
my $s := newstruct(84);
say $s;
say $s.member;
We build the lib & run the Raku program to get
$ gcc -Wall -Wextra -pedantic -shared -o libstruct.so -fPIC mod_struct.c
$ raku prog.raku
42
MyStruct.new(member => 84)
84
(took the liberty to rename C file to "mod_struct.c")
Seems good. But there's an issue: now that a dynamic allocation was made, responsibility to deliver it back arises. And we need to do it ourselves with a C-bridged freer:
When a CStruct-based type is used as the return type of a native function, the memory is not managed for you by the GC.
So
/* addendum to mod_struct.c */
void free_struct(struct MyStruct* s) {
free(s);
}
Noting that, since the struct itself didn't have dynamic allocations on its members (as it only has an integer), we didn't do further freeing.
Now the Raku program needs to be aware of this, and use it:
# prog.raku
use NativeCall;
my constant LIB = "./libstruct.so";
class MyStruct is repr("CStruct") {
has int32 $.member;
}
# C bridge
sub newint(int32) returns int32 is native(LIB) { * }
sub newstruct(int32) returns MyStruct is native(LIB) { * }
sub free_struct(MyStruct) is native(LIB) { * }; # <-- new!
say newint(42);
my $s := newstruct(84);
say $s;
say $s.member;
# ... after some time
free_struct($s);
say "successfully freed struct";
and the output follows as
42
MyStruct.new(member => 84)
84
successfully freed struct
Manually keeping track of MyStruct objects to remember freeing them after some time might be cumbersome; that would be writing C! In the Raku level, we already have a class representing the struct; then we can add a DESTROY submethod to it that frees itself whenever garbage collector deems necessary:
class MyStruct is repr("CStruct") {
has int32 $.member;
submethod DESTROY {
free_struct(self);
}
}
With this addition, no manual calls to free_struct is needed (in fact, better not because it might lead double freeing which is undefined behaviour on C level).
P.S. your main C file might be revisioned, e.g., a header file seems in order but that's out of scope or that was only a demonstrative example who knows. In either case, thanks for providing an MRE and welcome to the website.
In addition to great #Mustafa's answer.
I found another way to solve my problem: we can allocate structure in raku and pass it to C function. Here is an example, file mod_struct.c:
#include <stdint.h>
struct MyStruct {
int32_t member;
};
void writestruct(struct MyStruct *outputStruct, int32_t arg) {
outputStruct->member = arg;
}
File usestruct.raku:
#!/bin/env raku
use NativeCall;
class MyStruct is repr('CStruct') {
has int32 $.member;
}
sub writestruct(MyStruct is rw, int32) is native('./libstruct.so') { * }
my $myStruct = MyStruct.new;
writestruct($myStruct, 42);
say $myStruct.member;
Compile and run it:
$ gcc -Wall -Wextra -pedantic -shared -o libstruct.so -fPIC mod_struct.c
$ ./usestruct.raku
42

C++ Builder Function error [bcc32 - Ambiguity error] inside dll file

I am creating a currency converter Win32 program in Embarcadero C++Builder. I wrote a function for transforming date from format specified on user PC to YYYY-MM-DD format. I need that part because of API settings.
When I have this function inside my project it works fine, but I need to have that function inside a DLL.
This is how my code looks like:
#pragma hdrstop
#pragma argsused
#include <SysUtils.hpp>
extern DELPHI_PACKAGE void __fastcall DecodeDate(const System::TDateTime DateTime, System::Word &Year, System::Word &Month, System::Word &Day);
extern "C" UnicodeString __declspec (dllexport) __stdcall datum(TDateTime dat) {
Word dan, mjesec, godina;
UnicodeString datum, datum_dan, datum_mjesec, datum_godina;
DecodeDate(dat, godina, mjesec, dan);
if (dan<=9 && mjesec<=9) {
datum_dan="0"+IntToStr(dan);
datum_mjesec="0"+IntToStr(mjesec);
}
if (dan<=9 && mjesec>9) {
datum_dan="0"+IntToStr(dan);
datum_mjesec=IntToStr(mjesec);
}
if (dan>9 && mjesec<=9) {
datum_dan=IntToStr(dan);
datum_mjesec="0"+IntToStr(mjesec);
}
if (dan>9 && mjesec>9) {
datum_dan=IntToStr(dan);
datum_mjesec=IntToStr(mjesec);
}
datum_godina=IntToStr(godina);
return datum_godina+"-"+datum_mjesec+"-"+datum_dan;
}
extern "C" int _libmain(unsigned long reason)
{
return 1;
}
`
I've included SysUtils.hpp and declared DecodeDate() function, without those lines I have a million errors. But with code looking like this, I am getting this error, which I can't get rid of:
[bcc32 Error] File1.cpp(30): E2015 Ambiguity between '_fastcall System::Sysutils::DecodeDate(const System::TDateTime,unsigned short &,unsigned short &,unsigned short &) at c:\program files (x86)\embarcadero\studio\19.0\include\windows\rtl\System.SysUtils.hpp:3466' and '_fastcall DecodeDate(const System::TDateTime,unsigned short &,unsigned short &,unsigned short &) at File1.cpp:25'
Full parser context
File1.cpp(27): parsing: System::UnicodeString __stdcall datum(System::TDateTime)
Can you help me to get rid of that error?
The error message is self-explanatory. You have two functions with the same name in scope, and the compiler doesn't know which one you want to use on line 30 because the parameters you are passing in satisfy both function declarations.
To fix the error, you can change this line:
DecodeDate(dat, godina, mjesec, dan);
To either this:
System::Sysutils::DecodeDate(dat, godina, mjesec, dan);
Or this:
dat.DecodeDate(&godina, &mjesec, &dan);
However, either way, you should get rid of your extern declaration for DecodeDate(), as it doesn't belong in this code at all. You are not implementing DecodeDate() yourself, you are just using the one provided by the RTL. There is already a declaration for DecodeDate() in SysUtils.hpp, which you are #include'ing in your code. That is all the compiler needs.
Just make sure you are linking to the RTL/VCL libraries to resolve the function during the linker stage after compiling. You should have enabled VCL support when you created the DLL project. If you didn't, recreate your project and enable it.
BTW, there is a MUCH easier way to implement your function logic - instead of manually pulling apart the TDateTime and reconstituting its components, just use the SysUtils::FormatDateTime() function or the TDateTime::FormatString() method instead, eg:
UnicodeString __stdcall datum(TDateTime dat)
{
return FormatDateTime(_D("yyyy'-'mm'-'dd"), dat);
}
UnicodeString __stdcall datum(TDateTime dat)
{
return dat.FormatString(_D("yyyy'-'mm'-'dd"));
}
That being said, this code is still wrong, because it is not safe to pass non-POD types, like UnicodeString, over the DLL boundary like you are doing. You need to re-think your DLL function design to use only interop-safe POD types. In this case, change your function to either:
take a wchar_t* as input from the caller, and just fill in the memory block with the desired characters. Let the caller allocate the actual buffer and pass it in to your DLL for populating:
#pragma hdrstop
#pragma argsused
#include <SysUtils.hpp>
extern "C" __declspec(dllexport) int __stdcall datum(double dat, wchar_t *buffer, int buflen)
{
UnicodeString s = FormatDateTime(_D("yyyy'-'mm'-'dd"), dat);
if (!buffer) return s.Length() + 1;
StrLCopy(buffer, s.c_str(), buflen-1);
return StrLen(buffer);
}
extern "C" int _libmain(unsigned long reason)
{
return 1;
}
wchar_t buffer[12] = {};
datum(SomeDateValueHere, buffer, 12);
// use buffer as needed...
int len = datum(SomeDateValueHere, NULL, 0);
wchar_t *buffer = new wchar_t[len];
int len = datum(SomeDateValueHere, buffer, len);
// use buffer as needed...
delete[] buffer;
allocate a wchar_t[] buffer to hold the desired characters, and then return a wchar_t* pointer to that buffer to the caller. Then export a second function that the caller can pass the returned wchar_t* back to you so you can free it correctly.
#pragma hdrstop
#pragma argsused
#include <SysUtils.hpp>
extern "C" __declspec(dllexport) wchar_t* __stdcall datum(double dat)
{
UnicodeString s = FormatDateTime("yyyy'-'mm'-'dd", dat);
wchar_t* buffer = new wchar_t[s.Length()+1];
StrLCopy(buffer, s.c_str(), s.Length());
return buffer;
}
extern "C" __declspec(dllexport) void __stdcall free_datum(wchar_t *dat)
{
delete[] dat;
}
extern "C" int _libmain(unsigned long reason)
{
return 1;
}
wchar_t *buffer = datum(SomeDateValueHere);
// use buffer as needed...
free_datum(buffer);

Googlemock - multiple actions in WillOnce cause build error

I am already using gtest for some time but recently wanted to try out gmock. I am trying to mock class with method that returns value but also returns something in output parameter through reference. Here is my small code.
#include <vector>
#include "gmock/gmock.h"
#include "gtest/gtest.h"
using namespace ::testing;
class AReal
{
public:
virtual bool foo(std::vector<int>& v) const = 0;
};
class AMock : public AReal
{
public:
MOCK_CONST_METHOD1(foo, bool(std::vector<int>&));
};
class B
{
public:
B(AReal* _a) : a(_a) {}
bool foo(std::vector<int>& v) const { return a->foo(v); }
private:
AReal* a;
};
class FooTest : public Test {};
TEST_F(FooTest,
DummyTestVector) {
AMock a;
B b(&a);
std::vector<int> exp = { 1, 2, 3 };
EXPECT_CALL(a, foo(_))
.Times(1)
.WillOnce(AllOf(SetArgReferee<0>(exp), Return(true)));
std::vector<int> load;
EXPECT_TRUE(a.foo(load));
EXPECT_EQ(exp, load);
}
int main(int argc, char** argv) {
::testing::InitGoogleMock(&argc, argv);
return RUN_ALL_TESTS();
}
However, this code gives me this error.
$ g++ -Wall -Wextra -std=c++14 -I. -o test test.cpp gmock-gtest-all.cc -lpthread
test.cpp: In member function ‘virtual void FooTest_DummyTestVector_Test::TestBody()’:
test.cpp:40:61: error: no matching function for call to ‘testing::internal::TypedExpectation<bool(std::vector<int>&)>::WillOnce(testing::internal::AllOfResult2<testing::SetArgRefereeActionP<0, std::vector<int> >, testing::internal::ReturnAction<bool> >::type)’
.WillOnce(AllOf(SetArgReferee<0>(exp), Return(true)));
^
In file included from test.cpp:2:0:
gmock/gmock.h:10172:21: note: candidate: testing::internal::TypedExpectation<F>& testing::internal::TypedExpectation<F>::WillOnce(const testing::Action<F>&) [with F = bool(std::vector<int>&)]
TypedExpectation& WillOnce(const Action<F>& action) {
^
gmock/gmock.h:10172:21: note: no known conversion for argument 1 from ‘testing::internal::AllOfResult2<testing::SetArgRefereeActionP<0, std::vector<int> >, testing::internal::ReturnAction<bool> >::type {aka testing::internal::BothOfMatcher<testing::SetArgRefereeActionP<0, std::vector<int> >, testing::internal::ReturnAction<bool> >}’ to ‘const testing::Action<bool(std::vector<int>&)>&’
If I don't use AllOf but rather specify just one action, either SetArgReferee or Return, everything works just fine. The use of AllOf causes this kind of error. I have found about AllOf here gmock multiple in-out parameters SetArgReferee and basically my code is same as the answer.
After one whole afternoon of trying everything I found out it was just my stupidity. All the time I somehow thought that AllOf == DoAll. Realized it just now.

Static Parameter Function Specialization in D

I've read somewhere that D supports specialization of functions to calls where arguments are compile-time constants. Typical use of this is in matrix power functions (if exponent is 2 x*x is often faster than the general case).
I want this in my member function
bool opIndexAssign(bool b, size_t i) #trusted pure nothrow in {
assert(i < len); // TODO: Add static assert(i < len) when i is constant
} body {
b ? bts(ptr, i) : btr(ptr, i);
return b;
}
of a statically sized BitSet struct I'm writing. This in order to, when possible, get compile-time bounds checking on the index variable i. I thought
bool opIndexAssign(bool b, const size_t i) #trusted pure nothrow in {
static assert(i < len);
} body {
b ? bts(ptr, i) : btr(ptr, i);
return b;
}
would suffice but then DMD complains as follows
dmd -debug -gc -gs -unittest -D -Dd/home/per/.emacs.d/auto-builds/dmd/Debug-Boundscheck-Unittest/home/per/Work/justd/ -w -main ~/Work/justd/bitset.d /home/per/Work/justd/assert_ex.d -of/home/per/.emacs.d/auto-builds/dmd/Debug-Boundscheck-Unittest/home/per/Work/justd/bitset
/home/per/Work/justd/bitset.d(58): Error: bitset.BitSet!2.BitSet.opIndexAssign called with argument types (bool, int) matches both:
/home/per/Work/justd/bitset.d(49): opIndexAssign(bool b, ulong i)
and:
/home/per/Work/justd/bitset.d(65): opIndexAssign(bool b, const(ulong) i)
/home/per/Work/justd/bitset.d(66): Error: variable i cannot be read at compile time
/home/per/Work/justd/bitset.d(66): while evaluating: static assert(i < 2LU)
/home/per/Work/justd/bitset.d(58): Error: bitset.BitSet!2.BitSet.opIndexAssign called with argument types (bool, int) matches both:
/home/per/Work/justd/bitset.d(49): opIndexAssign(bool b, ulong i)
Do I have to make parameter i a template parameter, say using type U, and then use static if someTypeTrait!U. I tried this but isMutable!Index always evaluates to true.
import std.traits: isIntegral;
bool opIndexAssign(Index)(bool b, Index i) #trusted pure nothrow if (isIntegral!Index) in {
import std.traits: isMutable;
// See also: http://stackoverflow.com/questions/19906516/static-parameter-function-specialization-in-d
static if (isMutable!Index) {
assert(i < len);
} else {
import std.conv: to;
static assert(i < len,
"Index " ~ to!string(i) ~ " must be smaller than BitSet length " ~ to!string(len));
}
} body {
b ? bts(ptr, i) : btr(ptr, i);
return b;
}
What you're trying to do doesn't really work. You can do a template value parameter:
void foo(int i)() { /* use i at compile time */ }
but then you can't pass a runtime value to it, and it has different call syntax: foo!2 vs foo(2).
The closest you can get is is CTFE:
int foo(int i) { return i; }
enum something = foo(2); // works, evaluated at compile time
int s = foo(2); // also works, but runs at runtime.
Inside the function, there is a magic if(__ctfe) { running at compile time } else { at runtime}, but again, this isn't if there's a literal, it is if the function is run in a CT context, e.g., assigning the result to an enum constant.
But, otherwise, an int literal is still a mutable int as far as the function is concerned. So what you're specifically trying to do won't work in D as it is right now. (There's been some talk about wanting a way to tell if it is a literal, but as far as I know, there's no plan to actually do it.)

Trying to parse OpenCV YAML ouput with yaml-cpp

I've got a series of OpenCv generated YAML files and would like to parse them with yaml-cpp
I'm doing okay on simple stuff, but the matrix representation is proving difficult.
# Center of table
tableCenter: !!opencv-matrix
rows: 1
cols: 2
dt: f
data: [ 240, 240]
This should map into the vector
240
240
with type float. My code looks like:
#include "yaml.h"
#include <fstream>
#include <string>
struct Matrix {
int x;
};
void operator >> (const YAML::Node& node, Matrix& matrix) {
unsigned rows;
node["rows"] >> rows;
}
int main()
{
std::ifstream fin("monsters.yaml");
YAML::Parser parser(fin);
YAML::Node doc;
Matrix m;
doc["tableCenter"] >> m;
return 0;
}
But I get
terminate called after throwing an instance of 'YAML::BadDereference'
what(): yaml-cpp: error at line 0, column 0: bad dereference
Abort trap
I searched around for some documentation for yaml-cpp, but there doesn't seem to be any, aside from a short introductory example on parsing and emitting. Unfortunately, neither of these two help in this particular circumstance.
As I understand, the !! indicate that this is a user-defined type, but I don't see with yaml-cpp how to parse that.
You have to tell yaml-cpp how to parse this type. Since C++ isn't dynamically typed, it can't detect what data type you want and create it from scratch - you have to tell it directly. Tagging a node is really only for yourself, not for the parser (it'll just faithfully store it for you).
I'm not really sure how an OpenCV matrix is stored, but if it's something like this:
class Matrix {
public:
Matrix(unsigned r, unsigned c, const std::vector<float>& d): rows(r), cols(c), data(d) { /* init */ }
Matrix(const Matrix&) { /* copy */ }
~Matrix() { /* delete */ }
Matrix& operator = (const Matrix&) { /* assign */ }
private:
unsigned rows, cols;
std::vector<float> data;
};
then you can write something like
void operator >> (const YAML::Node& node, Matrix& matrix) {
unsigned rows, cols;
std::vector<float> data;
node["rows"] >> rows;
node["cols"] >> cols;
node["data"] >> data;
matrix = Matrix(rows, cols, data);
}
Edit It appears that you're ok up until here; but you're missing the step where the parser loads the information into the YAML::Node. Instead, se it like:
std::ifstream fin("monsters.yaml");
YAML::Parser parser(fin);
YAML::Node doc;
parser.GetNextDocument(doc); // <-- this line was missing!
Matrix m;
doc["tableCenter"] >> m;
Note: I'm guessing dt: f means "data type is float". If that's the case, it'll really depend on how the Matrix class handles this. If you have a different class for each data type (or a templated class), you'll have to read that field first, and then choose which type to instantiate. (If you know it'll always be float, that'll make your life easier, of course.)