2023 update The last person to edit this Q deleted the critically important "LATEST LATEST UPDATE" part that #zentrunix had added near the top. I'm reinstating it.
LATEST LATEST UPDATE
Please see my answer below.
Thanks to everyone who took the time to answer and understand this question.
Original question
Say I have my event-driven TCP communications library in C.
From my Raku application, I can call a function in the C library using NativeCall.
my $server = create-server("127.0.0.1", 4000);
Now, from my callback in C (say onAccept) I want to call out to a Raku function in my application (say on-accept(connection) where connection will be a pointer to a C struct).
So, how can I do that: call my Raku function on-accept from my C function onAccept ?
ps. I tried posting using a simple title "How to call Raku code from C code", but for whatever reason stackoverflow.com wouldn't let me do it. Because of that I concocted this fancy title.
I was creating a 32-bit DLL.
We have to explicitly tell CMake to configure a 64-bit build.
cmake -G "Visual Studio 14 2015 Win64" ..
Anyway, now that the code runs, it's not really what I asked for, because the callback is still in C.
It seems that what I asked for it's not really possible.
I tried to use the approach suggested by Haakon, though I'm afraid I don't understand how it would work.
I'm in Windows, and unfortunately, Raku can't find my dlls, even if I put them in C:\Windows\System32. It finds "msvcrt" (C runtime), but not my dlls.
The dll code (Visual Studio 2015).
#include <stdio.h>
#define EXPORTED __declspec(dllexport)
typedef int (*proto)(const char*);
proto raku_callback;
extern EXPORTED void set_callback(proto);
extern EXPORTED void foo(void);
void set_callback(proto arg)
{
printf("In set_callback()..\n");
raku_callback = arg;
}
void foo(void)
{
printf("In foo()..\n");
int res = raku_callback("hello");
printf("Raku return value: %d\n", res);
}
Cmake code for the
CMAKE_MINIMUM_REQUIRED (VERSION 3.1)
add_library (my_c_dll SHARED my_c_dll.c)
Raku code.
use v6.d;
use NativeCall;
sub set_callback(&callback (Str --> int32))
is native("./my_c_dll"){ * }
sub foo()
is native("./my_c_dll"){ * }
sub callback(Str $str --> Int) {
say "Raku callback.. got string: {$str} from C";
return 32;
}
## sub _getch() returns int32 is native("msvcrt") {*};
## print "-> ";
## say "got ", _getch();
set_callback(&callback);
# foo();
When I run
$ raku test-dll.raku
Cannot locate native library '(null)': error 0xc1
in method setup at D:\tools\raku\share\perl6\core\sources
\947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 298
in block set_callback at D:\tools\raku\share\perl6\core\sources
\947BDAB9F96E0E5FCCB383124F923A6BF6F8D76B (NativeCall) line 594
in block <unit> at test-dll.raku line 21
Raku version.
$ raku -v
This is Rakudo version 2020.05.1 built on MoarVM version 2020.05
implementing Raku 6.d.
Another approach could be to save a callback statically in the C library, for example (libmylib.c):
#include <stdio.h>
static int (*raku_callback)(char *arg);
void set_callback(int (*callback)(char * arg)) {
printf("In set_callback()..\n");
raku_callback = callback;
}
void foo() {
printf("In foo()..\n");
int res = raku_callback("hello");
printf("Raku return value: %d\n", res);
}
Then from Raku:
use v6;
use NativeCall;
sub set_callback(&callback (Str --> int32)) is native('./libmylib.so') { * }
sub foo() is native('./libmylib.so') { * }
sub callback(Str $str --> Int) {
say "Raku callback.. got string: {$str} from C";
return 32;
}
set_callback(&callback);
foo();
Output:
In set_callback()..
In foo()..
Raku callback.. got string: hello from C
Raku return value: 32
Raku is a compiled language; depending on the implementation you've got, it will be compiled to MoarVM, JVM or Javascript. Through compilation, Raku code becomes bytecode in the corresponding virtual machine. So it's never, actually, binary code.
However, Raku code seems to be cleverly organized in a way that an object is actually a pointer to a C endpoint, as proved by Haakon Hagland answer.
WRT to your latest problem, please bear in mind that what you are calling is not a path, but a name that is converted to a navive shared library name and also uses local library path conventions to look for them (it's `PATH' on Windows). So if it's not finding it, add local path to it of simply copy the DLL to one of the searched directories.
First of all, my apologies to #Håkon and #raiph.
Sorry for being so obtuse. :)
Håkon's answer does indeed answer my question, although for whatever reason I have failed to see that until now.
Now the code I played with in order to understand Håkon's solution.
// my_c_dll.c
// be sure to create a 64-bit dll
#include <stdio.h>
#define EXPORTED __declspec(dllexport)
typedef int (*proto)(const char*);
proto raku_function;
extern EXPORTED void install_raku_function(proto);
extern EXPORTED void start_c_processing(void);
void install_raku_function(proto arg)
{
printf("installing raku function\n");
raku_function = arg;
}
void start_c_processing(void)
{
printf("* ----> starting C processing..\n");
for (int i = 0; i < 100; i++)
{
printf("* %d calling raku function\n", i);
int res = raku_function("hello");
printf("* %d raku function returned: %d\n", i, res);
Sleep(1000);
}
}
# test-dll.raku
use v6.d;
use NativeCall;
sub install_raku_function(&raku_function (Str --> int32))
is native("./my_c_dll.dll") { * }
sub start_c_processing()
is native("./my_c_dll.dll") { * }
sub my_raku_function(Str $str --> Int)
{
say "# raku function called from C with parameter [{$str}]";
return 32;
}
install_raku_function &my_raku_function;
start { start_c_processing; }
for ^1000 -> $i
{
say "# $i idling in raku";
sleep 1;
}
$ raku test-dll.raku
installing raku function
# 0 idling in raku
* ----> starting C processing..
* 0 calling raku function
# 0 raku function called from C with parameter [hello]
* 0 raku function returned: 32
# 1 idling in raku
* 1 calling raku function
# 1 raku function called from C with parameter [hello]
* 1 raku function returned: 32
# 2 idling in raku
* 2 calling raku function
# 2 raku function called from C with parameter [hello]
* 2 raku function returned: 32
# 3 idling in raku
* 3 calling raku function
# 3 raku function called from C with parameter [hello]
* 3 raku function returned: 32
# 4 idling in raku
* 4 calling raku function
# 4 raku function called from C with parameter [hello]
* 4 raku function returned: 32
# 5 idling in raku
* 5 calling raku function
# 5 raku function called from C with parameter [hello]
* 5 raku function returned: 32
^CTerminate batch job (Y/N)?
^C
What amazes me is that the Raku signature for my_raku_function maps cleanly to the C signature ... isn't Raku wonderful ? :)
Related
I'm trying to return struct from shared library written in C. This is simple code, for testing of returning structure and simple int32, libstruct.c, compiled by gcc -shared -Wl,-soname,libstruct.so.1 -o libstruct.so.1 libstruct.c:
#include <stdint.h>
int32_t newint(int32_t arg) {
return arg;
}
struct MyStruct {
int32_t member;
};
struct MyStruct newstruct(int32_t arg) {
struct MyStruct myStruct;
myStruct.member = arg;
return(myStruct);
}
I can use this library with simple C program, usestruct.c, compiled by gcc -o usestruct usestruct.c ./libstruct.so.1:
#include <stdio.h>
#include <stdint.h>
struct MyStruct {
int32_t member;
};
extern struct MyStruct newstruct(int32_t);
extern int32_t newint(int32_t);
int main() {
printf("%d\n", newint(42));
struct MyStruct myStruct;
myStruct = newstruct(42);
printf("%d\n", myStruct.member);
return 0;
}
I can launch it with LD_LIBRARY_PATH=./ ./usestruct, and it works correctly, prints two values. Now, let's to write analogous program in raku, usestruct.raku:
#!/bin/env raku
use NativeCall;
sub newint(int32) returns int32 is native('./libstruct.so.1') { * }
say newint(42);
class MyStruct is repr('CStruct') {
has int32 $.member;
}
sub newstruct(int32) returns MyStruct is native('./libstruct.so.1') { * }
say newstruct(42).member;
This prints first 42, but then terminates with segmentation fault.
In C this example works, but I'm not expert in C, maybe I forgot something, some compile options? Or is this a bug of rakudo?
NativeCall interface requires that transaction of C structs be made with pointers:
CStruct objects are passed to native functions by reference and native functions must also return CStruct objects by reference.
Your C function, however, returns a new struct by value. Then, i guess, this is tried to be interpreted as a memory address as it expects a pointer, and tries to read/write from wild memory areas, hence the segfault.
You can pointerize your function as:
struct MyStruct* newstruct(int32_t val) {
/* dynamically allocating now */
struct MyStruct *stru = malloc(sizeof *stru);
stru->member = val;
return stru;
}
with #include <stdlib.h> at the very top for malloc. Raku program is essentially the same modulo some aesthetics:
# prog.raku
use NativeCall;
my constant LIB = "./libstruct.so";
class MyStruct is repr("CStruct") {
has int32 $.member;
}
# C bridge
sub newint(int32) returns int32 is native(LIB) { * }
sub newstruct(int32) returns MyStruct is native(LIB) { * }
say newint(42);
my $s := newstruct(84);
say $s;
say $s.member;
We build the lib & run the Raku program to get
$ gcc -Wall -Wextra -pedantic -shared -o libstruct.so -fPIC mod_struct.c
$ raku prog.raku
42
MyStruct.new(member => 84)
84
(took the liberty to rename C file to "mod_struct.c")
Seems good. But there's an issue: now that a dynamic allocation was made, responsibility to deliver it back arises. And we need to do it ourselves with a C-bridged freer:
When a CStruct-based type is used as the return type of a native function, the memory is not managed for you by the GC.
So
/* addendum to mod_struct.c */
void free_struct(struct MyStruct* s) {
free(s);
}
Noting that, since the struct itself didn't have dynamic allocations on its members (as it only has an integer), we didn't do further freeing.
Now the Raku program needs to be aware of this, and use it:
# prog.raku
use NativeCall;
my constant LIB = "./libstruct.so";
class MyStruct is repr("CStruct") {
has int32 $.member;
}
# C bridge
sub newint(int32) returns int32 is native(LIB) { * }
sub newstruct(int32) returns MyStruct is native(LIB) { * }
sub free_struct(MyStruct) is native(LIB) { * }; # <-- new!
say newint(42);
my $s := newstruct(84);
say $s;
say $s.member;
# ... after some time
free_struct($s);
say "successfully freed struct";
and the output follows as
42
MyStruct.new(member => 84)
84
successfully freed struct
Manually keeping track of MyStruct objects to remember freeing them after some time might be cumbersome; that would be writing C! In the Raku level, we already have a class representing the struct; then we can add a DESTROY submethod to it that frees itself whenever garbage collector deems necessary:
class MyStruct is repr("CStruct") {
has int32 $.member;
submethod DESTROY {
free_struct(self);
}
}
With this addition, no manual calls to free_struct is needed (in fact, better not because it might lead double freeing which is undefined behaviour on C level).
P.S. your main C file might be revisioned, e.g., a header file seems in order but that's out of scope or that was only a demonstrative example who knows. In either case, thanks for providing an MRE and welcome to the website.
In addition to great #Mustafa's answer.
I found another way to solve my problem: we can allocate structure in raku and pass it to C function. Here is an example, file mod_struct.c:
#include <stdint.h>
struct MyStruct {
int32_t member;
};
void writestruct(struct MyStruct *outputStruct, int32_t arg) {
outputStruct->member = arg;
}
File usestruct.raku:
#!/bin/env raku
use NativeCall;
class MyStruct is repr('CStruct') {
has int32 $.member;
}
sub writestruct(MyStruct is rw, int32) is native('./libstruct.so') { * }
my $myStruct = MyStruct.new;
writestruct($myStruct, 42);
say $myStruct.member;
Compile and run it:
$ gcc -Wall -Wextra -pedantic -shared -o libstruct.so -fPIC mod_struct.c
$ ./usestruct.raku
42
I have this C code:
typedef struct {
double dat[2];
} gsl_complex;
gsl_complex gsl_poly_complex_eval(const double c[], const int len, const gsl_complex z);
The C function returns a whole struct, not just a pointer, so I cannot write the Raku declaration as:
sub gsl_poly_complex_eval(CArray[num64] $c, int32 $len, gsl_complex $z --> gsl_complex)
is native(LIB) is export { * }
Any suggestion?
For that you need a CStruct. The P5localtime module contains a more elaborate example.
The problem
Some C APIs work with structs using a three-phase approach, passing around structs by reference, like this:
struct mystruct *init_mystruct(arguments, ...);
double compute(struct mystruct *);
void clean_mystruct(struct mystruct *);
This way the implementation hides the data structure, but this comes with a price: the final users have to keep track of their pointers and remember to clean up after themselves, or the program will leak memory.
Another approach is the one that the library I was interfacing used: return the data on the stack, so it can be assigned to an auto variable and automatically discarded when it goes out of scope.
In this case the API is modeled as a two-phase operation:
struct mystruct init_mystruct(arguments, ...);
double compute(struct mystruct);
The structure is passed on the stack, by value and there's no need to clean up afterwards.
But Raku's NativeCall interface is only able to use C structs passing them by reference, hence the problem.
The solution
This is not a clean solution, because it steps back into the first approach described, the three-phase one, but it's the only one I have been able to devise so far.
Here I consider two C functions from the library's API: the first creates a complex number as a struct, the second adds up two numbers.
First I wrote a tiny C code interface, the file complex.c:
#include <gsl/gsl_complex.h>
#include <gsl/gsl_complex_math.h>
#include <stdlib.h>
gsl_complex *alloc_gsl_complex(void)
{
gsl_complex *c = malloc(sizeof(gsl_complex));
return c;
}
void free_gsl_complex(gsl_complex *c)
{
free(c);
}
void mgsl_complex_rect(double x, double y, gsl_complex *res)
{
gsl_complex ret = gsl_complex_rect(x, y);
*res = ret;
}
void mgsl_complex_add(gsl_complex *a, gsl_complex *b, gsl_complex *res)
{
*res = gsl_complex_add(*a, *b);
}
I compiled it this way:
gcc -c -fPIC complex.c
gcc -shared -o libcomplex.so complex.o -lgsl
Note the final -lgsl used to link the libgsl C library I am interfacing to.
Then I wrote the Raku low-level interface:
#!/usr/bin/env raku
use NativeCall;
constant LIB = ('/mydir/libcomplex.so');
class gsl_complex is repr('CStruct') {
HAS num64 #.dat[2] is CArray;
}
sub mgsl_complex_rect(num64 $x, num64 $y, gsl_complex $c) is native(LIB) { * }
sub mgsl_complex_add(gsl_complex $a, gsl_complex $b, gsl_complex $res) is native(LIB) { * }
sub alloc_gsl_complex(--> gsl_complex) is native(LIB) { * }
sub free_gsl_complex(gsl_complex $c) is native(LIB) { * }
my gsl_complex $c1 = alloc_gsl_complex;
mgsl_complex_rect(1e0, 2e0, $c1);
say "{$c1.dat[0], $c1.dat[1]}"; # output: 1 2
my gsl_complex $c2 = alloc_gsl_complex;
mgsl_complex_rect(1e0, 2e0, $c2);
say "{$c2.dat[0], $c2.dat[1]}"; # output: 1 2
my gsl_complex $res = alloc_gsl_complex;
mgsl_complex_add($c1, $c2, $res);
say "{$res.dat[0], $res.dat[1]}"; # output: 2 4
free_gsl_complex($c1);
free_gsl_complex($c2);
free_gsl_complex($res);
Note that I had to free explicitly the three data structures I created, spoiling the original C API careful design.
This is a follow-up question to "How to declare native array of fixed size in Perl 6?".
In that question it was discussed how to incorporate an array of a fixed size into a CStruct. In this answer it was suggested to use HAS to inline a CArray in the CStruct. When I tested this idea, I ran into some strange behavior that could not be resolved in the comments section below the question, so I decided to write it up as a new question. Here is is my C test library code:
slib.c:
#include <stdio.h>
struct myStruct
{
int A;
int B[3];
int C;
};
void use_struct (struct myStruct *s) {
printf("sizeof(struct myStruct): %ld\n", sizeof( struct myStruct ));
printf("sizeof(struct myStruct *): %ld\n", sizeof( struct myStruct *));
printf("A = %d\n", s->A);
printf("B[0] = %d\n", s->B[0]);
printf("B[1] = %d\n", s->B[1]);
printf("B[2] = %d\n", s->B[2]);
printf("C = %d\n", s->C);
}
To generate a shared library from this i used:
gcc -c -fpic slib.c
gcc -shared -o libslib.so slib.o
Then, the Perl 6 code:
p.p6:
use v6;
use NativeCall;
class myStruct is repr('CStruct') {
has int32 $.A is rw;
HAS int32 #.B[3] is CArray is rw;
has int32 $.C is rw;
}
sub use_struct(myStruct $s) is native("./libslib.so") { * };
my $s = myStruct.new();
$s.A = 1;
$s.B[0] = 2;
$s.B[1] = 3;
$s.B[2] = 4;
$s.C = 5;
say "Expected size of Perl 6 struct: ", (nativesizeof(int32) * 5);
say "Actual size of Perl 6 struct: ", nativesizeof( $s );
say 'Number of elements of $s.B: ', $s.B.elems;
say "B[0] = ", $s.B[0];
say "B[1] = ", $s.B[1];
say "B[2] = ", $s.B[2];
say "Calling library function..";
say "--------------------------";
use_struct( $s );
The output from the script is:
Expected size of Perl 6 struct: 20
Actual size of Perl 6 struct: 24
Number of elements of $s.B: 3
B[0] = 2
B[1] = 3
B[2] = 4
Calling library function..
--------------------------
sizeof(struct myStruct): 20
sizeof(struct myStruct *): 8
A = 1
B[0] = 0 # <-- Expected 2
B[1] = 653252032 # <-- Expected 3
B[2] = 22030 # <-- Expected 4
C = 5
Questions:
Why does nativesizeof( $s ) give 24 (and not the expected value of 20)?
Why is the content of the array B in the structure not as expected when printed from the C function?
Note:
I am using Ubuntu 18.04 and Perl 6 Rakudo version 2018.04.01, but have also tested with version 2018.05
Your code is correct. I just fixed that bug in MoarVM, and added tests to rakudo, similar to your code:
In C:
typedef struct {
int a;
int b[3];
int c;
} InlinedArrayInStruct;
In Perl 6:
class InlinedArrayInStruct is repr('CStruct') {
has int32 $.a is rw;
HAS int32 #.b[3] is CArray;
has int32 $.c is rw;
}
See these patches:
https://github.com/MoarVM/MoarVM/commit/ac3d3c76954fa3c1b1db14ea999bf3248c2eda1c
https://github.com/rakudo/rakudo/commit/f8b79306cc1900b7991490eef822480f304a56d9
If you are not building rakudo (and also NQP and MoarVM) directly from latest source from github, you probably have to wait for the 2018.08 release that will appear here: https://rakudo.org/files
I'm trying to write a plugin that requires evaluating combinatorial circuits. From what I can gather ConstEval is the tool which does this. However, the API is not so clear to me. Is there somewhere a rundown of the members of ConstEval and what they do?
(Asked by jeremysalwen on github).
Using the ConstEval class is actually quite easy. You create a ConstEval object for a given module and set the known values using the void ConstEval::set(SigSpec, Const) method. After all the known signals have been set, the bool ConstEval::eval(SigSpec&, SigSpec&) method can be used to evaluate nets. The eval() method returns true when the evaluation was successful and replaces the net(s) in the first argument with the constant values the net evaluates to. Otherwise it returns false and sets the 2nd argument to the list of nets that need to be set in order to continue evaluation.
The methods push() and pop() can be used for creating local contexts for set(). The method stop() can be used to declare signals at which the evaluation should stop, even when there are combinatoric cells driving the net.
The following simple Yosys plugin demonstrates how to use the ConstEval API (evaldemo.cc):
#include "kernel/yosys.h"
#include "kernel/consteval.h"
USING_YOSYS_NAMESPACE
PRIVATE_NAMESPACE_BEGIN
struct EvalDemoPass : public Pass
{
EvalDemoPass() : Pass("evaldemo") { }
virtual void execute(vector<string>, Design *design)
{
Module *module = design->top_module();
if (module == nullptr)
log_error("No top module found!\n");
Wire *wire_a = module->wire("\\A");
Wire *wire_y = module->wire("\\Y");
if (wire_a == nullptr)
log_error("No wire A found!\n");
if (wire_y == nullptr)
log_error("No wire Y found!\n");
ConstEval ce(module);
for (int v = 0; v < 4; v++) {
ce.push();
ce.set(wire_a, Const(v, GetSize(wire_a)));
SigSpec sig_y = wire_y, sig_undef;
if (ce.eval(sig_y, sig_undef))
log("Eval results for A=%d: Y=%s\n", v, log_signal(sig_y));
else
log("Eval failed for A=%d: Missing value for %s\n", v, log_signal(sig_undef));
ce.pop();
}
}
} EvalDemoPass;
PRIVATE_NAMESPACE_END
Example usage:
$ cat > evaldemo.v <<EOT
module main(input [1:0] A, input [7:0] B, C, D, output [7:0] Y);
assign Y = A == 0 ? B : A == 1 ? C : A == 2 ? D : 42;
endmodule
EOT
$ yosys-config --build evaldemo.so evaldemo.cc
$ yosys -m evaldemo.so -p evaldemo evaldemo.v
...
-- Running command `evaldemo' --
Eval failed for A=0: Missing value for \B
Eval failed for A=1: Missing value for \C
Eval failed for A=2: Missing value for \D
Eval results for A=3: Y=8'00101010
I have seen that GCC is not able to detect pure mathematical functions and it needs you to provide the attribute "const" to indicate that.
What compilers can detect pure mathematical functions and optimize them (without telling you so)?
To do so is inherently risky in languages that have pointers and lack global compilation & analysis. So, if a an operation is declared non-const, the compiler must assume it could have side-effects.
Example:
//getx.cpp
int GetX(int input)
{
int* pData = (int*) input;
*pData = 50;
return 0;
}
// gety.cpp
int GetY(int input)
{
return GetX(input + 4);
}
// main.cpp
int main()
{
int arg[] { 0, 4 };
return GetY((int)arg);
}
The compiler while compiling GetY can't tell that GetX treats its argument as a pointer and dereferences and modifies data in a non-functional, side-effect-prone manner. That information is only available during linking so you'd have to re-invent the concept of linking to include a lot of code generation and analysis to support such a feature.
It's not really (afaik) the compiler that does this, but when writing C# in Visual Studio when using the plugin ReSharper, you can get compile time hints that indicate that it is possible to declare something as const. On the other hand, that doesn't go under the category "without telling you so", so it might not be what you're looking for...
It seems that gcc now does: doing "gcc -O2 -S" on the following code, and reading the assembly, the call to foo() from within test() is identified as pure and moved outside of the loop:
#include <stdio.h>
double __attribute__((noinline)) foo(double x)
{
x = x + 1;
x = x * x;
if (x > 20)
x -= 1;
x -= x * x;
return x;
}
void test(int iters, double x)
{
int i;
for (i = 0; i < iters; ++i) {
printf("%g\n", foo(x));
}
}
This is Fedora 22, gcc 5.1.1, x86_64. I haven't tried, but with -flto, I would expect this to work across compilation units.
Also, it is worth noting that today gcc has the command line options -Wsuggest-attribute=pure and -Wsuggest-attribute=const.