We have been using the stomp client developed by Julian Lyndon-Smith/dot.r for a couple of years.
So far, we have been sending unencrypted messages.
We would now like to send messages over TLS. So naturally, I've changed the port from 61613 to 61614 which is the secured one but this give the following error.
None of the widgets used in the WAIT-FOR statement are in a state
(such as SENSITIVE) such that the specified event can occur. WAIT-FOR
terminated. (4123)
This error occur on line
WAIT-FOR "U2":U OF SocketHandle PAUSE 5. in method CONNECT() of class dotr.Stomp.StompConnection.
In the cloudAMQP server log we have the following log when attempting to connect using port 61614.
2022-03-14 14:07:36.915534+00:00 [noti] <0.23840.19> TLS server: In state hello at tls_record.erl:558 generated SERVER ALERT: Fatal - Unexpected Message
2022-03-14 14:07:36.915534+00:00 [noti] <0.23840.19> - {unsupported_record_type,67}
Here is what the log say when we connect using the unsecured port 61613. (I've hidden IP address by replacing it with x)
2022-03-14 14:08:05.388817+00:00 [info] <0.23882.19> accepting STOMP connection <0.23882.19> (xxx.xx.xxx.xxx:63837 -> xx.xx.xx.x:61613)
2022-03-14 14:08:10.445381+00:00 [info] <0.23882.19> closing STOMP connection <0.23882.19> (xxx.xx.xxx.xxx:63837 -> xx.xx.xx.x:61613)
The server log also contains the following warning that occur at night time.
certificate chain verification is not enabled for this TLS connection.
Please see https://rabbitmq.com/ssl.html for more information.
but according to documentation, this warning can be ignored because they don't recommend client certs at the moment.
Using Julian's stomp client or another, can we sent message over TLS? If yes, how?
Sorry if the question is a little broad, but my knowledge on TLS are limited and master google didn't had a solution for this one.
Here is the code of the class StompConnection we are using.
/*
Copyright (c) 2011-2012, Julian Lyndon-Smith (julian+maia#dotr.com)
http://www.dotr.com
All rights reserved.
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction except as noted below, including without limitation
the rights to use,copy, modify, merge, publish, distribute,
and/or sublicense, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The Software and/or source code cannot be copied in whole and
sold without meaningful modification for a profit.
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with
the distribution.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
*/
USING Progress.Lang.*.
USING dotr.Stomp.*.
USING dotr.Stomp.Interface.*.
ROUTINE-LEVEL ON ERROR UNDO, THROW.
CLASS dotr.Stomp.StompConnection FINAL:
DEF PRIVATE VAR StompConfig AS StompConfig NO-UNDO.
DEF PRIVATE VAR ServerResponseFrames AS CHAR INIT "CONNECTED,RECEIPT,ERROR":U NO-UNDO.
DEF PRIVATE VAR DisconnectID AS CHAR NO-UNDO.
DEF PUBLIC PROPERTY UserName AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY Password AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY ConnectionID AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY ClientID AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY StompVersion AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY StompServer AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY ErrorMessage AS CHAR NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY ConnectionEstablished AS LOGI NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY ConnectedBroker AS CHAR NO-UNDO
GET():
RETURN SUBST("&1:&2":U,StompConfig:StompActiveServer,StompConfig:StompActivePort).
END GET. PRIVATE SET.
DEF PUBLIC PROPERTY SocketHandle AS HANDLE NO-UNDO GET. PRIVATE SET.
DEF PUBLIC PROPERTY SocketHandler AS HANDLE NO-UNDO GET. PRIVATE SET.
DEF PUBLIC EVENT NewMessage SIGNATURE VOID (p_Message AS dotr.Stomp.StompMessage).
DEF PUBLIC EVENT NewRawMessage SIGNATURE VOID (p_Message AS LONGCHAR).
DEF PUBLIC EVENT DisconnectedEvent SIGNATURE VOID ().
DEF PUBLIC EVENT ConnectedEvent SIGNATURE VOID ().
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
CONSTRUCTOR StompConnection(p_StompConfig AS dotr.Stomp.StompConfig,
p_ClientID AS CHAR,
p_UserName AS CHAR,
p_Password AS CHAR):
IF p_UserName <> "":U THEN ASSIGN p_StompConfig:login = p_UserName.
IF p_Password <> "":U THEN ASSIGN p_StompConfig:passcode = p_Password.
ASSIGN THIS-OBJECT:StompConfig = p_StompConfig
THIS-OBJECT:UserName = p_StompConfig:login
THIS-OBJECT:Password = p_StompConfig:passcode
THIS-OBJECT:ClientId = (IF p_ClientID = "":U THEN GUID ELSE p_ClientID).
THIS-OBJECT:CONNECT().
END CONSTRUCTOR.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
CONSTRUCTOR StompConnection(p_ClientID AS CHAR,p_UserName AS CHAR,p_Password AS CHAR):
THIS-OBJECT(dotr.Stomp.StompConfig:Default,p_ClientID,p_UserName,p_Password).
END CONSTRUCTOR.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
DESTRUCTOR StompConnection():
THIS-OBJECT:DISCONNECT() NO-ERROR.
IF VALID-HANDLE(SocketHandle) THEN
DO: SocketHandle:DISCONNECT() NO-ERROR.
DELETE OBJECT SocketHandle NO-ERROR.
END.
IF VALID-HANDLE(SocketHandler) THEN
DO: RUN shutdown IN SocketHandler.
DELETE PROCEDURE SocketHandler NO-ERROR.
END.
END DESTRUCTOR.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PUBLIC VOID Disconnect():
DEF VAR ID AS CHAR NO-UNDO.
ConnectionEstablished = FALSE.
ASSIGN id = GUID.
SendFrame(SUBST("DISCONNECT~nreceipt:&1~n~n":U,id)).
ASSIGN DisconnectId = id. /** no more frames to be sent */
IF VALID-HANDLE(SocketHandle) THEN
DO: SocketHandle:DISCONNECT() NO-ERROR.
DELETE OBJECT SocketHandle NO-ERROR.
END.
IF VALID-HANDLE(SocketHandler) THEN
DO: RUN shutdown IN SocketHandler.
DELETE PROCEDURE SocketHandler NO-ERROR.
END.
CATCH e AS Progress.Lang.AppError : /** swallow all errors, as we're disconnecting anyway */
DELETE OBJECT e NO-ERROR.
END CATCH.
END METHOD.
/** Raise the SocketDisconnected event
*/
METHOD PUBLIC VOID DoSocketDisconnected():
ConnectionEstablished = FALSE.
DisconnectedEvent:PUBLISH() NO-ERROR.
END METHOD.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PUBLIC VOID ProcessData(p_Message AS LONGCHAR):
DEF VAR lv_Frame AS CHAR NO-UNDO.
DEF VAR lv_Dest AS CHAR NO-UNDO.
DEF VAR lv_Header AS CHAR NO-UNDO.
DEF VAR lv_Body AS LONGCHAR NO-UNDO.
DEF VAR lv_Index AS INT NO-UNDO.
ASSIGN lv_Frame = ENTRY(1,p_message,"~n":U).
ASSIGN lv_Frame = ENTRY(2,p_message,"~n":U) WHEN lv_Frame = "":U
lv_index = INDEX(p_Message,"~n~n":U)
lv_Body = IF lv_index = 0 THEN "":U ELSE TRIM(SUBSTRING(p_message,lv_index + 1))
lv_Header = TRIM(SUBSTRING(p_message,1,lv_index))
lv_Index = INDEX(lv_Header,"destination:":U).
ASSIGN lv_Dest = ENTRY(2,ENTRY(1,SUBSTRING(lv_Header,lv_index),"~n":U),":":U) WHEN lv_index > 0
lv_Dest = SUBSTR(lv_Dest,INDEX(lv_Dest,"/",2) + 1) WHEN lv_index > 0 .
IF lv_header = "":U AND INDEX(ServerResponseFrames,lv_Frame) > 0 THEN
lv_Header = STRING(TRIM(SUBSTRING(p_message,LENGTH(lv_Frame) + 1))).
CASE lv_frame:
WHEN "CONNECTED" THEN
DO: setConnectionDetails(lv_Header).
ConnectionEstablished = TRUE.
ConnectedEvent:PUBLISH() NO-ERROR.
APPLY "U2":U TO SocketHandle.
END.
WHEN "RECEIPT" THEN
DO: APPLY "U1":U TO SocketHandle.
END.
WHEN "ERROR" THEN
DO: ErrorMessage = lv_Body.
APPLY "U1":U TO SocketHandle.
END.
END CASE.
NewRawMessage:Publish(p_Message) NO-ERROR.
THIS-OBJECT:MessageParsed(lv_Frame,lv_Dest,lv_Header,lv_Body) NO-ERROR.
END METHOD.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PROTECTED VOID MessageParsed(p_Frame AS CHAR,p_Dest AS CHAR,p_Header AS CHAR, p_Body AS LONGCHAR):
NewMessage:Publish(NEW dotr.Stomp.StompMessage(p_frame,p_Dest,p_Header,p_Body)) NO-ERROR.
END METHOD.
/*-----------------------------------------------------------------------------
Purpose: Send message
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PUBLIC VOID SendFrame(INPUT p_Data AS LONGCHAR):
DEF VAR lv_MemPtr AS MEMPTR NO-UNDO.
DEF VAR lv_ToWrite AS INT NO-UNDO.
DEF VAR lv_offset AS INT NO-UNDO.
IF DisconnectId <> "":U THEN RETURN. /** Clients MUST NOT send any more frames after the DISCONNECT frame is sent. */
IF NOT VALID-HANDLE(SocketHandle) OR p_Data = "":U OR p_Data = ? THEN RETURN.
ASSIGN lv_ToWrite = PutStringIntoMemoryPointer(p_Data, lv_MemPtr)
lv_offset = 0.
DO WHILE lv_ToWrite > 0:
SocketHandle:WRITE(lv_MemPtr,lv_offset + 1, lv_ToWrite) NO-ERROR.
IF SocketHandle:BYTES-WRITTEN = 0 THEN
UNDO, THROW NEW AppError("IO Exception":U,0).
ASSIGN lv_offset = lv_offset + SocketHandle:BYTES-WRITTEN
lv_ToWrite = lv_ToWrite - SocketHandle:BYTES-WRITTEN.
END.
FINALLY:
SET-SIZE(lv_MemPtr) = 0.
END FINALLY.
END METHOD.
/*-----------------------------------------------------------------------------
Purpose: Put string into given memory pointer.
Returns: Return the number of bytes in the memory pointer.
Notes: Make sure character are converted from current session encoding to UTF-8 for french characters (accent).
-----------------------------------------------------------------------------*/
METHOD PRIVATE INT PutStringIntoMemoryPointer(INPUT p_Data AS LONGCHAR, INPUT p_MemPtr AS MEMPTR):
DEF VAR lv_DataUTF8 AS LONGCHAR NO-UNDO.
DEF VAR lv_NbBytesToWrite AS INT NO-UNDO.
DEF VAR lv_NbCharToWrite AS INT NO-UNDO.
DEF VAR lv_NbCharWritten AS INT NO-UNDO.
DEF VAR lv_NbBytesWritten AS INT NO-UNDO.
DEF VAR vl_CurrentCharacter AS CHAR NO-UNDO.
FIX-CODEPAGE (lv_DataUTF8) = 'utf-8'.
lv_DataUTF8 = p_Data.
ASSIGN lv_NbBytesToWrite = LENGTH(lv_DataUTF8, "RAW":U) + 1 /* add 1 bytes for 0-terminated string */
lv_NbCharToWrite = LENGTH(lv_DataUTF8, "CHARACTER":U)
lv_NbCharWritten = 0
lv_NbBytesWritten = 0.
SET-SIZE(p_MemPtr) = lv_NbBytesToWrite.
//Line `COPY-LOB FROM p_Data TO p_MemPtr CONVERT TARGET CODEPAGE "UTF-8".` does work.
//However, it is so slow that it cause error(s) when using it.
//Thus, we convert encoding character by character with the following loop instead.
DO lv_NbCharWritten = 1 TO lv_NbCharToWrite:
vl_CurrentCharacter = CHR(ASC(STRING(SUBSTRING(lv_DataUTF8, lv_NbCharWritten, 1, "CHARACTER")), "UTF-8":U, SESSION:CPINTERNAL), "UTF-8":U, "UTF-8":U).
PUT-STRING(p_MemPtr, lv_NbBytesWritten + 1) = vl_CurrentCharacter.
lv_NbBytesWritten = lv_NbBytesWritten + LENGTH(vl_CurrentCharacter, "RAW").
END.
RETURN lv_NbBytesToWrite.
END METHOD.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PUBLIC VOID CONNECT():
IF NOT VALID-HANDLE(SocketHandle) THEN
DO: CREATE SOCKET SocketHandle.
RUN dotr/Stomp/stompSocketProc.p PERSISTENT SET SocketHandler (THIS-OBJECT).
SocketHandle:SET-READ-RESPONSE-PROCEDURE(IF StompConfig:LargeMessageSupport THEN "SocketBigReadHandler":U
ELSE "SocketReadHandler":U,SocketHandler).
END.
SocketHandle:CONNECT(SUBSTITUTE("-H &1 -S &2":U, StompConfig:StompServer, StompConfig:StompPort)) NO-ERROR.
IF ERROR-STATUS:NUM-MESSAGES > 0 OR NOT SocketHandle:CONNECTED() THEN
DO:
/* try connecting using the slave connection details, if set */
IF StompConfig:FailoverSupported THEN
DO: SocketHandle:CONNECT(SUBSTITUTE("-H &1 -S &2":U, StompConfig:StompSlaveServer, StompConfig:StompSlavePort)) NO-ERROR.
StompConfig:SetActive(StompConfig:StompSlaveServer,StompConfig:StompSlavePort).
END.
IF ERROR-STATUS:NUM-MESSAGES > 0 OR NOT SocketHandle:CONNECTED() THEN
UNDO, THROW NEW AppError(SUBSTITUTE("Cannot connect to Stomp Server with Server:&1 Port:&2":U,
StompConfig:Stompserver + "/" + StompConfig:StompSlaveServer,
StompConfig:StompPort + "/" + StompConfig:StompSlavePort),0).
END.
ELSE StompConfig:SetActive(StompConfig:Stompserver,StompConfig:StompPort).
SocketHandle:SET-SOCKET-OPTION ("SO-RCVBUF":U,STRING(1048576)).
SocketHandle:SET-SOCKET-OPTION ("TCP-NODELAY":U,"TRUE":U).
SocketHandle:SET-SOCKET-OPTION ("SO-KEEPALIVE":U,"TRUE":U).
SendFrame(SUBSTITUTE("CONNECT~naccept-version:1.0,1.1,2.0~nhost:&3~nclient-id:&4~nlogin:&1~npasscode:&2~n~n":U, THIS-OBJECT:UserName, THIS-OBJECT:Password,StompConfig:StompHost,THIS-OBJECT:ClientID)).
WAIT-FOR "U2":U OF SocketHandle PAUSE 5.
IF ConnectionEstablished = FALSE THEN
UNDO, THROW NEW AppError(SUBSTITUTE("Cannot connect to Stomp Server with Server:&1 Port:&2":U,
StompConfig:Stompserver + "/" + StompConfig:StompSlaveServer,
StompConfig:StompPort + "/" + StompConfig:StompSlavePort),0).
END METHOD.
/*-----------------------------------------------------------------------------
Purpose:
Parameters:
Notes:
-----------------------------------------------------------------------------*/
METHOD PRIVATE VOID setConnectionDetails (p_Header AS CHAR):
DEF VAR i AS INT NO-UNDO.
DEF VAR lv_header AS CHAR NO-UNDO.
DO i = 1 TO NUM-ENTRIES(p_Header,"~n":U):
lv_header = ENTRY(i,p_Header,"~n":U).
CASE ENTRY(1,lv_header,":":U):
WHEN "session":U THEN THIS-OBJECT:ConnectionID = SUBSTRING(lv_header,INDEX(lv_header,":":U) + 1).
WHEN "server":U THEN THIS-OBJECT:stompServer = SUBSTRING(lv_header,INDEX(lv_header,":":U) + 1).
WHEN "version":U THEN THIS-OBJECT:stompVersion = SUBSTRING(lv_header,INDEX(lv_header,":":U) + 1).
END CASE.
END.
END METHOD.
END CLASS.
Related
Note: This post is similar, but not quite the same as a more open-ended questions asked on Reddit: https://www.reddit.com/r/rakulang/comments/vvpikh/looking_for_guidance_on_getting_nativecall/
I'm trying to use the md4c c library to process a markdown file with its md_parse function. I'm having no success, and the program just quietly dies. I don't think I'm calling it with the right arguments.
Documentation for the function is here: https://github.com/mity/md4c/wiki/Embedding-Parser%3A-Calling-MD4C
I'd like to at least figure out the minimum amount of code needed to do this without error. This is my latest attempt, though I've tried many:
use v6.d;
use NativeCall;
sub md_parse(str, int32, Pointer is rw ) is native('md4c') returns int32 { * }
md_parse('hello', 5, Pointer.new());
say 'hi'; # this never gets printed
md4c is a SAX-like streaming parser that calls your functions when it encounters markdown elements. If you call it with an uninitialised Pointer, or with an uninitialised CStruct then the code will SEGV when the md4c library tries to call a null function pointer.
The README says:
The main provided function is md_parse(). It takes a text in the
Markdown syntax and a pointer to a structure which provides pointers
to several callback functions.
As md_parse() processes the input, it calls the callbacks (when
entering or leaving any Markdown block or span; and when outputting
any textual content of the document), allowing application to convert
it into another format or render it onto the screen.
The function signature of md_parse is:
int md_parse(const MD_CHAR* text, MD_SIZE size, const MD_PARSER* parser, void* userdata);
In order for md_parse() to work, you will need to:
define a native CStruct that matches the MD_PARSER type definition
create an instance of this CStruct
initialise all the function pointers with Raku functions that have the right function signature
call md_parse() with the initialised CStruct instance as the third parameter
The 4th parameter to md_parse() is void* userdata which is a pointer that you provide which gets passed back to you as the last parameter of each of the callback functions. My guess is that it's optional and if you pass a null value then you'll get called back with a null userdata parameter in each callback.
Followup
This turned into an interesting rabbit hole to fall down.
The code that makes it possible to pass a Raku sub as a callback parameter to a native function is quite complex and relies on MoarVM ops to build and cache the FFI callback trampoline. This is a piece of code that marshals the C calling convention parameters into a call that MoarVM can dispatch to a Raku sub.
It will be a sizeable task to implement equivalent functionality to provide some kind of nativecast that will generate the required callback trampoline and return a Pointer that can be assigned into a CStruct.
But we can cheat
We can use a simple C function to return the pointer to a generated callback trampoline as if it was for a normal callback sub. We can then store this pointer in our CStruct and our problem is solved. The generated trampoline is specific to the function signature of the Raku sub we want to call, so we need to generate a different NativeCall binding for each function signature we need.
The C function:
void* get_pointer(void* p)
{
return p;
}
Binding a NativeCall sub for the function signature we need:
sub get_enter_leave_fn(&func (uint32, Pointer, Pointer))
is native('./getpointer') is symbol('get_pointer') returns Pointer { * }
Initialising a CStruct attribute:
$!enter_block := get_enter_leave_fn(&enter_block);
Putting it all together:
use NativeCall;
enum BlockType < DOC QUOTE UL OL LI HR H CODE HTML P TABLE THEAD TBODY TR TH TD >;
enum SpanType < EM STRONG A IMG SPAN_CODE DEL SPAN_LATEXMATH LATEXMATH_DISPLAY WIKILINK SPAN_U >;
enum TextType < NORMAL NULLCHAR BR SOFTBR ENTITY TEXT_CODE TEXT_HTML TEXT_LATEXMATH >;
sub enter_block(uint32 $type, Pointer $detail, Pointer $userdata --> int32) {
say "enter block { BlockType($type) }";
}
sub leave_block(uint32 $type, Pointer $detail, Pointer $userdata --> int32) {
say "leave block { BlockType($type) }";
}
sub enter_span(uint32 $type, Pointer $detail, Pointer $userdata --> int32) {
say "enter span { SpanType($type) }";
}
sub leave_span(uint32 $type, Pointer $detail, Pointer $userdata --> int32) {
say "leave span { SpanType($type) }";
}
sub text(uint32 $type, str $text, uint32 $size, Pointer $userdata --> int32) {
say "text '{$text.substr(0..^$size)}'";
}
sub debug_log(str $msg, Pointer $userdata --> int32) {
note $msg;
}
#
# Cast functions that are specific to the required function signature.
#
# Makes use of a utility C function that returns its `void*` parameter, compiled
# into a shared library called libgetpointer.dylib (on MacOS)
#
# gcc -shared -o libgetpointer.dylib get_pointer.c
#
# void* get_pointer(void* p)
# {
# return p;
# }
#
# Each cast function uses NativeCall to build an FFI callback trampoline that gets
# cached in an MVMThreadContext. The generated callback code is specific to the
# function signature of the Raku function that will be called.
#
sub get_enter_leave_fn(&func (uint32, Pointer, Pointer))
is native('./getpointer') is symbol('get_pointer') returns Pointer { * }
sub get_text_fn(&func (uint32, str, uint32, Pointer))
is native('./getpointer') is symbol('get_pointer') returns Pointer { * }
sub get_debug_fn(&func (str, Pointer))
is native('./getpointer') is symbol('get_pointer') returns Pointer { * }
class MD_PARSER is repr('CStruct') {
has uint32 $!abi_version; # unsigned int abi_version
has uint32 $!flags; # unsigned int flags
has Pointer $!enter_block; # F:int ( )* enter_block
has Pointer $!leave_block; # F:int ( )* leave_block
has Pointer $!enter_span; # F:int ( )* enter_span
has Pointer $!leave_span; # F:int ( )* leave_span
has Pointer $!text; # F:int ( )* text
has Pointer $!debug_log; # F:void ( )* debug_log
has Pointer $!syntax; # F:void ( )* syntax
submethod TWEAK() {
$!abi_version = 0;
$!flags = 0;
$!enter_block := get_enter_leave_fn(&enter_block);
$!leave_block := get_enter_leave_fn(&leave_block);
$!enter_span := get_enter_leave_fn(&enter_span);
$!leave_span := get_enter_leave_fn(&leave_span);
$!text := get_text_fn(&text);
$!debug_log := get_debug_fn(&debug_log);
}
}
sub md_parse(str, uint32, MD_PARSER, Pointer is rw) is native('md4c') returns int { * }
my $parser = MD_PARSER.new;
my $md = '
# Heading
## Sub Heading
hello *world*
';
md_parse($md, $md.chars, $parser, Pointer.new);
The output:
./md4c.raku
enter block DOC
enter block H
text 'Heading'
leave block H
enter block H
text 'Sub Heading'
leave block H
enter block P
text 'hello '
enter span EM
text 'world'
leave span EM
leave block P
leave block DOC
In summary, it's possible. I'm not sure if I'm proud of this or horrified by it. I think a long-term solution will require refactoring the callback trampoline generator into a separate nqp op that can be exposed to Raku as a nativewrap style operation.
I have a FreeRADIUS C language module that implements MOD_AUTHENTICATE and MOD_AUTHORIZE methods for custom auth purpose. I need the ability to programmatically add VSAs to the Access-Accept reply.
I have toyed a bit with radius_pair_create() and fr_pair_add() methods (see snippet below) but that didn’t yield any change to the reply content, possibly because I specified ad-hoc values that don’t exist in a vendor-specific dictionary. Or because I didn’t use them correctly.
My FreeRADIUS version is 3_0_19
Any information, pointers and, especially, syntax samples will be highly appreciated.
void test_vsa(REQUEST *request)
{
VALUE_PAIR *vp = NULL;
vp = radius_pair_create(request->reply, NULL, 18, 0);
if (vp)
{
log("Created VALUE_PAIR");
vp->vp_integer = 96;
fr_pair_add(&request->reply->vps, vp);
}
else
{
log("Failed to create VALUE_PAIR");
}
}
So first off you're writing an integer value to a string attribute, which is wrong. The only reason why the server isn't SEGVing is because the length of the VP has been left at zero, so the RADIUS encoder doesn't bother dereferencing the char * inside the pair that's meant to contain the pair's value.
fr_pair_make is the easier function to use here, as it takes both the attribute name and value as strings, so you don't need to worry about the C types.
The code snippet below should do what you want.
void test_avp(REQUEST *request)
{
VALUE_PAIR *vp = NULL;
vp = fr_pair_make(request->reply, &request->reply->vps, "Reply-Message", "Hello from FreeRADIUS", T_OP_SET);
if (vp)
{
log("Created VALUE_PAIR");
}
else
{
log("Failed to create VALUE_PAIR");
}
}
For a bit more of an explanation, lets look at the doxygen header:
/** Create a VALUE_PAIR from ASCII strings
*
* Converts an attribute string identifier (with an optional tag qualifier)
* and value string into a VALUE_PAIR.
*
* The string value is parsed according to the type of VALUE_PAIR being created.
*
* #param[in] ctx for talloc
* #param[in] vps list where the attribute will be added (optional)
* #param[in] attribute name.
* #param[in] value attribute value (may be NULL if value will be set later).
* #param[in] op to assign to new VALUE_PAIR.
* #return a new VALUE_PAIR.
*/
VALUE_PAIR *fr_pair_make(TALLOC_CTX *ctx, VALUE_PAIR **vps,
char const *attribute, char const *value, FR_TOKEN op)
ctx - This is the packet or request that the vps will belong to. If you're adding attributes to the request it should be request->packet, reply would be request->reply, control would be request.
vps - If specified, this will be which list to insert the new VP into. If this is NULL fr_pair_make will just return the pair and let you insert it into a list.
attribute - The name of the attribute as a string.
value - The value of the attribute as a string. For non-string types, fr_pair_make will attempt to perform a conversion. So, for example, passing "12345" for an integer type, will result in the integer value 12345 being written to an int field in the attribute.
op - You'll usually want to us T_OP_SET which means overwrite existing instances of the same attribute. See the T_OP_* values of FR_TOKEN and the code that uses them, if you want to understand the different operators and what they do.
I'm trying to add a "clone" system call to the xv6 os. The call creates a new kernel thread which shares the calling process’s address space. The following is my code in proc.c
int clone(void(*fcn)(void*), void* arg, void* stack)
{
int i, pid;
struct proc *np;
int *myarg;
int *myret;
if((np = allocproc()) == 0)
return -1;
np->pgdir = proc->pgdir; //Here's where it tell's me proc is undefined
np->sz = proc->sz;
np->parent = proc;
*np->tf = *proc->tf;
np->stack = stack;
np->tf->eax = 0;
np->tf->eip = (int)fcn;
myret = stack + 4096 - 2 * sizeof(int *);
*myret = 0xFFFFFFFF;
myarg = stack + 4096 - sizeof(int *);
*myarg = (int)arg;
np->tf->esp = (int)stack + PGSIZE - 2 * sizeof(int *);
np->tf->ebp = np->tf->esp;
np->isthread = 1;
for(i = 0; i < NOFILE; i++)
if(proc->ofile[i])
np->ofile[i] = filedup(proc->ofile[i]);
np->cwd = idup(proc->cwd);
safestrcpy(np->name, proc->name, sizeof(proc->name));
pid = np->pid;
acquire(&ptable.lock);
np->state = RUNNABLE;
release(&ptable.lock);
return pid;
}
Most of the implementations I found look just like this, however, whenever I try to make it tells me that 'proc' is undefined. Most implementations of clone that I've seen look nearly identical, with all of them utilizing proc. I'd be happy to share my sysproc.c code as well if that would help in any way.
Thank you!
This has nothing to do with your system call's implementation because proc global variable is being set by the scheduler right before resuming a selected "runnable" process.
The reason for null would probably be because calling this function from a wrong context.
A system call implementation is expected to be executed from a wrapping function named sys_mysysfunc that syscall function called due to a system call inerrupt initiated by a user application code.
Please share with us your entire implementation flow for additional assistance.
I am running libgit2 v0.23.0. I am calling method git_index_add_all which takes following parameters:
git_index * index
const git_strarray * pathspec
unsigned int flags
git_index_matched_path_cb callback
void * payload
I am not able to get how do I need to create last void *payload parameter
My code is :
git_index *idx = NULL;
git_index_matched_path_cb matched_cb = NULL;
int error = 0;
error = git_index_open(&idx, "repofolder/.git/index");
char *paths[] = {"repofolder/*"};
git_strarray arr = {paths, 1};
error = git_index_add_all(idx, &arr, GIT_INDEX_ADD_DEFAULT,matched_cb, ?);
could anybody suggest me, what should be way to create or get payload type object ?
The payload argument is the standard way of creating a closure in C. Your callback will receive whatever pointer you put in as its payload argument. It should be a pointer to whatever variable/structure you need for the callback to do its work.
If you don't need any data, then pass in NULL.
I'd like to create 9-digit numeric ids that are unique across machines. I'm currently using a database sequence for this, but am wondering if it could be done without one. The sequences will be used for X12 EDI transactions, so they don't have to be unique forever. Maybe even only unique for 24 hours.
My only idea:
Each server has a 2 digit server identifier.
Each server maintains a file that essentially keeps track of a local sequence.
id = + <7 digit sequence which wraps>
My biggest problem with this is what to do if the hard-drive fails. I wouldn't know where it left off.
All of my other ideas essentially end up re-creating a centralized database sequence.
Any thoughts?
The Following
{XX}{dd}{HHmm}{N}
Where {XX} is the machine number {dd} is the day of the month {HHmm} current time (24hr) and {N} a sequential number.
A hd crash will take more than a minute so starting at 0 again is not a problem.
You can also replace {dd} with {ss} for seconds, depending on requirements. Uniqueness period vs. requests per minute.
If HD fails you can just set new and unused 2 digit server identifier and be sure that the number is unique (for 24 hours at least)
How about generating GUIDs (ensures uniqueness) and then using some sort of hash function to turn the GUID into a 9-digit number?
Just off the top of my head...
Use a variation on:
md5(uniqid(rand(), true));
Just a thought.
In my recent project I also come across this requirement, to generate N digit long sequence number without any database.
This is actually a good Interview question, because there are consideration on performance and software crash recovery. Further Reading if interested.
The following code has these features:
Prefix each sequence with a prefix.
Sequence cache like Oracle Sequence.
Most importantly, there is recovery logic to resume sequence from software crash.
Complete implementation attached:
import java.util.concurrent.atomic.AtomicLong;
import org.apache.commons.lang.StringUtils;
/**
* This is a customized Sequence Generator which simulates Oracle DB Sequence Generator. However the master sequence
* is stored locally in the file as there is no access to Oracle database. The output format is "prefix" + number.
* <p>
* <u><b>Sample output:</u></b><br>
* 1. FixLengthIDSequence(null,null,15,0,99,0) will generate 15, 16, ... 99, 00<br>
* 2. FixLengthIDSequence(null,"K",1,1,99,0) will generate K01, K02, ... K99, K01<br>
* 3. FixLengthIDSequence(null,"SG",100,2,9999,100) will generate SG0100, SG0101, ... SG8057, (in case server crashes, the new init value will start from last cache value+1) SG8101, ... SG9999, SG0002<br>
*/
public final class FixLengthIDSequence {
private static String FNAME;
private static String PREFIX;
private static AtomicLong SEQ_ID;
private static long MINVALUE;
private static long MAXVALUE;
private static long CACHEVALUE;
// some internal working values.
private int iMaxLength; // max numeric length excluding prefix, for left padding zeros.
private long lNextSnapshot; // to keep track of when to update sequence value to file.
private static boolean bInit = false; // to enable ShutdownHook routine after program has properly initialized
static {
// Inspiration from http://stackoverflow.com/questions/22416826/sequence-generator-in-java-for-unique-id#35697336.
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
if (bInit) { // Without this, saveToLocal may hit NullPointerException.
saveToLocal(SEQ_ID.longValue());
}
}));
}
/**
* This POJO style constructor should be initialized via Spring Singleton. Otherwise, rewrite this constructor into Singleton design pattern.
*
* #param sFilename This is the absolute file path to store the sequence number. To reset the sequence, this file needs to be removed manually.
* #param prefix The hard-coded identifier.
* #param initvalue
* #param minvalue
* #param maxvalue
* #param cache
* #throws Exception
*/
public FixLengthIDSequence(String sFilename, String prefix, long initvalue, long minvalue, long maxvalue, int cache) throws Exception {
bInit = false;
FNAME = (sFilename==null)?"C:\\Temp\\sequence.txt":sFilename;
PREFIX = (prefix==null)?"":prefix;
SEQ_ID = new AtomicLong(initvalue);
MINVALUE = minvalue;
MAXVALUE = maxvalue; iMaxLength = Long.toString(MAXVALUE).length();
CACHEVALUE = (cache <= 0)?1:cache; lNextSnapshot = roundUpNumberByMultipleValue(initvalue, cache); // Internal cache is always 1, equals no cache.
// If sequence file exists and valid, restore the saved sequence.
java.io.File f = new java.io.File(FNAME);
if (f.exists()) {
String[] saSavedSequence = loadToString().split(",");
if (saSavedSequence.length != 6) {
throw new Exception("Local Sequence file is not valid");
}
PREFIX = saSavedSequence[0];
//SEQ_ID = new AtomicLong(Long.parseLong(saSavedSequence[1])); // savedInitValue
MINVALUE = Long.parseLong(saSavedSequence[2]);
MAXVALUE = Long.parseLong(saSavedSequence[3]); iMaxLength = Long.toString(MAXVALUE).length();
CACHEVALUE = Long.parseLong(saSavedSequence[4]);
lNextSnapshot = Long.parseLong(saSavedSequence[5]);
// For sequence number recovery
// The rule to determine to continue using SEQ_ID or lNextSnapshot as subsequent sequence number:
// If savedInitValue = savedSnapshot, it was saved by ShutdownHook -> use SEQ_ID.
// Else if saveInitValue < savedSnapshot, it was saved by periodic Snapshot -> use lNextSnapshot+1.
if (saSavedSequence[1].equals(saSavedSequence[5])) {
long previousSEQ = Long.parseLong(saSavedSequence[1]);
SEQ_ID = new AtomicLong(previousSEQ);
lNextSnapshot = roundUpNumberByMultipleValue(previousSEQ,CACHEVALUE);
} else {
SEQ_ID = new AtomicLong(lNextSnapshot+1); // SEQ_ID starts fresh from lNextSnapshot+!.
lNextSnapshot = roundUpNumberByMultipleValue(SEQ_ID.longValue(),CACHEVALUE);
}
}
// Catch invalid values.
if (minvalue < 0) {
throw new Exception("MINVALUE cannot be less than 0");
}
if (maxvalue < 0) {
throw new Exception("MAXVALUE cannot be less than 0");
}
if (minvalue >= maxvalue) {
throw new Exception("MINVALUE cannot be greater than MAXVALUE");
}
if (cache >= maxvalue) {
throw new Exception("CACHE value cannot be greater than MAXVALUE");
}
// Save the next Snapshot.
saveToLocal(lNextSnapshot);
bInit = true;
}
/**
* Equivalent to Oracle Sequence nextval.
* #return String because Next Value is usually left padded with zeros, e.g. "00001".
*/
public String nextVal() {
if (SEQ_ID.longValue() > MAXVALUE) {
SEQ_ID.set(MINVALUE);
lNextSnapshot = roundUpNumberByMultipleValue(MINVALUE,CACHEVALUE);
}
if (SEQ_ID.longValue() > lNextSnapshot) {
lNextSnapshot = roundUpNumberByMultipleValue(lNextSnapshot,CACHEVALUE);
saveToLocal(lNextSnapshot);
}
return PREFIX.concat(StringUtils.leftPad(Long.toString(SEQ_ID.getAndIncrement()),iMaxLength,"0"));
}
/**
* Store sequence value into the local file. This routine is called either by Snapshot or ShutdownHook routines.<br>
* If called by Snapshot, currentCount == Snapshot.<br>
* If called by ShutdownHook, currentCount == current SEQ_ID.
* #param currentCount - This value is inserted by either Snapshot or ShutdownHook routines.
*/
private static void saveToLocal (long currentCount) {
try (java.io.Writer w = new java.io.BufferedWriter(new java.io.OutputStreamWriter(new java.io.FileOutputStream(FNAME), "utf-8"))) {
w.write(PREFIX + "," + SEQ_ID.longValue() + "," + MINVALUE + "," + MAXVALUE + "," + CACHEVALUE + "," + currentCount);
w.flush();
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* Load the sequence file content into String.
* #return
*/
private String loadToString() {
try {
return new String(java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(FNAME)));
} catch (Exception e) {
e.printStackTrace();
}
return "";
}
/**
* Utility method to round up num to next multiple value. This method is used to calculate the next cache value.
* <p>
* (Reference: http://stackoverflow.com/questions/18407634/rounding-up-to-the-nearest-hundred)
* <p>
* <u><b>Sample output:</b></u>
* <pre>
* System.out.println(roundUpNumberByMultipleValue(9,10)); = 10
* System.out.println(roundUpNumberByMultipleValue(10,10)); = 20
* System.out.println(roundUpNumberByMultipleValue(19,10)); = 20
* System.out.println(roundUpNumberByMultipleValue(100,10)); = 110
* System.out.println(roundUpNumberByMultipleValue(109,10)); = 110
* System.out.println(roundUpNumberByMultipleValue(110,10)); = 120
* System.out.println(roundUpNumberByMultipleValue(119,10)); = 120
* </pre>
*
* #param num Value must be greater and equals to positive integer 1.
* #param multiple Value must be greater and equals to positive integer 1.
* #return
*/
private long roundUpNumberByMultipleValue(long num, long multiple) {
if (num<=0) num=1;
if (multiple<=0) multiple=1;
if (num % multiple != 0) {
long division = (long) ((num / multiple) + 1);
return division * multiple;
} else {
return num + multiple;
}
}
/**
* Main method for testing purpose.
* #param args
*/
public static void main(String[] args) throws Exception {
//FixLengthIDSequence(Filename, prefix, initvalue, minvalue, maxvalue, cache)
FixLengthIDSequence seq = new FixLengthIDSequence(null,"H",50,1,999,10);
for (int i=0; i<12; i++) {
System.out.println(seq.nextVal());
Thread.sleep(1000);
//if (i==8) { System.exit(0); }
}
}
}
To test the code, let the sequence run normally. You can press Ctrl+C to simulate the server crash. The next sequence number will continue from NextSnapshot+1.
Cold you use the first 9 digits of some other source of unique data like:
a random number
System Time
Uptime
Having thaught about it for two seconds, none of those are unique on there own but you could use them as seed values for hash functions as was suggested in another answer.