Cro::WebSocket::Client doesn't work - raku

Created a websocket server with "cro sub".
Wrote this client:
use v6;
use Cro::WebSocket::Client;
constant WS-PORT = '20000';
constant WS-ADDRESS = 'localhost';
constant WS-PATH = 'chat';
constant WS-URL = 'ws://' ~ WS-ADDRESS ~ ':' ~ WS-PORT ~ '/' ~ WS-PATH;
constant TIMEOUT-TO-CONNECT = 5; # seconds
my $timeout;
my $connection-attempt;
await Promise.anyof(
$connection-attempt = Cro::WebSocket::Client.connect(WS-URL),
$timeout = Promise.in(TIMEOUT-TO-CONNECT));
if $timeout.status == Kept
{
say "* could not connect to server in ', TIMEOUT-TO-CONNECT, ' seconds";
exit 1;
}
if $connection-attempt.status != Kept
{
say "* error ", $connection-attempt.cause,
" when trying to connect to server";
exit 1;
}
my $connection = $connection-attempt.result;
my $peer = WS-ADDRESS ~ ':' ~ WS-PORT;
say '* connected with ', $peer;
my $counter = 0;
my $message-supplier = Supplier::Preserving.new;
my $has-message-to-send = $message-supplier.Supply;
$message-supplier.emit(1);
react
{
whenever $has-message-to-send
{
$counter++;
$connection.send($counter);
say "* ok, sent message ", $counter, " to server";
}
whenever $connection.messages -> $reply
{
say '* received reply=[' ~ $reply ~ '] from server';
$message-supplier.emit(1);
}
} # react
I see with tcpdump the response code 101 (switching protocols) from the server, but I don't see the message sent from the client to the server.
So, what am I doing wrong ?
Another question, shoudn't "$connection.send" return a Promise or something ? What if there's an error when sending ?
And another question: it seems the server only understands IPV6 addresses...how to make it understand IPV4 addresses ?
That's it, for now.
UPDATE
As per Takao's advice, changing
$connection.send($counter)
to
$connection.send($counter.Str)
solves the problem (though I tried it on another program, not this one).

Let's resolve this piece by piece.
Firstly, your code looks correct to me, except for a couple of tiny bits.
When I reproduced your code, it indeed did not work, so I tried it with cro trace . instead of cro run .. You can find info about that mode in official docs.
The alternative way is to just set CRO_TRACE=1 environment variable.
So during debug I saw this error:
[TRACE(anon 1)] Cro::HTTP::ResponseParser QUIT No applicable body serializer could be found for this message
As it says, the body you sent could not be serialized. So I looked into what are you sending: $counter. $counter in your code is Int, so we need to make it Str before, doing simple $counter.Str makes your example work.
Also, note that you are sending a message on every reply, and echo server (default one you created using cro stub) also sends a reply for every incoming message, so your example sends messages endlessly. To prevent that you may consider adding a condition under which you will no longer send things, but well, it is a test example anyway, so up to you.
As for your other questions:
Another question, shoudn't "$connection.send" return a Promise or something?
It should not, I'll write out some cro's architecture details to explain it next. As you may know from reading docs, cro pipeline is basically just a bunch of Cro::Transform-wrapped supplies. Inside of Cro::Websocket::Client::Connection, send method just sends a thing directly into Cro::Source of the whole pipeline, you cannot go wrong with a simple $supplier.emit($message)(the real implementation of this method looks very close to this line). The thing you bumped into occurred further in the pipeline. I am sure it is not a nice user experience to hide exceptions of such cases, so I'll consider making a patch to propagate the exception, so it'd be easier to catch(although you always can use debug mode).
it seems the server only understands IPV6 addresses...how to make it understand IPV4 addresses ?
I am not sure about that, please open a new question.

Related

Raku Cro service subscribing to data "in the background" general guidance

I am attempting to put together a Cro service that has a react/whenever block consuming data "in the background" So unlike many examples of websocket usage with Cro, this has nothing to do with routes that may be accessed via the browser.
My use case is to consume message received via an MQTT topic and do some processing with them. At a later stage in development I might create a supply out of this data, but for now, when data is received it will be stored in a variable and dependant on certain conditions, be sent to another service via a http post.
My thought was to include a provider() in the Cro::HTTP::Server setup like so:
use Cro::HTTP::Log::File;
use Cro::HTTP::Server;
use Routes;
use DataProvider; # Here
my Cro::Service $http = Cro::HTTP::Server.new(
http => <1.1>,
host => ...,
port => ...,
application => [routes(), provider()], # Made this into an array of subs?
after => [
Cro::HTTP::Log::File.new(logs => $*OUT, errors => $*ERR)
]
);
And in the DataProvider.pm6:
use MQTT::Client;
sub provider() is export {
my $mqtt = MQTT::Client.new: server => 'localhost';
react {
whenever $mqtt.subscribe('some/mqtt/topic') {
say "+ topic: { .<topic> } => { .<message>.decode("utf8-c8") }";
}
}
}
This throws a bunch of errors:
A react block:
in sub provider at DataProvider.pm6 (DataProvider) line 5
in block <unit> at service.p6 line 26
Died because of the exception:
Invocant of method 'write' must be an object instance of type
'IO::Socket::Async', not a type object of type 'IO::Socket::Async'. Did
you forget a '.new'?
in method subscribe at /home/cam/raku/share/perl6/site/sources/42C762836A951A1C11586214B78AD34262EC465F (MQTT::Client) line 133
in sub provider at DataProvider.pm6 (DataProvider) line 6
in block <unit> at service.p6 line 26
To be perfectly honest, I am totally guessing that this is how I would approach the need to subscribe to data in the background of a Cro service, but I was not able to find any information on what might be considered the recommended approach.
Initially I had my react/whenever block in the main service.pm6 file, but that did not seem right. And needed to be wrapped in a start{} block because as I have just learned, react is blocking :) and cro was not able to actually start.
But following the pattern of how Routes are implemented seemed logical, but I am missing something. The error speaks about setting up a new method, but I'm not convinced that is the root cause. Routes.pm6 does not have a constructor.
Can anyone point me in the right direction please?
Thanks to all who have provided information, this has been a very valuable learning exercise.
The approach of passing additional sub routines, along side router() in the application parameter to Cro::HTTP::Server.new gave further trouble. (an array is not allowed, and broke routing)
Instead, I have moved the background work into a class of it's own, and given it a start and stop method more akin to Cro::HTTP::Server.
My new approach:
service.pm6
use Cro::HTTP::Log::File;
use Cro::HTTP::Server;
use Routes;
use KlineDataSubscriber; # Moved mqtt functionality here
use Database;
my $dsn = "host=localhost port=5432 dbname=act user=.. password=..";
my $dbh = Database.new :$dsn;
my $mqtt-host = 'localhost';
my $subscriber = KlineDataSubscriber.new :$mqtt-host;
$subscriber.start; # Inspired by $http.start below
my Cro::Service $http = Cro::HTTP::Server.new(
http => <1.1>,
host => ...,
port => ...,
application => routes($dbh), # Basically back the way it was originally
after => [
Cro::HTTP::Log::File.new(logs => $*OUT, errors => $*ERR)
]
);
$http.start;
say "Listening at...";
react {
whenever signal(SIGINT) {
say "Shutting down...";
$subscriber.stop;
$http.stop;
done;
}
}
And in KlineDataSubscriber.pm6
use MQTT::Client;
class KlineDataSubscriber {
has Str $.mqtt-host is required;
has MQTT::Client $.mqtt = Nil;
submethod TWEAK() {
$!mqtt = MQTT::Client.new: server => $!mqtt-host;
await $!mqtt.connect;
}
method start(Str $topic = 'act/feed/exchange/binance/kline-closed/+/json') {
start {
react {
whenever $!mqtt.subscribe($topic) {
say "+ topic: { .<topic> } => { .<message>.decode("utf8-c8") }";
}
}
}
}
method stop() {
# TODO Figure how to unsubscribe and cleanup nicely
}
}
This feels much more "Cro idiomatic" to me, but I would be happy to be corrected.
More importantly, it works as expected and I feel is somewhat future proof. I should be able to create a supply to make real-time data available to the router, and push data to any connected web clients.
I also intend to have a http GET endpoint /status with various checks to ensure everything healthy
The root cause
The error speaks about setting up a new method, but I'm not convinced that is the root cause.
It's not about setting up a new method. It's about a value that should be defined instead being undefined. That typically means a failure to attempt to initialize it, which typically means a failure to call .new.
Can anyone point me in the right direction please?
Hopefully this question helps.
Finding information on a recommended approach
I am totally guessing that this is how I would approach the need to subscribe to data in the background of a Cro service, but I was not able to find any information on what might be considered the recommended approach.
It might be helpful for you to list which of the get-up-to-speed steps you've followed from Getting started with Cro, including the basics but also the "Learn about" steps at the end.
The error message
A react block:
in sub provider ...
Died because of the exception:
...
in method subscribe ...
The error message begins with the built in react construct reporting that it caught an exception (and handled it by throwing its own exception in response). A "backtrace" corresponding to where the react appeared in your code is provided indented from the initial "A react block:".
The error message continues with the react construct summarizing its own exception (Died because ...) and explains itself by reporting the original exception, further indented, in subsequent lines. This includes another backtrace, this time one corresponding to the original exception, which will likely have occurred on a different thread with a different callstack.
(All of Raku's structured multithreading constructs[1] use this two part error reporting approach for exceptions they catch and handle by throwing another exception.)
The first backtrace indicates the react line:
in sub provider at DataProvider.pm6 (DataProvider) line 5
use MQTT::Client;
sub provider() is export {
my $mqtt = MQTT::Client.new: server => 'localhost';
react {
The second backtrace is about the original exception:
Invocant of method 'write' must be an object instance of type
'IO::Socket::Async', not a type object of type 'IO::Socket::Async'. ...
in method subscribe at ... (MQTT::Client) line 133
This reports that the write method called on line 133 of MQTT::Client requires its invocant is an instance of type 'IO::Socket::Async'. The value it got was of that type but was not an instance, but instead a "type object". (All values of non-native types are either type objects or instances of their type.).
The error message concludes with:
Did you forget a '.new'?
This is a succinct hint based on the reality that 99 times out of a hundred the reason a type object is encountered when an instance is required is that code has failed to initialize a variable. (One of the things type objects are used for is to serve the role of "undefined" in languages like Perl.)
So, can you see why something that should have been an initialized instance of 'IO::Socket::Async' is instead an uninitialized one?
Footnotes
[1] Raku's constructs for parallelism, concurrency, and asynchrony follow the structured programming paradigm. See Parallelism, Concurrency, and Asynchrony in Raku for Jonathan Worthington's video presentation of this overall approach. Structured constructs like react can cleanly observe, contain, and manage events that occur anywhere within their execution scope, including errors such as error exceptions, even if they happen on other threads.
You seem to be fine now but when I first saw this I made this https://github.com/jonathanstowe/Cro-MQTT which turns the MQTT client into a first class Cro service.
I haven't released it yet but it may be instructive.

Problem with Prefix in Discord.js (including .toUpperCase())

Basically, I've been developing a bot for several weeks now using the discord.js library and recently encountered a small but crucial issue. Essentially when I declare my argument, I also made it so that the message content (message.content) would be capitalized using .toUpperCase(). Essentially in doing so regardless of the type of prefix you would enter (symbol wise) it would all be read by the program as valid.
For example only: !help - should work, however if I would enter .help, it would also be read as valid.
In any case, here's the code. I appreciate all the help!
bot.on('message', message =>{
let args = message.content.toUpperCase().substring(PREFIX.length).split(" ");
const sender = message.member;
switch(args[0])
{
case 'HELP':
message.reply("I've sent you some documentation on all the commands that you can use...").then(d_msg => {d_msg.delete(3000); });
message.delete(3000);
const attachment = new Attachment('./UtilityBot_Documentation.txt')
message.author.send('[Education] Bot - Documentation');
message.author.send(attachment);
break;
}
})
The discord.js tutorial covers an extremely similar problem to what you're trying to do. I recommend you check it out. The page I linked in specific is doing a very similar thing to you, but it's worth giving the whole thing a read through if you haven't done so already. In general, I would include the following line just above where you established args.
if (!message.content.startsWith(PREFIX)) return;
What I'm doing here is saying if the message does not start with the prefix, stop running until a new message is sent. I might be missing something, but certainly check out the tutorial. It's really well written.
https://discordjs.guide/creating-your-bot/commands-with-user-input.html#basic-arguments

Finding out the port an asynchronous socket is binding to?

Since version 6.d of Perl 6, you can use port 0 to ask the interpreter to find a port to bind for you:
my $socket = IO::Socket::Async.listen($SOCKET_ADDR, 0);
However, $socket is a Supply with no information on the low-level socket it's using. What is the way of finding which port is it binding to?
When you tap the $socket variable you get back a (currently undocumented) IO::Socket::Async::ListenSocket object. This has a couple of methods socket-port and socket-host which are Promises then when they resolve have the correct values.
We can probably tidy up the docs to indicate this.
Example :
my $s = IO::Socket::Async.listen("127.0.0.1",0);
my $t = $s.tap;
my $p = await $t.socket-port;
$p.say;

Values lifetime in mod_perl

.pm file:
package fo_condition_editor;
use utf8;
use diagnostics -trace;
use strict;
use warnings FATAL => 'all';
{...}
use Encode;
my $msg = {};
return 1;
{..}
sub ..() {
$msg->{saved} = 1;
I use this pm to show popup. When form is submitted, popup is refreshed.
In my local server everything works fine, but in other server i had problem with variables $msg. $msg is empty during printing, but when i submit again in $msg are old things.
I think is problem with apache configuration.
The probloem - if I get this correctly - is that the code
my $msg = {};
is only executed when this package is required/used for the first time. After that (in the current mod_perl Instance) this wont be executed any more, and $msg keeps whatever value it has for the next requests.
There are a lot of ways to work around this problem. One schema, I use some times, is to define a "tear-down/reset" method for each "package / module-Entity" I use. In the package itself I push a reference of this method to a global Variable. And in my "core-Handler" called by mod_perl I have a tear-down/reset method, which iterates over the registered handlers and calls them to reset the data.
HTH
Georg

How to flush current output in RApache?

I'm testing using RApache as an SSE (Server Sent Events) and similar (long poll, comet, etc.) back-end. I seem to be stuck on how to flush my output. Is it possible?
Here is my test R script:
setContentType("text/plain")
repeat{
cat(format(Sys.time()),"\n")
#sendBin(paste(format(Sys.time()),"\n"))
flush(stdout())
Sys.sleep(1)
}
My Rapache.conf entry is:
<Location /rtest/sse>
Options -MultiViews
SetHandler r-handler
RFileHandler /var/www/local/rtest/sse.r
</Location>
And I test it using either wget or curl:
wget -O - http://localhost/rtest/sse
curl http://localhost/rtest/sse
Both just sit there, meaning nothing is being sent.
Using sendBin() made no change, and neither did using flush().
If I change repeat to for(i in 1:5) then it sits there for 5 seconds and then shows 5 timestamps (spaced one second apart). So, I believe everything else is working fine and this is purely a buffering issue.
UPDATE: Looking at this with fresh eyes after 5 months, I think I could have described the problem more clearly: the problem is that RApache appears to be buffering all the output, and not sending anything until the R script exits. To be useful for streaming it has to send data out of Apache and on to the client each time flush() is called, i.e. while the R script is still running.
So, my question is: is there a way to get RApache to behave like that?
UPDATE 2 I tried adding flush.console() before or after the flush(stdout()) but no difference. I also tried setStatus(status=200L) at the top. And I tried SERVER$no_cache=T;SERVER$no_local_copy=T; at the top of the script. Again it made no difference. (Yes, none of those should have helped, but it never hurts to try!)
Here is a link to how PHP implements flush when it is running as an Apache module:
http://git.php.net/?p=php-src.git;a=blob;f=sapi/apache2handler/sapi_apache2.c#l290
I think the key point is that there is a call to ap_rflush(r). I'm guessing that RApache is not making the ap_rflush() call.
You are passing the wrong MIME type. Try changing with
setContentType("text/event-stream")
EDIT1:
this is the attempt, (still unsuccessful) I mentioned in the comment below, to implement SSE in Rook.
<%
res$header('Content-Type', 'text/event-stream')
res$header('Cache-Control', 'no-cache')
res$header('Connection', 'keep-alive')
A <- 1
sendMessage <- function(){
while(A<=4){
cat("id: ", Sys.time(), "\n", "data: hello\n\n", sep="")
A <- A+1
flush(stdout())
Sys.sleep(1)
}
}
-%>
<% sendMessage() %>
the while loop condition was supposed to be always TRUE but I'm having your same problem so I had to do a finite loop...
The good new is I DO have data reaching the browser. I can tell by looking, in developer tools, at the Content-Length in the Response Header section. it says 114 for the above code and you change, say, "Hello" in "Hello!" it'll say 118.
The js code is: (you'll need JQuery as well)
$(document).ready(function(){
$("button").click(function(){
var source = new EventSource("../R/sse.Rhtml");
source.onopen = function(event){
console.log("readyState: " + source.readyState);
}
source.onmessage = function(event){
$("#div").append(event.data);
};
source.onerror = function(event){
console.log(event);
};
});
});
So, in essence
1) The connection is open (readyState 1)
2) Buffering is still there
3) Data (after buffering) reaches the browser but an error happens in receiving them properly.
EIDT2:
it's interesting to note that brew()ing the above .Rhtml file the output is not buffered. There must be a configuration the in the web server (both the R internal and Apache) that buffer the data flows.
As a side note, flush is not even needed, cat's output defaults to stout(). So the options are:
Web server configuration
The R equivalent of the PHP ob_flush(); which is always used in any PHP implementation I've seen. this is example