Connecting to a UDP data stream with Mathematica - udp

I have an app on my iPhone called iSeismometer which reads the iPhone's accelerometers and acts as a server which streams this data via UDP (I can set the IP address and port number). The question is how to read this data stream with Mathematica? Apparently, Dreeves has been looking into this 12 years ago, so I imagine something must have happened in the meantime.
Update
I got two great answers so far; one from WReach and one from Mark McClure. Both are using JLink to get at the data. This seems like a fine approach. However, I was reminded of some work I did on the WII balance board. Using a few free programs (GlovePIE and PPJoy) I got this bluetooth peripheral to appear as a joystick to Windows, and therefore also to Mathematica (via ControllerState). Of course, bluetooth and UDP are quite different, but could something along the same lines be made to work too?

JLink is definitely the way to go. I prefer to keep my Java code and my Mathematica code separate by compiling a Java programwhich I then call from Mathematica. I set up a Notebook and companion Java program that you can grab here:
http://facstaff.unca.edu/mcmcclur/UDPFiles.tar.gz
Here is the essential Mathematica code:
Needs["JLink`"];
InstallJava[];
AddToClassPath[NotebookDirectory[]];
udpReader = JavaNew["myClient"];
i = 0;
While[True && i++ < 100,
Print[udpReader#udpReadOne[10552]]]
The updReader class is defined by the following Java code.
// A simple UDP client to read from iseismometer:
// http://www.iseismometer.com/
// You can run this from the command line via "java myClient"
// to check that your iseismometer setup is correct or you can
// call the the udpReadOne method from another program.
import java.io.*;
import java.net.*;
import java.util.*;
public class myClient {
public static void main() throws IOException {
DatagramSocket socket = new DatagramSocket(10552);
byte[] buffer = new byte[500];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
while(true) {
socket.receive(packet);
String received = new String(packet.getData(), 0, packet.getLength());
System.out.println(received);
}
}
public static String udpReadOne(int port) throws IOException {
DatagramSocket socket = new DatagramSocket(port);
byte[] buffer = new byte[100];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
socket.receive(packet);
String received = new String(packet.getData(), 0, packet.getLength());
socket.close();
return received;
}
}
Note that you can use the main method of the myClient class to check that your setup is working without Mathematica, essentially taking one potential issue out of the loop.

Assuming the set-up discussed in a blog entry on the iSeismometer web site, a couple of options come to mind.
Import
The first option would be to use an external program to capture the packets, and then use Import to bring in the results, e.g.
Import["!someexternalprog", "Lines"]
Alas, the Python program mentioned in the blog post will not work well here since it runs in an endless loop that must be manually terminated. The Import approach would only work if that program were modified to stop after a fixed number of packets or a time limit or something.
JLink
An alternate approach can be implemented without leaving the comfy Mathematica environment by using JLink. Well, perhaps it is a stretch to say that we are staying within Mathematica since a fair amount of funny-looking Java code is mixed in with the Mathematica code. Nevertheless, it does illustrate the utility of the built-in Java distribution that ships with every copy of Mathematica:
Needs["JLink`"]
LoadJavaClass["java.util.Arrays"];
ClearAll#ListenToISeismometer
ListenToISeismometer[port_] :=
JavaBlock#Module[{socket, packet, listen, record = Null, listening = True}
, packet = JavaNew["java.net.DatagramPacket", JavaNew["[B", 1024], 1024]
; listen[] :=
If[$Failed =!= Quiet[socket#receive[packet], Java::excptn]
, record =
JavaNew[
"java.lang.String"
, java`util`Arrays`copyOfRange ## packet /# {getData[], getOffset[], getLength[]}
]#toString[] // Sow
]
; Row[{Button["Stop", listening = False], Dynamic[record]}, " "] // PrintTemporary
; AbortProtect[
socket = JavaNew["java.net.DatagramSocket", port]
; socket#setSoTimeout[1000]
; Reap[While[listening, listen[]]; socket#close[]][[2, 1]]
]
]
Some shortcuts have been taken with respect to exception handling, packet decoding and the like in order to keep this example at a manageable length.
ListenToISeismometer needs to be given the UDP port number to listen upon. Let's use the same port as in the blog post, 10552:
In[33]:= data = ListenToISeismometer[10552];
The function will listen to all UDP events on that port until told to stop. A button is presented for this purpose, with each packet flashing by along side as received.
When the button is pressed, the function returns a list of the packets received:
In[34]:= data // Column
Out[34]= 1,83575.099,0.029,0.044,0.094
1,83575.781,0.056,0.033,0.099
1,83575.924,0.047,0.054,0.094
1,83575.613,0.096,0.092,0.057
1,83575.748,0.073,0.049,0.061
1,83575.577,0.008,0.089,0.020
...
JLink makes this possible, but there is no escaping the fact that the use of JLink requires a working knowledge of Java.

Related

Apache Beam : RabbitMqIO watermark doesn't advance

I need some help please. I'm trying to use Apache beam with RabbitMqIO source (version 2.11.0) and AfterWatermark.pastEndOfWindow trigger. It seems like the RabbitMqIO's watermark doesn't advance and remain the same. Because of this behavior, the AfterWatermark trigger doesn't work. When I use others triggers which doesn't take watermark in consideration, that works (eg: AfterProcessingTime, AfterPane) Below, my code, thanks :
public class Main {
private static final Logger LOGGER = LoggerFactory.getLogger(Main.class);
// Window declaration with trigger
public static Window<RabbitMqMessage> window() {
return Window. <RabbitMqMessage>into(FixedWindows.of(Duration.standardSeconds(60)))
.triggering(AfterWatermark.pastEndOfWindow())
.withAllowedLateness(Duration.ZERO)
.accumulatingFiredPanes();
}
public static void main(String[] args) {
SpringApplication.run(Main.class, args);
// pipeline creation
PipelineOptions options = PipelineOptionsFactory.fromArgs(args).create();
Pipeline pipeline = Pipeline.create(options);
// Using RabbitMqIO
PCollection<RabbitMqMessage> messages = pipeline
.apply(RabbitMqIO.read().withUri("amqp://guest:guest#localhost:5672").withQueue("test"));
PCollection<RabbitMqMessage> windowedData = messages.apply("Windowing", window());
windowedData.apply(Combine.globally(new MyCombine()).withoutDefaults());
pipeline.run();
}
}
class MyCombine implements SerializableFunction<Iterable<RabbitMqMessage>, RabbitMqMessage> {
private static final Logger LOGGER = LoggerFactory.getLogger(MyCombineKafka.class);
/**
*
*/
private static final long serialVersionUID = 6143898367853230506L;
#Override
public RabbitMqMessage apply(Iterable<RabbitMqMessage> input) {
LOGGER.info("After trigger launched");
return null;
}
}
I spent a lot of time looking into this. After opening https://issues.apache.org/jira/browse/BEAM-8347 I left some notes in the ticket on what I think the problems are with the current implementation.
Re-stated here:
The documentation for UnboundedSource.getWatermark reads:
[watermark] can be approximate. If records are read that violate this guarantee, they will be considered late, which will affect how
they will be processed. ...
However, this value should be as late as possible. Downstream windows may not be able to close until this watermark passes their
end.
For example, a source may know that the records it reads will be in timestamp order. In this case, the watermark can be the timestamp
of the last record read. For a source that does not have natural
timestamps, timestamps can be set to the time of reading, in which
case the watermark is the current clock time.
The implementation in UnboundedRabbitMqReader uses the oldest timestamp as the watermark, in violation of the above suggestion.
Further, the timestamp applied is delivery time, which should be monotonically increasing. We should reliably be able to increase the watermark on every message delivered, which mostly solves the issue.
Finally, we can make provisions for increasing the watermark even when no messages have come in. In the event where there are no new messages, it should be ok to advance the watermark following the approach taken in the kafka io TimestampPolicyFactory when the stream is 'caught up'. In this case, we would increment the watermark to, e.g., max(current watermark, NOW - 2 seconds) when we see no new messages, just to ensure windows/triggers can fire without requiring new data.
Unfortunately, it's difficult to make these slight modifications locally as the Rabbit implementations are closed to extension, and are mostly private or package-private.
Update: I've opened a PR upstream to address this. Changes here: https://github.com/apache/beam/pull/9820

How to send message to all players on server? (Server-side)

How would I send a message to all of the players on the server? (When trying to do so, it only outputs to the console, and I believe that is because the mod is not installed on the client side.)
I have been trying to make a mod for a 1.7.10 server (To put in a 1.7.10 mod pack) that can message all of the players online. I have looked this question up, and have not found any answers.
#SideOnly(Side.SERVER)
#SubscribeEvent
public void onDeath(PlayerEvent.Clone event)
{
if (event.wasDeath) {
final String[] messages = {"Oh boiss we got a respawner O_O", "How dare ye respawn on me?", "GAAH! You died again!", "._. Just why...", "Was taht me or waas that you? -.-","Why isn't this in hardcore mode? It should be..."};
Random random = new Random();
int index = random.nextInt(messages.length);
ChatComponentText text = new ChatComponentText(messages[index]);
ChatStyle style = new ChatStyle();
style.setColor(EnumChatFormatting.LIGHT_PURPLE);
text.setChatStyle(style);
FMLCommonHandler.instance().getMinecraftServerInstance().getConfigurationManager().sendChatMsg(text);
System.out.println("Respawned");
}
}
I expect that the server will send a message to all, but only outputs to the console.
A really quick way this could be handled is to create an EventHandler for when a player joins. Then add them to an ArrayList. Then when they leave (Check for kick / quit event). Remove them from the ArrayList. By having an arraylist you can run through this and message every player.
Your 'System.out.println("Respawned");` line will only print to the console.
I believe what you are looking to do is the following:
1) Loop through all the players on the server.
2) Send each player the calculated message.
I'm not 100% sure how to access the player-list off the top of my head, but you need to access the FMLServerHandler and get the player-list, OR (the better way) access the EntityPlayer objects connected to the player's current world and do the above steps. The second method would only work for the current world, so if you wanted to access all the connections to the server, the first method is the way to go.

Cro::WebSocket::Client doesn't work

Created a websocket server with "cro sub".
Wrote this client:
use v6;
use Cro::WebSocket::Client;
constant WS-PORT = '20000';
constant WS-ADDRESS = 'localhost';
constant WS-PATH = 'chat';
constant WS-URL = 'ws://' ~ WS-ADDRESS ~ ':' ~ WS-PORT ~ '/' ~ WS-PATH;
constant TIMEOUT-TO-CONNECT = 5; # seconds
my $timeout;
my $connection-attempt;
await Promise.anyof(
$connection-attempt = Cro::WebSocket::Client.connect(WS-URL),
$timeout = Promise.in(TIMEOUT-TO-CONNECT));
if $timeout.status == Kept
{
say "* could not connect to server in ', TIMEOUT-TO-CONNECT, ' seconds";
exit 1;
}
if $connection-attempt.status != Kept
{
say "* error ", $connection-attempt.cause,
" when trying to connect to server";
exit 1;
}
my $connection = $connection-attempt.result;
my $peer = WS-ADDRESS ~ ':' ~ WS-PORT;
say '* connected with ', $peer;
my $counter = 0;
my $message-supplier = Supplier::Preserving.new;
my $has-message-to-send = $message-supplier.Supply;
$message-supplier.emit(1);
react
{
whenever $has-message-to-send
{
$counter++;
$connection.send($counter);
say "* ok, sent message ", $counter, " to server";
}
whenever $connection.messages -> $reply
{
say '* received reply=[' ~ $reply ~ '] from server';
$message-supplier.emit(1);
}
} # react
I see with tcpdump the response code 101 (switching protocols) from the server, but I don't see the message sent from the client to the server.
So, what am I doing wrong ?
Another question, shoudn't "$connection.send" return a Promise or something ? What if there's an error when sending ?
And another question: it seems the server only understands IPV6 addresses...how to make it understand IPV4 addresses ?
That's it, for now.
UPDATE
As per Takao's advice, changing
$connection.send($counter)
to
$connection.send($counter.Str)
solves the problem (though I tried it on another program, not this one).
Let's resolve this piece by piece.
Firstly, your code looks correct to me, except for a couple of tiny bits.
When I reproduced your code, it indeed did not work, so I tried it with cro trace . instead of cro run .. You can find info about that mode in official docs.
The alternative way is to just set CRO_TRACE=1 environment variable.
So during debug I saw this error:
[TRACE(anon 1)] Cro::HTTP::ResponseParser QUIT No applicable body serializer could be found for this message
As it says, the body you sent could not be serialized. So I looked into what are you sending: $counter. $counter in your code is Int, so we need to make it Str before, doing simple $counter.Str makes your example work.
Also, note that you are sending a message on every reply, and echo server (default one you created using cro stub) also sends a reply for every incoming message, so your example sends messages endlessly. To prevent that you may consider adding a condition under which you will no longer send things, but well, it is a test example anyway, so up to you.
As for your other questions:
Another question, shoudn't "$connection.send" return a Promise or something?
It should not, I'll write out some cro's architecture details to explain it next. As you may know from reading docs, cro pipeline is basically just a bunch of Cro::Transform-wrapped supplies. Inside of Cro::Websocket::Client::Connection, send method just sends a thing directly into Cro::Source of the whole pipeline, you cannot go wrong with a simple $supplier.emit($message)(the real implementation of this method looks very close to this line). The thing you bumped into occurred further in the pipeline. I am sure it is not a nice user experience to hide exceptions of such cases, so I'll consider making a patch to propagate the exception, so it'd be easier to catch(although you always can use debug mode).
it seems the server only understands IPV6 addresses...how to make it understand IPV4 addresses ?
I am not sure about that, please open a new question.

Consumable channel

Use Case
Android fragment that consumes items of T from a ReceiveChannel<T>. Once consumed, the Ts should be removed from the ReceiveChannel<T>.
I need a ReceiveChannel<T> that supports consuming items from it. It should function as a FIFO queue.
I currently attach to the channel from my UI like:
launch(uiJob) { channel.consumeEach{ /** ... */ } }
I detach by calling uiJob.cancel().
Desired behavior:
val channel = Channel<Int>(UNLIMITED)
channel.send(1)
channel.send(2)
// ui attaches, receives `1` and `2`
channel.send(3) // ui immediately receives `3`
// ui detaches
channel.send(4)
channel.send(5)
// ui attaches, receiving `4` and `5`
Unfortunately, when I detach from the channel, the channel is closed. This causes .send(4) and .send(5) to throw exceptions because the channel is closed. I want to be able to detach from the channel and have it remain usable. How can I do this?
Channel<Int>(UNLIMITED) fits my use case perfect, except that is closes the channel when it is unsubscribed from. I want the channel to remain open. Is this possible?
Channel.consumeEach method calls Channel.consume method which has this line in documentation:
Makes sure that the given block consumes all elements from the given channel by always invoking cancel after the execution of the block.
So the solution is to simply not use consume[Each]. For example you can do:
launch(uiJob) { for (it in channel) { /** ... */ } }
You can use BroadcastChannel. However, you need to specify a limited size (such as 1), as UNLIMITED and 0 (for rendez-vous) are not supported by BroadcastChannel.
You can also use ConflatedBroadcastChannel which always gives the latest value it had to new subscribers, like LiveData is doing.
BTW, is it a big deal if you new Fragment instance receives only the latest value? If not, then just go with ConflatedBroadcastChannel. Otherwise, none of BroacastChannels may suit your use case (try it and see if you get the behavior you're looking for).

Save to buffer instead of file when recording aac with Audio Queue Services

My goal is to record audio in AAC format and send it over a network connection as a stream.
I'm using Audio Queue Services and have based my code on the SpeakHere example. I know that for writing to a file it uses the AudioFileWritePackets() function.
Here's the callback function:
void MyInputBufferHandler(void* inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp * inStartTime,
UInt32 inNumPackets,
const AudioStreamPacketDescription* inPacketDesc) {
Recorder *aqr = (Recorder *)inUserData;
if (inNumPackets > 0) {
// write packets to file
AudioFileWritePackets(aqr->mRecordFile, FALSE,
inBuffer->mAudioDataByteSize,
inPacketDesc, aqr->mRecordPacket,
&inNumPackets, inBuffer->mAudioData);
aqr->mRecordPacket += inNumPackets;
}
// if we're not stopping, re-enqueue the buffe so that it gets filled again
if ([aqr IsRunning])
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
}
At first I thought AudioFileWritePackets works by directly writing the content of inBuffer->mAudioData. However when I manually write just the contents of mAudioData to a file the decoding doesn't seem to work.
On examining and comparing the raw data of what AudioFileWritePackets writes to a file and of me just writing mAudioData to a file, they seem to have attached a header to it.
As I can't find out how AudioFileWritePackets() works inside, my question is, how can I write the recorded audio to a buffer (or stream, if you prefer to call it this way) instead of a file, so that I can later send it over a network connection?
Again, in short here's what I need: record audio in aac format and stream the audio over network connection.
Thanks! I've been searching my head blue...
P.S: please if you point me to existing projects, make sure they are what I need. I really feel like I've been looking for all possible projects out there >.<
This should help.
Check the "HandleInputBuffer" method in "SpeechToTextModule.m".