I'm trying to write my first script in Zeek which would allow to make statistics out of TLS packet segments sent and received by client in local network (quantity of packets with same size, list of dest ip by packets sent). Unfortunately, I'm unable to find proper Event or guide which would help me to find a solution for this. May I get an advise of this one?
Zeek has a few packet-level events that might get you started:
https://docs.zeek.org/en/current/scripts/base/bif/event.bif.zeek.html#id-new_packet
https://docs.zeek.org/en/current/scripts/base/bif/event.bif.zeek.html#id-raw_packet
https://docs.zeek.org/en/current/scripts/base/bif/event.bif.zeek.html#id-packet_contents
Note the warning that comes with these events: they incur high per-event overhead since they'll be generated for every packet, so they're most likely not suitable for deployment on live traffic.
Related
I'm currently working a network protocol which includes a client-to-client system with auto-discovering of clients on the current local network.
Right now, I'm periodically broadsting over 255.255.255.255 and if a client doesn't emit for 30 seconds I consider it dead (then offline). The goal is to keep an up-to-date list of clients runing. It's working well using UDP, but UDP does not ensure that the packets have been sucessfully delivered. So when it comes to the WiFi parts of the network, I sometimes have "false postivives" of dead clients. Currently I've reduced the time between 2 broadcasts to solve the issue (still not working well), but I don't find this clean.
Is there anything I can do to keep a list of "online" clients without this risk of "false positives" ?
To minimize the false positives, due to dropped packets you should alter a little bit the logic of your heartbeat protocol.
Rather than relying on a single packet broadcast per N seconds, you can send a burst 3 or more packets immediately one after the other every N seconds. This is an approach that ping and traceroute tools follow. With this method you decrease significantly the probability of a lost announcement from a peer.
Furthermore, you can specify a certain number of lost announcements that your application can afford. Also, in order to minimize the possibility of packet loss using wireless network, try to minimize as much as possible the size of the broadcast UDP packet.
You can turn this over, so you will broadcast "ServerIsUp" message
and every client than can register on server. When client is going offline it will unregister, otherwise you can consider it alive.
I am creating a server on a ST Cortex M3 device. I am using the lwip API and FreeRTOS. All is working, but the response time is way off. I am currently using lwip 1.3.2 and FreeRTOS 7.3.
A single client connects to the server and must have some time-critical data sent frequently. These packets are on the order of 6 or so bytes. Other times, I am sending upwards of 20K.
The problem I am having is that these smaller packets seem to be taking forever to be sent. I assume this is because lwip is waiting for more data to be enqueued to make more efficient transmissions. I cannot wait around for 2 or 3 seconds for the data to be sent; the client is expecting the data nominally in a few micro-seconds or milli-seconds.
I have tried using lwip_send and lwip_write. (I understand that one is the same as the other with a flag passed at the end. Just had to try...) I have tried setting TCP_NODELAY on the socket to no avail. I tried to set SO_SNDLOWAT to '1', but this always returned -1, so I do not think it is supported.
I do not want to redo all of my code using TCP RAW. Is there a way to invoke the tcp_output() function outside of TCP RAW mode? Is there any way to speed things up or is this just how slow lwip TCP with small packets is?
Any and all suggestions are welcome. Thanks.
--EDIT--
I would also like to add that once I am ready to transmit, I make sure that my TX task in FreeRTOS is at the highest priority. There are no other tasks running up to the point at which I call lwip_send/write.
I'm fairly experienced with bare metal lwIP on xilinx and lwip does not wait to send things out. It will pump packets out as fast as your interrupts are acknowledged based on the ethernet hardware. I've been using UDP only. What is coming to mind though, is your problem might be on the receive end. If you are doing TCP, maybe those small packets are coming out late because you are having receive issues. What you need to do is find in the code the lowest level point at which ethernet is transmit, put a general purpose output toggle on that. Then also put a general purpose output toggle on when a ethernet packet is received. Look at the signals on a scope. If it confirms your hypothesis, then move the output toggles around to narrow down the issue. Wash, rinse and repeat until you are down to where the issue its. It's crude and time consuming, but oftentimes this brute force approach solves many "impossible" embedded software problems, due to pure determination. Good luck!
I'm experimenting with webRTC and it seems that there's an arbitrary limit to how many bytes can be sent in each message. This guy whose example I used chose a limit of 100 (plus some) bytes. In my tests it seems to be close to 200 bytes. However from reading on TCP and UDP those protocols support packages of up to around 65kb and even when taking the MTU for different types of networks into account it should still be a lot more space available than ~200 bytes.
The only source I've found that mentions a hard limit is this WebRTC Data Channel Protocol draft but it only says TBD.
So my questions are:
if there's any source that specifies the current message size limit in any browser?
if I can assume that the limit is always the same, and if not if there's any way my app can be made aware of the limit?
The sharefest project found a way around the rate throttling - you can modify the outgoing offer to change the bandwidth setting (per http://www.ietf.org/rfc/rfc2327.txt)
Details here: https://github.com/Peer5/ShareFest/blob/master/public/js/peerConnectionImplChrome.js#L201
From my own experience you're still limited to ~800 bytes per message.
I've been testing sending jpegs to chrome 57 over the data channel, and messages up to 64k seem to be reliable now.
The webRTC data channel does have a reliability mechanism, it uses SCTP over DTLS (over UDP) - SCTP lets you set reliability and ordering behaviour, but by default WebRTC uses ordered+reliable - meaning you get similar semantics to that of TCP - except that the message boundaries are preserved - at least in theory.
In practice Chrome may deliver partial messages up to the javascript if it runs out of space so it is best to check that you have a complete message before processing it.
Hi I am writing a program that will send a file from client to server using UDP socket using different packet sizes for example 512B, 1KB and 2KB and i don't want use fixed buffer size in the receiver(server).I need some codes in Java that will allow both server and client to agree upon a packet size before transfer start. Many thanks
Don't you forget that UDP packets may be fragmented, duplicated and lost? There is a whole bunch of things to take care of, starting with lost packet retransmissions.
I hate to give a "don't do this" kind of answers, but for this one, just use TCP. And if you want some user-level "packets", you can have them with TCP also (prefix each one with its length, that's enough).
I am building an Arduino-based device that needs to send data over the internet to a remote server. It needs to do this as frequently as possible but also use as little bandwidth as possible. It will probably work over GSM/EDGE (cellular networking).
The data I'd like to send is about 40 bytes in size - really minimal. I'd like to send this packet to the server about once a minute, but also receive a packet of around that size in response once in a while.
The bandwidth on my server is no problem - the bandwidth on the device's internet connection is, i.e. the cellular data.
Do headers on mobile requests and responses count as part of the bandwidth?
Yes, the total packet size is all data that is sent. Assuming a TCP packet you lose 20 bytes right from the start. If you get intimate with Wireshark you can see exactly what's happening.