I would like to use WinPCap library for "reliable" UDP receiving in my C++ application. All examples that I found, using this library for capturing and then proceding. Is there any way (example) how to configure PCap for streaming mode and receive UDP only and on uder defined port or how to solve this. In this time I have reliable UDP server able to receiving 0.5Gb/s. But on slower PC I have a packet lose I can see packets in ethereal but not in application.
thanks
vsm
I assume that you have already tried all of the more standard methods of increasing the number of datagrams that you are able to process? Things like increasing the recv buffer size, speeding up the processing that you do per datagram and using IOCP to allow you to bring more threads to bear on the problem or using RIO if you can target Windows 8?
If so then using WinPCap might work but it sounds like a bit of an extreme solution.
What you need to do is create a filter so that you only capture the datagrams that you are interested in... The docs include examples which use filters.
I have server from here: http://www.gamedev.net/topic/533159-article-using-udp-with-iocp/. This code working with IOCP. Its working fine on WIndows XP. There is no problem with receiving 0.5Gb/s. But now on Win7 is little unreliable. Sometimes there are packets positions error. (my device generating udp packets and in its payload there is PacketNumber - number increasing with each packet. When error occured i write all packet numbers into file. I can see for exmaple: 10,11,290,13,14... ). Is there any known differences in WinXP and Win7 for IOCP and multi threading? Or do you konw any free UDP server with IOCP processing?
In procedding loop I only adding packets into buffer and checking their numbers.
Related
In the last couple of weeks I've been developing a boot loader that performs a firmware update on a certain device. The setup is as follows:
The firmware binary and its respective SHA1 hash are stored in a web server;
The device is composed of an ESP8266 and a STM32 microcontroller (STM32F401 or STM32F030, there two hardware versions, but the one I'm using is the F401). The ESP is used only with AT+ commands, i.e., I did not built it's firmware, just used the latest version from Espressif.
The idea is that, the STM32 bootloader should use the ESP to download the firmware hash and binary from the webserver and then boot the firmware if the hash is OK. The download is made using the ESP in passive mode, i.e. the STM has to manually request X bytes to read from the ESP buffer, currently I'm using 1 MTU (1460 bytes).
At first, the connection to the webserver was made using HTTP and everything worked perfectly, however, I had to change it to HTTPS, and that's where the problem starts. After the STM has received around 100kB of the firmware (which has 110kB), the ESP only provides 30 bytes per request (which should be around 1 MTU), thus, making the download time extremely large.
I've already did some digging trying to find out if this is related to the ESP, but didn't find anything. Also, the point where this 30 byte download rate starts to happen isn't always at the 100kB mark, I've tested with a 170kB firmware and It started to happen at 160kB ish, so, it looks like it's always the last 10kB.
I've also added some delays in the firmware when the packet size becomes smaller than 1 MTU, to give more time for the ESP to process the packet, since the SSL decryption stuff takes longer to process; but it did not help.
My question is: is there some characteristic in the HTTPS/SSL protocols that reduces the packet length? What could be the causes of what is happening here?
I am new to Wireshark and capturing packets and all Stuff. Let me get it to the straight.
I have a hardware which outputs its data over Ethernet using a UDP Broadcast. I Can directly plug a Ethernet Cable to a In-line RJ-45 Coupler (attached to the hardware) and my PC Running Wireshark.
REQUIREMENTS : I need to Capture RAW Data which my hardware is broadcasting so that it can be given to other team so as to know the format in which it is providing for further post processing.
What I Did : Initially , I connected the Ethernet Cable from my home and Started capturing the packets which didn't make any sense to me.
Can you please point out if I am going in correct direction ? Sorry if its a very basic question, but raw data from the hardware is important for my further tasks....
As far as any software can understand a wire you will always get a packet. Between you (in front of a computer) and the cable in the in the RJ-45 jack sits a NIC (network interface controller, i.e. your network card).
Your Ethernet NIC will read the current on the cable (in manchester encoding for ethernet) and synchronize itself to any Ehternet traffic on that cable. What does "synchronizing" mane in there? In front of any Ehternet traffic come 64 alternate bits of 0s and 1s which are meant to synchronize the clocks on both communicating NICs. Without proper clock synchronization some data may be misinterpreted.
But why I am talking about clock synchronization? Because if you want the data as RAW as it is on the cable you will not get it. A NIC will never send any synchronization bit to the rest of the computer, therefore it is absolutely impossible to read exactly what is on the cable by using software.
On the other hand I find hard to believe you want the RAW data as RAW as that. After the synchronization bits come an Ethernet encapsulated packed. Yup, Ethernet uses packets. They're link layer packets (layer 2 in OSI).
And wireshark gives you exactly that (in most cases, see note at the end for two exceptions to this rule): every Ehternet packet that the NIC understands, manages to sync, and manages to read without collision is sent to the kernel and then read by wireshark. A cable has electrical interference and has no provision against collisions (it's just a piece of cooper!) therefore the NIC abstracts things like interferences and collisions.
I'll repeat it once more: After abstracting the synchronization bits, sender collisions (which turn the cable into one huge interference) and plain interferences; all that remains is a stream of packets, one after the other.
Extra Notes
NICs sometimes do ignore some Ethernet packets: packets that are not directed to their MAC. This can be changed by enabling promiscuous mode (available in most NICs). This is irrelevant for broadcast packets.
There are exception to the rule of wireshark getting all the traffic coming from the NIC:
If the traffic comes incredibly quick, wireshark may drop out of kernel schedule and not see some packets. It happens, nothign can be done about it.
If you listen on all interfaces (as opposed to selecting a single interface to listen at), wireshark will strip the Ethernet (or Wifi) headers. This is a wireshark hack needed to make output files uniform (and possible to be read by other applications).
TL;DR, wireshark output (pcap) is pretty much just the stream of packets that it got from the NIC, one after the other. That is as RAW as you can get with software.
I am creating a server on a ST Cortex M3 device. I am using the lwip API and FreeRTOS. All is working, but the response time is way off. I am currently using lwip 1.3.2 and FreeRTOS 7.3.
A single client connects to the server and must have some time-critical data sent frequently. These packets are on the order of 6 or so bytes. Other times, I am sending upwards of 20K.
The problem I am having is that these smaller packets seem to be taking forever to be sent. I assume this is because lwip is waiting for more data to be enqueued to make more efficient transmissions. I cannot wait around for 2 or 3 seconds for the data to be sent; the client is expecting the data nominally in a few micro-seconds or milli-seconds.
I have tried using lwip_send and lwip_write. (I understand that one is the same as the other with a flag passed at the end. Just had to try...) I have tried setting TCP_NODELAY on the socket to no avail. I tried to set SO_SNDLOWAT to '1', but this always returned -1, so I do not think it is supported.
I do not want to redo all of my code using TCP RAW. Is there a way to invoke the tcp_output() function outside of TCP RAW mode? Is there any way to speed things up or is this just how slow lwip TCP with small packets is?
Any and all suggestions are welcome. Thanks.
--EDIT--
I would also like to add that once I am ready to transmit, I make sure that my TX task in FreeRTOS is at the highest priority. There are no other tasks running up to the point at which I call lwip_send/write.
I'm fairly experienced with bare metal lwIP on xilinx and lwip does not wait to send things out. It will pump packets out as fast as your interrupts are acknowledged based on the ethernet hardware. I've been using UDP only. What is coming to mind though, is your problem might be on the receive end. If you are doing TCP, maybe those small packets are coming out late because you are having receive issues. What you need to do is find in the code the lowest level point at which ethernet is transmit, put a general purpose output toggle on that. Then also put a general purpose output toggle on when a ethernet packet is received. Look at the signals on a scope. If it confirms your hypothesis, then move the output toggles around to narrow down the issue. Wash, rinse and repeat until you are down to where the issue its. It's crude and time consuming, but oftentimes this brute force approach solves many "impossible" embedded software problems, due to pure determination. Good luck!
Hi I am writing a program that will send a file from client to server using UDP socket using different packet sizes for example 512B, 1KB and 2KB and i don't want use fixed buffer size in the receiver(server).I need some codes in Java that will allow both server and client to agree upon a packet size before transfer start. Many thanks
Don't you forget that UDP packets may be fragmented, duplicated and lost? There is a whole bunch of things to take care of, starting with lost packet retransmissions.
I hate to give a "don't do this" kind of answers, but for this one, just use TCP. And if you want some user-level "packets", you can have them with TCP also (prefix each one with its length, that's enough).
I am creating an app that acts as a remote control for a lighting console and I need to send commands to the console over UDP. The protocol that I am using has its own custom header. How do I create the data packet with header and message to send over UDP? Thanks!
If you are trying to test the protocol, without writing any code, I suggest you use WireShark.
The probably most powerful solution you can use is scapy, which is a python module that allows very advanced packet crafting and manipulation. See its documentation or search the interwebs for examples to find out how to generate arbitrary packets and transmit them.
If you can't use python for some reason, there are multiple command line tools for packet generation, one other example being nping (documentation), the brother of nmap, the popular network scanner. nping has options to generate UDP packets with arbitrary payloads, with can be specified as a hex string, for example.
There may be other options as well. It would be good to know more details like the operating system you're on or where you get your input data from, and in which format.