Clent side of client-server application running multiple sockets hangs under the load. Analysis shows one of communication threads hanging on async_connect with the following stack trace:
ntdll.dll!ZwWaitForSingleObject() + 0xa bytes
mswsock.dll!SockWaitForSingleObject() + 0x13c bytes
mswsock.dll!WSPAccept() + 0x2a3a bytes
ws2_32.dll!WSAAccept() + 0xaf bytes
ws2_32.dll!accept() + 0x12 bytes
boost::asio::detail::socket_select_interrupter::open_descriptors() Line 88 + 0x31 bytes C++
boost::asio::detail::select_reactor::select_reactor(boost::asio::io_service & io_service={...}) Line 47 + 0x90 bytes C++
boost::asio::detail::service_registry::create<boost::asio::detail::select_reactor>(boost::asio::io_service & owner={...}) Line 81 + 0x1f bytes C++
boost::asio::detail::service_registry::do_use_service(const boost::asio::io_service::service::key & key={...}, boost::asio::io_service::service * (boost::asio::io_service &)* factory=0x0000000036cbdda8) Line 123 + 0x6 bytes C++
boost::asio::detail::win_iocp_socket_service_base::start_connect_op(boost::asio::detail::win_iocp_socket_service_base::base_implementation_type & impl={...}, boost::asio::detail::reactor_op * op=0x000000002b9cea30, const sockaddr * addr=0x000000002870f5f0, unsigned __int64 addrlen=16) Line 550 + 0x45 bytes C++
boost::asio::detail::win_iocp_socket_service<boost::asio::ip::tcp>::async_connect<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const & __ptr64>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > >(boost::asio::detail::win_iocp_socket_service<boost::asio::ip::tcp>::implementation_type & impl={...}, const boost::asio::ip::basic_endpoint<boost::asio::ip::tcp> & peer_endpoint={...}, boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const &>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > * handler=0x000000002b9cea30) Line 497 C++
boost::asio::stream_socket_service<boost::asio::ip::tcp>::async_connect<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const & __ptr64>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > >(boost::asio::detail::win_iocp_socket_service<boost::asio::ip::tcp>::implementation_type & impl={...}, const boost::asio::ip::basic_endpoint<boost::asio::ip::tcp> & peer_endpoint={...}, const boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const &>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > & handler={...}) Line 209 C++
boost::asio::basic_socket<boost::asio::ip::tcp,boost::asio::stream_socket_service<boost::asio::ip::tcp> >::async_connect<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const & __ptr64>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > >(const boost::asio::ip::basic_endpoint<boost::asio::ip::tcp> & peer_endpoint={...}, const boost::asio::detail::wrapped_handler<boost::asio::io_service::strand,boost::_bi::bind_t<void,boost::_mfi::mf1<void,Replication::PTCPChannel,boost::system::error_code const &>,boost::_bi::list2<boost::_bi::value<boost::shared_ptr<Replication::PTCPChannel> >,boost::arg<1> > > > & handler={...}) Line 723 C++
Can anyone suggest why is internal socket connection hanging? The hanging happens after a few hours of the load test and is eventually blocking the whole application.
Since this is an internal socket (listening on loopback IP), can anyone explain who is supposed to connect to it and when? Is there a some kind of internal limit of how many connection to loopback ports is possible by asio socket services?
Is there a preferred way to shut down hanging thread in this case?
The app in question is windows app currently running onWin2k8R2 or Win 2k12 R2, boost version used is 1.54.
Could a firewall potentially be blocking connection to the target port? That is one possibility mentioned in this bug report
Look for maxconnection (outbound limit = 2 on outbound for asp.net)
Related
I have an application running on STM32F429ZIT6 using USB stack to communicate with PC client.
MCU receives one type of message of 686 bytes every second and receives another type of message of 14 bytes afterwards with 0.5 seconds of delay between messages. The 14 bytes message is a heartbeat so it needs to replied by MCU.
It happens that after 5 to 10 minutes of continuous operation, MCU is not able to send data because
hcdc->TxState is always busy. Reception works fine.
During Rx interruption, application only adds data to ring buffer, so that this buffer is later serialized and processed by main function.
static int8_t CDC_Receive_HS(uint8_t* Buf, uint32_t *Len) {
/* USER CODE BEGIN 11 */
/* Message RX Completed, Send it to Ring Buffer to be processed at FMC_Run()*/
for(uint16_t i = 0; i < *Len; i++){
ring_push(RMP_RXRingBuffer, (uint8_t *) &Buf[i]);
}
USBD_CDC_SetRxBuffer(&hUsbDeviceHS, &Buf[0]);
USBD_CDC_ReceivePacket(&hUsbDeviceHS);
return (USBD_OK);
/* USER CODE END 11 */ }
USB TX is also kept as simple as possible:
uint8_t CDC_Transmit_HS(uint8_t\* Buf, uint16_t Len) {
uint8_t result = USBD_OK;
/\* USER CODE BEGIN 12 */
USBD_CDC_HandleTypeDef hcdc = (USBD_CDC_HandleTypeDef*)hUsbDeviceHS.pClassData;
if (hcdc-\>TxState != 0)
{
ZF_LOGE("Tx failed, resource busy\\n\\r"); return USBD_BUSY;
}
USBD_CDC_SetTxBuffer(&hUsbDeviceHS, Buf, Len);
result = USBD_CDC_TransmitPacket(&hUsbDeviceHS);
ZF_LOGD("TX Message Result:%d\\n\\r", result);
/ USER CODE END 12 \*/
return result;
}
I'm using latest HAL Drivers and software from CubeIDE (1.27.1).
I have tried expanding heap min size from 0x200 to larger values but result is the same.
Also Line Coding is set according to what recommended values:
case CDC_SET_LINE_CODING:
LineCoding.bitrate = (uint32_t) (pbuf[0] | (pbuf[1] << 8) | (pbuf[2] << 16) | (pbuf[3] << 24));
LineCoding.format = pbuf[4];
LineCoding.paritytype = pbuf[5];
LineCoding.datatype = pbuf[6];
ZF_LOGD("Line Coding Set\n\r");
break;
case CDC_GET_LINE_CODING:
pbuf[0] = (uint8_t) (LineCoding.bitrate);
pbuf[1] = (uint8_t) (LineCoding.bitrate >> 8);
pbuf[2] = (uint8_t) (LineCoding.bitrate >> 16);
pbuf[3] = (uint8_t) (LineCoding.bitrate >> 24);
pbuf[4] = LineCoding.format;
pbuf[5] = LineCoding.paritytype;
pbuf[6] = LineCoding.datatype;
ZF_LOGD("Line Coding Get\n\r");
break;
Thanks in advance, any support is appreciated.
I don't know enough about the STM32 libraries to really check your code, but I suspect you are forgetting to read the bytes transmitted by the STM32 on PC side. Try opening a terminal program like PuTTY and connecting to the STM32's virtual serial port. Otherwise, the Windows USB-to-serial driver (usbser.sys) will eventually have its buffers filled with data from your device and it will stop requesting more, at which point the buffers on your device will fill up as well.
I am trying to get USB device code working on my stm32f401 microcontroller. So far, I'm able to read the packets from the host, send the first device request, receive and apply the assigned address. After that, the host requests for the device descriptor again, and my device doesn't response to that for some reason. I've been fighting this issue for weeks now, seems like I need help with it.
In my code, after initializing the peripherals, I set up my endpoint 0 and set its Tx FIFO size to 16. When the device receives a packet, it reads the packet status and reads the packet content in function setup_packet_handler(). This function prints the content word by word:
uint32_t data = *fifo;
printf("Rx from FIFO: 0x%08x\n", data);
void rxflvl_handler() {
uint32_t packet_status = USB_OTG_FS->GRXSTSP;
uint8_t endpoint_number = (packet_status & USB_OTG_GRXSTSP_EPNUM_Msk) >> USB_OTG_GRXSTSP_EPNUM_Pos;
uint16_t bcnt = (packet_status & USB_OTG_GRXSTSP_BCNT_Msk) >> USB_OTG_GRXSTSP_BCNT_Pos;
uint16_t pktsts = (packet_status & USB_OTG_GRXSTSP_PKTSTS_Msk) >> USB_OTG_GRXSTSP_PKTSTS_Pos;
printf("Received packet of status 0x%02x\n", pktsts);
switch (pktsts) {
case 0x06: // SETUP packet (includes data)
setup_packet_handler(endpoint_number, bcnt);
break;
case 0x02: // OUT packet (includes data)
break;
case 0x04: // SETUP stage has completed
case 0x03: // OUT transfer has completed
// Re-enable the endpoint
OUT_ENDPOINT(endpoint_number)->DOEPCTL |= USB_OTG_DOEPCTL_CNAK;
OUT_ENDPOINT(endpoint_number)->DOEPCTL |= USB_OTG_DOEPCTL_EPENA;
break;
}
}
My main() function watches the current number of packets to send and free space of the Tx FIFO for endpoint 0:
int main(void)
{
/* Pins initialization */
/* USB initialization */
in_packets = (in_endpoint->DIEPTSIZ & USB_OTG_DIEPTSIZ_PKTCNT_Msk) >> USB_OTG_DIEPTSIZ_PKTCNT_Pos;
fifo_space = in_endpoint->DTXFSTS & USB_OTG_DTXFSTS_INEPTFSAV_Msk;
printf("FIFO free space: %i\n", fifo_space);
while (1)
{
// Checking for USB interrupts
usb_poll();
// Notify if packets number changed
uint32_t npkts = (in_endpoint->DIEPTSIZ & USB_OTG_DIEPTSIZ_PKTCNT_Msk) >> USB_OTG_DIEPTSIZ_PKTCNT_Pos;
if (npkts != in_packets) {
printf("Packets number update: %i\n", npkts);
in_packets = npkts;
}
// Notify if FIFO space changed
uint16_t new_space = in_endpoint->DTXFSTS & USB_OTG_DTXFSTS_INEPTFSAV_Msk;
if (new_space != fifo_space) {
fifo_space = new_space;
printf("FIFO free space: %i\n", new_space);
}
}
}
Now the broken part, the code which sends the device descriptor. The function handle_descriptor_request() is called when the device descriptor request is received.
The function sets the size to transmit to 18; packet count to 3 (endpoint 0 max packet size is configured to 8); enables the endpoint and fills the FIFO. After that, prints the FIFO free space, to ensure it has decreased.
void handle_descriptor_request(void) {
printf("Received descriptor request\n");
uint32_t* fifo = (USB_OTG_FS_PERIPH_BASE + USB_OTG_FIFO_BASE +
0 * 0x1000);
USB_OTG_INEndpointTypeDef* in_endpoint = (USB_OTG_INEndpointTypeDef*)(
USB_OTG_FS_PERIPH_BASE +
USB_OTG_IN_ENDPOINT_BASE +
0
);
// flush_txfifo(0);
// in_endpoint->DIEPTSIZ = 0;
// in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_SNAK;
// in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_EPDIS;
printf("Manual PKTCNT to 3\n");
// Set Tx size to 18
in_endpoint->DIEPTSIZ &= ~USB_OTG_DIEPTSIZ_XFRSIZ_Msk;
in_endpoint->DIEPTSIZ &= ~USB_OTG_DIEPTSIZ_PKTCNT_Msk;
in_endpoint->DIEPTSIZ |= 18 << USB_OTG_DIEPTSIZ_XFRSIZ_Pos;
// Send 3 packets (8 bytes max for each)
in_endpoint->DIEPTSIZ |= 3 << USB_OTG_DIEPTSIZ_PKTCNT_Pos;
// Enable transmission
in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_CNAK;
in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_EPENA;
// Write a valid device descriptor to the Tx FIFO
*fifo = 0x02000112;
*fifo = 0x08000000;
*fifo = 0x13aa6666;
*fifo = 0x00000000;
*fifo = 0x0000012c;
fifo_space = in_endpoint->DTXFSTS & USB_OTG_DTXFSTS_INEPTFSAV_Msk;
printf("FIFO free space: %i\n", fifo_space);
}
Below is the log I'm getting from the device. I see that the first device request is transmitted successfully, the FIFO size drops from 16 to 11, than back to 16. Number of packets to send decreases from 3 to 2 (although it should decrease to 0, right?). I also see this descriptor with Wireshark. Then the device address is received and answered by a ZLP packet; then something goes wrong. My FIFO space keeps decreasing with every descriptor request, the packets doesn't seem to leave the device.
USB reset received
FIFO free space: 16
INEPNE Endpoint interrupt
TXFE Endpoint interrupt
// First descriptor request. Sent successfully
Received packet of status 0x06
// Device descriptor request in binary form
Rx from FIFO: 0x01000680
Rx from FIFO: 0x00400000
// Output from handle_descriptor_request()
Received descriptor request
Manual PKTCNT to 3
FIFO free space: 11
// Output from main()
Packets number update: 2
FIFO free space: 16
// Output from rxflvl_handler()
Received packet of status 0x04
Received packet of status 0x02
Received packet of status 0x03
INEPNE Endpoint interrupt
TXFE Endpoint interrupt
// Received address
Received packet of status 0x06
// Set address request in binary form
Rx from FIFO: 0x001d0500
Rx from FIFO: 0x00000000
Received address: 29
Writing zero-length packet
Manual PKTCNT to 1
Packets number update: 1
Received packet of status 0x04
Packets number update: 0
TXFRC Endpoint interrupt
TXFE Endpoint interrupt
// Another device descriptor request
Received packet of status 0x06
Rx from FIFO: 0x01000680
Rx from FIFO: 0x00120000
Received descriptor request
Manual PKTCNT to 3
FIFO free space: 11
Packets number update: 3
Received packet of status 0x04
// Looks unsuccessful. Try again...
Received packet of status 0x06
Rx from FIFO: 0x01000680
Rx from FIFO: 0x00120000
Received descriptor request
Manual PKTCNT to 3
FIFO free space: 6
// And so on. The FIFO free space decreases to 0
Here are the Wireshark views of the first and second descriptor responses. The second one fails with EOVERFLOW (-75). Sometimes I see -71 error when run 'dmesg' on my host. Can't figure out what is the reason for that.
If I will be flushing the 0th FIFO before the descriptor transmission, i.e. uncomment the
flush_txfifo(0);
Thing doesn't change a lot: the FIFO free space stops decreasing below 11, but the packets doesn't seem to be sent.
If I recover the endpoint before the descriptor tranmission, uncommenting the
in_endpoint->DIEPTSIZ = 0;
in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_SNAK;
in_endpoint->DIEPCTL |= USB_OTG_DIEPCTL_EPDIS;
then starting at the second try, the device seems to strart sending something, but still something wrong. The FIFO size increases to 15 words instead of 16
// ...
Rx from FIFO: 0x01000680
Rx from FIFO: 0x00120000
Received descriptor request
Manual PKTCNT to 3
FIFO free space: 11
FIFO free space: 15
Received packet of status 0x04
Received packet of status 0x02
Received packet of status 0x03
Also I can't understand why the number of packets drops from 3 to 2 for the successful transmission. I can see in Wireshark whole descriptor (18 bytes) and the endpoint size is configured to 8.
Could you point me to things I may need to check?
CONTEXT
I'm using a code written to work with a GPS module that connects to the Arduino through serial communication. The module starts each packet with a header (0xb5, 0x62), continues with the information you requested and ends with to bytes of checksum, CK_A, and CK_B. I don't understand the code that calculates that checksum. More info about the algorithm of checksum (8-Bit Fletcher Algorithm) in the module protocol (https://www.u-blox.com/sites/default/files/products/documents/u-blox7-V14_ReceiverDescriptionProtocolSpec_%28GPS.G7-SW-12001%29_Public.pdf), page 74 (87 with index).
MORE INFO
Just wanted to understand the code, it works fine. In the UBX protocol, I mentioned there is also a piece of code that explains how it works (isn't write in c++)
struct NAV_POSLLH {
//Here goes the struct
};
NAV_POSLLH posllh;
void calcChecksum(unsigned char* CK) {
memset(CK, 0, 2);
for (int i = 0; i < (int)sizeof(NAV_POSLLH); i++) {
CK[0] += ((unsigned char*)(&posllh))[i];
CK[1] += CK[0];
}
}
In the link you provide, you can find a link to RFC 1145, containing that Fletcher 8 bit algorithm as well and explaining
It can be shown that at the end of the loop A will contain the 8-bit
1's complement sum of all octets in the datagram, and that B will
contain (n)*D[0] + (n-1)*D[1] + ... + D[n-1].
n = sizeof byte D[];
Quote adjusted to C syntax
Try it with a couple of bytes, pen and paper, and you'll see :)
I have used the ICMP example provided in the ASIO documentation to create a simple ping utility. However, the example covers IPv4 only and I have a hard time to make it work with IPv6.
Upgrading the ICMP header class to support IPv6 requires a minor change - the only difference between ICMP and ICMPv6 header is the different enumeration of ICMP types. However, I have a problem computing the checksum that needs to be incorporated in the ICMPv6 header.
For IPv4 the checksum is based on the ICMP header and payload. However, for IPv6 the checksum should include the IPv6 pseudo-header before the ICMPv6 header and payload. The ICMPv6 checksum function needs to know the source and destination address that will be in the IPv6 header. However, we have no control over what goes into the IPv6 header. How can this be done in Asio-Boost?
For reference please find below the function for IPv4 checksum calculation.
void compute_checksum(icmp_header& header, Iterator body_begin, Iterator body_end)
{
unsigned int sum = (header.type() << 8) + header.code()
+ header.identifier() + header.sequence_number();
Iterator body_iter = body_begin;
while (body_iter != body_end)
{
sum += (static_cast<unsigned char>(*body_iter++) << 8);
if (body_iter != body_end)
sum += static_cast<unsigned char>(*body_iter++);
}
sum = (sum >> 16) + (sum & 0xFFFF);
sum += (sum >> 16);
header.checksum(static_cast<unsigned short>(~sum));
}
[EDIT]
What are the consequences if the checksum is not calculated correctly? Will the target host send echo reply if the echo request has invalid checksum?
If the checksum is incorrect, a typical IPv6 implementation will drop the packet. So, it is a serious issue.
If you insist on crafting the packet yourself, you'll have to do it
completely. This incldues finding the source IP address, to put it in
the pseudo-header before computing the checksum. Here is how I do it
in C, by calling connect() for my intended destination address
(even when I use UDP, so it should work for ICMP):
/* Get the source IP addresse chosen by the system (for verbose display, and
* for checksumming) */
if (connect(sd, destination->ai_addr, destination->ai_addrlen) < 0) {
fprintf(stderr, "Cannot connect the socket: %s\n", strerror(errno));
abort();
}
source = malloc(sizeof(struct addrinfo));
source->ai_addr = malloc(sizeof(struct sockaddr_storage));
source_len = sizeof(struct sockaddr_storage);
if (getsockname(sd, source->ai_addr, &source_len) < 0) {
fprintf(stderr, "Cannot getsockname: %s\n", strerror(errno));
abort();
}
then, later:
sockaddr6 = (struct sockaddr_in6 *) source->ai_addr;
op6.ip.ip6_src = sockaddr6->sin6_addr;
and:
op6.udp.check =
checksum6(op6.ip, op6.udp, (u_int8_t *) & message, messagesize);
I'm trying to experiment with using splice (man 2 splice) to copy data from a UDP socket directly to a file. Unfortunately the first call to splice() returns EINVAL.
The man page states:
EINVAL Target file system doesn't support splicing; target file is opened in
append mode; neither of the descriptors refers to a pipe; or offset
given for nonseekable device.
However, I believe none of those conditions apply. I'm using Fedora 15 (kernel 2.6.40-4) so I believe splice() is supported on all filesystems. The target file should be irrelevant in the first call to splice, but for completeness I'm opening it via open(path, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR). Both calls use a pipe and neither call uses an offset besides NULL.
Here's my sample code:
int sz = splice(sock_fd, 0, mPipeFds[1], 0, 8192, SPLICE_F_MORE);
if (-1 == sz)
{
int err = errno;
LOG4CXX_ERROR(spLogger, "splice from: " << strerror(err));
return 0;
}
sz = splice(mPipeFds[0], 0, file_fd, 0, sz, SPLICE_F_MORE);
if (-1 == sz)
{
int err = errno;
LOG4CXX_ERROR(spLogger, "splice to: " << strerror(err));
}
return 0;
sock_fd is initialized by the following psuedocode:
int sock_fd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
setsockopt(sock_fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one));
fcntl(sock_fd, F_SETFL, flags | O_NONBLOCK);
bind(sock_fd, ...);
Possibly related is that this code snippet is running inside a libevent loop. libevent is using epoll() to determine if the UDP socket is hot.
Found my answer. tl;dr - UDP isn't supported on the inbound side.
After enough Googling I stumbled upon a forum discussion and some test code which prints out a table of in/out fd types and their support:
$ ./a.out
in\out pipe reg chr unix tcp udp
pipe yes yes yes yes yes yes
reg yes no no no no no
chr yes no no no no no
unix no no no no no no
tcp yes no no no no no
udp no no no no no no
Yeah, it is definitely not supported for reading from a UDP socket, even in the latest kernels. References to the kernel source follow.
splice invokes do_splice in the kernel, which calls do_splice_to, which calls the splice_read member in the file_operations structure for the file.
For sockets, that structure is defined as socket_file_ops in net/socket.c, which initializes the splice_read field to sock_splice_read.
That function, in turn, contains this line of code:
if (unlikely(!sock->ops->splice_read))
return -EINVAL;
The ops field of the socket is a struct proto_ops. For an IPv4 UDP socket, it is initialized to inet_dgram_ops in net/ipv4/af_inet.c. Finally, that structure does not explicitly initialize the splice_read field of struct proto_ops; i.e., it initializes it to zero.
So sock_splice_read returns -EINVAL, and that propagates up.