Winsock2, BitCoin Select() returns data to read, Recv() returns 0 bytes - bitcoin

I made a connection to BitCoin node via WinSock2. I sent the proper "getaddr" message and then the server responds, the replied data are ready to read, because Select() notifies this, but when I call Recv() there are 0 bytes read.
My code is working OK on localhost test server. The incomplete "getaddr" message (less than 24 bytes) is NOT replied by BitCoin node, only proper message, but I can't read the reply with Recv(). After returning 0 bytes, the Select() still returns there are data to read.
My code is divided into DLL which uses Winsock2 and the main() function.
Here are key fragments:
struct CMessageHeader
{
uint32_t magic;
char command[12];
uint32_t payload;
uint32_t checksum;
};
CSocket *sock = new CSocket();
int actual; /* Actually read/written bytes */
sock->connect("109.173.41.43", 8333);
CMessageHeader msg = { 0xf9beb4d9, "getaddr\0\0\0\0", 0, 0x5df6e0e2 }, rcv = { 0 };
actual = sock->send((const char *)&msg, sizeof(msg));
actual = sock->select(2, 0); /* Select read with 2 seconds waiting time */
actual = sock->receive((char *)&rcv, sizeof(rcv));
The key fragment of DLL code:
int CSocket::receive(char *buf, int len)
{
int actual;
if ((actual = ::recv(sock, buf, len, 0)) == SOCKET_ERROR) {
std::ostringstream s;
s << "Nie mozna odebrac " << len << " bajtow.";
throw(CError(s));
}
return(actual);
}

If select() reports the socket is readable, and then recv() returns 0 afterwards, that means the peer gracefully closed the connection on their end (ie, sent a FIN packet to you), so you need to close your socket.
On a side note, recv() can return fewer bytes than requested, so your receive() function should call recv() in a loop until all of the expected bytes have actually been received, or an error occurs (same with send(), too).

Related

STM32 USB Tx Busy

I have an application running on STM32F429ZIT6 using USB stack to communicate with PC client.
MCU receives one type of message of 686 bytes every second and receives another type of message of 14 bytes afterwards with 0.5 seconds of delay between messages. The 14 bytes message is a heartbeat so it needs to replied by MCU.
It happens that after 5 to 10 minutes of continuous operation, MCU is not able to send data because
hcdc->TxState is always busy. Reception works fine.
During Rx interruption, application only adds data to ring buffer, so that this buffer is later serialized and processed by main function.
static int8_t CDC_Receive_HS(uint8_t* Buf, uint32_t *Len) {
/* USER CODE BEGIN 11 */
/* Message RX Completed, Send it to Ring Buffer to be processed at FMC_Run()*/
for(uint16_t i = 0; i < *Len; i++){
ring_push(RMP_RXRingBuffer, (uint8_t *) &Buf[i]);
}
USBD_CDC_SetRxBuffer(&hUsbDeviceHS, &Buf[0]);
USBD_CDC_ReceivePacket(&hUsbDeviceHS);
return (USBD_OK);
/* USER CODE END 11 */ }
USB TX is also kept as simple as possible:
uint8_t CDC_Transmit_HS(uint8_t\* Buf, uint16_t Len) {
uint8_t result = USBD_OK;
/\* USER CODE BEGIN 12 */
USBD_CDC_HandleTypeDef hcdc = (USBD_CDC_HandleTypeDef*)hUsbDeviceHS.pClassData;
if (hcdc-\>TxState != 0)
{
ZF_LOGE("Tx failed, resource busy\\n\\r"); return USBD_BUSY;
}
USBD_CDC_SetTxBuffer(&hUsbDeviceHS, Buf, Len);
result = USBD_CDC_TransmitPacket(&hUsbDeviceHS);
ZF_LOGD("TX Message Result:%d\\n\\r", result);
/ USER CODE END 12 \*/
return result;
}
I'm using latest HAL Drivers and software from CubeIDE (1.27.1).
I have tried expanding heap min size from 0x200 to larger values but result is the same.
Also Line Coding is set according to what recommended values:
case CDC_SET_LINE_CODING:
LineCoding.bitrate = (uint32_t) (pbuf[0] | (pbuf[1] << 8) | (pbuf[2] << 16) | (pbuf[3] << 24));
LineCoding.format = pbuf[4];
LineCoding.paritytype = pbuf[5];
LineCoding.datatype = pbuf[6];
ZF_LOGD("Line Coding Set\n\r");
break;
case CDC_GET_LINE_CODING:
pbuf[0] = (uint8_t) (LineCoding.bitrate);
pbuf[1] = (uint8_t) (LineCoding.bitrate >> 8);
pbuf[2] = (uint8_t) (LineCoding.bitrate >> 16);
pbuf[3] = (uint8_t) (LineCoding.bitrate >> 24);
pbuf[4] = LineCoding.format;
pbuf[5] = LineCoding.paritytype;
pbuf[6] = LineCoding.datatype;
ZF_LOGD("Line Coding Get\n\r");
break;
Thanks in advance, any support is appreciated.
I don't know enough about the STM32 libraries to really check your code, but I suspect you are forgetting to read the bytes transmitted by the STM32 on PC side. Try opening a terminal program like PuTTY and connecting to the STM32's virtual serial port. Otherwise, the Windows USB-to-serial driver (usbser.sys) will eventually have its buffers filled with data from your device and it will stop requesting more, at which point the buffers on your device will fill up as well.

Bulk transfer sending too much (multiple of usb packet?)

Problem I am trying to solve
I am sending data over usb with libusb_bulk_transfer, with something like this:
int sent = 0;
int bulk_result = libusb_bulk_transfer(handle, endpoint_out->bEndpointAddress, buffer,
buffer_len, &sent, 5000);
and I receive those transfers on the other side in Kotlin (Android).
Most of the time, it works: I send a buffer of side, say, 289 bytes, and on the other side I receive 289 bytes.
Sometimes, however, I receive too much. Say I send 1536 bytes, and I receive 1719 bytes.
My solution that does not work
My understanding (e.g. from here) is that "A bulk transfer is considered complete when it has transferred the exact amount of data requested, transferred a packet less than the maximum endpoint size, or transferred a zero-length packet".
And because 1536 is a multiple of 64 (and all the wrong packets I receive are multiples of 64), I thought that this was my issue. So I went for sending a zero-length packet after I send a buffer that is a multiple of the maximum endpoint size. And I duly noted that the maximum endpoint size is not necessarily 64, so I wanted to detect it.
Here is my "solution":
int sent = 0;
int bulk_result = libusb_bulk_transfer(handle, endpoint_out->bEndpointAddress, buffer,
buffer_len, &sent, 5000);
if (sent % get_usb_packet_size() == 0) {
libusb_bulk_transfer(handle, endpoint_out->bEndpointAddress, nullptr, 0, &sent, 5000);
}
With the simple get_usb_packet_size() below, which happens to be 256:
int get_usb_packet_size() { return endpoint_out->wMaxPacketSize; }
Still, that does not seem to work! The return code of both libusb_bulk_transfer is 0 (success), the first one says it sent buffer_len bytes (as expected), and the second one says it sent 0 bytes (as expected).
But my receiver still receives packets that are longer than what is expected. I tried using 64 instead of 256 (therefore sending more zero-length packets), but I still get that same problem.
What am I missing?
The issue was due to concurrency: two threads were calling my code above, and therefore sometimes one thread would not have time to send the zero-length packet right after its packet.
So this actually seems to work:
int sent = 0;
int bulk_result = libusb_bulk_transfer(handle, endpoint_out->bEndpointAddress, buffer,
buffer_len, &sent, 5000);
if (sent % get_usb_packet_size() == 0) {
libusb_bulk_transfer(handle, endpoint_out->bEndpointAddress, nullptr, 0, &sent, 5000);
}
with
int get_usb_packet_size() { return endpoint_out->wMaxPacketSize; }

How to receive data over Ethernet using LWIP, UDP

I'm trying to send data to and from my computer and an STM32H745 over Ethernet using LwIP and UDP. I have successfully configured the card and right now I can send data from the card to a Python script running on the computer. However, I don't understand how udp_recv works <udp,lwip> or how to receive data with UDP on LwIP in general, and I can't find examples that do just that. Where is the data being received? Should I even use udp_recv?
In the main loop I run MX_LWIP_Process, which runs ethernetif_input which somehow handles the received data, but I don't understand where it puts it.
Below is the main code, just for reference.
const char* message = "a";
HAL_GPIO_TogglePin(GPIOE, GPIO_PIN_1); // orange
ip_addr_t PC_IPADDR;
IP_ADDR4(&PC_IPADDR, 192, 168, 1, 200);
u16_t port = 8000;
struct udp_pcb* my_udp = udp_new();
struct pbuf* udp_buffer = NULL;
/* Infinite loop */
for (;; )
{
MX_LWIP_Process();
HAL_GPIO_TogglePin(GPIOE, GPIO_PIN_1); // orange
HAL_Delay(1000);
udp_buffer = pbuf_alloc(PBUF_TRANSPORT, strlen(message), PBUF_RAM);
if (udp_buffer != NULL)
{
memcpy(udp_buffer->payload, message, strlen(message));
udp_sendto(my_udp, udp_buffer,&PC_IPADDR, port);
pbuf_free(udp_buffer);
}
//udp_recv (struct udp_pcb *pcb, udp_recv_fn recv, void *recv_arg)
}
udp_recv() does not actually receive UDP datagrams (despite its name). It registers a callback function that will then be called by MX_LWIP_Process() when a datagram has been buffered. It would better be called udp_set_recv_callback(), but it is what it is.
To that end you should call it once before your executive loop:
udp_bind( my_udp, IP_ADDR_ANY, port ) ;
udp_recv( my_udp, udp_receive_callback, NULL ) ;
/* Infinite loop */
for (;; )
{
// Run the CubeMX LwIP stack
MX_LWIP_Process() ;
...
}
Where udp_receive_callback is a function that will be invoked on receipt of a datagram:
void udp_receive_callback( void* arg, // User argument - udp_recv `arg` parameter
struct udp_pcb* upcb, // Receiving Protocol Control Block
struct pbuf* p, // Pointer to Datagram
const ip_addr_t* addr, // Address of sender
u16_t port ) // Sender port
{
// Process datagram here (non-blocking code)
...
// Must free receive pbuf before return
pbuf_free(p);
}
Examples include:
https://gist.github.com/iwanbk/1399729
https://github.com/STMicroelectronics/STM32CubeF2/blob/master/Projects/STM322xG_EVAL/Applications/LwIP/LwIP_UDP_Echo_Client/Src/udp_echoclient.c
Documentation can be found at https://www.nongnu.org/lwip/2_0_x/group__udp__raw.html

Memory use increases when sending messages using ActiveMQ-cpp

When using ActiveMQ-cpp all of the ActiveMQ clients that are created and send messages using cms::MessageProducer gradually increase the memory usage. Right now that looks to be about 4Kb per message send. There does not appear to be any memory leaks with valgrind and the memory increase will continue until the program is terminated or uses all available system memory.
The memory increase happens when the messages are sent and not received by any other ActiveMQ client and when messages are just sent by the producer with no other consumer. It also appears that the act of creating a producer can lead to the memory increase. Here is example code of a call to Publish that leads to the memory increase. I have also tried just using a member session_ variable that is used to create destinations and producers instead of creating a new session every time.
void ActiveMqClient::Publish(std::string type,
void* input, size_t len) {
if(type == "") {
ead::eadwarn() << "ActiveMqClient::Publish() - Attempting to publish to "
"empty string topic. Please check your message topic." << std::endl;
}
cms::Session* session = connection_->createSession(
cms::Session::AUTO_ACKNOWLEDGE);
//creates a destination and producer
cms::Destination* destination(session->createTopic(type));
cms::MessageProducer* producer(session->createProducer(destination));
producer->setDeliveryMode(cms::DeliveryMode::PERSISTENT);
//creates message and sets properties
std::unique_ptr<cms::BytesMessage> message(session->createBytesMessage());
//gets byte array from input
size_t size_to_write = 0;
unsigned char* body = (unsigned char*) input;
if(io_handler_ != nullptr) {
body =
io_handler_->ConvertBodyForPublish(type, input, &len, &size_to_write);
}
//writes the bytes of input
message->writeBytes(const_cast<const unsigned char*>(body), 0,
size_to_write);
//gets byte array from input
unsigned char* payload = (unsigned char*) input;
if(io_handler_ != nullptr) {
payload = io_handler_->ConvertPayloadForPublish(type,
input,
len,
&size_to_write);
}
//writes the bytes of input
if (size_to_write != 0) {
message->writeBytes(payload, 0, size_to_write);
}
//sets the message type of the message
message->setStringProperty("MsgType", type);
//sets the message size
message->setIntProperty("size", len);
//sets the byte pointer to the beginning of the byte array
message->reset();
producer->send(message.get());
//calls sentcallback if it exists
if(io_handler_ != nullptr) {
io_handler_->HandleMessageSent(type, reinterpret_cast<char*>(body), len);
}
//clears memory
delete producer;
delete destination;
delete session;
}
So any ideas on why the memory would steadily keep increasing when utilizing the MessageProducer in this way. No matter how I use this pattern it seems to keep increasing the memory use. Thanks in advance for any help with this!

STM32F4: SD-Card using FatFs and USB fails

(also asked on SE: Electrical Engineering)
In my application, I've set up a STM32F4, SD-Card and USB-CDC (all with CubeMX).
Using a PC, I send commands to the STM32, which then does things on the SD-Card.
The commands are handled using a "communicationBuffer" (implemented by me) which waits for commands over USB, UART, ... and sets a flag, when a \n character was received. The main loop polls for this flag and if it is set, a parser handles the command. So far, so good.
When I send commands via UART, it works fine, and I can get a list of the files on the SD-Card or perform other access via FatFs without a problem.
The problem occurs, when I receive a command via USB-CDC. The parser works as expected, but FatFs claims FR_NO_FILESYSTEM (13) in f_opendir.
Also other FatFs commands fail with this error-code.
After one failed USB-command, commands via UART will also fail. It seems, as if the USB somehow crashes the initialized SD-Card-driver.
Any idea how I can resolve this behaviour? Or a starting point for debugging?
My USB-Implementation:
I'm using CubeMX, and therefore use the prescribed way to initialize the USB-CDC interface:
main() calls MX_USB_DEVICE_Init(void).
In usbd_conf.c I've got:
void HAL_PCD_MspInit(PCD_HandleTypeDef* pcdHandle)
{
GPIO_InitTypeDef GPIO_InitStruct;
if(pcdHandle->Instance==USB_OTG_FS)
{
/* USER CODE BEGIN USB_OTG_FS_MspInit 0 */
/* USER CODE END USB_OTG_FS_MspInit 0 */
/**USB_OTG_FS GPIO Configuration
PA11 ------> USB_OTG_FS_DM
PA12 ------> USB_OTG_FS_DP
*/
GPIO_InitStruct.Pin = OTG_FS_DM_Pin|OTG_FS_DP_Pin;
GPIO_InitStruct.Mode = GPIO_MODE_AF_PP;
GPIO_InitStruct.Pull = GPIO_NOPULL;
GPIO_InitStruct.Speed = GPIO_SPEED_FREQ_LOW;
GPIO_InitStruct.Alternate = GPIO_AF10_OTG_FS;
HAL_GPIO_Init(GPIOA, &GPIO_InitStruct);
/* Peripheral clock enable */
__HAL_RCC_USB_OTG_FS_CLK_ENABLE();
/* Peripheral interrupt init */
HAL_NVIC_SetPriority(OTG_FS_IRQn, 7, 1);
HAL_NVIC_EnableIRQ(OTG_FS_IRQn);
/* USER CODE BEGIN USB_OTG_FS_MspInit 1 */
/* USER CODE END USB_OTG_FS_MspInit 1 */
}
}
and the receive-process is implemented in usbd_cdc_if.c as follows:
static int8_t CDC_Receive_FS (uint8_t* Buf, uint32_t *Len)
{
/* USER CODE BEGIN 6 */
mRootObject->mUsbBuffer->fillBuffer(Buf, *Len);
USBD_CDC_ReceivePacket(&hUsbDeviceFS);
return (USBD_OK);
/* USER CODE END 6 */
}
fillBuffer is implemented as follows (I use the same implementation for UART and USB transfer - with separate instances for the respective interfaces. mBuf is an instance-variable of type std::vector<char>):
void commBuf::fillBuffer(uint8_t *buf, size_t len)
{
// Check if last fill has timed out
if(SystemTime::getMS() - lastActionTime > timeout) {
mBuf.clear();
}
lastActionTime = SystemTime::getMS();
// Fill new content
mBuf.insert(mBuf.end(), buf, buf + len);
uint32_t done = 0;
while(!done) {
for(auto i = mBuf.end() - len, ee = mBuf.end(); i != ee; ++i) {
if(*i == '\n') {
newCommand = true;
myCommand = std::string((char*) &mBuf[0],i - mBuf.begin() + 1);
mBuf.erase(mBuf.begin(), mBuf.begin() + (i - mBuf.begin() + 1));
break;
}
}
done = 1;
}
}
I resolved the problem:
In usb_cdc_if.c the #define APP_RX_DATA_SIZE was set to 4 (for some unknown reason). As this is lower than the packet size, incoming packets of a larger size than 4 bytes were overwriting my memory.
It happened, that the following portion of my memory was the FATFS* FatFs[] pointer-list to the initialized FATFS-Filesystem structs.
So subsequently the address to this struct was overwritten, when a command of 5 or more bytes arrived.
Phew, that was a tough one.