I'm using Apache Mina framework for communication between server and client, when I test the programe, I found if my transmission frequency is too fast, the received packets will merge to a big one, which should be each separate package.
My client used to send message, code like below:
public static void main(String[] args) {
// IoConnector instance
IoConnector connector = new NioSocketConnector();
connector.setConnectTimeoutMillis(3000); // 连接超时时间
// configure filters
// connector.getFilterChain().addLast("coderc", new ProtocolCodecFilter(
// new TextLineCodecFactory(Charset.forName("UTF-8"))));
DefaultIoFilterChainBuilder builder = (DefaultIoFilterChainBuilder) connector.getFilterChainBuilder();
Map<String, IoFilter> filters = new LinkedHashMap<>();
filters.put("protocolCodecFilter", new ProtocolCodecFilter(new TcpCodecFactory()));
filters.put("executor", new ExecutorFilter());
filters.put("loggingFilter", new LoggingFilter());
builder.setFilters(filters);
connector.setFilterChainBuilder(builder);
// set handler
connector.setHandler(new TcpServerHandler());
// connect to the server
ConnectFuture future = connector.connect(new InetSocketAddress(address, port));
future.awaitUninterruptibly(); // waiting for connection
IoSession session = future.getSession();
byte[] data = new byte[10];
for (int i = 0; i < 100; ++i) {
data[0] = (byte)i; // first byte means the message sequence number
for (int j = 1; j < 10; ++j) {
data[j] = (byte) (j + 10);
}
// try {
// Thread.sleep(100);
// } catch (InterruptedException e) {
// e.printStackTrace();
// }
session.write(data);
}
session.getCloseFuture().awaitUninterruptibly(); // wait for close
connector.dispose();
}
When I comment the sleep code, which means I send message frequently, My received message will like below:
The message sequence number is 0, 1, 6..., If I uncomment the sleep code, the message sequence number is 0, 1, 2, 3...
What's the difference? How to avoid the mixed packets in fast transmission?
You are using a TCP socket which is a stream channel versus UDP which is a message channel. Your messages are being combined because that is how TCP works on all platforms to effectively create as few network packets as possible.
Unless the messages are out of order, then this is the expected behavior.
Related
I am reading samples from a software defined radio device that delivers two channels of Int16 (shorts) that represent audio. I am trying to send this stream to an audio output device on a PC.
I already have this working with a different set of tools but would like to use NAudio in its place since there are other capabilities I could use.
The code starts with the software defined radio method placing an array of shorts (i.e. int16) arranged as left channel then right channel. The array size is 65536, the sample rate is 192,000.
When the packet is received it is place on a Blocking collection to be picked up by a thread that sends out the audio.
So for the purpose of this question, the thread starts by reading the blocking collection which returns 65536 shorts.
StartStream = true;
waveformat = new WaveFormat(192000, 16, 2);
bs = new BufferedWaveProvider(waveformat);
bs.BufferLength = 65536 * 4;
_waveOut.DeviceNumber = 4;
while (true)
{
dspPacket = dsp_Queue.Take(); //take int16[] of queue
int j = 0;
try
{
for (int i = 0; i < 32768; i += 2)
{
byte[] qDataByte = BitConverter.GetBytes(dspShort[i]);
dspBytes[j] = qDataByte[0];
dspBytes[j + 1] = qDataByte[1];
j += 2;
}
if (startStream)
{
bs.AddSamples(dspBytes, 0, 65536);
_waveOut.Init(bs);
_waveOut.Play();
startStream = false;
}
else
{
bs.AddSamples(dspBytes, 0, 65536);
}
}
catch (Exception ex)
{
}
// SendData(dspPacket); //if this is uncommented, everything works correctly with the old method.
}
Now I can feed the output of the waveout device to a virtual audio cable which then feeds it into some spectrum analyser software and then this is what you will see with here:
When I use the old routines, I get a solid line and the sound does not break up.
So, am I doing something wrong here?
Thanks, Tom
I'm working on an esp32 FreeRTOS application with two tasks. Its purpose is to take UART messages received from a peripheral device and transmit them via mqtt to a central broker.
The first task reads input from Serial1, processes the contents into a message structure, and adds it to a FreeRTOS queue:
typedef struct {
int length;
char buffer[AZ_EL_MAX_MESSAGE_LENGTH];
} tag_message_t;
void uart_read_task(void * pvParameters){
BaseType_t xStatus;
tag_message_t tag_message;
while(true) {
while(Serial1.available())
{
first_char = Serial1.read();
if (first_char == '+') // Indicates the beginning of a message
{
for(int i = 0; i < AZ_EL_MAX_MESSAGE_LENGTH; i++)
{
message_buffer[i] = Serial1.read();
if (message_buffer[i] == '\n') // End of message received
{
ESP_LOGV(TAG, "Message found: %s", message_buffer);
strncpy(tag_message.buffer, message_buffer, i + 1);
tag_message.length = i + 1;
xStatus = xQueueSend(xMessagesToSendQueue, (void*) &tag_message, 0);
if (xStatus != pdPASS)
ESP_LOGW(TAG, "Failed to queue message.");
break;
}
}
}
}
vTaskDelay(pdMS_TO_TICKS(20)); // Wait the minimum BLE advertisement period for messages to come in, i.e. 20ms
}
}
The main loop() (which is technically the second FreeRTOS task) then attempts to receive from that queue and transmit over MQTT to a local broker:
void setup()
{
Serial.begin(115200);
// Configure and start WiFi
configure_network();
connect_network();
// Configure the MQTT connection
configure_mqtt_client();
// Configure and create the inter-task queues
xMessagesToSendQueue = xQueueCreate(100, sizeof(tag_message_t));
if (xMessagesToSendQueue == NULL) {
ESP_LOGE(TAG, "Unable to create messaging queue. Will not create UART handling message queue.");
delay(10000);
esp_restart();
} else {
ESP_LOGI(TAG, "Messaging queue generated");
configure_uart();
xTaskCreate(uart_read_task, "UART_Processing", 20000, NULL, 1, NULL);
}
}
void loop()
{
const TickType_t xTicksToWait = pdMS_TO_TICKS(100); // milliseconds to wait
tag_message_t received_message;
if (network_connected) {
connect_mqtt_client();
while(mqtt_client.connected())
{
mqtt_client.loop();
// Process messages on the xMessagesToSendQueue
if (xMessagesToSendQueue != NULL)
{
ESP_LOGI(TAG, "Processing message");
if (xQueueReceive(xMessagesToSendQueue, &received_message, xTicksToWait) == pdPASS)
{
ESP_LOGD(TAG, "Received message, transmitting.");
if(!mqtt_client.publish("aoa", received_message.buffer, received_message.length));
ESP_LOGW(TAG, "Failed transmission.");
}
else
{
vTaskDelay(pdMS_TO_TICKS(50));
}
}
else
{
ESP_LOGE(TAG, "Messages queue is null.");
}
}
} else {
ESP_LOGE(TAG, "WARNING Device not connected to the network. Reconnecting.");
connect_network();
}
delay(5000);
}
I've verified that the MQTT broker works, that it connects to WiFi, and it can properly read messages from Serial1. HOWEVER, the xQueueReceive() call in loop() throws a LoadProhibited exception every time it's called.
Can anyone tell me what I'm getting wrong here?
All, thank you for your help. It turns out this wasn't a FreeRTOS issue. After a little research (i.e. reading this and watching a more experienced engineer explain things: [link]https://hackaday.com/2017/08/17/secret-serial-port-for-arduinoesp32/) it turns out ESP32 Serial1 pins are connected to flash memory.
Every time I tried Serial1.read() or Serial1.readBytesUntil() the ESP32 crashed. Turns out reading the flash is taboo?
I replaced '''Serial1.read()''' with '''Serial2.read()''' and others. That fixed everything. Now I'm off to optimizing my queues!
You are trying to receive an address on the Queue without casting it.
2 solutions: You either declare received_message as a Pointer:
tag_message_t* received_message;
...
received_message = new tag_message_t;
if (xQueueReceive(xMessagesToSendQueue, received_message, xTicksToWait) == pdPASS)
and don't forget to delete it right after usage.
Or you can cast it after receiving :
if (xQueueReceive(xMessagesToSendQueue, &received_message, xTicksToWait) == pdPASS)
{
received_message = *(static_cast<tag_message_t*>(&received_message));
ESP_LOGD(TAG, "Received message, transmitting.");
if(!mqtt_client.publish("aoa", received_message.buffer, received_message.length));
ESP_LOGW(TAG, "Failed transmission.");
}
or any other sort of dereferencing you might wanna try.
There is also the possibility xQueueReceive is already doing that! So let's be sure of what is going on by adding this:
ESP_LOGD(TAG, "Received message, transmitting. %s", received_message.buffer);
Right after you get the message.
I don't think tag_message is being deleted inside the task, so if the struct is still valid and present, if you properly parse/cast its address, you should be able to acquire the message without any issues.
I'm trying to send and receive UDP packets through the same endpoint. As far as I know this should be possible. But I can not get it to work with the asio library (version 1.20.0).
This is what I do:
asio::io_context io_context;
asio::ip::udp::socket* udpSendRecvSocket = new asio::ip::udp::socket(io_context, asio::ip::udp::endpoint(asio::ip::udp::v4(), 7782));
asio::error_code ec;
char data[1000];
//
// send packet
//
std::string ipAddress = "127.0.0.1";
asio::ip::address ip_address = asio::ip::address::from_string(ipAddress);
asio::ip::udp::endpoint remoteTarget_endpoint(ip_address, 5500);
udpSendRecvSocket->send_to(asio::buffer(data, 50), remoteTarget_endpoint, 0, ec);
if (ec) {
return 0;
}
//
// receive packets
//
size_t avLen = udpSendRecvSocket->available(ec);
while (avLen) {
asio::ip::udp::endpoint remote_endpoint;
size_t length = udpSendRecvSocket->receive_from(asio::buffer(data, 1000), remote_endpoint, 0, ec);
int p = remote_endpoint.port();
if (ec) {
return 0;
}
avLen -= length;
}
The receive does not work correctly. I do receive a packet that I send (from some other app). I know because avLen gets the right value. But when executing the receive_from(), if fails. And the port number in p gets the value 5500. This is the value of the target port of the send_to() call that was executed before.
The strange thing is that when I remove the send_to() call, the receive does work correctly and the p will reflect the correct port number of the sending application.
Is this a bug?
I have issue to read serial data from Arduino Uno in UWP C#. Sometimes when I start app I get corrupted data.
But in arduino monitor always is good.
Corrupted data is like this:
57
5
But should be 557.
Arduino codes:
String digx, digx2, digx3, digx1 ;
void setup() {
delay(500);
Serial.begin(9600);
}
void loop() {
int sensorValue = analogRead(A1);
sensorValue = map(sensorValue, 0, 1024, 0, 999);
digx1 = String(sensorValue % 10);
digx2 = String((sensorValue / 10) % 10);
digx3 = String((sensorValue / 100) % 10);
digx = (digx3 + digx2 + digx1);
Serial.println(digx);
Serial.flush();
delay(100);
}
And windows universal codes:
public sealed partial class MainPage : Page
{
public DataReader dataReader;
public SerialDevice serialPort;
private string data;
private async void InitializeConnection()
{
var aqs = SerialDevice.GetDeviceSelectorFromUsbVidPid(0x2341, 0x0043);
var info = await DeviceInformation.FindAllAsync(aqs);
serialPort = await SerialDevice.FromIdAsync(info[0].Id);
Thread.Sleep(100);
serialPort.DataBits = 8;
serialPort.BaudRate = 9600;
serialPort.Parity = SerialParity.None;
serialPort.Handshake = SerialHandshake.None;
serialPort.StopBits = SerialStopBitCount.One;
serialPort.ReadTimeout = TimeSpan.FromMilliseconds(1000);
serialPort.WriteTimeout = TimeSpan.FromMilliseconds(1000);
Thread.Sleep(100);
dataReader = new DataReader(serialPort.InputStream);
while (true)
{
await ReadAsync();
}
}
private async Task ReadAsync()
{
try
{
uint readBufferLength = 5;
dataReader.InputStreamOptions = InputStreamOptions.Partial;
//dataReader.UnicodeEncoding = UnicodeEncoding.Utf8;
var loadAsyncTask = dataReader.LoadAsync(readBufferLength).AsTask();
var ReadAsyncBytes = await loadAsyncTask;
if (ReadAsyncBytes > 0)
{
data = dataReader.ReadString(ReadAsyncBytes);
serialValue.Text = data;
}
}
catch (Exception e)
{
}
}
}
My idea was if it happened, Program skip that data. I tried different buffer length but more or less than 5 will always receive incorrect.
Thanks for any help.
Two methods that can fix your problem.
Method1: Make your UART communication robust
That there is no chance of an error but this is a tricky and time-consuming task in the case of windows software communicating with Arduino.
But still have some tricks which can help you because I have tried them and some of them work but please don't ask for the reason why it works.
Change your UART baud rate from 9600 to something else (mine if works perfect at 38400).
Replace Serial.begin(9600); to Serial.begin(38400,SERIAL_8N1)where 8 is data bit, 1 is stop bit and N is parity none in SERIAL_8N1. And you can also try different settings in data bit, stop bit, and parity. check here
don't send data after converting it into a string Serial.println(digx);. Send data in form of bytes array use Serial.write(digx,len);. check here
Method2: Add CRC Packet
At the end of each data packet and then send. When it reaches the windows software first fetch all UART data then calculate CRC from that data and then just compare calculated CRC and received CRC.If CRC matches then the data is correct and if not then ignore data and wait for new data.
you can check for CRC calculation its direct formula base implement and for little head start check this library by vinmenn.
I used two things to fix the problem,
1. In Windows codes added SerialDevice.IsDataTerminalReadyEnabled = true property to read bytes with correct order. From Microsoft docs
Here's a link!
2. As dharmik helped, I used array and send it with Serial.write(array, arrayLength) in Arduino and read them with dataReader.ReadBytes(array[]) and right buffer size in Windows app.
And result is so stable even with much lower Delays.
I am using STM32F1 (STM32F103C8T6) in order to develop a project using FreeRTOS.
The following is my GPIO and USART1 interface configuration:
__GPIOA_CLK_ENABLE();
__USART1_CLK_ENABLE();
GPIO_InitTypeDef GPIO_InitStruct;
GPIO_InitStruct.Pin = GPIO_PIN_9;
GPIO_InitStruct.Mode = GPIO_MODE_AF_PP;
GPIO_InitStruct.Speed = GPIO_SPEED_HIGH;
HAL_GPIO_Init(GPIOA, &GPIO_InitStruct);
GPIO_InitStruct.Pin = GPIO_PIN_10;
GPIO_InitStruct.Mode = GPIO_MODE_INPUT;
GPIO_InitStruct.Pull = GPIO_NOPULL;
HAL_GPIO_Init(GPIOA, &GPIO_InitStruct);
huart1.Instance = USART1;
huart1.Init.BaudRate = 9600;//115200;
huart1.Init.WordLength = UART_WORDLENGTH_8B;
huart1.Init.StopBits = UART_STOPBITS_1;
huart1.Init.Parity = UART_PARITY_NONE;
huart1.Init.Mode = UART_MODE_TX_RX;
huart1.Init.HwFlowCtl = UART_HWCONTROL_NONE;
HAL_UART_Init(&huart1);
HAL_NVIC_SetPriority(USART1_IRQn, 0, 0);
HAL_NVIC_EnableIRQ(USART1_IRQn);
The question is: Why does UART transmit work before threads start but not after threads started or from threads? I want to transmit data from threads. i.e
int main(void)
{
Initializations();
//THIS WORKS!!
uart_transmit_buffer[0] = 'H';
uart_transmit_buffer[1] = 'R';
uart_transmit_buffer[2] = '#';
uint8_t nums_in_tr_buf = 0;
nums_in_tr_buf = sizeof(uart_transmit_buffer)/sizeof(uint8_t);
state = HAL_UART_Transmit(&huart1, uart_transmit_buffer, nums_in_tr_buf, 5000);
StartAllThreads();
osKernelStart();
for (;;);
}
static void A_Random_Thread(void const *argument)
{
for(;;)
{
if (conditionsMet()) //Executed once when a proper response received.
{
//BUT NOT THIS :(!!
uart_transmit_buffer[0] = 'H';
uart_transmit_buffer[1] = 'R';
uart_transmit_buffer[2] = '#';
uint8_t nums_in_tr_buf = 0;
nums_in_tr_buf = sizeof(uart_transmit_buffer)/sizeof(uint8_t);
state = HAL_UART_Transmit(&huart1, uart_transmit_buffer, nums_in_tr_buf, 5000);
}
}
}
I have made sure no thread is in deadlock. The problem is UART_HAL_Transmit gives HAL_BUSY state.
Furthermore, I have dedicated one thread to receiving and parsing information from UART RX and I suspect this might be a cause of the problem. The following is the code:
static void UART_Receive_Thread(void const *argument)
{
uint32_t count;
(void) argument;
int j = 0, word_length = 0;
for (;;)
{
if (uart_line_ready == 0)
{
HAL_UART_Receive(&huart1, uart_receive_buffer, UART_RX_BUFFER_SIZE, 0xFFFF);
if (uart_receive_buffer[0] != 0)
{
if (uart_receive_buffer[0] != END_OF_WORD_CHAR)
{
uart_line_buffer[k] = uart_receive_buffer[0];
uart_receive_buffer[0] = 0;
k++;
}
else
{
uart_receive_buffer[0] = 0;
uart_line_ready = 1;
word_length = k;
k = 0;
}
}
}
if (uart_line_ready == 1)
{
//osThreadSuspend(OLEDThreadHandle);
for (j = 0; j <= word_length; j++)
{
UART_RECEIVED_COMMAND[j] = uart_line_buffer[j];
}
for (j = 0; j <= word_length; j++)
{
uart_line_buffer[j] = 0;
}
uart_line_ready = 0;
RECEIVED_COMMAND = ParseReceivedCommand(UART_RECEIVED_COMMAND);
if (RECEIVED_COMMAND != _ID_)
{
AssignReceivedData (word_length); //Results in uint8_t * RECEIVED_DATA
}
//osThreadResume(OLEDThreadHandle);
}
//Should be no delay in order not to miss any data..
}
}
Another cause to the problem I suspect could be related to interrupts of the system (Also please notice initialization part, I configured NVIC):
void USART1_IRQHandler(void)
{
HAL_UART_IRQHandler(&huart1);
}
Any help or guidance to this issue would be highly appreciated. Thanks in advance.
From what I can see HAL_UART_Transmit would've worked with the F4 HAL (v1.4.2) if it weren't for __HAL_LOCK(huart). The RX thread would lock the handle and then the TX thread would try to lock as well and return HAL_BUSY. HAL_UART_Transmit_IT and HAL_UART_Receive_IT don't lock the handle for the duration of the transmit/receive.
Which may cause problems with the State member, as it is non-atomically updated by the helper functions UART_Receive_IT and UART_Transmit_IT. Though I don't think it would affect operation.
You could modify the function to allow simultaneous RX and TX. You'd have to update this every time they release a new version of the HAL.
The problem is that the ST HAL isn't meant to be used with an RTOS. It says so in the definition of the macro __HAL_LOCK. Redefining it to use the RTOS's mutexes might worth trying as well. Same with HAL_Delay() to use the RTOS's thread sleep function.
In general though, sending via a blocking function in a thread should be fine, but I would not receive data using a blocking function in a thread. You are bound to experience overrun errors that way.
Likewise, if you put too much processing in the receive interrupt you might also run into overrun errors. I prefer using DMA for reception, followed by interrupts if I've run out of DMA streams. The interrupt only copies the data to a buffer, similarly to the DMA. A processRxData thread is then used to process the actual data.
When using FreeRTOS, you have to set interrupt priority to 5 or above, because below 5 is reserved for the OS.
So change your code to set the priority to:
HAL_NVIC_SetPriority(USART1_IRQn, 5, 0);
The problem turned out to be something to do with blocking statements.
Since UART_Receive_Thread has HAL_UART_Receive inside and that is blocking the thread until something is received, that results in a busy HAL (hence, the HAL_BUSY state).
The solution was using non-blocking statements without changing anything else.
i.e. using HAL_UART_Receive_IT and HAL_UART_Transmit_IT at the same time and ignoring blocking statements worked.
Thanks for all suggestions that lead to this solution.