Best practice for buffering data to be sent on UART [closed] - embedded

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm working on an embedded project using an STM32F7 device, writing bare metal C.
I want to be able to send data to a UART at any point in the program for debugging purposes, without blocking while the data is sent. I'm using DMA to try to minimise the cpu time used for this.
Currently I'm filling the data into a FIFO queue, and then initiating a DMA request to send the data directly from the FIFO queue to the UART.
The issue with this is I can't set up the DMA to read from both the start and end of the FIFO buffer, in the case where the middle of the FIFO is unused and a message wraps from the end of the buffer to the start.
The two solutions to this would be to set up the first DMA request to read from the head of the FIFO to the end of the buffer, and then once that is complete, read from the start of the buffer to the tail of the FIFO.
The other way to do it would be to memcpy() out the bytes to be sent to another buffer, where they are all sequential, then initiate a single DMA request to send all the data at once.
Both of these would probably work but I'm looking for insight on what the best approach would be here.

The implementation I've usually chosen is similar to what you have proposed:
The logging functions creates a text and adds it to circular buffer.
DMA is used for the UART transmission. DMA is setup to send a contiguous chunk of data.
Whenever the DMA finishes, an interrupt is triggered. It first frees up the transmitted data in the circular buffer. Then it checks if more data needs to be transmitted. If so, it is immediately started again with new data.
Pseudo code:
tx_len = 0;
void log_message(const char* msg)
{
circ_buf_add(msg);
start_tx();
}
void start_tx()
{
if (tx_len > 0)
return; // already transmitting
const char* start;
int len;
circ_buf_get_chunk(&start, &tx_len);
if (tx_len == 0)
return;
uart_tx_dma(start, tx_len);
}
void dma_interrupt_handler()
{
circ_buf_remove(tx_len);
tx_len = 0;
start_tx();
}
It usually makes sense to limit the length of the transmitted chunk. The shorter it is, the sooner space is freed up in the circular buffer.

The examples proposed so far are fire and forget. In the case where your code needs to know if the data has been send. We have used the following structure in which the fifo holds structs pointing to the data.
This way your data is held by the code sending it. It is able to monitor the transmission but it is also responsible for not using the data until the transmission is complete.
A different advantage is that you do not have to allocate a buffer in advance. Only two pointers are required to point to the start en end of the linked list structure.
Some meta code:
enum transmission_state {
Unused,
WaitToBeSend,
Sending,
Done,
Error // Optional but handy
}
struct data_to_send
{
// Point to your data.
data* data_pointer;
// Set the length of your data.
int length;
// What is the current state of this transmission.
transmission_state state;
// Pointer to the next data to be send creating a linked list.
// Only have the send and dma functions use this.
data_to_send* next;
};
// Definition of the fifo.
data_to_send* fifo_first = null;
data_to_send* fifo_end = null;
// Use this function in your code to add data to be send.
void send(data_to_send* dts)
{
if(null == fifo_first) {
fifo_first = dts;
fifo_end = dts;
dts.next = null;
start_dma_transfer(fifo_first);
}
else {
fifo_end.next = dts;
fifo_end = dts;
dts.state = WaitToBeSend;
dts.next = null;
}
};
// Start a transfer.
void start_dma_transfer(data_to_send* dts)
{
dts->state = Sending;
// Do some DMA stuff to start the transmission.
dma_transfer(dts->data, dts->length)
}
// The interrupt handler called when the dma is done.
void dma_interrupt_handler()
{
fifo_first->state = Done;
if(null != fifo_first->next) {
// Send the next data.
fifo_first = fifo_first->next;
start_dma_transfer(fifo_first);
}
else {
// No new data to be send.
fifo_first = null;
fifo_end = null;
}
}
int main()
{
// Setup a transmission
byte data[3] = {x,y,z};
data_to_send transmission = default_dts; // Set to some default.
transmission.data = data;
transmission.length = 3;
send(&transmission);
// Do other important things.
// Later periodically check the transmission.
if(Done == transmission.status) {
// You could use the data for something else or send new data.
}
}
This structure can also be used for I2C and SPI in which case you can add responses to the data_to_send struct and check for a response and act upon it.

Related

How to optimize the code for reading SPI through ARDUINO in SLAVE mode

Not important:
I am doing a project to integrate a bluetooth module into a car radio pioneer. I understand perfectly well that it's easier to buy a new one =) but it's not interesting. At the moment, the byproduct was an adapter on arduino of resistor buttons, which the pioneer did not understand. The same adapter also controls the bluetooth board, it can switch the track forward and backward (there is no button on the steering wheel for pause). Now I want the bluetooth to turn on only in AUX mode. But there is a problem, which mode can be understood only by reading the signal from the SPI bus of the commutation microcircuit. I was able to read this data using arduino nano. I do not have an analyzer, but it is not necessary that I would understand something additional with it.
Essence of the question:
Using the scientific poke method, I found sequences indicating the launch of a particular mode, for example:
10110011
1
111
1000000
I'm sure I'm doing it wrong, but in the meantime I get duplicate results. But, when I try to detect them using IF, the nano speed is not enough and the chip starts to pass data.
#include "SPI.h"
bool flag01, flag02, flag03, flag11, flag12, flag13, flag31, flag32, flag33;
void setup (void)
{
Serial.begin(9600);
pinMode(MISO, OUTPUT);
SPCR |= _BV(SPE);
SPI.attachInterrupt();
}
// Вызываем функцию обработки прерываний по вектору SPI
// STC - Serial Transfer Comlete
ISR(SPI_STC_vect)
{
// Получаем байт из регистра данных SPI
byte c = SPDR;
Serial.println(c, BIN);
if (c == 0b1) {
Serial.println("1 ok");
flag11 = true;
} else {
flag11 = false;
}
if (c == 0b11 && flag11) {
Serial.println("11 ok");
flag12 = true;
} else {
flag12 = false;
flag11 = false;
}
if (c == 0b1100000 && flag11 && flag12) {
Serial.println("1100000 ok");
flag13 = true;
} else {
flag13 = false;
flag12 = false;
flag11 = false;
}
}
void loop(void)
{}
I myself am scared to look at this code, but I cannot think of anything better. It seems like I heard about some kind of buffer, but I don't know how to screw it to this solution. After all, the data packets go with dropping the CS signal and I can’t figure out how to determine the beginning and end of the packet from the commands in order to write it to a buffer or array and only then go through it with a comparison.
I will be grateful if someone will tell me at least in which direction to move.
There is also esp8266, but there is a limitation on the size of a data packet of 32 bits in a slave mode and I do not know how to get around it correctly.
So, actually the question.
How can I optimize the code so that the arduino has time to process the data and I can catch the pattern?
Perhaps, if we implement reading of data of arbitrary length on esp8266, or at least fill them to the required length, it would help me. But I still can't figure it out with the spi.slave library.
First you should keep your ISR as short as possible, certainly don't use Serial print inside the ISR.
Secondly, if you don't know exactly how long the data is, then you need to have a buffer to capture the data and try to determine the data length first before you trying to analysis it.
volatile uint8_t byteCount = 0;
volatile bool dataReady = false;
byte data[32];
// SPI interrupt routine
ISR (SPI_STC_vect)
{
data[byteCount++] = SPDR;
dataReady = true;
}
void setup (void)
{
// your SPI and Serial setup code
}
void loop (void)
{
// for determine the data stream length
if (dataReady) {
Serial.println(byteCount);
dataReady = false;
}
}
Once you know how long the data stream is (let assumed it is 15-byte long), you can then change your sketch slightly to capture the data and analysis it.
volatile uint8_t byteCount = 0;
volatile bool dataReady = false;
byte data[32];
// SPI interrupt routine
ISR (SPI_STC_vect)
{
data[byteCount++] = SPDR;
if (byteCount == 15)
dataReady = true;
}
void loop (void)
{
if (dataReady) {
dataReady = false;
// do your analysis here
}
}

fatfs f_write returns FR_DISK_ERR when passing a pointer to data in a mail queue

I'm trying to use FreeRTOS to write ADC data to SD card on the STM32F7 and I'm using V1 of the CMSIS-RTOS API. I'm using mail queues and I have a struct that holds an array.
typedef struct
{
uint16_t data[2048];
} ADC_DATA;
on the ADC half/Full complete interrupts, I add the data to the queue and I have a consumer task that writes this data to the sd card. My issue is in my Consumer Task, I have to do a memcpy to another array and then write the contents of that array to the sd card.
void vConsumer(void const * argument)
{
ADC_DATA *rx_data;
for(;;)
{
writeEvent = osMailGet(adcDataMailId, osWaitForever);
if(writeEvent.status == osEventMail)
{
// write Data to SD
rx_data = writeEvent.value.p;
memcpy(sd_buff, rx_data->data, sizeof(sd_buff));
if(wav_write_result == FR_OK)
{
if( f_write(&wavFile, (uint8_t *)sd_buff, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
{
file_size+=bytes_written;
}
}
osMailFree(adcDataMailId, rx_data);
}
}
This works as intended but if I try to change this line to
f_write(&wavFile, (uint8_t *)rx_data->data, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
so as to get rid of the memcpy, f_write returns FR_DISK_ERR. Can anyone help shine a light on why this happens, I feel like the extra memcpy is useless and you should just be able to pass the pointer to the queue straight to f_write.
So just a few thoughts here:
memcpy
Usually I copy only the necessary amount of data. If I have the size of the actual data I'll add a boundary check and pass it to memcpy.
Your problem
I am just guessing here, but if you check the struct definition, the data field has the type uint16_t and you cast it to a byte pointer. Also the FatFs documentation expects a void* for the type of buf.
EDIT: Could you post more details of sd_buff

Write UART on PIC18

I need help with the uart communication I am trying to implement on my Proteus simulation. I use a PIC18f4520 and I want to display on the virtual terminal the values that have been calculated by the microcontroller.
Here a snap of my design on Proteus
Right now, this is how my UART code looks like :
#define _XTAL_FREQ 20000000
#define _BAUDRATE 9600
void Configuration_ISR(void) {
IPR1bits.TMR1IP = 1; // TMR1 Overflow Interrupt Priority - High
PIE1bits.TMR1IE = 1; // TMR1 Overflow Interrupt Enable
PIR1bits.TMR1IF = 0; // TMR1 Overflow Interrupt Flag
// 0 = TMR1 register did not overflow
// 1 = TMR1 register overflowed (must be cleared in software)
RCONbits.IPEN = 1; // Interrupt Priority High level
INTCONbits.PEIE = 1; // Enables all low-priority peripheral interrupts
//INTCONbits.GIE = 1; // Enables all high-priority interrupts
}
void Configuration_UART(void) {
TRISCbits.TRISC6 = 0;
TRISCbits.TRISC7 = 1;
SPBRG = ((_XTAL_FREQ/16)/_BAUDRATE)-1;
//RCSTA REG
RCSTAbits.SPEN = 1; // enable serial port pins
RCSTAbits.RX9 = 0;
//TXSTA REG
TXSTAbits.BRGH = 1; // fast baudrate
TXSTAbits.SYNC = 0; // asynchronous
TXSTAbits.TX9 = 0; // 8-bit transmission
TXSTAbits.TXEN = 1; // enble transmitter
}
void WriteByte_UART(unsigned char ch) {
while(!PIR1bits.TXIF); // Wait for TXIF flag Set which indicates
// TXREG register is empty
TXREG = ch; // Transmitt data to UART
}
void WriteString_UART(char *data) {
while(*data){
WriteByte_UART(*data++);
}
}
unsigned char ReceiveByte_UART(void) {
if(RCSTAbits.OERR) {
RCSTAbits.CREN = 0;
RCSTAbits.CREN = 1;
}
while(!PIR1bits.RCIF); //Wait for a byte
return RCREG;
}
And in the main loop :
while(1) {
WriteByte_UART('a'); // This works. I can see the As in the terminal
WriteString_UART("Hello World !"); //Nothing displayed :(
}//end while(1)
I have tried different solution for WriteString_UART but none has worked so far.
I don't want to use printf cause it impacts other operations I'm doing with the PIC by adding delay.
So I really want to make it work with WriteString_UART.
In the end I would like to have someting like "Error rate is : [a value]%" on the terminal.
Thanks for your help, and please tell me if something isn't clear.
In your WriteByte_UART() function, try polling the TRMT bit. In particular, change:
while(!PIR1bits.TXIF);
to
while(!TXSTA1bits.TRMT);
I don't know if this is your particular issue, but there exists a race-condition due to the fact that TXIF is not immediately cleared upon loading TXREG. Another option would be to try:
...
Nop();
while(!PIR1bits.TXIF);
...
EDIT BASED ON COMMENTS
The issue is due to the fact that the PIC18 utilizes two different pointer types based on data memory and program memory. Try changing your declaration to void WriteString_UART(const rom char * data) and see what happens. You will need to change your WriteByte_UART() declaration as well, to void WriteByte_UART(const unsigned char ch).
Add delay of few miliseconds after line
TXREG = ch;
verify that pointer *data of WriteString_UART(char *data) actually point to
string "Hello World !".
It seems you found a solution, but the reason why it wasn't working in the first place is still not clear. What compiler are you using?
I learned the hard way that C18 and XC8 are used differently regarding memory spaces. With both compilers, a string declared literally like char string[]="Hello!", will be stored in ROM (program memory). They differ in the way functions use strings.
C18 string functions will have variants to access strings either in RAM or ROM (for example strcpypgm2ram, strcpyram2pgm, etc.). XC8 on the other hand, does the job for you and you will not need to use specific functions to choose which memory you want to access.
If you are using C18, I would highly recommend you switch to XC8, which is more recent and easier to work with. If you still want to use C18 or another compiler which requires you to deal with program/data memory spaces, then here below are two solutions you may want to try. The C18 datasheet says that putsUSART prints a string from data memory to USART. The function putrsUSART will print a string from program memory. So you can simply use putrsUSART to print your string.
You may also want to try the following, which consists in copying your string from program memory to data memory (it may be a waste of memory if your application is tight on memory though) :
char pgmstring[] = "Hello";
char datstring[16];
strcpypgm2ram(datstring, pgmstring);
putsUSART(datstring);
In this example, the pointers pgmstring and datstring will be stored in data memory. The string "Hello" will be stored in program memory. So even if the pointer pgmstring itself is in data memory, it initially points to a memory address (the address of "Hello"). The only way to point to this same string in data memory is to create a copy of it in data memory. This is because a function accepting a string stored in data memory (such as putsUSART) can NOT be used directly with a string stored in program memory.
I hope this could help you understand a bit better how to work with Harvard microprocessors, where program and data memories are separated.

How do I send strings between my osx app and arduino continuously?

I made a cocoa application that generates a list of instructions for an arduino uno to execute. Because the list is too large to fit at one time within the arduino's memory, I am trying to send the arduino an individual instruction at a time.
Arduino:
void setup(){
Serial.begin(9600);
}
void loop(){
if(Serial.available() > 0){
String in = Serial.readString();
delay(10);
Serial.print(in);
}
Serial.print("A");
}
Cocoa App
(Code after the serial port is open and working)
- (void)incomingTextUpdateThread: (NSThread *) parentThread {
// mark that the thread is running
readThreadRunning = TRUE;
const int BUFFER_SIZE = 1;
char byte_buffer[BUFFER_SIZE]; // buffer for holding incoming data
long numBytes=0; // number of bytes read during read
// assign a high priority to this thread
[NSThread setThreadPriority:1.0];
// this will loop unitl the serial port closes
while(TRUE) {
// read() blocks until some data is available or the port is closed
numBytes = read(serialFileDescriptor, byte_buffer, BUFFER_SIZE); // read up to the size of the buffer
if(numBytes>0) {
if(byte_buffer[0] == 'A'){
Coordinate *c = [coordinates objectAtIndex:coordinateIndex];
[self writeString:[self formateCoordinateString:c]];
coordinateIndex++;
}
}
if(coordinateIndex == coordinates.count){
close(serialFileDescriptor);
break;
}
}
readThreadRunning = FALSE;
}
I run the arduino code first and it prints a bunch of 'A's in the serial console. However once I start the cocoa app, it stops printing 'A's and doesn't do anything.
When I set a breakpoint within the while loop, the arduino starts printing the 'A's again. I continue to step within the while loop, and the "instruction" string is sent correctly to the arduino.
My issue is that this only works when I set this break point. Otherwise both my cocoa app and arduino app go into a stand-still.
Thanks for any and all advice! Please feel free to ask for clarification.

OSX Serial read freeze / hang

I'm writing a serial communication wrapper class in Objective-C. To list all serial available modems and setup the connection I'm using pretty much the same code as used in this example project by Apple.
I could read and write the ways apple does it. But I want to implement a loop on a second thread and write to the stream if a NSString *writeString longer 0 and read after write if bytes are available.
I got writing working quite straight forward. I just used the write function declared in unistd.h.
Reading will not work. Whenever I call read(), the function hangs and my loop does not proceed.
Here is the code used in my loop:
- (void)runInCOMLoop {
do {
// write
} while (bytesWritten < strlen([_writeString UTF8String]));
NSMutableString *readString = [NSMutableString string];
ssize_t bytesRead = 0;
ssize_t readB = 0;
char buffer[256];
do {
readB = read(_fileDescriptor, &buffer, sizeof(buffer));
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this function hangs
bytesRead += readB;
if (readB == -1 {
// error
}
else if (readB > 0) {
if(buffer[bytesRead - 1] == '\r' ]] buffer[bytesRead - 1] == '\n') {
break;
}
[readString appendString:[NSString stringWithUTF8String:buffer]];
}
} while (readB > 0);
What am I doing wrong here?
read() will block if there is nothing to read. Apple probably has their own of doing things, but you can use select() to see if there is anything to read on _fileDescriptor. Google around for examples on how to use select.
Here's one link on StackOverflow:
Can someone give me an example of how select() is alerted to an fd becoming "ready"
This excerpt from the select man is pertains:
To effect a poll, the timeout argument should be
non-nil, pointing to a zero-valued timeval structure. Timeout is not
changed by select(), and may be reused on subsequent calls, however it is
good style to re-initialize it before each invocation of select().
You can set the non-blocking flag (O_NONBLOCK) on the file descriptor using fcntl() to keep read() from waiting for data, but if you do that, you have to continuously poll looking for data, which is obviously bad from a CPU usage standpoint. As Charlie Burns' answer explains, the best solution is to use select() which will allow your program to efficiently wait until there is some data to be read on the port's file descriptor. Here's some example code taken from my own Objective-C serial port class, ORSSerialPort (slightly modified):
fd_set localReadFDSet;
FD_ZERO(&localReadFDSet);
FD_SET(self.fileDescriptor, &localReadFDSet);
timeout.tv_sec = 0;
timeout.tv_usec = 100000; // Check to see if port closed every 100ms
result = select(localPortFD+1, &localReadFDSet, NULL, NULL, &timeout);
if (!self.isOpen) break; // Port closed while select call was waiting
if (result < 0) {
// Handle error
}
if (result == 0 || !FD_ISSET(localPortFD, &localReadFDSet)) continue;
// Data is available
char buf[1024];
long lengthRead = read(localPortFD, buf, sizeof(buf));
NSData *readData = nil;
if (lengthRead>0) readData = [NSData dataWithBytes:buf length:lengthRead];
Note that select() indicates that data is available by returning. So, your program will sit suspended at the select() call while no data is available. The program is not hung, that's how it's supposed to work. If you need to do other things while select() is waiting, you should put the select() call on a different queue/thread from the other work you need to do. ORSSerialPort does this.