I'm trying to use FreeRTOS to write ADC data to SD card on the STM32F7 and I'm using V1 of the CMSIS-RTOS API. I'm using mail queues and I have a struct that holds an array.
typedef struct
{
uint16_t data[2048];
} ADC_DATA;
on the ADC half/Full complete interrupts, I add the data to the queue and I have a consumer task that writes this data to the sd card. My issue is in my Consumer Task, I have to do a memcpy to another array and then write the contents of that array to the sd card.
void vConsumer(void const * argument)
{
ADC_DATA *rx_data;
for(;;)
{
writeEvent = osMailGet(adcDataMailId, osWaitForever);
if(writeEvent.status == osEventMail)
{
// write Data to SD
rx_data = writeEvent.value.p;
memcpy(sd_buff, rx_data->data, sizeof(sd_buff));
if(wav_write_result == FR_OK)
{
if( f_write(&wavFile, (uint8_t *)sd_buff, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
{
file_size+=bytes_written;
}
}
osMailFree(adcDataMailId, rx_data);
}
}
This works as intended but if I try to change this line to
f_write(&wavFile, (uint8_t *)rx_data->data, SD_WRITE_BUF_SIZE, (void*)&bytes_written) == FR_OK)
so as to get rid of the memcpy, f_write returns FR_DISK_ERR. Can anyone help shine a light on why this happens, I feel like the extra memcpy is useless and you should just be able to pass the pointer to the queue straight to f_write.
So just a few thoughts here:
memcpy
Usually I copy only the necessary amount of data. If I have the size of the actual data I'll add a boundary check and pass it to memcpy.
Your problem
I am just guessing here, but if you check the struct definition, the data field has the type uint16_t and you cast it to a byte pointer. Also the FatFs documentation expects a void* for the type of buf.
EDIT: Could you post more details of sd_buff
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm working on an embedded project using an STM32F7 device, writing bare metal C.
I want to be able to send data to a UART at any point in the program for debugging purposes, without blocking while the data is sent. I'm using DMA to try to minimise the cpu time used for this.
Currently I'm filling the data into a FIFO queue, and then initiating a DMA request to send the data directly from the FIFO queue to the UART.
The issue with this is I can't set up the DMA to read from both the start and end of the FIFO buffer, in the case where the middle of the FIFO is unused and a message wraps from the end of the buffer to the start.
The two solutions to this would be to set up the first DMA request to read from the head of the FIFO to the end of the buffer, and then once that is complete, read from the start of the buffer to the tail of the FIFO.
The other way to do it would be to memcpy() out the bytes to be sent to another buffer, where they are all sequential, then initiate a single DMA request to send all the data at once.
Both of these would probably work but I'm looking for insight on what the best approach would be here.
The implementation I've usually chosen is similar to what you have proposed:
The logging functions creates a text and adds it to circular buffer.
DMA is used for the UART transmission. DMA is setup to send a contiguous chunk of data.
Whenever the DMA finishes, an interrupt is triggered. It first frees up the transmitted data in the circular buffer. Then it checks if more data needs to be transmitted. If so, it is immediately started again with new data.
Pseudo code:
tx_len = 0;
void log_message(const char* msg)
{
circ_buf_add(msg);
start_tx();
}
void start_tx()
{
if (tx_len > 0)
return; // already transmitting
const char* start;
int len;
circ_buf_get_chunk(&start, &tx_len);
if (tx_len == 0)
return;
uart_tx_dma(start, tx_len);
}
void dma_interrupt_handler()
{
circ_buf_remove(tx_len);
tx_len = 0;
start_tx();
}
It usually makes sense to limit the length of the transmitted chunk. The shorter it is, the sooner space is freed up in the circular buffer.
The examples proposed so far are fire and forget. In the case where your code needs to know if the data has been send. We have used the following structure in which the fifo holds structs pointing to the data.
This way your data is held by the code sending it. It is able to monitor the transmission but it is also responsible for not using the data until the transmission is complete.
A different advantage is that you do not have to allocate a buffer in advance. Only two pointers are required to point to the start en end of the linked list structure.
Some meta code:
enum transmission_state {
Unused,
WaitToBeSend,
Sending,
Done,
Error // Optional but handy
}
struct data_to_send
{
// Point to your data.
data* data_pointer;
// Set the length of your data.
int length;
// What is the current state of this transmission.
transmission_state state;
// Pointer to the next data to be send creating a linked list.
// Only have the send and dma functions use this.
data_to_send* next;
};
// Definition of the fifo.
data_to_send* fifo_first = null;
data_to_send* fifo_end = null;
// Use this function in your code to add data to be send.
void send(data_to_send* dts)
{
if(null == fifo_first) {
fifo_first = dts;
fifo_end = dts;
dts.next = null;
start_dma_transfer(fifo_first);
}
else {
fifo_end.next = dts;
fifo_end = dts;
dts.state = WaitToBeSend;
dts.next = null;
}
};
// Start a transfer.
void start_dma_transfer(data_to_send* dts)
{
dts->state = Sending;
// Do some DMA stuff to start the transmission.
dma_transfer(dts->data, dts->length)
}
// The interrupt handler called when the dma is done.
void dma_interrupt_handler()
{
fifo_first->state = Done;
if(null != fifo_first->next) {
// Send the next data.
fifo_first = fifo_first->next;
start_dma_transfer(fifo_first);
}
else {
// No new data to be send.
fifo_first = null;
fifo_end = null;
}
}
int main()
{
// Setup a transmission
byte data[3] = {x,y,z};
data_to_send transmission = default_dts; // Set to some default.
transmission.data = data;
transmission.length = 3;
send(&transmission);
// Do other important things.
// Later periodically check the transmission.
if(Done == transmission.status) {
// You could use the data for something else or send new data.
}
}
This structure can also be used for I2C and SPI in which case you can add responses to the data_to_send struct and check for a response and act upon it.
I wanted to ask, how will behave DMA SPI rx in STM32 in following situation.
I have a specified (for example) 96 Bytes array called A which is intended to store the data received from the SPI. I turn on my circular SPI DMA which operates on each Byte, is configured to 96 Byte.
Is it possible, when DMA will fill my 96 Bytes array, the Transfer Complete interrupt will went off, to quickly copy the 96 Byte array to another - B, before circular DMA will start writing to A(and destroy the data saved in B)?
I want to transfer(every time when I will get new data from A in B) data from B quickly over USB to PC.
I'm just thinking how to transmit continous data stream SPI from STM32 over USB to PC, because a block of 96 Bytes of data transferred by USB once per certain time is easier I think than stream in real time SPI to USB by STM32? I don't know it's even possible
For that to work, you would have to be able to guarantee that you can copy all the data before the next SPI byte is received and transferred to the start of the buffer. Whether that were possible would depend on the clock speed of the processor and the speed of the SPI, and be able to guarantee that no higher priority interrupts occur that might delay the transfer. To be safe it would need an exceptionally slow SPI speed, and in that case would probably not need to use DMA at all.
All in all it is a bad idea and entirely unnecessary. The DMA controller has a "half-transfer" interrupt for exactly this purpose. You will get the HT interrupt when the first 48 bytes are transferred, and the DMA will continue transferring the remaining 48 bytes while you copy lower half buffer. When you get the transfer complete you transfer the upper half. That extends the time you have to transfer the data from the receive time of a single byte to the receive time of 48 bytes.
If you actually need 96 bytes on each transfer, then you simply make your buffer 192 bytes long (2 x 96).
In pseudo-code:
#define BUFFER_LENGTH 96
char DMA_Buffer[2][BUFFER_LENGTH] ;
void DMA_IRQHandler()
{
if( DMA_IT_Flag(DMA_HT) == SET )
{
memcpy( B, DMA_Buffer[0], BUFFER_LENGTH ) ;
Clear_IT_Flag(DMA_HT) ;
}
else if( DMA_IT_Flag(DMA_TC) == SET )
{
memcpy( B, DMA_Buffer[1], BUFFER_LENGTH ) ;
Clear_IT_Flag(DMA_TC) ;
}
}
With respect to transferring the data to a PC over USB, first of all you need to be sure that your USB transfer rate is at least as fast or faster than the SPI transfer rate. It is likely that the USB transfer is less deterministic (because it is controlled by the PC host - that is you can only output data on the USB when the host explicitly asks for it), so even if the the average transfer rate is sufficient, there may be latency that requires further buffering, so rather then simply copying from the DMA buffer A to a USB buffer B, you may need a circular buffer or FIFO queue to feed the USB. On the other hand, if you already have the buffer DMA_Buffer[0], DMA_Buffer[1] and B you already effectively have a FIFO of three blocks of 96 bytes, which may be sufficient
In one of my projects I faced a similar problem. The task was to transfer data coming from an external ADC chip (connected with SPI) to PC over full speed USB. The data was (8 ch x 16-bit) and I was requested to achieve the fastest sampling frequency possible.
I ended up with a triple buffer solution. There is 4 possible states a buffer can be in:
READY: Buffer is full with data, ready to be send over USB
SENT: Buffer is already sent and outdated
IN_USE: DMA (requested by SPI) is currently filling this buffer
NEXT: This buffer is considered empty and will be used when IN_USE is full.
As the timing of the USB request can't be synchonized with with the SPI process, I believe a double buffer solution wouldn't work. If you don't have a NEXT buffer, by the time you decide to send the READY buffer, DMA may finish filling the IN_USE buffer and start corrupting the READY buffer. But in a triple buffer solution, READY buffer is safe to send over USB, as it won't be filled even the current IN_USE buffer is full.
So the buffer states look like this as the time passes:
Buf0 Buf1 Buf2
==== ==== ====
READY IN_USE NEXT
SENT IN_USE NEXT
NEXT READY IN_USE
NEXT SENT IN_USE
IN_USE NEXT READY
Of course, if the PC don't start USB requests fast enough, you may still loose a READY buffer as soon as it turns into NEXT (before becoming SENT). PC sends USB IN requests asynchronously with no info about the current buffer states. If there is no READY buffer (it's in SENT state), the STM32 responds with a ZLP (zero length package) and the PC tries again after 1 ms delay.
For the implementation on STM32, I use double buffered mode and I modify M0AR & M1AR registers in the DMA Transfer Complete ISR to address 3 buffers.
BTW, I used (3 x 4000) bytes buffers and achieved 32 kHz sampling frequency at the end. USB is configured as vendor specific class and it uses bulk transfers.
Generally using circular DMA only works if you trigger on the half full/half empty, otherwise you don't have enough time to copy information out of the buffer.
I would recommend against copying the data out the buffer during the interrupt. Rather use the data directly from the buffer without an additional copy step.
If you do the copy in the interrupt, you are blocking other lower priority interrupts during the copy. On a STM32 a simple naive byte copy of 48 bytes may take additional 48*6 ~ 300 clock cycles.
If you track the buffers read and write positions independently, you just need to update a single pointer and post a delayed a notification call to the consumer of the buffer.
If you want a longer period then don't use circular DMA, rather use normal DMA in 48 byte blocks and implement circular byte buffer as a data structure.
I did this for a USART at 460k baud that receives asynchronously variable length packets. If you ensure that the producer only updates the write pointer and the consumer only updates the read pointer you can avoid data races in most of it. Note that the read and write of an aligned <=32 bit variable on cortex m3/m4 is atomic.
The included code is a simplified version of the circular buffer with DMA support that I used. It is limited to buffer sizes that are 2^n and uses Templates and C++11 functionality so it may not be suitable depending on your development/platform constraints.
To use the buffer call getDmaReadBlock() or getDMAwriteBlock() and get the DMA memory address and block length. Once the DMA completes use skipRead() / skipWrite() to increment the read or write pointers by the actual amount that was transferred.
/**
* Creates a circular buffer. There is a read pointer and a write pointer
* The buffer is full when the write pointer is = read pointer -1
*/
template<uint16_t SIZE=256>
class CircularByteBuffer {
public:
struct MemBlock {
uint8_t *blockStart;
uint16_t blockLength;
};
private:
uint8_t *_data;
uint16_t _readIndex;
uint16_t _writeIndex;
static constexpr uint16_t _mask = SIZE - 1;
// is the circular buffer a power of 2
static_assert((SIZE & (SIZE - 1)) == 0);
public:
CircularByteBuffer &operator=(const CircularByteBuffer &) = default;
CircularByteBuffer(uint8_t (&data)[SIZE]);
CircularByteBuffer(const CircularByteBuffer &) = default;
~CircularByteBuffer() = default;
private:
static uint16_t wrapIndex(int32_t index);
public:
/*
* The number of byte available to be read. Writing bytes to the buffer can only increase this amount.
*/
uint16_t readBytesAvail() const;
/**
* Return the number of bytes that can still be written. Reading bytes can only increase this amount.
*/
uint16_t writeBytesAvail() const;
/**
* Read a byte from the buffer and increment the read pointer
*/
uint8_t readByte();
/**
* Write a byte to the buffer and increment the write pointer. Throws away the byte if there is no space left.
* #param byte
*/
void writeByte(uint8_t byte);
/**
* Provide read only access to the buffer without incrementing the pointer. Whilst memory accesses outside the
* allocated memeory can be performed. Garbage data can still be read if that byte does not contain valid data
* #param pos the offset from teh current read pointer
* #return the byte at the given offset in the buffer.
*/
uint8_t operator[](uint32_t pos) const;
/**
* INcrement the read pointer by a given amount
*/
void skipRead(uint16_t amount);
/**
* Increment the read pointer by a given amount
*/
void skipWrite(uint16_t amount);
/**
* Get the start and lenght of the memeory block used for DMA writes into the queue.
* #return
*/
MemBlock getDmaWriteBlock();
/**
* Get the start and lenght of the memeory block used for DMA reads from the queue.
* #return
*/
MemBlock getDmaReadBlock();
};
// CircularByteBuffer
// ------------------
template<uint16_t SIZE>
inline CircularByteBuffer<SIZE>::CircularByteBuffer(uint8_t (&data)[SIZE]):
_data(data),
_readIndex(0),
_writeIndex(0) {
}
template<uint16_t SIZE>
inline uint16_t CircularByteBuffer<SIZE>::wrapIndex(int32_t index){
return static_cast<uint16_t>(index & _mask);
}
template<uint16_t SIZE>
inline uint16_t CircularByteBuffer<SIZE>::readBytesAvail() const {
return wrapIndex(_writeIndex - _readIndex);
}
template<uint16_t SIZE>
inline uint16_t CircularByteBuffer<SIZE>::writeBytesAvail() const {
return wrapIndex(_readIndex - _writeIndex - 1);
}
template<uint16_t SIZE>
inline uint8_t CircularByteBuffer<SIZE>::readByte() {
if (readBytesAvail()) {
uint8_t result = _data[_readIndex];
_readIndex = wrapIndex(_readIndex+1);
return result;
} else {
return 0;
}
}
template<uint16_t SIZE>
inline void CircularByteBuffer<SIZE>::writeByte(uint8_t byte) {
if (writeBytesAvail()) {
_data[_writeIndex] = byte;
_writeIndex = wrapIndex(_writeIndex+1);
}
}
template<uint16_t SIZE>
inline uint8_t CircularByteBuffer<SIZE>::operator[](uint32_t pos) const {
return _data[wrapIndex(_readIndex + pos)];
}
template<uint16_t SIZE>
inline void CircularByteBuffer<SIZE>::skipRead(uint16_t amount) {
_readIndex = wrapIndex(_readIndex+ amount);
}
template<uint16_t SIZE>
inline void CircularByteBuffer<SIZE>::skipWrite(uint16_t amount) {
_writeIndex = wrapIndex(_writeIndex+ amount);
}
template <uint16_t SIZE>
inline typename CircularByteBuffer<SIZE>::MemBlock CircularByteBuffer<SIZE>::getDmaWriteBlock(){
uint16_t len = static_cast<uint16_t>(SIZE - _writeIndex);
// full is (write == (read -1)) so on wrap around we need to ensure that we stop 1 off from the read pointer.
if( _readIndex == 0){
len = static_cast<uint16_t>(len - 1);
}
if( _readIndex > _writeIndex){
len = static_cast<uint16_t>(_readIndex - _writeIndex - 1);
}
return {&_data[_writeIndex], len};
}
template <uint16_t SIZE>
inline typename CircularByteBuffer<SIZE>::MemBlock CircularByteBuffer<SIZE>::getDmaReadBlock(){
if( _readIndex > _writeIndex){
return {&_data[_readIndex], static_cast<uint16_t>(SIZE- _readIndex)};
} else {
return {&_data[_readIndex], static_cast<uint16_t>(_writeIndex - _readIndex)};
}
}
`
I need help with the uart communication I am trying to implement on my Proteus simulation. I use a PIC18f4520 and I want to display on the virtual terminal the values that have been calculated by the microcontroller.
Here a snap of my design on Proteus
Right now, this is how my UART code looks like :
#define _XTAL_FREQ 20000000
#define _BAUDRATE 9600
void Configuration_ISR(void) {
IPR1bits.TMR1IP = 1; // TMR1 Overflow Interrupt Priority - High
PIE1bits.TMR1IE = 1; // TMR1 Overflow Interrupt Enable
PIR1bits.TMR1IF = 0; // TMR1 Overflow Interrupt Flag
// 0 = TMR1 register did not overflow
// 1 = TMR1 register overflowed (must be cleared in software)
RCONbits.IPEN = 1; // Interrupt Priority High level
INTCONbits.PEIE = 1; // Enables all low-priority peripheral interrupts
//INTCONbits.GIE = 1; // Enables all high-priority interrupts
}
void Configuration_UART(void) {
TRISCbits.TRISC6 = 0;
TRISCbits.TRISC7 = 1;
SPBRG = ((_XTAL_FREQ/16)/_BAUDRATE)-1;
//RCSTA REG
RCSTAbits.SPEN = 1; // enable serial port pins
RCSTAbits.RX9 = 0;
//TXSTA REG
TXSTAbits.BRGH = 1; // fast baudrate
TXSTAbits.SYNC = 0; // asynchronous
TXSTAbits.TX9 = 0; // 8-bit transmission
TXSTAbits.TXEN = 1; // enble transmitter
}
void WriteByte_UART(unsigned char ch) {
while(!PIR1bits.TXIF); // Wait for TXIF flag Set which indicates
// TXREG register is empty
TXREG = ch; // Transmitt data to UART
}
void WriteString_UART(char *data) {
while(*data){
WriteByte_UART(*data++);
}
}
unsigned char ReceiveByte_UART(void) {
if(RCSTAbits.OERR) {
RCSTAbits.CREN = 0;
RCSTAbits.CREN = 1;
}
while(!PIR1bits.RCIF); //Wait for a byte
return RCREG;
}
And in the main loop :
while(1) {
WriteByte_UART('a'); // This works. I can see the As in the terminal
WriteString_UART("Hello World !"); //Nothing displayed :(
}//end while(1)
I have tried different solution for WriteString_UART but none has worked so far.
I don't want to use printf cause it impacts other operations I'm doing with the PIC by adding delay.
So I really want to make it work with WriteString_UART.
In the end I would like to have someting like "Error rate is : [a value]%" on the terminal.
Thanks for your help, and please tell me if something isn't clear.
In your WriteByte_UART() function, try polling the TRMT bit. In particular, change:
while(!PIR1bits.TXIF);
to
while(!TXSTA1bits.TRMT);
I don't know if this is your particular issue, but there exists a race-condition due to the fact that TXIF is not immediately cleared upon loading TXREG. Another option would be to try:
...
Nop();
while(!PIR1bits.TXIF);
...
EDIT BASED ON COMMENTS
The issue is due to the fact that the PIC18 utilizes two different pointer types based on data memory and program memory. Try changing your declaration to void WriteString_UART(const rom char * data) and see what happens. You will need to change your WriteByte_UART() declaration as well, to void WriteByte_UART(const unsigned char ch).
Add delay of few miliseconds after line
TXREG = ch;
verify that pointer *data of WriteString_UART(char *data) actually point to
string "Hello World !".
It seems you found a solution, but the reason why it wasn't working in the first place is still not clear. What compiler are you using?
I learned the hard way that C18 and XC8 are used differently regarding memory spaces. With both compilers, a string declared literally like char string[]="Hello!", will be stored in ROM (program memory). They differ in the way functions use strings.
C18 string functions will have variants to access strings either in RAM or ROM (for example strcpypgm2ram, strcpyram2pgm, etc.). XC8 on the other hand, does the job for you and you will not need to use specific functions to choose which memory you want to access.
If you are using C18, I would highly recommend you switch to XC8, which is more recent and easier to work with. If you still want to use C18 or another compiler which requires you to deal with program/data memory spaces, then here below are two solutions you may want to try. The C18 datasheet says that putsUSART prints a string from data memory to USART. The function putrsUSART will print a string from program memory. So you can simply use putrsUSART to print your string.
You may also want to try the following, which consists in copying your string from program memory to data memory (it may be a waste of memory if your application is tight on memory though) :
char pgmstring[] = "Hello";
char datstring[16];
strcpypgm2ram(datstring, pgmstring);
putsUSART(datstring);
In this example, the pointers pgmstring and datstring will be stored in data memory. The string "Hello" will be stored in program memory. So even if the pointer pgmstring itself is in data memory, it initially points to a memory address (the address of "Hello"). The only way to point to this same string in data memory is to create a copy of it in data memory. This is because a function accepting a string stored in data memory (such as putsUSART) can NOT be used directly with a string stored in program memory.
I hope this could help you understand a bit better how to work with Harvard microprocessors, where program and data memories are separated.
My code is as follows:
boost::asio::streambuf b1;
boost::asio::async_read_until(upstream_socket_, b1, '#',
boost::bind(&bridge::handle_upstream_read, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
void handle_upstream1_read(const boost::system::error_code& error,
const size_t& bytes_transferred)
{
if (!error)
{
async_write(downstream_socket_,
b2,
boost::bind(&bridge::handle_downstream_write,
shared_from_this(),
boost::asio::placeholders::error));
}
else
close();
}
According to the documentation of async_read_until, http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/async_read_until/overload1.html,
After a successful async_read_until operation, the streambuf may contain additional data beyond the delimiter. An application will typically leave that data in the streambuf for a subsequent async_read_until operation to examine.
I know that the streambuf may contain additional data beyond the delimiter, but, in my case, will it write those additional data (the data beyond the char'#') to the downstream_socket_ inside the async_write operation? Or will async_write function be smart enough not to write those additional data until the next time the handle_upstream1_read function is being called?
According to the approaches in the documentation, the data in streambuf are stored in the istream first ( std::istream response_stream(&streambuf); )
and then put it into a string by using std::getline() funciton.
Do I really need to store the streambuf in istream first and then convert it into a string and then convert it back to char arrary (so that I can send the char array to the downstream_socket_ ) instead of just using the async_write to write the data( up to but not including the delimter, '#' ) to the downstream_socket_ ?
I prefer the second approach so that I don't need to make several conversion on the data. However, it seems that something is wrong when I tried the second approach.
My ideal case is that:
upstream_socket_ received xxxx#yyyy by using async_read_until
xxxx# is written to the downstream_socket_
upstream_socket_ received zzzz#kkkk by using async_read_until
yyyyzzzz# is written to the downstream_socket_
It seems that async_write operation still writes the data beyond the delimiter to the downstream_socket_. (but I am not 100% sure about this)
I appreciate if anyone can provide a little help !
The async_write() overload being used is considered complete when all of the streambuf's data, its input sequence, has been written to the WriteStream (socket). It is equivalent to calling:
boost::asio::async_write(stream, streambuf,
boost::asio::transfer_all(), handler);
One can limit the amount of bytes written and consumed from the streambuf object by calling this async_write() overload with the boost::asio::transfer_exactly completion condition:
boost::asio::async_write(stream, streambuf,
boost::asio::transfer_exactly(n), handler);
Alternatively, one can write directly from the streambuf's input sequence. However, one will need to explicitly consume from the streambuf.
boost::asio::async_write(stream,
boost::asio::buffer(streambuf.data(), n), handler);
// Within the completion handler...
streambuf.consume(n);
Note that when the async_read_until() operation completes, the completion handler's bytes_transferred argument contains the number of bytes in the streambuf's input sequence up to and including the delimiter, or 0 if an error occurred.
Here is a complete example demonstrating using both approaches. The example is written using synchronous operations in an attempt to simplify the flow:
#include <iostream>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
// This example is not interested in the handlers, so provide a noop function
// that will be passed to bind to meet the handler concept requirements.
void noop() {}
/// #brief Helper function that extracts a string from a streambuf.
std::string make_string(
boost::asio::streambuf& streambuf,
std::size_t n)
{
return std::string(
boost::asio::buffers_begin(streambuf.data()),
boost::asio::buffers_begin(streambuf.data()) + n);
}
int main()
{
using boost::asio::ip::tcp;
boost::asio::io_service io_service;
// Create all I/O objects.
tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), 0));
tcp::socket server_socket(io_service);
tcp::socket client_socket(io_service);
// Connect client and server sockets.
acceptor.async_accept(server_socket, boost::bind(&noop));
client_socket.async_connect(acceptor.local_endpoint(), boost::bind(&noop));
io_service.run();
// Mockup write_buffer as if it read "xxxx#yyyy" with read_until()
// using '#' as a delimiter.
boost::asio::streambuf write_buffer;
std::ostream output(&write_buffer);
output << "xxxx#yyyy";
assert(write_buffer.size() == 9);
auto bytes_transferred = 5;
// Write to server.
boost::asio::write(server_socket, write_buffer,
boost::asio::transfer_exactly(bytes_transferred));
// Verify write operation consumed part of the input sequence.
assert(write_buffer.size() == 4);
// Read from client.
boost::asio::streambuf read_buffer;
bytes_transferred = boost::asio::read(
client_socket, read_buffer.prepare(bytes_transferred));
read_buffer.commit(bytes_transferred);
// Copy from the read buffers input sequence.
std::cout << "Read: " <<
make_string(read_buffer, bytes_transferred) << std::endl;
read_buffer.consume(bytes_transferred);
// Mockup write_buffer as if it read "zzzz#kkkk" with read_until()
// using '#' as a delimiter.
output << "zzzz#kkkk";
assert(write_buffer.size() == 13);
bytes_transferred = 9; // yyyyzzzz#
// Write to server.
boost::asio::write(server_socket, buffer(write_buffer.data(),
bytes_transferred));
// Verify write operation did not consume the input sequence.
assert(write_buffer.size() == 13);
write_buffer.consume(bytes_transferred);
// Read from client.
bytes_transferred = boost::asio::read(
client_socket, read_buffer.prepare(bytes_transferred));
read_buffer.commit(bytes_transferred);
// Copy from the read buffers input sequence.
std::cout << "Read: " <<
make_string(read_buffer, bytes_transferred) << std::endl;
read_buffer.consume(bytes_transferred);
}
Output:
Read: xxxx#
Read: yyyyzzzz#
A few other notes:
The streambuf owns the memory, and std::istream and std::ostream use the memory. Using streams may be a good idea when one needs to extract formatted input or insert formatted output. For instance, when one wishes to read the string "123" as an integer 123.
One can directly access the streambuf's input sequence and iterate over it. In the example above, I use boost::asio::buffers_begin() to help construct a std::string by iterating over a streambuf's input sequence.
std::string(
boost::asio::buffers_begin(streambuf.data()),
boost::asio::buffers_begin(streambuf.data()) + n);
A stream-based transport protocol is being used, so handle incoming data as a stream. Be aware that even if the intermediary server reframes messages and sends "xxxx#" in one write operation and "yyyyzzzz#" in a subsequent write operation, the downstream may read "xxxx#yyyy" in a single read operation.
I should use Objective-C to read some slowly growing file (under Mac OS X).
"Slowly" means that I read to EOF before it grows bigger.
In means of POSIX code in plain syncronous C I can do it as following:
while(1)
{
res = select(fd+1,&fdset,NULL,&fdset,some_timeout);
if(res > 0)
{
len = read(fd,buf,sizeof(buf));
if (len>0)
{
printf("Could read %u bytes. Continue.\n", len);
}
else
{
sleep(some_timeout_in_sec);
}
}
}
Now I want to re-write this in some asynchronous manner, using NSInputSource or some other async Objective-C technique.
The problem with NSInputSource: If I use scheduleInRunLoop: method then once I get NSStreamEventEndEncountered event, I stop receiving any events.
Can I still use NSInputSource or should I pass to using NSFileHandle somehow or what would you recommend ?
I see a few problems.
1) some_Timeout, for select() needs to be a struct timeval *.
2) for sleep() some_timeout needs to be an integer number of seconds.
3) the value in some_timeout is decremented via select() (which is why the last parameter is a pointer to the struct timeval*. And that struct needs to be re-initialized before each call to select().
4) the parameters to select() are highest fd of interest+1, then three separate struct fd_set * objects. The first is for input files, the second is for output files, the third is for exceptions, however, the posted code is using the same struct fd_set for both the inputs and the exceptions, This probably will not be what is needed.
When the above problems are corrected, the code should work.