sprintf() and WriteFile() affecting string Buffer - printf

I have a very weird problem which I cannot seem to figure out. Unfortunately, I'm not even sure how to describe it without describing my entire application. What I am trying to do is:
1) read a byte from the serial port
2) store each char into tagBuffer as they are read
3) run a query using tagBuffer to see what type of tag it is (book or shelf tag)
4) depending on the type of tag, output a series of bytes corresponding to the type of tag
Most of my code is implemented and I can get the right tag code sent back out the serial port. But there are two lines that I've added as debug statements which when I tried to remove them, they cause my program to stop working.
The lines are the two lines at the very bottom:
sprintf(buf,"%s!\n", tagBuffer);
WriteFile(hSerial,buf,strlen(buf), &dwBytesWritten,&ovWrite);
If I try to remove them, "tagBuffer" will only store the last character as oppose being a buffer. Same thing with the next line, WriteFile().
I thought sprintf and WriteFile are I/O functions and would have no effect on variables.
I'm stuck and I need help to fix this.
//keep polling as long as stop character '-' is not read
while(szRxChar != '-')
{
// Check if a read is outstanding
if (HasOverlappedIoCompleted(&ovRead))
{
// Issue a serial port read
if (!ReadFile(hSerial,&szRxChar,1,
&dwBytesRead,&ovRead))
{
DWORD dwErr = GetLastError();
if (dwErr!=ERROR_IO_PENDING)
return dwErr;
}
}
// resets tagBuffer in case tagBuffer is out of sync
time_t t_time = time(0);
char buf[50];
if (HasOverlappedIoCompleted(&ovWrite))
{
i=0;
}
// Wait 5 seconds for serial input
if (!(HasOverlappedIoCompleted(&ovRead)))
{
WaitForSingleObject(hReadEvent,RESET_TIME);
}
// Check if serial input has arrived
if (GetOverlappedResult(hSerial,&ovRead,
&dwBytesRead,FALSE))
{
// Wait for the write
GetOverlappedResult(hSerial,&ovWrite,
&dwBytesWritten,TRUE);
if( strlen(tagBuffer) >= PACKET_LENGTH )
{
i = 0;
}
//load tagBuffer with byte stream
tagBuffer[i] = szRxChar;
i++;
tagBuffer[i] = 0; //char arrays are \0 terminated
//run query with tagBuffer
sprintf(query,"select type from rfid where rfidnum=\"");
strcat(query, tagBuffer);
strcat(query, "\"");
mysql_real_query(&mysql,query,(unsigned int)strlen(query));
//process result and send back to handheld
res = mysql_use_result(&mysql);
while(row = mysql_fetch_row(res))
{
printf("result of query is %s\n",row[0]);
string str = "";
str = string(row[0]);
if( str == "book" )
{
WriteFile(hSerial,BOOK_INDICATOR,strlen(BOOK_INDICATOR),
&dwBytesWritten,&ovWrite);
}
else if ( str == "shelf" )
{
WriteFile(hSerial,SHELF_INDICATOR,strlen(SHELF_INDICATOR),
&dwBytesWritten,&ovWrite);
}
else //this else doesn't work
{
WriteFile(hSerial,NOK,strlen(NOK),
&dwBytesWritten,&ovWrite);
}
}
mysql_free_result(res);
// Display a response to input
//printf("query is %s!\n", query);
//printf("strlen(tagBuffer) is %d!\n", strlen(tagBuffer));
//without these, tagBuffer only holds the last character
sprintf(buf,"%s!\n", tagBuffer);
WriteFile(hSerial,buf,strlen(buf), &dwBytesWritten,&ovWrite);
}
}
With those two lines, my output looks like this:
s sh she shel shelf shelf0 shelf00 BOOKCODE shelf0001
Without them, I figured out that tagBuffer and buf only stores the most recent character at any one time.
Any help at all will be greatly appreciated. Thanks.

Where are you allocating tagbuffer, how large is it?
It's possible that you are overwriting 'buf' because you are writing past the end of tagbuffer.

It seems unlikely that those two lines would have that effect on a correct program - maybe you haven't allocated sufficient space in buf for the whole length of the string in tagBuffer? This might cause a buffer overrun that is disguising the real problem?

The first thing I'd say is a piece of general advice: bugs aren't always where you think they are. If you've got something going on that doesn't seem to make sense, it often means that your assumptions somewhere else are wrong.
Here, it does seem very unlikely that an sprintf() and a WriteFile() will change the state of the "buf" array variable. However, those two lines of test code do write to "hSerial", while your main loop also reads from "hSerial". That sounds like a recipie for changing the behaviour of your program.
Suggestion: Change your lines of debugging output to store the output somewhere else: to a dialog box, or to a log file, or similar. Debugging output should generally not go to files used in the core logic, as that's too likely to change how the core logic behaves.

In my opinion, the real problem here is that you're trying to read and write the serial port from a single thread, and this is making the code more complex than it needs to be. I suggest that you read the following articles and reconsider your design:
Serial Port I/O from Joseph Newcomer's website.
Serial Communications in Win32 from MSDN.
In a multithreaded implementation, whenever the reader thread reads a message from the serial port you would then post it to your application's main thread. The main thread would then parse the message and query the database, and then queue an appropriate response to the writer thread.
This may sound more complex than your current design, but really it isn't, as Newcomer explains.
I hope this helps!

Related

Write UART on PIC18

I need help with the uart communication I am trying to implement on my Proteus simulation. I use a PIC18f4520 and I want to display on the virtual terminal the values that have been calculated by the microcontroller.
Here a snap of my design on Proteus
Right now, this is how my UART code looks like :
#define _XTAL_FREQ 20000000
#define _BAUDRATE 9600
void Configuration_ISR(void) {
IPR1bits.TMR1IP = 1; // TMR1 Overflow Interrupt Priority - High
PIE1bits.TMR1IE = 1; // TMR1 Overflow Interrupt Enable
PIR1bits.TMR1IF = 0; // TMR1 Overflow Interrupt Flag
// 0 = TMR1 register did not overflow
// 1 = TMR1 register overflowed (must be cleared in software)
RCONbits.IPEN = 1; // Interrupt Priority High level
INTCONbits.PEIE = 1; // Enables all low-priority peripheral interrupts
//INTCONbits.GIE = 1; // Enables all high-priority interrupts
}
void Configuration_UART(void) {
TRISCbits.TRISC6 = 0;
TRISCbits.TRISC7 = 1;
SPBRG = ((_XTAL_FREQ/16)/_BAUDRATE)-1;
//RCSTA REG
RCSTAbits.SPEN = 1; // enable serial port pins
RCSTAbits.RX9 = 0;
//TXSTA REG
TXSTAbits.BRGH = 1; // fast baudrate
TXSTAbits.SYNC = 0; // asynchronous
TXSTAbits.TX9 = 0; // 8-bit transmission
TXSTAbits.TXEN = 1; // enble transmitter
}
void WriteByte_UART(unsigned char ch) {
while(!PIR1bits.TXIF); // Wait for TXIF flag Set which indicates
// TXREG register is empty
TXREG = ch; // Transmitt data to UART
}
void WriteString_UART(char *data) {
while(*data){
WriteByte_UART(*data++);
}
}
unsigned char ReceiveByte_UART(void) {
if(RCSTAbits.OERR) {
RCSTAbits.CREN = 0;
RCSTAbits.CREN = 1;
}
while(!PIR1bits.RCIF); //Wait for a byte
return RCREG;
}
And in the main loop :
while(1) {
WriteByte_UART('a'); // This works. I can see the As in the terminal
WriteString_UART("Hello World !"); //Nothing displayed :(
}//end while(1)
I have tried different solution for WriteString_UART but none has worked so far.
I don't want to use printf cause it impacts other operations I'm doing with the PIC by adding delay.
So I really want to make it work with WriteString_UART.
In the end I would like to have someting like "Error rate is : [a value]%" on the terminal.
Thanks for your help, and please tell me if something isn't clear.
In your WriteByte_UART() function, try polling the TRMT bit. In particular, change:
while(!PIR1bits.TXIF);
to
while(!TXSTA1bits.TRMT);
I don't know if this is your particular issue, but there exists a race-condition due to the fact that TXIF is not immediately cleared upon loading TXREG. Another option would be to try:
...
Nop();
while(!PIR1bits.TXIF);
...
EDIT BASED ON COMMENTS
The issue is due to the fact that the PIC18 utilizes two different pointer types based on data memory and program memory. Try changing your declaration to void WriteString_UART(const rom char * data) and see what happens. You will need to change your WriteByte_UART() declaration as well, to void WriteByte_UART(const unsigned char ch).
Add delay of few miliseconds after line
TXREG = ch;
verify that pointer *data of WriteString_UART(char *data) actually point to
string "Hello World !".
It seems you found a solution, but the reason why it wasn't working in the first place is still not clear. What compiler are you using?
I learned the hard way that C18 and XC8 are used differently regarding memory spaces. With both compilers, a string declared literally like char string[]="Hello!", will be stored in ROM (program memory). They differ in the way functions use strings.
C18 string functions will have variants to access strings either in RAM or ROM (for example strcpypgm2ram, strcpyram2pgm, etc.). XC8 on the other hand, does the job for you and you will not need to use specific functions to choose which memory you want to access.
If you are using C18, I would highly recommend you switch to XC8, which is more recent and easier to work with. If you still want to use C18 or another compiler which requires you to deal with program/data memory spaces, then here below are two solutions you may want to try. The C18 datasheet says that putsUSART prints a string from data memory to USART. The function putrsUSART will print a string from program memory. So you can simply use putrsUSART to print your string.
You may also want to try the following, which consists in copying your string from program memory to data memory (it may be a waste of memory if your application is tight on memory though) :
char pgmstring[] = "Hello";
char datstring[16];
strcpypgm2ram(datstring, pgmstring);
putsUSART(datstring);
In this example, the pointers pgmstring and datstring will be stored in data memory. The string "Hello" will be stored in program memory. So even if the pointer pgmstring itself is in data memory, it initially points to a memory address (the address of "Hello"). The only way to point to this same string in data memory is to create a copy of it in data memory. This is because a function accepting a string stored in data memory (such as putsUSART) can NOT be used directly with a string stored in program memory.
I hope this could help you understand a bit better how to work with Harvard microprocessors, where program and data memories are separated.

issues about using async_write right after async_read_until

My code is as follows:
boost::asio::streambuf b1;
boost::asio::async_read_until(upstream_socket_, b1, '#',
boost::bind(&bridge::handle_upstream_read, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
void handle_upstream1_read(const boost::system::error_code& error,
const size_t& bytes_transferred)
{
if (!error)
{
async_write(downstream_socket_,
b2,
boost::bind(&bridge::handle_downstream_write,
shared_from_this(),
boost::asio::placeholders::error));
}
else
close();
}
According to the documentation of async_read_until, http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/async_read_until/overload1.html,
After a successful async_read_until operation, the streambuf may contain additional data beyond the delimiter. An application will typically leave that data in the streambuf for a subsequent async_read_until operation to examine.
I know that the streambuf may contain additional data beyond the delimiter, but, in my case, will it write those additional data (the data beyond the char'#') to the downstream_socket_ inside the async_write operation? Or will async_write function be smart enough not to write those additional data until the next time the handle_upstream1_read function is being called?
According to the approaches in the documentation, the data in streambuf are stored in the istream first ( std::istream response_stream(&streambuf); )
and then put it into a string by using std::getline() funciton.
Do I really need to store the streambuf in istream first and then convert it into a string and then convert it back to char arrary (so that I can send the char array to the downstream_socket_ ) instead of just using the async_write to write the data( up to but not including the delimter, '#' ) to the downstream_socket_ ?
I prefer the second approach so that I don't need to make several conversion on the data. However, it seems that something is wrong when I tried the second approach.
My ideal case is that:
upstream_socket_ received xxxx#yyyy by using async_read_until
xxxx# is written to the downstream_socket_
upstream_socket_ received zzzz#kkkk by using async_read_until
yyyyzzzz# is written to the downstream_socket_
It seems that async_write operation still writes the data beyond the delimiter to the downstream_socket_. (but I am not 100% sure about this)
I appreciate if anyone can provide a little help !
The async_write() overload being used is considered complete when all of the streambuf's data, its input sequence, has been written to the WriteStream (socket). It is equivalent to calling:
boost::asio::async_write(stream, streambuf,
boost::asio::transfer_all(), handler);
One can limit the amount of bytes written and consumed from the streambuf object by calling this async_write() overload with the boost::asio::transfer_exactly completion condition:
boost::asio::async_write(stream, streambuf,
boost::asio::transfer_exactly(n), handler);
Alternatively, one can write directly from the streambuf's input sequence. However, one will need to explicitly consume from the streambuf.
boost::asio::async_write(stream,
boost::asio::buffer(streambuf.data(), n), handler);
// Within the completion handler...
streambuf.consume(n);
Note that when the async_read_until() operation completes, the completion handler's bytes_transferred argument contains the number of bytes in the streambuf's input sequence up to and including the delimiter, or 0 if an error occurred.
Here is a complete example demonstrating using both approaches. The example is written using synchronous operations in an attempt to simplify the flow:
#include <iostream>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
// This example is not interested in the handlers, so provide a noop function
// that will be passed to bind to meet the handler concept requirements.
void noop() {}
/// #brief Helper function that extracts a string from a streambuf.
std::string make_string(
boost::asio::streambuf& streambuf,
std::size_t n)
{
return std::string(
boost::asio::buffers_begin(streambuf.data()),
boost::asio::buffers_begin(streambuf.data()) + n);
}
int main()
{
using boost::asio::ip::tcp;
boost::asio::io_service io_service;
// Create all I/O objects.
tcp::acceptor acceptor(io_service, tcp::endpoint(tcp::v4(), 0));
tcp::socket server_socket(io_service);
tcp::socket client_socket(io_service);
// Connect client and server sockets.
acceptor.async_accept(server_socket, boost::bind(&noop));
client_socket.async_connect(acceptor.local_endpoint(), boost::bind(&noop));
io_service.run();
// Mockup write_buffer as if it read "xxxx#yyyy" with read_until()
// using '#' as a delimiter.
boost::asio::streambuf write_buffer;
std::ostream output(&write_buffer);
output << "xxxx#yyyy";
assert(write_buffer.size() == 9);
auto bytes_transferred = 5;
// Write to server.
boost::asio::write(server_socket, write_buffer,
boost::asio::transfer_exactly(bytes_transferred));
// Verify write operation consumed part of the input sequence.
assert(write_buffer.size() == 4);
// Read from client.
boost::asio::streambuf read_buffer;
bytes_transferred = boost::asio::read(
client_socket, read_buffer.prepare(bytes_transferred));
read_buffer.commit(bytes_transferred);
// Copy from the read buffers input sequence.
std::cout << "Read: " <<
make_string(read_buffer, bytes_transferred) << std::endl;
read_buffer.consume(bytes_transferred);
// Mockup write_buffer as if it read "zzzz#kkkk" with read_until()
// using '#' as a delimiter.
output << "zzzz#kkkk";
assert(write_buffer.size() == 13);
bytes_transferred = 9; // yyyyzzzz#
// Write to server.
boost::asio::write(server_socket, buffer(write_buffer.data(),
bytes_transferred));
// Verify write operation did not consume the input sequence.
assert(write_buffer.size() == 13);
write_buffer.consume(bytes_transferred);
// Read from client.
bytes_transferred = boost::asio::read(
client_socket, read_buffer.prepare(bytes_transferred));
read_buffer.commit(bytes_transferred);
// Copy from the read buffers input sequence.
std::cout << "Read: " <<
make_string(read_buffer, bytes_transferred) << std::endl;
read_buffer.consume(bytes_transferred);
}
Output:
Read: xxxx#
Read: yyyyzzzz#
A few other notes:
The streambuf owns the memory, and std::istream and std::ostream use the memory. Using streams may be a good idea when one needs to extract formatted input or insert formatted output. For instance, when one wishes to read the string "123" as an integer 123.
One can directly access the streambuf's input sequence and iterate over it. In the example above, I use boost::asio::buffers_begin() to help construct a std::string by iterating over a streambuf's input sequence.
std::string(
boost::asio::buffers_begin(streambuf.data()),
boost::asio::buffers_begin(streambuf.data()) + n);
A stream-based transport protocol is being used, so handle incoming data as a stream. Be aware that even if the intermediary server reframes messages and sends "xxxx#" in one write operation and "yyyyzzzz#" in a subsequent write operation, the downstream may read "xxxx#yyyy" in a single read operation.

Reading dynamically growing file using NSInputStream

I should use Objective-C to read some slowly growing file (under Mac OS X).
"Slowly" means that I read to EOF before it grows bigger.
In means of POSIX code in plain syncronous C I can do it as following:
while(1)
{
res = select(fd+1,&fdset,NULL,&fdset,some_timeout);
if(res > 0)
{
len = read(fd,buf,sizeof(buf));
if (len>0)
{
printf("Could read %u bytes. Continue.\n", len);
}
else
{
sleep(some_timeout_in_sec);
}
}
}
Now I want to re-write this in some asynchronous manner, using NSInputSource or some other async Objective-C technique.
The problem with NSInputSource: If I use scheduleInRunLoop: method then once I get NSStreamEventEndEncountered event, I stop receiving any events.
Can I still use NSInputSource or should I pass to using NSFileHandle somehow or what would you recommend ?
I see a few problems.
1) some_Timeout, for select() needs to be a struct timeval *.
2) for sleep() some_timeout needs to be an integer number of seconds.
3) the value in some_timeout is decremented via select() (which is why the last parameter is a pointer to the struct timeval*. And that struct needs to be re-initialized before each call to select().
4) the parameters to select() are highest fd of interest+1, then three separate struct fd_set * objects. The first is for input files, the second is for output files, the third is for exceptions, however, the posted code is using the same struct fd_set for both the inputs and the exceptions, This probably will not be what is needed.
When the above problems are corrected, the code should work.

OSX Serial read freeze / hang

I'm writing a serial communication wrapper class in Objective-C. To list all serial available modems and setup the connection I'm using pretty much the same code as used in this example project by Apple.
I could read and write the ways apple does it. But I want to implement a loop on a second thread and write to the stream if a NSString *writeString longer 0 and read after write if bytes are available.
I got writing working quite straight forward. I just used the write function declared in unistd.h.
Reading will not work. Whenever I call read(), the function hangs and my loop does not proceed.
Here is the code used in my loop:
- (void)runInCOMLoop {
do {
// write
} while (bytesWritten < strlen([_writeString UTF8String]));
NSMutableString *readString = [NSMutableString string];
ssize_t bytesRead = 0;
ssize_t readB = 0;
char buffer[256];
do {
readB = read(_fileDescriptor, &buffer, sizeof(buffer));
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this function hangs
bytesRead += readB;
if (readB == -1 {
// error
}
else if (readB > 0) {
if(buffer[bytesRead - 1] == '\r' ]] buffer[bytesRead - 1] == '\n') {
break;
}
[readString appendString:[NSString stringWithUTF8String:buffer]];
}
} while (readB > 0);
What am I doing wrong here?
read() will block if there is nothing to read. Apple probably has their own of doing things, but you can use select() to see if there is anything to read on _fileDescriptor. Google around for examples on how to use select.
Here's one link on StackOverflow:
Can someone give me an example of how select() is alerted to an fd becoming "ready"
This excerpt from the select man is pertains:
To effect a poll, the timeout argument should be
non-nil, pointing to a zero-valued timeval structure. Timeout is not
changed by select(), and may be reused on subsequent calls, however it is
good style to re-initialize it before each invocation of select().
You can set the non-blocking flag (O_NONBLOCK) on the file descriptor using fcntl() to keep read() from waiting for data, but if you do that, you have to continuously poll looking for data, which is obviously bad from a CPU usage standpoint. As Charlie Burns' answer explains, the best solution is to use select() which will allow your program to efficiently wait until there is some data to be read on the port's file descriptor. Here's some example code taken from my own Objective-C serial port class, ORSSerialPort (slightly modified):
fd_set localReadFDSet;
FD_ZERO(&localReadFDSet);
FD_SET(self.fileDescriptor, &localReadFDSet);
timeout.tv_sec = 0;
timeout.tv_usec = 100000; // Check to see if port closed every 100ms
result = select(localPortFD+1, &localReadFDSet, NULL, NULL, &timeout);
if (!self.isOpen) break; // Port closed while select call was waiting
if (result < 0) {
// Handle error
}
if (result == 0 || !FD_ISSET(localPortFD, &localReadFDSet)) continue;
// Data is available
char buf[1024];
long lengthRead = read(localPortFD, buf, sizeof(buf));
NSData *readData = nil;
if (lengthRead>0) readData = [NSData dataWithBytes:buf length:lengthRead];
Note that select() indicates that data is available by returning. So, your program will sit suspended at the select() call while no data is available. The program is not hung, that's how it's supposed to work. If you need to do other things while select() is waiting, you should put the select() call on a different queue/thread from the other work you need to do. ORSSerialPort does this.

How to check for End-of-File using NSFileHandle's readabilityHandler?

I am reading data from a NSFileHandle (from a NSPipe) using a readabilityHandler block:
fileHandle.readabilityHandler = ^( NSFileHandle *handle ) {
[self processData: [handle availableData]];
}
This works fine, I get all the data I expect fed to my processData method. The problem is that I need to know when the last chunk of data was read. availableData should return an empty NSData instance if it reached end-of-file, but the problem is that the reachability handler is not called again on EOF.
I can’t find anything about how to get some kind of notification or callback on EOF. So what am I missing? Is Apple really providing an asynchronous reading API without an EOF callback?
By the way, I cannot use the runloop based readInBackgroundAndNotify method since I don’t have a runloop available. If I cannot get this to work with the NSFileHandle API I probably will directly use a dispatch source to do the IO.
I personally compare current file offset with current file position and stop reading.
extension FileHandle {
func stopReadingIfPassedEOF() {
let pos = offsetInFile
let len = seekToEndOfFile()
if pos < len {
// Resume reading.
seek(toFileOffset: pos)
}
else {
// Stop.
// File offset pointer stays at the EOF.
readabilityHandler = nil
}
}
}
I couldn't understand why it's been designed in this way for a long time, but now I think this could be intentional.
In my opinion, Apple basically defines FileHandle as an infinite stream, therefore, EOF is not well defined unless you close the file. FileHandle seem to be more like a "channel".
It's also unclear what happens if another process appends/delete some data to/from the file while you're reading from it. What would be the EOF in this case? As far as I find, there's no mention about this case in Apple documentation. As far as I know, there's no true exclusive file I/O lock in macOS like other Unix-like systems.
In my opinion, availableData can return empty data at any time if I/O is not fast enough, and readabilityHandler just don't care about EOF.
I believe the accepted answer is actually incorrect. The readabilityHandler is indeed called when EOF is reached. That is signaled by having availableData be of 0 size.
Here’s a simple playground that attests to this.
import Foundation
import PlaygroundSupport
let pipe = Pipe()
pipe.fileHandleForReading.readabilityHandler = { fh in
let d = fh.availableData
print("Data length: \(d.count)")
if (d.count == 0) {
fh.readabilityHandler = nil
}
}
pipe.fileHandleForWriting.write("Hello".data(using: .utf8)!)
pipe.fileHandleForWriting.closeFile()
PlaygroundPage.current.needsIndefiniteExecution = true
I'm afraid you're out of luck doing this with NSFileHandle if you can't use readInBackgroundAndNotify.
Two solutions I see:
Create a runloop and then use readInBackgroundAndNotify.
Roll your own implementation using dispatch_io_*