How do I get something similar to Tail -f using NSTask - objective-c

I need to read the last added line to a log file, in realtime, and capture that line being added.
Something similar to Tail -f.
So my first attempt was to use Tail -f using NSTask.
I can't see any output using the code below:
NSTask *server = [[NSTask alloc] init];
[server setLaunchPath:#"/usr/bin/tail"];
[server setArguments:[NSArray arrayWithObjects:#"-f", #"/path/to/my/LogFile.txt",nil]];
NSPipe *outputPipe = [NSPipe pipe];
[server setStandardInput:[NSPipe pipe]];
[server setStandardOutput:outputPipe];
[server launch];
[server waitUntilExit];
[server release];
NSData *outputData = [[outputPipe fileHandleForReading] readDataToEndOfFile];
NSString *outputString = [[[NSString alloc] initWithData:outputData encoding:NSUTF8StringEncoding] autorelease];
NSLog (#"Output \n%#", outputString);
I can see the output as expected when using:
[server setLaunchPath:#"/bin/ls"];
How can i capture the output of that tail NSTask?
Is there any alternative to this method, where I can open a stream to file and each time a line is added, output it on screen? (basic logging functionality)

This is a little tricky to do your way, as readDataToEndOfFile will wait until tail closes the output stream before returning, but tail -f never closes the output stream (stdout). However, this is actually pretty simple to do with basic C I/O code, so I whipped up a simple FileTailer class that you can check out. It's not anything fancy, but it should show you how it's done. Here're the sources for FileTailer.h, FileTailer.m, and a test driver.
The meat of the class is pretty simple. You pass it a block, and it reads a character from the stream (if possible) and passes it to the block; if EOF has been reached, it waits a number of seconds (determined by refresh) and then tries to read the stream again.
- (void)readIndefinitely:(void (^)(int ch))action
{
long pos = 0L;
int ch = 0;
while (1) {
fseek(in, pos, SEEK_SET);
int ch = fgetc(in);
pos = ftell(in);
if (ch != EOF) {
action(ch);
} else {
[NSThread sleepForTimeInterval:refresh];
}
}
}
You can call it pretty simply, like this:
FileTailer *tail = [[[FileTailer alloc] initWithStream:stdin refreshPeriod:3.0] autorelease];
[tail readIndefinitely:^ void (int ch) { printf("%c", ch); }];
(Caveat: I wrote the FileTailer class pretty fast, so it's kind of ugly right now and should be cleaned up a bit, but it should serve as a decent example on how to read a file indefinitely, à la tail -f.)

Here's a way to use "tail -f logfile" via NSTask in Objective-C:
asynctask.m -- sample code that shows how to implement asynchronous stdin, stdout & stderr streams for processing data with NSTask
...
Being a GUI-less application (i.e. a Foundation-based command line tool), asynctask.m runs an NSRunLoop manually
to enable the use of asynchronous "waitForDataInBackgroundAndNotify" notifications. In addition, asynctask.m
uses pthread_create(3) and pthread_detach(3) for writing more than 64 KB to the stdin of an NSTask.
Source code of asynctask.m available at: http://www.cocoadev.com/index.pl?NSPipe

Related

NSTask Race Condition With ReadabilityHandler Block

Basic Setup
I use NSTask to run a process that optimizes images. This process writes output data to stdout. I use the readabilityHandler property of NSTask to capture that data. Here is the abbreviated setup:
NSTask *task = [[NSTask alloc] init];
[task setArguments:arguments]; // arguments defined above
NSPipe *errorPipe = [NSPipe pipe];
[task setStandardError:errorPipe];
NSFileHandle *errorFileHandle = [errorPipe fileHandleForReading];
NSPipe *outputPipe = [NSPipe pipe];
[task setStandardOutput:outputPipe];
NSFileHandle *outputFileHandle = [outputPipe fileHandleForReading];
NSMutableData *outputData = [[NSMutableData alloc] init];
NSMutableData *errorOutputData = [[NSMutableData alloc] init];
outputFileHandle.readabilityHandler = ^void(NSFileHandle *handle) {
NSLog(#"Appending data for %#", inputPath.lastPathComponent);
[outputData appendData:handle.availableData];
};
errorFileHandle.readabilityHandler = ^void(NSFileHandle *handle) {
[errorOutputData appendData:handle.availableData];
};
I then call NSTask like this:
[task setLaunchPath:_pathToJPEGOptim];
[task launch];
[task waitUntilExit];
(This is all done on a background dispatch queue). Next I examine the return values of NSTask:
if ([task terminationStatus] == 0)
{
newSize = outputData.length;
if (newSize <= 0)
{
NSString *errorString = [[NSString alloc] initWithData:errorOutputData encoding:NSUTF8StringEncoding];
NSLog(#"ERROR string: %#", errorString);
}
// Truncated for brevity...
}
The Problem
Approximately 98% of the time, this works perfectly. However, it appears that -waitUntilExit CAN fire before the readabilityHandler block is run. Here is a screenshot showing that the readability handler is running AFTER the task has exited:
So this is clearly a race condition between the dispatch queue running the readabilityHandler and the dispatch queue where I've fired off my NSTask. My question is: how the hell can I determine that the readabilityHandler is done? How do I beat this race condition if, when NSTask tells me it's done, it may not be done?
NOTE:
I am aware that NSTask has an optional completionHandler block. But the docs state that this block is not guaranteed to run before -waitUntilExit returns, which implies that it CAN begin running even SOONER than -waitUntilExit. This would make the race condition even more likely.
Update: Modern macOS:
availableData no longer has the issues I describe below. I'm unsure precisely when they were resolved, but at least Monterey works correctly. The approach described below is for older releases of macOS.
Additionally, with the modern Swift concurrency system in place and the new paradigm of "threads can always make forward progress", using semaphores like below should be a last resort. If you can, use NSTask's completionHandler API. I have no FORMAL guarantee that the readability handlers will complete before the completionHandler is called, but they seem to in practice, at least on modern macOS. Your mileage may vary.
Old Advice:
Ok, after much trial-and-error, here's the correct way to handle it:
1. Do not Use -AvailableData
In your readability handler blocks, do not use the -availableData method. This has weird side effects, will sometimes not capture all available data, and will interfere with the system's attempt to call the handler with an empty NSData object to signal the closing of the pipe because -availableData blocks until data is actually available.
2. Use -readDataOfLength:
Instead, use -readDataOfLength:NSUIntegerMax in your readability handler blocks. With this approach, the handler correctly receives an empty NSData object that you can use to detect the closing of the pipe and signal a semaphore.
3. Beware macOS 10.12!
There is a bug that Apple fixed in 10.13 that is absolutely critical here: on old versions of macOS, the readability handlers are never called if there is no data to read. That is, they never get called with zero-length data to indicate that they’re finished. That results in a permanent hang using the semaphore approach because the semaphore is never incremented. To combat this, I test for macOS 10.12 or below and, if I’m running on an old OS, I use a single call to dispatch_semaphore_wait() that is paired with a single call to dispatch_semaphore_signal() in NSTask’s completionHandler block. I have that completion block sleep for 0.2 seconds to allow the handlers to execute. That’s obviously a godawfully ugly hack, but it works. If I’m on 10.13 plus, I have different readability handlers that signal the semaphore (once from the error handler and once from the normal output handler) and I still signal the semaphore from the completionHandler block. These are paired with 3 calls to dispatch_semaphore_wait() after I launch the task. In this case, no delay is required in the completion block because macOS correctly calls the readability handlers with zero-length data when the fileHandle is done.
Example:
(Note: assume stuff is defined as in my original question example. This code is shortened for readability.)
// Create the semaphore
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
// Define a handler to collect output data from our NSTask
outputFileHandle.readabilityHandler = ^void(NSFileHandle *handle)
{
// DO NOT use -availableData in these handlers.
NSData *newData = [handle readDataOfLength:NSUIntegerMax];
if (newData.length == 0)
{
// end of data signal is an empty data object.
outputFileHandle.readabilityHandler = nil;
dispatch_semaphore_signal(sema);
}
else
{
[outputData appendData:newData];
}
};
// Repeat the above for the 'errorFileHandle' readabilityHandler.
[task launch];
// two calls to wait because we are going to signal the semaphore once when
// our 'outputFileHandle' pipe closes and once when our 'errorFileHandle' pipe closes
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
// ... do stuff when the task is done AND the pipes have finished handling data.
// After doing stuff, release the semaphore
dispatch_release(sema);
sema = NULL;
Create a semaphore with an initial value of 0. In the readability handlers, check if the data object returned from availableData has length 0. If it does, that means end of file. In that case, signal the semaphore.
Then, after waitUntilExit returns, wait on the semaphore twice (once for each pipe you're reading). When those waits return, you've got all of the data.

Run user defined command with NSTask

I would like to execute a terminal command specified by the user. For example, the user might write killall "TextEdit" or say "Hello world!" in a text field, and I want to execute that command.
NSTask is the way to go, except I have two problems with it:
First: the arguments. Right now I'm doing this:
NSArray* args = [commandString componentsSeparatedByString: #" "];
[task setArguments: [args subarrayWithRange: NSMakeRange(1, [args count] - 1)]]; // First one is the command name
Is this the way to do it? I don't think I've had problems with this yet, but I doesn't look like it's safe. Imagine this: the user writes killall 'Address Book' but the command receives as arguments 'Address and Book'?? That doesn't work. So, what should I do instead? How can I safely parse the arguments?
Second: the launch path. It's much more user-friendly to only have to write the name of the command, instead of the complete path to it. So I want to support that, which means finding out programmatically the full path for a command having only it's name. For that I wrote a category on NSTask like this:
+ (NSString*)completePathForExec: (NSString*)exec
{
NSTask* task = [[NSTask alloc] init];
NSPipe* pipe = [[NSPipe alloc] init];
NSArray* args = [NSArray arrayWithObject: exec];
[task setLaunchPath: #"/usr/bin/which"];
[task setArguments: args];
[task setStandardOutput: pipe];
[task setStandardError: pipe];
[task launch];
[task waitUntilExit];
NSFileHandle* file = [pipe fileHandleForReading];
NSString* result = [[NSString alloc] initWithData: [file readDataToEndOfFile] encoding: NSASCIIStringEncoding];
if ([result length]) {
if ([result hasSuffix: #"\n"]) { result = [result substringWithRange: NSMakeRange(0, [result length] - 1)]; }
return result;
}
else { return exec; }
}
This seems to works well. However, how can I be sure that this path: /usr/bin/which will always work? I mean: will it work on 10.6, 10.7, 10.8, etc? I think I had a problem once where the path to a shell command changed with the system version, and you can never be too careful.
If the path is guaranteed to stay the same, then this isn't a problem. If it changes, then how can I know the 'path to the path-finder'?
It'll be far easier for you to not re-invent the command line parsing wheel. But, of course, going down the route of executing arbitrary user entered code is a security nightmare (tempered by the fact that the user has access to the system and, thus, could probably just run Terminal directly).
Specifically, have NSTask wrap an invocation of one of the shells with the command line option to have it execute an arbitrary string.
sh -c "ls -alF"
This would allow you to pass the path to sh as your launch path, which is in a fixed location on every system. The #"-c" argument tells sh to parse the next argument as a script and, of course, the next argument is whatever the user entered.
Note, this will also give the user the ability to pipe stuff, too.

InterProcess Communication on MacOSX Lion

I'm trying to figure out how to set up IPC between my custom app and a pre-made program.
I'm using MacOSX Lion 10.7.2 and Xcode 4.2.1.
It doesn't matter actually what program exactly, since I believe that a similar reasoning may be applied to any kind of external process.
For testing purposes I'm using a simple bash script:
#test.sh
echo "Starting"
while read out
do
echo $out
done
What I would like to achieve is to redirect input and output of this script, using my app to send inputs to it and read its outputs.
I tried to use NSTask,NSPipe and NSFileHandle as follows:
-(void)awakeFromNib {
task = [[NSTask alloc] init];
readPipe = [NSPipe pipe];
writePipe = [NSPipe pipe];
[task setStandardOutput:readPipe];
[task setStandardInput:writePipe];
[task setLaunchPath:#"/path/test.sh"];
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(read:)
name:NSFileHandleReadCompletionNotification
object:nil];
[[readPipe fileHandleForReading] readInBackgroundAndNotify];
[task launch];
}
-(IBAction)write:(id)sender {
NSLog(#"Write called: %d %#\n",[task isRunning],writePipe);
NSFileHandle *writeHandle = [writePipe fileHandleForWriting];
NSString *message = #"someString";
[writeHandle writeData:[message dataUsingEncoding:NSUTF8StringEncoding] ];
}
-(void)read:(NSNotification*)notification {
NSString *output = [[NSString alloc] initWithData:[[notification userInfo] valueForKey: NSFileHandleNotificationDataItem]
encoding:NSUTF8StringEncoding];
NSLog(#"%#",output);
[output release];
[[notification object] readInBackgroundAndNotify];
}
but I'm able only to read the output of test.sh, not to send it any input.
Actually any other example I saw on the web is pretty similar to my code, so I'm not sure if this issue is due to some mistake(s) of mine or to other issues (like app's sandboxing of MacOS Lion).
I've checked XPC documentation, but, according to my researches, in order to use XPC API to IPC, both sides should connect to the same service.
That's not what I'm looking for since I don't want to alter the script in any way, I just want redirect its input and output.
Is my issue due to the lack of XPC and/or to app's sandboxing?
If yes, is there a way to use XPC without modifying the script?
If no, then may somebody explain me what I'm doing wrong?
You don't need XPC for this. Won't make any difference.
Is your script / external process able to read input when you pipe something to it on the command line
% echo "foobar" | /path/test.sh
?
How much data are you sending to it. Writing will be buffered. IIRC -synchronizeFile will flush buffers -- same as fsync(2).

Write to NSTasks standard input after launch

I am currently trying to wrap my head around the hole NSTask, NSPipe, NSFileHandle business. So I thought I write a little tool, which can compile and run C code. I also wanted to be able to redirect my stdout and stdin to a text view.
Here is what I got so far.
I used code from this post to redirect my stdio: What is the best way to redirect stdout to NSTextView in Cocoa?
NSPipe *inputPipe = [NSPipe pipe];
// redirect stdin to input pipe file handle
dup2([[inputPipe fileHandleForReading] fileDescriptor], STDIN_FILENO);
// curInputHandle is an instance variable of type NSFileHandle
curInputHandle = [inputPipe fileHandleForWriting];
NSPipe *outputPipe = [NSPipe pipe];
NSFileHandle *readHandle = [outputPipe fileHandleForReading];
[readHandle waitForDataInBackgroundAndNotify];
// redirect stdout to output pipe file handle
dup2([[outputPipe fileHandleForWriting] fileDescriptor], STDOUT_FILENO);
// Instead of writing to curInputHandle here I would like to do it later
// when my C program hits a scanf
[curInputHandle writeData:[#"123" dataUsingEncoding:NSUTF8StringEncoding]];
NSTask *runTask = [[[NSTask alloc] init] autorelease];
[runTask setLaunchPath:target]; // target was declared earlier
[runTask setArguments:[NSArray array]];
[runTask launch];
NSNotificationCenter *center = [NSNotificationCenter defaultCenter];
[center addObserver:self selector:#selector(stdoutDataAvailable:) name:NSFileHandleReadCompletionNotification object:readHandle];
And here the stdoutDataAvailable method
- (void)stdoutDataAvailable:(NSNotification *)notification
{
NSFileHandle *handle = (NSFileHandle *)[notification object];
NSString *str = [[NSString alloc] initWithData:[handle availableData] encoding:NSUTF8StringEncoding];
[handle waitForDataInBackgroundAndNotify];
// consoleView is an NSTextView
[self.consoleView setString:[[self.consoleView string] stringByAppendingFormat:#"Output:\n%#", str]];
}
This Program is working just fine. It is running the C program printing the stdout to my text view and reading "123" from my inputPipe. Like indicated in my comment above I would like to provide the input while the task is running once it is needed.
So there are two questions now.
Is there a way to get a notification as soon as somebody tries to read data from my inputPipe?
If the answer to 1 is no, is there a different approach I can try? Maybe using a class other than NSTask?
Any help, sample code, links to other resources are highly appreciated!
I'm not sure whether you can detect a "pull" on an NSPipe. I do have a vague sense that polling for write-availability with select() or using kqueue to look for I/O availability events on the underlying file descriptor of your NSFileHandle might do the trick, but I'm not very familiar with using those facilities in this way.
Do you have to support arbitrary C programs, or is it a special daemon or something you've developed?
If it's your own program, you could watch for requests for feedback on outputPipe, or just blast input onto the inputPipe as you find out what it is you want to send, and let the C program consume it when it's ready; if it's somebody else's code, you may be able to hook scanf and friends using a link-time method (since it's code you're compiling) like the one described in Appendix A-4 of:
http://www.cs.umd.edu/Library/TRs/CS-TR-4585/CS-TR-4585.pdf
The gist of it is to make a .dylib with your custom I/O functions (which may send some sigil to your app indicating that they need input), link that into the built program, set an environment variable (DYLD_BIND_AT_LAUNCH=YES) for the launched task, and run it. Once you've got those hooks in, you can provide whatever conveniences you want for your host program.

Getting exit status after launching app with NSWorkspace launchApplicationAtURL

I'm kind of new at Mac programming. I am porting a plugin to OSX. I need my application to launch a second app (which I do not control the source for) and then get its exit code. NSWorkspace launchApplicationAtURL works great to launch it with the needed arguments but I can't see how to get the exit code. Is there a way to get it after setting up notification for termination of the second app? I see tools for getting an exit code using NSTask instead. Should I be using that?
The NSWorkspace methods are really for launching independent applications; use NSTask to "run another program as a subprocess and ... monitor that program’s execution" as per the docs.
Here is a simple method to launch an executable and return its standard output - it blocks waiting for completion:
// Arguments:
// atPath: full pathname of executable
// arguments: array of arguments to pass, or nil if none
// Return:
// the standard output, or nil if any error
+ (NSString *) runCommand:(NSString *)atPath withArguments:(NSArray *)arguments
{
NSTask *task = [NSTask new];
NSPipe *pipe = [NSPipe new];
[task setStandardOutput:pipe]; // pipe standard output
[task setLaunchPath:atPath]; // set path
if(arguments != nil)
[task setArguments:arguments]; // set arguments
[task launch]; // execute
NSData *data = [[pipe fileHandleForReading] readDataToEndOfFile]; // read standard output
[task waitUntilExit]; // wait for completion
if ([task terminationStatus] != 0) // check termination status
return nil;
if (data == nil)
return nil;
return [NSString stringWithUTF8Data:data]; // return stdout as string
}
You may not want to block, especially if this is your main UI thread, supply standard input etc.
In fact, this property of the NSTask should do the trick: terminationStatus
From Apple's doc:
Returns the exit status returned by the receiver’s executable.
(int)terminationStatus
I tested it and it works ok. Watch out to test if the task is running first, otherwise an exception will be launched.
if (![aTask isRunning]) {
int status = [aTask terminationStatus];
if (status == ATASK_SUCCESS_VALUE)
NSLog(#"Task succeeded.");
else
NSLog(#"Task failed.");
}
Hope it helps.