Reading NSTask output persistent memory usage - objective-c

I have a simple NSTask which runs a shell command, and a NSPipe file handle for reading which is used to read the output of the command and writes out to a file. When this command terminates and the output is written to file, Activity Monitor and Xcode memory monitor shows 700mb of usage which never goes down. I have wrapped the NSData output in an autoreleasepool and the whole task is in it's own function so there is no reason I can see for the high usage when the task is finished. I have profiled this with instruments and there are apparently no leaks and only a few KB of persistent memory after the command runs.
- (void)extractLogs forDate:(NSDate *)date
{
__block NSError* err;
NSDate *collectionEndDate = [[NSCalendar currentCalendar]dateByAddingUnit:NSCalendarUnitDay
value:1
toDate:_collectionStartDate
options:0];
[NSString stringWithFormat:#"%#",collectionEndDate];
NSPipe *pipe = [NSPipe new];
NSTask *logRetrievalTask = [[NSTask alloc]init];
[logRetrievalTask setLaunchPath:#"/bin/bash"];
NSString *cmd = [[NSString alloc]initWithFormat:#"/usr/bin/log show --signpost --no-pager --source --style syslog --start %# --end %# --predicate \"%#\" | %#",
[_collectionStartDate logFormattedDateString],
[collectionEndDate logFormattedDateString],
[NSString stringWithFormat:#"subsystem == '%#'", "com.xxx.xxxxx"],
LOG_TRANSFORM_COMMAND];
[logRetrievalTask setArguments:#[#"-c",
cmd]];
[logRetrievalTask setStandardOutput:pipe];
NSURL *logFileUrl = [self logFilePathForSubsystem:system usingDateString:[_collectionStartDate sduLogFormattedDateString]];
__block FileStreamCompressor *compressor = [[FileStreamCompressor alloc]
initWithFilePath:[logFileUrl path]
maxFileSize:MB_IN_BYTES([self maxFileSize])
currentTotalBytes: _totalLogBytes];
if (compressor == nil)
{
[[self logger]indicateFailureWith:[NSString stringWithFormat:#"Error: Could not create log file for: %#", system]];
return;
}
[logRetrievalTask launch];
NSData *logData = [[pipe fileHandleForReading]readDataToEndOfFile];
[logRetrievalTask terminate];
if ([logData length] == LOG_COMMAND_OUTPUT_HEADER_SIZE)
{
[compressor finishWritingStreamToFile];
[self removeLogFileForSystem:system
usingDateString:[_collectionStartDate logFormattedDateString]
logFileErrorType:#"Empty log file"];
} else {
_totalLogBytes = [compressor compressData:logData error:&err];
if (err != nil) {
[compressor finishWritingStreamToFile];
[[self logger]indicateWarningWith:[err localizedDescription]];
[self removeLogFileForSystem:system
usingDateString:[_collectionStartDate logFormattedDateString]
logFileErrorType:#"Corrupt logfile"];
} else {
[[self logger]indicateSuccessWith:[NSString stringWithFormat:#"Finished processing log: %#-%#.log.gz",
system,
[_collectionStartDate logFormattedDateString]]];
}
}
[[pipe fileHandleForReading]closeFile];
[[pipe fileHandleForWriting]closeFile];
}

Related

OSX Xcode: Sandbox Error when creating DMG

I have an OSX xcode project, which works perfectly until I turn the sandbox on.
Under Capabilities, I have turn on Sandbox, and File access:
User Selected File - Read/Write
My project is like this, when user clicks on a button, the NSSavePanel will popup, user specifies the save directory and file name (ie. myfile.dmg), and once he or she clicks on save, the program will create dmg file using NSTask and hdiutil to the specific path. Here is my code:
__block NSSavePanel *saveDlg = [NSSavePanel savePanel];
// Enable options in the dialog.
[saveDlg setCanCreateDirectories:YES];
[saveDlg setAllowedFileTypes:[NSArray arrayWithObject:#"dmg"]];
[saveDlg setCanSelectHiddenExtension:YES];
[saveDlg beginSheetModalForWindow:self.view.window completionHandler:^ (NSInteger result) {
if (result == NSFileHandlingPanelOKButton) {
BOOL bSuccess = [self checkErrorHandler: saveDlg.URL optionToCheck: 8];
if (bSuccess) {
[self createDMG:saveDlg.URL];
}
}
saveDlg = nil;
}];
Here is my createDMG method:
- (void)createDMG:(NSURL *)myurl{
self.url = myurl;
self.createTask = [[NSTask alloc] init];
NSPipe *outputPipe = [NSPipe pipe];
NSPipe *errorPipe = [NSPipe pipe];
NSPipe *writePipe = [NSPipe pipe];
self.fhCreateOutput = [outputPipe fileHandleForReading];
self.fhCreateError = [errorPipe fileHandleForReading];
NSNotificationCenter *nc = [NSNotificationCenter defaultCenter];
[nc addObserver:self selector:#selector(createNotifiedForStdOutput:) name:NSFileHandleReadCompletionNotification object:self.fhCreateOutput];
[nc addObserver:self selector:#selector(createNotifiedForStdError:) name:NSFileHandleReadCompletionNotification object:self.fhCreateError];
[nc addObserver:self selector:#selector(createNotifiedForComplete:) name:NSTaskDidTerminateNotification object:self.createTask];
NSFileHandle *writeHandle = [writePipe fileHandleForWriting];
NSString *pw = [self getPlainTextPassword:nil];
[writeHandle writeData: [[NSString stringWithFormat:#"%#\0\n", pw] dataUsingEncoding:NSUTF8StringEncoding]];
[writeHandle closeFile];
[self.createTask setStandardInput:writePipe];
NSString *directoryPath = [self.url.path stringByDeletingLastPathComponent];
NSString *fileName = [[self.url absoluteString] lastPathComponent];
[self.createTask setLaunchPath: #"/usr/bin/hdiutil"];
[self.createTask setCurrentDirectoryPath:directoryPath];
[self.createTask setArguments:[NSArray arrayWithObjects:#"create", #"-puppetstrings", #"-encryption",#"AES-128", #"-size", #"100m", #"-verbose", #"-volname", #"test", fileName, #"-fs", #"HFS+J", #"-stdinpass", nil]];
[self.createTask setStandardOutput: outputPipe];
[self.createTask setStandardError:errorPipe];
[self.createTask launch];
[self.fhCreateOutput readInBackgroundAndNotify];
[self.fhCreateError readInBackgroundAndNotify];
}
and I got the following error:
2015-02-18 10:51:47.374 My DMG[41725:2016417] -[MyDMG
createNotifiedForStdError:] verbose: 2015-02-18 10:51:46.745 diskimages-helper[41741:2016793] ERROR: couldn't connect to framework.
2015-02-18 10:51:47.376 My DMG[41725:2016417] -[MyDMG createNotifiedForStdError:] verbose: 2015-02-18 10:51:46.745 diskimages-helper[41741:2016793] DIHelper: setupConnectionToFrameworkWithUUID: failed
2015-02-18 10:51:46.747 hdiutil[41739:2016783] helper died
DIHLDiskImageCreate() returned 10
(null)
2015-02-18 10:51:47.665 My DMG[41725:2016417] -[MyDMG createNotifiedForStdError:] verbose: hdiutil: create: returning 10
2015-02-18 10:51:47.682 My DMG[41725:2016417] -[MyDMG createNotifiedForStdError:] verbose: hdiutil: create failed - No child processes
2015-02-18 10:51:47.687 My DMG[41725:2016417] task completed or was stopped with exit code 1
Some people said that NSTask doesn't work well with sandbox, but I have a couple other project that runs NSTask in sandbox and they worked. Also if you look at the mac app store, there is a similar app which creates dmg (user drags a folder and dmg file created)
Any help is appreciated :)

echo Does Not Work with NSTask and readInBackgroundAndNotify

I have the following Obj-C code and its log output. Can anyone tell me why I'm not getting any output from the NSFileHandle?
#implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
[self performSelectorInBackground:#selector(startTask:) withObject:nil];
}
- (void) startTask: (id) sender
{
NSPipe *pipe = [[NSPipe alloc] init];
NSFileHandle *fh = pipe.fileHandleForReading;
[fh readInBackgroundAndNotify];
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(output:) name:NSFileHandleReadCompletionNotification object:fh];
NSTask *echoTask = [[NSTask alloc] init];
echoTask.standardOutput = pipe;
echoTask.standardError = [[NSPipe alloc] init];
echoTask.launchPath = #"/bin/echo";
echoTask.arguments = #[#"hello world!"];
NSLog(#"launching...");
[echoTask launch];
[echoTask waitUntilExit];
NSLog(#"finished.");
}
- (void) output:(NSNotification *)notification
{
NSFileHandle *fh = notification.object;
NSLog(#"fh: %#", fh);
NSString *output = [[NSString alloc] initWithData:[fh readDataToEndOfFile] encoding:NSUTF8StringEncoding];
NSLog(#"output: '%#'", output);
}
#end
log:
2014-12-16 10:19:58.154 SubProcess2[14893:704393] launching...
2014-12-16 10:19:58.165 SubProcess2[14893:704393] fh: <NSConcreteFileHandle: 0x6080000e9e80>
2014-12-16 10:19:58.165 SubProcess2[14893:704393] output: ''
2014-12-16 10:19:58.166 SubProcess2[14893:704393] finished.
If I do it synchronously or using the approach in https://stackoverflow.com/a/16274541/1015200 I could get it to work.
Any other techniques and variations(such as launching task without performSelectorInBackground) have failed.
I really want to see if I can get it to work using the notification.
So if I can get any help that'd be great.
The data that has already been read is passed to the notification in the userInfo dictionary under the key NSFileHandleNotificationDataItem, you should be accessing that and not attempting to read further data. E.g. something like:
- (void) output:(NSNotification *)notification
{
NSString *output = [[NSString alloc]
initWithData:notification.userInfo[NSFileHandleNotificationDataItem]
encoding:NSUTF8StringEncoding];
NSLog(#"output: '%#'", output);
}
HTH

NSTask for SSH using PTY

I'm trying to write an app that will programmatically log in to a remote device using SSH much like an expect script (I know I can use expect but I would like to do this in Obj-c).
I have researched a lot on this and know that I need to use a pty. The code I have works fine for telnet but I can't seem to get ssh to work. It seems as though SSH is not using the pty to ask for the password. When I execute the following code I see the device asking for the password, but I don't see my NSLog output.
I'm very new to this and probably over my head, but I'd really appreciate anyone who can help me get this working.
#import <Foundation/Foundation.h>
#import <util.h>
#interface NSTask (PTY)
- (NSFileHandle *)masterSideOfPTYOrError:(NSError **)error;
#end
#implementation NSTask (PTY)
- (NSFileHandle *)masterSideOfPTYOrError:(NSError *__autoreleasing *)error {
int fdMaster, fdSlave;
int rc = openpty(&fdMaster, &fdSlave, NULL, NULL, NULL);
if (rc != 0) {
if (error) {
*error = [NSError errorWithDomain:NSPOSIXErrorDomain code:errno userInfo:nil];
}
return NULL;
}
fcntl(fdMaster, F_SETFD, FD_CLOEXEC);
fcntl(fdSlave, F_SETFD, FD_CLOEXEC);
NSFileHandle *masterHandle = [[NSFileHandle alloc] initWithFileDescriptor:fdMaster closeOnDealloc:YES];
NSFileHandle *slaveHandle = [[NSFileHandle alloc] initWithFileDescriptor:fdSlave closeOnDealloc:YES];
self.standardInput = slaveHandle;
self.standardOutput = slaveHandle;
return masterHandle;
}
#end
int main(int argc, const char * argv[])
{
#autoreleasepool {
NSTask *task = [[NSTask alloc] init];
[task setLaunchPath:#"/usr/bin/ssh"];
[task setArguments:#[#"user#192.168.1.1"]];
NSError *error;
NSFileHandle *masterHandle = [task masterSideOfPTYOrError:&error];
if (!masterHandle) {
NSLog(#"error: could not set up PTY for task: %#", error);
exit(0);
}
[task launch];
[masterHandle waitForDataInBackgroundAndNotify];
NSMutableString *buff = [[NSMutableString alloc] init];
[[NSNotificationCenter defaultCenter] addObserverForName:NSFileHandleDataAvailableNotification
object:masterHandle queue:nil
usingBlock:^(NSNotification *note)
{
NSData *outData = [masterHandle availableData];
NSString *outStr = [[NSString alloc] initWithData:outData encoding:NSUTF8StringEncoding];
[buff appendString:outStr];
NSLog(#"output: %#", outStr);
NSRegularExpression *regex = [NSRegularExpression regularExpressionWithPattern:#"sername:"
options:NSRegularExpressionCaseInsensitive
error:nil];
NSTextCheckingResult *match = [regex firstMatchInString:buff
options:0
range:NSMakeRange(0, [buff length])];
if (match) {
NSLog(#"got a match!!");
[buff setString:#""];
[masterHandle writeData:[#"bhughes\n" dataUsingEncoding:NSUTF8StringEncoding]];
}
NSLog(#"Exiting function.\n");
[masterHandle waitForDataInBackgroundAndNotify];
}];
[task waitUntilExit];
NSLog(#"Program complete.\n");
}
return 0;
}
As far as I know, NSTask does not support pty ability. Working with something like ssh requires interactive pty device context.
The simplest way to do this is forkpty, but this forks process itself, so it cannot be used with NSTask.
Finally I wrote a wrapper class that manages forkpty. That forks a child process and calls forkpty and execve.
Here's my implementation: https://github.com/eonil/PseudoTeletypewriter.Swift
You can read/write using single master device file handle.
I confirmed that sudo is working, and I believe ssh also should work fine.

Async execution of shell command not working properly

So, this is my code :
- (void)runCmd:(NSString *)cmd withArgs:(NSArray *)args
{
NSLog(#"\nRunning ::\n\tCmd : %#\n\tArgs : %#",cmd, args);
[theSpinner start];
if (task)
{
[task interrupt];
}
else
{
task = [[NSTask alloc] init];
[task setLaunchPath:cmd];
[task setArguments:args];
[pipe release];
pipe = [[NSPipe alloc] init];
[task setStandardOutput:pipe];
NSFileHandle* fh = [pipe fileHandleForReading];
NSNotificationCenter* nc;
nc = [NSNotificationCenter defaultCenter];
[nc removeObserver:self];
[nc addObserver:self
selector:#selector(dataReady:)
name:NSFileHandleReadCompletionNotification
object:fh];
[nc addObserver:self selector:#selector(dataAvailable:) name:NSFileHandleDataAvailableNotification object:fh];
[nc addObserver:self
selector:#selector(taskTerminated:)
name:NSTaskDidTerminateNotification
object:task];
[task launch];
[fh readInBackgroundAndNotify];
}
}
- (void)dataAvailable:(NSNotification*)n
{
NSLog(#"Data Available : %#",n);
}
- (void)dataReady:(NSNotification*)n
{
NSData* d;
d = [[n userInfo] valueForKey:NSFileHandleNotificationDataItem];
NSLog(#"Data Ready : %#",n);
if ([d length])
{
NSLog(#"Data Ready : %#",[[NSString alloc] initWithData:d encoding:NSUTF8StringEncoding]);
}
}
- (void) taskTerminated:(NSNotification*)note
{
NSLog(#"Task Terminated : %#",note);
[task release];
task = nil;
[theSpinner stop];
NSAlert *alert = [[[NSAlert alloc] init] autorelease];
[alert setMessageText:[NSString stringWithFormat:#"Command finished"]];
[alert runModal];
}
I've tried running a command (e.g. the php interpreter at /usr/bin/php) with arguments (e.g. the file to be interpreted by php test.php).
The thing is :
The script runs fine
BUT, I'm receiving a Data Ready and Task Terminated
notification BEFORE I've managed to get all the output. (I mean,
the dataReady: function fetches just the first part of the
output and the rest of it is nowhere to be found...)
I basically want to be reading, asynchronously, all output - WHILE the command is running.
Any Ideas? What am I doing wrong?
Thanks!
You use readInBackgroundAndNotify to schedule your reading. This method reads only one buffer full of data and notifies. You either need to call readInBackgroundAndNotify in your notification method again to read more data or you need to use readToEndOfFileInBackgroundAndNotify if you want to receive all the data at once.
There's a new API since 10.7, so you can avoid using NSNotifications.
task.standardOutput = [NSPipe pipe];
[[task.standardOutput fileHandleForReading] setReadabilityHandler:^(NSFileHandle *file) {
NSData *data = [file availableData]; // this will read to EOF, so call only once
NSLog(#"Task output! %#", [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]);
// if you're collecting the whole output of a task, you may store it on a property
[self.taskOutput appendData:data];
}];
IMPORTANT:
When your task terminates, you have to set readabilityHandler block to nil; otherwise, you'll encounter high CPU usage, as the reading will never stop.
[task setTerminationHandler:^(NSTask *task) {
// do your stuff on completion
[task.standardOutput fileHandleForReading].readabilityHandler = nil;
[task.standardError fileHandleForReading].readabilityHandler = nil;
}];

Relaunching a cocoa app

I have an application that checks its command line parameters and stores values in persistent stores. One of those is a password that I don't want sticking around for people to see with 'ps' and friends. The approach I'm currently looking at is to, after I've stored the values I need, relaunch the process without the command line parameters. My naive approach is this, where args[0] is the path to the application:
NSTask *task = [[NSTask alloc] init];
[task setLaunchPath:[args objectAtIndex:0]];
[task launch];
[task release];
[NSApp terminate:nil];
The child is run. However, when my app is terminated the child doesn't seem to orphan but gets stuck. Am I just way off on this one?
More info: So it seems that when I call [NSApp terminate:nil] the NSTask that was launched gets stuck, but if I just exit() then it works fine. However, I'm concerned that things that are open (keychain, plist, etc.) will be in a bad state if I do that.
And note that lots of example code out there is about some watchdog-like process that restarts a separate process when needed. I'm trying to restart the current process that's already running from within that same process.
There are plenty of examples on the web, but this one (also below) looks like it has all the code you need. There are more detailed explanations out there, as well.
// gcc -Wall -arch i386 -arch ppc -mmacosx-version-min=10.4 -Os -framework AppKit -o relaunch relaunch.m
#import <AppKit/AppKit.h>
#interface TerminationListener : NSObject
{
const char *executablePath;
pid_t parentProcessId;
}
- (void) relaunch;
#end
#implementation TerminationListener
- (id) initWithExecutablePath:(const char *)execPath parentProcessId:(pid_t)ppid
{
self = [super init];
if (self != nil) {
executablePath = execPath;
parentProcessId = ppid;
// This adds the input source required by the run loop
[[[NSWorkspace sharedWorkspace] notificationCenter] addObserver:self selector:#selector(applicationDidTerminate:) name:NSWorkspaceDidTerminateApplicationNotification object:nil];
if (getppid() == 1) {
// ppid is launchd (1) => parent terminated already
[self relaunch];
}
}
return self;
}
- (void) applicationDidTerminate:(NSNotification *)notification
{
if (parentProcessId == [[[notification userInfo] valueForKey:#"NSApplicationProcessIdentifier"] intValue]) {
// parent just terminated
[self relaunch];
}
}
- (void) relaunch
{
[[NSWorkspace sharedWorkspace] launchApplication:[NSString stringWithUTF8String:executablePath]];
exit(0);
}
#end
int main (int argc, const char * argv[])
{
if (argc != 3) return EXIT_FAILURE;
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
[[[TerminationListener alloc] initWithExecutablePath:argv[1] parentProcessId:atoi(argv[2])] autorelease];
[[NSApplication sharedApplication] run];
[pool release];
return EXIT_SUCCESS;
}
I know its a bit late to answer but this answer may help others. Here is a cool trick that can help you.
By using the terminal command, just open your application as a new instance and terminate the current instance.
This is how it is done:
....
NSString* path = [[NSBundle mainBundle] bundlePath];
NSString* cmd = [NSString stringWithFormat:#"open -n %#", path];
[self runCommand:cmd];
exit(0);
}
/// temrinal function
-(NSString*)runCommand:(NSString*)commandToRun;
{
NSTask *task;
task = [[NSTask alloc] init];
[task setLaunchPath: #"/bin/sh"];
NSArray *arguments = [NSArray arrayWithObjects:
#"-c" ,
[NSString stringWithFormat:#"%#", commandToRun],
nil];
NSLog(#"run command: %#",commandToRun);
[task setArguments: arguments];
NSPipe *pipe;
pipe = [NSPipe pipe];
[task setStandardOutput: pipe];
NSFileHandle *file;
file = [pipe fileHandleForReading];
[task launch];
NSData *data;
data = [file readDataToEndOfFile];
NSString *output;
output = [[NSString alloc] initWithData: data encoding: NSUTF8StringEncoding];
return output;
}
Create an external process that launches yours when it terminates. Then terminate. Launching Cocoa programs with NSTask doesn't work quite right.
For anyone who still wants to use NSTask to relaunch,I found one possible way: Please DO NOT set NSPipe of the NSTask,because the NSTask will terminate the app itself,once the app terminated,the NSTask that started might get stuck there.
At least for me,after I removed the NSPipe settings,my app relaunch successfully.
The following is what I do:
1. Write a command line tool which has 3 parameters: app bundle id,app full path,please note that in this command line you need to terminate the app and wait for a while to make sure it is really terminated before launch the new one,I keep checking app.isTerminated and sleep(1) if it's not terminated.
Launch the Command line tool in app using NSTask,and set the parameters accorddingly,Don't use NSPipe,simply create NSTask and launch
The app relaunches now