Storing arbitrary metadata with a plain-text file - objective-c

I am writing a text-editor and I would need to store a few pieces of information (generally just a few strings; the storage needn't be particularly durable) with each file the app saves (without that being part of the text-file as other apps might read it and the info is only specific to my app).
How would I go about this?
More info: I have a NSDocument set up and I would like to simply store a NSString instance variable as a per file meta-datum. Based on the answers below I've come up with this, which is currently buggy and causes the program to crash on startup:
#import <sys/xattr.h>
#interface MyDocument : NSDocument {
NSString *metadatum;
}
#implementation MyDocument
- (BOOL)writeToURL:(NSURL *)url ofType:(NSString *)type error:(NSError **)err
{
BOOL output = [super writeToURL:url ofType:type error:err];
if(!setxattr([[url path] cStringUsingEncoding:NSUTF8StringEncoding],
"eu.gampleman.xattrs.style",
[metadatum cStringUsingEncoding:NSUTF8StringEncoding],
sizeof(char) * [styleName length], 0, 0))
{
NSLog(#"Write failure");
}
return output;
}
- (BOOL)readFromURL:(NSURL *)url ofType:(NSString *)type error:(NSError **)err {
char *output;
ssize_t bytes = getxattr([[url path] cStringUsingEncoding:NSUTF8StringEncoding],
"eu.gampleman.xattrs.style", &output, 1024, 0, 0);
if (bytes > 0) {
metadatum = [[NSString alloc] initWithBytes:output length:bytes
encoding:NSUTF8StringEncoding]; // <- crashes here with "EXC_BAD_ACCESS"
}
return [super readFromURL:url ofType:type error: err];
}
// ...
// fairly standard -dataOfType:error: and
// -readFromData:ofType:error: implementations
PS: If your answer is really good (with sample code, etc.), I will award a 100rep bounty.

Use extended attributes. See setxattr().
Here's a sample call to write a string:
NSData* encodedString = [theString dataUsingEncoding:NSUTF8StringEncoding];
int rc = setxattr("/path/to/your/file", "com.yourcompany.yourapp.yourattributename", [encodedString bytes], [encodedString length], 0, 0);
if (rc)
/* handle error */;
To read a string:
ssize_t len = getxattr("/path/to/your/file", "com.yourcompany.yourapp.yourattributename", NULL, 0, 0, 0);
if (len < 0)
/* handle error */;
NSMutableData* data = [NSMutableData dataWithLength:len];
len = getxattr("/path/to/your/file", "com.yourcompany.yourapp.yourattributename", [data mutableBytes], len, 0, 0);
NSString* string = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
PS: Don't you have to set the bounty on the question before it's answered?

There are several places to store this kind of information on Mac. The most simple, of course, is to store it in your own separate metadata database. Of course there are challenges if the file moves. Since 10.6, however, you can use Bookmarks to address this problem. A Bookmark (see NSURL) allows you to keep a reference to a file even if it is moved (even across application restarts). Prior to 10.6 there was the Alias Manager, but it couldn't create new aliases; just read ones that Finder created.
The next common solution is to create metadata files. So if I have foo.txt, then you create a sibling .foo.txt.metadata to hold the extra info. Several trade-offs there if the files can be moved around.
Next is Spotlight, which can be used to attach arbitrary information to files. The actual tool here is xattr (see the man pages for setxattr and its relatives). These basically absorb several things that used to be done with Resource Forks (except xattr is supposed to just be metadata).

Related

Create file with fixed size in Cocoa

My MAC app need create file that has fixed size. Example : I want to create file has name : test.txt, fixed size : 1069 bytes. How can i do that? I use below code to write file :
NSError *err;
NSString* arrayText = [writeArray componentsJoinedByString: #"\n"];
[filemgr createFileAtPath:[NSString stringWithFormat:#"%#/test.txt",sd_url] contents:nil attributes:nil];
[arrayText writeToFile:[NSString stringWithFormat:#"%#/test.txt",sd_url] atomically:YES encoding:NSUTF8StringEncoding error:&err];
Thanks
This function will write some junk data to file.
+(BOOL)writeToFile:(NSString *)path withSize:(size_t)bytes;
{
FILE *file = fopen([path UTF8String], "wb");
if(file == NULL)
return NO;
void *data = malloc(bytes); // check for NULL!
if (data==NULL) {
return NO;
}
fwrite(data, 1, bytes, file);
fclose(file);
return YES;
}
If you dont want to use fwrite
+(BOOL)writeToFile:(NSString *)path withSize:(size_t)bytes;
{
void *data = malloc(bytes); // check for NULL!
if (data==NULL) {
return NO;
}
NSData *ldata = [NSData dataWithBytes:data length:bytes];
[ldata writeToFile:path atomically:NO];
return YES;
}
In order for the file to contain a certain number of bytes, you must write that many bytes to the file. There's no way to make a file's size be something other than the number of bytes it contains.
You can use FSAllocateFork (or fcntl with F_PREALLOCATE) to reserve space to write into. You'd use this, for example, if you were implementing your own download logic (if, for some reason, NSURLDownload wasn't good enough) and wanted to (a) make sure enough space is available for the file you're downloading and (b) grab it before something else does.
But, even that doesn't actually change the size of the file, just ensures (if successful) that your writes will not fail for insufficient space.
The only way to truly grow a file is to write to it.
The best way to do that is to use either -[NSData writeToURL:options:error:], which is the easy way if you have the data all ready at once, or NSFileHandle, which will enable you to write the data in chunks rather than having to build it all up in memory first.

Data corruption when reading realtime H.264 output from AVAssetWriter

I'm using some tricks to try to read the raw output of an AVAssetWriter while it is being written to disk. When I reassemble the individual files by concatenating them, the resulting file is the same exact number of bytes as the AVAssetWriter's output file. However, the reassembled file will not play in QuickTime or be parsed by FFmpeg because there is data corruption. A few bytes here and there have been changed, rendering the resulting file unusable. I assume this is occurring on the EOF boundary of each read, but it isn't consistent corruption.
I plan to eventually use code similar to this to parse out individual H.264 NAL units from the encoder to packetize them and send them over RTP, however if I can't trust the data being read from disk I might have to use another solution.
Is there an explanation/fix for this data corruption? And are there any other resources/links you have found on how to parse the NAL units to packetize over RTP?
Full code here: AVAppleEncoder.m
// Modified from
// http://www.davidhamrick.com/2011/10/13/Monitoring-Files-With-GCD-Being-Edited-With-A-Text-Editor.html
- (void)watchOutputFileHandle
{
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
int fildes = open([[movieURL path] UTF8String], O_EVTONLY);
source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE,fildes,
DISPATCH_VNODE_DELETE | DISPATCH_VNODE_WRITE | DISPATCH_VNODE_EXTEND | DISPATCH_VNODE_ATTRIB | DISPATCH_VNODE_LINK | DISPATCH_VNODE_RENAME | DISPATCH_VNODE_REVOKE,
queue);
dispatch_source_set_event_handler(source, ^
{
unsigned long flags = dispatch_source_get_data(source);
if(flags & DISPATCH_VNODE_DELETE)
{
dispatch_source_cancel(source);
//[blockSelf watchStyleSheet:path];
}
if(flags & DISPATCH_VNODE_EXTEND)
{
//NSLog(#"File size changed");
NSError *error = nil;
NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingFromURL:movieURL error:&error];
if (error) {
[self showError:error];
}
[fileHandle seekToFileOffset:fileOffset];
NSData *newData = [fileHandle readDataToEndOfFile];
if ([newData length] > 0) {
NSLog(#"newData (%lld): %d bytes", fileOffset, [newData length]);
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *basePath = ([paths count] > 0) ? [paths objectAtIndex:0] : nil;
NSString *movieName = [NSString stringWithFormat:#"%d.%lld.%d.mp4", fileNumber, fileOffset, [newData length]];
NSString *path = [NSString stringWithFormat:#"%#/%#", basePath, movieName];
[newData writeToFile:path atomically:NO];
fileNumber++;
fileOffset = [fileHandle offsetInFile];
}
}
});
dispatch_source_set_cancel_handler(source, ^(void)
{
close(fildes);
});
dispatch_resume(source);
}
Here are some similar questions I have found, but don't exactly answer my question:
Get PTS from raw H264 mdat generated by iOS AVAssetWriter
streaming video FROM an iPhone
Parsing h.264 NAL units from a quicktime MOV file
Realtime Audio/Video Streaming FROM iPhone to another device (Browser, or iPhone)
When I eventually figure this out, I will release an open source library to assist people who try to do this in the future.
Thank you!
Update: The corruption doesn't happen at the EOF boundary. It seems like parts of the file are re-written after finishWriting is called. This first file was chunked at 4KB, so the area changed isn't anywhere near an EOF boundary. It seems to be corrupted near new "moov" elements as well when movieFragmentInterval is enabled.
Correct file on the left, broken file on the right.
I ended up abandoning the "read while it's written" approach in favor of a manual chunking approach where I call finishWriting every 5 seconds on a background thread. I was able to drop a negligible number of frames using a method originally described here:
- (void) segmentRecording:(NSTimer*)timer {
AVAssetWriter *tempAssetWriter = self.assetWriter;
AVAssetWriterInput *tempAudioEncoder = self.audioEncoder;
AVAssetWriterInput *tempVideoEncoder = self.videoEncoder;
self.assetWriter = queuedAssetWriter;
self.audioEncoder = queuedAudioEncoder;
self.videoEncoder = queuedVideoEncoder;
//NSLog(#"Switching encoders");
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
[tempAudioEncoder markAsFinished];
[tempVideoEncoder markAsFinished];
if (tempAssetWriter.status == AVAssetWriterStatusWriting) {
if(![tempAssetWriter finishWriting]) {
[self showError:[tempAssetWriter error]];
}
}
if (self.readyToRecordAudio && self.readyToRecordVideo) {
NSError *error = nil;
self.queuedAssetWriter = [[AVAssetWriter alloc] initWithURL:[self newMovieURL] fileType:(NSString *)kUTTypeMPEG4 error:&error];
if (error) {
[self showError:error];
}
self.queuedVideoEncoder = [self setupVideoEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:videoFormatDescription bitsPerSecond:videoBPS];
self.queuedAudioEncoder = [self setupAudioEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:audioFormatDescription bitsPerSecond:audioBPS];
//NSLog(#"Encoder switch finished");
}
});
}
Full source code: https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/AVSegmentingAppleEncoder.m
When reading a MOV file that is ACTIVELY recording on iOS, you MUST check the 4 bytes mentioned for changes, and re-write this four bytes, then check for additional data in file, and send additional data. Then when done, truncate the file to the file size written.
Obviously this depends on where you are sending the file. I use a send (offset,number of bytes) to receiver. So I send "additional data", "more additional data", ... , new data at (24,4), "more additional data".
Typically iOS only writes the 4 byte (size of data section) record when file is about to be closed (aka after last media write). (see info on "Quicktime atoms"). Unfortunately, this also means the MOV file is not PLAYABLE until recording is completed (and movie descriptors written at END of file).

Is it possible to glob a directory in iOS and have the results cached by the OS?

Many methods of reading from a filesystem using NSFileManager and lower level APIs on iOS involve built-in caching, so reading from directories that haven't changed can be quite fast, even if there's lots of items in the directories.
I have a situation where I want to be able to enumerate files in a directory using a glob:
i.e. the folder has files named like this:
1-1-0.png
1-2-0.png
1-3-0.png
1-3-1.png
2-2-1.png
5-1-1.png
5-1-2.png
5-2-1.png
5-3-0.png
6-1-1.png
...
1501-5-2.png
I might want to get all filenames matching 5-*-1.png, which would give me back 5-1-1.png and 5-2-1.png.
Loading the whole directory listing then doing the globbing in RAM is pretty straightforward, but is there a method for doing this at an OS level that would have caching built-in, so repeated calls to the same glob would give cached (faster) results?
You can use the glob() function as outlined in this gist: https://gist.github.com/bkyle/293959
#include <glob.h>
+ (NSArray*) arrayWithFilesMatchingPattern: (NSString*) pattern inDirectory: (NSString*) directory {
NSMutableArray* files = [NSMutableArray array];
glob_t gt;
NSString* globPathComponent = [NSString stringWithFormat: #"/%#", pattern];
NSString* expandedDirectory = [directory stringByExpandingTildeInPath];
const char* fullPattern = [[expandedDirectory stringByAppendingPathComponent: globPathComponent] UTF8String];
if (glob(fullPattern, 0, NULL, &gt) == 0) {
int i;
for (i=0; i<gt.gl_matchc; i++) {
size_t len = strlen(gt.gl_pathv[i]);
NSString* filename = [[NSFileManager defaultManager] stringWithFileSystemRepresentation: gt.gl_pathv[i] length: len];
[files addObject: filename];
}
}
globfree(&gt);
return [NSArray arrayWithArray: files];
}

AVAudioPlayer refuses to play modified wav file

The first time i call this method file1 will be nil and file2 will be returned. When this hapens the file will play normally (so the calling of this method should be fine). But when i call it for the second time it will return an NSURL which the AVAudioPlayer does not play. My guess is I have missed something in the header. In the debugging mode i have seen that the totalLength is exactly as long as the data's length.
+(NSURL *)mergeFile1:(NSURL *)file1 withFile2:(NSURL *)file2 {
if(file1 == nil) {
return [file2 copy];
}
NSData * wav1Data = [NSData dataWithContentsOfURL:file1];
NSData * wav2Data = [NSData dataWithContentsOfURL:file2];
int wav1DataSize = [wav1Data length] - 46;
int wav2DataSize = [wav2Data length] - 46;
if (wav1DataSize <= 0 || wav2DataSize <= 0) {
return nil;
}
NSMutableData * soundFileData = [NSMutableData dataWithData:[wav1Data subdataWithRange:NSMakeRange(0, 46)]];
[soundFileData appendData:[wav1Data subdataWithRange:NSMakeRange(46, wav1DataSize)]];
[soundFileData appendData:[wav2Data subdataWithRange:NSMakeRange(46, wav2DataSize)]];
unsigned int totalLength = [soundFileData length];
NSLog(#"Calculated: %d - Real: %d", totalLength, [soundFileData length]);
[soundFileData replaceBytesInRange:NSMakeRange(4, 4)
withBytes:&(UInt32){NSSwapHostIntToLittle(totalLength-8)}];
[soundFileData replaceBytesInRange:NSMakeRange(42, 4)
withBytes:&(UInt32){NSSwapHostIntToLittle(totalLength)}];
[soundFileData writeToURL:file1 atomically:YES];
return [file1 copy];
}
If anyone sees something that can be of help it would be much appreciated!
Any questions will be answered asap.
EDIT
I know there are 2 sorts of wav headers: 44 bytes or 46 bytes. I have tried both.
EDIT
I have looked at the Audio File Services Reference which contains a lot of nice stuff i might want to use, but i can't figure out how to use all this. I'm not really known with c. Hope anyone could help me out with this.
EDIT
An example of a merged wav file is found here: 7--443522512
Looks like your WAV file includes a broken FLLR chunk before the data chunk, or at least VLC thinks the FLLR chunk is over 2GB large so it tries to skip to the next chunk which is beyond the file end.
Maybe you should try to create WAV files without FLLR chunk before merging, the kAudioFileFlags_DontPageAlignAudioData seams to make Audio File Services skip it.
Another option is to extract the data chunks and write a new wav file, a did a proof of concept implementation here: https://gist.github.com/1555889

Cocoa Touch Bonjour how to deal with NSNetService addresses and uint8_t

I'm attempting to make an iOS app communicate with a server that uses Bonjour and uses HTTP commands. So far I have been able to find the local domain and locate the particular service I'm looking for. I am able to resolve the address of the service, but I don't know how to get something useful out of the address. The address from the NSNetService is a NSData object and I have no idea what to do with it. I need to send commands like GET and PUT. What cocoa classes handle things like this?
I also tried getting input and output streams from the Service, but they seem to be extremely low level streams and I don't know how to properly deal with buffers and all that.
[service getInputStream:&inputStream outputStream:&outputStream]
the NSOutputStream write method takes in a uint8_t buffer which I have no idea how to create.
the NSInputStream read method returns a uint8_t buffer and I don't know how to interpret it.
I am able to communicate with this server using terminal commands. For instance, sending it the command LIST causes it to print out the list of files I am looking for. How do I send and get information like this in Cocoa?
To write data to the output stream, therefore sending it to the server:
NSString * stringToSend = #"Hello World!\n"; //The "\n" lets the receiving method described below function correctly. I don't know if you need it or not.
NSData * dataToSend = [stringToSend dataUsingEncoding:NSUTF8StringEncoding];
if (outputStream) {
int remainingToWrite = [dataToSend length];
void * marker = (void *)[dataToSend bytes];
while (0 < remainingToWrite) {
int actuallyWritten = 0;
actuallyWritten = [outputStream write:marker maxLength:remainingToWrite];
remainingToWrite -= actuallyWritten;
marker += actuallyWritten;
}
}
You can send any data like this, just put it in a NSData object.
To receive data from the server use this code in the input stream's NSStreamDelegate:
- (void)stream:(NSStream *)aStream handleEvent:(NSStreamEvent)streamEvent {
NSInputStream * istream;
NSOutputStream * ostream;
switch(streamEvent) {
case NSStreamEventHasBytesAvailable:;
istream = (NSInputStream *)aStream;
ostream = (NSOutputStream *)CFDictionaryGetValue(connections, istream);
uint8_t buffer[2048];
int actuallyRead = [istream read:(uint8_t *)buffer maxLength:2048];
if (actuallyRead > 0) {
NSData *data;
data = [NSData dataWithBytes:buffer length:actuallyRead];
NSString *string = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]autorelease];
string = [string stringByReplacingOccurrencesOfString:#"\n" withString:#""];
//Do something with the string...
}
break;
case NSStreamEventEndEncountered:;
istream = (NSInputStream *)aStream;
ostream = nil;
if (CFDictionaryGetValueIfPresent(connections, istream, (const void **)&ostream)) {
[self shutdownInputStream:istream outputStream:ostream];
}
break;
case NSStreamEventHasSpaceAvailable:
case NSStreamEventErrorOccurred:
case NSStreamEventOpenCompleted:
case NSStreamEventNone:
default:
break;
}
}
Take a look at Apple's CocoaEcho Sample Code. It should help you.