What I am hoping to do may or may not be possible but I'll give it a shot. I am attempting to load huge multiple gigabyte text files. I am currently using an memory mapped NSData and only loading portions at a time and just infinite scrolling through it and load the currently visible chunk and it all is happy and it barely breaks a sweat with memory (<30mb).
Next I wanted to add editing of the data. I casually assumed I could just edit that memory mapped data and have it reflect on disk. This does not seem to be the case. Is there a way I can say hey the data changed! update the file. I am a little vague on how the file and NSData are linked.
Here is how it is currently working.
// create the memory mapped data:
self.fileData = [NSMutableData dataWithContentsOfURL:url options:NSDataReadingMappedAlways error:outError];
//load a chunk
self.currentRange = NSMakeRange(0, 4096);
void* data = malloc(4096);
[self.fileData getBytes:data range:self.currentRange];
NSString* currentView = [[NSString alloc] initWithBytes:data length:4096 encoding:NSUTF8StringEncoding];
//replace a chunk:
NSData* currentData = [[self.textArea string] dataUsingEncoding:NSUTF8StringEncoding];
[self.fileData replaceBytesInRange:self.currentRange withBytes:[currentData bytes] length:[currentData length]];
//If I close and re-open it doesn't actually work
Is there a way of doing this or would it be possible to create an NSOutputStream to the URL and somehow stream portions of it back to disk? Is there another way to write/save memory mapped data? If I try to do a writeToURL: or comparable function it will load it all to actual memory which is really no good.
Related
Just a question I had regarding general programming.
I could go for either of the options:
for(some_conditions)
{
NSFileManager * fm = [[NSFileManager alloc] init];
BOOL result = [fm moveItemAtPath:x toPath:y error:&err];
}
Or I could go for:
NSFileManager * fm = [[NSFileManager alloc] init];
for(some_conditions)
{
BOOL result = [fm moveItemAtPath:x toPath:y error:&err];
}
What I want to know is, are there any computational differences between the two with respect to time and space taken to execute the two options?
Thanks a bunch in advance :)
If you don't plan to use NSFileManagerDelegate, consider using the defaultManager:
NSFileManager *fm = NSFileManager.defaultManager;
Not only it is initialized once, but you can imagine that it might keep some internal in-memory caches to speed up certain operations.
If interested about performance you should add (and print) timings to your code and try it with real data. My guess is that here the time to alloc/init is negligible compared to the moveItemAtPath.
The space allocation can be checked in Xcode allocations instrument or debugging memory graph. My guess is that it is the same for both, because in the first variant fm is destroyed at the end of each loop iteration.
I have some NSData output that I would like to convert to a string.
NSString * test = [NSString stringWithFormat:#"myfile.txt];
NSData *myData = [[NSData alloc] initWithContentsOfFile:(#"%#", test)];
This NSData was saved to a file, it looks like : 21fa9731 27c67c00 da1c3349 d82470eb 56f97b88 559f406c 6abecbb7 de020007 47a4541d 99c9c5e7 883f8bf1 165fba39
Do you know a way to get this string back as it was in "myfile.txt" ?
Thanks !
If data file somewhere on HDD (not in app bundle) you must provide full path to your data file.
NSString * test = [NSString stringWithFormat:#"myfile.txt];
NSData *myData = [[NSData alloc] initWithContentsOfFile:(#"%#", test)];
NSString* myString = [NSString alloc]initWithData:myData
encoding: NSUTF8StringEncoding];
Choose encoding in which you save you data file.
I don't know what your problem ist. You could share with us how the file was written so that we can get a bit closer.
However, you are doing much too complicated and therefore error prone. The following would to for reading a file with a constant name fo myfile.txt.
NSData *myData = [[NSData alloc] initWithContentsOfFile:#"myfile.txt"];
Frankly I don't even know in which directory myfile.txt is expected to be. And there are of course much better ways to deal with constant literals than using literals directly in the code.
BTW, what do you actually receive in myData? null?
(This is more of a comment than an answer, but the formatting is much better in answers.)
I use the NSTask to run shell command and output the data via NSPipe. At first, I using bellow method to read output data, it is no any problem.
- (void)outputAvailable:(NSNotification *)aNotification {
NSString *newOutput;
NSMutableData *allData = [[NSMutableData alloc] init];
NSData *taskData = nil;
if((taskData = [readHandle availableData]) && [taskData length])
newOutput = [[NSString alloc] initWithData:allData encoding:NSASCIIStringEncoding];
NSLog(#"%#", newOutput);
[readHandle readInBackgroundAndNotify];
}
The problem about the method is that it only output 4096 bytes data. So I using while loop to get more data, modify the method like this：
- (void)outputAvailable:(NSNotification *)aNotification {
NSString *newOutput;
NSMutableData *allData; //Added.
NSData *taskData = nil;
while ((taskData = [readHandle availableData]) && [taskData length]) {
[allData appendData:taskData];
}
newOutput = [[NSString alloc] initWithData:allData encoding:NSASCIIStringEncoding];
NSLog(#"%#", newOutput);
[readHandle readInBackgroundAndNotify];
}
Then problem occurs: the program is blocking in the while loop and can not perform the following statements. I ensure that allData is what I wanted, but after appending the last data chunk, it is blocking.
Could you give me some solutions? Thanks.
Your while() loop effectively blocks further notifications, causing the whole program to block waiting for something to flush the buffer.
You should readInBackgroundAndNotify, then pull off availableBytes on each notification, appending it to your NSMutableData (which is likely held in an instance variable). When you handle the notification, don't attempt to wait for more data or do any kind of a while loop. The system will notify you when more data is available.
I.e. the system pushes data to you, you do not pull data from the system.
Ahh... OK. You should still only pull data when there is data available. Your while() loop is doing that. Not enough coffee. My bad.
The final block is most likely because your external process is not closing the pipe; no EOF is received and, thus, the program is waiting forever for more data that never arrives.
Either:
make sure the background task exits
detect when you've received enough data and terminate the process
If you are doing some kind of conversion program (say, tr) where you write data on the processes standard input, then you might need to close the standard input pipe.
I am trying to read a text file containing characters in billions. Using the function
contentOfFile is not working, as my application get crashed due to it.
So anybody please send me the sample code so that I get the chunks according to my requirement.Whichever i need i wanna get that one only.
please reply as soon as possible.
I'm guessing this is an iOS app. In that case, you are likely hitting the memory limit by calling contentsOfFile: because that method is trying to read the entire contents of the file into a variable (memory). Remember that on iOS your app must play nice and if it decides to consume too much memory, then the watchdog process will kill your app to save the device from rebooting (which happens because there is no disk to swap to on iOS devices).
Have you had a look at NSFileHandle? NSFileHandle supports seeking within a text a file. With some simple iteration you can use the following to methods to seek within the file and read chunks of data:
- (NSData *)readDataOfLength:(NSUInteger)length;
- (void)seekToFileOffset:(unsigned long long)offset;
It might look something like this. Assume pathToFile is an NSString containing the path to the text file to be read in.
uint64 offset = 0;
uint32 chunkSize = 1024; //Read 1KB chunks.
NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:pathToFile];
NSData *data = [handle readDataOfLength:chunkSize];
while ([data length] > 0)
{
//Make sure for the next line you choose the appropriate string encoding.
NSString *dataString = [[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] autorelease];
/* PERFORM STRING PROCESSING HERE */
/* END STRING PROCESSING */
offset += [data length];
[handle seekToFileOffset:offset];
data = [handle readDataOfLength:chunkSize];
}
[handle closeFile];
A good idea is to look at the textedit source because I've opened massive files with it before and there should be a way to do it. Not sure why your app is crashing though. It shouldn't have a problem.
I have been searching for many days on how to save my apps data. I found some stuff but it was very complicated and badly explained. I need that when I completely close my apps all the data I entered in the text field are still there when I open my apps again. I tried a tutorial but this only let me save about 8 textfields and I need to save thousands I am starting Objective-C and Xcode so if somebody want to give me an answer please make it very precise.
Alright, what I'd suggest would be putting all the data from your text fields into an array and saving that to a file, then loading it when you re-open the app.
The first thing you need is a save file. This function will create one for you.
-(NSString*) saveFilePath{
NSString* path = [NSString stringWithFormat:#"%#%#",
[[NSBundle mainBundle] resourcePath],
#"myfilename.plist"];
return path;}
Now that that's done you need to create your saving array. Hopefully you have your thousands of textfields already fitted into an array of some sort. If not, this will be a painful process regardless of how you tackle it. But anyway... (Here, labelArray will be the array of all your text fields/labels/etc.)
NSMutableArray* myArray = [[NSMutableArray alloc]init];
int i = 0;
while(i < labelArray.count){
[myArray addObject: [labelArray objectAtIndex: i].text];
i ++;
}
[myArray writeToFile:[self saveFilePath] atomically:YES];
[myArray release];
And the loading code would be something along the lines of
NSMutableArray* myArray = [[NSMutableArray arrayWithContentsOfFile:[self saveFilePath]]retain];
Then you'd simply load the data back into your array of text fields.
Hope this helps.
It sounds like your application architecture may be unsound if you are planning on saving thousands of text fields' data in the fraction of a second you get while your app is closing. It would probably be better to save these as the user enters the data instead of waiting to save all the data at once.
To get the path you are going to write ( or read from! ) to, you do the following:
NSString *writableDBPath = [[NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0] stringByAppendingPathComponent:#"MyFile.extension"];
And then use a method like "writeToFile:automically:" of NSString or NSDictionary etc. to write to the path.