I'm using the GPUImage framework and I've noticed that the compiler automatically skips everything that is within the brackets of the setColorAverageProcessingFinishedBlock. It completely skips over these contents and continues on, executing everything else in the code. Once everything else has been executed, it comes back to the content within the brackets. Obviously, this has unintended side effects.
NSMutableArray *redValues = [NSMutableArray array];
NSMutableArray *arrayOne = [NSMutableArray array];
NSUInteger arrayOneLength = [arrayOne count];
__block int counter = 0;
int amount = 1;
float totalOne, diffForAverage;
NSInteger j;
GPUImageVideoCamera *videoCamera = [[GPUImageVideoCamera alloc] initWithSessionPreset:AVCaptureSessionPreset640x480 cameraPosition:AVCaptureDevicePositionBack];
videoCamera.outputImageOrientation = UIInterfaceOrientationPortrait;
GPUImageAverageColor *averageColor = [[GPUImageAverageColor alloc] init];
[averageColor setColorAverageProcessingFinishedBlock:^(CGFloat redComponent, CGFloat greenComponent, CGFloat blueComponent, CGFloat alphaComponent, CMTime frameTime)
{ // the compiler runs until here, then skips everything within these brackets
NSLog(#"%f", redComponent);
[redValues addObject:#(redComponent * 255)];
}]; // after the brackets close, it executes everything that is below this
// once everything below this has been executed, it goes back to the brackets and executes
// everything between them
[videoCamera addTarget:averageColor];
[videoCamera startCameraCapture];
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, 27 * NSEC_PER_SEC), dispatch_get_main_queue(), ^{
[videoCamera stopCameraCapture];
});
totalOne = [redValues[24] floatValue];
float average = totalOne / amount;
NSUInteger redValuesLength = [redValues count];
for (j = (counter + 24); j < (redValuesLength - 24); j++)
{
diffForAverage = average - [redValues[j + 1] floatValue];
if (diffForAverage > -1 && diffForAverage < 1)
{
totalOne += [redValues[j + 1] floatValue];
amount++;
[arrayOne addObject:[NSNumber numberWithInt:(j - 24)]];
counter++;
}
}
How can I solve this problem?
There are two issues with the above code: a memory management one, and a misunderstanding of how blocks work.
First, you're creating a GPUImageVideoCamera instance within a method, but not retaining it as an instance variable. I'm going to assume this is code using automatic reference counting, and if that's true, this camera instance will be deallocated the instant your method is finished. At best, you'll capture maybe one frame from the camera before this is deallocated. At worst, this will crash as the camera and the entire filter chain attached to it are deallocated mid-operation.
Make an instance variable on your containing class and assign your GPUImageVideoCamera instance to it to have it last long enough to be useful.
The second issue with the above is a misunderstanding about how and when blocks will execute. Blocks are merely sections of code you can pass around, and they don't necessarily execute in serial with the rest of the code around them.
In this case, the block you're providing is a callback that will be triggered after every frame of video is processed through the average color operation. This processing takes place asynchronously on a background queue, and you have to design your code to acknowledge this.
If you want X values to be built up, have each measurement be added to an array inside that block, and then within the block check for X values to be reached. At that point, average and do whatever with them. Basically, add a check within the block and move the code you have after it into the block to be run whenever the count is greater than X. You may wish to stop camera capture at that point, if that's all you need.
The code you post is working exactly as it is supposed to work. The color average processing takes a while so it is done on a background thread so the main thread isn't stalled. After the processing is done, then the block is called.
Any code that shouldn't be executed until after the processing is done needs to go inside the block.
Related
I have two arrays: array1 and array2. Each object of arrays is an array too (2D arrays). In this way I multiple them. So how I have big arrays I use dispatch_apply. Every time i receive different results include a right result. Maybe somebody knows how to fix it?
dispatch_apply([array2 count], queue, ^(size_t j)
{
k = 0;
for (int l = 0; l < [[array1 objectAtIndex:0] count]; l++) {
k += [[[array1 objectAtIndex:i] objectAtIndex:l] intValue] *
[[[array2 objectAtIndex:j] objectAtIndex:l] intValue];
}
kNSNumber = [NSNumber numberWithInt:k];
[multipliedArrayInto replaceObjectAtIndex:j withObject:kNSNumber];
});
[resulArray insertObject:multipliedArrayInto atIndex:i];
}
There's two things, I can suggest, and I bet one of them (or both) is the overarching solution to your problem.
First, I would declare k local to the block, so there would be no question that you are overwriting it or not. You likely have the same problem with kNSNumber inside the block. If you are just using that NSNumber instance to slam into the multipliedArrayInto accumulator, you may as well remove kNSNumber, and use #(k) in it's place (if only to be more readable). Similarly, make sure multipliedArrayInto is declared just before the dispatch_apply, in what looks like an outer for loop (where ever i is coming from). And finally, make sure resulArray is instantiated, or otherwise readied just before that outer for loop.
Second, is queue a concurrent or serial queue? If you are using dispatch_apply like a parallel-executing for/enumeration -- which is likely, I think, so you are taking about handling "big arrays" efficiently -- then you are practically guaranteeing that k is being overwritten. If you change it to serial, it may work as designed. If you want it to be parallel, you will need to move the declaration of your k accumulator inside the block, and make sure the declaration of other variables makes sense, too.
Update to reflect question updates:
#antonytonies ideally, your followup answer on this thread should be moved into the question itself, so that people can follow this thread easier.
So, it looks like what I described is exactly your problem.
The global queues are all concurrent queues, which means that (hypothetically) all the dispatch blocks are executing at once, and the contents of k and other variables are getting blown away depending on how the order of the blocks executes.
I've taken your update (in the "answer" you added), and modified it to probably work:
// I renamed your method, because nameless parameters pain me. This is cosmetic, and doesn't
// matter for the problem at hand.
- (NSMutableArray *)multiplicationArrays:(NSMutableArray *)array vector:(NSMutableArray *)vector
{
// IMHO, you want to set resultArray to nil here. Another option is to set it to nil in the
// else case, below. Properties in Objective-C are initalized to nil,0,false,etc; you can
// rely on ARC to initialize pointer to objc objects on the stack, too. However, someone
// reading this code may or may not know that. IMHO, using the explicitly assignement makes it
// clear that you're going to be returning `nil` or an instance of `NSMutableArray`.
NSMutableArray *resultArray = nil;
if ([[array objectAtIndex:0] count] == [vector count]) {
// Nicely done w/ pre-allocating the result array here, so that there's no question
// of the indexes matches the results later on.
resultArray = [[NSMutableArray alloc] initWithCapacity:[array count]];
for (int i=0; i < [array count]; i++) {
[resultArray insertObject:[NSNull null] atIndex:i];
}
// 'queue' here is a concurrent queue. This means that you are proclaiming to the runtime
// that the blocks being executed are able to operate correctly w/o interference from each
// other. This is also thought of in terms of parallel execution: all these blocks may run
// **at once**. This *also* means, that you must not share storage between them.
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_apply([array count], queue, ^(size_t j) {
// Moved 'result' inside the block.
NSInteger result = 0;
for (int l = 0; l < [[array objectAtIndex:0] count]; l++) {
// These array reads are **NOT** thread safe. They probably don't cause must trouble in
// practice, but you may want to reconfigure this.
result += [[[array objectAtIndex:j] objectAtIndex:l] intValue] * [[vector objectAtIndex:l] intValue];
}
// The replace of the object into resultArray is **NOT** thread-safe.
// This probably hasn't caused you much trouble, since you can guarantee that
// you aren't writing at the same index. However, I would strongly suggest to
// change this to be thread-safe.
[resultArray replaceObjectAtIndex:j withObject:#(result)];
});
}
else {
NSLog(#"matrix count isn't correspond");
}
return resultArray;
}
Finally: consider just using Apple's Accelerate framework for this sort of problem solving. It's available on OSX and iOS, so you should have all of your bases covered.
it's the same thing if I multiple 2D-array and vector
-(NSMutableArray*)multiplicationArraysWithVector:(NSMutableArray *)array :(NSMutableArray *)vector
{
NSMutableArray* resultArray;
if ([[array objectAtIndex:0] count] == [vector count])
{
resultArray = [[NSMutableArray alloc] initWithCapacity:[array count]];
for (int i=0; i < [array count]; i++) {
[resultArray insertObject:[NSNull null] atIndex:i];
}
__block NSInteger result;
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_apply([array count], queue, ^(size_t j)
{
result = 0;
for (int l = 0; l < [[array objectAtIndex:0] count]; l++) {
result += [[[array objectAtIndex:j] objectAtIndex:l] intValue] * [[vector objectAtIndex:l]intValue];
}
[resultArray replaceObjectAtIndex:j withObject:#(result)];
});
}
else
{
NSLog(#"matrix count isn't correspond");
}
return resultArray;
}
In this case I can get a right or wrong data result.
I am trying to write a command line application in Objective-C for a university project. The project needs matrix manipulation so I have written a class to handle all the matrix methods (Addition and multiplication and such). My matrix methods look like this:
- (NSArray *)sumMatrices:(NSArray *)matrices
{
NSMutableArray *sum = [[NSMutableArray alloc] init];
NSInteger cols = [matrices[0][0] count];
NSInteger rows = [matrices[0] count];
for (int i = 0; i < rows; i++) {
NSMutableArray *row = [[NSMutableArray alloc] init];
for (int j = 0; j < cols; j++) {
CGFloat value = 0.0;
for (NSArray *array in matrices) {
value += [array[i][j] doubleValue];
}
[row addObject:[NSNumber numberWithDouble:value]];
}
[sum addObject:[row copy]];
row = nil;
}
return [sum copy];
}
However theres is a massive problem with this programme, I having used objective-c for iOS expect ARC to handle all my memory allocation and deallocation without a problem, however in this case the NSMutableArray 'sum' is never being deallocated, and because this method is being called in a loop which runs 10's of thousands of times (Modelling a double pendulum using RK4) the memory usage builds up and makes the program extremely slow.
Is there any reason this NSMutableArray isn't being deallocated once this method has returned?
Your problem is less about this code and more about the code surrounding it. Let's assume for a moment that your code around it looks like this:
NSArray *matricies; //Declared somewhere else;
NSMutableArray *results = [[NSMutableArray alloc] init];
for (int i=0; i < [matricies count] - 1; i++) {
for (int j=i+1; j < [matricies count]; i++) {
NSArray *sum = [self sumMatrices:#[matricies[i], matricies[j]]];
[results addObject:sum];
}
}
The actual operations that I'm performing are not particularly relevant to this example. The code pattern is. You'll notice I'm using a nested "tight" loop. Control never returns to the run loop until AFTER all calculations are complete. Without ARC, your memory would be freed as soon as the last release was performed, excluding autoreleased objects. With ARC, your memory is not freed until control is returned to the runloop, much the same way autoreleased objects used to. As a result, your code will appear to leak, until processing is complete and the system decides it should release your memory. If the CPU is perpetually under a heavy load, it may not clean up memory until it absolutely has to.
There are a few cleaver ways to use #autoreleasepool to help in this case, but that will make your code significantly slower. Additionally, Objective C is a fairly slow language for objects and method calls. If you are using this code heavily, you should convert it into C or C++ and manage the memory yourself.
without going into much detail you can try to use autoreleasepool
https://developer.apple.com/library/mac/documentation/cocoa/conceptual/memorymgmt/articles/mmAutoreleasePools.html
i would use copy if i want to preserve an array which gets modified but in your case do you really need it ?
As before, we usually do something for loop like this:
for (int i = 0 ; i < 5; i ++) {
NSNumber * number = [[NSNumber alloc] initWithInt:i];
[muArray addObject:number];
[number release];
}
But under ARC, there is no release. Can I manage memory without an #autorelease block, like this (directly remove release statement):
for (int i = 0 ; i < 5; i ++) {
NSNumber * number = [[NSNumber alloc] initWithInt:i];
[muArray addObject:number];
}
Similarly, is the #autoreleasepool necessary, like this?
for (int i = 0 ; i < 5; i ++) {
#autoreleasepool{
NSNumber * number = [NSNumber numberWithInt:i];
[muArray addObject:number];
}
}
Your second code block (compiled with ARC), has exactly the same semantics as your first code block (compiled with MRC).
Under ARC, when you set a strong object reference to nil, or when a strong object reference is destroyed, ARC takes care of sending the release message for you. In both examples, the number variable is destroyed at the end of the loop body, so (in the second example) ARC releases the objected that number referenced.
In your third example, the #autoreleasepool will cause the returned NSNumber to be released on each pass through the loop. This might be necessary for a large number of loop iterations. For just five iterations, each creating a single NSNumber, it's not necessary.
Yes, ARC will translate your second example into the first.
You can read more here (apple docs) and here (dr. dobbs).
The autorelease pool example is should also be equivalent, llvm docs
Im trying to make a animation with pictures using this code:
NSMutableArray *imageArray = [[NSMutableArray alloc] initWithCapacity:22];
int i;
for (i =1; i < 22; i++) {
[imageArray addObject:[UIImage imageNamed:[NSString stringWithFormat:#"fbAnimation%d.png", i]]];
//make it animate
self.fbLogin.animationImages = [NSArray arrayWithArray:imageArray];
self.fbLogin.animationDuration =0.8;
self.fbLogin.animationRepeatCount = 5;
[self.fbLogin startAnimating];
}
int j;
NSMutableArray *imageLoading = [[NSMutableArray alloc] initWithCapacity:4];
for (j = 1; j < 4; j++) {
[imageLoading addObject:[UIImage imageNamed:[NSString stringWithFormat:#"loading%d.png", j]]];
//make it animate
self.fbLogin.animationImages = [NSArray arrayWithArray:imageLoading];
self.fbLogin.animationDuration = 1;
self.fbLogin.animationRepeatCount = 5;
[self.fbLogin startAnimating];
}
}
The problem is that either way I've tried it so far just one animation gets executed on the screen. I want the first animation to run and then the second "Loading..." until all other data from facebook is ready.
Firstly, you are calling [self.fbLogin startAnimating] and resetting the animationImages array each time you go round each for loop. You should move them outside the loops - you only need to start each animation once.
Secondly, the second for loop will be executed immediately after the first, so the result I would expect would be for the second animation to appear, and not the first.
One way of getting round the problem would be to put the code for the second animation in a separate method, and then arrange for it to be called after the first has finished by calling:
[self performSelector:#selector(animation2) withObject:nil afterDelay:durationOfPreviousAnimation];
at the end of the first for loop. I'm assuming that since this is animation code we're guaranteed to be on the main thread.
So, I'm trying to do a simple calculation over previously recorded audio (from an AVAsset) in order to create a wave form visual. I currently do this by averaging a set of samples, the size of which is determined by dividing the audio file size by the resolution I want for the wave form.
This all works fine, except for one problem....it's too slow. Running on a 3GS, processing an audio file takes about 3% of the time it takes to play it, which is way to slow (for example, a 1 hour audio file takes about 2.5 minutes to process). I've tried to optimize the method as much as possible but it's not working. I'll post the code I use to process the file. Maybe someone will be able to help with that but what I'm really looking for is a way to process the file without having to go over every single byte. So, say given a resolution of 2,000 I'd want to access the file and take a sample at each of the 2,000 points. I think this would be a lot quicker, especially if the file is larger. But the only way I know to get the raw data is to access the audio file in a linear manner. Any ideas? Here's the code I use to process the file (note, all class vars begin with '_'):
So I've completely changed this question. I belatedly realized that AVAssetReader has a timeRange property that's used for "seeking", which is exactly what I was looking for (see original question above). Furthermore, the question has been asked and answered (I just didn't find it before) and I don't want to duplicate questions. However, I'm still having a problem. My app freezes for a while and then eventually crashes when ever I try to copyNextSampleBuffer. I'm not sure what's going on. I don't seem to be in any kind of recursion loop, it just never returns from the function call. Checking the logs show give me this error:
Exception Type: 00000020
Exception Codes: 0x8badf00d
Highlighted Thread: 0
Application Specific Information:
App[10570] has active assertions beyond permitted time:
{(
<SBProcessAssertion: 0xddd9300> identifier: Suspending process: App[10570] permittedBackgroundDuration: 10.000000 reason: suspend owner pid:52 preventSuspend preventThrottleDownCPU preventThrottleDownUI
)}
I use a time profiler on the app and yep, it just sits there with a minimal amount of processing. Can't quite figure out what's going on. It's important to note that this doesn't occur if I don't set the timeRange property of AVAssetReader. I've checked and the values for timeRange are valid, but setting it is causing the problem for some reason. Here's my processing code:
- (void) processSampleData{
if (!_asset || CMTimeGetSeconds(_asset.duration) <= 0) return;
NSError *error = nil;
AVAssetTrack *songTrack = _asset.tracks.firstObject;
if (!songTrack) return;
NSDictionary *outputSettingsDict = [[NSDictionary alloc] initWithObjectsAndKeys:
[NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
[NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,
nil];
UInt32 sampleRate = 44100.0;
_channelCount = 1;
NSArray *formatDesc = songTrack.formatDescriptions;
for(unsigned int i = 0; i < [formatDesc count]; ++i) {
CMAudioFormatDescriptionRef item = (__bridge_retained CMAudioFormatDescriptionRef)[formatDesc objectAtIndex:i];
const AudioStreamBasicDescription* fmtDesc = CMAudioFormatDescriptionGetStreamBasicDescription (item);
if(fmtDesc ) {
sampleRate = fmtDesc->mSampleRate;
_channelCount = fmtDesc->mChannelsPerFrame;
}
CFRelease(item);
}
UInt32 bytesPerSample = 2 * _channelCount; //Bytes are hard coded by AVLinearPCMBitDepthKey
_normalizedMax = 0;
_sampledData = [[NSMutableData alloc] init];
SInt16 *channels[_channelCount];
char *sampleRef;
SInt16 *samples;
NSInteger sampleTally = 0;
SInt16 cTotal;
_sampleCount = DefaultSampleSize * [UIScreen mainScreen].scale;
NSTimeInterval intervalBetweenSamples = _asset.duration.value / _sampleCount;
NSTimeInterval sampleSize = fmax(100, intervalBetweenSamples / _sampleCount);
double assetTimeScale = _asset.duration.timescale;
CMTimeRange timeRange = CMTimeRangeMake(CMTimeMake(0, assetTimeScale), CMTimeMake(sampleSize, assetTimeScale));
SInt16 totals[_channelCount];
#autoreleasepool {
for (int i = 0; i < _sampleCount; i++) {
AVAssetReader *reader = [AVAssetReader assetReaderWithAsset:_asset error:&error];
AVAssetReaderTrackOutput *trackOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:songTrack outputSettings:outputSettingsDict];
[reader addOutput:trackOutput];
reader.timeRange = timeRange;
[reader startReading];
while (reader.status == AVAssetReaderStatusReading) {
CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];
if (sampleBufferRef){
CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef);
size_t length = CMBlockBufferGetDataLength(blockBufferRef);
int sampleCount = length / bytesPerSample;
for (int i = 0; i < sampleCount ; i += _channelCount) {
CMBlockBufferAccessDataBytes(blockBufferRef, i * bytesPerSample, _channelCount, channels, &sampleRef);
samples = (SInt16 *)sampleRef;
for (int channel = 0; channel < _channelCount; channel++)
totals[channel] += samples[channel];
sampleTally++;
}
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
}
for (int i = 0; i < _channelCount; i++){
cTotal = abs(totals[i] / sampleTally);
if (cTotal > _normalizedMax) _normalizedMax = cTotal;
[_sampledData appendBytes:&cTotal length:sizeof(cTotal)];
totals[i] = 0;
}
sampleTally = 0;
timeRange.start = CMTimeMake((intervalBetweenSamples * (i + 1)) - sampleSize, assetTimeScale); //Take the sample just before the interval
}
}
_assetNeedsProcessing = NO;
}
I finally figured out why. Apparently there is some sort of 'minimum' duration you can specify for the timeRange of an AVAssetReader. I'm not sure what exactly that minimum is, somewhere above 1,000 but less than 5,000. It's possible that the minimum changes with the duration of the asset...honestly I'm not sure. Instead, I kept the duration (which is infinity) the same and simply changed the start time. Instead of processing the whole sample, I copy only one buffer block, process that and then seek to the next time. I'm still having trouble with the code, but I'll post that as another question if I can't figure it out.