Working CIAreaHistogram / CIHistogramDisplayFilter example for NSImage - objective-c

Somehow I cannot figure out to get an actual, meaningful histogram image from an NSImage input using the CIAreaHistogram and CIHistogramDisplayFilter filters.
I read Apple's "Core Image Filter Reference" and the relevant posts here on SO, but whatever I try I get no meaningful output.
Here's my code so far:
- (void) testHist3:(NSImage *)image {
CIContext* context = [[NSGraphicsContext currentContext] CIContext];
NSBitmapImageRep *rep = [image bitmapImageRepresentation];
CIImage *ciImage = [[CIImage alloc] initWithBitmapImageRep:rep];
ciImage = [CIFilter filterWithName:#"CIAreaHistogram" keysAndValues:kCIInputImageKey, ciImage, #"inputExtent", ciImage.extent, #"inputScale", [NSNumber numberWithFloat:1.0], #"inputCount", [NSNumber numberWithFloat:256.0], nil].outputImage;
ciImage = [CIFilter filterWithName:#"CIHistogramDisplayFilter" keysAndValues:kCIInputImageKey, ciImage, #"inputHeight", [NSNumber numberWithFloat:100.0], #"inputHighLimit", [NSNumber numberWithFloat:1.0], #"inputLowLimit", [NSNumber numberWithFloat:0.0], nil].outputImage;
CGImageRef cgImage2 = [context createCGImage:ciImage fromRect:ciImage.extent];
NSImage *img2 = [[NSImage alloc] initWithCGImage:cgImage2 size:ciImage.extent.size];
NSLog(#"Histogram image: %#", img2);
self.histImage = img2;
}
What I get is a 64x100 image with zero representations (=invisible). If I create the CI context with
CIContext *context = [[CIContext alloc] init];
then the resulting image is grey, but at least it does have a representation:
Histogram image: <NSImage 0x6100002612c0 Size={64, 100} Reps=(
"<NSCGImageSnapshotRep:0x6100002620c0 cgImage=<CGImage 0x6100001a1880>>" )>
The input image is a 1024x768 JPEG image.
I have little experience with Core Image or Core Graphics, so the mistake might be with the conversion back to NSImage... any ideas?
Edit 2016-10-26: With rickster's very comprehensive answer I was able to make a lot of progress.
Indeed it was the inputExtent parameter that was messing up my result. Supplying a CIVector there solved the problem. I found that you cannot leave that to the default either; I don't know what the default value is, but it is not the input image's full size. (I found that out by running an image and a mirrored version of it through the filter; I got different histograms.)
Edit 2016-10-28:
So, I've got a working, displayable histogram now; my next step will be to figure out how the "intermediate" histogram (the 256x1 pixel image coming out of the filter) can contain the actual histogram information even though all but the last pixel are always (0, 0, 0, 0).

I presume the [image bitmapImageRepresentation] in your code is a local category method that's roughly equivalent to (NSBitmapImageRep *)image.representations[0]? Otherwise, first make sure that you're getting the right input.
Next, it looks like you're passing the raw output of ciImage.extent into your filter parameters — given that said parameter expects a CIVector object and not a CGRect struct, you're probably borking the input to your filter at run time. You can get a bit more useful diagnostics for such problems by using the dictionary-based filter methods filterWithName:withInputParameters or imageByApplyingFilter:withInputParameters: — that way, if you try to pass nil for a filter key or pass something that isn't a proper object, you'll get a compile-time error. The latter gives you an easy way to go straight from input image to output image, or chain filters, without creating intermediary CIFilter objects and needing to set the input image on each.
A related tip: most of the parameters you're passing are the default values for those filters, so you can pass only the values you need:
CIImage *hist = [inputImage imageByApplyingFilter:#"CIAreaHistogram"
withInputParameters:#{ #"inputCount": #256 }];
CIImage *outputImage = [hist imageByApplyingFilter:#"CIHistogramDisplayFilter"
withInputParameters:nil];
Finally, you might still get an almost-all-gray image out of CIHistogramDisplayFilter depending on what your input image looks like, because all of the histogram bins may have very small bars. I get the following for Lenna:
Increasing the value for kCIInputScaleKey can help with that.
Also, you don't need to go through CGImage to get from CIImage to NSImage — create an NSCIImageRep instead and AppKit will automatically manage a CIContext behind the scenes when it comes time to render the image for display/output.
// input from NSImage
NSBitmapImageRep *inRep = [nsImage bitmapImageRepresentation];
CIImage *inputImage = [[CIImage alloc] initWithBitmapImageRep:inRep];
CIImage *outputImage = // filter, rinse, repeat
// output to NSImage
NSCIImageRep *outRep = [NSCIImageRep imageRepWithCIImage: outputImage];
NSImage *outNSImage = [[NSImage alloc] init];
[outNSImage addRepresentation: outRep];

Related

How can I extract image information dumped in NSData

I have as an input the dump of an image in an NSData object. Now, I want to extract relevant information of the image from this object like number of pixels, no. of bits per pixel, etc.
Can anyone tell me how to extract this info from the NSData object dump?
P.S.: I have gone through this documentation of the NSData class, but could not isolate out the relevant methods.
So the easiest way is to actually build the UIImage object from the NSData then extract the info from the UIImage then.
UIImage* image = [UIImage imageWithData:yourData];
NSLog(#"Image is %dx%d",image.size.width, image.size.height);
If you are only interested in the properties of the image but don't want to actually build its representation and only get the properties, take a look at CGImageSource
#import <ImageIO/ImageIO.h>
CGImageSourceRef imgSrc = CGImageSourceCreateWithData((__bridge CFDataRef)data, NULL);
size_t nbImages = CGImageSourceGetCount(imgSrc);
for(size_t idx=0; idx<nbImages; ++idx)
{
NSDictionary* props = (__bridge NSDictionary*)CGImageSourceCopyPropertiesAtIndex(imgSrc, idx, NULL);
NSLog(#"properties for image %lu in imageSource: %#", idx, props);
}
CFRelease(imgSrc);
[EDIT] For this to work, obviously add the ImageIO.framework to your "Link Binary With Libraries"
Convert the data to UIImage, then take a look at this post.
UIImage *image = [UIImage imageWithData:data];

How do I create a valid CGImageSourceRef from an ALAssetRepresentation?

I'm trying to use CGImageSourceCreateThumbnailAtIndex to efficiently create a resized version of an image. I have some existing code that does this with images from disk, and now I'm trying to use an image that comes from ALAssetsLibrary.
Here's my code:
ALAsset *asset;
ALAssetRepresentation *representation = [asset defaultRepresentation];
CGImageRef imageRef = [representation fullResolutionImage];
CGDataProviderRef provider = CGImageGetDataProvider(imageRef);
CGImageSourceRef sourceRef = CGImageSourceCreateWithDataProvider(provider, NULL);
NSDictionary *resizeOptions = #{
kCGImageSourceCreateThumbnailWithTransform : #YES,
kCGImageSourceCreateThumbnailFromImageAlways : #YES,
kCGImageSourceThumbnailMaxPixelSize : #(2100)
};
CGImageRef resizedImage = CGImageSourceCreateThumbnailAtIndex(source, 0, resizeOptions);
The problem is that resizedImage is null, and CGImageSourceGetCount(sourceRef) returns 0. The data provider does have quite a bit of data in it, though, and the data does appear to be valid image data. The ALAsset comes from an iPhone 4S camera roll.
What am I missing? Why does CGImageSourceCreateWithDataProvider() create an image source with 0 images?
CGImageSource is for deserializing serialized images, such as JPEGs, PNGs, and whatnot.
CGImageGetDataProvider returns (the provider of) the raw pixel data of the image. It does not return serialized bytes in some external format. CGImageSource has no way to know what pixel format (color space, bits-per-component, alpha layout, etc.) any given raw pixel data is in.
You could try getting the URL of the asset rep and giving that to CGImageSourceCreateWithURL. If that doesn't work (e.g., not a file URL), you'll have to run the image through a CGImageDestination and create a CGImageSource with wherever you put the output.
(The one other thing to try would be to see whether the rep's filename is actually a full path, the way Cocoa often misuses the term. But you probably shouldn't count on that.)
One thing you might try is the asset rep's CGImageWithOptions: method.
The documentation claims that:
This method returns the biggest, best representation available, unadjusted in any way.
But it says that about fullResolutionImage, too, and I'm not sure why this class would have both methods if they both do the same thing. I wonder if it's a copy-and-paste error.
Try CGImageWithOptions: with a bunch of thumbnail-creating options and see what happens.
Option #3 would be the rep's fullScreenImage. Depending on what sort of “thumbnail” you need, it may be cheaper and/or simpler to just use this, which will be no bigger than (approximately) the size of the device's screen.
This can also help...
ALAssetRepresentation* rep = [asset defaultRepresentation];
NSDictionary* options = [[NSDictionary alloc] initWithObjectsAndKeys:
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailWithTransform,
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailFromImageAlways,
(id)[NSNumber numberWithDouble:400], (id)kCGImageSourceThumbnailMaxPixelSize,
nil];
CGImageRef image = [rep CGImageWithOptions:options];

rotate CGImage on disk using CGImageSource/CGImageDestination?

I'm working on an application that needs to take a picture using UIImagePickerController, display a thumbnail of it in the app, and also submit that picture to a server using ASIFormDataRequest (from ASIHTTPRequest).
I want to use setFile:withFileName:andContentType:forKey: from ASIFormDataRequest since in my experience it's faster than trying to submit an image using UIImageJPEGRepresentation and submitting raw NSData. To that end I'm using CGImageDestination and creating an image destination with a url, saving that image to disk, and then uploading that file on disk.
In order to create the thumbnail I'm using CGImageSourceCreateThumbnailAtIndex (see docs) and creating an image source with the path of the file I just saved.
My problem is that no matter what options I pass into the image destination or the thumbnail creation call, my thumbnail always comes out rotated 90 degrees counterclockwise. The uploaded image is also rotated. I've tried explicitly setting the orientation in the options of the image using CGImageDestinationSetProperties but it doesn't seem to take. The only solution I've found is to rotate the image in memory, but I really want to avoid that since doing so doubles the time it takes for the thumbnail+saving operation to finish. Ideally I'd like to be able to rotate the image on disk.
Am I missing something in how I'm using CGImageDestination or CGImageSource? I'll post some code below.
Saving the image:
NSURL *filePath = [NSURL fileURLWithPath:self.imagePath];
CGImageRef imageRef = [self.image CGImage];
CGImageDestinationRef ref = CGImageDestinationCreateWithURL((CFURLRef)filePath, kUTTypeJPEG, 1, NULL);
CGImageDestinationAddImage(ref, imageRef, NULL);
NSDictionary *props = [[NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithFloat:1.0], kCGImageDestinationLossyCompressionQuality,
nil] retain];
//Note that setting kCGImagePropertyOrientation didn't work here for me
CGImageDestinationSetProperties(ref, (CFDictionaryRef) props);
CGImageDestinationFinalize(ref);
CFRelease(ref);
Generating the thumbnail
CGImageSourceRef imageSource = CGImageSourceCreateWithURL((CFURLRef)filePath, NULL);
if (!imageSource)
return;
CFDictionaryRef options = (CFDictionaryRef)[NSDictionary dictionaryWithObjectsAndKeys:
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailWithTransform,
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailFromImageIfAbsent,
(id)[NSNumber numberWithFloat:THUMBNAIL_SIDE_LENGTH], (id)kCGImageSourceThumbnailMaxPixelSize, nil];
CGImageRef imgRef = CGImageSourceCreateThumbnailAtIndex(imageSource, 0, options);
UIImage *thumb = [UIImage imageWithCGImage:imgRef];
CGImageRelease(imgRef);
CFRelease(imageSource);
And then to upload the image I just use
[request setFile:path withFileName:fileName andContentType:contentType forKey:#"photo"];
where path is the path to the file saved with the code above.
As far as I know and after trying lots of different things, this cannot be done with current public APIs and has to be done in memory.

How to get [UIImage imageWithContentsOfFile:] and High Res Images working

As many people are complaining it seems that in the Apple SDK for the Retina Display there's a bug and imageWithContentsOfFile actually does not automatically load the 2x images.
I've stumbled into a nice post how to make a function which detects UIScreen scale factor and properly loads low or high res images ( http://atastypixel.com/blog/uiimage-resolution-independence-and-the-iphone-4s-retina-display/ ), but the solution loads a 2x image and still has the scale factor of the image set to 1.0 and this results to a 2x images scaled 2 times (so, 4 times bigger than what it has to look like)
imageNamed seems to accurately load low and high res images, but is no option for me.
Does anybody have a solution for loading low/high res images not using the automatic loading of imageNamed or imageWithContentsOfFile ? (Or eventually solution how to make imageWithContentsOfFile work correct)
Ok, actual solution found by Michael here :
http://atastypixel.com/blog/uiimage-resolution-independence-and-the-iphone-4s-retina-display/
He figured out that UIImage has the method "initWithCGImage" which also takes a scale factor as input (I guess the only method where you can set yourself the scale factor)
[UIImage initWithCGImage:scale:orientation:]
And this seems to work great, you can custom load your high res images and just set that the scale factor is 2.0
The problem with imageWithContentsOfFile is that since it currently does not work properly, we can't trust it even when it's fixed (because some users will still have an older iOS on their devices)
We just ran into this here at work.
Here is my work-around that seems to hold water:
NSString *imgFile = ...path to your file;
NSData *imgData = [[NSData alloc] initWithContentsOfFile:imgFile];
UIImage *img = [[UIImage alloc] initWithData:imgData];
imageWithContentsOfFile works properly (considering #2x images with correct scale) starting iOS 4.1 and onwards.
Enhancing Lisa Rossellis's answer to keep retina images at desired size (not scaling them up):
NSString *imagePath = ...Path to your image
UIImage *image = [UIImage imageWithData:[NSData dataWithContentsOfFile:imagePath] scale:[UIScreen mainScreen].scale];
I've developed a drop-in workaround for this problem.
It uses method swizzling to replace the behavior of the "imageWithContentsOfFile:" method of UIImage.
It works fine on iPhones/iPods pre/post retina.
Not sure about the iPad.
Hope this is of help.
#import </usr/include/objc/objc-class.h>
#implementation NSString(LoadHighDef)
/** If self is the path to an image, returns the nominal path to the high-res variant of that image */
-(NSString*) stringByInsertingHighResPathModifier {
NSString *path = [self stringByDeletingPathExtension];
// We determine whether a device modifier is present, and in case it is, where is
// the "split position" at which the "#2x" token is to be added
NSArray *deviceModifiers = [NSArray arrayWithObjects:#"~iphone", #"~ipad", nil];
NSInteger splitIdx = [path length];
for (NSString *modifier in deviceModifiers) {
if ([path hasSuffix:modifier]) {
splitIdx -= [modifier length];
break;
}
}
// We insert the "#2x" token in the string at the proper position; if no
// device modifier is present the token is added at the end of the string
NSString *highDefPath = [NSString stringWithFormat:#"%##2x%#",[path substringToIndex:splitIdx], [path substringFromIndex:splitIdx]];
// We possibly add the extension, if there is any extension at all
NSString *ext = [self pathExtension];
return [ext length]>0? [highDefPath stringByAppendingPathExtension:ext] : highDefPath;
}
#end
#implementation UIImage (LoadHighDef)
/* Upon loading this category, the implementation of "imageWithContentsOfFile:" is exchanged with the implementation
* of our custom "imageWithContentsOfFile_custom:" method, whereby we replace and fix the behavior of the system selector. */
+(void)load {
Method originalMethod = class_getClassMethod([UIImage class], #selector(imageWithContentsOfFile:));
Method replacementMethod = class_getClassMethod([UIImage class], #selector(imageWithContentsOfFile_custom:));
method_exchangeImplementations(replacementMethod, originalMethod);
}
/** This method works just like the system "imageWithContentsOfFile:", but it loads the high-res version of the image
* instead of the default one in case the device's screen is high-res and the high-res variant of the image is present.
*
* We assume that the original "imageWithContentsOfFile:" implementation properly sets the "scale" factor upon
* loading a "#2x" image . (this is its behavior as of OS 4.0.1).
*
* Note: The "imageWithContentsOfFile_custom:" invocations in this code are not recursive calls by virtue of
* method swizzling. In fact, the original UIImage implementation of "imageWithContentsOfFile:" gets called.
*/
+ (UIImage*) imageWithContentsOfFile_custom:(NSString*)imgName {
// If high-res is supported by the device...
UIScreen *screen = [UIScreen mainScreen];
if ([screen respondsToSelector:#selector(scale)] && [screen scale]>=2.0) {
// then we look for the high-res version of the image first
UIImage *hiDefImg = [UIImage imageWithContentsOfFile_custom:[imgName stringByInsertingHighResPathModifier]];
// If such high-res version exists, we return it
// The scale factor will be correctly set because once you give imageWithContentsOfFile:
// the full hi-res path it properly takes it into account
if (hiDefImg!=nil)
return hiDefImg;
}
// If the device does not support high-res of it does but there is
// no high-res variant of imgName, we return the base version
return [UIImage imageWithContentsOfFile_custom:imgName];
}
#end
[UIImage imageWithContentsOfFile:] doesn't load #2x graphics if you specify an absolute path.
Here is a solution:
- (UIImage *)loadRetinaImageIfAvailable:(NSString *)path {
NSString *retinaPath = [[path stringByDeletingLastPathComponent] stringByAppendingPathComponent:[NSString stringWithFormat:#"%##2x.%#", [[path lastPathComponent] stringByDeletingPathExtension], [path pathExtension]]];
if( [UIScreen mainScreen].scale == 2.0 && [[NSFileManager defaultManager] fileExistsAtPath:retinaPath] == YES)
return [[[UIImage alloc] initWithCGImage:[[UIImage imageWithData:[NSData dataWithContentsOfFile:retinaPath]] CGImage] scale:2.0 orientation:UIImageOrientationUp] autorelease];
else
return [UIImage imageWithContentsOfFile:path];
}
Credit goes to Christof Dorner for his simple solution (which I modified and pasted here).

Error saving NSImage as NSData

I am using the following code to save a frame of a movie to my desktop:
NSCIImageRep *imageRep = [NSCIImageRep imageRepWithCIImage:[CIImage imageWithCVImageBuffer:imageBuffer]];
NSImage *image = [[[NSImage alloc] initWithSize:[imageRep size]] autorelease];
[image addRepresentation:imageRep];
CVBufferRelease(imageBuffer);
NSArray *representations = [image representations];
NSData *bitmapData = [NSBitmapImageRep representationOfImageRepsInArray:representations usingType:NSJPEGFileType properties:nil];
[bitmapData writeToFile:#"/Users/ricky/Desktop/MyImage.jpeg" atomically:YES];
At the second last line of code, I receive the following messages in the console, with no result being saved to the desktop:
<Error>: CGImageDestinationFinalize image destination does not have enough images
CGImageDestinationFinalize failed for output type 'public.jpeg'
The NSImage is still an allocated object for the entire method call, so I'm not sure why I am receiving complaints about insufficient amount of images.
I'd appreciate any help.
Thanks in advance,
Ricky.
I think the source of the problem is that you're passing an array of NSCIImageRep objects to representationOfImageRepsInArray:usingType:properties:, which I believe expects an array of NSBitmapImageRep objects.
What you want to do is create an NSBitmapImageRep from your CIImage. Then you can use that to write to disk. That would be roughly:
CIImage *myImage = [CIImage imageWithCVImageBuffer:imageBuffer];
NSBitmapImageRep *bitmapRep = [[NSBitmapImageRep alloc] initWithCIImage:myImage];
NSData *jpegData [bitmapRep representationUsingType:NSJPEGFileType properties:nil];
[jpegData writeToFile:#"/Users/ricky/Desktop/MyImage.jpeg" atomically:YES];
Of course, you'd want to handle any error cases and probably pass a properties dictionary to fine-tune the JPEG creation.
I'm sorry i don't really know why your code doesn't work, but approaching it a different way (and i think more efficiently than your CVImageBuffer to CIImage to NSCIImageRep to NSImage to NSData, albeit at a slightly lower level):-
CVImageBuffer to CGImage
CGImage to jpg file
I don't have code ready made to do this but extracting the right stuff from those examples should be straight forward.