I'm trying to use CGImageSourceCreateThumbnailAtIndex to efficiently create a resized version of an image. I have some existing code that does this with images from disk, and now I'm trying to use an image that comes from ALAssetsLibrary.
Here's my code:
ALAsset *asset;
ALAssetRepresentation *representation = [asset defaultRepresentation];
CGImageRef imageRef = [representation fullResolutionImage];
CGDataProviderRef provider = CGImageGetDataProvider(imageRef);
CGImageSourceRef sourceRef = CGImageSourceCreateWithDataProvider(provider, NULL);
NSDictionary *resizeOptions = #{
kCGImageSourceCreateThumbnailWithTransform : #YES,
kCGImageSourceCreateThumbnailFromImageAlways : #YES,
kCGImageSourceThumbnailMaxPixelSize : #(2100)
};
CGImageRef resizedImage = CGImageSourceCreateThumbnailAtIndex(source, 0, resizeOptions);
The problem is that resizedImage is null, and CGImageSourceGetCount(sourceRef) returns 0. The data provider does have quite a bit of data in it, though, and the data does appear to be valid image data. The ALAsset comes from an iPhone 4S camera roll.
What am I missing? Why does CGImageSourceCreateWithDataProvider() create an image source with 0 images?
CGImageSource is for deserializing serialized images, such as JPEGs, PNGs, and whatnot.
CGImageGetDataProvider returns (the provider of) the raw pixel data of the image. It does not return serialized bytes in some external format. CGImageSource has no way to know what pixel format (color space, bits-per-component, alpha layout, etc.) any given raw pixel data is in.
You could try getting the URL of the asset rep and giving that to CGImageSourceCreateWithURL. If that doesn't work (e.g., not a file URL), you'll have to run the image through a CGImageDestination and create a CGImageSource with wherever you put the output.
(The one other thing to try would be to see whether the rep's filename is actually a full path, the way Cocoa often misuses the term. But you probably shouldn't count on that.)
One thing you might try is the asset rep's CGImageWithOptions: method.
The documentation claims that:
This method returns the biggest, best representation available, unadjusted in any way.
But it says that about fullResolutionImage, too, and I'm not sure why this class would have both methods if they both do the same thing. I wonder if it's a copy-and-paste error.
Try CGImageWithOptions: with a bunch of thumbnail-creating options and see what happens.
Option #3 would be the rep's fullScreenImage. Depending on what sort of “thumbnail” you need, it may be cheaper and/or simpler to just use this, which will be no bigger than (approximately) the size of the device's screen.
This can also help...
ALAssetRepresentation* rep = [asset defaultRepresentation];
NSDictionary* options = [[NSDictionary alloc] initWithObjectsAndKeys:
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailWithTransform,
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailFromImageAlways,
(id)[NSNumber numberWithDouble:400], (id)kCGImageSourceThumbnailMaxPixelSize,
nil];
CGImageRef image = [rep CGImageWithOptions:options];
Related
Somehow I cannot figure out to get an actual, meaningful histogram image from an NSImage input using the CIAreaHistogram and CIHistogramDisplayFilter filters.
I read Apple's "Core Image Filter Reference" and the relevant posts here on SO, but whatever I try I get no meaningful output.
Here's my code so far:
- (void) testHist3:(NSImage *)image {
CIContext* context = [[NSGraphicsContext currentContext] CIContext];
NSBitmapImageRep *rep = [image bitmapImageRepresentation];
CIImage *ciImage = [[CIImage alloc] initWithBitmapImageRep:rep];
ciImage = [CIFilter filterWithName:#"CIAreaHistogram" keysAndValues:kCIInputImageKey, ciImage, #"inputExtent", ciImage.extent, #"inputScale", [NSNumber numberWithFloat:1.0], #"inputCount", [NSNumber numberWithFloat:256.0], nil].outputImage;
ciImage = [CIFilter filterWithName:#"CIHistogramDisplayFilter" keysAndValues:kCIInputImageKey, ciImage, #"inputHeight", [NSNumber numberWithFloat:100.0], #"inputHighLimit", [NSNumber numberWithFloat:1.0], #"inputLowLimit", [NSNumber numberWithFloat:0.0], nil].outputImage;
CGImageRef cgImage2 = [context createCGImage:ciImage fromRect:ciImage.extent];
NSImage *img2 = [[NSImage alloc] initWithCGImage:cgImage2 size:ciImage.extent.size];
NSLog(#"Histogram image: %#", img2);
self.histImage = img2;
}
What I get is a 64x100 image with zero representations (=invisible). If I create the CI context with
CIContext *context = [[CIContext alloc] init];
then the resulting image is grey, but at least it does have a representation:
Histogram image: <NSImage 0x6100002612c0 Size={64, 100} Reps=(
"<NSCGImageSnapshotRep:0x6100002620c0 cgImage=<CGImage 0x6100001a1880>>" )>
The input image is a 1024x768 JPEG image.
I have little experience with Core Image or Core Graphics, so the mistake might be with the conversion back to NSImage... any ideas?
Edit 2016-10-26: With rickster's very comprehensive answer I was able to make a lot of progress.
Indeed it was the inputExtent parameter that was messing up my result. Supplying a CIVector there solved the problem. I found that you cannot leave that to the default either; I don't know what the default value is, but it is not the input image's full size. (I found that out by running an image and a mirrored version of it through the filter; I got different histograms.)
Edit 2016-10-28:
So, I've got a working, displayable histogram now; my next step will be to figure out how the "intermediate" histogram (the 256x1 pixel image coming out of the filter) can contain the actual histogram information even though all but the last pixel are always (0, 0, 0, 0).
I presume the [image bitmapImageRepresentation] in your code is a local category method that's roughly equivalent to (NSBitmapImageRep *)image.representations[0]? Otherwise, first make sure that you're getting the right input.
Next, it looks like you're passing the raw output of ciImage.extent into your filter parameters — given that said parameter expects a CIVector object and not a CGRect struct, you're probably borking the input to your filter at run time. You can get a bit more useful diagnostics for such problems by using the dictionary-based filter methods filterWithName:withInputParameters or imageByApplyingFilter:withInputParameters: — that way, if you try to pass nil for a filter key or pass something that isn't a proper object, you'll get a compile-time error. The latter gives you an easy way to go straight from input image to output image, or chain filters, without creating intermediary CIFilter objects and needing to set the input image on each.
A related tip: most of the parameters you're passing are the default values for those filters, so you can pass only the values you need:
CIImage *hist = [inputImage imageByApplyingFilter:#"CIAreaHistogram"
withInputParameters:#{ #"inputCount": #256 }];
CIImage *outputImage = [hist imageByApplyingFilter:#"CIHistogramDisplayFilter"
withInputParameters:nil];
Finally, you might still get an almost-all-gray image out of CIHistogramDisplayFilter depending on what your input image looks like, because all of the histogram bins may have very small bars. I get the following for Lenna:
Increasing the value for kCIInputScaleKey can help with that.
Also, you don't need to go through CGImage to get from CIImage to NSImage — create an NSCIImageRep instead and AppKit will automatically manage a CIContext behind the scenes when it comes time to render the image for display/output.
// input from NSImage
NSBitmapImageRep *inRep = [nsImage bitmapImageRepresentation];
CIImage *inputImage = [[CIImage alloc] initWithBitmapImageRep:inRep];
CIImage *outputImage = // filter, rinse, repeat
// output to NSImage
NSCIImageRep *outRep = [NSCIImageRep imageRepWithCIImage: outputImage];
NSImage *outNSImage = [[NSImage alloc] init];
[outNSImage addRepresentation: outRep];
I have an animated GIF successfully loaded into an NSData or NSBitmapImageRep object. Reference for NSBitmapImageRep
I've figured how to return data like the number of frames in that gif using:
NSNumber *frames = [bitmapRep valueForProperty:#"NSImageFrameCount"];
However, I'm a bit confused as to how I can actually access that frame as its own object.
I think one of these two methods will help, but I'm not actually sure how they'll get the individual frame for me.
+ representationOfImageRepsInArray:usingType:properties:
– representationUsingType:properties:
Any help appreciated. Thanks
I've figured how to return data like the number of frames in that gif using:
NSNumber *frames = [bitmapRep valueForProperty:#"NSImageFrameCount"];
However, I'm a bit confused as to how I can actually access that frame as its own object.
To have access to a special frame indexOfFrame ( 0 <= indexOfFrame < [frames intValue] ) you only need to set the NSImageCurrentFrame and you are done. There is no need to use CG-functions or make copies of frames. You can stay in the object oriented Cocoa world. A small example shows the duration of all GIF frames:
NSNumber *frames = [bitmapRep valueForProperty:#"NSImageFrameCount"];
if( frames!=nil ){ // bitmapRep is a Gif imageRep
for( NSUInteger i=0; i<[frames intValue]; i++ ){
[bitmapRep setProperty:NSImageCurrentFrame
withValue:[NSNumber numberWithUnsignedInt:i] ];
NSLog(#"%2d duration=%#",
i, [bitmapRep valueForProperty:NSImageCurrentFrameDuration] );
}
}
Another example: write all frames of a GIF image as PNG files to the filesystem:
NSNumber *frames = [bitmapRep valueForProperty:#"NSImageFrameCount"];
if( frames!=nil ){ // bitmapRep is a Gif imageRep
for( NSUInteger i=0; i<[frames intValue]; i++ ){
[bitmapRep setProperty:NSImageCurrentFrame
withValue:[NSNumber numberWithUnsignedInt:i] ];
NSData *repData = [bitmapRep representationUsingType:NSPNGFileType
properties:nil];
[repData writeToFile:
[NSString stringWithFormat:#"/tmp/gif_%02d.png", i ] atomically:YES];
}
}
I've figured how to return data like the number of frames in that gif using:
NSNumber *frames = [bitmapRep valueForProperty:#"NSImageFrameCount"];
However, I'm a bit confused as to how I can actually access that frame as its own object.
As far as I know, you can't—not from an NSBitmapImageRep.
Instead, create a CGImageSource from the GIF data, and use CGImageSourceCreateImageAtIndex to extract each frame (preferably as you need it).
Alternatively, you might try setting the NSImageCurrentFrame property. If you need a rep for each frame, make as many copies as there are frames (minus one, since you have the original), and set each rep's current frame to a different number. But I haven't tried that, so I'm not sure it will actually work.
Basically, NSBitmapImageRep's GIF support is weird, so you should just use CGImageSource.
I think one of these two methods will help, but I'm not actually sure how they'll get the individual frame for me.
+ representationOfImageRepsInArray:usingType:properties:
– representationUsingType:properties:
No, those methods are for serializing an image (or image rep). They're for writing data out, not reading it in. (Notice what constants those methods expect in their type parameters.)
If you want to have a look at some working source code for a GIF decoder for iOS (works for MacOSX too) then you can find it AVGIF89A2MvidResourceLoader.m at github. The approach is to use the ImageIO framework and call CGImageSourceCreateWithData() along with CGImageSourceCreateImageAtIndex() to get access to the Nth gif image in the file. But, there are some tricky details related to detecting if a transparent pixel appears in the GIF and how to write the results to a file to avoid running out of memory if the GIF is really long that might not be obvious.
Apparently with CGImages you can add kCGImagePropertyPNGXPixelsPerMeter and kCGImagePropertyPNGYPixelsPerMeter properties to the image and they will get added to the pHYs chunk of the png. But UIImageWriteToSavedPhotosAlbum doesn't directly accept a CGImage.
I've been trying to load the image as a UIImage, then create a CGImage from that, add the properties, and then create a new UIImage from that data and saving that to the photo album.
An image is getting written to the photo album, but without the PPM settings. I can verify this in the preview app on MacOS, or using ImageMagick's identify.
ObjC isn't my domain, so here's what I've cobbled together from similar stackoverflow questions.
UIImage *uiImage = [UIImage imageWithContentsOfFile:path];
NSDictionary* properties = [NSDictionary dictionaryWithObjectsAndKeys:
[NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:5905], (NSString *)kCGImagePropertyPNGXPixelsPerMeter, /* this doesn't work */
[NSNumber numberWithInt:5905], (NSString *)kCGImagePropertyPNGYPixelsPerMeter, /* this doesn't work */
#"This works when saved to disk.", (NSString *)kCGImagePropertyPNGDescription,
nil],(NSString*) kCGImagePropertyPNGDictionary,
nil];
NSMutableData* imageData = [NSMutableData data];
CGImageDestinationRef imageDest = CGImageDestinationCreateWithData((CFMutableDataRef) imageData, kUTTypePNG, 1, NULL);
CGImageDestinationAddImage(imageDest, uiImage.CGImage, (CFDictionaryRef) properties);
CGImageDestinationSetProperties(imageDest, (CFDictionaryRef) properties); /* I'm setting the properties twice because I was going crazy */
CGImageDestinationFinalize(imageDest);
UIImageWriteToSavedPhotosAlbum( [UIImage imageWithData:imageData], nil, NULL, NULL );
// This is a test to see if the data is getting stored at all.
CGImageDestinationRef diskDest = CGImageDestinationCreateWithURL(
(CFURLRef)[NSURL fileURLWithPath:[NSString stringWithFormat:#"%#.dpi.png",path]], kUTTypePNG, 1, NULL);
CGImageDestinationAddImage(diskDest, uiImage.CGImage, (CFDictionaryRef) properties);
CGImageDestinationSetProperties(diskDest, (CFDictionaryRef) properties);
CGImageDestinationFinalize(diskDest);
Updated: Revised and simplified the code a bit. Still saves image to photo album and disk, but the pixels per meter value is not saving. The description block gets saved to disk, but appears to get stripped when written to the photo album.
Update 2: By saving the file as a jpeg and adding TIFF X/Y Resolution tags to it I've been able to get the version saved to disk to store a PPI value other than 72. This information is still stripped off when saving to the photo album.
Comparing a photo taken via iOS Camera app, one taken via Instagram, and one saved by my code, I noticed that the Camera app version has a ton of TIFF/EXIF tags, and the Instagram and my one have the same limited set of tags. This leads me to the conclusion that iOS strips these tags intentionally. A definitive answer would help me sleep at night though.
The answer is don't use UIImageWriteToSavedPhotosAlbum.
There's an ALAssetsLibrary method
[writeImageDataToSavedPhotosAlbum: metadata: completionBlock:]
The metadata is a dictionary in the same format as the one passed to CGImageDestinationSetProperties. PNG and JFIF density settings may be broken, I'm using kCGImagePropertyTIFFXResolution.
I'm working on an application that needs to take a picture using UIImagePickerController, display a thumbnail of it in the app, and also submit that picture to a server using ASIFormDataRequest (from ASIHTTPRequest).
I want to use setFile:withFileName:andContentType:forKey: from ASIFormDataRequest since in my experience it's faster than trying to submit an image using UIImageJPEGRepresentation and submitting raw NSData. To that end I'm using CGImageDestination and creating an image destination with a url, saving that image to disk, and then uploading that file on disk.
In order to create the thumbnail I'm using CGImageSourceCreateThumbnailAtIndex (see docs) and creating an image source with the path of the file I just saved.
My problem is that no matter what options I pass into the image destination or the thumbnail creation call, my thumbnail always comes out rotated 90 degrees counterclockwise. The uploaded image is also rotated. I've tried explicitly setting the orientation in the options of the image using CGImageDestinationSetProperties but it doesn't seem to take. The only solution I've found is to rotate the image in memory, but I really want to avoid that since doing so doubles the time it takes for the thumbnail+saving operation to finish. Ideally I'd like to be able to rotate the image on disk.
Am I missing something in how I'm using CGImageDestination or CGImageSource? I'll post some code below.
Saving the image:
NSURL *filePath = [NSURL fileURLWithPath:self.imagePath];
CGImageRef imageRef = [self.image CGImage];
CGImageDestinationRef ref = CGImageDestinationCreateWithURL((CFURLRef)filePath, kUTTypeJPEG, 1, NULL);
CGImageDestinationAddImage(ref, imageRef, NULL);
NSDictionary *props = [[NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithFloat:1.0], kCGImageDestinationLossyCompressionQuality,
nil] retain];
//Note that setting kCGImagePropertyOrientation didn't work here for me
CGImageDestinationSetProperties(ref, (CFDictionaryRef) props);
CGImageDestinationFinalize(ref);
CFRelease(ref);
Generating the thumbnail
CGImageSourceRef imageSource = CGImageSourceCreateWithURL((CFURLRef)filePath, NULL);
if (!imageSource)
return;
CFDictionaryRef options = (CFDictionaryRef)[NSDictionary dictionaryWithObjectsAndKeys:
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailWithTransform,
(id)kCFBooleanTrue, (id)kCGImageSourceCreateThumbnailFromImageIfAbsent,
(id)[NSNumber numberWithFloat:THUMBNAIL_SIDE_LENGTH], (id)kCGImageSourceThumbnailMaxPixelSize, nil];
CGImageRef imgRef = CGImageSourceCreateThumbnailAtIndex(imageSource, 0, options);
UIImage *thumb = [UIImage imageWithCGImage:imgRef];
CGImageRelease(imgRef);
CFRelease(imageSource);
And then to upload the image I just use
[request setFile:path withFileName:fileName andContentType:contentType forKey:#"photo"];
where path is the path to the file saved with the code above.
As far as I know and after trying lots of different things, this cannot be done with current public APIs and has to be done in memory.
I can create UIImage from NSData using [UIImage imageWithData:] or [UIImage initWithData:] methods.
I wonder if I can get the NSData back from an existing UIImage?
Something on the line of NSData *myData = [myImage getData];
NSData *imageData = UIImageJPEGRepresentation(image, 0.7); // 0.7 is JPG quality
or
NSData *imageData = UIImagePNGRepresentation(image);
Depending if you want your data in PNG format or JPG format.
When initialising a UIImage object with init(data: originalData), that originalData will be converted into raw data in some kind of internal format. These data can be retrieved later with
let rawData = myImage.cgImage?.dataProvider?.data as Data?
However because the rawData is raw, it is going to be even larger than when using UIImagePNGRepresentation.
Swift 4.2
let dataPng = image.pngData() // return image as PNG. May return nil if image has no CGImageRef or invalid bitmap format
let dataJpg = image.jpegData(compressionQuality: 1) // return image as JPEG. May return nil if image has no CGImageRef or invalid bitmap format. compression is 0(most)..1(least)
Just because I stumbled upon this and i like swift :)
Here is the swift translation of Caroiline's post.
var imageData = UIImagePNGRepresentation(image)
Or
var imageData = UIImageJPEGRepresentation(image, 0.7)
You can expect that a UIImage is an object formatted for display and so won't be using the original data (which is probably in PNG or JPEG format) but more likely a pixel array or some other internal format. In other words, UIImage(data: foo) will not retain foo.
If you just want to use it elsewhere in your program, the original UIImage will do fine (I presume that's not actually the case here)
If you want to serialise, UIImagePNGRepresentation(...) will work but will be oversized if the original was a JPEG; UIImageJPEGRepresentation(...) will often result in slightly oversize data and is slightly lossy if your original was PNG. It should be okay to pick one based on the way the image will be displayed and the format you expect to be provided. If you happen to be using PNG in and want PNG out, you should get a good file size and almost identical data, special PNG chunks aside.
If you want to get an exact copy of the original data (perhaps to save a file after thumbnailing, or to SHA1 it), then you need to retain it separately. You might do something like:
var image:UIImage
var imageData:NSData {
didSet {
image = UIImage(data: imageData)
}
}
The only solution I found to serialize/unserialize a UIImage (via Data) is by using this solution.
You can then serialize/unserialize regardless of how the UIImage was created by using the extension method on UIImage:
let originalImage: UIImage = ...
let cgData = image.cgImage!.png!
let image = UIImage(data: cgData)!
Things have changed since the above answer was given, for those still looking because they share CipherCom's concern: iOS 5 has added the CIImage property.