I'm learning about drawing UIImages and CGImages, using CIFilters etc. To test my knowledge I made a small test app with sliders that programmatically change the color of a potion sprite and display it on screen (using a CIHueBlendMode CIFilter). After I finished, I wanted to cleanup the relatively lengthy code and noticed that instead of going from the filter's outputted CIImage to a CGImage and then a UIImage, I could go directly from a CIImage to UIImage using UIImage's imageWithCIImage: method.
However, when I tried to draw the resultant UIImage using drawInRect:, nothing was drawn. Going through the CGImage stage rectifies this, of course. My understanding of this is that making a UIImage from a CIImage results in a NULL CGImage property in the UIImage, which is used in drawInRect:. Is this correct? If so, is there a better way to display a CIImage than to go through CGImage followed by UIImage? I could just draw a CGImage made with the CIImage, but that would flip the image, which leads to another question. Currently, I wrap anything I draw in a UIImage first to take care of flipping. Is there another, more efficient way?
Too Long; Didn't Read: Is there a better way to draw CIImages other than turning it into a CGImage, then a UIImage and drawing that? What's the best way to handle flipping when drawing CGImages?
Thanks to anyone who can answer some of my questions. :)
After doing some research into what a CIImage is, I realize now that you cannot skip the step of making a CGImage from the CIImage, and even if you could, it wouldn't really be any more efficient, since you'd still have to process the CIImage regardless. A CIImage is not really an image, as noted in Apple's documentation, which is processed when it's turned into a CGImage. That's also why if I use Time Profiler on my project I see that 99% of my time in my drawRect: method is spent on createCGImage:, and not using CIFilters.
As for the most efficient way to cope with the coordinate system change between Core Graphics and the iPhone, it seems that wrapping the object in a UIImage instance is the easiest (not sure about best) way to go. It's simple, and relatively efficient. Another option would be to transform the graphics context.
If I don't get a better answer than my own within three days, I'll mark it as accepted.
Related
I wrote the following code to apply a Sepia filter to an image:
- (void)applySepiaFilter {
// Set previous image
NSData *buffer = [NSKeyedArchiver archivedDataWithRootObject: self.mainImage.image];
[_images push:[NSKeyedUnarchiver unarchiveObjectWithData: buffer]];
UIImage* u = self.mainImage.image;
CIImage *image = [[CIImage alloc] initWithCGImage:u.CGImage];
CIFilter *filter = [CIFilter filterWithName:#"CISepiaTone"
keysAndValues: kCIInputImageKey, image,
#"inputIntensity", #0.8, nil];
CIImage *outputImage = [filter outputImage];
self.mainImage.image = [self imageFromCIImage:outputImage];
}
- (UIImage *)imageFromCIImage:(CIImage *)ciImage {
CIContext *ciContext = [CIContext contextWithOptions:nil];
CGImageRef cgImage = [ciContext createCGImage:ciImage fromRect:[ciImage extent]];
UIImage *image = [UIImage imageWithCGImage:cgImage];
CGImageRelease(cgImage);
return image;
}
When I run this code it seems to lag for 1-2 seconds. I heard that core image is faster than core graphics but I am unimpressed with the rendering time. I was wondering if this would be faster processing in CoreGraphics or even OpenCV(which is being used elsewhere in the project)? If not is there any way I can optimize this code to run faster?
I can almost guarantee it will be slower in Core Graphics than using Core Image, depending on the size of the image. If the image is small, Core Graphics may be fine, but if you are doing a lot of processing, it will be much slower than rendering using the GPU.
Core Image is very fast, however, you have to be very conscious of what is going on. Most of the performance hit with Core Image is due to setting up of the context, and copying images to/from Core Image. In addition to just copying bytes, Core Image may be converting between image formats as well.
Your code is doing the following every time:
Creating a CIContext. (slow)
Taking bytes from a CGImage and creating a CIImage.
Copying image data to GPU (slow).
Processing Sepia filter (fast).
Copying result image back to CGImage. (slow)
This is not a recipe for peak performance. Bytes from CGImage will typically live in CPU memory, but Core Image wants to use the GPU for its processing.
An excellent reference for performance considerations are provided in Getting the Best Performance documentation for Core Image:
Don’t create a CIContext object every time you render.
Contexts store a lot of state information; it’s more efficient to reuse them.
Evaluate whether you app needs color management. Don’t use it unless you need it. See Does Your App Need Color Management?.
Avoid Core Animation animations while rendering CIImage objects with a GPU context.
If you need to use both simultaneously, you can set up both to use the CPU.
Make sure images don’t exceed CPU and GPU limits. (iOS)
Use smaller images when possible.
Performance scales with the number of output pixels. You can have Core Image render into a smaller view, texture, or framebuffer. Allow Core Animation to upscale to display size.
Use Core Graphics or Image I/O functions to crop or downsample, such as the functions CGImageCreateWithImageInRect or CGImageSourceCreateThumbnailAtIndex.
The UIImageView class works best with static images.
If your app needs to get the best performance, use lower-level APIs.
Avoid unnecessary texture transfers between the CPU and GPU.
Render to a rectangle that is the same size as the source image before applying a contents scale factor.
Consider using simpler filters that can produce results similar to algorithmic filters.
For example, CIColorCube can produce output similar to CISepiaTone, and do so more efficiently.
Take advantage of the support for YUV image in iOS 6.0 and later.
If you demand real-time processing performance, you will want to use an OpenGL view that CoreImage can render its output to, and read your image bytes directly into the GPU instead of pulling it from a CGImage. Using a GLKView, and overriding drawRect: is a fairly simple solution to get a view that Core Image can render directly to. Keeping data on the GPU is the best way to get peak performance out of Core Image.
Try to reuse as much as possible. Keep a CIContext around for subsequent renders (like the doc says). If you end up using an OpenGL view, these are also things you may want to re-use as much as possible.
You may also be able to get better performance by using software rendering. Software rendering would avoid a copy to/from GPU. [CIContext contextWithOptions:#{kCIContextUseSoftwareRenderer: #(YES)}] However, this will have performance limitations in the actual render, since the CPU render is usually slower than a GPU render.
So, you can choose your level of difficulty to get maximum performance. The best performance can be more challenging, but a few tweaks may get you to "acceptable" performance for your use case.
I am trying to save the view with its subview, but the saved image is little bit blurry (especially the label's text)
I tried all the solutions given in stackoverflow - no use.
Can anyone help me on the same?
I am using the below code
UIGraphicsBeginImageContextWithOptions(view.bounds.size, NO, 0);
[view drawViewHierarchyInRect:view.bounds afterScreenUpdates:YES];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
And getting the blurred text, also the picture quality is low.
You could try a higher resolution image. It should be fine if you compress a high resolution image to down, but scaling up a low resolution image to a larger size will generally blur the image contents, as it stretches everything.
The preferred approach is [UIView snapshotViewAfterScreenUpdates:]. You should only use drawViewHierarchyInRect:afterScreenUpdates: if you plan to apply additional effects.
That said, there are several likely causes, depending on how you're manipulating or saving the image. For example, saving text in JPEG format will cause blurriness. Rotating or scaling the image without great care can make the text blurry. Drawing the image incorrectly (for instance, failing to pixel-align it) can make the text blurry. You should simplify your problem if you're making multiple steps, and validate the quality at each step. To discuss it further on StackOverflow, you need to provide details on how you're manipulating and displaying the image, not just how you generate it.
Text is extremely susceptible to artifacts. If you must take pictures of it (something you generally should avoid if at all possible), you should make sure to manipulate it as little as possible. It is always better to manipulate the text before it's drawn rather than after.
I am using CIContext method - (void)drawImage:(CIImage *)im inRect:(CGRect)dest fromRect:(CGRect)src to draw my image to screen. But I need to implement zoom-in/zoom-out method. How could I achieve it? I think zoom-in could be achieved increasing dest rect, because apple docs says:
The image is scaled to fill the destination rectangle.
But what about zoom-out? Because if dest rectangle is scaled down, then image is drawn in it's actual size, but only part of image is visible then (part that fits in dest rectangle).
What could you suggest?
You may try using this for image resizing (zooming). Hope this helps you.
Take a look at this little toy app I made
It's to demonstrate the NSImage version of your CIContext method:
- (void)drawInRect:(NSRect)dstRect
fromRect:(NSRect)srcRect
operation:(NSCompositingOperation)op
fraction:(CGFloat)delta
I did this to find out exactly how the rects relate to each other. It's interactive, you can play with the sliders and move/zoom the images. Not a solution, but it might help you work things out.
You can use CIFilter to resize your CIImage before drawing. Quartz Composer comes with a good example of using this filter. Just look up the description of the Image Resize filter in QC.
EDIT:
Another good filter for scaling is CILanczosScaleTransform. There is a snippet demonstrating basic usage.
I am creating a drawing app and have run into a problem. I have an array of curves; each curve keeps an array of points, and each point keeps its color, thickness, and coords.
When I drawRect: is called, I redraw all the curves from this array. The problem is that this array is getting huge, and the app slows down.
My idea is to, at the end of each redrawing, save the current context as an image, free the curves array, and at the next redraw, use that image as the background. Ultimately, I don't need the curves array at all, just an array of the curves in progress. Is this possible? Or maybe there is another way to do it?
You can render the corresponding layer of your view as image to update in on the next iteration. Sure it is better in this case to use UIImageView as yourViewToSaveAsImage. In this case you could get this process even easier...
UIView *view = yourViewToSaveAsImage;
UIGraphicsBeginImageContext(view.bounds.size);
[view.layer renderInContext:UIGraphicsGetCurrentContext()];
UIImage *result = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
A path contains only information on points, so if you want to track variations in rendering you need a separate list of objects.
I achieved this by creating an NSArray* of my own custom objects that included fields such as: an NSBezierPath* (to capture the points and simplify drawing the segment), a CGPathDrawingMode to use for the segment, and information on the color and line size.
Then when I draw, I iterate over the elements of the array, set the context's current colors, and call either stroke or fill on the current element's NSBezierPath* depending on how I configured that segment.
I would also like to know if there's a faster way but this approach certainly works well.
I need a method for resizing UIImage like in photoshop with "nearest neighbour" resampling. I was looking for some, but everything I found was about CoreGraphics thicks to improve bicubic resampling quality. I have pixel-style design in my app, and a lot of stuff I create by pixel and then enlarge it with x5 multiplier (and it takes a lot of time, so I even close to writing a script for Photoshop). For example:
>
But I really don't need this like result of resampling:
Maybe anyone will show me the right way.
When you draw your image into a graphics context, you can set the graphics context's interpolation quality to "none", like this (e.g. in a view's drawRect method):
CGContextRef c = UIGraphicsGetCurrentContext();
CGContextSetInterpolationQuality(c, kCGInterpolationNone);
UIImage *image = [UIImage imageNamed:#"pixels.png"];
[image drawInRect:self.bounds];
If you need the result as a UIImage (e.g. to assign it to a built-in UI control), you could do this with UIGraphicsBeginImageContext (you'll find lots of examples for that).
An alternative would be to set the magnificationFilter property of an image view's layer:
pixelatedImageView.layer.magnificationFilter = kCAFilterNearest;
This is probably faster and more memory-efficient, because you don't need to redraw the image.