Ok so I am in a bind and I have been staring at this for too long to think clearly.
Basically, all I need is a method that takes in an NSArray of images and writes an mp4 or other video format to disk.
I am not worried about monitoring the user's gesture or anything like that. The images already exist on disk -- I just need to do the conversion.
I understand that I need an AVWriter and something like:
- (CVPixelBufferRef) pixelBufferFromCGImage: (CGImageRef) image
{
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVPixelBufferCreate(kCFAllocatorDefault, CGImageGetWidth(image),
CGImageGetHeight(image), kCVPixelFormatType_32ARGB, (CFDictionaryRef) options,
&pxbuffer);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, CGImageGetWidth(image),
CGImageGetHeight(image), 8, 4*CGImageGetWidth(image), rgbColorSpace,
kCGImageAlphaNoneSkipFirst);
CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
// CGAffineTransform flipVertical = CGAffineTransformMake(
// 1, 0, 0, -1, 0, CGImageGetHeight(image)
// );
// CGContextConcatCTM(context, flipVertical);
// CGAffineTransform flipHorizontal = CGAffineTransformMake(
// -1.0, 0.0, 0.0, 1.0, CGImageGetWidth(image), 0.0
// );
//
// CGContextConcatCTM(context, flipHorizontal);
CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}
I just can't see how to put it all into one clean helper. And before you ask I have seen: How to display video from images and the like, but just running the images in a UIImageView is not what I am looking for here. This needs to be App Store legal, so FFMPeg is not an option.
Any thoughts / guidance?
https://github.com/meslater/MSImageMovieEncoder encodes images to H.264.
You have to provide images as context refs or CVPixelBufferRefs (see sample code here) implementing
-(BOOL)nextFrameInCVPixelBuffer:(CVPixelBufferRef*)pixelBuf;
Each call to this method should return a buffer and YES. If there are no more images return NO, and you'll find the movie on disk.
For more details, read How do I export UIImage array as a movie? (it's basically the same code) or read the github files.
Related
I am trying to send an image through sockets to a server, but the image doesn't appear well on the server side.
I am using this code, what can be the problem? I am sending text to the server and it works, but the image appears unformatted.
UIImage *imageResized=[self resizeImage:editedImage newSize:CGSizeMake(64,32)];
NSData *imageData = UIImageJPEGRepresentation(imageResized, 1.0f);
NSString *encodedString = [imageData base64EncodedStringWithOptions:nil];
NSString *finalStr= [NSString stringWithFormat:#"image:%#",encodedString];
NSData* data = [finalStr dataUsingEncoding:NSASCIIStringEncoding];
[tcpsocket initNetworkCommunication:[tcpSocket linkWebserviceInet1] porta:[tcpSocket linkWebport]];
[tcpsocket sendNSData:data];
- (UIImage *)resizeImage:(UIImage*)image newSize:(CGSize)newSize {
CGRect newRect = CGRectIntegral(CGRectMake(0, 0, newSize.width, newSize.height));
CGImageRef imageRef = image.CGImage;
UIGraphicsBeginImageContextWithOptions(newSize, NO, 0);
CGContextRef context = UIGraphicsGetCurrentContext();
// Set the quality level to use when rescaling
CGContextSetInterpolationQuality(context, kCGInterpolationHigh);
CGAffineTransform flipVertical = CGAffineTransformMake(1, 0, 0, -1, 0, newSize.height);
CGContextConcatCTM(context, flipVertical);
// Draw into the context; this scales the image
CGContextDrawImage(context, newRect, imageRef);
// Get the resized image from the context and a UIImage
CGImageRef newImageRef = CGBitmapContextCreateImage(context);
UIImage *newImage = [UIImage imageWithCGImage:newImageRef];
CGImageRelease(newImageRef);
UIGraphicsEndImageContext();
return newImage;
}
I've written a helper class to record a NSView and save it to a QuickTime file. The view is recorded fine to a QuickTime movie but the output is skewed for some reason. The core of my class is below, and the output is this:
- (void) captureImage
{
[self getCGImageFromView];
pixelBuffer = [self getPixelBufferFromCGImage:viewCGImage size:CGRectMake(0, 0, mViewRect.size.width, mViewRect.size.height).size];
if(pixelBuffer) {
if(![adapter appendPixelBuffer:pixelBuffer withPresentationTime:CMTimeMake(mCurrentFrame, 20)])
NSLog(#"AVAssetWriterInputPixelBufferAdaptor: Failed to append pixel buffer.");
CFRelease(pixelBuffer);
mCurrentFrame++;
}
}
- (void) getCGImageFromView
{
viewBitmapImageRep = [currentView bitmapImageRepForCachingDisplayInRect:mViewRect];
[currentView cacheDisplayInRect:mViewRect toBitmapImageRep:viewBitmapImageRep];
viewBitmapFormat = [viewBitmapImageRep bitmapFormat];
viewCGImage = [viewBitmapImageRep CGImage];
}
- (CVPixelBufferRef)getPixelBufferFromCGImage:(CGImageRef)image size:(CGSize)size
{
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferOpenGLCompatibilityKey,
[NSNumber numberWithInt:size.width], kCVPixelBufferWidthKey,
[NSNumber numberWithInt:size.height], kCVPixelBufferHeightKey,
nil];
CVPixelBufferRef pxbuffer = NULL;
CVReturn status = CVPixelBufferCreate(NULL, size.width, size.height, kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options, &pxbuffer);
NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);
CVPixelBufferLockBaseAddress(pxbuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
NSParameterAssert(pxdata != NULL);
CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(pxdata, size.width, size.height, 8, 4*size.width, rgbColorSpace, kCGImageAlphaPremultipliedFirst);
NSParameterAssert(context);
[[currentView layer] renderInContext:context];
//CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image), CGImageGetHeight(image)), image);
CGColorSpaceRelease(rgbColorSpace);
CGContextRelease(context);
CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
return pxbuffer;
}
The problem with this was this line.
4*size.width,
The size.width was an odd number, not a multiple of 4. The values passed into CGBitmapContextCreate need to be a multiple of 4.
4*ceil(size.width / 4) * 4
That solved the problem.
I have some well-working code for reading the screen or offscreen buffer and saving the result to the iPad photo album as a PNG with transparency. The images appear perfectly when viewed in the ipad photo viewer or any other image viewer. However, within ipad's native photo viewing app the thumbnails show portions of other images from the album in the transparent sections of the thumbnail.
Has anyone else experienced this problem, and if so found a fix for it? Here's my offscreen (partial) code for generating the images:
EAGLContext *myContext = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1];
[EAGLContext setCurrentContext:myContext];
[... set up render buffer code removed for display ...]
[EAGLContext setCurrentContext:myContext];
ImageTextureManager *imageManager = [[ImageTextureManager alloc] init];
[imageManager loadImageTexture:gAppModel.currentImageRef];
[imageManager release];
glBindRenderbufferOES(GL_RENDERBUFFER_OES, offscreenColorRenderbuffer);
[self renderTransformedImage]; // render the image to the buffer
[myContext presentRenderbuffer:GL_RENDERBUFFER_OES];
// grab image from frameBuffer and return it as UIImage
NSInteger x = 0, y = 0;
NSInteger dataLength = width * height * 4;
GLubyte *data = (GLubyte*)malloc(dataLength * sizeof(GLubyte));
glPixelStorei(GL_PACK_ALIGNMENT, 4);
glReadPixels(x, y, width, height, GL_RGBA, GL_UNSIGNED_BYTE, data);
CGDataProviderRef ref = CGDataProviderCreateWithData(NULL, data, dataLength, NULL);
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
CGImageRef iref = CGImageCreate(width, height, 8, 32, width * 4, colorspace,
kCGBitmapByteOrder32Big | kCGImageAlphaPremultipliedLast,
ref, NULL, true, kCGRenderingIntentDefault);
UIGraphicsBeginImageContext(CGSizeMake(width, height));
CGContextRef cgcontext = UIGraphicsGetCurrentContext();
CGContextSetBlendMode(cgcontext, kCGBlendModeCopy);
CGContextDrawImage(cgcontext, CGRectMake(0.0, 0.0, width, height), iref);
UIImage *image = UIGraphicsGetImageFromCurrentImageContext(); // this call creates an AutoRelease UIImage
NSData* imdata = UIImagePNGRepresentation(image); // get PNG representation
UIImage* myImagePNG = [UIImage imageWithData:imdata]; // wrap UIImage around PNG representation
UIImageWriteToSavedPhotosAlbum(myImagePNG, nil, nil, nil);
UIGraphicsEndImageContext();
Thanks to medvedNick for his offscreen rendering code: Drawing into OpenGL ES framebuffer and getting UIImage from it on iPhone
I'm trying to display texts in a CGContext and I want to know how to wrap the texts. How can I make this possible?
Here's my code:
NSString *text1 = #"These three first articles describe the God in whom we believe. The pictures of the triangle represents the Triune God), of Jesus Christ and of the dove representing the Holy Spirit are grouped together in order to manifest their intimate relationships and unity.";
- (void) renderPageAtIndex:(NSUInteger)index inContext:(CGContextRef)ctx {
if(index>[images count]-1)return;
UIImage *image = [images objectAtIndex:index];
CGRect imageRect = CGRectMake(0, 0, image.size.width, image.size.height);
CGAffineTransform transform = aspectFit(imageRect,
CGContextGetClipBoundingBox(ctx));
NSString *text1 = [script objectAtIndex:index];
char* text = (char *)[text1 cStringUsingEncoding:NSASCIIStringEncoding];
CGContextSelectFont(ctx, "Arial", 30, kCGEncodingMacRoman);
CGContextConcatCTM(ctx, transform);
CGContextDrawImage(ctx, imageRect, [image CGImage]);
CGContextSetTextPosition(ctx, 0.0f, round(30 / 4.0f));
CGContextSetTextDrawingMode(ctx, kCGTextFill);
CGContextSetRGBFillColor(ctx, 0, 0, 0, 1);
CGContextShowTextAtPoint(ctx,10,10,text, strlen(text));
}
If you need more advanced text formatting (like wrapping), use Core Text instead. To display a simple string in a rectangle, use
CTFramesetterRef framesetter = CTFramesetterCreateWithAttributedString(attrString);
CGPath p = CGPathCreateMutable();
CGPathAddRect(p, rect);
CTFrameRef frame = CTFramesetterCreateFrame(framesetter, CFRangeMake(0,0), p, NULL);
CTFrameDraw(frame, context);
I have an NSImageView which I get an image for from an NSOpenPanel. That works great.
Now, how can I take that NSImage, half its size and save it as the same format in the same directory as the original as well?
If you can help at all with anything I'd appreciate it, thanks.
Check the ImageCrop sample project from Matt Gemmell:
http://mattgemmell.com/source/
Nice example how to resize / crop images.
Finally you can use something like this to save the result (dirty sample):
// Write to TIF
[[resultImg TIFFRepresentation] writeToFile:#"/Users/Anne/Desktop/Result.tif" atomically:YES];
// Write to JPG
NSData *imageData = [resultImg TIFFRepresentation];
NSBitmapImageRep *imageRep = [NSBitmapImageRep imageRepWithData:imageData];
NSDictionary *imageProps = [NSDictionary dictionaryWithObject:[NSNumber numberWithFloat:0.9] forKey:NSImageCompressionFactor];
imageData = [imageRep representationUsingType:NSJPEGFileType properties:imageProps];
[imageData writeToFile:#"/Users/Anne/Desktop/Result.jpg" atomically:NO];
Since NSImage objects are immutable you will have to:
Create a Core Graphics context the size of the new image.
Draw the NSImage into the CGContext. It should automatically scale it for you.
Create an NSImage from that context
Write out the new NSImage
Don't forget to release any temporary objects you allocated.
There are definitely other options, but this is the first one that came to mind.
+(NSImage*) resize:(NSImage*)aImage scale:(CGFloat)aScale
{
NSImageView* kView = [[NSImageView alloc] initWithFrame:NSMakeRect(0, 0, aImage.size.width * aScale, aImage.size.height* aScale)];
[kView setImageScaling:NSImageScaleProportionallyUpOrDown];
[kView setImage:aImage];
NSRect kRect = kView.frame;
NSBitmapImageRep* kRep = [kView bitmapImageRepForCachingDisplayInRect:kRect];
[kView cacheDisplayInRect:kRect toBitmapImageRep:kRep];
NSData* kData = [kRep representationUsingType:NSJPEGFileType properties:nil];
return [[NSImage alloc] initWithData:kData];
}
Here is a specific implementation
-(NSImage*)resizeImage:(NSImage*)input by:(CGFloat)factor
{
NSSize size = NSZeroSize;
size.width = input.size.width*factor;
size.height = input.size.height*factor;
NSImage *ret = [[NSImage alloc] initWithSize:size];
[ret lockFocus];
NSAffineTransform *transform = [NSAffineTransform transform];
[transform scaleBy:factor];
[transform concat];
[input drawAtPoint:NSZeroPoint fromRect:NSZeroRect operation:NSCompositeCopy fraction:1.0];
[ret unlockFocus];
return [ret autorelease];
}
Keep in mind that this is pixel based, with HiDPI the scaling must be taken into account, it is simple to obtain :
-(CGFloat)pixelScaling
{
NSRect pixelBounds = [self convertRectToBacking:self.bounds];
return pixelBounds.size.width/self.bounds.size.width;
}
Apple has source code for downscaling and saving images found here
http://developer.apple.com/library/mac/#samplecode/Reducer/Introduction/Intro.html
Here is some code that makes a more extensive use of Core Graphics than other answers. It's made according to hints in Mark Thalman's answer to this question.
This code downscales an NSImage based on a target image width. It's somewhat nasty, but still useful as an extra sample for documenting how to draw an NSImage in a CGContext, and how to write contents of CGBitmapContext and CGImage into a file.
You may want to add extra error checking. I didn't need it for my use case.
- (void)generateThumbnailForImage:(NSImage*)image atPath:(NSString*)newFilePath forWidth:(int)width
{
CGSize size = CGSizeMake(width, image.size.height * (float)width / (float)image.size.width);
CGColorSpaceRef rgbColorspace = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedLast;
CGContextRef context = CGBitmapContextCreate(NULL, size.width, size.height, 8, size.width * 4, rgbColorspace, bitmapInfo);
NSGraphicsContext * graphicsContext = [NSGraphicsContext graphicsContextWithGraphicsPort:context flipped:NO];
[NSGraphicsContext setCurrentContext:graphicsContext];
[image drawInRect:NSMakeRect(0, 0, size.width, size.height) fromRect:NSMakeRect(0, 0, image.size.width, image.size.height) operation:NSCompositeCopy fraction:1.0];
CGImageRef outImage = CGBitmapContextCreateImage(context);
CFURLRef outURL = (CFURLRef)[NSURL fileURLWithPath:newFilePath];
CGImageDestinationRef outDestination = CGImageDestinationCreateWithURL(outURL, kUTTypeJPEG, 1, NULL);
CGImageDestinationAddImage(outDestination, outImage, NULL);
if(!CGImageDestinationFinalize(outDestination))
{
NSLog(#"Failed to write image to %#", newFilePath);
}
CFRelease(outDestination);
CGImageRelease(outImage);
CGContextRelease(context);
CGColorSpaceRelease(rgbColorspace);
}
To resize image
- (NSImage *)scaleImage:(NSImage *)anImage newSize:(NSSize)newSize
{
NSImage *sourceImage = anImage;
if ([sourceImage isValid])
{
if (anImage.size.width == newSize.width && anImage.size.height == newSize.height && newSize.width <= 0 && newSize.height <= 0) {
return anImage;
}
NSRect oldRect = NSMakeRect(0.0, 0.0, anImage.size.width, anImage.size.height);
NSRect newRect = NSMakeRect(0,0,newSize.width,newSize.height);
NSImage *newImage = [[NSImage alloc] initWithSize:newSize];
[newImage lockFocus];
[sourceImage drawInRect:newRect fromRect:oldRect operation:NSCompositeCopy fraction:1.0];
[newImage unlockFocus];
return newImage;
}
}