Porting CVOpenGLESTextureCacheCreateTextureFromImage to normal OpenGL ES methods - objective-c

Question

I'm working on an app that feeds video frames coming from the camera into an OpenGL ES texture to create some effects. The app uses Core Video and its many convenience methods to create OpenGL textures and caches.
Now I'd like to use static images instead of the video buffer, and I'm looking for standard OpenGL methods in place of those available with Core Video.
For example:
CVOpenGLESTextureRef;
CVOpenGLESTextureCacheRef;
CVOpenGLESTextureCacheCreateTextureFromImage();
CVOpenGLESTextureGetTarget();
CVOpenGLESTextureGetName();
CVOpenGLESTextureCacheFlush();
What are the standard OpenGL types and methods that I can use in place of the code above to load and create a texture from a normale image?
Thanks
*EDIT*
Following Brad's suggestion I was able to do some progress... but still doesn't work correctly. I believe the problem is that the shader is expecting two separate textures to work properly. One for the Y plane and one for the UV plane.
Here's the code used to set them up
// Y-plane
glActiveTexture(GL_TEXTURE0);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, _videoTextureCache, pixelBuffer, NULL, GL_TEXTURE_2D, GL_RED_EXT, _textureWidth, _textureHeight, GL_RED_EXT, GL_UNSIGNED_BYTE, 0, &_lumaTexture);
glBindTexture(CVOpenGLESTextureGetTarget(_lumaTexture), CVOpenGLESTextureGetName(_lumaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
// UV-plane
glActiveTexture(GL_TEXTURE1);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, _videoTextureCache, pixelBuffer, NULL, GL_TEXTURE_2D, GL_RG_EXT, _textureWidth/2, _textureHeight/2, GL_RG_EXT, GL_UNSIGNED_BYTE, 1, &_chromaTexture);
glBindTexture(CVOpenGLESTextureGetTarget(_chromaTexture), CVOpenGLESTextureGetName(_chromaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
So I changed it this way:
here's the setup
glActiveTexture(GL_TEXTURE0);
glGenTextures(1, &_lumaImageTexture);
glBindTexture(GL_TEXTURE_2D, _lumaImageTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glActiveTexture(GL_TEXTURE1);
glGenTextures(1, &_chromaImageTexture);
glBindTexture(GL_TEXTURE_2D, _chromaImageTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
and here is how I create them
// Y-plane
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _lumaImageTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _textureWidth, _textureHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, imageData);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
// UV-plane
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _chromaImageTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _textureWidth, _textureHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, imageData);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
I guess the problem is in one of the parameters in the glTexImage2D function. I tried different combinations with different results... but never the correct one :)

Answer 1

All those methods do is hand you an OpenGL ES texture that corresponds to a CVPixelBuffer. Before these were introduced in iOS 5.0, we had to manually grab the pixel data from a CVPixelBuffer and upload that to a texture. You'd use the same process for a static image pulled from another source (unless you wanted to use some of the GLKit convenience classes).
The following is code that I've used for uploading static images. First, you need to get the image into a BGRA byte array:
if (shouldRedrawUsingCoreGraphics)
{
// For resized image, redraw
imageData = (GLubyte *) calloc(1, (int)pixelSizeToUseForTexture.width * (int)pixelSizeToUseForTexture.height * 4);
CGColorSpaceRef genericRGBColorspace = CGColorSpaceCreateDeviceRGB();
CGContextRef imageContext = CGBitmapContextCreate(imageData, (int)pixelSizeToUseForTexture.width, (int)pixelSizeToUseForTexture.height, 8, (int)pixelSizeToUseForTexture.width * 4, genericRGBColorspace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
CGContextDrawImage(imageContext, CGRectMake(0.0, 0.0, pixelSizeToUseForTexture.width, pixelSizeToUseForTexture.height), newImageSource);
CGContextRelease(imageContext);
CGColorSpaceRelease(genericRGBColorspace);
}
else
{
// Access the raw image bytes directly
dataFromImageDataProvider = CGDataProviderCopyData(CGImageGetDataProvider(newImageSource));
imageData = (GLubyte *)CFDataGetBytePtr(dataFromImageDataProvider);
}
In the above code, newImageSource is a CGImageRef. This shows two different paths for extracting bitmap data, with the first path surprisingly being faster in every benchmark I've run.
Once you have that, you simply need to upload it to your OpenGL ES texture:
glBindTexture(GL_TEXTURE_2D, outputTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (int)pixelSizeToUseForTexture.width, (int)pixelSizeToUseForTexture.height, 0, GL_BGRA, GL_UNSIGNED_BYTE, imageData);
This assumes that you've already created outputTexture using something like the following:
glActiveTexture(GL_TEXTURE0);
glGenTextures(1, &outputTexture);
glBindTexture(GL_TEXTURE_2D, outputTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
// This is necessary for non-power-of-two textures
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
and that the correct EAGLContext is associated with the current thread by using +setCurrentContext:.
If I may make a suggestion, why not take a look at my open source framework that handles all of this for you in order to do image processing tasks? It might save you some effort here.