How can i read data by glReadPixels? - opengl-es-2.0

I'm working on openGL es on Android.
Now i meet a problem. I defined a float array, which is used to pass to fragment shader.
float[] data = new float[texWidth*texHeight];
// test data
for (int i = 0; i < data.length; i++) {
data[i] = 0.123f;
}
1. initTexture:
glGenTextures...
glBindTexture...
glTexParameteri...
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, fb);
2.FBO:
glGenBuffers...
glBindFramebuffer...
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texId, 0);
3.onDrawFrame:
glUseProgram(mProgram);...
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);...
IntBuffer fb = BufferUtils.iBufferAllocateDirect(texWidth*texHeight);
glReadPixels(0, 0, texWidth, texHeight, GL_RGBA, GL_UNSIGNED_BYTE, fb);
System.out.println(Integer.toHexString(fb.get(0)));
System.out.println(Integer.toHexString(fb.get(1)));
System.out.println(Integer.toHexString(fb.get(2)));
fragment shader:
precision mediump float;
uniform sampler2D sTexture;
varying vec2 vTexCoord;
void main()
{
tex = texture2D(sTexture, vTexCoord.st);
vec4 color = tex;
gl_FragColor = color;
}
So, how can i get the float data(0.123f, which i defined before) whith glReadPixels? Now what i get is ff000000(ABGR), so i suspect shader doesn't get the data through this way. Can someone tell me why and how can i deal with it? i am a newbie on it and really appreciate it.

Your main problem happens before glReadPixels(). The primary issue is with the way you use glTexImage2D():
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0,
GL_RGBA, GL_UNSIGNED_BYTE, fb);
The GL_UNSIGNED_BYTE value for the 8th argument specifies that the data passed in consists of unsigned bytes. However, the values in your buffer are floats. So your float values are interpreted as bytes, which can't possibly end well because they are completely different formats, with different sizes and memory layouts.
Now, you might be tempted to do this instead:
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0,
GL_RGBA, GL_FLOAT, fb);
This would work in desktop OpenGL, which supports implicit format conversions as part of specifying texture data. But it is not supported in OpenGL ES. In ES 2.0, GL_FLOAT is not even a legal value for the format argument. In ES 3.0, it is legal, but only for internal formats that actually store floats, like GL_RGBA16F or GL_RGBA32F. It is an error to use it in combination with the GL_RGBA internal format (3rd argument).
So unless you use float textures in ES 3.0 (which consume much more memory), you need to convert your original data to bytes. If you have float values between 0.0 and 1.0, you can do that by multiplying them by 255, and rounding to the next integer.
Then you can read them back also as bytes with glReadPixels(), and should get the same values again.

Related

CGImageCreate with CGColorSpaceCreateDeviceGray on iOS12

I was using CGImageCreate with CGColorSpaceCreateDeviceGray to convert a buffer (CVPixelBufferRef) to grayscale image. It was very fast and did work well until iOS 12... now the returned image is empty.
The code look like this:
bitmapInfo = kCGBitmapByteOrder32Little | kCGImageAlphaNoneSkipFirst;
CGDataProviderRef provider = CGDataProviderCreateWithData((void *)i_PixelBuffer,
sourceBaseAddr,
sourceRowBytes * height,
ReleaseCVPixelBuffer);
retImage = CGImageCreate(width,
height,
8,
32,
sourceRowBytes,
CGColorSpaceCreateDeviceGray(),
bitmapInfo,
provider,
NULL,
true,
kCGRenderingIntentDefault);
CGDataProviderRelease(provider);
This is a known bug in iOS 12? If device gray is no supported anymore in this function, can you suggest me another way to do it?
Note that conversion should take less than 0.1 seconds for a 4K image.
Thanks in advance!
According to the list of Supported Pixel Formats in the Quartz 2D Programming Guide, iOS doesn't support 32 bits per pixel with gray color spaces. And even on macOS, 32 bpp gray requires the use of kCGBitmapFloatComponents (and float data).
Is your data really 32 bpp? If so, is it float? What are you using for bitmapInfo?
I would not expect CGImageCreate() to "convert" a buffer, including to grayscale. The parameters you're supplying are telling it how to interpret the data. If you're not using floating-point components, I suspect it was just taking one of the color channels and interpreting that as the gray level and ignoring the other components. So, it wasn't a proper grayscale conversion.
Apple's advice is to create an image that properly represents the image; create a bitmap context with the colorspace, pixel layout, and bitmap info you desire; draw the former into the latter; and create the final image from the context.
I finally found a workaround for my purpose. Note that the CVPixelBuffer is coming from the video camera.
Changed camera output pixel format to
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
(AVCaptureVideoDataOutput)
Extract the Y plane from YpCbCr
Build a CGImage with the Y plane
Code:
// some code
colorSpace = CGColorSpaceCreateDeviceGray();
sourceRowBytes = CVPixelBufferGetBytesPerRowOfPlane(i_PixelBuffer, 0);
sourceBaseAddr = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(i_PixelBuffer,0);
bitmapInfo = kCGImageByteOrderDefault;
// some code
CGContextRef context = CGBitmapContextCreate(sourceBaseAddr,
width,
height,
8,
sourceRowBytes,
colorSpace,
bitmapInfo);
retImage = CGBitmapContextCreateImage(context);
// some code
You can also look at this related post:
420YpCbCr8BiPlanarVideoRange To YUV420 ?/How to copy Y and Cbcr plane to Single plane?

UIImageEffects: white image when Gaussian radius above 280, vImageBoxConvolve_ARGB8888 issue?

I'm using the Gaussian blur algorithm found in Apple's UIImageEffects example:
CGFloat inputRadius = blurRadius * inputImageScale;
if (inputRadius - 2. < __FLT_EPSILON__)
inputRadius = 2.;
uint32_t radius = floor((inputRadius * 3. * sqrt(2 * M_PI) / 4 + 0.5) / 2);
radius |= 1; // force radius to be odd so that the three box-blur methodology works.
NSInteger tempBufferSize = vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, NULL, 0, 0, radius, radius, NULL, kvImageGetTempBufferSize | kvImageEdgeExtend);
void *tempBuffer = malloc(tempBufferSize);
vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
vImageBoxConvolve_ARGB8888(outputBuffer, inputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
free(tempBuffer);
vImage_Buffer *temp = inputBuffer;
inputBuffer = outputBuffer;
outputBuffer = temp;
I'm also working with some fairly large images. Unfortunately, when the radius gets over 280, the blurred image suddenly becomes almost completely blank, regardless of the resolution. What's going on here? Does vImageBoxConvolve_ARGB8888 have an undocumented kernel width/height limit? Or does it have to do with the way the box kernel width is computed from the radius?
EDIT:
Found a similar question here: vImageBoxConvolve: errors when kernel size > 255. A Gaussian radius of 280 roughly translates to a 260 size kernel, so that part matches up.
The box and tent convolves can run into a problem where the value modulo overflows the 31-bit accumulator. However 255 seems a bit narrow for that. There should be another 7 bits of headroom at least for 255x255. Certainly, check the error code returned by the function. If it says everything is fine, then this seems bug worthy. Attach some sample code to help Apple reproduce the problem to help ensure it is fixed.

Why is drawing a circle procedurally slower than reading from a texture?

I am making a app where i need to draw a lot of circles in the screen, so i had the idea of replacing the texture of the triangles i am using to draw the texture with a function to draw a circle. However after testing, it turned out to be slower than picking the values off a texture (although the quality is vastly superior). Is it a problem with the way i produce the circle, or reading from a texture is really a lot faster? (about twice as fast)
new code:
precision mediump float;
uniform sampler2D u_Texture;
varying vec4 vColor;
varying vec2 vTexCoordinate;
void main() {
gl_FragColor = vec4(0, 0, 0, 0);
mediump float thing = vTexCoordinate.x * vTexCoordinate.x + vTexCoordinate.y * vTexCoordinate.y;
if(thing < 1.0 && thing > 0.9){
gl_FragColor = vec4(0, 0, 0, 1);
}
if(thing < 0.9){
gl_FragColor = vec4(1, 1, 1, 1) * vColor;
}
};
old code:
gl_FragColor = texture2D(u_Texture, vTexCoordinate) * vColor;
obs: i didn't bother to rename vTexCoordinate, so it now have a value of [-1, 1] where it was [0, 1]
Conditional branches are really expensive on the GPU, since there's no branch prediction, and probably other reasons too. Also, texture lookups latency can often be hidden under shader processing overhead, so it might actually work faster in the end. So it's best to avoid branches and loops in GLSL if you can.

How to handle the orthographic projection when auto-rotating screen?

I have this method for performing the ortho projection:
void myGL::ApplyOrtho(float maxX, float maxY) const
{
float a = 1.0f / maxX;
float b = 1.0f / maxY;
float ortho[16] = {
a, 0, 0, 0,
0, b, 0, 0,
0, 0, -1, 0,
0, 0, 0, 1};
GLint projectionUniform = glGetUniformLocation(m_simpleProgram, "Projection");
glUniformMatrix4fv(projectionUniform, 1, 0, &ortho[0]);
}
It works fine for iPad screen when I do this:
ApplyOrtho(2, 2*1024/768);
Here's my rendered image:
However, when I rotate to landscape, it looks like this:
Now my assumption is this is because the ApplyOrtho matrix is setting a fixed projection and that projection does not rotate while the image is rotating within that projection, thus getting displayed fatter.
Incidentally, this is the rotation:
void myGL::ApplyRotation(float degrees) const
{
float radians = degrees * 3.14159f / 180.0f;
float s = std::sin(radians);
float c = std::cos(radians);
float zRotation[16] = {
c, s, 0, 0,
-s, c, 0, 0,
0, 0, 1, 0,
0, 0, 0, 1
};
GLint modelviewUniform = glGetUniformLocation(m_simpleProgram, "Modelview");
glUniformMatrix4fv(modelviewUniform, 1, 0, &zRotation[0]);
}
It is used right before drawing.
So I experimented and tried this at the same time I rotate:
ApplyOrtho(2*1024/768, 2);
However this has no effect whatsoever, even though the rotation is definitely happening at the same time. My image remains "fat".
Is my interpretation of why the fatness is happening correct?
How to handle the orthographic projection when auto-rotating screen?
UDPATE: Also tried this on iPhone using the 2/3 dimensions of the screen (not iPhone 5) and using ApplyOrtho(2,3) and ApplyOrtho(3,2) but the "fat" triangle in landscape remains.
Also: the viewport is setup just once, before the first Ortho:
glViewport(0, 0, width, height);
Where width and height are the dimensions of the Portrait screen.
The cause of the above discrepancies is that the orthographic projection is not matching the width and height ratio of the screen, thus the X and Y coordinates are not the same screen size. Making the orthographic ratio match the viewport ratio resolves this issue. As a result, when rotating, the image will remain exactly the same shape and size.

glreadpixel gl_depth_component returns 0?

I am looking for a solution to intersection point of a cube and a line. So i used
GLES20.glReadPixels(touchX, touchY, 1, 1, GLES20.GL_DEPTH_COMPONENT, GLES20.GL_FLOAT, zz);
and i showed the zz , but result was 0. so how could i get the depth buffer value of a Cube when i touched on the cube(actually on the 2d screen). I use GLES20 and Android API level15.And My code is below.
ByteBuffer PixelBuffer = ByteBuffer.allocateDirect(4);
ByteBuffer zBuffer = ByteBuffer.allocateDirect(4);
PixelBuffer.order(ByteOrder.nativeOrder());
PixelBuffer.position(0);
zBuffer.order(ByteOrder.nativeOrder());
zBuffer.position(0);
FloatBuffer zz;
zz = zBuffer.asFloatBuffer();
GLES20.glReadPixels(touchX, touchY, 1, 1, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, PixelBuffer);
GLES20.glReadPixels(touchX, touchY, 1, 1, GLES20.GL_DEPTH_COMPONENT, GLES20.GL_FLOAT, zz);
by the way picking color works fine.
Thanks!
You forget to prepare target framebuffer to read... Try like this:
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glPixelStorei(GL_PACK_ALIGNMENT, 1);
glReadPixels(0, 0, deviceWidth, deviceHeight, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
Or just write simple shader and render your zbuffer data into your FBO, simething like
gl_FragColor = vec4(vec3(gl_FragCoord.z), 1.0);
and then read color information form this FBO...