From OpenGL ES 2.0 specification section 4.4.5:
"Formats not listed in table 4.5, including compressed internal formats. are not
color-, depth-, or stencil-renderable, no matter which components they contain."
Then there are extensions that extend this table such as:
OES_depth24
OES_depth32
OES_rgb8_rgba8
ARM_rgba8
If I understood the specification correctly, table 4.5 affects both texture and renderbuffer formats. And in that case, for example, RGB and RGBA textures with 8-bits per component are not color-renderable unless the extension OES_rgb8_rgba8 (or ARM_rgba8 for RGBA) is supported.
On a test device that supports OES_rgb8_rgba8 the following texture formats are valid, i.e., framebuffer complete when attached to FBO as the color attachment:
RGB 565
RGB 888
RGBA 4444
RGBA 5551
RGBA 8888
And these were not:
Alpha 8
Luminance 8
LuminanceAlpha 88
The results match my assumptions (at least on 1 device) but I would like to know if I understood the specification correctly or is this working by accident?
Yes, your assumptions are correct.
The list of renderable formats in the official specification are just what an OpenGL ES 2.0 implementation is required to support. Most support many more than what is listed there.
However, no implementation of OpenGL / OpenGL ES supports alpha or luminance textures as color-renderable. You can replicate the behavior with texture swizzle extensions and/or GLSL vector swizzling. The extension: EXT_texture_rg adds Red and Red/Green image formats that are useable by textures and renderbuffers. These two formats are very useful when you want to draw into a one or two-channel image format using an FBO (since GL_LUMINANCE, GL_ALPHA and GL_LUMINANCE_ALPHA are not color-renderable formats).
Generally speaking, the set of renderbuffer image formats is a subset of texture image formats. You can use compressed image formats (optional in ES 2.0), luminance and alpha image formats for texturing and pixel transfer operations but they are not supported by renderbuffers. This means that while you can draw using these textures, you cannot draw into them by attaching them to an FBO. To draw into a texture, it must have an image format that is both a renderbuffer format and texture format.
On a historical note, there are renderbuffer formats that are/were not usable as textures. Multisample formats are one example, multisample texturing was added after the initial FBO specification was created.
Related
In a Vulkan program, fragment shaders generally output single-precision floating-point colors in the range 0.0 to 1.0 to each red/blue/green channel, and these are then written to (blended into) the swapchain image that is then presented to screen. The floating point values are encoded into bits according to the format of the swapchain image (specified when the swapchain is created).
When I change my swapchain format from VK_FORMAT_B8G8R8A8_UNORM to VK_FORMAT_B8G8R8A8_SRGB I observe that the overall brightness of the frames is greatly increased, and also there are some minor color shifts.
My understanding of the SRGB format was that it was a lot like the UNORM format just having a different mapping of floating point values to 8-bit integers, such that it had higher color resolution in some areas and less in others, but the actually meaning of the "pre-encoded" RGB floating-point values remained unchanged.
So I'm a little suprised about the brightness increase. Is my understanding of SRGB encoding wrong? and/or is such a brightness increase is expected vs UNORM?
or maybe I have a bug and a brightness increase is not expected?
Update:
I've observed that if I use SRGB swapchain images and also load my images/textures in VK_FORMAT_B8G8R8A8_SRGB format rather than VK_FORMAT_B8G8R8A8_UNORM then the extra brightness goes away. It looks the same as if I use VK_FORMAT_B8G8R8A8_UNORM swapchain images and load my images/textures in VK_FORMAT_B8G8R8A8_UNORM format.
Also, if I put the swapchain image into VK_FORMAT_B8G8R8A8_UNORM format and then load the images/textures with VK_FORMAT_B8G8R8A8_SRGB, the frames look extra dark / almost black.
Some clarity about what is going on would be helpful.
This is a colorspace and display issue.
Fragment shaders are assumed to be writing values in a linear RGB colorspace. As such, if you are rendering to an image that has a linear RGB colorspace (UNORM), the values your FS produces are interpreted directly. When you render to an image which has an sRGB colorspace, you are writing values from one space (linear) into another space (sRGB). As such, these values are automatically converted into the sRGB colorspace. It's no different from transforming a position from model space to world space or whatever.
What is different is the fact that you've been looking at your scene incorrectly. See, odds are very good that your swapchain's VkSurfaceFormat::colorSpace value is VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.
VkSurfaceFormat::colorspace tells you how the display engine will interpret the pixel data in swapchain images you present. This is not a setting you provide; this is the display engine telling you something about how it is going to interpret the values you send it.
I say "odds are very good" that it is sRGB because, outside of extensions, this is the only possible value. You are rendering to an sRGB display device whether you like it or not.
So if you write to a UNORM image, the display device will read the actual bits of data and interpret them as if they are in the sRGB colorspace. This operation only makes sense if the data your fragment shader wrote itself is in the sRGB colorspace.
However, that's generally not how FS's generate data. The lighting computations you compute only make sense on color values in a linear RGB colorspace. So unless you wrote your FS to deliberately do sRGB conversion after computing the final color value, odds are good that all of your results have been in a linear RGB colorspace. And that's what you've been writing to your framebuffer.
And then the display engine mangles it.
By using an sRGB image as your destination, you force a colorspace conversion from linear RGB to sRGB, which will then be interpreted by the display engine as sRGB values. This means that your lighting equations are finally producing the correct results.
Failure to do gamma-correct rendering properly (including the source texture images which are almost certainly also in the sRGB colorspace, as this is the default colorspace for most image editors. The exceptions would be for things like gloss-maps, normal maps, or other images that aren't storing "colors".) leads to cases where linear light attention appears more correct than quadratic attenuation, even though quadratic is how reality works.
This is gamma correction.
Using a swapchain with VK_FORMAT_B8G8R8A8_SRGB leverages the ability to to apply gamma correction as the final step in your render pipeline. This happens for you automatically behind the scenes.
That is the only place you want gamma correction to happen. Make sure your shaders are not applying gamma correction. You might see it as:
color = pow(color, vec3(1.0/2.2));
If your swapchain does the gamma correction, you do not need todo it in your shaders.
Most images are SRGB (pictures, color textures, etc). Linear images are for specific data, like a blue noise texture or heightmap.
Load SRGB images w/ VK_FORMAT_R8G8B8A8_SRGB
Load LINEAR images w/ VK_FORMAT_R8G8B8A8_UNORM
No shader conversion is required if the rules outlined above are followed.
I have a line art texture applied to an object in 3D space. The default behavior is for the object and the texture to receive perspective scaling based on the perspective model view projection matrix. Is there any established technique to keep the positioning and scaling of the 3D object, while keeping the line width constant relative to the screen? The desired effect is as though a pen (fixed screen width) were used to trace a path on the 3D object.
Would something like SDF-based font rendering help?
Or maybe some kind of projective texture mapping?
Or render the object and texture to a buffer and expand the lines using edge detection?
Unfortunately, I'm using OGL ES 2, so I can't use a geom shader or anything like that.
The solution I came up with is inspired by procedural SDF generation, like #Felipe suggested, combined with Chris Green's Improved Alpha-Tested Magnification for Vector Textures and Special Effects.
Basically I hand draw shapes into textures using pure red, green, and blue. Then I render the scene using those textures, and generate an SDF on the fly in a second render pass. The SDF generation uses Green's algorithm with a small spread to improve performance. The SDF is then passed to a final render pass that thresholds and antialiases the SDF per Green's approach, using fwidth to maintain a constant line weight regardless of the distance of the object to the camera.
Since the original question was just for the approach/concept, I'm not posting an example at the moment. But I'll see if I can put together a shadertoy sometime soon.
You could create the texture procedurally in a fragment shader and use the size of a pixel for interpolations.
See:
FabriceNeyret's blog
I'm working on porting a graphics system based on CoreGraphics to use Metal instead. However, I've noticed that there seems to be some kind of color system mixup when I load .png files that do not have an alpha channel. .png's with an alpha channel work fine, although I have to do some swizzling as my Metal context uses BGR colors. When I load and display a texture from an alpha-less .png, the color components appear to be out of order. Textures that were supposed to appear red appear blue and vice-versa, leading me to believe the order of the color components has been swapped.
Does anyone have any idea why this might be happening? There was no issue on the previous CG-based system.
Can anybody explain how rendering differs from rasterization especially in the context of font rendering (why not font rasterization)?
Can rendering be called a special technique (like greyscale rendering and subpixel rendering) before the rasterizer rasterizes the image?
Rendering is a broad term that generally means transforming computer-readable information, for example objects in a 3d scene, to one or more images.
Rasterization is a more specific term that typically means the process of transforming a vector (curve based) image to a rasterized (pixel based) image.
Rendering involves performing the calculations for vectors and shape geometry for the elements to be drawn.
Rasterizing involves converting the rendered vectors and shapes into pixel bit maps for display.
How would I draw a UIImage in Core Graphics with dimensions 16x16 filled with random pixels at random coordinates and random grayscale color? This seems slightly impossible to do at the moment...
EDIT: Perhaps I should start with a diagonal line texture? My problem is filling in each pixel one by one. Doesn't seem doable in Core Graphics.
Create a buffer as many bytes long as you want pixels (so, in this case, 16 * 16).
Fill this buffer by reading from /dev/random.
Pass this buffer to the CGImageCreate function using kCGImageAlphaNone.
Once you have created a CGImage, it is trivial to create a UIImage from it. Depending on your requirements, you can actually create up to eight “random” UIImages from the same CGImage by specifying different orientation values.
ETA: You might also try creating a two-byte-per-pixel buffer and image. Then, by using each of the endianness flags, you can create two “random” CGImages from the same buffer, for a total of 16 “random” UIImages. However, I don't know whether two-byte-per-pixel no-alpha grayscale is supported on any version of iOS; the Quartz 2D Programming Guide lists only Mac OS X version numbers.