What would be the most efficient way to remap the colors of an image to a gradient for iOS? This is defined as "apply a color lookup table to the image" in the Image Magic docs, and generally I think. Is there something built in core image for instance to do this? I know it can be done with ImageMagick code using convert -clut, but not certain that is the most efficient way to do it.
the result of remapping the image to a gradient is as pictured here:
http://owolf.net/uploads/ny.jpg
The basic formula, copied from fraxel's comment is:
1.Open your image as grayscale, and RGB
2.Convert the RGB image to HSV (Hue, Saturation, Value/Brightness) color space. This is a cylindrical space, with hue represented by a single value on the polar axis.
3.Set the hue channel to the grayscale image we already opened, this is the crucial step.
4.Set value, and saturation channels both to maximal values.
5.Convert back to RGB space (otherwise display will be incorrect).
Related
In a Vulkan program, fragment shaders generally output single-precision floating-point colors in the range 0.0 to 1.0 to each red/blue/green channel, and these are then written to (blended into) the swapchain image that is then presented to screen. The floating point values are encoded into bits according to the format of the swapchain image (specified when the swapchain is created).
When I change my swapchain format from VK_FORMAT_B8G8R8A8_UNORM to VK_FORMAT_B8G8R8A8_SRGB I observe that the overall brightness of the frames is greatly increased, and also there are some minor color shifts.
My understanding of the SRGB format was that it was a lot like the UNORM format just having a different mapping of floating point values to 8-bit integers, such that it had higher color resolution in some areas and less in others, but the actually meaning of the "pre-encoded" RGB floating-point values remained unchanged.
So I'm a little suprised about the brightness increase. Is my understanding of SRGB encoding wrong? and/or is such a brightness increase is expected vs UNORM?
or maybe I have a bug and a brightness increase is not expected?
Update:
I've observed that if I use SRGB swapchain images and also load my images/textures in VK_FORMAT_B8G8R8A8_SRGB format rather than VK_FORMAT_B8G8R8A8_UNORM then the extra brightness goes away. It looks the same as if I use VK_FORMAT_B8G8R8A8_UNORM swapchain images and load my images/textures in VK_FORMAT_B8G8R8A8_UNORM format.
Also, if I put the swapchain image into VK_FORMAT_B8G8R8A8_UNORM format and then load the images/textures with VK_FORMAT_B8G8R8A8_SRGB, the frames look extra dark / almost black.
Some clarity about what is going on would be helpful.
This is a colorspace and display issue.
Fragment shaders are assumed to be writing values in a linear RGB colorspace. As such, if you are rendering to an image that has a linear RGB colorspace (UNORM), the values your FS produces are interpreted directly. When you render to an image which has an sRGB colorspace, you are writing values from one space (linear) into another space (sRGB). As such, these values are automatically converted into the sRGB colorspace. It's no different from transforming a position from model space to world space or whatever.
What is different is the fact that you've been looking at your scene incorrectly. See, odds are very good that your swapchain's VkSurfaceFormat::colorSpace value is VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.
VkSurfaceFormat::colorspace tells you how the display engine will interpret the pixel data in swapchain images you present. This is not a setting you provide; this is the display engine telling you something about how it is going to interpret the values you send it.
I say "odds are very good" that it is sRGB because, outside of extensions, this is the only possible value. You are rendering to an sRGB display device whether you like it or not.
So if you write to a UNORM image, the display device will read the actual bits of data and interpret them as if they are in the sRGB colorspace. This operation only makes sense if the data your fragment shader wrote itself is in the sRGB colorspace.
However, that's generally not how FS's generate data. The lighting computations you compute only make sense on color values in a linear RGB colorspace. So unless you wrote your FS to deliberately do sRGB conversion after computing the final color value, odds are good that all of your results have been in a linear RGB colorspace. And that's what you've been writing to your framebuffer.
And then the display engine mangles it.
By using an sRGB image as your destination, you force a colorspace conversion from linear RGB to sRGB, which will then be interpreted by the display engine as sRGB values. This means that your lighting equations are finally producing the correct results.
Failure to do gamma-correct rendering properly (including the source texture images which are almost certainly also in the sRGB colorspace, as this is the default colorspace for most image editors. The exceptions would be for things like gloss-maps, normal maps, or other images that aren't storing "colors".) leads to cases where linear light attention appears more correct than quadratic attenuation, even though quadratic is how reality works.
This is gamma correction.
Using a swapchain with VK_FORMAT_B8G8R8A8_SRGB leverages the ability to to apply gamma correction as the final step in your render pipeline. This happens for you automatically behind the scenes.
That is the only place you want gamma correction to happen. Make sure your shaders are not applying gamma correction. You might see it as:
color = pow(color, vec3(1.0/2.2));
If your swapchain does the gamma correction, you do not need todo it in your shaders.
Most images are SRGB (pictures, color textures, etc). Linear images are for specific data, like a blue noise texture or heightmap.
Load SRGB images w/ VK_FORMAT_R8G8B8A8_SRGB
Load LINEAR images w/ VK_FORMAT_R8G8B8A8_UNORM
No shader conversion is required if the rules outlined above are followed.
I've seen many versions of multicolored gradient like images, that are both non linear and heavily stylized. Usually in the form of layered blob like shapes.
My guess as to how they achieve this effect is
drawing intersecting blob like shapes
masking gradients on the shapes
interpolating the colors on the image.
However as you'll notice by the distinct lines in the image the interpolated effect only appears in certain regions of the image. This effect is what I would like to achieve in metal.
One approach is to draw your solid colors and then apply a zoom or motion blur CoreImage filter to achieve the effect of a gradient, leaving some detail by where you place the center (for zoom) or the angle you set (for motion).
Here's an example of a before and a couple afters. The original image in this case is drawn with 2D function plotting but you could easily use a static input image/video-frame, draw an image with filled bezier paths, etc.
The second image uses a CIZoomBlur, input center pt just off image center at (240, 220), with amount set to 134.9.
The CIMotionBlur filter also produces some interesting gradient effects. Here's the same input image, with CIMotionBlur inputRadius 57.6 and inputAngle -0.415.
I think this could achieve what you're after providing you set up the original solid-color image as you like and are able to figure out optimal settings for the filters (angle, center pt etc.).
I'm trying to use dcraw on a color image (e.g.CR or NEF) to extract raw monochrome data for image processing.
With parameters -4 -D -c I get an image with a checkerboard as shown below:
When unzoomed, the image data is correct, except for the checkboard pattern in all images from different cameras.
The above image was produced using -T and zooming in the resulting .tiff file in File Viewer Plus. In practice, I'm reading the .pgm file directly and getting the same checkboard.
What aren't I understanding? Does this have something to do with Bayer filtering?
Yes, this is due to Bayer filtering and no demosaicing. For example, Green areas will have green pixels brighter than red according to the Bayer pattern, whereas red areas will have green pixels dark.
To get some kind of correct grayscale (or color) image, intensity has to be weighed over a 2x2 area (in standard Bayer). What you are looking for cannot be achieved without the demosaicing step.
Your best bet is to extract a color image, then turn it into grayscale.
In Photoshop you can set a layer's blending mode to be "Hue". If that layer is, for example, filled with blue then it seems to take the layer below and makes it all blue wherever a non-whiteish color exists.
I'm wondering what it's actually doing though. If I have a background layer with a pixel aarrggbb and the layer on top of that is set to blend mode "Hue" and there's a pixel aarrggbb on that layer, how are those two values combined to give the result that we see?
It doesn't just drop the rrggbb from the layer below. If it did that it'd color white and black as well. It also wouldn't allow color variations through.
If a background pixel is 0xff00ff00 and the corresponding hue layer pixel is 0xff0000ff then I'm assuming the end result will just be 0xff0000ff because the ff blue replaces the ff green. But, if the background pixel is 0x55112233 and the hue layer pixel is 0xff0000ff, how does it come up with the shade of blue that it comes up with?
The reason I ask is that I'd like to take various images and change the hue of the image programmatically in my app. Rather than storing 8 different versions of the same image with different colors, I'd like to store one image and color it as needed.
I've been researching a way to replicate that blending mode in javascript/canvas but I've only come up with the "colorize" filter/blend mode. (Examples below)
Colorize algorithm:
convert the colors from RGB to HSL;
change the Hue value to the wanted one (in my case 172⁰ or 0.477);
revert the update HSL to RGB
Note: this is ok on the desktop but it's noticeably slow on a smartphone, I found.
You can see the difference by comparing these three images. Original:
colorize:
Fireworks' "blend hue" algorithm (which I think is the same as Photoshop's):
The colorize filter might be a good substitute.
RGB/HSL conversion question
Hue/Chroma and HSL on Wikipedia
I found an algorithm to convert RGB to HSV here:
http://www.cs.rit.edu/~ncs/color/t_convert.html
Of course, at the bottom of that page it mentions that the Java Color object already has methods for converting between RGB and HSV, so I just used that.
With modern hardware, what is the fastest way to draw an image with a "bitmask", i.e., a mask that specifies whether a given pixel will be drawn or not (this could be extracted from "magic pink" pixels, for example) using OpenGL?
Should I just use alpha blending and set invisible pixels to a=0?
Should I use the old "AND black/white mask then OR image on black bg" technique?
Should I use the alpha pass test?
Should I use a shader?
This matters because I'm planning on drawing massive quantities of such images - as much as I can afford to.
If the mask and the texture are always the same (e.g. for splatting), you probably should use blending with a pre-multiplied color values. This usually is just saturated adding the texture with the background (no need to multiply per-pixel).
You should definitely use the alpha pass test - by default it's set to something like >0.08, so you'll automatically get this if you set your pixels to 0.0 alpha.