32 bit/component Images with CGImageCreate are actually only 8 bit/component - objective-c

For the past 4 to 5 hours I've been wrestling with this very bizarre issue. I have a an array of bytes which contain pixel values out of which I'll like to make an image of. The array represents 32 bit per component values. There is no Alpha channel, so the image is 96 bits/pixel.
I have specified all of this to the CGImageCreate function as follows:
CGImageRef img = CGImageCreate(width, height, 32, 96, bytesPerRow, space, kCGImageAlphaNone , provider, NULL, NO, kCGRenderingIntentDefault);
bytesPerRow is 3*width*4. This is so because there are 3 components per pixel, and each component takes 4 bytes (32 bits). So, total bytes per row is 3*4*width. The data provider is defined as follows:
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,bitmapData,3*4*width*height,NULL);
This is where things get bizarre. In my array, I am explicity setting the values to be 0x000000FF (for all 3 channels) and yet, the image is coming out to be completely white. If I set the value to 0xFFFFFF00, the image comes out to be black. This is telling me that the program is, for some reason, not reading all of the 4 bytes for each component and is instead reading the least significant byte. I have tried all sorts of combinations - even including an Alpha channel, but it has made no difference to this.
The program is blind to this: 0xAAAAAA00. It simply reads this as 0. When I'm explicity specifying that the bits per component are 32 bits, shouldn't the function take this into account and actually read 4 bytes from the array?
The bytes array is defined as: bitmapData = (char*)malloc(bytesPerRow*height); And I am assigning values to the array as follows
for(i=0;i<width*height;i++)
{
*((unsigned int *)(bitmapData + 12*i + 0)) = 0xFFFFFF00;
*((unsigned int *)(bitmapData + 12*i + 4)) = 0xFFFFFF00;
*((unsigned int *)(bitmapData + 12*i + 8)) = 0xFFFFFF00;
}
Note that I address the array as an int to address 4 bytes of memory. i is multiplied by 12 because there are 12 bytes per pixel. The addition of 4 and 8 allow the loop to address the green and blue channels. Note that I have inspected the memory of the array in the debugger and that seems to be perfectly OK. The loop is writing to 4 bytes. Any sort of pointers to this would be MOST helpful. My ultimate goal is to be able to read 32 bit FITS files - for which I already have the program written. I am only testing the above code with the above array.
Here the code in its entirety if it matters. This is in drawRect:(NSRect)dirtyRect method of my custom view:
int width, height, bytesPerRow;
int i;
width = 256;
height = 256;
bytesPerRow = 3*width*4;
char *bitmapData;
bitmapData = (char*)malloc(bytesPerRow*height);
for(i=0;i<width*height;i++)
{
*((unsigned int *)(bitmapData + 12*i + 0)) = 0xFFFFFF00;
*((unsigned int *)(bitmapData + 12*i + 4)) = 0xFFFFFF00;
*((unsigned int *)(bitmapData + 12*i + 8)) = 0xFFFFFF00;
}
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,bitmapData,3*4*width*height,NULL);
CGColorSpaceRef space = CGColorSpaceCreateDeviceRGB();
CGImageRef img = CGImageCreate(width, height, 32, 96, bytesPerRow, space, kCGImageAlphaNone, provider, NULL, NO, kCGRenderingIntentDefault);
CGColorSpaceRelease(space);
CGDataProviderRelease(provider);
CGContextRef theContext = [[NSGraphicsContext currentContext] graphicsPort];
CGContextDrawImage(theContext, CGRectMake(0,0,width,height), img);

I see a few things worth pointing out:
First, the Quartz 2D Programming Guide doesn't list 96-bpp RGB as a supported format. You might try 128-bpp RGB.
Second, you're working on a little-endian system*, which means LSB comes first. Change the values to which you set each component to 0x33000000EE and you will see a light grey (EE), not a dark grey (33).
Most importantly, bbum is absolutely right when he points out that your display can't render that range of color**. It's getting squashed down to 8-bpc just for display. If it's correct in memory, then it's correct in memory.
*: More's the pity. R.I.P PPC.
**: Maybe NASA has one that can?

Related

finding bounding box of centroid with limited information

I have detected blob keypoints in opencv c++. The centroid displays fine. How do I then draw a bounding box around the detected blob if I only have the blob center coordinates? I can't work backwards from center because of too many unknowns(or so I believe).
threshold(imageUndistorted, binary_image, 30, 255, THRESH_BINARY);
Ptr<SimpleBlobDetector> detector = SimpleBlobDetector::create(params);
// Detect blob
detector->detect(binary_image, binary_keypoints);
drawKeypoints(binary_image, binary_keypoints, bin_image_keypoints, Scalar(0, 0, 255), DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
//draw BBox ?
What am I overlooking to draw the bounding box around the single blob?
I said:
I can't work backwards from center because of too many unknowns(or so I believe).
There is not limited information if blob size is used: keypoints.size which returns the diameter of the blob in question. Though there might be some inaccurate results with highly asymmetric or lopsided targets, this worked well for me b/c I used spheroid objects. Moments/ is probably the better approached for the asymmetrical targets.
keypoints.size should not be confused with keypoints.size(). The latter does a count in the vector of objects in my case the former is the diameter. Using both.
Using the diameter I can then calculate the rest with no problem:
float TLx = (ctr_x - r);
float TLy = (ctr_y - r);
float BRx = (ctr_x + r);
float Bry = (ctr_y + r);
Point TLp(TLx-10, TLy-10); //works fine without but more visible with enhancement
Point BRp(BRx+10, Bry+10); //same here
std::cout << "Top Left: " << TLp << std::endl << "Right Lower:" << BRp << std::endl;
cv::rectangle(bin_with_keypoints, TLp, BRp, cv::Scalar(0, 255, 0));
imshow("With Green Bounding Box:", bin_with_keypoints);
TLp = top left point with 10px adjustments to make box bigger.
BRp = bottom right point
TLx, TLy are calculated from blob center coordinates as well as BRps. If you are going to use multiple targets would suggest contours approach (with the moments). I have 1 - 2 blobs to keep track of which is a lot easier but keeps resource usage down.
Rectangle drawing function can also work with Rect (diameter = keypoint.size)
Rect r(TLp, BRp, center_x + diameter/2, center_y+diamter/2) // r(TLc, BRc, width, heigth)
cv::rectangle(bin_with_keypoints, rect, cv::Scalar(0, 255, 0));

CGImageCreate with CGColorSpaceCreateDeviceGray on iOS12

I was using CGImageCreate with CGColorSpaceCreateDeviceGray to convert a buffer (CVPixelBufferRef) to grayscale image. It was very fast and did work well until iOS 12... now the returned image is empty.
The code look like this:
bitmapInfo = kCGBitmapByteOrder32Little | kCGImageAlphaNoneSkipFirst;
CGDataProviderRef provider = CGDataProviderCreateWithData((void *)i_PixelBuffer,
sourceBaseAddr,
sourceRowBytes * height,
ReleaseCVPixelBuffer);
retImage = CGImageCreate(width,
height,
8,
32,
sourceRowBytes,
CGColorSpaceCreateDeviceGray(),
bitmapInfo,
provider,
NULL,
true,
kCGRenderingIntentDefault);
CGDataProviderRelease(provider);
This is a known bug in iOS 12? If device gray is no supported anymore in this function, can you suggest me another way to do it?
Note that conversion should take less than 0.1 seconds for a 4K image.
Thanks in advance!
According to the list of Supported Pixel Formats in the Quartz 2D Programming Guide, iOS doesn't support 32 bits per pixel with gray color spaces. And even on macOS, 32 bpp gray requires the use of kCGBitmapFloatComponents (and float data).
Is your data really 32 bpp? If so, is it float? What are you using for bitmapInfo?
I would not expect CGImageCreate() to "convert" a buffer, including to grayscale. The parameters you're supplying are telling it how to interpret the data. If you're not using floating-point components, I suspect it was just taking one of the color channels and interpreting that as the gray level and ignoring the other components. So, it wasn't a proper grayscale conversion.
Apple's advice is to create an image that properly represents the image; create a bitmap context with the colorspace, pixel layout, and bitmap info you desire; draw the former into the latter; and create the final image from the context.
I finally found a workaround for my purpose. Note that the CVPixelBuffer is coming from the video camera.
Changed camera output pixel format to
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
(AVCaptureVideoDataOutput)
Extract the Y plane from YpCbCr
Build a CGImage with the Y plane
Code:
// some code
colorSpace = CGColorSpaceCreateDeviceGray();
sourceRowBytes = CVPixelBufferGetBytesPerRowOfPlane(i_PixelBuffer, 0);
sourceBaseAddr = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(i_PixelBuffer,0);
bitmapInfo = kCGImageByteOrderDefault;
// some code
CGContextRef context = CGBitmapContextCreate(sourceBaseAddr,
width,
height,
8,
sourceRowBytes,
colorSpace,
bitmapInfo);
retImage = CGBitmapContextCreateImage(context);
// some code
You can also look at this related post:
420YpCbCr8BiPlanarVideoRange To YUV420 ?/How to copy Y and Cbcr plane to Single plane?

UIImageEffects: white image when Gaussian radius above 280, vImageBoxConvolve_ARGB8888 issue?

I'm using the Gaussian blur algorithm found in Apple's UIImageEffects example:
CGFloat inputRadius = blurRadius * inputImageScale;
if (inputRadius - 2. < __FLT_EPSILON__)
inputRadius = 2.;
uint32_t radius = floor((inputRadius * 3. * sqrt(2 * M_PI) / 4 + 0.5) / 2);
radius |= 1; // force radius to be odd so that the three box-blur methodology works.
NSInteger tempBufferSize = vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, NULL, 0, 0, radius, radius, NULL, kvImageGetTempBufferSize | kvImageEdgeExtend);
void *tempBuffer = malloc(tempBufferSize);
vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
vImageBoxConvolve_ARGB8888(outputBuffer, inputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
vImageBoxConvolve_ARGB8888(inputBuffer, outputBuffer, tempBuffer, 0, 0, radius, radius, NULL, kvImageEdgeExtend);
free(tempBuffer);
vImage_Buffer *temp = inputBuffer;
inputBuffer = outputBuffer;
outputBuffer = temp;
I'm also working with some fairly large images. Unfortunately, when the radius gets over 280, the blurred image suddenly becomes almost completely blank, regardless of the resolution. What's going on here? Does vImageBoxConvolve_ARGB8888 have an undocumented kernel width/height limit? Or does it have to do with the way the box kernel width is computed from the radius?
EDIT:
Found a similar question here: vImageBoxConvolve: errors when kernel size > 255. A Gaussian radius of 280 roughly translates to a 260 size kernel, so that part matches up.
The box and tent convolves can run into a problem where the value modulo overflows the 31-bit accumulator. However 255 seems a bit narrow for that. There should be another 7 bits of headroom at least for 255x255. Certainly, check the error code returned by the function. If it says everything is fine, then this seems bug worthy. Attach some sample code to help Apple reproduce the problem to help ensure it is fixed.

How can i read data by glReadPixels?

I'm working on openGL es on Android.
Now i meet a problem. I defined a float array, which is used to pass to fragment shader.
float[] data = new float[texWidth*texHeight];
// test data
for (int i = 0; i < data.length; i++) {
data[i] = 0.123f;
}
1. initTexture:
glGenTextures...
glBindTexture...
glTexParameteri...
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, fb);
2.FBO:
glGenBuffers...
glBindFramebuffer...
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texId, 0);
3.onDrawFrame:
glUseProgram(mProgram);...
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);...
IntBuffer fb = BufferUtils.iBufferAllocateDirect(texWidth*texHeight);
glReadPixels(0, 0, texWidth, texHeight, GL_RGBA, GL_UNSIGNED_BYTE, fb);
System.out.println(Integer.toHexString(fb.get(0)));
System.out.println(Integer.toHexString(fb.get(1)));
System.out.println(Integer.toHexString(fb.get(2)));
fragment shader:
precision mediump float;
uniform sampler2D sTexture;
varying vec2 vTexCoord;
void main()
{
tex = texture2D(sTexture, vTexCoord.st);
vec4 color = tex;
gl_FragColor = color;
}
So, how can i get the float data(0.123f, which i defined before) whith glReadPixels? Now what i get is ff000000(ABGR), so i suspect shader doesn't get the data through this way. Can someone tell me why and how can i deal with it? i am a newbie on it and really appreciate it.
Your main problem happens before glReadPixels(). The primary issue is with the way you use glTexImage2D():
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0,
GL_RGBA, GL_UNSIGNED_BYTE, fb);
The GL_UNSIGNED_BYTE value for the 8th argument specifies that the data passed in consists of unsigned bytes. However, the values in your buffer are floats. So your float values are interpreted as bytes, which can't possibly end well because they are completely different formats, with different sizes and memory layouts.
Now, you might be tempted to do this instead:
FloatBuffer fb = BufferUtils.array2FloatBuffer(data);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight, 0,
GL_RGBA, GL_FLOAT, fb);
This would work in desktop OpenGL, which supports implicit format conversions as part of specifying texture data. But it is not supported in OpenGL ES. In ES 2.0, GL_FLOAT is not even a legal value for the format argument. In ES 3.0, it is legal, but only for internal formats that actually store floats, like GL_RGBA16F or GL_RGBA32F. It is an error to use it in combination with the GL_RGBA internal format (3rd argument).
So unless you use float textures in ES 3.0 (which consume much more memory), you need to convert your original data to bytes. If you have float values between 0.0 and 1.0, you can do that by multiplying them by 255, and rounding to the next integer.
Then you can read them back also as bytes with glReadPixels(), and should get the same values again.

NSBitmapImageRep: Inconsistent set of values

I am trying to read a 12-bit grayscale (DICOM:MONOCHROME2) image. I can read DICOM RGB files fine. When I attempt to load a grayscale image into NSBitmapImageRep, I get the following error message:
Inconsistent set of values to create NSBitmapImageRep
I have the following code fragment:
NSBitmapImageRep *rep = [[NSBitmapImageRep alloc]
initWithBitmapDataPlanes : nil
pixelsWide : width
pixelsHigh : height
bitsPerSample : bitsStored
samplesPerPixel : 1
hasAlpha : NO
isPlanar : NO
colorSpaceName : NSCalibratedWhiteColorSpace
bytesPerRow : width * bitsAllocated / 8
bitsPerPixel : bitsAllocated];
With these values:
width = 256
height = 256
bitsStored = 12
bitsAllocated = 16
Nothing seems inconsistent to me. I have verified that the image is: width*height*2 in length. So I am pretty sure that it is in a 2-byte grayscale format. I have tried many variations of the parameters, but nothing works. If I change "bitsPerSample" to 16, the error message goes away, but I get a solid black image. The closest success that I have been able to achieve, is to set "bitsPerPixel" to zero. When I do this, I successfully produce an image but it is clearly incorrectly rendered (you can barely make out the original image). Please some suggestions!! I have tried a long time to get this to work and have checked the Stack overflow and the web (many times). Thanks very much for any help!
SOLUTION:
After the very helpful suggestions from LEADTOOLS Support, I was able to solve my problem. Here is the code fragment that works (assuming a MONOCHROME2 DICOM image):
// If, and only if, MONOCHROME2:
NSBitmapImageRep *imageRep = [[NSBitmapImageRep alloc]
initWithBitmapDataPlanes : &pixelData
pixelsWide : width
pixelsHigh : height
bitsPerSample : bitsAllocated /*bitsStored-this will not work*/
samplesPerPixel : samplesPerPixel
hasAlpha : NO
isPlanar : NO
colorSpaceName : NSCalibratedWhiteColorSpace
bytesPerRow : width * bitsAllocated / 8
bitsPerPixel : bitsAllocated];
int scale = USHRT_MAX / largestImagePixelValue;
uint16_t *ptr = (uint16_t *)imageRep.bitmapData;
for (int i = 0; i < width * height; i++) *ptr++ *= scale;
It is important to know about the Transfer Syntax (0002:0010) and Number of frames in the dataset. Also, try to get the value length and VR for Pixel Data (7FE0:0010) element. Using value length of the pixel data element you will be able to validate your calculation for uncompressed image.
As for displaying the image, you will also need the value for High Bit (0028:0102) and Pixel Representation (0028:0103). An image could be 16-bit allocated, 12-bit stored, high bit set to 15 and have one sample per pixel. That means 4 lest significant bits of each word do not contain pixel data. Pixel Representation when set to 1 means sign bit is the high bit in pixel sample.
In addition, you many need to apply modality LUT transformation (rescale slope and rescale intercept for linear transformation) when present in the dataset to prepare the data for display. At the end, you apply the VOI LUT transformation (Window center and Window Width) to display the image.