Metal fragment shader receives incorrect interpolated vertex z values - objective-c

I have a simple vertex shader that outputs position and color and a fragment shader that outputs color. There is also depth testing enabled with a texture that I created.
struct VertexOut {
float4 v_vertex [[position]];
float4 v_color;
float4 vsVertex;
};
vertex VertexOut vertexShader(uint vertexID [[ vertex_id ]],
const device RigidFormatPos* vertices [[ buffer(0) ]],
const device StdVertexData* vertexData [[ buffer(1) ]],
constant SettingsUBO& sceneUniforms [[ buffer(2) ]],
constant MeshUBO& perMesh [[ buffer(3) ]]) {
VertexOut out;
out.v_vertex = sceneUniforms.cameras[0].projectFromWorld * perMesh.worldFromModel * float4(vertices[vertexID].pos.xyz, 1.0);
out.v_color = float4(float3(vertexData[vertexID].color.rgb, 1.f));
out.vsVertex = out.v_vertex;
return out;
}
fragment half4 fragmentShader(VertexOut in [[stage_in]]) {
return half4(half3(in.v_color.rgb), 1.f);
}
The screen dimensions are 800x600.
Xcode debugger shows the value of in.v_vertex.z = ~857.3434 while it has to be in the range [0,1] after the divide by w. The value of in.vsVertex which is in clip space that I have manually cached and output from the vertex shader shows correct z value that after divide by w would be in the range [0,1].
This causes depth test to fail.
Has anyone encountered anything like this. Any ideas on where to look next?
If I manually output depth values to the depth buffer using depth = in.vsVertex.z / in.vsVertex.w depth test works as expected.
I am rendering to a CAMetalDrawable render target color attachement, using a depth texture that I created.

Related

FATAL EXCEPTION: inference. Cannot copy to a TensorFlowLite tensor from a Java Buffer

I was developing an Image Classifier app in Android Studio with MNIST-fashion database, but I have a little problem. When I try to run the app, I have this common error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_conv2d_input:0) with 3136 bytes from a Java Buffer with 9408 bytes.
I know this might be the mismatch of input tensor from the model and the buffer that I have in my code. But It's too confusing because my code automatically fits the size of the image from the model and all the info it needs. So I was wondering where is the error...
// Reads type and shape of input and output tensors, respectively.
int imageTensorIndex = 0;
int[] imageShape = tflite.getInputTensor(imageTensorIndex).shape(); // {1, height, width, 1}
imageSizeY = imageShape[1];
imageSizeX = imageShape[2];
DataType imageDataType = tflite.getInputTensor(imageTensorIndex).dataType();
int probabilityTensorIndex = 0;
int[] probabilityShape =
tflite.getOutputTensor(probabilityTensorIndex).shape(); // {1, 10}
DataType probabilityDataType = tflite.getOutputTensor(probabilityTensorIndex).dataType();
// Creates the input tensor.
inputImageBuffer = new TensorImage(imageDataType);
Maybe this is the problem... I'm creating imageShape like this {1, height, width, 1}, and the data type is FLOAT32, so it is supposed to be {1, height, width, 4}, right?
Another reason could be the metadata. But I populate the model with metadata and I have a .json and I don't know how to use it.
Note: If u want the note book to do the .tflite, there u go.
The tensor buffer size is determined by datasize (float32: 4bytes) * flat size of the tensor shape (1 * height * width * 1).
So the above code snippet needs to prepare an float input tensor data with the shape (1, height, width, 1) instead of the shape (1, height, width, 4).

Can you change the bounds of a Sampler in a Metal Shader?

In the fragment function of a Metal Shader file, is there a way to redefine the "bounds" of the texture with respect to what the sample will consider it's normalized coordinates to be?
By default, a value of 0,0 for the sample is the top-left "pixel" and 1,1 is the bottom right "pixel" of the texture. However, I'm re-using textures for drawing and at any given render pass there's only a portion of the texture that contains the relevant data.
For example, in a texture of width: 500 and height: 500, I might have only copied data into the region of 0,0,250,250. In my fragment function, I'd like the sampler to interpret a normalized coordinate of 1.0 to be 250 and not 500. Is that possible?
I realize I can just change the sampler to use pixel addressing, but that comes with a few restrictions as noted in the Metal Shader Specification.
No, but if you know the region you want to sample from, it's quite easy to do a little math in the shader to fix up your sampling coordinates. This is used often with texture atlases.
Suppose you have an image that's 500x500 and you want to sample the bottom-right 125x125 region (just to make things more interesting). You could pass this sampling region in as a float4, storing the bounds as (left, top, width, height) in the xyzw components. In this case, the bounds would be (375, 375, 125, 125). Your incoming texture coordinates are "normalized" with respect to this square. The shader simply scales and biases these coordinates into texel coordinates, then normalizes them to the dimensions of the whole texture:
fragment float4 fragment_main(FragmentParams in [[stage_in]],
texture2d<float, access::sample> tex2d [[texture(0)]],
sampler sampler2d [[sampler(0)]],
// ...
constant float4 &spriteBounds [[buffer(0)]])
{
// original coordinates, normalized with respect to subimage
float2 texCoords = in.texCoords;
// texture dimensions
float2 texSize = float2(tex2d.get_width(), tex2d.get_height());
// adjusted texture coordinates, normalized with respect to full texture
texCoords = (texCoords * spriteBounds.zw + spriteBounds.xy) / texSize;
// sample color at modified coordinates
float4 color = tex2d.sample(sampler2d, texCoords);
// ...
}

NSBitmapImageRep: Inconsistent set of values

I am trying to read a 12-bit grayscale (DICOM:MONOCHROME2) image. I can read DICOM RGB files fine. When I attempt to load a grayscale image into NSBitmapImageRep, I get the following error message:
Inconsistent set of values to create NSBitmapImageRep
I have the following code fragment:
NSBitmapImageRep *rep = [[NSBitmapImageRep alloc]
initWithBitmapDataPlanes : nil
pixelsWide : width
pixelsHigh : height
bitsPerSample : bitsStored
samplesPerPixel : 1
hasAlpha : NO
isPlanar : NO
colorSpaceName : NSCalibratedWhiteColorSpace
bytesPerRow : width * bitsAllocated / 8
bitsPerPixel : bitsAllocated];
With these values:
width = 256
height = 256
bitsStored = 12
bitsAllocated = 16
Nothing seems inconsistent to me. I have verified that the image is: width*height*2 in length. So I am pretty sure that it is in a 2-byte grayscale format. I have tried many variations of the parameters, but nothing works. If I change "bitsPerSample" to 16, the error message goes away, but I get a solid black image. The closest success that I have been able to achieve, is to set "bitsPerPixel" to zero. When I do this, I successfully produce an image but it is clearly incorrectly rendered (you can barely make out the original image). Please some suggestions!! I have tried a long time to get this to work and have checked the Stack overflow and the web (many times). Thanks very much for any help!
SOLUTION:
After the very helpful suggestions from LEADTOOLS Support, I was able to solve my problem. Here is the code fragment that works (assuming a MONOCHROME2 DICOM image):
// If, and only if, MONOCHROME2:
NSBitmapImageRep *imageRep = [[NSBitmapImageRep alloc]
initWithBitmapDataPlanes : &pixelData
pixelsWide : width
pixelsHigh : height
bitsPerSample : bitsAllocated /*bitsStored-this will not work*/
samplesPerPixel : samplesPerPixel
hasAlpha : NO
isPlanar : NO
colorSpaceName : NSCalibratedWhiteColorSpace
bytesPerRow : width * bitsAllocated / 8
bitsPerPixel : bitsAllocated];
int scale = USHRT_MAX / largestImagePixelValue;
uint16_t *ptr = (uint16_t *)imageRep.bitmapData;
for (int i = 0; i < width * height; i++) *ptr++ *= scale;
It is important to know about the Transfer Syntax (0002:0010) and Number of frames in the dataset. Also, try to get the value length and VR for Pixel Data (7FE0:0010) element. Using value length of the pixel data element you will be able to validate your calculation for uncompressed image.
As for displaying the image, you will also need the value for High Bit (0028:0102) and Pixel Representation (0028:0103). An image could be 16-bit allocated, 12-bit stored, high bit set to 15 and have one sample per pixel. That means 4 lest significant bits of each word do not contain pixel data. Pixel Representation when set to 1 means sign bit is the high bit in pixel sample.
In addition, you many need to apply modality LUT transformation (rescale slope and rescale intercept for linear transformation) when present in the dataset to prepare the data for display. At the end, you apply the VOI LUT transformation (Window center and Window Width) to display the image.

VRML IndexedLineSet thickness

I want to increase the IndexedLineSet thickness here is my code:
geometry IndexedLineSet {
coord DEF Line Coordinate {
point [0 0 0, 0 0 0]
}
coordIndex [ 0, 1, -1 ]
}
I have tried lineWidth but it doesn't work is there any other attribute I can use?
Thanks.
Unfortunately, as I understand it (it's been a long while) if you want to use an IndexedLineSet you can't control the thickness. There's no thickness attribute for an IndexedLineSet and the appearance is implementation dependent. You'd have to have boxes, or extruded circles/ellipses to get a thickness.

Initialising a C Struct Array - Objective C - OpenGLES

I have the following Vertex struct in my OpenGL ES app :
typedef struct Vertex {
float Position[3];
float Color[4];
} Vertex;
In my header I then declare :
Vertex *Vertices;
Then in my init method :
int array = 4;
Vertices = (Vertex *)malloc(array * sizeof(Vertex));
I then later setup the mesh as follows, where vertices array in this case has 4 vertices :
- (void)setupMesh {
int count = 0;
for (VerticeObject * object in verticesArray) {
Vertices[count].Position[0] = object.x;
Vertices[count].Position[1] = object.y;
Vertices[count].Position[2] = object.z;
Vertices[count].Color[0] = 0.9f;
Vertices[count].Color[1] = 0.9f;
Vertices[count].Color[2] = 0.9f;
Vertices[count].Color[3] = 1.0f;
count ++;
}
}
Can anyone spot what I am doing wrong here ? When I pass this Vertices object to OpenGL nothing is drawn, whereas if I hard code the Vertices array as :
Vertex Vertices [] = {
{{0.0, 0.0, 0}, {0.9, 0.9, 0.9, 1}},
{{0.0 , 0.0 , 0}, {0.9, 0.9, 0.9, 1}},
{{0.0, 0.0, 0}, {0.9, 0.9, 0.9, 1}},
{{0.0, 0.0, 0}, {0.9, 0.9, 0.9, 1}},
};
Everything works ?
I think the problem is that before you had a array allocated on the stack where now you have a pointer(memory address) to a block of memory on the heap. So when you wright stuff like sizeof(Vertices) your original sizeof(Vertices) would result in 4 vertices each holding 3 floats position and 4 floats color -> 4 * (3 + 4) * 4(float = 4 bytes) = 112 bytes. Where sizeof(aPointer) = 4 bytes. OpenGL is a C library an not super easy to work with so you should really brush up on you C skills before trying to get it running. Also there in a GLKView class now days that will make all the setup allot easier.
glBufferData(GL_ARRAY_BUFFER, sizeof(Vertices), Vertices, GL_STATIC_DRAW);
Try to allocate same size as the array of vertices. In your case 4 * sizeof(Vertex).
glBufferData(GL_ARRAY_BUFFER, sizeof(Vertex) * 4, Vertices, GL_STATIC_DRAW);
If that doesn't work you can easily fix the problem by replacing your dynamically allocated array for a statically allocated since you know at compile time how big it needs to be.
Vertex Vertices[4];
Then set the values in your loop as you do.