Vulkan input attachment from previous subpass - vulkan

I am trying to use the color attachment created in subpass 0 as an input attachment in subpass 1. However there is an issue that i cant get past.
My problem currently is the following, i try to clear attachment 0 at the beginning of the pass using VK_ATTACHMENT_LOAD_OP_CLEAR however gives the error.
Cannot clear attachment 0 with invalid first layout VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL.
Which seems weird to me, attachment 0 does not get layout VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL untill subpass 1 where it is a input attachment, while the clear should have already happened when the layout was still VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL (AFAIK).
I have now tried to just continue running after the validation error, and attachment 0 does get cleared, which makes me even more unsure. I can apparently just ignore the validation error, but something weird might be going on that could cause problems later on so I am not confidant enough to just ignore it.
Here is the minimal code that will give the error:
VkAttachmentDescription attachments[1] {};
attachments[0].format = VK_FORMAT_R16G16B16A16_SFLOAT;
attachments[0].samples = VK_SAMPLE_COUNT_1_BIT;
attachments[0].loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
attachments[0].storeOp = VK_ATTACHMENT_STORE_OP_STORE;
attachments[0].stencilLoadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;
attachments[0].stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
attachments[0].initialLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
attachments[0].finalLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
VkAttachmentReference pass_0_ColorAttachments[1];
pass_0_ColorAttachments[0].attachment = 0;
pass_0_ColorAttachments[0].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
VkAttachmentReference pass_1_InputAttachments[1];
pass_1_InputAttachments[0].attachment = 0;
pass_1_InputAttachments[0].layout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
VkSubpassDescription subpasses[2] {};
subpasses[0].colorAttachmentCount = 1;
subpasses[0].pColorAttachments = pass_0_ColorAttachments;
subpasses[0].pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
subpasses[1].inputAttachmentCount = 1;
subpasses[1].pInputAttachments = pass_1_InputAttachments;
subpasses[1].pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
VkSubpassDependency subpassDependancies[1] {};
subpassDependancies[0].srcSubpass = 0;
subpassDependancies[0].dstSubpass = 1;
subpassDependancies[0].srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
subpassDependancies[0].dstStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
subpassDependancies[0].srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
subpassDependancies[0].dstAccessMask = VK_ACCESS_INPUT_ATTACHMENT_READ_BIT;
subpassDependancies[0].dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT;
VkRenderPassCreateInfo renderpassCreateInfo {};
renderpassCreateInfo.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO;
renderpassCreateInfo.attachmentCount = 1;
renderpassCreateInfo.pAttachments = attachments;
renderpassCreateInfo.subpassCount = 2;
renderpassCreateInfo.pSubpasses = subpasses;
renderpassCreateInfo.dependencyCount = 1;
renderpassCreateInfo.pDependencies = subpassDependancies;
VkRenderPass renderPass;
vkCreateRenderPass(device, &renderpassCreateInfo, nullptr, &renderPass);
The next part is only relevant if it is impossible to specify VK_ATTACHMENT_LOAD_OP_CLEAR for an attachment used as color attachment first and in a later subpass as input attachment (i see no reason why this would be impossible, unless vulkan does the load operation each subpass). So this is kind of a separate problem.
I could, instead of using VK_ATTACHMENT_LOAD_OP_CLEAR, manually clear the attachment using vkCmdClearAttachments and use VK_ATTACHMENT_LOAD_OP_DONT_CARE for the load operation of the attachment.
I have a crash error on calling vkCmdClearAttachments
I started the command buffer recording and the renderpass, and in the first subpass I call:
VkClearAttachment clearAtts[] = {{VK_IMAGE_ASPECT_COLOR_BIT, 1, {0,0,0,0}}};
VkClearRect rect = {{{0,0}, {1,1}}, 0, 1};
vkCmdClearAttachments(vkCommandBuffer, 1, clearAtts, 1, &rect);
{1,1} as the extent to show this is not the problem

You're having the same problem as was discovered here. It's a bug in the validation layer in 1.0.17. It would appear that the current Github head has a fix, but that fix is not in the 1.0.21 release.
But you still need to fix the stuff below ;)
subpassDependancies[0].dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_READ_BIT;
You are not going to read the image as a color attachment. You're going to read it as an input attachment. From the fragment shader. So that would mean VK_ACCESS_INPUT_ATTACHMENT_READ_BIT.
If that fixes the error, then I'd guess the validation layer got confused by your oddball dependency access mask. It therefore couldn't establish a valid dependency chain between subpasses 0 and 1, and thus thought that subpass 1 was first.
Also, your srcStageMask and dstStageMask bits are way over-specified. Every graphics stage is not going to write to the image, and every graphics stage is not going to access it after. You only wrote to the image as a color attachment, and you're only going to read from it in the fragment shader.
So your srcStageMask ought to just be VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, and your dstStageMask ought to be VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT.

Related

How to work around ImageJ run("HSB stack") error/ bug?

I am working on a macro for ImageJ. The goal is to take colour scans with several seeds on them and crop around the seeds to get several equally sized images with one seed on each.
This is the basic idea for the macro: prompt to select folder with scans (info about the seed is in the name of the image) > threshold to select seeds > crop around each seed on the original image > save all of the cropped images in a folder (name of the cropped images still containing the information of the name of the original image)
When I run the code below, I get an error for line 31: run("HSB stack");
The error informs me about supported conversions and shows that in order to run this command I need to start with an RGB image. However, according to Fiji > Image > Type, my images are RGBs. A coding error in that part also seems unlikely since it was written with the recording function in ImageJ.
Error message
According to what I found for the error, this seems to concern a recurring bug in the software, specific to the commands run("HSB stack") and run("RGB stack") in macros.
We have tried running this on ImageJ 2.3.0/1.53s as well as 1.53q on MacOS and Windows and always got the same problem.
If it is not a software problem, where is the error? Or if it is, do you have any suggestions for workarounds or a different program that could perform the same job?
The images I am working with are colour scans, 600dpi, white background with between 1 and 90 seeds on each scan. They are large tiff images (107.4 MB) but look like this:
Example scan image
I am not sure if it is helpful, but the code is below. There are probably still errors in the latter part that I could not yet get to because I can't get past the problem in line 31.
// Directory
dir=getDirectory("Choose a data folder");
list = getFileList(dir);
processed_dir_name = dir + "Cropped" + File.separator;
print(processed_dir_name);
File.makeDirectory(processed_dir_name);
// Batch
for (i=0; j<list.length; i++) {
print(i + ":" + dir+list[i]};
// Open images
run("Bio-Formats Importer", "open=" + dir+list[i] + "color_mode=Default view =Hyperstack");
// Crop edge, set general cropping parameters, scale
makeRectangle(108, 60, 4908, 6888);
run("Crop");
main = getTitle():
default_crop_width = 350;
default_crop_height = 350;
run("Set Scale...", "distance=600 known=25.4 unit=mm global");
//Thresholding
run("Color Threshold...");
//Color Thresholder 2.3.0/1.53q
// Autogenerated macro, single images only!
min=newArray(3);
max=newArray(3);
filter=newArray(3);
a=getTitle();
run("HSB stack");
run("Convert Stack to images");
selectWindow("Hue");
rename("0");
selectWindow("Saturation");
rename("1");
selectWindow("Brightness");
rename("2");
min[0]=0;
max[0]=255;
filter[0]="pass";
min[1]=0;
max[1]=255;
filter[1]="pass";
min[2]=0;
max[2]=193;
filter[2]="pass";
for (i=0;j<3;i++){
selectWindow(""+i);
The problem lies in the fact that your image is a hyperstack, and the color thresholding doesn't know how to work with that.
There are a few options you could try: Open the image as an 8-bit RGB, e.g. via open(dir+list[i]); or split the channels of the hyperstack and threshold each separately. Based on your sample image, I assume the first option makes more sense.
The following is an edited version of your code that works for the sample that you've provided:
// Directory
dir=getDirectory("Choose a data folder");
list = getFileList(dir);
processed_dir_name = dir + "Cropped" + File.separator;
print(processed_dir_name);
File.makeDirectory(processed_dir_name);
// Batch
for (i=0; i<list.length; i++)
{
if (!File.isDirectory(dir+list[i])) // Ignore directories such as processed_dir_name
{
print(i + ":" + dir+list[i]);
// Open images
open(dir+list[i]);
// Crop edge, set general cropping parameters, scale
makeRectangle(108, 60, 4908, 6888);
run("Crop");
main = getTitle();
default_crop_width = 350;
default_crop_height = 350;
run("Set Scale...", "distance=600 known=25.4 unit=mm global");
//Thresholding
//run("Color Threshold...");
//Color Thresholder 2.3.0/1.53q
// Autogenerated macro, single images only!
min=newArray(3);
max=newArray(3);
filter=newArray(3);
a=getTitle();
run("HSB Stack");
run("Convert Stack to Images");
selectWindow("Hue");
rename("0");
selectWindow("Saturation");
rename("1");
selectWindow("Brightness");
rename("2");
min[0]=0;
max[0]=255;
filter[0]="pass";
min[1]=0;
max[1]=255;
filter[1]="pass";
min[2]=0;
max[2]=193;
filter[2]="pass";
for (j=0;j<3;j++){
selectWindow(""+j);
}
}
}

Vulkan Instanced Rendering Weird Depth Buffer Behaviour

I am rendering a single mesh test object that contains 8000 cubes, 1 monkey, and 1 plane. I checked normals and winding orders. all seems correct for mesh.
if I render a single instance, the depth buffer works correctly. if I render using instanced when instant number increase depth buffer doesn't work correctly.
//Depth Attachment
//Create Depth Image
depthImage = new Image(
physicalDevice_,
logicalDevice_,
VK_IMAGE_TYPE_2D,
depthFormat,
VK_IMAGE_TILING_OPTIMAL,
swapChainExtent.width,
swapChainExtent.height,
1,
sampleCount_, //do we really need to sample depth????
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT,
VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
depthImageView = new ImageView(
logicalDevice_,
VK_IMAGE_VIEW_TYPE_2D,
depthImage->handle,
depthFormat,
1,
VK_IMAGE_ASPECT_DEPTH_BIT);
//Render pass Depth Attachment
VkAttachmentDescription depthAttachment{};
depthAttachment.format = depthFormat;
depthAttachment.samples = VK_SAMPLE_COUNT_1_BIT;
depthAttachment.loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
depthAttachment.storeOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
depthAttachment.stencilLoadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;
depthAttachment.stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
depthAttachment.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
depthAttachment.finalLayout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL;
//Graphics Pipeline
VkPipelineDepthStencilStateCreateInfo depthStencil{};
depthStencil.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO;
depthStencil.depthTestEnable = VK_TRUE;
depthStencil.depthWriteEnable = VK_TRUE;
depthStencil.depthCompareOp = VK_COMPARE_OP_LESS;
depthStencil.depthBoundsTestEnable = VK_FALSE;
depthStencil.minDepthBounds = 0.0f; // Optional
depthStencil.maxDepthBounds = 1.0f; // Optional
depthStencil.stencilTestEnable = VK_FALSE;
depthStencil.front = {}; // Optional
depthStencil.back = {}; // Optional
Below the screen is the view from the 100th instance of that mesh.
The higher the instance number, the more the depth buffer deteriorates.
The result is still the same if I use a different model.
What am I missing?
Result:
Depth Buffer:
For those who may struggle with similar issue. I found a solution.
Actually, this issue is not related to instancing directly but it's a bit confusing issue becomes visible as the instance index increases.
the cause is projection matrix z-near value was so small.
...
fieldOfView = 65.0f;
znear = 0.001f; // ! this much small value cause this problem
zfar = 5000.0f;
...
changed z-near to an acceptable value. Problem solved.
...
fieldOfView = 65.0f;
znear = 0.1f;
zfar = 5000.0f;
...

VkSubpassDependency required for depth attachment, but not for color attachment

I have a very simple setup - in fact one of the earlier stages of vkguide.dev. So a single render pass with a single subpass. Those have one color and one depth attachment.
The color AttachmentDescription has .initialLayout = VK_IMAGE_LAYOUT_UNDEFINED and .finalLayout = VK_IMAGE_LAYOUT_PRESENT_SRC_KHR. In its SubPassDescription, it has .layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.
The depth AttachmentDescription has .initialLayout = VK_IMAGE_LAYOUT_UNDEFINED and .finalLayout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL. In its SubPassDescription, it has .layout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL.
I'm trying to understand the subpass dependencies, so I don't want to rely on the implicit ones the spec provides but provide my own, minimal ones.
When I provide "zeroed out" subpass dependencies, I get a the following error with sync validation turned on:
Validation Error: [ SYNC-HAZARD-WRITE_AFTER_WRITE ] Object 0: handle = 0x83d4ee000000000b, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0xfdf9f5e1 | vkCmdBeginRenderPass: Hazard WRITE_AFTER_WRITE vs. layout transition in subpass 0 for attachment 1 aspect depth during load with loadOp VK_ATTACHMENT_LOAD_OP_CLEAR.
So he's complaining about the depth attachment (attachment 1). And I can fix this with the following set of explicit subpass dependencies:
constexpr VkSubpassDependency in_dependency{
.srcSubpass = VK_SUBPASS_EXTERNAL,
.dstSubpass = 0,
.srcStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,
.dstStageMask = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT,
.srcAccessMask = 0,
.dstAccessMask = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT
};
constexpr VkSubpassDependency out_dependency = {
.srcSubpass = 0,
.dstSubpass = VK_SUBPASS_EXTERNAL,
.srcStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,
.dstStageMask = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
.srcAccessMask = 0,
.dstAccessMask = 0,
.dependencyFlags = 0
};
The second of course is only there to not use the second implicit one and does nothing. The first one is an explicit dependency between things before the renderpass and the first/only subpass. But I don't really understand - a couple of questions:
does a srcAccessMask = 0 really define a real dependency? it's empty after all, or are things different with a VK_SUBPASS_EXTERNAL?
Why do I only need a dependency on the depth- and not on the color attachment?
I made the image below and am aware that color attachment gets loaded and stored in the same pipeline stage, and that the depth attachment does now. However I still don't understand why that eliminates the need for a dependency. That pipeline stage could still overlap between two renderpasses right?
does a srcAccessMask = 0 really define a real dependency? it's empty after all, or are things different with a VK_SUBPASS_EXTERNAL?
Yes it does. It is called an execution dependency (as opposed to memory dependency). It becomes relevant in dependency chaining.
Consider this code:
vkCmdPipelineBarrier(
srcStage = WHATEVER_STAGE, srcAccess = MEMORY_WRITE,
dstStage = COOL_STAGE, dstAccess = 0
);
vkCmdPipelineBarrier(
srcStage = COOL_STAGE, srcAccess = 0,
dstStage = WHATEVER_ELSE_STAGE, dstAccess = MEMORY_READ
);
It is equivalent to a single dependency:
vkCmdPipelineBarrier(
srcStage = WHATEVER_STAGE, srcAccess = MEMORY_WRITE,
dstStage = WHATEVER_ELSE_STAGE, dstAccess = MEMORY_READ
);
Why do I only need a dependency on the depth- and not on the color attachment?
The validation might be implemented imperfectly. Or likely the color attachment is treated differently in your code (the color images are presented, while the depth is not). Which either sneaks the dependency there, or makes it less trivial for the validator to catch.

Cannonjs move body forward according to its quaternion

I'm struggling with Cannonjs physics lib, I got a sphere body with a changing quaternion, I just want that body to move forward according to its quaternion. I found many topics related to that but none of the suggested codes is working.
Is it a simple way to achieve that simple task?
So far I tried the solution given here but the using of vmult() method do not change the vector3 at all...
body.quaternion = new CANNON.Quaternion(0,1,0,0); // Whatever value you put here will not change the result of the vmult() operation below
var localVelocity = new CANNON.Vec3(0, 0, 1);
var worldVelocity = body.quaternion.vmult(localVelocity); //-> worldVelocity = (0,0,1)
body.velocity.copy(worldVelocity);
Try to set and not copy the body.velocity, this way:
let directionVector = new CANNON.Vec3(0, 0, 1);
directionVector.z -= moveDistance;
directionVector = vehicleBody.quaternion.vmult( directionVector );
vehicleBody.velocity.set( directionVector.x, directionVector.y, directionVector.z );

What the equivalent of OpenGL glStencilFunc in Metal?

How to translate to Metal the following OpenGL instruction:
glStencilFunc(Func, Ref, Mask);
Let's try to extend Hamid's correct answer with some actual Swift code...
Assuming you already have the basic Metal setup done:
let metalView: MTKView = /* ... */
let commandBuffer: MTLCommandBuffer = /* ... */
let device: MTLDevice = metalView.device!
Let's create a MTLTexture to store the stencil buffer:
let stencilTexture: MTLTexture
// Set up a texture for rendering the *8 bit unsigned* stencil buffer.
let stencilTextureDescriptor = MTLTextureDescriptor()
stencilTextureDescriptor.textureType = .type2D
stencilTextureDescriptor.width = Int(metalView.frame.width)
stencilTextureDescriptor.height = Int(metalView.frame.height)
stencilTextureDescriptor.pixelFormat = .stencil8
stencilTextureDescriptor.storageMode = .private
// Set this option if you use the given texture
// as a stencil render target in any render pass.
stencilTextureDescriptor.usage = [.renderTarget]
stencilTexture = device.makeTexture(descriptor: stencilTextureDescriptor)
Create the MTLRenderPipelineState as usual but be sure to set the stencil attachment too:
let rendePipelineState: MTLRenderPipelineState
let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
/* ...standard render pipeline configuration (e.g., render states, etc)... */
renderPipelineDescriptor.stencilAttachmentPixelFormat = .stencil8
rendePipelineState = try! device.makeRenderPipelineState(descriptor: renderPipelineDescriptor)
Next configure the MTLRenderPassDescriptor and create a MTLRenderCommandEncoder:
let renderEncoder: MTLRenderCommandEncoder
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0, 0, 0, 1)
renderPassDescriptor.colorAttachments[0].storeAction = .store
// Configure these depending on your rendering pass stencil semantics.
renderPassDescriptor.stencilAttachment.texture = stencilTexture
renderPassDescriptor.stencilAttachment.clearStencil = 0
renderPassDescriptor.stencilAttachment.loadAction = .clear
renderPassDescriptor.stencilAttachment.storeAction = .store
renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.setRenderPipelineState(rendePipelineState)
Now create a MTLDepthStencilState with the desired stencil buffer configuration:
let depthAndStencilState: MTLDepthStencilState
let stencilDescriptor = MTLStencilDescriptor()
stencilDescriptor.depthStencilPassOperation = .incrementClamp
// ## glStencilFunc(FUNC, _, _) ##
stencilDescriptor.stencilCompareFunction = .equal
// ## glStencilFunc(_, _, MASK) ##
stencilDescriptor.readMask = 0xFF_FF_FF_FF
let depthAndStencilDescriptor = MTLDepthStencilDescriptor()
depthAndStencilDescriptor.depthCompareFunction = .always
depthAndStencilDescriptor.isDepthWriteEnabled = false
depthAndStencilDescriptor.frontFaceStencil = stencilDescriptor
depthAndStencilDescriptor.backFaceStencil = stencilDescriptor
depthAndStencilState = device.makeDepthStencilState(descriptor: depthAndStencilDescriptor)!
renderEncoder.setDepthStencilState(depthAndStencilState)
// ## glStencilFunc(_, REF, _) ##
renderEncoder.setStencilReferenceValue(1)
Finally, do the rendering as usual:
/* ...encode rendering primitives... */
renderEncoder.endEncoding()
MTLStencilDescriptor an object that defines the front-facing or back-facing stencil operations of a depth and stencil state object.
A stencil test is a comparison between a masked reference value and a
masked value stored in a stencil attachment. (A value is masked by
performing a logical AND operation on it with the readMask value.) The
MTLStencilDescriptor object defines how to update the contents of the
stencil attachment, based on the results of the stencil test and the
depth test.
The stencilCompareFunction property defines the stencil test. The
stencilFailureOperation, depthFailureOperation, and
depthStencilPassOperation properties specify what to do to a stencil
value stored in the stencil attachment for three different test
outcomes: if the stencil test fails, if the stencil test passes and
the depth test fails, or if both stencil and depth tests succeed,
respectively. writeMask determines which stencil bits can be modified
as the result of a stencil operation.
I see where to put the value of Func and Mask with
MTLStencilDescriptor, but where go the value of REF ?
setStencilReferenceValue sets a stencil reference value for both front and back stencil comparison tests.