I'm trying to figure out how to share some push constants between different shader stages, by setting up multiple push constant ranges. I success when I use one single range with VkFlags VK_SHADER_STAGE_ALL, but I'm not sure if this is the correct way?
Here's an example of what I'm trying to achieve:
Fragment shader:
layout(push_constant) uniform fragmentPushConstants {
layout(offset = 0) float time;
layout(offset = 4) vec4 color;
} u_pushConstants;
Vertex shader:
layout(push_constant) uniform vertexPushConstants {
layout(offset = 0) float time;
} u_pushConstants;
For this example, how many ranges should I provide vkCreatePipelineLayout with and how should I structure them?
It's working if one single range (0 - 20) with VK_SHADER_STAGE_ALL is provided to the pipeline layout info structure. I cannot find any examples what so ever of multi-range usage, except when two different ranges without any overlapping ranges are being used. What's the purpose of ranges at all if I just could use one single range(0 - max) anyway with VK_SHADER_STAGE_ALL?
Related
Right now I am writing out to a colour buffer in the fragment shader, which is a float format.
layout (location = 0) out vec4 outColour;
I need to have a way to write the object's id to a framebuffer for picking. There are a number of ways I've thought about doing this. I can compile two versions for each shader, one a normal one, and another for the picking, which basically only needs to do the vertex position transformations and then skip everything else, lighting calculations, texturing, etc. This probably isn't ideal because this is essentially doubling the number of shaders I have to write.
An easier method I've thought is to do a conditional branch (preferably over a specialisation constant), and for picking purposes compile a picking version of the graphics pipeline with the picking boolean value set to true. This sounds better. For the ordinary passes I can write to multiple attachments. Will it be best to compile that picking pipeline with a new render pass that writes to only one framebuffer attachment, an integer one? If I swap the render pass for one that writes an integer at attachment 0 instead of the float 4 can I alias this in the fragment shader?
layout (location = 0) out vec4 outColour;
layout (location = 0) out ivec4 out_id;
void main()
{
vec4 colour;
int object_id;
if (bPicking)
out_id = ivec4(object_id, 0, 0, 0); // y, z, w not used
else
out_colour = colour;
}
I'm guessing I really need a different render pass because instead of writing to a R32G32B32A32_SFLOAT image I'm writing to a R8_UINT image for the IDs. This is really confusing what's the best way to do this?
I am trying to write a Vulkan renderer, I use glslangValidator with HLSL for shaders and am trying to implement push constants.
[[vk::push_constant]]
cbuffer cbFragment {
float4 imageColor;
float4 aaaa;
};
[[vk::push_constant]]
cbuffer cbMatrices {
float4 bbbb;
};
The annotation "[[vk::push_constant]]" works, I use spirv_reflect for reflection and both push constants show up and they work as intended.
The problem I'm having is that they seemingly overlap, if I assign "bbbb" a value, "imageColor" is affected in exactly the same way and vice versa. In the reflection data both push constant blocks have the offset 0, which explains the issue. However, I seem to be completely unable to change the offset of either of the push constants.
[[vk::offset(x)]] does not work at all, it neither affects the individual member offsets nor the offset of the push constants. The only offset that works at all is HLSL's built in "packoffset", which only applies to the buffer members. And although it might actually be a solution to just offset the members of one of the push constants to be outside the range of the other, I hardly believe that can be a sensible solution as it's also causing the validation layer to fail because offsetting the individual member simply increases the size of the push constant unnecessarily and the overlap itself is still present.
I would greatly appreciate any help on this matter and am willing to provide any necessary clarification, thank you very much!
Push constants live in a single chunk of contiguous memory. The compiler doesn't try to append multiple blocks into that memory; like with the GLSL syntax, it's intended to just have one block containing all the push constant data.
This is consistent with other places where the compiler has to pack variables in a block: it only packs within a block, not across multiple blocks. Two separate non-pushconstant cbuffers would refer to two distinct buffers in memory, with contents that begin at offset zero within their individual buffer. There's only one "push constant buffer", hence you should only decorate one cbuffer with vk::push_constant.
Out of curiosity, will it be more efficient to write shader variables like this :
lowp vec4 tC = texture2D(uTexture, vTexCoord); // texture color
or
lowp vec4 textureColor = texture2D(uTexture, vTexCoord); // texture color
Note that I wrote variable tC because it has less characters than variable textureColor
I understand in programming language like C/ObjC, it doesn't matter, but what about shader, since you can query the attributes / uniform names.
It shouldn't make a measurable difference. After linking your program during initialization, query the locations of attributes/uniforms, and keep the result around with the program handle. From then on, neither your app nor the driver will be touching the name strings, just the integer locations.
Even if you re-query locations every time you need to change an attrib binding or uniform value, the difference between a short and "moderate" name length likely won't make much difference compared to the other costs of doing the lookup and binding/value change.
I need to develop an application where a user (physiotherapist) will perform a movement in front of the Kinect, I'll write the data movement in the database and then the patient will try to imitate this motion. The system will calculate the similarity between the movement recorded and executed.
My first idea is, during recording (each 5 second, by example), to store the position (x, y, z) of the points and then compare them in the execution time(by patient).
I know that this approach is too simple, because I imagine that in people of different sizes the skeleton is recognized differently, so the comparison is not reliable.
My question is about the best way to compare a saved motion with a movement executed (on the fly).
I have done this, where a doctors frame is projected onto the patients frame, but with the whole skeleton this doesn't work so well because of different bone heights :/. The code can be found here. It is in beta 2 code, the more current version can be found here, although it is not currently working perfectly
As for comparing, do something like this
for (int i = 0; i < patientList.Count; i++)
{
int diff = (int)Math.Abs(patientList[i] - doctorList[i]);
if (diff < 100) //or whatever number you want
{
Debug.WriteLine("Good Job");
}
}
I have abandoned the idea of a whole figure because of the bone heights mentioned by Fixus, so my current program looks some thing like:
EDIT
This is the concept of camparing two movements with kinect and calculate a similarity between the two movements I explain in depth.
Suppose I have the following 2 points, point A (0, 0, 0) and point B (1, 1, 1). Now I want to find the difference from point A to B, so I would subtract all of the X, Y, and Z numbers, so the difference is 1 X 1 Y 1 Z. That is the simple stuff. Now to implement it. The code I have written above, I would implement like this.
//get patient hand coordinates
double patienthandX = Canvas.GetLeft(patienthand);
double patienthandY = Canvas.GetTop(patienthand);
//get doctor hand coordinates
double doctorhandX = Canvas.GetLeft(doctorhand);
double doctorhandY = Canvas.GetTop(doctorhand);
//compare difference for each x and y
//take Absolute value so that it is positive
double diffhandX = Math.Abs(patienthandX - doctorhandX);
double diffhandY = Math.Abs(patienthandY - doctorhandY);
Now here comes another issue. The doctor coordinates are always the same, but what if the patient isn't standing where the doctor coordinates were recorded? Now we implement more simple math. Take this simple example. suppose I want point A(8, 2) to move to point B(4, 12). You multiply the x and y's of A to get to B. So I would multiply the X by .5, and the Y by 6. So for Kinect, I would put a element on the patients hip, then compare this to the doctors hip. Then multiply all of the doctor joints by that number to achieve the doctor joints on top of the patients (more or less). For example
double whatToMultiplyX = (double) doctorhipX / patienthipX;
double whatToMultiplyY = (double) doctorhipY / patienthipY;
This is all pretty simple, but bringing it together is the harder part. So far we, 1) Scale the doctor frames on top of the patient frames, 2) Calculate the difference. 3) Compare the difference throughout the entire rep. and 4) Reset for the next rep. This seems simple but it is not. To calculate the entire difference for the rep, do something like this:
//get patient hand coordinates
double patienthandX = Canvas.GetLeft(patienthand);
double patienthandY = Canvas.GetTop(patienthand);
//get doctor hand coordinates
double doctorhandX = Canvas.GetLeft(doctorhand);
double doctorhandY = Canvas.GetTop(doctorhand);
//compare difference for each x and y
//take Absolute value so that it is positive
double diffhandX = Math.Abs(patienthandX - doctorhandX);
double diffhandY = Math.Abs(patienthandY - doctrorhandY);
//+= so that it keeps adding to it.
totaldiffhandX += diffhandX;
totaldiffhandY += diffhandY;
Now we can compare, and say:
if (totaldiffhandX < 1000 && totaldiffhandY < 1000) //keep numbers pretty high since it is an entire rep
{
//reset difference
totaldiffhandX = 0;
totaldiffhandY = 0;
//tell the patient good job
Debug.WriteLine("Good Job");
}
This is pretty easy, but keep in mind you must do this for every single joint's x and y. Otherwise it will not work. Hope this Helps.
First of all remember that people are diffrent. Every person has diffrent height, width, weight, diffrent bones length etc etc
You`re code probably will never work cause of this.
Secondly you need to think more geometrically. Don`t think about points only, think with vectors, their directions. Each movement is movent of some vectors in some directions.
Then the proportion. You need to configure application for each user.
You have some pattern. The patter is your physiotherapist. You need to remember not only his movements but also his body. Arm length, leg length, distances etc. Each user that will be using your app also need to me mesured. Having all this data you can compare movement by scaling sizes and comparing directions of movent
Of course remember that there are some very simple moves like for example. They can be recognized by simple mathematic by checking actual position of the hand and checking direction of the movement. You need for this 3 control points and you`re at home :)
Gesture recognizing isn`t a simple thing
For a program I'm writing, I need to be able to trace a virtual line (that is not straight) that an object must travel along. I was thinking to use NSBezierPath to draw the line, but I cannot find a way to get any point along the line, which I must do so I can move the object along it.
Can anyone suggest a way to find a point along an NSBezierPath? If thats not possible, can anyone suggest a method to do the above?
EDIT: The below code is still accurate, but there are much faster ways to calculate it. See Introduction to Fast Bezier and Even Faster Bezier.
There are two ways to approach this. If you just need to move something along the line, use a CAKeyframeAnimation. This is pretty straightforward and you never need to calculate the points.
If on the other hand you actually need to know the point for some reason, you have to calculate the Bézier yourself. For an example, you can pull the sample code for Chapter 18 from iOS 5 Programming Pushing the Limits. (It is written for iOS, but it applies equally to Mac.) Look in CurvyTextView.m.
Given control points P0_ through P3_, and an offset between 0 and 1 (see below), pointForOffset: will give you the point along the path:
static double Bezier(double t, double P0, double P1, double P2,
double P3) {
return
pow(1-t, 3) * P0
+ 3 * pow(1-t, 2) * t * P1
+ 3 * (1-t) * pow(t, 2) * P2
+ pow(t, 3) * P3;
}
- (CGPoint)pointForOffset:(double)t {
double x = Bezier(t, P0_.x, P1_.x, P2_.x, P3_.x);
double y = Bezier(t, P0_.y, P1_.y, P2_.y, P3_.y);
return CGPointMake(x, y);
}
NOTE: This code violates one of my cardinal rules of always using accessors rather than accessing ivars directly. It's because in it's called many thousands of times, and eliminating the method call has a significant performance impact.
"Offset" is not a trivial thing to work out. It does not proceed linearly along the curve. If you need evenly spaced points along the curve, you'll need to calculate the correct offset for each point. This is done with this routine:
// Simplistic routine to find the offset along Bezier that is
// aDistance away from aPoint. anOffset is the offset used to
// generate aPoint, and saves us the trouble of recalculating it
// This routine just walks forward until it finds a point at least
// aDistance away. Good optimizations here would reduce the number
// of guesses, but this is tricky since if we go too far out, the
// curve might loop back on leading to incorrect results. Tuning
// kStep is good start.
- (double)offsetAtDistance:(double)aDistance
fromPoint:(CGPoint)aPoint
offset:(double)anOffset {
const double kStep = 0.001; // 0.0001 - 0.001 work well
double newDistance = 0;
double newOffset = anOffset + kStep;
while (newDistance <= aDistance && newOffset < 1.0) {
newOffset += kStep;
newDistance = Distance(aPoint,
[self pointForOffset:newOffset]);
}
return newOffset;
}
I leave Distance() as an exercise for the reader, but it's in the example code of course.
The referenced code also provides BezierPrime() and angleForOffset: if you need those. Chapter 18 of iOS:PTL covers this in more detail as part of a discussion on how to draw text along an arbitrary path.