Vulkan default coord system for vertex positions - vulkan

Question

I'm studing Vulkan coordinate system stuff by working on a toy renderer.
I'm confused about coordinates of vertex positions.
Online Vulkan info, such as this:
https://matthewwellings.com/blog/the-new-vulkan-coordinate-system/
...mention that +X is right, +Y is down, and +Z is back.
In my renderer, +Z is pointing forward and I can't figure out why.
I have a triangle defined like this:
// CCW is facing forward
std::vector<PosColorVertex> vertexBuffer = {
{{ 0.0f, -1.0f, 0.0f}, {1.0f, 0.0f, 0.0f}},
{{-1.0f, 1.0f, 0.0f}, {0.0f, 1.0f, 0.0f}},
{{ 1.0f, 1.0f, -5.0f}, {0.0f, 0.0f, 1.0f}},
};
That -5(Z) moves the vertex back, into the screen. It should be +5 that does that.
The coordinate system seems to be like this:
If I place the camera at the origin, it looks like this:
Another shot, with the camera away from the triangle (view translated by -4 on Z).
Some relevant code. Both model & view matrix are identity.
VS:
outColor = inColor;
gl_Position = ubo.projectionMatrix * ubo.viewMatrix * ubo.modelMatrix * vec4(inPos.xyz, 1.0);
FS:
outFragColor = vec4(inColor, 1.0);
Projection is:
glm::perspective(glm::radians(60.0f), w/h, 0.1, 256.0);

Answer 1

Clip, and normalized device coordinates
Clip coordinates are those we get from the vertex shader. Normalized device coordinates (NDC) are the same, but divided by w. There are two common user options (left-handed, and right-handed):
What "up" means is actually up to you. But if you want it to be compatible with virtually all presentation engines, you want "up" to mean -y after the viewport transform (so in NDH your "up" should be -y in case of vanilla viewport transform, or it should be +y in case viewport transform later flips it).
The choice of "up" being always -y in framebuffer\image coordinates is because surface coordinates on virtually all presentation engines assume upper-left origin:
Many image file formats also assume the same.
World coordinates
World coordinates are the ones that are perfectly up to you. And via the vertex shader you transform the world coordinates into the clip coordinates Vulkan can process.
You did this via your glm::perspective.
Let's first actually see what we have:
std::vector<PosColorVertex> vertexBuffer = {
{{ 0.0f, -1.0f, 0.0f}, {1.0f, 0.0f, 0.0f}},
{{-1.0f, 1.0f, 0.0f}, {0.0f, 1.0f, 0.0f}},
{{ 1.0f, 1.0f, -5.0f}, {0.0f, 0.0f, 1.0f}},
};
Now, this is again highly up to interpretation what this actually means. We need another "up" reference direction.
To stay sane, we would perhaps like "up" to be in the increasing direction of y. So that means we got some red corner at the bottom. We got one green corner at upper left. And we got one blue corner at upper right. Or so I assume was the author's intent.
Additionally to stay sane we would prefer right-handed coordinate system. So, if we have chosen that +y means "up" and +x means "right", then -z gotta be "front" (and +z gotta be "back"):
(This is something that matches e.g. how Blender have world coordinates.) Now we got ourselves in bit of a pickle though. Our z is negative instead of positive as required by NDH. And our y points the other way than in NDH compared to "up". Whatever transform we do, we need to make these match.
What does glm::perspective() do
glm::perspective() primarily does make it look, well perspectivy. But for that it needs to do couple of assumptions.
The easy one is depth. In Vulkan in NDC it is zero to one. Conveniently there is GLM_FORCE_DEPTH_ZERO_TO_ONE. That instucts perspective() that it should map your near and far to 0 and 1, respectively. (The defualt is -1 to 1, which would not work in Vulkan unless manually corrected.) The x and y are still always -1 to 1.
Second choice is handedness. Right-handed is default. Left-handed needs GLM_FORCE_LEFT_HANDED. Or it can be used explicitly for the single function, e.g. perspectiveRH(). It is slightly misleading. What this actually means here is that "right-handed" implies "front" being -z. And "left-handed" means the projection assumes "front" is into positive direction of z:
Third choice is what is "up". glm::perspective() does not do anything with this actually, and the polarity of y stays the same througout the transform. If we want +y to mean "up", we need to do this manually. Either we can make use of the viewport flip feature, or we can bake it into the view-projection matrix: proj[1][1] = -proj[1][1].
How to test this stuff
It is actually pretty straightforwar to test this. This code can be used for the purpose:
#include <cmath>
#include <iostream>
#define GLM_FORCE_RADIANS
#define GLM_FORCE_DEPTH_ZERO_TO_ONE
//#define GLM_FORCE_LEFT_HANDED
#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>
int main(){
#ifdef GLM_FORCE_LEFT_HANDED
const float near = 1.0f;
const float far = 2.0f;
#else //right-handed
const float near = -1.0f;
const float far = -2.0f;
#endif
glm::vec3 right { 1.0f, 0.0f, near};
glm::vec3 left {-1.0f, 0.0f, near};
glm::vec3 up { 0.0f, 1.0f, near};
glm::vec3 down { 0.0f, -1.0f, near};
glm::vec3 front { 0.0f, 0.0f, (far-near) + near };
glm::vec3 back { 0.0f, 0.0f, (near-far) + near };
const auto xform = glm::perspective(glm::radians(60.0f), 1.0f, std::abs(near), std::abs(far));
auto r = xform * glm::vec4(right , 1.0f);
auto l = xform * glm::vec4(left , 1.0f);
auto u = xform * glm::vec4(up , 1.0f);
auto d = xform * glm::vec4(down , 1.0f);
auto f = xform * glm::vec4(front , 1.0f);
auto b = xform * glm::vec4(back , 1.0f);
std::cout << "Right to clip: (" << r.x << ", " << r.y << ", " << r.z << ", " << r.w << ")\n";
std::cout << "Left to clip: (" << l.x << ", " << l.y << ", " << l.z << ", " << l.w << ")\n";
std::cout << "Up to clip: (" << u.x << ", " << u.y << ", " << u.z << ", " << u.w << ")\n";
std::cout << "Down to clip: (" << d.x << ", " << d.y << ", " << d.z << ", " << d.w << ")\n";
std::cout << "Front to clip: (" << f.x << ", " << f.y << ", " << f.z << ", " << f.w << ")\n";
std::cout << "Back to clip: (" << b.x << ", " << b.y << ", " << b.z << ", " << b.w << ")\n";
}
For both handedness settings, we get:
Right to clip: (1.73205, 0, 0, 1)
Left to clip: (-1.73205, 0, 0, 1)
Up to clip: (0, 1.73205, 0, 1)
Down to clip: (0, -1.73205, 0, 1)
Front to clip: (0, 0, 2, 2)
Back to clip: (0, 0, -2, 0)
Ups, it all gets clipped away. But anyway "right" and "up" is positive number. So yea, we might want to flip y some way to be compatible with presentation engine coordinates. "front" is (0,0,1) direction, and "back" stops existing.
Note what changes in the code is the direction of z as used in the world coordinates.