Related
I am using Assimp to load an FBX model with animation (created in Blender) into my DirectX 12 game, but I'm experiencing a very frustrating bug with the animation rendered by the game application.
The test model is a simple 'flagpole' containing four bones like so:
Bone0 -> Bone1 -> Bone2 -> Bone3
The model renders correctly in its rest pose when the keyframe animation is bypassed.
The model also renders and animates properly when the animation rotates the model only by the root bone (Bone0).
However, when importing a model that rotates at the first joint (i.e. at Bone1), the vertices clustered around each joint seem 'stuck' in their original positions, while the vertices surrounding the 'bones' proper appear to follow through with the correct animation.
The result is a crappy zigzag of stretched geometry like so:
Instead the model should resemble an 'allen-key' shape at the end of its animation pose, as shown by the same model rendered in the AssimpViewer utility tool:
Since the model is rendering correctly in AssimpViewer, it's reasonable to assume there are no issues with the FBX file exported by Blender. I then checked and confirmed that the vertices 'stuck' around the joints did indeed have their vertex weights correctly assigned by the game loading code.
The C++ model loading and animation code is based on the popular OGLDev tutorial: https://ogldev.org/www/tutorial38/tutorial38.html
Now the infuriating thing is, since the AssimpViewer tool was correctly rendering the model animation, I also copied in the SceneAnimator and AnimEvaluator classes from that tool to generate the final bone transforms via that code branch as well... only to end up with exactly the same zigzag bug in the game!
I'm reasonably confident there aren't any issues with finding the bone hierarchy structure at initialization, so here are the key functions that traverse the hierarchy and interpolate key frames each frame.
VOID Mesh::ReadNodeHeirarchy(FLOAT animationTime, CONST aiNode* pNode, CONST aiAnimation* pAnim, CONST aiMatrix4x4 parentTransform)
{
std::string nodeName(pNode->mName.data);
// nodeTransform is a relative transform to parent node space
aiMatrix4x4 nodeTransform = pNode->mTransformation;
CONST aiNodeAnim* pNodeAnim = FindNodeAnim(pAnim, nodeName);
if (pNodeAnim)
{
// Interpolate scaling and generate scaling transformation matrix
aiVector3D scaling(1.f, 1.f, 1.f);
CalcInterpolatedScaling(scaling, animationTime, pNodeAnim);
// Interpolate rotation and generate rotation transformation matrix
aiQuaternion rotationQ (1.f, 0.f, 0.f, 0.f);
CalcInterpolatedRotation(rotationQ, animationTime, pNodeAnim);
// Interpolate translation and generate translation transformation matrix
aiVector3D translat(0.f, 0.f, 0.f);
CalcInterpolatedPosition(translat, animationTime, pNodeAnim);
// build the SRT transform matrix
nodeTransform = aiMatrix4x4(rotationQ.GetMatrix());
nodeTransform.a1 *= scaling.x; nodeTransform.b1 *= scaling.x; nodeTransform.c1 *= scaling.x;
nodeTransform.a2 *= scaling.y; nodeTransform.b2 *= scaling.y; nodeTransform.c2 *= scaling.y;
nodeTransform.a3 *= scaling.z; nodeTransform.b3 *= scaling.z; nodeTransform.c3 *= scaling.z;
nodeTransform.a4 = translat.x; nodeTransform.b4 = translat.y; nodeTransform.c4 = translat.z;
}
aiMatrix4x4 globalTransform = parentTransform * nodeTransform;
if (m_boneMapping.find(nodeName) != m_boneMapping.end())
{
UINT boneIndex = m_boneMapping[nodeName];
// the global inverse transform returns us to mesh space!!!
m_boneInfo[boneIndex].FinalTransform = m_globalInverseTransform * globalTransform * m_boneInfo[boneIndex].BoneOffset;
//m_boneInfo[boneIndex].FinalTransform = m_boneInfo[boneIndex].BoneOffset * globalTransform * m_globalInverseTransform;
m_shaderTransforms[boneIndex] = aiMatrixToSimpleMatrix(m_boneInfo[boneIndex].FinalTransform);
}
for (UINT i = 0u; i < pNode->mNumChildren; i++)
{
ReadNodeHeirarchy(animationTime, pNode->mChildren[i], pAnim, globalTransform);
}
}
VOID Mesh::CalcInterpolatedRotation(aiQuaternion& out, FLOAT animationTime, CONST aiNodeAnim* pNodeAnim)
{
UINT rotationKeys = pNodeAnim->mNumRotationKeys;
// we need at least two values to interpolate...
if (rotationKeys == 1u)
{
CONST aiQuaternion& key = pNodeAnim->mRotationKeys[0u].mValue;
out = key;
return;
}
UINT rotationIndex = FindRotation(animationTime, pNodeAnim);
UINT nextRotationIndex = (rotationIndex + 1u) % rotationKeys;
assert(nextRotationIndex < rotationKeys);
CONST aiQuatKey& key = pNodeAnim->mRotationKeys[rotationIndex];
CONST aiQuatKey& nextKey = pNodeAnim->mRotationKeys[nextRotationIndex];
FLOAT deltaTime = FLOAT(nextKey.mTime) - FLOAT(key.mTime);
FLOAT factor = (animationTime - FLOAT(key.mTime)) / deltaTime;
assert(factor >= 0.f && factor <= 1.f);
aiQuaternion::Interpolate(out, key.mValue, nextKey.mValue, factor);
}
I've just included the rotation interpolation here, since the scaling and translation functions are identical. For those unaware, Assimp's aiMatrix4x4 type follows a column-vector math convention, so I haven't messed with original matrix multiplication order.
About the only deviation between my code and the two Assimp-based code branches I've adopted is the requirement to convert the final transforms from aiMatrix4x4 types into a DirectXTK SimpleMath Matrix (really an XMMATRIX) with this conversion function:
Matrix Mesh::aiMatrixToSimpleMatrix(CONST aiMatrix4x4 m)
{
return Matrix
(m.a1, m.a2, m.a3, m.a4,
m.b1, m.b2, m.b3, m.b4,
m.c1, m.c2, m.c3, m.c4,
m.d1, m.d2, m.d3, m.d4);
}
Because of the column-vector orientation of aiMatrix4x4 Assimp matrices, the final bone transforms are not transposed for HLSL consumption. The array of final bone transforms are passed to the skinning vertex shader constant buffer as follows.
commandList->SetPipelineState(m_psoForwardSkinned.Get()); // set PSO
// Update vertex shader with current bone transforms
CONST std::vector<Matrix> transforms = m_assimpModel.GetShaderTransforms();
VSBonePassConstants vsBoneConstants{};
for (UINT i = 0; i < m_assimpModel.GetNumBones(); i++)
{
// We do not transpose bone matrices for HLSL because the original
// Assimp matrices are column-vector matrices.
vsBoneConstants.boneTransforms[i] = transforms[i];
//vsBoneConstants.boneTransforms[i] = transforms[i].Transpose();
//vsBoneConstants.boneTransforms[i] = Matrix::Identity;
}
GraphicsResource vsBoneCB = m_graphicsMemory->AllocateConstant(vsBoneConstants);
vsPerObjects.gWorld = m_assimp_world.Transpose(); // vertex shader per object constant
vsPerObjectCB = m_graphicsMemory->AllocateConstant(vsPerObjects);
commandList->SetGraphicsRootConstantBufferView(RootParameterIndex::VSBoneConstantBuffer, vsBoneCB.GpuAddress());
commandList->SetGraphicsRootConstantBufferView(RootParameterIndex::VSPerObjConstBuffer, vsPerObjectCB.GpuAddress());
//commandList->SetGraphicsRootDescriptorTable(RootParameterIndex::ObjectSRV, m_shaderTextureHeap->GetGpuHandle(ShaderTexDescriptors::SuzanneDiffuse));
commandList->SetGraphicsRootDescriptorTable(RootParameterIndex::ObjectSRV, m_shaderTextureHeap->GetGpuHandle(ShaderTexDescriptors::DefaultDiffuse));
for (UINT i = 0; i < m_assimpModel.GetMeshSize(); i++)
{
commandList->IASetVertexBuffers(0u, 1u, &m_assimpModel.meshEntries[i].GetVertexBufferView());
commandList->IASetIndexBuffer(&m_assimpModel.meshEntries[i].GetIndexBufferView());
commandList->IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
commandList->DrawIndexedInstanced(m_assimpModel.meshEntries[i].GetIndexCount(), 1u, 0u, 0u, 0u);
}
Please note I am using the Graphics Resource memory management helper object found in the DirectXTK12 library in the code above. Finally, here's the skinning vertex shader I'm using.
// Luna (2016) lighting model adapted from Moller
#define MAX_BONES 4
// vertex shader constant data that varies per object
cbuffer cbVSPerObject : register(b3)
{
float4x4 gWorld;
//float4x4 gTexTransform;
}
// vertex shader constant data that varies per frame
cbuffer cbVSPerFrame : register(b5)
{
float4x4 gViewProj;
float4x4 gShadowTransform;
}
// bone matrix constant data that varies per object
cbuffer cbVSBonesPerObject : register(b9)
{
float4x4 gBoneTransforms[MAX_BONES];
}
struct VertexIn
{
float3 posL : SV_POSITION;
float3 normalL : NORMAL;
float2 texCoord : TEXCOORD0;
float3 tangentU : TANGENT;
float4 boneWeights : BONEWEIGHT;
uint4 boneIndices : BONEINDEX;
};
struct VertexOut
{
float4 posH : SV_POSITION;
//float3 posW : POSITION;
float4 shadowPosH : POSITION0;
float3 posW : POSITION1;
float3 normalW : NORMAL;
float2 texCoord : TEXCOORD0;
float3 tangentW : TANGENT;
};
VertexOut VS_main(VertexIn vin)
{
VertexOut vout = (VertexOut)0.f;
// Perform vertex skinning.
// Ignore BoneWeights.w and instead calculate the last weight value
// to ensure all bone weights sum to unity.
float4 weights = vin.boneWeights;
//weights.w = 1.f - dot(weights.xyz, float3(1.f, 1.f, 1.f));
//float4 weights = { 0.f, 0.f, 0.f, 0.f };
//weights.x = vin.boneWeights.x;
//weights.y = vin.boneWeights.y;
//weights.z = vin.boneWeights.z;
weights.w = 1.f - (weights.x + weights.y + weights.z);
float4 localPos = float4(vin.posL, 1.f);
float3 localNrm = vin.normalL;
float3 localTan = vin.tangentU;
float3 objPos = mul(localPos, (float4x3)gBoneTransforms[vin.boneIndices.x]).xyz * weights.x;
objPos += mul(localPos, (float4x3)gBoneTransforms[vin.boneIndices.y]).xyz * weights.y;
objPos += mul(localPos, (float4x3)gBoneTransforms[vin.boneIndices.z]).xyz * weights.z;
objPos += mul(localPos, (float4x3)gBoneTransforms[vin.boneIndices.w]).xyz * weights.w;
float3 objNrm = mul(localNrm, (float3x3)gBoneTransforms[vin.boneIndices.x]) * weights.x;
objNrm += mul(localNrm, (float3x3)gBoneTransforms[vin.boneIndices.y]) * weights.y;
objNrm += mul(localNrm, (float3x3)gBoneTransforms[vin.boneIndices.z]) * weights.z;
objNrm += mul(localNrm, (float3x3)gBoneTransforms[vin.boneIndices.w]) * weights.w;
float3 objTan = mul(localTan, (float3x3)gBoneTransforms[vin.boneIndices.x]) * weights.x;
objTan += mul(localTan, (float3x3)gBoneTransforms[vin.boneIndices.y]) * weights.y;
objTan += mul(localTan, (float3x3)gBoneTransforms[vin.boneIndices.z]) * weights.z;
objTan += mul(localTan, (float3x3)gBoneTransforms[vin.boneIndices.w]) * weights.w;
vin.posL = objPos;
vin.normalL = objNrm;
vin.tangentU.xyz = objTan;
//vin.posL = posL;
//vin.normalL = normalL;
//vin.tangentU.xyz = tangentL;
// End vertex skinning
// transform to world space
float4 posW = mul(float4(vin.posL, 1.f), gWorld);
vout.posW = posW.xyz;
// assumes nonuniform scaling, otherwise needs inverse-transpose of world matrix
vout.normalW = mul(vin.normalL, (float3x3)gWorld);
vout.tangentW = mul(vin.tangentU, (float3x3)gWorld);
// transform to homogenous clip space
vout.posH = mul(posW, gViewProj);
// pass texcoords to pixel shader
vout.texCoord = vin.texCoord;
//float4 texC = mul(float4(vin.TexC, 0.0f, 1.0f), gTexTransform);
//vout.TexC = mul(texC, gMatTransform).xy;
// generate projective tex-coords to project shadow map onto scene
vout.shadowPosH = mul(posW, gShadowTransform);
return vout;
}
Some last tests I tried before posting:
I tested the code with a Collada (DAE) model exported from Blender, only to observe the same distorted zigzagging in the Win32 desktop application.
I also confirmed the aiScene object for the loaded model returns an identity matrix for the global root transform (also verified in AssimpViewer).
I have stared at this code for about a week and am going out of my mind! Really hoping someone can spot what I have missed. If you need more code or info, please ask!
This seems to be a bug with the published code in the tutorials / documentation. It would be great if you could open an issue-report here: Assimp-Projectpage on GitHub .
It's taken almost another two weeks of pain, but I finally found the bug. It was in my own code, and it was self-inflicted. Before I show the solution, I should explain the further troubleshooting I did to get there.
After losing faith with Assimp (even though the AssimpViewer tool was animating my model correctly), I turned to the FBX SDK. The FBX ViewScene command line utility tool that's available as part of the SDK was also showing and animating my model properly, so I had hope...
So after a few days reviewing the FBX SDK tutorials, and taking another week to write an FBX importer for my Windows desktop game, I loaded my model and... saw exactly the same zig-zag animation anomaly as the version loaded by Assimp!
This frustrating outcome meant I could at least eliminate Assimp and the FBX SDK as the source of the problem, and focus again on the vertex shader. The shader I'm using for vertex skinning was adopted from the 'Character Animation' chapter of Frank Luna's text. It was identical in every way, which led me to recheck the C++ vertex structure declared on the application side...
Here's the C++ vertex declaration for skinned vertices:
struct Vertex
{
// added constructors
Vertex() = default;
Vertex(FLOAT x, FLOAT y, FLOAT z,
FLOAT nx, FLOAT ny, FLOAT nz,
FLOAT u, FLOAT v,
FLOAT tx, FLOAT ty, FLOAT tz) :
Pos(x, y, z),
Normal(nx, ny, nz),
TexC(u, v),
Tangent(tx, ty, tz) {}
Vertex(DirectX::SimpleMath::Vector3 pos,
DirectX::SimpleMath::Vector3 normal,
DirectX::SimpleMath::Vector2 texC,
DirectX::SimpleMath::Vector3 tangent) :
Pos(pos), Normal(normal), TexC(texC), Tangent(tangent) {}
DirectX::SimpleMath::Vector3 Pos;
DirectX::SimpleMath::Vector3 Normal;
DirectX::SimpleMath::Vector2 TexC;
DirectX::SimpleMath::Vector3 Tangent;
FLOAT BoneWeights[4];
BYTE BoneIndices[4];
//UINT BoneIndices[4]; <--- YOU HAVE CAUSED ME A MONTH OF PAIN
};
Quite early on, being confused by Luna's use of BYTE to store the array of bone indices, I changed this structure element to UINT, figuring this still matched the input declaration shown here:
static CONST D3D12_INPUT_ELEMENT_DESC inputElementDescSkinned[] =
{
{ "SV_POSITION", 0u, DXGI_FORMAT_R32G32B32_FLOAT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
{ "NORMAL", 0u, DXGI_FORMAT_R32G32B32_FLOAT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
{ "TEXCOORD", 0u, DXGI_FORMAT_R32G32_FLOAT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
{ "TANGENT", 0u, DXGI_FORMAT_R32G32B32_FLOAT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
//{ "BINORMAL", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0 },
{ "BONEWEIGHT", 0u, DXGI_FORMAT_R32G32B32A32_FLOAT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
{ "BONEINDEX", 0u, DXGI_FORMAT_R8G8B8A8_UINT, 0u, D3D12_APPEND_ALIGNED_ELEMENT, D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0u },
};
Here was the bug. By declaring UINT in the vertex structure for bone indices, four bytes were being assigned to store each bone index. But in the vertex input declaration, the DXGI_FORMAT_R8G8B8A8_UINT format specified for the "BONEINDEX" was assigning one byte per index. I suspect this data type and format size mismatch was resulting in only one valid bone index being able to fit in the BONEINDEX element, and so only one index value was passed to the vertex shader each frame, instead of the whole array of four indices for correct bone transform lookups.
So now I've learned... the hard way... why Luna had declared an array of BYTE for bone indices in the original C++ vertex structure.
I hope this experience will be of value to someone else, and always be careful changing code from your original learning sources.
How do I get a distortion like what a fisheye lens does to a view with a SCNCamera in Scene Kit?
Something like this kind of "bowing" of the imagery:
// as Rickster pointed out, this kind of distortion is known as "Barrel Distortion".
From the docs, this is the part that got me intrigued by the possibility of doing this kind of distortion with the camera:
If you compute your own projection transform matrix, you can use this
method to set it directly, overriding the transformation synthesized
from the camera’s geometric properties.
Unfortunately I know nothing about the powers and possibilities of computing ones own projection transform matrix. I'm hoping it's possible to do this kind of distortion via it... but dunno, hence the question.
Any other means via a camera is ideal. Too. Wanting to avoid post processing trickery and get the more "organic" look of this kind of distortion when the camera rotates and moves through the scene.
See any skateboarding video for how this looks in real life.
What you are looking for is called Barrel Distrortion.
There are a few ways of doing this, all of them using GLSL shaders.
You can either use classic OpenGL code, such as this example for the Occulus Rift (you will need to change the shader a little bit), or my personal favorite: SCNTechnique.
Create a technique containing a Barrel Fragment Shader (.fsh), and set its draw parameter to DRAW_QUAD. Then, simply apply the technique to your camera.
You can find an example of Barrel Distortion shader here : http://www.geeks3d.com/20140213/glsl-shader-library-fish-eye-and-dome-and-barrel-distortion-post-processing-filters/2/
EDIT: here's a sample code:
barrel.json (this should go in your scnassets bundle)
{
"passes" : {
"barrel" : {
"outputs" : {
"color" : "COLOR"
},
"inputs" : {
"colorSampler" : "COLOR",
"noiseSampler" : "noiseSymbol",
"a_position" : "a_position-symbol"
},
"program" : "art.scnassets/barrel",
"draw" : "DRAW_QUAD"
}
},
"sequence" : [
"barrel"
],
"symbols" : {
"a_position-symbol" : {
"semantic" : "vertex"
},
"noiseSymbol" : {
"image" : "noise.png",
"type" : "sampler2D"
},
"barrelPower" : {
"type" : "float"
}
}
}
barrel.vsh
attribute vec4 a_position;
varying vec2 uv;
void main() {
gl_Position = a_position;
uv = a_position.xy;
}
barrel.fsh
// Adapted from :
// http://www.geeks3d.com/20140213/glsl-shader-library-fish-eye-and-dome-and-barrel-distortion-post-processing-filters/2/
uniform sampler2D colorSampler;
const float PI = 3.1415926535;
uniform float barrelPower;
varying vec2 uv;
vec2 Distort(vec2 p)
{
float theta = atan(p.y, p.x);
float radius = length(p);
radius = pow(radius, barrelPower);
p.x = radius * cos(theta);
p.y = radius * sin(theta);
return 0.5 * (p + 1.0);
}
void main() {
vec2 rg = 2.0 * uv.xy - 1.0;
vec2 uv2;
float d = length(xy);
if (d < 1.0){
uv2 = Distort(xy);
}else{
uv2 = uv.xy;
}
gl_FragColor = texture2D(colorSampler, uv2);
}
something.m
NSURL *url = [[NSBundle mainBundle] URLForResource:#"art.scnassets/barrel" withExtension:#"json"];
NSDictionary *tecDic = [NSJSONSerialization JSONObjectWithData:[NSData dataWithContentsOfURL: url] options:nil error:nil];
SCNTechnique* technique = [SCNTechnique techniqueWithDictionary:tecDic];
[technique setValue: [NSNumber numberWithFloat:0.5] forKey:#"barrelPower"];
cameraNode.technique = technique;
So, I've messed around with three.js, works out great. The only thing I can't figure out, is how to make a camera with a real fisheye effect.
How is that possible? camera.setLens()?
The fish eye effect can be achieved using Giliam de Carpentier's shader for lens distortion.
Shader code:
function getDistortionShaderDefinition()
{
return {
uniforms: {
"tDiffuse": { type: "t", value: null },
"strength": { type: "f", value: 0 },
"height": { type: "f", value: 1 },
"aspectRatio": { type: "f", value: 1 },
"cylindricalRatio": { type: "f", value: 1 }
},
vertexShader: [
"uniform float strength;", // s: 0 = perspective, 1 = stereographic
"uniform float height;", // h: tan(verticalFOVInRadians / 2)
"uniform float aspectRatio;", // a: screenWidth / screenHeight
"uniform float cylindricalRatio;", // c: cylindrical distortion ratio. 1 = spherical
"varying vec3 vUV;", // output to interpolate over screen
"varying vec2 vUVDot;", // output to interpolate over screen
"void main() {",
"gl_Position = projectionMatrix * (modelViewMatrix * vec4(position, 1.0));",
"float scaledHeight = strength * height;",
"float cylAspectRatio = aspectRatio * cylindricalRatio;",
"float aspectDiagSq = aspectRatio * aspectRatio + 1.0;",
"float diagSq = scaledHeight * scaledHeight * aspectDiagSq;",
"vec2 signedUV = (2.0 * uv + vec2(-1.0, -1.0));",
"float z = 0.5 * sqrt(diagSq + 1.0) + 0.5;",
"float ny = (z - 1.0) / (cylAspectRatio * cylAspectRatio + 1.0);",
"vUVDot = sqrt(ny) * vec2(cylAspectRatio, 1.0) * signedUV;",
"vUV = vec3(0.5, 0.5, 1.0) * z + vec3(-0.5, -0.5, 0.0);",
"vUV.xy += uv;",
"}"
].join("\n"),
fragmentShader: [
"uniform sampler2D tDiffuse;", // sampler of rendered scene?s render target
"varying vec3 vUV;", // interpolated vertex output data
"varying vec2 vUVDot;", // interpolated vertex output data
"void main() {",
"vec3 uv = dot(vUVDot, vUVDot) * vec3(-0.5, -0.5, -1.0) + vUV;",
"gl_FragColor = texture2DProj(tDiffuse, uv);",
"}"
].join("\n")
};
}
One way to setup the effect using effect composer (assuming scene and renderer have been been created):
// Create camera
camera = new THREE.PerspectiveCamera( 100, window.innerWidth / window.innerHeight, 1, 1000000 );
camera.position.z = 800;
// Create effect composer
composer = new THREE.EffectComposer( renderer );
composer.addPass( new THREE.RenderPass( scene, camera ) );
// Add distortion effect to effect composer
var effect = new THREE.ShaderPass( getDistortionShaderDefinition() );
composer.addPass( effect );
effect.renderToScreen = true;
// Setup distortion effect
var horizontalFOV = 140;
var strength = 0.5;
var cylindricalRatio = 2;
var height = Math.tan(THREE.Math.degToRad(horizontalFOV) / 2) / camera.aspect;
camera.fov = Math.atan(height) * 2 * 180 / 3.1415926535;
camera.updateProjectionMatrix();
effect.uniforms[ "strength" ].value = strength;
effect.uniforms[ "height" ].value = height;
effect.uniforms[ "aspectRatio" ].value = camera.aspect;
effect.uniforms[ "cylindricalRatio" ].value = cylindricalRatio;
Following script are needed and they can be found for example from three.js GitHub page:
<script src="examples/js/postprocessing/EffectComposer.js"></script>
<script src="examples/js/postprocessing/RenderPass.js"></script>
<script src="examples/js/postprocessing/MaskPass.js"></script>
<script src="examples/js/postprocessing/ShaderPass.js"></script>
<script src="examples/js/shaders/CopyShader.js"></script>
Link to Giliam's example: http://www.decarpentier.nl/downloads/lensdistortion-webgl/lensdistortion-webgl.html
Link to Giliam's article about lens distortion: http://www.decarpentier.nl/lens-distortion
Image of my test where lens distortion effect is used:
Put a camera inside a reflective sphere. Make sure the sphere is double sided. Parent the camera and sphere together if you want to move it around your scene. Works like a charm:
http://tileableart.com/code/NOCosmos/test.html
borrowed from:
http://mrdoob.github.io/three.js/examples/webgl_materials_cubemap_dynamic2.html
cubeCamera = new THREE.CubeCamera( 1, 3000, 1024);
cubeCamera.renderTarget.minFilter = THREE.LinearMipMapLinearFilter;
scene.add( cubeCamera );
camParent.add(cubeCamera);
var material = new THREE.MeshBasicMaterial( { envMap: cubeCamera.renderTarget } );
material.side = THREE.DoubleSide;
sphere = new THREE.Mesh( new THREE.SphereGeometry( 2, 60, 30 ), material );
It's possible to get the fisheye effect with a high Field of View.
var fishCamera = new THREE.PerspectiveCamera( 110, window.innerWidth / window.innerHeight, 1, 1100 );
var normalCamera = new THREE.PerspectiveCamera( 50, window.innerWidth / window.innerHeight, 1, 1100 );
or set
camera.fov = 110
camera.updateProjectionMatrix();
Live Example here:
http://mrdoob.github.com/three.js/examples/canvas_geometry_panorama_fisheye.html
One way is to set a large field of view on the camera:
new THREE.PerspectiveCamera(140, ... )
This will not technically be a fisheye effect, but it may be the effect you're looking for.
In a real camera lens, getting a large field of view without distorsion would likely make the lens pretty expensive, but in computer graphics, it's the easy way.
A real fisheye lens distorts the image so that straight line become curved, like in this image:
If you want to create an actual fisheye effect with this kind of distorsion, you would have to modify the geometry, as in Three.js's fisheye example. In that example, the geometry is actually modified beforehand, but for a more advanced scene, you'd want to use a vertex shader to update the vertices on the fly.
A wide angle lens generally have a very low focus length.
To achieve an extreme wide angle we need to reduce focus length.
Note that fish eye lens is an extreme wide angle lens.
To reduce focus length(or to achieve extreme wide angles), one can just increase FOV (field of view), as FOV is inversely proportional to focus length.
example:
var camera_1 = new THREE.PerspectiveCamera( 45, width / height, 1, 1000 );
var camera_2 = new THREE.PerspectiveCamera( 80, width / height, 1, 1000 );
Here camera_2 is a wider angle setup.
Note
To achieve desired effect, one may have to adjust camera position.
I am trying to rotate the camera around to X-axis of the scene.
At this point my code is like this:
rotation += 0.05;
camera.position.y = Math.sin(rotation) * 500;
camera.position.z = Math.cos(rotation) * 500;
This makes the camera move around but during the rotation something weird happens and either the camera flips, or it skips some part of the imaginary circle it's following.
You have only provided a snippet of code, so I have to make some assumptions about what you are doing.
This code:
rotation += 0.05;
camera.position.x = 0;
camera.position.y = Math.sin(rotation) * 500;
camera.position.z = Math.cos(rotation) * 500;
camera.lookAt( scene.position ); // the origin
will cause the "flipping" you refer to because the camera is trying to remain "right side up", and it will quickly change orientation as it passes over the "north pole."
If you offset the camera's x-coordinate like so,
camera.position.x = 200;
the camera behavior will appear more natural to you.
Three.js tries to keep the camera facing up. When you pass 0 along the z-axis, it'll "fix" the camera's rotation. You can just check and reset the camera's angle manually.
camera.lookAt( scene.position ); // the origin
if (camera.position.z < 0) {
camera.rotation.z = 0;
}
I'm sure this is not the best solution, but if anyone else runs across this question while playing with three.js (like I just did), it'll give one step further.
This works for me, I hope it helps.
Rotating around X-Axis:
var x_axis = new THREE.Vector3( 1, 0, 0 );
var quaternion = new THREE.Quaternion;
camera.position.applyQuaternion(quaternion.setFromAxisAngle(x_axis, rotation_speed));
camera.up.applyQuaternion(quaternion.setFromAxisAngle(x_axis, rotation_speed));
Rotating around Y-Axis:
var y_axis = new THREE.Vector3( 0, 1, 0 );
camera.position.applyQuaternion(quaternion.setFromAxisAngle(y_axis, angle));
Rotating around Z-Axis:
var z_axis = new THREE.Vector3( 0, 0, 1 );
camera.up.applyQuaternion(quaternion.setFromAxisAngle(z_axis, angle));
I wanted to move my camera to a new location while having the camera look at a particular object, and this is what I came up with [make sure to load tween.js]:
/**
* Helper to move camera
* #param loc Vec3 - where to move the camera; has x, y, z attrs
* #param lookAt Vec3 - where the camera should look; has x, y, z attrs
* #param duration int - duration of transition in ms
**/
function flyTo(loc, lookAt, duration) {
// Use initial camera quaternion as the slerp starting point
var startQuaternion = camera.quaternion.clone();
// Use dummy camera focused on target as the slerp ending point
var dummyCamera = camera.clone();
dummyCamera.position.set(loc.x, loc.y, loc.z);
// set the dummy camera quaternion
var rotObjectMatrix = new THREE.Matrix4();
rotObjectMatrix.makeRotationFromQuaternion(startQuaternion);
dummyCamera.quaternion.setFromRotationMatrix(rotObjectMatrix);
dummyCamera.up.set(camera)
console.log(camera.quaternion, dummyCamera.quaternion);
// create dummy controls to avoid mutating main controls
var dummyControls = new THREE.TrackballControls(dummyCamera);
dummyControls.target.set(loc.x, loc.y, loc.z);
dummyControls.update();
// Animate between the start and end quaternions
new TWEEN.Tween(camera.position)
.to(loc, duration)
.onUpdate(function(timestamp) {
// Slerp the camera quaternion for smooth transition.
// `timestamp` is the eased time value from the tween.
THREE.Quaternion.slerp(startQuaternion, dummyCamera.quaternion, camera.quaternion, timestamp);
camera.lookAt(lookAt);
})
.onComplete(function() {
controls.target = new THREE.Vector3(scene.children[1].position-0.001);
camera.lookAt(lookAt);
}).start();
}
Example usage:
var pos = {
x: -4.3,
y: 1.7,
z: 7.3,
};
var lookAt = scene.children[1].position;
flyTo(pos, lookAt, 60000);
Then in your update()/render() function, call TWEEN.update();
Full example
I wondering if there is a way to optimize this vertex shader.
This vertex shader projects (in the light direction) a vertex to the far plane if it is in the shadow.
The aim of this shader is to create a shadow volume object that enclose the shadow of the object itself.
void main(void) {
vec3 lightDir = (gl_ModelViewMatrix * gl_Vertex
- gl_LightSource[0].position).xyz;
// if the vertex is lit
if ( dot(lightDir, gl_NormalMatrix * gl_Normal) < 0.01 ) {
// don't move it
gl_Position = ftransform();
} else {
// move it far, is the light direction
vec4 fin = gl_ProjectionMatrix * (
gl_ModelViewMatrix * gl_Vertex
+ vec4(normalize(lightDir) * 100000.0, 0.0)
);
if ( fin.z > fin.w ) // if fin is behind the far plane
fin.z = fin.w; // move to the far plane (needed for z-fail algo.)
gl_Position = fin;
}
}
If you don't want do touch your principal algorithm (as Michael Daum suggested in his comment) you could replace some parts of your code:
uniform mat4 infiniteProjectionMatrix;
if(...) {
...
} else {
gl_Position = infiniteProjectionMatrix * vec4(lightDir, 0.0);
}
where infiniteProjectionMatrix is a customized projection matrix where the far plane has been set to infinity (see http://www.terathon.com/gdc07_lengyel.pdf on slide 7) and looks something like:
* 0 0 0
0 * 0 0
0 0 -1 *
0 0 -1 0
Since you are projecting to infinity you don't need the "100000.0" scaling factor and the "gl_ModelViewMatrix * gl_Vertex" offset can be neglected (compared to the infinite length of the lightDir vector).