Why transform normals with the transpose of the inverse of the modelview matrix?

Question

I am working on some shaders, and I need to transform normals.

I read in few tutorials the way you transform normals is you multiply them with the transpose of the inverse of the modelview matrix. But I can't find explanation of why is that so, and what is the logic behind that?

Solution 1

Take a look at this tutorial:

https://paroj.github.io/gltut/Illumination/Tut09%20Normal%20Transformation.html

You can imagine that when the surface of a sphere stretches (so the sphere is scaled along one axis or something similar) the normals of that surface will all 'bend' towards each other. It turns out you need to invert the scale applied to the normals to achieve this. This is the same as transforming with the Inverse Transpose Matrix. The link above shows how to derive the inverse transpose matrix from this.

Also note that when the scale is uniform, you can simply pass the original matrix as normal matrix. Imagine the same sphere being scaled uniformly along all axes, the surface will not stretch or bend, nor will the normals.

Solution 2

It flows from the definition of a normal.

Suppose you have the normal, N, and a vector, V, a tangent vector at the same position on the object as the normal. Then by definition N·V = 0.

Tangent vectors run in the same direction as the surface of an object. So if your surface is planar then the tangent is the difference between two identifiable points on the object. So if V = Q - R where Q and R are points on the surface then if you transform the object by B:

V' = BQ - BR
   = B(Q - R)
   = BV

The same logic applies for non-planar surfaces by considering limits.

In this case suppose you intend to transform the model by the matrix B. So B will be applied to the geometry. Then to figure out what to do to the normals you need to solve for the matrix, A so that:

(AN)·(BV) = 0

Turning that into a row versus column thing to eliminate the explicit dot product:

[tranpose(AN)](BV) = 0

Pull the transpose outside, eliminate the brackets:

transpose(N)*transpose(A)*B*V = 0

So that's "the transpose of the normal" [product with] "the transpose of the known transformation matrix" [product with] "the transformation we're solving for" [product with] "the vector on the surface of the model" = 0

But we started by stating that transpose(N)*V = 0, since that's the same as saying that N·V = 0. So to satisfy our constraints we need the middle part of the expression — transpose(A)*B — to go away.

Hence we can conclude that:

 transpose(A)*B = identity
 => transpose(A) = identity*inverse(B)
 => transpose(A) = inverse(B)
 => A = transpose(inverse(B))

Solution 3

My favorite proof is below where N is the normal and V is a tangent vector. Since they are perpendicular their dot product is zero. M is any 3x3 invertible transformation (M^-1 * M = I). N' and V' are the vectors transformed by M.

To get some intuition, consider the shear transformation below.

Note that this does not apply to tangent vectors.

Solution 4

If the model matrix is made of translation, rotation and scale, you don't need to do inverse transpose to calculate normal matrix. Simply divide the normal by squared scale and multiply by model matrix and we are done. You can extend that to any matrix with perpendicular axes, just calculate squared scale for each axes of the matrix you are using instead.

I wrote the details in my blog: https://lxjk.github.io/2017/10/01/Stop-Using-Normal-Matrix.html

View more solutions

Comments

[Tommy]

@NicolBolas V is, as I said, a vector on the surface, not a location. So, yes, N.V is the dot product. I could equally have phrased it "a vector along the surface". As you point out, alternative interpretations are immediately ruled out through the formal mention of the dot product.

[Tommy]

(though technically "a vector along the surface" would be less accurate since it presupposes the surface is flat; I'll edit to explicitly mention the word 'tangent')

[Tommy]

@NicolBolas on the contrary, "You intend to transform the model by the matrix A" seems quite clear to me. Your question makes the implicit point that I'd verbally described A and B the wrong way around though.

[Nicol Bolas]

One more question. If B is the model transform matrix, how do we know that the tangent V needs to be multiplied by it and not the normal N?

[Tommy]

@NicolBolas good point. I mean, you and I both know it's because one definition of the tangent is the limit between the difference between two points that are h units apart as h goes towards zero but it's not a complete answer unless I edit that in there.

[Tommy]

Modelview matrices may also scale, including in different amounts in different directions, not necessarily oriented with axes. i.e. the top left 3x3 is not necessarily orthonormal. If you limit yourself to special orthogonal matrices then your solution is sufficient.

[marcianx]

There's a typo on the last one. It should have said $(M^-T N)^T MV = 0$ (transpose is missing around the produce of the first two matrices).

[wcochran]

For any orthonormal transformation M (e.g., rotation) the inverse-tranpose M^(-T) = M. Uniform scaling will only change the magnitude of the normal, not the direction. Translation as the last operation doesn't effect the upper 3x3 part so its moot. Everything else you should be using the inverse-transpose -- why not use the most general solution that works in all cases?

[eric]

Surely you don't need to do anything special for uniform scale. For non-uniform scale this method is FASTER then sending over a normal matrix to shader or doing inverse transpose in shader. For model matrix this is a common case. If the model matrix does not meet the condition then unfortunately we have to fall back to inverse transpose.

[wcochran]

To compute one 3x3 inverse transpose to transform perhaps thousands of vertex normals is inconsequential. A non-inform scale following a rotation will not preserve normal directions.

[eric]

That's 2 instructions (mul, div) versus 8 instructions (3 cross, 1 dot, 3 mul, 1 div), more than 4 times faster. (Both method need a matrix multiply at the end, so ignore that part) How important this is depends on your application. Correctness is proved in my blog post, read it if you are interested, it is exactly the same result as 3x3 inverse transpose.

[wcochran]

Transforming normals with a 3x3 matrix is one matrix-vector multiply (3 dot products in GLSL) -- cost of 3 multiplies (at most) on a SIMD GPU. Most of the cost comes when you renormalize the resulting vector after the multiply - GPU's are particularly fast at that (dot, rsqrt, dot) -- rsqrt is probably the most expensive, but there are tricks for that and will be done in silicon (h14s.p5r.org/2012/09/0x5f3759df.html)

[wcochran]

I liked your blog, but are you computing the inverse transform for every vertex? I usually don't do that in the vertex shader -- I pre-compute the normal matrix (only needed if model or view matrix change) and pass it to the shader.

[eric]

I used to do that as well, calculate normal matrix per geometry on CPU and send it over to shader. Then I find out this way, which is cheap enough that I can afford it to be calculated in vertex shader. It will save me from sending an extra matrix to shader per geometry, which I think is a win :) But yeah as you said it won't work for general matrix, so it is trade-off decision.