Hessian Matrix of the image

Question

I am wondering what information does an Hessian Matrix of an image provides? Does it provide the information of the stable points? What is Hessian matrix used for?

Solution

Hessian matrix describes the 2nd order local image intensity variations around the selected voxel. For the obtained Hessian matrix, eigenvector decomposition extracts an orthonormal coordinate system that is aligned with the second order structure of the image. Having the eigenvalues and knowing the (assumed) model of the structure to be detected and the resulting theoretical behavior of the eigenvalues, the decision can be made if the analyzed voxel belongs to the structure being searched.

The figure below illustrates the correspondence between eigenvalues of the hessian operation on the image and the local features (corner, edge, or flat region).

The Hessian operator is also widely used in 3D images, and it can reflect more local features:

It is widely used in vessel detection in medical images. For more details, please see M.Rudzki et al's Vessel Detection Method Based on Eigenvalues of the Hessian Matrix and its Applicability to Airway Tree Segmentation

Comments

[Royi]

@lennon310, Where did you get those figures from? They are great.

[Soltius]

Could you please elaborate on "eigenvector decomposition extracts an orthonormal coordinate system that is aligned with the second order structure of the image" ? Why is this ? I get that an eigenvector decomposition extracts an orthonomal coordinate system which is strongly related to the second order structure (since we're working with the Hessian), but how come these eigenvectors align with the structures we're looking for (eg in a tube, one eigenvector follows the tube while the two others are orthogonal) ?

[Yuri Feldman]

Figure is good but slightly misleading: eigenvalues have nothing to do with the edge direction (eigenvectors do, but they are part of matrix R above, and their order is not shown). Edge direction w.r.t. image axes can most readily be extracted either by diagonal terms of the Hessian, or the gradient.

[Yves Daoust]

This answer is wrong. It describes the Structure Tensor, not the Hessian.

[Mario Viti]

This answer is WRONG. The matrix shown is not the Hessian but the Structure Tensor as @YvesDaoust pointed out. It comes from the work of Harris for corner detection semanticscholar.org/paper/…