FindChessboardCorners cannot detect chessboard on very large images by long focal length lens

Question

I can use FindChessboardCorners functions for images that less than 15 Mega pixel such like 2k x 1.5k. however when I use it on the image from DSLR, the resolution at 3700x5300, it doesn't work.

I tried to use resize() to reduce the image size directly, then it works.

Obviously there's some hard coded or bug in the OpenCV source code.

Could you help me to figure it out, or point me to a patch for this ?

I found someone posted a similar issue in 2006, here, so it looks like the problem still remains.

The code I used is like

found = findChessboardCorners( viewGray, boardSize, ptvec,
                                CV_CALIB_CB_ADAPTIVE_THRESH + CV_CALIB_CB_FILTER_QUADS + CV_CALIB_CB_NORMALIZE_IMAGE + CV_CALIB_CB_FAST_CHECK);

Update

Just here to clarify. I think the algorithm works on large image resolution, but it fails when the chessboard occupy larger proportion of the image. For example, when I use a 50mm fixed lens on the same camera position, FindChessboardCorners never fails. After I change it to 100mm fixed lens, the function starts to stop detecting the pattern. I think it relates to the proportion or the focal length.

The image below is the 100mm lens result.

Update 2

I added a sharpen filter to the large image, and it starts to fix the problem.

Firstly I used

//do a sharpen filter for the large resolution image
if (viewGray.cols > 1500)
{
  Mat temp ;
  GaussianBlur(viewGray,temp, Size(0,0), 105) ; //hardcoded filter size, to be tested on 50 mm lens
  addWeighted(viewGray, 1.8, temp, -0.8,0,viewGray) ; //hardcoded weight, to be tested.
//imwrite("test"+ imageList[k][i], viewGray) ;

}

found = findChessboardCorners( viewGray, boardSize, ptvec,
CV_CALIB_CB_ADAPTIVE_THRESH + CV_CALIB_CB_FILTER_QUADS + CV_CALIB_CB_NORMALIZE_IMAGE + CV_CALIB_CB_FAST_CHECK);

Uploaded the image:

A jpg image at original resolution 3744 x 5616, if this site force convert, then make sure you are using at the correct resolution.

Solution 1

A few points.

1. Down-sizing, as you noticed, helps the detector. That is because the corner-detection filters used in OpenCV to find the corners have fixed size, and that size of convolution mask may be too small to detect your corners - the full-size image may actually look "smooth" at that scale, particularly where it is slightly blurry. However, by downscaling you throw away some corner location accuracy.
2. For the same reason, sharpening helps too. However, it also goes against accuracy, because it adds bias to the subpixel positions of the corners - even in the ideal case with no noise. To convince yourself that this is the case, consider the 1D analogue: the intensity of the image around a corner (in 1D, a sharp black-white transition) looks ideally like a sigmoid curve (a ramp with smooth corners), and you want to find the location of its inflection point. Sharpening makes the curve steeper, which in general will move that point's location. Things get worse when you take into account that sharpening generally amplifies noise.
3. The likely correct way to proceed is to start at a lower resolution (i.e. downsizing), then scale up the positions of the corners thus found, and use them as the initial estimates for a run of cvFindCornersSubpix at full resolution.

Solution 2

If you have access to the OpenCV source and can rebuild it, then maybe you can debug the behavior of cvFindChessboardCorners.

You have to #define DEBUG_CHESSBOARD and then you will have some helps in understanding the algorithm.

I think OpenCV 2.4 has this capability (see for example https://github.com/Itseez/opencv/blob/2.4/modules/calib3d/src/calibinit.cpp).

Furthermore, even if it doesn't seem to apply to your case, OpenCV doc gives a requirement for the calibration target:

Note: The function requires white space (like a square-thick border, the wider the better) around the board to make the detection more robust in various environments. Otherwise, if there is no border and the background is dark, the outer black squares cannot be segmented properly and so the square grouping and ordering algorithm fails.

http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#findchessboardcorners

The chessboard in the question has an even number of inner corner both for rows (6 corners) and columns (8 corners) while a reference OpenCV chessboard has an even/odd number of corners, i.e. it is 9x6, I do not know wether this can be a problem.