c++ opencv pixel value operation - auxiliary document corner judgment

There is a set of corner coordinates of corners of vector < point > type (obtained by fitting the document border and calculating the intersection)
There is also a binary document contour map of Mat type (the document image is obtained by Canny contour extraction + findContours() contour detection function)
Since the corner points are obtained by calculating the intersection point of the straight line after Hough transform, some points are not on the real document corner points. It is necessary to judge which corner points find the right position and which do not.

For this reason, it is temporarily considered to try to calculate by obtaining the pixel value near the corner in the document outline.

The document outline (binarization) is as follows

First, refer to the pixel access method of Mat type image

The code is as follows:

Write the way to obtain 8 * 8 pixels near the corner

//vector<Point> corners;// Corner
//Mat Outlineimg;// Binary graph
Mat PointMat(Size(8, 8), Outlineimg.type());//The 8 * 8 matrix is used to store images near corners

int BeginY = corners[num].y - 4, BeginX = corners[num].x - 4;//Take point start position
int EndY = corners[num].y + 4 , EndX = corners[num].x + 4;//Take point termination bit

for (int i =  BeginY ; i < EndY; i++)
{
	for (int j = BeginX; j < EndX; j++)
	{
		PointMat.at<uchar>(i - BeginY, j - BeginX) = Outlineimg.at<uchar>(i, j);
	}
}
imshow("show PointMat", PointMat);//Print images near corners

The image near the corner displayed is as follows:

Existing problem: if the corner is adjacent to the image boundary, the access to the image pixels may cross the boundary, resulting in program crash

Solution: the offset required to obtain the 8 * 8 pixel image near the corners can be calculated according to the current coordinates of corners and the size of Mat image

The code is as follows

vector<Point> DocScan::CalPixValue(Mat& Outlineimg, vector<Point>& corners)//Calculate the pixel value of corners
{
	Mat PointMat(Size(8, 8), Outlineimg.type());
	vector<Point>  Markcorners;
	for (char num=0;num<corners.size();num++)
	{
		int dx, dy;//Offset
        //---------------Offset calculation-------------
		if (corners[num].x - 0 < 4) {
			dx = 4 - corners[num].x;
		}else if (Outlineimg.cols - corners[num].x < 4) {
			dx = Outlineimg.cols - corners[num].x - 4;
		}else
			dx = 0;

		if (corners[num].y - 0 < 4) {
			dy = 4 - corners[num].y;
		}else if (Outlineimg.rows - corners[num].y < 4) {
			dy = Outlineimg.rows - corners[num].y - 4;
		}else
			dy = 0;

		int BeginY = corners[num].y - 4 + dy, BeginX = corners[num].x - 4 + dx;//Take point start position
		int EndY = corners[num].y + 4 + dy, EndX = corners[num].x + 4 + dx;//Take point termination bit
		//----------------Access pixels near corners-----------------
		int whitenum = 0;
		for (int i = BeginY; i < EndY; i++)
		{
			for (int j = BeginX; j < EndX; j++)
			{
				PointMat.at<uchar>(i - BeginY, j - BeginX) = Outlineimg.at<uchar>(i, j);
				if (Outlineimg.at<uchar>(i, j) == 255)//White pixels found
				{
					++whitenum;
				}
			}
		}
		if (whitenum >= 9)//If the number of white pixels > = 9, it may be corner points (not too accurate, the specific parameters can be modified according to individual needs)
		{
			cout << "got edge,whitenum= " << whitenum << endl;
			Markcorners.push_back(corners[num]);//Place qualified corners into Markcorners
		}
	}
	
	imshow("show PointMat", PointMat);//Print images near corners
	return Markcorners;//Returns the corner that meets the criteria
}

Get the Markcorners returned by the function, and then mark them with a yellow circle on the figure
As shown in the figure below, yellow is the possible corner and red is the wrong corner (the purple red point is the endpoint of the line segment fitted by Hough transform, please ignore). It can be seen that the correct corner can be distinguished in general.

However, corners in this case cannot be detected (number of pixels = 7)

Then, it is impossible to meet the demand simply by using pixel calculation. You can consider using the method of finding lines by Hough transform to calculate the angle between the two lines to judge whether they are corners.