In slam, optical flow tracking judges the dynamics of an object in the image, which mainly includes three functions:

1. goodFeaturesToTrack function

Function: extract corners at pixel level in the input image, and support harris corners and Shi Tomasi corners.

Function prototype:

void goodFeaturesToTrack( InputArray image, OutputArray corners,  
                          int maxCorners, double qualityLevel, double minDistance,  
						  InputArray mask=noArray(), int blockSize=3,  
                          bool useHarrisDetector=false, double k=0.04 ); 

Parameter Description:

• The first parameter: the input parameter of this function. The input is gray image;
• The second parameter: the output parameter of this function. The output is corner, and the saving type is vector or array;
• The third parameter: indicates the maximum number of corners extracted by the function;
• The fourth parameter: indicates the quality grade of the detected corner (usually the value between 0.10 and 0.01, which cannot be greater than 1.0);
• The fifth parameter: indicates the minimum spacing between two corner points, in pixels. Pixels with spacing less than this pixel value will be merged;
• The sixth parameter: generally cv::Mat type. The dimension is consistent with the input. Corner detection is not performed when the mask value is 0;
• The seventh parameter: indicates the size of the area involved in the calculation of corner points. The common value is 3. However, if the image resolution is high, a larger value can be considered;
• The eighth parameter: indicates whether Harris corner detection is used. If false, Shi Tomasi corner is used;
• The ninth parameter: the intermediate parameter used to extract Harris corner, generally taking the empirical value of 0.04 ~ 0.06. When it is a Shi Tomasi corner, this parameter has no effect;

2. cv::cornerSubPix() function

Function: since the corner extracted by goodFeaturesToTrack function is pixel level, in some cases such as slam, the corner applied at pixel level does not meet the accuracy requirements. cornerSubPix() function can further optimize the pixel level corner to make it reach sub-pixel level.

Function prototype:

void cornerSubPix(InputArray image,
					InputOutputArray corners, 
						Size winSize, 
							Size zeroZone, 
								TermCriteria criteria)

Parameter Description:

• The first parameter: indicates the input image. The input images of the above two functions must be consistent;
• The second parameter: represents the input corner, which is generally the corner extracted by goodFeaturesToTrack function. This parameter also stores the output sub-pixel corner;
• The third parameter: represents half of the area side length considered in calculating sub-pixel corners;
• The fourth parameter: represents half the size of the dead zone. If the value is cv::Size(-1,-1), it means there is no such area;
• The fifth parameter: indicates the condition for calculating the sub-pixel corner to stop iteration. Either the iteration number is greater than a certain set value, or the accuracy reaches a certain set value, or even a combination of them;

3. calcOpticalFlowPyrLK() function

Function: LK optical flow tracking based on image pyramid

Function prototype:

void cv::calcOpticalFlowPyrLK	(	
InputArray 	prevImg,
InputArray 	nextImg,
InputArray 	prevPts,
InputOutputArray 	nextPts,
OutputArray 	status,
OutputArray 	err,
Size 	winSize = Size(21, 21),
int 	maxLevel = 3,
TermCriteria 	criteria = TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, 30, 0.01),
)	

Parameter Description:

• The first parameter: the gray image of the previous frame as the benchmark;
• The second parameter: the gray image used for optical flow tracking, which is the same size and type as the gray image of the previous frame;
• The third parameter: the corner detected in the previous frame;
• The fourth parameter: the corner of gray image output for optical flow tracking
• The fifth parameter: indicates the state vector. Set the state of the corner with successful tracking to 1, otherwise set to 0;
• Sixth parameter: output error vector;
• The seventh parameter: the size of the search window of each pyramid;
• The eighth parameter: indicates the height of the pyramid;
• The ninth parameter: Specifies the termination condition of the iterative search algorithm;

example:

cv::goodFeaturesToTrack(
mImGrayPre, 
lastFramePoints, 
1000, 
0.01, 
8,
cv::Mat(),
3,
false, 
0.04);  

cv::cornerSubPix(
mImGrayPre, 
lastFramePoints, 
cv::Size(5, 5), 
cv::Size(-1, -1), 
cv::TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 30, 0.01));

cv::calcOpticalFlowPyrLK(
mImGrayPre, 
imGray, 
lastFramePoints, 
trackedPoints, 
state, 
err, 
cv::Size(21, 21), 
3, 
cv::TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 30, 0.01));