Previously, we talked about the two most commonly used image binarization algorithms, "Otsu method" and "optimal threshold iteration method", and made some improvements to the "optimal threshold iteration method", so that it can distinguish the foreground and background to a certain extent in the case of uneven illumination. However, in some cases, it can not be well distinguished:
Threshold method for image binarization
In this paper, we refer to the following literature, and combined with our improved "optimal threshold iteration method" above, we use C++/Opencv to realize an algorithm with better binarization effect in the case of uneven illumination.
[binarization of text images under uneven illumination] he Zhiming
01
—
Overall processing flow
The overall processing flow mainly includes illumination compensation -- > denoising -- > Image Enhancement -- > optimal threshold iterative acquisition threshold -- > binarization, as shown in the following figure. The first three steps are preprocessing, and the last two steps are binarization.
Below, we will discuss the principle and implementation of the above steps in turn.
02
—
Illumination compensation
According to the references mentioned above, illumination compensation is divided into two steps: calculating background information -- > using background information to compensate illumination.
The first is to calculate the background information. Here, the premise is that the background information is higher than the pixel value of the foreground information. If the image is not in this case, the black-and-white pixel value of the image can be reversed. The core idea is to take the rectangular neighborhood of each point in the image, then sort the pixel values of all points in the rectangular neighborhood from large to small, take the pixel values with large values of the top five pixels, and calculate the average value as the background value of the point.
Implementation code:
//SRC -- original image / / win_ 2 -- the length and width of the rectangular neighborhood window are (2 * win_2 + 1) mat get_ Background (mat SRC, int win_2) {int winsize = 2 * win_2 + 1; mat src_tmp; / / to enable all points in the original graph to get complete rectangular neighborhoods, first fill the edge of the original graph with copymakeorder (SRC, src_tmp, win_2, win_2, win_2, border_replicate); / / median filter denoising to reduce the background magazine medianBlur(src_tmp, src_tmp, 9);
Mat dst(src.size(), CV_8UC1); for (int i = win_2; i < src_tmp.rows-win_2; i++) { uchar *pd = dst.ptr<uchar>(i-win_2); for (int j = win_2; j < src_tmp.cols-win_2; j++) { Mat tmp; //Intercept the rectangular neighborhood SRC around each point_ tmp(Rect(j-win_2, i-win_2, winsize, winsize)). copyTo(tmp); / / convert two-dimensional matrix into one-dimensional data TMP reshape(1, 1). copyTo(tmp); / / sort cv::sort(tmp, tmp, CV_SORT_EVERY_ROW + CV_SORT_ASCENDING); uchar *p = tmp. ptr<uchar>(0); / / take the first five pixel values after sorting and calculate the average value as the background value. pd[j - win_2] = (uchar)((p[tmp.cols - 1] + p[tmp.cols - 2] + p[tmp.cols - 3] + p[tmp.cols - 4] + p[tmp.cols - 5]) * 0.2);}}
return dst;}In this paper, we use a blood vessel image with uneven illumination to test the effect of the algorithm. As shown in the figure below, we can see that the illumination information of the original image is uneven, so the brightness of different areas on the calculated background image is different.
Original drawing
Background map
Then, the background information is used to compensate the illumination. For each point in the original image, the pixel value after illumination compensation is calculated by using the points at the same coordinate position in the background image. Assuming that the pixel value of the original image is x, the pixel value of the same position point in the background image is xb, and the pixel value after illumination compensation is y, the processing flow is shown in the following figure:
The following piecewise function formula is used to calculate the value of k:
The blood vessel map and its background map after light compensation are as follows. It can be seen that the light intensity distribution of the background map after light compensation is much more uniform than that of the original map.
Image after illumination compensation
Background image of image after illumination compensation
03
—
Denoising
The commonly used denoising algorithms include mean filter, Gaussian filter, median filter and nonlocal mean filter. In order to preserve the vascular edge better, we use nonlocal mean filter to denoise the vascular image. We talked about the principle and implementation of nonlocal mean filtering earlier (Opencv also has the ready-made fastNlMeansDenoising function implementation):
Principle and C + + implementation of nonlocal mean filtering (NL means) algorithm
CUDA optimization acceleration of nonlocal mean filtering (NL means) algorithm
Denoise the image after illumination compensation, and the results are as follows:
Denoised image
04
—
image enhancement
Common image enhancement algorithms include histogram equalization, Laplace sharpening, Gamma correction, USM sharpening and so on. I have tried these algorithms and feel that the effect of USM sharpening is relatively better. Therefore, we use USM algorithm to enhance the denoised image.
In the image, usually, the edge area belongs to high-frequency signal and other areas belong to low-frequency signal. If the low-frequency signal is subtracted from the original signal, the rest is high-frequency signal, that is, the edge information is highlighted. Gaussian filtering can be regarded as a low-pass filter. After Gaussian filtering, the image becomes blurred, that is, the high-frequency signal is filtered out and the remaining low-frequency signal. At this time, the image after Gaussian filtering can be subtracted from the original image to obtain the high-frequency signal image, that is, the image with enhanced edge, which is the basic principle of USM algorithm.
Code implementation:
//IMG -- original image / / DST -- enhanced image / / alpha1 -- original image coefficient, usually 1.0~1.5//alpha2 -- Gaussian filtered image coefficient, usually -0.5 ~ -0.9 void img_ ehance(Mat img, Mat &dst, double alpha1, double alpha2){ Mat blur; GaussianBlur(img, blur, Size(0, 0), 30); addWeighted(img, alpha1, blur, alpha2, 0, dst);}The denoised image is enhanced, and the effect is as follows:
Enhanced image
05
—
The optimal threshold iterative method is used to obtain the threshold
Previously, we have talked about the principle and implementation of the optimal threshold iterative method:
Threshold method for image binarization
Here we go directly to the code: / / src -- original image
//T -- threshold at the end of iteration / / a -- parameter controlling the range of foreground area. The larger a, the more areas are divided into foreground_ threld(Mat src, float T, float a){ const int level = 256;
float hist[level] = { 0 };
float sum = 0.0; for (int i = 0; i < src.rows; i++) //Statistical histogram {uchar * P = Src. PTR < uchar > (I); for (int j = 0; J < Src. Cols; j + +) {hist [P [J] + = 1.0; sum + = P [J];}}
float hist_add[level] = { 0 }; hist_add[0] = hist[0]; for (int i = 1; i < level; i++) //Cumulative statistical histogram {hist_add[i] = hist_add[i - 1] + hist[i];}
const int img_size = src.rows*src.cols; float T_current = sum / img_size; //Calculate the pixel average value as the initial value of the threshold value ＾ float ＾ t_ next = 0; float S0 = 0.0; float n0 = 0.0; float S1 = 0.0; float n1 = 0.0; float d = abs(T_current - T_next); float count = 0;
while (d >= T && count < 1000) { count++;
uchar TT = (uchar)(T_current);
n1 = hist_add[TT]; //Number of points with pixel value less than or equal to TT n0 = img_size - hist_add[TT]; // Number of points with pixel value greater than TT
S1 = 0.0; for (int i = 0; i <= TT; i++) //Use the cumulative histogram to quickly calculate the sum of pixel values of all points with pixel values less than or equal to the threshold value in the image {S1 + = I * hist [i];} S0 = sum - S1; // Calculate the sum of pixel values of all points in the image whose pixel values are greater than the threshold
float m0 = S0 / n0; //Calculate the average value of all pixels in the image whose values are greater than the threshold, float m1 = S1 / n1// Calculate the average value of all pixels in the image whose values are less than or equal to the threshold
T_next = m1 + (m0 - m1)*a; //Parameter a controls the weights of m0 and m1
d = abs(T_current - T_next); //The absolute value t of the difference between the threshold value obtained in the current iteration and the threshold value obtained in the next iteration_ current = T_ next; // Assign the threshold value of the next iteration to the threshold value of the current iteration}
return T_current;}After obtaining the threshold using the optimal threshold iteration method, the image can be binarized:
void threld_img(Mat src, Mat &dst, float a, double h){ //Get background map {Mat} BG = get_background(src, 35); / / illumination compensation ， Mat ， GB = guangqiang_buchang(src, bg); / / denoising fastnlmeans denoising (GB, GB, h, 7, 25)// Image enhancement img_ehance(gb, gb, 1.0, -0.5); / / get threshold float TT = get_threld(gb, 0.01, a); // Binarization} threshold(gb, dst, TT, 255, THRESH_BINARY_INV);}The binarization results of comparing whether to make illumination compensation are as follows. Yes, when illumination compensation is not made, the binarization results are not good. The places with uneven illumination are originally the background and are also divided into prospects. There is no such problem when light compensation is made.
Binarization results without illumination compensation
Binarization results for illumination compensation