pexels-dinnow-9469740.jpg

Business background

Since the end of last year, our team has been making an intelligent hardware that can automatically take pictures of e-commerce products. After taking photos, the set of pictures of commodities will be displayed on the e-commerce platform.

For the commodity pictures to be displayed, we have high requirements for the quality of the pictures themselves. For example, ambiguous pictures cannot be displayed. Therefore, we need a method of image blur detection, which is convenient for us to screen the available images.

We use Laplacian based algorithm to detect whether the image is blurred. It's easier to call because OpenCV has a built-in Laplacian function.

Laplacian operator

The mapping to find the second derivative of multivariate functions is called Laplacian operator, which is equivalent to the derivative of second-order Sobel operator.

Definition of Laplacian operator:

Laplacian operator

We obtain the Laplace operator of discrete function by making difference on the second derivative of Laplace operator in X and Y directions.

Take the x direction as an example: First order difference: f'(x) = f(x) - f(x - 1) Second order difference: f''(x) = f'(x+1) - f'(x) = (f(x + 1) - f(x)) - (f(x) - f(x - 1)) After simplification: F '' (x) = f (x - 1) - 2 (f (x)) + F (x + 1)

Extract the previous coefficient: [1, - 2, 1]

Similarly, the coefficients in the y direction [1, - 2,1] can be obtained

The Laplace matrix is obtained by superposition

laplacian matrix

Laplace 3x3 convolution kernel.

Image blur detection algorithm

The main idea of the algorithm: first convert the image into a gray image, and then the gray image of a single channel will get a response map after being calculated by the Laplace 3x3 convolution kernel just calculated, and finally calculate the variance of the response map.

Based on the variance and the threshold set according to experience, whether the image is blurred can be determined. For the same type of commodity pictures, the same threshold can be used. The threshold may need to be adjusted for pictures taken in different commodities and different environments.

bool isImageBlurry(cv::Mat& img, double threshold)
{
    cv::Mat matImageGray;
    cv::cvtColor(img, matImageGray, COLOR_BGR2GRAY);
    cv::Mat dst, abs_dst;
    cv::Laplacian(matImageGray, dst, CV_16S, 3);
    cv::convertScaleAbs( dst, abs_dst );
    cv::Mat tmp_m, tmp_sd;
    double sd = 0;
    cv::meanStdDev(dst, tmp_m, tmp_sd);
    sd = tmp_sd.at<double>(0,0); // variance
    return ((sd * sd) <= threshold);
}

Find a blurred picture and write a simple example to test

test.jpeg

using namespace std;
using namespace cv;

bool isImageBlurry(cv::Mat& img, double threshold=49.0);

int main(int argc,char *argv[])
{
    String imageName;
    cout << "Enter the image file name: " << endl;
    cin >> imageName;
    // read the image
    Mat image = imread(imageName);

    double time = (double)getTickCount();

    bool result = isImageBlurry(image);
    time = ((double)getTickCount() - time) / getTickFrequency();
    cout << "The time taken is:" << time << "s" << endl;
    cout << "result: " << result << endl;
    return 0;
}

bool isImageBlurry(cv::Mat& img, double threshold)
{
    cv::Mat matImageGray;
    cv::cvtColor(img, matImageGray, COLOR_BGR2GRAY);
    cv::Mat dst, abs_dst;
    cv::Laplacian(matImageGray, dst, CV_16S, 3);
    cv::convertScaleAbs( dst, abs_dst );
    cv::Mat tmp_m, tmp_sd;
    double m = 0, sd = 0;
    cv::meanStdDev(dst, tmp_m, tmp_sd);
    m = tmp_m.at<double>(0,0); 
    sd = tmp_sd.at<double>(0,0);
    std::cout << "sd * sd: " << sd * sd << std::endl;
    return ((sd * sd) <= threshold);
}

Execution results:

Enter the image file name: 
test.jpeg
sd * sd: 31.0646
 Time taken: 0.0219034s
result: 1

It can be determined that the picture is blurred through the above procedure.

Finally, the main language used by our team is Java/Kotlin, and we need to write a jni to call the function.

summary

Laplacian is a common method of image blur detection without reference image. In addition, Brenner, Tenengrad, SMD, SMD2, etc. can also be used.