Many friends will encounter a variety of problems when developing face recognition system. Now we take Android platform for example, which uses HongSoft's free offline face recognition SDK development, to introduce how to develop a face recognition system with image detection, video picture face attribute detection, face registration recognition and other functions.
I. Acquisition of SDK
1. Application Address for ArcFace 2.0
https://ai.arcsoft.com.cn/product/arcface.html
2. Fill in the application for information and submit it it
After the application is approved, you can download the SDK and view APP_ID and SDK_KEY.
<br/>
II. Introduction of SDK Function
Rainbow Soft ArcFace 2.0 Android SDK includes face detection, age information detection, gender information detection, three-dimensional face angle detection, biopsy detection, face feature extraction and face feature comparison. SDK also supports static graph mode detection and video stream mode detection.
<br/>
3. Using SDK
1. Engineering Configuration
Put the jar file and so file in the corresponding directory, and add the jar to the project dependency
<br/>
2. Activating Engine
FaceEngine faceEngine = new FaceEngine(); //Activation method calls network and file operation for the first time, and only file operation for the next time. It is suggested that it should not be placed in the main thread. int activeCode = faceEngine.active(SettingsActivity.this, Constants.APP_ID, Constants.SDK_KEY); if (activeCode == ErrorInfo.MOK || activeCode == ErrorInfo.MERR_ASF_ALREADY_ACTIVATED) { //Successful or activated activation }else{ //Failure of activation }
3. Initialization engine, recommended at startup
faceEngine = new FaceEngine(); int afCode = faceEngine.init(context.getApplicationContext(), FaceEngine.ASF_DETECT_MODE_VIDEO, FaceEngine.ASF_OP_0_HIGHER_EXT, 16, 10, FaceEngine.ASF_FACE_RECOGNITION | FaceEngine.ASF_FACE_DETECT | FaceEngine.ASF_FACE3DANGLE|....); if (afCode != ErrorInfo.MOK) { //Successful initialization }else{ //initialization failed }
4. Face Detection
List faceInfoList = new ArrayList<>(); int code = faceEngine.detectFaces(nv21, width, height, FaceEngine.CP_PAF_NV21, faceInfoList); if (code == ErrorInfo.MOK && faceInfoList.size() >0) { //Face Detection Successfully and Face Detection }else{ //Failure or failure of face detection }
5. Feature extraction
FaceFeature faceFeature = new FaceFeature(); int code = faceEngine.extractFaceFeature(nv21, width, height, FaceEngine.CP_PAF_NV21, faceInfo, faceFeature); if (code == ErrorInfo.MOK) { //Successful feature extraction }else{ //Failure of feature extraction, you can view the reason according to code }
6. Characteristic comparison
public void compareFace(FaceFeature faceFeature1,FaceFeature faceFeature2){ FaceSimilar faceSimilar = new FaceSimilar(); int code = faceEngine.compareFaceFeature(faceFeature1, faceFeature2, faceSimilar); if (code == ErrorInfo.MOK){ //If the comparison is successful, you can see the similarity in faceSimilar }else{ //Comparing failures, you can see the reason according to code } }
7. Detection of three-dimensional angle, age and gender in vivo and face
int faceProcessCode = faceEngine.process(nv21, width, height, FaceEngine.CP_PAF_NV21, faceInfoList, FaceEngine.ASF_AGE | FaceEngine.ASF_GENDER | FaceEngine.ASF_FACE3DANGLE | FaceEngine.ASF_LIVENESS); if (faceProcessCode != ErrorInfo.MOK){ //Failure }else{ //The process succeeds and results are available List ageInfoList = new ArrayList<>(); List genderInfoList = new ArrayList<>(); List face3DAngleList = new ArrayList<>(); List faceLivenessInfoList = new ArrayList<>(); int ageCode = faceEngine.getAge(ageInfoList); int genderCode = faceEngine.getGender(genderInfoList); int face3DAngleCode = faceEngine.getFace3DAngle(face3DAngleList); int livenessCode = faceEngine.getLiveness(faceLivenessInfoList); //Error code checking to determine whether all successful if ((ageCode | genderCode | face3DAngleCode | livenessCode) != ErrorInfo.MOK) { return; }else{ //All the tests are successful and the results can be obtained from ageInfoList, genderInfoList, face3D AngleList and faceLivenessInfoList. } }
8. Destroy the engine and recommend execution on exit
if (faceEngine != null) { int faceEngineCode = faceEngine.unInit(); Log.i(TAG, "unInitEngine: " + faceEngineCode); }
<br/>
IV. NOTES
1. Selection of Detection Mode
Initialization of the engine requires an incoming detection mode (video stream mode or picture mode), in addition to identifying functional modules.( extractFaceFeature and compareFaceFeature ) Other functions are distinguished by detection modes. For face detection, age detection, gender detection, face three-dimensional angle, biopsy detection, using video stream mode processing speed is faster. But the biopsy detection of video stream mode is special: although the result can be obtained immediately after processing, the value returned after the first frame of a video stream is unknown, and the real algorithm result is the value obtained after a period of time.
2. Multithread Use of Engine
Algorithmic functions in the same function module of a single engine do not support multithreaded calls and cannot be destroyed during calls. If multithreaded calls are required, multiple engines need to be created.
For example:
- Face Detection in Line 1 (detectFaces) At the same time, feature parsing can be done in thread 2. (extractFaceFeature).
- Face Detection in Line 1 (detectFaces) Face detection cannot be performed simultaneously in thread 2 (detectFaces).
- Feature analysis on thread 1 (extractFaceFeature) Thread 2 cannot be destroyed at the same time (unInit)
3. Runtime report java.lang.UnsatisfiedLinkError
java.lang.UnsatisfiedLinkError is a common jni-related error. If this error occurs, there are generally the following reasons:
- There are several ABI subdirectories in the dynamic library file directory or the related dynamic library file directory under the Android project directory, but the list of files in the subdirectory is different.
Detailed introduction:
First, when an application is installed on a device, only the best so libraries supported by the CPU architecture of the device will be copied to the local lib. For example, in the local library directory of a project, there are four dynamic library files under armeabi-v7: a.so,b.so,c.so,d.so, and two dynamic library files under arm64-v8a: c.so,d.so. A device supports arm64-v8a, arm64-v8a is better than armeabi-v7, so only so files in arm64-v8a directory are copied during installation. It is no problem to load C.so and d.so, but when loading A.so or b.so, because there are no such files in arm64-v8a directory, it will be reported to java.lang.UnsatisfiisfiedLinkError.
The following are some possible scenarios for dynamic library file storage:
Solution:
If the dynamic library directory is not reassigned in build.gradle, the default path for the dynamic library will be projectName - > moduleName - > SRC - > main - > jniLibs.
Make sure that the armeabi-v7a directory under the dynamic library directory or the armeabi-v7a-compatible directory contains ArcFace 2.0 Android SDK-related dynamic library files, and that the list of file names under each ABI directory is the same.
- Although the dynamic library file was loaded successfully, the corresponding native method could not be found.
Detailed introduction:
For example, in the com.arcsoft.Test class, there is a method defined as:
private native int add(int a,int b)，
When writing C++ code, the corresponding content is:
extern "C" JNIEXPORT jint JNICALL Java_com_arcsoft_Test_add(JNIEnv *env, jobject, jint a, jint b, jint c)，
Native defines a method with one more parameter. The corresponding Java method will be private native int add(int a, int b, int c). The signatures of the two methods are inconsistent. Therefore, when calling add in Java, the corresponding native method will not be found.
Solution:
Generally, when using SDK, this problem is caused by using jars or dynamic libraries in different versions or platforms. In order to ensure the consistency of method signatures, make sure that jars and dynamic libraries are files in the same version of the same platform.
- Relevant native methods are defined in Java, but dynamic library files are not loaded
Detailed introduction:
For example, in the com.arcsoft.Test class, there is a method defined as:
private native int add(int a,int b)，
One of the corresponding C++ codes is:
Extern "C" JNIEXPORT jint JNICALL Java_com_arcsoft_Test_add (JNIEnv*env, jobject, jint a, jint b), but has been unable to load, it may also be that the dynamic library is not loaded to find the native method.
Solution:
Dynamic libraries need to be loaded before so files are called, usually in a static code block of a class.
<br/>
Reference documents:
Android Platform Development Guide:
https://ai.arcsoft.com.cn/man...