Share the entire process of TensorFlow Android side compilation

Turn over the wall, must turn over the wall, and ensure download speed!
First Note: I am a deep learning Xiao Bai, but I don't know about deep learning TensorFlow. Because the company needs to put the image recognition model on the mobile device, it tried to compile it, and fortunately it succeeded in the end!

Reference

Guidance for Android-side compilation in TensorFlow GitHub
Of course, problems during compilation can occur issues Find in

Compilation environment

System: mac os 10.14
bazel: 0.24.1
TensorFlow: 1.14.0
Android SDK: 23
Android NDK: 17c
Python: 3.6.3
jdk: 1.8

Environment Setup

1. Install Bazel 0.24.1

Follow the steps on the official mac installation, why install 0.24.1, because other I tried, not suitable.

2. Install jdk1.8

3. Install python 3.6.3

4. Install Android SDK 23

Use Android Studio or the official website to download and install

5. Install Android NDK

Unzip to sdk installation directory after downloading from official website or Baidu NDK 17
Why must it be 17c, because I've tried everything else, but it's just a waste of emotion.

6. Download TensorFlow version 1.14.0 release

The reason I chose 1.14.0 was that I successfully compiled both iOS and Android on my mac for this version
PS: I've tried other versions too, missing files during compilation.

Compilation process

1. Empty Compile Cache

If you've compiled it before, it's best to use it. If you don't, you don't know what's going to happen.

bazel clean --expunge

2. configure the compilation environment required during TensorFlow compilation

Follow the steps to configure your computer configuration. Here are some configurations on my mac:
Terminal Input:

./configure

WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.24.1 installed.
Please specify the location of python. [Default is /usr/local/opt/python@2/bin/python2.7]: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6

Found possible Python library paths:
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
Please input the desired Python library path to use.  Default is [/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages]

Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: n
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: n
Clang will not be downloaded.

Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y
Searching for NDK and SDK installations.

Please specify the home path of the Android NDK to use. [Default is /Users/mask/library/Android/Sdk/ndk-bundle]: /Users/mask/Library/Android/sdk/android-ndk-r17c 

Please specify the (min) Android NDK API level to use. [Available levels: ['14', '15', '16', '17', '18', '19', '21', '22', '23', '24', '26', '27', '28']] [Default is 18]: 21

Please specify the home path of the Android SDK to use. [Default is /Users/mask/library/Android/Sdk]: /Users/mask/Library/Android/sdk 

Please specify the Android SDK API level to use. [Available levels: ['23', '26', '29']] [Default is 29]: 23

Please specify an Android build tools version to use. [Available versions: ['.DS_Store', '28.0.3', '29.0.2']] [Default is 29.0.2]: 28.0.3

Do you wish to build TensorFlow with iOS support? [y/N]: n
No iOS support will be enabled for TensorFlow.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=gdr         	# Build with GDR support.
	--config=verbs       	# Build with libverbs support.
	--config=ngraph      	# Build with Intel nGraph support.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=noaws       	# Disable AWS S3 filesystem support.
	--config=nogcp       	# Disable GCP support.
	--config=nohdfs      	# Disable HDFS support.
	--config=noignite    	# Disable Apache Ignite support.
	--config=nokafka     	# Disable Apache Kafka support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished

3. Start compiling

Terminal Input:

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so \
   --verbose_failures \
   --crosstool_top=//external:android/crosstool \
   --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
   --cxxopt=-std=c++11 \
   --cpu=armeabi-v7a

Start Compiling
After a long wait, previous compilations generally do not have problems, but at the end of the day:

ERROR: /Users/mask/Desktop/tensorflow-1.14.0Android/tensorflow/contrib/android/BUILD:60:1: Linking of rule '//tensorflow/contrib/android:libtensorflow_inference.so' failed (Exit 1): clang failed: error executing command 
  (cd /private/var/tmp/_bazel_mask/acbc0c8b666a3bbc6e9563ed36a12d36/execroot/org_tensorflow && \
  exec env - \
    ANDROID_BUILD_TOOLS_VERSION=28.0.3 \
    ANDROID_NDK_API_LEVEL=21 \
    ANDROID_NDK_HOME=/Users/mask/Library/Android/sdk/android-ndk-r17c \
    ANDROID_SDK_API_LEVEL=23 \
    ANDROID_SDK_HOME=/Users/mask/Library/Android/sdk \
    PATH=/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/mask/bin:/Users/mask/Library/Android/sdk/android-ndk-r17c/ \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 \
    PYTHON_LIB_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages \
    TF_CONFIGURE_IOS=0 \
    TF_DOWNLOAD_CLANG=0 \
    TF_NEED_CUDA=0 \
    TF_NEED_OPENCL_SYCL=0 \
    TF_NEED_ROCM=0 \
  external/androidndk/ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -shared -o bazel-out/armeabi-v7a-opt/bin/tensorflow/contrib/android/libtensorflow_inference.so -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/contrib/android/libandroid_tensorflow_inference_jni.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/java/src/main/native/libnative.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libandroid_tensorflow_lib.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/kernels/libandroid_tensorflow_kernels.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libandroid_tensorflow_lib_lite.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libplatform_base.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/hash/libhash.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/hash/libcity.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/types/libbad_variant_access.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/container/libraw_hash_set.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/types/libbad_optional_access.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/container/libhashtablez_sampler.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/synchronization/libsynchronization.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/synchronization/libgraphcycles_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libstacktrace.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libsymbolize.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libdebugging_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libdemangle_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libmalloc_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/libtime.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/internal/cctz/libtime_zone.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/internal/cctz/libcivil_time.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/strings/libstrings.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/strings/libinternal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libthrow_delegate.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libbase.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libdynamic_annotations.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libspinlock_wait.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/numeric/libint128.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libstats_calculator_portable.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/double_conversion/libdouble-conversion.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/farmhash_archive/libfarmhash.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/nsync/libnsync_cpp.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libprotos_all_proto_cc_impl.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/liberror_codes_proto_cc_impl.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/fft2d/libfft2d.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/protobuf_archive/libprotobuf.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/protobuf_archive/libprotobuf_lite.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/zlib_archive/libzlib.a -Wl,-no-whole-archive external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libandroid_support.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_static.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++abi.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libunwind.a -landroid -latomic -ldl -llog -lm -z defs -s -Wl,--gc-sections -Wl,--version-script,tensorflow/contrib/android/jni/version_script.lds -ldl -lz -pthread -pthread -pthread -framework Foundation -lm -lm -target armv7-none-linux-androideabi -Wl,--fix-cortex-a8 -gcc-toolchain external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64 -no-canonical-prefixes -Lexternal/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a -static-libgcc '--sysroot=external/androidndk/ndk/platforms/android-21/arch-arm')
Execution platform: @bazel_tools//platforms:host_platform
external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot open Foundation: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Target //tensorflow/contrib/android:libtensorflow_inference.so failed to build
INFO: Elapsed time: 2413.791s, Critical Path: 324.00s
INFO: 942 processes: 942 local.
FAILED: Build did NOT complete successfully

Solution:
Open file: bazel-tensorflow/external/com_google_absl/absl/time/internal/cctz/Build.bazel
Delete the following Blocks:

config_setting(
	name = "osx",
	constraint_values = [
		"@bazel_tools//platforms:osx",
	],
)

config_setting(
     name = "ios",
     constraint_values = [
         "@bazel_tools//platforms:ios",
     ],
 )

linkopts = select({
     ":osx": [
         "-framework Foundation",
     ],
     ":ios": [
         "-framework Foundation",
     ],
     "//conditions:default": [],
 }),

Re-enter after deletion, compile:

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so \
   --verbose_failures \
   --crosstool_top=//external:android/crosstool \
   --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
   --cxxopt=-std=c++11 \
   --cpu=armeabi-v7a

Next, the.so file will compile successfully!
Location: bazel-bin/tensorflow/contrib/android/libtensorflow_inference.so
It's best to copy one, since the next compilation of.jar will overwrite the directory
Compile.jar
Terminal Input:

bazel build //tensorflow/contrib/android:android_tensorflow_inference_java

This process is quick and generally does not cause problems. Compile the results:
bazel-bin/tensorflow/contrib/android/libandroid_tensorflow_inference_java.jar
Of course, there is also libandroid_tensorflow_inference_java-native-header.jar

Supplementary Instructions

I have some questions when delivering to Android developers:
1. To support other cpus, modify the compile command to compile separately: - cpu = armeabi...
2. For CPUs that do not support 64-bit operations such as armeabi-v7a, if 64-bit operations are used in the pb model, errors will occur ~. How to solve this problem: If you don't want to find a manufacturer or development, choose your own..