Structure of this document:

- Demand background
  - Attacking Python
  - Java and Python
- to Python accelerate
  - Looking for direction
  - Jython?
- Python->Native code
  - Overall thinking
  - Hands on
  - automation
- key problem
  - import Question of
  - Python GIL problem
- Test effect
- summary

Demand background

Attacking Python

With the rise of artificial intelligence, Python, a once small programming language, is a second spring.

The development framework of machine learning / deep learning based on tensorflow and python has become popular, which has helped python, a programming language that was once good at reptiles (Python powder is not angry), cut through thorns and thorns all the way on the TIOBE programming language ranking list, ranked among the top three, second only to Java and C, and cut down the powerful enemies such as C + +, JavaScript, PHP and c# etc.

Of course, xuanyuanjun always does not advocate the competitive comparison between programming languages. Each language has its own advantages and disadvantages and its own application fields.
On the other hand, the data of TIOBE statistics can not represent the actual situation in China. The above example only reflects the current popularity of Python.

Java or Python

Back to our needs, there are Python R & D teams and Java R & D teams in many enterprises. The python team is responsible for the development of artificial intelligence algorithms, while the Java team is responsible for the engineering of algorithms, providing interfaces for higher-level applications through engineering packaging.
You may want to ask, why not directly use Java for AI development? Two teams. In fact, frameworks including TensorFlow have gradually begun to support the Java platform, and it is not impossible to use Java for AI development (in fact, many teams are already doing so). However, due to historical reasons, there are not many people doing AI development, and most of them are Python technology stacks, and Python's AI development ecology has been relatively perfect, Therefore, in many companies, the algorithm team and engineering team have to use different languages.

Now it's time to raise the important question of this article: how does the Java engineering team call Python's algorithmic capabilities?
Basically, there is only one answer: Python starts a Web service through frameworks such as Django/Flask, and Java interacts with it through Restful API

The above method can solve the problem, but the performance problem comes with it. Especially after the increase in the number of users and a large number of concurrent interface access, network access and Python code execution speed will become the bottleneck of the whole project.

Of course, companies with good money can stack performance with hardware. If one doesn't work, deploy more Python Web services.

In addition, is there a more affordable solution? This is the problem to be discussed in this article.
Speed up Python
Looking for direction

Among the above performance bottlenecks, there are two main reasons that hinder execution speed:

• Accessing through the network is not as fast as calling internal modules directly
• Python is interpreted and executed. It can't get up quickly

As we all know, Python is an interpreted scripting language. Generally speaking, in terms of execution speed:
Interpretive language < intermediate byte code language < locally compiled language
Naturally, there are two directions we should strive for:

Can I call directly locally without network access
Python does not interpret execution

Combining the above two points, our goal is also clear:
Convert Python code into modules that Java can call directly locally
For Java, there are two types that can be called locally:

• Java code package
• Native code module

In fact, what we usually call Python refers to CPython, which is interpreted and executed by the interpreter developed by C language. In addition, in addition to C language, many other programming languages can also develop virtual machines to interpret and execute Python scripts according to Python language specifications:

• Cpython: interpreter written in C language
• Jython: an interpreter written in Java
  Ir
• onPython: .NET platform interpreter
• PyPy: Python's own interpreter (chicken lays egg, egg lays chicken)

Jython?
If you can directly execute Python scripts in the JVM, the interaction with Java business code is naturally the simplest. However, subsequent research found that this road was soon blocked:

Python 3.0 is not supported Syntax above 0
If the third-party library referenced in the python source code contains C language extensions, it will not be able to provide support, such as numpy

This road won't work. There's another way: convert Python code into Native code blocks, and Java calls them through JNI's interface.

Python - > native code

Overall thinking
First convert the Python source code into C code, then compile the C code into binary module so/dll with GCC, then encapsulate the Java Native Interface, convert it into Jar package with Jar packaging command, and then Java can call it directly.

The process is not complex, but to achieve this goal completely, there is a key problem to be solved:
How can Python code be converted to C code?
Finally, it's the turn of the protagonist of this article. A core tool to be used is called python
Note that Python here is not the same as CPython mentioned earlier. CPython in a narrow sense refers to the Python interpreter written in C language. It is our default Python script interpreter under Windows and Linux.
Python is a third-party library of Python. You can install it through PIP install python.
Officially, Python is a superset of Python language specification, which can encode Python+C pyx script is converted to C code, which is mainly used to optimize the performance of Python script or Python calls C function library.
It sounds a little complicated and a little convoluted, but it doesn't matter. Just get one core point: Python can convert Python scripts into C code
Let's look at an experiment:

# FileName: test.py
def TestFunction():
  print("this is print from python script")

Convert the above code through Python to generate test c. Look like this:

The code is very long and not easy to read. Here is only a screenshot.

Hands on

1. Prepare Python source code

FileName: Test.py
# Example code: convert the input string to uppercase
def logic(param):
  print('this is a logic function')
  print('param is [%s]' % param)
  return param.upper()

# Interface function, the interface exported to Java Native
def JNI_API_TestFunction(param):
  print("enter JNI_API_test_function")
  result = logic(param)
  print("leave JNI_API_test_function")
  return result

Note 1: a convention is used in the python source code: JNI_API_ The function beginning with the prefix represents the interface function to be exported by the Python code module for external calls. The purpose of this is to enable our Python one click to Jar package system to automatically identify which interfaces to extract as export functions.
Note 2: the input of this kind of interface function is a string of python str type, and the output is the same. This makes it easy to transplant the previous RESTful interface with JSON as a parameter. The advantage of using JSON is that it can encapsulate parameters and support a variety of complex parameter forms without overloading different interface functions for external calls.
Note 3: it should also be noted that the interface function is prefixed with JNI_API_ Later, function naming should not be based on python's usual underline naming method, but use hump naming method. Note that this is not a recommendation, but a requirement. The reason will be mentioned later.
2. Prepare a main C Documents
The function of this file is to encapsulate the code generated by Python conversion into the style of Java JNI interface for the use of Java in the next step.

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
#include <Python.h>
#include <stdio.h>

#ifndef _Included_main
#define _Included_main
#ifdef __cplusplus
extern "C" {
#endif

#if PY_MAJOR_VERSION < 3
# define MODINIT(name)  init ## name
#else
# define MODINIT(name)  PyInit_ ## name
#endif
PyMODINIT_FUNC  MODINIT(Test)(void);

JNIEXPORT void JNICALL Java_Test_initModule
(JNIEnv *env, jobject obj) {
  PyImport_AppendInittab("Test", MODINIT(Test));
  Py_Initialize();

  PyRun_SimpleString("import os");
  PyRun_SimpleString("__name__ = \"__main__\"");
  PyRun_SimpleString("import sys");
  PyRun_SimpleString("sys.path.append('./')");

  PyObject* m = PyInit_Test_Test();
  if (!PyModule_Check(m)) {
      PyModuleDef *mdef = (PyModuleDef *) m;
      PyObject *modname = PyUnicode_FromString("__main__");
      m = NULL;
      if (modname) {
        m = PyModule_NewObject(modname);
        Py_DECREF(modname);
        if (m) PyModule_ExecDef(m, mdef);
      }
  }
  PyEval_InitThreads();
}


JNIEXPORT void JNICALL Java_Test_uninitModule
(JNIEnv *env, jobject obj) {
  Py_Finalize();
}

JNIEXPORT jstring JNICALL Java_Test_testFunction
(JNIEnv *env, jobject obj, jstring string)
{
  const char* param = (char*)(*env)->GetStringUTFChars(env, string, NULL);
  static PyObject *s_pmodule = NULL;
  static PyObject *s_pfunc = NULL;
  if (!s_pmodule || !s_pfunc) {
    s_pmodule = PyImport_ImportModule("Test");
    s_pfunc = PyObject_GetAttrString(s_pmodule, "JNI_API_testFunction");
  }
  PyObject *pyRet = PyObject_CallFunction(s_pfunc, "s", param);
  (*env)->ReleaseStringUTFChars(env, string, param);
  if (pyRet) {
    jstring retJstring = (*env)->NewStringUTF(env, PyUnicode_AsUTF8(pyRet));
    Py_DECREF(pyRet);
    return retJstring;
  } else {
    PyErr_Print();
    return (*env)->NewStringUTF(env, "error");
  }
}
#ifdef __cplusplus
}
#endif
#endif

There are three functions in this file:

Java_Test_initModule: python initialization work
Java_Test_uninitModule: python uninitialization
Java_Test_testFunction: the real business interface encapsulates the JNI defined in the original Python_ API_ The call of testfuncion function is also responsible for the conversion of parameter jstring type at JNI level.

According to the JNI interface specification, the C function naming at the native level needs to conform to the following form:

// QualifiedClassName: full class name
// MethodName: JNI interface function name
void
JNICALL
Java_QualifiedClassName_MethodName(JNIEnv*, jobject);

So in main The definitions in the C file need to be named as above, which is why it was emphasized that the python interface function cannot be named with an underscore, which will lead to the JNI interface unable to find the corresponding native function.
3. Compile and generate dynamic library with Python tool
Add a little preparation: remove the suffix from the Python source file Change py to pyx
python source code test Pyx and main c files are ready, and then it's time for python to debut. It will automatically convert all pyx files into c file, combined with our own main c file, calling gcc internally to generate a dynamic binary library file.
Python needs to prepare a setup Py file to configure the translation information, including input file, output file, compilation parameters, including directory and link directory, as shown below:

from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension

sourcefiles = ['Test.pyx', 'main.c']

extensions = [Extension("libTest", sourcefiles, 
  include_dirs=['/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/include',
    '/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/include/darwin/',
    '/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m'],
  library_dirs=['/Library/Frameworks/Python.framework/Versions/3.6/lib/'],
  libraries=['python3.6m'])]

setup(ext_modules=cythonize(extensions, language_level = 3))

Note: this involves the compilation of Python binary code. You need to link the python library
Note: JNI related data structure definitions are involved here, and Java JNI directory needs to be included
setup. After the PY file is ready, execute the following command to start conversion + compilation:

python3.6 setup.py build_ext --inplace

Generate the dynamic library file we need: libtest so
4. Prepare the interface file for Java JNI calls
An interface needs to be defined for the use of Java business code, as shown below:

// FileName: Test.java
public class Test {
  public native void initModule();
  public native void uninitModule();
  public native String testFunction(String param);
｝

To this step, the purpose of calling in Java has been realized. Before calling the business interface, we need to call initModule to initialize the native level Python.

import Test;
public class Demo {
    public void main(String[] args) {
        System.load("libTest.so");
        Test tester = new Test();
        tester.initModule();
        String result = tester.testFunction("this is called from java");
        tester.uninitModule();

        System.out.println(result);
    }
}

Output:

enter JNI_API_test_function
this is a logic function
param is [this is called from java]
leave JNI_API_test_function
THIS IS CALLED FROM JAVA!

Successful implementation of calling Python code in Java!
5. Packaged as Jar package
It's not enough to do the above. In order to have a better use experience, we'll take another step forward and package it into a Jar package.
First, the original JNI interface file needs to be expanded to add a static method loadLibrary to automatically release and load so files.

// FileName: Test.java
public class Test {
  public native void initModule();
  public native void uninitModule();
  public native String testFunction(String param);
  public synchronized static void loadLibrary() throws IOException {
    // Implementation strategy
  ｝
}

Then convert the above interface file into a java class file:

javac Test.java

Finally, prepare to place the class files and so files in the Test directory and package them:

jar -cvf Test.jar ./Test

automation
The above five steps are really troublesome if you have to do them manually every time! Fortunately, we can write Python scripts to completely automate this process, and really achieve Python one click conversion of Jar packages
Due to space limitations, only the key of automation process is mentioned here:

Automatically scan and extract interface functions to be exported from python source code
main.c,setup.py and JNI interface java files need to be generated automatically (they can be built quickly in the form of defined templates + parameters), and the corresponding relationship between module names and function names needs to be handled well

key problem
1.import problem
The case shown above is only a single py file. In practice, our project usually has multiple py files, and these files usually constitute a complex directory level, with various import relationships among them.
One of the biggest pitfalls of Python is that the directory level information of the code file will be lost in the file code processed by it. As shown in the figure below, there is no difference between the code converted by C.py and the code generated by m/C.py.

This brings a very big problem: if the C.py module in the m directory is referenced in the A.py or B.py code, the loss of directory information will cause them to report an error when executing import m.C and cannot find the corresponding module!
Fortunately, experiments show that in the above figure, if modules A, B and C are in the same directory, import can execute correctly.
Xuanyuanjun once tried to read the source code of python, modify it and retain the directory information, so that the generated C code can still be import ed normally. However, due to the lack of time and understanding of the mechanism of Python interpreter, he chose to give up after some attempts.
I stuck with this problem for a long time and finally chose a stupid way: expand the tree code level directory into a flat directory structure. For the example in the above figure, the expanded directory structure becomes

A.py
B.py
m_C.py

This alone is not enough. It is also necessary to amend all references to C in A and B to M_ Reference to C.
This seems very simple, but the actual situation is much more complex than this. In Python, import is not only as simple as import, but has various complex forms:

import package
import module
import package.module
import module.class / function
import package.module.class / function
import package.*
import module.*
from module import *
from module import module
from package import *
from package import module
from package.module import class / function
...

In addition, there may be a way to reference directly through the module in the code.
The price of expanding into a flat structure is to deal with all the above situations! Xuanyuan Jun had no choice but to make such a bad decision. If you guys have a better solution, please don't hesitate to give me advice.
2.Python GIL problem
The jar package after Python conversion was used in actual production, but then a problem was found:
Whenever the Java concurrency count goes up, the JVM always crashes from time to time
After analyzing the crash information, it is found that the crash is in the Python converted code in the Native code.

Is it a python bug?
Are there pits in the converted code?
Or is there a problem with the above import correction?

The dark cloud of collapse hung over my head for a long time. Calm down and think:
Why does it crash only after the test is normal and no problems are found?
Looking at the crash log again, I find that in the native code, the exception always occurs where malloc allocates memory. Is it difficult that the memory is damaged?
It was also found that only functional testing was completed during testing, and no concurrent stress testing was conducted, and the crash scenario was always in a multi concurrent environment. If multiple threads access the JNI interface, the Native code will be executed in multiple thread contexts.
Suddenly an alert: 99% has something to do with Python's GIL lock!

As we all know, due to historical reasons, python was born in the 1990s. At that time, the concept of multithreading was far less popular than today. As a product of this era, Python is a single threaded product.
Although Python also has a multithreading library that allows the creation of multiple threads, because the C language interpreter is not thread safe in memory management, there is a very important lock inside the interpreter that restricts Python's multithreading, so the so-called multithreading is actually only a pit for everyone to take turns.
The original GIL was scheduled and managed by the interpreter. Now it has been transformed into C code. Who is responsible for managing the security of multithreading?
Since Python provides a set of interfaces for C language calls, allowing Python scripts to be executed in C programs, check the API documentation to see if you can find the answer.
Fortunately, I found it:
Get GIL lock:

Release GIL lock:

The GIL lock needs to be obtained at the JNI call entry, and the GIL lock needs to be released when the interface exits.
After adding the control of GIL lock, the annoying Crash problem is finally solved!
Test effect
Prepare as like as two peas py files, the same algorithm function, one through Flask Web interface, (Web service deployed in local 127.0.0.1, minimizing network latency), and another through the above process to Jar package. The two files are the same.
In the Java service, the two interfaces are called 100 times respectively, and the whole test work is carried out 10 times. The execution time is counted:

In the above test, in order to further distinguish the delay caused by the network and the delay of the code execution itself, the timing is made at the entrance and exit of the algorithm function, and before Java executes the interface call and obtains the results. In this way, the proportion of the time of the algorithm execution itself in the whole interface call process can be calculated.

It can be seen from the results that for the interface access through the Web API, the execution time of the algorithm itself accounts for only 30% +, and most of the time is spent on network overhead (sending and receiving of data packets, scheduling processing of flash framework, etc.).

Through the local call of JNI interface, the execution time of the algorithm accounts for more than 80% of the execution time of the whole interface, while the interface conversion process of Java JNI only takes 10% + time, which effectively improves the efficiency and reduces the waste of additional time.

In addition, looking at the execution part of the algorithm itself, the execution time of the same code converted into Native code is 300 ~ 500 μ s. CPython explained that the execution time was 2000 ~ 4000 μ s. It is also very different.

summary
This paper provides a new idea of Java calling Python code for reference only. Its maturity and stability need to be discussed. Accessing through HTTP Restful interface is still the first choice for cross language docking.
As for the methods in the article, interested friends are welcome to leave messages.