2021SC@SDUSC

rpc procedure in seafile

According to the previous analysis, I already know that one of the functions of seafile server is to provide rpc to the front end, that is, seahub. Therefore, in the source code of seahub, there should also be a call to searpc. Next, the relevant parts of seahub source code are analyzed.

It is learned from the document that searpc itself does not have the ability to create servers and clients. It needs to manually use socket s and other libraries to establish port listening, and the client needs to establish a connection to the server itself.

Therefore, the main purpose is to find the code to establish the connection part.

Searchpc mechanism

In the source code, you can see that in many places, you can directly call the api in seafile server through import seaserv; Therefore, it can be inferred that when seahub starts running, it has been connected to the rpc server.

Through full-text search in the project, it is found that seaserv is directly import ed in all files. Therefore, there should be a configuration file to map the file defining seaserv in seafile server to the seahub project path.

The speculation is related to the search path during import.

Yes, sys path. Insert for full-text search, found in setting Py file exists

central_conf_dir = os.environ.get('SEAFILE_CENTRAL_CONF_DIR', '')
...
try:
    if os.path.exists(central_conf_dir):
        sys.path.insert(0, central_conf_dir)
    import seahub_settings

That is, the search path during import is added, so it can be concluded that the seaserv module is from central_conf_dir imported, while central_conf_dir comes from the environment variable; Because SEAFILE_CENTRAL_CONF_DIR is obviously not the environment variable of the system itself, so it is speculated that it is set in seafile server.

Retrieve seafile in the server project_ CENTRAL_ CONF_ DIR

SEAFILE_CENTRAL_CONF_DIR

In seafile controller C in the document

setup_env ()
{
    g_setenv ("CCNET_CONF_DIR", ctl->config_dir, TRUE);
    g_setenv ("SEAFILE_CONF_DIR", ctl->seafile_dir, TRUE);
    g_setenv ("SEAFILE_CENTRAL_CONF_DIR", ctl->central_config_dir, TRUE);
    g_setenv ("SEAFILE_RPC_PIPE_PATH", ctl->rpc_pipe_path, TRUE);

    char *seahub_dir = g_build_filename (installpath, "seahub", NULL);
    char *seafdav_conf = g_build_filename (ctl->central_config_dir, "seafdav.conf", NULL);
    g_setenv ("SEAHUB_DIR", seahub_dir, TRUE);
    g_setenv ("SEAFDAV_CONF", seafdav_conf, TRUE);

    setup_python_path();
}

At seaverctl Py file

class ServerCtl(object):
    def get_seaserv_envs(self):
        envs = dict(os.environ)
        envs.update({
            'SEAFILE_CENTRAL_CONF_DIR': self.central_conf_dir,
            'CCNET_CONF_DIR': self.ccnet_conf_dir,
            'SEAFILE_CONF_DIR': self.seafile_conf_dir,
        })
        return envs

In run Py file

def start_and_test_with_db(db):
    fileservers = ('go_fileserver', 'c_fileserver')
    for fileserver in fileservers:
        shell('rm -rf {}/*'.format(INSTALLDIR))
        info('Setting up seafile server with %s database, use %s', db, fileserver)
        server = ServerCtl(
            TOPDIR,
            SeafileServer().projectdir,
            INSTALLDIR,
            fileserver,
            db=db,
            # Use the newly built seaf-server (to avoid "make install" each time when developping locally)
            seaf_server_bin=join(SeafileServer().projectdir, 'server/seaf-server')
        )
        server.setup()
        with server.run():
            info('Testing with %s database', db)
            with cd(SeafileServer().projectdir):
                shell('py.test', env=server.get_seaserv_envs())

In the above three files, the environment variable seafile is created_ CENTRAL_ CONF_ Dir and update it according to the actual operation of the project

• SEAFILE_CENTRAL_CONF_DIR is: '< seafile Path > / conf‘

run-serhub.sh

. setenv.sh

./manage.py runserver

The purpose of this document is to run the seahub project, where/ manage.py runserver is used to run django projects, so it is speculated that setenv The code you are looking for exists in the SH file

###　setenv.sh

export CCNET_CONF_DIR=/home/plt/dev/ccnet/seafile/tests/basic/conf2
export SEAFILE_CONF_DIR=/home/plt/dev/ccnet/seafile/tests/basic/conf2/seafile-data
export PYTHONPATH=/opt/lib/python2.6/site-packages:thirdpart:$PYTHONPATH

This file indicates ccnet configuration directory, seafile configuration directory and python path. Through previous analysis, it is known that ccnet is an internal RPC service process, so it can be basically determined that the above speculation is correct.

Libevhtp overview

libevent

In readme, developers call libevhtp an alternative to libevet's existing HTTP API. Therefore, to understand libevhtp, you first need to understand libevent. The introduction in Baidu Encyclopedia is as follows:

Libevent is a lightweight open source high-performance event notification library written in C language, There are the following highlights: event driven (event driven), high performance; lightweight, focusing on the network, not as bulky as ACE; the source code is quite refined and easy to read; cross platform, supporting Windows, Linux, * BSD and Mac Os; supporting a variety of I/O multiplexing technologies, such as epoll, poll, dev/poll, select and kqueue; supporting I/O, timer and signal events; registering event priority.
Libevent has been widely used as the underlying network library; For example, memcached, Vomit, Nylon, Netchat, and so on.

The official website is as follows:

The libevent API provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout has been reached. Furthermore, libevent also support callbacks due to signals or regular timeouts.

According to these introductions, we can know the function of libevent: when a specific event of a file descriptor occurs or a timing event occurs, libevnet will automatically execute the callback function specified by the user to process the event.

• File descriptor: the medium through which the kernel accesses files. It is a nonnegative integer.

The application scenario of libevent is the event notification of the network server

However, libevent does not consider creating a complete HTTP server when it is built. At the same time, it also has some defects in use. Therefore, libevhtp appears. It tries to solve the problems of the former and provides the backward compatibility of libevent.