This article was published on Byte cloud official account

preface

Some time ago, due to the upgrade of an SDK used by the project, when running a use case using PyCharm+unittest, it can run and output results, but it has been unable to exit the use case. With the in-depth investigation, it is found that the threads in this SDK are "causing trouble".

Reproduce with simple code

For simplicity, the following code (Python 2) contains simple thread logic and a use case to reproduce the problems encountered:

# coding: utf-8
import threading
import time
import unittest


def tick():
    while True:
        print('tick')
        time.sleep(3)


t = threading.Thread(target=tick)
t.start()


class TestString(unittest.TestCase):
    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

This code starts a thread and outputs tick every 3 seconds. On the other hand, a use case is defined to determine the upper() method of the string. If the thread logic is deleted, the use case can end normally; On the contrary, PyCharm shows that the use case is executed successfully, but it has been unable to exit the use case, as shown in the following figure:

Why not quit?

Before running the use case, a new thread must be started to execute the tick() function. Since this function uses the while loop to continuously output strings, it is not difficult to infer that the use case framework has been waiting for the end of the thread when exiting, resulting in the failure of the use case to exit.

To test this idea, take a look at the entry code of the PyCharm run case. Different operating systems, PyCharm (community version, professional version) and use case entry file paths under single test framework are different. The path of the use case entry file for unittest in PyCharm Community Edition on Mac is "/ applications / PyCharm CE. App / contents / plugins / Python CE / helpers / PyCharm / _jb_unittest_runner. Py". The content of the file is as follows:

# coding=utf-8
import os
import sys
from unittest import main

from _jb_runner_tools import jb_start_tests, jb_doc_args, JB_DISABLE_BUFFERING, PROJECT_DIR
from teamcity import unittestpy

if __name__ == '__main__':
    path, targets, additional_args = jb_start_tests()

    args = ["python -m unittest"]
    if path:
        assert os.path.exists(path), "{0}: No such file or directory".format(path)
        if sys.version_info > (3, 0) and os.path.isfile(path):
            # In Py3 it is possible to run script directly which is much more stable than discovery machinery
            # For example it supports hyphens in file names PY-23549
            additional_args = [path] + additional_args
        else:
            discovery_args = ["discover", "-s"]
            # Unittest in py2 does not support running script directly (and folders in py2 and py3),
            # but it can use "discover" to find all tests in some folder (optionally filtering by script)
            if os.path.isfile(path):
                discovery_args += [os.path.dirname(path), "-p", os.path.basename(path)]
            else:
                discovery_args.append(path)
            discovery_args += ["-t", PROJECT_DIR]  # To force unit calculate path relative to this folder
            additional_args = discovery_args + additional_args
    elif targets:
        additional_args += targets
    args += additional_args
    jb_doc_args("unittests", args)
    # Working dir should be on path, that is how unittest work when launched from command line
    sys.path.insert(0, PROJECT_DIR)
    sys.exit(main(argv=args, module=None, testRunner=unittestpy.TeamcityTestRunner, buffer=not JB_DISABLE_BUFFERING))

The previous logic is mainly used to combine the parameters of running cases. For the problems encountered in this paper, the key is the last line main(argv=args, module=None, testRunner=unittestpy.TeamcityTestRunner, buffer=not JB_DISABLE_BUFFERING). Here, main is unittest Testprogram, and the relevant core contents are as follows:

class TestProgram(object):
    """A command-line program that runs a set of tests; this is primarily
       for making test modules conveniently executable.
    """
    USAGE = USAGE_FROM_MODULE

    # defaults for testing
    failfast = catchbreak = buffer = progName = None

    def __init__(self, module='__main__', defaultTest=None, argv=None,
                    testRunner=None, testLoader=loader.defaultTestLoader,
                    exit=True, verbosity=1, failfast=None, catchbreak=None,
                    buffer=None):
        ...
        self.exit = exit
        ...
        self.parseArgs(argv)
        self.runTests()

    def runTests(self):
        ...
        self.result = testRunner.run(self.test)
        if self.exit:
            sys.exit(not self.result.wasSuccessful())

PyCharm's_ jb_ unittest_ runner. When py calls main() (i.e. TestProgram()), no exit parameter is passed in, so the default value is True. At the end of specifying runTests() to run the use case, determine the exit code (0 or 1) according to the use case result, and then call sys Exit() exits the use case execution process. If you break here, you will find that you have been stuck in this sentence.

sys. The function of exit () is to exit the current thread. If other threads do not end, the process will not end naturally. Obviously, the thread where the tick function is located has not been explicitly exited, which leads to the phenomenon that the use case has been successfully run but cannot exit.

How to solve it?

Now that you understand the reason, the solution is ready to come out.

Method 1: thread logic is not executed when running the use case

If the thread logic for executing periodic tasks is not required by the use case, it can be controlled through environment variables, configuration files, etc. when running the use case, the thread logic is not executed, so as to avoid that the use case cannot exit.

Method 2: explicitly exit the process instead of the thread

Use OS_ Exit (n) exits the process. It should be noted that this method does not call to clean up the logic, refresh the standard IO cache, etc. it is usually used in the child process after fork(). Since unit testing has no special requirements for the process, the test cases here generally will not cause side effects.

We can simply modify it_ jb_ unittest_ runner. The last logic of Py explicitly specifies exit=False, that is, unittest is not allowed to call sys Exit(), but call os_ exit().

prog = main(argv=args, module=None, testRunner=unittestpy.TeamcityTestRunner, buffer=not JB_DISABLE_BUFFERING, exit=False)
os._exit(not prog.result.wasSuccessful())

Method 3: exit the thread gracefully

_ jb_unittest_runner.py sends a SIGKILL signal to the current process at the end of the use case. When the use case thread receives this signal, it executes the cleaning logic (if necessary) to exit gracefully, and then exit the process. This method will be discussed in detail in another article. This article knows this idea.

summary

If the thread logic of running periodic tasks is triggered when executing the use case through PyCharm, the use case execution will be completed but cannot exit. Reason in. Sys Exit () is used to exit the current thread instead of the process. If a thread does not exit, the process cannot exit. There are no more than three solutions: do not execute thread logic, exit the process or exit the thread gracefully.