I accidentally wrote a Python test loader

Python is a powerful programming language with a vast range of applications. Python is great but for someone like me who is coming from a C/C++ background, Python syntax is bit off. (Yes yes I know, it is just indentation without semicolons (;) and curly braces ({})).

Long story short, I decided to give Python a go, and as you know code will be followed by unit tests and Python has a rich set of unit testing tools. The tool I decided to go with is unittest (read more here). It is an inbuilt module to Python.

It is pretty easy to write unit tests with unittest I will put a minimalistic test code here just to get a sense.

# app/tests/test1.py 
import unittest
class TestGreaterThan(unittest.TestCase):
      def test_three_greater_than_two(self):
          self.assertTrue(3 > 2)
if __name__ == '__main__':
     unittest.main()

Run the test file simply typing python app/tests/test1.py in a command line (If you are in the root directory). If all is well you will see OK in the command line with the time it took to run the test.

Let’s say I want another test but in a different file

# app/tests/nested_directory/test2.py
import unittest
class TestLessThan(unittest.TestCase):
      def test_three_less_than_2(self):
          self.assertFalse(3 < 2)
if __name__ == '__main__':
     unittest.main()

We can run the python command python app/tests/nested_directory/test2.py to run the test.

We have two tests and need a way to combine those results. So how are we going to do this? It is very easy,

python -m unittest discover ./app/tests -v

(make sure you have a __init__.py inside the nested_direcotry folder)

Fortunately or unfortunately I did not find the above command (No need to call me stupid, it was a beginners mistake)

So I looked around on the internet and found this. The way it suggests was,

# app/test_runner.py
import unittest
import tests.test1 as test1
import tests.nested_directory.test2 as test2

loader = unittest.TestLoader() 
suite  = unittest.TestSuite()  
suite.addTests(loader.loadTestsFromModule(test1)) suite.addTests(loader.loadTestsFromModule(test2)) 
runner = unittest.TextTestRunner(verbosity=3) 
result = runner.run(suite)

Run this as python app/test_runner.py and it will show OK with time to run two tests. voilà!

This solution is not extensible since we need to import individual test modules and if we have 100 test files then 100 imports. (Not going to work)

This is where I started writing my own test loader.

Here is my start,

# app/test_loader.py
import os
def load_unit_test_modules(unit_test_directory):
    test_modules = []
    # Load files in the directory
    files_in_directory = os.listdir(unit_test_directory)
    test_files = filter(is_test_file, files_in_directory)
    test_module_names = list(map(map_to_module_name, test_files))
    # Append each test module
    for test_module in test_module_names:
        # Append the module prefix to test module name
        test_modules.append(test_module)
     return test_modules

Code is staright forward right?,

• First I lists the files in the unit_test_directory param
• Then I filter out test files
• Then I map the test files to module names
• Then module names are returned in a list

Let see is_test_file and map_to_module_name functions.

import re
# test file pattern
file_pattern = re.compile("test[a-zA-Z0-9_]*.py")

def is_test_file(file_name): return file_pattern.match(file_name)

This function simply checks if a file in the directory follows test name pattern by using a regex. Nothing fancy.

def map_to_module_name(file_name): return file_name.split('.')[0]

Since the filtered test files have the .py extension and modules do not include that, we need to remove it. map_to_module_name simply does that.

So far so good.

But there is a problem (like always)

We have not considered the nested folders. Let’s append that to existing function.

# app/test_runner.py
import os
def load_unit_test_modules(unit_test_directory, module_prefix=''):
    test_modules = []
    # Load files in the directory
    files_in_directory = os.listdir(unit_test_directory)
    # If no files, return empty list
    if not files_in_directory:
        return []
    nested_directories = filter(is_directory, files_in_directory)
    for nested_directory in nested_directories:
        # nested modules are accessed using '.', 
        # there add the prefix
        nested_test_modules = load_unit_test_modules(
              unit_test_directory.joinpath(nested_directory),
              nested_directory + '.')
        for nested_test_module in nested_test_modules:
            test_modules.append(module_prefix + nested_test_module)
    test_files = filter(is_test_file, files_in_directory)
    test_module_names = list(map(map_to_module_name, test_files))
    
    # Append each test module
    for test_module in test_module_names:
        # Append the module prefix to test module name
        test_modules.append(module_prefix + test_module)
    return test_modules

Let’s note the changes,

• There is a logic to check if there are no files in the unit_test_directory, this is placed because this function is called recursively and we need a guard condition (Otherwise a stack overflow)
• We filter out directories and for each directory we call the function recursively and append modules inside them to test_modules

If you check the function signature there is new change, a new parameter has been added module_prefix . When we access python files inside directories we need to use dot (.) to access the relevant module. Ex: test2.py file is accessed as tests.nested_directory.test2 . The extra param module_prefix is used to achieve this.

See that when calling the function recursively, we append a .

nested_test_modules = load_unit_test_modules(
              unit_test_directory.joinpath(nested_directory),
              nested_directory + '.')

There is a new is_directory function that will return true if there are no extensions in the file name.

def is_directory(file_name): return file_name.find('.') == -1

See the full code for the test loader here.

Now let’s see the test runner code. It is staright forward.

# app/test_runner.py
import unittest
import importlib
from pathlib import Path
from test_loader import load_unit_test_modules
# Define a loader to load files and suite to load suites
loader = unittest.TestLoader()
suite = unittest.TestSuite()
# current_directory = Path(__file__)
current_file = Path(__file__)
current_directory = current_file.parent
unit_test_directory = current_directory.joinpath('tests')
test_modules = load_unit_test_modules(unit_test_directory, 'tests.')
for test_module in test_modules:
    module = importlib.import_module(test_module)
    suite.addTest(loader.loadTestsFromModule(module))
runner = unittest.TextTestRunner(verbosity=2)
result = runner.run(suite)

In a summary

• tests is the test module folder, it is passed to the load_unit_test_modules function
• since the module prefix is also test. we pass it as the second paramater
• Iterate over all the modules and use importlib.import_module function to load the relevant test module

Other parts of the code remains the same.

If you now run python app/test_runner.py you will get the same output as before OK with time to run two tests.

And that’s how I accidently wrote a test loader.

Is that effort all in vain?

No. With bit customization now I have a generic module loader. I can configure types of modules I want to load, what directories I need to check. (At least that’s what I think)

Thank you for reading.

See the full code here.