Testing Services Before Deployment

Iteratively debugging a service by repeatedly deploying and calling entrypoints can become a somewhat slow and tedious process, having to wait a short time for every deploy. To help ensure that services are in a working state before deployment, we recommend that you use an automated testing framework. When creating a new project with mvi init a test/ directory is created. It contains two files: base_functionality_test.py that performes base tests for things that are necessary for these services to work, and user_tests.py for the user to add their own tests.

Writing Tests

The tests use pytest which requires the tests to lay in a folder called tests/, in modules that end with _test.py and the names of the test functions should start with test_. Tests use the assert statement to see if code behaves as expected. Let’s use the hello service that is generated with mvi init for an example test

import logging
from mvi import service
from mvi.communication import notify, Severity

logger = logging.getLogger(__name__)

service.add_parameter("greeting_phrase", "Hello")

@service.entrypoint
def hello(name: str) -> str:
    greeting_phrase = service.get_parameter("greeting_phrase")
    if name == "World":
        notify(
            msg="Someone is trying to greet the World, too time consuming. Skipping!",
            severity=Severity.WARNING,
            emails=None,
        )
        return "Greeting failed"
    logger.info(f"Greeting someone with the name: {name}")
    return f"{greeting_phrase} {name}"


if __name__ == "__main__":
    service.run()

The output of the hello() function can be tested with the following function

# <project_path>/tests/user_tests

def test_hello():
    # Test that service.hello("Bob") returns what is expected
    assert service.hello("Bob") == "Hello Bob"

We can run the tests using:

>>> mvi test <project_path> 

Which is equivalent to:

>>> cd <project_path> 
>>> python3 -m pytest . 

Functions That Do Not Work Locally

There are three ways to run an MVI service. We have seen how to deploy and run your services to a manager using mvi deploy many times now, but you could also run it with python <project_path>/service.py in which case it will be run as it’s own webserver on localhost if service.py ends with mvi.service.run(). If it does not have mvi.service.run() it will behave like a normal python module and the functions therein can be used however you’d like. The if __name__ == "__main__" clause that surrounds mvi.service.run() allows us to import service.py freely and use it’s functions. However, the SDK will only have full functionality if run on a manager. The table below shows the SDK functions that cannot run locally without a manager:

Functions	Module	Localhost	Manager
mvi.communication.call_service()	No	No	Yes
mvi.communication.notify()	No	No	Yes
mvi.service.store()	No	Yes	Yes
mvi.service.call_every()	No*	Yes	Yes

*Will not repeat without starting the service with mvi.service.run() but it won’t cause any errors

Mocking Functions to Run Services Locally

To be able to test services that depend on some of the functions listed above, we have to use something called mocking, which means to templorarily replace code to do something else. In the case of the hello service we would get an error if notify() was called. We could mock notify() to do nothing and then check if it was called to test that a notification would be posted to the manager. In mvi.testing there is a function patch() that is included in MVI for convenience from unittest.mock and is used for mocking

# <project_path>/tests/user_tests
import service
from mvi.testing import patch


def test_notify_called():
    # Test that notify() is called when running service.hello("World")
    with patch("service.notify") as notify:
        service.hello("World")
        notify.assert_called()

We import service.py and patch() and then in the test function we create a session scope for the patch to revert the change back once we are done. In patch() we give a path to the function to mock, in this case the notify() function that we imported in service.py. We can then call the hello() function as normal and use the mock object notify to check if notify() was called in service.hello(). The process for mocking store() is very similar, because neither store() or notify() have any return values.

call_service() on the other hand, will often have return values and testing a service that uses it requires you to mock a problem specific return value. Let’s take a look at an example service

import logging
from mvi import service
from mvi.communication import call_service

logger = logging.getLogger(__name__)

@service.entrypoint
def ping_service(service_name: str) -> str:
    logger.info(f"Pinging: {service_name}")
    response = call_service(
        service_name=service_name,
        entrypoint_name="ping",
    )
    return response


if __name__ == "__main__":
    service.run()

This service has an entrypoint that pings another service to see if it is responding. To create an automated test for this we need to mock call_service() to return something that we would expect the ping() entrypoint of the target service to return in a real scenario

# <project_path>/tests/user_tests
import service
from mvi.testing import patch


def test_ping_service():
    with patch("service.call_service") as call_service:
        call_service.return_value = "Responding"
        service.ping_service("test_service")
        call_service.assert_called()

Here we mock call_service() in the same way as we did with notify() but we add a return value to the mock object call_service to make it return that when it is called in service.ping_service().

The patch() function returns an object of the Mock class from unittest.mock.

More Testing

Refer to the pytest documentation and its many addons if you would like to do more testing. To learn more about mocking and the patch() function, take a look at unittest.mock.