Parametrized unit tests using pytest for Xpress Insight apps

Community Manager

2 years ago

Unit testing allows developers to test individual components of their program in a systematic and automated manner. In an earlier post, we showed how to set up test mode for Xpress Insight-Python application development and reflected on the importance of running parametrized unit tests in an external environment that does not get deployed to the Insight server. The pytestpackage in Python offers a structure that enables an easy set up and testing of functions and objects in a non-production environment. In this article, we will show how you can use pytestto run parametrized unit tests for your Insight-Python application. You can also find a tutorial in the online documentation.

View the video tutorial on “Parametrized unit tests for Insight-Python apps”

Before you set up automated testing using pytest, make sure you have the pytest package installed, and that you have added the installation path to your environment variables. Then, follow these instructions:

1. In your project root directory, create a new subdirectory named “test”:

this folder will be the location for unit tests. This folder will not be distributed when you deploy the app

2. Within the new “test” subdirectory, create a new Python file:

the name of this file must start with the prefix “test_”(for example, “test_application.py”)

3. In the header of the new Python file, import the pytest and xpressinsight packages:

import pytestimport xpressinsight as xi4. Make your source Python module(s) visible to the test file:

import the nativeosand sys packages and then append the python_source directory to the list of directories the interpreter will search in for modules:

import osimport syssys.path.append(os.path.join(os.path.dirname(__file__),'..','python_source'))from 
application import InsightApp

5. Move the contents of the __main__function from your source Python file to the test_application.py file and redefine it as a standard function (again use the prefix “test_”):

it must accept a tmp_pathparameter, which specifies the name of a temporary directory that pytestwill create for each test (stored in “C:\Users\<USER>\AppData\Local\Temp”). This will avoid permission errors when attempting to the delete/create the default temporary working directory when performing multiple tests in a single script run
write an assert statement as a Boolean expression that tests whether some outcome matches the expectations for a given value (or set of values)

def test_app(tmp_path):
     app = xi.create_app(InsightApp, app_work_dir=str(tmp_path))
     app.call_exec_mode('LOAD')

     app.MaxHighRisk = 0.2
     app.data_connector.save_input()
     app.call_exec_mode('RUN')    
 
assert app.TotalReturn == pytest.approx(11.05)

6. Define the main function of the new test_application.py file to invoke pytest.main()
 to run the tests when the file is run:if __name__ == "__main__":
    pytest.main()

To run the test file, open a terminal prompt and have the newly created “test” subdirectory as the active working directory, then run “pytest”. Pytest will look for all files that start with the prefix “test_” in the active working directory, and then perform tests in all functions that also start with the prefix “test_”:

PROJECTROOTDIR\TEST> pytest

Optionally use the -v flag to show the result (passed or failed) for each test, and the -s flag to show the run log for each test:

PROJECTROOTDIR\TEST> pytest -v -s

Alternatively, run the test_application.py file directly from your IDE. As long as the “test” file suddirectory is the active working directory, pytest will perform the tests and output the corresponding logs.

Manual setup of individual tests might not be suitable for real-world, large-scale applications where tests must run automatically on several files and functions on a frequent basis. Parameterized unit tests using pytest make it possible to automate bulks of tests with multiple parameters, eliminating the need for extensive and redundant test code. To set up parametrized unit tests using pytest, edit the previous test_application.py following the steps below:

1. In your test_application.py file, after the package import statements, insert the decorator @pytest.mark.parametrize

next define the names and values of the parameters that you want to use to validate the test
you can define as many parameters as necessary, and assign them different data types such as string, list, or dictionaries

import osimport sysimport pytestimport xpressinsight as xisys.path.append(os.path.join(os.path.dirname(__file__),'..','python_source'))from application import InsightApp@pytest.mark.parametrize(
    ('input_value', 'expected_output'),
    (
            (0.1, 0.0),
            (0.2, 13.42),
            (0.3, 14.24),
    )
)

2. Change the definition of the test_app function to accept the defined parameters in addition to tmp_path

use the passed values in your script and assert statement. Note that you must use the same variable names as in the decorator

def test_app(input_value: float, expected_output: float, tmp_path: pathlib.Path):
    """ Parametrized testing """

    app = xi.create_app(InsightApp, app_work_dir=str(tmp_path))
    app.call_exec_mode('LOAD')

    app.InputEntity = input_value
    app.data_connector.save_input()
    app.call_exec_mode('RUN')

    assert app.ResultEntity == pytest.approx(expected_output)
3. Run the test_application.py file in your IDE or run “pytest” in a terminal prompt using 
the –v flag to obtain the results for all tests, including whether each test passed or 
failed for each individual instance:PROJECTROOTDIR\TEST> pytest –vView the video tutorial on “Parametrized unit tests for Insight-Python apps”

If you are curious and want to try it yourself, then you are more than welcome to try 
the community edition  of FICO Xpress Insight and implement this tutorial on top of 
the Portfolio Optimization example app using the files available in the FICO Xpress 
Optimization Examples Repository.