Overview of pytest framework?

Pytest is a testing framework for Python that allows you to write straightforward and expandable tests. It offers functions including parameterization, automatic test discovery, fixture support, and thorough test reporting. Pytest provides straightforward syntax and robust features for testing Python programs to simplify and improve the efficiency of test development. In the Python community, it’s frequently used for testing a wide range of projects, from simple scripts to large-scale apps. In this article we will learn to use pytest marker feature.

What is pytest marker feature or pytest.mark decorator?

One of the most useful features in the pytest testing framework is the pytest.mark decorator, which lets you add markers to your tests and regulate how they behave. Tests can be categorized using markers, and they can even be conditionally executed based on user-defined criteria or skipped entirely under some circumstances.

An overview of some of the pytest.mark decorator’s features is provided below:

Test Categorization: Tests can be categorized using markers, which makes it simpler to run test groups or to exclude test groups. As an example, you may designate some tests as sanity tests (@pytest.mark.sanity) or regression tests (@pytest.mark.regression).

Test Skipping: Under some circumstances, markers can be used to bypass some tests. For example, you may have an unimplemented test or one that relies on unavailable external resources. To skip such tests, mark them with @pytest.mark.skip or @pytest.mark.skipif(condition).

Test Failure: Expected failure tests are marked with the pytest.mark.xfail marker in the Pytest testing framework. This comes in handy when you wish to write and run tests to keep track of features that are either not yet implemented or are known to be broken. When running tests with the ‘pytest xfail’ marker, the tests will be executed, but if any of them fail, it won’t be considered a test failure, and the reason for the failure will be reported.

Conditional Execution: Tests can also be conditionally executed using markers in accordance with predetermined standards. To execute the same test with different input parameters, for example, use @pytest.mark.parametrize. Alternatively, you can use custom markers with conditional logic to run tests only when certain environment variables or configuration settings are met.

You can check the usage of all the built-in markers on the command line. You can visit https://docs.pytest.org/en/7.1.x/example/markers.html to learn more on pytest marker

Below is a example sample code which uses different markers like @pytest.mark.sanity, @pytest.mark.regression, @pytest.mark.skip, @pytest.mark.parametrize

import pytest

@pytest.mark.sanity
def test_sanity1():
    print("san1")

@pytest.mark.regression
def test_regression():
    print("reg1")

@pytest.mark.sanity
def test_sanity2():
    print("san2")

@pytest.mark.skip(reason= "as feature is not implemented")
def test_feature():
    print("feature1")

@pytest.mark.parametrize("input_val, output", [(1,3),(3,5),(5,7)])
def test_add(input_val, output):

The whole execution results:

C:\pytest>python -m pytest -s test_marker.py -v -s --disable-pytest-warnings
========================================================================= test session starts =========================================================================
platform win32 -- Python 3.8.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python38\python.exe
cachedir: .pytest_cache
rootdir: C:\pytest
collected 7 items

test_marker.py::test_sanity1 san1
PASSED
test_marker.py::test_regression reg1
PASSED
test_marker.py::test_sanity2 san2
PASSED
test_marker.py::test_feature SKIPPED (as feature is not implemented)
test_marker.py::test_add[1-3] PASSED
test_marker.py::test_add[3-5] PASSED
test_marker.py::test_add[5-7] PASSED

============================================================== 6 passed, 1 skipped, 3 warnings in 0.02s ===============================================================

The execution results with the sanity marker are:

C:\pytest>python -m pytest -s test_marker.py -m sanity -v -s --disable-pytest-warnings
========================================================================= test session starts =========================================================================
platform win32 -- Python 3.8.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python38\python.exe
cachedir: .pytest_cache
rootdir: C:\pytest
collected 7 items / 5 deselected / 2 selected

test_marker.py::test_sanity1 san1
PASSED
test_marker.py::test_sanity2 san2
PASSED

============================================================= 2 passed, 5 deselected, 3 warnings in 0.00s =============================================================

The execution result with parametrized marker:

C:\pytest>python -m pytest -s test_marker.py -m parametrize -v -s --disable-pytest-warnings
========================================================================= test session starts =========================================================================
platform win32 -- Python 3.8.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python38\python.exe
cachedir: .pytest_cache
rootdir: C:\pytest
collected 7 items / 4 deselected / 3 selected

test_marker.py::test_add[1-3] PASSED
test_marker.py::test_add[3-5] PASSED
test_marker.py::test_add[5-7] PASSED

============================================================= 3 passed, 4 deselected, 3 warnings in 0.02s =============================================================

Sometimes we may need to include a test case in both sanity and regression test run. Multiple pytest marker can also be declared with a function. You can call sanity and regression both custom marker with a single test case.

@pytest.mark.sanity
@pytest.mark.regression
def test_regression():
    print("reg1")

The above test case will run with both the sanity and regression pytest marker.

C:\pytest>python -m pytest -s test_marker.py -m sanity -v -s --disable-pytest-warnings
========================================================================= test session starts =========================================================================
platform win32 -- Python 3.8.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python38\python.exe
cachedir: .pytest_cache
rootdir: C:\pytest
collected 7 items / 4 deselected / 3 selected

test_marker.py::test_sanity1 san1
PASSED
test_marker.py::test_regression reg1
PASSED
test_marker.py::test_sanity2 san2
PASSED

============================================================= 3 passed, 4 deselected, 4 warnings in 0.02s =============================================================


C:\pytest>python -m pytest -s test_marker.py -m regression -v -s --disable-pytest-warnings
========================================================================= test session starts =========================================================================
platform win32 -- Python 3.8.2, pytest-8.1.1, pluggy-1.4.0 -- C:\Program Files\Python38\python.exe
cachedir: .pytest_cache
rootdir: C:\pytest
collected 7 items / 6 deselected / 1 selected

test_marker.py::test_regression reg1
PASSED

============================================================= 1 passed, 6 deselected, 4 warnings in 0.02s =============================================================

You may like the TestRail and RTM test execution automation blog https://dasfascination.com/rtm-test-management-execution-automation/ and https://dasfascination.com/streamline-test-management-testrail-automation/

Conclusion

Overall, pytest marker are an extremely useful tool that helps developers collaborate more effectively, improves code quality, and expedites the testing process. They offer a practical means of classifying exams, managing expectations, and regulating their conduct. You may build reliable and manageable test suites that provide users with confidence in the caliber and dependability of your software by making good use of markers.