Introduction

A complete test management tool called TestRail was created to assist teams in effectively planning, coordinating, and monitoring their software testing activities. Testers may build, organize, and run test cases on this unified platform. They can also track test results and report issues. TestRail automation enhances the speed and productivity of the tester.

Benefits of Test Rail

Centralized Test Repository: Teams can easily access and manage their testing assets in one location using TestRail’s role as a centralized repository for all test cases. This streamlines the testing process by doing away with the requirement for disorganized papers or spreadsheets.

Test Case Organization: TestRail enables testers to arrange test cases logically into sections and suites, which facilitates the management and classification of tests according to various parameters like features, modules, or priorities.

Test Execution and Tracking: TestRail offers tools for organizing and carrying out test runs. These tools enable testers to designate test cases for designated team members, monitor the status of tests, and methodically record test outcomes. Teams can use this to guarantee comprehensive test coverage and track the progress of testing tasks in real time.

Reporting: TestRail offers customizable reporting features that enable teams to produce meaningful metrics and reports to monitor the status of testing, spot bottlenecks, and make informed decisions.

Integration Capabilities: TestRail allows teams to automate and optimize their testing workflows by integrating with a wide range of testing tools, issue tracking systems, and continuous integration platforms. Integrations with tools like Jira, Jenkins, and Selenium facilitate automation and provide a unified view of development and testing activities.

All things considered, TestRail is essential for teams to easily manage their testing procedures, foster better teamwork, guarantee test quality, and produce software that meets market demands.

Components of TestRail

Test-rail test management typically includes the following components:

Test Cases: test-rail enables users to effectively generate and arrange test cases. The cornerstone of the testing procedure, test cases include comprehensive guidelines on how to carry out tests, anticipated results, and any necessary preconditions or prerequisites.

Test Suites: test-rail users can arrange related test cases into test suites, which they can then further organize into sections. While sections enable users to organize test cases according to criteria like functionality, features, or modules, test suites aid in maintaining structure and organization within a project.

Test Runs: In test-rail, a test run is a group of related test cases that are run simultaneously. Users can plan and execute test runs for a range of purposes, such as regression, feature, and release testing. Teams can assign tests to team members, oversee their execution, and keep tabs on their progress with the aid of test runs.

Test Plans: By combining several test runs into one, test plans offer a higher-level overview of testing activity. Teams may better organize their testing efforts across several projects, releases, and milestones by using test plans. They let users specify the goals of the tests, allocate resources, and plan the timing of test runs.

Test Results: TestRail enables users to methodically record and monitor test outcomes. Test results include details regarding test execution outcomes, such as pass/fail status, comments, and attachments. Users have two options for logging test results: either manually or automatically, by integrating TestRail with automated testing solutions.

Management of Defects: TestRail has tools for handling problems and flaws found during testing. Within TestRail, users can immediately record problems and provide information about the defect, including its severity, priority, reproducibility steps, and related test cases. For easy defect management, test-rail also provides connectivity with well-known issue tracking solutions.

Reports and Metrics: TestRail offers capabilities for reporting and metrics that let users assess the status of testing, spot patterns, and make informed decisions based on data. To display test results, manage important data, and communicate insights with stakeholders, users can create personalized reports, graphs, and dashboards.

Customization and Configuration: test-rail provides the adaptability needed to customize and configure it to meet the specific requirements of various teams and projects. To fit TestRail’s unique requirements and testing procedures, users can alter fields, workflows, templates, and permissions.

TestRail Automation: Execution status

In this article, we will look at how to completely automate test execution through TestRail automation. It is possible to include attachments, such as a log of the test result, in addition to status or remark updates. The test rail provides test-rail API endpoints for managing test cases and importing execution results from automated processes. For the automation, we’ll use the Python Request Module, although you can use any language or program that supports HTTP requests.

For details on Test Rail API, which is used for TestRail Automation, visit https://support.testrail.com/hc/en-us/articles/7077083596436-Introduction-to-the-TestRail-API

TestRail anticipates that the API’s standard HTTP basic authentication will be used to supply the authentication credentials. TestRail offers two methods for authenticating API queries.

• Username and password
• Username and API key

The step-by-step guide for TestRail Automation

• To create TestRail API key, please login to TestRail site and go to ‘My Settings’ under the profile. Click on the API key and create a key.

• Create a sample Test Plan, test run included in the test plan and add some cases to the test run.
• Download and install python from https://www.python.org/downloads/
• Install ‘requests’ module of Python which simulates the GET, PUT, POST etc. HTTP commands.
  • pip install requests
• Download the Testrail API finding zip  file from https://support.testrail.com/hc/en-us/articles/7077135088660-Binding-Python#01G68HGSB80CB8ZNYZ0KTA71VC
• Extract the downloaded file and copy testrail.py file from “testrail-api-master\testrail-api-master\python\3.x” folder. This file contains functions to access TestRail’s API from Python.
• To identify specific projects, cases, test runs, and other information, you must include an ID value in the request URL of TestRail API services. To find the ID for project, test case, etc. you need to open the project/test case/test plan and you can find the ID as below, We need to develop the code to find the IDs

• Now, to update the result, we must complete the code. The run id for a case varies, but the case ID is always fixed, so to locate it we must find the case ID from the test run and update the status.

Below is the configuration file to update the specific test run.

testrail_user = ''
testrail_password = ''
testrail_project = Automation
testrail_testplan = 'Automation sample test run'
testrail_testrun = 'Test Run 11/28/2019'

•  To initialize the APIClient mentioned in testrail.py,

def get_testrail_client(user, passwd):
    #client = testrail.APIClient('https://irdeto.testrail.com/')
    client = APIClient(testrail_rul)
    client.user = user
    client.password = passwd
    return client

• TestRail Automation: Sample code to get the project ID.

def get_project_id(client, project_name):
    "Get the project ID using project name"

    project_id=None
    data = client.send_get('get_projects')
    proj_dic = data['projects']
    for project in proj_dic:
        if project['name'] == project_name:
            project_id = project['id']
            print("project:",project_id)

    return project_id

• Sample code to get the run ID from that project.

def get_run_id(client, test_run_name, project_name, test_plan_name):
    "Get the run ID using test name and project name"
    run_id = None
    project_id = get_project_id(client,project_name)
    if(test_plan_name == ''):
        try:
            test_runs_data = client.send_get('get_runs/%s' % (project_id))
            test_runs = test_runs_data['runs']
        except:
            print ('Exception in getting test run information.')
            return None
        else:
            for test_run in test_runs:
                if test_run['name'] == test_run_name:
                    run_id = test_run['id']
                    break
            return run_id
    else:
        try:
            test_plans_data = client.send_get('get_plans/%s' % (project_id))
            test_plans = test_plans_data['plans']
            for test_plan in test_plans:
                if test_plan['name'] == test_plan_name:
                    plan_id = test_plan['id']
                    print("plan:",plan_id)
                    break
            
            test_plan_details = client.send_get('get_plan/%s' % (plan_id))
            for test_plan_entries in test_plan_details['entries']:
                for test_run in test_plan_entries['runs']:
                    if test_run['name'] == test_run_name:
                        run_id = test_run['id']
                        print("run:",run_id)
                        break
            return run_id
        except:
            print ('Exception in getting test run information from test plan.')
            return None

• To update the result, we need to pass the test case ID from the test case script. Sample code for the same.

def update_testresult(client, case_id, run_id, result_flag, msg=""):
    "Update TestRail for a given run_id and case_id"
    
    # Update the result in TestRail using send_post function.
    # Parameters for add_result_for_case is the combination of runid and case id.
    # status_id is 1 for Passed, 2 For Blocked, 4 for Retest and 5 for Failed
    status_id = 1 if result_flag is True else 5

    if run_id is not None:
        try:
            result = client.send_post(
                'add_result_for_case/%s/%s' % (run_id, case_id),
                {'status_id': status_id, 'comment': msg})

        except:
            print ('Exception in update test result in test rail.')
        else:
            print ('Updated test result for case: %s in test run: %s with msg:%s' % (case_id, run_id, msg))

• Now, you need to just call the functions one by one to update the results.

def testrail_update_result(user, passwd, project, testplan, testrun, caseid, result):

        client = get_testrail_client(user, passwd)
        run_id = get_run_id(client, testrun, project, testplan)

        update_testresult(client, caseid, run_id, result, "")

• You can also attach a log with a test result. Sample code to attach a log file.

def attach_log(client, case_id, run_id, logs):
    "Attach log to a test result"
    results = client.send_get('get_results_for_case/%s/%s' % (run_id, case_id))
    for result in results:
        result_id = result['id']
        break;

    # Update test logs to result

    if result_id is not None:
        try:
            result = client.send_post('add_attachment_to_result/%s' % (result_id), logs)
        except Exception, e:
            print 'Exception in attaching log to test result.'
            print 'PYTHON SAYS: '
            print e
        else:

• Verify in your TestRail browser that the mentioned test case should be updated as per your result.

TestRail Automation can be achieved by using the above steps. Along with the submission of the test result you can create test runs and plans programmatically using the TestRail API.

You may like RTM test management execution automation https://dasfascination.com/rtm-test-management-automation/