Customize Test Run Templates

Test Run Templates can save a great deal of time over time. You configure the properties of the template once, and then all Test Runs based on the template are preconfigured with the same properties (some of which Test Run creators can modify in Test Runs based on the template.)

Access Test Run templates

  1. Open the project containing the Test Cases that will be tested with Test Runs.

  2. In Navigation, click Test Runs. The Test Runs page opens. The top section of the page displays a table of existing Test Runs.

  3. On the toolbar of the Test Runs page, click Manage Templates. The table of Test Runs is replaced by a table of Test Run Templates.

    You can filter Test Run Templates using query bubbles.

A Test Run Template is essentially a Test Run, with configurable properties and page design. The main difference is that a template itself cannot be executed. Only Test Runs created from it can be executed. So your goal in customizing a Test Run Template is to set it up exactly as if it were a Test Run. You can customize any existing Test Run Template by modifying the Test Run Properties, or by modifying the underlying Widgets and so forth that comprise the Test Run page, or both.

Any existing Test Run Template or Test Run can serve as the basis for a Test Run Template. To create one from an existing template, simply create a copy of the existing template and customize that. If you want to customize one of Polarion's default Test Run Templates, always create a copy for customization so you don't lose the default template. To create a Test Run Template from an existing Test Run, simply open it, click (Actions), and select Save As Template.

Create a copy of a Test Run Template to customize:

  1. Access Test Run Templates as described above, and select the template you want to duplicate and customize in the table of Test Run Templates.

  2. On the toolbar of the detail of the selected template (bottom part of the page), click (Actions) > Duplicate Template or Save as Template. The text of the menu item depends on the Polarion version you are using, and/or the type of the template's page: LiveReport or Classic Wiki.

  3. In the dialog box, provide a unique ID for the new Test Run Template and a human-friendly title, and then click Duplicate Test Run Template.

After creating the duplicate Test Run Template, you can proceed to modify it as described in the next sections. If workflow is configured for the Test Run type, the Duplicate Template action resets the Status to the configured initial status, and invokes the configured initial workflow action, if any.

Modify Test Run Template Properties

Test Run Properties include such data as Test Run type, testing platform and environment, and Test Parameters. Every Test Run created using the modified Test Run Template will have the properties and values as specified in the template. A simple graphical user interface is provided for editing the template properties.

  1. Access a Test Run Template as previously described.

  2. With the desired Test Run Template selected in the table of templates, click Properties on the toolbar of the template detail pane (lower part of the page).

You can edit the title and the property fields in place, or click Edit to place all non-read only fields into edit mode.

The values for some properties like test Type are defined in the global or project Testing configuration in Administration. You can select from configured values, but you cannot change the values themselves. For more information, see Administrator's Guide+ Configure Testing.

In a Test Run template, the Status field is not editable if a workflow configuration for Test Runs exists.

Specifying Test Case selection

In the Test Run Template properties, it is important to specify the method of populating new Test Runs (created by users and based on the template) with Test Cases to be executed. You do this by setting a value for the Select Test Cases field. The field presents the following list of options:

  • Manually: Creators of new Test Runs based on the template can manually select the Test Cases to be executed, or the Test Cases may be manually selected by the author of the Test Run template, in which case they are preselected when a new Test Run is created from the template. When this option is selected a multivalued field named Project Span appears. If you only need to select Test Cases from the current project, you can leave that field empty. If you want to be able to select Test Cases that are defined in one or more different projects, then specify those projects in Project Span. If, in addition to the other projects, you also want to select Test Cases from the current project, then you must include it as one of the values specified in the Project Span field.

  • By Query on Create: Test Cases are selected by a query which automatically runs when a new Test Run based on the template is created. The set of waiting Test Cases of the Test Run is static and does not change if new Test Cases that meet the query criteria are added after a Test Run is created, but before it is executed. Nor will any be added to the set if the Test Run is executed multiple times, even if new Test Cases that would meet the criteria have subsequently been added in the system since the last execution of the Test Run.

    When you select this option you must specify Lucene query syntax items in the Query field that will select the Test Cases that testers must execute in Test Runs based on the Template. You must specify this query before you can save changes to the Template.

  • By Query on Execute: Test Cases for Test Runs are selected by a query which automatically runs when a Test Run based on the template is executed. The set of Test Cases waiting to be executed is not static. Waiting Test Cases are taken from the current set of Test Cases in the system meeting the query criteria at the time a user executes the Test Run, and could even change during the execution each time the view refreshes.

    When you select this option you must specify Lucene query syntax items in the Query field that will select the Test Cases that testers must execute in Test Runs based on the Template. You must specify this query before you can save changes to the Template.

    When this option is selected the multivalued field Project Span appears. If you only need to select Test Cases from the current project, you can leave that field empty. If you want to be able to select Test Cases that are defined in one or more different projects, then specify those projects in Project Span. If, in addition to the other projects, you also want to select Test Cases from the current project, then you must include it as one of the values specified in the Project Span field. The query entered in the Query field should be valid for all specified projects, otherwise some desired Test Cases may not be added to the Test Run, or some Test Cases may be added that are not relevant. For example, if you specify a query like (type:testcase AND status:active AND testType:(manual user)), but in one of the projects the test type is configured as manual ui and not manual user, Test Cases from that project are not be added to your Test Run.

  • From LiveDoc on Create: All Test Cases contained in the specified Document are selected for execution when a new Test Run based on the template iscreated. The set of waiting Test Cases of the Test Run is static and does not change if new Test Cases are added to the Document after a Test Run is created, but before it is executed. Nor will any be added to the set if the Test Run is executed multiple times.

    Specify the Document in the required Document field of the Test Run Template properties. Select a document from the drop-down menu. Preview it by clicking on the icon beside it, or just start typing a word contained within the document's name. Documents located outside the default Space will include the Space name before it. (Separated by a "/"). You can optionally specify a query in the Query field to select a subset of the Test Cases contained in the specified Document. For example, you might query for Test Cases with a Status value that is not "Draft", such as NOT status:draft.

    Figure 20.5. Dynamic Document Selector

    If the list of documents is extensive and covers several Spaces, just type the Space name. Only documents within that space are listed.

  • From LiveDoc on Execute: Test Cases appearing in the specified Document, including Test Cases referenced from another Document, are selected for execution when a Test Run based on the template is executed. The set of Test Cases waiting to be executed is not static. Waiting Test Cases are taken from the current set of Test Cases appearing in the Document at the time a user executes the Test Run, and could even change during the execution each time the view refreshes. Referenced Test Case items can be from projects other than the Test Run and the specified Document, or frozen to a particular revision of the referenced Test Case, or both.

    Specify the Document in the required Document field of the Test Run Template properties. (See From LiveDoc on Create for the drop-down box's behavior.) You can optionally specify a query in the Query field to select a subset of the Test Cases contained in the specified Document. For example, for a Smoke test, you might query for open manual type Test Cases, like status:open AND type: manual.

  • Automation: Test Runs based on the template are created by an automated process and results of test execution are imported to Polarion.

Caution:

For 'By Query on Execute' or 'From LiveDoc on Execute' selection types. (Dynamic Test Runs):

  • When closing dynamic Test Runs by setting a previously unset finished date, usually by a workflow action, the number of waiting Test Cases gets stored in the Test Run as empty test records and no longer changes.

  • When reopening a dynamic Test Run, by clearing a previously set "Finished On" date – usually also by a workflow action, the number of waiting Test Cases becomes live again and corresponds to the current number of unexecuted Test Cases matching the Test Run query.

  • The query to retrieve or count awaiting Test Cases will therefore differ for finished and unfinished dynamic Test Runs.

Modify a Test Run Template's page

Test Runs and Templates are implemented as LiveReport type Pages. You can use the full range of editing features and Widgets to customize the layout and content of a Test Run Template.

  1. In Navigation, select Test Runs.

  2. On the page toolbar, click Manage Templates. Then, in the table of Test Run Templates, select the one you want to modify.

  3. On the toolbar of the detail pane, click (Actions) Customize Test Run Page.

An instance of the Page Designer opens in the detail panel of the Page. You can then modify the Page according to your needs. For more information, see: Working With Pages.