Evaluations

In this book, all questions about evaluation are answered. Please note that "evaluation" may have a different name in your case, such as study or trial.

General

General

What is an evaluation?

evaluations.png

An evaluation can be viewed as a collection of tests and itself contains only a limited amount of data such as a name, start and end, etc. In addition, templates for testers, test objects and categories can optionally be entered, which can then be used in a test. This saves you double work if the tests are structured similarly/identically. The status of an evaluation is automatically determined and displayed based on the status of the tests.

General

How do I create a new evaluation?

Planer-UI-Evaluation-anlegen.png

To create a new evaluation, click on Evaluations in the left navigation bar. Then click on the button at the top right called "+ Create evaluation" to open a popup window.

Basic data

*Alternative reviewers can be used if, for example, several people are testing a test object together but only have one device available. A tester who is not logged in on the device now wants to give feedback and it should also be assigned to him in the system. In this case, the entering tester selects the appropriate name under "Alternative assessor".

Test objects

The screen is divided in two. On the left side you will find all objects from the data pool. You can also search this list. You can add a test object from the data pool to the evaluation by double-clicking or using drag & drop. You can also create a new test object.

On the right side you will find all test objects that are assigned to the evaluation. You will see the picture, the name, the key value and the attributes. You can also search this list. If an added test object is not used in at least one test, you can remove the test object from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm). With a click on the pencil symbol you can edit details about the vehicle and, for example, enter values for the attributes assigned to the vehicle.

Tester

The screen is also divided into two here. On the left side you will find all available testers from the data pool and you can search this list. You can add a tester from the data pool to the evaluation by double-clicking or using drag & drop. You can also create a new tester.

On the right side you will find all testers who are assigned to the evaluation. You will see the first name, last name, company/department, e-mail address and whether the user has permissions in the currently selected division. You can also search this list. If an added tester is not yet assigned to at least one test, you can remove the tester from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm).

Categories

The screen is also split in two here. On the left you will find all available categories and category sets. You can search this list. You can add a category to the evaluation by double-clicking or using drag & drop.

On the right side you will find all categories and subcategories that are assigned to the evaluation. If added, you will see an icon and the name of the category. If a down arrow icon appears in front of the icon, it is a category with subcategories. This means that this list can be expanded. If there is no arrow, it is a normal category or subcategory. In the end, a category hierarchy can be built that contains many sub-levels. If an added category is not yet assigned to at least one test, you can remove the category from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm).


Clicking on Save closes the pop-up window and takes you to the overview page.


Words marked in bold indicate mandatory entries.

General

How do I edit an evaluation?

Planer-Ui-edit-evaluation-adhoc-tests.png

To edit an existing evaluation, click Evaluations in the left navigation bar. Then click on the eye icon or double-click on the corresponding line. Now a new page opens. You will see the overview page of the selected evaluation. It is divided into an upper and a lower section. In the upper section you will find some information like the current status of the evaluation. The lower section is divided into four categories:

You can now switch to edit mode to be able to edit the evaluation. To do so, click on the "Edit" button in the upper right corner. A popup with four tabs will open.

Basic data

*Alternate evaluators can be used if, for example, several people are testing a test object together but only have one device available. A tester who is not logged in on the device now wants to give feedback and it should also be assigned to him in the system. In this case, the entering tester selects the appropriate name under "Alternative assessor".

Test objects

The screen is divided in two. On the left side you will find all objects from the data pool. You can also search this list. You can add a test object from the data pool to the evaluation by double-clicking or using drag & drop. You can also create a new test object.

On the right side you will find all test objects that are assigned to the evaluation. You will see the picture, the name, the key value and the attributes. You can also search this list. If an added test object is not used in at least one test, you can remove the test object from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm).

Tester

The screen is also divided into two here. On the left side you will find all available testers from the data pool and you can search this list. You can add a tester from the data pool to the evaluation by double-clicking or using drag & drop. You can also create a new tester.

On the right side you will find all testers who are assigned to the evaluation. You will see the first name, last name, company/department and e-mail address. You can also search this list. If an added tester is not yet assigned to at least one test, you can remove the tester from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm).

Categories

The screen is also split in two here. On the left you will find all available categories and you can category sets. You can search this list. You can add a category to the evaluation by double-clicking or using drag & drop.

On the right side you will find all categories and subcategories that are assigned to the evaluation. If added, you will see an icon and the name of the category. If a down arrow icon appears in front of the icon, it is a category with subcategories. This means that this list can be expanded. If there is no arrow, it is a normal category or subcategory. In the end, a category hierarchy can be built that contains many sub-levels. If an added category is not yet assigned to at least one test, you can remove the category from the evaluation by clicking on the X symbol (wait a moment until the symbol turns red and press again to confirm).

Clicking on Save closes the pop-up window and takes you to the overview page.

Words marked in bold indicate mandatory entries.

General

Status of an evaluation

evaluations.pngThe status of an evaluation is set automatically based on the status of the tests.

Tests

Tests

How do I create a new test?

create-test.png

To create a new test, you must first select the desired evaluation. Within an evaluation you will find four different tabs: tests, testers, test objects and categories. At this point, stay in the already active tab "Tests" and click on the button "Create test". A new popup will open with six different tabs.

Remember to click on the Save button after the editing is done.

Tests

How do I edit an existing test?

edit-test.png

To edit an existing test, you must first select the corresponding evaluation. In the detailed view of the evaluation, you then double-click on the line or on the eye symbol of the corresponding test. Now a new page opens with detailed information about the test. On the top right there is a button called "Edit". However, this button can only be clicked if you change the status of the evaluation. To do this, click on "Stop". Please be aware that testers can no longer participate in a stopped test.

Remember to click on the Save button after the editing is done.

Tests

Status of a test

edit-test.png

You can change the status of a test if necessary by navigating to the test detail page. There you will see the current status of the test halfway up on the left. To the right, you will find a series of buttons that allow you to change the status of the test. Below is a brief explanation of each status.

PUBLISHED: This means that a test is live and testers can participate in it.
STOPPED: Testers can no longer participate in the test, but the status can be changed back to RELEASED at any time.
MEETING BREAK: Select this option if you have chosen an iterative approach and want to conduct a discussion of the results during the discussion break. No further participation by testers is possible during this time.
COMPLETE: If all testers have participated and you want to close the test, activate this status

Tests

Activating the "Tester can change reviewer" feature

Activate the checkbox for “Tester can change reviewer” in a test or evaluation if you want to use this function.

This makes it possible for testers to select another tester from a list and assign free feedback to them, without them having to be logged in themselves. This is useful, for example, if several testers share an iPhone/iPad for joint assessment.

IVA-Beurteiler ändern-Evaluation-EN.png

IVA-Beurteiler ändern- Test-DE .png

Tests

Feedback validation

create-test.png

If you want to activate the feedback validation, first navigate to the corresponding test. There you can set the checkmark on 'Use feedback validation' in the Edit mode. You then have the option of setting the status of the test to MEETING BREAK whenever you like. During this time, you can then review the free feedback received and mark it as validated if necessary. This will indicate to future testers that you do not need to report this specific item again.

Tests

Selecting a Test Type

IVA-Testtyp auswählen-EN.png

If you create a new test, you have the option of defining it as one of the following three test types:

Public test
Internal test

Tester requires:

Closed test
Tests

Feedback configuration for tests

Once you have opened the “Create test” or “Edit test” pop-up, you will see that it is divided into two sections:

To continue configuring the feedback, navigate to “Feedback” in the top tab bar.

IVA-Evaluations-Test-Feedback.png

In the left column you can see the different types of feedback:

You can switch between the different types at any time and you have several configuration options depending on the feedback type.


In the right area you can see:


There are three tabs in the middle of the pop-up:

IVA-Evaluationen-Test-Additional-Feedback-for-Testers.png

Additional fields can be assigned when creating or editing a test by going to the “Feedback” tab in the top tab bar and clicking on the “Additional fields for testers” tab at the bottom of the tab bar. This allows the planner to drag and drop or double-click the additional attributes from the left-hand section into the right-hand pop-up page.

Test objects

Test objects

How do I assign a new test object?

testobjekte-bearb.png

To assign a new test object to an existing evaluation, you must first navigate to the corresponding evaluation (Evaluations -> Select corresponding evaluation from list). Then go to the evaluation detail page. Now click on the "Edit" button in the upper right corner. A popup with four tabs will now open: Basic Data, Test Objects, Testers and Categories. Now navigate to the tab "Test objects". In the large column you will now see a list of test objects that have already been added. On the left side you can browse the test objects data pool and add a test object to the current evaluation by using drag&drop or by double-clicking. Especially if you plan to create multiple test objects within an evaluation, this option saves work. You can then, when you edit or create a test, simply take over all test objects from the evaluation. In addition, you can of course also create a new test object at this point. At the end, remember to press the Save button.

Test objects

How do I edit an existing test object list?

testobjekte-bearb.png

To assign a new test object to an existing evaluation, you must first navigate to the corresponding evaluation (Evaluations -> Select corresponding evaluation from list). Then go to the evaluation detail page. Now click on the "Edit" button in the upper right corner. A popup with four tabs will now open: Basic Data, Test objects, Testers and Categories. Now navigate to the tab "Test objects". In the large column you will now see a list of test objects that have already been added. On the left side you can browse the test objects data pool and add a test object to the current evaluation by using drag&drop or by double-clicking. Especially if you plan to create multiple test objects within an evaluation, this option saves work. You can then, in the test processing, simply take over all test objects from the evaluation. In addition, you can of course also create a new test object at this point. At the end, remember to press the Save button.

Tester

Tester

How do I assign a new tester?

assign-new-tester.png

To assign a new tester to an existing evaluation, you have to navigate to the corresponding evaluation (Evaluations -> Select corresponding evaluation from list). Then you will see the evaluation detail page. Now click on the 'Edit' button in the upper right corner. A popup with four tabs will now open: Basic Data, Test Objects, Testers and Categories. Now navigate to the tab 'Tester'. In the large column you will now see a list of testers that have already been added. On the left side, you can browse the tester data pool and add a tester to the current evaluation by using drag and drop or by double-clicking. Especially if you plan to create multiple tests within an evaluation, this option saves work. You can then, in the test editing, simply take over all testers from the evaluation. In addition, you can of course also create a new tester at this point. At the end, remember to press the 'Save' button.

Tester

How do I edit an existing tester list?

assign-new-tester.png

To assign a new tester to an existing evaluation, you must first navigate to the corresponding evaluation (Evaluations -> Select corresponding evaluation from list). Then you will see the evaluation detail page. Now click on the 'Edit' button in the upper right corner. A popup with four tabs will now open: Basic Data, Test Objects, Tester and Categories. Now navigate to the tab 'Tester'. In the large column you will now see a list of testers that have already been added. On the left side, you can browse the tester data pool and add a tester to the current evaluation by using drag and drop or by double-clicking. Especially if you plan to create multiple tests within an evaluation, this option saves work. You can then, in the test editing, simply take over all testers from the evaluation. In addition, you can of course also create a new tester at this point. At the end, remember to press the 'Save' button.