Test Plan - Incident Response v0.6 + 7

0.0

Initial version

08/07

0.1

Added tests for v0.7.

Added test steps for the test cases.

Test plan ready for review.

08/16

1.0

Tests updated.

Tests run and completed

08/21

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

Pass

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

Pass

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

Pass

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

Pass

9

Plugin upgrades normally from v0.4 to v0.6

1. Install version 0.4 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

Pass

10

Plugin upgrades normally from v0.5 to v0.6

1. Install version 0.5 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

Pass

11

Incidents started with the prior plugin versions are still active and available in v0.6

1. Start a few incident with plugin version 0.5.

2. Upgrade plugin to v0.6.

3. Open up the incident RHS to see the incident list.

4. Verify that incidents started with version 0.5 are still available in v0.6.

Pass

12

Incidents ended with prior plugin versions are not available in v0.6

1. Start a few incident with plugin version 0.5.

2. Leave some as active and end a few of them.

3. Upgrade plugin to v0.6.

4. Open up the incident RHS to see the incident list.

5. Verify that incidents started with version 0.5 are still available in v0.6.

6. Verify that incidents ended in step 2 are not visible in the list.

Pass

13

Playbook started with the older plugin version is still available upon plugin upgrade

1. Create a few playbooks with plugin version 0.5.

2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Pass

Test ID

Test Case

Test Steps

Result

Notes

14

Playbook creator can edit playbook

1. Log in as user-0.

2. Launch playbook backstage (using any method).

3. Click the “New Playbook” button.

4. Enter “P1” for Playbook Name.

5. Enter stage “1”.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Bring up the incident creation modal (using any method).

9. Verify that the playbook dropdown has “P1”.

10. From the playbook backstage, find P1.

11. Click P1 to edit the playbook.

12. Change the playbook name to “PB-Private”.

13. Click “Save Playbook”.

14. Verify that the playbook in the backstage shows up as “PB-Private”.

15. Switch to a normal channel.

16. Bring up the incident creation modal.

17. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

Pass

15

Any playbook member can edit playbook to add a new member

Continue from the above test.

1. From the playbook backstage, find PB-Private.

2. Click on the playbook to open the “Edit Playbook” view

3. In the “Members” field, add @user-1.

4. Save playbook.

5. Log in as user-1.

6. Bring up the incident creation modal.

7. Verify that user-1 can see “PB-Private” in the playbook dropdown.

8. Launch playbook backstage.

9. Find “PB-Private” and click to open the “Edit Playbook” view.

10. In the “Members” field, add @user-2.

11. Click “Save Playbook”.

12. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

Pass

16

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” and click to open the “Edit Playbook” view.

4. From the “Members” field, remove @user-1.

5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

Pass

17

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

1. As user-2, launch playbook backstage.

2. Find “PB-Private” and click to open the “Edit Playbook” view.

3. From the “Members” field, remove all users including user-0 and user-2.

4. Verify that the playbook cannot be saved.

5. Add user-1 to the Members field.

6. Save playbook.

Pass

The “Save Playbook” button should stay inactive unless there is at least one member

18

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Toggle the “Create Public Incident” to enable the setting.

4. Save playbook.

5. Click the playbook again to view the edit mode.

6. Verify that the “Create Public Incident” setting is enabled.

Pass

19

Any playbook member can edit playbook to add a new stage

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Click + Add new stage.

4. Change the stage name from “New stage” to “Stage 2”.

5. Save playbook.

6. Click the playbook again to view the edit mode.

7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

Pass

20

A playbook can have multiple stages with the same stage name

1. As user-1, launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “PB-New” for Playbook Name.

4. Enter “Stage 1” as the 1st stage name.

5. Click “Add new stage”.

6. Enter “Stage 1” as the 2nd stage name.

7. ??

Pass

Valid?

@Ian Tao (Unlicensed) confirm?

21

A playbook member can edit playbook to add a new checklist item to a stage added by themself

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Under “Stage 1”, click “Add new checklist item”.

4. Enter “S1-C1” as the checklist item.

5. Click “Save Playbook”.

6. From the playbook list in the backstage find “PB-Private”.

7. Click to view the edit view.

8. Verify that “Stage 1” has “S1-C1” as the checklist item.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the checklist item as been added to “Stage 1”.

Pass

22

Any playbook member can edit playbook to add a new checklist item to a stage added by a different member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 1”, click “Add new checklist item”.

6. Enter “S1-U2” as the checklist item.

7. Click “Save Playbook”.

8. From the playbook list in the backstage find “PB-Private”.

9. Click to view the edit view.

10. Verify that “Stage 1” has “S1-U2” as the checklist item.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify the checklist item has been added to “Stage 1”.

Pass

23

A playbook member cannot edit playbook such that the playbook name field is empty

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Delete the playbook name “PB-Private”.

5. Verify that the “Save Playbook” button becomes deactivate.

6. Click on the “Save Playbook” button.

7. Verify that the playbook cannot be saved, and the user is still in the edit view.

8. Enter a space in the playbook name field.

9. Repeat steps 5-7.

Pass

24

Any playbook member can reorder the checklist items within a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-C1” and drag it under “S1-U2”.

5. Verify that “S1-C1” is successfully moved under “S1-U2”.

6. Click the hamburger icon next to “S1-U2” and drag it above “S1-C1”.

7. Verify that “S1-U2” is successfully moved under “S1-C1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the reordering was successful.

Pass

25

Any playbook member can reorder the checklist items within different stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-U2” and drag it under the 2nd stage “Stage 2”.

5. Verify that “S1-U2” is successfully moved under “Stage 2”.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the reordering was successful.

Pass

Cannot be reordered under a different stage without a step:

MM-28024: A "step" cannot be dragged-dropped into a stage with no stepClosed

26

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to the checklist item “S1-C1” and drag it right above “Stage 1”.

5. Verify that the checklist item does not move from its original spot under “Stage 1”.

6. Drag and drop “S1-C1” outside stage area.

7. Verify that the checklist item does not move from its original spot under “Stage 1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the order of the checklist items is normal.

Pass

27

Any playbook member can delete a checklist item

Continue from the above test.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 2” find “S1-U2” and hover the mouse over so that the “X” appears next to it.

6. Click the “X” for “S1-U2”.

7. Verify that the checklist item “S1-U2” is deleted.

8. Click “Save Playbook”.

9. Click “PB-Private” to go to edit view.

10. Verify that the checklist item “S1-U2” is not available in any stage.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify that “S1-U2” is not available in any stage.

Pass

28

A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” next to all checklist items of all stages.

5. Verify that all checklist items are deleted.

6. Click “Save Playbook”.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that none of stages have any checklist items.

N/A

Invalid.

29

Any playbook member can reorder the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Add a new stage “Stage 3”.

5. Click the hamburger icon next to “Stage 1” and drag it below “Stage 3”.

6. Verify that “Stage 1” is successfully moved under “Stage 3”.

7. Click the hamburger icon next to “Stage 3” and drag it above “Stage 2”.

8. Verify that “Stage 3” is successfully moved above “Stage 2”.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the stage reordering was successful.

Pass

30

A playbook member cannot reorder the playbook such that the stages can be nested

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to “Stage 1” and drag it under “Stage 3” to align it within the checklist items of “Stage 3”.

5. Verify that “Stage 1” is not moved from its original position.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the stage reordering was not successful.

Pass

Verify that the stages cannot be stacked inside another stage

Step 5: The dragged item gets dropped below (not inside) whatever stage item it was dragged into.

31

Any playbook member can delete a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 3”.

5. Verify that “Stage 3” is deleted.

6. Save playbook.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that the incident created does not have “Stage 3”.

Pass

32

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 1”.

5. Click the “X” button next to “Stage 2”.

6. Verify that the “Save Playbook” button becomes inactive.

7. Click on the “Save Playbook”.

8. Verify that the user is still in the edit view of “PB-Private” and the action was not saved.

Fail

MM-28025: Starting an incident with playbook that has no stage loads a blank pageClosed

33

Any playbook member can delete playbook

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Under “Actions” for the playbook, click the ellipses.

5. Click “Delete”.

6. Verify that a “Confirm Playbook Deletion” modal appears.

7. Click “Delete Playbook”.

8. Verify that “PB-Private” is not present in the playbook list.

9. Switch to a normal channel view.

10. Open up an incident modal.

11. Verify that “PB-Private” is not available in the playbook dropdown.

Pass

34

A playbook member can create playbook with a large number of stages

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Click “Add new stage” 20 or more times to create a long list of stages.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Start an incident using “PB-N1” as the playbook.

9. Verify that the Stages dropdown in the incident RHS has all the stages added in step 5.

10. Verify the Stage dropdown displays the stages properly with scrollbar.

Pass

Verify playbook with a large number of stages in the edit page

35

A playbook member can create playbook with a large number of checklist items within the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-N1” and click to go to edit view.

3. For the first stage, add at least 20 checklist items.

4. Save playbook.

5. Switch to a normal channel.

6. Start an incident using “PB-N1” as the playbook.

7. In the incident RHS, ensure that the first stage is selected from the dropdown.

8. Verify that all the checklist items added in step 3 are available.

Pass

Verify playbook with a large number of checklist items within the stages in the edit page

36

The components in the playbook edit page render normally

Verify the appearance of the components in the playbook edit page.

Pass

Test ID

Test Case

Test Steps

Result

Notes

37

An active stage is annotated with Activelabel in the RHS

1. Log in as user-1.

2. Bring up the incident creation modal (using any method).

3. Click the “Create a playbook” button.

4. Enter “PB1” as the playbook name.

5. Enter “S1” as the first stage.

6. Add a new step and call it “S2”.

7. Add another step and call it “S3”.

8. Save playbook.

9. Switch to a normal channel view.

10. Start a new incident “I1” with “PB1”.

11. In the incident RHS, verify that “S1” under “Stages” has an Activelabel next to it.

Pass

38

Non-active stages are not annotated with Active labels

Continue with the above test.

1. From the incident RHS for “I1”, click the “Stages” dropdown menu.

2. Verify that “S2” and “S3” don’t have Activelabel next to them.

Pass

39

Selecting a non-active stage from the RHS dropdown shows a Make Active option prompt

Continue with the above test.

1. From “Stages” in the incident RHS for “I1”, select “S2”.

2. Verify a “Make Active” button appears above the dropdown box.

Pass

40

Incident commander can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Click the “Make Active” button.

2. Verify that there is now an Active label next to “S2”.

3. Click the “Stages” dropdown.

4. Verify that there is no longer an Active label next to “S1”.

Pass

41

Workflow member can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Invite user-2 to the “I1” incident channel.

2. Log in as user-2.

3. From the “Stages” in the incident RHS for “I1”, select “S3”.

4. Verify a “Make Active” button appears above the dropdown box.

5. Click the “Make Active” button.

6. Verify that there is now an Active label next to “S3”.

7. Verify that there is no longer an Active label next to “S1” or “S2”.

8. Log in as user-1.

9. Verify that “S3” has an Active label next to it.

Pass

42

Workflow member cannot can set a different stage as Active in an incident that has been ended

Continue with the above test.

1. As user-1, end the incident “I1”.

2. From the “Stages” in the incident RHS for “I1”, select “S2”.

3. Verify that “Make Active” button does not appear above the dropdown box (or even if it appears, it’s greyed out?)

Fail

@Ian Tao (Unlicensed) confirm?

Updated to “can set a different stage…”

MM-28030: Active stage cannot be changed once an incident is endedClosed

43

Workflow member can set a different stage as Active in an incident that has been restarted

Continue with the above test.

1. From the incident RHS, click “Restart Incident”.

2. Verify that the incident has restarted: The “Restart Incident” button has changed to “End Incident”.

3. Select “S2” from the Stages dropdown in the RHS.

4. Click the “Make Active” button.

5. Verify that “S2” has the Active label next to it.

Pass

44

Plugin posts a message in the incident channel when changing an active stage

Continue from the above test.

1. Verify that setting “S2” as the active stage created a post that the stage has been set as active in the incident channel.

2. Verify that the post were also created in the former tests while setting different stages as active.

Pass

45

Workflow member views the active stage by default in incident RHS

Continue with the above test.

1. Log in as user-2.

2. Open the RHS for “I1”.

3. Verify that when the RHS for “I1” opens, the Stages dropdown shows “S2” by default.

Pass

46

When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the Active annotation

Continue with the above test.

1. While viewing the “I1” RHS as user-2, have user-1 set “S1” as the active stage.

2. Verify that user-2 continues to see “S2” still in the RHS but now without the Active label.

Pass

47

In case of only one stage, the stage is set as active by default

1. As user-1, create a new playbook “P2”.

2. Have only one stage “S1” in “P2”.

3. Save playbook.

4. Switch to a normal channel view.

5. Start a new incident “I2” with “P2”.

6. Verify that when “I2” is successfully started, the RHS shows the “Stages” dropdown shows “S1 active” as the default stage.

Pass

48

In case of multiple stages, the first stage is set as active by default

1. From the playbook backstage, create a new playbook “P3”.

2. Add stages “S1”, “S2”, “S3” and “S4”.

3. Save playbook.

4. Switch to normal channel view.

5. Start an incident “I3” using “P3”.

6. When “I3” is successfully started, verify that the RHS shows “S1” as the active stage in the dropdown.

Pass

49

Switching between the stages does not set the stage as active

Continue from the above test.

1. From the “Stages” dropdown, select “S2”.

2. Verify that when “S2” loads in the RHS, there is no Active label.

3. From the dropdown, select “S3”.

4. Verify that when “S3” loads in the RHS, there is no Active label.

5. From the dropdown, select “S4”.

6. Verify that when “S4” loads in the RHS, there is no Active label.

7. From the dropdown, select “S1”.

8. Verify that when “S1” loads in the RHS, there is Active label.

Pass

50

Selecting an active stage from the RHS dropdown does not show a Make Active option prompt

Continue from the above test.

1. Select “S2” from the incident RHS.

2. Verify that Make Active option appears above the “Stages” dropdown box.

3. Select “S1” from the incident RHS.

4. Verify that when “S1” is selected, the Make Active option appears above the Stages dropdown.

Pass

51

Workflow member can check items off of stages that are active

Continue from the above test.

1. Launch playbook backstage.

2. Find “P3” and click to edit.

3. Add checklist items in all the stages of the playbook “P3”.

4. Switch to a normal channel view.

5. Start a new incident “I4” with playbook P3.

6. From the incident RHS, set “S2” as the active stage.

7. Click on the checkboxes of items in “S2” to check off.

8. Verify that the items in “S2” are successfully checked off.

Pass

52

Workflow member can uncheck items of stages that are not active

Continue from the above test.

1. Ensure that “S1” is not an active stage for “I4”.

2. From the incident RHS for “I4”, select “S1”.

3. Click on the checkboxes of items in “S1” to check off.

4. Verify that the items in “S1” are successfully checked off.

Pass

53

Workflow member can check off items of an ongoing incident

Pass

Covered by the above test

54

Workflow member cannot can checkoff items of an ended incident

Continue from the above test.

1. From the incident RHS for “I4” click “End Incident”.

2. Ensure that “I4” has been ended successfully.

3. Select “S3” from the incident RHS.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items cannot get checked off (checkboxes are inactive)?

Pass

Valid?

@Ian Tao (Unlicensed) confirm?

Updated to “can checkoff…”

55

Workflow member can checkoff items of a restarted incident

Continue from the above test.

1. From the incident RHS for “I4” click “Restart Incident”.

2. Ensure that “I4” has been restarted successfully.

3. Select “S3” from the incident RHS if not already selected.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items are checked off successfully.

Pass

Test ID

Test Case

Test Steps

Result

Notes

56

The Stages dropdown in the RHS has all the stages for that incident

Pass

Covered by #34

57

Selecting a different stage in the RHS shows the checklist corresponding to that stage

1. As user-1, launch playbook backstage.

2. Create a new playbook “P5”.

3. Add stage “S1”.

4. Under “S1”, add the following checklist items:

   1. “C1” with slash command /away.

   2. “C2” (no slash command).

5. Add stage “S2” (don’t add any checklist item).

6. Add stage “S3”.

7. Under “S3”, add the following checklist items:

   1. “Ca”

   2. “Cb”

   3. “Cc”.

8. Switch to a normal channel view.

9. Start an incident “I5” using “P5”.

10. From the incident RHS for I5, while “S1” is selected, verify that the checklist items it contains are “C1” (with the option to run the /away command), and “C2”.

11. Now from the “Stages” dropdown, select “S2”.

12. Verify that “S2” has no checklist items under the dropdown.

13. Select “S3”.

14. Verify that “S3” has a checklist with “Ca”, “Cb” and “Cc”.

Pass

58

Saving a playbook with only one stage and leaving the stage name empty, will create an incident with stage “Default Stage”

1. Launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “P6” for the playbook name.

4. Do not change anything else (there should already be a stage “Default Stage”).

5. Click “Save Playbook”.

6. Switch to a normal channel view.

7. Start a new incident “I6” with “P6”.

8. Open up the RHS for “I6”.

9. Verify that the selected active stage is called “Default Stage” in the dropdown box.

Pass

59

Any workflow member can switch between the stages in the RHS in an ongoing incident

Continue from the above test.

1. As user-1, launch playbook backstage.

2. Select “P5” and click to edit.

3. Under “Members”, add user-2.

4. Save playbook.

5. Log in as user-2.

6. Start a new incident “I7” with “P5”.

7. From the “I7” incident RHS, select “S3”.

8. Click the checkboxes to checkoff items.

9. Verify that the items under “S3” are successfully checked off.

Pass

60

Any workflow member can switch between the stages in the RHS in an incident that has been ended

Continue from the above test.

1. As user-1, end incident “I7”.

2. Switch to user-2.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

Pass

61

Any workflow member can switch between the stages in the RHS in an incident that has been restarted

Continue from the above test.

1. As user-2, restart incident “I7”.

2. Switch to user-1.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

Pass

62

Selecting a different stage will not change the default active stage view upon page refresh

Continue from the above test.

1. As user-1, set “S3” as the active stage.

2. Select “S1” from the RHS.

3. Verify that the checklist for “S1” are loaded in the RHS.

4. Navigate to a different channel.

5. Navigate back to the “I7” incident channel.

6. Verify that “S3” is selected in the “Stages” dropdown box.

7. Verify that the checklist for “S3” is loaded in the RHS.

8. Select “S2”.

9. Refresh the channel.

10. Verify that “S3” is selected in the “Stages” dropdown box.

11. Verify that the checklist for “S3” is loaded in the RHS.

Pass

Refreshing the page or navigating back to the incident channel will always show the Active stage in the RHS and not the stage selected earlier

63

Selecting a different stage will not change the stage view for a different workflow member

Continue from the above test.

1. Add user-2 to the incident channel of “I7”.

2. While user-1 is viewing the incident channel for “I7”, have user-2 switch between the stages in the “I7” RHS.

3. Verify that user-1 continues to see only the active stage in the incident RHS and that user-2 switching between the stages has no effect on user-1’s RHS view.

Pass

64

Stage title should state the current and total number of stages as X/Y

Continue from the above test.

1. In the “I7” incident RHS, verify that the current and total number of stages are listed above the “Stages” dropdown.

Pass

Test ID

Test Case

Test Steps

Result

Notes

65

Checklist item in the RHS has a dropdown for assignee

Continue from the above tests.

1. As user-1 start a new incident “N1” using “P5” from the above tests.

2. In the incident RHS for “N1”, verify that the checklist items for “S1” has a “No Assignee” (assignee) dropdown field under “C1”.

3. Select “S2” from the “Stages” dropdown.

4. Since “S2” has no checklist item, verify there is no assignee dropdown field.

5. Select “S3” from the “Stages” dropdown.

6. Verify that all the three checklist items for “S3” have the assignee dropdown fields under the checklist items.

Pass

66

By default, checklist item does not have any assignee

Continue from the above tests.

1. Verify that non of the checklist items for “N1”, “S1”, “S2” and “S3”, do not have any assignee on the checlist items.

Pass

67

The assignee dropdown shows the list of workflow members

Continue from the above tests.

1. While “S3” is selected, click on the “No Assignee” dropdown for “Ca”.

2. Verify that dropdown opens up with user-1 in the list.

3. Verify there is a search attached to the dropdown when it opens.

Pass

68

The assignee dropdown does not show users that are not workflow members

Continue from the above test.

1. In the dropdown, verify that there is no other user listed other than user-1.

2. In the search box for “No assignee”, type the user name for user-2.

3. Verify that user-2 does not appear in the dropdown.

Pass

69

A checklist item can be checked off without an assignee

Continue from the above test.

1. From the stages dropdown, select “S1”.

2. Ensure that the checklist items of “S1” have no assignees.

3. Click on the “C1” and “C2” checkboxes.

4. Verify that the items are checked off.

Pass

70

A checklist item can be unchecked without an assignee

Continue from the above test.

1. Continue with “S1” selected.

2. Ensure that the checklist items of “S1” have no assignees.

3. Uncheck the previously checked “C1” and “C2”.

4. Verify that the items are unchecked.

Pass

71

Workflow member can select him/herself as the checklist assignee

Continue from the above test.

1. Continue with “S1” selected.

2. For item “C1” click the assignee dropdown.

3. Select user-1 (current user) from the list.

4. Verify that the assignee dropdown field shows user-1 as the assignee.

Pass

72

Workflow member can select a different workflow member as the checklist assignee

Continue from the above test.

1. Invite user-2 in the “N1” incident channel.

2. In the “N1” incident RHS, under “C2”, click the assignee dropdown.

3. Verify that user-2 is now available in the list.

4. Select user-2.

5. Verify that user-2 is now assigned checklist item “C2”.

Pass

73

Workflow member can change assignee from him/herself to ‘No Assignee’ for a checklist item

Continue from the above test.

1. Switch to user-2.

2. Find the “N1” incident channel navigate to it.

3. From the “N1” incident RHS, select “S1” if not already selected as the stage.

4. Click the assignee dropdown for “C2”.

5. Verify there is a “No Assignee” option available under the search bar.

6. Click on “No Assignee”.

7. Verify that “C2” has no assignee.

Pass

74

Workflow member can change another workflow member to ‘No Assignee’ for a checklist item

Continue from the above test.

1. As user-2, assign checklist item “C2” to the him/herself.

2. Switch to user-1.

3. Find incident channel for “N1”.

4. Navigate to it.

5. From the incident RHS, select stage “S1” if not already selected.

6. Ensure user-2 is the assignee for “C2”.

7. Under “C2”, click the assignee dropdown.

8. Click on “No Assignee”.

9. Verify that the item has no assignee.

Pass

75

Plugin posts a message in incident channel when an assignee is selected

Continue from the above tests.

1. Verify that when assigning a checklist item to a workflow member generates a plugin message to notify the action in the center channel.

Pass

76

Plugin posts a message in incident channel when ‘No Assignee’ is selected

Continue from the above tests.

1. Verify that when selecting “No Assignee” for a checklist item, the plugin generates a message to notify the action in the center channel.

Pass

77

Workflow member cannot select multiple assignees

Continue from the above tests.

1. From the incident RHS for “N1”, click the assignee dropdown for “C1”.

2. Click user-1.

3. Click user-2.

4. Verify that clicking a different user highlight only that user in the dropdown.

5. Press and hold the CMD (or Ctrl) button and click on user-2.

6. Verify that clicking user-2 highlight only user-2.

7. Verify that more than 1 user cannot be selected as an assignee for the checklist item.

Pass

78

Workflow member cannot select a different assignee in an incident that is ended

Continue from the above tests.

1. End incident “N1”.

2. Once the incident has ended, click on the assignee dropdown for “C1”.

3. Verify that the assignee dropdown does not open up anymore (??).

4. (Even if it opens up, verify that a different user cannot be assigned to the checklist item).

Pass

79

Workflow member can select a different assignee in an incident that is restarted

Continue for the above tests.

1. Restart incident “N1”.

2. Once the incident has restarted, click on the assignee dropdown for “C2”.

3. Verify that the assignee dropdown is active and can be expanded.

4. Select user-2 as the assignee.

5. Verify that user-2 is successfully assigned “C2”.

Pass

80

Assignee does not change when a workflow member leaves the workflow channel

Continue form the above tests.

1. Log in as user-2.

2. Find and navigate to the “N1” incident channel.

3. Verify that user-2 is assigned to checklist item “C2” (under stage “S1”).

4. In the center channel post textbox, type “/leave” and hit enter to leave the channel.

5. Verify that user-2 cannot find the “N1” incident channel.

6. Log out.

7. Log in as user-1.

8. Find the “N1” incident channel and navigate to it.

9. Verify that user-2 is still the assignee for “C2” under “S1”.

Pass

81

The assignee dropdown list gets updated when a workflow member is kicked from the channel

Continue from the above tests.

1. Invite user-2 to “N1” incident channel again.

2. From the incident RHS, select stage “S3”.

3. Assign item “Ca” to user-2.

4. In the center channel post textbox, type “/kick” and hit enter to remove user-2 from the channel.

5. Under “S3”, click on the assignee list for “Ca”.

6. Verify that user-2 does now appear in the assignee list.

Pass

82

Assignee does not change when a workflow member is kicked from the workflow channel

Continue from the above tests.

1. Select stage “S1”.

2. Verify that “C2” still shows user-2 as the assignee.

3. Select stage “S3”.

4. Verify that “Ca” still shows user-2 as the assignee.

Pass

83

The assignee dropdown list gets updated when a workflow member leaves the channel

Continue from the above tests.

1. With user-2 still removed from the incident channel, click the assignee dropdown for “Cc” under stage “S3”.

2. Verify that user-2 is not in the assignee list.

Pass

84

The assignee dropdown list gets updated when a new member is added to the workflow channel

Continue from the above tests.

1. Invite user-2 back to the “N1” incident channel.

2. In the incident RHS, click the assignee dropdown for “Cc”.

3. Verify that user-2 is now available in the assignee list.

4. Assign “Cc” to user-2

Pass

85

An assignee can checkoff a checklist item

Continue from the above tests.

1. Log in as user-2.

2. From the “N1” incident RHS, find “Cc” item under “S3”.

3. Click the checkbox for “Cc”.

4. Verify that the checklist item is successfully checked off.

Pass

86

An assignee can uncheck a checklist item

Continue from the above tests.

1. As user-2, uncheck item “Cc”.

2. Verify that the checklist item is successfully unchecked.

Pass

87

A non-assignee can checkoff a checklist item

Continue from the above tests.

1. As user-2 uncheck all item assigned to user-2.

2. Log out.

3. Log in as user-1.

4. Find all items assigned to user-2.

5. Click the checkboxes for items assigned to user-2.

6. Verify that the items are successfully checked off.

Pass

88

A non-assignee can uncheck a checklist item

Continue from the above tests.

1. As user-1, uncheck all the boxes for items assigned to user-2.

2. Verify that the items are successfully unchecked.

Pass

89

Anything in the backstage about assignee?

N/A

At the moment you can’t assign an item in the backstage, but that seems useful. @Ian Tao (Unlicensed) ?

Test ID

Test Case

Test Steps

Result

Notes

90

A newly added checklist item in a playbook has a slash command field

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Under the preexisting “Default Stage”, click “Add new checklist item”.

6. Enter “I-1” as the checklist item name.

7. Hit enter.

8. Verify that two new fields, one for slash command, and the other for step description appear underneath.

9. Add more checklist items.

10. Verify that every new checklist item added has the slash command and step description fields.

Pass

91

A playbook can be edited to add a slash command to any checklist item

Continue from the above test.

1. Type /away in the slash command field for one of the checklist items.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

4. Type other valid slash commands in the fields for other checklist items.

5. Verify that each slash command filed can successfully save the slash commands.

Pass

92

A checklist item configured with a slash command shows the slash command for that item in the RHS

Continue from the above tests.

1. Switch to a normal channel.

2. Start a new incident using “PB-N1”.

3. Open the incident RHS.

4. Verify that all checklist items configured with slash commands in the above tests, show the slash command under the checklist item.

Pass

93

A checklist item can only be configured with one slash command at most

1. Create a new playbook “P1a”.

2. Add a checklist item “C1”.

3. In the slash command field for “C1”, enter the following: /away /online /shrug.

4. Verify that a warning appears that the checklist item can only be configured with one slash command.

N/A

Valid?

I think this should be allowed (there’s no technical reason it can’t). @Ian Tao (Unlicensed) ?

MM-27963: A checklist item can only be configured with one slash command at mostClosed

94

Typing an invalid slash command

N/A

There’s nothing preventing the user from typing an invalid slash command.

Test ID

Test Case

Test Steps

Result

Notes

95

Typing a / in the slash command field brings up a list of existing slash commands

1. Log in as user-1.

2. Launch playbooks backstage.

3. Click “New Playbook”.

4. Enter “PBa” as the playbook name.

5. Edit the name of the default stage to “S1”.

6. Click “Add new stage”.

7. Enter “C1” for checklist item name.

8. In the slash command field, enter /

9. Verify that a autocomplete list of all available slash commands pop up as soon as entering /.

Pass

96

Typing a non-/ character in the slash command field?

N/A

See #94

97

Selecting from the slash command list auto completes the slash command

Continue from the above tests.

1. In the slash command field, enter /.

2. From the autocomplete list, click /dnd.

3. Verify that the slash command field is autocompleted with /dnd.

Pass

98

Typing a non-existing slash command shows a “invalid slash command” error

N/A

Valid?

We do not validate the slash commands at the moment.

99

The slash command field can be left empty

Continue from the above tests.

1. Remove the contents of “C1” so that the slash command field is empty.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

Pass

Test ID

Test Case

Test Steps

Result

Notes

100

A checklist item configured without a slash command does not have a Run button next to it

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “Apb” as the playbook name.

5. Change the name of “Default Stage” to “S1”.

6. Click “Add new checklist item”.

7. Enter “C1” as the item name.

8. Don’t add anything for the slash command.

9. Save playbook.

10. Switch to normal channel view.

11. Start a new incident using “Apb”.

12. Once the incident is started, in the incident RHS, examine the checklist item “C1” under “S1”.

13. Verify that “C1” does not have any “Run” button.

Pass

101

A checklist item configured with a valid slash command has a Run button next to it

Continue from the above test.

1. Launch playbook.

2. Find and click “Apb” to edit.

3. Add slash command /away for “C1”.

4. Switch to normal channel view.

5. Start a new incident using “Apb”.

6. Once the incident is started, in the incident RHS, verify that “C1” show /away under it which has a Run button next to it.

Pass

102

A checklist item configured with an invalid slash command does not have a Run button next to it

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Add a new checklist item “C2” under “S1”.

4. Add slash command invalid text for “C2”.

5. Save playbook.

6. Switch to normal channel view.

7. Start a new incident using “Apb”.

8. Once the incident has started, find “C2” under “S1”.

9. Verify that the slash command for “C2” doesn’t have a Run button next to it.

N/A

This doesn’t happen currently. It would require us to spend mana on validating the slash command.

103

A checklist item configured with a nonexistent slash command does not have a Runbutton next to it

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Add a new checklist item “C3” under “S1”.

4. Add slash command /invalidcommand for “C3”.

5. Save playbook.

6. Switch to normal channel view.

7. Start a new incident “Ia1” using “Apb”.

8. Once the incident has started, find “C3” under “S1”.

9. Verify that the slash command for “C3” doesn’t have a Run button next to it.

N/A

See #102

104

Clicking the Run button for a checklist slash command runs the slash command

Continue from the above test.

1. Open up the incident RHS for “Ia1”.

2. Find “C1” under stage “S1”.

3. Click the Run button next to “C1”.

4. Verify that clicking the Run button executes so that the status of user-1 is now set to “away”.

Pass

105

Plugin posts a message in the incident channel when a slash command Runis clicked

Continue from the above test.

1. Verify that plugin posts a message in the “Ia1” incident channel to notify that the slash command for “C1” has been run.

N/A

Ticket created

A message that the checklist item slash command was run (not the slash command system message)

@Ian Tao (Unlicensed) should we make a ticket for this feature? We currently don’t do it, but it makes sense.

MM-27964: Plugin posts a message in the incident channel when a slash command `Run` button is clickedClosed

106

Running a slash command with system message attached to it posts the slash command system message in the incident channel

Continue from the above test.

1. Verify that the system message for the /away command “System: You are now away (Only visible to you)” is posted in the incident channel.

Pass

107

In case of two valid slash commands, running the command from the checklist item will only run the first command successfully

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C5”.

4. Add the following for “C5’s” slash command: /online /away.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C5” in the incident RHS.

8. Click the Run button next to “C5”.

9. Verify that the status of user-1 is now set to “online” and not “away”.

Pass

This is in sync with the core feature behavior

Seems correct to me (ChrisP)

108

Running an invalid slash command from RHS checklist shows an ‘invalid slash command’ error

Depends on the steps in #102.

N/A

109

A slash command can be run multiple times from the RHS checklist

Continue from the above tests.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C6”.

4. Add /shrug as the slash command for “C6”.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C6” in the incident RHS.

8. Click the Run button for “C6”.

9. Verify that the /shrug command is executed successfully and a shrug message is posted in the center channel.

10. Repeat steps 8-9 multiple times.

Pass

110

A plugin post notifies when a slash command fails to run successfully

Verify that when a slash command fails to execute by clicking the Run button, the plugin posts a message in the center channel notifying that the execution has failed.

Fail

MM-28037: Running an invalid slash command does not show error to userClosed

Test ID

Test Case

Test Steps

Result

Notes

111

Checklist item description

1. Create a playbook with checklist items that have markdown descriptions.

2. Start an incident using such playbook.

3. Make sure that hovering over the icon changes its opacity.

4. Verify that clicking on the icon opens up the popover.

5. Verify that the popover renders markdown correctly (included channel links and mentions).

6. Verify that clicking on the X icon closes the popover.

Pass

MM-27016: Step DescriptionsClosed

MM-28029: Step descriptions overlap each over in the RHSClosed

112

Add pagination to playbook list

1. Navigate to backstage -> playbooks.

2. Create more than 16 playbooks.

3. Observe the pagination options at the bottom.

Pass

https://mattermost.atlassian.net/browse/MM-27344

113

Browse Playbook templates

1. Navigate to backstage > playbooks.

2. Click on any of the new templates.

Pass

https://mattermost.atlassian.net/browse/MM-27028

114

Channel link capability to the workflow markdown

A channel link in the checklist text (it has to be correct), such as ~town-square, should:

• be correctly highlighted like it is in the center channel

• be clickable and bring the user to that channel, /without/ refreshing the app.

Pass

https://mattermost.atlassian.net/browse/MM-26924

115

Test new playbook creation with Blank Playbook template

Sanity tests with Blank Playbook template.

Pass

https://mattermost.atlassian.net/browse/MM-28009

116

Test new playbook creation with Incident Response Playbook

Sanity tests with Incident Response Playbook template.

Pass

117

Show workflow duration

1. Verify that total duration of the workflow is displayed in the RHS.

2. This value should update in realtime without requiring any refresh. Show days, hours, minutes and seconds as necessary.

Pass

https://mattermost.atlassian.net/browse/MM-27014

118

Return paginated response for all list endpoints

1. Make sure you can see multiple incident pages in backstage.

2. Make sure you can view existing playbooks.

Pass

https://mattermost.atlassian.net/browse/MM-26179

https://mattermost.atlassian.net/browse/MM-28038

119

Empty playbook landing

1. Open backstage.

2. Click playbooks.

3. Make sure no playbooks exists to see the empty state.

Pass

https://mattermost.atlassian.net/browse/MM-27027

120

Sortable names, stages, steps in playbook backstage

1. Navigate to the backstage.

2. Verify the playbook list can be sorted by the columns.

Pass

https://mattermost.atlassian.net/browse/MM-27029

121

Active label is visible in the “Stages” dropdown box for a long stage name

Pass

https://mattermost.atlassian.net/browse/MM-28026