Test Plan - Incident Response v0.6 + 7

0.0

Initial version

08/07

0.1

Added tests for v0.7.

Added test steps for the test cases.

Test plan ready for review.

08/16

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

9

Plugin upgrades normally from v0.4 to v0.6

1. Install version 0.4 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

10

Plugin upgrades normally from v0.5 to v0.6

1. Install version 0.5 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

11

Incidents started with the prior plugin versions are still active and available in v0.6

1. Start a few incident with plugin version 0.5.

2. Upgrade plugin to v0.6.

3. Open up the incident RHS to see the incident list.

4. Verify that incidents started with version 0.5 are still available in v0.6.

12

Incidents ended with prior plugin versions are not available in v0.6

1. Start a few incident with plugin version 0.5.

2. Leave some as active and end a few of them.

3. Upgrade plugin to v0.6.

4. Open up the incident RHS to see the incident list.

5. Verify that incidents started with version 0.5 are still available in v0.6.

6. Verify that incidents ended in step 2 are not visible in the list.

13

Playbook started with the older plugin version is still available upon plugin upgrade

1. Create a few playbooks with plugin version 0.5.

2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Test ID

Test Case

Test Steps

Result

Notes

14

Playbook creator can edit playbook

1. Log in as user-0.

2. Launch playbook backstage (using any method).

3. Click the “New Playbook” button.

4. Enter “P1” for Playbook Name.

5. Click “Save Playbook”.

6. Switch to a normal channel.

7. Bring up the incident creation modal (using any method).

8. Verify that the playbook dropdown has “P1”.

9. From the playbook backstage, find P1.

10. Click P1 to edit the playbook.

11. Change the playbook name to “PB-Private”.

12. Click “Save Playbook”.

13. Verify that the playbook in the backstage shows up as “PB-Private”.

14. Switch to a normal channel.

15. Bring up the incident creation modal.

16. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

15

Any playbook member can edit playbook to add a new member

Continue from the above test.

1. From the playbook backstage, find PB-Private.

2. Click on the playbook to open the “Edit Playbook” view

3. In the “Members” field, add @user-1.

4. Log in as user-1.

5. Bring up the incident creation modal.

6. Verify that user-1 can see “PB-Private” in the playbook dropdown.

7. Launch playbook backstage.

8. Find “PB-Private” and click to open the “Edit Playbook” view.

9. In the “Members” field, add @user-2.

10. Click “Save Playbook”.

11. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

16

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” and click to open the “Edit Playbook” view.

4. From the “Members” field, remove @user-1.

5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

17

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

1. As user-2, launch playbook backstage.

2. Find “PB-Private” and click to open the “Edit Playbook” view.

3. From the “Members” field, remove all users including user-0 and user-2.

4. Verify that the playbook cannot be saved.

5. Add user-1 to the Members field.

6. Save playbook.

The “Save Playbook” button should stay inactive unless there is at least one member

18

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Toggle the “Create Public Incident” to enable the setting.

4. Save playbook.

5. Click the playbook again to view the edit mode.

6. Verify that the “Create Public Incident” setting is enabled.

19

Any playbook member can edit playbook to add a new stage

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Click + Add new stage.

4. Change the stage name from “New stage” to “Stage 2”.

5. Save playbook.

6. Click the playbook again to view the edit mode.

7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

20

A playbook can have multiple stages with the same stage name

1. As user-1, launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “PB-New” for Playbook Name.

4. Enter “Stage 1” as the 1st stage name.

5. Click “Add new stage”.

6. Enter “Stage 1” as the 2nd stage name.

7. ??

Valid?

Ian Tao (Unlicensed) confirm?

21

A playbook member can edit playbook to add a new checklist item to a stage added by themself

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Under “Stage 1”, click “Add new checklist item”.

4. Enter “S1-C1” as the checklist item.

5. Click “Save Playbook”.

6. From the playbook list in the backstage find “PB-Private”.

7. Click to view the edit view.

8. Verify that “Stage 1” has “S1-C1” as the checklist item.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the checklist item as been added to “Stage 1”.

22

Any playbook member can edit playbook to add a new checklist item to a stage added by a different member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 1”, click “Add new checklist item”.

6. Enter “S1-U2” as the checklist item.

7. Click “Save Playbook”.

8. From the playbook list in the backstage find “PB-Private”.

9. Click to view the edit view.

10. Verify that “Stage 1” has “S1-U2” as the checklist item.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify the checklist item has been added to “Stage 1”.

23

A playbook member cannot edit playbook such that the playbook name field is empty

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Delete the playbook name “PB-Private”.

5. Verify that the “Save Playbook” button becomes deactivate.

6. Click on the “Save Playbook” button.

7. Verify that the playbook cannot be saved, and the user is still in the edit view.

8. Enter a space in the playbook name field.

9. Repeat steps 5-7.

24

Any playbook member can reorder the checklist items within a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-C1” and drag it under “S1-U2”.

5. Verify that “S1-C1” is successfully moved under “S1-U2”.

6. Click the hamburger icon next to “S1-U2” and drag it above “S1-C1”.

7. Verify that “S1-U2” is successfully moved under “S1-C1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the reordering was successful.

25

Any playbook member can reorder the checklist items within different stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-U2” and drag it under the 2nd stage “Stage 2”.

5. Verify that “S1-U2” is successfully moved under “Stage 2”.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the reordering was successful.

26

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to the checklist item “S1-C1” and drag it right above “Stage 1”.

5. Verify that the checklist item does not move from its original spot under “Stage 1”.

6. Drag and drop “S1-C1” outside stage area.

7. Verify that the checklist item does not move from its original spot under “Stage 1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the order of the checklist items is normal.

27

Any playbook member can delete a checklist item

Continue from the above test.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 2” find “S1-U2” and hover the mouse over so that the “X” appears next to it.

6. Click the “X” for “S1-U2”.

7. Verify that the checklist item “S1-U2” is deleted.

8. Click “Save Playbook”.

9. Click “PB-Private” to go to edit view.

10. Verify that the checklist item “S1-U2” is not available in any stage.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify that “S1-U2” is not available in any stage.

28

A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” next to all checklist items of all stages.

5. Verify that all checklist items are deleted.

6. Click “Save Playbook”.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that none of stages have any checklist items.

29

Any playbook member can reorder the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Add a new stage “Stage 3”.

5. Click the hamburger icon next to “Stage 1” and drag it below “Stage 3”.

6. Verify that “Stage 1” is successfully moved under “Stage 3”.

7. Click the hamburger icon next to “Stage 3” and drag it above “Stage 2”.

8. Verify that “Stage 3” is successfully moved above “Stage 2”.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the stage reordering was successful.

30

A playbook member cannot reorder the playbook such that the stages can be nested

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to “Stage 1” and drag it under “Stage 3” to align it within the checklist items of “Stage 3”.

5. Verify that “Stage 1” is not moved from its original position.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the stage reordering was not successful.

Verify that the stages cannot be stacked inside another stage

31

Any playbook member can delete a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 3”.

5. Verify that “Stage 3” is deleted.

6. Save playbook.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that the incident created does not have “Stage 3”.

32

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 1”.

5. Click the “X” button next to “Stage 2”.

6. Verify that the “Save Playbook” button becomes inactive.

7. Click on the “Save Playbook”.

8. Verify that the user is still in the edit view of “PB-Private” and the action was not saved.

33

Any playbook member can delete playbook

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Under “Actions” for the playbook, click the ellipses.

5. Click “Delete”.

6. Verify that a “Confirm Playbook Deletion” modal appears.

7. Click “Delete Playbook”.

8. Verify that “PB-Private” is not present in the playbook list.

9. Switch to a normal channel view.

10. Open up an incident modal.

11. Verify that “PB-Private” is not available in the playbook dropdown.

34

A playbook member can create playbook with a large number of stages

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Click “Add new stage” 20 or more times to create a long list of stages.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Start an incident using “PB-N1” as the playbook.

9. Verify that the Stages dropdown in the incident RHS has all the stages added in step 5.

10. Verify the Stage dropdown displays the stages properly with scrollbar.

Verify playbook with a large number of stages in the edit page

35

A playbook member can create playbook with a large number of checklist items within the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-N1” and click to go to edit view.

3. For the first stage, add at least 20 checklist items.

4. Save playbook.

5. Switch to a normal channel.

6. Start an incident using “PB-N1” as the playbook.

7. In the incident RHS, ensure that the first stage is selected from the dropdown.

8. Verify that all the checklist items added in step 3 are available.

Verify playbook with a large number of checklist items within the stages in the edit page

36

The components in the playbook edit page render normally

Verify the appearance of the components in the playbook edit page.

Test ID

Test Case

Test Steps

Result

Notes

37

An active stage is annotated with Activelabel in the RHS

1. Log in as user-1.

2. Bring up the incident creation modal (using any method).

3. Click the “Create a playbook” button.

4. Enter “PB1” as the playbook name.

5. Enter “S1” as the first stage.

6. Add a new step and call it “S2”.

7. Add another step and call it “S3”.

8. Save playbook.

9. Switch to a normal channel view.

10. Start a new incident “I1” with “PB1”.

11. In the incident RHS, verify that “S1” under “Stages” has an Activelabel next to it.

38

Non-active stages are not annotated with Active labels

Continue with the above test.

1. From the incident RHS for “I1”, click the “Stages” dropdown menu.

2. Verify that “S2” and “S3” don’t have Activelabel next to them.

39

Selecting a non-active stage from the RHS dropdown shows a Make Active option prompt

Continue with the above test.

1. From “Stages” in the incident RHS for “I1”, select “S2”.

2. Verify a “Make Active” button appears above the dropdown box.

40

Incident commander can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Click the “Make Active” button.

2. Verify that there is now an Active label next to “S2”.

3. Click the “Stages” dropdown.

4. Verify that there is no longer an Active label next to “S1”.

41

Workflow member can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Invite user-2 to the “I1” incident channel.

2. Log in as user-2.

3. From the “Stages” in the incident RHS for “I1”, select “S3”.

4. Verify a “Make Active” button appears above the dropdown box.

5. Click the “Make Active” button.

6. Verify that there is now an Active label next to “S3”.

7. Verify that there is no longer an Active label next to “S1” or “S2”.

8. Log in as user-1.

9. Verify that “S3” has an Active label next to it.

42

Workflow member cannot set a different stage as Active in an incident that has been ended

Continue with the above test.

1. As user-1, end the incident “I1”.

2. From the “Stages” in the incident RHS for “I1”, select “S2”.

3. Verify that “Make Active” button does not appear above the dropdown box (or even if it appears, it’s greyed out?)

Ian Tao (Unlicensed) confirm?

43

Workflow member can set a different stage as Active in an incident that has been restarted

Continue with the above test.

1. From the incident RHS, click “Restart Incident”.

2. Verify that the incident has restarted: The “Restart Incident” button has changed to “End Incident”.

3. Select “S2” from the Stages dropdown in the RHS.

4. Click the “Make Active” button.

5. Verify that “S2” has the Active label next to it.

44

Plugin posts a message in the incident channel when changing an active stage

Continue from the above test.

1. Verify that setting “S2” as the active stage created a post that the stage has been set as active in the incident channel.

2. Verify that the post were also created in the former tests while setting different stages as active.

45

Workflow member views the active stage by default in incident RHS

Continue with the above test.

1. Log in as user-2.

2. Open the RHS for “I1”.

3. Verify that when the RHS for “I1” opens, the Stages dropdown shows “S2” by default.

46

When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the Active annotation

Continue with the above test.

1. While viewing the “I1” RHS as user-2, have user-1 set “S1” as the active stage.

2. Verify that user-2 continues to see “S2” still in the RHS but now without the Active label.

47

In case of only one stage, the stage is set as active by default

1. As user-1, create a new playbook “P2”.

2. Have only one stage “S1” in “P2”.

3. Save playbook.

4. Switch to a normal channel view.

5. Start a new incident “I2” with “P2”.

6. Verify that when “I2” is successfully started, the RHS shows the “Stages” dropdown shows “S1 active” as the default stage.

48

In case of multiple stages, the first stage is set as active by default

1. From the playbook backstage, create a new playbook “P3”.

2. Add stages “S1”, “S2”, “S3” and “S4”.

3. Save playbook.

4. Switch to normal channel view.

5. Start an incident “I3” using “P3”.

6. When “I3” is successfully started, verify that the RHS shows “S1” as the active stage in the dropdown.

49

Switching between the stages does not set the stage as active

Continue from the above test.

1. From the “Stages” dropdown, select “S2”.

2. Verify that when “S2” loads in the RHS, there is no Active label.

3. From the dropdown, select “S3”.

4. Verify that when “S3” loads in the RHS, there is no Active label.

5. From the dropdown, select “S4”.

6. Verify that when “S4” loads in the RHS, there is no Active label.

7. From the dropdown, select “S1”.

8. Verify that when “S1” loads in the RHS, there is Active label.

50

Selecting an active stage from the RHS dropdown does not show a Make Active option prompt

Continue from the above test.

1. Select “S2” from the incident RHS.

2. Verify that Make Active option appears above the “Stages” dropdown box.

3. Select “S1” from the incident RHS.

4. Verify that when “S1” is selected, the Make Active option appears above the Stages dropdown.

51

Workflow member can check items off of stages that are active

Continue from the above test.

1. Launch playbook backstage.

2. Find “P3” and click to edit.

3. Add checklist items in all the stages of the playbook “P3”.

4. Switch to a normal channel view.

5. Start a new incident “I4” with playbook P3.

6. From the incident RHS, set “S2” as the active stage.

7. Click on the checkboxes of items in “S2” to check off.

8. Verify that the items in “S2” are successfully checked off.

52

Workflow member can uncheck items of stages that are not active

Continue from the above test.

1. Ensure that “S1” is not an active stage for “I4”.

2. From the incident RHS for “I4”, select “S1”.

3. Click on the checkboxes of items in “S1” to check off.

4. Verify that the items in “S1” are successfully checked off.

53

Workflow member can check off items of an ongoing incident

Covered by the above test

54

Workflow member cannot checkoff items of an ended incident

Continue from the above test.

1. From the incident RHS for “I4” click “End Incident”.

2. Ensure that “I4” has been ended successfully.

3. Select “S3” from the incident RHS.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items cannot get checked off (checkboxes are inactive)?

Valid?

Ian Tao (Unlicensed) confirm?

55

Workflow member can checkoff items of a restarted incident

Continue from the above test.

1. From the incident RHS for “I4” click “Restart Incident”.

2. Ensure that “I4” has been restarted successfully.

3. Select “S3” from the incident RHS if not already selected.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items are checked off successfully.

Test ID

Test Case

Test Steps

Result

Notes

56

The Stages dropdown in the RHS has all the stages for that incident

Covered by #34

57

Selecting a different the stage in the RHS shows the checklist corresponding to that stage

1. As user-1, launch playbook backstage.

2. Create a new playbook “P5”.

3. Add stage “S1”.

4. Under “S1”, add the following checklist items:

   1. “C1” with slash command /away.

   2. “C2” (no slash command).

5. Add stage “S2” (don’t add any checklist item).

6. Add stage “S3”.

7. Under “S3”, add the following checklist items:

   1. “Ca”

   2. “Cb”

   3. “Cc”.

8. Switch to a normal channel view.

9. Start an incident “I5” using “P5”.

10. From the incident RHS for I5, while “S1” is selected, verify that the checklist items it contains are “C1” (with the option to run the /away command), and “C2”.

11. Now from the “Stages” dropdown, select “S2”.

12. Verify that “S2” has no checklist items under the dropdown.

13. Select “S3”.

14. Verify that “S3” has a checklist with “Ca”, “Cb” and “Cc”.

58

Saving a playbook with only one stage and leaving the stage name empty, will create an incident with stage “Default Stage”

1. Launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “P6” for the playbook name.

4. Do not change anything else (there should already be a stage “Default Stage”).

5. Click “Save Playbook”.

6. Switch to a normal channel view.

7. Start a new incident “I6” with “P6”.

8. Open up the RHS for “I6”.

9. Verify that the selected active stage is called “Default Stage” in the dropdown box.

59

Any workflow member can switch between the stages in the RHS in an ongoing incident

Continue from the above test.

1. As user-1, launch playbook backstage.

2. Select “P5” and click to edit.

3. Under “Members”, add user-2.

4. Save playbook.

5. Log in as user-2.

6. Start a new incident “I7” with “P5”.

7. From the “I7” incident RHS, select “S3”.

8. Click the checkboxes to checkoff items.

9. Verify that the items under “S3” are successfully checked off.

60

Any workflow member can switch between the stages in the RHS in an incident that has been ended

Continue from the above test.

1. As user-1, end incident “I7”.

2. Switch to user-2.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

61

Any workflow member can switch between the stages in the RHS in an incident that has been restarted

Continue from the above test.

1. As user-2, restart incident “I7”.

2. Switch to user-1.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

62

Selecting a different stage will not change the default active stage view upon page refresh

Continue from the above test.

1. As user-1, set “S3” as the active stage.

2. Select “S1” from the RHS.

3. Verify that the checklist for “S1” are loaded in the RHS.

4. Navigate to a different channel.

5. Navigate back to the “I7” incident channel.

6. Verify that “S3” is selected in the “Stages” dropdown box.

7. Verify that the checklist for “S3” is loaded in the RHS.

8. Select “S2”.

9. Refresh the channel.

10. Verify that “S3” is selected in the “Stages” dropdown box.

11. Verify that the checklist for “S3” is loaded in the RHS.

Refreshing the page or navigating back to the incident channel will always show the Active stage in the RHS and not the stage selected earlier

63

Selecting a different stage will not change the stage view for a different workflow member

Continue from the above test.

1. Add user-2 to the incident channel of “I7”.

2. While user-1 is viewing the incident channel for “I7”, have user-2 switch between the stages in the “I7” RHS.

3. Verify that user-1 continues to see only the active stage in the incident RHS and that user-2 switching between the stages has no effect on user-1’s RHS view.

64

Stage title should state the current and total number of stages as X/Y

Continue from the above test.

1. In the “I7” incident RHS, verify that the current and total number of stages are listed above the “Stages” dropdown.

Test ID

Test Case

Test Steps

Result

Notes

65

Checklist item in the RHS has a dropdown for assignee

Continue from the above tests.

1. As user-1 start a new incident “N1” using “P5” from the above tests.

2. In the incident RHS for “N1”, verify that the checklist items for “S1” has a “No Assignee” (assignee) dropdown field under “C1”.

3. Select “S2” from the “Stages” dropdown.

4. Since “S2” has no checklist item, verify there is no assignee dropdown field.

5. Select “S3” from the “Stages” dropdown.

6. Verify that all the three checklist items for “S3” have the assignee dropdown fields under the checklist items.

66

By default, checklist item does not have any assignee

Continue from the above tests.

1. Verify that non of the checklist items for “N1”, “S1”, “S2” and “S3”, do not have any assignee on the checlist items.

67

The assignee dropdown shows the list of workflow members

Continue from the above tests.

1. While “S3” is selected, click on the “No Assignee” dropdown for “Ca”.

2. Verify that dropdown opens up with user-1 in the list.

3. Verify there is a search attached to the dropdown when it opens.

68

The assignee dropdown does not show users that are not workflow members

Continue from the above test.

1. In the dropdown, verify that there is no other user listed other than user-1.

2. In the search box for “No assignee”, type the user name for user-2.

3. Verify that user-2 does not appear in the dropdown.

69

A checklist item can be checked off without an assignee

Continue from the above test.

1. From the stages dropdown, select “S1”.

2. Ensure that the checklist items of “S1” have no assignees.

3. Click on the “C1” and “C2” checkboxes.

4. Verify that the items are checked off.

70

A checklist item can be unchecked without an assignee

Continue from the above test.

1. Continue with “S1” selected.

2. Ensure that the checklist items of “S1” have no assignees.

3. Uncheck the previously checked “C1” and “C2”.

4. Verify that the items are unchecked.

71

Workflow member can select him/herself as the checklist assignee

Continue from the above test.

1. Continue with “S1” selected.

2. For item “C1” click the assignee dropdown.

3. Select user-1 (current user) from the list.

4. Verify that the assignee dropdown field shows user-1 as the assignee.

72

Workflow member can select a different workflow member as the checklist assignee

Continue from the above test.

1. Invite user-2 in the “N1” incident channel.

2. In the “N1” incident RHS, under “C2”, click the assignee dropdown.

3. Verify that user-2 is now available in the list.

4. Select user-2.

5. Verify that user-2 is now assigned checklist item “C2”.

73

Workflow member can change assignee from him/herself to ‘No Assignee’ for a checklist item

Continue from the above test.

1. Switch to user-2.

2. Find the “N1” incident channel navigate to it.

3. From the “N1” incident RHS, select “S1” if not already selected as the stage.

4. Click the assignee dropdown for “C2”.

5. Verify there is a “No Assignee” option available under the search bar.

6. Click on “No Assignee”.

7. Verify that “C2” has no assignee.

74

Workflow member can change another workflow member to ‘No Assignee’ for a checklist item

Continue from the above test.

1. As user-2, assign checklist item “C2” to the him/herself.

2. Switch to user-1.

3. Find incident channel for “N1”.

4. Navigate to it.

5. From the incident RHS, select stage “S1” if not already selected.

6. Ensure user-2 is the assignee for “C2”.

7. Under “C2”, click the assignee dropdown.

8. Click on “No Assignee”.

9. Verify that the item has no assignee.

75

Plugin posts a message in incident channel when an assignee is selected

Continue from the above tests.

1. Verify that when assigning a checklist item to a workflow member generates a plugin message to notify the action in the center channel.

76

Plugin posts a message in incident channel when ‘No Assignee’ is selected

Continue from the above tests.

1. Verify that when selecting “No Assignee” for a checklist item, the plugin generates a message to notify the action in the center channel.

77

Workflow member cannot select multiple assignees

Continue from the above tests.

1. From the incident RHS for “N1”, click the assignee dropdown for “C1”.

2. Click user-1.

3. Click user-2.

4. Verify that clicking a different user highlight only that user in the dropdown.

5. Press and hold the CMD (or Ctrl) button and click on user-2.

6. Verify that clicking user-2 highlight only user-2.

7. Verify that more than 1 user cannot be selected as an assignee for the checklist item.

78

Workflow member cannot select a different assignee in an incident that is ended

Continue from the above tests.

1. End incident “N1”.

2. Once the incident has ended, click on the assignee dropdown for “C1”.

3. Verify that the assignee dropdown does not open up anymore (??).

4. (Even if it opens up, verify that a different user cannot be assigned to the checklist item).

Valid for ended incident?

79

Workflow member can select a different assignee in an incident that is restarted

Continue for the above tests.

1. Restart incident “N1”.

2. Once the incident has restarted, click on the assignee dropdown for “C2”.

3. Verify that the assignee dropdown is active and can be expanded.

4. Select user-2 as the assignee.

5. Verify that user-2 is successfully assigned “C2”.

80

Assignee does not change when a workflow member leaves the workflow channel

Continue form the above tests.

1. Log in as user-2.

2. Find and navigate to the “N1” incident channel.

3. Verify that user-2 is assigned to checklist item “C2” (under stage “S1”).

4. In the center channel post textbox, type “/leave” and hit enter to leave the channel.

5. Verify that user-2 cannot find the “N1” incident channel.

6. Log out.

7. Log in as user-1.

8. Find the “N1” incident channel and navigate to it.

9. Verify that user-2 is still the assignee for “C2” under “S1”.

81

The assignee dropdown list gets updated when a workflow member is kicked from the channel

Continue from the above tests.

1. Invite user-2 to “N1” incident channel again.

2. From the incident RHS, select stage “S3”.

3. Assign item “Ca” to user-2.

4. In the center channel post textbox, type “/kick” and hit enter to remove user-2 from the channel.

5. Under “S3”, click on the assignee list for “Ca”.

6. Verify that user-2 does now appear in the assignee list.

82

Assignee does not change when a workflow member is kicked from the workflow channel

Continue from the above tests.

1. Select stage “S1”.

2. Verify that “C2” still shows user-2 as the assignee.

3. Select stage “S3”.

4. Verify that “Ca” still shows user-2 as the assignee.

83

The assignee dropdown list gets updated when a workflow member leaves the channel

Continue from the above tests.

1. With user-2 still removed from the incident channel, click the assignee dropdown for “Cc” under stage “S3”.

2. Verify that user-2 is not in the assignee list.

84

The assignee dropdown list gets updated when a new member is added to the workflow channel

Continue from the above tests.

1. Invite user-2 back to the “N1” incident channel.

2. In the incident RHS, click the assignee dropdown for “Cc”.

3. Verify that user-2 is now available in the assignee list.

4. Assign “Cc” to user-2

85

An assignee can checkoff a checklist item

Continue from the above tests.

1. Log in as user-2.

2. From the “N1” incident RHS, find “Cc” item under “S3”.

3. Click the checkbox for “Cc”.

4. Verify that the checklist item is successfully checked off.

86

An assignee can uncheck a checklist item

Continue from the above tests.

1. As user-2, uncheck item “Cc”.

2. Verify that the checklist item is successfully unchecked.

87

A non-assignee can checkoff a checklist item

Continue from the above tests.

1. As user-2 uncheck all item assigned to user-2.

2. Log out.

3. Log in as user-1.

4. Find all items assigned to user-2.

5. Click the checkboxes for items assigned to user-2.

6. Verify that the items are successfully checked off.

88

A non-assignee can uncheck a checklist item

Continue from the above tests.

1. As user-1, uncheck all the boxes for items assigned to user-2.

2. Verify that the items are successfully unchecked.

89

Anything in the backstage about assignee?

At the moment you can’t assign an item in the backstage, but that seems useful. Ian Tao (Unlicensed) ?

Test ID

Test Case

Test Steps

Result

Notes

90

A newly added checklist item in a playbook has a slash command field

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Under the preexisting “Default Stage”, click “Add new checklist item”.

6. Enter “I-1” as the checklist item name.

7. Hit enter.

8. Verify that two new fields, one for slash command, and the other for step description appear underneath.

9. Add more checklist items.

10. Verify that every new checklist item added has the slash command and step description fields.

91

A playbook can be edited to add a slash command to any checklist item

Continue from the above test.

1. Type /away in the slash command field for one of the checklist items.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

4. Type other valid slash commands in the fields for other checklist items.

5. Verify that each slash command filed can successfully save the slash commands.

92

A checklist item configured with a slash command shows the slash command for that item in the RHS

Continue from the above tests.

1. Switch to a normal channel.

2. Start a new incident using “PB-N1”.

3. Open the incident RHS.

4. Verify that all checklist items configured with slash commands in the above tests, show the slash command under the checklist item.

93

A checklist item can only be configured with one slash command at most

1. Create a new playbook “P1a”.

2. Add a checklist item “C1”.

3. In the slash command field for “C1”, enter the following: /away /online /shrug.

4. Verify that a warning appears that the checklist item can only be configured with one slash command.

Valid?

I think this should be allowed (there’s no technical reason it can’t). Ian Tao (Unlicensed) ?

94

Typing an invalid slash command

There’s nothing preventing the user from typing an invalid slash command.

Test ID

Test Case

Test Steps

Result

Notes

95

Typing a / in the slash command field brings up a list of existing slash commands

1. Log in as user-1.

2. Launch playbooks backstage.

3. Click “New Playbook”.

4. Enter “PBa” as the playbook name.

5. Edit the name of the default stage to “S1”.

6. Click “Add new stage”.

7. Enter “C1” for checklist item name.

8. In the slash command field, enter /

9. Verify that a autocomplete list of all available slash commands pop up as soon as entering /.

96

Typing a non-/ character in the slash command field?

Should this be allowed?

97

Selecting from the slash command list auto completes the slash command

Continue from the above tests.

1. In the slash command field, enter /.

2. From the autocomplete list, click /dnd.

3. Verify that the slash command field is autocompleted with /dnd.

98

Typing a non-existing slash command shows a “invalid slash command” error

Valid?

We do not validate the slash commands at the moment.

99

The slash command field can be left empty

Continue from the above tests.

1. Remove the contents of “C1” so that the slash command field is empty.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

Test ID

Test Case

Test Steps

Result

Notes

100

A checklist item configured without a slash command does not have a Run button next to it

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “Apb” as the playbook name.

5. Change the name of “Default Stage” to “S1”.

6. Click “Add new checklist item”.

7. Enter “C1” as the item name.

8. Don’t add anything for the slash command.

9. Save playbook.

10. Switch to normal channel view.

11. Start a new incident using “Apb”.

12. Once the incident is started, in the incident RHS, examine the checklist item “C1” under “S1”.

13. Verify that “C1” does not have any “Run” button.

101

A checklist item configured with a valid slash command has a Run button next to it

Continue from the above test.

1. Launch playbook.

2. Find and click “Apb” to edit.

3. Add slash command /away for “C1”.

4. Switch to normal channel view.

5. Start a new incident using “Apb”.

6. Once the incident is started, in the incident RHS, verify that “C1” show /away under it which has a Run button next to it.

102

A checklist item configured with an invalid slash command does not have a Run button next to it

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Add a new checklist item “C2” under “S1”.

4. Add slash command invalid text for “C2”.

5. Save playbook.

6. Switch to normal channel view.

7. Start a new incident using “Apb”.

8. Once the incident has started, find “C2” under “S1”.

9. Verify that the slash command for “C2” doesn’t have a Run button next to it.

This doesn’t happen currently. It would require us to spend mana on validating the slash command.

103

A checklist item configured with a nonexistent slash command does not have a Runbutton next to it

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Add a new checklist item “C3” under “S1”.

4. Add slash command /invalidcommand for “C3”.

5. Save playbook.

6. Switch to normal channel view.

7. Start a new incident “Ia1” using “Apb”.

8. Once the incident has started, find “C3” under “S1”.

9. Verify that the slash command for “C3” doesn’t have a Run button next to it.

104

Clicking the Run button for a checklist slash command runs the slash command

Continue from the above test.

1. Open up the incident RHS for “Ia1”.

2. Find “C1” under stage “S1”.

3. Click the Run button next to “C1”.

4. Verify that clicking the Run button executes so that the status of user-1 is now set to “away”.

105

Plugin posts a message in the incident channel when a slash command Runis clicked

Continue from the above test.

1. Verify that plugin posts a message in the “Ia1” incident channel to notify that the slash command for “C1” has been run.

A message that the checklist item slash command was run (not the slash command system message)

Ian Tao (Unlicensed) should we make a ticket for this feature? We currently don’t do it, but it makes sense.

106

Running a slash command with system message attached to it post the slash command system message in the incident channel

Continue from the above test.

1. Verify that the system message for the /away command “System: You are now away (Only visible to you)” is posted in the incident channel.

107

In case of two valid slash commands, the running the command from the checklist item will only run the first command successfully

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C5”.

4. Add the following for “C5’s” slash command: /online /away.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C5” in the incident RHS.

8. Click the Run button next to “C5”.

9. Verify that the status of user-1 is now set to “online” and not “away”.

This is in sync with the core feature behavior

Seems correct to me (ChrisP)

108

Running an invalid slash command from RHS checklist shows an ‘invalid slash command’ error

Depends on the steps in #102.

Like core feature?

109

A slash command can be run multiple times from the RHS checklist

Continue from the above tests.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C6”.

4. Add /shrug as the slash command for “C6”.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C6” in the incident RHS.

8. Click the Run button for “C6”.

9. Verify that the /shrug command is executed successfully and a shrug message is posted in the center channel.

10. Repeat steps 8-9 multiple times.

110

A plugin post notifies when a slash command fails to run successfully

Verify that when a slash command fails to execute by clicking the Run button, the plugin posts a message in the center channel notifying that the execution has failed.

Test ID

Test Case

Test Steps

Result

Notes

111

Checklist item description

1. Create a playbook with checklist items that have markdown descriptions.

2. Start an incident using such playbook.

3. Make sure that hovering over the icon changes its opacity.

4. Verify that clicking on the icon opens up the popover.

5. Verify that the popover renders markdown correctly (included channel links and mentions).

6. Verify that clicking on the X icon closes the popover.

https://mattermost.atlassian.net/browse/MM-27016

112

Add pagination to playbook list

1. Navigate to backstage -> playbooks.

2. Create more than 16 playbooks.

3. Observe the pagination options at the bottom.

https://mattermost.atlassian.net/browse/MM-27344

113

Browse Playbook templates

1. Navigate to backstage > playbooks.

2. Click on any of the new templates.

https://mattermost.atlassian.net/browse/MM-27028

114

Channel link capability to the workflow markdown

A channel link in the checklist text (it has to be correct), such as ~town-square, should:

• be correctly highlighted like it is in the center channel

• be clickable and bring the user to that channel, /without/ refreshing the app.

https://mattermost.atlassian.net/browse/MM-26924

115

Test new playbook creation with Blank Playbook template

116

Test new playbook creation with Incident Response Playbook

117

Show workflow duration

1. Verify that total duration of the workflow is displayed in the RHS.

2. This value should update in realtime without requiring any refresh. Show days, hours, minutes and seconds as necessary.

https://mattermost.atlassian.net/browse/MM-27014

118

Return paginated response for all list endpoints

1. Make sure you can see multiple incident pages in backstage.

2. Make sure you can view existing playbooks.

https://mattermost.atlassian.net/browse/MM-26179

https://docs.google.com/document/d/1Kp2PmE8CDNUSUiviui4GhL51z5OmYf5QqrcxFu2YFJU/edit#heading=h.t5ecjz3pjyu3

119

Empty playbook landing

1. Open backstage.

2. Click playbooks.

3. Make sure no playbooks exists to see the empty state.

https://mattermost.atlassian.net/browse/MM-27027