Test Plan - Incident Response v0.6 + 7

0.0

Initial version

08/07

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

9

Plugin upgrades normally from v0.4 to v0.6

1. Install version 0.4 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

10

Plugin upgrades normally from v0.5 to v0.6

1. Install version 0.5 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

11

Incidents started with the prior plugin versions are still active and available in v0.6

1. Start a few incident with plugin version 0.5.

2. Upgrade plugin to v0.6.

3. Open up the incident RHS to see the incident list.

4. Verify that incidents started with version 0.5 are still available in v0.6.

12

Incidents ended with prior plugin versions are not available in v0.6

1. Start a few incident with plugin version 0.5.

2. Leave some as active and end a few of them.

3. Upgrade plugin to v0.6.

4. Open up the incident RHS to see the incident list.

5. Verify that incidents started with version 0.5 are still available in v0.6.

6. Verify that incidents ended in step 2 are not visible in the list.

13

Playbook started with the older plugin version is still available upon plugin upgrade

1. Create a few playbooks with plugin version 0.5.

2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Test ID

Test Case

Test Steps

Result

Notes

14

Playbook creator can edit playbook

1. Log in as user-0.

2. Launch playbook backstage (using any method).

3. Click the “New Playbook” button.

4. Enter “P1” for Playbook Name.

5. Click “Save Playbook”.

6. Switch to a normal channel.

7. Bring up the incident creation modal (using any method).

8. Verify that the playbook dropdown has “P1”.

9. From the playbook backstage, find P1.

10. Click P1 to edit the playbook.

11. Change the playbook name to “PB-Private”.

12. Click “Save Playbook”.

13. Verify that the playbook in the backstage shows up as “PB-Private”.

14. Switch to a normal channel.

15. Bring up the incident creation modal.

16. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

15

Any playbook member can edit playbook to add a new member

Continue from the above test.

1. From the playbook backstage, find PB-Private.

2. Click on the playbook to open the “Edit Playbook” view

3. In the “Members” field, add @user-1.

4. Log in as user-1.

5. Bring up the incident creation modal.

6. Verify that user-1 can see “PB-Private” in the playbook dropdown.

7. Launch playbook backstage.

8. Find “PB-Private” and click to open the “Edit Playbook” view.

9. In the “Members” field, add @user-2.

10. Click “Save Playbook”.

11. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

16

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” and click to open the “Edit Playbook” view.

4. From the “Members” field, remove @user-1.

5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

17

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

1. As user-2, launch playbook backstage.

2. Find “PB-Private” and click to open the “Edit Playbook” view.

3. From the “Members” field, remove all users including user-0 and user-2.

4. Verify that the playbook cannot be saved.

5. Add user-1 to the Members field.

6. Save playbook.

The “Save Playbook” button should stay inactive unless there is at least one member

18

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Toggle the “Create Public Incident” to enable the setting.

4. Save playbook.

5. Click the playbook again to view the edit mode.

6. Verify that the “Create Public Incident” setting is enabled.

19

Any playbook member can edit playbook to add a new stage

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Click + Add new stage.

4. Change the stage name from “New stage” to “Stage 2”.

5. Save playbook.

6. Click the playbook again to view the edit mode.

7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

20

A playbook can have multiple stages with the same stage name

1. As user-1, launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “PB-New” for Playbook Name.

4. Enter “Stage 1” as the 1st stage name.

5. Click “Add new stage”.

6. Enter “Stage 1” as the 2nd stage name.

7. ??

Valid?

21

A playbook member can edit playbook to add a new checklist item to a stage added by themself

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Under “Stage 1”, click “Add new checklist item”.

4. Enter “S1-C1” as the checklist item.

5. Click “Save Playbook”.

6. From the playbook list in the backstage find “PB-Private”.

7. Click to view the edit view.

8. Verify that “Stage 1” has “S1-C1” as the checklist item.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the checklist item as been added to “Stage 1”.

22

Any playbook member can edit playbook to add a new checklist item to a stage added by a different member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 1”, click “Add new checklist item”.

6. Enter “S1-U2” as the checklist item.

7. Click “Save Playbook”.

8. From the playbook list in the backstage find “PB-Private”.

9. Click to view the edit view.

10. Verify that “Stage 1” has “S1-U2” as the checklist item.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify the checklist item has been added to “Stage 1”.

23

A playbook member cannot edit playbook such that the playbook name field is empty

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Delete the playbook name “PB-Private”.

5. Verify that the “Save Playbook” button becomes deactivate.

6. Click on the “Save Playbook” button.

7. Verify that the playbook cannot be saved, and the user is still in the edit view.

8. Enter a space in the playbook name field.

9. Repeat steps 5-7.

24

Any playbook member can reorder the checklist items within a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-C1” and drag it under “S1-U2”.

5. Verify that “S1-C1” is successfully moved under “S1-U2”.

6. Click the hamburger icon next to “S1-U2” and drag it above “S1-C1”.

7. Verify that “S1-U2” is successfully moved under “S1-C1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the reordering was successful.

25

Any playbook member can reorder the checklist items within different stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-U2” and drag it under the 2nd stage “Stage 2”.

5. Verify that “S1-U2” is successfully moved under “Stage 2”.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the reordering was successful.

26

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to the checklist item “S1-C1” and drag it right above “Stage 1”.

5. Verify that the checklist item does not move from its original spot under “Stage 1”.

6. Drag and drop “S1-C1” outside stage area.

7. Verify that the checklist item does not move from its original spot under “Stage 1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the order of the checklist items is normal.

27

Any playbook member can delete a checklist item

Continue from the above test.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 2” find “S1-U2” and hover the mouse over so that the “X” appears next to it.

6. Click the “X” for “S1-U2”.

7. Verify that the checklist item “S1-U2” is deleted.

8. Click “Save Playbook”.

9. Click “PB-Private” to go to edit view.

10. Verify that the checklist item “S1-U2” is not available in any stage.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify that “S1-U2” is not available in any stage.

28

A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” next to all checklist items of all stages.

5. Verify that all checklist items are deleted.

6. Click “Save Playbook”.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that none of stages have any checklist items.

29

Any playbook member can reorder the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Add a new stage “Stage 3”.

5. Click the hamburger icon next to “Stage 1” and drag it below “Stage 3”.

6. Verify that “Stage 1” is successfully moved under “Stage 3”.

7. Click the hamburger icon next to “Stage 3” and drag it above “Stage 2”.

8. Verify that “Stage 3” is successfully moved above “Stage 2”.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the stage reordering was successful.

30

A playbook member cannot reorder the playbook such that the stages can be nested

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to “Stage 1” and drag it under “Stage 3” to align it within the checklist items of “Stage 3”.

5. Verify that “Stage 1” is not moved from its original position.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the stage reordering was not successful.

Verify that the stages cannot be stacked inside another stage

31

Any playbook member can delete a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 3”.

5. Verify that “Stage 3” is deleted.

6. Save playbook.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that the incident created does not have “Stage 3”.

32

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 1”.

5. Click the “X” button next to “Stage 2”.

6. Verify that the “Save Playbook” button becomes inactive.

7. Click on the “Save Playbook”.

8. Verify that the user is still in the edit view of “PB-Private” and the action was not saved.

33

Any playbook member can delete playbook

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Under “Actions” for the playbook, click the ellipses.

5. Click “Delete”.

6. Verify that a “Confirm Playbook Deletion” modal appears.

7. Click “Delete Playbook”.

8. Verify that “PB-Private” is not present in the playbook list.

9. Switch to a normal channel view.

10. Open up an incident modal.

11. Verify that “PB-Private” is not available in the playbook dropdown.

34

A playbook member can create playbook with a large number of stages

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Click “Add new stage” 20 or more times to create a long list of stages.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Start an incident using “PB-N1” as the playbook.

9. Verify that the Stages dropdown in the incident RHS has all the stages added in step 5.

10. Verify the Stage dropdown displays the stages properly with scrollbar.

Verify playbook with a large number of stages in the edit page

35

A playbook member can create playbook with a large number of checklist items within the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-N1” and click to go to edit view.

3. For the first stage, add at least 20 checklist items.

4. Save playbook.

5. Switch to a normal channel.

6. Start an incident using “PB-N1” as the playbook.

7. In the incident RHS, ensure that the first stage is selected from the dropdown.

8. Verify that all the checklist items added in step 3 are available.

Verify playbook with a large number of checklist items within the stages in the edit page

36

The components in the playbook edit page render normally

Verify the appearance of the components in the playbook edit page.

Test ID

Test Case

Test Steps

Result

Notes

37

An active stage is annotated with Activelabel in the RHS

38

Non-active stages are not annotated with Active labels

39

Incident commander can set a different stage as Active in an ongoing incident

40

Workflow member can set a different stage as Active in an ongoing incident

41

Workflow member cannot set a different stage as Active in an incident that has been ended

42

Workflow member can set a different stage as Active in an incident that has been restarted

43

Plugin posts a message in the incident channel when changing an active stage

44

Workflow member views the active stage by default in incident RHS

45

When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the Active annotation

46

In case of only one stage, the stage is set as active by default

47

In case of multiple stages, the first stage is set as active by default

48

Switching between the stages does not set the stage as active

49

Selecting a non-active stage from the RHS dropdown shows a Make Active option prompt

50

Selecting an active stage from the RHS dropdown does not show a Make Active option prompt

51

Workflow member can check items off of stages that are active

52

Workflow member can uncheck items of stages that are not active

53

Workflow member can check off items of an ongoing incident

54

Workflow member cannot checkoff items of an ended incident

55

Workflow member can checkoff items of a restarted incident

Test ID

Test Case

Test Steps

Result

Notes

56

The Stages dropdown in the RHS has all the stages for that incident

57

Selecting a different the stage in the RHS shows the checklist corresponding to that stage

58

Saving a playbook with only one stage and leaving the stage name empty, will create an incident with stage “Default Stage”

59

Any workflow member can switch between the stages in the RHS in an ongoing incident

60

Any workflow member can switch between the stages in the RHS in an incident that has been ended

61

Any workflow member can switch between the stages in the RHS in an incident that has been restarted

62

Selecting a different stage will not change the default stage view upon page refresh

Refreshing the page or navigating back to the incident channel will always show the Active stage in the RHS and not the stage selected earlier

63

Selecting a different stage will not change the stage view for a different workflow member

Test ID

Test Case

Test Steps

Result

Notes

64

Checklist item in the RHS has a dropdown for assignee

65

The assignee dropdown shows the list of workflow members

66

The assignee dropdown does not show users that are not workflow members

67

By default, checklist item does not have any assignee

68

A checklist item can be checked off without an assignee

69

A checklist item can be unchecked without an assignee

70

Workflow member can select him/herself as the checklist assignee

71

Workflow member can select a different workflow member as the checklist assignee

72

Workflow member can change assignee from him/herself to ‘No Assignee’ for a checklist item

73

Workflow member can change another workflow member to ‘No Assignee’ for a checklist item

74

Plugin posts a message in incident channel when an assignee is selected

75

Plugin posts a message in incident channel when ‘No Assignee’ is selected

76

Workflow member cannot select multiple assignees

77

Workflow member cannot select a different assignee in an incident that is ended

78

Workflow member can select a different assignee in an incident that is restarted

79

Assignee does not change when a workflow member leaves the workflow channel

80

Assignee does not change when a workflow member is kicked from the workflow channel

81

The assignee dropdown list gets updated when a new member is added to the workflow channel

82

The assignee dropdown list gets updated when a workflow member leaves the channel

83

The assignee dropdown list gets updated when a workflow member is kicked from the channel

84

An assignee can checkoff a checklist item

85

An assignee can uncheck a checklist item

86

A non-assignee can checkoff a checklist item

87

A non-assignee can uncheck a checklist item

88

Anything in the backstage about assignee?

Test ID

Test Case

Test Steps

Result

Notes

89

A newly added checklist item in a playbook has a slash command field

90

A playbook can be edited to add a slash command to any checklist item

91

A checklist item configured with a slash command shows the slash command for that item in the RHS

92

A checklist item can only be configured with one slash command at most

93

Typing an invalid slash command

Test ID

Test Case

Test Steps

Result

Notes

Typing a / in the slash command field brings up a list of existing slash commands

Typing a non-/ character in the slash command field?

Should this be allowed?

Selecting from the slash command list auto completes the slash command

Typing a non-existing slash command shows a “invalid slash command” error

The slash command field can be left empty

Test ID

Test Case

Test Steps

Result

Notes

A checklist item configured without a slash command does not have a Run button next to it

A checklist item configured with a valid slash command has a Run button next to it

A checklist item configured with an invalid slash command does not have a Run button next to it

A checklist item configured with a nonexistent slash command does not have a Runbutton next to it

Clicking the Run button for a checklist slash command runs the slash command

Plugin posts a message in the incident channel when a slash command Runis clicked

A message that the checklist item slash command was run (not the slash command system message)

Running a slash command with system message attached to it post the slash command system message in the incident channel

In case of two valid slash commands, the running the command from the checklist item will only run the first command successfully

This is in sync with the core feature behavior

Running an invalid slash command from RHS checklist shows an ‘invalid slash command’ error

Like core feature?

A slash command can be run multiple times from the RHS checklist

A plugin post notifies when a slash command fails to run successfully

Test ID

Test Case

Test Steps

Result

Notes