Page Comparison

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

9

Plugin upgrades normally from v0.4 to v0.6

1. Install version 0.4 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

10

Plugin upgrades normally from v0.5 to v0.6

1. Install version 0.5 of the plugin.

2. Upgrade to the latest version.

3. Verify that the plugin upgrade happens without any error.

11

Incidents started with the prior plugin versions are still active and available in v0.6

1. Start a few incident with plugin version 0.5.

2. Upgrade plugin to v0.6.

3. Open up the incident RHS to see the incident list.

4. Verify that incidents started with version 0.5 are still available in v0.6.

12

Incidents ended with prior plugin versions are not available in v0.6

1. Start a few incident with plugin version 0.5.

2. Leave some as active and end a few of them.

3. Upgrade plugin to v0.6.

4. Open up the incident RHS to see the incident list.

5. Verify that incidents started with version 0.5 are still available in v0.6.

6. Verify that incidents ended in step 2 are not visible in the list.

13

Playbook started with the older plugin version is still available upon plugin upgrade

1. Create a few playbooks with plugin version 0.5.

2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Test ID

Test Case

Test Steps

Result

Notes

14

Playbook creator can edit playbook

1. Log in as user-0.

2. Launch playbook backstage (using any method).

3. Click the “New Playbook” button.

4. Enter “P1” for Playbook Name.

5. Click “Save Playbook”.

6. Switch to a normal channel.

7. Bring up the incident creation modal (using any method).

8. Verify that the playbook dropdown has “P1”.

9. From the playbook backstage, find P1.

10. Click P1 to edit the playbook.

11. Change the playbook name to “PB-Private”.

12. Click “Save Playbook”.

13. Verify that the playbook in the backstage shows up as “PB-Private”.

14. Switch to a normal channel.

15. Bring up the incident creation modal.

16. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

15

Any playbook member can edit playbook to add a new member

Continue from the above test.

1. From the playbook backstage, find PB-Private.

2. Click on the playbook to open the “Edit Playbook” view

3. In the “Members” field, add @user-1.

4. Log in as user-1.

5. Bring up the incident creation modal.

6. Verify that user-1 can see “PB-Private” in the playbook dropdown.

7. Launch playbook backstage.

8. Find “PB-Private” and click to open the “Edit Playbook” view.

9. In the “Members” field, add @user-2.

10. Click “Save Playbook”.

11. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

16

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” and click to open the “Edit Playbook” view.

4. From the “Members” field, remove @user-1.

5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

17

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

1. As user-2, launch playbook backstage.

2. Find “PB-Private” and click to open the “Edit Playbook” view.

3. From the “Members” field, remove all users including user-0 and user-2.

4. Verify that the playbook cannot be saved.

5. Add user-1 to the Members field.

6. Save playbook.

The “Save Playbook” button should stay inactive unless there is at least one member

18

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Toggle the “Create Public Incident” to enable the setting.

4. Save playbook.

5. Click the playbook again to view the edit mode.

6. Verify that the “Create Public Incident” setting is enabled.

19

Any playbook member can edit playbook to add a new checklist item to a stage added by a different memberAny new stage

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Click + Add new stage.

4. Change the stage name from “New stage” to “Stage 2”.

5. Save playbook.

6. Click the playbook again to view the edit mode.

7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

20

A playbook can have multiple stages with the same stage name

Valid?

21

A playbook member can edit playbook to add a new checklist item to a stage A added by themself

22

Any playbook member can edit playbook to add a new checklist item to a stage added by themselfa different member

23

A playbook member cannot edit playbook such that the playbook name field is empty

24

Any playbook member can reorder the checklist items within a stage

25

Any playbook member can reorder the checklist items within different stages

26

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

27

Any playbook member can delete a checklist item

28

A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all

29

Any playbook member can reorder the stages

30

A playbook member cannot reorder the playbook such that the stages can be nested

Verify that the stages cannot be stacked inside another stage

31

Any playbook member can delete a stage

32

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

33

Any playbook member can delete playbook

34

A playbook member can create playbook with a large number of stages

Verify playbook with a large number of stages in the edit page

35

A playbook member can create playbook with a large number of checklist items within the stages

Verify playbook with a large number of checklist items within the stages in the edit page

36

The components in the playbook edit page render normally

Verify the appearance of the components in the playbook edit page

Test ID

Test Case

Test Steps

Result

Notes

37

An active stage is annotated with Activelabel in the RHS

38

Non-active stages are not annotated with Active labels

39

Incident commander can set a different stage as Active in an ongoing incidentWorkflow owner can change the active stage in incident RHS

40

Workflow member can set a different stage as Active in an ongoing incident

41

Workflow member cannot set a different stage as Active in an incident that has been ended

42

Workflow member can set a different stage as Active in an incident that has been restarted

43

Plugin posts a message in the incident channel when changing an active stage

44

Workflow member views the active stage by default in incident RHS

45

When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the Active annotation

46

In case of only one stage, the stage is set as active by default

47

In case of multiple stages, the first stage is set as active by default

48

Switching between the stages does not set the stage as active

49

Selecting a non-active stage from the RHS dropdown shows a Make Active option prompt

50

Selecting an active stage from the RHS dropdown does not show a Make Active option prompt

51

Workflow member can check items off of stages that are active

52

Workflow member can uncheck items of stages that are not active

53

Workflow member can check off items of an ongoing incident

54

Workflow member cannot checkoff items of an ended incident

55

Workflow member can checkoff items of a restarted incident