/
Test Plan - Incident Response v0.6 + 7

Test Plan - Incident Response v0.6 + 7

Document version 1.0

Status: Test Completed

Document Version

Description

Date

Document Version

Description

Date

0.0

Initial version

08/07

0.1

Added tests for v0.7.

Added test steps for the test cases.

Test plan ready for review.

08/16

1.0

Tests updated.

Tests run and completed

08/21

References

Related Tests:

Summary

This document details the tests for Incident Response plugin v0.6.0. The tests are derived from the user stories in the requirements for v0.6. The MVP of the Incident Response plugin will be tested against the relevant tests included in this test plan if the feature is included.

Scope

This document covers the cases for the user stories as outlined in the Incident Response MVP Overview for v.0.6.0 of the plugin only.

Glossary

  • Admin - A user with system administrator privileges.

  • Normal user - A user with non-system administrator privileges.

  • Test Server - A list of test server versions used in testing including Mattermost server and Marketplace Server

  • Main menu - Hamburger menu on the LHS.

  • Incident Response icon - The Incident Response plugin icon that appears on the channel header

  • Commander - The user who owns an incident. The user who starts the incident is designated the commander by default.

  • Incident member/incident channel member - A user who has been added to the incident channel.

  • Non-incident user - A user who has not been added to the incident channel.

  • Active incident - An incident that has not been ended.

  • Ongoing incident - Same as Active Incident.

  • Inactive incident - An incident that has been ended.

  • Incident RHS - The RHS sidebar that opens upon clicking the incident response icon and displays incidents.

  • Incident Backstage - The backstage view that shows the incident list.

  • Playbook Backstage - The backstage view that shows the playbook list.

Assumptions

The tests in this test plans are written with the assumption that:

  1. The tests outlined in this document is to be executed against the Incident Response plugin v0.6.0.

  2. This version of the plugin can be installed and used in Mattermost instances that aren’t equipped with any enterprise license.

  3. The Incident Response plugin can function as a standalone plugin that does not require to talk to any other Mattermost plugin.

  4. This version of the Incident Response plugin will function properly only in the latest version of Mattermost.

  5. The tests will be executed in the latest version of Mattermost.

  6. Testing is primarily done on browser (Chrome), with spot checks on other web browsers, desktop app, RN mobile app and mobile web browser app.

  7. The tests in this document are executed as a normal user (see glossary above) unless otherwise specified in the tests.

  8. The test cases that pass for a normal user will also pass for a system administrator.

  9. Plugin uploads is enabled in server instance

    1. “EnableUploads” option under “PluginSettings” in mattermost-server’s config.json is set to 'true'.

  10. The channel export feature defined in the user story is dependent on the existence of the Channel Export plugin being installed and enabled in the test server.

  11. Private channels cannot be converted into public channel, hence private incidents cannot be converted to public incidents.

Setup

Document Setup

For the practicality of running tests easily, test cases are arranged in sequential order when necessary, following the preceding test case.

Test Setup

The following setup will be necessary in order to begin testing:

  • A Mattermost test server running the latest version.

  • The test server may not need to be equipped with an EE license.

  • Incident Plugin v0.6.

  • Channel Export Plugin v0.2

Test Server: https://ir-test.test.mattermost.cloud/

Test Server (upgrade tests): https://ir-upgrade-test.test.mattermost.cloud/

Build Hash: fb453d
EE Build Hash: 7251aa
Webapp Build Hash: d0bcba

Incident Response plugin version: v0.6.0-alpha.3 (build hash d60f2f)

Test Date: 08/21/2020

Tests

Test Area - Plugin Setup

Test Area - Plugin Setup

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

  1. Log in to a non-enterprise Mattermost, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

 

2

Admin can install plugin in an E10 instance

  1. Log in to Mattermost that has an E10 license, as admin.

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin is successfully installed.

Pass

 

3

Admin can install plugin in an E20 instance

  1. Log in to Mattermost that has an E20 license, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

 

4

Admin can enable plugin

  1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

  2. Verify that the plugin starts normally without errors.

Pass

 

5

Plugin functionality is available to admin

  1. Switch to a normal channel.

  2. Type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

6

Plugin functionality is available to normal user

  1. Log in as a non-admin user.

  2. In a normal channel, type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

7

Admin can disable plugin

  1. Log out and log back in as admin.

  2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  3. Click on the “Disable” button.

  4. Verify that the plugin is disabled.

  5. In the main channel’s post input box, type /incident.

  6. Verify that /incident option is not present in the slash command list.

Pass

 

8

Admin can remove plugin

  1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  2. Click on the “Remove” button.

  3. Verify that the plugin is no longer installed in the instance.

Pass

 

9

Plugin upgrades normally from v0.4 to v0.6

  1. Install version 0.4 of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

Pass

 

10

Plugin upgrades normally from v0.5 to v0.6

  1. Install version 0.5 of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

Pass

 

11

Incidents started with the prior plugin versions are still active and available in v0.6

  1. Start a few incident with plugin version 0.5.

  2. Upgrade plugin to v0.6.

  3. Open up the incident RHS to see the incident list.

  4. Verify that incidents started with version 0.5 are still available in v0.6.

Pass

 

12

Incidents ended with prior plugin versions are not available in v0.6

  1. Start a few incident with plugin version 0.5.

  2. Leave some as active and end a few of them.

  3. Upgrade plugin to v0.6.

  4. Open up the incident RHS to see the incident list.

  5. Verify that incidents started with version 0.5 are still available in v0.6.

  6. Verify that incidents ended in step 2 are not visible in the list.

Pass

 

13

Playbook started with the older plugin version is still available upon plugin upgrade

  1. Create a few playbooks with plugin version 0.5.

  2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Pass

 

Test Area - Create stages when editing a playbook

Test Area - Create stages when editing a playbook

Test ID

Test Case

Test Steps

Result

Notes

14

Playbook creator can edit playbook

  1. Log in as user-0.

  2. Launch playbook backstage (using any method).

  3. Click the “New Playbook” button.

  4. Enter “P1” for Playbook Name.

  5. Enter stage “1”.

  6. Click “Save Playbook”.

  7. Switch to a normal channel.

  8. Bring up the incident creation modal (using any method).

  9. Verify that the playbook dropdown has “P1”.

  10. From the playbook backstage, find P1.

  11. Click P1 to edit the playbook.

  12. Change the playbook name to “PB-Private”.

  13. Click “Save Playbook”.

  14. Verify that the playbook in the backstage shows up as “PB-Private”.

  15. Switch to a normal channel.

  16. Bring up the incident creation modal.

  17. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

Pass

 

15

Any playbook member can edit playbook to add a new member

Continue from the above test.

  1. From the playbook backstage, find PB-Private.

  2. Click on the playbook to open the “Edit Playbook” view

  3. In the “Members” field, add @user-1.

  4. Save playbook.

  5. Log in as user-1.

  6. Bring up the incident creation modal.

  7. Verify that user-1 can see “PB-Private” in the playbook dropdown.

  8. Launch playbook backstage.

  9. Find “PB-Private” and click to open the “Edit Playbook” view.

  10. In the “Members” field, add @user-2.

  11. Click “Save Playbook”.

  12. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

Pass

 

16

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

  1. Log in as user-2.

  2. Launch playbook backstage.

  3. Find “PB-Private” and click to open the “Edit Playbook” view.

  4. From the “Members” field, remove @user-1.

  5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

Pass

 

17

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

  1. As user-2, launch playbook backstage.

  2. Find “PB-Private” and click to open the “Edit Playbook” view.

  3. From the “Members” field, remove all users including user-0 and user-2.

  4. Verify that the playbook cannot be saved.

  5. Add user-1 to the Members field.

  6. Save playbook.

Pass

The “Save Playbook” button should stay inactive unless there is at least one member

18

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

  1. As user-1, launch the playbook backstage.

  2. Find “PB-Private” and click to open the edit view.

  3. Toggle the “Create Public Incident” to enable the setting.

  4. Save playbook.

  5. Click the playbook again to view the edit mode.

  6. Verify that the “Create Public Incident” setting is enabled.

Pass

 

19

Any playbook member can edit playbook to add a new stage

Continue from the above test.

  1. As user-1, launch the playbook backstage.

  2. Find “PB-Private” and click to open the edit view.

  3. Click + Add new stage.

  4. Change the stage name from “New stage” to “Stage 2”.

  5. Save playbook.

  6. Click the playbook again to view the edit mode.

  7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

Pass

 

20

A playbook can have multiple stages with the same stage name

  1. As user-1, launch the playbook backstage.

  2. Click “New Playbook”.

  3. Enter “PB-New” for Playbook Name.

  4. Enter “Stage 1” as the 1st stage name.

  5. Click “Add new stage”.

  6. Enter “Stage 1” as the 2nd stage name.

  7. ??

Pass

Valid?

@Ian Tao (Unlicensed) confirm?

21

A playbook member can edit playbook to add a new checklist item to a stage added by themself

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Under “Stage 1”, click “Add new checklist item”.

  4. Enter “S1-C1” as the checklist item.

  5. Click “Save Playbook”.

  6. From the playbook list in the backstage find “PB-Private”.

  7. Click to view the edit view.

  8. Verify that “Stage 1” has “S1-C1” as the checklist item.

  9. Switch to a normal channel view.

  10. Start an incident with “PB-Private” to verify the checklist item as been added to “Stage 1”.

Pass

 

22

Any playbook member can edit playbook to add a new checklist item to a stage added by a different member

Continue from the above test.

  1. Log in as user-2.

  2. Launch playbook backstage.

  3. Find “PB-Private” from the playbook list.

  4. Click to open edit view.

  5. Under “Stage 1”, click “Add new checklist item”.

  6. Enter “S1-U2” as the checklist item.

  7. Click “Save Playbook”.

  8. From the playbook list in the backstage find “PB-Private”.

  9. Click to view the edit view.

  10. Verify that “Stage 1” has “S1-U2” as the checklist item.

  11. Switch to a normal channel view.

  12. Start an incident with “PB-Private” to verify the checklist item has been added to “Stage 1”.

Pass

 

23

A playbook member cannot edit playbook such that the playbook name field is empty

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Delete the playbook name “PB-Private”.

  5. Verify that the “Save Playbook” button becomes deactivate.

  6. Click on the “Save Playbook” button.

  7. Verify that the playbook cannot be saved, and the user is still in the edit view.

  8. Enter a space in the playbook name field.

  9. Repeat steps 5-7.

Pass

 

24

Any playbook member can reorder the checklist items within a stage

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-C1” and drag it under “S1-U2”.

  5. Verify that “S1-C1” is successfully moved under “S1-U2”.

  6. Click the hamburger icon next to “S1-U2” and drag it above “S1-C1”.

  7. Verify that “S1-U2” is successfully moved under “S1-C1”.

  8. Switch to a normal channel view.

  9. Start an incident with “PB-Private” to verify the reordering was successful.

Pass

 

25

Any playbook member can reorder the checklist items within different stages

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-U2” and drag it under the 2nd stage “Stage 2”.

  5. Verify that “S1-U2” is successfully moved under “Stage 2”.

  6. Switch to a normal channel view.

  7. Start an incident with “PB-Private” to verify the reordering was successful.

Pass

Cannot be reordered under a different stage without a step:

https://mattermost.atlassian.net/browse/MM-28024

26

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Click the hamburger icon next to the checklist item “S1-C1” and drag it right above “Stage 1”.

  5. Verify that the checklist item does not move from its original spot under “Stage 1”.

  6. Drag and drop “S1-C1” outside stage area.

  7. Verify that the checklist item does not move from its original spot under “Stage 1”.

  8. Switch to a normal channel view.

  9. Start an incident with “PB-Private” to verify the order of the checklist items is normal.

Pass

 

27

Any playbook member can delete a checklist item

Continue from the above test.

  1. Log in as user-1.

  2. Launch playbook backstage.

  3. Find “PB-Private” from the playbook list.

  4. Click to open edit view.

  5. Under “Stage 2” find “S1-U2” and hover the mouse over so that the “X” appears next to it.

  6. Click the “X” for “S1-U2”.

  7. Verify that the checklist item “S1-U2” is deleted.

  8. Click “Save Playbook”.

  9. Click “PB-Private” to go to edit view.

  10. Verify that the checklist item “S1-U2” is not available in any stage.

  11. Switch to a normal channel view.

  12. Start an incident with “PB-Private” to verify that “S1-U2” is not available in any stage.

Pass

 

28

A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Click the “X” next to all checklist items of all stages.

  5. Verify that all checklist items are deleted.

  6. Click “Save Playbook”.

  7. Switch to a normal channel view.

  8. Start an incident with “PB-Private” to verify that none of stages have any checklist items.

N/A

Invalid.

29

Any playbook member can reorder the stages

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Add a new stage “Stage 3”.

  5. Click the hamburger icon next to “Stage 1” and drag it below “Stage 3”.

  6. Verify that “Stage 1” is successfully moved under “Stage 3”.

  7. Click the hamburger icon next to “Stage 3” and drag it above “Stage 2”.

  8. Verify that “Stage 3” is successfully moved above “Stage 2”.

  9. Switch to a normal channel view.

  10. Start an incident with “PB-Private” to verify the stage reordering was successful.

Pass

 

30

A playbook member cannot reorder the playbook such that the stages can be nested

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Click the hamburger icon next to “Stage 1” and drag it under “Stage 3” to align it within the checklist items of “Stage 3”.

  5. Verify that “Stage 1” is not moved from its original position.

  6. Switch to a normal channel view.

  7. Start an incident with “PB-Private” to verify the stage reordering was not successful.

Pass

Verify that the stages cannot be stacked inside another stage

 

Step 5: The dragged item gets dropped below (not inside) whatever stage item it was dragged into.

31

Any playbook member can delete a stage

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Click the “X” button next to “Stage 3”.

  5. Verify that “Stage 3” is deleted.

  6. Save playbook.

  7. Switch to a normal channel view.

  8. Start an incident with “PB-Private” to verify that the incident created does not have “Stage 3”.

Pass

 

32

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

Continue from the above test.

  1. Launch playbook backstage.

  2. Find “PB-Private” from the playbook list.

  3. Click to open edit view.

  4. Click the “X” button next to “Stage 1”.

  5. Click the “X” button next to “Stage 2”.

  6. Verify that the “Save Playbook” button becomes inactive.

  7. Click on the “Save Playbook”.

  8. Verify that the user is still in the edit view of “PB-Private” and the action was not saved.

Fail

https://mattermost.atlassian.net/browse/MM-28025

33

Any playbook member can delete playbook