Test Plan - Incident Response v0.6 + 7

Document version 0.0

Status: Test Development in Progress

Document Version	Description	Date
0.0	Initial version	08/07

References

Related Tests:

Summary

This document details the tests for Incident Response plugin v0.6.0. The tests are derived from the user stories in the requirements for v0.6. The MVP of the Incident Response plugin will be tested against the relevant tests included in this test plan if the feature is included.

Scope

This document covers the cases for the user stories as outlined in the Incident Response MVP Overview for v.0.6.0 of the plugin only.

Glossary

Admin - A user with system administrator privileges.
Normal user - A user with non-system administrator privileges.
Test Server - A list of test server versions used in testing including Mattermost server and Marketplace Server
Main menu - Hamburger menu on the LHS.
Incident Response icon - The Incident Response plugin icon that appears on the channel header
Commander - The user who owns an incident. The user who starts the incident is designated the commander by default.
Incident member/incident channel member - A user who has been added to the incident channel.
Non-incident user - A user who has not been added to the incident channel.
Active incident - An incident that has not been ended.
Ongoing incident - Same as Active Incident.
Inactive incident - An incident that has been ended.
Incident RHS - The RHS sidebar that opens upon clicking the incident response icon and displays incidents.
Incident Backstage - The backstage view that shows the incident list.
Playbook Backstage - The backstage view that shows the playbook list.

Assumptions

The tests in this test plans are written with the assumption that:

The tests outlined in this document is to be executed against the Incident Response plugin v0.6.0.
This version of the plugin can be installed and used in Mattermost instances that aren’t equipped with any enterprise license.
The Incident Response plugin can function as a standalone plugin that does not require to talk to any other Mattermost plugin.
This version of the Incident Response plugin will function properly only in the latest version of Mattermost.
The tests will be executed in the latest version of Mattermost.
Testing is primarily done on browser (Chrome), with spot checks on other web browsers, desktop app, RN mobile app and mobile web browser app.
The tests in this document are executed as a normal user (see glossary above) unless otherwise specified in the tests.
The test cases that pass for a normal user will also pass for a system administrator.
Plugin uploads is enabled in server instance
1. “EnableUploads” option under “PluginSettings” in mattermost-server’s config.json is set to 'true'.
The channel export feature defined in the user story is dependent on the existence of the Channel Export plugin being installed and enabled in the test server.
Private channels cannot be converted into public channel, hence private incidents cannot be converted to public incidents.

Setup

Document Setup

For the practicality of running tests easily, test cases are arranged in sequential order when necessary, following the preceding test case.

Test Setup

The following setup will be necessary in order to begin testing:

A Mattermost test server running the latest version.
The test server may not need to be equipped with an EE license.
Incident Plugin v0.6.
Channel Export Plugin v0.2

Test Server:

Test Server (upgrade tests):

Build Hash:
EE Build Hash:
Webapp Build Hash:

Test Date:

Tests

Test Area - Plugin Setup
Test ID	Test Case	Test Steps	Result	Notes
1	Admin can install plugin in a non-EE instance	Log in to a non-enterprise Mattermost, as admin From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin. Verify that the plugin can be installed.
2	Admin can install plugin in an E10 instance	Log in to Mattermost that has an E10 license, as admin. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin. Verify that the plugin is successfully installed.
3	Admin can install plugin in an E20 instance	Log in to Mattermost that has an E20 license, as admin From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin. Verify that the plugin can be installed.
4	Admin can enable plugin	Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button. Verify that the plugin starts normally without errors.
5	Plugin functionality is available to admin	Switch to a normal channel. Type `/incident`in the post input box. Verify that `/incident`is listed as a slash command option. Verify that the channel header has Incident Response icon for RHS. Create a post. Click post action menu for the post. Verify that `Start Incident` is available as a post action menu option.
6	Plugin functionality is available to normal user	Log in as a non-admin user. In a normal channel, type `/incident`in the post input box. Verify that `/incident`is listed as a slash command option. Verify that the channel header has Incident Response icon for RHS. Create a post. Click post action menu for the post. Verify that `Start Incident` is available as a post action menu option.
7	Admin can disable plugin	Log out and log back in as admin. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin. Click on the “Disable” button. Verify that the plugin is disabled. In the main channel’s post input box, type `/incident`. Verify that `/incident` option is not present in the slash command list.
8	Admin can remove plugin	From System Console > PLUGINS > Plugin Management, find the Incident Response plugin. Click on the “Remove” button. Verify that the plugin is no longer installed in the instance.
9	Plugin upgrades normally from v0.4 to v0.6	Install version 0.4 of the plugin. Upgrade to the latest version. Verify that the plugin upgrade happens without any error.
10	Plugin upgrades normally from v0.5 to v0.6	Install version 0.5 of the plugin. Upgrade to the latest version. Verify that the plugin upgrade happens without any error.
11	Incidents started with the prior plugin versions are still active and available in v0.6	Start a few incident with plugin version 0.5. Upgrade plugin to v0.6. Open up the incident RHS to see the incident list. Verify that incidents started with version 0.5 are still available in v0.6.
12	Incidents ended with prior plugin versions are not available in v0.6	Start a few incident with plugin version 0.5. Leave some as active and end a few of them. Upgrade plugin to v0.6. Open up the incident RHS to see the incident list. Verify that incidents started with version 0.5 are still available in v0.6. Verify that incidents ended in step 2 are not visible in the list.
13	Playbook started with the older plugin version is still available upon plugin upgrade	Create a few playbooks with plugin version 0.5. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Test Area - Create stages when editing a playbook
Test ID	Test Case	Test Steps	Result	Notes
14	Playbook creator can edit playbook	Log in as user-0. Launch playbook backstage (using any method). Click the “New Playbook” button. Enter “P1” for Playbook Name. Click “Save Playbook”. Switch to a normal channel. Bring up the incident creation modal (using any method). Verify that the playbook dropdown has “P1”. From the playbook backstage, find P1. Click P1 to edit the playbook. Change the playbook name to “PB-Private”. Click “Save Playbook”. Verify that the playbook in the backstage shows up as “PB-Private”. Switch to a normal channel. Bring up the incident creation modal. Verify that the playbook dropdown does not have “P1” but “PB-Private”.
15	Any playbook member can edit playbook to add a new member	Continue from the above test. From the playbook backstage, find PB-Private. Click on the playbook to open the “Edit Playbook” view In the “Members” field, add `@user-1`. Log in as user-1. Bring up the incident creation modal. Verify that user-1 can see “PB-Private” in the playbook dropdown. Launch playbook backstage. Find “PB-Private” and click to open the “Edit Playbook” view. In the “Members” field, add `@user-2`. Click “Save Playbook”. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.
16	Any playbook member can edit playbook to remove an existing member	Continue from the above test. Log in as user-2. Launch playbook backstage. Find “PB-Private” and click to open the “Edit Playbook” view. From the “Members” field, remove `@user-1`. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.
17	A playbook member cannot edit playbook to remove all the members of the playbook	Continue from the above test. As user-2, launch playbook backstage. Find “PB-Private” and click to open the “Edit Playbook” view. From the “Members” field, remove all users including user-0 and user-2. Verify that the playbook cannot be saved. Add user-1 to the Members field. Save playbook.		The “Save Playbook” button should stay inactive unless there is at least one member
18	Any playbook member can edit playbook to change the private/public incident setting	Continue from the above test. As user-1, launch the playbook backstage. Find “PB-Private” and click to open the edit view. Toggle the “Create Public Incident” to enable the setting. Save playbook. Click the playbook again to view the edit mode. Verify that the “Create Public Incident” setting is enabled.
19	Any playbook member can edit playbook to add a new stage	Continue from the above test. As user-1, launch the playbook backstage. Find “PB-Private” and click to open the edit view. Click `+ Add new stage`. Change the stage name from “New stage” to “Stage 2”. Save playbook. Click the playbook again to view the edit mode. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.
20	A playbook can have multiple stages with the same stage name			Valid?
21	A playbook member can edit playbook to add a new checklist item to a stage added by themself
22	Any playbook member can edit playbook to add a new checklist item to a stage added by a different member
23	A playbook member cannot edit playbook such that the playbook name field is empty
24	Any playbook member can reorder the checklist items within a stage
25	Any playbook member can reorder the checklist items within different stages
26	A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage
27	Any playbook member can delete a checklist item
28	A playbook member can edit the playbook to delete all the checklist items such that all the stages can have no checklist item at all
29	Any playbook member can reorder the stages
30	A playbook member cannot reorder the playbook such that the stages can be nested			Verify that the stages cannot be stacked inside another stage
31	Any playbook member can delete a stage
32	A playbook member cannot delete the last remaining stage such that the playbook has no stages at all
33	Any playbook member can delete playbook
34	A playbook member can create playbook with a large number of stages			Verify playbook with a large number of stages in the edit page
35	A playbook member can create playbook with a large number of checklist items within the stages			Verify playbook with a large number of checklist items within the stages in the edit page
36	The components in the playbook edit page render normally			Verify the appearance of the components in the playbook edit page

Test Area - Set active workflow stage
Test ID	Test Case	Test Steps	Result	Notes
37	An active stage is annotated with `Active`label in the RHS
38	Non-active stages are not annotated with `Active` labels
39	Incident commander can set a different stage as `Active` in an ongoing incident
40	Workflow member can set a different stage as `Active` in an ongoing incident
41	Workflow member cannot set a different stage as `Active` in an incident that has been ended
42	Workflow member can set a different stage as `Active` in an incident that has been restarted
43	Plugin posts a message in the incident channel when changing an active stage
44	Workflow member views the active stage by default in incident RHS
45	When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the `Active` annotation
46	In case of only one stage, the stage is set as active by default
47	In case of multiple stages, the first stage is set as active by default
48	Switching between the stages does not set the stage as active
49	Selecting a non-active stage from the RHS dropdown shows a `Make Active` option prompt
50	Selecting an active stage from the RHS dropdown does not show a `Make Active` option prompt
51	Workflow member can check items off of stages that are active
52	Workflow member can uncheck items of stages that are not active
53	Workflow member can check off items of an ongoing incident
54	Workflow member cannot checkoff items of an ended incident
55	Workflow member can checkoff items of a restarted incident

Test Area - View steps in stages during a workflow
Test ID	Test Case	Test Steps	Result	Notes
56	The `Stages` dropdown in the RHS has all the stages for that incident
57	Selecting a different the stage in the RHS shows the checklist corresponding to that stage
58	Saving a playbook with only one stage and leaving the stage name empty, will create an incident with stage “Default Stage”
59	Any workflow member can switch between the stages in the RHS in an ongoing incident
60	Any workflow member can switch between the stages in the RHS in an incident that has been ended
61	Any workflow member can switch between the stages in the RHS in an incident that has been restarted
62	Selecting a different stage will not change the default stage view upon page refresh			Refreshing the page or navigating back to the incident channel will always show the `Active` stage in the RHS and not the stage selected earlier
63	Selecting a different stage will not change the stage view for a different workflow member

Test Area - Assign step owners
Test ID	Test Case	Test Steps	Result	Notes
64	Checklist item in the RHS has a dropdown for assignee
65	The assignee dropdown shows the list of workflow members
66	The assignee dropdown does not show users that are not workflow members
67	By default, checklist item does not have any assignee
68	A checklist item can be checked off without an assignee
69	A checklist item can be unchecked without an assignee
70	Workflow member can select him/herself as the checklist assignee
71	Workflow member can select a different workflow member as the checklist assignee
72	Workflow member can change assignee from him/herself to ‘No Assignee’ for a checklist item
73	Workflow member can change another workflow member to ‘No Assignee’ for a checklist item
74	Plugin posts a message in incident channel when an assignee is selected
75	Plugin posts a message in incident channel when ‘No Assignee’ is selected
76	Workflow member cannot select multiple assignees
77	Workflow member cannot select a different assignee in an incident that is ended
78	Workflow member can select a different assignee in an incident that is restarted
79	Assignee does not change when a workflow member leaves the workflow channel
80	Assignee does not change when a workflow member is kicked from the workflow channel
81	The assignee dropdown list gets updated when a new member is added to the workflow channel
82	The assignee dropdown list gets updated when a workflow member leaves the channel
83	The assignee dropdown list gets updated when a workflow member is kicked from the channel
84	An assignee can checkoff a checklist item
85	An assignee can uncheck a checklist item
86	A non-assignee can checkoff a checklist item
87	A non-assignee can uncheck a checklist item
88	Anything in the backstage about assignee?

Test Area - Attach a slash command to each step when editing playbook
Test ID	Test Case	Test Steps	Result	Notes
89	A playbook can be edited to add a slash command to any checklist item
90	A newly added checklist item in a playbook has a slash command field
91	A checklist item configured with a slash command shows the slash command for that item in the RHS
92	A checklist item can only be configured with one slash command at most
93	In case of two valid slash commands, the running the command from the checklist item will only run the first command successfully			This is in sync with the core feature behavior
94	Typing an invalid slash command
95	Running an invalid slash command from RHS checklist shows an ‘invalid slash command’ error			Like core feature?
96	A slash command can be run multiple times from the RHS checklist
97	A plugin post notifies when a slash command fails to run successfully
98	https://mattermost.atlassian.net/browse/MM-26733 : “any textbox” what is this?

Test Area - Browse existing slash commands in playbook editing
Test ID	Test Case	Test Steps	Result	Notes
99	Typing a `/` in the slash command field brings up a list of existing slash commands
100	Typing a non-/ character in the slash command field?			Should this be allowed?
101	Selecting from the slash command list auto completes the slash command
102	Typing a non-existing slash command shows a “invalid slash command” error
103	The slash command field can be left empty

Test Area - Execute the attached slash command with a button click
Test ID	Test Case	Test Steps	Result	Notes

Test Area - Tests/bugs outside of requirements and user stories
Test ID	Test Case	Test Steps	Result	Notes