Document version 1.0
Status: Test In Progress
Document Version | Description | Date |
---|---|---|
0.0 | Initial version | 05/11 |
0.1 | Test development done. In review. | 05/13 |
0.2 |
| 05/15 |
1.0 | Test in progress | 05/20 |
...
The following setup will be necessary in order to begin testing:
A Mattermost v5.23 test server: https://ir-test-523.test.mattermost.cloud/
The test server is equipped with an E10 E20 license for the plugin to be installed.
Incident Plugin v0.4.0.:
Channel Export Plugin v0.2
Test Server
...
Test Server (upgrade tests):
Build Hash: ed1576d007a09b07574b18784b9eabb3d8c9db53
...
Test Area - Plugin Setup | ||||
Test ID | Test Case | Test Procedure | Result | Notes |
---|---|---|---|---|
1 | Admin can install plugin in a non-EE instance |
| ||
2 | Admin can install plugin in an E10 instance |
| ||
3 | Admin can install plugin in an E20 instance |
| Pass |
|
4 | Admin can enable plugin |
| Pass |
|
5 | Plugin functionality is available to admin |
| Pass |
|
6 | Plugin functionality is available to normal user |
|
| |
7 | Admin can disable plugin |
| Pass |
|
8 | Admin can remove plugin |
| Pass |
|
9 | Plugin upgrades normally from v0.3 to v0.4 |
| ||
10 | Incidents started with older plugin version are still active and available in v0.4 |
|
| |
+ | Incidents started with older plugin version are displayed correctly in the backstage incident list view | Continue from the above test.
| ||
+ | Incidents started with older plugin version info is correct in the backstage incident summary page | Continue from the above test.
| ||
11 | Incidents ended with plugin v0.3 are not available in v0.4 |
|
| |
12 | Playbook started with the older plugin version is still available upon plugin upgrade |
|
Login as an admin user
Enable the Incident Response plugin
Navigate to System Console > PLUGINS > Plugin Management.
In “Installed Plugins” find Incident Response and click “Enabled”.
Log out.
Test Area - Monitor As a user of a team, I can see a list of all past and current incidents within the team so that the information is not lost. | ||||
---|---|---|---|---|
Test ID | Test Case | Test Procedure | Result | Notes |
13 | Clicking playbook button on RHS opens Playbook backstage by default |
| Pass | |
Clicking “Incidents & Playbooks Backstage” in the main menu opens Incidents backstage |
| Pass | ||
14 | Backstage shows a tab for incidents | Continue from the above test.
| Pass | |
15 | Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab | Continue from the above test.
| Pass | |
16 | Clicking on “Incidents” tab on backstage LHS loads the list of all incidents | Continue from the above test.
| Pass | |
17 | Incidents list view displays the team name the incidents belong to | Continue from the above test.
| Pass | |
18 | Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander | Continue from the above test.
| Pass | |
19 | The incident list in the backstage shows all active incidents that belong to the current team only |
| Pass | |
20 | The incident list in the backstage shows all inactive incidents that belong to the current team only |
| Pass | |
21 | Active incidents show an “Ongoing” status in the | Continue from the above test.
| Pass | |
22 | Inactive incidents (ended incidents) show an “Ended” status in the | Continue from the above test.
| Pass | |
23 | The incident list in the backstage shows updated end timestamp for an incident | Continue from the above test.
| Pass | |
24 | An active incident shows “--” in End Date column | Continue from the above test.
| Pass | |
25 | The incident list in the backstage shows updated commander for an incident |
| Pass | |
26 | The incident names in the “Name” column render as clickable links | Continue from the above test.
| Fail | |
27 | Long incident name wraps in the “Name” column displaying the entire incident name |
| Fail | |
As the incident plugin, the channel is automatically archived when the incident ends so that it reduces clutter. | ||||
28 | Channel is archived when ending an incident with slash command |
| N/A | Moved to 0.5 |
29 | Channel is archived when ending an incident from RHS |
| N/A | Moved to 0.5 |
30 | The incident channel shows up under archived channel once the incident is ended |
| N/A | Moved to 0.5 |
31 | Incident channels do not appear in LHS when archived channel display is turned off |
| Test Area N/A | Moved to 0.5 |
Test Area - Coordinate As a user, I can export channel transcript from the incident detail page so that it can be saved for record | ||||||||
---|---|---|---|---|---|---|---|---|
Test ID | Test Case | Test Procedure | Result | Notes | ||||
32 | When Channel Export plugin is not installed, the “Export Incident Channel” link is inactive |
| Pass | |||||
33 | When Channel Export plugin is not installed, hovering over the “Export Incident Channel” link shows a message to “install the channel export plugin” | Continue from the above test.
| Pass | |||||
34 | When Channel Export plugin is installed in an instance with no EE license, the “Export Incident Channel” link is inactive |
| ||||||
35 | When Channel Export plugin is installed in an instance with no EE license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required” | Continue from the above test.
| ||||||
36 | When Channel Export plugin is installed in an instance with an E10 license, the “Export Incident” link is inactive |
| ||||||
37 | When Channel Export plugin is installed in an instance with an E10 license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required” | Continue from the above test.
| ||||||
38 | When Channel export plugin is installed but not enabled, the “Export Incident Channel” link is inactive |
| Pass | |||||
39 | When Channel Export plugin is installed but not enabled, hovering over the “Export Incident Channel” shows “enable the plugin” message | Continue from the above test.
| Fail | |||||
40 | When Channel Export plugin is installed and enabled, an active “Export Incident Channel” link and icon is available in the incident details view as an active link |
| Pass | |||||
41 | When Channel Export plugin is installed and enabled, hovering over the “Export Incident Channel” link shows a message that the action will download the channel content | Continue from the above test.
| N/A | I think I added this on my own for consistency, but is this valid? | ||||
42 | A system admin can export the incident channel the admin is a member of by clicking “Export Incident Channel” link |
| The features of the Channel Export plugin were tested in Test Plan - Channel Export v0.0.1 | |||||
43 | A system admin can export the incident channel the admin is NOT a member of by clicking “Export Incident Channel” link |
| ||||||
44 | An incident member can export the incident channel by clicking “Export Incident Channel” link |
| ||||||
45 | The exported csv file captures the current status of the incident |
| This feature was tested in Test Plan - Channel Export v0.0.1 | |||||
46 | “Export Incident Channel” button is inactive for non-incident user |
| ||||||
47 | A non-incident user sees a “no access” message upon hovering on the “Export Incident Channel” button | Continue from the above test.
| Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage. Can someone confirm? | |||||
48 | Exporting an ended incident channel from backstage exports the archived channel |
| ||||||
49 | Non-member cannot export an ended incident channel |
| Same as #47. Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage. Can someone please confirm? | |||||
50 | Verify the appearance of Once the Channel Export plugin is disabled, the “Export Incident Channel” link and icon appear in dark theme |
| 51 | Verify the appearance of the “Export Incident Channel” link and icon appear in should deactivate again |
| Fail | ||
50 | Verify the appearance of the “Export Incident Channel” link and icon appear in dark theme |
| Fail | |||||
51 | Verify the appearance of the “Export Incident Channel” link and icon appear in light theme |
| Pass | |||||
52 | Verify the appearance of the “Export Incident Channel” link and icon appear in other themes |
| Fail | |||||
As the incident commander, I receive an incident summary and a link to more info after ending an incident so that I know where to go for post-mortem. | ||||||||
Test Area - Measure As a user, I can see a summary of the following on incident detail page so that I can get the gist at a glance:
| ||||
---|---|---|---|---|
Test ID | Test Case | Test Procedure | Result | Notes |
53 | User cannot view incident summary if the user is not an incident member |
| Is this test case correct since we’re limiting a non-user’s access to the incident summary. | |
54 | From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an active incident if the user is an incident member |
| ||
55 | From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an inactive incident |
| ||
56 | Incident summary page displays the incident name as the page heading | Continue from the above test.
| ||
57 | Incident summary page has a “<“ button that brings a user back to the incidents list view | Continue from the above test.
| ||
58 | Incident summary page bears a link to the incident channel | Continue from the above test.
| ||
59 | Hovering on the incident channel link shows a tooltip suggesting that it’s a shortcut to the incident channel | Continue from the above test.
| ||
60 | Clicking on incident channel link of active incidents brings the user to the incident channel | Continue from the above test.
| ||
61 | Clicking on incident channel link of inactive incidents, when |
| ||
62 | Clicking on incident channel link of inactive incidents, when | ??? Should the link be inactive and in greyscale if false? | ||
63 | Incident summary shows the current status of the incident |
| ||
64 | Incident summary shows the current commander of the incident |
| ||
65 | For active incidents, incident summary shows the duration of how long the incident has been active for |
| ||
66 | For inactive incidents, incident summary shows the duration of how long the incident remained active for before the incident ended |
| ||
67 | For active incidents, incident summary shows the current number of members in the incident channel | Continue from the above test.
| ||
68 | For an inactive incident, incident summary shows the number of member in the incident channel when the incident was ended | Continue from the above test.
| ||
69 | For active incident, incident summary shows the total number of messages in the incident channel |
| ||
70 | For inactive incident, incident summary shows the total number of messages in the archived channel of the inactive incident | Continue from the above test.
|
...