/
Test Plan - Incident Response v0.4.0

Test Plan - Incident Response v0.4.0

Document version 1.1

Status: Tests Done.

Document Version

Description

Date

Document Version

Description

Date

0.0

Initial version

05/11

0.1

Test development done. In review.

05/13

0.2

  • Tests updated based on Maria’s review

  • Updated closed bugs for test

05/15

1.0

Test in progress

05/20

1.1

Tests done

05/20

References

Related Tests:

Summary

This document details the tests for Incident Response plugin v0.4.0. The tests are derived from the user stories in Incident Response overview. The MVP of the Incident Response plugin will be tested against all tests included in this test plan.

Scope

This document covers the cases for the user stories as outlined in the Incident Response MVP Overview for v.0.4.0 of the plugin only. v0.4.0 includes features that works in conjunction with the Channel Export plugin, the tests for which as a stand-alone plugin have been covered in the test plan for the plugin (see reference above). Thus, the tests in this document will validate the channel export features that are tied to the Incident Response plugin.

Glossary

  • Admin - A user with system administrator privileges.

  • Normal user - A user with non-system administrator privileges.

  • Test Server - A list of test server versions used in testing including Mattermost server and Marketplace Server

  • Main menu - Hamburger menu on the LHS.

  • Incident Response icon - The Incident Response plugin icon that appears on the channel header

  • Commander - The user who owns an incident. The user who starts the incident is designated the commander by default.

  • Incident member/incident channel member - A user who has been added to the incident channel.

  • Non-incident user - A user who has not been added to the incident channel.

  • Active incident - An incident that has not been ended.

  • Ongoing incident - Same as Active Incident.

  • Inactive incident - An incident that has been ended.

  • Incident RHS - The RHS sidebar that opens upon clicking the incident response icon and displays incidents.

  • Incident Backstage - The backstage view that shows the incident list.

  • Playbook Backstage - The backstage view that shows the playbook list.

Assumptions

The tests in this test plans are written with the assumption that:

  1. The tests outlined in this document is to be executed against the Incident Response plugin v0.4.0.

  2. This version of the plugin can be installed and used in Mattermost instances that aren’t equipped with any enterprise license.

  3. The Incident Response plugin can function as a standalone plugin that does not require to talk to any other plugin.

  4. The Incident Response plugin will function in Mattermost version 5.12 and later

  5. The tests will be executed in Mattermost version 5.23.

  6. Testing is primarily done on the webapp, with spot checks on the desktop app, RN mobile app and mobile web browser app.

  7. The tests in this document are executed as a normal user (see glossary above) unless otherwise specified.

  8. The test cases that pass for a normal user will also pass for a system administrator.

  9. Plugin uploads is enabled in server instance

    1. “EnableUploads” option under “PluginSettings” in mattermost-server’s config.json is set to 'true'.

  10. The channel export feature defined in the user story is dependent on the existence of the Channel Export plugin being installed and enabled in the test server.

Setup

Document Setup

For the practicality of running tests easily, test cases are arranged in sequential order when necessary, following the preceding test case.

Test Setup

The following setup will be necessary in order to begin testing:

Test Date: 05/20/2020

Tests

Test Area - Plugin Setup

Test ID

Test Case

Test Procedure

Result

Notes

1

Admin can install plugin in a non-EE instance

  1. Log in to a non-enterprise Mattermost, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

 

2

Admin can install plugin in an E10 instance

  1. Log in to Mattermost that has an E10 license, as admin.

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin is successfully installed.

Pass

 

3

Admin can install plugin in an E20 instance

  1. Log in to Mattermost that has an E20 license, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

 

4

Admin can enable plugin

  1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.2.0 and click on the “Enable” button.

  2. Verify that the plugin starts normally without errors.

Pass

 

5

Plugin functionality is available to admin

  1. Switch to a normal channel.

  2. Type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

6

Plugin functionality is available to normal user

  1. Log in as a non-admin user.

  2. In a normal channel, type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

7

Admin can disable plugin

  1. Log out and log back in as admin.

  2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  3. Click on the “Disable” button.

  4. Verify that the plugin is disabled.

  5. In the main channel’s post input box, type /incident.

  6. Verify that /incident option is not present in the slash command list.

Pass

 

8

Admin can remove plugin

  1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  2. Click on the “Remove” button.

  3. Verify that the plugin is no longer installed in the instance.

Pass

 

9

Plugin upgrades normally from v0.3 to v0.4

  1. Install older version of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

Pass

MM-25403: The backstage page sometimes renders wider than normalClosed

10

Incidents started with older plugin version are still active and available in v0.4

  1. Start a few incident with plugin version 0.3.

  2. For at least one of the incidents, add a few checklist items and check some of them.

  3. Upgrade plugin to v0.4.

  4. Open up the incident RHS to see the incident list.

  5. Verify that incidents started with version 0.3 are still available on the RHS.

  6. In the incident details on the RHS, verify that the state of the checklist and the checklist items are the same as step 2 before upgrading the plugin.

Pass

 

11

Incidents started with older plugin version are displayed correctly in the backstage incident list view

Continue from the above test.

  1. Launch the incident backstage.

  2. Verify that incident info in the list view is correct in all columns.

Pass

 

12

Incidents started with older plugin version info is correct in the backstage incident summary page

Continue from the above test.

  1. Click on one of the incidents created in the above tests to go to the incident summary page.

  2. Verify that the info is correct in all sections.

  3. Click the channel export link.

  4. Verify that the correct channel is exported in the csv file.

  5. Click on the incident channel link.

  6. Verify that the correct incident channel loads.

  7. Click the link to the incident channel.

Pass

 

13

Incidents ended with plugin v0.3 are not available in v0.4

  1. Start a few incident with plugin version 0.3.

  2. Leave some as active and end a few of them.

  3. Upgrade plugin to v0.4.

  4. Open up the incident RHS to see the incident list.

  5. Verify that incidents started with version 0.3 are still available in v0.4.

  6. Verify that incidents ended in step 2 are not visible in the list.

Pass

 

14

Playbook started with the older plugin version is still available upon plugin upgrade

  1. Create a few playbooks with plugin version 0.3.

  2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

Pass

 

  • Login as an admin user

  • Enable the Incident Response plugin

    • Navigate to System Console > PLUGINS > Plugin Management.

    • In “Installed Plugins” find Incident Response and click “Enabled”.

  • Log out.

Test Area - Monitor

As a user of a team, I can see a list of all past and current incidents within the team so that the information is not lost.

Test Area - Monitor

As a user of a team, I can see a list of all past and current incidents within the team so that the information is not lost.

Test ID

Test Case

Test Procedure

Result

Notes

15

Clicking playbook button on RHS opens Playbook backstage by default

  1. Login as a normal user.

  2. Go to Team-A.

  3. From the channel header, click the incident icon to open the incident RHS.

  4. Click the playbook icon in the RHS header.

  5. Verify Playbooks backstage is launched.

  6. Verify the backstage shows a list of playbooks if they exist.

  7. Verify the Playbooks tab is highlighted on the backstage LHS.

Pass

 

16

Clicking “Incidents & Playbooks Backstage” in the main menu opens Incidents backstage

  1. Login as a normal user.

  2. Go to Team-A.

  3. Verify Incidents backstage is launched.

  4. Verify the Incidents tab is highlighted on the backstage LHS.

  5. Verify the backstage shows a list of incidents if they exist.

Pass

 

17

Backstage shows a tab for incidents

Continue from the above test.

  1. Create incidents a-inc-1, a-inc-2, a-inc-3.

  2. Note the time of incident creation for each incidents for later tests.

  3. Verify that there is an “Incidents” tab in the backstage LHS in addition to the Playbooks tab.

Pass

 

18

Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab

Continue from the above test.

  1. Click on the “Incidents” tab on the LHS.

  2. Verify that the highlight shifts from Playbooks to Incidents.

Pass

 

19

Clicking on “Incidents” tab on backstage LHS loads the list of all incidents

Continue from the above test.

  1. Verify that clicking on the “Incidents” tab on the LHS loads the incidents list view.

  2. Verify that all incidents from Team-A including a-inc-1, a-inc-2, a-inc-3 are in the list.

Pass

 

20

Incidents list view displays the team name the incidents belong to

Continue from the above test.

  1. Verify that the team’s name - “Team-A” is displayed in the backstage header.

Pass

 

21

Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

Continue from the above test.

  1. Verify that the incidents backstage shows incident details including name, status, start date, end date and commander for each incident.

Pass

 

22

The incident list in the backstage shows all active incidents that belong to the current team only

  1. Switch to Team-B.

  2. Create new incidents: Inc-a, Inc-b and Inc-c

  3. Launch the incident backstage from within Team-B.

  4. Verify that the incident list only shows the three incidents created in step 2.

  5. Switch to Team-A.

  6. Launch the incident backstage from within Team-A.

  7. Verify that the incident list only shows the incidents created within Team-A.

  8. Verify that the incident list does not show the incidents created in step 2.

  9. Verify that the incident list shows all currently active incidents in the list.

Pass

MM-24555: List past and current incidents within the teamClosed

23

The incident list in the backstage shows all inactive incidents that belong to the current team only

  1. Switch to Team-B.

  2. End incident Inc-b and Inc-c.

  3. Note the incident end time for later tests.

  4. Launch the incident backstage from within Team-B.

  5. Verify that the incident list shows all three incidents including both active and inactive incidents.

  6. Verify none of the incidents from Team-A are in the list.

  7. Switch to Team-A.

  8. End incident a-inc-2.

  9. Note the incident end time for later tests.

  10. Launch the incident backstage.

  11. Verify that the incident list shows all incidents created and ended with Team-A.

Pass

MM-24555: List past and current incidents within the teamClosed

24

Active incidents show an “Ongoing” status in the Status column in the backstage

Continue from the above test.

  1. Launch the incident backstage from Team-A.

  2. Verify that incidents a-inc-1, and a-inc-3 show the status as “Ongoing” in the “Status” column.

  3. Launch the incident backstage from Team-B.

  4. Verify that incident Inc-a shows “Ongoing” status.

Pass

 

25

Inactive incidents (ended incidents) show an “Ended” status in the Statuscolumn in the backstage.

Continue from the above test.

  1. Launch the incident backstage from Team-A.

  2. Verify that incident a-inc-2 shows the status as “Ended” in the “Status” column.

  3. Launch the incident backstage from Team-B.

  4. Verify that incidents Inc-b and Inc-c show “Ended” in the “Status” column.

Pass

 

26

The incident list in the backstage shows updated end timestamp for an incident

Continue from the above test.

  1. Launch the incident backstage from Team-A.

  2. Verify that the “End Date” for a-inc-2 is accurate.

Pass

 

27

An active incident shows “--” in End Date column

Continue from the above test.

  1. Verify that a-inc-1 and a-inc-3 show “Ongoing” under “End Date”.

Pass

 

28

The incident list in the backstage shows updated commander for an incident

  1. Go to the normal channel view.

  2. Invite user-B to the incident channel of a-inc-3.

  3. Launch incident RHS.

  4. Click on a-inc-3 to see the details.

  5. Change the commander to user-B.

  6. Launch the incident backstage.

  7. In the incident list, find a-inc-3.

  8. Verify that user-B is listed in the “Commander” column.

Pass

 

29

The incident names in the “Name” column render as clickable links

Continue from the above test.

  1. Verify that all the incident names in the “Name” column are rendered as clickable links.

Fail

MM-25391: Incident names in the backstage list view render as normal text instead of linksClosed

30

Long incident name wraps in the “Name” column displaying the entire incident name

  1. Start a new incident such that the number of characters exceed the column width of “Name”.

  2. Launch the incident backstage.

  3. In the incident list view, verify that the name of the incident wraps in the next line so that the entire incident name is visible.

Fail

MM-25392: Long incident names will overflow into other column in backstageClosed

MM-25393: Long incident names wrap into the next line with the same spacing as the next incident below Closed

As the incident plugin, the channel is automatically archived when the incident ends so that it reduces clutter.

31

Channel is archived when ending an incident with slash command

  1. Set ExperimentalViewArchivedChannels to true or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to true.

  2. Start a new incident using any method.

  3. Locate the channel in the LHS.

  4. End the incident using the /incident end command.

  5. Click “Confirm” in the confirmation dialog.

  6. Verify that the incident ended.

    1. Incident no longer appears in the RHS list.

    2. Status of the incident changed to “Ended” in the backstage.

  7. Verify that the incident channel is archived:

    1. In the LHS, verify the incident channel is listed with the archived icon.

    2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

N/A

Moved to 0.5

32

Channel is archived when ending an incident from RHS

  1. Start a new incident using any method.

  2. Locate the channel in the LHS.

  3. End the incident from the RHS.

  4. Click “Confirm” in the confirmation dialog.

  5. Verify that the incident ended.

    1. Incident no longer appears in the RHS list.

    2. Status of the incident changed to “Ended” in the backstage.

  6. Verify that the incident channel is archived:

    1. In the LHS, verify the incident channel is listed with the archived icon.

    2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

N/A

Moved to 0.5

33

The incident channel shows up under archived channel once the incident is ended

  1. On the LHS click “more” under Private Channels.

  2. Ensure “Show: Archived Channels” is selected in the dropdown.

  3. Verify the channels of the incidents ended in the above 2 tests are listed.

N/A

Moved to 0.5

34

Incident channels do not appear in LHS when archived channel display is turned off

  1. Set ExperimentalViewArchivedChannels to false or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to false.

  2. Switch to normal channel view.

  3. Verify that the channels of the incidents ended in the tests above do not appear on the LHS.

N/A

Moved to 0.5

Test Area - Coordinate

As a user, I can export channel transcript from the incident detail page so that it can be saved for record

Test Area - Coordinate

As a user, I can export channel transcript from the incident detail page so that it can be saved for record

Test ID

Test Case

Test Procedure

Result

Notes

35

When Channel Export plugin is not installed, the “Export Incident Channel” link is inactive

  1. Ensure that the Channel Export plugin is not installed.

  2. Log in as an admin.

  3. Launch the incident backstage.

  4. Click on a-inc-1 to view the incident summary.

  5. Verify that the “Export Incident Channel” link and icon appear in greyscale.

  6. Click on the link.

  7. Verify that the link is inactive and nothing happens upon clicking.

  8. Logout and log in as an incident member.

  9. Repeat steps 3-7 for an incident the user is a member of.

Pass

 

36

When Channel Export plugin is not installed, hovering over the “Export Incident Channel” link shows a message to “install the channel export plugin”

Continue from the above test.

  1. Login as an admin.

  2. Launch incident backstage and click on an incident.

  3. Hover over the “Export Incident Channel” link.

  4. Verify that the tooltip shows the message “Install and enable the channel export plugin to support exporting this incident.”

  5. Log out and login as a normal user.

  6. Find an incident the user is a member of.

  7. Repeat steps 3-4.

Pass

 

37

When Channel Export plugin is installed in an instance with no EE license, the “Export Incident Channel” link is inactive

  1. Ensure the server has no EE license.

  2. Log in as an admin.

  3. Install and enable the Channel Export plugin.

  4. Launch the incident backstage.

  5. Click on a-inc-1 to view the incident summary.

  6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

  7. Click on the link.

  8. Verify that the link is inactive and nothing happens upon clicking.

  9. Logout and log in as an incident member.

  10. Repeat steps 4-8 for an incident the user is a member of.