Versions Compared

Version Old Version 13 New Version Current
Changes made by

Prapti Shrestha (Deactivated)

Prapti Shrestha (Deactivated)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

...

...

...

...

...

...

...

...

...

...

Document version

...

1.1

Status:

...

Tests Done.

Document Version

Description

Date

0.0

Initial version

05/11

0.1

Test development done. In review.

05/13

0.2

  • Tests updated based on Maria’s review

  • Updated closed bugs for test

05/15

1.0

Test in progress

05/20

1.1

Tests done

05/20

References

...

...

...

Related Tests:

...

...

...

Summary

This document details the tests for Incident Response plugin v0.4.0. The tests are derived from the user stories in Incident Response overview. The MVP of the Incident Response plugin will be tested against all tests included in this test plan.

...

The following setup will be necessary in order to begin testing:

Test Server:

Test Server (upgrade tests):

...

Test Date: 05/20/2020

Tests

Test Area - Plugin Setup

Test ID

Test Case

Test Procedure

Result

Notes

1

Admin can install plugin in a non-EE instance

  1. Log in to a non-enterprise Mattermost, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

2

Admin can install plugin in an E10 instance

  1. Log in to Mattermost that has an E10 license, as admin.

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin is successfully installed.

Pass

3

Admin can install plugin in an E20 instance

  1. Log in to Mattermost that has an E20 license, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

 

4

Admin can enable plugin

  1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.2.0 and click on the “Enable” button.

  2. Verify that the plugin starts normally without errors.

Pass

 

5

Plugin functionality is available to admin

  1. Switch to a normal channel.

  2. Type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

6

Plugin functionality is available to normal user

  1. Log in as a non-admin user.

  2. In a normal channel, type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

 

7

Admin can disable plugin

  1. Log out and log back in as admin.

  2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  3. Click on the “Disable” button.

  4. Verify that the plugin is disabled.

  5. In the main channel’s post input box, type /incident.

  6. Verify that /incident option is not present in the slash command list.

Pass

 

8

Admin can remove plugin

  1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  2. Click on the “Remove” button.

  3. Verify that the plugin is no longer installed in the instance.

Pass

 

9

Plugin upgrades normally from v0.3 to v0.4

  1. Install older version of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

Pass

https://mattermost.atlassian.net/browse/MM-25403

10

Incidents started with

the

older plugin version

is

are still active and available in v0.

3

11

Incidents ended with plugin v0.2 are not available in v0.3

  • Start a few incident with plugin version 0.3.

    • Leave some as active and end a few

    4

    1. Start a few incident with plugin version 0.3.

  • Upgrade plugin to v0.4.

  • Open up the incident RHS to see the incident list.

  • Verify that incidents started with version 0.3 are still available in v0.4.

    •  

    1. For at least one of the incidents, add a few checklist items and check some of them.

    2. Upgrade plugin to v0.4.

    3. Open up the incident RHS to see the incident list.

    4. Verify that incidents started with version 0.3 are still available

    in v0.4.

  • Verify that incidents ended in step 2 are not visible in the list.

    •  

    12

    Playbook started with the older plugin version is still available upon plugin upgrade

    1. Create a few playbooks with plugin version 0.3.

    2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

    • Login as an admin user

    • Enable the Incident Response plugin

      • Navigate to System Console > PLUGINS > Plugin Management.

      • In “Installed Plugins” find Incident Response and click “Enabled”.

    • Log out.

    Test Area - Monitor

    As a user of a team, I can see a list of all past and current incidents within the team so that the information is not lost.

    (Epic: https://mattermost.atlassian.net/browse/MM-24554?jql=project%20%3D%20MM%20AND%20fixVersion%20%3D%20%22IR%20(v0.4.0)%22 )

    Test ID

    Test Case

    Test Procedure

    Result

    Notes

    13

    Clicking playbook button on RHS opens Playbook backstage by default

    1. Login as a normal user.

    2. Go to Team-A.

    3. From the channel header, click the incident icon to open the incident RHS.

    4. Click the playbook icon in the RHS header.

    5. Verify Playbooks backstage is launched.

    6. Verify the backstage shows a list of playbooks if they exist.

    7. Verify the Playbooks tab is highlighted on the backstage LHS.

    14

    Backstage shows a tab for incidents

    Continue from the above test.

    1. Create incidents a-inc-1, a-inc-2, a-inc-3.

    2. Note the time of incident creation for each incidents for later tests.

    3. Verify that there is an “Incidents” tab in the backstage LHS in addition to the Playbooks tab.

    15

    Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab

    Continue from the above test.

    1. Click on the “Incidents” tab on the LHS.

    2. Verify that the highlight shifts from Playbooks to Incidents.

    16

    Clicking on “Incidents” tab on backstage LHS loads the list of all incidents

    Continue from the above test.

    1. Verify that clicking on the “Incidents” tab on the LHS loads the incidents list view.

    2. Verify that all incidents from Team-A including a-inc-1, a-inc-2, a-inc-3 are in the list.

    17

    Incidents list view displays the team name the incidents belong to

    Continue from the above test.

    1. Verify that the team’s name - “Team-A” is displayed in the backstage header.

    18

    Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

    Continue from the above test.

    1. Verify that the incidents backstage shows incident details including name, status, start date, end date and commander for each incident.

    19

    The incident list in the backstage shows all active incidents that belong to the current team only

    1. Switch to Team-B.

    2. Create new incidents: Inc-a, Inc-b and Inc-c

    3. Launch the incident backstage from within Team-B.

    4. Verify that the incident list only shows the three incidents created in step 2.

    5. Switch to Team-A.

    6. Launch the incident backstage from within Team-A.

    7. Verify that the incident list only shows the incidents created within Team-A.

    8. Verify that the incident list does not show the incidents created in step 2.

    9. Verify that the incident list shows all currently active incidents in the list.

    https://mattermost.atlassian.net/browse/MM-24555

    20

    The incident list in the backstage shows all inactive incidents that belong to the current team only

    1. Switch to Team-B.

    2. End incident Inc-b and Inc-c.

    3. Note the incident end time for later tests.

    4. Launch the incident backstage from within Team-B.

    5. Verify that the incident list shows all three incidents including both active and inactive incidents.

    6. Verify none of the incidents from Team-A are in the list.

    7. Switch to Team-A.

    8. End incident a-inc-2.

    9. Note the incident end time for later tests.

    10. Launch the incident backstage.

    11. Verify that the incident list shows all incidents created and ended with Team-A.

    1. on the RHS.

    2. In the incident details on the RHS, verify that the state of the checklist and the checklist items are the same as step 2 before upgrading the plugin.

    Pass

     

    11

    Incidents started with older plugin version are displayed correctly in the backstage incident list view

    Continue from the above test.

    1. Launch the incident backstage.

    2. Verify that incident info in the list view is correct in all columns.

    Pass

    12

    Incidents started with older plugin version info is correct in the backstage incident summary page

    Continue from the above test.

    1. Click on one of the incidents created in the above tests to go to the incident summary page.

    2. Verify that the info is correct in all sections.

    3. Click the channel export link.

    4. Verify that the correct channel is exported in the csv file.

    5. Click on the incident channel link.

    6. Verify that the correct incident channel loads.

    7. Click the link to the incident channel.

    Pass

    13

    Incidents ended with plugin v0.3 are not available in v0.4

    1. Start a few incident with plugin version 0.3.

    2. Leave some as active and end a few of them.

    3. Upgrade plugin to v0.4.

    4. Open up the incident RHS to see the incident list.

    5. Verify that incidents started with version 0.3 are still available in v0.4.

    6. Verify that incidents ended in step 2 are not visible in the list.

    Pass

     

    14

    Playbook started with the older plugin version is still available upon plugin upgrade

    1. Create a few playbooks with plugin version 0.3.

    2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

    Pass

    • Login as an admin user

    • Enable the Incident Response plugin

      • Navigate to System Console > PLUGINS > Plugin Management.

      • In “Installed Plugins” find Incident Response and click “Enabled”.

    • Log out.

    in the “Status” column

    Test Area - Monitor

    As a user of a team, I can see a list of all past and current incidents within the team so that the information is not lost.

    Test ID

    Test Case

    Test Procedure

    Result

    Notes

    15

    Clicking playbook button on RHS opens Playbook backstage by default

    1. Login as a normal user.

    2. Go to Team-A.

    3. From the channel header, click the incident icon to open the incident RHS.

    4. Click the playbook icon in the RHS header.

    5. Verify Playbooks backstage is launched.

    6. Verify the backstage shows a list of playbooks if they exist.

    7. Verify the Playbooks tab is highlighted on the backstage LHS.

    Pass

    16

    Clicking “Incidents & Playbooks Backstage” in the main menu opens Incidents backstage

    1. Login as a normal user.

    2. Go to Team-A.

    3. Verify Incidents backstage is launched.

    4. Verify the Incidents tab is highlighted on the backstage LHS.

    5. Verify the backstage shows a list of incidents if they exist.

    Pass

    17

    Backstage shows a tab for incidents

    Continue from the above test.

    1. Create incidents a-inc-1, a-inc-2, a-inc-3.

    2. Note the time of incident creation for each incidents for later tests.

    3. Verify that there is an “Incidents” tab in the backstage LHS in addition to the Playbooks tab.

    Pass

    18

    Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab

    Continue from the above test.

    1. Click on the “Incidents” tab on the LHS.

    2. Verify that the highlight shifts from Playbooks to Incidents.

    Pass

    19

    Clicking on “Incidents” tab on backstage LHS loads the list of all incidents

    Continue from the above test.

    1. Verify that clicking on the “Incidents” tab on the LHS loads the incidents list view.

    2. Verify that all incidents from Team-A including a-inc-1, a-inc-2, a-inc-3 are in the list.

    Pass

    20

    Incidents list view displays the team name the incidents belong to

    Continue from the above test.

    1. Verify that the team’s name - “Team-A” is displayed in the backstage header.

    Pass

    21

    Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

    Continue from the above test.

    1. Verify that the incidents backstage shows incident details including name, status, start date, end date and commander for each incident.

    Pass

    22

    The incident list in the backstage shows all active incidents that belong to the current team only

    1. Switch to Team-B.

    2. Create new incidents: Inc-a, Inc-b and Inc-c

    3. Launch the incident backstage from within Team-B.

    4. Verify that the incident list only shows the three incidents created in step 2.

    5. Switch to Team-A.

    6. Launch the incident backstage from within Team-A.

    7. Verify that the incident list only shows the incidents created within Team-A.

    8. Verify that the incident list does not show the incidents created in step 2.

    9. Verify that the incident list shows all currently active incidents in the list.

    Pass

    https://mattermost.atlassian.net/browse/MM-24555

    21

    Active incidents show an “Ongoing” status in the Status column in the backstage

    Continue from the above test.

  • Launch the incident backstage from Team-A.

    • Verify that incidents a-inc-1, and a-inc-3 show the status as “Ongoing”

    23

    The incident list in the backstage shows all inactive incidents that belong to the current team only

    1. Switch to Team-B.

    2. End incident Inc-b and Inc-c.

    3. Note the incident end time for later tests.

    4. Launch the incident backstage from within Team-B.

    5. Verify that the incident

    Inc-a shows “Ongoing” status.

    22

    Inactive incidents (ended incidents) show an “Ended”

    1. list shows all three incidents including both active and inactive incidents.

    2. Verify none of the incidents from Team-A are in the list.

    3. Switch to Team-A.

    4. End incident a-inc-2.

    5. Note the incident end time for later tests.

    6. Launch the incident backstage.

    7. Verify that the incident list shows all incidents created and ended with Team-A.

    Pass

    https://mattermost.atlassian.net/browse/MM-24555

    24

    Active incidents show an “Ongoing” status in the Status column in the backstage

    .

    Continue from the above test.

    1. Launch the incident backstage from Team-A.

    2. Verify that

    incident

    1. incidents a-inc-1, and a-inc-

    2 shows

    1. 3 show the status as “Ongoing” in the “Status” column.

    2. Launch the incident backstage from Team-B.

    3. Verify that incident Inc-a shows “Ongoing” status.

    Pass

    25

    Inactive incidents (ended incidents) show an “Ended” status in the Statuscolumn in the backstage.

    Continue from the above test.

    1. Launch the incident backstage from Team-A.

    2. Verify that incident a-inc-2 shows the status as “Ended” in the “Status” column.

    3. Launch the incident backstage from Team-B.

    4. Verify that incidents Inc-b and Inc-c show “Ended” in the “Status” column.

    Pass

    23

    26

    The incident list in the backstage shows updated end timestamp for an incident

    Continue from the above test.

    1. Launch the incident backstage from Team-A.

    2. Verify that the “End Date” for a-inc-2 is accurate.

    24

    Pass

    27

    An active incident shows

    “Ongoing”

    “--” in End Date column

    Continue from the above test.

    1. Verify that a-inc-1 and a-inc-3 show “Ongoing” under “End Date”.

    25

    Pass

    28

    The incident list in the backstage shows updated commander for an incident

    1. Go to the normal channel view.

    2. Invite user-B to the incident channel of a-inc-3.

    3. Launch incident RHS.

    4. Click on a-inc-3 to see the details.

    5. Change the commander to user-B.

    6. Launch the incident backstage.

    7. In the incident list, find a-inc-3.

    8. Verify that user-B is listed in the “Commander” column.

    Pass

    26

    29

    The incident names in the “Name” column render as clickable links

    Continue from the above test.

    1. Verify that all the incident names in the “Name” column

    as

    1. are rendered as clickable links.

    Fail

    https://mattermost.atlassian.

    27

    net/browse/MM-25391

    30

    Long incident name wraps in the “Name” column displaying the entire incident name

    1. Start a new incident such that the number of characters exceed the column width of “Name”.

    2. Launch the incident backstage.

    3. In the incident list view, verify that the name of the incident wraps in the next line so that the entire incident name is visible.

    Fail

    https://mattermost.atlassian.net/browse/MM-25392

    https://mattermost.atlassian.net/browse/MM-25393

    As the incident plugin, the channel is automatically archived when the incident ends so that it reduces clutter.

    28

    31

    Channel is archived when ending an incident with slash command

    1. Set ExperimentalViewArchivedChannels to true or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to true.

    2. Start a new incident using any method.

    3. Locate the channel in the LHS.

    4. End the incident using the /incident end command.

    5. Click “Confirm” in the confirmation dialog.

    6. Verify that the incident ended.

      1. Incident no longer appears in the RHS list.

      2. Status of the incident changed to “Ended” in the backstage.

    7. Verify that the incident channel is archived:

      1. In the LHS, verify the incident channel is listed with the archived icon.

      2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

    29

    N/A

    Moved to 0.5

    32

    Channel is archived when ending an incident from RHS

    1. Start a new incident using any method.

    2. Locate the channel in the LHS.

    3. End the incident from the RHS.

    4. Click “Confirm” in the confirmation dialog.

    5. Verify that the incident ended.

      1. Incident no longer appears in the RHS list.

      2. Status of the incident changed to “Ended” in the backstage.

    6. Verify that the incident channel is archived:

      1. In the LHS, verify the incident channel is listed with the archived icon.

      2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

    30

    N/A

    Moved to 0.5

    33

    The incident channel shows up under archived channel once the incident is ended

    1. On the LHS click “more” under Private Channels.

    2. Ensure “Show: Archived Channels” is selected in the dropdown.

    3. Verify the channels of the incidents ended in the above 2 tests are listed.

    31

    Incident channels

    N/A

    Moved to 0.5

    34

    Incident channels do not appear in LHS when archived channel display is turned off

    1. Set ExperimentalViewArchivedChannels to false or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to false.

    2. Switch to normal channel view.

    3. Verify that the channels of the incidents ended in the tests above do not appear on the LHS.

    N/A

    Moved to 0.5

    Test Area - Coordinate

    As a user, I can export channel transcript from the incident detail page so that it can be saved for record

    (Epic: https://mattermost.atlassian.net/browse/MM-24784?jql=project%20%3D%20MM%20AND%20fixVersion%20%3D%20%22IR%20(v0.4.0)%22 )

    Test ID

    Test Case

    Test Procedure

    Result

    Notes

    3233

    35

    When Channel Export plugin is not installed, the “Export Incident Channel” link is inactive

    1. Ensure that the Channel Export plugin is not installed.

    2. Log in as an admin.

    3. Launch the incident backstage.

    4. Click on a-inc-1 to view the incident summary.

    5. Verify that the “Export Incident Channel” link and icon appear in greyscale.

    6. Click on the link.

    7. Verify that the link is inactive and nothing happens upon clicking.

    8. Logout and log in as an incident member.

    9. Repeat steps 3-7 for an incident the user is a member of.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 5-7.

    • Pass

    36

    When Channel Export plugin is not installed, hovering over the “Export Incident Channel” link shows a message to “install the channel export plugin”

    Continue from the above test.

    1. Login as an admin.

    2. Launch incident backstage and click on an incident.

    3. Hover over the “Export Incident Channel” link.

    4. Verify that the tooltip shows the message “Install and enable the channel export plugin to support exporting this incident.”

    5. Log out and login as a normal user.

    6. Find an incident the user is a member of.

    7. Repeat steps 3-4.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 3-4.

    • 34

    35

    Pass

    37

    When Channel Export plugin is installed in an instance with no EE license, the “Export Incident Channel” link is inactive

    1. Ensure the server has no EE license.

    2. Log in as an admin.

    3. Install and enable the Channel Export plugin.

    4. Launch the incident backstage.

    5. Click on a-inc-1 to view the incident summary.

    6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

    7. Click on the link.

    8. Verify that the link is inactive and nothing happens upon clicking.

    9. Logout and log in as an incident member.

    10. Repeat steps 4-8 for an incident the user is a member of.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 4-8.

    • Pass

    38

    When Channel Export plugin is installed in an instance with no EE license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

    Continue from the above test.

    1. Login as an admin.

    2. Launch incident backstage and click on an incident.

    3. Hover over the “Export Incident Channel” link.

    4. Verify that the tooltip shows the message “Exporting an incident channel requires a Mattermost Enterprise E20 license.”

    5. Log out and login as a normal user.

    6. Find an incident the user is a member of.

    7. Repeat steps 3-4.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 3-4.

    • 36

    Pass

    39

    When Channel Export plugin is installed in an instance with an E10 license, the “Export Incident” link is inactive

    1. Ensure the server has an E10 license.

    2. Log in as an admin.

    3. Install and enable the Channel Export plugin.

    4. Launch the incident backstage.

    5. Click on a-inc-1 to view the incident summary.

    6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

    7. Click on the link.

    8. Verify that the link is inactive and nothing happens upon clicking.

    9. Logout and log in as an incident member.

    10. Repeat steps 4-8 for an incident the user is a member of.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 4-8.

    • 37

    When Channel

    Pass

    40

    When Channel Export plugin is installed in an instance with an E10 license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

    Continue from the above test.

    1. Login as an admin.

    2. Launch incident backstage and click on an incident.

    3. Hover over the “Export Incident Channel” link.

    4. Verify that the tooltip shows the message “Exporting an incident channel requires a Mattermost Enterprise E20 license.”

    5. Log out and login as a normal user.

    6. Find an incident the user is a member of.

    7. Repeat steps 3-4.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 3-4.

    • 38

    39

    Pass

    41

    When Channel export plugin is installed but not enabled, the “Export Incident Channel” link is inactive

    1. Ensure the server has an E20 license.

    2. Log in as an admin.

    3. Install the Channel Export plugin but do not enable it.

    4. Launch the incident backstage.

    5. Click on a-inc-1 to view the incident summary.

    6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

    7. Click on the link.

    8. Verify that the link is inactive and nothing happens upon clicking.

    9. Logout and log in as an incident member.

    10. Repeat steps 4-8 for an incident the user is a member of.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 4-8.

    • 40

    Pass

    42

    When Channel Export plugin is installed but not enabled, hovering over the “Export Incident Channel” shows “enable the plugin” message

    Continue from the above test.

    1. Login as an admin.

    2. Launch incident backstage and click on an incident.

    3. Hover over the “Export Incident Channel” link.

    4. Verify that the tooltip shows the message “Enable the channel export plugin to support exporting this incident.”

    5. Log out and login as a normal user.

    6. Find an incident the user is a member of.

    7. Repeat steps 3-4.

  • Go back to the incident list view.

  • Find an incident the user is not a member of.

  • Repeat steps 3-4.

    • Fail

    https://mattermost.atlassian.net/browse/MM-25395

    43

    When Channel Export plugin is installed and enabled, an active “Export Incident Channel” link and icon is available in the incident details view as an active link

    1. Ensure the server has an E20 license.

    2. Log in as an admin.

    3. Install the Channel Export and enable it.

    4. Launch the incident backstage.

    5. Click on a-inc-1 to view the incident summary.

    6. Verify that the “Export Incident Channel” link and icon do not appear in greyscale.

    7. Logout and log in as an incident member.

    8. Repeat steps 4-7 for an incident the user is a member of.

    Pass

    41

    44

    When Channel Export plugin is installed and enabled, hovering over the “Export Incident Channel” link shows a message that the action will download the channel content

    Continue from the above test.

    1. Login as an admin.

    2. Launch incident backstage and click on an incident.

    3. Hover over the “Export Incident Channel” link.

    4. Verify that the tooltip shows the message that the action will download the channel content.

    5. Log out and login as a normal user.

    6. Find an incident the user is a member of.

    7. Repeat steps 3-4.

    N/A

    I think I added this on my own for consistency, but is this valid?

    42

    45

    A system admin can export the incident channel the admin is a member of by clicking “Export Incident Channel” link

    1. Log in as an admin.

    2. Launch incident backstage.

    3. Click on an incident the admin is a member of.

    4. Verify that the link is not in greyscale.

    5. Click the “Export Incident Channel” link.

    6. Verify that the channel export action begins.

    7. Verify that a csv file is downloaded.

    8. Click on the channel export icon

      Image Modified

    9. Verify steps 4, 6 and 7.

    Pass

    The features of the Channel Export plugin were tested in Test Plan - Channel Export v0.0.1

    43

    46

    A system admin can export the incident channel the admin is NOT a member of by clicking “Export Incident Channel” link

    1. Log in as an admin.

    2. Launch incident backstage.

    3. Click on an incident the admin is not a member of.

    4. Verify that the link is not in greyscale.

    5. Click the “Export Incident Channel” link.

    6. Verify that the channel export action begins.

    7. Verify that a csv file is downloaded.

    8. Click on the channel export icon

      Image Modified

    9. Verify steps 4, 6 and 7.

    44

    Fail

    https://mattermost.atlassian.net/browse/MM-25398

    47

    An incident member can export the incident channel by clicking “Export Incident Channel” link

    1. Log in as a normal user.

    2. Launch incident backstage.

    3. Click on an incident the user is a member of.

    4. Verify that the link is not in greyscale.

    5. Click the “Export Incident Channel” link.

    6. Verify that the channel export action begins.

    7. Verify that a csv file is downloaded.

    8. Click on the channel export icon

      Image Modified

    9. Verify steps 4, 6 and 7.

    Pass

    45

    48

    The exported csv file captures the

    current status

    contents of the incident channel

    1. Verify the contents of the csv file.

    Pass

    This feature was tested in Test Plan - Channel Export v0.0.1

    46

    49

    “Export Incident Channel” button is inactive for non-incident user

    1. Log in as a normal user.

    2. Launch incident backstage.

    3. Click on an incident the user is not a member of.

    4. Verify that the link is in greyscale.

    5. Click the “Export Incident Channel” link.

    6. Verify that nothing happens.

    7. Click on the channel export icon

      Image Modified

    8. Verify nothing happens.

    47

    N/A

    50

    A non-incident user sees a “no access” message upon hovering on the “Export Incident Channel” button

    Continue from the above test.

    1. Hover over the “Export Incident Channel” link.

    2. Verify that the tooltip shows the message “You do not have access to this incident channel. Contact the commander to export this incident.“

    N/A

    Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage. Can someone confirm?

    48

    51

    Exporting an ended incident channel from backstage exports the archived channel

    1. From the incident backstage, find incident a-inc-e (or a different incident that has been ended).

    2. Click to view the summary.

    3. Click the “Export Incident Channel”.

    4. Verify that a csv file is downloaded with the contents of the archived channel.

    49

    Pass

    52

    Non-member cannot export an ended incident channel

    1. As user-B start incident “user-b-inc”.

    2. Do not add user-A to the incident channel.

    3. End the incident “user-b-inc”.

    4. Login as user-A.

    5. Launch the incident backstage.

    6. Find the incident “user-b-inc”.

    7. Click to view the summary.

    8. Verify that the “Export Incident Channel” link is inactive and appear in greyscale.

    9. Click the link.

    10. Verify that nothing happens.

    N/A

    Same as #47. Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage. Can someone please confirm?

    50

    53

    Verify the appearance of

    Once the Channel Export plugin is disabled, the “Export Incident Channel” link

    and icon appear in dark theme

    1. Switch to a dark theme.

    2. Verify that the “Export Incident Channel” is visible according to the theme.

    51

    Verify the appearance of the “Export Incident Channel” link and icon appear in light theme

    1. Switch to a light theme.

    2. Verify that the “Export Incident Channel” is visible according to the theme.

    52

    should deactivate again

    1. Install the Channel Export plugin but do not enable yet.

    2. Navigate to an incident's details view in the backstage.

    3. The "Export Incident Channel" link in backstage details view is inactive.

    4. Enable the Channel Export plugin.

    5. Navigate to the incident's details view in the backstage.

    6. The "Export Incident Channel" link in the details page is now active (not greyscale and functional).

    7. Disable the Channel Export plugin.

    8. Navigate to the incident's details view in the backstage.

    Fail

    https://mattermost.atlassian.net/browse/MM-25396

    54

    Verify the appearance of the “Export Incident Channel” link and icon appear in

    other themes

    dark theme

    1. Switch to a

    custom

    1. dark theme.

    2. Verify that the “Export Incident Channel” is visible according to the theme.

    As the incident commander, I receive an incident summary and a link to more info after ending an incident so that I know where to go for post-mortem.
    https://mattermost.atlassian.net

    Fail

    https://mattermost.atlassian.net/browse/MM-

    24409

    Test Area - Measure

    As a user, I can see a summary of the following on incident detail page so that I can get the gist at a glance:

    • Channel, start time, end time, duration, commander, number of people in the channel

    (Epic:

    25359

    55

    Verify the appearance of the “Export Incident Channel” link and icon appear in light theme

    1. Switch to a light theme.

    2. Verify that the “Export Incident Channel” is visible according to the theme.

    Pass

    56

    Verify the appearance of the “Export Incident Channel” link and icon appear in other themes

    1. Switch to a custom theme.

    2. Verify that the “Export Incident Channel” is visible according to the theme.

    Fail

    https://mattermost.atlassian.net/browse/MM-

    24554?jql=project%20%3D%20MM%20AND%20fixVersion%20%3D%20%22IR%20(v0.4.0)%22 )

    Test ID

    Test Case

    Test Procedure

    Result

    Notes

    53

    User cannot view incident summary if the user is not an incident member

  • Create an incident as user-B.

  • Do not add user-A to the incident channel.

  • Log in as user-A.

  • Launch the incident backstage.

  • Find the incident from step 1.

  • Click on the incident.

  • Verify that user-A is not taken to the incident’s summary page.

    • (Any message to contact the commander to view the summary?)

    25359

    As the incident commander, I receive an incident summary and a link to more info after ending an incident so that I know where to go for post-mortem.
    https://mattermost.atlassian.net/browse/MM-24409
    https://mattermost.atlassian.net/browse/MM-24411

    Test Area - Measure

    As a user, I can see a summary of the following on incident detail page so that I can get the gist at a glance:

    • Channel, start time, end time, duration, commander, number of people in the channel

    Test ID

    Test Case

    Test Procedure

    Result

    Notes

    57

    User cannot view incident summary if the user is not an incident member

    1. Create an incident as user-B.

    2. Do not add user-A to the incident channel.

    3. Log in as user-A.

    4. Launch the incident backstage.

    5. Find the incident from step 1.

    6. Click on the incident.

    7. Verify that user-A is taken to a blank incident page where no incident info is visible.

    8. Verify that there is a message to contact the commander to request access

    Pass

    Is this test case correct since we’re limiting a non-user’s access to the incident summary.

    54

    58

    From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an active incident if the user is an incident member

    1. Log in as user-A.

    2. Ensure that “a-inc-1” is still an active incident

    3. Launch the incident backstage.

    4. Find the incident “a-inc-1”.

    5. Click on the incident name.

    6. Verify that the incident summary page loads.

    Pass

    55

    59

    From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an inactive incident

    1. Log in as user-A.

    2. Launch the incident backstage.

    3. Find the incident “a-inc-2” (the ended incident).

    4. Click on the incident name.

    5. Verify that the incident summary page loads.

    Pass

    56

    60

    Incident summary page displays the incident name as the page heading

    Continue from the above test.

    1. In the incident summary page, verify that the heading of the page is the name of the incident.

    57

    Pass

    61

    Incident summary page has a “<“ button that brings a user back to the incidents list view

    Continue from the above test.

    1. In the incident summary page, verify there is a “<“ button beside the incident name header.

    2. Click on the “<“ button.

    3. Verify that the user is brought back to the incident list view in the backstage.

    Pass

    58

    62

    Incident summary page bears a link to the incident channel

    Continue from the above test.

    1. Click on the incident “a-inc-3” (still an active incident) to view the summary page.

    2. Verify that a link to the incident channel is present beside the incident name header.

      Image Modified

    Pass

    59

    63

    Hovering on the incident channel link shows a tooltip suggesting that it’s a shortcut to the incident channel

    Continue from the above test.

    1. Hover over the incident channel link icon.

    2. Verify that a tooltip shows “Go to incident channel”.

    60

    Pass

    64

    Clicking on incident channel link of active incidents brings the user to the incident channel

    Continue from the above test.

    1. Click on the incident channel link.

    2. Verify the user lands on the correct incident channel.

    Pass

    61

    65

    Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:truebrings the user to the incident channel

    1. As an admin, ensure that the ExperimentalViewArchiveChanels is set to true in the config or from System Console > Site Configuration > Users and Teams, set “Allow users to view archived channels” to true.

    2. Log in as user-A.

    3. Launch the incident backstage.

    4. Find the incident “a-inc-2” (ended incident).

    5. Click to view the summary.

    6. Click on the incident channel shortcut link.

    7. Verify that the user lands on the archived channel of the incident channel.

    N/A

    https://mattermost.atlassian.net/browse/MM-24408

    62

    66

    Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:falseshows a message that “the channel is not accessible, contact system admin”

    N/A

    ??? Should the link be inactive and in greyscale if false?

    63

    67

    Incident summary shows the current status of the incident

    1. From the incident list view.

    2. Find and click “a-inc-1”.

    3. In the incident summary page, verify that the status shows “Ongoing”.

    4. Go back to the incident list view.

    5. Find and click “a-inc-2”.

    6. In the incident summary page, verify that the status shows “Ended”.

    Pass

    64

    68

    Incident summary shows the current commander of the incident

    1. As user-B start an incident.

    2. Launch the incident backstage.

    3. Find and click the incident from step 1.

    4. Verify the summary page shows user-B as the commander.

    5. Go to the incident channel.

    6. Add user-A.

    7. Launch the incident RHS and click the incident from step 1.

    8. Change the commander to user-A.

    9. Launch the incident backstage.

    10. Find and click the incident from step 1.

    11. Verify the summary page shows user-A as the commander.

    Pass

    65

    69

    For active incidents, incident summary shows the duration of how long the incident has been active for

    1. Log in as user-A.

    2. Launch the incident backstage.

    3. Find and click “a-inc-3”.

    4. In the summary page, verify that the “Duration” section shows the duration the incident has been active for.

    5. Verify the status in the “Duration” section says “Ongoing”.

    Pass

    66

    70

    For inactive incidents, incident summary shows the duration of how long the incident remained active for before the incident ended

    1. Log in as user-A.

    2. Launch the incident backstage.

    3. Find and click “a-inc-2” (inactive incident).

    4. In the summary page, verify that the “Duration” section shows the duration the incident had been active for.

    5. Verify there is no “Ongoing” status in the “Duration” section.

    Pass

    67

    71

    For active incidents, incident summary shows the current number of members in the incident channel

    Continue from the above test.

    1. Find and click “a-inc-3”.

    2. In the summary page, verify that the “Members Involved” section shows “1” as the number of members involved in the incident.

    3. Go to the incident channel of “a-inc-3”.

    4. Add user-B and admin to the channel.

    5. Launch incident backstage.

    6. Find and click “a-inc-3”.

    7. In the summary page, verify that the “Members Involved” section shows “3” as the number of members involved in the incident.

    Pass

    68

    72

    For an inactive incident, incident summary shows the number of

    member

    members in the incident channel when the incident was ended

    Continue from the above test.

    1. Kick admin from the “a-inc-3” incident's channel.

    2. End incident “a-inc-3”.

    3. Launch incident backstage.

    4. Find and click “a-inc-3”.

    5. In the summary page, verify that the “Members Involved” section shows “2” as the number of members involved in the incident.

    69

    ???

    This will need to be tested once the channel-archive-on-ending feature is finalized

    73

    For active incident, incident summary shows the total number of messages in the incident channel

    1. Create some posts in the incident channel for “a-inc-1”.

    2. Make a note of the number of messages.

    3. Launch the incident backstage.

    4. Find “a-inc-1” and click to view the summary.

    5. Verify that the number of messages shown in the “Messages” section matches the numbers noted in step 2.

    70

    For inactive incident, incident summary shows the total number

    What counts as a message? Do bot messages count? In that case, some numbers are not showing up correctly.

    Image AddedImage AddedImage Added

    Why is the first one 2 messages? Shouldn’t it be 1?

    74

    For inactive incident, incident summary shows the total number of messages in the archived channel of the inactive incident

    Continue from the above test.

    1. End the incident “a-inc-1”.

    2. Launch the incident backstage.

    3. Find “a-inc-1” and click to view the summary.

    4. Verify that the number of messages shown in the “Messages” section matches the numbers noted in the test above.

    This will need to be tested once the channel-archive-on-ending feature is finalized

    Test Area - Tests/bugs outside of user stories

    Test ID

    Test Case

    Test Procedure

    Result

    Ticket

    Notes

    75

    Playbook name too long

    Playbook name empty

    https://mattermost.atlassian.net/browse/MM-25402

    76

    Clicking checklist item timestamp should permalink jump to corresponding post

    1. Click on the incident icon on the channel header to bring up the the incident RHS.

    2. Create a new incident with the following checklist items:

      1. item 1

      2. item 2

      3. item 3

    3. Check the first item (item 1) in the checklist.

    4. Verify a timestamp showing the time the item was checked appears beside the checklist item.

    5. Verify a bot message “user-1 checked off checklist item 'item 1'“ appears in the center channel.

    6. In the incident channel create a few posts.

    7. In the RHS, check the third item (item 3).

    8. Verify a timestamp showing the time the item was checked appears beside the checklist item.

    9. Verify a bot message “user-1 checked off checklist item 'item 3'“ appears in the center channel.

    10. In the incident channel create a few posts so that the bot message that says the checklist item was checked off hides beyond the post view.

    11. In the RHS, click on the timestamp beside the first checklist item (item 1).

    12. Verify that the post jumps to (and highlights?) the bot message in step 5.

    13. In the RHS, click on the timestamp beside the third checklist item (item 3).

    14. Verify that the post jumps to (highlights?) the message in step 9.

    Pass

    https://mattermost.atlassian.net/browse/MM-24423

    77

    Add a message indicating no checklist items in Incidents RHS

    https://mattermost.atlassian.net/browse/MM-24502

    Moved to 0.5

    78

    Display incident checklist timeline graph

    https://mattermost.atlassian.net/browse/MM-24577

    Ticket is still open

    79

    Support incident list pagination

    1. Create a new team so that there are no incidents.

    2. Create 24 incidents.

    3. Launch the incident backstage.

      1. From the channel header, click the incident icon.

      2. In the incident RHS, click the playbook icon to launch the backstage.

      3. Click the “Incidents” tab on the LHS.

    4. Verify that all 24 incidents created in step 2 are displayed in the list on the same page.

    5. Verify there is no “Next” and “Previous” buttons at the bottom of the page.

    6. Create one more incident so that the total number of incidents are 25.

    7. In the incident backstage, verify that all 25 incidents are displayed in the same page.

    8. Verify there is no “Next” and “Previous” buttons at the bottom of the page.

    9. Create one more incident so that the total number of incidents are now 26.

    10. In the incident backstage, scroll to the bottom of the page.

    11. Verify there are 25 incidents displayed on the first incident list page.

    12. Verify there is no “Previous” button.

    13. Verify there is a “Next” button.

    14. Click on the “Next” button.

    15. Verify that page 2 of the incident lists page loads.

    16. Verify that there is only 1 incident in the 2nd page.

    17. Verify there is “Previous” button in the 2nd page.

    18. Create incidents so that there are now a total of 51 incidents.

    19. In the 2nd page of the incident list page in the backstage, verify there is “Previous” button and a “Next” button.

    20. Verify both “Previous” and “Next” button

    are functional.

    1. are functional.

    https://mattermost.atlassian.net/browse/MM-24580

    Moved to 0.5

    80

    Support incident list sorting on End Date

    1. Create a few incidents at varying times.

    2. Launch the incident backstage to view the incident list.

    3. Verify that incidents are listed with the newest one on top by default.

    4. Click “End Date” in the header.

    5. Verify that the incident list is rearranged such that the last one that was ended is now at the top of the list, followed by other ‘Ended’ incidents in descending order of the time the incidents were ended, followed by the ‘Ongoing’ incidents (descending order of creation?)

    6. Verify that all other incident info are still associated with the incidents correctly.

    7. Click “End Date” again.

    8. Verify that the incident list is rearranged such that the first incident to end is now at the top of the list, followed by other ‘Ended’ incidents in ascending order of the time the incidents were ended, followed by the ‘Ongoing’ incidents (ascending order of creation?).

    9. Verify that all other incident info are still associated with the incidents correctly.

    https://mattermost.atlassian.net/browse/MM-24581

    Moved to 0.5

    81

    Support incident list sorting on Start Date

    Continue from the test for “Support incident list sorting on End Date”

    1. Click “Start Date” in the header.

    2. Verify that the incident list is rearranged such that the first incident to get created is now at the top of the list, followed by other incidents (ongoing or ended) in ascending order of the time the incidents were created.

    3. Verify that all other incident info are still associated with the incidents correctly.

    https://mattermost.atlassian.net/browse/MM-24581

    Moved to 0.5

    82

    Omit playbooks table header if empty

    https://mattermost.atlassian.net/browse/MM-24721

    Ticket is still open

    83

    Reduce in-channel checklist notifications

    Pass

    https://mattermost.atlassian.net/browse/MM-24731

    84

    Desktop: Channel Export does not download file

    https://mattermost.atlassian.net/browse/MM-25257

    Can still repro

    85

    Checklist items can be created with empty string

    Pass

    https://mattermost.atlassian.net/browse/MM-24908

    86

    Filter incidents to active team

    Pass

    https://mattermost.atlassian.net/browse/MM-25024

    From ticket desc: The incidents list in the backstage currently provides no `teamId` to the server, resulting in the admin seeing incidents across all teams, but non-admins seeing nothing.

    87

    Incident list on RHS is empty on Safari

    Pass

    https://mattermost.atlassian.net/browse/MM-24743

    88

    Playbook can be created with an empty string char

    Pass

    https://mattermost.atlassian.net/browse/MM-

    24580

    Support incident list sorting on End Date

  • Create a few incidents at varying times.

  • Launch the incident backstage to view the incident list.

  • Verify that incidents are listed with the newest one on top by default.

  • Click “End Date” in the header.

  • Verify that the incident list is rearranged such that the last one that was ended is now at the top of the list, followed by other ‘Ended’ incidents in descending order of the time the incidents were ended, followed by the ‘Ongoing’ incidents (descending order of creation?)

  • Verify that all other incident info are still associated with the incidents correctly.

  • Click “End Date” again.

  • Verify that the incident list is rearranged such that the first incident to end is now at the top of the list, followed by other ‘Ended’ incidents in ascending order of the time the incidents were ended, followed by the ‘Ongoing’ incidents (ascending order of creation?).

    • Verify that all other incident info are still associated with the incidents correctly.

    24907

    89

    IR: Playbook with empty name is created

    1. Go to New Playbook

    2. Type something in the name.

    3. Erase all characters in the name.

    4. Click Save.

    Observed: Playbook is created with an empty name.
    Expected: It should not allow playbook to be created with no name.

    Pass

    https://mattermost.atlassian.net/browse/MM-

    24581

    Support incident list sorting on Start Date

    Continue from the test for “Support incident list sorting on End Date”

  • Click “Start Date” in the header.

  • Verify that the incident list is rearranged such that the first incident to get created is now at the top of the list, followed by other incidents (ongoing or ended) in ascending order of the time the incidents were created.

    • Verify that all other incident info are still associated with the incidents correctly.

    24904

    90

    Desktop app: Export Incident Channel does not download the csv file

    https://mattermost.atlassian.net/browse/MM-25399

    Works in browsers, but not in desktop app

    Duplicate of https://mattermost.atlassian.net/browse/MM-25257

    91

    Long incident name pushes the components of the header outside the normal view

    https://mattermost.atlassian.net/browse/MM-

    24581Omit playbooks table header if empty

    25400

    92

    Clicking checklist item timestamp when viewing the "item checked" post in incident channel reloads the page

    https://mattermost.atlassian.net/browse/MM-

    24721Reduce in-channel checklist notifications

    25401

    93

    https://mattermost.atlassian.net/browse/MM-

    24731

    25403

    ...

    IR (v0.4.0) Bugs:

    Jira Legacy
    serverSystem JIRA
    maximumIssues20
    columnskey,summary,type,assignee,reporter,status,resolution
    jqlQueryissuetype = Bug AND project = MM AND fixVersion = "IR (v0.4.0)"
    serverIdfa8b0166-b019-31be-aef3-0e1e83e7ecff