Page Comparison

Test ID

Test Case

Test Procedure

Result

Notes

Clicking playbook button on RHS opens Playbook backstage by default

Backstage shows a tab for incidents

Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab

Clicking on “Incidents” tab on backstage LHS loads the list of all incidents

The incidents list view displays the team name the incidents belong to

The incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

The incident list in the backstage shows all active incidents that belong to the current team only

https://mattermost.atlassian.net/browse/MM-24555

The incident list in the backstage shows all inactive incidents that belong to the current team only

https://mattermost.atlassian.net/browse/MM-24555

Active incidents show an “Ongoing” status in the Status column in the backstage

Inactive incidents (ended incidents) show an “Ended” status in the Statuscolumn in the backstage.

The incident list in the backstage shows updated end timestamp for an incident

An active incident shows “Ongoing” in End Date column

The incident list in the backstage shows updated commander for an incident

The incident names in the “Name” column render as clickable links

Long incident name wraps in the “Name” column displaying the entire incident name

As the incident plugin, the channel is automatically archived when the incident ends so that it reduces clutter.

Channel is archived when ending an incident with slash command

Channel is archived when ending an incident from RHS

The incident channel is no longer available on the LHS once the incident is ended

The incident channel shows up under archived channel once the incident is ended

1. Login as a normal user.

2. Go to Team-A.

3. From the channel header, click the incident icon to open the incident RHS.

4. Click the playbook icon in the RHS header.

5. Verify Playbooks backstage is launched.

6. Verify the backstage shows a list of playbooks if they exist.

7. Verify the Playbooks tab is highlighted on the backstage LHS.

Backstage shows a tab for incidents

Continue from the above test.

1. Create incidents a-inc-1, a-inc-2, a-inc-3.

2. Note the time of incident creation for each incidents for later tests.

3. Verify that there is an “Incidents” tab in the backstage LHS in addition to the Playbooks tab.

Clicking on “Incidents” tab on backstage LHS highlights the Incidents tab

Continue from the above test.

1. Click on the “Incidents” tab on the LHS.

2. Verify that the highlight shifts from Playbooks to Incidents.

Clicking on “Incidents” tab on backstage LHS loads the list of all incidents

Continue from the above test.

1. Verify that clicking on the “Incidents” tab on the LHS loads the incidents list view.

2. Verify that all incidents from Team-A including a-inc-1, a-inc-2, a-inc-3 are in the list.

Incidents list view displays the team name the incidents belong to

Continue from the above test.

1. Verify that the team’s name - “Team-A” is displayed in the backstage header.

Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

Continue from the above test.

1. Verify that the incidents backstage shows incident details including name, status, start date, end date and commander for each incident.

The incident list in the backstage shows all active incidents that belong to the current team only

1. Switch to Team-B.

2. Create new incidents: Inc-a, Inc-b and Inc-c

3. Launch the incident backstage from within Team-B.

4. Verify that the incident list only shows the three incidents created in step 2.

5. Switch to Team-A.

6. Launch the incident backstage from within Team-A.

7. Verify that the incident list only shows the incidents created within Team-A.

8. Verify that the incident list does not show the incidents created in step 2.

9. Verify that the incident list shows all currently active incidents in the list.

https://mattermost.atlassian.net/browse/MM-24555

The incident list in the backstage shows all inactive incidents that belong to the current team only

1. Switch to Team-B.

2. End incident Inc-b and Inc-c.

3. Note the incident end time for later tests.

4. Launch the incident backstage from within Team-B.

5. Verify that the incident list shows all three incidents including both active and inactive incidents.

6. Verify none of the incidents from Team-A are in the list.

7. Switch to Team-A.

8. End incident a-inc-2.

9. Note the incident end time for later tests.

10. Launch the incident backstage.

11. Verify that the incident list shows all incidents created and ended with Team-A.

https://mattermost.atlassian.net/browse/MM-24555

Active incidents show an “Ongoing” status in the Status column in the backstage

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that incidents a-inc-1, and a-inc-3 show the status as “Ongoing” in the “Status” column.

3. Launch the incident backstage from Team-B.

4. Verify that incident Inc-a shows “Ongoing” status.

Inactive incidents (ended incidents) show an “Ended” status in the Statuscolumn in the backstage.

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that incident a-inc-2 shows the status as “Ended” in the “Status” column.

3. Launch the incident backstage from Team-B.

4. Verify that incidents Inc-b and Inc-c show “Ended” in the “Status” column.

The incident list in the backstage shows updated end timestamp for an incident

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that the “End Date” for a-inc-2 is accurate.

An active incident shows “Ongoing” in End Date column

Continue from the above test.

1. Verify that a-inc-1 and a-inc-3 show “Ongoing” under “End Date”.

The incident list in the backstage shows updated commander for an incident

1. Go to the normal channel view.

2. Invite user-B to the incident channel of a-inc-3.

3. Launch incident RHS.

4. Click on a-inc-3 to see the details.

5. Change the commander to user-B.

6. Launch the incident backstage.

7. In the incident list, find a-inc-3.

8. Verify that user-B is listed in the “Commander” column.

The incident names in the “Name” column render as clickable links

Continue from the above test.

1. Verify that all the incident names in the “Name” column as rendered as clickable links.

Long incident name wraps in the “Name” column displaying the entire incident name

1. Start a new incident such that the number of characters exceed the column width of “Name”.

2. Launch the incident backstage.

3. In the incident list view, verify that the name of the incident wraps in the next line so that the entire incident name is visible.

As the incident plugin, the channel is automatically archived when the incident ends so that it reduces clutter.

Channel is archived when ending an incident with slash command

1. Set ExperimentalViewArchivedChannels to true or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to true.

2. Start a new incident using any method.

3. Locate the channel in the LHS.

4. End the incident using the /incident end command.

5. Click “Confirm” in the confirmation dialog.

6. Verify that the incident ended.

   1. Incident no longer appears in the RHS list.

   2. Status of the incident changed to “Ended” in the backstage.

7. Verify that the incident channel is archived:

   1. In the LHS, verify the incident channel is listed with the archived icon.

   2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

Channel is archived when ending an incident from RHS

1. Start a new incident using any method.

2. Locate the channel in the LHS.

3. End the incident from the RHS.

4. Click “Confirm” in the confirmation dialog.

5. Verify that the incident ended.

   1. Incident no longer appears in the RHS list.

   2. Status of the incident changed to “Ended” in the backstage.

6. Verify that the incident channel is archived:

   1. In the LHS, verify the incident channel is listed with the archived icon.

   2. Running CLI command ./mattermost channel list [team-name] shows the channel as '(archived)'.

The incident channel shows up under archived channel once the incident is ended

1. On the LHS click “more” under Private Channels.

2. Ensure “Show: Archived Channels” is selected in the dropdown.

3. Verify the channels of the incidents ended in the above 2 tests are listed.

Incident channels do not appear in LHS when archived channel display is turned off

1. Set ExperimentalViewArchivedChannels to false or, from System Console > Site Configuration > User and Teams, set “Allow users to view archived channels” to false.

2. Switch to normal channel view.

3. Verify that the channels of the incidents ended in the tests above do not appear on the LHS.

Test ID

Test Case

Test Procedure

Result

Notes

When Channel Export plugin is not installed, the “Export Incident Channel” link is inactive

1. Ensure that the Channel Export plugin is not installed.

2. Log in as an admin.

3. Launch the incident backstage.

4. Click on a-inc-1 to view the incident summary.

5. Verify that the “Export Incident Channel” link and icon appear in greyscale.

6. Click on the link.

7. Verify that the link is inactive and nothing happens upon clicking.

8. Logout and log in as an incident member.

9. Repeat steps 3-7 for an incident the user is a member of.

10. Go back to the incident list view.

11. Find an incident the user is not a member of.

12. Repeat steps 5-7.

When Channel Export plugin is not installed, hovering over the “Export Incident Channel” link shows a message to “install the channel export plugin”

Continue from the above test.

1. Login as an admin.

2. Launch incident backstage and click on an incident.

3. Hover over the “Export Incident Channel” link.

4. Verify that the tooltip shows the message “Install and enable the channel export plugin to support exporting this incident.”

5. Log out and login as a normal user.

6. Find an incident the user is a member of.

7. Repeat steps 3-4.

8. Go back to the incident list view.

9. Find an incident the user is not a member of.

10. Repeat steps 3-4.

When Channel Export plugin is installed in an instance with no EE license, the “Export Incident Channel” link is inactive

1. Ensure the server has no EE license.

2. Log in as an admin.

3. Install and enable the Channel Export plugin.

4. Launch the incident backstage.

5. Click on a-inc-1 to view the incident summary.

6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

7. Click on the link.

8. Verify that the link is inactive and nothing happens upon clicking.

9. Logout and log in as an incident member.

10. Repeat steps 4-8 for an incident the user is a member of.

11. Go back to the incident list view.

12. Find an incident the user is not a member of.

13. Repeat steps 4-8.

When Channel Export plugin is installed in an instance with no EE license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

Continue from the above test.

1. Login as an admin.

2. Launch incident backstage and click on an incident.

3. Hover over the “Export Incident Channel” link.

4. Verify that the tooltip shows the message “Exporting an incident channel requires a Mattermost Enterprise E20 license.”

5. Log out and login as a normal user.

6. Find an incident the user is a member of.

7. Repeat steps 3-4.

8. Go back to the incident list view.

9. Find an incident the user is not a member of.

10. Repeat steps 3-4.

When Channel Export plugin is installed in an instance with an E10 license, the “Export Incident” link is inactive

1. Ensure the server has an E10 license.

2. Log in as an admin.

3. Install and enable the Channel Export plugin.

4. Launch the incident backstage.

5. Click on a-inc-1 to view the incident summary.

6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

7. Click on the link.

8. Verify that the link is inactive and nothing happens upon clicking.

9. Logout and log in as an incident member.

10. Repeat steps 4-8 for an incident the user is a member of.

11. Go back to the incident list view.

12. Find an incident the user is not a member of.

13. Repeat steps 4-8.

When Channel Export plugin is installed in an instance with an E10 license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

Continue from the above test.

1. Login as an admin.

2. Launch incident backstage and click on an incident.

3. Hover over the “Export Incident Channel” link.

4. Verify that the tooltip shows the message “Exporting an incident channel requires a Mattermost Enterprise E20 license.”

5. Log out and login as a normal user.

6. Find an incident the user is a member of.

7. Repeat steps 3-4.

8. Go back to the incident list view.

9. Find an incident the user is not a member of.

10. Repeat steps 3-4.

When Channel export plugin is installed but not enabled, the “Export Incident Channel” link is inactive

1. Ensure the server has an E20 license.

2. Log in as an admin.

3. Install the Channel Export plugin but do not enable it.

4. Launch the incident backstage.

5. Click on a-inc-1 to view the incident summary.

6. Verify that the “Export Incident Channel” link and icon appear in greyscale.

7. Click on the link.

8. Verify that the link is inactive and nothing happens upon clicking.

9. Logout and log in as an incident member.

10. Repeat steps 4-8 for an incident the user is a member of.

11. Go back to the incident list view.

12. Find an incident the user is not a member of.

13. Repeat steps 4-8.

When Channel Export plugin is installed but not enabled, hovering over the “Export Incident Channel” shows “enable the plugin” message

Continue from the above test.

1. Login as an admin.

2. Launch incident backstage and click on an incident.

3. Hover over the “Export Incident Channel” link.

4. Verify that the tooltip shows the message “Enable the channel export plugin to support exporting this incident.”

5. Log out and login as a normal user.

6. Find an incident the user is a member of.

7. Repeat steps 3-4.

8. Go back to the incident list view.

9. Find an incident the user is not a member of.

10. Repeat steps 3-4.

When Channel Export plugin is installed and enabled, an active “Export Incident Channel” link and icon is available in the incident details view as an active link

1. Ensure the server has an E20 license.

2. Log in as an admin.

3. Install the Channel Export and enable it.

4. Launch the incident backstage.

5. Click on a-inc-1 to view the incident summary.

6. Verify that the “Export Incident Channel” link and icon do not appear in greyscale.

7. Logout and log in as an incident member.

8. Repeat steps 4-7 for an incident the user is a member of.

When Channel Export plugin is installed and enabled, hovering over the “Export Incident Channel” link shows a message that the action will download the channel content

Continue from the above test.

1. Login as an admin.

2. Launch incident backstage and click on an incident.

3. Hover over the “Export Incident Channel” link.

4. Verify that the tooltip shows the message that the action will download the channel content.

5. Log out and login as a normal user.

6. Find an incident the user is a member of.

7. Repeat steps 3-4.

I think I added this on my own for consistency

A system admin can export the incident channel the admin is a member of by clicking “Export Incident Channel” link

1. Log in as an admin.

2. Launch incident backstage.

3. Click on an incident the admin is a member of.

4. Verify that the link is not in greyscale.

5. Click the “Export Incident Channel” link.

6. Verify that the channel export action begins.

7. Verify that a csv file is downloaded.

8. Click on the channel export icon

   Image Added

9. Verify steps 4, 6 and 7.

The features of the Channel Export plugin were tested in Test Plan - Channel Export v0.0.1

A system admin can export the incident channel the admin is NOT a member of by clicking “Export Incident Channel” link

1. Log in as an admin.

2. Launch incident backstage.

3. Click on an incident the admin is not a member of.

4. Verify that the link is not in greyscale.

5. Click the “Export Incident Channel” link.

6. Verify that the channel export action begins.

7. Verify that a csv file is downloaded.

8. Click on the channel export icon

   Image Added

9. Verify steps 4, 6 and 7.

An incident member can export the incident channel by clicking “Export Incident Channel” link

1. Log in as a normal user.

2. Launch incident backstage.

3. Click on an incident the user is a member of.

4. Verify that the link is not in greyscale.

5. Click the “Export Incident Channel” link.

6. Verify that the channel export action begins.

7. Verify that a csv file is downloaded.

8. Click on the channel export icon

   Image Added

9. Verify steps 4, 6 and 7.

The exported csv file captures the current status of the incident

1. Verify the contents of the csv file.

This feature was tested in Test Plan - Channel Export v0.0.1

“Export Incident Channel” button is inactive for non-incident user

1. Log in as a normal user.

2. Launch incident backstage.

3. Click on an incident the user is not a member of.

4. Verify that the link is in greyscale.

5. Click the “Export Incident Channel” link.

6. Verify that nothing happens.

7. Click on the channel export icon

   Image Added

8. Verify nothing happens.

A non-incident user sees a “no access” message upon hovering on the “Export Incident Channel” button

Continue from the above test.

1. Hover over the “Export Incident Channel” link.

2. Verify that the tooltip shows the message “You do not have access to this incident channel. Contact the commander to export this incident.“

Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage.

Exporting an ended incident channel from backstage exports the archived channel

1. From the incident backstage, find incident a-inc-e (or a different incident that has been ended).

2. Click to view the summary.

3. Click the “Export Incident Channel”.

4. Verify that a csv file is downloaded with the contents of the archived channel.

Non-member cannot export an ended incident channel

1. As user-B start incident “user-b-inc”.

2. Do not add user-A to the incident channel.

3. End the incident “user-b-inc”.

4. Login as user-A.

5. Launch the incident backstage.

6. Find the incident “user-b-inc”.

7. Click to view the summary.

8. Verify that the “Export Incident Channel” link is inactive and appear in greyscale.

9. Click the link.

10. Verify that nothing happens.

Don’t think this test is valid anymore since a non-member cannot see the summary at all in the backstage.

Verify the appearance of the “Export Incident Channel” link and icon appear in dark theme

1. Switch to a dark theme.

2. Verify that the “Export Incident Channel” is visible according to the theme.

Verify the appearance of the “Export Incident Channel” link and icon appear in light theme

1. Switch to a light theme.

2. Verify that the “Export Incident Channel” is visible according to the theme.

Verify the appearance of the “Export Incident Channel” link and icon appear in other themes

1. Switch to a custom theme.

2. Verify that the “Export Incident Channel” is visible according to the theme.

As the incident commander, I receive an incident summary and a link to more info after ending an incident so that I know where to go for post-mortem.
https://mattermost.atlassian.net/browse/MM-24409

Test ID

Test Case

Test Procedure

Result

Notes

When Channel Export plugin is not installed, the “Export Incident Channel” link is inactive and appears in greyscale

To admin

To incident member

To non-member

When Channel Export plugin is not installed, hovering over the “Export Incident Channel” link shows a message to “install the channel export plugin”

To admin

To incident member

To non-member

Install the channel export plugin to support exporting this incident.

When Channel Export plugin is installed in an instance with no EE license, the “Export Incident Channel” link is inactive and appears in greyscale

To admin

To incident member

To non-member

When Channel Export plugin is installed in an instance with no EE license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

To admin

To incident member

To non-member

Exporting an incident channel requires a Mattermost Enterprise E20 license.

When Channel Export plugin is installed in an instance with an E10 license, the “Export Incident” link is inactive and appears in greyscale

To admin

To incident member

To non-member

When Channel Export plugin is installed in an instance with an E10 license, hovering on the “Export Incident Channel” link shows a message that “E20 license is required”

To admin

To incident member

To non-member

Exporting an incident channel requires a Mattermost Enterprise E20 license.

When Channel export plugin is installed but not enabled, the “Export Incident Channel” link is inactive and appears in greyscale

To admin

To incident member

To non-member

When Channel Export plugin is installed but not enabled, hovering over the “Export Incident Channel” shows “enable the plugin” message

To admin

To incident member

To non-member

Enable the channel export plugin to support exporting this incident.

When Channel Export plugin is installed and enabled, an active “Export Incident Channel” link and icon is available in the backstage incident list view

When Channel Export plugin is installed and enabled, hovering over the “Export Incident Channel” link shows a message that the action will download the channel content

An incident member can export the incident channel by clicking “Export Incident Channel” link

Link is not in greyscale

Shows action when clicked

Clicking the “Export Incident Channel” link downloads the csv file of the incident channel

The exported csv file captures the current status of the incident

Admin can export the incident without being a member of the incident channel

“Export Incident Channel” button shows as a grey-inactive button to non-incident user

Should not be able to click

A non-incident user sees a “no access” message upon hovering on the “Export Incident Channel” button

You do not have access to this incident channel. Contact the commander to export this incident.

Verify the appearance of the “Export Incident Channel” link and icon appear in dark theme

Verify the appearance of the “Export Incident Channel” link and icon appear in other themes

Exporting an ended incident channel from backstage

As the incident commander, I receive an incident summary and a link to more info after ending an incident so that I know where to go for post-mortem.
https://mattermost.atlassian.net/browse/MM-24409

Test ID

Test Case

Test Procedure

Result

Notes

User cannot view incident summary if the user is not an incident member

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an active incident if the user is an incident member

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an inactive incident

Incident summary page displays the incident name as the page heading

Incident summary page has a “<“ button that brings a user back to the incidents list view

Incident summary page bears a link to the incident channel

Hovering on the incident channel link shows a tooltip suggesting that it’s a shortcut to the incident channel

Clicking on incident channel link of active incidents brings the user to the incident channel

Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:truebrings the user to the incident channel

https://mattermost.atlassian.net/browse/MM-24408

Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:falseshows a message that “the channel is not accessible, contact system admin”

???

Incident summary shows the current status of the incident

Test Procedure

Result

Notes

User cannot view incident summary if the user is not an incident member

1. Create an incident as user-B.

2. Do not add user-A to the incident channel.

3. Log in as user-A.

4. Launch the incident backstage.

5. Find the incident from step 1.

6. Click on the incident.

7. Verify that user-A is not taken to the incident’s summary page.

8. (Any message to contact the commander to view the summary?)

Is this test case correct since we’re limiting a non-user’s access to the incident summary.

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an active incident if the user is an incident member

1. Log in as user-A.

2. Ensure that “a-inc-1” is still an active incident

3. Launch the incident backstage.

4. Find the incident “a-inc-1”.

5. Click on the incident name.

6. Verify that the incident summary page loads.

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an inactive incident

1. Log in as user-A.

2. Launch the incident backstage.

3. Find the incident “a-inc-2” (the ended incident).

4. Click on the incident name.

5. Verify that the incident summary page loads.

Incident summary page displays the incident name as the page heading

Continue from the above test.

1. In the incident summary page, verify that the heading of the page is the name of the incident.

Incident summary page has a “<“ button that brings a user back to the incidents list view

Continue from the above test.

1. In the incident summary page, verify there is a “<“ button beside the incident name header.

2. Click on the “<“ button.

3. Verify that the user is brought back to the incident list view in the backstage.

Incident summary page bears a link to the incident channel

Continue from the above test.

1. Click on the incident “a-inc-3” (still an active incident) to view the summary page.

2. Verify that a link to the incident channel is present beside the incident name header.

   Image Added

Hovering on the incident channel link shows a tooltip suggesting that it’s a shortcut to the incident channel

Continue from the above test.

1. Hover over the incident channel link icon.

2. Verify that a tooltip shows “Go to incident channel”.

Clicking on incident channel link of active incidents brings the user to the incident channel

Continue from the above test.

1. Click on the incident channel link.

2. Verify the user lands on the correct incident channel.

Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:truebrings the user to the incident channel

1. As an admin, ensure that the ExperimentalViewArchiveChanels is set to true in the config or from System Console > Site Configuration > Users and Teams, set “Allow users to view archived channels” to true.

2. Log in as user-A.

3. Launch the incident backstage.

4. Find the incident “a-inc-2” (ended incident).

5. Click to view the summary.

6. Click on the incident channel shortcut link.

7. Verify that the user lands on the archived channel of the incident channel.

https://mattermost.atlassian.net/browse/MM-24408

Clicking on incident channel link of inactive incidents, when ”ExperimentalViewArchiveChannels”:falseshows a message that “the channel is not accessible, contact system admin”

??? Should the link be inactive and in greyscale if false?

Incident summary shows the current status of the incident

1. From the incident list view.

2. Find and click “a-inc-1”.

3. In the incident summary page, verify that the status shows “Ongoing”.

4. Go back to the incident list view.

5. Find and click “a-inc-2”.

6. In the incident summary page, verify that the status shows “Ended”.

Incident summary shows the current commander of the incident

1. As user-B start an incident.

2. Launch the incident backstage.

3. Find and click the incident from step 1.

4. Verify the summary page shows user-B as the commander.

5. Go to the incident channel.

6. Add user-A.

7. Launch the incident RHS and click the incident from step 1.

8. Change the commander to user-A.

9. Launch the incident backstage.

10. Find and click the incident from step 1.

11. Verify the summary page shows user-A as the commander.

For active incidents, incident summary shows the duration of how long the incident has been active for

For inactive incidents, incident summary shows the duration of how long the incident remained active for before the incident ended

For active incidents, incident summary shows the current number of members in the incident channel

For an inactive incident, incident summary shows the number of member in the incident channel when the incident was ended

For active incident, incident summary shows the total number of messages in the incident channel

For inactive incident, incident summary shows the total number of messages in the archived channel of the inactive incident