Versions Compared

Version Old Version 15 New Version 16
Changes made by

Prapti Shrestha (Deactivated)

Prapti Shrestha (Deactivated)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Document version 0.1

Status: Test

...

in Progress…

Document Version

Description

Date

0.0

Initial version

08/07

0.1

Added tests for v0.7.

Added test steps for the test cases.

Test plan ready for review.

08/16

...

  • A Mattermost test server running the latest version.

  • The test server may not need to be equipped with an EE license.

  • Incident Plugin v0.6.

  • Channel Export Plugin v0.2

Test Server: https://ir-test.test.mattermost.cloud/

Test Server (upgrade tests):

Build Hash:  fb453d
EE Build Hash:  7251aa
Webapp Build Hash: : d0bcba

Incident Response plugin version: v0.6.0-alpha.3 (build hash d60f2f)

Test Date: 08/20/2020

Tests

Test Area - Plugin Setup

Test ID

Test Case

Test Steps

Result

Notes

1

Admin can install plugin in a non-EE instance

  1. Log in to a non-enterprise Mattermost, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

2

Admin can install plugin in an E10 instance

  1. Log in to Mattermost that has an E10 license, as admin.

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

  1. Log in to Mattermost that has an E20 license, as admin

  2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

  3. Verify that the plugin can be installed.

Pass

4

Admin can enable plugin

  1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.6 and click on the “Enable” button.

  2. Verify that the plugin starts normally without errors.

Pass

5

Plugin functionality is available to admin

  1. Switch to a normal channel.

  2. Type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

Pass

6

Plugin functionality is available to normal user

  1. Log in as a non-admin user.

  2. In a normal channel, type /incidentin the post input box.

  3. Verify that /incidentis listed as a slash command option.

  4. Verify that the channel header has Incident Response icon for RHS.

  5. Create a post.

  6. Click post action menu for the post.

  7. Verify that Start Incident is available as a post action menu option.

7

Admin can disable plugin

  1. Log out and log back in as admin.

  2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  3. Click on the “Disable” button.

  4. Verify that the plugin is disabled.

  5. In the main channel’s post input box, type /incident.

  6. Verify that /incident option is not present in the slash command list.

Pass

8

Admin can remove plugin

  1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

  2. Click on the “Remove” button.

  3. Verify that the plugin is no longer installed in the instance.

Pass

9

Plugin upgrades normally from v0.4 to v0.6

  1. Install version 0.4 of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

10

Plugin upgrades normally from v0.5 to v0.6

  1. Install version 0.5 of the plugin.

  2. Upgrade to the latest version.

  3. Verify that the plugin upgrade happens without any error.

11

Incidents started with the prior plugin versions are still active and available in v0.6

  1. Start a few incident with plugin version 0.5.

  2. Upgrade plugin to v0.6.

  3. Open up the incident RHS to see the incident list.

  4. Verify that incidents started with version 0.5 are still available in v0.6.

12

Incidents ended with prior plugin versions are not available in v0.6

  1. Start a few incident with plugin version 0.5.

  2. Leave some as active and end a few of them.

  3. Upgrade plugin to v0.6.

  4. Open up the incident RHS to see the incident list.

  5. Verify that incidents started with version 0.5 are still available in v0.6.

  6. Verify that incidents ended in step 2 are not visible in the list.

13

Playbook started with the older plugin version is still available upon plugin upgrade

  1. Create a few playbooks with plugin version 0.5.

  2. Upon plugin upgrade, verify that the playbooks created in step 1 are still available in the backstage.

...

Test Area (v0.7) - Execute the attached slash command with a button click

The tests below might need to get adjusted (or added) to match Revised step execution: https://mattermost.atlassian.net/browse/MM-27012

Test ID

Test Case

Test Steps

Result

Notes

100

A checklist item configured without a slash command does not have a Run button next to it

  1. Log in as user-1.

  2. Launch playbook backstage.

  3. Click “New Playbook”.

  4. Enter “Apb” as the playbook name.

  5. Change the name of “Default Stage” to “S1”.

  6. Click “Add new checklist item”.

  7. Enter “C1” as the item name.

  8. Don’t add anything for the slash command.

  9. Save playbook.

  10. Switch to normal channel view.

  11. Start a new incident using “Apb”.

  12. Once the incident is started, in the incident RHS, examine the checklist item “C1” under “S1”.

  13. Verify that “C1” does not have any “Run” button.

101

A checklist item configured with a valid slash command has a Run button next to it

Continue from the above test.

  1. Launch playbook.

  2. Find and click “Apb” to edit.

  3. Add slash command /away for “C1”.

  4. Switch to normal channel view.

  5. Start a new incident using “Apb”.

  6. Once the incident is started, in the incident RHS, verify that “C1” show /away under it which has a Run button next to it.

102

A checklist item configured with an invalid slash command does not have a Run button next to it

Continue from the above test.

  1. Launch playbook backstage.

  2. Find and click “Apb” to edit.

  3. Add a new checklist item “C2” under “S1”.

  4. Add slash command invalid text for “C2”.

  5. Save playbook.

  6. Switch to normal channel view.

  7. Start a new incident using “Apb”.

  8. Once the incident has started, find “C2” under “S1”.

  9. Verify that the slash command for “C2” doesn’t have a Run button next to it.

This doesn’t happen currently. It would require us to spend mana on validating the slash command.

103

A checklist item configured with a nonexistent slash command does not have a Runbutton next to it

Continue from the above test.

  1. Launch playbook backstage.

  2. Find and click “Apb” to edit.

  3. Add a new checklist item “C3” under “S1”.

  4. Add slash command /invalidcommand for “C3”.

  5. Save playbook.

  6. Switch to normal channel view.

  7. Start a new incident “Ia1” using “Apb”.

  8. Once the incident has started, find “C3” under “S1”.

  9. Verify that the slash command for “C3” doesn’t have a Run button next to it.

104

Clicking the Run button for a checklist slash command runs the slash command

Continue from the above test.

  1. Open up the incident RHS for “Ia1”.

  2. Find “C1” under stage “S1”.

  3. Click the Run button next to “C1”.

  4. Verify that clicking the Run button executes so that the status of user-1 is now set to “away”.

105

Plugin posts a message in the incident channel when a slash command Runis clicked

Continue from the above test.

  1. Verify that plugin posts a message in the “Ia1” incident channel to notify that the slash command for “C1” has been run.

A message that the checklist item slash command was run (not the slash command system message)

Ian Tao (Unlicensed) should we make a ticket for this feature? We currently don’t do it, but it makes sense.

https://mattermost.atlassian.net/browse/MM-27964

106

Running a slash command with system message attached to it post the slash command system message in the incident channel

Continue from the above test.

  1. Verify that the system message for the /away command “System: You are now away (Only visible to you)” is posted in the incident channel.

107

In case of two valid slash commands, the running the command from the checklist item will only run the first command successfully

Continue from the above test.

  1. Launch playbook backstage.

  2. Find and click “Apb” to edit.

  3. Under “S1” add a new checklist item “C5”.

  4. Add the following for “C5’s” slash command: /online /away.

  5. Switch to normal channel view.

  6. Start a new incident using “Apb”.

  7. Once the incident has been started, find “C5” in the incident RHS.

  8. Click the Run button next to “C5”.

  9. Verify that the status of user-1 is now set to “online” and not “away”.

This is in sync with the core feature behavior

Seems correct to me (ChrisP)

108

Running an invalid slash command from RHS checklist shows an ‘invalid slash command’ error

Depends on the steps in #102.

Like core feature?

109

A slash command can be run multiple times from the RHS checklist

Continue from the above tests.

  1. Launch playbook backstage.

  2. Find and click “Apb” to edit.

  3. Under “S1” add a new checklist item “C6”.

  4. Add /shrug as the slash command for “C6”.

  5. Switch to normal channel view.

  6. Start a new incident using “Apb”.

  7. Once the incident has been started, find “C6” in the incident RHS.

  8. Click the Run button for “C6”.

  9. Verify that the /shrug command is executed successfully and a shrug message is posted in the center channel.

  10. Repeat steps 8-9 multiple times.

110

A plugin post notifies when a slash command fails to run successfully

Verify that when a slash command fails to execute by clicking the Run button, the plugin posts a message in the center channel notifying that the execution has failed.

...