Test Plan - Incident Response v0.1.0

0.0

Initial version

03/24

0.1

• Updated based on @Maria Nunez 's feedback.

• Questions remaining on test cases 2, 12, 14, 20, 21.

• Tests updated on 13, 15, 18, 25, 26, 29, 30, 31.

03/25

0.2

• Updated document to reflect the new versioning of the Incident Response plugin.

• Added test cases based on Ali’s feedback: 12, 13, 14, 15, 17, 19, 20, 21, 22, 25, 26, 27, 28, 36, 37, 38, 52, 55, 57 (marked with colored cells)

• Questions remaining on test cases: 2
  
  

03/31

1.0

1. Tests updated for v0.1.0

2. Test results updated with tests for v0.1.0

04/03

Test ID

Test Case

Test Procedure

Result

Notes

1

System administrator can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

Pass

2

System administrator can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

3

System administrator cannot can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin cannot can be installed.

Pass

Updated this test since the plugin v0.1.0 can be used in a non-EE instance.

4

System administrator can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

Pass

5

System administrator can access the plugin functionality

1. Navigate to a team’s channel.

2. Verify that `/incident` slash command is present in the command list.

   1. In the post text box in the center channel, type `/` and scroll down the list of available commands.

   2. `/incident` should be listed as a command in alphabetical order.

3. Post/find a message in any center channel.

4. Click on the post menu.

5. Verify that the Start incident option is present

6. Verify that the Incident Response plugin icon appears on the channel header area (see screenshot below)

Open

Pass

6

When no incidents have been started, the Incident Response RHS indicates that there are no incidents

1. When no incidents have been started, click on the Incident Response icon on the header.

2. Verify that the incident response RHS opens successfully.

3. Verify that the RHS has a message that there are no incidents to show.

4. Verify that there is a Create new incidentbutton.

Open

Pass

7

System administrator can start an incident

1. In the center channel, create an incident-response post with the following slash command: `/incident start`

2. Verify that a dialog prompt “Incident Details” asks for the incident’s channel name.

3. Give a channel name.

4. Click “Start Incident”.

5. Verify that an incident channel with the name chosen in step 3 is created.

6. Verify that the admin user is added to the incident channel created above in step b.

7. Verify that when the admin user clicks on the Incident Response plugin icon in the header, it opens up the RHS listing incident started in step a above.

Fail

/incident start did not create an incident

MM-23852: Incidents are sometimes failing to start with slash command and RHSClosed

8

System administrator can disable plugin

1. Navigate to System Console > PLUGINS > Plugin Management.

2. In the “Installed Plugins” section, find Incident Response and click on the “Disable” button.

3. Verify that the plugin gets disabled successfully without errors.

Pass

9

Incident data is preserved even after disabling the plugin

1. Navigate to the main channel.

2. In the center channel post text box, type `/incident`.

3. Verify that the `/incident` is not listed in the slash command list.

4. Verify that the Incident Response plugin icon is not available in the channel header.

5. Verify that the incident-response channel created in step 4.b still exists.

Pass

10

Incident data is not accessible after disabling the plugin

1. Navigate to the main channel.

2. In the center channel post text box, type `/incident`.

3. Verify that the `/incident` is not listed in the slash command list.

4. Try accessing the data from backdoor and verify it’s not accessible

11

Plugin functionality is available to normal user

1. Enable the plugin if disabled.

2. Log out if already logged in.

3. Log in as a normal user.

4. In the main channel, type `/incident`.

5. Verify that `/incident` is available in the command list.

6. Verify that the Incident Response plugin icon is present in the channel header.

Pass

Test ID

Test Case

Test Procedure

Result

Notes

12

Incident Response modal opens when starting an incident with slash command

1. As normal user, in center channel, type /incident start command and hit enter.

2. Verify that the Incident Details modal pops up as shown in the following screenshot.

Pass

13

Incident Response modal opens when starting an incident from RHS

1. Ensure that there are no active incidents.

2. Open up the Incident Response RHS by clicking on the Incident Response icon in channel header.

3. Click on the Create new incident button or the + button on RHS.

4. Verify that the Incident Details modal pops up.

Pass

14

Incident Response modal opens when starting an incident from post menu

1. From any post in center channel, click the post menu.

2. Click on the Start Incidentoption.

3. Verify that the Incident Details modal pops up.

Pass

15

Channel Name input box is focused when incident response modal pops up

1. Start an incident using any method to bring up the Incident Details modal.

2. Verify that the Channel Name input box is focused.

Fail

16

Incident is started when enter is pressed on Incident Response modal

1. Start an incident using any method to bring up the Incident Details modal.

2. Provide a channel name.

3. Press enter.

4. Verify that the incident appears in the Incident Response RHS.

5. Verify that a new incident channel with the channel name provided in step 2 appears in the LHS.

Fail

17

Incident creation is cancelled when esc is pressed on Incident Response modal

1. Bring up the Incident Details modal using any method.

2. Provide a channel name.

3. Press esc key.

4. Verify that the modal is dismissed.

5. Verify that the incident is not started.

   1. There is no such incident in the RHS.

   2. There is no channel for the incident in the LHS.

Pass

18

A normal user can start an incident with slash command in a public channel

1. Login as a normal user.

2. Open a public channel.

3. In a regular channel type the following command: `/incident start`

4. When the Incident Detailsmodal pops up, provide a channel name and click Start Incident.

5. Verify that the incident channel is created.

6. Verify that the user is added to the incident channel.

Fail

See test # 7

19

A normal user can start an incident with slash command in a private channel

1. Open a private channel.

2. Repeat steps 3-6 from test # 18.

Fail

See test # 7

20

A normal user can start an incident with slash command in a group message

1. Open up a group message.

2. Repeat steps 3-6 from test # 18.

Fail

See test # 7

21

A normal user can start an incident with slash command in a direct message

1. Open up a direct message with another user.

2. Repeat steps 3-6 from test # 18.

Fail

See test # 7

22

A normal user can start an incident with slash command in direct message to self

1. Open up the direct message with the user her/himself.

2. Repeat steps 3-6 from test # 18.

Pass

23

A normal user can start an incident from RHS in a public channel

1. Go to a public channel.

2. Click on the Incident Response icon on the channel header and open the RHS.

3. Click on the `+Create new incident` button to start a new incident

4. Verify that a new incident is listed in the RHS

5. Verify that a new incident channel is created

6. Verify that the user who started the incident in step d is added to the incident channel.

Pass

24

A normal user can start an incident from RHS in a private channel

1. Go to a private channel.

2. Click on the Incident Response icon on the channel header and open the RHS.

3. Click on the `+Create new incident` button to start a new incident

4. Verify that a new incident is listed in the RHS

5. Verify that a new incident channel is created

6. Verify that the user who started the incident in step d is added to the incident channel.

Pass

25

A normal user can start an incident from RHS in a group message channel

1. Go to a group message channel.

2. Click on the Incident Response icon on the channel header and open the RHS.

3. Click on the `+Create new incident` button to start a new incident

4. Verify that a new incident is listed in the RHS

5. Verify that a new incident channel is created

6. Verify that the user who started the incident in step d is added to the incident channel.

Fail

MM-23854: Non-admin user cannot start an incident from RHS in GM and DM channelsClosed

26

A normal user can start an incident from RHS in a direct message channel

1. Go to a direct message channel.

2. Click on the Incident Response icon on the channel header and open the RHS.

3. Click on the `+Create new incident` button to start a new incident

4. Verify that a new incident is listed in the RHS

5. Verify that a new incident channel is created

6. Verify that the user who started the incident in step d is added to the incident channel.

Fail

MM-23854: Non-admin user cannot start an incident from RHS in GM and DM channelsClosed

27

A normal user can start an incident from RHS in a direct message channel to self

1. Go to the direct message to self channel.

2. Click on the Incident Response icon on the channel header and open the RHS.

3. Click on the `+Create new incident` button to start a new incident

4. Verify that a new incident is listed in the RHS

5. Verify that a new incident channel is created

6. Verify that the user who started the incident in step d is added to the incident channel.

Fail

MM-23854: Non-admin user cannot start an incident from RHS in GM and DM channelsClosed

28

A normal user can start an incident from a post action menu a public channel

1. Open the Incident Response RHS (by clicking on the Incident Response icon in the header)

2. Create a post in a normal public channel

3. From the post menu, click on the `...` menu

4. Click on Start Incident.

5. Incident Details modal prompts for a new channel name.

6. Verify that a new incident is listed in the RHS

7. Verify that a new incident channel is created

8. Verify that the user who started the incident in step 4 is added to the incident channel.

9. Verify that the post where the incident was started in step 3 above, is linked in the new incident channel header.

10. → In the new incident channel, a post is created that links to in the original post where the incident was started?

Pass

29

A normal user can start an incident from a post action menu a private channel

1. Create a post in a private channel.

2. Repeat steps 3-10 from test # 24.

Pass

30

A normal user can start an incident from a post action menu a group message

1. Create a post in a group message.

2. Repeat steps 3-10 from test # 24.

Fail

MM-23855: Non-admin user cannot start an incident from post menu in GM and DM channelsClosed

31

A normal user can start an incident from a post action menu a direct message

1. Create a post in a direct message to a different user.

2. Repeat steps 3-10 from test # 24.

Fail

MM-23855: Non-admin user cannot start an incident from post menu in GM and DM channelsClosed

32

A normal user can start an incident from a post action menu a direct message to self

1. Create a post in a direct message to the user her/himself.

2. Repeat steps 3-10 from test # 24.

Fail

MM-23855: Non-admin user cannot start an incident from post menu in GM and DM channelsClosed

33

The user who started an incident is assigned/marked as the commander of the incident

1. Start a new incident (using any method).

2. Click on the Incident Response icon in the header to open the RHS.

3. Find the incident created in step a.

4. Verify that the user who started the incident in step a is listed as the commander of the incident.

Pass

34

A new private channel is created when an incident is started

1. Start an incident using slash command.

2. Verify that a private channel is created that is associated with the newly started incident.

3. Start an incident using the incident RHS.

4. Verify that a private channel is created that is associated with the newly started incident.

5. Start an incident using the post action menu.

6. Verify that a private channel is created that is associated with the newly started incident.

Pass

35

User who started the incident is auto-invited to the incident channel

1. Start an incident using any method.

2. Once the channel is created, verify that the user who started the incident is automatically invited to that channel.

Pass

36

The incident channel can be renamed

1. Start an incident (using any method).

2. Open the incident RHS.

3. Find the incident started in step 1 and click for details.

4. Go to the incident channel.

5. Click on the channel name in the header to bring up the channel’s drop down menu.

6. Click on “Rename Channel”.

7. Rename the channel to “New Incident Channel”.

8. Change the URL to “new-incident-channel”

9. Click “Save”.

10. Verify that the channel name is changed to “New Incident Channel” on the header.

11. Verify that the channel name is changed to “New Incident Channel” on LHS.

12. Verify that the channel name is changed to “New Incident Channel” on RHS incident details.

13. Visit the new URL of the channel set in step 8.

14. Verify that the New Incident Channel loads in the center space.

Pass

37

The incident channel is linked properly when renamed

1. In a new browser window, visit the URL as set in step 8 from the test above.

2. Verify that the user lands on the New Incident Channel.

3. Switch to a different channel in the same team.

4. In the Incident RHS, click on the channel link with the new channel name.

5. Verify that the user lands on the New Incident Channel.

Pass

38

A normal user can end an incident started by the same user using slash command

1. In the channel header, click on the Incident Response icon.

2. Verify that the incident started in the test case above, is listed in the RHS section.

3. Go to the incident response channel for an incident that has been started.

4. In the channel, post the following command: `/incident end`.

5. Verify that the incident has been ended

   1. In the incident RHS, verify the incident is deleted.

6. Verify that the message has been posted in the channel indicating that the incident has been ended.

Pass

39

A normal user can end an incident started by the same user from RHS

1. Start a new incident.

2. In the channel header, click on the Incident Response icon.

3. Verify that the incident started in step 1 is listed in the RHS section.

4. Click on the incident to go into the details.

5. Click on the End Incident button at the bottom of the RHS.

6. Verify that the incident has been ended

   1. In the incident RHS, verify the incident is deleted.

7. Verify that the message has been posted in the channel indicating that the incident has been ended.

Fail

The End Incident button on the RHS only appears when I click on the Channel in the details.

40

A normal user can end an incident started by another user

1. As a different user, start an incident (using any method).

2. Log out and log in as normal user.

3. Open Incident Response RHS.

4. Verify that the incident started in step 1 is not present in the list.

5. Log out and log in as the different user in step 1.

6. In the incident channel created in step 1, add normal user.

7. Log out and log in as normal user.

8. Verify that the incident channel from step 1 appears on the LHS.

9. Open the Incident Response RHS.

10. Find the incident from step 1.

11. Verify there is End Incident button.

12. Click on the End Incident button.

13. Verify that the incident is ended.

Pass

41

The incident list reloads on RHS when an incident is ended while viewing its details

1. As a normal user start an incident using any method.

2. Open the incident RHS.

3. Click on the incident started in step 1 to open it’s details.

4. With the RHS incident details open, in the center channel, end the incident with the /incident end command.

5. Verify that the RHS reloads the list of active incidents.

6. Verify that the incident ended in step 4 is not listed in RHS.

Fail

MM-23858: Incident ended with slash command while viewing details in RHS does not close out of details viewClosed

42

Trying to end an incident that has already been ended shows error message

1. Ensure that there is an active incident that the normal user started (or has rights to end).

2. End the incident using any method (slash command or RHS).

3. Go to the incident channel for the incident ended in step 2.

4. Issue the /incident end slash command in the channel.

5. Verify that there is an error message that the incident was not found.

Fail

MM-23859: A closed incident can be closed again from the RHSClosed

43

Issuing an /incident end command from a non-incident channel shows error message

1. As a normal user, start an incident using any method.

2. Go to a channel that is not an incident-response channel.

3. Issue the /incident end slash command in the channel.

4. Verify that there is an error message that the incident was not found.

Pass

44

Clicking the Incident Response plugin icon on header toggles the RHS open and close

1. Click the Incident Response icon in the channel header.

2. Verify that the Incident Response RHS opens up.

3. Click the Incident Response icon in the channel header again.

4. Verify that the Incident Response RHS closes.

Pass

45

Clicking on the Incident Response RHS back button takes user back to incident list

1. Have some incidents already started.

2. Open Incident Response RHS.

3. Click on an active incident.

4. When the incident details open up, click on the back button on the RHS.

5. Verify that the list of active incidents is loaded in RHS.

Pass

46

An incident channel with an existing channel name cannot be created

1. Start a new incident using any method.

2. Name the incident channel Incident 1

3. Start another incident.

4. In the Incidents Details modal input channel name as Incident 1

5. Verify that a new incident is not created.

   1. The new incident is not listed in the RHS.

   2. The new incident channel is not present in LHS.

6. Verify that there is an error in the modal indicating that the channel name has already been taken and use a different name.

7. Input channel name as Incident 2

8. Verify that the new incident is started and listed in RHS as Incident 2.

9. Verify the new incident channel Incident 2 appears on LHS.

Resolved

MM-23672: Creating an incident with the same name as a previous incident fails silentlyClosed

47

Incident Response RHS shows incidents for respective teams

1. As normal user, in team X create a few incidents.

2. In team X, open RHS by clicking on the Incident Response icon in the header.

3. Verify that all incidents created in step 1 are listed with normal user as the commander.

4. As a different user, in team Y create a few incidents.

5. Verify that incidents are created.

6. Switch to team X (as different user in step 4).

7. Open the Incident Response RHS.

8. Verify that only incidents created in step 1 are listed.

9. Log out as the different user and log in as normal user.

10. Go to team Y.

11. Open the Incident Response RHS.

12. Verify that incidents created in step 4 are listed.

13. Switch to team X.

14. Verify that incident created in step 1 are listed.

Pass

Duplicate of test # 52

Test ID

Test Case

Test Procedure

Result

Notes

48

An incident appears on the RHS when started without having to refresh the list

1. Open the incidents RHS by clicking on the Incidents Response icon on the channel header.

2. With the incidents RHS open, on the center channel, start a new incident with the slash command.

3. Verify that the newly started incident in step b appears in the RHS incident list as soon as it is started by the slash command.

Pass

49

An incident disappears from the RHS when ended without having to refresh the list

1. Go to a channel of an active incident.

2. Open the incident RHS by clicking on the Incident Response plugin icon.

3. The incident must be listed on the RHS.

4. In the channel, post the command `/incident end`.

5. Verify that the incident disappears from the RHS without refreshing.

Pass

50

A normal user can see all the started incidents in RHS

1. Start a new incident (using any method) as admin.

2. Start a new incident as a different normal user.

3. As normal user, In the channel header, click on the Incident Response icon to open the RHS.

4. Verify that the incident started in steps a and b, along with any previously started incidents are listed in the RHS section.

Pass

51

A normal user can view the details of incidents in RHS from anywhere in the team.

1. Have a few incidents started as different users (similar to the above test cases).

2. Open the incident RHS.

3. Click on an incident started by a different user.

4. Verify that the details of the incident are displayed.

Pass

52

A normal user cannot view the incidents in another team

1. As a normal user, create a new incident in Team A.

2. Switch to team B.

3. Verify that the incident RHS does not have the incident created in step 1.

4. As a different user create a new incident in Team B.

5. Switch to team A.

6. Verify that the incident RHS does not have the incident created in step 4.

Pass

See test # 47

53

Incident details in RHS lists the incident channels associated with it

1. Ensure that there are some incidents have been started.

2. As a normal user, open the Incident Response RHS

   1.  Click on the Incident Response icon in the header

3. Click on an incident to expand the incident’s details.

4. Verify that the channels associated with the incident are listed under details.

5. Verify that the commanders are commander is listed under details.

Pass

54

A normal user can access an incident channel started by the same user

1. As a normal user, start a new incident (using any method).

2. Open the incident RHS (by clicking on the icon in header).

3. Find the incident started in step a.

4. Open the incident details by clicking on the incident.

5. Click on the listed channels associated to the incident.

6. Verify that the user is taken to the correct channel.

Pass

55

A normal user can access an incident channel started by a different user only when added to the incident channel

1. As admin, start a new incident (using any method).

2. As the normal user, open the incident RHS (by clicking on the icon in header).

3. Find the incident started in step a.

4. Open the incident details by clicking on the incident.

5. Click on the listed channel associated to the incident.

6. Verify that the user cannot see the associated incident channel.

7. Log in as admin.

8. Add user in step 2 to the incident channel started in step 1.

9. Log back in as normal user.

10. From the RHS, find the incident started in step 1.

11. Click on the listed channel associated to the incident.

12. Verify that the normal user is successfully taken to the incident channel.

Pass

56

A normal user can’t access an incident channel once removed

1. As admin, start a new incident (using any method).

2. Add normal user to the incident channel created.

3. Log in as normal user.

4. Verify that the normal user can access the incident channel.

5. Log in as admin.

6. Remove the normal user in step 3 from the incident channel.

7. Log back in as normal user.

8. Verify that the user cannot access the incident channel anymore.

9. As admin repeat steps 1-4 in browser A.

10. In a separate browser B log in as normal user.

11. Verify user can access the incident channel.

12. Copy the link address of the incident channel.

13. Switch to a different channel.

14. As admin in browser A, remove user from incident channel.

15. As normal user in browser B, without logging out, verify that the incident channel is no longer available in LHS.

16. In a new tab in Browser B, paste the channel address copied in step 12.

17. Verify that the user cannot access the incident channel.

Pass

Incident channel is still present in details view.

MM-23861: Incident channel is still visible in details view when user is kicked from the channelClosed

57

A normal user can’t end an incident after being removed from the incident channel

1. As admin in browser A, start a new incident using any method.

2. In the incident channel for the incident started in step 1, add the normal user.

3. In browser B, log in as normal user.

4. Verify that the incident channel appears on the LHS.

5. Open the Incident Response RHS.

6. Find the incident from step 1 and click for details.

7. Verify that the End incident button exists.

8. In browser A (as admin), remove the normal user from the incident channel in step 2.

9. In browser B, as normal user, verify that the incident channel no longer present in LHS.

10. Open the Incident Response RHS.

11. Find the incident from step 1 and click for details.

12. Verify that the End incident button does not exist.

Pass

58

The incident list on the RHS shows the commander without opening the incident details.

1. As different users, start a few incidents.

2. Log in as normal user.

3. Open the incident RHS from the Incident icon on the header.

4. Verify that each incident in the list has the user who started the incident shown as the commanders of the respective incidents.

Pass

Test ID

Test Case

Test Procedure

Result

Notes

59

A message is posted to the channel when the incident starts

1. As a normal user, start an incident using any method.

2. Once the channel is created, verify that a message is posted in the incident channel with the name of the commander and the timestamp of when the incident was started.

Fail

The incident channel does not have a “Commander” of the incident mentioned explicitly. This is different from what is in the user stories: “As the Incident plugin, a message is posted to the channel when the incident starts and ends so there is a timestamp record of who made the change.”

MM-23862: The incident channel does not mention the commander explicitlyClosed

60

An message is posted in the channel where the incident was started

1. In channel A, start an incident using the /incident start command.

2. Verify that in channel A (not the incident channel created after step 1), an ephemeral message is posted including the name of the commander and the incident creation timestamp.

3. In channel A, start an incident using the post action menu.

4. Verify that in channel A, an ephemeral message is posted including the name of the commander and the incident creation timestamp.

5. In channel B, create an incident from the RHS.

6. Verify that in channel B, an ephemeral message is posted including the name of the commander and the incident creation timestamp.

Pass

61

A message is posted to the channel when the incident ends using slash command

1. Go to an incident channel.

2. End the incident with the slash command.

3. Verify that a message is posted in the channel with the name of the user who ended the incident and the timestamp of when the incident was ended.

Pass

62

A message is posted to the channel when the incident ends using RHS

1. As normal user, start an incident using any method.

2. Open up the Incident RHS.

3. Find the incident from step 1.

4. End the incident by clicking on End incident button.

5. Verify that in the incident channel, a message is posted that the incident has been ended including the name of the commander and the incident end timestamp.

Pass

63

An “invalid channel name” error is displayed for channel names that are not valid

1. Start a channel using any method.

2. In the "Incident Details" modal, enter a channel name longer than allowed number of characters.

3. Click `Start Incident`.

4. Observe the error message in the center channel.

5. Verify that an error is displayed indicating an invalid channel name.

Fail

MM-23864: Giving a long channel name for an incident gives wrong error message: "... channel already exists... "Closed

64

Channel name in Incident Details modal cannot be empty

1. Start a channel using any method.

2. In the "Incident Details" modal, leave the ‘Channel Name’ field empty.

3. Click `Start Incident`.

4. Verify that the incident cannot be started and an error for required field appears in the modal.

Pass