Test Plan - Incident Response v0.2.0

0.0

Initial version

04/14

0.1

• Updated to reflect response from the sync-up on 04/16

• Removed tests for v0.3.0

04/16

0.2

Tests updated to reflect realtime checklist item editing.

04/17

1.0

• Tests updated

• Tests run

04/17

Test ID

Test Case

Test Procedure

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

Plugin is Enabled by default

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

Pass

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.2.0 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

Pass

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

Pass

UI funkyness - MM-24338: Post action menu shows a blank section at the end after corresponding plugin is disabledClosed

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

Pass

Test ID

Test Case

Test Procedure

Result

Notes

9

User is taken to incident channel after starting an incident

1. As a normal user, start an incident.

2. Once the incident is started, verify that the user is taken to the new incident channel.

3. If the same user has multiple sessions open, it would only change to the new incident channel in the session where the incident was started.

Pass

Did not always get navigated to incident channel when starting from post action menu

10

In RN mobile app, user is notified with an ephemeral message with link to the incident channel

1. Start an incident in RN mobile app.

2. Verify that the user remains in the same channel where the incident was started.

3. Verify that an ephemeral message is shown with the link to the incident channel.

Pass

11

A newly started incident does not have any checklist item

1. As a normal user, start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step1 and click.

4. Verify that the incident does not have any checklist.

5. Verify that the “Checklist” header in RHS has an “edit” button beside it.

6. Verify that there is an “Add new checklist item”.

Pass

12

Commander can create a checklist for an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name and hit enter.

7. Verify that the checklist item is created.

8. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

9. Go back to incident list by clicking on “<“ button.

10. Click on the incident again to open the details view.

11. Verify the checklist item added remains in the list.

12. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

Pass

13

Commander cannot create a checklist for an incident from a channel different from the incident channel

1. Find an active incident or start new.

2. Click to view details.

3. Verify the checklist can be edited and new checklist items can be added.

4. Navigate to a different channel from the incident channel.

5. Verify that the checklist is now converted to read-only

   1. Verify that the “edit” button no longer exists.

   2. Verify that the “Add new checklist item” button no longer exists.

   3. Verify that the “End Incident” button is faded/inactive.

   4. Verify that there is a message to navigate to the incident channel to make changes.

6. Click on the incident channel link.

7. Verify that the checklist is not read-only.

Pass

14

Checklist “edit” button does not work when checklist is empty

1. Start a new incident.

2. Do not add any item to the checklist.

3. Click on “edit”.

4. Verify a message alerts users that empty checklist cannot be edited.

Pass

15

Checklist item creation can be cancelled with the Esc key

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist item” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name.

7. Press escape.

8. Verify that the checklist item is not created.

Pass

16

Commander can create checklist item from within the channel incident view

1. Start a new incident (the commander should be taken to the incident channel).

2. On RHS click on the incident to view details.

3. Click “Add new checklist item” button.

4. Enter a checklist item and hit enter.

5. Verify the item appears on the list.

6. Go to the incident list by clicking “<“.

7. Click on the incident again to view details.

8. Verify that the item added in step 4 is still there.

Pass

17

Checklist item can be added in edit view.

1. Start a new incident.

2. Click on it on RHS to view details.

3. While the list is empty, click the “edit” button beside the Checklist header.

4. Click on the “Add new checklist item” button.

5. Enter an item.

6. Click on “done” button next to Checklist header.

7. Verify item added now has a checkbox next to it.

8. Go back to incident list view by clicking on the “<“ button.

9. Click the incident again to view details.

   1. Verify the item entered in step 5 still exists.

Pass

18

Commander can remove checklist item from an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Create a new checklist with a few items.

5. Click the “edit” button next to the Checklist header

6. Click the “X” button to remove the checklist item.

7. Verify that the checklist item is removed from the incident.

8. Click on “done”.

9. Verify that the removed items are not in the list.

Pass

19

User only has read-only access to an incident checklist if user is not an incident channel member

1. As user A, start an incident (don’t add user B to incident channel).

2. Create a checklist by clicking the “Add new checklist” button.

3. As user B, open the Incident Response RHS.

4. Click on the incident started in step 1.

5. Verify that the checklist appears as setup in step 2.

6. Verify that there is no “edit”, “Add new checklist item” and “End Incident” buttons in details view.

Pass

20

User cannot add checklist item to a new incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Do not add any items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify there is no “Add new checklist item” button for user B.

Pass

21

User cannot add items to a checklist, with existing items, for an incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that the checklist setup by user A in step 2 is visible.

5. Verify that there is no button to add new checklist item.

Pass

22

User cannot remove checklist item from an incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that checklist items setup in step 2 does not have the “edit” button beside the Checklist header.

Pass

23

Incident checklist is no longer read-only once non-commander is added to incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items.

3. As user B, click on the incident started in step 1 to view details.

4. User B only has read-only access to the checklist.

5. Have user A added user B to the incident channel.

6. As user B, verify that the incident channel appears in the details.

   1. Verify that “End Incident” (although deactivated) appears in the details.

Pass

24

User can add checklist item to a new incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Do not add any item to the incident checklist.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify there is “Add new checklist item” button for user B.

6. Click on the “Add new checklist item”.

7. Verify that an input box asks for the checklist item name.

8. Input a checklist item name and hit enter.

9. Verify that the checklist item is created.

10. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

Pass

25

User can add items to a checklist, with existing items, for an incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist and add a few items to it.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 2 is visible.

6. Verify there is “Add new checklist item” button at the end of the checklist for user B.

7. Click on the “Add new checklist item”.

8. Verify that an input box asks for the checklist item name.

9. Input a checklist item name and hit enter.

10. Verify that the checklist item is created.

11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

Pass

26

User can edit the checklist item for an incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist

3. Add an item “Test 123” to the checklist.

4. Add user B to the incident channel.

5. As user B, click on the incident started by user A in step 1 to open the incident details.

6. Verify there is an “edit” button beside the Checklist header.

7. Click on the “edit” button.

8. Click checklist item “Test 123” and edit it to “Test ABC”

9. Click “done” beside the Checklist header.

10. Click the “<“ button beside the incident name to go back to the incident list.

11. Click on the incident again to bring up the details.

12. Verify that the changes made in step 8 persisted.

Pass

27

User can move items around in the checklist once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist

3. Add 3 items “Test”, “123” and “ABC” to the checklist in that order.

4. Add user B to the incident channel.

5. As user B, click on the incident started by user A in step 1 to open the incident details.

6. Verify there is an “edit” button beside the Checklist header.

7. Click on the “edit” button.

8. Click on the hamburger menu icon beside “123” and drag it to the top of the list.

9. Verify that list now appears in the order of “123”, “Test” and “ABC”.

10. Click on the hamburger menu icon beside “ABC” and drag it to the middle of the list.

11. Verify that the list now appears in the order of “123”, “ABC” and “Test”.

12. Click “done”

13. Verify that the list remains in the same order as mentioned in step 11.

14. Click the “<“ button beside the incident name to go back to the incident list.

15. Click on the incident again to bring up the details.

16. Verify that the changes made in step 11 persisted.

Pass

28

User can remove item added by different user once added to the incident channel.

1. As user A, start a new incident.

2. Create a checklist and add a few items to the checklist.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 2 is visible.

6. Verify that the checklist items have an “edit” button beside the Checklist header.

7. Click on “edit”.

8. Click on the “X” button beside the checklist item to remove the items.

9. Verify that the items are removed from the checklist.

Pass

29

Checklist item added reflects accurately for user not added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Do not add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

Pass

30

Checklist item edited reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, edit an item in the checklist.

2. As user B viewing the RHS, verify that the checklist is updated accurately to reflect the edit by user A.

Pass

31

Checklist item rearranged reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, rearrange the order of the checklist items.

2. As user B viewing the RHS, verify that the checklist is rearranged accurately to reflect the rearrangement by user A.

Pass

32

Checklist item removed reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, remove an item from the checklist.

2. As user B viewing the RHS, verify that the item is updated accurately to reflect the removed item from the checklist by user A.

Pass

33

Checklist changes reflect accurately for a different user added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

Pass

34

Checklist can be added by 2 users at the same time when they are both incident channel members

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. Have both users click “edit”.

8. Have both users enter item titles simultaneously before pressing enter.

9. Have both users press enter almost around the same time.

10. Verify that the checklist is updated accurately for each change made in step 7.

Pass

35

Checklist can be edited by 2 users at the same time when they are both incident channel members

Continue from the above test.

1. As any user add the following items to the checklist in the same order: “123”, “ABC”, “Test”.

2. Have both users go in the edit mode for the checklist.

3. Have both users click “123” to start editing.

4. As user A, edit it to “123 A”.

5. Verify that the change is reflected accurately in real time to user B.

6. As user A, edit the same item to “B XYZ”.

7. Verify that the change is reflected accurately in real time to user A.

8. Save the changes made by user B.

9. As user B, drag the “Test” item below “B XYZ”.

10. As user A, drag ABC to the top of the list.

11. Have both users go back to the incident list by clicking the “<“ button.

12. Have both users go to the incident’s detail view.

13. Verify the the checklist appears in the following order: “ABC“, “B XYZ“, “Test”

Pass

Test ID

Test Case

Test Procedure

Result

Notes

36

Commander can check off items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Verify that the commander can check items off.

Pass

37

Commander can uncheck items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Check off the items.

4. Verify that the commander can uncheck items.

Pass

38

Non-commander who is not added to incident channel cannot check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Log in as user B.

4. Verify that user B cannot check off items in step 2.

Pass

39

Non-commander who is not added to incident channel cannot uncheck items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check off a few items in the list.

4. Log in as user B.

5. Verify that user B cannot uncheck items in step 3.

Pass

40

Non-commander user, who is added to the incident channel, can check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check items off.

Pass

41

Non-commander user, who is added to the incident channel, can uncheck items

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check off items added in step 3.

5. Verify that user B can uncheck items checked off in step 4.

Pass

42

Non-commander user, who is added to the incident channel, can uncheck items checked off by the commander

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check items on the list.

4. Add user B to the incident channel.

5. Verify that user B can uncheck items that were checked off in step 3.

Pass

43

Ending an incident shows a confirmation modal

1. Start an incident.

2. Open the RHS and find the incident just started.

3. Click to view details.

4. Verify there is an “End Incident” button at the bottom.

5. Click on the button.

6. Verify a confirmation modal pops up asking whether to end the incident.

7. Go to the incident channel.

8. In the post input box type /incident end

9. Verify a confirmation modal pops up asking whether to end the incident.

Pass

44

End incident confirmation modal can be dismissed by clicking Cancel

1. Click on an incident’s End Incident button.

2. Once the confirmation modal pops up, click on Cancel.

3. Verify that the confirmation modal gets cancelled.

4. Verify that the incident is not ended.

Pass

45

End incident confirmation modal can dismissed by pressing Esc

1. Click on an incident’s End Incident button.

2. Once the confirmation modal pops up, press Esc.

3. Verify that the confirmation modal gets cancelled.

4. Verify that the incident is not ended.

Pass

46

Commander can end incident

Continue from the above test:

1. Click to confirm ending the incident.

2. Verify that the incident is ended.

(Incident end verifications in Test Plan - Incident Response v0.1.0 )

Pass

47

Non-commander cannot can end incident from RHS

1. As user A start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Click the incident to view details.

4. Verify that there is no “End Incident” button.

5. As user A, add user B to the incident channel.

6. As user B, navigate to the incident channel.

7. In the RHS, find the incident and click to view details.

8. Verify there is no “End Incident” button.

9. Click on the “End Incident” button and confirm to end.

10. Verify that the incident is ended.

Pass

48

Non-commander cannot can end incident with /incident command.

1. As user A, start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Navigate to any channel.

4. In the post input box, type `/incident'.

5. Verify that there is no /incidentoption present in the slash commands.

6. As user A, add user B to the incident channel.

7. As user B, navigate to the incident channel.

8. In the post input box, type `/incident'.

9. Verify that there is no /incidentoption present in the slash commands.

10. In incident channel, issue /incident endcommand.

11. When the end-incident-confirmation pops up, confirm to end.

12. Verify that the incident is ended.

Pass

49

Incident cannot be ended from outside the incident channel

1. Create an incident.

2. Ensure the user is navigated to the incident channel.

3. Open RHS and find the incident.

4. Click on it to view the details.

5. Verify the “End Incident” button exists in RHS.

6. Navigate to a different channel than the incident channel.

7. Verify that the “End Incident” changes to an inactive button, notifying the user that the incident can only be ended from within the incident channel.

Pass

Test ID

Test Case

Test Procedure

Result

Notes

50

A bot message is posted in incident channel when an item is added to an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. In the incident channel, verify that a bot message similar to the following is posted:

@username added checklist item " to [checklist item title]"

Pass

51

“Item added to incident checklist” post in incident channel shows the username of user who added the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who added the checklist item.

Pass

52

A bot message is posted in incident channel when an item in checklist is checked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username checked off checklist item "[checklist item title]"

Pass

53

“Item checked” post in incident channel shows the username of user who checked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who checked off the checklist item.

Pass

54

Checking an item shows the timestamp to the right of the checklist item of when the item was checked off

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was checked off is displayed beside the checklist item title.

Fail

For v0.3.0
MM-24210: Display time last checked beside checklists.Closed

55

A bot message is posted in incident channel when an item in checklist is unchecked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. Uncheck the item.

8. In the incident channel, verify that a bot message similar to the following is posted:

@username unchecked item "[checklist item title]"

Pass

56

“Item checked” post in incident channel shows the username of user who unchecked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who unchecked the checklist item.

Pass

57

A bot message is posted in incident channel when an item is removed from an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Remove the checklist item.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username removed checklist item "[checklist item title]"

Pass

58

“Item removed from incident checklist” post in incident channel shows the username of who removed the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who removed the checklist item.

Pass

59

“Item removed from incident checklist” post in incident channel shows the timestamp of when the item was removed

Continue from the above test:

1. Verify that the bot message from the above test shows the timestamp of when the item was removed from the checklist.

Pass

60

Starting a next incident will start with empty checklist

1. Start an incident.

2. Create a checklist for the incident.

3. Check a few items and leave some unchecked.

4. Start a new incident.

5. View the incident details of the new incident.

6. Verify that the incident does not have any checklist.

Pass

Test Case/Bug Title

Test Procedure

Note

Channel links refresh current channel if incident channel is archived before ending incident when "view archived channels" is false

1. From System Console > Users and Teams set "Allow users to view archived channels" to false (is false by default).

2. Start an incident.

3. Archive the incident channel.

4. Open incident RHS and find the incident from step 2.

5. Click on incident to open details.

6. Click on the incident channel (the one that was archived in step 3).

Observed: The current channel page refreshes.

Expected: User is notified that the "view archived channels" should be enabled to navigate to the channel.

MM-24339: Channel links refresh current channel if incident channel is archived before ending incident when "view archived channels" is falseClosed

"Internal error" message when starting an incident from system message post action menu

1. Find a system message

   1. For e.g., start a new incident.

   2. b. In the newly created incident channel, find the first post from "System".

2. Go to the post action menu and click.

3. Click on the "Start incident" option.

4. Enter a channel name.

5. Click "Start Incident".

MM-24342: "Internal error" message when starting an incident from system message post action menuClosed