Test Plan - Incident Response v0.2.0

0.0

Initial version

04/14

0.1

• Updated to reflect response from the sync-up on 04/16

• Removed tests for v0.3.0

04/16

0.2

Tests updated to reflect realtime checklist item editing.

04/17

1.0

0/17

Test ID

Test Case

Test Procedure

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

Pass

Plugin is Enabled by default

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.2.0 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

Pass

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

Pass

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

Pass

UI funkyness

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

Pass

Test ID

Test Case

Test Procedure

Result

Notes

9

User is taken to incident channel after starting an incident

1. As a normal user, start an incident.

2. Once the incident is started, verify that the user is taken to the new incident channel.

3. If the same user has multiple sessions open, it would only change to the new incident channel in the session where the incident was started.

10

In RN mobile app, user is notified with an ephemeral message with link to the incident channel

1. Start an incident in RN mobile app.

2. Verify that the user remains in the same channel where the incident was started.

3. Verify that an ephemeral message is shown with the link to the incident channel.

11

A newly started incident does not have any checklist item

1. As a normal user, start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step1 and click.

4. Verify that the incident does not have any checklist.

5. Verify that the “Checklist” header in RHS has an “edit” button beside it.

6. Verify that there is an “Add new checklist item”.

12

Commander can create a checklist for an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name and hit enter.

7. Verify that the checklist item is created.

8. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

9. Click on the “done” button by the Checklist header.

10. Verify the checklist item added remains in the list.

11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

13

Commander can create a checklist for an incident from a channel different from the incident channel

1. Start a new incident.

2. Navigate to a different channel from the incident channel.

3. Open up the Incident RHS.

4. Find the incident started in step 1 and click on it to open the details.

5. Click the “Add new checklist” button.

6. Verify that an input box asks for the checklist item name.

7. Input a checklist item name and hit enter.

8. Verify that the checklist item is created.

9. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

14

Checklist item creation can be cancelled with the Esc key

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist item” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name.

7. Press escape.

8. Verify that the checklist item is not created.

15

Commander can remove checklist item from an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Create a new checklist with a few items.

5. Click the “edit” button next to the Checklist header

6. Click the “X” button to remove the checklist item.

7. Verify that the checklist item is removed from the incident.

16

Checklist item can be removed from a different channel than the incident channel

1. Start a new incident.

2. Open up the Incident RHS.

3. Create a checklist and add a few items to it.

4. Navigate to a different channel from the incident channel.

5. Find the incident started in step 1 and click on it to open the details.

6. Click the “edit” button next to the Checklist header

7. Click the “X” button to remove the checklist item.

8. Verify that the checklist item is removed from the incident.

17

User can see checklist in RHS for active incidents

1. For this step, as user A, ensure there are a few incidents already started.

2. For each incident add checklists.

3. As user B, open the Incident Response RHS.

4. Click on the incidents one at a time.

5. Verify that each incident has checklists as setup in step 2.

18

User can access read-only incident checklist even if user is not an incident channel member

1. As user A, start an incident (don’t add user B to incident channel).

2. Create a checklist by clicking the “Add new checklist” button.

3. As user B, open the Incident Response RHS.

4. Click on the incident started in step 1.

5. Verify that the checklist appears as setup in step 2.

19

User cannot add checklist item to a new incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Do not add any items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify there is no “Add new checklist item” button for user B.

20

User cannot add items to a checklist, with existing items, for an incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that the checklist setup by user A in step 2 is visible.

5. Verify that there is no button to add new checklist item.

21

User cannot remove checklist item from an incident started by a different user if user is not added to the incident channel

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that checklist items setup in step 2 does not have the “edit” button beside the Checklist header.

22

User can add checklist item to a new incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Do not add any item to the incident checklist.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify there is “Add new checklist item” button for user B.

6. Click on the “Add new checklist item”.

7. Verify that an input box asks for the checklist item name.

8. Input a checklist item name and hit enter.

9. Verify that the checklist item is created.

10. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

23

User can add items to a checklist, with existing items, for an incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist and add a few items to it.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 2 is visible.

6. Verify there is “Add new checklist item” button at the end of the checklist for user B.

7. Click on the “Add new checklist item”.

8. Verify that an input box asks for the checklist item name.

9. Input a checklist item name and hit enter.

10. Verify that the checklist item is created.

11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

24

User can edit the checklist item for an incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist

3. Add an item “Test 123” to the checklist.

4. Add user B to the incident channel.

5. As user B, click on the incident started by user A in step 1 to open the incident details.

6. Verify there is an “edit” button beside the Checklist header.

7. Click on the “edit” button.

8. Click checklist item “Test 123” and edit it to “Test ABC”

9. Click “done” beside the Checklist header.

10. Click the “<“ button beside the incident name to go back to the incident list.

11. Click on the incident again to bring up the details.

12. Verify that the changes made in step 8 persisted.

25

User can move items around in the checklist once added to the incident channel

1. As user A, start a new incident.

2. Create a checklist

3. Add 3 items “Test”, “123” and “ABC” to the checklist in that order.

4. Add user B to the incident channel.

5. As user B, click on the incident started by user A in step 1 to open the incident details.

6. Verify there is an “edit” button beside the Checklist header.

7. Click on the “edit” button.

8. Click on the hamburger menu icon beside “123” and drag it to the top of the list.

9. Verify that list now appears in the order of “123”, “Test” and “ABC”.

10. Click on the hamburger menu icon beside “ABC” and drag it to the middle of the list.

11. Verify that the list now appears in the order of “123”, “ABC” and “Test”.

12. Click “done”

13. Verify that the list remains in the same order as mentioned in step 11.

14. Click the “<“ button beside the incident name to go back to the incident list.

15. Click on the incident again to bring up the details.

16. Verify that the changes made in step 11 persisted.

26

User can remove item added by different user once added to the incident channel.

1. As user A, start a new incident.

2. Create a checklist and add a few items to the checklist.

3. Add user B to the incident channel.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 2 is visible.

6. Verify that the checklist items have an “edit” button beside the Checklist header.

7. Click on “edit”.

8. Click on the “X” button beside the checklist item to remove the items.

9. Verify that the items are removed from the checklist.

27

Checklist item added reflects accurately for user not added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Do not add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

28

Checklist item edited reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, edit an item in the checklist.

2. As user B viewing the RHS, verify that the checklist is updated accurately to reflect the edit by user A.

29

Checklist item rearranged reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, rearrange the order of the checklist items.

2. As user B viewing the RHS, verify that the checklist is rearranged accurately to reflect the rearrangement by user A.

30

Checklist item removed reflects accurately for user not added to incident channel

Continue from the above test.

1. As user A, remove an item from the checklist.

2. As user B viewing the RHS, verify that the item is updated accurately to reflect the removed item from the checklist by user A.

31

Checklist changes reflect accurately for a different user added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

32

Checklist can be added by 2 users at the same time when they are both incident channel members

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. Have both users add items to the checklist.

8. Verify that the checklist is updated accurately for each change made in step 7.

33

Checklist can be edited by 2 users at the same time when they are both incident channel members

Continue from the above test.

1. As any user add the following items to the checklist in the same order: “123”, “ABC”, “Test”.

2. Have both users go in the edit mode for the checklist.

3. Have both users click “123” to start editing.

4. As user A, edit it to “123 A”.

5. Verify that the change is reflected accurately in real time to user B.

6. As user A, edit the same item to “B XYZ”.

7. Verify that the change is reflected accurately in real time to user A.

8. Save the changes made by user B.

9. As user B, drag the “Test” item below “B XYZ”.

10. As user A, drag ABC to the top of the list.

11. Have both users go back to the incident list by clicking the “<“ button.

12. Have both users go to the incident’s detail view.

13. Verify the the checklist appears in the following order: “ABC“, “B XYZ“, “Test”

Test ID

Test Case

Test Procedure

Result

Notes

34

Commander can check off items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Verify that the commander can check items off.

35

Commander can uncheck items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Check off the items.

4. Verify that the commander can uncheck items.

36

Non-commander who is not added to incident channel cannot check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Log in as user B.

4. Verify that user B cannot check off items in step 2.

37

Non-commander who is not added to incident channel cannot uncheck items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check off a few items in the list.

4. Log in as user B.

5. Verify that user B cannot uncheck items in step 3.

38

Non-commander user, who is added to the incident channel, can check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check items off.

39

Non-commander user, who is added to the incident channel, can uncheck items

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check off items added in step 3.

5. Verify that user B can uncheck items checked off in step 4.

40

Non-commander user, who is added to the incident channel, can uncheck items checked off by the commander

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check items on the list.

4. Add user B to the incident channel.

5. Verify that user B can uncheck items that were checked off in step 3.

41

Ending an incident shows a confirmation modal

1. Start an incident.

2. Open the RHS and find the incident just started.

3. Click to view details.

4. Verify there is an “End Incident” button at the bottom.

5. Click on the button.

6. Verify a confirmation modal pops up asking whether to end the incident.

7. Go to the incident channel.

8. In the post input box type /incident end

9. Verify a confirmation modal pops up asking whether to end the incident.

42

Commander can end incident

Continue from the above test:

1. Click to confirm ending the incident.

2. Verify that the incident is ended.

(Incident end verifications in Test Plan - Incident Response v0.1.0 )

43

Non-commander cannot can end incident from RHS

1. As user A start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Click the incident to view details.

4. Verify that there is no “End Incident” button.

5. As user A, add user B to the incident channel.

6. As user B, navigate to the incident channel.

7. In the RHS, find the incident and click to view details.

8. Verify there is no “End Incident” button.

9. Click on the “End Incident” button and confirm to end.

10. Verify that the incident is ended.

Fix from 0.1.0

44

Non-commander cannot can end incident with /incident command.

1. As user A, start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Navigate to any channel.

4. In the post input box, type `/incident'.

5. Verify that there is no /incidentoption present in the slash commands.

6. As user A, add user B to the incident channel.

7. As user B, navigate to the incident channel.

8. In the post input box, type `/incident'.

9. Verify that there is no /incidentoption present in the slash commands.

10. In incident channel, issue /incident endcommand.

11. When the end-incident-confirmation pops up, confirm to end.

12. Verify that the incident is ended.

45

Incident cannot be ended from outside the incident channel

1. Create an incident.

2. Ensure the user is navigated to the incident channel.

3. Open RHS and find the incident.

4. Click on it to view the details.

5. Verify the “End Incident” button exists in RHS.

6. Navigate to a different channel than the incident channel.

7. Verify that the “End Incident” changes to an inactive button, notifying the user that the incident can only be ended from within the incident channel.

Test ID

Test Case

Test Procedure

Result

Notes

46

A bot message is posted in incident channel when an item is added to an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. In the incident channel, verify that a bot message similar to the following is posted:

@username added checklist item "[checklist item title]"

47

“Item added to incident checklist” post in incident channel shows the username of user who added the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who added the checklist item.

48

A bot message is posted in incident channel when an item in checklist is checked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username checked off checklist item "[checklist item title]"

49

“Item checked” post in incident channel shows the username of user who checked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who checked off the checklist item.

50

Checking an item shows the timestamp to the right of the checklist item of when the item was checked off

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was checked off is displayed beside the checklist item title.

51

A bot message is posted in incident channel when an item in checklist is unchecked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. Uncheck the item.

8. In the incident channel, verify that a bot message similar to the following is posted:

@username unchecked item "[checklist item title]"

52

“Item checked” post in incident channel shows the username of user who unchecked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who unchecked the checklist item.

53

A bot message is posted in incident channel when an item is removed from an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Remove the checklist item.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username removed checklist item "[checklist item title]"

54

“Item removed from incident checklist” post in incident channel shows the username of who removed the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who removed the checklist item.

55

“Item removed from incident checklist” post in incident channel shows the timestamp of when the item was removed

Continue from the above test:

1. Verify that the bot message from the above test shows the timestamp of when the item was removed from the checklist.

56

Starting a next incident will start with empty checklist

1. Start an incident.

2. Create a checklist for the incident.

3. Check a few items and leave some unchecked.

4. Start a new incident.

5. View the incident details of the new incident.

6. Verify that the incident does not have any checklist.