Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Document version 0.2

Status: Test plan development in progressv0.2 done.

Document Version

Description

Date

0.0

Initial version

04/14

0.1

  • Updated to reflect response from the sync-up on 04/16

  • Removed tests for v0.3.0

04/16

0.2Update in progress

Tests updated to reflect realtime checklist item editing.

04/17

Summary

This document details the tests for Incident Response plugin v0.2.0. The tests are derived from the user stories in Incident Responseoverview.

...

Test

Test Area - Monitor

Test ID

Test Case

Test Procedure

Result

Notes

9

User is taken to incident channel after starting an incident

  1. As a normal user, start an incident.

  2. Once the incident is started, verify that the user is taken to the new incident channel.

  3. If the same user has multiple sessions open, it would only change to the new incident channel in the session where the incident was started.

10

In RN mobile app, user is notified with an ephemeral message with link to the incident channel

  1. Start an incident in RN mobile app.

  2. Verify that the user remains in the same channel where the incident was started.

  3. Verify that an ephemeral message is shown with the link to the incident channel.

11

A newly started incident does not have any checklist item

  1. As a normal user, start a new incident.

  2. Open up the Incident RHS.

  3. Find the incident started in step1 and click.

  4. Verify that the incident does not have any checklist.

  5. Verify that the “Checklist” header in RHS has an “edit” button beside it.

  6. Verify that there is an “Add new checklist item”.

12

Commander can create a checklist for an incident started by them

  1. Start a new incident.

  2. Open up the Incident RHS.

  3. Find the incident started in step 1 and click on it to open the details.

  4. Click the “Add new checklist” button.

  5. Verify that an input box asks for the checklist item name.

  6. Input a checklist item name and hit enter.

  7. Verify that the checklist item is created.

  8. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

  9. Click on the “done” button by the Checklist header.

  10. Verify the checklist item added remains in the list.

  11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

13

Commander can create a checklist for an incident from a channel different from the incident channel

  1. Start a new incident.

  2. Navigate to a different channel from the incident channel.

  3. Open up the Incident RHS.

  4. Find the incident started in step 1 and click on it to open the details.

  5. Click the “Add new checklist” button.

  6. Verify that an input box asks for the checklist item name.

  7. Input a checklist item name and hit enter.

  8. Verify that the checklist item is created.

  9. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

14

Checklist item creation can be cancelled with the Esc key

  1. Start a new incident.

  2. Open up the Incident RHS.

  3. Find the incident started in step 1 and click on it to open the details.

  4. Click the “Add new checklist item” button.

  5. Verify that an input box asks for the checklist item name.

  6. Input a checklist item name.

  7. Press escape.

  8. Verify that the checklist item is not created.

15

Commander can remove checklist item from an incident started by them

  1. Start a new incident.

  2. Open up the Incident RHS.

  3. Find the incident started in step 1 and click on it to open the details.

  4. Create a new checklist with a few items.

  5. Click the “edit” button next to the Checklist header

  6. Click the “X” button to remove the checklist item.

  7. Verify that the checklist item is removed from the incident.

16

Checklist item can be removed from a different channel than the incident channel

  1. Start a new incident.

  2. Open up the Incident RHS.

  3. Create a checklist and add a few items to it.

  4. Navigate to a different channel from the incident channel.

  5. Find the incident started in step 1 and click on it to open the details.

  6. Click the “edit” button next to the Checklist header

  7. Click the “X” button to remove the checklist item.

  8. Verify that the checklist item is removed from the incident.

17

User can see checklist in RHS for active incidents

  1. For this step, as user A, ensure there are a few incidents already started.

  2. For each incident add checklists.

  3. As user B, open the Incident Response RHS.

  4. Click on the incidents one at a time.

  5. Verify that each incident has checklists as setup in step 2.

18

User can access read-only incident checklist even if user is not an incident channel member

  1. As user A, start an incident (don’t add user B to incident channel).

  2. Create a checklist by clicking the “Add new checklist” button.

  3. As user B, open the Incident Response RHS.

  4. Click on the incident started in step 1.

  5. Verify that the checklist appears as setup in step 2.

19

User cannot add checklist item to a new incident started by a different user if user is not added to the incident channel

  1. As user A, start a new incident (don’t add user B to incident channel).

  2. Do not add any items to the checklist.

  3. As user B, click on the incident started by user A in step 1 to open the incident details.

  4. Verify there is no “Add new checklist item” button for user B.

20

User cannot add items to a checklist, with existing items, for an incident started by a different user if user is not added to the incident channel

  1. As user A, start a new incident (don’t add user B to incident channel).

  2. Create a checklist and add a few items to the checklist.

  3. As user B, click on the incident started by user A in step 1 to open the incident details.

  4. Verify that the checklist setup by user A in step 2 is visible.

  5. Verify that there is no button to add new checklist item.

21

User cannot remove checklist item from an incident started by a different user if user is not added to the incident channel

  1. As user A, start a new incident (don’t add user B to incident channel).

  2. Create a checklist and add a few items to the checklist.

  3. As user B, click on the incident started by user A in step 1 to open the incident details.

  4. Verify that checklist items setup in step 2 does not have the “edit” button beside the Checklist header.

22

User can add checklist item to a new incident started by a different user once added to the incident channel

  1. As user A, start a new incident.

  2. Do not add any item to the incident checklist.

  3. Add user B to the incident channel.

  4. As user B, click on the incident started by user A in step 1 to open the incident details.

  5. Verify there is “Add new checklist item” button for user B.

  6. Click on the “Add new checklist item”.

  7. Verify that an input box asks for the checklist item name.

  8. Input a checklist item name and hit enter.

  9. Verify that the checklist item is created.

  10. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

23

User can add items to a checklist, with existing items, for an incident started by a different user once added to the incident channel

  1. As user A, start a new incident.

  2. Create a checklist and add a few items to it.

  3. Add user B to the incident channel.

  4. As user B, click on the incident started by user A in step 1 to open the incident details.

  5. Verify that the checklist setup in step 2 is visible.

  6. Verify there is “Add new checklist item” button at the end of the checklist for user B.

  7. Click on the “Add new checklist item”.

  8. Verify that an input box asks for the checklist item name.

  9. Input a checklist item name and hit enter.

  10. Verify that the checklist item is created.

  11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

24

User can edit the checklist item for an incident started by a different user once added to the incident channel

  1. As user A, start a new incident.

  2. Create a checklist

  3. Add an item “Test 123” to the checklist.

  4. Add user B to the incident channel.

  5. As user B, click on the incident started by user A in step 1 to open the incident details.

  6. Verify there is an “edit” button beside the Checklist header.

  7. Click on the “edit” button.

  8. Click checklist item “Test 123” and edit it to “Test ABC”

  9. Click “done” beside the Checklist header.

  10. Click the “<“ button beside the incident name to go back to the incident list.

  11. Click on the incident again to bring up the details.

  12. Verify that the changes made in step 8 persisted.

25

User can move items around in the checklist once added to the incident channel

  1. As user A, start a new incident.

  2. Create a checklist

  3. Add 3 items “Test”, “123” and “ABC” to the checklist in that order.

  4. Add user B to the incident channel.

  5. As user B, click on the incident started by user A in step 1 to open the incident details.

  6. Verify there is an “edit” button beside the Checklist header.

  7. Click on the “edit” button.

  8. Click on the hamburger menu icon beside “123” and drag it to the top of the list.

  9. Verify that list now appears in the order of “123”, “Test” and “ABC”.

  10. Click on the hamburger menu icon beside “ABC” and drag it to the middle of the list.

  11. Verify that the list now appears in the order of “123”, “ABC” and “Test”.

  12. Click “done”

  13. Verify that the list remains in the same order as mentioned in step 11.

  14. Click the “<“ button beside the incident name to go back to the incident list.

  15. Click on the incident again to bring up the details.

  16. Verify that the changes made in step 11 persisted.

2426

User can remove item added by different user once added to the incident channel.

  1. As user A, start a new incident.

  2. Create a checklist and add a few items to the checklist.

  3. Add user B to the incident channel.

  4. As user B, click on the incident started by user A in step 1 to open the incident details.

  5. Verify that the checklist setup in step 2 is visible.

  6. Verify that the checklist items have an “edit” button beside the Checklist header.

  7. Click on “edit”.

  8. Click on the “X” button beside the checklist item to remove the items.

  9. Verify that the items are removed from the checklist.

2527

Checklist item added reflects accurately for user not added to incident channel

  1. Open two separate browser windows.

  2. Log in as 2 different users (user A and user B) in the two browsers.

  3. In both browsers, open up the Incident Response RHS.

  4. As user A, create a new incident.

  5. Do not add user B to the incident channel.

  6. As user B, find the incident created in step 4 by user B and click to view the details.

  7. While user B is also viewing the RHS, have user A create a checklist with a few items.

  8. As user B verify that the checklist is updated accurately.

  9. Reversing the actions of user A and B, repeat steps 4-7.

  10. Repeat the tests for checklist item removal.

28

Checklist item edited reflects accurately for user not added to incident channel

Continue from the above test.

  1. As user A, edit an item in the checklist.

  2. As user B viewing the RHS, verify that the checklist is updated accurately to reflect the edit by user A.

29

Checklist item rearranged reflects accurately for user not added to incident channel

Continue from the above test.

  1. As user A, rearrange the order of the checklist items.

  2. As user B viewing the RHS, verify that the checklist is rearranged accurately to reflect the rearrangement by user A.

30

Checklist item removed reflects accurately for user not added to incident channel

Continue from the above test.

  1. As user A, remove an item from the checklist.

  2. As user B viewing the RHS, verify that the item is updated accurately to reflect the removed item from the checklist by user A.

2631

Checklist changes reflect accurately for a different user added to incident channel

  1. Open two separate browser windows.

  2. Log in as 2 different users (user A and user B) in the two browsers.

  3. In both browsers, open up the Incident Response RHS.

  4. As user A, create a new incident.

  5. Add user B to the incident channel.

  6. As user B, find the incident created in step 4 by user B and click to view the details.

  7. While user B is also viewing the RHS, have user A create a checklist with a few items.

  8. As user B verify that the checklist is updated accurately.

  9. Reversing the actions of user A and B, repeat steps 4-7.

  10. Repeat the tests for checklist item removal.

2732

Checklist can be edited added by 2 users at the same time when they are both incident channel members

  1. Open two separate browser windows.

  2. Log in as 2 different users (user A and user B) in the two browsers.

  3. In both browsers, open up the Incident Response RHS.

  4. As user A, create a new incident.

  5. Add user B to the incident channel.

  6. As user B, find the incident created in step 4 by user B and click to view the details.

  7. Have both users add and remove checklist items for to the incidentchecklist.

  8. Verify that the checklist is updated accurately for each change made in step 7.

Test Area - Coordinate

Test ID

Test Case

33

Checklist can be edited by 2 users at the same time when they are both incident channel members

Continue from the above test.

  1. As any user add the following items to the checklist in the same order: “123”, “ABC”, “Test”.

  2. Have both users go in the edit mode for the checklist.

  3. Have both users click “123” to start editing.

  4. As user A, edit it to “123 A”.

  5. Verify that the change is reflected accurately in real time to user B.

  6. As user A, edit the same item to “B XYZ”.

  7. Verify that the change is reflected accurately in real time to user A.

  8. Save the changes made by user B.

  9. As user B, drag the “Test” item below “B XYZ”.

  10. As user A, drag ABC to the top of the list.

  11. Have both users go back to the incident list by clicking the “<“ button.

  12. Have both users go to the incident’s detail view.

  13. Verify the the checklist appears in the following order: “ABC“, “B XYZ“, “Test”

Test Area - Coordinate

Test ID

Test Case

Test Procedure

Result

Notes

2834

Commander can check off items in incident checklist

  1. Start an incident.

  2. Create a checklist with a few items.

  3. Verify that the commander can check items off.

2935

Commander can uncheck items in incident checklist

  1. Start an incident.

  2. Create a checklist with a few items.

  3. Check off the items.

  4. Verify that the commander can uncheck items.

3036

Non-commander who is not added to incident channel cannot check off items in incident checklist

  1. Start an incident as user A.

  2. Create a checklist with a few items.

  3. Log in as user B.

  4. Verify that user B cannot check off items in step 2.

3137

Non-commander who is not added to incident channel cannot uncheck items in incident checklist

  1. Start an incident as user A.

  2. Create a checklist with a few items.

  3. Check off a few items in the list.

  4. Log in as user B.

  5. Verify that user B cannot uncheck items in step 3.

3238

Non-commander user, who is added to the incident channel, can check off items in incident checklist

  1. Start an incident as user A.

  2. Create a checklist with a few items.

  3. Add user B to the incident channel.

  4. Verify that user B can check items off.

3339

Non-commander user, who is added to the incident channel, can uncheck items

  1. Start an incident as user A.

  2. Create a checklist with a few items.

  3. Add user B to the incident channel.

  4. Verify that user B can check off items added in step 3.

  5. Verify that user B can uncheck items checked off in step 4.

3440

Non-commander user, who is added to the incident channel, can uncheck items checked off by the commander

  1. Start an incident as user A.

  2. Create a checklist with a few items.

  3. Check items on the list.

  4. Add user B to the incident channel.

  5. Verify that user B can uncheck items that were checked off in step 3.

3541

Ending an incident shows a confirmation modal

  1. Start an incident.

  2. Open the RHS and find the incident just started.

  3. Click to view details.

  4. Verify there is an “End Incident” button at the bottom.

  5. Click on the button.

  6. Verify a confirmation modal pops up asking whether to end the incident.

  7. Go to the incident channel.

  8. In the post input box type /incident end

  9. Verify a confirmation modal pops up asking whether to end the incident.

3642

Commander can end incident

Continue from the above test:

  1. Click to confirm ending the incident.

  2. Verify that the incident is ended.

(Incident end verifications in Test Plan - Incident Response v0.1.0 )

3743

Non-commander cannot can end incident from RHS

  1. As user A start an incident.

  2. As user B, verify that the incident is visible in RHS.

  3. Click the incident to view details.

  4. Verify that there is no “End Incident” button.

  5. As user A, add user B to the incident channel.

  6. As user B, navigate to the incident channel.

  7. In the RHS, find the incident and click to view details.

  8. Verify there is no “End Incident” button.

  9. Click on the “End Incident” button and confirm to end.

  10. Verify that the incident is ended.

Fix from 0.1.0

3844

Non-commander cannot can end incident with /incident command.

  1. As user A, start an incident.

  2. As user B, verify that the incident is visible in RHS.

  3. Navigate to any channel.

  4. In the post input box, type `/incident'.

  5. Verify that there is no /incidentoption present in the slash commands.

  6. As user A, add user B to the incident channel.

  7. As user B, navigate to the incident channel.

  8. In the post input box, type `/incident'.

  9. Verify that there is no /incidentoption present in the slash commands.

  10. In incident channel, issue /incident endcommand.

  11. When the end-incident-confirmation pops up, confirm to end.

  12. Verify that the incident is ended.

3945

Incident cannot be ended from outside the incident channel

  1. Create an incident.

  2. Ensure the user is navigated to the incident channel.

  3. Open RHS and find the incident.

  4. Click on it to view the details.

  5. Verify the “End Incident” button exists in RHS.

  6. Navigate to a different channel than the incident channel.

  7. Verify that the “End Incident” changes to an inactive button, notifying the user that the incident can only be ended from within the incident channel.

Test Area - Measure

Test ID

Test Case

Test Procedure

Result

Notes

4046

A bot message is posted in incident channel when an item is added to an incident checklist

  1. As a normal user, start an incident.

  2. The user is navigated to the incident channel.

  3. Open the Incident Response RHS.

  4. Find the incident started in step 1 and click to open details.

  5. Create a checklist and add an item.

  6. In the incident channel, verify that a bot message similar to the following is posted:

@username added checklist item "[checklist item title]"

4147

“Item added to incident checklist” post in incident channel shows the username of user who added the item

Continue from the above test:

  1. Verify that the bot message from the above test shows the username of the user who added the checklist item.

42

Checklist item shows the timestamp of when the item was added

Continue from the above test:

  1. In the RHS, verify that the correct timestamp of when the item was added to the checklist is displayed beside the checklist item title.

https://mattermost.atlassian.net/browse/MM-23476 only talks about timestamp for checking off item

43
  1. from the above test shows the username of the user who added the checklist item.

48

A bot message is posted in incident channel when an item in checklist is checked

  1. As a normal user, start an incident.

  2. The user is navigated to the incident channel.

  3. Open the Incident Response RHS.

  4. Find the incident started in step 1 and click to open details.

  5. Create a checklist and add an item.

  6. Check the item in step 5.

  7. In the incident channel, verify that a bot message similar to the following is posted:

@username checked off checklist item "[checklist item title]"

4449

“Item checked” post in incident channel shows the username of user who checked the item

Continue from the above test:

  1. Verify that the bot message from the above test shows the username of the user who checked off the checklist item.

4550

Checking an item shows the timestamp to the right of the checklist item of when the item was checked off

Continue from the above test:

  1. In the RHS, verify that the correct timestamp of when the item was checked off is displayed beside the checklist item title.

4651

A bot message is posted in incident channel when an item in checklist is unchecked

  1. As a normal user, start an incident.

  2. The user is navigated to the incident channel.

  3. Open the Incident Response RHS.

  4. Find the incident started in step 1 and click to open details.

  5. Create a checklist and add an item.

  6. Check the item in step 5.

  7. Uncheck the item.

  8. In the incident channel, verify that a bot message similar to the following is posted:

@username unchecked item "[checklist item title]"

4752

“Item checked” post in incident channel shows the username of user who unchecked the item

Continue from the above test:

  1. Verify that the bot message from the above test shows the username of the user who unchecked the checklist item.

48

Unchecking an item shows the timestamp to the right of the checklist item of when the item was uncheckedchannel shows the username of user who unchecked the item

Continue from the above test:

  1. In the RHS, verify that the correct timestamp of when the item was unchecked is displayed beside Verify that the bot message from the above test shows the username of the user who unchecked the checklist item title.

https://mattermost.atlassian.net/browse/MM-23476 only talks about timestamp for checking off item

49

53

A bot message is posted in incident channel when an item is removed from an incident checklist

  1. As a normal user, start an incident.

  2. The user is navigated to the incident channel.

  3. Open the Incident Response RHS.

  4. Find the incident started in step 1 and click to open details.

  5. Create a checklist and add an item.

  6. Remove the checklist item.

  7. In the incident channel, verify that a bot message similar to the following is posted:

@username removed checklist item "[checklist item title]"

5054

“Item removed from incident checklist” post in incident channel shows the username of who removed the item

Continue from the above test:

  1. Verify that the bot message from the above test shows the username of the user who removed the checklist item.

5155

“Item removed from incident checklist” post in incident channel shows the timestamp of when the item was removed

Continue from the above test:

  1. Verify that the bot message from the above test shows the timestamp of when the item was removed from the checklist.

5256

Starting a next incident will start with empty checklist

  1. Start an incident.

  2. Create a checklist for the incident.

  3. Check a few items and leave some unchecked.

  4. Start a new incident.

  5. View the incident details of the new incident.

  6. Verify that the incident does not have any checklist.

...