Test Plan - Incident Response v0.2.0

0.0

Initial version

04/14

0.1

• Updated to reflect response from the sync-up on 04/16

• Removed tests for v0.3.0

04/16

Test ID

Test Case

Test Procedure

Result

Notes

1

Admin can install plugin in a non-EE instance

1. Log in to a non-enterprise Mattermost, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

2

Admin can install plugin in an E10 instance

1. Log in to Mattermost that has an E10 license, as admin.

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin is successfully installed.

3

Admin can install plugin in an E20 instance

1. Log in to Mattermost that has an E20 license, as admin

2. From System Console > PLUGINS > Plugin Management, upload the Incident Response plugin.

3. Verify that the plugin can be installed.

4

Admin can enable plugin

1. Once installed in the “Installed Plugins” section, find the Incident Response plugin v0.2.0 and click on the “Enable” button.

2. Verify that the plugin starts normally without errors.

5

Plugin functionality is available to admin

1. Switch to a normal channel.

2. Type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

6

Plugin functionality is available to normal user

1. Log in as a non-admin user.

2. In a normal channel, type /incidentin the post input box.

3. Verify that /incidentis listed as a slash command option.

4. Verify that the channel header has Incident Response icon for RHS.

5. Create a post.

6. Click post action menu for the post.

7. Verify that Start Incident is available as a post action menu option.

7

Admin can disable plugin

1. Log out and log back in as admin.

2. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

3. Click on the “Disable” button.

4. Verify that the plugin is disabled.

5. In the main channel’s post input box, type /incident.

6. Verify that /incident option is not present in the slash command list.

8

Admin can remove plugin

1. From System Console > PLUGINS > Plugin Management, find the Incident Response plugin.

2. Click on the “Remove” button.

3. Verify that the plugin is no longer installed in the instance.

Test ID

Test Case

Test Procedure

Result

Notes

9

User is taken to incident channel after starting an incident

1. As a normal user, start an incident.

2. Once the incident is started, verify that the user is taken to the new incident channel.

3. If the same user has multiple sessions open, it would only change to the new incident channel in the session where the incident was started.

10

In RN mobile app, user is notified with an ephemeral message with link to the incident channel

1. Start an incident in RN mobile app.

2. Verify that the user remains in the same channel where the incident was started.

3. Verify that an ephemeral message is shown with the link to the incident channel.

11

A newly started incident does not have any checklist

1. As a normal user, start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step1 and click.

4. Verify that the incident does not have any checklist.

5. Verify that there is an “Add new checklist item”.

12

Commander can create a checklist for an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name and hit enter.

7. Verify that the checklist item is created.

8. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

13

Commander can create a checklist for an incident from a different channel than the incident channel

1. Start a new incident.

2. Navigate to a different channel from the incident channel.

3. Open up the Incident RHS.

4. Find the incident started in step 1 and click on it to open the details.

5. Click the “Add new checklist” button.

6. Verify that an input box asks for the checklist item name.

7. Input a checklist item name and hit enter.

8. Verify that the checklist item is created.

9. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

14

Checklist item creation can be cancelled.

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Click the “Add new checklist item” button.

5. Verify that an input box asks for the checklist item name.

6. Input a checklist item name.

7. Press escape.

8. Verify that the checklist item is not created.

15

Commander can remove checklist item from an incident started by them

1. Start a new incident.

2. Open up the Incident RHS.

3. Find the incident started in step 1 and click on it to open the details.

4. Create a new checklist with a few items.

5. Click the button to remove the checklist item.

6. Verify that the checklist item is removed from the incident.

16

Checklist item can be removed from a different channel than the incident channel

1. Start a new incident.

2. Open up the Incident RHS.

3. Create a checklist and add a few items to it.

4. Navigate to a different channel from the incident channel.

5. Find the incident started in step 1 and click on it to open the details.

6. Click the button to remove the checklist item.

7. Verify that the checklist item is removed from the incident.

17

User can see checklist in RHS for active incidents

1. For this step, ensure there are a few incidents already started.

2. For each incident add checklists.

3. As normal user, open the Incident Response RHS.

4. Click on the incidents one at a time.

5. Verify that each incident has checklists as setup in step 2.

18

User can access read-only incident checklist even if user is not an incident channel member

1. As user A, start an incident (don’t add user B to incident channel).

2. Create a checklist by clicking the “Add new checklist” button.

3. As user B, open the Incident Response RHS.

4. Click on the incident started in step 1.

5. Verify that the checklist appears as setup in step 2.

19

User cannot add checklist to an new incident started by a different user

1. As user A, start a new incident (don’t add user B to incident channel).

2. As user B, click on the incident started by user A in step 1 to open the incident details.

3. Verify there is no “Add new checklist item” button for user B.

20

User cannot add items to a checklist, with existing items, for an incident started by a different user

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that the checklist setup by user A in step 2 is visible.

5. Verify that there is no button to add new checklist item.

21

User cannot remove checklist item from an incident started by a different user

1. As user A, start a new incident (don’t add user B to incident channel).

2. Create a checklist and add a few items to the checklist.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify that checklist items setup in step 2 does not have the “-” button to remove the items.

22

User can add checklist item to a new incident started by a different user once added to the incident channel

1. As user A, start a new incident.

2. Add user B to the incident channel.

3. As user B, click on the incident started by user A in step 1 to open the incident details.

4. Verify there is “Add new checklist item” button for user B.

5. Click on the “Add new checklist item”.

6. Verify that an input box asks for the checklist item name.

7. Input a checklist item name and hit enter.

8. Verify that the checklist item is created.

9. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

23

User can add items to a checklist, with existing items, for an incident started by a different user

1. As user A, start a new incident.

2. Add user B to the incident channel.

3. Create a checklist and add a few items to the checklist.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 3 is visible.

6. Verify there is “Add new checklist item” button at the end of the checklist for user B.

7. Click on the “Add new checklist item”.

8. Verify that an input box asks for the checklist item name.

9. Input a checklist item name and hit enter.

10. Verify that the checklist item is created.

11. Verify that the “Add new checklist item” button is present at the bottom of the checklist.

24

User can remove item from an incident once added to the incident channel.

1. As user A, start a new incident.

2. Add user B to the incident channel.

3. Create a checklist and add a few items to the checklist.

4. As user B, click on the incident started by user A in step 1 to open the incident details.

5. Verify that the checklist setup in step 3 is visible.

6. Verify that the checklist items have a “-” button to remove the items.

7. Click on the button to remove the items.

8. Verify that the items are removed from the checklist.

25

Checklist changes reflect accurately for a different user not added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Do not add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

26

Checklist changes reflect accurately for a different user added to incident channel

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. While user B is also viewing the RHS, have user A create a checklist with a few items.

8. As user B verify that the checklist is updated accurately.

9. Reversing the actions of user A and B, repeat steps 4-7.

10. Repeat the tests for checklist item removal.

27

Checklist can be edited by 2 users at the same time

1. Open two separate browser windows.

2. Log in as 2 different users (user A and user B) in the two browsers.

3. In both browsers, open up the Incident Response RHS.

4. As user A, create a new incident.

5. Add user B to the incident channel.

6. As user B, find the incident created in step 4 by user B and click to view the details.

7. Have both users add and remove checklist items for the incident.

8. Verify that the checklist is updated accurately for each change made in step 7.

Test ID

Test Case

Test Procedure

Result

Notes

28

Commander can check off items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Verify that the commander can check items off.

29

Commander can uncheck items in incident checklist

1. Start an incident.

2. Create a checklist with a few items.

3. Check off the items.

4. Verify that the commander can uncheck items.

30

Non-commander who is not added to incident channel cannot check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Log in as user B.

4. Verify that user B cannot check off items in step 2.

31

Non-commander who is not added to incident channel cannot uncheck items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check off a few items in the list.

4. Log in as user B.

5. Verify that user B cannot uncheck items in step 3.

32

Non-commander user, who is added to the incident channel, can check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check items off.

33

Non-commander user, who is added to the incident channel, can uncheck items

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check off items added in step 3.

5. Verify that user B can uncheck items checked off in step 4.

34

Non-commander user, who is added to the incident channel, can uncheck items checked off by the commander

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check items on the list.

4. Add user B to the incident channel.

5. Verify that user B can uncheck items that were checked off in step 3.

35

Ending an incident shows a confirmation modal

1. Start an incident.

2. Open the RHS and find the incident just started.

3. Click to view details.

4. Verify there is an “End Incident” button at the bottom.

5. Click on the button.

6. Verify a confirmation modal pops up asking whether to end the incident.

7. Go to the incident channel.

8. In the post input box type /incident end

9. Verify a confirmation modal pops up asking whether to end the incident.

36

Commander can end incident

Continue from the above test:

1. Click to confirm ending the incident.

2. Verify that the incident is ended.

(Incident end verifications in Test Plan - Incident Response v0.1.0 )

37

Non-commander cannot can end incident from RHS

1. As user A start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Click the incident to view details.

4. Verify that there is no “End Incident” button.

5. As user A, add user B to the incident channel.

6. As user B, navigate to the incident channel.

7. In the RHS, find the incident and click to view details.

8. Verify there is no “End Incident” button.

9. Click on the “End Incident” button and confirm to end.

10. Verify that the incident is ended.

Fix from 0.1.0

38

Non-commander cannot can end incident with /incident command.

1. As user A, start an incident.

2. As user B, verify that the incident is visible in RHS.

3. Navigate to any channel.

4. In the post input box, type `/incident'.

5. Verify that there is no /incidentoption present in the slash commands.

6. As user A, add user B to the incident channel.

7. As user B, navigate to the incident channel.

8. In the post input box, type `/incident'.

9. Verify that there is no /incidentoption present in the slash commands.

10. In incident channel, issue /incident endcommand.

11. When the end-incident-confirmation pops up, confirm to end.

12. Verify that the incident is ended.

39

Incident cannot be ended from outside the incident channel

1. Create an incident.

2. Ensure the user is navigated to the incident channel.

3. Open RHS and find the incident.

4. Click on it to view the details.

5. Verify the “End Incident” button exists in RHS.

6. Navigate to a different channel than the incident channel.

7. Verify that the “End Incident” changes to an inactive button, notifying the user that the incident can only be ended from within the incident channel.

Test ID

Test Case

Test Procedure

Result

Notes

40

A bot message is posted in incident channel when an item is added to an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. In the incident channel, verify that a bot message similar to the following is posted:

@username added checklist item "[checklist item title]"

41

“Item added to incident checklist” post in incident channel shows the username of user who added the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who added the checklist item.

42

Checklist item shows the timestamp of when the item was added

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was added to the checklist is displayed beside the checklist item title.

https://mattermost.atlassian.net/browse/MM-23476 only talks about timestamp for checking off item

43

A bot message is posted in incident channel when an item in checklist is checked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username checked off checklist item "[checklist item title]"

44

“Item checked” post in incident channel shows the username of user who checked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who checked off the checklist item.

45

Checking an item shows the timestamp to the right of the checklist item of when the item was checked off

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was checked off is displayed beside the checklist item title.

46

A bot message is posted in incident channel when an item in checklist is unchecked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. Uncheck the item.

8. In the incident channel, verify that a bot message similar to the following is posted:

@username unchecked item "[checklist item title]"

47

“Item checked” post in incident channel shows the username of user who unchecked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who unchecked the checklist item.

48

Unchecking an item shows the timestamp to the right of the checklist item of when the item was unchecked

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was unchecked is displayed beside the checklist item title.

https://mattermost.atlassian.net/browse/MM-23476 only talks about timestamp for checking off item

49

A bot message is posted in incident channel when an item is removed from an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Remove the checklist item.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username removed checklist item "[checklist item title]"

50

“Item removed from incident checklist” post in incident channel shows the username of who removed the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who removed the checklist item.

51

“Item removed from incident checklist” post in incident channel shows the timestamp of when the item was removed

Continue from the above test:

1. Verify that the bot message from the above test shows the timestamp of when the item was removed from the checklist.

52

Starting a next incident will start with empty checklist

1. Start an incident.

2. Create a checklist for the incident.

3. Check a few items and leave some unchecked.

4. Start a new incident.

5. View the incident details of the new incident.

6. Verify that the incident does not have any checklist.