Test Plan - Incident Response v1.0

0.0

Initial version

10/06

1.0

Test Completed (for v1.0 updated features)

10/14

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

1

/incident is available as slash command option in mobile

1. Go to the main channel view in mobile app.

2. Under the main input box, tap the / icon.

3. Verify that the slash command suggestion box pops up.

4. Scroll down.

5. Verify that incident slash command is available.

Pass

2

Typing /incident should load the available slash command options

• In mobile

  1. In the main channel input box, type /incident

  2. In the pop up for incident [command], verify that the available slash command options are shown.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

3

The slash command options in the popup is scrollable

• In mobile

  1. In the main channel input box, type /incident

  2. Tap the /incident [command] option in the pop up.

  3. Verify the pop up changes to the available command options.

  4. Touch and scroll the pop up options up and down.

  5. Verify that all options appear normally.

• In webapp, scroll + up/down arrows work

  1. Type /incident with a space.

  2. Verify the pop up changes to the available command options.

  3. Verify that the pop up can be scrolled up and down.

  4. Verify that using the up-down arrow keys highlight the intended slash command option.

Pass

4

“Execute current command” is highlighted by default in pop up options

• In mobile - ? There is no “Execute current command”

• In webapp

  1. Type /incident (with a space).

  2. When the slash command pop up changes to the available options, verify that “Execute Current Command” is highlighted by default.

Pass in webapp

https://mattermost.atlassian.net/browse/MM-29629

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

5

/incident start is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that start is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

6

Issuing /incident start opens up the incident creation screen

• In mobile

  1. Issue an /incident start command by typing the command in the main input box and tapping the ‘send’ icon.

  2. Verify that the main screen changes into the ‘Incident Details’ incident creation screen.

• In webapp

  1. Issue an /incident start command.

  2. Verify that the incident creation modal opens up.

Pass

https://mattermost.atlassian.net/browse/MM-29633

7

Incident creation screen on mobile shows all the fields as the the incident creation page on webapp

• In mobile

  1. Issue an /incident start command to open up the incident creation screen.

  2. Verify that the following fileds are present:

     1. Commander

     2. “Create a playbook” link

     3. Playbook dropdown

     4. Incident Name

     5. Incident Description

Pass

8

Tap “Create a playbook” link in incident creation screen takes the user to the playbook creation screen?

• In mobile - continue from the above test

  1. Tap on the “Create a playbook” link in the incident creation screen.

  2. Verify that the user is taken to the playbook creation screen (in mobile webapp?)

Fail

Says “Link not found in server”

https://mattermost.atlassian.net/browse/MM-29380

9

Starting an incident requires a playbook to be selected

• In mobile

  1. Issue /incident start command to open up the incident creation screen.

  2. Do not select any playbook from the “Playbook” dropdown.

  3. Enter “MI-1” as the incident name.

  4. Tap “Start Incident”.

  5. Verify that the incident did not start.

  6. Verify that the incident creation screen is still open.

  7. Verify that an error message notifies the user that Playbook “field is required”.

Pass

10

Starting an incident requires an incident name

• In mobile - continue from the above test

  1. Select any playbook from the “Playbook” dropdown.

  2. Leave the incident name field empty.

  3. Tap “Start Incident”.

  4. Verify that the incident did not start.

  5. Verify that the incident creation screen is still open.

  6. Verify that an error message notifies the user that Incident Name “field is required”.

Pass

11

User can start an incident on mobile app using /incident start command

• In mobile - continue from the above test

  1. In the incident creation screen, ensure that a playbook is selected and an incident name has been entered.

  2. Verify that the incident has started.

  3. Verify that the incident creation screen is no longer open.

Pass

13

When incident is started from mobile app, a system message is posted in the channel from where incident is started

• In mobile

  1. As user-1, go to channel A.

  2. Select any playbook.

  3. Enter “M-In-a” as the incident name.

  4. Start the incident.

  5. Once an incident is started and the incident creation screen is closed, verify that the user is back in channel A.

  6. Verify that the user sees a system message that “Incident M-In-a started in ~M-In-a”.

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

14

/incident info is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that info is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

15

On mobile app, issuing /incident info command in incident channel posts an ephemeral message including the incident details

1. As user-1, create a new playbook with only 1 Stage S1 and tasks T1 and T2.

2. Start incident “M2”.

3. Navigate to the “M2” incident channel.

4. Issue an /incident info command.

5. Verify that a system message displays the following accurately:

   1. Incident name: M2

   2. Duration of incident

   3. Commander of incident: user-1

   4. Stage: S1

   5. Tasks: T1, T2

Pass

16

On mobile, issuing /incident info command in non-incident channel posts a message that the info is only available in incident channels

1. Go to a non-incident channel.

2. Issue an /incident info command.

3. Verify that a system message notifies the user that the details of an incident can only be viewed in an incident channel.

Pass

17

On webapp, issuing /incident info command in incident channel opens incident RHS if not already open

1. Log in as user-1 in webapp.

2. Go to the “M2” incident channel.

3. Ensure that the incident RHS is not open (close if needed).

4. Issue an /incident info command.

5. Verify that the incident RHS opens showing the incident details.

Pass

Yes - E2E test PR

18

On webapp, issuing /incident info command in incident channel with a non-incident RHS open, opens incident RHS

Continue from the above test

1. From the channel header, click the “Pinned posts” icon.

2. Issue an /incident info command.

3. Verify that the incident RHS opens showing the incident details.

4. From the channel header, click the “Saved posts” icon.

5. Issue an /incident info command.

6. Verify that the incident RHS opens showing the incident details.

7. From the channel header, click the “@ Recent mentions” icon.

8. Issue an /incident info command.

9. Verify that the incident RHS opens showing the incident details.

10. From the channel header, search for a random term to trigger the search RHS.

11. Issue an /incident info command.

12. Verify that the incident RHS opens showing the incident details.

Pass

Yes - https://github.com/mattermost/mattermost-plugin-incident-response/blob/master/tests-e2e/cypress/integration/frontstage/slash_command/info_spec.js

19

On webapp, issuing /incident info command in incident channel with incident RHS open posts a message that the incident details is already open in incident RHS

Continue from the above test

1. Ensure that the incident RHS is open.

2. Issue an /incident info command.

3. Verify that the incident bot notifies the user that the “incident details are already open in the RHS”.

Pass

Yes - https://github.com/mattermost/mattermost-plugin-incident-response/blob/master/tests-e2e/cypress/integration/frontstage/slash_command/info_spec.js

20

On webapp, issuing /incident info command in non-incident channel posts a message that the info is only available in incident channels

Continue from the above test

1. On webapp, navigate to a non-incident channel.

2. Issue an /incident info command.

3. Verify that the incident bot notifies the user that the incident details can only be viewed in an incident channel.

Pass

Yes - E2E test PR

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

21

/incident list is available as a slash command

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that list is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

22

Issuing an /incident list command when there is no incidents (fresh server) should show “There are no ongoing incidents in [team name] team” message for the user.

0. Install a fresh server.

• In mobile

  1. Go to any channel and issue an /incident list command.

  2. Verify that the incident bot notifies the user that there are no ongoing incidents for the current team.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

23

Issuing /incident list in non-incident channel shows on going incidents

1. Start a few incidents (< 10 incidents).

2. In mobile, go to a non-incident channel.

3. Issue an /incident list command.

4. Verify that the incident bot posts a message with the incidents started in step 1 with the details including:

   1. link to the incident channel

   2. active stage

   3. duration

   4. commander of the incident

5. In webapp, repeat steps 3-4.

Pass

24

Issuing /incident list in incident channel shows on going incidents

Continue from the above test

1. In mobile, go to an incident channel.

2. Issue an /incident list command.

3. Verify that the incident bot posts a message with the incidents started in step 1 with the details including:

   1. link to the incident channel

   2. active stage

   3. duration

   4. commander of the incident

4. In webapp, repeat steps 2-3.

Pass

Issuing /incident list in incident channel shows all ongoin incidents if there are <= 10 ongoing incidents

Pass

Issuing /incident list show 10 of the most recent ongoing incidents if there are >10 ongoing incidents

Pass

25

Issuing /incident list does not show any incident that has been ended

Continue from the above test

1. End an incident from the above test.

2. In mobile, go to an incident channel.

3. Issue an /incident list command.

4. Verify that the incident bot does not list the incident that was ended in step 1.

5. End another incident.

6. Repeat steps 3-4

7. In webapp issue an /incident list command.

8. Verify that the incidents ended in steps 1 and 5 are not listed by the incident bot.

Pass

26

Issuing /incident list shows incidents that have been restarted

Continue from the above test

1. Restart the incidents ended in the above test.

2. In mobile, issue an /incident list command.

3. Verify that the incident bot lists all incidents.

4. In webapp, issue an /incident list command.

5. Verify that the incident bot lists all incidents.

Pass

27

Issuing /incident list does not show incidents from another team

1. In mobile, as user-1 switch to a different team, Team Y; create one if needed.

2. Issue an /incident list command.

3. Verify that the incident bot notifies that there are no ongoing incidents in Team Y.

4. In webapp, switch to Team Y.

5. Repeat steps 2-3.

Pass

28

Issuing /incident list does not show incidents for another user

1. Log in as user-2 (a user that has not started any incident).

2. Issue an `/incident list` command in any channel.

3. Verify that the incident bot notifies that there are no ongoing incidents in the current team.

Pass

29

/incident list should show all ongoing incidents for a large number of incidents

1. Create a large number of incidents (>50).

2. In mobile, issue an `/incident list` command.

3. Verify that the incident bot lists all ongoing incidents.

4. In webapp, repeat steps 2-3.

Invalid because only 10 of the most recent ongoing incidents are shown.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

30

/incident commander is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that commander is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

31

Issuing /incident commander in non-incident channel shows “Commander is only available in incident channel”

• In mobile

  1. Go to a non-incident channel

  2. Issue an /incident commander command.

  3. Verify that the incident bot notifies the user that the commander can only be viewed in the incident channels.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

32

Issuing /incident commander in incident channel shows the current commander of the incident

In mobile

1. Go to an incident channel

2. Issue an /incident commander command.

3. Verify that the incident bot posts a message including the current commander of the incident.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

33

/incident commande [@user-name] in non-incident channel shows “You can only change the commander from the incident channel” message

• In mobile

  1. Go to a non-incident channel.

  2. Issue an /incident commander @sysadmin command.

  3. Verify that the incident bot posts a message that the commander can only be changed from the incident channel.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

34

/incident commander [@user-name] in incident channel with [@user-name] not added to incident channel shows [@user-name] must be a part of this channel.. message

• In mobile

  1. Go to an incident channel.

  2. Issue an /incident commander @user-2 command.

  3. Verify that the incident bot posts a message that user-2 must be an incident member to make them the commander.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

35

/incident commander [@user-name] in incident channel with [@user-name] already added to the channel changes the commander to that user

• In mobile

1. Go to an incident channel.

2. Invite user-2 to the incident channel.

3. Issue an /incident commander @user-2 command.

4. Verify that the incident bot posts a message that user-1 changed the incident commander from user-1 to user-2.

5. Verify that the new commander of the incident is user-2.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

36

/incident commander [@user-name] in incident channel with [@user-name] of the current commander shows “`[@user-name]` is already the commander of this incident”

Continue from the above test

• In mobile

  1. In the same incident channel as the above test, issue another /incident commander @user-2 command.

  2. Verify that the incident bot posts a message that user-2 is already the commander of the incident.

  3. Verify that user-2 is still the commander of the incident.

• In webapp

  1. Repeat mobile. tetst steps above in webapp.

Pass

37

An incident channel member can see a dropdown for users under “Commander” in details view

Covered in older release tests (will be reorganized later).

38

Incident channel members are listed in the user dropdown list

39

An incident channel member can be searched by a commander in the user dropdown list

(0.3)

40

A non-incident channel member cannot be searched in the user dropdown list

41

A bot message is posted to the incident channel when commander is changed.

1. Log in as user A.

2. Start a new incident.

3. Add user B to the incident channel.

4. From the incident RHS, find the incident started in step 2 and see the details.

5. From the user dropdown, pick user B.

6. Verify that in the incident channel, the Incident bot posted a message that “user A changed the incident commander from user A to user B”.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

User is able to change a private incident name in mobile app

1. Log in as user-1.

2. Start a private incident “I-P-1”.

3. Go to the incident backstage and verify the incident is created.

4. In mobile app, go to the “I-P-1” incident channel.

5. Tap the “chevron” icon next to the channel name in the channel header.

6. Tap “Edit Channel”.

7. Edit the channel name to “New I1”.

8. From webapp, go to the incident backstage.

9. Verify that incident “I-P-1” does not exist.

10. Verify that the new incident “New I1” exists and ongoing.

Pass

https://mattermost.atlassian.net/browse/MM-29698

User is able to change a public incident name in mobile app

1. Start a public incident “Public I1”.

2. Go to the incident backstage and verify the incident is created.

3. In mobile app, go to the “Public I1” incident channel.

4. Tap the “chevron” icon next to the channel name in the channel header.

5. Tap “Edit Channel”.

6. Edit the channel name to “New public 1”.

7. From webapp, go to the incident backstage.

8. Verify that incident “Public I1” does not exist.

9. Verify that the new incident “New public 1” exists and ongoing.

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

42

Clicking on the incident icon when there are no incidents, opens the incident RHS to the “welcome” message

1. Install a fresh server.

2. As user-1, click the incident icon in the channel header.

3. Verify that the incident RHS opens with a “welcome” message to the user.

Pass

43

With incidents ongoing, clicking on the incident icon when user is in non-incident channel RHS opens up with the incident list for the user

Continue from the above test.

1. Create a playbook.

2. Start a new incident “I1”.

3. Navigate to a non-incident channel.

4. From the channel header, click the incident icon.

5. Verify that the incident RHS opens up with the list view showing the incidents user-1 is a member of.

Pass

44

With incidents ongoing, clicking on the incident icon when user is in incident channel with RHS open, shows the current incident details

Continue from the above test.

1. Go to the incident channel for “I1”.

2. Click the incident icon in the channel header.

3. Verify that the incident RHS opens with the incident list view, listing the incident created in the above test.

Pass

45

Switching from one incident channel to another incident channel with the RHS details view open will switch to the selected incident’s detail view in the RHS

Continue fromt he above test.

1. Create incident “I2”.

2. Click the incident icon in the channel header to open the details view for “I2” in incident RHS.

3. Navigate to incident channel for “I1”.

4. Verify that the details view in the incident RHS switches from “I2” to that of “I1”

Pass

46

The incident RHS list view has a “Start Incident” button in the RHS header

Continue from the above test.

1. Open the incident RHS list view.

2. Verify that the RHS header has a “Start Incident” button.

Pass

47

The “Start Incident” button in the RHS header opens up the incident creation modal

Continue from the above test.

1. Click the “Start Incident” button in the incident RHS header.

2. Verify that the incident creation modal opens.

Pass

48

The incident RHS list view has a “Create Playbook” button

Continue from the above test.

1. Go to an incident channel.

2. Open the incident RHS list view.

3. On the top right of the incident RHS, click the three dot menu.

4. Verify a “Create Playbook” button exists.

5. Click the button.

6. Verify that the playbook backstage opens.

Pass

49

A user can only see the incident he/she is a member of in the incident RHS list view

Continue from the above test.

1. Log in as user-3.

2. Click the incident icon in the channel header to open the incident RHS.

3. Verify that there are no ongoing incidents.

4. Create a new incident “I4” (create a playbook if necessary).

5. Click the incident icon in the channel header.

6. Verify that the incident RHS only lists “I4” in the list view.

7. Create a new incident “I5”.

8. Verify that the incident RHS lists “I4” and “I5” in the list view.

Pass

50

A user cannot see a public incident in the incident RHS if he/she is not a member of that incident

1. As user-1, create a public incident “PI1”.

2. Do not add any other user to the incident channel.

3. Log out and log in as user-2.

4. Open up the incident RHS list view.

5. Verify that incident “PI1” is not listed in the incident RHS for user-2.

Pass

51

“Click here to see all incidents” link in the bottom of the incident RHS list view takes the user to the incident backstage

1. As user-1, open up the incident RHS list view.

2. Verify that there is a “Click here to see all incidents” link in the RHS.

3. Click the link.

4. Verify that the incidents backstage page loads listing all the incidents that user-1 is a member of.

Pass

52

The incident RHS details view has a “back arrow” that brings a user to the incident list view in RHS

Continue from the above test.

1. As user-1, go to the incident channel for “PI1”.

2. If the incident details view is not already open in the incident RHS, open it up.

3. Verify that in the incident RHS header, there is a “<“ arrow.

4. Click the “<“ arrow.

5. Verify that the incident details view switches to the incidents list view.

Pass

53

Viewing the incident list from an incident channel highlights the current incident

Continue from the above test.

1. As user-1, start another incident “I-a1”.

2. Go to the incident channel for “I-a1”.

3. Open up the incident list view in the incident RHS.

4. Verify that in the list view, the entry for “I-a1” is highlighted.

5. Verify that none of the other incidents that user-1 is a member of is highlighted.

Pass

54

“Go to Incident Channel” takes the user to the associated incident channel

Continue from the above test.

1. From the incident list in the RHS, find the entry for “PI1”.

2. Click the “Go to incident Channel” button for “PI1”.

3. Verify that the incident channel for “PI1” is loaded.

4. Verify that the incident RHS changes to the incident details views for “PI1”.

Pass

56

“Websocket updates should reflect changes automatically.” → what are the websocket updates?

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

/incident stage is available as slash command

Pass

/incident stage next is available as slash command

Pass

/incident stage prev is available as slash command

Pass

Running /incident stage prev while in the first stage shows the ephemeral post that “The active stage is the first one…”

Pass

Running /incident stage prev while in the non-first stage sets the active stage to the previous stage

Pass

Running `/incident stage next' while in the non-final stage sets the active stage to the next stage

Pass

Running /incident stage next in the last stage of an incident shows the ephemeral post that “The active stage is the last one.”

Pass

Running /incident stage next in an incident in a stage that has tasks unchecked, asks the user to confirm the action

Pass

Running /incident stage prev in an incident with only one stage shows the ephemeral post that “The active stage is the first one.”

Pass

Running /incident stage next in an incident with only one stage shows the ephemeral post that “The active stage is the last one.”

Pass

Running /incident stage [option] in a non-incident channel shows an ephemeral post “You can only change an incident stage from within the incident's channel. (Only visible to you)”

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

A cloud server has the IR plugin v1.0 installed by default

Verify as system administrator

A cloud server has the IR plugin enabled by default

The IR plugin can be disabled in a cloud server

The IR plugin can be removed from a cloud server

The IR plugin can be installed in an on-premises server that is equipped with an E20 license

The IR plugin can be enabled normally in an on-premises server that is equipped with an E20 license

The IR plugin can be disabled in an on-premises server that is equipped with an E20 license

The IR plugin can be removed from an on-premises server that is equipped with an E20 license.

The IR plugin cannot be enabled in an on-premises server that is equipped with an E10 license

The IR plugin can be removed from an on-premises server that is equipped with an E10 license

The IR plugin cannot be enabled in an on-premises EE server that is not equipped with any license

The IR plugin can be removed from an on-premises EE server that is not equipped with any license

The IR plugin cannot be enabled in an on-premises TE server

The IR plugin can be removed from an on-premises TE server

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

System administrator can access the plugin functionality

Non-admin user can access the plugin functionality

Guest user can access the plugin functionality

Other user roles being introduced in 5.28 can access the plugin functionality

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Channel Name input box is focused when incident response modal pops up

1. Start an incident using any method to bring up the Incident Details modal.

2. Verify that the Channel Name input box is focused.

(0.1)

Incident is started when enter is pressed on Incident Response modal

1. Start an incident using any method to bring up the Incident Details modal.

2. Provide a channel name.

3. Press enter.

4. Verify that the incident is created.

(0.1)

Incident creation is cancelled when esc is pressed on Incident Response modal

1. Bring up the Incident Details modal using any method.

2. Provide a channel name.

3. Press esc key.

4. Verify that the modal is dismissed.

5. Verify that the incident is not started.

(0.1)

Channel name in Incident Details modal cannot be empty

1. Start a channel using any method.

2. In the "Incident Details" modal, leave the ‘Channel Name’ field empty.

3. Click `Start Incident`.

4. Verify that the incident cannot be started and an error for required field appears in the modal.

Incident Response modal opens when starting an incident with slash command in public channel

(0.1)

Incident Response modal opens when starting an incident with slash command in private channel

(0.1)

Yes

Incident Response modal opens when starting an incident with slash command in group message channel

(0.1)

Yes

Incident Response modal opens when starting an incident with slash command in direct message channel

(0.1)

Yes

Incident Response modal opens when starting an incident with slash command in direct message with self

(0.1)

Yes

Incident Response modal opens when starting an incident from RHS in a public channel

(0.1)

Yes

Incident Response modal opens when starting an incident from RHS in a private channel

(0.1)

Yes

Incident Response modal opens when starting an incident from RHS in a GM channel

(0.1)

Yes

Incident Response modal opens when starting an incident from RHS in a DM channel

(0.1)

Yes

Incident Response modal opens when starting an incident from RHS in a DM with self

(0.1)

Yes

Incident Response modal opens when starting an incident from post menu in public channel

(0.1)

Yes

Incident Response modal opens when starting an incident from post menu in private channel

(0.1)

Yes

Incident Response modal opens when starting an incident from post menu in GM channel

(0.1)

Yes

Incident Response modal opens when starting an incident from post menu in DM channel

(0.1)

Yes

Incident Response modal opens when starting an incident from post menu in DM with self

(0.1)

Yes

User who started an incident is the commander of the incident by default

(0.2)

A message is posted to the channel when the incident starts

1. As a normal user, start an incident using any method.

2. Once the channel is created, verify that a message is posted in the incident channel with the name of the commander and the timestamp of when the incident was started.

(0.2)

User is taken to incident channel after starting an incident

1. As a normal user, start an incident.

2. Once the incident is started, verify that the user is taken to the new incident channel.

3. If the same user has multiple sessions open, it would only change to the new incident channel in the session where the incident was started.

(0.2)

In RN mobile app, user is notified with an ephemeral message with link to the incident channel

1. Start an incident in RN mobile app.

2. Verify that the user remains in the same channel where the incident was started.

3. Verify that an ephemeral message is shown with the link to the incident channel.

(0.2)

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

The incident channel can be renamed

1. Start an incident (using any method).

2. Open the incident RHS.

3. Find the incident started in step 1 and click for details.

4. Go to the incident channel.

5. Click on the channel name in the header to bring up the channel’s drop down menu.

6. Click on “Rename Channel”.

7. Rename the channel to “New Incident Channel”.

8. Change the URL to “new-incident-channel”

9. Click “Save”.

10. Verify that the channel name is changed to “New Incident Channel” on the header.

11. Verify that the channel name is changed to “New Incident Channel” on LHS.

12. Verify that the channel name is changed to “New Incident Channel” on RHS incident details.

13. Visit the new URL of the channel set in step 8.

14. Verify that the New Incident Channel loads in the center space.

(0.2)

The incident channel is linked properly when renamed

1. In a new browser window, visit the URL as set in step 8 from the test above.

2. Verify that the user lands on the New Incident Channel.

3. Switch to a different channel in the same team.

4. In the Incident RHS, click on the channel link with the new channel name.

5. Verify that the user lands on the New Incident Channel.

(0.2)

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Issuing an /incident end command from a non-incident channel shows “incident from within the incident's channel” message

1. As a normal user, start an incident using any method.

2. Go to a channel that is not an incident-response channel.

3. Issue the /incident end slash command in the channel.

4. Verify that “incident from within the incident's channel”.

(0.2)

A message is posted to the channel when the incident ends using slash command

1. Go to an incident channel.

2. End the incident with the slash command.

3. Verify that a message is posted in the channel with the name of the user who ended the incident and the timestamp of when the incident was ended.

(0.2)

A message is posted to the channel when the incident ends using RHS

1. As normal user, start an incident using any method.

2. Open up the Incident RHS.

3. Find the incident from step 1.

4. End the incident by clicking on End incident button.

5. Verify that in the incident channel, a message is posted that the incident has been ended including the name of the commander and the incident end timestamp.

(0.2)

Ending an incident shows a confirmation modal

1. Start an incident.

2. Open the RHS and find the incident just started.

3. Click to view details.

4. Verify there is an “End Incident” button at the bottom.

5. Click on the button.

6. Verify a confirmation modal pops up asking whether to end the incident.

7. Go to the incident channel.

8. In the post input box type /incident end

9. Verify a confirmation modal pops up asking whether to end the incident.

(0.2)

End incident confirmation modal can be dismissed by clicking Cancel

1. Click on an incident’s End Incident button.

2. Once the confirmation modal pops up, click on Cancel.

3. Verify that the confirmation modal gets cancelled.

4. Verify that the incident is not ended.

(0.2)

End incident confirmation modal can dismissed by pressing Esc

1. Click on an incident’s End Incident button.

2. Once the confirmation modal pops up, press Esc.

3. Verify that the confirmation modal gets cancelled.

4. Verify that the incident is not ended.

(0.2)

Incident member can end incident

(0.2)

Non-member cannot end incident

(0.2)

Incident cannot be ended from outside the incident channel

1. Create an incident.

2. Ensure the user is navigated to the incident channel.

3. Open RHS and find the incident.

4. Click on it to view the details.

5. Verify the “End Incident” button exists in RHS.

6. Navigate to a different channel than the incident channel.

7. Verify that the “End Incident” changes to an inactive button, notifying the user that the incident can only be ended from within the incident channel.

(0.2)

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Clicking the Incident Response plugin icon on header toggles the RHS open and close

1. Click the Incident Response icon in the channel header.

2. Verify that the Incident Response RHS opens up.

3. Click the Incident Response icon in the channel header again.

4. Verify that the Incident Response RHS closes.

(0.2)

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Incident Response RHS shows incidents for respective teams

1. As normal user, in team X create a few incidents.

2. In team X, open RHS by clicking on the Incident Response icon in the header.

3. Verify that all incidents created in step 1 are listed with normal user as the commander.

4. As a different user, in team Y create a few incidents.

5. Verify that incidents are created.

6. Switch to team X (as different user in step 4).

7. Open the Incident Response RHS.

8. Verify that only incidents created in step 1 are listed.

9. Log out as the different user and log in as normal user.

10. Go to team Y.

11. Open the Incident Response RHS.

12. Verify that incidents created in step 4 are listed.

13. Switch to team X.

14. Verify that incident created in step 1 are listed.

(0.2)

An incident appears on the RHS when started without having to refresh the list

1. Open the incidents RHS by clicking on the Incidents Response icon on the channel header.

2. With the incidents RHS open, on the center channel, start a new incident with the slash command.

3. Verify that the newly started incident in step b appears in the RHS incident list as soon as it is started by the slash command.

(0.2)

An incident disappears from the RHS when ended without having to refresh the list

1. Go to a channel of an active incident.

2. Open the incident RHS by clicking on the Incident Response plugin icon.

3. The incident must be listed on the RHS.

4. In the channel, post the command `/incident end`.

5. Verify that the incident disappears from the RHS without refreshing.

(0.2)

User who is not added to incident channel cannot check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Log in as user B.

4. Verify that user B cannot check off items in step 2.

(0.2)

User who is not added to incident channel cannot uncheck items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check off a few items in the list.

4. Log in as user B.

5. Verify that user B cannot uncheck items in step 3.

(0.2)

User, who is added to the incident channel, can check off items in incident checklist

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Add user B to the incident channel.

4. Verify that user B can check items off.

(0.2)

User, who is added to the incident channel, can uncheck items checked off by the commander

1. Start an incident as user A.

2. Create a checklist with a few items.

3. Check items on the list.

4. Add user B to the incident channel.

5. Verify that user B can uncheck items that were checked off in step 3.

(0.2)

An active stage is annotated with Activelabel in the RHS

1. Log in as user-1.

2. Bring up the incident creation modal (using any method).

3. Click the “Create a playbook” button.

4. Enter “PB1” as the playbook name.

5. Enter “S1” as the first stage.

6. Add a new step and call it “S2”.

7. Add another step and call it “S3”.

8. Save playbook.

9. Switch to a normal channel view.

10. Start a new incident “I1” with “PB1”.

11. In the incident RHS, verify that “S1” under “Stages” has an Activelabel next to it.

Non-active stages are not annotated with Active labels

Continue with the above test.

1. From the incident RHS for “I1”, click the “Stages” dropdown menu.

2. Verify that “S2” and “S3” don’t have Activelabel next to them.

Selecting a non-active stage from the RHS dropdown shows a Make Active option prompt

Continue with the above test.

1. From “Stages” in the incident RHS for “I1”, select “S2”.

2. Verify a “Make Active” button appears above the dropdown box.

Incident commander can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Click the “Make Active” button.

2. Verify that there is now an Active label next to “S2”.

3. Click the “Stages” dropdown.

4. Verify that there is no longer an Active label next to “S1”.

Workflow member can set a different stage as Active in an ongoing incident

Continue with the above test.

1. Invite user-2 to the “I1” incident channel.

2. Log in as user-2.

3. From the “Stages” in the incident RHS for “I1”, select “S3”.

4. Verify a “Make Active” button appears above the dropdown box.

5. Click the “Make Active” button.

6. Verify that there is now an Active label next to “S3”.

7. Verify that there is no longer an Active label next to “S1” or “S2”.

8. Log in as user-1.

9. Verify that “S3” has an Active label next to it.

Workflow member cannot can set a different stage as Active in an incident that has been ended

Continue with the above test.

1. As user-1, end the incident “I1”.

2. From the “Stages” in the incident RHS for “I1”, select “S2”.

3. Verify that “Make Active” button does not appear above the dropdown box (or even if it appears, it’s greyed out?)

Workflow member can set a different stage as Active in an incident that has been restarted

Continue with the above test.

1. From the incident RHS, click “Restart Incident”.

2. Verify that the incident has restarted: The “Restart Incident” button has changed to “End Incident”.

3. Select “S2” from the Stages dropdown in the RHS.

4. Click the “Make Active” button.

5. Verify that “S2” has the Active label next to it.

Plugin posts a message in the incident channel when changing an active stage

Continue from the above test.

1. Verify that setting “S2” as the active stage created a post that the stage has been set as active in the incident channel.

2. Verify that the post were also created in the former tests while setting different stages as active.

Workflow member views the active stage by default in incident RHS

Continue with the above test.

1. Log in as user-2.

2. Open the RHS for “I1”.

3. Verify that when the RHS for “I1” opens, the Stages dropdown shows “S2” by default.

When an active stage is changed by a different user, the stage selected in the RHS remains as is but without the Active annotation

Continue with the above test.

1. While viewing the “I1” RHS as user-2, have user-1 set “S1” as the active stage.

2. Verify that user-2 continues to see “S2” still in the RHS but now without the Active label.

In case of only one stage, the stage is set as active by default

1. As user-1, create a new playbook “P2”.

2. Have only one stage “S1” in “P2”.

3. Save playbook.

4. Switch to a normal channel view.

5. Start a new incident “I2” with “P2”.

6. Verify that when “I2” is successfully started, the RHS shows the “Stages” dropdown shows “S1 active” as the default stage.

In case of multiple stages, the first stage is set as active by default

1. From the playbook backstage, create a new playbook “P3”.

2. Add stages “S1”, “S2”, “S3” and “S4”.

3. Save playbook.

4. Switch to normal channel view.

5. Start an incident “I3” using “P3”.

6. When “I3” is successfully started, verify that the RHS shows “S1” as the active stage in the dropdown.

Switching between the stages does not set the stage as active

Continue from the above test.

1. From the “Stages” dropdown, select “S2”.

2. Verify that when “S2” loads in the RHS, there is no Active label.

3. From the dropdown, select “S3”.

4. Verify that when “S3” loads in the RHS, there is no Active label.

5. From the dropdown, select “S4”.

6. Verify that when “S4” loads in the RHS, there is no Active label.

7. From the dropdown, select “S1”.

8. Verify that when “S1” loads in the RHS, there is Active label.

Selecting an active stage from the RHS dropdown does not show a Make Active option prompt

Continue from the above test.

1. Select “S2” from the incident RHS.

2. Verify that Make Active option appears above the “Stages” dropdown box.

3. Select “S1” from the incident RHS.

4. Verify that when “S1” is selected, the Make Active option appears above the Stages dropdown.

Workflow member can check items off of stages that are active

Continue from the above test.

1. Launch playbook backstage.

2. Find “P3” and click to edit.

3. Add checklist items in all the stages of the playbook “P3”.

4. Switch to a normal channel view.

5. Start a new incident “I4” with playbook P3.

6. From the incident RHS, set “S2” as the active stage.

7. Click on the checkboxes of items in “S2” to check off.

8. Verify that the items in “S2” are successfully checked off.

Workflow member can uncheck items of stages that are not active

Continue from the above test.

1. Ensure that “S1” is not an active stage for “I4”.

2. From the incident RHS for “I4”, select “S1”.

3. Click on the checkboxes of items in “S1” to check off.

4. Verify that the items in “S1” are successfully checked off.

Workflow member can check off items of an ongoing incident

Workflow member cannot can checkoff items of an ended incident

Continue from the above test.

1. From the incident RHS for “I4” click “End Incident”.

2. Ensure that “I4” has been ended successfully.

3. Select “S3” from the incident RHS.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items cannot get checked off (checkboxes are inactive)?

Workflow member can checkoff items of a restarted incident

Continue from the above test.

1. From the incident RHS for “I4” click “Restart Incident”.

2. Ensure that “I4” has been restarted successfully.

3. Select “S3” from the incident RHS if not already selected.

4. Click on the checkboxes of the items in “S3”.

5. Verify that the items are checked off successfully.

The Stages dropdown in the RHS has all the stages for that incident

Selecting a different stage in the RHS shows the checklist corresponding to that stage

1. As user-1, launch playbook backstage.

2. Create a new playbook “P5”.

3. Add stage “S1”.

4. Under “S1”, add the following checklist items:

   1. “C1” with slash command /away.

   2. “C2” (no slash command).

5. Add stage “S2” (don’t add any checklist item).

6. Add stage “S3”.

7. Under “S3”, add the following checklist items:

   1. “Ca”

   2. “Cb”

   3. “Cc”.

8. Switch to a normal channel view.

9. Start an incident “I5” using “P5”.

10. From the incident RHS for I5, while “S1” is selected, verify that the checklist items it contains are “C1” (with the option to run the /away command), and “C2”.

11. Now from the “Stages” dropdown, select “S2”.

12. Verify that “S2” has no checklist items under the dropdown.

13. Select “S3”.

14. Verify that “S3” has a checklist with “Ca”, “Cb” and “Cc”.

Saving a playbook with only one stage and leaving the stage name empty, will create an incident with stage “Default Stage”

1. Launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “P6” for the playbook name.

4. Do not change anything else (there should already be a stage “Default Stage”).

5. Click “Save Playbook”.

6. Switch to a normal channel view.

7. Start a new incident “I6” with “P6”.

8. Open up the RHS for “I6”.

9. Verify that the selected active stage is called “Default Stage” in the dropdown box.

Any workflow member can switch between the stages in the RHS in an ongoing incident

Continue from the above test.

1. As user-1, launch playbook backstage.

2. Select “P5” and click to edit.

3. Under “Members”, add user-2.

4. Save playbook.

5. Log in as user-2.

6. Start a new incident “I7” with “P5”.

7. From the “I7” incident RHS, select “S3”.

8. Click the checkboxes to checkoff items.

9. Verify that the items under “S3” are successfully checked off.

Any workflow member can switch between the stages in the RHS in an incident that has been ended

Continue from the above test.

1. As user-1, end incident “I7”.

2. Switch to user-2.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

Any workflow member can switch between the stages in the RHS in an incident that has been restarted

Continue from the above test.

1. As user-2, restart incident “I7”.

2. Switch to user-1.

3. From the “I7” incident RHS, click on the “Stages” dropdown.

4. Select “S1”.

5. Verify that the checklist for “S1” is loaded.

6. Select “S2”.

7. Verify that the checklist for “S2” is loaded.

8. Select “S3”.

9. Verify that the checklist for “S3” is loaded.

Selecting a different stage will not change the default active stage view upon page refresh

Continue from the above test.

1. As user-1, set “S3” as the active stage.

2. Select “S1” from the RHS.

3. Verify that the checklist for “S1” are loaded in the RHS.

4. Navigate to a different channel.

5. Navigate back to the “I7” incident channel.

6. Verify that “S3” is selected in the “Stages” dropdown box.

7. Verify that the checklist for “S3” is loaded in the RHS.

8. Select “S2”.

9. Refresh the channel.

10. Verify that “S3” is selected in the “Stages” dropdown box.

11. Verify that the checklist for “S3” is loaded in the RHS.

Selecting a different stage will not change the stage view for a different workflow member

Continue from the above test.

1. Add user-2 to the incident channel of “I7”.

2. While user-1 is viewing the incident channel for “I7”, have user-2 switch between the stages in the “I7” RHS.

3. Verify that user-1 continues to see only the active stage in the incident RHS and that user-2 switching between the stages has no effect on user-1’s RHS view.

Stage title should state the current and total number of stages as X/Y

Continue from the above test.

1. In the “I7” incident RHS, verify that the current and total number of stages are listed above the “Stages” dropdown.

Step owners

Checklist item in the RHS has a dropdown for assignee

Continue from the above tests.

1. As user-1 start a new incident “N1” using “P5” from the above tests.

2. In the incident RHS for “N1”, verify that the checklist items for “S1” has a “No Assignee” (assignee) dropdown field under “C1”.

3. Select “S2” from the “Stages” dropdown.

4. Since “S2” has no checklist item, verify there is no assignee dropdown field.

5. Select “S3” from the “Stages” dropdown.

6. Verify that all the three checklist items for “S3” have the assignee dropdown fields under the checklist items.

By default, checklist item does not have any assignee

Continue from the above tests.

1. Verify that non of the checklist items for “N1”, “S1”, “S2” and “S3”, do not have any assignee on the checlist items.

The assignee dropdown shows the list of workflow members

Continue from the above tests.

1. While “S3” is selected, click on the “No Assignee” dropdown for “Ca”.

2. Verify that dropdown opens up with user-1 in the list.

3. Verify there is a search attached to the dropdown when it opens.

The assignee dropdown does not show users that are not workflow members

Continue from the above test.

1. In the dropdown, verify that there is no other user listed other than user-1.

2. In the search box for “No assignee”, type the user name for user-2.

3. Verify that user-2 does not appear in the dropdown.

A checklist item can be checked off without an assignee

Continue from the above test.

1. From the stages dropdown, select “S1”.

2. Ensure that the checklist items of “S1” have no assignees.

3. Click on the “C1” and “C2” checkboxes.

4. Verify that the items are checked off.

A checklist item can be unchecked without an assignee

Continue from the above test.

1. Continue with “S1” selected.

2. Ensure that the checklist items of “S1” have no assignees.

3. Uncheck the previously checked “C1” and “C2”.

4. Verify that the items are unchecked.

Workflow member can select him/herself as the checklist assignee

Continue from the above test.

1. Continue with “S1” selected.

2. For item “C1” click the assignee dropdown.

3. Select user-1 (current user) from the list.

4. Verify that the assignee dropdown field shows user-1 as the assignee.

Workflow member can select a different workflow member as the checklist assignee

Continue from the above test.

1. Invite user-2 in the “N1” incident channel.

2. In the “N1” incident RHS, under “C2”, click the assignee dropdown.

3. Verify that user-2 is now available in the list.

4. Select user-2.

5. Verify that user-2 is now assigned checklist item “C2”.

Workflow member can change assignee from him/herself to ‘No Assignee’ for a checklist item

Continue from the above test.

1. Switch to user-2.

2. Find the “N1” incident channel navigate to it.

3. From the “N1” incident RHS, select “S1” if not already selected as the stage.

4. Click the assignee dropdown for “C2”.

5. Verify there is a “No Assignee” option available under the search bar.

6. Click on “No Assignee”.

7. Verify that “C2” has no assignee.

Workflow member can change another workflow member to ‘No Assignee’ for a checklist item

Continue from the above test.

1. As user-2, assign checklist item “C2” to the him/herself.

2. Switch to user-1.

3. Find incident channel for “N1”.

4. Navigate to it.

5. From the incident RHS, select stage “S1” if not already selected.

6. Ensure user-2 is the assignee for “C2”.

7. Under “C2”, click the assignee dropdown.

8. Click on “No Assignee”.

9. Verify that the item has no assignee.

Plugin posts a message in incident channel when an assignee is selected

Continue from the above tests.

1. Verify that when assigning a checklist item to a workflow member generates a plugin message to notify the action in the center channel.

Plugin posts a message in incident channel when ‘No Assignee’ is selected

Continue from the above tests.

1. Verify that when selecting “No Assignee” for a checklist item, the plugin generates a message to notify the action in the center channel.

Workflow member cannot select multiple assignees

Continue from the above tests.

1. From the incident RHS for “N1”, click the assignee dropdown for “C1”.

2. Click user-1.

3. Click user-2.

4. Verify that clicking a different user highlight only that user in the dropdown.

5. Press and hold the CMD (or Ctrl) button and click on user-2.

6. Verify that clicking user-2 highlight only user-2.

7. Verify that more than 1 user cannot be selected as an assignee for the checklist item.

Workflow member cannot select a different assignee in an incident that is ended

Continue from the above tests.

1. End incident “N1”.

2. Once the incident has ended, click on the assignee dropdown for “C1”.

3. Verify that the assignee dropdown does not open up anymore (??).

4. (Even if it opens up, verify that a different user cannot be assigned to the checklist item).

Workflow member can select a different assignee in an incident that is restarted

Continue for the above tests.

1. Restart incident “N1”.

2. Once the incident has restarted, click on the assignee dropdown for “C2”.

3. Verify that the assignee dropdown is active and can be expanded.

4. Select user-2 as the assignee.

5. Verify that user-2 is successfully assigned “C2”.

Assignee does not change when a workflow member leaves the workflow channel

Continue form the above tests.

1. Log in as user-2.

2. Find and navigate to the “N1” incident channel.

3. Verify that user-2 is assigned to checklist item “C2” (under stage “S1”).

4. In the center channel post textbox, type “/leave” and hit enter to leave the channel.

5. Verify that user-2 cannot find the “N1” incident channel.

6. Log out.

7. Log in as user-1.

8. Find the “N1” incident channel and navigate to it.

9. Verify that user-2 is still the assignee for “C2” under “S1”.

The assignee dropdown list gets updated when a workflow member is kicked from the channel

Continue from the above tests.

1. Invite user-2 to “N1” incident channel again.

2. From the incident RHS, select stage “S3”.

3. Assign item “Ca” to user-2.

4. In the center channel post textbox, type “/kick” and hit enter to remove user-2 from the channel.

5. Under “S3”, click on the assignee list for “Ca”.

6. Verify that user-2 does now appear in the assignee list.

Assignee does not change when a workflow member is kicked from the workflow channel

Continue from the above tests.

1. Select stage “S1”.

2. Verify that “C2” still shows user-2 as the assignee.

3. Select stage “S3”.

4. Verify that “Ca” still shows user-2 as the assignee.

The assignee dropdown list gets updated when a workflow member leaves the channel

Continue from the above tests.

1. With user-2 still removed from the incident channel, click the assignee dropdown for “Cc” under stage “S3”.

2. Verify that user-2 is not in the assignee list.

The assignee dropdown list gets updated when a new member is added to the workflow channel

Continue from the above tests.

1. Invite user-2 back to the “N1” incident channel.

2. In the incident RHS, click the assignee dropdown for “Cc”.

3. Verify that user-2 is now available in the assignee list.

4. Assign “Cc” to user-2

An assignee can checkoff a checklist item

Continue from the above tests.

1. Log in as user-2.

2. From the “N1” incident RHS, find “Cc” item under “S3”.

3. Click the checkbox for “Cc”.

4. Verify that the checklist item is successfully checked off.

An assignee can uncheck a checklist item

Continue from the above tests.

1. As user-2, uncheck item “Cc”.

2. Verify that the checklist item is successfully unchecked.

A non-assignee can checkoff a checklist item

Continue from the above tests.

1. As user-2 uncheck all item assigned to user-2.

2. Log out.

3. Log in as user-1.

4. Find all items assigned to user-2.

5. Click the checkboxes for items assigned to user-2.

6. Verify that the items are successfully checked off.

A non-assignee can uncheck a checklist item

Continue from the above tests.

1. As user-1, uncheck all the boxes for items assigned to user-2.

2. Verify that the items are successfully unchecked.

Slash command execution

A checklist item configured without a slash command does not have a Run button next to it

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “Apb” as the playbook name.

5. Change the name of “Default Stage” to “S1”.

6. Click “Add new checklist item”.

7. Enter “C1” as the item name.

8. Don’t add anything for the slash command.

9. Save playbook.

10. Switch to normal channel view.

11. Start a new incident using “Apb”.

12. Once the incident is started, in the incident RHS, examine the checklist item “C1” under “S1”.

13. Verify that “C1” does not have any “Run” button.

A checklist item configured with a valid slash command has a Run button next to it

Continue from the above test.

1. Launch playbook.

2. Find and click “Apb” to edit.

3. Add slash command /away for “C1”.

4. Switch to normal channel view.

5. Start a new incident using “Apb”.

6. Once the incident is started, in the incident RHS, verify that “C1” show /away under it which has a Run button next to it.

Clicking the Run button for a checklist slash command runs the slash command

Continue from the above test.

1. Open up the incident RHS for “Ia1”.

2. Find “C1” under stage “S1”.

3. Click the Run button next to “C1”.

4. Verify that clicking the Run button executes so that the status of user-1 is now set to “away”.

Running a slash command with system message attached to it posts the slash command system message in the incident channel

Continue from the above test.

1. Verify that the system message for the /away command “System: You are now away (Only visible to you)” is posted in the incident channel.

In case of two valid slash commands, running the command from the checklist item will only run the first command successfully

Continue from the above test.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C5”.

4. Add the following for “C5’s” slash command: /online /away.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C5” in the incident RHS.

8. Click the Run button next to “C5”.

9. Verify that the status of user-1 is now set to “online” and not “away”.

A slash command can be run multiple times from the RHS checklist

Continue from the above tests.

1. Launch playbook backstage.

2. Find and click “Apb” to edit.

3. Under “S1” add a new checklist item “C6”.

4. Add /shrug as the slash command for “C6”.

5. Switch to normal channel view.

6. Start a new incident using “Apb”.

7. Once the incident has been started, find “C6” in the incident RHS.

8. Click the Run button for “C6”.

9. Verify that the /shrug command is executed successfully and a shrug message is posted in the center channel.

10. Repeat steps 8-9 multiple times.

A plugin post notifies when a slash command fails to run successfully

Verify that when a slash command fails to execute by clicking the Run button, the plugin posts a message in the center channel notifying that the execution has failed.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

A normal user cannot view the incidents in another team

1. As a normal user, create a new incident in Team A.

2. Switch to team B.

3. Verify that the incident RHS does not have the incident created in step 1.

4. As a different user create a new incident in Team B.

5. Switch to team A.

6. Verify that the incident RHS does not have the incident created in step 4.

(0.2)

Incident details in RHS lists the incident channels associated with it

1. Ensure that there are some incidents have been started.

2. As a normal user, open the Incident Response RHS

   1.  Click on the Incident Response icon in the header

3. Click on an incident to expand the incident’s details.

4. Verify that the channels associated with the incident are listed under details.

5. Verify that the commanders are commander is listed under details.

(0.2)

A playbook created in Team X is not visible for Team Y’s incident creation

1. As user A, go to Team X.

2. Create a playbook “PB-X”.

3. Verify that PB-X is accessible in the Playbook dropdown during incident creation.

4. Switch to Team Y.

5. Start an incident using any method.

6. Verify that “PB-X” is not present in the Playbook dropdown.

A playbook created in Team X is not visible in Team Y’s Playbook Backstage

1. As user A, go to Team X.

2. Create a playbook “PB-X”.

3. Verify that PB-X is accessible in the Playbook dropdown during incident creation.

4. Go to Playbook backstage.

5. Verify that PB-X is available in playbooks list.

6. Switch to Team Y.

7. Go to playbook backstage.

8. Verify that PB-X is not available in playbooks list.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

A normal user can’t end an incident after being removed from the incident channel

1. As admin in browser A, start a new incident using any method.

2. In the incident channel for the incident started in step 1, add the normal user.

3. In browser B, log in as normal user.

4. Verify that the incident channel appears on the LHS.

5. Open the Incident Response RHS.

6. Find the incident from step 1 and click for details.

7. Verify that the End incident button exists.

8. In browser A (as admin), remove the normal user from the incident channel in step 2.

9. In browser B, as normal user, verify that the incident channel no longer present in LHS.

10. Open the Incident Response RHS.

11. Find the incident from step 1 and click for details.

12. Verify that the End incident button does not exist.

(0.2)

User can make a public incident

1. As user-1, create a public playbook as follows:

   1. Launch the playbook backstage.

   2. Click “New Playbook”.

   3. Provide name of the playbook - “Public PB 1“.

   4. Select “Public” channel type.

   5. Enter a few checklist items.

   6. Click “Save”.

2. Switch to the main channel view.

3. Create an incident by selecting the playbook “Public PB 1”.

4. Do not add any other user to the incident channel.

5. As user-2, verify that the incident is visible in the incident backstage.

6. Click on the incident.

7. Verify that the incident details can’t be viewed.

8. Verify that the incident details can be viewed.

User can make a private incident

1. As user-1, launch the playbook backstage.

2. Click “New Playbook”.

3. Provide a playbook name “Private-PB-1”.

4. Select “Private” channel type.

5. Add some checklist items.

6. Save playbook.

7. Create an incident by selecting the playbook “Private-PB-1”.

8. Log in as user-2.

9. Launch incident backstage.

10. Verify that the incident is not visible in the incident list.

Converting a public incident channel into private, makes the incident private

1. As a sysadmin, create a public incident “Pub-1”.

2. Log in as user-1.

3. Join the “Pub-1” incident channel to become a participant.

4. Convert it into a private incident

   1. Go to Pub-1 channel

   2. From the channel header, click the dropdown

   3. From the menu, select “Convert to Private Channel”

   4. In the “Convert public to a Private Channel” confirmation box, select “Yes..”

5. Verify access in backstage

   1. Launch incident backstage.

   2. Verify that the incident is visible in the incident list.

   3. Click on the incident.

   4. Verify that the incident details are accurately displayed.

Non participant cannot access a previously public incident converted to a private incident

Continue the test from above.

1. Log in as user-2.

2. Verify incident is not accessible in backstage.

   1. Launch incident backstage.

   2. Verify that the incident is not visible in the incident list.

Non participant cannot access a new private incident

1. As user-1, bring up the incident modal.

2. Give it the channel name “Pr-In-1”.

3. From the Playbook dropdown, select “Private-PB-1”.

4. Click “Start Incident”.

5. Do not add anyone else to the incident channel.

6. Log in as user-2.

7. Verify that the incident “Pr-In-1” is not visible in the incident RHS.

8. Launch incident backstage.

9. Verify that the incident “Pr-In-1” is not visible in the incident list.

Private incidents cannot be searched in backstage by non-participants.

Continue from the above test.

1. In the incident backstage, search for “Pub-1”.

2. Verify that “Pub-1” incident is not present in the search result.

3. Search for “Pr-In-1” in the backstage.

4. Verify that “Pr-In-1” incident is not present in the search result.

Public incidents are visible only within the incident’s team

1. As user-1, create a public incident in Team X.

2. Launch incident backstage.

3. Verify that the incident is visible in the incident list.

4. Switch to Team Y.

5. Launch the incident backstage.

6. Verify that the incident is not visible in the incident list.

7. As user-2, go to Team X.

8. Launch the incident backstage.

9. Verify that the incident is visible in the incident list.

10. Switch to Team Z.

11. Verify that the incident is not visible in the incident list.

Private incidents are visible only within the incident’s team

1. As user-1, create a private incident using playbook with private setting on.

2. Launch incident backstage.

3. Verify that the incident is visible in the incident list.

4. Switch to Team Y.

5. Launch the incident backstage.

6. Verify that the incident is not visible in the incident list.

7. As user-2, go to Team X.

8. Launch the incident backstage.

9. Verify that the incident is not visible in the incident list.

10. Switch to Team Z.

11. Verify that the incident is not visible in the incident list.

Public incidents cannot be searched in a different team’s incident backstage

1. As user-1, create a public incident “Pub C” in Team Y.

2. Launch incident backstage.

3. Verify that the incident is visible in the incident list.

4. Search for “Pub C” in the backstage.

5. Verify that “Pub C” is present in the search result.

6. Switch to Team X.

7. Launch the incident backstage.

8. Verify that the incident is not visible in the incident list.

9. Search for “Pub C” in the backstage.

10. Verify that “Pub C” is not present in the search result.

System administrator cannot see a private incident

1. As user-1, create a private incident “Pr A”.

2. Add a few checklist items.

3. Log in as sysadmin.

4. Launch incident backstage.

5. Verify that sysadmin cannot see “Pr A” in the incident list.

System administrator can join a private incident

Continue from the above test.

1. As sysadmin, use the permalink to the incident created in the above test.

2. Verify that sysadmin can successfully join the channel.

System administrator can view a private incident details

Continue from the above test.

1. As sysadmin, open up the incident RHS.

2. Verify that sysadmin can see the incident details for “Pr A” on the RHS.

3. Launch incident backstage.

4. Click on “Pr A”.

5. Verify that sysadmin can see the details of “Pr A” correctly in the details view in backstage.

System administrator can check and uncheck items in private incidents

Continue from the above test.

1. As sysadmin, open up the incident RHS to view the details of “Pr A”.

2. Check an item from the checklist.

3. Verify that sysadmin can successfully check an item off the list.

4. Uncheck an item from the checklist.

5. Verify that sysadmin can successfully uncheck an item off the list.

6. Launch the incident backstage.

7. Find and click “Pr A” to view the details.

8. Verify that the items checked off and unchecked are reflected properly in the graph.

System administrator can end a private incident

Continue from the above test.

1. As sysadmin, open the incident details of “Pr A” on the RHS.

2. Click “End Incident”.

3. Verify that the incident is ended successfully.

4. Launch the incident backstage.

5. Verify that the incident end state under “Status” is properly reflected in the list view.

6. Verify that the incident end state is properly reflected in the incident details view.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

A bot message is posted in incident channel when an item in checklist is checked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username checked off checklist item "[checklist item title]"

(0.2)

“Item checked” post in incident channel shows the username of user who checked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who checked off the checklist item.

(0.2)

Checking an item shows the timestamp to the right of the checklist item of when the item was checked off

Continue from the above test:

1. In the RHS, verify that the correct timestamp of when the item was checked off is displayed beside the checklist item title.

(0.2)

A bot message is posted in incident channel when an item in checklist is unchecked

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Check the item in step 5.

7. Uncheck the item.

8. In the incident channel, verify that a bot message similar to the following is posted:

@username unchecked item "[checklist item title]"

(0.2)

“Item checked” post in incident channel shows the username of user who unchecked the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who unchecked the checklist item.

(0.2)

A bot message is posted in incident channel when an item is removed from an incident checklist

1. As a normal user, start an incident.

2. The user is navigated to the incident channel.

3. Open the Incident Response RHS.

4. Find the incident started in step 1 and click to open details.

5. Create a checklist and add an item.

6. Remove the checklist item.

7. In the incident channel, verify that a bot message similar to the following is posted:

@username removed checklist item "[checklist item title]"

(0.2)

“Item removed from incident checklist” post in incident channel shows the username of who removed the item

Continue from the above test:

1. Verify that the bot message from the above test shows the username of the user who removed the checklist item.

(0.2)

“Item removed from incident checklist” post in incident channel shows the timestamp of when the item was removed

Continue from the above test:

1. Verify that the bot message from the above test shows the timestamp of when the item was removed from the checklist.

(0.2)

Items checked more than a day ago shows the day along with timestamp of when the item was checked

?? copied from v0.3. needs to be updated.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Clicking “Incidents & Playbooks Backstage” in the main menu opens Incidents backstage

1. Login as a normal user.

2. Go to Team-A.

3. Verify Incidents backstage is launched.

4. Verify the Incidents tab is highlighted on the backstage LHS.

5. Verify the backstage shows a list of incidents if they exist.

Backstage shows a tab for incidents

Continue from the above test.

1. Create incidents a-inc-1, a-inc-2, a-inc-3.

2. Note the time of incident creation for each incidents for later tests.

3. Verify that there is an “Incidents” tab in the backstage LHS in addition to the Playbooks tab.

Clicking on “Incidents” tab on backstage header highlights the Incidents tab

Continue from the above test.

1. Click on the “Incidents” tab on the LHS.

2. Verify that the highlight shifts from Playbooks to Incidents.

Clicking on “Incidents” tab on backstage header loads the list of all incidents

Continue from the above test.

1. Verify that clicking on the “Incidents” tab on the LHS loads the incidents list view.

2. Verify that all incidents from Team-A including a-inc-1, a-inc-2, a-inc-3 are in the list.

Yes

When no playbook exists, the backstage shows an empty playbook list

1. Ensure that there are no predefined playbooks yet.

2. Click on the playbook icon to open up the Playbooks backstage.

3. Verify that the main Playbooks view shows an empty playbook list.

4. Verify that a “New Playbook” button appears on the Playbooks backstage → is this not valid anymore?

(0.3)

A “delete playbook confirmation” dialog pops up when deleting a playbook

1. Go to Playbook backstage by clicking on the playbook icon on RHS.

2. From the list, find playbook “PB-2”.

3. Click “delete” for “PB-2”.

4. Verify that a confirmation pop up appears.

(0.3)

A “delete playbook” can be canceled

1. In the confirmation dialog click the “Cancel” button.

2. Verify that the playbook is not deleted and it is still there in the playbook list.

3. Switch to normal MM view.

4. Start an incident using any method.

5. In the incident interactive dialog, verify that the playbook still appears on the Playbook dropdown.

(0.3)

A playbook can be deleted from the backstage

1. Go to Playbook backstage by clicking on the playbook icon on RHS.

2. From the list, find playbook “PB-2”.

3. Click “delete” for “PB-2”.

4. In the confirmation dialog click “Confirm”.

5. Verify that the playbook no longer appears in the Playbooks list.

6. Switch to normal MM view.

7. Start an incident using any method.

8. In the incident interactive dialog, verify that the playbook no longer appears on the Playbook dropdown.

(0.3)

A deleted playbook does not affect previously started incident using that playbook

Continue from the above test.

1. In the incident RHS, find incidents I-3 and I-4 from above tests.

2. Verify that the checklists for I-3 and I-4 are not changed.

(0.3)

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Incidents list view displays the team name the incidents belong to

Incident list in the backstage displays the following columns: name, status, start timestamp, end timestamp and the commander

Continue from the above test.

1. Verify that the incidents backstage shows incident details including name, status, start date, end date and commander for each incident.

The incident list in the backstage shows all ongoing incidents that belong to the current team only

1. Switch to Team-B.

2. Create new incidents: Inc-a, Inc-b and Inc-c

3. Launch the incident backstage from within Team-B.

4. Verify that the incident list only shows the three incidents created in step 2.

5. Switch to Team-A.

6. Launch the incident backstage from within Team-A.

7. Verify that the incident list only shows the incidents created within Team-A.

8. Verify that the incident list does not show the incidents created in step 2.

9. Verify that the incident list shows all currently active incidents in the list.

The incident list in the backstage shows all inactive incidents that belong to the current team only

1. Switch to Team-B.

2. End incident Inc-b and Inc-c.

3. Note the incident end time for later tests.

4. Launch the incident backstage from within Team-B.

5. Verify that the incident list shows all three incidents including both active and inactive incidents.

6. Verify none of the incidents from Team-A are in the list.

7. Switch to Team-A.

8. End incident a-inc-2.

9. Note the incident end time for later tests.

10. Launch the incident backstage.

11. Verify that the incident list shows all incidents created and ended with Team-A.

Active incidents show an “Ongoing” status in the Status column in the backstage

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that incidents a-inc-1, and a-inc-3 show the status as “Ongoing” in the “Status” column.

3. Launch the incident backstage from Team-B.

4. Verify that incident Inc-a shows “Ongoing” status.

Inactive incidents (ended incidents) show an “Ended” status in the Statuscolumn in the backstage.

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that incident a-inc-2 shows the status as “Ended” in the “Status” column.

3. Launch the incident backstage from Team-B.

4. Verify that incidents Inc-b and Inc-c show “Ended” in the “Status” column.

The incident list in the backstage shows updated end timestamp for an incident

Continue from the above test.

1. Launch the incident backstage from Team-A.

2. Verify that the “End Date” for a-inc-2 is accurate.

An active incident shows “--” in End Date column

Continue from the above test.

1. Verify that a-inc-1 and a-inc-3 show “Ongoing” under “End Date”.

The incident list in the backstage shows updated commander for an incident

1. Go to the normal channel view.

2. Invite user-B to the incident channel of a-inc-3.

3. Launch incident RHS.

4. Click on a-inc-3 to see the details.

5. Change the commander to user-B.

6. Launch the incident backstage.

7. In the incident list, find a-inc-3.

8. Verify that user-B is listed in the “Commander” column.

The incident names in the “Name” column render as clickable links

Continue from the above test.

1. Verify that all the incident names in the “Name” column are rendered as clickable links.

Long incident name wraps in the “Name” column displaying the entire incident name

1. Start a new incident such that the number of characters exceed the column width of “Name”.

2. Launch the incident backstage.

3. In the incident list view, verify that the name of the incident wraps in the next line so that the entire incident name is visible.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

User cannot view incident summary if the user is not an incident member

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an active incident if the user is an incident member

From the incidents list in the backstage, user can navigate to incident summary upon clicking the incident name of an inactive incident

Incident summary page displays the incident name as the page heading

Incident summary page has a “<“ button that brings a user back to the incidents list view

Incident summary page bears a link to the incident channel

Hovering on the incident channel link shows a tooltip suggesting that it’s a shortcut to the incident channel

Clicking on incident channel link of active incidents brings the user to the incident channel

Incident summary shows the current status of the incident

Incident summary shows the current commander of the incident

For active incidents, incident summary shows the duration of how long the incident has been active for

For inactive incidents, incident summary shows the duration of how long the incident remained active for before the incident ended

For active incidents, incident summary shows the current number of members in the incident channel

For an inactive incident, incident summary shows the number of members in the incident channel when the incident was ended

For active incident, incident summary shows the total number of messages in the incident channel

For inactive incident, incident summary shows the total number of messages in the archived channel of the inactive incident

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Canceling new playbook does not save the playbook

1. Click on “New Playbook” button.

2. Enter “Playbook 1” for Playbook name.

3. Add a few checklist items.

4. Click “Cancel”.

5. Verify that the user is brought back to the Playbooks view on the backstage.

6. Verify that “Playbook 1” does not exist as a playbook.

7. Repeat steps 1 - 3.

8. Without saving, from the top, click on the '<' button.

9. Verify that the user is brought back to the Playbooks view on the backstage.

10. Verify that “Playbook 1” does not exist as a playbook.

(0.3)

Clicking on “ ← Back to Mattermost” brings user back to normal channel

Continue from the above test.

1. From the Playbooks backstage, click on the “ ← Back to Mattermost” button.

2. Verify that the user is brought back to Mattermost’s channel view.

(0.3)

A new playbook can be saved from the Playbook backstage

Continue from the above test.

1. Click on the “New Playbook” button.

2. Enter “PB-1” for Playbook name.

3. Add “Started”, “In progress”, and “Analysis” as checklist items.

4. Click “Save Playbook”.

5. Verify that the backstage “Playbooks” view shows PB-1 listed.

(0.3)

A playbook can be edited from the Playbook backstage

Continue from the above test.

1. Click on “edit” for PB-1 in the backstage.

2. Click “edit” for the checklist.

3. Add a new item “Resolved” at the end of the checklist.

4. Click “done”.

5. Click “Save Playbook”.

6. Click “edit” for PB-1 again.

7. Verify that the list shows all 4 items: “Started, “In progress”, “Analysis” and “Resolved”.

(0.3)

Playbook can be edited to change public incident creation setting to private

Continue from the above tests.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find playbook “PB 1”.

4. Click “Edit”.

5. Disable the button to “Create Public Incident”.

6. Click “Save Playbook”.

7. Switch to normal channel view.

8. Bring up the incident creation dialog.

9. Create incident “Incident PB3”.

10. From the playbook dropdown, select “PB 1”.

11. Click “Start Incident”.

12. Verify that the incident created has the correct checklist items.

13. Log in as user-2.

14. Launch incident backstage.

15. Verify user-2 cannot see “Incident PB3” in the incident list.

Playbook can be edited to change private incident creation setting to public

Continue from the above tests.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find playbook “PB 2”.

4. Click “Edit”.

5. Enable the button to “Create Public Incident”.

6. Click “Save Playbook”.

7. Switch to normal channel view.

8. Bring up the incident creation dialog.

9. Create incident “Incident PB4”.

10. From the playbook dropdown, select “PB 2”.

11. Click “Start Incident”.

12. Verify that the incident created has the correct checklist items.

13. Log in as user-2.

14. Launch incident backstage.

15. Verify user-2 can see “Incident PB4” in the incident list.

Playbook creator can edit playbook

1. Log in as user-0.

2. Launch playbook backstage (using any method).

3. Click the “New Playbook” button.

4. Enter “P1” for Playbook Name.

5. Enter stage “1”.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Bring up the incident creation modal (using any method).

9. Verify that the playbook dropdown has “P1”.

10. From the playbook backstage, find P1.

11. Click P1 to edit the playbook.

12. Change the playbook name to “PB-Private”.

13. Click “Save Playbook”.

14. Verify that the playbook in the backstage shows up as “PB-Private”.

15. Switch to a normal channel.

16. Bring up the incident creation modal.

17. Verify that the playbook dropdown does not have “P1” but “PB-Private”.

Any playbook member can edit playbook to add a new member

Continue from the above test.

1. From the playbook backstage, find PB-Private.

2. Click on the playbook to open the “Edit Playbook” view

3. In the “Members” field, add @user-1.

4. Save playbook.

5. Log in as user-1.

6. Bring up the incident creation modal.

7. Verify that user-1 can see “PB-Private” in the playbook dropdown.

8. Launch playbook backstage.

9. Find “PB-Private” and click to open the “Edit Playbook” view.

10. In the “Members” field, add @user-2.

11. Click “Save Playbook”.

12. As user-2, verify that “PB-Private” is available in the playbook dropdown of the incident creation modal.

Any playbook member can edit playbook to remove an existing member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” and click to open the “Edit Playbook” view.

4. From the “Members” field, remove @user-1.

5. As user-1, verify that “PB-Private” is not available in the playbook dropdown of the incident creation modal.

A playbook member cannot edit playbook to remove all the members of the playbook

Continue from the above test.

1. As user-2, launch playbook backstage.

2. Find “PB-Private” and click to open the “Edit Playbook” view.

3. From the “Members” field, remove all users including user-0 and user-2.

4. Verify that the playbook cannot be saved.

5. Add user-1 to the Members field.

6. Save playbook.

Any playbook member can edit playbook to change the private/public incident setting

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Toggle the “Create Public Incident” to enable the setting.

4. Save playbook.

5. Click the playbook again to view the edit mode.

6. Verify that the “Create Public Incident” setting is enabled.

Any playbook member can edit playbook to add a new stage

Continue from the above test.

1. As user-1, launch the playbook backstage.

2. Find “PB-Private” and click to open the edit view.

3. Click + Add new stage.

4. Change the stage name from “New stage” to “Stage 2”.

5. Save playbook.

6. Click the playbook again to view the edit mode.

7. Verify that the playbook contains “Default Stage” and “Stage 2” for stages.

A playbook can have multiple stages with the same stage name

1. As user-1, launch the playbook backstage.

2. Click “New Playbook”.

3. Enter “PB-New” for Playbook Name.

4. Enter “Stage 1” as the 1st stage name.

5. Click “Add new stage”.

6. Enter “Stage 1” as the 2nd stage name.

7. ??

A playbook member can edit playbook to add a new checklist item to a stage added by themself

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Under “Stage 1”, click “Add new checklist item”.

4. Enter “S1-C1” as the checklist item.

5. Click “Save Playbook”.

6. From the playbook list in the backstage find “PB-Private”.

7. Click to view the edit view.

8. Verify that “Stage 1” has “S1-C1” as the checklist item.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the checklist item as been added to “Stage 1”.

Any playbook member can edit playbook to add a new checklist item to a stage added by a different member

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 1”, click “Add new checklist item”.

6. Enter “S1-U2” as the checklist item.

7. Click “Save Playbook”.

8. From the playbook list in the backstage find “PB-Private”.

9. Click to view the edit view.

10. Verify that “Stage 1” has “S1-U2” as the checklist item.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify the checklist item has been added to “Stage 1”.

A playbook member cannot edit playbook such that the playbook name field is empty

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Delete the playbook name “PB-Private”.

5. Verify that the “Save Playbook” button becomes deactivate.

6. Click on the “Save Playbook” button.

7. Verify that the playbook cannot be saved, and the user is still in the edit view.

8. Enter a space in the playbook name field.

9. Repeat steps 5-7.

Any playbook member can reorder the checklist items within a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-C1” and drag it under “S1-U2”.

5. Verify that “S1-C1” is successfully moved under “S1-U2”.

6. Click the hamburger icon next to “S1-U2” and drag it above “S1-C1”.

7. Verify that “S1-U2” is successfully moved under “S1-C1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the reordering was successful.

Any playbook member can reorder the checklist items within different stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Under “Stage 1”, click the hamburger icon next to the checklist item “S1-U2” and drag it under the 2nd stage “Stage 2”.

5. Verify that “S1-U2” is successfully moved under “Stage 2”.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the reordering was successful.

A playbook member cannot reorder the playbook such that the checklist items can be pulled outside a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to the checklist item “S1-C1” and drag it right above “Stage 1”.

5. Verify that the checklist item does not move from its original spot under “Stage 1”.

6. Drag and drop “S1-C1” outside stage area.

7. Verify that the checklist item does not move from its original spot under “Stage 1”.

8. Switch to a normal channel view.

9. Start an incident with “PB-Private” to verify the order of the checklist items is normal.

Any playbook member can delete a checklist item

Continue from the above test.

1. Log in as user-1.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Click to open edit view.

5. Under “Stage 2” find “S1-U2” and hover the mouse over so that the “X” appears next to it.

6. Click the “X” for “S1-U2”.

7. Verify that the checklist item “S1-U2” is deleted.

8. Click “Save Playbook”.

9. Click “PB-Private” to go to edit view.

10. Verify that the checklist item “S1-U2” is not available in any stage.

11. Switch to a normal channel view.

12. Start an incident with “PB-Private” to verify that “S1-U2” is not available in any stage.

Any playbook member can reorder the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Add a new stage “Stage 3”.

5. Click the hamburger icon next to “Stage 1” and drag it below “Stage 3”.

6. Verify that “Stage 1” is successfully moved under “Stage 3”.

7. Click the hamburger icon next to “Stage 3” and drag it above “Stage 2”.

8. Verify that “Stage 3” is successfully moved above “Stage 2”.

9. Switch to a normal channel view.

10. Start an incident with “PB-Private” to verify the stage reordering was successful.

A playbook member cannot reorder the playbook such that the stages can be nested

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the hamburger icon next to “Stage 1” and drag it under “Stage 3” to align it within the checklist items of “Stage 3”.

5. Verify that “Stage 1” is not moved from its original position.

6. Switch to a normal channel view.

7. Start an incident with “PB-Private” to verify the stage reordering was not successful.

Any playbook member can delete a stage

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 3”.

5. Verify that “Stage 3” is deleted.

6. Save playbook.

7. Switch to a normal channel view.

8. Start an incident with “PB-Private” to verify that the incident created does not have “Stage 3”.

A playbook member cannot delete the last remaining stage such that the playbook has no stages at all

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-Private” from the playbook list.

3. Click to open edit view.

4. Click the “X” button next to “Stage 1”.

5. Click the “X” button next to “Stage 2”.

6. Verify that the “Save Playbook” button becomes inactive.

7. Click on the “Save Playbook”.

8. Verify that the user is still in the edit view of “PB-Private” and the action was not saved.

Any playbook member can delete playbook

Continue from the above test.

1. Log in as user-2.

2. Launch playbook backstage.

3. Find “PB-Private” from the playbook list.

4. Under “Actions” for the playbook, click the ellipses.

5. Click “Delete”.

6. Verify that a “Confirm Playbook Deletion” modal appears.

7. Click “Delete Playbook”.

8. Verify that “PB-Private” is not present in the playbook list.

9. Switch to a normal channel view.

10. Open up an incident modal.

11. Verify that “PB-Private” is not available in the playbook dropdown.

A playbook member can create playbook with a large number of stages

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Click “Add new stage” 20 or more times to create a long list of stages.

6. Click “Save Playbook”.

7. Switch to a normal channel.

8. Start an incident using “PB-N1” as the playbook.

9. Verify that the Stages dropdown in the incident RHS has all the stages added in step 5.

10. Verify the Stage dropdown displays the stages properly with scrollbar.

A playbook member can create playbook with a large number of checklist items within the stages

Continue from the above test.

1. Launch playbook backstage.

2. Find “PB-N1” and click to go to edit view.

3. For the first stage, add at least 20 checklist items.

4. Save playbook.

5. Switch to a normal channel.

6. Start an incident using “PB-N1” as the playbook.

7. In the incident RHS, ensure that the first stage is selected from the dropdown.

8. Verify that all the checklist items added in step 3 are available.

The components in the playbook edit page render normally

Verify the appearance of the components in the playbook edit page.

Slash commands

A newly added checklist item in a playbook has a slash command field

1. Log in as user-1.

2. Launch playbook backstage.

3. Click “New Playbook”.

4. Enter “PB-N1” as the playbook name.

5. Under the preexisting “Default Stage”, click “Add new checklist item”.

6. Enter “I-1” as the checklist item name.

7. Hit enter.

8. Verify that two new fields, one for slash command, and the other for step description appear underneath.

9. Add more checklist items.

10. Verify that every new checklist item added has the slash command and step description fields.

A playbook can be edited to add a slash command to any checklist item

Continue from the above test.

1. Type /away in the slash command field for one of the checklist items.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

4. Type other valid slash commands in the fields for other checklist items.

5. Verify that each slash command filed can successfully save the slash commands.

A checklist item configured with a slash command shows the slash command for that item in the RHS

Continue from the above tests.

1. Switch to a normal channel.

2. Start a new incident using “PB-N1”.

3. Open the incident RHS.

4. Verify that all checklist items configured with slash commands in the above tests, show the slash command under the checklist item.

Focusing in the slash command field brings up a list of existing slash commands

1. Log in as user-1.

2. Launch playbooks backstage.

3. Click “New Playbook”.

4. Enter “PBa” as the playbook name.

5. Edit the name of the default stage to “S1”.

6. Click “Add new stage”.

7. Enter “C1” for checklist item name.

8. Click the slash command input field.

9. Verify that a autocomplete list of all available slash commands pop up as soon as entering /.

Selecting from the slash command list auto completes the slash command

Continue from the above tests.

1. In the slash command field, enter /.

2. From the autocomplete list, click /dnd.

3. Verify that the slash command field is autocompleted with /dnd.

The slash command field can be left empty

Continue from the above tests.

1. Remove the contents of “C1” so that the slash command field is empty.

2. Click “Save Playbook”.

3. Verify that the playbook is saved successfully.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Users who have access to a playbook can limit playbook access to specific users

1. Log in as team admin.

2. Create a playbook “TA 1”.

3. Add checklist items “1”, “2” and “3”.

4. Invite user-2 in the playbook.

5. Click “Save Playbook”.

6. Login as user-1.

7. Launch playbook backstage.

8. Verify that user-1 cannot find “TA 1” in the backstage.

9. Switch to normal channel view.

10. Bring up the incident creation dialog.

11. Verify that user-1 cannot find “TA 1” in the playbook dropdown.

12. Login as user-2.

13. Launch playbook backstage.

14. Verify that user-1 can find “TA 1” in the backstage.

15. Switch to normal channel view.

16. Bring up the incident creation dialog.

17. Verify that user-1 can find “TA 1” in the playbook dropdown.

Any playbook member can edit users who have access to playbook

Continue from the above test.

1. As team admin, launch playbook backstage.

2. Find “TA 1”.

3. Click “Edit”.

4. Invite user-1 to manage the playbook “TA 1”.

5. Click “Save Playbook”.

6. Verify that user-1 can now find “TA 1” in the backstage.

7. Switch to normal channel view.

8. Bring up the incident creation dialog.

9. Verify that user-1 can now find “TA 1” in the playbook dropdown.

10. Launch the playbook backstage.

11. Find and click on “TA-1”.

12. Remove user-2 from the members list of the playbook.

13. As user-2, verify that “TA-1” is no longer available in the incident creation modal.

14. As user-2, verify that “TA-1” is no longer available in the team’s playbook backstage.

15. As user-1, add user-2 to the playbook again.

16. As user-2, verify that “TA-1” is now available in the incident creation modal.

17. As user-2, verify that “TA-1” is available in the playbook backstage.

User removed by any member of a playbook can no longer access playbook

Continue from the above test.

1. As user-1, launch playbook backstage.

2. Find “TA 1”.

3. Click “Edit”.

4. Remove team admin from the playbook “TA 1”.

5. Click “Save Playbook”.

6. Verify that user-2 cannot find “TA 1” in the backstage.

7. Switch to normal channel view.

8. Bring up the incident creation dialog.

9. Verify that team admin cannot find “TA 1” in the playbook dropdown.

Playbook member can limit playbook access to specific users

1. Log in as team admin.

2. Create a playbook “Sa 1”.

3. Add checklist items “a”, “b” and “c”.

4. Invite user-2 in the playbook.

5. Click “Save Playbook”.

6. Login as user-1.

7. Launch playbook backstage.

8. Verify that user-1 cannot find “Sa 1” in the backstage.

9. Switch to normal channel view.

10. Bring up the incident creation dialog.

11. Verify that user-1 cannot find “Sa 1” in the playbook dropdown.

12. Login as user-2.

13. Launch playbook backstage.

14. Verify that user-2 can find “Sa 1” in the backstage.

15. Switch to normal channel view.

16. Bring up the incident creation dialog.

17. Verify that user-2 can find “Sa 1” in the playbook dropdown.