Page Comparison

0.0

Initial version

10/06

1.0

Test Completed (for v1.0 updated features)

10/14

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

1

/incident is available as slash command option in mobile

1. Go to the main channel view in mobile app.

2. Under the main input box, tap the / icon.

3. Verify that the slash command suggestion box pops up.

4. Scroll down.

5. Verify that incident slash command is available.

Pass

2

Typing /incident should load the available slash command options

• In mobile

  1. In the main channel input box, type /incident

  2. In the pop up for incident [command], verify that the available slash command options are shown.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

3

The slash command options in the popup is scrollable

• In mobile

  1. In the main channel input box, type /incident

  2. Tap the /incident [command] option in the pop up.

  3. Verify the pop up changes to the available command options.

  4. Touch and scroll the pop up options up and down.

  5. Verify that all options appear normally.

• In webapp, scroll + up/down arrows work

  1. Type /incident with a space.

  2. Verify the pop up changes to the available command options.

  3. Verify that the pop up can be scrolled up and down.

  4. Verify that using the up-down arrow keys highlight the intended slash command option.

Pass

4

“Execute current command” is highlighted by default in pop up options

• In mobile - ? There is no “Execute current command”

• In webapp

  1. Type /incident (with a space).

  2. When the slash command pop up changes to the available options, verify that “Execute Current Command” is highlighted by default.

Pass in webapp

https://mattermost.atlassian.net/browse/MM-29629

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

5

/incident start is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that start is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

6

Issuing /incident start opens up the incident creation screen

• In mobile

  1. Issue an /incident start command by typing the command in the main input box and tapping the ‘send’ icon.

  2. Verify that the main screen changes into the ‘Incident Details’ incident creation screen.

• In webapp

  1. Issue an /incident start command.

  2. Verify that the incident creation modal opens up.

Pass

https://mattermost.atlassian.net/browse/MM-29633

7

Incident creation screen on mobile shows all the fields as the the incident creation page on webapp

• In mobile

  1. Issue an /incident start command to open up the incident creation screen.

  2. Verify that the following fileds are present:

     1. Commander

     2. “Create a playbook” link

     3. Playbook dropdown

     4. Incident Name

     5. Incident Description

Pass

8

Tap “Create a playbook” link in incident creation screen takes the user to the playbook creation screen?

• In mobile - continue from the above test

  1. Tap on the “Create a playbook” link in the incident creation screen.

  2. Verify that the user is taken to the playbook creation screen (in mobile webapp?)

Fail

Says “Link not found in server”

https://mattermost.atlassian.net/browse/MM-29380

9

Starting an incident requires a playbook to be selected

• In mobile

  1. Issue /incident start command to open up the incident creation screen.

  2. Do not select any playbook from the “Playbook” dropdown.

  3. Enter “MI-1” as the incident name.

  4. Tap “Start Incident”.

  5. Verify that the incident did not start.

  6. Verify that the incident creation screen is still open.

  7. Verify that an error message notifies the user that Playbook “field is required”.

Pass

10

Starting an incident requires an incident name

• In mobile - continue from the above test

  1. Select any playbook from the “Playbook” dropdown.

  2. Leave the incident name field empty.

  3. Tap “Start Incident”.

  4. Verify that the incident did not start.

  5. Verify that the incident creation screen is still open.

  6. Verify that an error message notifies the user that Incident Name “field is required”.

Pass

11

User can start an incident on mobile app using /incident start command

• In mobile - continue from the above test

  1. In the incident creation screen, ensure that a playbook is selected and an incident name has been entered.

  2. Verify that the incident has started.

  3. Verify that the incident creation screen is no longer open.

12

User can start an incident on mobile using post menu option

Probably not valid?

Pass

13

When incident is started from mobile app, a system message is posted in the channel from where incident is started

• In mobile

  1. As user-1, go to channel A.

  2. Select any playbook.

  3. Enter “M-In-a” as the incident name.

  4. Start the incident.

  5. Once an incident is started and the incident creation screen is closed, verify that the user is back in channel A.

  6. Verify that the user sees a system message that “Incident M-In-a started in ~M-In-a”.

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

14

/incident info is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that info is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

15

On mobile app, issuing /incident info command in incident channel posts an ephemeral message including the incident details

1. As user-1, create a new playbook with only 1 Stage S1 and tasks T1 and T2.

2. Start incident “M2”.

3. Navigate to the “M2” incident channel.

4. Issue an /incident info command.

5. Verify that a system message displays the following accurately:

   1. Incident name: M2

   2. Duration of incident

   3. Commander of incident: user-1

   4. Stage: S1

   5. Tasks: T1, T2

Is it an ephemeral message or a normal bot message that gets posted? Looks like a bot message

16

On mobile, Pass

16

On mobile, issuing /incident info command in non-incident channel posts a message that the info is only available in incident channels

1. Go to a non-incident channel.

2. Issue an /incident info command.

3. Verify that a system message notifies the user that the details of an incident can only be viewed in an incident channel.

Pass

17

On webapp, issuing /incident info command in incident channel opens incident RHS if not already open

1. Log in as user-1 in webapp.

2. Go to the “M2” incident channel.

3. Ensure that the incident RHS is not open (close if needed).

4. Issue an /incident info command.

5. Verify that the incident RHS opens showing the incident details.

Pass

Yes - E2E test PR

18

On webapp, issuing /incident info command in incident channel with a non-incident RHS open, opens incident RHS

Continue from the above test

1. From the channel header, click the “Pinned posts” icon.

2. Issue an /incident info command.

3. Verify that the incident RHS opens showing the incident details.

4. From the channel header, click the “Saved posts” icon.

5. Issue an /incident info command.

6. Verify that the incident RHS opens showing the incident details.

7. From the channel header, click the “@ Recent mentions” icon.

8. Issue an /incident info command.

9. Verify that the incident RHS opens showing the incident details.

10. From the channel header, search for a random term to trigger the search RHS.

11. Issue an /incident info command.

12. Verify that the incident RHS opens showing the incident details.

Pass

Yes - https://github.com/mattermost/mattermost-plugin-incident-response/blob/master/tests-e2e/cypress/integration/frontstage/slash_command/info_spec.js

19

On webapp, issuing /incident info command in incident channel with incident RHS open posts a message that the incident details (or list?) is already open in incident RHS

Continue from the above test

1. Ensure that the incident RHS is open.

2. Issue an /incident info command.

3. Verify that the incident bot notifies the user that the “incident details are already open in the RHS”.

Pass

Yes - https://github.com/mattermost/mattermost-plugin-incident-response/blob/master/tests-e2e/cypress/integration/frontstage/slash_command/info_spec.js

20

On webapp, issuing /incident info command in non-incident channel posts a message that the info is only available in incident channels

Continue from the above test

1. On webapp, navigate to a non-incident channel.

2. Issue an /incident info command.

3. Verify that the incident bot notifies the user that the incident details can only be viewed in an incident channel.

Pass

Yes - E2E test PR

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

21

/incident list is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that list is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

22

Issuing an /incident list command when there is no incidents (fresh server) should show “There are no ongoing incidents in [team name] team” message for the user.

0. Install a fresh server.

• In mobile

  1. Go to any channel and issue an /incident list command.

  2. Verify that the incident bot notifies the user that there are no ongoing incidents for the current team.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

23

Issuing /incident list in non-incident channel shows all on going incidents

1. Start a few incidents (< 10 incidents).

2. In mobile, go to a non-incident channel.

3. Issue an /incident list command.

4. Verify that the incident bot posts a message with the incidents started in step 1 with the details including:

   1. link to the incident channel

   2. active stage

   3. duration

   4. commander of the incident

5. In webapp, repeat steps 3-4.

Pass

24

Issuing /incident list in incident channel shows all on going incidents

Continue from the above test

1. In mobile, go to an incident channel.

2. Issue an /incident list command.

3. Verify that the incident bot posts a message with the incidents started in step 1 with the details including:

   1. link to the incident channel

   2. active stage

   3. duration

   4. commander of the incident

4. In webapp, repeat steps 2-3.

25Pass

Issuing /incident list does not in incident channel shows all ongoin incidents if there are <= 10 ongoing incidents

Pass

Issuing /incident list show 10 of the most recent ongoing incidents if there are >10 ongoing incidents

Pass

25

Issuing /incident list does not show any incident that has been ended

Continue from the above test

1. End an incident from the above test.

2. In mobile, go to an incident channel.

3. Issue an /incident list command.

4. Verify that the incident bot does not list the incident that was ended in step 1.

5. End another incident.

6. Repeat steps 3-4

7. In webapp issue an /incident list command.

8. Verify that the incidents ended in steps 1 and 5 are not listed by the incident bot.

Pass

26

Issuing /incident list shows all incidents that have been restarted

Continue from the above test

1. Restart the incidents ended in the above test.

2. In mobile, issue an /incident list command.

3. Verify that the incident bot lists all incidents.

4. In webapp, issue an /incident list command.

5. Verify that the incident bot lists all incidents.

Pass

27

Issuing /incident list does not show incidents from another team

1. In mobile, as user-1 switch to a different team, Team Y; create one if needed.

2. Issue an /incident list command.

3. Verify that the incident bot notifies that there are no ongoing incidents in Team Y.

4. In webapp, switch to Team Y.

5. Repeat steps 2-3.

Pass

28

Issuing /incident list does not show incidents for another user

1. Log in as user-2 (a user that has not started any incident).

2. Issue an `/incident list` command in any channel.

3. Verify that the incident bot notifies that there are no ongoing incidents in the current team.

Pass

29

/incident list should show all ongoing incidents for a large number of incidents

1. Create a large number of incidents (>50).

2. In mobile, issue an `/incident list` command.

3. Verify that the incident bot lists all ongoing incidents.

4. In webapp, repeat steps 2-3.

Invalid because only 10 of the most recent ongoing incidents are shown.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

30

/incident commander is available as a slash command in mobile app

• In mobile

  1. Type /incident in the main input box.

  2. In the slash command options pop up, verify that commander is an available options.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

31

Issuing /incident commander in non-incident channel shows “Commander is only available in incident channel”

• In mobile

  1. Go to a non-incident channel

  2. Issue an /incident commander command.

  3. Verify that the incident bot notifies the user that the commander can only be viewed in the incident channels.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

32

Issuing /incident commander in incident channel shows the current commander of the incident

In mobile

1. Go to an incident channel

2. Issue an /incident commander command.

3. Verify that the incident bot posts a message including the current commander of the incident.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

33

/incident commande [@user-name] in non-incident channel shows “You can only change the commander from the incident channel” message

• In mobile

  1. Go to a non-incident channel.

  2. Issue an /incident commander @sysadmin command.

  3. Verify that the incident bot posts a message that the commander can only be changed from the incident channel.

• In webapp

  1. Repeat mobile test steps above in webapp.

Pass

34

/incident commander [@user-name] in incident channel with [@user-name] not added to incident channel shows [@user-name] must be a part of this channel.. message

• In mobile

  1. Go to an incident channel.

  2. Issue an /incident commander @user-2 command.

  3. Verify that the incident bot posts a message that user-2 must be an incident member to make them the commander.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

35

/incident commander [@user-name] in incident channel with [@user-name] already added to the channel changes the commander to that user

• In mobile

1. Go to an incident channel.

2. Invite user-2 to the incident channel.

3. Issue an /incident commander @user-2 command.

4. Verify that the incident bot posts a message that user-1 changed the incident commander from user-1 to user-2.

5. Verify that the new commander of the incident is user-2.

• In webapp

  1. Repeat mobile test steps above in webapp

Pass

36

/incident commander [@user-name] in incident channel with [@user-name] of the current commander shows “`[@user-name]` is already the commander of this incident”

Continue from the above test

• In mobile

  1. In the same incident channel as the above test, issue another /incident commander @user-2 command.

  2. Verify that the incident bot posts a message that user-2 is already the commander of the incident.

  3. Verify that user-2 is still the commander of the incident.

• In webapp

  1. Repeat mobile. tetst steps above in webapp.

Pass

37

An incident channel member can see a dropdown for users under “Commander” in details view

Covered in older release tests (will be reorganized later).

38

Incident channel members are listed in the user dropdown list

39

An incident channel member can be searched by a commander in the user dropdown list

(0.3)

40

A non-incident channel member cannot be searched in the user dropdown list

41

A bot message is posted to the incident channel when commander is changed.

1. Log in as user A.

2. Start a new incident.

3. Add user B to the incident channel.

4. From the incident RHS, find the incident started in step 2 and see the details.

5. From the user dropdown, pick user B.

6. Verify that in the incident channel, the Incident bot posted a message that “user A changed the incident commander from user A to user B”.

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

42

Clicking on the incident icon when there are no incidents, opens the incident RHS to the “welcome” message

1. Install a fresh server.

2. As user-1, click the incident icon in the channel header.

3. Verify that the incident RHS opens with a “welcome” message to the user.

43

With incidents ongoing, clicking on the incident icon when user is in non-incident channel RHS opens up with the incident list for the user

Continue from the above test.

1. Create a playbook.

2. Start a new incident “I1”.

3. Navigate to a non-incident channel.

4. From the channel header, click the incident icon.

5. Verify that the incident RHS opens up with the list view showing the incidents user-1 is a member of.

44

With incidents ongoing, clicking on the incident icon when user is in incident channel with RHS open, shows the current incident details

Continue from the above test.

1. Go to the incident channel for “I1”.

2. Click the incident icon in the channel header.

3. Verify that the incident RHS opens with the incident list view, listing the incident created in the above test.

45

Switching from one incident channel to another incident channel with the RHS details view open will switch to the selected incident’s detail view in the RHS

Continue fromt he above test.

1. Create incident “I2”.

2. Click the incident icon in the channel header to open the details view for “I2” in incident RHS.

3. Navigate to incident channel for “I1”.

4. Verify that the details view in the incident RHS switches from “I2” to that of “I1”

46

The incident RHS list view has a “Start Incident” button in the RHS header

Continue from the above test.

1. Open the incident RHS list view.

2. Verify that the RHS header has a “Start Incident” button.

47

The “Start Incident” button in the RHS header opens up the incident creation modal

Continue from the above test.

1. Click the “Start Incident” button in the incident RHS header.

2. Verify that the incident creation modal opens.

48

The incident RHS list view has a “Create Playbook” button

Continue from the above test.

1. Verify that the incident RHS has a “Create Playbook” button.

Is this valid?

49

A user can only see the incident he/she is a member of in the incident RHS list view

User is able to change a private incident name in mobile app

1. Log in as user-1.

2. Start a private incident “I-P-1”.

3. Go to the incident backstage and verify the incident is created.

4. In mobile app, go to the “I-P-1” incident channel.

5. Tap the “chevron” icon next to the channel name in the channel header.

6. Tap “Edit Channel”.

7. Edit the channel name to “New I1”.

8. From webapp, go to the incident backstage.

9. Verify that incident “I-P-1” does not exist.

10. Verify that the new incident “New I1” exists and ongoing.

Pass

https://mattermost.atlassian.net/browse/MM-29698

User is able to change a public incident name in mobile app

1. Start a public incident “Public I1”.

2. Go to the incident backstage and verify the incident is created.

3. In mobile app, go to the “Public I1” incident channel.

4. Tap the “chevron” icon next to the channel name in the channel header.

5. Tap “Edit Channel”.

6. Edit the channel name to “New public 1”.

7. From webapp, go to the incident backstage.

8. Verify that incident “Public I1” does not exist.

9. Verify that the new incident “New public 1” exists and ongoing.

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

42

Clicking on the incident icon when there are no incidents, opens the incident RHS to the “welcome” message

1. Install a fresh server.

2. As user-1, click the incident icon in the channel header.

3. Verify that the incident RHS opens with a “welcome” message to the user.

Pass

43

With incidents ongoing, clicking on the incident icon when user is in non-incident channel RHS opens up with the incident list for the user

Continue from the above test.

1. Create a playbook.

2. Start a new incident “I1”.

3. Navigate to a non-incident channel.

4. From the channel header, click the incident icon.

5. Verify that the incident RHS opens up with the list view showing the incidents user-1 is a member of.

Pass

44

With incidents ongoing, clicking on the incident icon when user is in incident channel with RHS open, shows the current incident details

51

“Click here to see all incidents” link in the bottom of the incident RHS list view takes the user to the incident backstage

Continue from the above test.

1. Go to the incident channel for “I1”.

2. Click the incident icon in the channel header to open the incident RHS.

3. Verify that there are no ongoing incidents.

4. Create a new incident “I4” (create a playbook if necessary).

5. Click the incident icon in the channel header.

6. Verify that the incident RHS only lists “I4” in the list view.

7. Create a new incident “I5”.

8. Verify that the incident RHS lists “I4” and “I5” in the list view.

50

A user cannot see a public incident in the incident RHS if he/she is not a member of that incident

1. As user-1, create a public incident “PI1”.

2. Do not add any other user to the incident channel.

3. Log out and log in as user-2.

4. Open up the incident RHS list view.

5. Verify that incident “PI1” is not listed in the incident RHS for user-2.

1. As user-1, open up the incident RHS opens with the incident list view, listing the incident created in the above test.

Pass

45

Switching from one incident channel to another incident channel with the RHS details view open will switch to the selected incident’s detail view in the RHS

Continue fromt he above test.

1. Create incident “I2”.

2. Click the incident icon in the channel header to open the details view for “I2” in incident RHS.

3. Navigate to incident channel for “I1”.

4. Verify that the details view in the incident RHS switches from “I2” to that of “I1”

Pass

46

The incident RHS list view has a “Start Incident” button in the RHS header

Continue from the above test.

1. Open the incident RHS list view.

2. Verify that there is a “Click here to see all incidents” link in the RHS.

3. Click the link.

4. Verify that the incidents backstage page loads listing all the incidents that user-1 is a member of.

52

The incident RHS details view has a “back arrow” that brings a user to the incident list view in RHS

Continue from the above test.

1. As user-1, go to the incident channel for “PI1”.

2. If the incident details view is not already open in the incident RHS, open it up.

3. Verify that in the incident RHS header, there is a “<“ arrow.

4. Click the “<“ arrow.

5. Verify that the incident details view switches to the incidents list view.

53

1. the RHS header has a “Start Incident” button.

Pass

47

The “Start Incident” button in the RHS header opens up the incident creation modal

Continue from the above test.

1. Click the “Start Incident” button in the incident RHS header.

2. Verify that the incident creation modal opens.

Pass

48

The incident RHS list view has a “Create Playbook” button

Continue from the above test.

1. Go to an incident channel.

2. Open the incident RHS list view.

3. On the top right of the incident RHS, click the three dot menu.

4. Verify a “Create Playbook” button exists.

5. Click the button.

6. Verify that the playbook backstage opens.

Pass

49

A user can only see the incident he/she is a member of in the incident RHS list view

Continue from the above test.

1. As Log in as user-1, start another incident “I-a1”3.Go to

2. Click the incident channel for “I-a1”.Open up the incident list view in the incident icon in the channel header to open the incident RHS.

3. Verify that in the list view, the entry for “I-a1” is highlighted.

4. Verify that none of the other incidents that user-1 is a member of is highlighted.

54

“Go to Incident Channel” takes the user to the associated incident channel

Continue from the above test.

1. From the incident list in the RHS, find the entry for “PI1”.

2. Click the “Go to incident Channel” button for “PI1”.

3. Verify that the incident channel for “PI1” is loaded.

4. Verify that the incident RHS changes to the incident details views for “PI1”.

55

“Go to Incident Channel” from a different team, takes the user to the correct incident channel in the right team.

Not sure about this behavior. Right now it shows a “Channel not found” error.

56

1. there are no ongoing incidents.

2. Create a new incident “I4” (create a playbook if necessary).

3. Click the incident icon in the channel header.

4. Verify that the incident RHS only lists “I4” in the list view.

5. Create a new incident “I5”.

6. Verify that the incident RHS lists “I4” and “I5” in the list view.

Pass

50

A user cannot see a public incident in the incident RHS if he/she is not a member of that incident

1. As user-1, create a public incident “PI1”.

2. Do not add any other user to the incident channel.

3. Log out and log in as user-2.

4. Open up the incident RHS list view.

5. Verify that incident “PI1” is not listed in the incident RHS for user-2.

Pass

51

“Click here to see all incidents” link in the bottom of the incident RHS list view takes the user to the incident backstage

1. As user-1, open up the incident RHS list view.

2. Verify that there is a “Click here to see all incidents” link in the RHS.

3. Click the link.

4. Verify that the incidents backstage page loads listing all the incidents that user-1 is a member of.

Pass

52

The incident RHS details view has a “back arrow” that brings a user to the incident list view in RHS

Continue from the above test.

1. As user-1, go to the incident channel for “PI1”.

2. If the incident details view is not already open in the incident RHS, open it up.

3. Verify that in the incident RHS header, there is a “<“ arrow.

4. Click the “<“ arrow.

5. Verify that the incident details view switches to the incidents list view.

Pass

53

Viewing the incident list from an incident channel highlights the current incident

Continue from the above test.

1. As user-1, start another incident “I-a1”.

2. Go to the incident channel for “I-a1”.

3. Open up the incident list view in the incident RHS.

4. Verify that in the list view, the entry for “I-a1” is highlighted.

5. Verify that none of the other incidents that user-1 is a member of is highlighted.

Pass

54

“Go to Incident Channel” takes the user to the associated incident channel

Continue from the above test.

1. From the incident list in the RHS, find the entry for “PI1”.

2. Click the “Go to incident Channel” button for “PI1”.

3. Verify that the incident channel for “PI1” is loaded.

4. Verify that the incident RHS changes to the incident details views for “PI1”.

Pass

56

“Websocket updates should reflect changes automatically.” → what are the websocket updates?

Pass

Test ID

Test Case

Test Steps

Result

Notes

E2E test done

/incident stage is available as slash command

Pass

/incident stage next is available as slash command

Pass

/incident stage prev is available as slash command

Pass

Running /incident stage prev while in the first stage shows the ephemeral post that “The active stage is the first one…”

Pass

Running /incident stage prev while in the non-first stage sets the active stage to the previous stage

Pass

Running `/incident stage next' while in the non-final stage sets the active stage to the next stage

Pass

Running /incident stage next in the last stage of an incident shows the ephemeral post that “The active stage is the last one.”

Pass

Running /incident stage next in an incident in a stage that has tasks unchecked, asks the user to confirm the action

Pass

Running /incident stage prev in an incident with only one stage shows the ephemeral post that “The active stage is the first one.”

Pass

Running /incident stage next in an incident with only one stage shows the ephemeral post that “The active stage is the last one.”

Pass

Running /incident stage [option] in a non-incident channel shows an ephemeral post “You can only change an incident stage from within the incident's channel. (Only visible to you)”

Pass