Document version 0.0
Status: Test Completed
Document Version | Description | Date |
---|---|---|
0.0 | Initial version | 10/06 |
...
Admin - A user with system administrator privileges.
Normal user - A user with non-system administrator privileges.
Test Server - A list of test server versions used in testing including Mattermost server and Marketplace Server
Main menu - Hamburger menu on the LHS.
Incident Response icon - The Incident Response plugin icon that appears on the channel header
Commander - The user who owns an incident. The user who starts the incident is designated the commander by default.
Incident member/incident channel member - A user who has been added to the incident channel.
Non-incident user - A user who has not been added to the incident channel.
Active incident - An incident that has not been ended.
Ongoing incident - Same as Active Incident.
Inactive incident - An incident that has been ended.
Incident RHS - The RHS sidebar that opens upon clicking the incident response icon and displays incidents.
Incident Backstage - The backstage view that shows the incident list.
Playbook Backstage - The backstage view that shows the playbook list.
Mobile - Mobile app in iOS and mobile app in Android
Assumptions
The tests in this test plans are written with the assumption that:
Test steps mentioning “mobile” will be executed in both iOS and Android RN mobile apps.
The tests outlined in this document is to be executed against the Incident Response plugin v1.0.
This version of the plugin can be installed and used in Mattermost instances that aren’t equipped with any enterprise license.
The Incident Response plugin can function as a standalone plugin that does not require to talk to any other Mattermost plugin.
This version of the Incident Response plugin will function properly only in the latest version of Mattermost.
The tests will be executed in the latest version of Mattermost.
Testing is primarily done on browser (Chrome), with spot checks on other web browsers, desktop app, RN mobile app and mobile web browser app.
The tests in this document are executed as a normal user (see glossary above) unless otherwise specified in the tests.
The test cases that pass for a normal user will also pass for a system administrator.
Plugin uploads is enabled in server instance
“EnableUploads” option under “PluginSettings” in mattermost-server’s config.json is set to 'true'.
The channel export feature defined in the user story is dependent on the existence of the Channel Export plugin being installed and enabled in the test server.
Private channels cannot be converted into public channel, hence private incidents cannot be converted to public incidents.
...
Test Area - IR mobile slash commands general test | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done | |||||||||||
|
| Covered in tests for | Typing |
| ||||||||||||
2 | Typing |
| The slash command options in the popup is scrollable | In mobile - In webapp, scroll + up/down arrows work - |
| Covered in tests for | ||||||||||
Test Area - User must be able to start an incident on mobile app | ||||||||||||||||
Test ID | Test Case | Test Steps | Result | Notes | E2E test done | |||||||||||
|
| Typing | Incident creation screen on mobile shows all the fields as the the incident creation page on webapp |
| Clicking “Create a playbook” link in incident creation screen takes the user to the playbook creation screen? | Fail | Says “Link not found in server” | Starting an incident requires a playbook to be selected | Starting an incident requires an incident name | User can start an incident on mobile app using | User can start an incident on mobile using post menu option | Probably not valid? | When incident is started from mobile app, a system message is posted in the channel from where incident is started
| |||
3 | The slash command options in the popup is scrollable |
| ||||||||||||||
4 | “Execute current command” is highlighted by default in pop up options |
|
Test Area - User must be able to see the list of tasks in the current stage in start an incident on mobile app | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done | ||||||||
5 |
|
| User can see incident details with | On mobile app, issuing |
| On mobile, issuing | On webapp, issuing | On webapp, issuing | On webapp, issuing | ||||
Test Area - User must be able to see all their incidents in mobile app | |||||||||||||
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
|
| Issuing an |
|
|
|
|
|
Test Area - User must be able to change commander in mobile app | |||||||||||||
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
| Test in both mobile and webapp |
| Test in both mobile and webapp | Issuing | Test in both mobile and webapp | Issuing | Test in both mobile and webapp |
| Test in both mobile and webapp |
| Test in both mobile and webapp |
| Test in both mobile and webapp |
| Test in both mobile and webapp | An incident channel member can see a dropdown for users under “Commander” in details view | Incident channel members are listed in the user dropdown list | An incident channel member can be searched by a commander in the user dropdown list | (0.3) | A non-incident channel member cannot be searched in the user dropdown list | A bot message is posted to the incident channel when commander is changed. |
|
Test Area - User must be able to change incident name in mobile app – is this still valid? | |||||||||||||
Test ID | Test Case | Test Steps | Result | Notes | E2E test done | ||||||||
Test Area - Incident List View for Team Members | |||||||||||||
Test ID | Test Case | Test Steps | Result | Notes | E2E test done | ||||||||
Clicking on the incident icon when there are no incidents, opens the incident RHS to the “welcome” message | With incidents ongoing, clicking on the incident icon when user is in non-incident channel RHS opens up with the incident list for the user | With incidents ongoing, clicking on he incident icon when user is in incident channel with RHS open, shows the current incident details | Switching from one incident channel to another incident channel with the RHS details view will switch to the selected incident’s detail view in the RHS | The incident RHS list view has a “Start Incident” button on the RHS header | The incident RHS list view has a “Create Playbook” button | A user can only see the incident he/she is a member of in the incident RHS list view | A user cannot see a public incident in the incident RHS if he/she is not a member of that incident | “Click here to see all incidents” link in the bottom of the incident RHS list view takes the user to the incident backstage | The incident RHS details view has a “back arrow” that brings a user to the incident list view in RHS | Viewing the incident list from an incident channel highlights the current incident
| |||
6 | Issuing |
| |||||||||||
7 | Incident creation screen on mobile shows all the fields as the the incident creation page on webapp |
| |||||||||||
8 | Tap “Create a playbook” link in incident creation screen takes the user to the playbook creation screen? |
| Fail | Says “Link not found in server” | |||||||||
9 | Starting an incident requires a playbook to be selected |
| |||||||||||
10 | Starting an incident requires an incident name |
| |||||||||||
11 | User can start an incident on mobile app using |
| |||||||||||
12 | User can start an incident on mobile using post menu option | Probably not valid? | |||||||||||
13 | When incident is started from mobile app, a system message is posted in the channel from where incident is started |
|
Test Area - User must be able to see the list of tasks in the current stage in mobile app | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
14 |
|
| |||
15 | On mobile app, issuing |
| Is it an ephemeral message or a normal bot message that gets posted? Looks like a bot message | ||
16 | On mobile, issuing |
| |||
17 | On webapp, issuing |
| |||
18 | On webapp, issuing | Continue from the above test
| |||
19 | On webapp, issuing | Continue from the above test
| |||
20 | On webapp, issuing | Continue from the above test
|
Test Area - User must be able to see all their incidents in mobile app | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
21 |
|
| |||
22 | Issuing an | 0. Install a fresh server.
| |||
23 | Issuing |
| |||
24 | Issuing | Continue from the above test
| |||
25 | Issuing | Continue from the above test
| |||
26 | Issuing | Continue from the above test
| |||
27 | Issuing |
| |||
28 | Issuing |
| |||
29 |
|
|
Test Area - User must be able to change commander in mobile app | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
30 |
|
| |||
31 | Issuing |
| |||
32 | Issuing | In mobile
| |||
33 |
|
| |||
34 |
|
| |||
35 |
|
| |||
36 |
| Continue from the above test
| |||
37 | An incident channel member can see a dropdown for users under “Commander” in details view | Covered in older release tests (will be reorganized later). | |||
38 | Incident channel members are listed in the user dropdown list | ||||
39 | An incident channel member can be searched by a commander in the user dropdown list | (0.3) | |||
40 | A non-incident channel member cannot be searched in the user dropdown list | ||||
41 | A bot message is posted to the incident channel when commander is changed. |
|
Test Area - User must be able to change incident name in mobile app – (this is in Confluence) is this still valid? | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
Test Area - Incident List View for Team Members (Webapp only) | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
42 | Clicking on the incident icon when there are no incidents, opens the incident RHS to the “welcome” message |
| |||
43 | With incidents ongoing, clicking on the incident icon when user is in non-incident channel RHS opens up with the incident list for the user | Continue from the above test.
| |||
44 | With incidents ongoing, clicking on the incident icon when user is in incident channel with RHS open, shows the current incident details | Continue from the above test.
| |||
45 | Switching from one incident channel to another incident channel with the RHS details view open will switch to the selected incident’s detail view in the RHS | Continue fromt he above test.
| |||
46 | The incident RHS list view has a “Start Incident” button in the RHS header | Continue from the above test.
| |||
47 | The “Start Incident” button in the RHS header opens up the incident creation modal | Continue from the above test.
| |||
48 | The incident RHS list view has a “Create Playbook” button | Continue from the above test.
| Is this valid? | ||
49 | A user can only see the incident he/she is a member of in the incident RHS list view | Continue from the above test.
| |||
50 | A user cannot see a public incident in the incident RHS if he/she is not a member of that incident |
| |||
51 | “Click here to see all incidents” link in the bottom of the incident RHS list view takes the user to the incident backstage |
| |||
52 | The incident RHS details view has a “back arrow” that brings a user to the incident list view in RHS | Continue from the above test.
| |||
53 | Viewing the incident list from an incident channel highlights the current incident | Continue from the above test.
| |||
54 | “Go to Incident Channel” takes the user to the associated incident channel | Continue from the above test.
| |||
55 | “Go to Incident Channel” from a different team, takes the user to the correct incident channel in the right team. | Not sure about this behavior. Right now it shows a “Channel not found” error. | |||
56 | “Websocket updates should reflect changes automatically.” → what are the websocket updates? |
Regression Tests:
Test Area - Plugin Setup | |||||
---|---|---|---|---|---|
Test ID | Test Case | Test Steps | Result | Notes | E2E test done |
A cloud server has the IR plugin v1.0 installed by default | Verify as system administrator | ||||
A cloud server has the IR plugin enabled by default | |||||
The IR plugin can be disabled in a cloud server | |||||
The IR plugin can be removed from a cloud server | |||||
The IR plugin can be installed in an on-premises server that is equipped with an E20 license | |||||
The IR plugin can be enabled normally in an on-premises server that is equipped with an E20 license | |||||
The IR plugin can be disabled in an on-premises server that is equipped with an E20 license | |||||
The IR plugin can be removed from an on-premises server that is equipped with an E20 license. | |||||
The IR plugin cannot be enabled in an on-premises server that is equipped with an E10 license | |||||
The IR plugin can be removed from an on-premises server that is equipped with an E10 license | |||||
The IR plugin cannot be enabled in an on-premises EE server that is not equipped with any license | |||||
The IR plugin can be removed from an on-premises EE server that is not equipped with any license | |||||
The IR plugin cannot be enabled in an on-premises TE server | |||||
The IR plugin can be removed from an on-premises TE server |
...