Microsoft Calendar User Statuses

Business Logic for Statuses in Mattermost

Microsoft Statuses

Microsoft can let us know a user's status at a given time using one of five available options.

free
tentative
busy
out of office
working elsewhere

Mattermost Implementation

online
away
offline
do not disturb

How the plugin currently handles these statuses:

- free
    if dnd, set online
- tentative
    if not dnd, set dnd
- busy
    if not dnd, set dnd
- out of office
    ?
- working elsewhere
    ?

Concerns

The update status functionality in Mattermost uses a boolean flag manual to represent that it is a manual action by the user that is changing the status. These manual actions always override whatever the current status is. The plugin API method UpdateUserStatus automatically uses the manual=true for the value of the flag, so it is the same as user manually doing it as far as the status handling is concerned. We may need to change the way the manual boolean flag is used, so the plugin may have slightly less precedence than the user’s manual choices.
If a user was previously busy but is now free, we need to determine if they should be set to online, away, or offline. Ideally we set them as whatever they were set to before we set them to dnd earlier.
In order for the user to be aware of when the plugin changes his status, the application could tell the user in-app that his status has been changed for him by the plugin.
The plugin needs to use all available info to determine if the user's status needs to change. This could include special settings that user can set themselves. Maybe a user config value that says “Microsoft Plugin, take the wheel for x hours.”
We need to keep track of if we’ve already taken care of the user’s current availability. i.e. we already set them to out of office for this event, don’t set it again if the user has set it to something else since then.

Custom Status Text

A hackathon submission for November 2019 introduces a custom status text to be displayed next to the username. This approach may be of use for the enumerations of Microsoft's statuses. It's also possible that we have the option to include emojis in the custom status as well. This is not yet in the product so it may not be a viable solution.

https://github.com/mattermost/mattermost-hackathon-nov2019/blob/master/hackathon-submissions/m-zubairahmed-set-custom-status.md

Automated Job

The job runs every 5 minutes to fetch and update user statuses.

In an HA environment, only one node should run the job. Using the RunOnSingleNode plugin helper (or a similar solution using a shared lock), we can synchronize the nodes and only allow one to run the job. https://github.com/mattermost/mattermost-server/pull/12969

The plugin uses app-only level permissions that are explicitly agreed to by the admin of the customer. We should build the auth flow to include admin consent so they don't have to configure in the azure portal: https://docs.microsoft.com/en-us/graph/auth-v2-service#3-get-administrator-consent

Performance

Microsoft Graph supports batch requesting on certain endpoints. The API supports 20 requests in one batch. Combining this with using the max 20 users per request, we can fetch 20 * 20 = 400 users' availabilities in one HTTP request. A smoke test of this request had a roundtrip of 1.18 seconds. Note: There could be an optimization on Microsoft's end affecting this time, as I only used one user but requested its availability 400 times.

https://docs.microsoft.com/en-us/graph/json-batching

Given we have 1,000 users, we can fetch them in 3 requests, resulting in about ~1.5s * 3 = ~4.5s. Given we have 10,000 users, we should be able to fetch all users' statuses in 25 requests, resulting in about ~1.5s * 25 = ~37.5s if done synchronously. Splitting up the work into goroutines should work nicely since each request is completely independent from the others.

Using the Plugin API method GetUserStatusesByIds, we can fetch all of the relevant statuses in one RPC call, and only update a user's status if it needs to change.

Fetching Free/Busy from Microsoft Graph

Microsoft Graph exposes an API to fetch multiple users' schedules.

https://docs.microsoft.com/en-us/graph/api/calendar-getschedule

Request:

POST https://graph.microsoft.com/v1.0/me/calendar/getSchedule

Request Body:

{        
    "schedules": ["test@mattermost.onmicrosoft.com", "test2@mattermost.onmicrosoft.com"],
    "startTime": {
        "dateTime": "2019-03-15T09:00:00",
        "timeZone": "Pacific Standard Time"
    },
    "endTime": {
        "dateTime": "2019-03-15T09:10:00",
        "timeZone": "Pacific Standard Time"
    },
    "availabilityViewInterval": 5
}

Response Body:

The availabilityView gives us a summary of the free/busy based on the intervals we request. Each digit represents a time slot of length availabilityViewInterval we specified in the request.

Value of each number:

0 = free
1 = tentative
2 = busy
3 = out of office
4 = working elsewhere.

We can fetch as many slots that makes sense (if we want to know more information than just "now"), but the first slot will be at the startTimevalue in the request, which would be "now". Using the first number of the availabilityView, we can determine what the user's status is at this moment.

TODO

Improve business logic of handling user statuses

Test the roundtrip of a batch request with realistic user data
Build the auth flow to include admin consent: https://docs.microsoft.com/en-us/graph/auth-v2-service#3-get-administrator-consent
Better error handling for requests, including batch requests. This includes notifying admins of failures.
Research and test rate limiting of Microsoft’s APIs
Test that the order of availabilities in each batch response is valid.

MS Calendar Integration