Resources

Overview

Content Flagging is a new Mattermost feature that allows any end user to manually flag (report) a message containing sensitive or inappropriate content for review by authorized moderators (referred to as Content Reviewers). The goal of this MVP is to provide a simple, user-initiated workflow to help prevent data spillage and enforce communication policies in secure deployments. When a message is flagged, Content Reviewers are immediately notified in a dedicated private channel so they can quickly evaluate the content and take appropriate action (such as removing the message). This MVP focuses strictly on manual flagging – there is no automated content detection or external DLP integration in this release. All functionality described is based on the approved design mockups, ensuring the specification aligns exactly with the intended UI and wording.

Key goals include:

Empower end users to report sensitive or harmful messages: Any user who spots a potential data spillage (e.g. classified information in a public channel) or a violation of policy can flag the message for moderator review.
Notify Content Reviewers for rapid response: Flagged messages are collected in real-time in a secure “Flagged Content” channel visible only to Content Reviewers (e.g. security team moderators or system admins). This provides a central queue for moderation.
Enable Content Reviewers to assess and act: Content Reviewers can view details about the flagged message (including which channel it was in, who posted it, and when) and then decide to remove the message or mark the flag as resolved. The UI guides the reviewer through resolving the flag, including confirming removal actions.
Maintain audit and transparency: The feature keeps a record of flagged content and actions taken. The original message author is notified that their post is under review (and later if it is removed), and the reporting user gets feedback that their flag has been received. This ensures transparency in the moderation process.

User Roles and UX Flows

The Content Flagging feature involves three primary user roles, each with distinct interactions:

System Administrator: Configures and enables the feature in the System Console, and designates the channel and members for content review.
End User: Flags a message via the UI when they encounter sensitive or inappropriate content.
Content Reviewer: Receives notifications of flagged messages, reviews the content details, and takes action (resolve or remove).

System Administrator UX Flow (Configuration)

The System Administrator is responsible for enabling the Content Flagging feature and configuring who can review flagged content and how notifications and behavior are handled. All configuration is done in the System Console under Site Configuration > Content Flagging. By default, Content Flagging is off and must be explicitly enabled.

Content Reviewers Configuration

Under Content Reviewers, the admin defines who will review flagged messages in the workspace. Mattermost will automatically create a special “Flagged content” channel (one per team irrespective of the below configuration) and add the selected users as content reviewers in each channel.

Content Reviewers <> Teams: The admin can choose to use one global set of reviewers for all teams (select True) or specify different reviewers per team (select False).
- If True, a single Reviewers list is shown. The admin adds user(s) who will act as Content Reviewers across the entire workspace (e.g., a dedicated compliance team).
  - Only users who are a part of all teams on the workspace will be allowed to be selected in this case.
- If False, the interface displays a separate Reviewers list for each team in the workspace, allowing the admin to assign reviewers on a per-team basis. Only users that are already part of the respective team should be allowed to be selected for that team.
Additional reviewers
- Two checkboxes allow automatically including certain roles as Content Reviewers:
  - System Administrators – If checked, all users with the System Admin role will be included as Content Reviewers (they will have access to all flagged content, in addition to any specific users listed).
  - Team Administrators – If checked, Team Admins are included as reviewers for their respective teams. This means the Team Admin will be added to the Flagged content channel for their team and can moderate flags in that team.
- Note: System Admins and Team admins added as reviewers via this configuration will not be shown as chips in the content reviewer input boxes above.
  - Edge case — Same reviewers for all teams is enabled + Team admins are selected in Additional reviewers: There should be a channel created in every team where the manually selected set of reviewers are added in each channel. These users will be the common content reviewers across all teams in the workspace. Any team admins for each team will also then be added as a reviewer given their team admin role to the respective team’s flagged content channel.

When the admin saves these settings, the system will update the membership of each Flagged content channel accordingly. The Flagged content channel is a private moderation channel (one per team) where flagged message records are posted for review. Only the configured Content Reviewers (and System Admins/Team Admins if those options are enabled) are members of this channel, ensuring flagged content is only visible to authorized moderators. Channel membership cannot be manually changed in the UI – a note in the channel header indicates “Channel access is managed through the Content Flagging configuration in the System Console.”

Implementation Recommendation

Management of content reviewers and the the flagged content channel can be done via ABAC under the wraps. How this can work:

Add a hidden custom attribute in the system for the role of content reviewers.
When the user selects a bunch of people as content reviewers in the above screen, add the value content reviewer to all of these users.
Also add the value for system admins and team admins, if selected.
If the user has selected Same reviewers for all teams, then there can be a single ABAC policy with auto-sync enabled which matches users based on the custom attribute and has the flagged content channels from all teams assigned to it.
If the user has selected different reviewers per teams, then there can be an ABAC policy per team with auto-sync enabled which matches users based on the custom attribute and has the flagged content channel from that teams assigned to it.

Notification Settings

The System Admin configures who gets notified by the Content Review bot when certain events occur in the flagging workflow. There are three events, each with checkboxes for Reviewer(s), Author, and Reporter roles:

Notify when content is flagged
Notify when content is removed
Notify when flag is dismissed

All notifications are sent via the Content Review bot, either as direct messages to the individuals (Author/Reporter) or posts in the Flagged content channel for the Reviewers.

Additional Settings

These settings control aspects of the flagging workflow behavior:

Reasons for flagging: The admin can configure the set of predefined reasons a reporter must choose from when flagging a message. The interface presents a list of reason tags.
The admin can tailor which of these reasons are available for users to select (there are a few default values that will be pre-configured). The chosen reason by the reporter will be recorded and displayed to Content Reviewers to inform their decision. All flags must have one of the provided reasons selected – the UI will require the user to pick one before submission. This ensures flags are categorized and can be reported on. Reasons for flagging must have at least 1 option before saving.
Require reporters to add comment: A boolean setting (True/False) determining whether the user flagging a message must provide a written explanation (comment) apart from the reason for flagging tag. If True, the Flag Message dialog will enforce that the Comment field is filled in before the user can submit a flag. If False, the comment will be optional (the dialog will still provide a Comment field but indicate it’s optional and allow submission without it).
Hide message while it is being reviewed: A boolean setting controlling channel visibility of flagged content. If True, as soon as a message is flagged, it is hidden from the original channel (removed from view for all users) while the review is pending. Only Content Reviewers can view it in the Flagged content channel record that they can review. If False, the message remains visible in the channel to all users even after being flagged, up until a Content Reviewer actively removes it (if they choose to do so).

End User UX Flow

End users encompass both the Reporter (the user who flags a message) and the Author (the user whose message gets flagged, if different). Any regular user (with permission to see the message) can flag a message for Content Review, and any user could potentially have their message flagged by someone else.

Flagging a Message (Reporter Perspective)

When an end user encounters a message that they believe should be removed due to sensitive or inappropriate content or any other reason, they can initiate the flagging process.

The user hovers over the message in question and opens the "More actions" menu.
In the list of actions, a new option is available: “Flag message”.
The flag action is only visible if Content Flagging is enabled in the System Console.
Once the user clicks Flag message, a modal dialog appears titled “Flag message”. This is a confirmation and input form that gathers the necessary information for the flag.
The modal shows the content of the message the user is about to flag. This is presented in a container to give context and confirmation of the correct message. It includes the message’s author name, timestamp, and content (and any attachment thumbnail if applicable).
“Reason for flagging this message:” A required field where the user selects a reason from a dropdown list. The label text is exactly as in the UI. The dropdown options correspond to the Reasons for flagging configured by the System Admin.
“Comment” field: A text area input for the reporter to add additional context about why they flagged the message.
- If the admin set comments as required, this field will be marked as required. If the comment is required and left blank, an inline validation message will prompt the user to fill it before submitting.

When the user clicks Submit, the dialog closes and the flagging action is processed:

The message is now flagged for review. If Hide message while being reviewed = True, the flagged message is immediately removed from the channel view. From all users’ perspective in the original channel, that message disappears (similar to if it were deleted).
If Hide while under review = False, then the message remains visible in the channel as-is. There is no indication in the channel UI that it’s flagged (the design does not show any badge or marker on the message). The flagging is essentially silent to other users in this case.
In both cases, the system creates a record of the flag in the Flagged content moderation channel for the Content Reviewers and triggers any configured notifications.
Confirmation to Reporter: The user who flagged the message gets immediate feedback confirming that the flag was successful. This appears as an ephemeral message (only visible to that user) in the context where they performed the action.
The same reporter cannot flag the same message again (the option will not be available if they open the menu again).

Flagged Message Author’s Perspective

When a user’s message is flagged by someone (or by themselves in rare cases), that user (the Author) is informed of the situation, provided the System Admin has left author notifications enabled.

Content Reviewer UX Flow

Content Reviewers have access to the private Flagged content channel in each team where they are reviewers. They are responsible for reviewing flagged messages, discussing if needed, and deciding whether to remove the content or allow it to remain.

Receiving a Flag Notification

When a message is flagged by a user, a record is automatically posted to the Flagged content channel of the corresponding team. The Content Reviewers in that team will see a new post in this channel (and the channel will indicate an unread message)

The format of the post contains key information about the flagged message:

Flag notification post title – For example, when Leonard Riley flags a message in the “UX Design” channel, the Flagged content channel will show a post titled: “@Leonard Riley flagged a message for review”
Status
Reason – the reason selected by the reporter.
Message – a preview of the flagged message itself. This includes the original author’s name and avatar (e.g., Gordon Walker), timestamp of the original message, and the content snippet or attachment name. If the message was long, there may be a “Show more” expander to view the full text.
Reporters comment
Flagged by – the user who reported it.
Flagged at – the timestamp when the message was flagged
Duration visible – this field indicates how long the message was visible in the channel before it was flagged

Open question: How do we handle multiple flags on the same message?

Reviewing Flagged Content

When a Content Reviewer sees a new flagged message post, they will perform the following to handle it:

Assess details - A reviewer can click on a flagged post to open a details view on the right-hand side.
Decide on action: The Content Reviewer(s) must choose to either remove the message or dismiss the flag (keep the message). In the UI, there are explicit controls for these actions associated with the flagged post.
1. If the reviewer clicks Remove (Delete) message: a confirmation dialog will appear to ensure this destructive action is intended.
  1. If the reviewer confirms (clicks Remove message in the dialog), the system proceeds to delete the message from the original channel permanently:
  2. The Flagged content channel post for this message updates its Status from "Pending review" to "Removed".
  3. Also, fields Reviewed by and Reviewed at are populated.
  4. Notifications on removal: The system triggers notifications according to the configuration.
2. If the reviewer clicks Keep message: this indicates the reviewer has decided the message does not need removal (the flag is not upheld). Once confirmed to keep:
  1. If the message was hidden, it is restored to the channel. The original message becomes visible again to all channel participants as if nothing happened. (The timestamp order would place it back in the original position)
  2. If the message was never hidden (hide=false scenario), then essentially no change occurs in the channel content (the message was always visible, and remains so).
  3. The Flagged content channel post updates its Status to "Dismissed".
  4. The post would also get Reviewed by [Name] and Reviewed at [time] filled in, similar to the removed case.
  5. Notifications on dismissal: The system triggers notifications according to the configuration.

c. If the reviewer clicks Run playbook: (Optional Action) This would be used if the flagged message indicates a larger incident (for example, a serious data spillage might trigger an incident response process). By clicking Run playbook, the reviewer can launch a predefined incident playbook (such as a Security Incident response) with context about this message. The flagged message information might be passed as parameters in the run summary.

The reviewer might run a playbook before or after taking the decision of removing or keeping the flagged message. The flagged message is not resolved by this action.

Edge Cases and Special Scenarios for Reviewers

Flagged message deleted by author before review: If the author manages to delete their message after it was flagged (this could happen if hide=false and they had permission to delete their own post), the system should handle this gracefully.
- The UI could update the status to something like “Removed” automatically with a note that the author deleted it.
- The Content Review bot could add a comment on the report like “Message was deleted by its author at [time].” The Content Reviewer should still be able to see all information if any further action is needed (like running a playbook).
Flagged message edited by author during review: For the MVP, we can either block editing on flagged messages or we can have the reviewer still act based on what was flagged (the original content is what they have in the record). The final decision might be to dismiss if the edit fixed the issue, but the system does not explicitly support showing reviewers any “edits” on a flagged message.
Flagging in Private Channels: Content Reviewers might normally not have access to a private conversation. However, when a user flags a message in such a context, the system surfaces that message to the reviewers via the flagged content channel. The Content Reviewers thus gain visibility into content they otherwise might not see, but this is by design for moderation purposes.
Multiple Content Reviewers: If multiple reviewers attempt actions on the same flag simultaneously (rare), the system should only allow one resolution.

Product Design Team

UX Spec: Data Spillage Handling — Content Flagging