Resources

• Figma file

• Prototype

Overview

Content Flagging is a Mattermost feature that allows any end user to manually flag (report) a message containing sensitive or inappropriate content for review by authorized moderators (Content Reviewers). The goal of this MVP is to provide a simple, user-initiated workflow to help prevent data spillage and enforce communication policies in secure deployments.

When a message is flagged, Content Reviewers are immediately notified via a direct message from the Content Review bot, instead of a team-specific channel. This bot DM serves as a centralized moderation queue: all flagged messages across all teams are aggregated in the Content Review DM for each reviewer (cross-team). The Content Review bot posts a flagged message report into the DM, containing a summary of the flagged post and relevant details for review. From there, Content Reviewers can assess the content and take appropriate action (such as assigning a reviewer, removing the message, or dismissing the flag).

Open

Key goals include:

• Empower end users to report sensitive or harmful messages: Any user who spots a potential data spillage (e.g. classified information in a public channel) or a violation of a policy can flag the message for review.

• Notify Content Reviewers: Flagged messages are collected in real-time in a DM with the Content Review bot for all authorized Content Reviewers.

• Enable Content Reviewers to assess and act: Content Reviewers can view details about the flagged message (which channel/team it was in, who posted it, who flagged it, when, etc.) and then decide to remove the message or mark the flag as resolved (dismiss the flag)

User Roles and UX Flows

The Content Flagging feature involves three primary user roles, each with distinct interactions:

• System Administrator: Configures and enables the feature in the System Console, and designates who will review flagged content. The System Admin defines notification preferences, as well as settings like available flag reasons and whether flagged messages are hidden from channels.

• End User: Flags a message via the UI when they encounter sensitive or inappropriate content.

• Content Reviewer: Receives notifications of flagged messages (via the Content Review bot DM), reviews the content details, and takes action (resolve by removal or dismissal). Content Reviewers are typically a small set of moderators (e.g. security team, compliance officers, system admins) designated by the System Admin.

System Administrator UX Flow (Configuration)

The System Administrator is responsible for enabling the Content Flagging feature and configuring who can review flagged content and how notifications and behavior are handled. All configuration is done in the System Console under Site Configuration > Content Flagging. By default, Content Flagging is off and must be explicitly enabled.

Open

Content Reviewers Configuration

Under Content Reviewers, the admin defines who will review flagged messages in the workspace. Mattermost will automatically create a special “Flagged content” channel (one per team irrespective of the below configuration) and add the selected users as content reviewers in each channel.

• Same reviewers for all teams: a single, global list of Content Reviewers for the entire workspace (select True for “Same reviewers for all teams”). The admin can add one or more users to the Reviewers list who will act as Content Reviewers across all teams.

  • Only users who are members of every team in the workspace can be selected in this case (to ensure they have access to any team content that might be flagged)

  • Open

    

     

     

• Different reviewers per team: a separate list of reviewers for each team (select False for “Same reviewers for all teams”). The interface will display a each team in the workspace along with an input box for each, allowing the admin to assign reviewers on a per-team basis.

  • Only users who are already members of a given team can be selected as reviewers for that team.

    Open

    

     

     

• Additional reviewers

  • Two checkboxes allow automatically including certain roles as Content Reviewers:

    • System Administrators – If checked, all users with the System Administrator role will automatically get flagged message reports from the Content Review bot—but only for teams they are already members of.

      • System Admins will not receive flagged messages from teams they do not belong to. Being a System Admin alone does not grant cross-team visibility. System Admins must be explicitly added as members of a team to receive flagged message reports from that team.

    • Team Administrators – If checked, all users with the Team Administrator role are automatically designated as Content Reviewers for their respective teams. Each Team Admin will only receive flagged message reports via the Content Review bot DM for the specific team(s) they administer.

  • Note: System Admins and Team admins added as reviewers via this configuration will not be shown as chips in the content reviewer input boxes above.

Notification Settings

The System Admin configures who gets notified by the Content Review bot when certain events occur in the flagging workflow. There are four event types, each with checkboxes to enable notifications for specific roles:

• Notify when content is flagged: Options to notify Reviewer(s), and the Author when a message is flagged.

• Notify when a reviewer is assigned: Option to notify Reviewer(s) when a flagged message is taken by a reviewer. If enabled, when one reviewer assigns themselves or another person to the flag, the other Content Reviewers are notified that a reviewer has been assigned by a reply in the flag’s thread from the Content Review bot.

• Notify when content is removed: Options to notify Reviewer(s), Author, and Reporter when a flagged message is removed by a reviewer. Typically, the Author and the original Reporter should be informed of the resolution.

• Notify when flag is dismissed

Regardless of these settings, the Content Reviewers will see the updates in the DM card and RHS details.

All notifications are sent via the Content Review bot, either as direct messages to the individuals or as replies to the original flagged message report DM.

Open

Additional Settings

These settings control aspects of the flagging workflow behavior:

• Reasons for flagging: The admin can configure the set of predefined reasons a reporter must choose from when flagging a message. The interface presents a list of reason tags.
  The admin can tailor which of these reasons are available for users to select (there are a few default values that will be pre-configured). The chosen reason by the reporter will be recorded and displayed to Content Reviewers to inform their decision. All flags must have one of the provided reasons selected – the UI will require the user to pick one before submission. This ensures flags are categorized and can be reported on. Reasons for flagging must have at least 1 option before saving.

• Require reporters to add comment: A boolean setting (True/False) determining whether the user flagging a message must provide a written explanation (comment) apart from the reason for flagging tag. If True, the Flag Message dialog will enforce that the Comment field is filled in before the user can submit a flag. If False, the comment will be optional (the dialog will still provide a Comment field but indicate it’s optional and allow submission without it).

• Require reviewers to add comment: A boolean setting (True/False) determining if the Content Reviewer must add a comment when resolving a flag. If True, when a reviewer takes the Remove Message action (or when dismissing a flag), the UI will require the Reviewer’s Comment field to be filled before finalizing the action. If False, reviewer comments are optional (the reviewer may still provide one, but can complete the action without writing anything). This setting ensures important decisions have justification recorded if desired.

• Hide message while it is being reviewed: A boolean setting controlling channel visibility of flagged content. If True, as soon as a message is flagged, it is hidden from the original channel (removed from view for all users) while the review is pending. Only Content Reviewers can view it in the flagged message report in the Content Review bot DM. If False, the message remains visible in the channel to all users even after being flagged, up until a Content Reviewer actively removes it (if they choose to do so).

Open

End User UX Flow

End users encompass both the Reporter (the user who flags a message) and the Author (the user whose message gets flagged, if different). Any regular user (with permission to see the message) can flag a message for Content Review, and any user could potentially have their message flagged by someone else.

Flagging a Message (Reporter Perspective)

Open

When an end user encounters a message that they believe should be removed due to sensitive or inappropriate content or any other reason, they can initiate the flagging process.

• The user hovers over the message in question and opens the "More actions" menu.

• In the list of actions, a new option is available: “Flag message”.

• The flag action is only visible if Content Flagging is enabled in the System Console.

• Once the user clicks Flag message, a modal dialog appears titled “Flag message”. This is a confirmation and input form that gathers the necessary information for the flag.

• The modal shows the content of the message the user is about to flag. This is presented in a container to give context and confirmation of the correct message. It includes the message’s author name, timestamp, and content (and any attachment thumbnail if applicable).

• “Reason for flagging this message:” A required field where the user selects a reason from a dropdown list. The label text is exactly as in the UI. The dropdown options correspond to the Reasons for flagging configured by the System Admin.

• “Comment” field: A text area input for the reporter to add additional context about why they flagged the message.

  • If the admin set comments as required, this field will be marked as required. If the comment is required and left blank, an inline validation message will prompt the user to fill it before submitting.

Open

When the user clicks Submit, the dialog closes and the flagging action is processed:

• The message is now flagged for review. If Hide message while being reviewed = True, the flagged message is immediately removed from the channel view. If the message has been received by a client, it should be replaced by the text (message hidden) (which goes away on refresh, similar to how it works for deleted messages). From all users’ perspective in the original channel, that message disappears (similar to if it were deleted).

• If Hide while under review = False, then the message remains visible in the channel as-is. There is no indication in the channel UI that it’s flagged (the design does not show any badge or marker on the message). The flagging is essentially silent to other users in this case.

• In both cases (hide or not), the Content Review bot posts a new flagged message report into the DM for the appropriate reviewers.

• Confirmation to Reporter: The user who flagged the message gets immediate feedback confirming that the flag was successful. This appears as an ephemeral message (only visible to that user) in the context where they performed the action.

• The same reporter cannot flag the same message again (the option will not be available if they open the menu again).

Flagged Message Author’s Perspective

When a user’s message is flagged by someone (or by themselves in rare cases), that user (the Author) is informed of the situation, provided the System Admin has left author notifications enabled.

Open

• The bot may include a preview of the message content in the DM (quoting the message text and showing its original posting timestamp and channel) so the author knows which message was flagged. However, if the message has been hidden/removed from the channel due to the flag, the message preview will not be shown in the notification.
  
  

  Open

  

Depending on configuration, the author may receive a second notification once the review is complete.

Content Reviewer UX Flow

Content Reviewers are the moderators designated to handle flagged messages. The DM with the Content Review bot gives them a view of all flagged post reports that they have been configured to handle (across teams). They are responsible for reviewing a flagged message, assigning it to themselves or another reviewer, and deciding whether to remove the content or allow it to remain (dismiss the flag).

Receiving a Flag Notification

Whenever any user flags a message, a flagged message report is automatically posted by the bot into the Content Review DM for all relevant reviewers .

Open

Each flagged message report appears as a card-style post in the bot DM. The reviewer can click on the card or the view details button to open a detailed view in the RHS. Specifically, the content of each report includes:

• Flag notification post title – For example, when Leonard Riley flags a message in the “UX Design” channel, the content review bot DM will show a post titled: “@Leonard Riley flagged a message for review”

• Status

• Reason – the reason selected by the reporter.

• Message – a preview of the flagged message itself. This includes the original author’s name and avatar (e.g., Gordon Walker), timestamp of the original message, the message content snippet, and attachments if any. If the message was long, there may be a “Show more” expander to view the full text.

• Reviewer – An assignment field indicating who is handling the review. Initially this may be blank or “Unassigned.” Content Reviewers can use this field to assign the flagged post to a specific reviewer (even themselves).

• Reporters comment

• Flagged by – the user who reported it.

• Flagged at – the timestamp when the message was flagged

• Duration visible – this field indicates how long the message was visible in the channel before it was flagged

• Seen by – The reviewer has the ability to see who saw the message before it was flagged. In the seen by field, the reviewer can click on the Show Users button retrieve the list of users who viewed the message in the channel before it was hidden. This may take a moment, after which it will display all users who are likely to have seen the post. This action is on-demand to avoid performance issues on every flag.

Open

Reviewing Flagged Content

When a Content Reviewer sees a new flagged message post, they will perform the following to handle it:

1. Assess details - A reviewer can click on a flagged post to open a details view on the right-hand side.

2. Claim or assign the report (optional): If multiple reviewers are in the team, one reviewer should take ownership. The reviewer can click the Reviewer field (or the Assign button) to assign it to themselves or to another reviewer from the list. Selecting a reviewer instantly update the card to show “Reviewer: @Name”. The status will change from “Pending” to “Reviewer Assigned“ . This assignment is immediately communicated to all other reviewers: the Content Review bot will post a reply the thread of that report.

3. Decide on action: After assessing the details, the Content Reviewer (assignee) must decide an action for the flagged message. There are two primary resolution actions: Remove Message or Keep Message, corresponding to whether the content will be taken down or allowed to remain.

   1. Remove Message: A confirmation dialog appears to prevent accidental deletion and warn the reviewer that this action will permanently delete the message for all users and cannot be undone

      1. The dialog also provides a Reviewer’s comment field (optional or required per settings) for the reviewer to enter a note explaining the removal.

      2. If the reviewer confirms (clicks Remove message in the dialog), the system proceeds to delete the message from the original channel permanently:

         1. The report post for this message updates its Status to "Removed".

         2. Also, fields Reviewed by, Reviewed at, and Reviewers Comment (if any was provided) are populated.

         3. Notifications on removal: The system triggers notifications according to the configuration.

   2. If the reviewer clicks Keep message: this indicates the reviewer has decided the message does not need removal (the flag is not upheld). Once confirmed to keep:

      1. If the message was hidden, it is restored to the channel. The original message becomes visible again to all channel participants as if nothing happened. (The timestamp order would place it back in the original position)

      2. If the message was never hidden (hide=false scenario), then essentially no change occurs in the channel content (the message was always visible, and remains so).

      3. The report’s Status is updated to “Flag Dismissed” to indicate the message was safe and kept.

      4. The post would also get Reviewed by, Reviewed at, and Reviewer’s comment filled in, similar to the removed case.

      5. Notifications on dismissal: The system triggers notifications according to the configuration.

 

 

Open

 

 

 

 

 

c. If the reviewer clicks Run playbook: (Optional Action) This would be used if the flagged message indicates a larger incident (for example, a serious data spillage might trigger an incident response process). By clicking Run playbook, the reviewer can launch a predefined incident playbook (such as a Security Incident response) with context about this message. The flagged message information might be passed as parameters in the run summary.

1. The reviewer might run a playbook before or after taking the decision of removing or keeping the flagged message. The flagged message is not resolved by this action.

Edge Cases and Special Scenarios for Reviewers

• Content of a flagged message may continue to be available in email notifications.

• Flagged message deleted by author before review: If the author manages to delete their message after it was flagged (this could happen if hide=false and they had permission to delete their own post), the system should handle this gracefully.

  • The UI could update the status to something like “Removed” automatically with a note that the author deleted it.

  • The Content Review bot can update the flag’s status to Removed automatically, and add a comment on the report like “Message was deleted by its author at [time].” The Content Reviewer should still be able to see all information if any further action is needed (like running a playbook).

  • Essentially, the flag is resolved because the content is no longer available.

• Flagged message edited by author during review: For the MVP, we can either block editing on flagged messages or we can have the reviewer still act based on what was flagged (the original content is what they have in the record). If the edit “fixes” the issue (e.g., they removed the sensitive info), the reviewer may choose to dismiss the flag, but that decision is manual.

• Flagging in Private Channels: Content Reviewers might normally not have access to a private conversation. However, when a user flags a message in such a context, the system surfaces that message to the reviewers via the Content Review bot. The Content Reviewers thus gain visibility into content they otherwise might not see, but this is by design for moderation purposes.

• Multiple flags on the same message: Once a message is flagged, subsequent attempts by other users to flag it would be prevented. In the UI, if a message is already flagged (active in the queue), other users will get a notice like “This message is already flagged for review” if they attempt to flag the message. Only one flagged report exists per message, and the first flagger is recorded as the Reporter. In a future enhancement, we might allow multiple reporters to contribute (and show a count of how many flagged it), but not in this MVP.

• Multiple Content Reviewers: If multiple reviewers attempt actions on the same flag simultaneously (rare), the system should only allow one resolution.

• Post-resolution actions: Once a flag is resolved, the main actions are disabled, but reviewers can still use the flagged message report for notes or run a playbook if needed. The record stays in the DM. There is no built-in “undo” for a resolved flag.

Open questions

• Do we need to delete the message from the data base after it is reviewed? Or can it just be hidden from the channel similar to how the delete message functionality works today.

  • Would we need it to stay for audit and investigation purposes?

• Do we need the original flagged message to be viewable to reviewers after it is removed?

• Should system admins (or anyone else) be able to delete posts from the Content Review bot?

• Is it expected that all on a flagged root message will also be deleted/hidden? Should they be preserved?

• Do we need the ability to enable/disable content flagging per team?

• Do we need this functionality in DMs/GMs? Any potential privacy concerns there since that message will now be visible to Content Reviewers.

• If flagged messages are configured to NOT be hidden while under review, should there be an indication that they have been flagged for review that is visible to all users?