/
Integrating Usability Testing into the Mattermost Product Design Team's Practices

Integrating Usability Testing into the Mattermost Product Design Team's Practices

  • In progress

    • Overview

    To enhance the usability and overall user experience of Mattermost, the Product Design team will bring back structured usability testing into our design process with more consistency. This initiative aims to identify usability issues early, assess design decisions with user feedback, and continuously improve our product based on user-centered insights.

    Usability Testing Framework

    We will adopt an iterative usability testing approach using Useberry as our primary tool. This framework includes:

    • Early-Stage Concept Validation: Testing wireframes and prototypes to validate assumptions before development.

    • Pre-Release Usability Testing: Evaluating new features before launch to ensure a seamless user experience.

    • Post-Launch Evaluations: Gathering feedback on live features (where possible) on the community server from internal dogfooding and community members to inform iterative improvements.

    Testing Audience

    • Customers (Where Possible): While direct access to end users within customer organizations is limited, we will collaborate with customer champions and administrators to gather indirect feedback.

    • Mattermost Community: Open-source contributors and community members will provide valuable insights on usability and workflows.

    • Internal Users: Mattermost employees, including support, sales, and engineering, who interact with the platform regularly as 'guerilla testing'.

    Testing Methodologies

    Our usability testing approach will include:

    • Task-Based Testing: Participants complete predefined tasks while we track their interactions and friction points.

    • Click Testing & Heatmaps: Useberry analytics to understand interaction patterns and identify confusing UI elements.

    • Surveys & Open-Ended Feedback: Collect qualitative insights from users on their experience and pain points.

    • A/B Testing: Comparing design variations to determine the most effective user experience.

    Process & Integration

    1. Define Test Objectives: Identify the goals of each usability test based on upcoming feature releases or design changes.

    2. Create Prototypes in Figma: Design interactive prototypes that reflect the intended user flows.

    3. Build Test in Useberry: Prepare the testing questions, tasks and instructions for users.

    4. Recruit Participants: Engage community members, internal users, and available customer representatives.

      1. User Feedback Channel

      2. QA Contributors

      3. Design Reviews

    5. Conduct Tests & Analyze Data: Run tests asynchronously, gather quantitative and qualitative insights, and identify usability issues.

    6. Report Findings & Iterate: Share results with the product team, prioritize issues, and refine designs accordingly.

    7. Continuous Improvement: Establish usability testing as a recurring practice, ensuring that insights drive product decisions over time.

    Expected Outcomes

    • Improved usability and reduced friction in Mattermost workflows.

    • Increased confidence in design decisions before implementation.

    • Stronger alignment between user needs and product development.

    • A scalable and repeatable approach to usability testing within the Product Design team.

    Next Steps

    • Formalize a usability testing schedule aligned with product development cycles.

    • Establish a repository for usability insights to track findings and improvements over time.

    • Train team members on Useberry best practices for consistent and effective testing.

    • Communicate testing results and insights regularly to stakeholders.

    By embedding usability testing into our design processes, we can make data-driven decisions that enhance the user experience and drive continuous improvement in Mattermost.