Multi-Source Alerts Research
A multi-method research programme uncovering root causes of alert complexity and shaping a new alerts architecture.
Impact & Outcomes
Overview
Project
at a glance
Smart Alerts are a core capability that enables PR and marketing professionals to monitor brand activity, detect risks, and respond quickly to emerging trends. However, over time the experience had become more complicated and less structured — the result of ongoing feature expansion and accumulated backend constraints.
Rather than jump to solutions, we approached the problem holistically. I led a research initiative spanning five methods, culminating in a cross-functional workshop that translated findings into a prioritised product roadmap. The work ran in close collaboration with a Product Manager, UX researchers, and engineering partners.
Each method provided a unique perspective on the ecosystem, enabling us to build a comprehensive understanding of user needs and system constraints.
Surveys & Usability Studies
Conducted with Research Team
User & Sales Rep Interviews
Led by Product Manager & Me
Peer Design Reviews
(Self-Initiated)
Expert Heuristic Evaluations
by Research Specialists
Cross-Functional Workshop
With PM, Eng. Manager & Research Team
General Survey & Card Sort
The team aimed to better understand how easy it is for users to create and manage Smart Alerts, with a focus on identifying pain points, simplifying the process, and exploring opportunities to consolidate or improve alert types.
Respondents: 28 Meltwater Alert users (3% response rate) Research format: General survey questions and a card-sorting exercise
Goals
- Identify user pain points
- Discover which Smart Alerts are most frequently used and the primary use cases they support
- Understand the perceived usefulness of alerts
- Determine whether any alerts could be consolidated
Key Findings
- Users experience significant friction due to alert overload and limited control.
- Users want greater flexibility, control, and transparency in how alerts are triggered and configured.
- Brand monitoring (68%) and crisis preparation (57%) are the top use cases, with Every Mention and Spike Detection serving as the primary supporting alerts.
- The highest priority for improvements is ensuring content accuracy (25%), followed by optimising alert thresholds (18%).
- Many alert types are underutilised due to lack of relevance or user understanding.
- 64% of users reported not using certain alerts because they do not apply to their workflow — not because the alerts themselves are ineffective.
Recommendations
- Provide fewer alert types with greater customization and control.
- Reduce overlap by consolidating similar alert types.
- Allow users to manage alert triggers through threshold settings or batching options.
- Make alerts more digestible by introducing AI summaries and improving existing explanations to clarify value.
Sales Representative Interviews
Three Sales and Premium Support representatives were interviewed through semi-structured sessions, workflow walkthroughs, and artifact review of existing alerts and searches. These are the power users of the system — managing alerts for large enterprise clients at scale — and the friction they experienced was acute.
Key findings: bulk operations are a critical unmet need, particularly during product launches, crises, and client onboarding where dozens of alerts must be created simultaneously. The 10-recipient limit forces representatives to duplicate alerts for the same search, creating compounding maintenance overhead. Searching for existing alerts is so broken that representatives resort to browser Ctrl+F as a workaround. External email management is particularly painful — no filtering, no bulk input, emails must be added individually. In extreme enterprise cases, setting up alerts for a single client took up to 1.5 days and required 7 people.
Recommendations: introduce bulk alert creation for search-based alerts; increase or remove the recipient limit; add native search and filtering to the alerts table; simplify external email entry with multi-email input; explore automation rules to reduce manual setup.
Search & Navigation Friction
Theme: Alerts are hard to find and manage
Raw Insights
- SR using Ctrl+F that takes ~20 minutes to find a search
- If the search isn't visible in preview, Ctrl+F won't catch it
- No way to find it aside from manually checking each one
- "All Alerts" category would be too crowded
Pattern Identified
- CRUD table lacks scalable search & filtering
- Visibility hierarchy issues
- Navigation doesn't match workflow mental model
Recipient Limit Constraints
Theme: System limits create operational complexity
Raw Insights
- 10-recipient limit forces duplicate alerts
- Multiple alerts created for same search due to recipient cap
- Backend architecture built around hard limits
- Increasing limits has heavy backend implications
- Decision historically constrained scalability
Pattern Identified
- Artificial limits create alert duplication
- Scalability problem, not UX surface problem
- Structural tech debt impacts experience
External Email Management Issues
Theme: External users are hard to manage
Raw Insights
- No filtering for external emails
- External emails must be added a few at a time
- No stored list of all external users
- Validation confusion (limit vs format first?)
- Email truncation makes visibility difficult
Pattern Identified
- External users treated differently
- No scalable filtering model
Bulk Alert Creation
Theme: Multi-alert creation is common for search-based alerts
Raw Insights
- Bulk creation needed for launches & crisis monitoring
- New search often requires multiple alerts
- Most common alerts: Spike Detection & Top Reach
- Bulk creation means search-based alerts only
- Extreme case: hundreds of alerts took ~1.5 days, ~7 people
Pattern Identified
- Search based alerts lacks bulk creation option
- Reduce Sales operational overhead
User Interviews
Six Meltwater customers across PR, corporate communications, and agency roles participated in moderated usability sessions with task-based scenarios covering alert creation, modification, and receipt. This method moved beyond stated preferences into observed behaviour — surfacing friction that users had normalised and could not easily articulate in a survey.
Key findings: Every Mention alerts are mission-critical for daily threat scanning. Alert usage is highly event-driven, with volume fluctuating dramatically around campaigns and crises. Users manually validate alerts daily to catch outliers the system misses. Regional comparison across US, EU, and APAC markets is essential for crisis strategy — a need the current system doesn't support well. Alert expiration is managed through naming conventions because the system offers no native lifecycle management. Executive reporting requires manual screenshots because there is no export path.
The Smart vs. System Alerts distinction was a persistent source of confusion — users had built incorrect mental models about which system was responsible for which notifications, leading to missed alerts and misplaced troubleshooting.
Recommendations
Reduce configuration friction
Improve mental model alignment
Increase system transparency and feedback
Support real workflows, not just alert mechanics
Introduce reporting and sharing capabilities
Peer Review
Two product designers from cross-functional teams evaluated the Smart Alerts experience through moderated usability sessions. The focus was on identifying friction points, validating interaction patterns, and uncovering inconsistencies across alert entry points.
Respondents: 2 Product Designers (cross-team peers) Research format: Moderated usability sessions with task-based scenarios covering alert creation, modification, and management across Explore and Alerts pages
Goals
- Identify usability issues in alert creation flows across multiple entry points
- Validate clarity of configurations and terminology
- Evaluate bulk creation behaviour and system feedback states
- Discover inconsistencies between Explore-based and Alerts-page experiences
- Identify opportunities to simplify navigation and reduce confusion
Key Findings
- Overall usability was perceived as clean and intuitive, but interaction inconsistencies created friction.
- Pre-selected alert types caused mixed reactions — helpful in context, confusing when unexpected.
- Navigation between Explore and Alerts contexts introduced cognitive load due to inconsistent system behaviour.
- Delivery configuration lacked clear defaults, increasing decision friction.
- System feedback states (processing, loading, counts) created uncertainty about whether actions succeeded.
- Structural differences between Smart Alerts and System Alerts contributed to mental model confusion.
Strengths Observed
- Alert configuration structure was generally clear and logical.
- Examples and previews were highly valued for understanding outcomes.
- Bulk creation concept was well understood and perceived as valuable.
- Visual hierarchy and layout were considered clean and modern.
Recommendations
- Standardise preselection logic across entry points.
- Keep users in context — prioritise modal overlays over navigation jumps.
- Clarify Smart vs System Alerts mental model through clearer grouping or labelling.
- Replace informational alert banners with more appropriate descriptive components.
- Improve discoverability of alert categories (e.g., tab visibility).
Expert Heuristic Evaluation
Two UX researchers conducted a structured heuristic evaluation based on Nielsen's 10 Usability Heuristics, covering the full alert lifecycle — from creation through management to receiving and interpreting alerts in email.
Reviewers: 2 UX Researchers Research format: Heuristic evaluation with task-based walkthroughs across Explore, Monitor, Alerts page, and email notifications
Goals
- Identify usability and workflow friction across the full alert lifecycle (create → manage → receive → interpret)
- Evaluate consistency and clarity across entry points and configurations
- Assess confidence signals and system feedback when alerts are created or triggered
- Identify opportunities to simplify alert selection and reduce cognitive overload
- Determine areas where alerts could be consolidated or clarified
Key Findings
- The most significant issues occur not during creation, but when users receive and interpret alerts.
- Users lack confidence alerts are functioning correctly due to weak feedback and unclear processing states.
- Navigation and workflows are inconsistent across entry points, creating mental overhead.
- Alert terminology relies heavily on internal jargon rather than user language.
- Email notifications are cognitively overloaded and hard to distinguish at a glance.
- Inconsistent UI patterns (bell icon behaviour, dropdown usage, field placement) violate expectations.
- Critical configuration details often appear below the fold while secondary settings are elevated.
- Absence of previews and guidance increases uncertainty throughout setup.
- Finding and managing existing alerts is inefficient due to weak search and cluttered lists.
Cross-Functional Synthesis Exercise
After completing the expert review, we facilitated a working session with Research and Product to translate findings into actionable direction. Participants generated solution ideas — some pre-seeded with research recommendations — then collaboratively mapped them to the core pain points identified in the evaluation. The exercise helped us cluster themes, distinguish quick wins from structural opportunities, and align on a simplified, use-case-driven alert framework.
Recommendations
Simplify Alert Creation
Improve System Feedback & Confidence
Make Alert Selection Easier
Improve Alert Consumption & Interpretation
Reduce Workflow Disruptions
Improve Findability & Management
Quick Wins Identified
Strategic Opportunities
Workshop & Strategic Synthesis
The cross-functional workshop was the final synthesis phase — bringing together Product, Design, Research, and Engineering to translate five streams of research into a shared, actionable product direction. Rather than another ideation session, the goal was alignment: on root causes, on what was feasible, and on a simplified alert framework.
The session began with a structured walkthrough of all research findings to ground decisions in evidence. We then mapped current alert types against actual user use cases, evaluated configuration overlap, and pressure-tested consolidation hypotheses against technical constraints and existing mental models.
Key findings: the current ecosystem contains too many alert types with overlapping purposes, creating cognitive overload. Users think in terms of goals — brand monitoring, crisis detection, campaign tracking — not alert names. Multiple alerts detect similar signals (volume change, sentiment change, reach), creating consolidation opportunities with manageable risk. Threshold control and AI summarisation were identified as the highest-leverage mechanisms for reducing alert fatigue.
The workshop produced a prioritised action framework across four dimensions: quick usability wins, consolidation candidates, structural architecture changes, and longer-term AI-powered capabilities. This became the research-backed roadmap that directly shaped the Smart Alerts Redesign.
The workshop translated research insights into a clear action framework across four dimensions:
- Structural complexity identified as primary friction driver
- Too many alert types create decision fatigue
- Feature-heavy model increases cognitive load
- Alert system does not align with user intent
- Configurability valued over proliferation
- Reduce number of alert types
- Shift to use-case-driven architecture
- Increase flexibility within fewer core alert models
- Simplify mental model across creation & management
- Alert types overlap in functionality
- Usage data used to assess consolidation risk
- Backend implications evaluated for each merge
- Low-adoption alerts lack clear value
- High-usage alerts require careful transition
- Pilot low-risk merges (Industry + Company Events)
- Phase evolution of high-usage alerts (Every Mention + Spike Detection)
- Retire or absorb low-adoption alerts
- Reduce system redundancy through structured consolidation
- System currently structured around feature categories
- Users think in goals, not alert types
- Alert fatigue linked to volume & unclear thresholds
- Low-usage alerts require careful transition
- Need for a guiding framework
- Start with user intent, not alert type
- Favor configurability over proliferation
- Reduce cognitive load at every decision point
- Introduce thresholds & AI summaries to manage fatigue
- Maintain user trust during phased evolution
- Reduce alert creation steps
- Improve terminology and clarity
- Surface most-used alerts contextually
- Improve feedback after alert creation
- Redesign alert architecture around use cases
- Introduce threshold customization
- Implement AI-assisted recommendations
- Consolidate overlapping alert types via phased rollout
- Redesign alert architecture around use cases
Next case study
Alert Forecasting & AI Insights
Shifting spike detection from reactive notification to proactive decision support — with forecasting and AI-powered insights.