Catching AI slip-ups before they cost insurers millions

How we started

We kicked off the project by collaborating with the client to define the user personas and understand their needs and expectations for the platform. We developed two personas in detail :

1. Carl - an executive persona  
2. Maya - a claims manager persona

We also mapped out key insurance processes like Claims Processing and Underwriting, to gain a clearer understanding of the challenges and data flow at each stage.

This foundational work helped us align the platform’s features with real-world business processes.

Discovery conclusions

The main conclusion of the discovery is that the primary pain point for the executive persona is the lack of overview and control over what is happening with the AI models.

The core of the OverseeAI platform is the Dashboard, which serves as the primary interface for monitoring model performance and the underwriting/claims process.

The dashboard displays key metrics, including AI model health indicators and anomaly rates real time.

We focused on providing a clean and intuitive layout, with visualizations like progress bars and charts that give executives and analysts a high-level view of AI performance and efficiency.

By streamlining the data presentation, we made sure that users could understand both the overall health of the system and the specific outcomes of individual models at each stage.

Anomalies are flagged throughout the underwriting and claims processes, and the Anomaly Detection page allows users to review these flags in real time.

The feature focuses on actionability, allowing users to mark anomalies as false positives or take corrective actions. Each flagged anomaly is accompanied by a feedback mechanism, where users can provide insights that improve model accuracy.

This continuous feedback loop ensures that the system evolves and adapts to new information.Key elements: