Fin Labs unveils Monitors: Transparency at the Heart of AI

Fin Labs Paris: A Step Towards Transparency with Monitors

In a recent announcement in Paris, Fin Labs introduced Monitors, a new product line that enhances Fin's existing offerings, which already include Insights and Recommendations. This comprehensive suite aims to improve observability and trust in Fin's operations, enabling businesses to better understand and control what their AI is doing.

With Monitors, companies can now choose which conversations, whether managed by Fin or by humans, should be reviewed. Thanks to Custom Scorecards, they can establish evaluation criteria that reflect their specific business priorities. This ensures monitoring of essential metrics, thus providing complete control over customer support quality.

In addition to Insights and Recommendations, Monitors provides all the necessary tools to analyze support operations, evaluate each interaction based on defined criteria, and strive for an impeccable customer experience.

The Growing Importance of Transparency and Control

The AI industry is rapidly evolving, with agents capable of handling increasingly complex tasks, including conversations with real-world consequences. Fin, for example, serves nearly 8,000 clients and boasts a 67% resolution rate, processing around 2 million customer requests each week, including in regulated sectors.

As agents take on more complex tasks at scale, observability becomes crucial. Currently, many support leaders cannot confidently answer fundamental questions about their agent's performance: Does it provide a good experience? Does it effectively resolve complex issues? Does it reliably represent the brand? Traditional infrastructures are no longer sufficient, and CSAT scores or QA samples are not scalable enough.

The challenge is to transform this "black box" into a transparent system. That’s why Fin was designed with transparency as a priority, allowing teams to understand and optimize the system autonomously.

Fin's Continuous Improvement Cycle

Analysis is a crucial step in uncovering what is really happening and initiating improvements. To trust an AI operation, three elements are essential:

• A comprehensive understanding of the interactions between Fin, the human team, and customers.
• A means to monitor and evaluate conversations based on criteria relevant to the business.
• AI-based recommendations to act on the findings.

Last year, Fin launched Insights and Recommendations to address these needs. Today, with the introduction of Monitors, the company completes a total observability system, thus opening the black box.

Monitors: Ensuring Standards Compliance in Every Interaction

Understanding a customer's feelings after a conversation is different from knowing whether that conversation was handled correctly. Both aspects are crucial for ensuring quality service.

Monitors offers a new QA capability that allows defining which conversations should be reviewed and evaluating them based on specific quality criteria. This system replaces ad hoc sampling methods and spreadsheet-based assessments with a scalable process, suited to the increasing volume of interactions.

Two key components make up this system: Monitors, which determines the conversations to be reviewed, and Custom Scorecards, which define the evaluation criteria.

Targeting the Right Conversations with Adequate Coverage

Random sampling has always been an imprecise method. When AI handles thousands of conversations each week, an arbitrary sample does not reliably capture edge cases or complex escalations.

With Monitors, companies can define how conversations are selected and evaluated. This may include targeting specific risk signals, such as "the customer showed signs of financial vulnerability" or "Fin repeated the same response without resolving the issue." Alternatively, generic and consistent samples can be created to assess quality over time. Criteria can be defined from an existing list of filters or described in natural language for more nuance.

Companies can combine these approaches to focus on the most important conversations while maintaining a structured and regular QA sample each week.

Custom Scorecards: Consistently Applying Your Standards

Every business has its own priorities and trade-offs, meaning a standardized quality grid may not necessarily reflect what customers truly value.

Custom Scorecards allow businesses to define what constitutes a "good" interaction for them and translate that into a personalized quality score for each conversation. Criteria can be automatically scored by AI, reviewed by a human, or both, within the same grid. This allows for combining scale and judgment in a single system.

Each conversation is then evaluated according to these criteria, and an overall quality score is calculated based on the company's configuration. Criteria can be weighted according to their importance, and some can be marked as critical, so that a single failure can invalidate the entire assessment if necessary.

The result is a unique and consistent quality score that reflects the company's standards, rather than a generic metric or a collection of disconnected checks. This allows for measuring quality over time and tracking the performance of both AI and human support against a consistent definition of "good."

Review Queue: Transforming Alerts into Corrective Actions

When conversations require human review based on the defined criteria, they are placed in the Review Queue. Each conversation corresponding to a Monitor is automatically assigned to the appropriate reviewer, with its scorecard attached and the review status tracked.

Reviewers work on the conversations directly in Intercom, filling out the grid criteria as they go. When a conversation fails, they mark it, add a note on what went wrong, and can suggest potential solutions, such as updating documentation. These conversations then move to a follow-up stage, where the team can apply corrections.

This ensures that no information is lost in a spreadsheet or a Slack thread, and that QA stops being a loop that ends with a score to become a loop that ends with improvement.

Reporting: Using QA as a Continuous Signal

Reporting connects quality scores to the entire operation. Companies can track review scores over time, across Monitors and Scorecards, and compare them directly with the CX Score, resolution rate, and other performance metrics.

Patterns that were previously invisible become clear: a specific topic consistently underperforming, a drop in quality coinciding with a recent change in the knowledge base, a team whose scores improve week after week. QA data becomes a continuous signal on how the operation is improving, rather than a one-off exercise that lives in a separate tool.

Future Perspectives

Monitors for Fin conversations are available today, and future developments are already underway.

• QA for human agents will bring the same structured evaluation to your human team's conversations, providing a consistent quality system across your entire support operation.

• Real-time alerts will notify you as soon as a conversation crosses a threshold you have set, before the issue reaches more customers.

• Knowledge base evaluation will link AI scoring directly to your content, so conversations are assessed against your latest policies and documentation, detecting inaccurate or outdated responses and providing clear justification tied to the relevant source.

Towards Total Transparency

Creating a perfect customer experience with AI requires transparency. It is essential to understand how the system works to maintain and improve quality over time. With Insights, Monitors, and Recommendations, this is now possible. This complete analytics suite allows visibility into what happens in each conversation, ensures it meets company standards, and identifies opportunities for improvement when necessary.

Everything in the analysis step of the continuous improvement cycle has been built in close collaboration with Fin's clients. Their use cases, feedback, and honest conversations about what wasn't working have been essential. This partnership is why Fin is the highest-performing agent on the market. And the company has no intention of slowing down.