Override Labs: AI for Secure Teen Consent

An AI to Prevent Sexual Assault: An Innovative Approach

Imagine an artificial intelligence capable of preventing sexual assaults before they even occur, while respecting users' privacy and without judgment. This is the challenge that Override Labs has taken on with its flagship product, Is This Okay? (ITO). This tool provides adolescents with a safe space to reflect on ambiguous sexual situations, relying on artificial intelligence and rigorous clinical research.

In an episode of Just Now Possible, Teresa Torres spoke with Priya Nakra, founder and product lead, and Olivia Rowley, AI advisor and board member of Override Labs. Their nonprofit organization is dedicated to preventing gender-based violence, and ITO is designed to provide non-judgmental support to young people.

A Data-Driven Development with Clinical Expertise

To design ITO, the Override Labs team analyzed 2,000 Reddit posts from various sub-forums. This approach validated the need for a tool that allows adolescents to reflect on consent issues. The organization collaborated with a licensed therapist to develop a risk assessment framework, ensuring that every aspect of the product is based on solid clinical principles.

Every design decision was made to prevent the tool from being used for harmful purposes. For example, ITO never offers a "green light" type of response that could be interpreted as permission to proceed. Instead, the tool guides users through a thorough reflection on their choices.

A South Star to Avoid the Worst

Unlike many tech companies that set "North Stars" to guide their development, Override Labs chose to define a "South Star." This approach involves identifying the worst possible outcome and designing the product to avoid it. This has allowed the team to focus on preventing harm rather than growth at any cost.

Clinical expertise played a crucial role in defining the tone and structure of the AI. The tool's response structure is based on motivational interviewing, comprising three parts: validate, reflect, and invite reflection.

A Privacy-by-Design Approach

One of the core principles of Override Labs is "privacy by design." This means that ITO does not require account creation, does not use cookies, and does not track users between sessions. This approach ensures that adolescents can use the tool confidentially, without fear of being monitored.

Risk classification is performed deterministically even before the AI Claude is invoked. This allows the tool to tailor its responses based on the level of risk while respecting users' privacy.

Measuring the Impact of Invisible Prevention

One of the major challenges facing Override Labs is measuring the impact of their product. Indeed, the success of ITO is reflected in the absence of negative events, making it difficult to assess its effectiveness. However, the organization remains committed to building an ecosystem of products aimed at preventing gender-based violence.

Override Labs is also developing a tool for women and a web game serving as the main funnel to raise user awareness. An institutional API layer is also in preparation to extend the impact of their solutions.

A Philanthropic Model for Sustainable Impact

The development of ITO was made possible through the support of the Fund to Prevent Sexual Assault, a philanthropic incubator that funded the first group of Override Labs projects. This nonprofit model allows the organization to make prevention-focused design decisions without the profitability constraints faced by traditional companies.

Priya Nakra left a 10-year tech career to found this organization, highlighting her commitment to preventing gender-based violence.