Schmidt Sciences: Call for Projects for More Transparent AI

Schmidt Sciences has recently launched a call for proposals for an ambitious pilot program focused on the interpretability of artificial intelligence (AI). The deadline for submitting proposals is set for Tuesday, May 26, 2026. This program seeks to explore new methods for detecting and mitigating deceptive behaviors of AI models, particularly large language models (LLMs). If this pilot project shows signs of significant progress, it could unlock much larger investments in this area.

Central Question and Program Objectives

The central question that Schmidt Sciences aims to address is: can we develop interpretability methods that detect deceptive behaviors exhibited by LLMs and guide their reasoning to eliminate these behaviors? The successful tools will need to generalize to realistic use cases, going beyond typical academic benchmarks. It is crucial that these interpretability tools outperform references that do not rely on access to weights, in order to prove that we can truly leverage our understanding of the internals of the models.

Research Agenda and Main Directions

Schmidt Sciences' research agenda outlines a research field with several key directions. Proposals do not need to match the topics of this agenda verbatim, but they are encouraged to explore any relevant technical method or evaluation that could advance our scientific understanding of deceptive behaviors in LLMs. We will focus particularly on three directions:

• Detection of Deceptive Behaviors: can we develop tools to detect these behaviors, defined as cases where there is a contradiction between what a model says (or does) and what it internally represents as true (or the best action)?

• Guiding Models to Improve Truthfulness: can we develop targeted guidance methods to intervene on the truthfulness of models? We would like to leverage a better mechanistic understanding of the models to develop mitigations for deceptive behaviors.

• Applications of Detection/Steering Methods: can new detection and steering techniques unlock AI use cases? When can these techniques practically enhance human-AI teams? When will having a more truthful AI improve outcomes in multi-agent systems?

Selection Criteria and Funding

Proposals will be evaluated by Schmidt Sciences staff and external reviewers based on several criteria:

• Alignment with the Research Agenda: Does the proposal clearly engage with the intent behind the scientific questions and objectives of the research agenda?

• Scientific Quality and Rigor: Is the proposed work technically sound, well-motivated, and capable of producing generalizable insights?

• Potential Impact: If successful, would it significantly advance AI interpretability or change the way risks are understood, measured, or managed?

• Feasibility and Scope: Is the project appropriately sized relative to the requested budget and duration?

• Team Expertise: Is the team well-positioned to execute the proposed work, with relevant technical expertise, sufficient capacity, and a time commitment proportional to the project's ambition?

• Cost-Effectiveness: Is the proposed budget reasonable and well-justified given the project's objectives and planned activities?

Funding for selected projects will range from $300,000 to $1 million, for a duration of 1 to 3 years.

Importance of Interpretability and Current Challenges

Research on interpretability is particularly promising for reducing risks associated with deceptive behaviors in LLMs. These models often mislead users, even on simple tasks with innocuous prompts. The results of interpretability analyses could also enable new forms of guidance for honesty. Promising methods have shown they can steer models toward truthfulness in a generalizable way, robustly optimize against reward signals to reduce deception, and allow for constrained fine-tuning of models that operates only on interpretable features.

However, we do not yet have universal detectors for deceptive behaviors, nor can we reliably steer models to be completely truthful. Thus, we aim to support research on relevant open problems.

Major Research Directions

• Defining "Deceptive Behaviors": we use this term to include instances of model generations known (by the model) to be factually incorrect, claims made with misleading confidence levels, deceptive assertions about the context of interaction, selective omission of relevant information, and other behaviors that models know to be deceptive to humans or AI monitors.

• Monitoring: this area covers research on monitoring and validating model reasoning. We expect monitoring to involve any black-box testing of models, white-box probing, gray-box analysis techniques, and other research on methods and evaluations.

• Steering: this area covers research on representative and weight-based interventions aimed at mitigating deceptive behaviors without undesirable consequences. We are particularly interested in methods that surpass traditional fine-tuning approaches by leveraging insights from interpretability analyses of model reasoning.

• Applications: this area covers work that applies detection and steering methods to draw new conclusions about trained models, training processes, or the utility of models for humans. We want interpretability techniques to reveal actionable insights.