Medical AI: Bristol Introduces the Learning Together Framework

Key Takeaways

1Researchers from Bristol propose Learning Ensemble, a framework for validating medical AIs inspired by medication leaflets

2Three areas need to be documented: limitations of use, equity among patient groups, clinical appropriateness

3Studies from 2021 and a triage case illustrate failures and biases that motivate this approach

4This framework is presented as a starting point requiring expertise, external review, and constant adjustments.

Why it matters : This framework aims to make the use of medical AIs more reliable and suited to clinical practice by anticipating risks of failure and inequalities.

Researchers from the University of Bristol have proposed a validation framework for medical AI systems, designed like a "package" similar to medication leaflets. Named Learning Ensemble, it aims to clarify usage limits, verify equity among patients, and assess clinical appropriateness before any deployment. Several documented cases of failure and bias serve as warnings and motivate this approach.

Clinical Reliability: An Ongoing Project with Expertise and Oversight

The development of reliable medical AIs is described as a process of trial and error, and this work is presented as a starting point. According to them, this project requires expertise, external review, and constant adjustments. They consider clinical appropriateness a priority, reminding that a system can technically function while being practically useless. One cited example shows a system that classified asthmatic patients with pneumonia as low risk for mortality. In the training data, these patients survived more often because they received particularly aggressive treatment in emergency rooms. For triaging new patients, this signal proved worthless.

Three Checks to Document Before Treating with AI

Learning Ensemble encompasses three components that developers must document and verify before any use on patients. The first component is the system's limits: targeted physician and clinic profiles, required hardware, and patient data used for training. The second component is reliability across all patient groups, beyond an overall average. The third component is appropriateness for daily clinical use, which conditions the system's relevance concerning the medical objective pursued.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Data Specificities and Inequalities: Pitfalls Already Observed

AI systems may succeed in initial tests but fail in clinical settings by capturing signals unrelated to the diagnosis. In 2021, it was shown that a tool intended to detect COVID in X-rays relied on ancillary details correlated with the diagnosis, then failed after deployment in another clinic. An average success rate can also mask recurring errors in certain groups: another 2021 study found lower disease detection in underserved populations. The deployment of such systems could have harmed patients already facing lower-quality care.

Medicine as a Model: Towards an Information Package for AI

The Bristol researchers believe that medicine has learned to use treatments with partially opaque functioning, and that AI could take inspiration from this. They propose systematically testing the reliability of medical AI systems and framing their use with an information package, akin to that of medications. In pharmaceuticals, a structured leaflet specifies conditions of use, dosage, timing, and patient group, transforming a substance into a reliable therapy. Transposed to AI, this package would aim to avoid irrelevant signals and underestimating risk, equipping developers with a common language and structure to detect problems earlier.