Brief IA
›

AI: 10% of Companies Say They Receive Explanations

Business & Startups·Tom Levy·

AI: 10% of Companies Say They Receive Explanations

AI: 10% of Companies Say They Receive Explanations
Key Takeaways
1In a survey of a recruitment software publisher, 82% of companies consider the transparency and explainability of AI to be important.
2Only 10% say they benefit from it regularly.
3The recruiter’s judgment prevails in 58% of disagreements with AI, according to this company.
Why it matters : a ranking guides the review of applications, while automated filtering directly determines which can proceed in the process.

Only 10% of companies surveyed in research by a recruitment software publisher report regularly benefiting from transparency and explainability for AI, while 82% consider these aspects important. The AI director of this unnamed company advocates for extensive automation of preparatory tasks but insists on maintaining human intervention to decide the fate of applications.

Employers Must Be Able to Explain Their Choices

In New York, regulations already govern automated tools involved in employment decisions. Illinois has expanded protections related to the use of AI in employment. In the European Union, many employment-related systems fall under the high-risk category of the AI regulation. Other jurisdictions are developing their own frameworks, also covered by labor law.

The specific requirements are expected to evolve, according to the publisher's executive. He urges organizations to understand how their tools work, the reasons behind the results obtained, how they are used, and the identities of those responsible for decisions. However, an explanation does not guarantee that a result is accurate or fair. It can enable review, challenge, and effective oversight.

Needs differ depending on the role in recruitment: understanding the priority assigned to a file for the recruiter, being able to justify a pre-selection for their manager, obtaining a substantial explanation for the candidate. An opaque score does not meet the first of these needs. For the executive, this capacity for explanation should also be a product requirement.

50% of Companies Cite Efficiency as the Primary Reason for Adopting AI

According to research conducted with Aptitude Research, 50% of companies cite efficiency as the primary reason for adopting AI. Teams particularly measure recruitment timelines, the volume of work manageable, and the cost of each hire. However, the executive warns against a management approach dominated solely by these indicators.

Candidate trust, which is harder to quantify, can also have a commercial impact. He reports that research mobilized by his company suggests discomfort among a significant proportion of candidates when AI decides on a hire. This does not demonstrate that they refrain from applying, nor that the remaining individuals are less qualified. The mentioned risk remains conditional: if strong candidates withdraw due to lack of trust, a seemingly more efficient process could deplete the talent pool.

On the recruiter side, the company's research indicates that their judgment prevails in 58% of cases of disagreement with AI. The director does not treat these arbitrations as automatic proof of a technical failure. He invokes George Box's formula, "all models are wrong, but some are useful": the usefulness of a model may lie in improving human judgment, without replacing it.

Classifying or Filtering Applications

The AI director, who claims to have dedicated his career to the responsible development of AI and machine learning, does not advocate for a general reduction in automation. The clients surveyed by his company even rank the efficiency of recruiters through AI as their top priority. Faced with time and volume constraints, he advocates for more automation upstream but refuses to conflate the number of tasks assigned to a tool with the authority granted to it. His caution pertains to decisions with significant effects made from incomplete information and imperfect models.

La lettre du soirGratuit · 19 h

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Both filtering and classification utilize the characteristics of positions and individuals, without granting the same power to the system. The former applies a pre-set threshold: some candidates proceed in the process, while others are excluded. It assumes fairly complete data, appropriate criteria, and a sufficiently reliable model near the threshold, conditions that the executive deems difficult to meet for decisions concerning individuals. The latter primarily directs the recruiter's attention toward certain files.

However, classification does not prevent practical exclusion: in the example given, a person ranked 8,000th is unlikely to be reviewed if only 100 files are consulted. The difference lies in the ability to adjust the limit as applications and the position are discovered. The company sometimes summarizes this distribution as "AI suggests, human decides." The executive calls for human participation in any important decision, particularly one that determines whether an application continues.

The Limits of Files and Tasks Assigned to AI

Most AI-based recruitment systems rely on a job description and a CV, two partial representations. The former may reuse an old document or a previous role, even before the team has identified the qualities distinguishing an excellent candidate. Some expectations only become clear during interviews. The CV, on the other hand, condenses years of activity, also reflecting the ability to narrate one’s journey and increasingly includes sections written or revised by generative AI.

These limitations do not negate the value of the documents; interviews provide complementary information. A model can compare the received elements and detect patterns, but it cannot retrieve information that was not present. The deductions it produces carry their own uncertainty.

The possible assistance remains extensive: searching for candidates and information, identifying relevant experience, comparing files against criteria, ordering them, producing a summary, or presenting elements to support a recommendation. AI can also prepare exchanges, assist with planning and communication, answer questions, or lighten administration. These uses can reduce preparatory work and, according to the director, improve both the quality and consistency of human decisions. He anticipates a growth in the automatable portion of recruitment with technical advancements.

Biases Can Be Embedded in Data History

Removing a human from an arbitration does not automatically render the whole process neutral. Biases can be embedded in the data history, their labeling, sampling, or variables indirectly used to represent other characteristics. They can also stem from the objectives assigned to the model, evaluation criteria, product choices, or human use of the results.

The executive therefore rejects the idea of a definitive correction during development. Rigorous engineering, improved data, careful evaluations, and continuous monitoring can reduce risk; design choices can also limit the model's power. A defective system risks, conversely, repeating the same error consistently and at scale. He advocates for incorporating bias control, human oversight, and accountability requirements from the design stage.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.