Explainable AI

What explainability means in practice

A screening tool that says "Candidate score: 72" is a black box. An explainable one says something like: "Meets 4 of 5 required criteria: customer service experience (3 years), weekend availability, bilingual English and Spanish, passed typing test at 45 WPM. Missing: valid driver's license." A recruiter can check each point against the application and overrule it if the tool misread something.

Good explanations in hiring are tied to job-related criteria, specific to the candidate, consistent (similar candidates get similar reasons) and written in plain language. Feature-importance charts that only a data scientist can read do not help a recruiter at 4:00 on a Friday.

Why it matters for hiring teams

• Human review. Human-in-the-loop only works if the human can see why the AI recommended something. Without reasons, review becomes rubber-stamping.
• Bias detection. When explanations show a tool leaning on employment gaps or particular zip codes, you can catch proxies for protected characteristics before they cause AI bias at scale.
• Candidate trust. Candidates who are rejected increasingly ask why. Clear, job-related reasons are easier to give and defend.
• Defensibility. If a selection procedure is challenged for disparate impact, the employer has to show it is job-related. That is hard to do for a model nobody can explain.

Legal drivers

The EU AI Act classifies AI systems used for recruitment and selection as high-risk. Deployers must use them under human oversight and, under the Act, people affected by certain high-risk AI decisions have a right to an explanation of the role the system played. GDPR Article 22 limits decisions based solely on automated processing that significantly affect people, and data subjects are entitled to meaningful information about the logic involved. In the US, New York City's Local Law 144 requires bias audits and candidate notices for automated employment decision tools, and from January 1, 2027 Colorado's AI law (SB 26-189) requires employers to explain, within 30 days, adverse decisions that automated tools materially influenced and to offer human review. Rules vary by jurisdiction and are changing, so confirm current obligations with counsel.

Questions to ask vendors

• What reasons does the tool show for each recommendation, and can recruiters see them on every candidate?
• Which inputs does the model use, and which are excluded (name, photo, age, address)?
• Can a recruiter override a recommendation, and is that logged?
• What bias testing has been done, by whom, and how often?
• Can you export decisions and reasons for audit?

These questions belong in your wider AI governance process, alongside security review and data retention.

Limits

Explanations can be incomplete. Some techniques approximate what a complex model did rather than show its exact reasoning, and a convincing explanation can still hide a biased pattern. Pair explanations with outcome monitoring, such as pass rates by group at each stage, rather than trusting either alone.