AI Bias Audit

What a bias audit actually tests

At its core, an audit asks one question: does this tool treat groups differently in outcomes, and if so, can you explain and justify the difference? It applies to anything that scores, ranks, filters or recommends people: resume screening, chatbot knockout logic, game-based and cognitive assessments, AI-scored video or voice interviews, and job ad targeting.

A useful audit has three layers:

1. Outcome analysis. Selection or scoring rates and impact ratios by sex, race and ethnicity, age (40 and over vs under 40) and, where data allows, disability and intersectional groups such as Black women.
2. Statistical testing. Whether gaps are larger than chance would explain, using tests such as Fisher's exact test or a two-standard-deviation rule, plus practical significance. Small samples can make a ratio swing wildly.
3. Root cause and validity. Which inputs drive the difference, whether those inputs are job-related, and whether a less discriminatory alternative would predict performance as well.

The first layer is what NYC Local Law 144 requires. The second and third are what protect you if a tool is challenged under Title VII.

The legal reference points

In the US, the starting point is the four-fifths rule from the Uniform Guidelines on Employee Selection Procedures (1978): a selection rate for any group below 80% of the highest group's rate is generally regarded as evidence of adverse impact. It is a rule of thumb, not a safe harbor. Courts and agencies also look at statistical significance, and a ratio above 0.80 can still be challenged when the sample is large.

Title VII's disparate impact framework applies to algorithms the same way it applies to paper tests. If a tool causes adverse impact, the employer must show it is job-related and consistent with business necessity, and the candidate can still win by showing an equally effective, less discriminatory alternative existed. Using a vendor's tool does not shift that liability away from the employer.

NYC Local Law 144 requires an independent audit at least annually, with public results. Illinois's amendments to its Human Rights Act, California's Civil Rights Council regulations on automated decision systems and the EU AI Act each push employers toward testing in different ways. Colorado's replacement AI law, effective January 1, 2027, adds notice, explanation and human review duties rather than a testing requirement. Requirements differ by jurisdiction and keep changing, so check current rules for each place you hire.

Worked example: auditing a resume ranking tool

A national retail pharmacy chain uses an AI tool that ranks applicants for pharmacy technician roles; recruiters review the top 40%. Over 12 months, 18,000 applicants were ranked.

The HR analytics lead, Marcus, pulls outcomes by group. The top-40% rate is 43% for white applicants, 41% for Hispanic applicants, 38% for Asian applicants and 31% for Black applicants. Impact ratios against the highest group: 0.95, 0.88 and 0.72. The Black applicant ratio fails the four-fifths rule, and with about 3,000 Black applicants it is clearly statistically significant.

Root cause work shows the model rewards 'continuous employment' and certain retail chain employers in work history. Gaps in employment and experience at independent pharmacies push candidates down, and neither predicts 90-day performance in the company's own data. The vendor retrains without the gap feature, the ratio rises to 0.86, and recruiters now review the top 50% while the change is validated. Marcus documents everything, because that file is the employer's defense if the tool is ever challenged.

Bias audit vs validation study

People confuse these constantly. A validation study shows that a tool predicts job performance: scores correlate with supervisor ratings, sales, quality scores or retention. A bias audit shows whether outcomes differ across groups.

You need both. A highly valid test with severe adverse impact can still be challenged if a less discriminatory alternative exists. A tool with perfect impact ratios that predicts nothing is a coin flip with extra steps. For high-volume roles, ask vendors for both: criterion validity evidence for jobs like yours, and adverse impact results from your own applicant pool, not a vendor-wide average.

Common mistakes

• Auditing the vendor's data, not yours. Impact depends on your applicant mix, cutoffs and how recruiters use the output.
• Testing only the tool, not the decision. If recruiters only call the top 20%, audit that cutoff, not the raw scores.
• Missing demographic data. Voluntary self-identification rates under 50% make results unreliable. Improve collection, and never infer race from names in place of it.
• Ignoring age and disability. Local Law 144 covers sex and race only, but federal law protects more groups.
• Running it once. Models, job mixes and applicant pools change. Audit at least annually and whenever you change a cutoff or model version.
• Auditing without authority to act. Decide in advance who can pause a tool.

How to run a practical audit

Inventory every automated step in the funnel and note what decision each one drives. Collect self-identified demographic data through a voluntary form kept separate from hiring decisions. For each step, calculate rates, impact ratios and significance tests by group, at the cutoff actually used.

Where a ratio falls below 0.80 or a gap is significant, investigate which features or questions drive it, check them against job analysis, and test alternatives such as a different cutoff or removing a feature. Track the fix in the next audit. Keep the whole file: data, method, findings, decisions and who made them. For the wider picture of how algorithms pick up bias in the first place, see AI bias.