Bias in AI hiring tools is a serious and legitimate concern. Early AI systems amplified historical biases, sometimes dramatically, by learning patterns from tainted data. Legislative bodies in New York City, Illinois, Colorado, and the EU have responded with laws requiring disclosure, auditing, and in some cases, pre-deployment bias testing.
We welcomed this scrutiny — and we commissioned an independent third-party audit of Braintrust AIR to verify what we believed about our own system. Here's what the audit found.
The Audit Process
The audit was conducted by an independent firm with no prior relationship with Braintrust. Auditors were given full access to AIR's scoring methodology, rubric design, training data documentation, and a dataset of historical interview evaluations across a demographically diverse candidate pool.
The evaluation assessed two primary questions: Does AIR produce statistically significant disparities in scoring outcomes across demographic groups? And does the overall system architecture comply with applicable regulatory frameworks, including GDPR, EEOC guidelines, and NYC Local Law 144?
The Findings
The auditors found zero adverse impact across every demographic group tested — including race, gender, national origin, and age. Selection rate equivalence was confirmed at 100%: candidates from every demographic group passed through AIR's initial screen at statistically equivalent rates when controlling for the competencies actually required for the role.
The auditors specifically noted that AIR's rubric-based, transcript-driven evaluation architecture was a key factor in this outcome. By evaluating the semantic content of what candidates say — scored strictly against predefined competency criteria — rather than relying on subjective human judgment or unscientific visual or acoustic analysis, the system avoids the primary vectors through which AI systems introduce bias.
Regulatory Compliance
On the compliance side, the audit confirmed full alignment with GDPR data processing requirements, EEOC disparate impact standards, and NYC Local Law 144's audit and transparency requirements. The audit report itself constitutes the disclosure required under LL 144 and is available to any enterprise client subject to that regulation.
What This Means for Enterprise Buyers
If your legal team or procurement process requires documented evidence of bias testing as a condition of deployment, we have it. If your jurisdiction requires an annual bias audit for automated employment decision tools, our audit history satisfies that requirement.
We believe rigorous independent auditing should be standard practice for any AI system used in high-stakes decisions. We publish our results because we think that transparency builds trust — and because the data supports our confidence in AIR's fairness.
Book a demo to request the full audit report and discuss compliance requirements specific to your jurisdiction and industry.