AI Hiring Tools Face Scrutiny Over Discrimination Concerns

Allegations of AI-assisted hiring discrimination have sparked concerns among human resources and legal experts, highlighting the potential for AI tools to perpetuate and even exacerbate long-standing patterns of exclusion. Law professor Pauline Kim warned that if AI systems are not designed with an awareness of bias risks, they could worsen existing discriminatory practices. This issue has come to the forefront with recent claims against workplace management software firm Workday, which is accused of using AI to discriminate against job applicants based on race, age, and disabilities.

Workday has denied these allegations, asserting that its AI tools do not make hiring decisions and that customers maintain full control over their hiring processes. The company has also received third-party accreditations for its commitment to developing AI responsibly. However, the concerns raised by the lawsuit and the broader discussion around AI hiring tools underscore the need for vigilance in ensuring that these technologies do not perpetuate biases.

Research from the University of Washington Information School provides stark evidence of AI’s potential for discrimination. A study found that AI-assisted resume screenings favored white-associated names in 85.1% of cases and female-associated names in only 11.1% of cases. In some settings, Black male participants were disadvantaged compared to their white male counterparts in up to 100% of cases. This research highlights the risks of training biased models on biased data, creating a feedback loop that can exacerbate discrimination.

The issue of AI hiring discrimination is not limited to Workday. Amazon has also faced allegations of discriminatory practices, with a letter sent to its executives claiming that the company’s AI processes for employee accommodations do not comply with the Americans with Disabilities Act. Amazon has denied these allegations, stating that its AI does not make final decisions around employee accommodations and that it follows robust guidelines to ensure fair AI use.

The root of AI’s biases often lies in human biases, according to Washington University law professor Pauline Kim. A 2023 meta-analysis by Northwestern University found persistent and pervasive biases in hiring practices, with employers calling back white applicants 36% more than Black applicants and 24% more than Latino applicants with identical resumes. This underscores the need for careful implementation and oversight of AI hiring tools to prevent the perpetuation of these biases.

Experts have called for increased regulation and transparency in the use of AI hiring tools. While existing laws prohibit both intentional and disparate impact discrimination, there are few formal regulations specifically addressing AI employment discrimination. Local efforts, such as a New York City ordinance banning the use of automated employment decision tools without a bias audit, have attempted to address this gap. Employment lawyer Melanie Ronen noted that other state and local laws have focused on increasing transparency and providing opt-out options for prospective employees.

Firms behind AI hiring and workplace assessments, such as PDRI and Bold, have taken steps to mitigate bias in their technologies. PDRI CEO Elaine Pulakos advocated for human raters to evaluate AI tools before implementation, while Bold’s Victor Schwartz emphasized the importance of guardrails, audits, and transparency. Schwartz also noted the potential for AI to diversify a company’s workforce if applied appropriately, citing research indicating that women are more likely than men to apply for jobs only when they meet all qualifications. By streamlining the application process, AI could remove barriers for those less likely to apply to certain positions, leveling the playing field.