2.3 Algorithms and artificial intelligence: Disability discrimination risks in hiring and how to prevent them

As organizations receive more and more applications for a single vacancy, they have no choice but to use software to automate the process of examining and narrowing down the pool of candidates.

For example, employers might use technology for a variety of goals, including sending targeted job vacancy notices to candidates, using computer-based tests to evaluate an applicant’s skills or abilities, analyzing and scoring applicants’ résumés, assessing whether a candidate meets the required job qualifications and recognizing candidates’ facial expressions in video interviews.

Many of these hiring applications use artificial intelligence (AI) and machine learning algorithms to perform tasks in place of a human being. These algorithms are programmed by humans who may not be fully aware of the needs of candidates with disabilities, and they may be trained on data from millions of biased hiring decisions made regarding previous applicants, thereby increasing the risk of adding more human bias into the hiring process and making those same biases more efficient.

As a result, candidates with disabilities may be at risk of being excluded because AI-based hiring software discounts them early in the process.

The report on artificial intelligence and persons with disabilities issued by the United Nations Special Rapporteur on the rights of persons with disabilities in December 2021 (A/HRC/49/52) lists some of the potential negative impacts of using AI algorithms in recruitment tools:

 Time-based assessments may penalize candidates with disabilities who need more time to complete assessments or who use assistive technologies.
AI-powered chatbots used in online interviews may not be accessible to keyboard-only candidates or to those using screen reading software.
Companies that develop AI-powered video screening tools do not normally test them with persons with disabilities before launch, meaning that the atypical attributes of some candidates with disabilities may not be recognized and such candidates may be excluded.
Some AI algorithms that measure the levels of eye contact and vocal enthusiasm of interviewees may be unable to read the facial expressions of persons with disabilities, thereby putting them at risk of exclusion.

Employers must avoid using hiring technologies that discriminate against persons with disabilities. For example, some employers use AI algorithms to predict which candidate might be a productive employee by comparing applicant data to that of current successful employees. As persons with disabilities have historically been excluded from many jobs, they may not form part of the employer’s current staff, meaning that there are no equivalent data to compare them to, which results in discrimination.

Recommendations for avoiding the discriminatory use of AI:

Tell candidates in advance about the type of hiring technology that will be used and how they will be evaluated
Provide candidates with enough information for them to determine whether they need to request reasonable accommodations.
Provide candidates with clear instructions for requesting reasonable accommodations.
Make sure that asking for reasonable accommodations does not have negative consequences for the candidate’s chances of getting the job.

Recommendations for government and policy-makers:

Explicitly prohibit discrimination against persons with disabilities in national AI regulations.
Consider imposing a moratorium on facial and emotion recognition technologies.
Mandate the provision of reasonable accommodations in the operation of AI tools.
Include disability-inclusive requirements in the public procurement of AI tools.
Build capacity among organizations of persons with disabilities to enable them to advocate the use of responsible and disability-inclusive AI.
Monitor and report on how AI is being used to advance the provisions of the CRPD.

Recommendations for the United Nations system and specialized agencies:

Include disability in AI strategies, and devise action plans to assess and address the impact of AI on persons with disabilities.
Cover the use of disability-inclusive AI in UNDIS.