How AI Can End Bias In Job Recruitment

Want a job? Chances are you going to have to win over both man and machine.

The ease of firing off a job application online has led to many employers having to sift through thousands of candidates. Early in the year, VICE Media posted a job for a new office manager. Within hours, they had received over two thousand applications. If a team of five, spent five minutes looking at each application, it would take them a working week to make any kind of decision, and that's before any interviews take place.

Enter AI. The most obvious advantage of which is the screening of CVs in ridiculously high-volume. Simply put, AI can crunch multiple data sources and come up with insights that the human brain cannot. Rather than simply scanning a CV, an AI platform could evaluate a candidate’s entire online presence, accessing fit more effectively in a fraction of the time. For these two reasons, every major employer is using the technology in one form or another.

AI in recruitment, however, has received a lot of flak in 2018, most infamously with Amazon’s flawed system hitting the headlines. Amazon, a company of seemingly limitless resources, could not stop its tool from discriminating against women. They fed the system a decade’s worth of CVs from Amazon applicants, and the tool taught itself to downgrade applications containing the word “women’s” and assign lower scores to graduates of women-only universities.

These revelations caused many companies, both large and small, to scrutinise their technology more actively. It’s true, AI comes with its pitfalls - which technology doesn’t? If it is implemented incorrectly, it can perpetuate bias and reinforce years of gender and racial inequality. But if various lessons are followed, it could have a completely revolutionary effect on one of the most important, yet antiquated sectors in business.

More Data, Less Problems

One reason Amazon fell afoul was because of their dataset, which simply wasn’t large or diverse enough. As mentioned above, models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.

In fact, a staggering 89% of Amazon’s engineering workforce is male. Clearly, using historic data will continue historic failings – no big surprise there. To develop a more diverse workforce, companies need to have a more diverse dataset – ideally, from a range of sources, entered into a completely independent platform to enforce auditing. Independence in this context is absolutely crucial – data which is owned solely by one party, in this case Amazon, is open to bias, error and sometimes manipulation.

Companies need to embrace independent sources for their data and be more open with their own – only through collaboration can they have a reliable dataset on which to assess and improve diversity, without jeopardising on quality.

Data Auditing

While the use of AI in recruitment is rapidly becoming the norm, the auditing of these back-end algorithms is lagging behind. A successful algorithm must incorporate two factors.

Firstly, the algorithm must be built in such a way that allows it to be analysed and its outcomes interpreted. Overly complex "black box" systems are self-defeating and make any analysis extremely difficult.

Secondly, companies must be willing to have their algorithms reviewed externally from an independent party. If companies can prove their algorithm is impartial, then it should become a competitive advantage when it comes to hiring. Much like being "GDPR ready" gave consumers peace of mind, auditing would provide a level of trust devoid in traditional recruitment.

A Force For Diversity

Contrary to most reports, AI can be a force for addressing gender and racial bias. If implemented correctly (following the steps laid out above and below), the technology is far fairer than humans. We all have an unconscious bias, which cannot be controlled and, in most cases, is extremely difficult to interpret. It’s human nature to like what we know and place value in strengths that mirror our own. Shockingly in the modern age, referrals remain the number one source of hires – proving that a form of nepotism remains rife.

Al, on the other hand, can be audited, giving HR departments tangible insight into bias and why hiring decisions are made. While Amazon’s tool has been widely condemned, the faults in the system were discernible in a way that is impossible to interpret with humans. Amazons found its tool was downgrading CVs with certain words – how many of us could offer that level of analysis after reading a CV or conducting an interview?

Having this granular level of accountability can, in turn, help combat lawsuits and inform internal policies. If you can prove that no bias was present, or point to how potential flaws have been mitigated, it becomes nigh on impossible to sue. Policies too can be governed by such insight – why are so few applications coming from women? Why is aggressive and singular wording preferred? And crucially, how can these trends be reversed?

Using AI, managers could also set criteria for applicants. This could include “cognitive diversity” (people with different backgrounds and therefore styles of problem-solving) when looking for candidates. AI could then make suggestions based on the employer’s priorities.

AI As Standard

Even with the negative reports, many companies are pushing the boundaries of AI in recruitment. These companies recognise the traditional recruitment model is broken - continuing to pursue the same avenues for candidates, simply cannot deliver the best results. Recruitment is playing catch-up with other industries, but AI and algorithmic technology is currently the most likely breakthrough.

While AI systems may not be ready to make autonomous hiring decisions, it is making rapid strides in making the process both faster and more efficient. Algorithmic rankings of candidates based on fit is fairer and quicker than human based CV reviews. As AI technology develops, it must have fairness placed at its heart. If at its core it focuses on diversity and equality, can it become an undeniable force for good in the workplace?

Ben Chatfield is CEO and Co-founder at Tempo

This article reflects the opinions of the author and is neither intended to be financial counsel nor a recommendation.