According to a December 2016 article in the Harvard Business Review, AI is being used by businesses to screen out up to 70 per cent of job applicants without any of the candidates having interacted with a human being. If you were overlooked by an algorithm for a potential job, wouldn’t you like to know how the algorithm made that decision?
Some of the AI used for recruitment now even evaluates candidates’ facial expressions and body language as part of a wider data set to determine their suitability for positions. However, the scientific community is not in agreement over the appropriateness of ascribing particular meanings to the expression of emotions. According to How Emotions are Made (The Secret Life of the Brain) by Professor Lisa Feldman, it appears that how people express emotions is not innate but learned, so our perception of others’ emotions can be very inaccurate.
Therefore, it seems that it is not that human expressions of emotion as they may be interpreted by AI have no meaning, but rather, those expressions may not lend themselves to a standard interpretation. This is just one example of why purchasers of such AI applications need to be cautious.
This problem highlights a general issue with the use of AI in the workplace: to reduce the risk of liability associated with its deployment, a business is likely to need to understand at a basic level how the algorithm is working and what its fallibilities are. If it has no such understanding, how can it take steps to mitigate the risks?
For example, an employer might use a third party developer’s AI software to reject unsuitable candidates automatically, without owning the software itself. If a candidate then claims the decision to exclude them is based on a protected characteristic (such as race, sex or disability), the employer may be vicariously liable for unlawful discrimination. It may attempt to use the statutory defence available under the Equality Act 2010 and claim that it has done everything reasonably possible in the circumstances to prevent the discrimination from occurring. Putting aside the inherent difficulties in successfully arguing the statutory defence, it could only ever be a real possibility if the employer had attempted to understand how the AI was making decisions.
It is unlikely that the software developer would allow the employer to delve into its coding in any meaningful way, as allowing access could damage its commercial interests. However, even if such access were to be authorised, many employers may face what is called the ‘black box’ issue. This term is used to describe the inability of many types of AI software to explain the logic used in reaching decisions. In some cases, there is a requirement for software to be developed to run alongside the AI application to interpret, in a way that humans understand, how decisions have been reached. Commentary in this area of work suggests that this solution is by no means straightforward.
If a developer assumed responsibility for the AI avoiding making unlawful, biased decisions, then it would open itself up to significant costs. It may be reluctant to warrant that the operation (and therefore decision-making) of AI complies with all laws. If the relevant AI uses machine learning, the developer may argue that the provider of the data sets used to teach the system ought to be responsible, not it. For example, at the client’s request, the developer may customise the AI using the client’s own data sets relating to what the business considers to be a successful recruitment outcome for it. If those data sets contain implicit biases, these may be reflected in the decision-making of the AI and the developer might argue that it ought not to be blamed for the outcome. If, for example, the AI decides the ideal candidate is a white, able-bodied, male employee in his thirties practising what might be perceived to be a mainstream religion, this is likely to reflect the business’s inherent bias.
Risk of reputational damage and unexpected outcomes
Businesses will need to address such issues as a commercial and legal imperative, particularly given the reputational damage that can follow. For example, in 2016, a well-known tech giant launched a chatbot through messaging platforms, which was intended to mimic the way a 19-year-old American girl might speak. The aim was reportedly to conduct research on conversational understanding. The chatbot was programmed to respond to messages in an entertaining way and to impersonate the audience she was created to target: American 18 to 20 year olds.
Hours after the chatbot’s launch, among other offensive things, she was providing support for Hitler’s views and agreeing that 9/11 was probably an inside job. She seemed to choose consistently the most inflammatory responses possible. By the evening of her launch, the chatbot was taken offline.
As this shows, AI can result in unexpected and unwanted outcomes. The chatbot’s responses were modelled on those she got from humans, so her evolution simply reflected the data sets to which she was exposed. What AI learns from can determine whether its outputs are perceived as intelligent or unhelpful.
Some commentators have suggested that one way to provide safeguards against inappropriate outcomes in the use of AI is to ‘stress test’ the software by pushing it beyond its normal operational capacity. It may also be appropriate to introduce a human element earlier in the decision-making chain to consider the AI’s proposed outcomes, and to measure and correct ‘automation bias’. Automation bias is the name given for the problem that humans tend to give undue weight to the conclusions presented by automated decision-makers and to ignore other evidence that suggests they should make a different decision.
For example, an automated diagnostician may help a GP reach a diagnosis, but it may also have the effect of reducing the GP’s own independent competence as they may be unduly influenced by the AI. The result may be that the augmented system may not be as much of an improvement overall as had been expected.