When bias gets baked in
There have been multiple events in which AI has created errors. "In 2015, 21 American courts adopted a risk assessment algorithm to help judges decide whether defendants should be jailed before their trials," explains Fayçal Boujemaa, Technology Strategist, Orange Labs Research. The following year an investigation found the system's use of historical data meant, "Black defendants were mistakenly being flagged at almost twice the rate as their white counterparts."
So, what contributes to such errors, and what approach should enterprises take to minimize them?
Françoise Soulié-Fogelman, Scientific Advisor at Hub France IA, identifies three primary ethical risks to AI: algorithmic bias, the need for transparency and fairness, and questions around liability.
Take algorithmic bias. Soulié-Fogelman argues its existence suggests that many more minor forms of bias in AI also exist. She believes algorithm designs may reflect the conscious and unconscious bias of data scientists who build them. Given AI cannot correct its own inherent programming, we need to measure the impact of those algorithms when in use – think about the court records mentioned above to illustrate this.
Another challenge in ethical AI is the need to expose the inherent decision tree. Machine intelligence needs to demonstrate how a decision is reached to facilitate necessary adjustments to the algorithm, to provide legal protection for regulated industries and an evidence trail for flawed decisions. Lack of transparency and algorithmic bias reinforces the importance of creating chains of responsibility that work in the real world and maintain scrutiny and accountability, particularly when things go wrong.
One way to cope may be to support data scientists with ethical advisors and specialists tasked with analyzing bias and discrimination. Mick Lévy, Business & Decision Business Innovation Director, proposes a team-based approach to data governance, with organizations trained to approach data as an asset. Another approach is to ensure humans remain central to decision making. Michael Sandel, Professor of Government at Harvard University, asks: "Are certain elements of human judgment indispensable in deciding some of the most important things in life?"
You need quality data
"Garbage in, garbage out," is a data scientist maxim. When it comes to AI, it means data that is fake, poorly measured, reflects societal or personal bias, or otherwise flawed, delivers poor results. Data must be consistent, precise and accurate. It must comply with privacy and regulatory standards and come from trusted sources. It must also be contextually relevant. AI built on 50-year-old data may be deeply relevant to historical events but irrelevant today. Nine out of ten companies already fear their databases contain too many errors. "It is easy to imagine, in the near future, sets of data from any information system being used to feed or train AI," said Didier Gaultier, Data Science and AI Director at Business & Decision.
"Numerous problems can impede proper AI training, such as false data, inconsistent data and missing (and/or non-representative) data. The list is far from exhaustive, and the danger is that this may not only skew results but also reproduce, maybe even intensify, human biases through algorithmic biases."
The scale of the ethical dilemma is prompting businesses and governments to seek to resolve the challenge. The European Commission is developing principles that support ethical AI. Orange and others are developing principles of governance for its use. The European High-Level Expert Group says AI should be:
- Lawful
- Technically and socially robust
- Ethical, which means fair and transparently explainable
Ensuring these standards requires product managers, data scientists and ethical advisors to work closely together on AI. Ethical considerations must be maintained across the development cycle, from defining context and objectives to data gathering, development and deployment. The Commission argues that any risk of bias should be identified and declared before any AI goes live.
Ethical requirements can be ranked in terms of criticality. Fairness, for example, requires that people and groups of people are not treated differently because of sensitive characteristics. The European Commission has published an Assessment List for Trustworthy AI. The idea is that data scientists and ethical advisors should integrate these requirements into the code, mitigating bias and ensuring decision transparency.
Recruitment tools are used by 99% of Fortune 500 firms, but a recent Harvard study revealed that some AI recruitment tools filter out potential hires with relevant experience because of limitations in the AI. HR teams now believe that many such systems have not been built to reflect the nuanced realities of the employment world. One way to mitigate this might be to build decision trees that manage bias by providing transparent explanations for how decisions are reached.
Consumers are growing suspicious about untrammeled data collection. In France, 57% of Internet users feel more tracked than before, 80% consider automated recommendation systems as intrusive, and 74% don't trust how their apps use their data.
Companies that reassure customers that their data is being used responsibly by AI could have an advantage. "This is how businesses will be able to differentiate themselves from less ethical ones: through complete transparency when it comes to data use and communication with the general public," explained Indexical’s Co-founder Emmanuel Dubois in a recent Business & Decision report. Ethical AI is not only the correct approach but may also prove profitable for those who create systems that support it.
Orange recently created a Data and AI Ethics Council, comprising 11 independent experts, which will develop ethics guidelines for the responsible use of data and AI at Orange and monitor its implementation within the Group's entities. Orange is also a founder of an international charter for inclusive AI, through the Arborus Endowment Fund. For more information, read about the importance of auditing algorithms, and how to implement ethical AI.
Jon Evans is a highly experienced technology journalist and editor. He has been writing for a living since 1994. These days you might read his daily regular Computerworld AppleHolic and opinion columns. Jon is also technology editor for men's interest magazine, Calibre Quarterly, and news editor for MacFormat magazine, which is the biggest UK Mac title. He's really interested in the impact of technology on the creative spark at the heart of the human experience. In 2010 he won an American Society of Business Publication Editors (Azbee) Award for his work at Computerworld.