Thoughts from the CEO – The Risks of AI Governance

When taking into account the immense power artificial intelligence technology can have, the issue of AI ethics and “what if” scenarios are frequently top-of-mind. As ideal as it would be to have absolute certainty that AI’s application is, and always will be, used for good that will benefit society –this is currently not a real-world guarantee.

So, the questions that often plague us are important and deserving of serious consideration. We must ask ourselves:
  • What happens if a government makes a wrong decision based on AI that affects millions of citizens?
  • Who –or what– is to blame?
  • How are we ensuring that our results are reliable?
When things go wrong, the finger is usually pointed at the algorithm or the machine learning model. And while it’s true that there can be a problem with the algorithm itself, we find that this is more than likely not the case –contrary to popular belief.

So, then, if the problems aren’t just with the algorithm, where do the risks lie? Let’s take a closer look…

Sources of Risk

  • We at Citibeats find that potential problems arise from 3 main areas:
Data Sources
If an algorithm produces biased results, chances are that the issue is in the data itself. We must question the source of the data and make sure that bias isn’t present from the get-go and spilling into the output data –through sources that include sample bias, stereotype bias or systematic value distortion. Faulty data, or an inaccurate weighing of mixed data sources, can produce a biased sample and may result in large-scale problems.

For instance, the demographic bias in the training data poses a major risk. If the data does not accurately represent the population for which it’s built to operate in, results may favor, or discriminate against, a certain gender or race.

A good example of a faulty data source is Microsoft and its experimental AI chatbot, Tay, that was released via Twitter in 2016. The results were not very pretty. The intention was for Tay to mimic the language patterns of a millennial female using Natural Language Understanding (NLU) and adaptive algorithms in an experiment to learn more about “conversational understanding” and AI design.

After just 16 hours, Tay was removed from the internet after her jovial exchange turned into a medley of insults –from sexism to racism. What went so horribly wrong so quickly? Initially-innocent Tay had been “corrupted” by twitter trolls – those malicious people who purposely stir up trouble online by starting arguments and inciting ill will. Knowing that AI is only as “smart” as the data it is fed, these trolls went about teaching Tay all the wrong things.

Despite the media outcry, the reality is that the AI worked. It listened, it learned, and it adapted; its responses were scarily human. The problem was that the data set was influenced by a devious source –internet trolls.

It all boils down to the need to create a solid and reliable data strategy. For us, that means working collaboratively with local experts in relevant industries and use cases that have the relevant knowledge and expertise. For example, teaming up with NTT Data for effective natural disaster relief efforts in Japan.
Algorithm Training
A machine learning model can be supervised, unsupervised or semi-supervised; the key difference between them is the application of training data –also known as labeled data. For supervised learning, labeled data is used to train the algorithm. In essence, the training data is structured to instruct the algorithm to learn the specific answer (output) from specific data (input).

Unsupervised learning trains the algorithm with data that isn’t classified or labeled –essentially allowing the algorithm to make deductions and to identify the underlying structure or pattern of a data set. In this case, there is no “right” answer –or output– to give it because it’s unknown.

The middle ground between the two is a semi-supervised machine learning system –which combines the two. At Citibeats, we develop custom algorithms for every context. We do this strategically as a safeguard against inherent bias within the algorithm.

In our use of semi-supervised machine learning systems, there is a huge amount of data provided, with most of it being unlabeled and some of it being labeled. The benefits of this type of system are that getting labeled data is time-consuming and labor-intensive, whereas unlabeled data is inexpensive and quick to collect. With many socially-impacting issues, speed is of the essence, so there is a clear advantage to having a partially unsupervised system. But, having a layer of supervision –a human being that is there to monitor the input before the output is executed– helps control for inaccuracy and bias.
Results interpretation
Even with the right data from reliable sources and high-performing algorithms, things can still spiral downwards in the results interpretation stage.

This can come about because:
  • the client isn’t trained or unaccustomed to navigating the data;
  • the data is not structured clearly enough to provide a concrete and actionable insight.
This is why we focus on teaming up with top-tier partners who help guide the process and demystify the client journey. We provide an API that processes structured raw data, alerts, real-time dashboard and reports to our partners –who in turn work with the client to create a model that will yield the most accurate and useful results for the issue at hand.

An example is FSD and Kenya, where Citibeats technology was implemented to process hundreds of thousands of customer complaints during 2018, generating regular alerts on the quality of issues being addressed –like fraud, scams and frozen accounts. Due to the effective and straight-forward style of reporting and analysis, FSD confirmed that 85% of those alerts were quality enough to warrant an investigation, decreasing their reaction time by 45 days.

Another example is our collaboration with the government of Navarra, Spain in hate speech detection. Our AI technology and detection system provided the context on the where who (via aggregated profiling) and what has been said against vulnerable communities. The information and insights extracted from this data were used directly to affect education budgets and policies that are helping to mitigate and counteract the effects of this hate speech.

Is it people or AI making the wrong decisions?

When we consider that algorithms, machine learning and AI have a direct and significant impact on governments’ decisions –thereby affecting millions of citizens– we realize that we need to apply plenty of focus and care to the areas where things can go wrong.
We have plenty of examples of governments making wrong decisions for a variety of different reasons. And though we may not be able to rely solely on AI and machine learning, we can address the pitfalls and be aware of the limitations. Only then, will we be able to maximize the good, minimize the bad and eradicate the ugly.
We at Citibeats plan on continuing to evolve, learn and improve in ways that better enable us to guide the decision-making process in the right direction. We will not restrict ourselves to think of the issue of ethics and risks only from a technology perspective –we must open our circle of inclusion to include other players like partners, clients, and data.