Guest author: Scott Robbins, PhD at TU Delft on AI and counter terrorism

Recent AI innovations have demonstrated algorithms successful at detecting cancerous moles, playing complicated games like chess, and speech recognition. If implemented responsibly, these algorithms will no doubt contribute to society in a positive way . The problem we face is one where we don’t have policy or guidelines which will get the best out of AI while avoiding negative ethical and societal impacts. Unfortunately we often lack the knowledge necessary to construct such policy. Before we can enact good policy surrounding AI we must have some knowledge of what the products made with these algorithms actually are.

Much of the popular (and academic) discussion surrounding AI is on the transparency of its decision making. Can we get these algorithms to explain themselves? Google has famously tried to take steps towards such an explanation by reversing the direction of their image recognition AI. The results were not exactly explanations, but these dream-like pictures were extremely interesting. Importantly, Europe has enacted legislation which gives people a right to explanation when an algorithm ‘significantly affects him or her’. Europe now has multiple heavily funded projects attempting to make AI explain itself. I am of course in favor of having explanations to important decisions like receiving a loan or prison sentencing. However, the solution may be as simple as not using AI for tasks which would require such an explanation – not to ‘open the black box’ of AI as is so frequently called for. This is because the power of AI is a direct result of its being able to function in a way that we do not understand. Constraining AI to human articulable explanations would erode the very power promised – the power that has resulted in machines classifying moles as malignant better than physicians, detecting signs of a heart-attack by listening into emergency calls, and detecting when faces don’t match passport photos.

The examples above all significantly affect us; however, it would be odd to require these algorithms to have human articulable explanations for their outputs. If we accept that the explanations for modern AI algorithms will remain opaque to us, then the question is how to use these algorithms in a responsible manner. Starting from here, we can see that there is a lot more that we don’t know about AI than the reasons for its decisions. With commercial AI (and machines powered by AI) we are often in the dark regarding its: training data, function, inputs, outputs, and boundaries. While it is unnecessary to require that we know all of this about any particular algorithm or machine – it is important that we at least know that we don’t know. And by ‘we’ I mean not only the company making the machines or algorithms, but the people who are legislating, using, and are subject to them.

Training Data

The data used to train machine learning algorithms is extremely important with regard to how that algorithm or machine will work. Two algorithms that share the exact same code could work wildly differently because they were trained using different datasets. A facial recognition system trained only using pictures of faces of old white men will not work very well for young black women. If someone is to buy a facial recognition algorithm then there should be some information about the faces used to train it. The number of faces and the breakdown of age, ethnicity, sex, etc. would be a basic start. The specifics regarding what information is needed about the training data will obviously vary depending on context and type of data.

The knowledge regarding training data will be important when implementing algorithms. Simply knowing that the training data lacks a certain demographic would hopefully cause one to test the system before using it on such a demographic or to restrict its use to demographics covered by the training data. For example, algorithms made to detect skin cancer were trained on images of moles mostly from fair-skinned patients – meaning the algorithm does poorly with regard to darker skinned patients. Whatever the reasons for this biased training data – it is important to know this before using such an algorithm is used on a dark-skinned patient.

Boundaries & Inputs

Machines – especially so-called ‘smart’ machines – increasingly have few boundaries. Digital assistants have access to so much of our data from so many sources that it may be more helpful to detail where they do not get their data from. Sometimes boundaries are set by the physical space a machine can navigate. For example, a Roomba vacuum will have the boundaries of one floor of a home or apartment. A user is given a limited space with which to make sure that the robot will function properly. More frequently these days is the virtual space that the algorithm can reach for its inputs. It may be scanning our email for appointments or grabbing data from a weather website.

Knowing what boundaries (if indeed there are some) algorithms and robots are constrained by is important for it is these boundaries which help us to understand the possible inputs the machine could receive. Sometimes, the machine is constrained by what we give it – meaning the machine has no autonomy to receive inputs. The malignant mole detecting algorithm, for example, receives an image of a mole that we manually give it. The contrast would be cameras in your bathroom constantly scanning for moles. While the algorithm is constrained by what it can ‘see’ via its camera, the expected inputs could be quite complicated.

An ‘input’ as I want to talk about it here is the combined data from all sensors. We, as humans, make decisions based on a number of factors. For example, we might put on a rain jacket because: it is raining, it is not too cold outside (otherwise we would opt for a heavy jacket), and we are going to be outside. A machine might be able to tell a user to wear a rain jacket based on the same data because it has a temperature sensor to sense how cold it is outside, a data feed from a weather website (to ‘sense’ that it is raining), and a microphone to hear the user say they need to go outside. It is the combination of this data which determines what output will be given.

With the case of autonomous cars, we see a situation where there are very few boundaries, and little knowledge about the possible inputs. Because cars can go anywhere roads take us, the various sensors on autonomous cars mixed with the extreme variability in states of affairs surrounding these roads, makes for limitless possible inputs. Some of these inputs have led to death – both for passengers and for bystanders. This points to the importance of knowing the possible outputs of these machines – and what exactly they are for (their function). A machine made to play a video game is quite benign. In that instance, what do we care about the possible inputs? No matter what, the outcome results in game play. However, when the outputs could result in death, or the function of the machine is to have moral import – then we better know a lot about these boundaries and inputs.

Function & Outputs

Knowledge of the functions and possible outputs of a machine is essential for being able to responsibly regulate, use, and be a bystander to AI powered machines. This is meant to ensure that the possible outputs all are in response to a task given to the AI system. In the AlphaGo example, the output is a move in the game of GO. We might be shocked by it making a particular move, but it is nonetheless a legal move in the game of GO. It would be strange if the task of AlphaGO were defined as “not letting an opposing player win” and instead of making a move its output was to mess up the board (because it knew there was no chance of winning and this was the only way to ensure that the other player did not win).

It can be easy to think that functions and outputs are equivalent. To illustrate the difference we can say that the function of a driverless car is to drive from point A to point B; however, this will involve many outputs. Each turn, acceleration, swerve, and brake is an output. These outputs can have, as previously mentioned, fatal outcomes.

Defined functions are of the utmost importance because it allows us to test the machines for efficacy. How well a machine functions is clearly salient with regard to its moral acceptability. If the malignant mole detecting algorithm was seldom successful at categorizing moles then it would be unethical to use it. Equally unethical is the use of the algorithm when we do not know how successful it is (i.e. use outside of a testing environment). Digital assistants (like Amazon’s Echo and Google’s Home) are prime examples of AI powered machines for which there is no function and the outputs are seemingly limitless. In one scary example a couple’s conversation was recorded by their Amazon Echo and that recording was sent off to a colleague. Not only was it unclear how this happened but the couple did not even know this was a possible output. How can people responsibly put such machines in their homes when they don’t know this basic information?

Outputs must also be evaluated alongside the boundaries. A drone, for example, may have a machine gun built in, giving it the capability to shoot bullets. This means a possible output is the shooting of bullets. At first glance many would consider this an inappropriate and perhaps unethical capability for a drone. But consider the case when the drone is operating in a bullet proof room with no humans present. A machine whose possible output is to shoot bullets may be acceptable if its only input is a user telling it to shoot and its boundaries are a bulletproof room. This example makes it clear why it is so important to have knowledge regarding the functions, outputs, boundaries, and inputs. We need all of this knowledge in order to make informed decisions regarding the acceptability and use of AI powered machines.

Move Slow and Fix Things

Forcing designers and engineers of AI powered machines to be transparent about these properties may sound too constrictive – and may ‘stifle innovation’. The word ‘stifle’ is thrown around whenever an idea or legislation would slow things down. While I don’t even think this would be a bad thing, I argue that articulating what we know about these properties actually helps us to focus in on the problems that AI is good at fixing. Digital assistants solve few problems but cause many more – because we don’t know basic things about them. Driverless cars have limitless inputs and potentially fatal outputs all in the service of doing something we can already do – and with public transportation we can do it safely, efficiently, and sustainably.

The further we constrain these algorithms to clearly defined functions with clear boundaries allowing for limited inputs and testable outputs, the better these algorithms are going to work. This is one reason that so many of the news stories around breakthroughs in AI are about their ability to play games – whether it is Go, Breakout, or Mario. Finding real world problems that resemble the clear win/loss scenarios in games is where we should be focusing the power of AI. The messy real-world of driving and talking to human beings is simply a waste of resources – a waste with demonstrably dangerous outcomes.