The monkey, the railroad, and responsible AI

In the 1880's, a South African railway company officially hired a baboon as an employee. Its duty was to set the rail switches that guided the trains to the right tracks, and during the monkey's nine years of service, it reportedly performed this duty without making a single mistake.

The fascination of this story lies in the descrepancy of the intelligence we attribute to the animal, and the risk and reputational damage any mistake would have had. Hiring a seemingly unqualified agent is without question a big liability, so it is reassuring to know that the railway officials only hired the baboon, whose name was Jack, after having thouroughly assessed its competence.

While hiring business monkeys never really took off as a concept, we do rely more and more on non-human assistants and tools to carry out our tasks: The rapid advances of artificial intelligence (AI) provides us with algorithms, bots and virtual assistants that take over increasingly complex and impactful tasks.

Entrusting non-human intelligence - be it artificial or simian - with tasks beyond its competence can have far-reaching and detrimental ramifications. As the first large legislative body to do so, the European Commision has recognized the risks that lie in unregulated AI and is preparing an act hedge and manage the sprawling influence of AI-backed systems in everyday life.

The EU AI act

The EU AI act is a proposed legislation to regulate use of AI inside the EU market. To develop this regulation, the Commission has adapted a technology-agnostic, risk-based approach that provides different levels or compliance standards based on the risk associated to the AI's role. These roles are classified into risk categories ranging from unacceptable risk (prohibited, such as social scoring) over high-risk (like credit-worthiness) to low risk (e.g. spam filters).

Depending on the risk, different cautionary measures, reporting and audits might be necessary. The exact requirements are still to be elaborated, and scope, timeline, and excemptions are still under discussion. Nonetheless, the general direction or the regulation is clear and provides a framework for businesses to prepare for the upcoming changes.

The reaction to these changes are mixed: Critics bemoan a stifling of European competitiveness and additional red tape, while proponents praise its ethical impact and the new opportunities this disruption offers. They also estimate that most systems will fall into the low-risk category, which will require minimal to no oversight.

What does that mean for your business?

Thanks to its powerful capabilities (and the marketing department's love for buzzwords), AI is very pervasive in today's IT landscape. Most software nowadays makes use of AI in one way or the other, and its effect on everyday business decisions is not always clear at first sights. Evaluating the risk these different systems pose in light of the new regulation requires a concerted effort of business process specialists, data scientists, and legal experts.

The legal interpretations of the act still lacks a clear delineation. As it was the case with GDPR when it was first introduced, nobody could foresee how the rules would be enforced until somebody would infringe on them. Nonetheless, from a business and data science standpoint, there are ways to prepare: awareness of the AI systems in use and assessment of their risk is a necessity. For continuous quality assurance, a standardized monitoring solution prevents deploying models whose performance drifts over time. And an estimation of the regulation's cost impact helps you to stay in budget.

At adesso, we understand the challenges and opportunities the EU AI act introduces and are happy to support your journey to responsible AI.

Read more:

• Trustworthy AI – a wild tension between human and machine interaction
• What legislative initiatives will change and the expected effects it will have on our AI and data projects