I recently joined and listened to OECD’s International Conference on AI in Work, Innovation, Productivity and Skills that took place on December 12 and 13. The conference brought in many voices in a multi-disciplinary manner and tackled the topic of AI from various angles. You can check out the conference’s agenda and the recording of each session online.
For this article, I am covering one discussion on ‘AI Incidents: A look at past mistakes to inform future AI governance’. If you have read my previous post then you know I am quite interested in governance options for AI development, so upon reading that this session would be part of this conference, I made sure to listen in and cover it on this blog for you. Their initial angle was that since 2022 OECD’s AI Incident Monitor (AIM) has seen a considerable increase in reports – which can be valuable data for policymakers to make decisions regarding governance.
The panel discussion featured four speakers:
- Elham Tabassi, Chief AI Advisor for the National Institute of Standards and Technology (NIST)
- Sean McGregor, Founding Director of the Digital Safety Research Institute
- Marko Grobelnik, AI Researcher, and co-leader at The Artificial Intelligence Lab at Jožef Stefan Institute (JSI)
- Jimena Viveros, Managing Director and CEO at IQuilibriumAI
This short discussion was moderated by Stephanie Ifayemi, a Senior Managing Director at Partnership on AI.
How can we address the risks and incidents of AI?
Ms Ifayemi opened the discussion by setting the context – how workings on frameworks are ongoing, and why incident reporting is important. She underlined that policymaking is too slow and reactive – but policymakers are understanding the risks of AI, even the most existential ones, and how the Hiroshima Code of Conduct, and the EU’s AI Act are addressing some concerns, but as all such documents they do contain loopholes – one example stems from simple things such as definitions: What can be considered as meaningful information (i.e. the ones to be reported)?
Ms Tabassi noted that AI is the most transformative technology there is, but she underlined quite explicitly that it does come with negative consequences and harms – ones we cannot necessarily quantify right now. This is an extremely important point to address – it is a concern that humanity knows way less about vulnerabilities than we should for responsible development. She argued for a general reporting framework that is concise, simple, flexible, and easy to understand for outsiders and utilizes a multi-stakeholder approach so people can understand what can go wrong. She raised the important question: Who is the system failing? Who is bearing the negative impact of AI? – questions we have attempted to look at on this blog ourselves as well.
Ms Viveros then voiced her opinion on the need for international governance to ensure global safety. Quoting UN officials – she mentioned that there is no sustainable development without global peace while pointing out that our existing AI data is limited to the civilian domains (countries might quite be reluctant to share their military findings after all). She argued for a need for incident reporting across jurisdictions for all systems that might be hazardous (which are concerns for stability and peace). In this approach, it is vital to have a multi-stakeholder approach, collection of data to mitigate future risk and incorporate these findings in AI training, while underlining that missing incident data is a risk itself.
What kind of incidents happen?
Mr Grobelnik explained OECD’s AIM and how they collect information. Their reporting is done by a quite wide scope of media reporting – distilling 150 thousand sources (amounting to a million articles a day!), which produce around 15-25 incidents each day. An issue is that media reporting might not pick up on smaller issues, nor on ones not reported in English. Their observation was that a big increase in incidents came with the appearance of ChatGPT – so-called soft AI incidents – where nobody got hurt (although these include deepfakes that might constitute actual harm, both personal and societal). He underlined that it is relatively inexpensive to create problematic content now, which is a concern for ethical AI.
An interesting point was raised that there is no increase in incidents with casualties yet, speculating this might be due to the lack of autonomy given to AI for the time being (such as laxer regulation for autonomous driving, or perhaps in healthcare). Mr Grobelnik pointed out, however, that their monitoring only extends to the civilian segment of our society and the number of deaths cannot be explained within the militarized framework of AI used.
As for the future of AIM, he addressed the need for expansion to cover more languages, not just English, as well as the use of LLMs to analyse the incidents. Further work is needed on experimentation of the possible impact of these reported incidents.
Mr McGregor talked about the usefulness of AI incidents in the form of it being data we can use to learn – and figure out what might happen, and what should be done should these systems fail. He also pointed out that governance makes it less likely for incidents to occur, and in its centre should these incidents be for the time being.
International effort for governance
At this point, Ms Ifayemi announced the results of an interactive poll of the audience – with 17 respondents reporting they have been negatively affected by AI, with 58 responding in the negative. I share the concern of one fellow audience member though – the lack of the I don’t know option in voting, arguing the possibility that the impact might be in the unconscious.
After this, the discussion was refocused on the international nature of governance.
Ms Viveros summed up the existing reporting system by saying it is fragmented and there are incomplete mechanisms of reporting right now (that it is both lagging and overlapping). She said the aim is a global framework with global governance, including obligatory incident reporting (in contrast to it being voluntary right now). Once again multi stakeholderism was invoked, so we could prevent harm in all domains. She pointed out that potential large-scale harm should be addressed with urgency.
Mr Grobelnik argued for the importance of international organizations – working as hubs where countries talk and build consensus in contrast with individual nations that might not have the same impact. A concern was raised that AI is developing faster than policymakers can react – he mentioned just a few shifts in geopolitics, the publishing of new AI models and functionalities just in the last few weeks, making it hard even for experts to keep track.
Ms Tabassi reminded everyone that AI does not know borders, and voiced a concern that a system trained on data from one part of the world might not be compatible everywhere as societal realities differ from country to country. She argued for representative data (so it can avoid bias). The main questions for organisations working with incident reporting should be ‘What went wrong?’ and ‘Whom did the system fail?’. She argued for the prioritisation of technologies that have low negative impact possibilities (and even those should be easily corrected).
Conclusions
A few audience questions were addressed – on how to ensure reporting is factual – right now by redundancies and manual checks. To prevent bad reporting Ms Viveros argued for an agency with a mandate for audits and reporting that can be reliable in the long run.
As Ms Ifayemi summed out the discussion, she argued that everyone should be thinking of incidents, and I can’t help but echo that thought. Right now, AI development is not as high stakes as it could be eventually, and we can learn quite much from incidents so developers can avoid fatal mistakes in future developments, to ensure proper AI alignment and ethical considerations. I was quite happy to hear that experts agree on the need for international cooperation to oversee development and the urgency of addressing the catastrophic outcomes of AI incidents.
What do you think – how can we learn from our AI mistakes that inform future development?
Feature image: geralt, Creative Commons Zero, via Pixabay