Frequently Asked Questions

Loss of control broadly refers to a situation in which AI systems are operating beyond the scope of reliable human direction or oversight. However, definitions in the broader AI literature differ in their emphasis and framing.→ Chapter 1, Section 1.A.1 — LoC Definitions in the Literature

Our report proposes three categories: Deviation (events that cause a small degree of harm or inconvenience but are relatively easy to contain), Bounded LoC (events that cause significant damage or suffering but are ultimately possible to contain), and Strict LoC (events that are maximally severe and permanent, capturing events that result in humanity as a whole becoming extinct). → Chapter 1, Section 1.B — Taxonomy of LoC

Our taxonomy is grounded in a systematic literature review of 130 works. We extracted concrete loss of control scenarios from the literature and plotted them on two axes according to their severity and their persistence. The taxonomy is based on the results of this mapping and research into existing definitions. → Chapter 1, Section 1.A.2 — LoC Scenarios in the Literature

The DAP framework is a practical framework for reducing loss of control risk today, by intervening on Deployment context, Affordances, and Permissions. Its implementation sidesteps uncertainties around AI capabilities and propensities, which may be difficult to measure and contextualize.→ Chapter 2, Section 2.B — Deployment, Affordances & Permissions

High-stakes deployment contexts are environments and use cases with a high degree of complexity and with limited time to respond and where there is a reasonable expectation that a malfunction of the AI system causes severe negative consequences. For example, critical national infrastructure or military applications could be considered high-stakes deployment contexts.→ Chapter 2, Section 2.B.1 — Deployment Context

Affordances are the environmental resources and opportunities available to an AI system for affecting the world (e.g. internet access, the ability to execute code, or control over physical systems). Limiting affordances to those that are strictly necessary for the task at hand is a direct lever for reducing LoC risk. → Chapter 2, Section 2.B.2 — Affordances

The more permissions an AI system is given, the less oversight humans have over it, since the AI system no longer needs to gain human approval before taking action. We recommend applying the principle of least privilege, i.e. restricting permissions to the minimum needed for a given task. Additionally, we recommend accounting for the possibility that advanced AI systems could manipulate users into granting additional permissions. → Chapter 2, Section 2.B.3 — Permissions

A state of vulnerability is a state in which AI systems have acquired or could independently acquire sufficient resources, affordances, permissions, and sufficient capabilities to cause LoC once a catalyst materializes.→ Chapter 3, Section 3.A — The Pathway to Vulnerability

First, because it seems likely that AI capabilities conducive to increasing the risk of loss of control will continue to improve (given current trends). Second, because it is likely that there will be increasing economic and strategic benefits from leveraging highly capable AI systems in more complex and high-stakes deployment contexts, where they are given broad affordances and permissions. → Chapter 3, Section 3.A — The Pathway to Vulnerability

A catalyst is required to trigger loss of control. The catalyst could be either misalignment, where the AI systems' goals and therefore its behaviours deviate from what humans intended, or pure malfunctions, where the AI system ceases to function as intended, absent misalignment. → Chapter 3, Section 3.B — The Catalyst Triggering LoC

Because it seems likely that we can never be certain that a state of vulnerability will not lead to a loss of control outcome, a sensible course of action may be to act as if we live in a world in which LoC could materialize, but has not yet — in other words, where loss of control is held in suspension.→ Chapter 3, Section 3.C — Living with a State of Vulnerability

We recommend a defense-in-depth approach that includes, at minimum, both governance and technical interventions. However, a detailed treatment of this is outside the scope of this report.→ Chapter 3, Section 3.D — Reflections