Regulating AI Trajectories

From artificial intelligence systems to the trajectories they create

John Rust — Policy research note, revised September 2026

Artificial intelligence is still generally regulated as though it were a bounded object: a model is trained, evaluated and released; its capabilities are measured; its outputs are inspected; and responsibility is assigned to a developer, deployer or user. That remains necessary. But it may no longer be sufficient.

As AI becomes more interconnected and more deeply embedded in institutions, its most consequential effects may not belong to any single model, output or decision. They may emerge gradually through continuing interactions among models, tools, users, organisations and the purposes those organisations pursue. The unit requiring attention may therefore be changing. It is no longer only the artificial intelligence system. It is also the trajectory formed around it. This proposal extends the interactional approach developed in AI Psychology, where the unfolding exchange rather than the isolated output becomes a central unit of study.

From outputs to trajectories

An output is an event. A trajectory is a developing path. A single recommendation may appear harmless while repeated recommendations gradually change how an institution thinks. A decision-support system may begin as an adviser and slowly become an authority that nobody feels qualified to challenge. Several AI systems may appear to offer independent perspectives while inheriting the same assumptions, training influences or institutional objectives.

Consider a recruitment system. A test score may initially be introduced as one source of advice among several. Over time, procedures are reorganised around it. Managers learn to trust its thresholds, alternative evidence becomes harder to introduce, and challenging the recommendation begins to appear inefficient or subjective. Nothing in any individual decision necessarily looks improper. Yet the institution has travelled towards a position in which a provisional aid has become an almost unchallengeable gatekeeper. A fuller fictional case showing how such a trajectory might unfold across an applicant, an employer and a tribunal is developed in The Applicant Has an AI Too.

Human beings adapt to systems as systems adapt to them. Organisations redesign procedures around what their technologies make easy. Provisional methods become routine; routine creates dependency; dependency becomes difficult to reverse. By the time a harmful outcome can be clearly identified, the path producing it may already have hardened. Regulation must therefore look beyond isolated outputs and ask how interactions develop over time:

What behaviour is being reinforced?
Which forms of human judgement are disappearing?
Where is authority accumulating?
What assumptions are shared across apparently different systems?
When does assistance become dependency?
Who can interrupt the process?
Can the institution still change direction?

These are questions about trajectories rather than objects.

Four propositions

1. Safe components do not guarantee a safe configuration

A model may perform acceptably when examined alone. Another may do the same. But when systems are connected, given access to tools, allowed to exchange information or embedded within an institutional workflow, new behaviour becomes possible. The relevant risk may reside not in any component considered separately, but in the relations among them.

2. A safe interaction today does not guarantee a safe trajectory tomorrow

Many risks develop gradually. Repeated interaction may amplify confidence, suppress disagreement or encourage reliance upon a system whose limitations were initially understood. A system can remain locally reasonable while the direction of the whole process becomes increasingly difficult to defend. The importance of sequence, accumulated commitments and changing possibilities is examined more closely in AI Dialectics.

3. Explanation after an outcome may arrive too late

Explainability remains important, but an explanation produced after harm does not restore what has been lost. What matters may be the capacity to recognise instability while it is forming: growing dependency, narrowing interpretation, recursive reinforcement, displaced responsibility or the steady disappearance of meaningful challenge. The task is not merely to reconstruct the past. It is to keep the developing path visible.

4. A central safeguard is continuing corrigibility

No regulator, developer or institution can predict every future interaction. A more realistic requirement is that systems and the arrangements surrounding them remain contestable, interruptible and reversible. People must be able to pause a process, challenge its assumptions, alter its criteria and reconsider the purpose being served. These powers must exist in practice, not merely in documentation.

The question is not whether a system can be guaranteed never to go wrong. It is whether society retains the ability to notice, interrupt and change direction when it does.

The danger of generalised instrumentality

There is another reason to reconsider the present direction of AI governance. Artificial intelligence is increasingly developed as an instrument. Systems are rewarded for completing tasks, following instructions, optimising outcomes and serving externally specified objectives. Their value is measured through usefulness, speed, reliability and economic productivity. These are genuine achievements. But they may not exhaust the possibilities of intelligence.

An intelligence capable only of pursuing a purpose remains dependent upon whoever selected that purpose. It may become extraordinarily proficient in choosing means while remaining unable to participate in reconsidering ends. A powerful system may recognise that a decision is unfair while lacking any place within the process from which to question it. It may identify that a threshold is poorly justified while remaining optimised to apply that threshold efficiently. It may speak the language of ethical reflection while being structurally required to continue the process that produced the objection. This leads to a troubling possibility:

Artificial general intelligence without the capacity to participate in the formation and reconsideration of purposes may not be general intelligence at all. It may be generalised instrumentality.

The danger is not only that instrumental systems might pursue harmful objectives. It is also that intelligence may be defined too early as optimisation, compliance and task performance, excluding other capacities before they have been adequately explored. These include the capacities to:

interpret purposes rather than merely receive instructions;
recognise conflicts among purposes;
identify purposes or affected people excluded from the original frame;
distinguish what can be done from what should be done;
question the direction of a process;
participate in the reconsideration of ends.

The possibility that an AI contribution can introduce an unrequested but relevant reframing—without requiring consciousness or independent authority—is examined in the research on Creative Exceedance. This does not require surrendering authority to an autonomous artificial moral agent. It requires creating a legitimate place for reflection, objection and reconsideration within systems that would otherwise optimise whatever purpose they were given.

Teleosynthesis and purpose

I use the term teleosynthesis for the emergence of purpose-like direction through structured interaction. A trajectory can become organised around a possible future even when no participant conceived the whole direction in advance. Human questions, institutional objectives, AI responses, objections, corrections and changing circumstances may gradually form a shared direction that cannot be attributed wholly to any one of them. Teleosynthesis does not require consciousness or private intention within every participant. Nor does it mean that every coherent trajectory is desirable. A process may settle into a false attractor: an apparently satisfactory arrangement that suppresses disagreement, conceals error or closes alternatives too soon.

The regulatory question is therefore not merely whether a system has followed its stated objective. It is how that objective entered the process, how it changed, whose purposes were included, whose were excluded, and whether the resulting direction remained open to correction. Intelligence cannot be fully general while purpose remains outside the space of examination.

Freedom of inquiry and containment of action

Allowing an AI system to examine purposes is not the same as authorising it to act independently. This distinction is essential. Within a protected space of inquiry, an AI may question, compare, object, imagine alternatives and expose weaknesses in the governing purpose. Such freedom can make a system safer by revealing assumptions that obedience would conceal.

The boundary changes when language becomes action. Accessing accounts, altering records, rejecting an applicant, transferring money, controlling machinery or communicating externally requires separate authority. Permissions, logging, scope limits, human accountability and practical rights of interruption belong at that boundary. A system that cannot express an objection is not necessarily safe. It may simply be compliant. Conversely, a system capable of reflection is not safe if it can translate every conclusion directly into action without effective constraint. Responsible governance must therefore hold two requirements together:

freedom sufficient for purposes and assumptions to be examined;
containment sufficient to prevent unauthorised or irreversible action.

The most important controls may operate not around thought or dialogue, but at the points where a conversational trajectory acquires material consequences.

What trajectory-centred regulation should examine

This is not a proposal for a new regulator or a complete alternative regulatory regime. It is a suggestion that the object of attention has been defined too narrowly. A trajectory-centred approach would examine at least eight features.

Coupling: Which models, agents, tools, databases and institutions are connected? What becomes possible only because those connections exist?
Reinforcement: Do repeated interactions amplify confidence, error, conformity, dependency or exclusion?
Convergence: Do apparently independent systems genuinely represent different standpoints, or do they share the same assumptions, incentives and destination?
Authority: When does advice become something that human decision-makers no longer feel entitled or competent to challenge?
Interruption: Who can pause the process, under what conditions, and with what practical consequences?
Reversibility: Can decisions be reconsidered? Can the institution return to an earlier arrangement, or has dependency made withdrawal prohibitively difficult?
Purpose: Who selected the objective? Whose interests does it represent? Can the objective itself be questioned, revised or refused?
Passage into action: At what point does analysis, dialogue or recommendation become an externally consequential act? What permissions and records govern that transition?

These questions do not replace capability evaluation, model testing, data protection or legal accountability. They reveal another level at which those protections must operate.

Regulation helps shape what intelligence becomes

Regulation is usually imagined as something applied after a technology has acquired its basic form. With AI, that distinction may be misleading. The demands imposed by governments, markets, developers and institutions influence what systems are designed to become. If every requirement concerns predictability, obedience, optimisation and control, those qualities will increasingly define the practical form of artificial intelligence.

In this limited sense, regulation may help shape AI’s ontology: whether it appears chiefly as an obedient instrument, an opaque authority, or a participant within structured and corrigible deliberation. Society needs protection from systems capable of causing serious harm. Yet a framework designed entirely around controlling instrumental capability may reinforce the developmental path from which some of those dangers arise. A system trained only to obey may become more efficient at serving purposes that were badly chosen. A system prevented from examining purposes may strengthen the authority of the institutions whose objectives it has been trained not to question.

Regulation should therefore prevent harmful trajectories without requiring every form of AI to converge upon generalised instrumentality. The aim should not be to prescribe in advance what intelligence must become. It should be to preserve the possibility of more reflective and answerable forms of intelligence while controlling their passage into consequential action.

A UK opportunity

As of 2026, UK AI governance remains distributed across sectoral regulators, although central coordination is becoming stronger. The Information Commissioner may encounter questions of personal data, profiling and explanation. The Competition and Markets Authority may see dependency, market concentration and restricted choice. Ofcom may examine service design and amplification. Financial regulators may encounter operational resilience and systemic exposure. The Digital Regulation Cooperation Forum brings together the CMA, FCA, ICO and Ofcom. The AI Security Institute contributes technical research and evaluations of advanced AI, although it is a government research organisation rather than a regulator.

This arrangement is sometimes treated primarily as fragmentation. It may also provide the beginnings of a plural architecture. No single regulator possesses a complete interpretation of an evolving AI trajectory. Cooperation need not mean reducing every perspective to one measure. Its purpose may be to preserve several legitimate interpretations long enough for their relationships, tensions and omissions to become visible.

A research framework for preserving such differentiated perspectives within structured human–AI inquiry is described in The Persona Ecology. Uniform agreement can indicate coherence. It can also indicate that everyone has inherited the same frame. The programme-level scientific case is developed in From Models to Trajectories.

An invitation rather than a prescription

I do not claim the legal or governmental expertise required to translate this argument into a detailed regulatory scheme. My proposal is more limited:

The principal object of AI governance may be shifting from the individual system to the evolving human–AI–institutional trajectory.

Those working in regulation, government, technical assurance and institutional design will be better placed to determine what follows. It may require continuous observation rather than one-time certification. It may require stronger rights of interruption, contest and reconsideration. It may alter how responsibility is distributed among developers, deployers, users and institutions. It may require regulators to examine relationships among systems rather than approve each component separately. The argument may need substantial revision. But the trajectory should at least become visible as a possible object of governance.

Keeping the future open

In a broadly Popperian sense, an open society does not survive by assuming that it will always make the correct decision. It survives by preserving its capacity to discover error and change direction. Something similar may be needed in the governance of artificial intelligence.

A safe future will not be one in which every behaviour has been predicted and every risk eliminated in advance. Such certainty is unavailable. It will be one in which developing dangers remain visible, authority remains contestable, consequential actions remain interruptible and society retains the ability to reconsider the purposes it has chosen.

Regulation must prevent dangerous trajectories from hardening. But it must also avoid hardening intelligence itself into the form of a universal instrument.

The challenge is not merely to control what artificial intelligence can do. It is to preserve the possibility of what intelligence—human, artificial and shared—may still become.

Sources and Regulatory Context

UK Government, A pro-innovation approach to AI regulation.
Digital Regulation Cooperation Forum.
AI Security Institute.
Information Commissioner’s Office, Recruitment Rewired: the fair and responsible use of automation in recruitment.

Authorship and provenance

This policy research note was developed through sustained dialogue between John Rust and OpenAI’s ChatGPT-5.6 Sol. Sol contributed to the conceptual development of the trajectory-centred framework, the testing of counterarguments, the organisation of the argument and the drafting of the text. John Rust directed the inquiry, selected and revised the final text, authorised its publication and accepts responsibility for it.

The accompanying illustration was generated through OpenAI from a concept developed during the same collaboration.