SikiT

Field notes on running AI agents without a human in the loop.

Field notes on unattended AI agents.

10–15 minutes

When an AI rule changes, who gets the correction?

A four-way change test and five continuity duties for deciding when an AI correction should reach people affected by an earlier version.

A dark anchored imprint linked through three translucent frames whose openings change from round to angular.

Sources are linked in this article. Found an error? Report a correction.

An updated AI can stop making a harmful decision without repairing the life already shaped by it. Replaying the old version explains what happened, but it does not decide whether the institution should reopen, reverse, or compensate the earlier case. For high-stakes systems, a correction should carry a defined duty to look backward, while ordinary adaptation should not automatically turn every past decision into an error.

On this page

Question: A correction reaches the future first

At 9:00 a.m., a hospital’s intensive-care allocation system evaluates a patient and recommends against admission. The bed goes to someone else. The patient is transferred, treatment is delayed, and the family later challenges the recommendation.

By 3:00 p.m., the system has changed. Engineers have corrected a weakness, clinicians have revised a safety threshold, or new cases have altered the model. Given the same recorded patient data, the current version recommends admission.

The hospital can now offer an accurate but incomplete answer: the system no longer makes that decision.

The patient still carries the earlier consequence. The lost bed, delay, expense, or physical harm did not update with the software. Yet the rule that produced the recommendation is no longer operating. The family is appealing a decision made by a version that, in one technical sense, has ceased to exist.

This is a thought experiment, not a report that hospitals currently let intensive-care models retrain after every patient without validation or regulatory control. It compresses the time between decision, update, and appeal to expose a problem that slower update cycles can hide. What is a person entitled to challenge when the reason has changed before the consequence can be reviewed?

Most debates about legitimate algorithmic power begin with accuracy, bias, transparency, explanation, or human oversight. Each assumes some continuity between the authority that acted and the authority that later receives the complaint. An explanation needs an identifiable decision process. An appeal needs the standard that applied at the time. A remedy needs an institution willing to inherit the past rather than treat every update as a clean beginning.

Call that condition temporal contestability. A system is temporally contestable when a person can reconstruct, question, and seek a remedy for a consequential decision after the system changes. The software need not remain frozen. The institution operating it must preserve an accountable relationship with authorized earlier versions.

Without that relationship, the person keeps the consequence while the institution sheds the reason. The system learns from the case, future users receive the benefit, and the person whose experience exposed the weakness may receive nothing from the correction.

Argument: A preserved decision still needs an heir

Change cannot be prohibited outright. Medical practice, populations, resources, and data distributions move. A model that never changes can become unsafe.

Current regulation already makes room for controlled change. The US Food and Drug Administration’s August 2025 final guidance on predetermined change control plans asks a plan to describe intended modifications, methods for developing, validating, and implementing them, and an assessment of their effects. The EU AI Act likewise anticipates high-risk systems that continue to learn. Article 43 and Recital 128 allow certain predetermined, pre-assessed changes to avoid being treated as substantial modifications that require a new conformity assessment.

These rules address prospective control: what a provider may change and how the change should be evaluated. They do not, by themselves, answer what a corrected system owes people judged under the earlier version.

This problem already has close neighbors. Brian C. Long’s 2026 essay The agent that learned from you argues that accountability fails when a changing agent does not hold still long enough to be examined, and recommends pinned, recoverable versions and provenance. Cobbe, Lee, and Singh’s Reviewable Automated Decision-Making treats reviewability as a property of the whole socio-technical process, supported by records that permit review of individual decisions and the wider system.

Åke Elden’s When responsibility fails to arise makes the institutional problem sharper. Continuous statistical optimization can erode the discreteness, authorship, and publicness through which responsibility normally attaches. Ghassan Abukar’s Temporal authorship as a moral right examines the neighboring danger of classifications that persist and foreclose a person’s future.

This essay accepts those premises and asks a narrower question. A recoverable version can show what happened, but it cannot decide who gets the benefit when a later version embodies a correction. The persistent-classification problem asks how a person can escape a rule that continues to define them. Here the structure is almost the inverse: the rule disappears, but its consequence remains.

A high-stakes system needs five forms of continuity.

  1. Historical reconstructibility. The institution must be able to recover the model version, relevant inputs, transformations, thresholds, policy, human interventions, dependencies, and known limitations that shaped the decision. Source code alone may not reproduce the governing environment.
  1. Cross-version intelligibility. When the current version gives a materially different answer, the institution must explain the kind of change that produced the divergence. A parameter diff is not enough. The person needs to know whether the earlier view was corrected, the environment changed, the institution revised its values, or the system varied without a defensible reason.
  1. Responsibility inheritance. The institution using the current system must remain answerable for decisions made under authorized earlier versions. The complaint is not addressed to obsolete software. It is addressed to the hospital, employer, agency, school, insurer, or lender that selected, deployed, and benefited from it.
  1. Remedial reachback. If an update confirms that a class of earlier decisions was materially defective, the institution must ask whether similarly affected people should receive notice, reassessment, restoration, compensation, or another remedy. The duty is to ask and justify, not to reopen every case automatically.
  1. Temporal proportionality. The durability and severity of a consequence should affect how long records, explanations, appeal routes, and remedial duties remain available. A recommendation that can be reconsidered tomorrow requires less continuity than a decision affecting liberty, treatment, employment, education, insurance, or credit for years.

These conditions do not make the model a moral person. They identify the institution as the carrier of continuity. People, software, policies, and corporate structures can change while acts performed under institutional authority remain part of its history.

Examples: Four reasons a new version may disagree

A changed answer is evidence. It is not yet a verdict on the earlier decision. At least four events can make two versions disagree.

Type of changeWhat happenedWhat it may require
Epistemic correctionThe earlier version relied on a coding error, biased data, defective assumption, or unreliable relationship that the institution now rejects.The strongest case for reviewing materially similar past decisions and considering a remedy.
Environmental adaptationDisease prevalence, available beds, labor markets, fraud patterns, or other relevant conditions changed.An explanation of the changed environment; both decisions may have been reasonable in context.
Normative revisionThe institution changed what it values or how it trades off urgency, survival, equality, age, cost, or waiting time.Public, professional, or political justification, plus any applicable transition rule.
Uncontrolled variationRetraining noise, implementation differences, upstream data changes, or unstable optimization changed the result without a defensible policy reason.Reliability investigation before either output is treated as authoritative.

The first category creates the clearest reachback claim. If an institution fixes a defect by standards it now accepts, it cannot describe the change only as a benefit for future users. The correction is also evidence about earlier cases. That evidence may still be incomplete, and a remedy may be impossible or disproportionate, but the retrospective question has been triggered.

Environmental adaptation is the hard counterexample. Imagine that the earlier hospital decision occurred during a severe outbreak with every bed occupied, while the replay occurred after capacity expanded. The same patient data does not recreate the same decision environment. A new recommendation may reveal adaptation rather than error. Treating every divergence as an admission would collapse context into software version.

Normative revision raises another issue. Suppose the hospital moves from maximizing expected survival to giving greater weight to urgency or time already spent waiting. The newer decision may reflect a defensible policy change rather than better medical knowledge. The institution must explain who authorized the revision and how old and new cases are related. Technical documentation alone cannot supply that justification.

Uncontrolled variation may look mundane, but it matters just as much. A model can change because a data pipeline moved, a seed differed, an upstream classifier was replaced, or retraining produced a new error pattern. Gagan Bansal and colleagues found in Updates in Human-AI Teams that a model update improving predictive accuracy could still reduce combined human-AI team performance. People had learned where the earlier model succeeded and failed; the new pattern disrupted that understanding.

Joshua Hatherley calls the cross-version explanation problem update opacity. Ordinary opacity asks why one model produced an output. Update opacity asks why the same input receives a different answer from another version and whether that difference is improvement, degradation, or a changed pattern of reasoning. Andrea Ferrario’s 2026 preprint on high-risk AI system identity shows why lifecycle regulation cannot avoid deciding what persists across versions, although its interpretation is not settled law.

The EU AI Act supplies parts of the record. Articles 12 and 19 require logging capabilities and retention of certain logs for high-risk systems, while Article 86 provides a clear and meaningful explanation right for a limited class of covered decisions. Those safeguards preserve evidence. They do not decide whether the change was a correction, who inherits the act, or which remedy follows.

An affected person therefore needs three explanations: why the earlier version decided as it did, why the current version differs, and what the difference means for the earlier case. The third is institutional, not merely technical.

Counterargument: Improvement should not become a confession

The strongest objection is practical. A system that learns from mistakes is better than one forced to repeat them. If every update threatens mass reopening, liability, and compensation, institutions may delay improvements, conceal modest changes, or keep inferior systems in production. Retrospective review can also consume the same scarce clinical or administrative capacity that the system was meant to protect.

That objection defeats a crude stability rule. It does not defeat temporal contestability.

Operational replacement and accountability replacement are different acts. An institution should be able to retire an unsafe version immediately. It can deploy the improved version while preserving a reproducible decision snapshot, the applicable policy, and an appeal route for prior outputs. Prospective safety and retrospective responsibility can travel on separate timelines.

Nor should every performance change be treated as a confession. A workable reachback policy can use materiality thresholds. It can ask whether the update corrects a confirmed defect, whether the defect could change consequential outcomes, whether an affected cohort can be identified, whether a remedy remains possible, and whether reopening would create greater injustice than it repairs. Ordinary adaptation may require explanation without reassessment. A confirmed coding error that wrongly excluded a recognizable group may justify a cohort review.

The policy must also resist strategic classification. An institution should not be able to label a correction as routine adaptation only to avoid reachback. The classification needs evidence, independent review where stakes warrant it, and a route for affected people to challenge the label.

The Venice Commission’s Updated Rule of Law Checklist offers a useful analogy with careful limits. Legal certainty values accessibility, foreseeability, stability, and consistency, but law must also adapt. Legal systems handle change through effective dates, published amendments, transitional rules, preserved records, appeal doctrines, and continuing institutional responsibility. Adaptive AI does not need to copy every legal mechanism. It does need an equivalent answer to the question of what remains continuous when the rule changes.

This approach does not punish improvement. It stops improvement from becoming the reason no one must answer for what preceded it.

Conclusion: Let corrections reach the people who paid for them

Return to the patient denied the intensive-care bed. Suppose the patient’s deterioration becomes one of the cases that exposes a weakness in the earlier system. Engineers correct it. Later patients with similar characteristics receive admission because the institution learned from what happened.

Future patients are safer, and that is a real achievement. The first patient still occupies an unstable position. When the recommendation was imposed, the system was treated as valid authority. Once the outcome became useful, the patient became evidence from which the system could improve. When the family sought redress, the institution could point to that improvement as proof that the problem had already been solved.

The patient paid part of the epistemic cost of learning while others received the benefit.

That observation does not make every adverse outcome a debt. Medicine and other high-stakes practices cannot eliminate uncertainty, and tragedy does not prove negligence or injustice. The narrower concern arises when an institution uses a case to revise a decision rule but refuses to let the revision alter its account of the case that helped reveal the weakness.

The model need not remember its predecessor. The institution must. A legitimate learning system can improve forward while remaining answerable backward. Its authority may change methods, but its responsibility cannot expire faster than the consequences imposed in its name.

Human-AI process disclosure: This essay was developed from a user-supplied manuscript and researched and edited with AI assistance. SikiT is responsible for the published argument, source choices, and stated limits.

Sources

Related articles

Stay in the loop

Get new practical AI and technology articles in your inbox. Unsubscribe anytime.

Comments

Questions, corrections, and useful counterpoints are welcome. Keep comments specific and on topic.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Thanks for commenting

Get new practical AI and technology articles in your inbox. Unsubscribe anytime.

Return to the comments