FrankenRobotics

Autopsy: counterfactual debugging

6,879 words19 subsections11 tables5 listingsPlan version 1.1

On this page19
27.1 What Autopsy owns27.2 Decisions27.3 Forks from snapshots27.4 Classifying a fork27.5 Interventions and the placebo27.6 Interventional contrasts and the flip rate27.7 Coalitions and budgeted Shapley summaries27.8 Design parameters re-run from the episode start27.9 Precision for attribution27.10 The first-divergence algorithm27.11 The decision model for attribution27.12 Time-travel debugging as an agent workflow27.13 What Autopsy does not claim27.14 Verbs and refusals27.15 Invariants and proof obligations27.16 Known answers27.17 Gates, kill gate and pilot27.18 Open questions27.19 Attribution in single precision on the originating tier

Crate frob-autopsy, layer 5.

27.1 What Autopsy owns

frob explain assembles what happened: the failure class, the energy ledger, saturation, slip, and the earliest anomaly before the terminal event, which it labels Estimated because ordering anomalies in time is a heuristic. Autopsy turns that heuristic into a measurement. It restores the world to a moment before the failure, changes one thing, runs forward, and records whether the failure still happens. A proposed cause is tested by intervention; it is never inferred from the order in which monitors or solver stages reported.

Autopsy owns the semantics of forks (their classification and the earliest tick at which a given change may be applied), the intervention vocabulary do(factor := value), placebo forks, interventional contrasts with their flip rates, coalition experiments and budgeted Shapley summaries, minimal repair sets, paired re-runs from the episode start for design parameters, the reading of first divergences on forks (§27.10; the algorithm and its verb belong to §29.8), and the transcript of a debugging session. It consumes snapshots, ring buffers, replay and the fork mechanism from Chronicle (§29), verdicts from Arena (§24), the typed world edits of §26.3 and the typed design edits of §14, the shrinker of §26.7, and the tiers of §16.

A value in this chapter that follows from a stated formula is a derived value and a known answer of §27.16; every other number carries its label.

Everything in this chapter is S except one claim. Forks, classification, contrasts with a stability report, coalition tables and the divergence algorithm are established technique applied carefully. The claim that an agent repairs designs in fewer turns with these tools than with logs is a HYPOTHESIS with its own gate (GATE-AUTOPSY-08).

Counterfactual replay from restorable simulator state is established practice, and incumbent simulators expose restorable state and opt-in run-to-run determinism on one GPU (FACT, Appendix H). Nothing here is claimed as new. What the engine must supply for it to work is listed as needs: an exact null replay, replay of one world outside the batch it ran in, forks that mix interventions on different model variants in one batch, and a double-precision tier that runs the same model.

27.2 Decisions

  • ADR-AUTOPSY-01. A cause is what an intervention removes, net of a placebo. A factor is named only when removing it prevents the failure more often than an irrelevant perturbation does.
  • ADR-AUTOPSY-02. Classify a fork before restoring state. Whether a change may be applied at a mid-episode state is decided from the model difference it makes, before any state is touched.
  • ADR-AUTOPSY-03. Design-parameter interventions re-run from the episode start. History-dependent state would otherwise answer a different question (§27.8).
  • ADR-AUTOPSY-04. Attribution reports a flip rate at round-off scale; double precision is the default for one failure. Closed-loop legged control amplifies round-off enough to flip marginal outcomes in single precision over long windows (§27.9). Single precision on the originating tier is admitted by a measured rule (the single-precision lane, §27.19).
  • ADR-AUTOPSY-05. Contrasts first; Shapley values are a summary on a budget. The table of what each intervention and each pair did is the result. A Shapley summary is computed only when that table does not already answer the question.
  • ADR-AUTOPSY-06. Diagnostic interventions are not repairs. Switching a torque limit off locates a cause; it is not an edit anyone can build. The two kinds are typed apart.
  • ADR-AUTOPSY-07. The cause is a cause in the model. Every attribution is scoped to the model, the tier and the declared factor list.

27.3 Forks from snapshots S

A fork is a parent episode, a fork tick and a change set. DESIGN:

  • State. A fork starts from a complete checkpoint in the sense of §29: physical state; contact feature memory and impulses; clock phases and packet queues; controller, filter and policy hidden states; actuator electrical and thermal states; terrain edits; the command cursor. Random streams are counter-based and keyed by address, so they have no state to save.
  • Where forks can start. At any stored snapshot; at any tick held in the per-world ring buffer; and at any tick of any replayable episode, by replaying the world from its address up to that tick. The ring buffer is an optimization, not a requirement. At a per-world state of tens of kilobytes (§36), twenty snapshots covering two seconds for each of 8,192 worlds would take several gigabytes, so full-state rings are kept for a reservoir of worlds and every other failing world is reconstructed by replay, at the cost of one episode prefix on the CPU.
  • Identity. A child world has the address of its parent extended by a fork path. It shares its prefix with the parent by reference. A fork never mutates its parent (INV-AUTOPSY-04).
  • Randomness. By default a child reads the same streams as its parent, so the intervention is the only difference. A replicate changes one declared thing more: a state perturbation of stated size, or the key of one named stream.
  • Batching. One failing world is forked K times into a batch of K children. This relies on the rule that a world's trajectory does not depend on the batch it ran in (§35). Children that carry different model variants use the mixed batches of §17.
  • Horizon. A fork starts at a lead Δ before the failure, from the ladder 0.25, 0.5, 1 and 2 s, and by default runs to one second past the parent's failure time (TARGET), or to the episode horizon when asked.

27.4 Classifying a fork S

The service classifies every requested change before restoring anything, and returns the class, the earliest valid fork tick, and the fields of the checkpoint that forced a later or earlier start. The five classes below are the intervention semantics, and this section owns them. The fork mechanism of §29.8 (snapshots, restore, the shared prefix, the verb run.fork) implements exactly these five and adds none.

Class What changes Where it may start State handling
Continuation Nothing (the null fork), or a placebo Any checkpoint whose physical, clock, sensor and policy contracts match the run Restored bit for bit
Exogenous intervention A future disturbance, a setpoint, a scene property the robot has not yet touched The fork tick Restored; nothing regenerated
Contact-affecting intervention Friction or compliance of a surface in contact at the fork tick; the contact law The fork tick Positions and velocities restored; contact memory discarded and the first solve run cold with a raised iteration cap; the fork is labelled
Information intervention Sensor noise or latency; an estimator replaced by privileged state The fork tick Packet queues and filter state re-initialized as the intervention states; labelled diagnostic where no hardware could realize it
Structural or design-parameter intervention Mass, inertia, geometry, a gear ratio, an actuator card, a sensor mount, controller gains or memory, policy weights The authored initial condition only The edited design is settled from its own reset cache (§24.7); paired exogenous draws are reused; contact memory, proprioception, sensor histories and policy actions are regenerated

A continuation whose contracts do not match is refused (FROB-REF-580). A structural change requested at a mid-episode tick is refused with the earliest valid tick, which is zero, and the list of fields that forced the restart (FROB-REF-581). Negative control: grafting the contact warm start of the 60 mm wheel of §24.10 onto the 75 mm design at the lip must return a structured refusal (FROB-REF-582), not a run.

27.5 Interventions and the placebo S

An intervention is written do(factor := value). A factor is a path into the world or the model whose mechanism is declared: an edit class of §26.3 for worlds, a typed edit of §14 for designs, or one of a short list of diagnostic factors such as torque_limit := off, estimator := privileged and latency := 0. Each intervention changes one declared mechanism, and the model difference it makes is computed and stored with the fork (INV-AUTOPSY-02). A change that would alter two mechanisms, such as a wet floor that changes both friction and depth returns, is split into two interventions or refused (FROB-REF-586).

Three forks give every intervention its meaning.

  • The null fork applies the empty intervention. It must reproduce the parent bit for bit on the originating profile (INV-AUTOPSY-01). If it does not, the record is unsound and nothing is attributed (FROB-REF-583).
  • A placebo fork applies a change that no physical cause could depend on: one state coordinate perturbed by a relative 10⁻¹². The placebo flip rate ρ₀ is the fraction of placebo forks whose verdict differs from the parent's. It measures how much of an apparent rescue is mere sensitivity.
  • Replicates of an intervention apply the same change with independent perturbations of the same size. A second series at a relative 10⁻⁷, the scale of single-precision round-off, is run for every intervention that changes the outcome.

27.6 Interventional contrasts and the flip rate S

For a failing world and an intervention c with R replicates, Autopsy reports three numbers.

Quantity Definition
Rescue rate The fraction of replicates of do(c) in which the requirement is not violated
Contrast The rescue rate minus the placebo flip rate ρ₀
Flip rate under perturbation The fraction of replicates whose verdict differs from the unperturbed fork of the same intervention, at 10⁻¹² and at 10⁻⁷

Over a set of failing worlds the rescue rate estimates the probability of necessity, P(no failure under do(¬c) | failure). A factor is named only when its contrast is tested above zero against the placebo forks, by an exact one-sided test at the fixed replicate count or by an e-process when forks are bought one at a time, with the level divided by the number of factors (INV-AUTOPSY-05).

Label Condition (TARGET thresholds)
but-for One factor with contrast at least 0.5, tested above the placebo, and a flip rate of at most 0.1 at 10⁻¹²
contributing Tested above the placebo with a smaller contrast
joint No single factor rescues and a coalition does
alternative Several single factors each rescue
precedes-snapshot No intervention at the fork tick rescues, and the same change applied from the episode start does
precision-sensitive The failure is not reproduced in double precision and is not admitted to the single-precision lane (§27.9, §27.19)
chaotic: unattributable ρ₀ above 0.2. No factor is named

Illustrative numbers: of 200 failing worlds re-run from their forks, do(torque_limit := off) prevents the failure in 164, do(friction := nominal) in 38, and a placebo in 12. Net of the placebo the contrasts are 0.76 and 0.13. For a marginal failure, or for causes that interact, the report names a set of factors and not a top factor.

27.7 Coalitions and budgeted Shapley summaries S

Causes interact, so single interventions are not enough. For a factor list N of size m, the value of a coalition S ⊆ N is v(S), the rescue rate when all of S are applied together.

Budget. DESIGN: at most 12 factors, all singles and all pairs. That is 78 coalitions; with 8 replicates each, 64 placebo forks and the null fork it comes to 689 forks, and the 10⁻⁷ series adds 8 forks for each coalition that rescues. With forks of two or three seconds this is a small batch on any tier. For each pair the interaction v({i, j}) − v({i}) − v({j}) + v(∅) is reported.

Shapley summary. The Shapley value of factor i is φᵢ = Σ over S ⊆ N∖{i} of |S|!·(m − |S| − 1)!/m! · [v(S ∪ {i}) − v(S)]. It is computed exactly for m ≤ 5, which needs 32 coalitions, and by sampled permutations with an interval otherwise. It is computed only when no single factor and no pair changes the outcome. Exact values over ten factors would take 2¹⁰ = 1,024 coalitions and leave no budget for a single replicate, so the stability the method depends on could not be measured.

What the summary cannot say. Two hand-checkable cases are in the counterexample catalogue (Appendix N).

  • Two factors that rescue only together (v of the pair is 1, of each single 0) and two factors that each rescue alone (v is 1 for either single and for the pair) both have Shapley values (½, ½). The repair sets differ: both factors in the first case, either one in the second. The coalition table distinguishes them and the summary does not.
  • With v(∅) = 0, v({1}) = 1, v({2}) = 0 and v({1, 2}) = 0 the values are (½, −½). They sum to v(N) − v(∅) = 0 as they must, a single-factor repair exists, and factor 2 has a negative value because applying it cancels the rescue. A Shapley value can be negative and never establishes that a factor is necessary.

FrankenSim's Shapley estimator is variance-based and says of itself that its effects "are not causal effects" (FACT: frankensim/crates/fs-uq/src/product_shapley.rs). It is not used here; the interventional estimator is a few hundred lines of our own.

Minimal repair sets. The shrinker of §26.7 runs over the intervention set with the test "the failure no longer occurs", and returns repair sets that are 1-minimal. A rescue at a fork is not a minimal repair in any stronger sense, and it is not evidence that an edited design passes the task; that takes §27.8 and then a study.

27.8 Design parameters re-run from the episode start S

An agent wants to know whether a different gear ratio would have prevented a thermal trip. Applying the new ratio one second before the trip answers a different question: what happens to a motor that was heated by the old ratio's currents and is then geared differently. Winding temperature, battery state of charge, wear, estimator bias, a policy's hidden state and the contact memory all carry the history of the old design, and none has a canonical image under the edit.

Counterexample, hand-checkable. One thermal node with time constant 60 s, ambient 25 °C and a trip at 115 °C. At a gear ratio of 12 the steady-state rise for the task's torque is 192 K, so the motor trips at 60·ln(192/102) = 37.95 s of a 45 s task. Heating scales with the inverse square of the ratio, so at 16 the rise is 108 K.

Experiment Result
Ratio 16 from the episode start Peak temperature 25 + 108·(1 − e^(−0.75)) = 82.0 °C at 45 s. The task passes with 33 K in hand
Ratio 16 applied at 36.95 s, one second before the trip The winding is already at 113.3 °C and still heading for 133 °C. It trips at 42.4 s. The fork says the edit does not prevent the failure

The fork is not wrong about the experiment it ran. It ran the wrong experiment. DESIGN: an intervention classified as structural or design-parameter is never applied to a mid-episode state (INV-AUTOPSY-03). autopsy.rerun does the following instead.

  1. Apply the typed edit, which gives a new DesignId and ModelId and a reset cache of its own.
  2. Re-run the failing worlds from the start on the same world addresses, so scene, disturbances and noise are paired, while events that depend on the robot's actions keep their own logically keyed streams.
  3. Re-run a sample of worlds that passed, because a repair can break what worked.
  4. Report the paired outcomes, labelled Estimated, and hand the pair of designs to eval.compare for a verdict under §24.12.

The controller must be able to act on the edited body without retraining: the baseline controller re-synthesized from the edited model (§22), a family policy (§23) or a sampling controller. A policy trained for the original body alone is refused for design factors (FROB-REF-587), since its failure on the new body would be attributed to the edit. A forced fork of a design parameter remains available in a diagnostic mode, labelled as such, because it is occasionally the question someone means to ask.

27.9 Precision for attribution S

FACT (Appendix H): learned continuous-control policies in closed loop have measured maximal Lyapunov exponents of 0.03 to 0.16 per control step on walker and cheetah tasks, and a perturbation of 5 × 10⁻⁴ degrees in a torso angle produces significant divergence after 50 steps. The uncontrolled systems are not chaotic; the sensitivity comes from the policy with contact. At 0.16 per step over 80 steps, which is 2 s at that benchmark's 25 ms control period, a perturbation grows by about 3.6 × 10⁵. Single-precision round-off of 6 × 10⁻⁸ then reaches about 2% of the state scale, enough to flip a marginal failure; double-precision round-off of 10⁻¹⁶ stays near 4 × 10⁻¹¹.

DESIGN: by default attribution runs on batch-cpu in double precision, or on the referee. For a failure recorded on a single-precision tier the procedure is:

  1. Replay the null fork on the originating backend and require identical digests.
  2. Replay the same checkpoint in double precision on batch-cpu and apply the decision-level equivalence test of §35: the same verdict, failure class and binding atom, with the failure time inside the recorded envelope.
  3. If the two agree, attribute in double precision. If they do not, the failure goes to the single-precision lane (§27.19) with the numerical-variant interventions first; only if that lane does not admit it is the failure labeled precision-sensitive, attribution refused (FROB-REF-584) and the world passed to the artifact guard of §26.6.

Every attribution reports its flip rates at 10⁻¹² and at 10⁻⁷, and the growth of the separation between the placebo forks and the parent over the fork horizon, so a reader sees how far the result is from the regime where nothing can be attributed.

27.10 The first-divergence algorithm S

Comparing two runs, a fork with its parent, one tier with another, or one backend with another, starts with the first place they differ. Chronicle's teleport does this for exact classes by stepping digests. The general algorithm covers tolerance classes and avoids one trap. §29.8 owns it and telemetry.divergence runs it; it is restated here because every attribution reads its output.

first_divergence(A, B, mode):
  align the committed ticks and events of A and B by stable identifiers; a variable with no counterpart
    is listed as unpaired and is never matched by position
  if mode is exact:
      c_t = H(c_{t-1}, d_t) is the chained digest of the per-tick state digests d_t, stored at checkpoints
      bisect over checkpoints for the least k with c_k(A) ≠ c_k(B)
      replay both runs from checkpoint k − 1 and compare d_t tick by tick: the first unequal tick is t1
  if mode is tolerance, with a scale per class of variable:
      for each block of ticks, in chronological order:
          [lo, hi] = bounds on the block maximum of the scaled error
          hi ≤ 1 : the block agrees; go on
          lo > 1 : a mismatch lies inside; descend into the block
          otherwise : unresolved; refine this block before looking at any later block
      where samples are missing, report the earliest possible interval and the coverage gap
  at t1, re-run the tick with per-stage digests or residuals: the first stage over tolerance is s1
  report t1, s1, the variables that differ under their physical names and units, and the first tick at
    which a requirement atom or a task decision differs

The trap. Bisection is valid only on a predicate that is monotone and decided. "Some tick up to t differs" is monotone. "The states differ at tick t" is not, because an error can vanish and return. Counterexample: two traces of eight ticks that differ at ticks 1 and 5 and agree elsewhere. A midpoint search on the pointwise predicate probes tick 3 or 4, sees agreement, moves right and reports tick 5. The chained digest reports tick 1. The same trap applies to design bisect over a lineage (§28), where the predicate must be "the outcome has changed by this edit" and each step must be decided by a study.

Rules for reading the result. The first recorded discrepancy is labelled with its temporal resolution. The order in which solver stages report residuals inside a tick is not evidence of a physical cause. For a fork against its parent the first divergent tick is the intervention tick or later, and the useful output is the order in which classes of variable leave tolerance and the first decision-level divergence. When topology differs between the runs, only shared interfaces and task observables are compared: a leg that exists in one run has no counterpart in the other.

27.11 The decision model for attribution

States: the failure has a single removable cause, a joint cause, or none that can be attributed. Actions: buy a fork, name a factor or a set, or decline. Loss: naming a wrong cause sends an agent to edit the wrong thing, which costs a design iteration and its compute; declining costs one fallback to the heuristic ordering.

Decision:          Which repair edit should the agent try first for this failure?
Artifacts:         Typed interventions; a fork ledger; the placebo flip rate; a rescue rate per factor and
                   per pair; minimal repair sets; one label from §27.6.
Obligations:       The empty intervention replays the failure bit for bit, so the starting point is exact.
                   Each intervention changes one declared mechanism (model difference on file).
                   A cause is named only when its contrast beats the placebo under a test at a stated level.
Failure modes:     Chaos makes every nudge look like a cause. The cause precedes the snapshot.
                   The failure does not survive double precision.
Fallback:          The first-anomaly ordering of §29, labelled Estimated.
Fallback trigger:  Null-replay mismatch; placebo flip rate above 0.2; failed equivalence in double precision.
Budgeted mode:     12 factors, singles and pairs, 8 replicates, 64 placebo forks: 689 forks. A Shapley
                   summary only if no single and no pair changes the outcome.
Baseline:          The heuristic ordering; and one-at-a-time contrasts without coalitions, to measure
                   whether ten forks answer as well as seven hundred.
Rollback:          A configuration flag to the heuristic; recorded forks remain valid experiments.
Verification:      Planted single and joint causes; a planted chaotic failure; the thermal counterexample.

27.12 Time-travel debugging as an agent workflow

The workflow is five moves: read the heuristic, fork, intervene, attribute, re-run. Each move is one verb with a bounded output, and the session is recorded as a transcript, schema frob.autopsy.transcript/1, that replays every fork from its address. A session on a quadruped in the wet-floor kitchen of NS-04:

frob explain run r:19e2 --budget 600
    # first anomaly (Estimated): slip at the left front foot at 11.1 s, 0.2 s before the fall
frob run fork r:19e2 --world 311 --before failure --dt 1.0s --json
    # fork f:a41c at 10.30 s; class continuation; null replay bitwise on gpu-cuda;
    # double precision on batch-cpu is decision-equivalent (fall at 11.31 s against 11.30 s)
frob autopsy intervene f:a41c --placebo 64 --replicates 8 --coalitions pairs --json
        --do 'scene./floor/patch[galley-2]/film := dry'
        --do 'model.joint[*.knee].torque_limit := off'
        --do 'sense.imu.latency := 0 s'
        --do 'model.joint[*.knee].gear_ratio := 16'
    # exit 2: three interventions ran from the fork; gear_ratio is a design parameter and was
    # refused with FROB-REF-581 (earliest valid fork tick 0; forced by actuator thermal state,
    # policy history and contact memory)
frob autopsy attribute f:a41c --json
{ "schema": "frob.autopsy.attribution/1", "fork": "f:a41c…", "tier": "batch-cpu", "precision": "f64",
  "horizon_s": "2", "forks_run": 137, "null_replay": "bitwise",
  "placebo": {"n": 64, "flips": 2, "rate": 0.031, "separation_growth": "2.1e4"},
  "contrasts": [
    {"do": "scene./floor/patch[galley-2]/film := dry", "class": "contact-affecting", "rescued": "8/8",
     "contrast": 0.97, "flip_rate": {"1e-12": 0.0, "1e-7": 0.0}, "label": "but-for"},
    {"do": "model.joint[*.knee].torque_limit := off", "class": "diagnostic", "rescued": "3/8",
     "contrast": 0.34, "flip_rate": {"1e-12": 0.125, "1e-7": 0.375}, "label": "contributing"},
    {"do": "sense.imu.latency := 0 s", "class": "information", "rescued": "0/8",
     "contrast": -0.03, "flip_rate": {"1e-12": 0.0, "1e-7": 0.0}, "label": "none"} ],
  "pairs": [ {"do": ["film := dry", "torque_limit := off"], "rescued": "8/8", "interaction": -0.34} ],
  "refused": [ {"do": "model.joint[*.knee].gear_ratio := 16", "code": "FROB-REF-581",
                "earliest_valid_fork_tick": 0} ],
  "evidence": {"class": "Estimated", "scope": "cause in the model at this tier, among the listed factors"},
  "next_actions": [
    {"command": "frob autopsy rerun r:19e2 --edit 'regear *.knee 16' --worlds failing,sample:64",
     "why": "A design parameter is tested from the episode start on the same world addresses."},
    {"command": "frob nemesis shrink r:19e2 --world 311",
     "why": "The film is a but-for cause; find the smallest world that still fails."} ] }
frob autopsy rerun r:19e2 --edit 'regear *.knee 16' --worlds failing,sample:64 --json
    # new design d:8e77 with its own reset cache; baseline controller re-synthesized;
    # 23 failing worlds and 64 sampled passing worlds on the same addresses:
    # 17 of 23 now pass; 63 of 64 still pass, 1 new timeout; label contributing, Estimated
frob eval compare quad-a quad-a@regear --task kitchen-tray
    # the verdict comes from a study (§24.12), not from the autopsy

The numbers in the transcript are illustrative. What the transcript fixes is the shape: the heuristic proposes, forks test, design parameters go back to the start, and a study decides. The viewer of §32 can overlay a fork on its parent as a ghost, synchronized on the fork tick.

27.13 What Autopsy does not claim

  • Not a cause in the world. The result is a cause in the model at a tier. FACT (Appendix H): causal models of robot failures learned from simulation reached a sim-to-real accuracy of 70% and 72% in one study, with variables chosen by the experimenter.
  • Not beyond the factor list. A failure whose cause is not among the declared factors yields no named factor. The planted no-cause control tests this.
  • Not a minimal repair. Repair sets are 1-minimal under the tested interventions.
  • Not necessity from a Shapley value. §27.7.
  • Not a passing design. A rescue at a fork, or a re-run on the failing worlds, is not a verdict on the task. A study is.
  • Not an answer in chaotic regimes. Above the placebo threshold nothing is attributed.
  • Not a proof. A contrast is a replayable experiment and has class Tested on its tier; an attribution drawn from contrasts has class Estimated.
  • Not a differentiator of the engine. An incumbent with restorable state and a thin layer can host the same method (§39 includes that arm).

27.14 Verbs and refusals

Verb Kind First milestone Notes
autopsy.intervene run M3 Runs interventions, placebo forks, replicates and coalitions from a fork handle made by run.fork (§29.8)
autopsy.attribute read M3 Contrasts, flip rates, pairs, labels, an optional Shapley summary
autopsy.repair run M3 Minimal repair sets by shrinking the intervention set
autopsy.rerun run M3 Paired re-run from the episode start for design-parameter and structural edits
autopsy.show read M3 The transcript of a session, token-budgeted

Autopsy registers no fork verb and no divergence verb. A fork point is made with run.fork (§29.8, M2), which returns the class of a proposed change set under §27.4, the earliest valid tick and the null-replay result; --classify-only restores nothing. The first divergence between two runs, in exact or tolerance mode (§27.10), is read with telemetry.divergence (§29.8, M2).

Code Refusal
FROB-REF-580 A continuation whose physical, clock, sensor or policy contract does not match the checkpoint
FROB-REF-581 A structural or design-parameter intervention requested at a mid-episode tick; the earliest valid tick and the forcing fields are returned
FROB-REF-582 State from one model grafted onto another
FROB-REF-583 The null fork does not reproduce its parent
FROB-REF-584 Attribution requested for a failure that neither lane admits, or under an uncontrolled determinism class
FROB-REF-586 An intervention changes more than one declared mechanism
FROB-REF-587 A design-parameter re-run with a policy that cannot act on the edited body without retraining
FROB-REF-880 An attribution cited as an established cause, or as a cause in the physical robot

27.15 Invariants and proof obligations

  • INV-AUTOPSY-01. The empty intervention replays its parent bit for bit on the originating profile; otherwise nothing is attributed.
  • INV-AUTOPSY-02. Every intervention changes one declared mechanism, and its model difference is stored with the fork.
  • INV-AUTOPSY-03. No structural or design-parameter intervention starts from a mid-episode state, except in the labelled diagnostic mode.
  • INV-AUTOPSY-04. A fork never mutates its parent; a child has its own address and shares the prefix by reference.
  • INV-AUTOPSY-05. A factor is named only when its contrast is tested above the placebo at a stated level and the placebo flip rate is at most 0.2.
  • INV-AUTOPSY-06. Every attribution carries its scope: the model, the tier, the precision and the factor list.
Obligation Statement Class sought
PO-AUTOPSY-01 The Shapley summary satisfies efficiency, the dummy property and symmetry on the computed coalition table Tested in exact rational arithmetic on integer counts
PO-AUTOPSY-02 first_divergence in exact mode returns the least tick at which the prefixes differ Tested with planted divergences; the monotonicity argument is on file in docs/derivations/
PO-AUTOPSY-03 In tolerance mode, an interval reported as agreeing has a true maximum scaled error of at most 1 Tested with outward-rounded block bounds

27.16 Known answers

Known answer Expected Negative control
Planted single causes: 200 failures in five classes (friction drop, torque cap, payload shift, latency, terrain step), ten candidate factors each The planted factor ranks first with label but-for A failure planted through a factor outside the list names no factor
Null fork Identical digests A checkpoint with its packet queue stripped does not replay and is refused
Chaotic placebo: a doubling map with failure defined on its fortieth iterate A 10⁻¹² placebo is amplified by 2⁴⁰ ≈ 1.1 × 10¹², about half of placebo forks flip, and the label is chaotic: unattributable The same map over five iterates attributes normally
Joint and alternative pairs Labels joint and alternative; Shapley values (½, ½) in both; repair sets {1, 2} and {1} or {2} A summary that reports only Shapley values cannot tell them apart
Negative Shapley value (½, −½) for the table of §27.7; the values sum to 0 exactly —
Thermal re-run 37.95 s; 113.3 °C at the fork; a trip at 42.4 s in the forced fork; 82.0 °C peak from the start A tool that reports the forced fork's result as the answer to the design question
Warm-start graft Refused with FROB-REF-582 —
First divergence Tick 1 for traces that differ at ticks 1 and 5 The pointwise midpoint search reports tick 5
Tolerance mode with a gap The earliest possible interval and the coverage gap A tool that skips the gap reports a later tick
Precision A marginal failure built to flip under 10⁻⁷ and not under 10⁻¹² is labelled precision-sensitive when it came from a single-precision run The same failure with a wide margin attributes normally

27.17 Gates, kill gate and pilot

Gate Claim Threshold (TARGET) Earliest
GATE-AUTOPSY-01 Known answer Null forks of 1,000 failing worlds replay with identical digests on the originating backend; a world forked alone and in a batch of 512 children gives identical digests M2
GATE-AUTOPSY-02 Known answer Classification: 40 catalogued pairs of intervention and state are classified correctly; the warm-start graft and a mid-episode gear change are refused with the right fields M2
GATE-AUTOPSY-03 Known answer Planted single causes: the planted factor ranks first in at least 90% of 200 failures in double precision, and in the single-precision lane for windows of at most 1 s, and the top factor changes in at most 30% of cases under 10⁻⁷ perturbation; the agreement between ten one-at-a-time forks and the full coalition table is recorded M3
GATE-AUTOPSY-04 Known answer Coalitions: planted joint and alternative causes are labelled correctly in at least 95 of 100; the Shapley identities hold exactly; the negative-value case is reproduced M3
GATE-AUTOPSY-05 Known answer Abstention: a planted chaotic failure and a planted no-cause failure name no factor in 100 of 100 M3
GATE-AUTOPSY-06 Known answer The thermal counterexample: all four numbers within 0.1 s and 0.1 K; the mid-episode request is refused and the re-run passes M3
GATE-AUTOPSY-07 Known answer First divergence: planted divergences found at the right tick and stage in 100 of 100 in exact mode; the pointwise mutant is caught; tolerance mode reports gaps M2
GATE-AUTOPSY-08 Design comparison On the planted-defect designs of the corpus (§24.9), agents given Autopsy reach an accepted repair in at most one third of the serial turns of agents given logs and explain alone, pre-registered, with an anytime-valid verdict M4
✖ KILL-AUTOPSY — Trigger: GATE-AUTOPSY-03 fails after one repair cycle: planted-cause accuracy below 90%, or the top factor changes in more than 30% of cases at 10⁻⁷. Fallback: single-factor paired re-runs from the episode start, with the first-anomaly ordering labelled Estimated; coalitions and Shapley summaries are removed M3

Pilot. The planted-cause test runs at paper scale before the coalition machinery is funded: one trained quadruped policy on a development oracle outside the product (§38) or on the CPU referee, 200 planted failures in five cause classes with ten candidate factors, one-at-a-time contrasts against exact Shapley over 1,024 coalitions, each repeated under 20 snapshot perturbations at 10⁻⁷ and at 10⁻¹². It takes about a day on a 32-core machine. Its falsifier is the trigger of KILL-AUTOPSY, and it also answers whether ten forks suffice. The ceiling of the benefit is stated in advance: the output is an accuracy, not a time. It can shorten a design decision only by removing agent turns spent on wrong repairs, so its ceiling is the share of turns that diagnosis takes in the phase ledger of §39. If GATE-AUTOPSY-08 fails, the tools stay, because they are cheap and exact, and the claim about turns is withdrawn.

27.18 Open questions

  • OPEN-AUTOPSY-1. Default replicate count and perturbation sizes. Owner: Autopsy lead. Decided by the pilot. Default: 8 replicates at 10⁻¹², and 8 at 10⁻⁷ for interventions that change the outcome.
  • OPEN-AUTOPSY-2. Whether one-at-a-time contrasts suffice, which would make pairs optional. Owner: Autopsy lead. Decided by GATE-AUTOPSY-03. Default: singles and pairs.
  • OPEN-AUTOPSY-3. Lyapunov exponents of our own controllers and policies on our tiers, which set the usable fork horizon. No measurement exists for trained quadruped policies in any engine we could find. Owner: Cortex lead. Default: leads of at most 2 s, with the growth of placebo separation reported in every attribution.
  • OPEN-AUTOPSY-4. The size of the snapshot reservoir on each device. Owner: Chronicle lead. Default: a reservoir sized by the budget estimator; every other fork point is reconstructed by replay.

27.19 Attribution in single precision on the originating tier

The single-precision lane F. This amends ADR-AUTOPSY-04 and §27.9. Double precision on the CPU remains the default for one failure, where it costs little. It is not a requirement of the evidence. What the evidence supports is a flip rate measured under perturbations at or above round-off scale (§27.5), and that measurement can be made on the tier where the failure happened.

Why it is admissible. FACT arithmetic on the exponent quoted in §27.9, 0.16 per control step: the growth factor e^(0.16·n) is 24.5 at n = 20, 602 at n = 40 and 3.6 × 10⁵ at n = 80. Single-precision round-off of 6 × 10⁻⁸ therefore reaches 1.5 × 10⁻⁶ of state scale after 20 steps, 3.6 × 10⁻⁵ after 40 and 2% after 80; at that benchmark's 25 ms period these are fork windows of 0.5 s, 1 s and 2 s. A verdict that moves under a relative change of 10⁻⁵ in state also moves under sensor noise and friction scatter two orders larger. It is marginal in any arithmetic, and the right output is the label marginal, not more digits. HYPOTHESIS: the exponent of our closed loops is of the same order. It was measured on walker and cheetah benchmarks under learned policies, and the gate below measures ours.

Procedure for a failure recorded on a single-precision tier.

  1. The null fork on the originating profile must give identical digests (unchanged, INV-AUTOPSY-01).
  2. R₀ placebo forks at a relative 10⁻⁷ on the originating tier over the fork window W (DESIGN default R₀ = 32). Record the placebo flip rate ρ₀ and the measured growth G(W) of the separation between placebo forks and parent.
  3. Admission. ρ₀ ≤ 0.1 and G(W)·u ≤ 10⁻⁴ of state scale, with u = 6 × 10⁻⁸. Attribution then runs on the originating tier in single precision with at least 16 replicates per intervention at 10⁻⁷, contrasts tested against the placebo as in §27.6, and the record carries tier: <backend>, precision: f32.
  4. Not admitted. Shorten W to the longest window that passes, within the bound the first-divergence algorithm of §27.10 gives on how late a fork may start, or escalate to double precision as in §27.9.
  5. A failure that double precision does not reproduce is no longer refused outright. It is attributed in this lane with the numerical-variant interventions first: half step, doubled solver iterations, the alternative contact law, the stricter step profile. A rescue by one of them gives the label tier-specific: <variant> and passes the world to the artifact guard of §26.6. Only when the lane is also not admitted is the failure labelled precision-sensitive and attribution refused.

A cross-check that can fail. One attribution in k (DESIGN default k = 8), and every attribution that enters a report, is repeated in double precision on batch-cpu; the top factor, its label and the sign of each contrast are compared, and the agreement rate is a ledger figure.

Known answers. (i) The linear map x ← 1.1735·x has exponent ln 1.1735 = 0.16 per step: after 40 steps a perturbation has grown by 602, and the measured G must match within 1%. (ii) Negative control: a pendulum released 5 × 10⁻⁸ rad from upright with no control, with the verdict "falls to the positive side". A placebo of 10⁻⁷ rad flips the side whenever its sign is negative, so ρ₀ is about one half; the lane must not admit the case, and the label is marginal. At a placebo of 10⁻¹² the side is stable, which is why the 10⁻¹² series alone would have hidden the problem.

Cost and ceiling. One failure with ten factors and 16 replicates is 160 forks of 2 s: 320 world-seconds, seconds on either tier, and the lane saves nothing. A cluster of 200 failures with pair interactions (55 interventions × 32 replicates × 2 s) is 7.0 × 10⁵ world-seconds: about 3 minutes on a device producing 4,000 world-seconds per second, against 12 minutes to 2 hours on batch-cpu at 1,000 to 100. Ceiling: the wall time of bulk attribution, by the ratio of device to CPU world-seconds; and attribution of tier-specific failures, which the double-precision rule could not attribute at all.

Owner and verb. frob-autopsy. autopsy.attribute --lane {f64, tier-f32, auto}; auto applies the admission rule and records which lane ran and why.

Gate Claim Threshold (TARGET) First milestone
GATE-AUTOPSY-09 Known answer; Design comparison The linear map and the pendulum control behave as stated. G(W) is recorded for the reference quadruped and humanoid policies at 0.5 s, 1 s and 2 s. On the planted-cause suite of GATE-AUTOPSY-03 with windows of at most 1 s, the lane names the same top factor as double precision in at least 95% of admitted cases. A planted solver artifact that double precision does not reproduce is labelled tier-specific with the right variant. Bulk attribution of 200 failures is at least 3 times faster in wall time than on batch-cpu M3
Kill gate Trigger Fallback
✖ KILL-ENSEMBLE Agreement with double precision below 90%; or fewer than half of the reference failures admitted at any window of 0.5 s or more; or a bulk speedup below 3 ADR-AUTOPSY-04 as first written: attribution in double precision only, and a failure not reproduced there goes to the artifact guard unattributed