FrankenRobotics

Atelier: the studio loop, analyses, comparison, lineage, cascade, decision cores

21,486 words30 subsections24 tables5 listingsPlan version 1.1

On this page30
28.1 What Atelier owns28.2 Decisions28.3 The loop28.4 Analyses28.5 Explain28.6 Comparing designs honestly (C-15)28.7 Lineage: diff, blame, bisect, merge28.8 Optimization28.9 The ladder of evaluation rungs28.10 The rank-agreement ledger28.11 The six-stage co-design cascade28.12 Screening audits28.13 Revalidation after edits (result.revalidate)28.14 The memo graph28.15 Campaign-level deduplication28.16 Experiment suggestion28.17 Registered predictions28.18 Reports28.19 Decision cores28.19.1 The core contract28.19.2 The eleven cores28.19.3 Composition: timescales and interference28.19.4 Methods rejected as ornamental or premature28.20 Refusals28.21 Invariants and proof obligations28.22 Known answers and negative controls28.23 Gates and kill gates28.24 Open questions28.25 Controller-budget curves and the controller-limited label28.26 The reuse ledger: three columns, never summed

28.1 What Atelier owns

Atelier is the design studio: the loop in which a design is checked, given a controller, simulated, explained, edited and compared, and every rule that decides how evaluation effort is spent inside that loop. It is two crates. frob-analyze holds the analyses A-01 to A-26, each of which returns a typed finding with ranked repairs. frob-opt holds comparison, lineage queries, optimization, the evaluation ladder and its allocation rules, the co-design cascade, revalidation, experiment suggestion, registered predictions, reports, and the compiler for the decision-core tables of §28.19.

Atelier owns Atelier uses and does not own
The order of the studio loop and the right of each stage to refuse the next Design terms, typed edits and the model compiler (§14)
Analyses A-01 to A-26 and the aggregate verb design.check The three tiers, the trust ladder and effort profiles (§16)
Paired comparison in two declared study types Family batches and campaign deduplication (§17)
Lineage queries: diff, blame, bisect, merge Baseline controllers, gait orbits and skills (§22); family policies and inheritance (§23)
The ladder of evaluation rungs, its allocation rule and the rank-agreement policy Tasks, requirements, batteries and the blinded qualification lane (§24)
The six-stage co-design cascade and its screening audits Mission queries and family certificates (§25)
Revalidation of verdicts after an edit (result.revalidate) Adversarial worlds and rare-failure estimation (§26); interventional attribution (§27)
The memo policy: what may be served from the memo graph, and when Every ledger and the memo store, which are tables of Chronicle (§29)
Experiment suggestion, registered predictions, reports Calibration against reality (§30); the daemon and its queue (§32)
The decision-core contract and the eleven decision rules DC-01 to DC-11 The mechanisms those rules govern, each in its own chapter

Where a decision core governs a mechanism that another chapter owns, this chapter fixes the decision rule (the table, its loss model, its obligations, its fallback) and the owning chapter fixes the mechanism.

The mechanisms of this chapter at a glance. Every Frontier row has its pilot, its ceiling and its kill gate in the section named.

Mechanism Tier Verb Gate Kill gate and fallback Section
Analyses A-01 to A-26 and check S, some F design.check, analyze.<name> GATE-ATEL-01, -02, -05 KILL-STATIC: advisory lints §28.4
Explain, with comparison and symmetry views S explain.run, explain.design, explain.compare GATE-ATEL-16 — §28.5
Paired comparison, two study types S eval.compare, eval.preanalyze GATE-ATEL-04, -06 — §28.6
Lineage queries and merge S design.diff, design.blame, design.bisect GATE-ATEL-04, -16 — §28.7
Continuous optimization, racing, archive S opt.start, opt.status, opt.archive GATE-ATEL-04, -13, -16 — §28.8
A family per dispatch; paired differences F opt.start --family Pilot of §17.13 KILL-FDGRAD (owner §17.13): rank-based search §28.8
The ladder: fixed table S opt.cascade GATE-ATEL-16 — §28.9
The ladder: conformal survivors, value of information F opt.cascade GATE-ATEL-08, -15 KILL-LADDER, KILL-VOI: the fixed table §28.9
Rank-agreement ledger and modes F eval.rank-agreement GATE-ATEL-07 KILL-RANK: uninformative everywhere §28.10
The cascade and screening audits S controls, F as a whole opt.cascade; nemesis.audit (owner §26.10) GATE-ATEL-09, -10, -16; GATE-NEMESIS-07 Stage by stage, as above §28.11, §28.12
Revalidation, grades 1 and 2 S result.revalidate GATE-ATEL-11 — §28.13
Revalidation, grade 3 F result.revalidate GATE-ATEL-11 KILL-REUSE: grades 1 and 2 §28.13
Memo graph S Every verb; --no-memo GATE-ATEL-12 Stop serving a node kind on one mismatch §28.14
Experiment suggestion F experiment.suggest (owner §25.3.1) GATE-ATEL-15 KILL-VOI: unranked proposals §28.16
Registered predictions S predict.register, predict.report GATE-ATEL-16 — §28.17
Reports S report.design, report.campaign GATE-ATEL-16 — §28.18
Decision cores DC-01 to DC-11 S and F cores.show, and the verbs they govern GATE-ATEL-13, -14 Each core's own fallback §28.19

28.2 Decisions

  • ADR-ATEL-01: the order of the loop is a rule. Check before you simulate, simulate before you train, train before you claim. Reason: each stage is cheaper than the next by roughly an order of magnitude (HYPOTHESIS; settled by the phase ledger of §39) and each can refuse the next.
  • ADR-ATEL-02: two study types, never mixed inside one study. A comparison is declared either as an anytime-valid e-process study or as a fixed-n exact study with a pre-analysis of which verdicts are reachable at that n. Reason: time-uniform inference and a sealed fixed-n rule compute different decisions; data collected under one is not re-read under the other.
  • ADR-ATEL-03: successive halving is the fixed baseline and the fallback of the ladder. Every adaptive allocation rule ships in shadow behind it and reverts to it on a named trigger. Reason: an adaptive rule can fail in a way that costs validity or the best design; the fixed table can only cost compute. One asymmetry governs every rule in this chapter: reporting an artifact-dependent result as established, or eliminating the best design, costs far more than a wasted higher-rung episode.
  • ADR-ATEL-04: a cheap rung discards; it does not select. A rung may choose finalists only in the strata where the rank-agreement ledger (§28.10) has measured that it can. Reason: no published measurement of rank agreement between a shared policy and trained specialists exists, and the nearest neighbours are weak (§28.10).
  • ADR-ATEL-05: a hash identifies; only an implication licenses skipping a recheck. A content hash says that an input or a result is the same object. It says nothing about whether a verdict survives a different input. Reuse of a verdict needs a dependency argument or an implication argument, recorded and independently checkable (§28.13).
  • ADR-ATEL-06: decision rules are compiled to tables. Every adaptive decision in the studio is compiled ahead of time into an integer guard table, an automaton or a monitor with a version, stored under registries/cores/, regenerated in continuous integration and diffed. The runtime evaluates tables. No decision reads the wall clock, device load or arrival order. Reason: replay, audit, and a rollback that is a configuration flag.
  • ADR-ATEL-07: parametric models rank experiments; distribution-free tests issue verdicts. The value-of-information model of DC-05 never issues a verdict, so a wrong model costs efficiency and never validity.
  • ADR-ATEL-08: agents are the discrete optimizer. Agents choose topology, leg count and actuator families. The studio supplies fast continuous optimization, population evaluation and honest comparison underneath.
  • ADR-ATEL-09: the controller gate. No adaptive controller is enabled by default without a replay scenario, evidence records, a calibration report and a demonstrated fallback. Adoption is always shadow (report both rules, act on the old), then canary, then default; the incumbent in each shadow run is the plan's own reference path.
  • ADR-ATEL-10: two planes. Kernels, step plans and physics never depend on the dispatcher, the memo policy or any estimator. The decision plane reads results and writes requests; it cannot change what a world computes.
  • ADR-ATEL-11: decision cores are named DC-01 to DC-11. Reason: the letter D already names determinism classes (D0 to D4) and doctrine rules; a second meaning would make "D2" ambiguous in exactly the sentences where both appear.

28.3 The loop

 design ──► check ──► synth controller ──► simulate ──► explain ──► edit ──► compare ──┐
   ▲        (static,   (seconds)           (batch or     (telemetry,  (typed)  (anytime-  │
   │         seconds)                       GPU)          energy,              valid or   │
   │                                                      forks)               fixed-n)   │
   └── revalidate ◄── lineage, archive, ladder, cascade, optimize, train ◄────────────────┘

The order is a rule, not a suggestion: check before you simulate, simulate before you train, train before you claim. Each stage is cheaper than the next by roughly an order of magnitude, and each can refuse the next. Two things are added to the loop of the first studio milestone. An edit does not only produce a new design; it produces an invalidation set, the verdicts and derived assets that the edit puts in question, and result.revalidate says which survive (§28.13). And a population of designs does not walk the loop one at a time; it climbs the ladder of §28.9 together, in mixed batches (§17), with the cascade of §28.11 deciding who climbs.

The loop is measured, not assumed. Every verb records its phase (authoring, check, synthesis, simulation, training, promotion, review) and its cost in the phase ledger that the ten-fold test reads (§39). A phase that accounts for less than nine tenths of elapsed time cannot deliver a ten-fold improvement alone even if its cost falls to zero, so the ledger is read before any speed claim is made about a stage.

28.4 Analyses

A-01 to A-18 and A-20 to A-25 need no batch simulation; the orbit solve of A-04 is a small CPU computation. A-19 and A-26 read runs. frob design check runs the ones marked ✓ and returns in under two seconds for a template-sized design (TARGET). An analysis marked "✓ when bound" joins check only when the request names the scene, budget or skill it needs.

ID Analysis Question it answers Mathematics In check Tier
A-01 Mobility How many degrees of freedom does it really have? Is any loop over-constrained? Rank of the constraint Jacobian at exact rational sample points; the subgroup lattice (C-01) ✓ S / F
A-02 Mass roll-up Mass, centre of mass and inertia by subtree; physical consistency C-03 ✓ S
A-03 Symmetry What is the symmetry group, how does it act, and what is the smallest edit that restores a near-symmetry? Which subgroup also preserves the sensors, the payload and the task? C-09; two nested groups, the mechanical group and its observation-preserving subgroup (§13, §20). Only the subgroup may be used for weight sharing, deduplication and gait orbits ✓ S
A-04 Gait atlas and orbits Which gaits can this body have, with what phase lags and stability margins? Does a periodic orbit exist under the baseline controller, and how does it move with a design parameter? C-10; periodic orbits by shooting on the compliant tier, full-space Floquet multipliers, continuation in design parameters (§22). Labelled a screening signal F
A-05 Workspace Can the feet or the hand reach what the tasks require (step height, shelf height)? C-11 S
A-06 Singularities Where are they, and how close does a nominal gait come? C-11 S
A-07 Maneuverability Growth vector, turning footprint, cost of a sideways shift, door and aisle passability C-07; with A-18 when a scene is bound ✓ for wheeled S / F
A-08 Static stability Support margins, tip-over angles on slopes, contact-wrench-cone margin with payload C-06; the result is typed exact, inner bound or outer bound, and names its load-sharing model ✓ S
A-09 Connection curvature Displacement per shape cycle for kinematic locomotors; reorientation capacity in flight C-08 F
A-10 Co-design closure Does the mass spiral close? What is the least mass and power? C-12 ✓ S
A-11 Actuator envelope Do the torque-speed points of standing, crouching, stepping up and nominal gaits fit inside each card, with thermal steady state? §15 ✓ S
A-12 Modes Natural frequencies and mode shapes at stance, block-diagonalized by symmetry; is actuator bandwidth sufficient? C-09 S
A-13 Self-collision and clearance Do limbs sweep through each other or the ground over their ranges? Swept charts (fs-motion, FACT) ✓ S
A-14 Sensor coverage Does the sensor suite see the footholds and the route ahead? Ray geometry S
A-15 Structural margin Would a link yield under the peak loads seen or bounded? Beam theory; finite elements through fs-solid; closed into the loop by §31 F
A-16 Sensitivity Which parameters matter most for a chosen metric? Automatic differentiation for static metrics; Morris screening and smoothed gradient estimates for simulated ones S / F
A-17 Observability and aliasing Can the state the task needs be reconstructed from the sensor suite at the declared rates and delays? Are there two admitted worlds with the same observation history that need different actions? Rank and conditioning of the observability Gramian along nominal motions; an alias witness as an explicit pair of worlds (§20) ✓ (structural rank) S / F
A-18 Clearance and route feasibility Does the footprint family fit the routes of a bound scene: fits, cannot fit, or unresolved, with the bottleneck named? The clearance complex of the scene (.fclear, §19) against the swept footprint; two-sided: a witness path or an impossibility cut ✓ when bound F
A-19 Rank robustness Does the ranking of the finalists survive a change of contact law, cone, stiffness, step or solver tolerance? Did any finalist profit from energy the simulator created? Re-evaluation of the top-k and the runners-up under model-form variants (§28.11, stage 5); the created-energy audit (§29.9) S / F
A-20 Energy-budget feasibility Can the mission's mandatory work be paid from the declared energy and power budget? Can the body deliver more energy into a contact than the safety budget allows? A lower bound on mandatory work (potential rise, unavoidable dissipation) against stored, injected and approach kinetic energy under the .fenergy budget; port quantities of C-12 ✓ when bound S / F
A-21 Deadline and latency closure Does the last informative observation arrive before the decision it must inform, on every admitted schedule? The seven instants of the clock contract (§20) composed along the sense-decide-actuate path in exact rational time, with the boundary ordering rule ✓ S
A-22 Sampled-loop stability Is each actuator's current and speed loop stable at the declared control period, feedforward law and reflected inertia? The one-tick map of the sampled-data loop with the hold modelled exactly; its spectral radius ✓ S
A-23 Shared-bus schedule Can all consumers on a shared electrical bus meet their demands by their deadlines, jointly? Joint time-indexed feasibility over the bus contract (exact enumeration when small, a linear program otherwise). Never a sum of peaks ✓ S
A-24 Task-wrench attainability Can this body produce the wrench a skill or a task needs inside torque, speed, friction and support limits? The cone program shared with contact (C-06, §22); infeasibility returns a separating dual vector ✓ when bound S
A-25 Identifiability Would a proposed experiment determine the card or scene parameters on which the pending decision depends? Rank of the sensitivity of predicted readings to the parameters over the proposed excitation; coverage of each regime of a switching law (§30) S / F
A-26 Regime atlas In which contact modes, slip states and saturation states did the design operate, where is a gradient trustworthy, and where does a ranking change? A partition of recorded operation by mode, slip, saturation and derivative validity; per region a local model, an error estimate and the shortest replayable failure F

Every analysis returns its result with an evidence class, the model identity, and, when it finds a problem, ranked repairs as typed edits. Four rules are added.

  1. A bound says which side it is on. A feasibility result is typed exact, inner (everything reported feasible is feasible) or outer (everything reported infeasible is infeasible), and it names the authority model it assumed. DESIGN, because the same body gives different answers under different load-sharing models: under independent foot forces a stance needs 155 N at the worst foot, and under equal passive springs the forces are (175, 115, 85, 25) N against a 160 N cap (FACT, executed stance enumeration; source register).
  2. An exclusion encloses every state and every energy source. A static or energy argument may exclude a design only for a model whose states and energy sources it encloses. A-20 therefore always includes approach kinetic energy.
  3. An analysis is a memo node. Its key is the model identity restricted to the fields it reads, its own version and its parameters; its determinism class is D0 for the exact analyses. An edit that does not touch those fields recomputes nothing (§28.14).
  4. A screening analysis is labelled as one. A-04 orbit results, A-17 Gramian conditioning and A-26 regions are screening signals. They may discard under the rules of §28.9 and may not establish a comparison.

Frontier analyses and their pilots. A-04, A-17's alias witness, A-18, A-24's witness and A-25 are piloted and gated in the chapters that own their mathematics (§22, §20, §19, §22, §30); A-15 in §31; A-09 remains among the first items to be cut if its payoff is not measured. A-26 is piloted here. Ceiling: the atlas computes nothing new about the robot; its whole value is fewer wrong gradient steps and fewer misread rankings. Pilot: on recorded runs of the reference quadruped, test whether a region's validity label predicts the sign agreement between a smoothed gradient estimate and a specialist-checked finite difference. Kill gate KILL-ATLAS: the label predicts no better than "trust nowhere" at a 95% lower bound; fallback: A-26 is a descriptive report with no validity label.

Shortcuts that are wrong. Each row is a hand-checkable case in which a plausible analysis gives the wrong answer. They are seeds of the negative-control registry (§38).

Shortcut Smallest case that breaks it What the analysis does instead
Support polygon plus total actuator capacity as a stance screen (A-08) In an exhaustive stance enumeration of 11,520 body and load cases the screen accepts 1,596 bodies that the exact classifier rejects (FACT, executed; source register) Per-contact force limits enter the cone program; the result is typed and names its load-sharing model
Adding peak currents on a shared bus (A-23) Two fingers each need one powered tick; the bus powers one finger per tick. With a one-tick deadline no joint schedule exists; with two ticks exactly two exist. A rule that adds both peaks rejects the two-tick design, which is feasible The joint time-indexed contract is kept; a prune is rejected if any discarded design is feasible under the same mission
Connection curvature as a test for a misaligned mecanum roller (A-09) A base with misaligned rollers still has spanning curvature, as a differential drive does The defect is a rank drop in the connection, which the growth vector of A-07 reports
The mechanical symmetry group as the policy's symmetry (A-03) A quadruped with one forward lidar has the four-element group mechanically and a two-element group in closed loop Both groups are stored on each robot-and-task pair; only the observation-preserving subgroup is used
A quasi-static energy exclusion without approach energy (A-20) An 8 kg body on a 12 mm rise needs 8 × 9.81 × 0.012 = 0.94176 J; a 0.90 J budget excludes the passive crossing; 0.05 J of approach kinetic energy removes the exclusion The budget includes kinetic energy at the start of the segment; the result is outer only for the enclosed model
Per-reading accuracy as the value of sensor redundancy (A-17) Seven independent readings, each right with probability 0.75, give a majority success of 3807/4096 ≈ 0.929; seven readings with a perfectly shared error stay at 0.75 The alias analysis reads the declared error dependence of the cards, not the per-reading accuracy
An instantaneous-current model of the drive loop (A-22) For a 75 mm wheel with total reflected inertia 0.00107890625 kg·m² and a current target held for 10 ms, the one-tick spectral radius is 1.4820 with target-current feedforward and 0.9207 with measured-current feedforward; the instantaneous model gives 2.6917 for both with the fixture's shaft damping (2.7075 without it) and misses the stable case (FACT, executed pilot; re-derived from the fixture before it is pinned) The hold is modelled exactly over the control period
Identifying friction from a convenient run (A-25) With drawbar force F = min(k_t·I/r, μ·N), k_t = 0.2 N·m/A and r = 0.06 m, runs at 1 A and 2 A under 30 N and 40 N give 3.33 N and 6.67 N for μ = 0.4 and for μ = 0.8; at 3 A under 10 N they give 4 N and 8 N The analysis reports zero sensitivity to μ for the first two runs and asks for an excitation in the traction-limited regime
A late observation with the right content (A-21) A packet that arrives at tick 1 serves decisions that commit at ticks 2 and 4; delayed to tick 3 it still serves the second and no longer serves the first Arrival instants are compared with decision instants per decision, in rational time

28.5 Explain

After a simulation, frob explain <run> assembles: the failure taxonomy with first-cause ordering (§29.10); the energy ledger (§29.9); the saturation report by joint; friction utilization and slip; the gap between estimated and privileged state; and a ranked list of typed edits predicted to help, computed from A-16. The list is the envelope's next_actions.

Four additions make the explanation harder to misread.

  • Description and cause are different outputs. The first-cause ordering is a description: it orders anomalies by time and is labelled Estimated. A causal statement ("torque saturation caused the fall") comes only from a controlled intervention, which is the fork machinery of §27 under decision core DC-06. explain offers that as a next_action; it does not imply it.
  • Explaining a comparison. frob explain compare <comparison> clusters the paired-episode differences by contact signature and scene region and names the two or three regimes in which the candidate gained or regressed, within a token budget. A paired difference is attributed to a regime only if both arms' traces for that world address are present; the address-keyed sample of §29.6 guarantees that the same worlds were traced for both arms.
  • Gait by symmetry mode. Joint trajectories are projected onto the isotypic components of the observation-preserving subgroup (A-03) and the energy per component is reported. One line tells the agent which gait emerged and how far a payload or a sensor broke the symmetry. It is a readability aid and makes no claim about performance.
  • The first physical consequence of an edit. When a run follows an edit, explain leads with the change the agent made, the first event at which the two lineages' traces part (the first-divergence algorithm of §29.8), and the evidence for the comparison.

28.6 Comparing designs honestly (C-15)

frob eval compare A B --task T pairs episodes by world address, so both designs meet the same scenes, the same disturbances and the same randomization draws. A comparison declares its study type before it draws an episode.

Study type anytime. A betting e-process runs per metric. The comparison stops when a threshold is crossed or the budget ends, cancels the unfinished episodes, and returns a verdict with the e-value, the confidence sequence for the effect, the number of pairs used, and the tier. Episodes are consumed in address order, never in completion order, so preemption and reordering by the daemon leave the e-value unchanged. A metric enters an e-process only with a bound on its paired difference that was fixed before the first observation (FACT: frankensim/crates/fs-eproc requires an explicit LossSpan, which "must not be estimated from the race data"); the span comes from the task definition (§24), and a metric without one is refused for this study type.

Study type fixed-n. The rule is frozen before sampling: n pairs, the counts b (candidate succeeds, baseline fails) and a (the reverse), the effect (b − a)/n, and exact Clopper–Pearson intervals on the two discordant probabilities at 97.5% two-sided each, with opposite endpoints subtracted, which gives at least 95% simultaneous coverage. The candidate is promoted if the lower endpoint exceeds the margin and hard feasibility holds; the verdict is a practical tie if the interval lies inside ±margin, inferior if the upper endpoint is below −margin, and otherwise inconclusive. Two rules attach.

  • Reachability pre-analysis. eval.preanalyze enumerates every outcome the frozen rule can produce at the declared n and margin and reports which verdicts are reachable. A study whose declared verdict set contains an unreachable verdict is refused unless the request acknowledges it. Known answer: at n = 64 with a five-point margin there are 2,145 count pairs, of which 521 promote, 521 are inferior, 1,103 are inconclusive and none is a practical tie, because perfect concordance gives [−0.0662, +0.0662]; promotion with a = 0 needs b ≥ 15; a tie first becomes reachable at n = 86, and only for (0, 0) among 3,828 pairs (FACT, executed enumeration; the endpoint 1 − 0.0125^(1/64) = 0.0662 and the counts C(66, 2) and C(88, 2) are reproduced by hand).
  • Exact curtailment. A fixed-n study may stop early only when every admissible completion of the unresolved labels yields the same verdict. No e-process may stop it.

Pairing rules. These are requirements on every paired study; violating one is a refusal, not a warning.

  1. Pair by (world, replicate). A counter that depends on training progress is not part of an address. Counterexample: seeds keyed by (world, epoch) on a 64-wide batch pair only morphologies whose indices agree modulo 64.
  2. Exogenous fields are indexed by physical coordinate, not by event order. Two robots on different paths must read one space-indexed material field at their own contact points; two tires on different paths need not receive equal friction samples at corresponding ticks, and a comparison that pairs "the k-th friction draw" has paired nothing.
  3. Each design is settled separately. The settled equilibrium is recomputed per design; a design that starts in penetration makes the pair unscored; a buildable design that stalls is a failed trial, not a missing one.
  4. Pairing imposes no minimum number of episodes per design.
  5. The accounting identity holds: a_only + b_only + both + neither + unscored = planned, each an integer, and the verdict states its denominator. An agent cannot optimize failures out of the denominator.
  6. Paired statistics are refused under an uncontrolled determinism class (§35). Both arms run at D2 or better on one backend, or at D3 with decision-level equivalence recorded for the task. A D4 run never enters a comparison.

Several metrics. The average of the per-metric e-values is an e-value under arbitrary dependence and tests the global null that the candidate is no better on any listed metric (FACT: fs-eproc::combine_average; the product is valid only for independent comparisons, combine_product). A claim about one named metric uses that metric's own e-process, with the e-value form of the Benjamini–Hochberg rule across metrics when several are claimed (FACT: fs-eproc::e_benjamini_hochberg, false discovery rate at most α under arbitrary dependence).

Several designs. A race over N candidates uses the mixture construction of fs-race::race_field: rounds are the only clock, every survivor consumes one observation per round in index order, a candidate's elimination evidence is the average of the pairwise e-processes against all original opponents, and an elimination needs that mixture to reach at least 1/α, so the true best is eliminated with probability at most α (FACT: frankensim/crates/fs-race/CONTRACT.md). An implementation that eliminates on any single pairwise crossing runs each comparison at α/(N − 1) instead.

What a comparison returns. The verdict (settled, tied, inconclusive, incomparable or infeasible, all legitimate dispositions); the statistic of its study type; the pairs used with the accounting identity; the tier; the first divergent event between the arms when one exists; each design's absolute hard-limit verdict beside the paired verdict, because a relative result does not establish that either design meets the mission; and a rank-robustness field (model-robust, model-conditional or unchecked, from A-19). A verdict from a fast tier is provisional until promoted (the trust ladder of §16, decision core DC-01); a promoted verdict also carries the estimate at referee fidelity from the tier-correction ledger (§29.11).

What a comparison does not control. One comparison is valid under optional stopping. It does nothing about an agent that submits fifty candidates to the same held-out worlds until one passes. That is the job of the blinded qualification lane of §24, with its fresh sealed world clusters and its error budget α_i = α / [i·(i+1)] for the i-th submission. A design comparison that is to be reported as a claim (§8) goes through that lane; eval.compare on discovery worlds is a development result.

A relation, not two scores F. "Can this replacement make a formerly successful delivery fail?" is a question about the pair, and the statistical comparison answers it only in distribution. The exact form, reachability of the outcome pair (success, failure) in the product of the two closed loops driven by the same exogenous inputs, is a mission query (§25). Atelier's part is the pairing discipline above, without which neither form means anything.

28.7 Lineage: diff, blame, bisect, merge

Design versions form a content-addressed graph whose edges are typed edits. design diff lowers the difference of two terms to a list of edits. design blame names the edit that introduced a parameter value. design bisect --predicate "<task> succeeds" walks a lineage path by binary search, deciding each step with the anytime-valid comparison, and returns the edit that changed the outcome (NS-06).

  • An edit returns its invalidation set. design.edit takes the expected parent and an idempotency key, refuses a stale parent, and returns the new immutable version, the structural and parameter diff, the symmetry effect, and the set of derived assets and stored verdicts whose dependency support the edit touches (§28.13). The agent sees what its change put in question before it spends anything.
  • Bisect is honest about non-monotone lineages. Binary search is correct only for a predicate that is monotone along the path. Counterexample: outcomes along six versions are pass, fail, pass, pass, pass, fail. The midpoint passes, the search goes right and reports the last edit, while the first edit also changed the outcome and the second changed it back. design bisect therefore returns a pair of adjacent versions with different decided outcomes, which is always a true statement, labelled a-change. It labels the result first-change only after the prefix has been scanned (every earlier version decided) or when the caller declares the predicate monotone, which is recorded as an assumption on the result.
  • Bisect controls its own error. Each probe is an e-process at α / ⌈log₂ L⌉ for a path of L edits, so the whole search errs with probability at most α by the union bound. An inconclusive probe ends the search with the bracket reached and the evidence so far.
  • Merge is order-independent. Lineages and archives from several agents merge as grow-only sets of versions and per-cell maxima under a total order (fitness, then DesignId). The merge is commutative, associative and idempotent, so the result does not depend on who synchronized first. FACT: fs-archive::MapElites::add replaces an elite only on strict improvement, so equal fitness keeps the first arrival; the total order removes that dependence on arrival.
  • Migration keeps old replay possible. A schema migration reads the old object and writes a new one with migration provenance. The old version stays addressable and replayable with the worker version that produced it.

28.8 Optimization

Agents are the discrete optimizer: they choose topology, leg count and actuator families. The studio supplies fast continuous optimization and honest evaluation underneath.

  • Continuous parameters. CMA-ES with restarts, Bayesian optimization with trust regions and multiple fidelities, and Riemannian quasi-Newton for static objectives on manifold-valued parameters, all from FrankenSim (FACT: fs-dfo with restart schedules, fs-bo with turbo and mf modules, fs-ascent::riemann), reached through the airlock.
  • Racing. Candidates in a generation are compared by e-processes and the dominated ones are cancelled mid-evaluation, so compute follows the survivors. The benefit depends on separation: the sibling's own battery asserts a saving of at least 2 on a separated field and below 1.5 on an inseparable one, with 11.7 and 1.03 measured on its fixtures (FACT: frankensim/crates/fs-race/tests/battery.rs, CONTRACT.md). The plan therefore expects two to five times fewer evaluations when candidates are separated and nothing otherwise (HYPOTHESIS; settled per campaign by the saving the race ledger records).
  • Quality-diversity. A MAP-Elites archive (fs-archive, FACT) over behaviour descriptors such as mass, speed, energy per metre and footprint. frob opt archive show gives an agent a map of different good designs instead of one optimum. Mutations are typed edits, including adding and removing a limb, and one generation is one mixed batch (§17) under the class's family policy (§23).
  • The fidelity ladder. Static analyses, then a shared family policy on a fast tier, then a fine-tuned policy, then a specialist, then promotion on the referee, then a rare-failure estimate for finalists. Each rung certifies or escalates. §28.9 specifies it.
  • Co-design with control. The inner loop is a controller that needs no per-design training: the synthesized baseline controller, which costs seconds, or the family policy of the design's morphology class. Policy fine-tuning is spent only on survivors and specialist training only on finalists.
  • A family per dispatch F. A neighbourhood of designs (a sweep, a finite-difference stencil with common random numbers, an evolutionary generation) is evaluated in one mixed batch, which cuts sequential agent rounds. A 20-parameter central stencil with 64 addresses per design is 41 × 64 = 2,624 worlds and fits one device. The ceiling is stated before any test: published population co-design reports compute gains of about three to four times, not ten (FACT, source register), and parameter sweeps inside one topology are something incumbents batch today. Paired finite-difference design gradients are used only over short horizons and smooth metrics, because paired worlds decouple under contact (a perturbation of 5 × 10⁻⁴ degrees is reported to diverge after 50 steps; FACT, source register). The pilot, its falsifier and the kill gate KILL-FDGRAD are specified in §17.13, which owns them: on one quadruped with 12 design parameters, the variance ratio of paired against independent central differences at horizons of 25 to 500 steps and the sign agreement over 20 replicate batches. The gate fires on a variance ratio below 2 at every horizon of 50 steps or more, on sign agreement below 70%, or when no end point is confirmed by a specialist; its fallback is rank-based search (CMA-ES, racing) with the stencil kept as a sensitivity report.
  • One verb for the whole campaign. frob opt cascade --spec <mission or task> --budget … runs the cascade of §28.11 from a specification to a ranked, typed result set, streaming progress with denominators. It is orchestration over the verbs of this chapter and is judged as such. The agent-facing orchestration verb is solve.run (§32.6.2), which calls opt.cascade and adds no mechanism. There is one A/B, not two: the one-verb arm of GATE-ATEL-03 is the second arm of GATE-TETHER-13, and KILL-SOLVE is decided on it. If agents with the one-verb form do no better than agents driving the verbs by hand in that A/B, it stays a convenience and is not counted as a lever.

28.9 The ladder of evaluation rungs

A design is never scored by the most expensive method first. Six rungs, each about an order of magnitude dearer than the one below, each allowed to decide only what its evidence supports.

Rung What scores the design Cost per design What it may decide
R0 Static design.check and the bound analyses Seconds of CPU; under 2 s for check (TARGET) Refuse a design; exclude with a typed bound. It never ranks survivors
R1 Shared policy The family policy of the design's morphology class, zero-shot, on a fast tier (§23); the synthesized baseline controller where the class has no family policy (§22) Tens of GPU-seconds (HYPOTHESIS; the phase ledger records it) Discard, under §28.10. Select only in strata where rank agreement has been measured
R2 Fine-tuned The family policy fine-tuned to the design at a declared fraction of specialist steps. Each fraction is its own rung for calibration Minutes Discard, and select among survivors
R3 Specialist A policy trained or inherited for this design alone One training run. The leading incumbent trains a quadruped velocity policy in about 410 s on one RTX 4090 and humanoids in under 30 minutes (FACT, source register) Rank finalists, as provisional
R4 Referee Paired re-evaluation on the referee at the same world addresses (DC-01), with the tier-correction estimate (§29.11) Referee episodes: 16 / 64 / 256 at the 50th / 95th / 99th percentile per established verdict (TARGET, unmeasured) Establish, or label tier-sensitive
R5 Rare failure A failure-rate bound for finalists (DC-04, §26) n = ⌈ln(1/δ)/p*⌉ nominal episodes for a bound p* at confidence 1 − δ Certify, refute or exhaust the bound, scoped to its tier

A sampling-based controller that needs no training is admitted as an R1 scorer and as an out-of-distribution fallback. It never stands in for a trained policy as the evaluator of record: at about 0.33 and 1 GPU-minutes per 20 s episode (quadruped, humanoid) at a 20 ms planning step, and about 1.7 and 5.2 at 4 ms (§22.8), a few hundred episodes cost more than training once (FACT, source register). Its own episodes are acceptance evidence only where it is the delivered controller, under a frozen planner record (§22.22). A policy may be trained under the labelled D4 training profile of §35; every score that enters the ladder is an evaluation on a D2 profile.

What the evidence says about the middle rungs (FACT, source register; every figure is from published work on other engines and is re-read before a gate cites it). Within one morphology class a family policy reaches 70 to 85% of specialist return zero-shot, and fine-tuning matches it after about 2 to 25% of a from-scratch run stopped at matched return. Across topology-diverse designs at affordable pretraining the zero-shot figure is 10 to 40% and the fine-tuning gain two to three times. Inheritance from a class prior costs about 2 to 25% of such a run; the one published parent-to-child figure is about 50%; no source measures a small typed parameter edit; after a single-limb topology edit the cost is 33 to 100%. Cheap rungs are biased as well as noisy: early evaluation underestimates slow learners, and a population scored under one shared controller drifts toward designs that controller steers easily.

Layer 1 of the allocation rule: the fixed table S. Successive halving with η = 3 inside each stratum: at each rung transition the top ⌈n/3⌉ designs by the lower rung's score advance, on a schedule written to the ledger before the campaign starts (FACT: fs-race::successive_halving returns a BracketLedger; its kills are rank-based and carry no e-process guarantee). The table is the baseline every adaptive rule is measured against, on the same population and the same addresses, in GPU-seconds to the same accepted decision and in audited top-k recall. It is also the fallback of every trigger in this chapter.

Layer 2: conformal survivor sets with an audit lane F. This layer owns the population stage. For each rung pair and stratum:

  1. The audit lane. A tenth of the rung's spend (DESIGN default) goes to designs drawn by a keyed random stream before any score exists. Each is evaluated at the higher rung at a fixed n and is never cancelled by an e-process. The audit lane is the only source of calibration. Calibrating on the designs that advanced is biased toward agreement and is forbidden.
  2. Calibration. With n audit designs, the residuals are rᵢ = |y_hi,i − ĝ(y_lo,i)|, where ĝ is a monotone map fitted on a disjoint part of the audit set. The margin q̂ is the ⌈(n + 1)(1 − α)⌉-th smallest residual. At α = 0.05 this needs n ≥ 19; with fewer the stratum falls back to layer 1.
  3. The survivor set. Every design whose lower-rung score is within 2·q̂ of the k-th best lower-rung score advances, where k is the number of finalists wanted. A rung cannot separate designs that are closer than twice its own error, so everything inside that margin goes up.
  4. The trust check. If at most 3k designs survive, the rung separates the top band and is trusted there. If more survive, the request buys more lower-rung episodes, or skips the rung for the top band, or, when the budget forbids both, truncates to 3k by lower-rung score and labels the campaign budget-truncated.
  5. The alarm. A test martingale on the audit designs' conformal p-values watches exchangeability. At a value of 20, which is 1/α at α = 0.05, the stratum's calibration is void, the stratum reverts to layer 1, and calibration restarts inside the new regime.

Obligations: coverage of at least 1 − α for designs exchangeable with the audit set, on average over calibration draws (with n = 39 the coverage averages 0.95 and falls below 0.885 in one calibration draw in twenty; arithmetic from the Beta law of split-conformal coverage); two designs separated by more than the margin at the lower rung are misordered at the higher rung with probability at most 2α; top-k recall is measured by the audit lane and never claimed from the theory.

Layer 3: value of information at the finalist stage F. When the survivors of the last population rung number at most 64 (DESIGN, the budgeted mode of DC-05), allocation passes, one way, to decision core DC-05: a table of experiments (more fast-tier pairs, referee pairs, a fine-tune, a bench measurement), each with a cost and a closed-form expected value of sample information, bought in order of value per cost until none clears the budget price. The handover is one-way: no design receives requests from both layers. Layer 3 orders work; every verdict still comes from DC-01 and the comparison rules of §28.6.

The numbers the evidence supports.

Quantity Figure Status
Saving from anytime-valid stopping and elimination Two to five times when candidates are separated; about none when they are not FACT for the sibling fixtures (fs-race battery: at least 2 asserted, 11.7 measured, on a separated field; 1.03 on an inseparable one); HYPOTHESIS for robot design populations
Information-gain selection against a capable agent Unsupported; plausibly below 1.5 times. A language-model tuner beat random search more often than Gaussian-process optimization at 10 and at 30 evaluations, and model-based search lost to a genetic algorithm on a robot co-design benchmark FACT, source register. Layer 3 is therefore a stop rule and a budget allocator
Referee cost saved by pairing with the fast tier Variance scales by 1 − ρ²: 1.96 times at ρ = 0.7, 5.3 at 0.9, 10 at 0.95. Published cross-fidelity correlation collapses at large dynamics gaps Arithmetic; ρ unmeasured. Expect about two times until ρ above 0.9 is measured
Rank agreement of R1 with specialists on the referee Unmeasured anywhere. Nearest neighbours: rank correlation 0.40 overall FACT, source register; §28.10
Share of a population that must leave R1 for R2 About the top quarter, not the top twentieth HYPOTHESIS inferred from the neighbours; settled by GATE-ATEL-07
Compute gain of published shared-policy population methods About three to four times FACT, source register

The ceiling, stated before any speed test. The ladder cannot cost less than one R1 score for every design, the audit lane, and R3 to R5 for the finalists. A worked case, illustrative and not a measurement: 81 designs with rungs of 10 s, 3 min and 30 min cost 40.5 h if every design is trained; halving at η = 3 costs 13.5 min + 81 min + 270 min ≈ 6.1 h; with q̂ = 0.04 on a unit-scaled score, a third-best lower-rung score of 0.74 puts the cut at 0.66, six designs survive, and with three finalists the ladder costs about 2.0 h (13.5 min + 18 min + 90 min), or about 2.2 h with a tenth added for the audit lane. With q̂ = 0.10 about twenty designs survive, more than 3k = 9, and layer 2 gives back most of its advantage. The size of q̂ is exactly what the rank-agreement ledger measures, which is why that experiment comes first.

Pilot and falsifier for layers 2 and 3. Log once, on an incumbent engine or on batch-cpu: 64 designs × 3 rungs × 32 addresses (about one GPU-day, TARGET). Replay four choosers over the log with no further simulation: an agent reading the log so far, the knowledge-gradient chooser, e-race elimination, and uniform allocation. Metric: evaluations until a true top-3 design is identified at 95% confidence, over 100 replays. KILL-LADDER fires if layer 2 does not reach halving's audited top-k recall at fewer GPU-seconds; KILL-VOI fires if the knowledge-gradient chooser is less than 1.5 times better than the agent. Fallbacks: layer 1 alone; value of information as a stop rule only.

28.10 The rank-agreement ledger

The ladder is only as good as the agreement between adjacent rungs, and that agreement has not been published for the case that matters. The ledger measures it and gives the measurement consequences.

rank_agreement row
  key        morphology class · task family · rung pair (lo, hi) · stratum (block of archive cells)
             policy and kernel versions · effort profile · battery version
  measures   n_designs
             Kendall τ on the top band, with a bootstrap interval    top band = top m of either rung,
             top-k recall of lo against hi                           m = max(10, 3k)
             Spearman ρ over the stratum
             mean signed residual per region (the bias of the cheap rung)
             conformal margin q̂ at α · exchangeability martingale value
             per-episode cross-tier correlation ρ_ep on paired addresses (for R3 → R4)
  provenance audit-lane draws · calibration experiment · claim_id · recording session

Rows come from two sources: the audit lane of every campaign, continuously, and one calibration experiment per morphology class. That experiment is the decisive early experiment of the program. Forty designs in one class are each scored by the shared policy on the fast tier, by a fine-tune at 5% of specialist steps, and by a specialist promoted on the referee; it reports Kendall τ on the top band, top-10 recall and bootstrap intervals, and ρ_ep on 256 paired episodes per design. It costs about 45 GPU-hours, or about 5 if the specialists of the family-policy experiment of §23 are reused (TARGET).

What published neighbours report (FACT, source register): the value predictions of a shared policy correlate with rollouts of the same policy at a rank correlation of 0.40 overall and 0.50, 0.60 and 0.83 on three robots, over 17 designs, which is an easier comparison than a shared policy against specialists; in architecture search with shared weights, sparse Kendall τ of 0.236, 0.740 and 0.159 on three benchmarks; a universal controller reaches up to 80% of individually trained controllers; and in the 17-design study the top design was the same under both scores. Agreement of 0.7 or better near the top is therefore not supported, top-1 agreement at modest correlation is plausible, and the default, until a stratum is measured, is that R1 neither selects nor discards.

Each rung pair in each stratum is in one of three modes. The thresholds 0.5 and 0.7 are the kill thresholds of the calibration experiment; they are the thresholds of §16.9, §22.18 and §23.11, fixed before data. All conditions are on the lower bootstrap bound.

Mode Condition What the lower rung may do
selective τ on the top band ≥ 0.7, with the winner check of §23.28 passed; calibration valid (n ≥ 19, martingale below 20); audited miss rate at most α The conformal survivor rule of §28.9
discard-only 0.5 ≤ τ < 0.7; or a fallback trigger of DC-02 or DC-03 fired The fixed table of layer 1: the top third advances, so the top quarter always does. Where a margin exists, a design within 2·q̂ of the k-th best is never eliminated. The audit lane keeps running
uninformative τ < 0.5; or the stratum is unmeasured; or the planted winner was lost in this stratum Nothing. The rung is skipped in this stratum, its budget moves up one rung, and designs it eliminated earlier in the campaign are re-opened through the audit lane

An unmeasured stratum is uninformative; the default fails closed, and a rung without a rank-agreement record is refused (FROB-REF-537). The synthesized baseline controller is a scorer like any other: when it serves as R1 its rank agreement with trained policies is measured in the same ledger and it has no exemption. Low agreement has four further consequences.

  • The report says so. Every campaign report prints the mode of every rung and stratum. Finalists chosen under any mode other than selective are described as "k survivors", never as "the top k".
  • Bias is handled by region. A region where the cheap rung systematically under-scores (bodies that are hard to steer with a shared controller) gets its own margin by per-bucket calibration; a bucket below the calibration floor refuses instead of extrapolating (FACT: fs-eproc::hardening::MondrianConformal implements this refusal).
  • The tier correction switches off. Where ρ_ep is below 0.7 on at least 200 paired addresses, the estimator of §29.11 is disabled for that stratum and promotion uses referee episodes alone.
  • The program is told. If GATE-ATEL-07 fails for every morphology class tested, KILL-RANK fires: R1 is uninformative everywhere, every candidate that survives static analysis is scored on the higher rung, the plan's benefit claim for the ladder shrinks to stopping and elimination among trained candidates, and §39 records which factor of the ten-fold thesis was lost.

28.11 The six-stage co-design cascade

The cascade is the ladder applied to a population, with a control built into every stage. opt.cascade runs it; each stage is also callable alone.

Stage Content Rungs Output Control built into the stage Owner
1 Deduplicate and check One canonical representative per orbit under the observation-preserving subgroup; the mission-observable digest; design.check with the bound analyses R0 Representatives; typed exclusions Two designs that must merge and two that must split §17, frob-analyze
2 Reuse and synthesize Memo hits for analyses, controllers, step plans and episodes; module contracts reused where the edit is local; bounded exact linkage synthesis when the brief asks for a mechanism R0 Reuse receipts; candidate set A mirror swap must reuse; a shared-bus edit must invalidate frob-opt, §14, §25
3 Screen Gait-orbit screening (A-04), family policy zero-shot, task-wrench attainability (A-24) R1 Survivor set with the mode of each stratum The blinded stratified audit of discards and the planted winner (§28.12) frob-opt, §22, §23
4 Race A mixed-morphology race on paired addresses; fine-tuning for survivors, specialists for finalists R1 to R3 Finalists, provisional A null race keeps the best with probability at least 1 − α frob-opt, §17
5 Rank robustness A-19: the top-k and the runners-up re-evaluated under model-form variants, with the created-energy audit R3, R4 Each finalist typed model-robust or model-conditional Planted cone and energy-pumping exploits must be caught frob-analyze, §16, §18
6 Qualify Sealed confirmation on fresh world clusters with pre-committed requests; promotion on the referee; the rare-failure bound R4, R5 A qualified design comparison with receipts False-promotion rate at most α under adaptive retries on no-signal worlds §24, §26

Stage 1 is cheap and its factor is modest. Ten leg types on six slots with dihedral symmetry give (10⁶ + 2·10 + 2·10² + 10³ + 3·10⁴ + 3·10³)/12 = 86,185 representatives, an 11.6-fold reduction; with only a sagittal mirror left, because a sensor broke the rest, (10⁶ + 10³)/2 = 500,500, a 2.0-fold reduction (exact counts). The stage is worth having and is not a large factor once sensors break the symmetry.

Stage 5 variants. (a) The alternative contact law, where the hard-contact batch law is available (§18); (b) the circular against the pyramidal friction cone; (c) contact stiffness at one half and at twice its value; (d) half the step; (e) one tenth of the solver tolerance. The stage costs at most a tenth of campaign compute (TARGET). A finalist is model-robust {variants, top-k overlap, energy audit} when the top-k set and the order of the top two survive every variant and no finalist's advantage coincides with created energy or cone excess (§29.9). Otherwise it is model-conditional, the report shows both orders as a fidelity-dependent ranking, and two things are blocked: the result cannot be reported as a design comparison without its condition, and it cannot spend prototype budget, so a build-packet request (§31) is refused. A design is preferred only if it wins under both qualified contact laws.

Every stage may end the campaign honestly. "Every admitted alternative fails", "tied within the margin" and "inconclusive at this budget" are results. A stage that cannot run its control (no planted winner available, no second contact law admitted) reports the control as not-run, and the result carries that.

28.12 Screening audits

A screen can throw away the winner, and nothing downstream will notice: an e-value between the eighth and ninth survivors says nothing about a design discarded at the first rung. Three audits make the loss observable. The first two are the screening audit that §26.10 owns (verb nemesis.audit, gate GATE-NEMESIS-07); this section applies it to the stages of the cascade and defines no second mechanism, verb or bound. The third is Atelier's own.

  1. The stratified audit of discards. The audit lane of §28.9, stratified by stage, reason for discard, morphology class and archive cell, at rates fixed in the campaign manifest. It is blinded: an audit design and a survivor are the same job type with the same budget, and the evaluating job cannot tell which it is running. Results feed the rank-agreement ledger. With no true top-k design among m audited discards, the rate of discarded winners is at most 1 − δ^(1/m) at confidence 1 − δ, the exact bound of §26.10 (9.5% at m = 30 and δ = 0.05).
  2. The planted winner. Before the campaign a design known from an earlier specialist-and-referee result to belong in the top k is inserted in a form the screen represents poorly: an unusual gait, or a morphology class with few examples in the family policy's training set. Its identity is sealed in the campaign manifest hash and revealed at close. It must reach the finalists. If it does not, the results are labelled screen-biased (the label of §26.10), the offending rung becomes uninformative in that stratum and the campaign re-opens. A stage whose audit is missing is treated the same way: Atelier does not trust a rung without a passed audit (GATE-ATEL-09).
  3. Prune soundness. An exclusion made by an analysis or a contract is wrong if any discarded design is independently feasible under the same mission, even when the eventual winner survives. Pruned designs are sampled and run at R1 or above. One feasible pruned design suspends that pruning rule for the campaign and files the case as a negative-control seed. The shared-bus case of §28.4 is the standing example.

28.13 Revalidation after edits (result.revalidate)

An edit is an event that puts evidence in question. The studio does not ask "is the cache still warm"; it asks which verdicts survive, which are revoked, and what the cheapest set of reruns is. frob result revalidate <verdict or claim> --after-edit <edit> answers with the surviving obligations and the licence for each, the invalidated regions, and the required reruns with their cost. The mission cluster's revalidation verb (§25) is this engine applied to a mission answer.

Every stored verdict has this shape (frob.verdict/1):

verdict record
  claim        claim_id · claim type (§8) · statement · quantifier prefix
  kind         inclusion (a witness exists) | exclusion (no witness can exist) | comparison | optimum | estimate
  assumptions  disturbance set · world set · hypothesis support · actuator limits · requirement ·
               policy class · information contract (channels and their instants) · catalog and prices ·
               model identity · tier · kernel versions · effort profile
  obligations  feasibility obligations   (the witness is valid)
               optimality obligations    (every cheaper rival is excluded; the cost bound; catalog membership)
               each with: evidence_id · dependency support (the exact fields it read) · checker
  provenance   evidence closure · decision-policy id · trace ids (§29.13)

Four grades of reuse.

Grade Licence to skip the recheck What is recorded Tier
1 Identity The obligation's full key is unchanged The memo hit (§28.14) S
2 Dependency Every changed field lies outside the obligation's recorded dependency support A mask over the obligations, checkable against the supports without rerunning anything S
3 Implication A changed field lies inside the support, and a polarity rule of the assumption lattice preserves the obligation The rule's name, the old and new assumption sets, the order relation between them and its check F
4 None — The rerun and its cost estimate —

The assumption lattice with polarity. Each component of the assumption set carries a partial order: inclusion for sets, pointwise order for limits, refinement for information. The table gives, for each direction of change, whether an inclusion result and an exclusion result survive.

Component Change Inclusion (a witness exists) Exclusion (no witness can exist)
Disturbance or world set Narrowed Preserved, with the same witness Recheck
Enlarged Recheck Preserved, if the policy class and the information contract are unchanged
Hypothesis support (admitted hidden parameters) Enlarged Recheck Preserved
Actuator limits Relaxed Preserved, if the old command stays realizable and the plant still realizes the old dynamics Recheck
Tightened Recheck Preserved
Requirement Relaxed Preserved Recheck
Tightened Recheck Preserved
Policy class Enlarged Preserved Recheck
Shrunk Preserved only if the witness policy is still in the class Preserved
Information contract Refined: more, earlier or better observations, the old ones still recoverable Preserved Recheck
Coarsened or delayed Recheck Preserved
Catalog and prices Part added or price lowered Preserved Recheck. This is where an optimum is lost: every cheaper rival must be excluded again
Part removed Preserved unless the witness uses the part Preserved; an optimum that used the part is void
Free space of the scene Enlarged Preserved, if the witness trace does not touch the changed region Recheck every exclusion that used the removed obstacle

Five rules govern the table.

  1. A rule applies only to a pure change in its direction. A larger motor relaxes a torque limit and also changes mass and rotor inertia. It is not a relaxation; the model diff sends it to grade 4. A room segment may guarantee a wheel-traction margin under a normal-load interval, and a changed motor can leave that interval through acceleration.
  2. Feasibility and optimality are separate obligations. A cost-9 cart may be the certified minimum while a cost-5 relay is excluded by a doorway. Widen the doorway: the cart and its trace remain valid, so its feasibility obligations survive; the obligation "every cheaper rival is excluded" read the doorway and is revoked. The verdict splits into feasible: preserved and optimal: revoked, recheck. A cheaper, newly feasible rival always revokes an incumbent's optimality claim, including when an amendment adds a catalog part.
  3. Joint edits need a joint check. Polarity rules compose only when the edited components have disjoint dependency supports for the obligation. A simultaneous edit to observation timing and to the floor law is checked jointly.
  4. Improving something can destroy a result. A better sensor preserves witnesses and can void an impossibility that rested on two worlds being indistinguishable.
  5. Unsafe reuse is a false accept, even if a later change restores feasibility. One false reused verdict disables the rule that licensed it until it is repaired, and a checker rejects an unknown rule name or a changed quantifier prefix even when every hash is consistent.

Counterexample: widening by a hair. In the docking fixture of §25 the gain is 1 or 2 and one velocity reading decides the action. Add the gain 33/32 to the hypothesis support: it is indistinguishable from gain 1 at an error radius of 1/64, and every one of the 34,590 positive decisions on each test grid is wrong for the amended mission (FACT, executed; source register). "Hypothesis support enlarged: recheck inclusions" is not a formality.

The negative result, kept on purpose. Fixture: two scenes whose decisions commit at ticks 2 and 4; a camera packet arrives at tick 1. Four states follow: the initial one; a display label is renamed; delivery is delayed to tick 2; delivery is delayed to tick 3. A semantic receipt and a conventional dependency graph given the same eligibility condition both evaluated 2, 0, 0, 1 scene pairs and recomputed the verdict 1, 0, 0, 1 times, and every verdict matched a fresh replay. An implementation whose key omitted delivery time kept a stale promotion at the fourth state (FACT, executed; source register). Three consequences are binding. On small cases a correctly keyed dependency graph ties a semantic receipt, so grade 3 has no demonstrated advantage. What decides safety is the completeness of the key: delivery time is a dependency of a decision. Grade 2 is therefore the default and is Solid; grade 3 is Frontier, behind a flag, and its open value is whether a useful non-dependence certificate can be obtained cheaply for a rich plant.

The ceiling, stated before any speed test. Rollouts and training are at least nine tenths of per-design compute and depend on every mass and geometry parameter, so exact reuse after a dynamics-affecting edit is at most about a tenth of compute (FACT for the phase shares of published loops, source register; HYPOTHESIS for this system until the phase ledger exists). Reuse is total for model analyses after a controller-only, reward-only or scene-only edit. Revalidation is an evidence rule and a hygiene measure. It is not a source of the ten-fold. Training avoided by a warm start is not reuse in this sense; it is reported in its own column of the reuse ledger (§28.26) and credited to inheritance once.

Pilot with a hard stop for grade 3. A campaign of 30 amendments is frozen before any implementation sees it: visual-only edits, camera timing, sole geometry, floor law, torque, shared noise, load authority, facility edits, and one apparently cosmetic change that advances a packet by one control tick. A separate session implements the incremental dependency graph. Fresh solving, grades 1 and 2, and grade 3 are run from the same cache state and dirty fraction; every decision is replayed; every incorrectly inherited success and every unnecessary invalidation is recorded. KILL-REUSE fires on any unsafe reuse, or if grade 3 saves less than a fifth of the recheck cost that grade 2 leaves (TARGET). Fallback: grades 1 and 2 only.

28.14 The memo graph

Memoization does not stop at the compiled model. One content-addressed graph covers everything downstream of it. The Memo trait is declared in frob-core, wraps each verb's executor in frob-verbs, and stores in frob-chronicle.

Node kind Key A hit is served within Note
Subterm compile Subterm content hash, compiler version, card versions D0 The incremental-compilation invariant of §14
Analysis The model identity restricted to the fields the analysis reads; analysis version; parameters D0 An edit outside those fields recomputes nothing
Controller synthesis Model identity, task interface, synthesis version D0 or same-backend D2
Step plan Shape class: topology bucket, capacities, contact law, sensor layout — A parameter edit recompiles nothing; parameters are per-world data (§17, §21)
Kernel binary Kernel source hash, compiler, driver, device family — The binary cache of §21
Scene compile Scene identity, compiler version D0 §19
Episode The full identity: model, scene, task, controller or policy, world address, tier, kernel versions, effort profile, device family Same-backend D2; D1 where the device family holds it Never across a D3 boundary
Comparison, summary The identities of its inputs and the rule version Derived Recomputed from memoized episodes; it reaches the identical e-value

Store policy.

  • Keys are derived, not written. A key is generated from the declared inputs of the node kind. Each kind has a property test that perturbs every declared input and requires the key to change, and a live-dependency test that perturbs every field the computation reads and requires the key to change or the output to stay the same. A missing input in a key is the failure this policy exists to prevent (§28.13, the stale promotion).
  • Hits are bit-identical or not served. Results are placed by index and random streams are keyed by address, so a hit equals a recomputation bit for bit inside D0 and same-backend D2. Tolerance-level reuse (FACT: fs-recompute::Store::can_skip serves a node cached under a looser requirement if it achieved enough) is allowed only for analyses that record an achieved error, never for episodes.
  • Shadow recomputation. One hit in 64 (DESIGN), chosen by key hash, is recomputed and compared. One digest mismatch stops serving that node kind and files a determinism violation (the trip-wire pattern of fs-recompute, FACT: StoreError::DeterminismViolation).
  • Loss model. States: valid hit, stale hit, miss. Actions: serve, recompute, shadow. A stale result served costs 100 units against the GPU-seconds of a needless recomputation (DESIGN ratio, recorded in the memo policy's contract).
  • Budget. 64 GB by default (DESIGN). On exhaustion the store evicts and never blocks. Eviction is the S3-FIFO policy FrankenSQLite's pager uses (FACT: frankensqlite/crates/fsqlite-pager/src/s3_fifo.rs), with admission weighted by recompute cost because misses are not equally expensive. Nodes in the evidence closure of a live claim are pinned (§29.16).
  • Rollout and rollback. Shadow, then D0 analyses, then D2 episodes. --no-memo disables serving; the store can be deleted at any time without losing a result that a claim cites, because cited evidence is pinned in Chronicle, not in the cache.
  • Baseline. Full recomputation per verb with compile-only memoization.

Where the reuse is, and is not: the comparison that follows a 2,048-world run of the previous version on the same scene reuses that version's episodes at the comparison's addresses; a controller-only edit reuses every model analysis; a mass or geometry edit reuses no rollout. Targets, unmeasured: design check after a one-parameter edit in 0.2 / 0.5 / 1.0 s at the 50th / 95th / 99th percentile (TARGET).

FACT: frankensim/crates/fs-recompute is a content-addressed Merkle graph with a skip oracle, pins that are never evicted and an independent checker. It is consumed through the airlock if admission passes (§33); otherwise the same node record is implemented in frob-chronicle.

28.15 Campaign-level deduplication

Two designs that the mission cannot tell apart are evaluated once. The mechanism belongs to Phylum (§17): one canonical representative per orbit under the observation-preserving subgroup, and a mission-observable digest, which is the compiled model restricted to the quantities that enter the mission's equations (total inertia, collision hull, joint and actuator parameters, port coordinates, sensor visibility, thermal and electrical bus, task objective), with results memoized on it. Atelier's part is three rules.

  • The digest is a memo key like any other and obeys the store policy of §28.14: derived from declared inputs, shadow-recomputed, stopped on one mismatch.
  • "Equal digests imply identical trajectories" is a structural rule with a bit-identical replay test. It is a statement about the simulator and is not offered as a proof obligation to any proof checker.
  • The factor is reported, not promised. A fixture built to have a large factor (ballast slugs that enter the dynamics only through count and first and second moments reduce 735,471 placements to 24,546 classes) shows the mechanism is exact; it says nothing about the factor on a real brief. Placements with equal first moment and different second moment must split, and adding a placement-dependent quantity to the mission enlarges the digest and splits classes.

28.16 Experiment suggestion

frob experiment suggest <designs or answer> --budget … returns a ranked list of information-gathering proposals, each with a testable prediction: more pairs at a rung, a referee batch, a fine-tune, a probe on a bench, a measurement of a card field, a cell of a mission answer to split. It is advisory. It never decides validity (ADR-ATEL-07). The verb is registered once, by §25.3.1, which owns it; this section is a consumer and specifies the ranking by decision value. Two kill gates bear on it and do not overlap: KILL-AUGUR-PROBE (§25.27) governs the probe generator, and KILL-VOI (§28.23) governs the ranking.

  • Ranking by decision value. Each proposal is an action with a cost and a variance component it reduces (numerical, statistical or model). Its value is the reduction in expected opportunity loss of the current choice per unit cost, and the list ends with STOP when nothing clears the price (FACT: frankensim/crates/fs-voi provides action_value, recommend with a stop threshold, expected_opportunity_loss and a knowledge-gradient routine over a finite Gaussian menu; its own comparison test is a three-design fixture, so no saving is measured).
  • Partition by decision, not by parameter. For a measurement, the compatible physical mechanisms are partitioned by the design decision they imply; the cheapest realizable intervention whose possible readings separate incompatible decisions is proposed; where observation sets still overlap, two witness mechanisms with opposing design margins are returned; when all mechanisms imply the same choice, identification is skipped. A-25 checks that the proposed excitation can identify what it claims to.
  • Measuring the target directly is always on the list. Counterexample: a short probe synthesized for three admitted material laws chose the right gripper pad in 675 of 675 in-class cases; a fourth, delayed law followed the same early trace and reversed all 225 held-out decisions; measuring the candidate pad at mission dwell was right in 900 of 900 (FACT, executed; source register). An in-class-perfect probe cannot expose structural error, so every suggestion states its model class and lists the direct measurement as a comparator with its cost.
  • Every suggestion registers its prediction. The predicted outcome interval of the chosen experiment is registered (§28.17) before it runs. The calibration of those intervals is the fallback trigger of DC-05.

28.17 Registered predictions

Before a run, an agent may register a prediction: frob predict register --metric cot-elec --interval 0.42 0.50 --confidence 0.8 --for <planned-run>. It is hashed and frozen before the run starts and scored afterwards with a proper scoring rule. frob predict report shows each agent's calibration. The mechanism costs almost nothing and answers a question nothing else does: whether an agent understands the robot it is designing or is searching blindly (adapted from FrankenMining's forecast ledger, FACT).

The studio registers its own predictions under the same rule: the intervals of sim.estimate, the outcome intervals of experiment.suggest, and the saving a race or a ladder expects. A refit after scoring opens a new prediction version; the old one stays scored. Calibration is put to work in two places. An agent whose 80% intervals cover far from 80% of the time is told so in next_actions, with the suggestion to buy evidence before the next edit. And the studio's own miscalibration is a fallback trigger (DC-05: coverage of 80% intervals outside 65 to 92% over 30 experiments).

28.18 Reports

frob report design <id> writes an evidence package: the design and its lineage, every analysis, scorecards, comparisons with their verdicts, certificates, device receipts and replay addresses, as a directory with an HTML index (FrankenSim's evidence-package pattern, FACT: fs-package, with a standalone checker in fs-checker).

A report is generated from the artifact graph of §29.13 and prints nothing it cannot resolve. Each statement carries its claim type, its claim_id, the boundary of its claim product (the weakest input, §8), the mode of every ladder rung that selected it, its rank-robustness type, its qualification receipt if it has one, and the machine-readable no-authorization flags of §12. frob report campaign <id> adds the phase ledger, the screening audit, the planted-winner outcome and the list of fallbacks that fired. Reports are deterministic: the same graph produces the same bytes. A losing experiment is reported with the same completeness as a winning one, so that it can be reproduced as readily.

28.19 Decision cores

Eleven decisions recur across the studio, and each one moves compute or changes what is reported as established. Each is given one mathematical family, compiled ahead of time into a small runtime artifact, with a loss model, proof obligations, a deterministic fallback and a budgeted mode. Nothing in this section is measured: failure signatures are predicted, substituted numbers are illustrative unless marked TARGET, and every core ships in shadow behind its baseline and is re-examined against the first profile at M2.

28.19.1 The core contract

A core is a record registries/cores/<id>.toml, validated by the existing contract check (§34), with these fields: the decision, its owner crate and its timescale; the family and its compiled artifacts (frob.core-table/1, with a version and a content hash); the loss model as declared states, actions and cost constants; the proof obligations by identifier; the deterministic fallback and its triggers; the budgeted mode; the adoption status (shadow, canary, default); a replay scenario, a calibration report and a demonstration manifest (commands that run in under 60 s, with checksums, and the claim_ids they support). Every decision a core makes is written as an evidence record: the identities it read, the table version, the ledger frontier, the action taken, and, while in shadow, the action the baseline took. Each record carries the core's decision-policy identifier and version (§29.13), so any verdict can be traced to the exact rule that produced it.

Three conventions hold for every core below. The parameters of fallbacks, triggers and budgets (30 pairs, a quantum of 5 s, a 3% gain and the like) are DESIGN defaults recorded in the core's contract; changing one is a new version of the decision policy. For a Frontier core the fallback is also its kill fallback: if the core's calibration fails GATE-ATEL-13 after one repair cycle, the core stays on its fallback, and the registry says so. And a core's pilot is its shadow run: it acts on nothing until the comparison of its actions with the baseline's has been read.

The eight families sit in three planes of at most three: evidence (sequential tests, conformal prediction and risk control, value of information), failure (rare-event simulation, counterfactual forks, submodular coverage) and platform (queueing, certified rewrites). Successive halving is a fixed table, not a family.

Core Decision Family Owner Timescale Tier
DC-01 Promote a fast-tier result, reject it as tier-sensitive, or buy another pair Sequential evidence frob-opt (eval.promote) Per episode batch S
DC-02 Which designs advance at a rung transition Fixed halving table; conformal survivors; value of information for finalists frob-opt Per rung transition S table, F adaptive
DC-03 How wide the survivor margin is for a rung pair and stratum, and which mode the rung is in Split conformal prediction with an audit lane frob-opt Per generation F
DC-04 Certify, refute or stop on a failure-rate bound Rare-event simulation frob-nemesis (§26) Per finalist S bound, F splitting
DC-05 Which experiment to buy next, or stop Bayesian value of information frob-opt (experiment.suggest) Per experiment F
DC-06 Which cause, and which repair edit, ranks first after a failure Counterfactual forks with a placebo frob-autopsy (§27) Per failure cluster S
DC-07 Whether a simulated result may carry a reality band, and when the band is void Conformal band with an exchangeability alarm frob-ground (§30) Per real batch F
DC-08 Which solver effort profile a run uses Risk-controlled profiles frob-dyn (§16) At ledger refresh F
DC-09 Which job runs when on a device Queueing with size classes and preempt-resume frobd (§32) Per quantum S
DC-10 Which scenes form a battery Submodular coverage frob-arena (§24) Per battery revision S
DC-11 Whether a kernel rewrite is accepted Certified rewrites frob-kir, derive/frob-derive (§21) At build S exact class, F real class

28.19.2 The eleven cores

DC-01 · Promote, reject or keep sampling.

  • Artifacts. One bit per paired episode: did the fast tier and the referee disagree at this address beyond the fidelity ledger's frozen envelope for the quantity? Two likelihood-ratio e-processes on the exceedance rate: E⁺ tests the null "rate ≥ p₁" and promotes on crossing; E⁻ tests the null "rate ≤ p₀" and labels the result tier-sensitive on crossing. Both compile to one integer guard table (n, k) → promote, reject, continue, per ledger stratum, with the envelope hash and the address schedule.
  • Loss model. States: faithful, sensitive. Actions: promote, reject, buy a pair at cost c. Thresholds α⁺ = c/(π·C_fp) and α⁻ = c/((1 − π)·C_fn), with π the ledger's share of sensitive results and C_fp, C_fn declared in the contract (OPEN-ATEL-1).
  • Obligations. Wrong promotion at most α⁺ and wrong rejection at most α⁻ under optional stopping, by Ville's inequality, provided episodes are chosen by address before outcomes exist and the envelope is frozen first. Independence between episodes is not needed: under any conditional exceedance probability p ≥ p₁ each factor of E⁺ has expectation (1 − p)(1 − p₀)/(1 − p₁) + p·p₀/p₁ ≤ 1.
  • Fallback and trigger. Fixed 30 pairs, promote on at most 2 exceedances (at p₁ = 0.30 this promotes wrongly with probability 0.0021 and at p₀ = 0.05 it promotes with probability 0.81; binomial arithmetic). Triggers: a missing ledger stratum; an identity mismatch; a failed null self-test of the table.
  • Budgeted mode. 32 pairs per result; on exhaustion the result stays provisional.
  • Worked example, illustrative. ln E⁺ = (n − k)·ln((1 − p₀)/(1 − p₁)) + k·ln(p₀/p₁). With p₀ = 0.05, p₁ = 0.30 and α⁺ = 2/(0.1 × 2,000) = 0.01, promotion takes 16 clean pairs, or 22 with one exceedance; at α⁻ = 0.05 two exceedances within the first three pairs reject. A clean referee episode multiplies the evidence by 1.36 and an exceedance divides it by 6. If C_fp is ten times smaller, or p₁ = 0.5, eight pairs suffice.
  • Primary risk. An envelope tuned after outcomes. Countermeasure: the envelope is hashed into the ledger before sampling. Negative control: the planted tier-sensitive policy of the fidelity gate (§16) must be caught.

DC-02 · Which designs advance.

  • Artifacts. The bracket schedule of the fixed table; per rung pair and stratum the survivor rule of §28.9 with its margin; the one-way handover marker to DC-05; the label budget-truncated.
  • Loss model. Eliminating a true top-k design, against the rung cost of promoting a poor one. The first is far dearer (ADR-ATEL-03).
  • Obligations. Those of §28.9; top-k recall is measured by the audit lane; no design is served by two layers.
  • Fallback and trigger. Halving at η = 3. Triggers: under 19 calibration points; martingale at or above 20; audited miss rate above α; survivors above 3k for two generations; a change of identity or scope.
  • Budgeted mode. Audit lane at a tenth of rung spend; survivors capped at 3k, then truncation by lower-rung score with the label.

DC-03 · The survivor margin and the mode of a rung.

  • Artifacts. The table of margins q̂ by (rung pair, stratum, α); the keyed list of audit draws; the exchangeability martingale; the mode automaton of §28.10 with hysteresis: a mode changes only at a generation boundary, and a stratum needs two consecutive generations of valid calibration to become selective.
  • Loss model. As DC-02, plus the cost of the audit lane itself.
  • Obligations. Coverage at least 1 − α for designs exchangeable with the audit set; exchangeability of the audit draws by construction, because they are drawn by key before scores exist; a pair separated by the margin is misordered with probability at most 2α; the automaton never changes inside a generation.
  • Fallback and trigger. discard-only with the fixed table. Triggers as DC-02.
  • Budgeted mode. The audit lane's tenth of rung spend. When that buys fewer than 19 audit designs in a stratum, strata are merged upward to the morphology class before the stratum falls back.
  • Failure modes. Calibration from survivors only; drift of the optimizer toward a region the audit set does not cover; starved strata. Each is detected by the martingale or by the calibration floor.
  • Worked example, illustrative. n = 39, α = 0.05: q̂ is the 38th smallest residual, say 0.04; the third-best lower-rung score is 0.74; the cut is 0.66.

DC-04 · Certify, refute or stop on a rare failure.

  • Artifacts. The certification table n = ⌈ln(1/δ)/p*⌉; a sequential binomial bound that ends certified, refuted or exhausted (FACT: fs-eproc::bernoulli provides a likelihood-mixture confidence sequence for a Bernoulli rate); a splitting schedule on a distress score, with fixed levels and child world addresses for forks; a failure-mode catalogue with a Good–Turing monitor of unseen mass (modes seen once divided by failures seen); a truncated-weight lower bound for refutation.
  • Loss model and split of budget. False certification far outweighs false refutation. Explore first, to a fifth of the budget or until unseen mass is at most 0.05; refute if the lower bound exceeds p*; spend the rest on nominal sampling, for finalists only.
  • Obligations. The bound holds at confidence 1 − δ for independent nominal episodes with a frozen failure predicate, and is scoped to its tier. Splitting supplies discovery and Estimated rates only. Failures are replayed one tier up. A bound refuses episodes selected by a battery or by a search.
  • Fallback and trigger. Crude Monte Carlo alone. Triggers: under 1% of forks reach the next level (the score is uninformative); forks without fresh randomness.
  • Budgeted mode. Exploration at most a fifth; on exhaustion a refusal that carries the bound reached.
  • Worked example, illustrative. At δ = 0.05, bounds of 10⁻⁴, 10⁻⁵ and 10⁻⁶ need 3 × 10⁴, 3 × 10⁵ and 3 × 10⁶ nominal episodes: 75 s, 12.5 min and 2.1 h at 400 episodes per second. Splitting to 10⁻⁶ with six levels of probability 0.1 and 10⁴ trajectories per level uses 6 × 10⁴ fragments at a coefficient of variation near 7% (cv² ≈ L(1 − p₀)/(p₀N)). Forks find failures much sooner than waiting; published adaptive stress testing reports 2 to 20 times fewer episodes (FACT, source register); 10 to 100 times at rates of 10⁻⁴ to 10⁻⁵ is a TARGET (§26.9). Proving absence has a fixed price that no sampler discounts without assuming where failures live. If throughput is a tenth of that figure, 10⁻⁶ costs 21 h: certify 10⁻⁵ and report 10⁻⁶ as Estimated.
  • Sibling facts. fs-uq has Gaussian importance sampling and multilevel Monte Carlo; no splitting code exists in the suite (FACT, to be re-read).

DC-05 · Which experiment next, or stop.

  • Artifacts. An experiment table (simulation at a rung, a bench measurement, a real batch) with cost in GPU-second equivalents at a declared exchange rate, a closed-form expected value of sample information and their ratio; a Gaussian posterior on the finalists' score differences with sampling, tier-bias and reality-gap variance components read from the ledgers; the stop rule: no ratio clears the budget price.
  • Loss model. Opportunity loss of the accepted design plus experiment cost.
  • Obligations. The value is zero when no outcome can change the choice. Its intervals are scored against registered predictions. It never issues a verdict.
  • Fallback and trigger. The fixed order: check, simulate, train, referee. Trigger: 80% intervals cover outside 65 to 92% over 30 experiments.
  • Budgeted mode. 64 candidates, closed form only.
  • Worked example, illustrative. EVSI = s̃·[φ(z) − z·Φ(−z)] with z = |m|/s̃ and s̃² the variance the experiment removes from Δ ~ N(m, s²). With m = 0.03 and s² = 0.03² (sampling) + 0.04² (tier bias), 200 more fast-tier pairs are worth 0.0013 for 1 GPU-second and 8 referee pairs are worth 0.0055 for 27; the fast tier wins sixfold per unit cost. A second fast batch is worth 10⁻⁵, so the referee goes next: cheap episodes are bought until only tier bias is left, which only the referee can shrink. At m ≥ 0.15 no experiment can change the choice: stop and decide. Assumed: per-pair noise 0.3 on the fast tier and 0.1 on the referee; 30 core-seconds per GPU-second.

DC-06 · Which cause and which repair rank first.

  • Artifacts. Typed interventions (do(torque limit off), do(friction := nominal), do(estimator := privileged)), each one declared mechanism wide; a fork ledger; a placebo flip rate ρ₀ from an intervention that should not matter; a probability of necessity per cause; minimal repair sets by delta debugging; the labels but-for, joint and chaotic: unattributable. The output is which repair edit ranks first.
  • Loss model. A wrong first repair costs an agent turn and its compute; naming a cause in a chaotic regime sends the agent after noise, which costs more than saying nothing.
  • Obligations. The empty intervention replays the failure bit for bit, so the starting point is exact. Each intervention changes one declared mechanism, checked by model diff. A cause is named only when its flip rate beats ρ₀ under a DC-01-style test. Design-parameter interventions re-run from the episode start, because winding temperature and the policy's own history differ; environment and controller-state interventions may fork from a snapshot under the classification of §29.8. Attribution arithmetic runs in double precision.
  • Fallback and trigger. First-anomaly ordering, labelled Estimated. Triggers: a replay mismatch; ρ₀ above 0.2.
  • Budgeted mode. 12 interventions, singles and pairs, about 700 forks. A coalition-value summary is computed only when no single or pair flips the outcome, and is labelled a description: such values can be negative and cannot establish necessity.
  • Worked example, illustrative. Of 200 failing worlds re-run from snapshots, removing the torque limit prevents 164, nominal friction prevents 38 and a placebo nudge prevents 12. Net of placebo: 0.76 and 0.13. If the placebo rate is near a cause's rate, nothing is attributed; if only the pair prevents the failure, the cause is joint.

DC-07 · Whether a result may carry a reality band.

  • Artifacts. A one-sided band, real ≥ sim − q̂, per scope key (robot, firmware, floor, policy); a test martingale on the band's conformal p-values; the table of scopes.
  • Loss model. Citing a simulated result as predictive outside its calibrated family, against withholding a usable prediction inside it.
  • Obligations. Coverage at least 1 − α within one calibrated robot and task family, varying by about six points at 20 calibration episodes (FACT, source register). Nothing for designs that were never built. Conditions of real episodes come from a pre-registered draw; DC-05 chooses only how many. The alarm ends the regime and calibration restarts inside the new one.
  • Fallback and trigger. reality: no-claim. Triggers: under 19 calibration points; martingale at or above 20; any change of scope key.
  • Budgeted mode. The number of real episodes is what DC-05 buys at the declared exchange rate. Below the calibration floor no band is issued; there is no smaller band for a smaller budget.

DC-08 · Which solver effort profile a run uses.

  • Artifacts. Declared effort profiles (iteration counts, tolerances, substeps), each its own model identity; a guard table that holds, per stratum, the cheapest profile whose risk bound (n·R̂ + 1)/(n + 1) is at most τ on the fidelity battery; a rule that re-runs unconverged worlds one profile up, flagged. If the chosen profile misses the throughput budget, the run shrinks its world count or is refused; the profile does not move.
  • Loss model. An under-converged result that passes as evidence, against wasted iterations.
  • Obligations. A monotone loss, enforced by taking the running maximum over higher profiles; expected loss at most τ for worlds exchangeable with the battery; the profile never reads the wall clock and never changes inside a run; the table changes only at a ledger refresh.
  • Fallback and trigger. The default profile of §16. Triggers: fewer than 19 battery points in the stratum; a change of identity.
  • Budgeted mode. At most one step up per world per run.

DC-09 · Which job runs when.

  • Artifacts. Size classes from the system's own --estimate (interactive: at most 30 s); preempt-resume at snapshot boundaries with a quantum of at most 5 s; deficit round-robin per agent, weighted by budget; an admission guard; a capacity table of wait by class against utilization; a monitor of estimate calibration, with a conformal interval on step time per device family.
  • Loss model. Agent idle time on interactive rungs, against preemption overhead and unfairness.
  • Obligations. Utilization below 0.8 per device; a resumed job keeps its digests; an ageing bound prevents starvation. Arrivals are assumed near-Poisson; swarms burst, so the capacity table carries Kingman's correction for arrival variability.
  • Fallback and trigger. Round-robin per agent, run to completion. Triggers: a fifth of jobs exceed twice their estimate; a resume digest differs.
  • Budgeted mode. One preemption per batch job per 30 s; batch share at least 30%.
  • Worked example, illustrative. One device at utilization 0.7 with 90% of jobs taking 10 s and 10% taking 30 min: a 10 s job waits about 2,000 s first-come-first-served (W = λ·E[S²]/(2(1 − ρ))), 621 s with non-preemptive priority, and 0.2 s plus one quantum with preempt-resume; long jobs slow by 3%. The wait is the remainder of the training run already on the device, and only preemption removes it. If a device is reserved for interactive work, or a resume costs several seconds, running to completion is right.
  • Scope. The core decides when a job's episodes run and never which episodes exist or in what order a study consumes them.

DC-10 · Which scenes form a battery.

  • Artifacts. A scene × failure-mode incidence matrix weighted by frequency, severity and designs affected; a lazy-greedy trace of marginal gains; a certificate OPT_k ≤ f(S) + the k largest remaining gains; one slot in five reserved for random held-out scenes.
  • Loss model. Uncovered failure-mode weight, against battery runtime.
  • Obligations. Weighted coverage is monotone and submodular, so greedy selection reaches (1 − 1/e)·OPT at a fixed number of scenes; with unequal scene costs the cost-benefit variant is used and its weaker ratio is stated; the matrix records the identities it was built under.
  • Fallback and trigger. The standard battery of §38. Trigger: a change of identity since the matrix was built.
  • Budgeted mode. Any greedy prefix is a valid battery.
  • Failure mode. A battery that overfits past failures; the held-out slots and the rule that DC-04 refuses battery-selected episodes contain it.
  • Worked example, illustrative. 40 modes of total weight 100; greedy gains of 31, 22, 14, 9, 6 and 4 cover 86; the six largest remaining gains sum to 12, so OPT₆ ≤ 98 and the battery is within 88% of the best six-scene battery, against a worst case of 63%.

DC-11 · Whether a kernel rewrite is accepted.

  • Artifacts. A closed rule table. Class E is bit-exact under round-to-nearest with contraction off: commutation, common-subexpression elimination, dead-code elimination, scaling by a power of two away from overflow and underflow, and scheduling. Class R is equal over the reals only: reassociation, distribution, fused multiply-add. A certificate is (source hash, target hash, rule trace), replayed by an independent checker.
  • Loss model. A silent bit change invalidates replay and both ledgers; a skipped rewrite costs a few per cent of speed.
  • Obligations. Each class-E rule is shown exact once on its domain, and digests match on a pinned corpus on every backend. A class-R rewrite needs a polynomial-identity certificate (C-14), an error bound, the fidelity battery and a new kernel version; a new version is a new identity and resets evidence.
  • Fallback and trigger. The unrewritten derived kernel. Triggers: a digest mismatch; a whole-step gain under 3%.
  • Budgeted mode. 60 s of search per kernel; one class-R candidate per release.
  • Worked example. In f32 with a = 10⁸, b = −10⁸, c = 1: (a + b) + c = 1 and a + (b + c) = 0, because the spacing of f32 near 10⁸ is 8. Sharing a subexpression used three times saves two evaluations and changes no bit. Regrouping makes a different kernel, which must be named as one.
  • Ceiling. The Lie-algebraic recursions are about 16 to 20% of an incumbent's step, so specializing them is worth about 1.2 times at most (FACT, source register). Class R and any wider search are worth their certificates only if our own M2 profile shows more than about a fifth of step time in derived arithmetic.

28.19.3 Composition: timescales and interference

Timescales, slowest first: DC-11 at build; DC-10 per battery revision; DC-08 at ledger refresh; DC-07 per real batch; DC-03 per generation; DC-02 per rung transition; DC-05 per experiment; DC-04 per finalist; DC-06 per failure cluster; DC-01 per episode batch; DC-09 per quantum. Each loop treats every slower one as a constant, and no loop writes a quantity that a faster loop is reading inside its own period. DC-04 → DC-06 → DC-10 is a one-way pipeline: failures found feed attribution, attributed modes feed the incidence matrix, and nothing flows back inside a campaign.

Where two controllers touch the same quantity, one rule assigns it and one test shows the rule holds.

Controllers and the shared quantity Rule Interference test
DC-02 and DC-03 survivors, DC-05: what runs next Conformal survivors own the population stage, value of information owns the finalists; the handover is one-way In a replayed campaign no design receives requests from both
DC-01 stopping, DC-03 calibration: higher-rung episodes Audit designs run at fixed n and are never cancelled No audit episode was stopped by an e-process
DC-01, DC-02, DC-03, DC-05 and DC-09: GPU-seconds, completion order DC-09 decides when, never what; e-values consume episodes in address order Random preemption and reordering leave digests and verdicts identical
DC-08 profile and DC-09: throughput DC-08 never reads the wall clock and never changes inside a run Under a saturated queue the profile and the digests are unchanged
DC-08, DC-11 and the ledgers of DC-01 to DC-03: model identity An identity change resets evidence and calibration; a class-E rewrite changes no identity Pooling across two model identities is refused; 1,000 worlds keep their digests after a class-E rewrite
DC-04 certification and DC-10: which scenes are sampled Bounds use nominal draws only A bound refuses battery-selected episodes
DC-07 band and DC-05: real episodes Conditions come from a pre-registered draw; DC-05 chooses only how many Assigned conditions equal the draw
DC-01 inside an N-way race: error rate The mixture construction of §28.6, or α/(N − 1) per comparison A null race of 81 keeps the best with probability at least 1 − α
The memo policy and every core: outcomes A hit is bit-identical to a recomputation, so no core can tell A partly memoized comparison reaches the identical e-value

Two tensions are reconciled by construction. Conformal exchangeability against drift: the alarm ends the regime and calibration restarts inside the new one. A parametric model (DC-05) beside distribution-free tests (DC-01 to DC-03): the parametric model never issues a verdict.

28.19.4 Methods rejected as ornamental or premature

Each was considered for a named core and is not built. The reason is recorded so the question is not reopened without new evidence.

Method Core Why not
E-graph saturation with a solver over real-arithmetic rewrites DC-11 Every accepted rewrite changes bits, so each needs an error bound, a battery run and a new kernel identity, while most step time is expected in the solve. Reopen only on the profile condition of DC-11
Stochastic superoptimization and counterexample-guided synthesis of kernels DC-11 No sound equivalence check exists for floating-point code
Extreme-value extrapolation of failure tails DC-04 Checking a fitted tail at 10⁻⁶ takes the same 3 × 10⁶ nominal episodes that certify directly
Defensive-mixture importance sampling as the certified bound DC-04 Its floor is ln(1/δ)/(βn), never below crude sampling on the nominal share β. It survives for refutation, by a lower bound, and for tightening an estimate once failures are observed (§26.9)
Gaussian-process surrogates over designs DC-02, DC-05 No validated kernel exists on typed designs of varying topology; continuous sub-problems already have fs-bo
A learned correction of the fast tier, or a learned step surrogate as a screen DC-01, DC-02 It hides exactly the artifacts the trust ladder exists to expose
Feedback control (PID or model-predictive) on solver effort DC-08 Wall-clock feedback into iteration counts makes results depend on load and breaks replay. A guard table with hysteresis passes
Bandit allocation (upper confidence bounds, posterior sampling) DC-02, DC-09 Regret is the wrong objective for identifying the best design, and a bandit on dispatch is a second controller on shared telemetry
Expected information gain DC-05 It buys knowledge that cannot change the choice; the estimator is doubly intractable and fails under misspecification
Kernel two-sample and optimal-transport drift tests DC-07 No power at twenty scalar residuals
Mechanism design for the queue DC-09 Estimates come from the system, not from agents
Coalition-value attribution as the primary output DC-06 Values can be negative and cannot establish necessity. Kept as a budgeted description
Adaptive-fidelity stopping as a decision procedure DC-01, DC-02 Stopping a pair early on cheap labels does not compute the sealed fixed-n decision, and coarse concordance has no sound relation to strict outcomes. Kept as a scheduling heuristic that orders work without touching validity
Reuse of a verdict by sensitivity margin §28.13 Withdrawn for want of a certificate; a matched dependency graph duplicated every saving
Replicated consensus, gossip membership, escrow counters for the daemon DC-09 One daemon per host; nothing is replicated
Copy-on-write device state for forks DC-06 Wrong bottleneck: copying 8,192 worlds at 30 kB is 0.25 GB

Not scored, each for a one-line reason: convex dual prices (one scalar, already the DC-05 threshold); Lyapunov and bifurcation analysis of solver effort (the residual monitor covers DC-08); observational causal adjustment (exact intervention is available); subset simulation by Markov chains (deferred unless forks carry no fresh randomness).

28.20 Refusals

Code Refused Ranked repair
FROB-REF-553 A comparison that mixes study types, or re-reads fixed-n data as a sequential study Declare one type; open a new study
FROB-REF-555 A fixed-n study whose declared verdict set contains an unreachable verdict Raise n to the first value at which it is reachable (reported), or acknowledge
FROB-REF-542 A paired study under an uncontrolled determinism class, or with arms in different classes Rerun both arms on a D2 profile
FROB-REF-543 An anytime study on a metric with no declared span Declare the span in the task
FROB-REF-556 A pairing keyed by event order or by a training counter Space-indexed fields; (world, replicate) addresses
FROB-REF-545 A rung asked to select in a stratum whose mode is not selective Run the audit lane, or accept "k survivors"
FROB-REF-546 Calibration from survivors, or an audit episode cancelled or re-weighted Redraw the audit lane by key
FROB-REF-557 A model-conditional result offered as a design comparison without its condition, or as input to a build packet Run A-19; report the condition
FROB-REF-548 Pooling evidence or calibration across two model identities Restart the ledger rows in the new identity
FROB-REF-549 A rare-failure bound over battery-selected or search-selected episodes Draw nominal episodes
FROB-REF-550 first-change asked of bisect without a prefix scan or a declared monotone predicate Scan, or declare and record the assumption
FROB-REF-551 An exclusion from an analysis whose model does not enclose a declared state or energy source Bind the missing source; accept unresolved
FROB-REF-870 Reuse of a verdict with no licence, or under a rule the registry does not know Recheck
FROB-REF-871 A decision-core table that is missing, stale against its contract, or not the registry version Regenerate; the fallback is in force meanwhile
FROB-REF-872 A value-of-information output cited as a verdict Run the comparison

28.21 Invariants and proof obligations

ID Invariant
INV-ATEL-01 No stage of the loop runs on a design the previous stage refused, unless the request carries an override that is recorded on every downstream result.
INV-ATEL-02 A study's statistic is a pure function of the per-address outcomes taken in address order. It does not depend on completion order, preemption, co-tenancy, shard count or memo hits.
INV-ATEL-03 No audit-lane episode is cancelled, re-weighted or selected after any score for its design exists.
INV-ATEL-04 No design receives allocation requests from both the population layer and the finalist layer.
INV-ATEL-05 Evidence and calibration are never pooled across model identities. A change of model, kernel version, effort profile or battery version resets the affected ledger rows.
INV-ATEL-06 A decision core's action is a pure function of its table version, the ledger state at a pinned frontier and addresses. It reads no clock.
INV-ATEL-07 Every reused verdict carries a grade and a licence; a grade-3 licence names a rule in the registry.
INV-ATEL-08 A memo hit is bit-identical to a recomputation inside its determinism class.
INV-ATEL-09 Lineage merge and archive merge are commutative, associative and idempotent.
INV-ATEL-10 Every paired study satisfies the accounting identity and states its denominator.
INV-ATEL-11 No verb of this chapter reads a confirmation world of the blinded qualification lane (§24).
INV-ATEL-12 A value-of-information quantity never appears in a verdict field.
ID Obligation How it is discharged
PO-ATEL-01 The DC-01 table keeps wrong promotion at most α⁺ and wrong rejection at most α⁻ under optional stopping. A written supermartingale derivation; 10⁴ null runs at p₁ and at p₀ with optional stopping; the table regenerated in continuous integration and diffed.
PO-ATEL-02 The fixed-n rule has simultaneous coverage of at least 95%, and the reachability report is exact. Exact enumeration in rational arithmetic at the declared n.
PO-ATEL-03 Split-conformal coverage holds for exchangeable audit designs, and its spread is the Beta law. Simulation over 10⁴ calibration draws; the 5th percentile at n = 39 is checked against 0.885.
PO-ATEL-04 In a race the true best is eliminated with probability at most α. The mixture derivation of fs-race; a null race of 81 over 512 replays within the exact binomial acceptance bound.
PO-ATEL-05 The greedy certificate OPT_k ≤ f(S) + the k largest remaining gains is correct. Independent recomputation of marginal gains in integer weights.
PO-ATEL-06 Each polarity rule preserves what it claims to preserve. A written derivation per rule; a positive and a negative fixture per rule; exhaustive check on the finite-game fragment of §25.
PO-ATEL-07 Incremental evaluation with the memo graph equals full recomputation. 10⁴ random edit sequences; shadow recomputation in production.
PO-ATEL-08 design.bisect errs with probability at most α and never reports first-change wrongly. Union bound derivation; 100 planted lineages, 20 of them non-monotone.
PO-ATEL-09 Scheduling does not change evidence. Random preemption, reordering and co-tenancy over 1,000 episodes in 100 schedules; digests and verdicts identical.
PO-ATEL-10 The risk bound of DC-08 holds for the monotonized loss. Simulation on the fidelity battery with a planted under-converged profile.

28.22 Known answers and negative controls

Subject Known answer Negative control that must fail
A-03 A quadruped with one forward lidar: four-element mechanical group, two-element observation-preserving subgroup A policy tied under the mechanical group is refused
A-07, A-09 A mecanum base with misaligned rollers: full-rank curvature, rank drop in the growth vector A curvature-only check passes the defective base
A-08 The stance enumeration: 11,520 of 11,520 classifications agree with the exact classifier; equal-spring forces (175, 115, 85, 25) N The polygon-and-total-capacity screen accepts 1,596 rejected bodies
A-17 Seven independent readings at 0.75: 3807/4096; fully shared error: 0.75 A redundancy score computed from per-reading accuracy
A-20 0.94176 J against a 0.90 J budget excludes; 0.05 J of approach energy removes the exclusion An exclusion issued with approach energy unbound
A-21 Packet at tick 1, decisions at ticks 2 and 4; delays to ticks 2 and 3 change the answer only at the second delay and only for the first decision A check that compares content and ignores arrival
A-22 Spectral radii 1.4820 and 0.9207 for the two feedforward laws The instantaneous-current model (2.6917 for both)
A-23 Two fingers, one bus: zero schedules at a one-tick deadline, exactly two at two ticks The additive-peak rule
A-25 Zero sensitivity to μ at 1 A and 2 A; 4 N against 8 N at 3 A under 10 N A fit that reports μ from the first two runs
A-18 A doorway narrower than every orientation of the footprint returns an impossibility cut that names the doorway; widened past the footprint it returns a witness path (fixtures in §19) A clearance test on a bounding ball for a rectangular footprint
A-24 A task wrench outside the attainable set returns a dual vector that separates it from every admissible actuation (fixtures in §22) An infeasible wrench reported as not-synthesizable with no witness
A-26 A run with one planted mode switch is partitioned into two regions with the boundary at the switch tick A gradient reported as trustworthy across the switch
A-19 A finalist whose advantage comes from diagonal traction on a pyramidal cone (up to √2·μ) is typed model-conditional The same campaign with stage 5 skipped reports it as the winner
Fixed-n rule 2,145 pairs at n = 64: 521, 521, 1,103, 0; first tie at n = 86 A study that declares "tie" reachable at n = 64
Zero discordances in 64 One-sided 95% bound 1 − 0.05^(1/64) = 0.04573 —
DC-01 table 16 clean pairs, or 22 with one exceedance, at p₀ = 0.05, p₁ = 0.30, α⁺ = 0.01 An envelope changed after the first outcome
Conformal margin n = 39, α = 0.05: the 38th residual Calibration from survivors only
Stage 1 86,185 and 500,500 representatives Deduplication under the mechanical group when a sensor breaks it
Deduplication digest 735,471 ballast placements give 24,546 classes Two placements with equal first moment and different second moment merged
Revalidation The four-state camera fixture: 2, 0, 0, 1 evaluations and 1, 0, 0, 1 recomputations The key without delivery time keeps a stale promotion
Revalidation Widening the doorway: the cost-9 cart stays feasible, its optimality is revoked An optimum kept after a cheaper rival became feasible
Bisect Outcomes pass, fail, pass, pass, pass, fail: an a-change at the last edit, no first-change without a scan first-change at the last edit
DC-09 Waits of about 2,000 s, 621 s and 0.2 s in the worked queue —
DC-10 Six gains cover 86 of 100; bound 98 A certificate computed from stale gains
DC-11 The f32 regrouping example gives 1 and 0 A reassociation admitted in class E

28.23 Gates and kill gates

Method gates have the same form: competing arms, a potency control that the method must catch and a clean case it must pass, a paired lower bound, and a fallback named in advance.

Gate Claim Threshold (TARGET)
GATE-ATEL-01 Design comparison BET-05 on the design corpus: area under the ROC curve at least 0.8 and at least 0.05 above the heuristic baseline at a 95% paired lower bound; planted defective designs flagged and a planted sound design passed.
GATE-ATEL-02 Known answer Each analysis A-01 to A-14 passes its known-answer tests; design check finishes in under 2 s at the 95th percentile on templates.
GATE-ATEL-03 Design comparison BET-10: the A/B of agent surfaces on the design benchmark of §24, pre-registered, with an anytime-valid verdict and an effect size: restricted mean time to an accepted answer lower by a factor of at least 1.5 on at least ten briefs. The incumbent arm belongs to §39, not to this gate.
GATE-ATEL-04 Known answer Comparison type-I error at or below α under optional stopping over 10⁴ null comparisons; design bisect finds the planted edit in at least 95 of 100 trials.
GATE-ATEL-05 Known answer Each analysis A-15 to A-26 passes its known answers (§28.22 and the owning chapters) and its negative control fails; the analyses marked ✓ keep design check under 2 s.
GATE-ATEL-06 Known answer The fixed-n planner reproduces the enumeration at n = 64 and n = 86 exactly; FROB-REF-553, -555, -542, -543 and -556 each fire on their planted violation, 100 of 100.
GATE-ATEL-07 Design comparison The rank-agreement experiment per morphology class: Kendall τ on the top band at least 0.7 for selective and at least 0.5 for discard-only, and ρ_ep at least 0.7, all at the lower bootstrap bound, on 40 designs. A class below 0.5 is uninformative.
GATE-ATEL-08 Design comparison Layer 2 against the fixed table on the same population and addresses: held-out coverage at least 1 − α − 0.03 per stratum; audited top-k recall no lower than halving's; fewer GPU-seconds to the same accepted decision at a 95% paired lower bound above 1; a planted over-scored design is caught at the next rung; synthetic drift trips the alarm.
GATE-ATEL-09 Known answer Atelier does not trust a rung whose screening audit (§26.10, GATE-NEMESIS-07) is missing or failed: in 100 of 100 planted campaigns a stage with no audit record, a missed planted winner or an audited miss rate above its tolerance has its rung set to uninformative in the affected strata and its results labelled screen-biased; a planted unsound prune is caught in 100 of 100; no audit episode is ever cancelled.
GATE-ATEL-10 Known answer A-19: every planted cone-excess and energy-pumping exploit is typed model-conditional; honest winners are typed model-robust; the stage costs at most 10% of campaign compute.
GATE-ATEL-11 Known answer Revalidation on the frozen 30-amendment campaign: every decision equals full recomputation; zero unsafe reuse; the stale-key mutant is caught; unnecessary invalidations are at most 20% of unaffected verdicts, counted and reported.
GATE-ATEL-12 Known answer, Performance PO-ATEL-07 holds; a shadow mismatch stops serving within one hit; design check after a one-parameter edit in 0.2 / 0.5 / 1.0 s at the 50th / 95th / 99th percentile.
GATE-ATEL-13 Known answer Each decision core: its table regenerates bit-identically; its null calibration holds (PO-ATEL-01 to -05, -10); its fallback is demonstrated by forcing each trigger; its shadow run records both actions.
GATE-ATEL-14 Known answer The nine interference tests of §28.19.3 pass on a replayed campaign.
GATE-ATEL-15 Design comparison The offline replay of §28.9: the knowledge-gradient chooser at least 1.5 times better than the agent chooser in evaluations to a true top-3; the value is exactly zero on a fixture where no outcome can change the choice.
GATE-ATEL-16 Known answer The cascade end to end on a seeded population: stage controls all run; the report prints every mode, every fallback and the phase ledger; merge of two agents' archives in both orders gives identical bytes.
Kill gate Trigger Fallback
✖ KILL-STATIC GATE-ATEL-01 fails (BET-05) design check verdicts are downgraded to advisory lints and the pruning claim is withdrawn
✖ KILL-RANK GATE-ATEL-07 fails for every morphology class tested R1 is uninformative everywhere and every candidate that survives static analysis is scored on the higher rung; the ladder's claim shrinks to stopping and elimination among trained candidates; §39 records the lost factor
✖ KILL-LADDER GATE-ATEL-08 fails after one repair cycle The fixed halving table is the allocation rule; the audit lane is kept as measurement
✖ KILL-VOI GATE-ATEL-15 fails Value of information is a stop rule only; experiment.suggest lists proposals unranked by value
✖ KILL-REUSE Any unsafe reuse in GATE-ATEL-11, or grade 3 saves less than a fifth of the recheck cost grade 2 leaves Grades 1 and 2 only
✖ KILL-FDGRAD Owner §17.13; this chapter is a consumer. The pilot of §17.13 meets its falsifier (variance ratio below 2 at every horizon of 50 steps or more, or sign agreement below 70%), or no end point is confirmed by a specialist Rank-based search (CMA-ES, racing), with the stencil kept as a sensitivity report
✖ KILL-ATLAS The A-26 pilot of §28.4 fails A-26 is a descriptive report with no validity label

Milestones (§43): GATE-ATEL-02 and -06 at M1, with the static analyses A-20 to A-23 of GATE-ATEL-05, the analysis part of GATE-ATEL-12 and the dependency grades of GATE-ATEL-11 on mission cells; the episode part of GATE-ATEL-12 at M2; GATE-ATEL-01 and -04 at M3, with the dependency grades of GATE-ATEL-11, the cores DC-01, DC-09 and DC-10 of GATE-ATEL-13, and the pilots of GATE-ATEL-07 and -08 on an incumbent engine or on batch-cpu; GATE-ATEL-03, -07 to -10, -14 to -16, the rest of -05, -11 and -13 at M4. Each kill gate is decided at the milestone of its gate and recorded as an ADR.

28.24 Open questions

  • OPEN-ATEL-1. The loss constants of DC-01 (C_fp, C_fn, the pair cost c) and the exchange rate between GPU-seconds and real experiment cost in DC-05. Owner: Atelier lead. Blocks GATE-ATEL-13. Default: the fixed 30-pair rule and the fixed experiment order.
  • OPEN-ATEL-2. The definition of a stratum (archive cell blocks, or morphology class by task family). The mode thresholds are no longer open: they are 0.7 and 0.5 (§28.10). Owner: Atelier lead. Blocks GATE-ATEL-07. Default: one stratum per morphology class; an unmeasured stratum is uninformative.
  • OPEN-ATEL-3. Whether fs-recompute, fs-voi and fs-race pass admission through the airlock on the toolchain T. Owner: suite lead. Blocks GATE-ATEL-12 and -15. Default: the node record, the closed-form value and the race are implemented locally against the same known answers.
  • OPEN-ATEL-4. Whether result.revalidate and the mission cluster's revalidation verb stay two verbs over one engine or become one. Owner: Tether lead. Blocks the verb registry freeze at M3. Default: two verbs, one engine.
  • OPEN-ATEL-5. The planted winner for morphology classes that have no earlier specialist result. Owner: Arena lead. Blocks GATE-ATEL-09 for those classes. Default: the control is reported not-run and finalists are "k survivors".
  • OPEN-ATEL-6. Whether a fine-tune fraction of 5% is the right R2 rung, or two R2 rungs are needed. Owner: Cortex lead. Decided by the calibration experiment. Default: one R2 rung at 5%.

28.25 Controller-budget curves and the controller-limited label

Controller-budget curves F. Every cheap rung of §28.9 scores a body together with a controller of fixed competence: the sampling controller at one sample count, a family policy with no adaptation, a fine-tune at one fraction of specialist steps. The score ranks body × controller, and a design that needs more controller effort to show what it can do is under-scored. §28.10 corrects this by region after the fact. The mechanism here measures it per design before the design is discarded.

The budget coordinate. ControllerId (§22) gains a budget coordinate b on a registered ladder b₀ < b₁ < b₂ with a scalar effort c(b). For mpc-sample, c = N_s·N_it·H model steps per tick, with the default ladder N_s ∈ {64, 256, 1,024} (DESIGN). For a learned policy, c is the fine-tuning fraction of specialist steps, default {0, 0.05, 0.2}, which are the R1 and R2 rungs already defined. A row is keyed by (design, world address, budget). Every budget runs on the same addresses, so differences across budgets are paired.

The curve. For design d in a morphology class,

J_d(b) = J_d∞ − a_d·(c(b)/c(b₀))^(−γ),     a_d ≥ 0,  γ > 0 shared by the class,  J_d non-decreasing in b.

For fixed γ the law is linear in (J_d∞, a_d): a two-parameter least-squares fit per design, with one residual degree of freedom at three budgets, and γ the minimizer of the pooled residual over the class. On a ladder with a constant effort ratio the three-point form is closed. With Δ₁ = J(b₁) − J(b₀), Δ₂ = J(b₂) − J(b₁) and ρ = Δ₂/Δ₁,

J∞ = J(b₀) + Δ₁/(1 − ρ),      saturation index  s_d = (J(b_max) − J(b₀)) / (J_d∞ − J(b₀)).

The saturation index is the share of the predicted gain already realized at the highest budget reached.

Worked case, illustrative. Design A scores 0.61, 0.70, 0.72 at the three budgets and design B scores 0.55, 0.68, 0.76. Then ρ_A = 0.02/0.09 = 0.222, J_A∞ = 0.61 + 0.09/0.778 = 0.726 and s_A = 0.11/0.116 = 0.95; ρ_B = 0.08/0.13 = 0.615, J_B∞ = 0.55 + 0.13/0.385 = 0.888 and s_B = 0.21/0.338 = 0.62. The fixed-budget verdict at b₀ prefers A by 0.06 and at b₂ prefers B by 0.04: the curves cross between b₁ and b₂. A is saturated; B is limited by its controller.

What the estimate may do. J_d∞ is Estimated. It steers allocation and never enters a verdict field, under the same invariant that binds value-of-information quantities (§28.21). Its interval is not the standard error of the fit: it is the conformal margin of §28.9 for the predictor ĝ = J_d∞, calibrated on audit-lane designs that were scored at the specialist rung. Where no calibration exists the interval is the metric's declared span, and the rule below reduces to the fixed table. The measured contrast at the highest budget reached goes through §28.6 and DC-01 unchanged and is the only quantity that can become established.

Escalation rule. Every candidate runs at b₀. A design moves to the next budget when the upper end of its interval for J_d∞ reaches the k-th best measured score of its stratum. A design whose fit is refused is treated as unsaturated and escalates. A design is flagged controller-limited when J_d∞ − J_d(b_max) exceeds the decision margin of the study; campaign reports print the flag beside the rank, and a controller-limited design is never described as inferior. Cost, illustrative: with an effort ratio of 4 and escalation of 30% of designs to b₁ and 10% to b₂, the mean cost per design is 1 + 0.3 × 4 + 0.1 × 16 = 3.8 units of b₀, against 16 for running every design at b₂. On the quadruped row of §22.8 (0.33 GPU-minutes per episode at 2,048 samples; cost proportional to the sample count) a design escalated through all three budgets on three addresses costs 3 × (64 + 256 + 1,024)/2,048 × 0.33 = 0.65 GPU-minutes (DESIGN arithmetic on that FACT), inside the three-episode screening allowance of §22.8.

Refusals of the fit. The fit is refused and the design labelled curve-unfit when its scores decrease by more than paired noise, when Δ₂ ≥ Δ₁ (ρ ≥ 1: the curve is still accelerating), or when Δ₁ is within noise of zero. Counterexample: a body that needs a threshold budget to find any gait scores 0.20, 0.20, 0.80. The two lower budgets extrapolate to J∞ = 0.20, wrong by at least 0.60, and the three-point form divides by zero. A saturating law does not describe a threshold. Such a design is escalated and is never discarded on its curve.

Owners and verbs. frob-control owns the budget coordinate; frob-opt owns the fit, the index and the escalation rule; rows are stored by frob-chronicle. eval.compare --budget-curve returns, per design, the scores by budget, J∞ with its interval, the saturation index and the flag. opt.cascade applies the escalation rule at stage 3, and explain.compare prints both curves.

Ceiling, pilot and gates. Ceiling, stated before any test: the rule cannot rank better than running every design at the top budget. Its benefit is the compute between that and the escalation rule, at most c(b₂) divided by the mean cost (16/3.8 = 4.2 in the illustration), and it is nothing when no two curves cross. Pilot before the work package is funded, on batch-cpu: 20 variants from two templates at N_s ∈ {32, 128, 512} on shared addresses. If no two curves cross, rankings do not depend on budget, the mechanism is not built, and §22.8 is the better for it. If more than a fifth of the curves are refused, the law is wrong for this controller.

Gate Claim Threshold (TARGET) First milestone
GATE-ATEL-17 Known answer On 10³ synthetic designs drawn from the law with planted (J∞, a, γ) and paired noise, the calibrated interval covers the planted J∞ at a rate of at least 1 − α − 0.03; the worked case returns 0.726, 0.888, 0.95 and 0.62; the threshold case is refused M3
GATE-ATEL-18 Design comparison On 30 designs in three classes with an oracle (a specialist per design over five seeds, and the sampling controller at 8,192 samples): top-3 regret at equal total compute at most half that of ranking at the base budget; J∞ closer to the oracle than J(b_max) for at least half the designs M4
Kill gate Trigger Fallback
✖ KILL-BUDGETCURVE The pilot shows no crossing; or at GATE-ATEL-18 regret at equal compute falls by less than 30%; or J∞ is further from the oracle than J(b_max) for more than half the designs Fixed-budget scores, with the saturation index printed as a description and no escalation rule

28.26 The reuse ledger: three columns, never summed

Three kinds of reuse, reported apart S. This extends §28.13 and §28.14. "Reuse after an edit" has meant three different things. Adding them, or reporting one under another's name, overstates what a cache does.

Column Definition What it is evidence of
Exact reuse Compute whose result is taken unchanged from the memo graph because no dependency changed: analyses, step plans, kernels, scene compiles, rollouts of untouched designs The memo graph (§28.14)
Evidence carried forward Verdicts that result.revalidate keeps without recomputation, each with the dependency receipt that justifies it The revalidation rule (§28.13)
Reacquisition avoided Training or optimization steps not spent because a child started from a parent's policy, orbit or solution, measured against a from-scratch control stopped at the same return Inheritance and warm starts (§23.12, §22.9)

Each column is a share of one declared estimand: total work per accepted child decision, in GPU-seconds and CPU-seconds, priced cold and warm separately. The third column is counted here and nowhere else; the phase ledger of §39 does not credit it again to inheritance. A reuse figure printed without its column name is refused in a report.

Arithmetic the ledger must reproduce. With rollouts and training at nine tenths of per-design compute, exact reuse after a dynamics-affecting edit is at most about one tenth. A kernel compile of 95.91 s beside 240 s of training is 28.6% of a cold run, and the cached 0.89 s is 0.37% of a warm one (FACT, Appendix H), so a cold figure flatters a cache. If a warm start avoids a fraction a of training, the third column is 0.9·a: 72% at a = 0.8. An edit therefore "reuses 70% of prior work" only through the third column, and only if inheritance holds for that class of edit (§23.12).

Owner and verbs. frob-chronicle keeps the ledger and frob-opt runs the from-scratch control. design.diff --reuse and report.campaign print the three columns per edit class: mass, mount, controller-only, mission-only, topology.

Gate Claim Threshold (TARGET) First milestone
GATE-ATEL-19 Known answer; Performance On one frozen lineage of 30 edits spanning the five classes, the three columns are reported cold and warm, with the from-scratch control run for at least one edit per class. The columns plus recomputed work sum to the measured total within 2%. A controller-only edit shows exact reuse of every model analysis, and a mass edit shows exact reuse of none of its rollouts M3