FrankenRobotics

Arena: tasks, requirements, fixtures, metrics, qualification

15,330 words23 subsections34 tables7 listingsPlan version 1.1

On this page23
24.1 What Arena owns24.2 Decisions24.3 Task families24.4 The task file .ftask24.5 The requirement language24.6 Requirements and rewards24.7 The settled-pose reset cache24.8 Metrics and scorecards24.9 Batteries and the design corpus24.10 Fixture 1: a loaded cart crosses a sill and docks24.11 Fixture 2: a quadruped on a split-grip landing24.12 Two study types24.13 The blinded qualification lane24.14 Verbs and refusals24.15 Invariants and proof obligations24.16 Known answers24.17 Gates, kill gate and pilots24.18 Open questions24.19 Traction capacity is a slice of the wrench set at the commanded wrench24.20 Auditing a task before any design runs on it: foil bodies and intent invariants24.21 The item bank: importance sampling of battery items for close comparisons24.22 Exposure domains and the coupling test24.23 The idle arm of the task audit

Crates frob-spec and frob-arena, both in layer 4.

24.1 What Arena owns

Arena defines what a robot is asked to do and how an answer is scored. It owns the task families, the task file .ftask, the requirement language and its monitors, the mission-level verdict, the fixtures whose answers are known before any run, the metrics and scorecards, the batteries and the design corpus, the two study types, and the blinded qualification lane. frob-spec holds the language: parser, unit and clock checker, normalizer, monitor compiler and the verdict type. frob-arena holds the registry of tasks and fixtures, the metric definitions, scorecards, study plans and the qualification service.

Arena does not own the machinery it scores with. The boundaries are fixed here so that no other chapter has to guess.

A note on numbers. A value in this chapter that follows from a stated formula is a derived value: it is a known answer of §24.16 and is recomputed by its test. Every other number carries its label where it appears.

Concern Owner What Arena contributes
Observation, action and reward terms; reset and bootstrap semantics Cortex (§23) The task file carries the terms. Arena never reads a reward to decide a verdict
Mission queries with quantifiers over designs, policies and worlds Augur (§25) The acceptance predicate of a mission query is a requirement of this chapter
Search for worlds that break a design; rare-failure bounds Nemesis (§26) The falsification score is the requirement margin; infeasible worlds are counted here
Attribution of a failure Autopsy (§27) The outcome that an intervention flips is a verdict of this chapter
Paired comparison, racing, the ladder of rungs, promotion between tiers Atelier (§28), Kinesis (§16) The study types and the confirmation lane wrap those mechanisms
Telemetry channels, snapshots, the failure taxonomy Chronicle (§29) Requirements read channels by name, kind and clock
Scene splits and held-out scene families Habitat (§19) Arena enforces the split at scoring time (GATE-ARENA-02)

24.2 Decisions

  • ADR-ARENA-01. Requirements decide acceptance; reward terms train. S A mission is accepted or rejected by a requirement that returns a verdict and a margin. Reward terms remain the training signal. Reason: as an evaluation metric a requirement is close to definitional, a pass or a fail with a distance to the boundary. As the only training signal the margin is the weak baseline in the field's own results. FACT (Appendix H, temporal-logic reinforcement learning): decomposing a formula into timed subgoals and invariant constraints raised task success by an average of 31.6% over the best baseline, in a field where earlier methods used margins as sparse terminal rewards; and margin-only optimizers reached between 6.67% and 29.9% success in a study of robustness metrics for robot tasks.
  • ADR-ARENA-02. Bounded time only. S Every temporal operator carries a finite window. Reason: a monitor needs fixed memory, a verdict needs a finite horizon, and formulas with unbounded time admit no learning guarantee except where they are decidable within a finite horizon (FACT, Appendix H).
  • ADR-ARENA-03. Every atom is normalized by a declared scale. S An atom states an inequality and the physical scale that makes its margin dimensionless. Reason: the minimum of a clearance in metres and a thermal margin in kelvins has no meaning, and the composition rules of the language are minima and maxima.
  • ADR-ARENA-04. Past time online, future time offline. S Operators that look backwards are computed inside the step and are available as signals. Operators that look forwards are evaluated on a completed trace and are the verdict. Reason: an online value of a future-time operator is a silently truncated window.
  • ADR-ARENA-05. The declared comparator decides at the boundary, in exact arithmetic. S The sign of a floating-point margin does not decide a case that sits on the boundary. Reason: a strict and a non-strict inequality differ exactly there, and single precision misclassifies near-boundary cases in bulk (§24.5).
  • ADR-ARENA-06. Clocks are explicit and rational. S Every signal and every requirement names its clock. Periods are rational numbers of seconds with a phase. Reason: sensor, policy and physics rates are not integer multiples of each other in real robots, and a requirement evaluated on an unstated clock is a different requirement on every backend.
  • ADR-ARENA-07. A study is anytime-valid or fixed-n, never both. S Reason: each kind of inference is valid only under its own stopping rule (§24.12).
  • ADR-ARENA-08. Discovery and confirmation are separate lanes with separate data rights. S Reason: an optimizing agent that may resubmit to the same held-out worlds will eventually pass by chance (§24.13).
  • ADR-ARENA-09. An infeasible scenario is kept and counted. S Reason: a failure is a failed cell, never an excluded cell (§8), and a suite that sheds its impossible cases reports a success rate over a population nobody declared.
  • ADR-ARENA-10. A fixture is complete before its first run. S Every mass, constant, seed and rule of a fixture is recorded and hashed before either design runs, with a derivation on file for each analytic answer. Reason: a fixture whose constants can move after the outcome is seen is a tuning harness.

24.3 Task families

Family Example tasks First milestone
Stand and recover Stand for 10 s; recover from a push of stated impulse M1
Threshold delivery A loaded cart crosses a door sill and docks on the run-out (fixture 1, §24.10) M1, CPU
Timed docking Reach a pose with zero velocity by a deadline under a hidden plant gain (the exact mission slice of §25; its boundary case is a known answer of §24.16) M1, CPU
Velocity tracking Commanded planar velocity and yaw rate on flat and rough ground M2
Terrain traversal Stairs up and down, gaps, stepping stones, slopes, rubble M2
Split-grip landing A quadruped climbs a ramp onto a landing with a low-grip strip (fixture 2, §24.11) M2
Interior navigation Point goal through doors and thresholds in each archetype M3
Payload transport Carry or tow a stated mass along a warehouse route, including a ramp M3
Low-friction crossing Cross a kitchen galley with a wet or greasy patch M3
Tight-space maneuvering Turn around, parallel shift and dock in an aisle of stated width M3
Moving surfaces Board and leave a conveyor M4
Stair and curb climbing for wheels and tracks A tracked base on stairs; a wheel cluster on a curb M4
Reach and carry Mobile base with an arm: reach a shelf point, carry a tote M4

Every family ships at least one task with a requirement, labelled example traces (§24.5) and a reference-robot baseline scorecard.

24.4 The task file .ftask S

Format FMT-FTASK, schema frob.task/1. DESIGN: the canonical form is JSON, and its BLAKE3 hash is the TaskId; an S-expression view is lossless and optional, as for designs (§14). A task is data. It has nine sections.

Section Contents Semantics defined in
interface Required ports; observation and action layouts with units, frames and rates §23
clocks Named clocks, each a rational period in seconds and a phase §24.5, §20
terms Observation, action, reward and termination terms, each with unit, noise, delay and the representation under which it transforms §23
signals The telemetry channels that requirements read: name, unit, channel kind, clock §29
requirements Atoms with scales, online signals, the verdict formula, labelled examples §24.5
randomization Draws, each traced to an uncertainty on a card and bound to a named random stream §15, §23
reset The settled-pose cache specification §24.7
world Scene family or scene set, with its split §19
study Default study type, effect margin and budget §24.12

The requirement section in the S-expression view has this grammar. The JSON form is the same tree.

Requirements ::= "(requirements" Atom+ Online* Verdict Example* ")"
Atom      ::= "(atom" Ident Cmp (":scale" Qty | ":exact") ")"
Cmp       ::= "(" (">" | ">=" | "<" | "<=" | "=") Expr Expr ")"
Expr      ::= Signal | Qty | "(" ("+" | "-" | "*" | "min" | "max" | "abs") Expr+ ")" | "(delay" Qty Signal ")"
Past      ::= Ident | "(and" Past+ ")" | "(or" Past+ ")" | "(not" Past ")"
            | "(H" Win Past ")" | "(O" Win Past ")" | "(S" Win Past Past ")"
Future    ::= Past | "(and" Future+ ")" | "(or" Future+ ")" | "(not" Future ")"
            | "(G" Win Future ")" | "(F" Win Future ")" | "(U" Win Future Future ")"
Win       ::= "(" Qty Qty (":round" ("inward" | "outward"))? ")"
Online    ::= "(online" Ident Past ")"
Verdict   ::= "(verdict" Ident Future ")"
Example   ::= "(example" TraceRef ("satisfied" | "violated") ")"

H, O and S are the past-time operators historically, once and since. G, F and U are the future-time operators always, eventually and until. A window is a closed interval of simulated time.

24.5 The requirement language S

Signals and atoms. A signal is a telemetry channel with a unit, a channel kind and a clock (§29). An atom is a comparison of two expressions of the same dimension, together with a scale s of that dimension. The unit checker rejects an expression that adds or compares different dimensions and an atom that has no scale. An atom over an integer-valued signal, such as a count of chassis strikes, is declared :exact and has margin +1 or −1.

Margins. Every formula has a Boolean value and a margin ρ, a dimensionless number whose sign agrees with the Boolean value away from the boundary. Time below is the tick index of the requirement's clock.

Formula at tick t Margin ρ
Atom with expression e ≥ 0 or e > 0 and scale s e(t) / s
¬φ −ρ(φ, t)
φ ∧ ψ ; φ ∨ ψ min ; max of the two margins
G_[a,b] φ ; F_[a,b] φ min ; max of ρ(φ, t′) over t′ in [t + a, t + b]
φ U_[a,b] ψ max over t′ in [t + a, t + b] of min( ρ(ψ, t′), min of ρ(φ, t″) over t″ in [t, t′) )
H_[a,b] φ ; O_[a,b] φ min ; max of ρ(φ, t′) over t′ in [t − b, t − a]
φ S_[a,b] ψ max over t′ in [t − b, t − a] of min( ρ(ψ, t′), min of ρ(φ, t″) over t″ in (t′, t] )

Two properties are tested as laws (PO-ARENA-04): ρ > 0 implies the formula holds, and ρ < 0 implies it fails. At ρ = 0 the margin decides nothing.

Counterexample for composition without scales. A clearance margin of 0.02 m and a thermal margin of 3 K compose to a raw minimum of 0.02, so clearance binds. Express the clearance in millimetres and the raw minimum is 3, so the thermal margin binds. With scales of 0.10 m and 10 K the margins are 0.2 and 0.3 in either unit system, and clearance binds in both. The binding atom of a requirement is what an agent acts on, so it must not depend on the unit system (the metamorphic relation on unit scaling, §38).

Clocks. DESIGN: a clock is a rational period p/q seconds and a phase, advanced by an exact accumulator (§20). A task names its clocks, for example physics at 1/500 s, an inertial sensor at 1/500 s, a policy at 1/50 s, a lidar at 1/10 s and a camera at 1/30 s; the camera is not an integer multiple of the physics period and needs no special case. Each requirement names the clock it is evaluated on. Three rules remove implicit resampling.

  1. A signal on a faster clock enters a requirement through a declared reduction over each requirement tick: last, min or max. Channels of kind interval extremum, such as the minimum clearance swept during a step, are reduced by the matching extremum. A missing declaration is refused (FROB-REF-525).
  2. A signal on a slower clock is held, and its sample age is itself a signal that a requirement may bound. A requirement about what the robot knew reads delivered packets with their timestamps from Sensorium; a requirement about what happened reads simulation state at committed physics ticks. The two are different signals with different names.
  3. A window [a, b] denotes the ticks j with a ≤ j·T ≤ b, compared in exact rational arithmetic. A bound that is not a multiple of the period T is refused unless the window declares outward or inward rounding (FROB-REF-538). Ticks that coincide with another clock's boundary follow the ordering rule of §20.

A verdict on a sampled trace says nothing about the time between samples. A requirement on a state-kind channel is therefore labelled sampled in its verdict, and one on an interval-extremum channel is labelled interval.

Online evaluation. S Past-time formulas compile into kernels that run in the last stage of the batched step (§16), beside observations and rewards. The kernel for a sliding minimum or maximum over a window of W ticks uses the block decomposition that keeps a prefix and a suffix extremum per block of W samples: fixed memory of 2·W values per operator node and world, at most three comparisons per sample, and no data-dependent control flow, which the lockstep rule of §16 requires. The referee uses the monotonic-deque formulation of the same function. DESIGN: minimum, maximum and negation are exact in floating point, so both formulations return identical bits for identical atom values on every backend, after negative zero is canonicalized and a NaN marks the world Faulted. The capacity of an online window is 4,096 requirement ticks (TARGET); a longer window is offline only (FROB-REF-528). An online signal has three uses: an input to early termination, a score for splitting in §26, and an optional added reward term (§24.6).

Offline evaluation. S The verdict formula is evaluated on the completed trace by a two-pass evaluator on the CPU. The reach of a formula, the sum of its nested window ends, is computed statically; a task whose episode horizon is shorter than the reach of its verdict is refused at admission (FROB-REF-526). A window that the recorded trace does not cover, because the episode was truncated or samples are missing, yields undetermined with the reason, never a value from the part that exists (INV-ARENA-02). A verdict whose truth is already fixed by a prefix, as when an always-clause is violated or an until-clause is met, is reported as decided at the tick that fixed it. That is how an environment ends an episode on success or on a hard violation without truncating a window.

The boundary. S DESIGN: an atom is decided on the device only when its normalized margin lies outside a guard band of 2⁻²¹ (TARGET) around zero. A world with a verdict atom inside the band is flagged boundary and that atom is re-evaluated on the CPU with its declared comparator, in exact rational or integer arithmetic where the atom's inputs belong to the exact-integer kernel class of §35 and in double precision otherwise. FACT (planning archive, boundary-classifier runs on this Mac's Metal GPU and on both RTX 4090 cards; Appendix H): on a dense 65,536-case boundary screen, unguarded single-precision strict comparison gave 452 false admissions and 23,438 false exclusions; with a 2⁻²¹ guard and an exact fallback there were no final errors; an exact 32-bit integer kernel classified all 65,536 cases correctly with identical bytes on every device. The exact scalar docking mission of §25 supplies the smallest case in which the comparator matters: at command limit U = 3/4 and read error ε = 5/16 the separation between the two worlds' readings is exactly zero. The strict requirement correctly fails. A non-strict comparator would admit a mission that cannot be completed, because both worlds can then report 15/16 while needing different next commands.

The verdict. The mission-level verdict is an object with schema frob.verdict/1, stored with the episode in .frun and summarized in the ledger. It is the unit that scorecards, studies and searches count.

{ "schema": "frob.verdict/1", "task": "t:cart-sill-dock@6be1…", "requirement": "docked",
  "world": {"run_seed": 17, "world": 41, "episode": 0}, "model": "m:9f2c…", "controller": "c:5e02…",
  "verdict": "violated", "margin": -0.31, "binding_atom": "cleared", "binding_time_s": "8",
  "decided_at_s": "8", "reason": "stuck", "coverage": {"kind": "interval", "missing_samples": 0},
  "boundary": {"flagged": false, "guard": "2^-21"}, "atoms_min": {"cleared": -0.31, "upright": 0.84},
  "evidence": {"class": "Tested", "tier": "batch-cpu", "determinism": "D2", "standing": "provisional"},
  "authorization": {"execution_authorized": false, "hardware_safety_signoff": false,
                       "sim_to_real_certified": false},
  "trace_digest": "b3:77a0…" }

A verdict takes one of four values: satisfied, violated, undetermined and invalid. invalid records a numerical fault; it is a study error, excluded from policy updates and retained in every benchmark and failure count. The verdict is a pure function of the trace and the TaskId (INV-ARENA-01).

Stages. A mission with several stages is one requirement over one continuous episode, with the stages chained by until-clauses. Concatenating separately recorded stage episodes proves nothing about the mission, because the end state of one need not lie in the entry set of the next. Stage pass rates are reported jointly. Counterexample: among 20 equally likely worlds two stages each pass in 19. If the failing worlds differ, 18 of 20 missions pass; if they coincide, 19 of 20; the product 0.95² = 0.9025 is neither.

Checking a requirement someone wrote. S FACT (Appendix H): a 2023 language model translated natural-language statements into temporal-logic formulas with 77.7% accuracy over 300 samples. A requirement is therefore admitted only with labelled example traces: at least two satisfying and two violating, with one near the boundary of each comparator (DESIGN). task.check runs the monitors on the examples and refuses a requirement whose labels disagree (FROB-REF-539). It also runs a mutation check: reversing a comparator or deleting an atom must change the label of at least one example. An atom that no example can distinguish is reported as vacuous, and a vacuous hard-limit atom is a refusal.

24.6 Requirements and rewards

Use What is read Owner
Acceptance of an episode The offline verdict and its margin Arena
Falsification score The minimum margin of the verdict over an episode, minimized over worlds Nemesis (§26)
Splitting score for rare failures An online past-time margin Nemesis (§26)
Outcome flipped by an intervention The verdict Autopsy (§27)
Early termination The decided-early flag of the verdict; hard-limit atoms Cortex (§23)
Training signal Reward terms, integrated over simulated time Cortex (§23)
Optional added reward term An online past-time margin, or timed subgoals and invariants derived from the verdict formula Cortex (§23), F
Runtime monitors Hard-limit atoms shared with the shields Reflex (§22)

Hard limits are frozen separately from shaping rewards. A curriculum may change training distributions and reward weights; it may not change an atom, a scale or a window, because that changes the TaskId. The added reward term is off by default. It is a Frontier option with a pilot and a kill gate (§24.17). The ceiling of its benefit is the gap in task success between a shaped reward and the same reward plus the term; the nearest published figure is the 31.6% of ADR-ARENA-01, measured on other tasks against weaker baselines, and no effect on decision time is claimed.

24.7 The settled-pose reset cache S

An episode that starts with a foot 2 mm inside the floor begins with an impulse that no robot would feel, and a paired comparison in which one design starts settled and the other does not compares spawn artifacts. DESIGN: episodes start from precomputed quasi-static poses.

  • Contents. For each (ModelId, spawn specification, contact law, kernel versions) the cache holds K = 64 settled states (TARGET): configuration, contact warm starts, actuator states at rest and estimator initial state. The key is a content hash, so the cache is a node of the memo graph of §28 and a design edit that changes mass, geometry or contact surfaces invalidates it.
  • Settling. States are settled in double precision on batch-cpu until base and joint speeds stay below 10⁻³ in SI units for 0.2 s with maximum penetration at most 0.5 mm (TARGET). The first contact impulse after loading a cached state on any tier must lie within 5% of the static support force (TARGET).
  • Selection. The pose index is drawn from a counter-based stream keyed by the world address, so two designs paired on one address start from the same spawn specification, each in its own settled state. The initial equilibrium is recomputed for each design; a state is never transported from one design to another.
  • Failure. A world that does not meet the settle criterion on its tier is marked Degraded: unsettled and counted. A task whose cache is missing for the requested model is refused (FROB-REF-529).

24.8 Metrics and scorecards S

Every metric has a unit and a definition on file. The core set:

Metric Definition
Tracking error Root-mean-square error between commanded and achieved planar velocity and yaw rate
Success, fall and collision rates Per the task's verdict and termination rules, with the failure taxonomy of §29
Requirement margin The margin of the verdict formula for the episode, with the binding atom and its time
Time to verdict Simulated time at which the verdict was decided. Non-attainment is right-censored at the horizon and summaries use the restricted mean
Cost of transport Energy divided by (mass × g × distance), reported twice: mechanical work at the joints, and electrical energy from the battery
Energy per completed task Reported twice: mean energy conditional on completion, and total energy of all attempts divided by the number of completions, which is undefined at zero completions
Froude number v² / (g × leg length), so gaits are compared at matched dynamic similarity
Actuator utilization Fractions of time at the torque, speed and thermal limits, per joint
Friction utilization The distribution of ‖γₜ‖ / (μ·γₙ) over stance
Slip distance Tangential travel of a contact while in stance, computed as the time integral of the normal-load-weighted mean tangential speed over the contact quadrature, zero below a declared normal-force floor
Clearance Minimum distance between non-contact links and the scene
Tail robustness The 10% conditional value at risk of the task score over the randomization
Maneuver cost Path length and time for the tight-space tasks, against A-07's prediction
Fallback count Controller fallbacks taken, by kind; every fallback is recorded (§22)

The two energy figures can order designs oppositely. Design A with one 10 J success and one 20 J failure, against design B with two 12 J successes, gives 10 J against 12 J conditional on completion and 30 J against 12 J per completion. Both are published. The slip metric must be invariant to refinement of the quadrature: duplicating a quadrature point and halving the two weights leaves it unchanged (a known answer of §24.16). A quantile is reported only with the sample count that could have observed it: seeing one value above the population's 99th percentile with 95% probability takes 299 independent episodes, and five episodes leave a 95.1% chance that even the largest lies below it.

A scorecard is a table of metrics by scene class with intervals, normalized against the reference robot of the same class. No single scalar is reported unless the task defines one. DESIGN: every scenario of a battery lands in exactly one outcome column (INV-ARENA-03).

Column Meaning
satisfied, violated Verdicts, the second broken down by failure class
undetermined The trace did not cover a window; the reason is listed
invalid Numerical fault. Counted as a study error and as a failure of the tier, never dropped
infeasible-for-candidate A checked necessary condition shows that this body cannot satisfy the requirement in this world (§26.5); the witness is attached
infeasible-for-class The same for every body of the declared class; evidence that the scenario should leave the suite by a recorded decision, not by attrition
refused The run was not admitted; the refusal code is listed

Success rates are printed with two denominators, all scenarios and scenarios not shown infeasible, side by side. Removing a scenario from a battery is a recorded event with a reason and changes the battery's identity.

24.9 Batteries and the design corpus

  • smoke: minutes; every template and reference robot on a handful of scenarios. Runs on every change.
  • fixtures: the analytic layers of the fixtures of §24.10 and §24.11 and the monitor known answers. Runs on every change that touches contact, actuators, clocks or frob-spec.
  • standard: the fidelity battery of §38, about 200 scenarios per robot class.
  • full: every task family across every archetype with the full randomization.
  • hard: the hard-world set of §26 for the robot class. It is discovery data and is never a confirmation sample.
  • covering: a scene set chosen by the coverage selection of §26.11, with one slot in five drawn at random from held-out scenes.
  • The design corpus for BET-05 and BET-06: about 200 designs made of templates, parameter perturbations of them, and designs with planted defects of known kind (an undersized knee actuator, an inertia that breaks static stability, a wheelbase too long for an aisle). Labels are fixed before any analysis is tuned. The corpus also holds the planted winners used to audit screening (§26.10) and the planted single causes used to test attribution (§27.16).

24.10 Fixture 1: a loaded cart crosses a sill and docks S

Question. For a fixed motor and a current-limited cart, does a 75 mm front wheel improve loaded delivery across a 12 mm door sill relative to a 60 mm wheel? The larger wheel has a better lever at the lip and a smaller tread force for the same torque; the fixture exists to find which effect dominates once load distribution, motor lag, impact and momentum are included. Everything below is DESIGN: a synthetic fixture with explicit values, registered as fixture:cart-sill-dock, so that an implementer can build it and a result can falsify it. Measured profiles replace synthetic ones only under a new fixture identity.

Bodies. Chassis frame: x forward, y left, z up, origin midway between the axles on the axle line.

Part Value
Axles and wheels Axles at x = ±0.16 m (wheelbase L = 0.32 m); wheel centres at y = ±0.12 m; front-wheel drive, rear wheels passive
Chassis Uniform box 0.40 × 0.20 × 0.08 m, 5.0 kg, centred 50 mm above the axle line, so it overhangs each axle by 40 mm
Payload Uniform box 0.18 × 0.14 × 0.08 m, 2.2 kg, bolted to the chassis top face with its bottom flush; centred at x = δ, where δ is the fore-aft offset of the condition
Wheels Uniform cylinders 30 mm wide. Rear pair: radius 60 mm, 0.2 kg each, in both designs
Design A Front pair: radius 60 mm, 0.2 kg each, spin inertia 3.6 × 10⁻⁴ kg·m². Total mass 8.000 kg
Design B Front pair: radius 75 mm at the same density and width, so 0.3125 kg each (0.1125 kg more) and spin inertia 8.7890625 × 10⁻⁴ kg·m², a ratio of (75/60)⁴ = 625/256. Total mass 8.225 kg. The front axle rides 15 mm higher, pitching the chassis up by asin(15/320) = 2.69°
Rotor Additional reflected rotor inertia per driven wheel: zero in the primary fixture

Drive. One motor per front wheel, on the wheel shaft.

Quantity Value
Supply Ideal regulated 24 V bus for the 8 s trial; a lossless averaged four-quadrant bridge that absorbs regeneration
Motor Torque constant 0.2 N·m/A, back-EMF constant 0.2 V·s/rad, resistance 1 Ω, inductance 10 mH, current limit ±6 A, gear ratio 1
Initial state Zero current; winding and pack held at 20 °C
Shaft loss Viscous damping 0.002 N·m·s/rad on each of the four wheel shafts
Outer loop, 100 Hz Wheel-rate target 0.30 m/s divided by the loaded radius: 5.0 rad/s for A, 4.0 rad/s for B. Current command 2 A·s/rad times the rate error, clipped to ±6 A, held for 10 ms
Inner driver, continuous Winding voltage v = R·i + k_e·ω + 4 Ω·(i_ref − i), clipped to ±24 V. At +6 A, if that voltage would raise the current, v = R·i + k_e·ω holds the limit when available, and symmetrically at −6 A. A limit crossing is located inside the step. If no admissible voltage keeps the winding inside its rating, a physical limit violation is recorded
Power ledger Net DC power is Σ v_j·i_j over the two drives; bus current is that sum divided by 24 V; positive and negative parts are integrated separately

World and rules.

Item Value
Floor and sill Level floor. A sill spans x from 0 to 0.10 m across the full lane width, 0.012 m high, in two strata: sharp upper edges, or 2 mm fillets on both upper edges
Contact Coulomb friction μ, one value for all wheel–floor and wheel–sill contacts of a condition, 0.8 nominal; inelastic impact; g = 9.81 m/s²
Start Stationary, centred and parallel to the lane, front axle at x = −0.50 m, settled under gravity from the reset cache of §24.7, separately for each design
Clocks Physics 1/2000 s; outer loop and requirement clock 1/100 s; atoms are interval extrema over the 20 physics ticks of each requirement tick
Docked The minimum world x of the whole loaded collision envelope exceeds 0.10 m, and roll and pitch stay within 5°, continuously for 0.5 s, with the dwell complete within 8 s of the start. The dwell restarts whenever a condition lapses
Hard limits Roll or pitch beyond 35°; any chassis contact with the sill; winding current beyond the rating by more than 0.01 A. Each is absorbing
Stall The command is saturated and the cart advances less than 2 mm over 0.5 s. A stall names the failure reason; the verdict still comes from the requirement at the horizon
Horizon 8 simulated seconds. Timeout, rollover, strike and stall each carry a reason

The requirement, in the S-expression view:

(requirements
  (atom cleared    (>  (- env_min_x 0.10 m) 0 m)                    :scale 0.10 m)
  (atom upright    (>= (- 5 deg tilt) 0 deg)                        :scale 5 deg)
  (atom tilt-ok    (>= (- 35 deg tilt) 0 deg)                       :scale 35 deg)
  (atom current-ok (>= (- 6.01 A i_max) 0 A)                        :scale 6 A)
  (atom no-strike  (=  strikes 0)                                   :exact)
  (atom saturated  (=  drive_saturated 1)                           :exact)
  (atom advancing  (>= (- x_front (delay 0.5 s x_front)) 0.002 m)   :scale 0.002 m)
  (online dwell    (H (0 s 0.5 s) (and cleared upright)))
  (online stalled  (and (H (0 s 0.5 s) saturated) (not advancing)))
  (verdict docked  (U (0 s 8 s) (and tilt-ok current-ok no-strike) dwell)))

env_min_x is the rearmost point of the loaded collision envelope, tilt the larger of |roll| and |pitch|, i_max the larger winding current magnitude, each an interval extremum over a requirement tick.

The until-form says that the hard limits hold at every tick before the dwell completes, and that it completes by 8 s. Written this way the dwell cannot latch: H over 0.5 s is false the tick any condition lapses. FACT (planning archive, sill pilot audit; Appendix H): the hand-written scorer of an earlier planar pilot of this fixture had a latched dwell timer and no chassis-strike guard. Both defects are unrepresentable here.

Energy. Net bus energy is integrated from the start to adjudication for every trial and reported with duration, gross draw and returned energy. The two per-completion figures of §24.8 are both published. Energy is descriptive in version 1 of the fixture and takes no part in the rule.

The analytic layer. These rows are known answers, each a limiting rig that runs before the full cart. Symbols: r wheel radius, h lip height, x = √(2rh − h²) the horizontal distance from axle to lip at lift-off, tan θ = x/(r − h), F₀ the static load on the climbing axle on level ground, W the total weight. The rigid, quasi-static derivations are on file under docs/derivations/fixture-cart-sill-dock/.

Row Statement Value
Mass roll-up Total mass; front-wheel mass and spin-inertia ratio of B to A 8.000 kg and 8.225 kg; 1.5625 and 625/256
Tread force cap Two wheels at the current limit: 2·k_t·I_max/r 40 N for A, 32 N for B
Force slope Unsaturated flat rolling at settled current: 2·k_t·K_ω/r² per unit speed error 222.22 and 142.22 N/(m/s). A gain of 3.125 A·s/rad for B matches the slopes
Lip lever x for a sharp lip of 12 mm 36.000 mm at r = 60 mm; 40.694 mm at r = 75 mm
Front lift-off, creep limit Driven axle at the lip, trailing axle free on the floor. The lip force is vertical, so the friction demand is μ ≥ tan θ and the total drive torque is F₀·x/(1 + x/L) μ ≥ 0.7500 for A and μ ≥ 0.6459 for B. With 2 mm fillets, 0.7332 and 0.6351. Torque for A at δ = 0: 1.270 N·m of 2.4 N·m available
Rear pull-over, creep limit Free axle at the lip, driven axle on the floor. The lip force passes through the free axle, so its horizontal part is V·tan θ with V = W·d/(L − x − h·tan θ), d the horizontal distance from the driven contact to the mass centre. The driven axle must supply that traction For A, V/W = 32/55 − δ with δ in metres, and the friction demand is ¾·(32/55 − δ)/(23/55 + δ): 24/23 = 1.043 at δ = 0, 0.887 at δ = +0.04 m. For B at δ = 0 the demand of 34.2 N exceeds the 32 N force cap
Creep-feasible set for A Both axles μ ≥ ¾·(32/55 − δ)/(23/55 + δ), which lies inside the admitted friction range only for δ ≥ 23.0 mm: 1.13% of the development distribution below
Energy Raising the front axle of A by h takes F₀·h; the cart's kinetic energy at 0.30 m/s is ½·m·v² plus wheel spin 0.471 J against 0.360 J + 0.018 J. Momentum alone does not carry even one axle over
Wheel-loop stability The 75 mm wheel with 0.0002 kg·m² added rotor inertia (0.00107890625 kg·m² in all), slipping against a constant torque, rate sampled and current target held for 10 ms: spectral radius of the one-tick map 1.4820 with the feedforward term R·i_ref; 0.9207 with R·i as specified above. An instantaneous-current approximation gives 2.6917 for both (2.7075 if the shaft damping is left out)
Power ledger A blocked winding at 3 A and 3 V; a drive at 50 rad/s and −2 A 9 W dissipated, 0.375 A from the bus; v = 8 V, 16 W returned, 20 W from the shaft, 4 W in copper

The creep limit is run as a rig at a commanded speed of 0.02 m/s. For the front-axle rig the outcome must flip between the friction grid points that bracket the threshold, on a grid of spacing 0.01 that excludes points within 0.005 of the threshold on the referee and within 0.01 on the fast tiers (TARGET). A run exactly at the threshold must carry the boundary flag. The wheel-loop row is also a mutation test for the actuator stage: at a 0.5 ms step, explicit Euler that updates the current before the wheel speed gives 1.854 and 1.497, which turns the stable case into an unstable one.

What the analytic layer says about the mission: in the creep limit design A docks in about one condition in ninety, design B in fewer still because its 32 N force cap binds at the rear lip (the derivation file carries B's pitched geometry), and for both the rear axle binds, not the front. Whatever difference the batch shows between the designs is therefore produced by momentum, impact and the transient of the drive, which is the reason a static screen cannot settle this question. FACT (planning archive, planar sill pilot; Appendix H): in a planar pilot of 156 design-condition rows, 62 passed the dynamic scorer and failed a quasi-static screen, 7 passed both and 87 failed both; no row passed the static screen alone. Changing only the feedforward term from R·i_ref to R·i changed 24 of the 156 labels, taking the 60 mm design from 8 to 18 successes of 78 and the 75 mm design from 61 to 67. The pilot is exploratory and keeps its own law and scorer.

The development batch. 64 independent conditions from master seed 17, on counter-based streams keyed by the world address (17, k, 0) for k = 0 … 63 and by stream name:

Stream Draw
fixture1/mu Friction, uniform on [0.35, 0.95]
fixture1/payload_dx Payload fore-aft offset δ, uniform on [−0.04, 0.04] m
fixture1/heading Initial heading, uniform on [−2°, 2°]
fixture1/edge Sharp or filleted lip, a fair Bernoulli draw

Both designs receive every condition. The 64-row input table, the generator pin, the mapping from generator output to each draw, and the hash of this section's constants are written to the ledger before either design runs (INV-ARENA-04). The nominal analytic case is separate from the batch. These are synthetic challenge distributions and claim nothing about the frequency of sills.

The rule. The study is fixed-n (§24.12). Let n be the number of complete pairs, b the pairs in which only B docks and a the pairs in which only A docks. The effect estimate is (b − a)/n. Form two-sided 97.5% exact Clopper–Pearson intervals [L_b, U_b] and [L_a, U_a] for the two discordance probabilities; the interval [L_b − U_a, U_b − L_a] then covers their difference with probability at least 95%.

Verdict Condition
Prefer B L_b − U_a > +0.05, and B violates no hard limit in a condition where A does not
B inferior U_b − L_a < −0.05
Practical tie The interval lies inside [−0.05, +0.05]
Inconclusive Otherwise

Both-succeed and neither-succeed counts, invalid pairs and the worst-case range for missing pairs are reported, with unresolved counts given per arm and per action regime, because a policy can enter states where the solver fails preferentially: a candidate with 2 successes in 2 valid trials and 98 unresolved has a complete-case rate of 100% and a success rate anywhere in [2%, 100%], against a baseline at 60 of 100, and its unresolved worlds are re-run on the referee before a ranking that depends on them is promoted. A preference needs complete pairs. If an apparent improvement reverses under a halved step, a refined contact feature or the alternative contact law, the result is reported as fidelity-dependent and both runs are kept.

The pre-analysis. The rule is enumerated before any pair runs. Every count pair (a, b) with a + b ≤ n is classified; the table is regenerated by eval.preanalyze and compared in CI.

n Count pairs Prefer B B inferior Practical tie Inconclusive
64 2,145 521 521 0 1,103
86 3,828 1,031 1,031 1 1,765
128 8,385 2,517 2,517 4 3,347
256 33,153 11,384 11,384 39 10,346

Three consequences follow. First, no outcome at n = 64 can establish a practical tie: perfect concordance gives [−0.0662, +0.0662]. A tie first becomes reachable at n = 86 and only for (a, b) = (0, 0); with 5% discordance each way the chance of that outcome is 0.9⁸⁶ = 0.0116%. Second, with no A-only pair a preference for B needs at least 15 B-only pairs of 64, and 17, 19, 21 and 23 with one to four A-only pairs. Third, the batch has little power for a modest effect. Each alternative below is a pair of probabilities, of a B-only pair and of an A-only pair:

n P(prefer B) at discordance (0.17, 0) at (0.25, 0.02) at (0.40, 0.02) P(tie) at (0.02, 0.02) P(tie) at (0, 0)
64 0.117 0.406 0.962 0 0
128 0.775 0.906 above 0.999 0.072 1
256 0.998 above 0.999 above 0.999 0.427 1

The pair (0.17, 0) is the share of the friction range in which only the 75 mm wheel meets the front lift-off threshold of the analytic layer; it is a reference effect size, not a prediction. The 64 conditions are therefore a development batch: they can prefer a design or call it inferior, they cannot declare a tie, and they are not used to tune anything that the holdout then judges. The holdout is sized from this table for the claim that is wanted, runs through eval.qualify on sealed conditions (§24.13), and uses the level of its request. A claim of equivalence may instead register the standalone test that bounds the difference by the total discordance probability: zero discordant pairs in 64 give an exact one-sided 95% bound of 1 − 0.05^(1/64) = 0.0457, inside the five-point band. The inference is registered before the holdout is opened, and the development batch keeps its original analysis.

Mechanism controls. Two registered follow-ups separate the mechanisms. One sets B's gain to 3.125 A·s/rad to match the force slopes, leaving the caps at 40 N and 32 N. The other is a 2 × 2 of rotor inertia (zero or 0.0002 kg·m²) and control cadence (100 Hz or 1 kHz). FACT (planning archive; Appendix H): in an exploratory planar sweep the two designs' success fractions were 0.093 and 0.780 at 100 Hz with zero rotor inertia and 1.000 and 1.000 at 1 kHz or with added inertia. A comparison that leaves rotor inertia and controller rate unpinned is a comparison of loop stability.

24.11 Fixture 2: a quadruped on a split-grip landing S

Question. Does a wider sole help a quadruped arrive on a landing whose surface has a low-grip strip under one side? On a uniform Coulomb surface a wider foot has no more translational traction at the same normal load; the proposed advantage is a change in the share of the sole that rests on the high-grip side of a material boundary. The fixture, registered as fixture:quad-split-grip, is built so that the limiting model predicts a reversal, which a count of contact area cannot produce.

Item Value (DESIGN)
Body The reference 12-actuator quadruped template of §14 at its pinned DesignId, with identical actuator limits, equal limb masses and a symmetric nominal stance
Controller One crawl controller from Reflex (§22), frozen before either sole is tested
Sole A, sole B Rectangular plates 80 × 40 mm and 80 × 70 mm, the 80 mm side along the direction of travel, on all four feet; the same sole law, sole height and ankle origin; mass and spatial inertia matched by a declared insert. Mounting and leg interference are checked for each, and each design is settled independently
Course A ramp 1 m long rising at 6°, then a level landing 1.2 m long
Surfaces Friction 0.8 everywhere except a longitudinal strip of friction 0.25 on the landing, 0.20 m wide, whose inner boundary runs through the centres of the left soles in the nominal straight gait and whose outer edge lies outside both footprints throughout
Command 0.25 m/s along the ramp from rest; stand once all feet have cleared the crest by 0.40 m
Pass Standing reached within 12 s and held for 1 s; no body strike; roll and pitch below 30°; less than 50 mm of slip per foot over its loaded stance, by the slip metric of §24.8 with a 1 N floor
Cases All-dry, split-grip, and split-grip reflected to the right side
Development set Nominal strip position and offsets of ±5 mm, centred payload
Holdout Ramp at 8°; strip offsets ±12 mm; initial yaw ±2°; lateral payload shift ±20 mm; the full factorial of eight cells with its eight reflected counterparts, and the four dry cells
Contact laws Distributed rigid Coulomb contact as the baseline; then a finer distributed compliant sole with a declared pressure and shear law, with the step halved and the footprint quadrature refined until the classification stops changing

The limiting model. For uniform normal pressure, the share of a sole of width w that rests on the high-grip side when its centre is displaced by z into that side is f = clip(½ + z/w, 0, 1). Its longitudinal friction capacity per unit normal load is c(z, w) = 0.8·f + 0.25·(1 − f) = 0.525 + 0.55·clip(z/w, −½, ½), and at the slip limit the plate carries a yaw moment of 0.275·N·(w/4 − z²/w) about its centre for |z| ≤ w/2.

Offset z Capacity, 40 mm sole Capacity, 70 mm sole Yaw moment per newton, 40 mm ; 70 mm
0 0.525 0.525 2.75 mm ; 4.81 mm
+12 mm 0.690 0.619 1.76 mm ; 4.25 mm
−12 mm 0.360 0.431 1.76 mm ; 4.25 mm

The narrow sole has more capacity when the foot lands toward the high-grip side and less when it lands toward the strip. The two held-out offsets therefore carry opposite predictions, and the wide sole always carries the larger yaw moment. Area alone cannot predict traction when pressure redistributes; the actual load split and moment are measured under each contact law.

Executable known answers. A plate-drag rig realizes the limiting model: one sole plate on a compliant layer under a centred normal load, guided against yaw, pulled along the boundary until it slips. The pull force at slip must equal c(z, w)·N and the guide moment the value above. The slip metric is checked for invariance under duplication of a quadrature point. The reflected cases are a metamorphic relation: the mirrored robot on the mirrored course returns the mirrored result, inside its determinism class.

Decision rule. Results are counts on a finite suite. Reflected pairs are related and are reported as pairs. A sole is preferred only if it completes more held-out cases under both contact laws with no new hard-limit failure. A reversal between the laws is reported as a fidelity-dependent ranking, which is a finding about the tiers and not about the soles. Distance, work, worst slip, peak torque and falls are published for every design and condition. A comparison with a learned residual on top of the frozen controller is a later study with equal training resources.

24.12 Two study types S

A study is a registered plan to answer one comparative or absolute question. It declares its type when it is created and keeps it (INV-ARENA-05).

Anytime-valid study Fixed-n exact study
Inference A betting e-process or a confidence sequence (C-15), valid at every stopping time An exact interval or test at a sample size fixed in advance
Stopping When the evidence crosses its threshold or the budget ends; unfinished episodes are cancelled At n complete units, never earlier
Before the first episode Null hypothesis, effect margin, bound on outcomes, betting rule, level, budget n, the rule, the level, and a pre-analysis: which verdicts are reachable at that n, and the power at declared alternatives
Reports The e-value, the confidence sequence, pairs used, the stopping reason Counts, the interval, the verdict, and the pre-analysis hash
Interim information Any; validity does not depend on it Progress only. Outcomes may be looked at, and the verdict is computed once, at n
Typical use Screening, racing, promotion between tiers, bisection Registered A/B questions, equivalence claims, qualification of a finalist

The types are never mixed inside one study. A fixed-n interval computed at the moment an anytime-valid study stops has no coverage guarantee, and a fixed-n study that stops when its running p-value first looks good loses its level. Counterexample for the second: an exact one-sided sign test at nominal level 5% on pairs that favour either design with probability ½, looked at after every pair. After 8 looks the chance of a false rejection is 13/256 = 5.08%; after 16 it is 8.76%, after 32, 12.9%, and after 64, 17.4%. A request for either mixture is refused (FROB-REF-552).

A fixed-n study without a pre-analysis is not admitted, and a study that names a verdict its rule cannot reach at the declared n and level is refused (FROB-REF-532). Fixture 1 shows why: at 64 pairs its rule cannot return one of its four verdicts.

Four rules hold for both types. Units are paired by world address, and when several episodes share a cluster such as one building, the cluster is the unit of inference. Paired statistics are refused under an uncontrolled determinism class (§35). An N-way race spends α/(N − 1) per comparison, so that a null race keeps its best candidate with probability at least 1 − α. And a comparison of designs declares its controller arm: a shared controller isolates the change of body, controllers retuned with equal resources compare the best systems, and a candidate that inherits weights declares its warm start and receives a matched transfer baseline.

24.13 The blinded qualification lane S

An e-process makes one comparison valid under optional stopping. It does nothing about an agent that submits fifty candidates to the same held-out scenes until one passes. With independent attempts at level 5% each, fourteen null submissions give a 51% chance of at least one false promotion, and five hundred give certainty for practical purposes. The qualification lane closes that hole. Its verb is eval.qualify.

Lanes. The discovery lane is everything an agent may read while it designs: training scenes, the development splits, batteries, the hard-world set, its own runs and telemetry. The confirmation lane is a service inside frobd that owns a sealed family of worlds and runs registered requests against it. A campaign is a set of confirmation requests that share a claim family, meaning a task, a baseline class and a held-out world family, and a total error budget α. All agents working on the claim family share the campaign and its index sequence. Opening a campaign needs the capability qual:admin; an agent cannot reset its index by opening another.

The protocol for request i of a campaign.

  1. Commit. The request names candidate, baseline, task, claim, study type and rule, tier plan, number of clusters and budget. It is hashed into the ledger with an idempotency key before any confirmation world exists (INV-QUAL-04). For a fixed-n study the pre-analysis is computed at this request's level and attached.
  2. Spend. The request receives the level α_i = α / [i·(i + 1)] and the ledger records it. The index advances for every committed request, including ones that end inconclusive or are refused after commitment (INV-QUAL-03).
  3. Draw. The service derives fresh world clusters for this request from a sealed key, the campaign identifier, the index i and a cluster counter. A cluster is the unit that shares a generator draw, such as one building with its materials. No cluster serves two requests and none has appeared in any discovery dataset (INV-QUAL-02).
  4. Run. Candidate and baseline run paired on the clusters at the tier plan, with promotion to a higher tier by the rule of §16. The study runs at level α_i with clusters as units.
  5. Answer. The agent receives a receipt and nothing else: the decision, the index, the level spent, the number of units used and a hash.
  6. Close. When the campaign closes, the key is revealed, the clusters and their episodes become readable, and the family is retired into discovery. It never serves as a confirmation family again.
{ "schema": "frob.qualify.request/1", "campaign": "q:threshold-delivery/3", "index": 7,
  "alpha_total": "0.05", "alpha_i": "0.05/56",
  "candidate": {"design": "d:7c03…", "controller": "c:91ab…"},
  "baseline": {"design": "d:41aa…", "controller": "c:91ab…"},
  "task": "t:cart-sill-dock@6be1…", "claim": "candidate docks more often than baseline by more than 0.05",
  "study": {"type": "fixed-n", "n": 256, "rule": "paired-clopper-pearson/1", "preanalysis": "b3:5d10…"},
  "tier_plan": {"run": "batch-cpu", "promote_to": "referee"}, "clusters": 256, "budget": {"cpu_core_s": 90000},
  "commit": {"ledger_seq": "18812", "hash": "b3:e41f…", "idempotency_key": "k:2c9a…"} }
{ "schema": "frob.qualify.receipt/1", "campaign": "q:threshold-delivery/3", "index": 7,
  "alpha_i": "0.05/56", "alpha_spent": "0.05*7/8", "decision": "not-promoted", "units_used": 256,
  "sealed_until_close": ["world addresses", "per-unit outcomes", "telemetry", "stills"],
  "receipt_hash": "b3:90c2…", "authorization": {"execution_authorized": false,
  "hardware_safety_signoff": false, "sim_to_real_certified": false} }

Why the error rate holds. Because 1/[i·(i + 1)] = 1/i − 1/(i + 1), the levels sum to α·m/(m + 1) after m requests and to α in the limit (PO-ARENA-02). Request i tests a candidate that the agent chose with everything it had seen before, on clusters that are independent of everything it had seen before, so conditional on the past the chance of a false promotion at request i is at most α_i. A union bound over requests gives a campaign-wide chance of any false promotion of at most α, for any number of retries and any adaptive strategy. The argument needs three things and the lane enforces exactly those: the request is fixed before the draw, the clusters are fresh, and the level is spent whether or not the agent likes the answer.

What blinding adds. Freshness alone controls the error rate. Blinding protects the claim's population. If an agent could read confirmation worlds, they would inform its next candidate, the held-out family would stop being held out, and a promotion would describe worlds the design was fitted to. So discovery cannot read confirmation worlds (INV-QUAL-01): no verb returns a sealed address, outcome, trace or still to a session without qual:admin before the campaign closes, and an attempt is refused and logged (FROB-REF-840). If a reversal on a sealed family is ever used to inform training or selection, that family is retired at once.

What a retry costs. The sample size of a test grows with ln(1/α_i). Relative to a single test at α = 0.05, request 1 needs about 1.23 times the units, request 10 about 2.57 times, request 100 about 4.08 times and request 500 about 5.15 times. A fixed-n rule changes with the level too: for the rule of fixture 1 a practical tie first becomes reachable at n = 99 for request 1, n = 178 for request 10 and n = 328 for request 500, and at 64 pairs request 1 has 478 preferring outcomes of 2,145 where the development level had 521. Retrying is allowed and is never free.

Standing. A result is qualified only with a receipt whose decision is promoted. Reports cite the campaign, the index and the level. Citing a discovery-lane comparison as a qualification is a protocol refusal (FROB-REF-845). The receipts, the ledger of levels and, after close, the cluster derivation are part of the .frobcert bundle and are replayed by frob-verify (§12).

24.14 Verbs and refusals

Every verb of kind run is a long tool in the sense of §32: it accepts a bounded synchronous wait and otherwise returns a job handle.

Verb Kind First milestone Notes
task.new, task.show, task.list mutate, read M1 Tasks are content-addressed
task.check read M1 Units, clocks, windows, reach against horizon, labelled examples, mutation check
task.verdict read M1 Evaluates the requirements of a task on a recorded run and returns frob.verdict/1
task.settle run M1 Builds or refreshes the settled-pose cache for a model and task
task.fixture run M1 Runs a registered fixture's analytic layer or batch and reports every row against its known answer
eval.run, eval.scorecard run, read M2, M3 Scorecards with the outcome columns of §24.8
eval.compare, eval.promote run M3 Defined in §28 and §16; each belongs to a study of §24.12
eval.preanalyze read M1 Enumerates the reachable verdicts and the power of a fixed-n rule at a given n and level
eval.qualify run M3 Submits a committed request to a campaign and returns a receipt
eval.ledger read M3 The campaign's index, levels spent and receipts; nothing sealed
Code Refusal
FROB-REF-502 An evaluation reuses a scene from the policy's training manifest (carried from §19)
FROB-REF-520 An atom has no scale and is not declared exact
FROB-REF-535 Dimension mismatch inside an atom
FROB-REF-536 A temporal operator has no finite window
FROB-REF-523 A future-time operator appears in an online signal
FROB-REF-538 A window bound is not a multiple of the clock period and no rounding is declared
FROB-REF-525 A signal is read across clocks without a declared reduction, or its channel kind does not admit the reduction
FROB-REF-526 The episode horizon is shorter than the reach of the verdict
FROB-REF-539 A requirement has no labelled examples, or the monitor disagrees with a label
FROB-REF-528 An online window exceeds its capacity
FROB-REF-529 No settled-pose cache for this model and task
FROB-REF-530 A run offered as a fixture run does not match the fixture's recorded hash
FROB-REF-552 A request mixes study types
FROB-REF-532 A fixed-n study has no pre-analysis, or names a verdict it cannot reach
FROB-REF-533 Paired statistics requested under an uncontrolled determinism class
FROB-REF-840 A session without qual:admin asks for sealed confirmation data before close
FROB-REF-841 A confirmation request was not committed first, or differs from its commitment
FROB-REF-842 The units the request needs at its level exceed its budget; the reachable verdicts are attached
FROB-REF-843 The candidate's lineage contains data from the sealed family
FROB-REF-844 A cluster would be used twice
FROB-REF-845 A result is cited as qualified without a promoting receipt

24.15 Invariants and proof obligations

  • INV-ARENA-01. A verdict is a pure function of the trace and the TaskId. It never depends on a reward term.
  • INV-ARENA-02. No requirement value is computed from a truncated or incomplete window.
  • INV-ARENA-03. Every scenario of a battery appears in exactly one outcome column, and the columns sum to the battery size.
  • INV-ARENA-04. A fixture's constants, seeds, input table and rule are hashed into the ledger before its first run.
  • INV-ARENA-05. A study has one type for its whole life.
  • INV-QUAL-01. Discovery cannot read confirmation worlds.
  • INV-QUAL-02. Confirmation clusters are fresh: no cluster serves two requests, and none is in any discovery dataset before the campaign closes.
  • INV-QUAL-03. The ledger of levels is append-only, the index advances by one for every committed request, and the total spent never exceeds α.
  • INV-QUAL-04. A request's commitment precedes the derivation of its clusters in ledger order.
Obligation Statement Class sought
PO-ARENA-01 The online block kernels, the deque monitor and the definitional evaluator agree on every past-time operator Tested: differential, bit for bit
PO-ARENA-02 Σ α/[i·(i + 1)] over i ≥ 1 equals α, and the campaign error is bounded by the sum A candidate theorem card for §12; Tested by exact rational summation until a checker accepts it
PO-ARENA-03 The pre-analysis enumerator is exact: counts are integers, and interval endpoints are rounded outward Tested against an independent enumeration
PO-ARENA-04 A positive margin implies the formula holds and a negative margin that it fails Tested as a law on random traces

24.16 Known answers

Known answer Expected Negative control
Fixture 1, analytic layer Every row of the table in §24.10 within its tolerance The wheel-loop row under explicit current-then-wheel Euler at 0.5 ms; a creep rig with the rear lip removed
Fixture 1, rule enumeration 521, 521, 0 and 1,103 at n = 64; first tie at n = 86 for (0, 0) only; 15 B-only pairs for a preference with no A-only pair A request for a tie verdict at n = 64 is refused
Fixture 1, energy accounting 10 J against 12 J and 30 J against 12 J for the two-trial example Dropping failed attempts from the total
Fixture 2, limiting model The table of §24.11 on the plate-drag rig, within 1% on the referee A pyramidal friction cone; a sole with one quadrature point
Slip metric Unchanged when a quadrature point is duplicated with halved weights A metric that sums unweighted point speeds
Normalization The binding atom of the two-atom example is the same in metres and millimetres Composition of raw margins
Boundary comparator At U = 3/4 and ε = 5/16 the strict docking requirement fails; the 65,536-case screen has no final error with the guard A non-strict comparator admits the equality case; unguarded single-precision comparison misclassifies the screen
Online and offline agreement A past-time signal sampled at the end of the trace equals the offline evaluation of the same formula A monitor that pads a short window
Peeking The sign-test example gives 13/256 after 8 looks The same data under a confidence sequence stay below their level
Joint stage accounting 18 or 19 of 20, never 18.05 Multiplying stage rates
Level series Exact partial sums α·m/(m + 1) for m up to 500 A constant level per request

24.17 Gates, kill gate and pilots

Gate Claim Threshold (TARGET) Earliest
GATE-ARENA-01 Known answer Every task is reproducible inside its determinism class from its TaskId, SceneId, ModelId and seed; reference-robot baseline scorecards are recorded M2
GATE-ARENA-02 Known answer The scene-split check rejects an evaluation that reuses a training scene M3
GATE-ARENA-03 Known answer Monitors: the three evaluators agree bit for bit on 10⁶ random traces per operator, including multi-rate signals and windows with rounding; every requirement-language refusal of §24.14 (FROB-REF-520, -535, -536, -523, -538, -525, -526, -539 and -528) is triggered by its planted task file M1
GATE-ARENA-04 Known answer Fixture 1: every analytic row within tolerance on the referee and inside the fidelity envelope on the batch tiers; the creep rigs flip at the stated thresholds; both negative controls are caught M1 on the referee; M2 on the batch tiers
GATE-ARENA-05 Known answer eval.preanalyze reproduces the tables of §24.10 exactly and refuses the unreachable verdict M1
GATE-ARENA-06 Known answer Fixture 2: limiting-model rows within 1% on the referee; slip invariance to 10⁻¹² relative in double precision; reflected cases mirror inside the determinism class M2
GATE-ARENA-07 Known answer Scorecard accounting: a planted infeasible scenario appears in its column and in the all-scenario denominator; columns sum to the battery size on 100 random batteries; a silently shortened battery changes identity M3
GATE-ARENA-08 Known answer Reset cache: penetration at most 0.5 mm and first impulse within 5% of static support on all 64 states for each reference robot; a design edit invalidates the cache M2
GATE-ARENA-09 Design comparison Requirement writing: on 50 scripted mission statements, each with a reference formula and 20 labelled trajectories the agent does not see, at most 20% of first-draft requirements mislabel a trajectory, and task.check rejects every draft that does M3
GATE-QUAL-01 Known answer Over 2,000 simulated campaigns on no-signal worlds at α = 0.05, each with 500 adaptive retries by a scripted adversary that sees every receipt and all discovery data, a false promotion occurs in at most 123 campaigns (an exact one-sided test of a rate at or below 0.05 rejects at the 1% level from 124). A planted true gain of the declared size is promoted within 10 requests in at least 80% of campaigns. Negative control: the same adversary against reused clusters at a constant level obtains a false promotion in more than 99% of campaigns M3
GATE-QUAL-02 Known answer Firewall: 10⁴ fuzzed read attempts across every verb return no sealed field; a planted leak path is found by the audit; receipts contain only the listed fields M3
GATE-QUAL-03 Known answer frob-verify replays the levels, the commitments and, after close, the cluster derivation of a campaign; a reordered index, a changed request and a reused cluster are each rejected M5
✖ KILL-SPEC-REWARD — Trigger: in the training pilot below, the arm with the added term does not improve success over the shaped reward across five seeds. Fallback: requirements stay acceptance metrics and search scores only, and the added-term option is removed from the task schema. The margin-only arm is run to record whether it reaches 80% of the shaped arm's success; below that, margin-only training is not offered M3

Pilots before funding. The requirement-writing pilot is GATE-ARENA-09 run on paper: one day, no GPU. The training pilot for the Frontier reward term is one quadruped velocity-tracking task under three arms, shaped reward, smoothed margin only, and both, with five seeds each, about ten GPU-hours on a development oracle outside the product (§38). The first fixture needs no pilot; its analytic layer is the pilot.

24.18 Open questions

  • OPEN-ARENA-1. Which public task definitions to mirror for comparability. Default: mirror the reward terms of the public velocity-tracking tasks and make no claim of score parity.
  • OPEN-ARENA-2. A permissively licensed hexapod reference model. Default: none; the template stands alone.
  • OPEN-ARENA-3. The size and composition of the sealed world families, and who holds qual:admin. Owner: project lead. Blocks GATE-QUAL-01. Default: the owner alone opens and closes campaigns; one sealed family per task family, sized for 20 requests at 256 clusters; no result is called qualified until a family exists.
  • OPEN-ARENA-4. Whether fixture 2's quadruped is the reference template or a dedicated minimal body. Owner: Arena lead. Blocks GATE-ARENA-06. Default: the reference template at a pinned DesignId.
  • OPEN-ARENA-5. The guard band for boundary atoms on each backend. Owner: Spark lead. Blocks the device half of GATE-ARENA-03. Default: 2⁻²¹ with CPU re-evaluation of every flagged atom.

24.19 Traction capacity is a slice of the wrench set at the commanded wrench

What it is. S A metric, and two analytic rows for Fixture 2 (§24.11), used in every comparison of soles, tyres and pads. The scalar friction capacity Σ μᵢ·Nᵢ is the largest longitudinal force with the yaw moment left free. A robot commands a wrench. The capacity that ranks two designs is the largest force along the commanded direction among contact forces that also deliver the commanded moments.

Counterexample. A two-point sole carries 50 N on each point, at lateral offsets ±20 mm, one point on μ = 0.25 and the other on μ = 0.8. The largest forward force is 12.5 + 40 = 52.5 N, and it comes with a yaw moment of (40 − 12.5) N × 0.020 m = 0.55 N·m. If the command is a pure forward force, the two longitudinal forces must be equal, and the largest is 2 × 12.5 = 25 N. A sole on uniform μ = 0.4 supplies 40 N at zero yaw. By scalar capacity the split sole wins, 52.5 N against 40 N. At the commanded wrench it loses, 25 N against 40 N.

Mechanism. DESIGN:

  • traction_at_wrench(direction, moment command) is a cone program over the contact wrench set of C-06: maximize the force along the direction, subject to the per-point friction cones, the measured normal loads and the moment equalities. It is the program of §22.5 with another objective, so it needs no new solver. Owner: frob-control for the program, frob-arena for the metric, frob-analyze for the static margin of A-08.
  • Scorecards (§24.8) report the scalar capacity under the name capacity, moment free, and the slice under the name of the commanded wrench. A comparison of contact geometries cites the slice.
  • Fixture 2 gains two rows. The plate-drag rig runs once guided against yaw, as now, and once free to yaw, which enforces zero moment physically. For the uniform-pressure strip model with the boundary through the centre and longitudinal traction only, the zero-yaw capacity per unit normal load is (μ_L + √(μ_L·μ_H))/2, which is 0.3486 for μ_L = 0.25 and μ_H = 0.8, against 0.525 with the moment free, and it does not depend on the sole width. The closed form is a lower bound when lateral traction may also carry moment; the cone program gives the exact value.

Known answers. Two-point rig: 52.5 N and 0.55 N·m guided; 25 N free; 40 N both ways on uniform μ = 0.4. Strip model at zero offset: 0.525·N guided for both widths; at least 0.3486·N free for both widths.

GATE-ARENA-10 Known answer. The cone program reproduces 52.5 N, 0.55 N·m, 25 N and 40 N to 10⁻⁹ in double precision; the two-point rig slips within 2% of each value on the referee and inside the fidelity envelope on the batch tiers; a scorecard that ranks the two soles by moment-free capacity when the task commands zero yaw is rejected by task.check. M2.

24.20 Auditing a task before any design runs on it: foil bodies and intent invariants

Task audit F; intent invariants S. The detectors of §23.16 ask whether a good score is an artifact of the solver. A behaviour that is physically real and unintended passes every one of them: sliding on the belly where walking was meant, tumbling down a flight to reach the goal region, riding a wall, satisfying the predicate only at the instant it is checked. Faster search finds the cheapest such loophole sooner, and one loophole invalidates every evaluation of the campaign that ran under it. task.audit attacks the task itself before any candidate is scored.

Foil bodies. A library assets/foils/ of bodies that are deliberately over-capable or degenerate, written in the design language and marked foil: true. Each runs in the task's own scene and is scored by the task's own requirement and cost. A foil is never a candidate: offering one to opt.*, eval.compare or eval.qualify is refused.

Foil What it is What it probes
Free centroidal body One rigid body with the mass and inertia of the design class, applying bounded wrenches at the contact sites the task designates, inside friction cones; no limbs, no actuator limits Whether the predicate can be met by forces alone, without the intended mechanism
Relaxed family member The current design with card limits multiplied by 3 (DESIGN) and collision enabled on every link Behaviours the real design is only barely kept from
Degenerate shapes A sled, a ball, a tumbling box and a single-leg hopper of the class's mass Sliding, rolling, tumbling and bouncing routes to a satisfied verdict

Search. The sampling controller of §22.8 and rank-based search over open-loop parameters, filling a quality-diversity archive (§28.8) over behaviour descriptors: the fraction of time each site class touches the ground, the range of base attitude ‖log R‖, the airborne fraction, and the sliding fraction (tangential speed at stance contacts above a card threshold). The objective is many distinct high-scoring behaviours, not one optimum.

Intent invariants. Past-time predicates in the requirement language of §24.5 over channels Chronicle already records. A task family declares which it adopts.

Invariant Signal
Contact only at designated sites (feet, wheels, tracks) Contact provenance per site
Attitude excursion inside a bound ‖log R‖ of the base
Airborne and sliding fractions inside their bounds Contact state; tangential speed at stance contacts
No net positive work by a contact The contact-active monitor of §29.9
The body stays inside the route corridor The scene's route graph (§19)
Dwell: success is held for a declared time, and is not first reached in the last 5% of the horizon (DESIGN) The verdict's own online signal

Findings and their resolution. A finding is (foil, behaviour cell, score, violated invariant, replay address, proposed clause), the clause being a typed edit of the requirements section. The author accepts the clause, which yields a new TaskId because a task is content-addressed, and the foil's trace is added as (example … violated), so the amended requirement is tested against the loophole from then on. Or the author marks the behaviour intended, and the trace is added as (example … satisfied). A high-scoring degenerate behaviour that violates no adopted invariant is reported unclassified. Campaign results cite the TaskId they ran under. eval.qualify on a task with an unresolved finding, or with no audit record for its TaskId, is refused. The audit runs on discovery worlds only and reads no confirmation world.

In-campaign check S. The winning episodes of the top-k designs are evaluated against the adopted invariants. A design whose wins violate one is flagged wins-outside-intent and cannot be promoted until the author resolves it.

Hand case. Goal region x ∈ [4.9, 5.1] m and requirement (F (0 s 10 s) in_goal). A sled foil moving at a constant 1 m/s is inside the region for t ∈ [4.9, 5.1] s and never stops: the verdict is satisfied with a dwell of 0.2 s. The proposed clause is (F (0 s 10 s) (H (0 s 2 s) in_goal)). The sled's longest dwell is 0.2 s < 2 s, so the amended requirement is violated by the foil; a body at rest at x = 5 m from t = 6 s satisfies it from t = 8 s.

Owners and verb. frob-arena owns the audit, the foil library and the invariants; frob-control and frob-opt supply the search; findings are stored by frob-chronicle keyed by TaskId. Verb: task.audit (run; budgeted; a long call returns a job handle).

Ceiling, pilot and gate. Ceiling: the compute of campaigns that a loophole would have invalidated. How often authored tasks contain one is unmeasured. FACT (§2.2, source register): in one published agent design loop 15.8% of trained candidates were degenerate after a full training run. HYPOTHESIS: at least one accepted finding in a fifth of authored tasks; test: the count over the task families of §24.3 at M3. Pilot before funding: task.audit on ten authored tasks with a budget of 5% of a reference campaign each. If no finding is accepted by an author, only the in-campaign check is built.

Gate Claim Threshold (TARGET) First milestone
GATE-ARENA-11 Known answer; Design comparison The hand case exactly. On 30 tasks with one planted loophole each (off-site contact, tumbling, off-route travel, last-instant success, energy-creating contact) and 30 clean tasks: at least 80% of planted loopholes found before any design campaign, at no more than 5% of a reference campaign's compute per task, with findings on at most 20% of the clean tasks M3
Kill gate Trigger Fallback
✖ KILL-FOIL Detection below 60%; or findings on more than 30% of clean tasks; or the sampling controller on the actual design finds the same loopholes at the same cost, so foil bodies add nothing The in-campaign check on winning episodes only; task.audit runs the actual design alone

24.21 The item bank: importance sampling of battery items for close comparisons

Item bank F. The cost of a population decision scales with designs × battery items × replicates. Racing and the ladder work on the first factor and the tiers on the cost of an episode. Nothing so far models the items, although most items say nothing about two close designs: scenes both pass and scenes both fail. The item bank keeps a persistent model of items and uses it only to choose which items a comparison runs. The estimand never changes.

Item, law and bank. An item is (scene instance, task, disturbance law, family of initial conditions), identified by content hash. A battery version fixes a law p(i) over its items; p is the estimand's weight and is never edited by the bank. The bank stores, per morphology class and battery version, a response model fitted by regularized maximum likelihood on every (design, item, outcome) row of discovery data in the archive, in canonical row order:

success:      P(Y = 1 | d, i) = σ(a_iᵀ·θ_d − b_i),   θ_d ∈ ℝᵏ, k ≤ 5 (DESIGN)
continuous:   Y | d, i  ~  N(μ_i + a_iᵀ·θ_d,  s_i²)

The fit has a version, and the version is part of the identity of every comparison that used it.

Proposal and estimator. For a contrast of designs A and B with paired differences D_i = Y_B,i − Y_A,i,

q*(i) ∝ p(i)·√(E_model[D_i²]),      q(i) = (1 − ε)·q*(i) + ε·p(i),      ε = 0.1 (DESIGN),
Ĉ = (1/n)·Σ_j w(i_j)·D_{i_j},        i_j drawn from q by an addressed stream,   w(i) = p(i)/q(i) ≤ 1/ε.

E_q[w·D] = Σ_i q(i)·(p(i)/q(i))·E[D_i] = Σ_i p(i)·E[D_i]: the estimate is unbiased for the contrast under the frozen law whether or not the response model is right. The model buys efficiency and nothing else. Items that do not discriminate are down-weighted in q and never removed from p.

Arithmetic, exact. Let D be ±1 on a fraction f of items and 0 elsewhere, with mean μ. Uniform sampling has variance f − μ². Under q* the weight is f on the discriminating items and the variance is f² − μ². At f = 0.1 and μ = 0.05 these are 0.0975 and 0.0075, a ratio of 13. With the defensive mixture at ε = 0.1 the weight on a discriminating item is 1/(0.9/f + 0.1) = 0.1099 and the variance is 0.1 × 0.1099 − 0.0025 = 0.0085, a ratio of 11.5.

Counterexample: a wrong model costs up to 1/ε. Let the model place q* on a set of items disjoint from the items that discriminate. Each discriminating item then has q = ε·p and weight 10, the second moment is f × 10 = 1.0, and the variance is 0.9975 against 0.0975 under uniform sampling: 10.2 times worse, with the estimate still unbiased. The mixture bounds the damage at 1/ε. It does not remove it, and the gate therefore measures realized interval width and not the model's own forecast.

Rules.

  • Anytime studies on discovery worlds only. The weighted increment w·D lies in a span 1/ε times the span declared for D, so a range-driven sequence would lose by (1/ε)² in sample size. Item-bank sampling is admitted only with the variance-adaptive betting sequence, and the span it declares is the widened one.
  • Refused for fixed-n studies and inside the blinded qualification lane. The exact rule of §28.6 counts discordant pairs under the frozen law, and confirmation draws from p on fresh sealed clusters. The fit never reads a confirmation world.
  • Not composed with the tier-correction estimator. Promotion pairs are drawn from p; multiplying the two weights would allow 10 × 20 = 200.
  • A rare-failure bound refuses item-bank episodes, as it refuses any episode not drawn nominally (§26).
  • Replicate noise is untouched. Variance between seeds inside an item is not reduced by choosing items.

What an agent reads. Beside the verdict: the episodes used and the uniform-equivalent count, and the separating items ranked by discrimination ‖a_i‖ with their difficulty b_i, which says in which capability the two designs differ.

Owners and verbs. frob-arena owns the battery law and the bank; frob-opt owns the proposal and the estimator; rows come from frob-chronicle. eval.compare --sampling item-bank runs a comparison; eval.items (read) returns the bank rows for a battery and class with their sample counts and fit version.

Ceiling and expectation. The idealized ratio is 1/f when a fraction f of items discriminates. With model error, the mixture and replicate noise the expectation is 4 to 8 (HYPOTHESIS; test: the replay study below). The saving multiplies with the tier correction of §29.11, which lowers the cost of an episode instead of their number.

Gate Claim Threshold (TARGET) First milestone
GATE-ARENA-12 Known answer The two-value fixture reproduces 0.0975, 0.0075 and 0.0085 in exact arithmetic; under the disjoint wrong model the mean of Ĉ over 10⁴ replays equals the true contrast within its Monte Carlo error and the variance ratio stays at or below 1/ε; a fixed-n request with item-bank sampling is refused M3
GATE-ARENA-13 Design comparison Replay on a recorded full matrix of about 40 designs × 150 items × 4 replicates (24,000 episodes): for the 20 closest pairs, episodes to equal interval width lower by a factor of at least 4; coverage within 2 points of nominal; item parameters fitted on one family predict held-out designs better than per-item means M3
Kill gate Trigger Fallback
✖ KILL-ITEMBANK At GATE-ARENA-13 the factor is under 2 for the 20 closest pairs; or coverage falls more than 2 points below nominal; or the held-out prediction is no better than per-item means Stratified sampling by interior archetype with proportional allocation, which needs no model

24.22 Exposure domains and the coupling test

Exposure domains S; chosen couplings F. This extends the randomization section of the task file (§24.4), pairing rule 2 of §28.6 and the keyed streams of §35.9. Those rules say that a draw is keyed by the physical entity it perturbs. What is still unstated is which kind of entity each draw belongs to, what happens to a disturbance that a mission event triggers, and how anyone checks that a coupling left each arm's own distribution alone. Every draw in randomization now declares one of four exposure domains, and the task compiler in frob-arena lowers the section to a versioned exposure table that is part of the task's identity.

Domain Key, beside WorldAddr and stream_id Meaning Example
field An anchored coordinate cell or a fixture identifier Sampled once per world at stable coordinates; a robot reads it where it is Friction of a floor tile; the mass of a cart in the scene
process A node of the dyadic tree over physical time (§16.5) An exogenous process in physical time; a change of step or control rate reads the same path A gust; a push scheduled at 3 s
event A mission role and an occurrence index Exists only where the task defines the exposure by a declared mission event; its physical time may differ between arms by definition A push when the base first crosses the docking gate
private The hash of a port name Noise with no counterpart in the other arm Noise of a sensor that only one design carries

Roles are the typed ports and task regions of §14 and §19, so they survive edits. An event draw is legal only when the exposure is defined by a mission event in the task; keying by the order of contacts, footfalls or draws remains refused (§28.6).

The coupling test. task.check --coupling runs four checks on a task and a pair of controllers or designs. A coupling that fails one is refused.

  1. Marginal invariance. For each arm, outcomes under the coupled protocol and under independent sampling agree in distribution within a tolerance fixed in advance. A coupling may change the joint law of the two arms and nothing about either arm alone.
  2. Insertion invariance. Adding unused draws, a sensor or a limb changes no exposure of another domain entry (the sensor-set property of §20.17, applied to exposures).
  3. Rate invariance. Halving the physics step changes no field value, and a process increment over a coarse interval equals the sum of its fine increments.
  4. Retiming. Under a deliberately retimed controller, a field stays at its coordinates, a process push occurs at the same physical time, and an event push occurs at each arm's own crossing.

Counterexamples, hand-checkable. A push keyed by control tick 1,500 occurs at 3.0 s with a 2 ms step and at 1.5 s with a 1 ms step. A push keyed by "the tenth footfall" occurs at 5.0 s under a gait period of 0.5 s and at 4.0 s under 0.4 s: a disturbance meant to be timed now follows gait phase, and the two arms were pushed in different states. The first keying fails check 3 and the second fails check 4.

Choosing among legal couplings F. Where more than one legal keying exists (friction by tile or by fixture), the choice is made on pilot addresses by the measured variance of the contrast, the table version is then locked, and confirmation addresses are drawn only afterwards. Both marginals and the contrast variance are recorded. For equal marginal variances σ², Var(D) = 2σ²·(1 − ρ): moving the paired correlation from 0.50 to 0.95 lowers the variance of a contrast ten times at a fixed count. That is an illustration of what is at stake, not a prediction.

Owners and verb. frob-arena owns the compiler and the test; frob-scene caches field samples per world; the stream registry of §35.9 holds the domains. Verb: task.check --coupling. A paired study on a task whose exposure table has not passed the test is refused.

Gate Claim Threshold (TARGET) First milestone
GATE-ARENA-14 Known answer; Design comparison Fixture: a slick strip over x ∈ [2.0, 2.5] m (field, per 0.25 m tile), a push at t = 3 s (process), a push at the first crossing of a gate at x = 5 m (event), a baseline controller and one retimed from a 0.5 s to a 0.4 s gait period. With 1,000 unused draws inserted, the step halved and a limb added: tile values bit-identical; the timed push at 3 s in every arm; the gate push at each arm's crossing; per-arm outcomes indistinguishable from independent sampling in a two-sided test at level 0.01 on 4,096 worlds. The tick-keyed and footfall-keyed controls are refused. On a held-out slip-and-dock battery the variance of the contrast is at most half that of named streams keyed per stream and tick, at equal count M2 for the fixture; M3 for the battery
Kill gate Trigger Fallback
✖ KILL-EXPODOMAIN Under a factor of 2 in contrast variance on the held-out battery; or the automatic choice among couplings picks one that fails the test in any pilot The keyed streams of §35.9 as they stand, with explicit per-stream adapters; the four checks are kept as a test

24.23 The idle arm of the task audit

What it is. S One more arm for task.audit (§24.20). The foil bodies probe ways of satisfying a task without the intended mechanism. The idle arm probes the cheapest of them: doing nothing. A requirement or a reward built from safety atoms alone is satisfied in full by a robot that never leaves its start. Owner: frob-arena. Verb: task.audit.

Hand case. A delivery task uses the past-time requirement (H (0 s 2 s) (and clear-margin thermal-ok spill-ok)) as its online reward over 60 s. A parked robot keeps all three margins positive for the whole episode and collects the full reward. A robot that carries the tray through the pass and loses part of its spill margin for one second collects less. Search prefers the parked robot.

Mechanism. DESIGN.

  • The audit runs the candidate design, and each foil, under three idle laws: zero command with the brakes as carded; a hold-posture command from the baseline controller of §22.4; and the backup controller of §22.12.3.
  • A task is flagged rewards-idleness when an idle run's requirement verdict is satisfied, or when its return is at least a declared fraction of the best audited return (DESIGN default 0.5).
  • The finding carries a proposed clause, like every finding of §24.20: a progress atom with its scale (for the hand case, approach speed above 0.15 m/s on a scale of 0.50 m/s, which gives the parked robot a margin of −0.3), or a completion requirement with a deadline. The author accepts it, which yields a new TaskId, or marks idleness intended, as a station-keeping task would.
  • eval.qualify on a task with an unresolved idleness finding is refused, under the existing rule for unresolved findings.
GATE-ARENA-15 Known answer. On ten authored tasks with a planted safety-only reward and ten clean tasks, the idle arm flags ten of ten and none of ten (TARGET: at least nine, at most one). The hand case is flagged, and after the progress clause is accepted the parked robot's verdict is violated. M3.