Arena: tasks, requirements, fixtures, metrics, qualification
15,330 words23 subsections34 tables7 listingsPlan version 1.1
On this page23
Crates frob-spec and frob-arena, both in layer 4.
24.1 What Arena owns
Arena defines what a robot is asked to do and how an answer is scored. It owns the task families, the task file
.ftask, the requirement language and its monitors, the mission-level verdict, the fixtures whose answers are known
before any run, the metrics and scorecards, the batteries and the design corpus, the two study types, and the blinded
qualification lane. frob-spec holds the language: parser, unit and clock checker, normalizer, monitor compiler and
the verdict type. frob-arena holds the registry of tasks and fixtures, the metric definitions, scorecards, study
plans and the qualification service.
Arena does not own the machinery it scores with. The boundaries are fixed here so that no other chapter has to guess.
A note on numbers. A value in this chapter that follows from a stated formula is a derived value: it is a known answer of §24.16 and is recomputed by its test. Every other number carries its label where it appears.
| Concern | Owner | What Arena contributes |
|---|---|---|
| Observation, action and reward terms; reset and bootstrap semantics | Cortex (§23) | The task file carries the terms. Arena never reads a reward to decide a verdict |
| Mission queries with quantifiers over designs, policies and worlds | Augur (§25) | The acceptance predicate of a mission query is a requirement of this chapter |
| Search for worlds that break a design; rare-failure bounds | Nemesis (§26) | The falsification score is the requirement margin; infeasible worlds are counted here |
| Attribution of a failure | Autopsy (§27) | The outcome that an intervention flips is a verdict of this chapter |
| Paired comparison, racing, the ladder of rungs, promotion between tiers | Atelier (§28), Kinesis (§16) | The study types and the confirmation lane wrap those mechanisms |
| Telemetry channels, snapshots, the failure taxonomy | Chronicle (§29) | Requirements read channels by name, kind and clock |
| Scene splits and held-out scene families | Habitat (§19) | Arena enforces the split at scoring time (GATE-ARENA-02) |
24.2 Decisions
- ADR-ARENA-01. Requirements decide acceptance; reward terms train. S A mission is accepted or rejected by a requirement that returns a verdict and a margin. Reward terms remain the training signal. Reason: as an evaluation metric a requirement is close to definitional, a pass or a fail with a distance to the boundary. As the only training signal the margin is the weak baseline in the field's own results. FACT (Appendix H, temporal-logic reinforcement learning): decomposing a formula into timed subgoals and invariant constraints raised task success by an average of 31.6% over the best baseline, in a field where earlier methods used margins as sparse terminal rewards; and margin-only optimizers reached between 6.67% and 29.9% success in a study of robustness metrics for robot tasks.
- ADR-ARENA-02. Bounded time only. S Every temporal operator carries a finite window. Reason: a monitor needs fixed memory, a verdict needs a finite horizon, and formulas with unbounded time admit no learning guarantee except where they are decidable within a finite horizon (FACT, Appendix H).
- ADR-ARENA-03. Every atom is normalized by a declared scale. S An atom states an inequality and the physical scale that makes its margin dimensionless. Reason: the minimum of a clearance in metres and a thermal margin in kelvins has no meaning, and the composition rules of the language are minima and maxima.
- ADR-ARENA-04. Past time online, future time offline. S Operators that look backwards are computed inside the step and are available as signals. Operators that look forwards are evaluated on a completed trace and are the verdict. Reason: an online value of a future-time operator is a silently truncated window.
- ADR-ARENA-05. The declared comparator decides at the boundary, in exact arithmetic. S The sign of a floating-point margin does not decide a case that sits on the boundary. Reason: a strict and a non-strict inequality differ exactly there, and single precision misclassifies near-boundary cases in bulk (§24.5).
- ADR-ARENA-06. Clocks are explicit and rational. S Every signal and every requirement names its clock. Periods are rational numbers of seconds with a phase. Reason: sensor, policy and physics rates are not integer multiples of each other in real robots, and a requirement evaluated on an unstated clock is a different requirement on every backend.
- ADR-ARENA-07. A study is anytime-valid or fixed-n, never both. S Reason: each kind of inference is valid only under its own stopping rule (§24.12).
- ADR-ARENA-08. Discovery and confirmation are separate lanes with separate data rights. S Reason: an optimizing agent that may resubmit to the same held-out worlds will eventually pass by chance (§24.13).
- ADR-ARENA-09. An infeasible scenario is kept and counted. S Reason: a failure is a failed cell, never an excluded cell (§8), and a suite that sheds its impossible cases reports a success rate over a population nobody declared.
- ADR-ARENA-10. A fixture is complete before its first run. S Every mass, constant, seed and rule of a fixture is recorded and hashed before either design runs, with a derivation on file for each analytic answer. Reason: a fixture whose constants can move after the outcome is seen is a tuning harness.
24.3 Task families
| Family | Example tasks | First milestone |
|---|---|---|
| Stand and recover | Stand for 10 s; recover from a push of stated impulse | M1 |
| Threshold delivery | A loaded cart crosses a door sill and docks on the run-out (fixture 1, §24.10) | M1, CPU |
| Timed docking | Reach a pose with zero velocity by a deadline under a hidden plant gain (the exact mission slice of §25; its boundary case is a known answer of §24.16) | M1, CPU |
| Velocity tracking | Commanded planar velocity and yaw rate on flat and rough ground | M2 |
| Terrain traversal | Stairs up and down, gaps, stepping stones, slopes, rubble | M2 |
| Split-grip landing | A quadruped climbs a ramp onto a landing with a low-grip strip (fixture 2, §24.11) | M2 |
| Interior navigation | Point goal through doors and thresholds in each archetype | M3 |
| Payload transport | Carry or tow a stated mass along a warehouse route, including a ramp | M3 |
| Low-friction crossing | Cross a kitchen galley with a wet or greasy patch | M3 |
| Tight-space maneuvering | Turn around, parallel shift and dock in an aisle of stated width | M3 |
| Moving surfaces | Board and leave a conveyor | M4 |
| Stair and curb climbing for wheels and tracks | A tracked base on stairs; a wheel cluster on a curb | M4 |
| Reach and carry | Mobile base with an arm: reach a shelf point, carry a tote | M4 |
Every family ships at least one task with a requirement, labelled example traces (§24.5) and a reference-robot baseline scorecard.
24.4 The task file .ftask S
Format FMT-FTASK, schema frob.task/1. DESIGN: the canonical form is JSON, and its BLAKE3 hash is the TaskId; an
S-expression view is lossless and optional, as for designs (§14). A task is data. It has nine sections.
| Section | Contents | Semantics defined in |
|---|---|---|
interface |
Required ports; observation and action layouts with units, frames and rates | §23 |
clocks |
Named clocks, each a rational period in seconds and a phase | §24.5, §20 |
terms |
Observation, action, reward and termination terms, each with unit, noise, delay and the representation under which it transforms | §23 |
signals |
The telemetry channels that requirements read: name, unit, channel kind, clock | §29 |
requirements |
Atoms with scales, online signals, the verdict formula, labelled examples | §24.5 |
randomization |
Draws, each traced to an uncertainty on a card and bound to a named random stream | §15, §23 |
reset |
The settled-pose cache specification | §24.7 |
world |
Scene family or scene set, with its split | §19 |
study |
Default study type, effect margin and budget | §24.12 |
The requirement section in the S-expression view has this grammar. The JSON form is the same tree.
Requirements ::= "(requirements" Atom+ Online* Verdict Example* ")"
Atom ::= "(atom" Ident Cmp (":scale" Qty | ":exact") ")"
Cmp ::= "(" (">" | ">=" | "<" | "<=" | "=") Expr Expr ")"
Expr ::= Signal | Qty | "(" ("+" | "-" | "*" | "min" | "max" | "abs") Expr+ ")" | "(delay" Qty Signal ")"
Past ::= Ident | "(and" Past+ ")" | "(or" Past+ ")" | "(not" Past ")"
| "(H" Win Past ")" | "(O" Win Past ")" | "(S" Win Past Past ")"
Future ::= Past | "(and" Future+ ")" | "(or" Future+ ")" | "(not" Future ")"
| "(G" Win Future ")" | "(F" Win Future ")" | "(U" Win Future Future ")"
Win ::= "(" Qty Qty (":round" ("inward" | "outward"))? ")"
Online ::= "(online" Ident Past ")"
Verdict ::= "(verdict" Ident Future ")"
Example ::= "(example" TraceRef ("satisfied" | "violated") ")"
H, O and S are the past-time operators historically, once and since. G, F and U are the future-time
operators always, eventually and until. A window is a closed interval of simulated time.
24.5 The requirement language S
Signals and atoms. A signal is a telemetry channel with a unit, a channel kind and a clock (§29). An atom is a
comparison of two expressions of the same dimension, together with a scale s of that dimension. The unit checker
rejects an expression that adds or compares different dimensions and an atom that has no scale. An atom over an
integer-valued signal, such as a count of chassis strikes, is declared :exact and has margin +1 or −1.
Margins. Every formula has a Boolean value and a margin ρ, a dimensionless number whose sign agrees with the Boolean value away from the boundary. Time below is the tick index of the requirement's clock.
| Formula at tick t | Margin ρ |
|---|---|
| Atom with expression e ≥ 0 or e > 0 and scale s | e(t) / s |
| ¬φ | −ρ(φ, t) |
| φ ∧ ψ ; φ ∨ ψ | min ; max of the two margins |
| G_[a,b] φ ; F_[a,b] φ | min ; max of ρ(φ, t′) over t′ in [t + a, t + b] |
| φ U_[a,b] ψ | max over t′ in [t + a, t + b] of min( ρ(ψ, t′), min of ρ(φ, t″) over t″ in [t, t′) ) |
| H_[a,b] φ ; O_[a,b] φ | min ; max of ρ(φ, t′) over t′ in [t − b, t − a] |
| φ S_[a,b] ψ | max over t′ in [t − b, t − a] of min( ρ(ψ, t′), min of ρ(φ, t″) over t″ in (t′, t] ) |
Two properties are tested as laws (PO-ARENA-04): ρ > 0 implies the formula holds, and ρ < 0 implies it fails. At ρ = 0 the margin decides nothing.
Counterexample for composition without scales. A clearance margin of 0.02 m and a thermal margin of 3 K compose to a raw minimum of 0.02, so clearance binds. Express the clearance in millimetres and the raw minimum is 3, so the thermal margin binds. With scales of 0.10 m and 10 K the margins are 0.2 and 0.3 in either unit system, and clearance binds in both. The binding atom of a requirement is what an agent acts on, so it must not depend on the unit system (the metamorphic relation on unit scaling, §38).
Clocks. DESIGN: a clock is a rational period p/q seconds and a phase, advanced by an exact accumulator (§20). A task names its clocks, for example physics at 1/500 s, an inertial sensor at 1/500 s, a policy at 1/50 s, a lidar at 1/10 s and a camera at 1/30 s; the camera is not an integer multiple of the physics period and needs no special case. Each requirement names the clock it is evaluated on. Three rules remove implicit resampling.
- A signal on a faster clock enters a requirement through a declared reduction over each requirement tick:
last,minormax. Channels of kind interval extremum, such as the minimum clearance swept during a step, are reduced by the matching extremum. A missing declaration is refused (FROB-REF-525). - A signal on a slower clock is held, and its sample age is itself a signal that a requirement may bound. A requirement about what the robot knew reads delivered packets with their timestamps from Sensorium; a requirement about what happened reads simulation state at committed physics ticks. The two are different signals with different names.
- A window [a, b] denotes the ticks j with a ≤ j·T ≤ b, compared in exact rational arithmetic. A bound that is not a multiple of the period T is refused unless the window declares outward or inward rounding (FROB-REF-538). Ticks that coincide with another clock's boundary follow the ordering rule of §20.
A verdict on a sampled trace says nothing about the time between samples. A requirement on a state-kind channel is
therefore labelled sampled in its verdict, and one on an interval-extremum channel is labelled interval.
Online evaluation. S Past-time formulas compile into kernels that run in the last stage of the batched step
(§16), beside observations and rewards. The kernel for a sliding minimum or maximum over a window of W ticks uses the
block decomposition that keeps a prefix and a suffix extremum per block of W samples: fixed memory of 2·W values per
operator node and world, at most three comparisons per sample, and no data-dependent control flow, which the lockstep
rule of §16 requires. The referee uses the monotonic-deque formulation of the same function. DESIGN: minimum, maximum
and negation are exact in floating point, so both formulations return identical bits for identical atom values on
every backend, after negative zero is canonicalized and a NaN marks the world Faulted. The capacity of an online
window is 4,096 requirement ticks (TARGET); a longer window is offline only (FROB-REF-528). An online signal has three
uses: an input to early termination, a score for splitting in §26, and an optional added reward term (§24.6).
Offline evaluation. S The verdict formula is evaluated on the completed trace by a two-pass evaluator on the
CPU. The reach of a formula, the sum of its nested window ends, is computed statically; a task whose episode horizon
is shorter than the reach of its verdict is refused at admission (FROB-REF-526). A window that the recorded trace does
not cover, because the episode was truncated or samples are missing, yields undetermined with the reason, never a
value from the part that exists (INV-ARENA-02). A verdict whose truth is already fixed by a prefix, as when an
always-clause is violated or an until-clause is met, is reported as decided at the tick that fixed it. That is how an
environment ends an episode on success or on a hard violation without truncating a window.
The boundary. S DESIGN: an atom is decided on the device only when its normalized margin lies outside a guard
band of 2⁻²¹ (TARGET) around zero. A world with a verdict atom inside the band is flagged boundary and that atom is
re-evaluated on the CPU with its declared comparator, in exact rational or integer arithmetic where the atom's inputs
belong to the exact-integer kernel class of §35 and in double precision otherwise. FACT (planning archive,
boundary-classifier runs on this Mac's Metal GPU and on both RTX 4090 cards; Appendix H): on a dense 65,536-case
boundary screen, unguarded single-precision strict comparison gave 452 false admissions and 23,438 false exclusions;
with a 2⁻²¹ guard and an exact fallback there were no final errors; an exact 32-bit integer kernel classified all
65,536 cases correctly with identical bytes on every device. The exact scalar docking mission of §25 supplies the
smallest case in which the comparator matters: at command limit U = 3/4 and read error ε = 5/16 the separation between
the two worlds' readings is exactly zero. The strict requirement correctly fails. A non-strict comparator would admit
a mission that cannot be completed, because both worlds can then report 15/16 while needing different next commands.
The verdict. The mission-level verdict is an object with schema frob.verdict/1, stored with the episode in
.frun and summarized in the ledger. It is the unit that scorecards, studies and searches count.
{ "schema": "frob.verdict/1", "task": "t:cart-sill-dock@6be1…", "requirement": "docked",
"world": {"run_seed": 17, "world": 41, "episode": 0}, "model": "m:9f2c…", "controller": "c:5e02…",
"verdict": "violated", "margin": -0.31, "binding_atom": "cleared", "binding_time_s": "8",
"decided_at_s": "8", "reason": "stuck", "coverage": {"kind": "interval", "missing_samples": 0},
"boundary": {"flagged": false, "guard": "2^-21"}, "atoms_min": {"cleared": -0.31, "upright": 0.84},
"evidence": {"class": "Tested", "tier": "batch-cpu", "determinism": "D2", "standing": "provisional"},
"authorization": {"execution_authorized": false, "hardware_safety_signoff": false,
"sim_to_real_certified": false},
"trace_digest": "b3:77a0…" }
A verdict takes one of four values: satisfied, violated, undetermined and invalid. invalid records a
numerical fault; it is a study error, excluded from policy updates and retained in every benchmark and failure count.
The verdict is a pure function of the trace and the TaskId (INV-ARENA-01).
Stages. A mission with several stages is one requirement over one continuous episode, with the stages chained by until-clauses. Concatenating separately recorded stage episodes proves nothing about the mission, because the end state of one need not lie in the entry set of the next. Stage pass rates are reported jointly. Counterexample: among 20 equally likely worlds two stages each pass in 19. If the failing worlds differ, 18 of 20 missions pass; if they coincide, 19 of 20; the product 0.95² = 0.9025 is neither.
Checking a requirement someone wrote. S FACT (Appendix H): a 2023 language model translated natural-language
statements into temporal-logic formulas with 77.7% accuracy over 300 samples. A requirement is therefore admitted only
with labelled example traces: at least two satisfying and two violating, with one near the boundary of each comparator
(DESIGN). task.check runs the monitors on the examples and refuses a requirement whose labels disagree
(FROB-REF-539). It also runs a mutation check: reversing a comparator or deleting an atom must change the label of at
least one example. An atom that no example can distinguish is reported as vacuous, and a vacuous hard-limit atom is a
refusal.
24.6 Requirements and rewards
| Use | What is read | Owner |
|---|---|---|
| Acceptance of an episode | The offline verdict and its margin | Arena |
| Falsification score | The minimum margin of the verdict over an episode, minimized over worlds | Nemesis (§26) |
| Splitting score for rare failures | An online past-time margin | Nemesis (§26) |
| Outcome flipped by an intervention | The verdict | Autopsy (§27) |
| Early termination | The decided-early flag of the verdict; hard-limit atoms | Cortex (§23) |
| Training signal | Reward terms, integrated over simulated time | Cortex (§23) |
| Optional added reward term | An online past-time margin, or timed subgoals and invariants derived from the verdict formula | Cortex (§23), F |
| Runtime monitors | Hard-limit atoms shared with the shields | Reflex (§22) |
Hard limits are frozen separately from shaping rewards. A curriculum may change training distributions and reward
weights; it may not change an atom, a scale or a window, because that changes the TaskId. The added reward term is
off by default. It is a Frontier option with a pilot and a kill gate (§24.17). The ceiling of its benefit is the gap
in task success between a shaped reward and the same reward plus the term; the nearest published figure is the 31.6%
of ADR-ARENA-01, measured on other tasks against weaker baselines, and no effect on decision time is claimed.
24.7 The settled-pose reset cache S
An episode that starts with a foot 2 mm inside the floor begins with an impulse that no robot would feel, and a paired comparison in which one design starts settled and the other does not compares spawn artifacts. DESIGN: episodes start from precomputed quasi-static poses.
- Contents. For each (
ModelId, spawn specification, contact law, kernel versions) the cache holds K = 64 settled states (TARGET): configuration, contact warm starts, actuator states at rest and estimator initial state. The key is a content hash, so the cache is a node of the memo graph of §28 and a design edit that changes mass, geometry or contact surfaces invalidates it. - Settling. States are settled in double precision on
batch-cpuuntil base and joint speeds stay below 10⁻³ in SI units for 0.2 s with maximum penetration at most 0.5 mm (TARGET). The first contact impulse after loading a cached state on any tier must lie within 5% of the static support force (TARGET). - Selection. The pose index is drawn from a counter-based stream keyed by the world address, so two designs paired on one address start from the same spawn specification, each in its own settled state. The initial equilibrium is recomputed for each design; a state is never transported from one design to another.
- Failure. A world that does not meet the settle criterion on its tier is marked
Degraded: unsettledand counted. A task whose cache is missing for the requested model is refused (FROB-REF-529).
24.8 Metrics and scorecards S
Every metric has a unit and a definition on file. The core set:
| Metric | Definition |
|---|---|
| Tracking error | Root-mean-square error between commanded and achieved planar velocity and yaw rate |
| Success, fall and collision rates | Per the task's verdict and termination rules, with the failure taxonomy of §29 |
| Requirement margin | The margin of the verdict formula for the episode, with the binding atom and its time |
| Time to verdict | Simulated time at which the verdict was decided. Non-attainment is right-censored at the horizon and summaries use the restricted mean |
| Cost of transport | Energy divided by (mass × g × distance), reported twice: mechanical work at the joints, and electrical energy from the battery |
| Energy per completed task | Reported twice: mean energy conditional on completion, and total energy of all attempts divided by the number of completions, which is undefined at zero completions |
| Froude number | v² / (g × leg length), so gaits are compared at matched dynamic similarity |
| Actuator utilization | Fractions of time at the torque, speed and thermal limits, per joint |
| Friction utilization | The distribution of ‖γₜ‖ / (μ·γₙ) over stance |
| Slip distance | Tangential travel of a contact while in stance, computed as the time integral of the normal-load-weighted mean tangential speed over the contact quadrature, zero below a declared normal-force floor |
| Clearance | Minimum distance between non-contact links and the scene |
| Tail robustness | The 10% conditional value at risk of the task score over the randomization |
| Maneuver cost | Path length and time for the tight-space tasks, against A-07's prediction |
| Fallback count | Controller fallbacks taken, by kind; every fallback is recorded (§22) |
The two energy figures can order designs oppositely. Design A with one 10 J success and one 20 J failure, against design B with two 12 J successes, gives 10 J against 12 J conditional on completion and 30 J against 12 J per completion. Both are published. The slip metric must be invariant to refinement of the quadrature: duplicating a quadrature point and halving the two weights leaves it unchanged (a known answer of §24.16). A quantile is reported only with the sample count that could have observed it: seeing one value above the population's 99th percentile with 95% probability takes 299 independent episodes, and five episodes leave a 95.1% chance that even the largest lies below it.
A scorecard is a table of metrics by scene class with intervals, normalized against the reference robot of the same class. No single scalar is reported unless the task defines one. DESIGN: every scenario of a battery lands in exactly one outcome column (INV-ARENA-03).
| Column | Meaning |
|---|---|
satisfied, violated |
Verdicts, the second broken down by failure class |
undetermined |
The trace did not cover a window; the reason is listed |
invalid |
Numerical fault. Counted as a study error and as a failure of the tier, never dropped |
infeasible-for-candidate |
A checked necessary condition shows that this body cannot satisfy the requirement in this world (§26.5); the witness is attached |
infeasible-for-class |
The same for every body of the declared class; evidence that the scenario should leave the suite by a recorded decision, not by attrition |
refused |
The run was not admitted; the refusal code is listed |
Success rates are printed with two denominators, all scenarios and scenarios not shown infeasible, side by side. Removing a scenario from a battery is a recorded event with a reason and changes the battery's identity.
24.9 Batteries and the design corpus
smoke: minutes; every template and reference robot on a handful of scenarios. Runs on every change.fixtures: the analytic layers of the fixtures of §24.10 and §24.11 and the monitor known answers. Runs on every change that touches contact, actuators, clocks orfrob-spec.standard: the fidelity battery of §38, about 200 scenarios per robot class.full: every task family across every archetype with the full randomization.hard: the hard-world set of §26 for the robot class. It is discovery data and is never a confirmation sample.covering: a scene set chosen by the coverage selection of §26.11, with one slot in five drawn at random from held-out scenes.- The design corpus for BET-05 and BET-06: about 200 designs made of templates, parameter perturbations of them, and designs with planted defects of known kind (an undersized knee actuator, an inertia that breaks static stability, a wheelbase too long for an aisle). Labels are fixed before any analysis is tuned. The corpus also holds the planted winners used to audit screening (§26.10) and the planted single causes used to test attribution (§27.16).
24.10 Fixture 1: a loaded cart crosses a sill and docks S
Question. For a fixed motor and a current-limited cart, does a 75 mm front wheel improve loaded delivery across a
12 mm door sill relative to a 60 mm wheel? The larger wheel has a better lever at the lip and a smaller tread force
for the same torque; the fixture exists to find which effect dominates once load distribution, motor lag, impact and
momentum are included. Everything below is DESIGN: a synthetic fixture with explicit values, registered as
fixture:cart-sill-dock, so that an implementer can build it and a result can falsify it. Measured profiles replace
synthetic ones only under a new fixture identity.
Bodies. Chassis frame: x forward, y left, z up, origin midway between the axles on the axle line.
| Part | Value |
|---|---|
| Axles and wheels | Axles at x = ±0.16 m (wheelbase L = 0.32 m); wheel centres at y = ±0.12 m; front-wheel drive, rear wheels passive |
| Chassis | Uniform box 0.40 × 0.20 × 0.08 m, 5.0 kg, centred 50 mm above the axle line, so it overhangs each axle by 40 mm |
| Payload | Uniform box 0.18 × 0.14 × 0.08 m, 2.2 kg, bolted to the chassis top face with its bottom flush; centred at x = δ, where δ is the fore-aft offset of the condition |
| Wheels | Uniform cylinders 30 mm wide. Rear pair: radius 60 mm, 0.2 kg each, in both designs |
| Design A | Front pair: radius 60 mm, 0.2 kg each, spin inertia 3.6 × 10⁻⁴ kg·m². Total mass 8.000 kg |
| Design B | Front pair: radius 75 mm at the same density and width, so 0.3125 kg each (0.1125 kg more) and spin inertia 8.7890625 × 10⁻⁴ kg·m², a ratio of (75/60)⁴ = 625/256. Total mass 8.225 kg. The front axle rides 15 mm higher, pitching the chassis up by asin(15/320) = 2.69° |
| Rotor | Additional reflected rotor inertia per driven wheel: zero in the primary fixture |
Drive. One motor per front wheel, on the wheel shaft.
| Quantity | Value |
|---|---|
| Supply | Ideal regulated 24 V bus for the 8 s trial; a lossless averaged four-quadrant bridge that absorbs regeneration |
| Motor | Torque constant 0.2 N·m/A, back-EMF constant 0.2 V·s/rad, resistance 1 Ω, inductance 10 mH, current limit ±6 A, gear ratio 1 |
| Initial state | Zero current; winding and pack held at 20 °C |
| Shaft loss | Viscous damping 0.002 N·m·s/rad on each of the four wheel shafts |
| Outer loop, 100 Hz | Wheel-rate target 0.30 m/s divided by the loaded radius: 5.0 rad/s for A, 4.0 rad/s for B. Current command 2 A·s/rad times the rate error, clipped to ±6 A, held for 10 ms |
| Inner driver, continuous | Winding voltage v = R·i + k_e·ω + 4 Ω·(i_ref − i), clipped to ±24 V. At +6 A, if that voltage would raise the current, v = R·i + k_e·ω holds the limit when available, and symmetrically at −6 A. A limit crossing is located inside the step. If no admissible voltage keeps the winding inside its rating, a physical limit violation is recorded |
| Power ledger | Net DC power is Σ v_j·i_j over the two drives; bus current is that sum divided by 24 V; positive and negative parts are integrated separately |
World and rules.
| Item | Value |
|---|---|
| Floor and sill | Level floor. A sill spans x from 0 to 0.10 m across the full lane width, 0.012 m high, in two strata: sharp upper edges, or 2 mm fillets on both upper edges |
| Contact | Coulomb friction μ, one value for all wheel–floor and wheel–sill contacts of a condition, 0.8 nominal; inelastic impact; g = 9.81 m/s² |
| Start | Stationary, centred and parallel to the lane, front axle at x = −0.50 m, settled under gravity from the reset cache of §24.7, separately for each design |
| Clocks | Physics 1/2000 s; outer loop and requirement clock 1/100 s; atoms are interval extrema over the 20 physics ticks of each requirement tick |
| Docked | The minimum world x of the whole loaded collision envelope exceeds 0.10 m, and roll and pitch stay within 5°, continuously for 0.5 s, with the dwell complete within 8 s of the start. The dwell restarts whenever a condition lapses |
| Hard limits | Roll or pitch beyond 35°; any chassis contact with the sill; winding current beyond the rating by more than 0.01 A. Each is absorbing |
| Stall | The command is saturated and the cart advances less than 2 mm over 0.5 s. A stall names the failure reason; the verdict still comes from the requirement at the horizon |
| Horizon | 8 simulated seconds. Timeout, rollover, strike and stall each carry a reason |
The requirement, in the S-expression view:
(requirements
(atom cleared (> (- env_min_x 0.10 m) 0 m) :scale 0.10 m)
(atom upright (>= (- 5 deg tilt) 0 deg) :scale 5 deg)
(atom tilt-ok (>= (- 35 deg tilt) 0 deg) :scale 35 deg)
(atom current-ok (>= (- 6.01 A i_max) 0 A) :scale 6 A)
(atom no-strike (= strikes 0) :exact)
(atom saturated (= drive_saturated 1) :exact)
(atom advancing (>= (- x_front (delay 0.5 s x_front)) 0.002 m) :scale 0.002 m)
(online dwell (H (0 s 0.5 s) (and cleared upright)))
(online stalled (and (H (0 s 0.5 s) saturated) (not advancing)))
(verdict docked (U (0 s 8 s) (and tilt-ok current-ok no-strike) dwell)))
env_min_x is the rearmost point of the loaded collision envelope, tilt the larger of |roll| and |pitch|, i_max
the larger winding current magnitude, each an interval extremum over a requirement tick.
The until-form says that the hard limits hold at every tick before the dwell completes, and that it completes by 8 s.
Written this way the dwell cannot latch: H over 0.5 s is false the tick any condition lapses. FACT (planning
archive, sill pilot audit; Appendix H): the hand-written scorer of an earlier planar pilot of this fixture had a
latched dwell timer and no chassis-strike guard. Both defects are unrepresentable here.
Energy. Net bus energy is integrated from the start to adjudication for every trial and reported with duration, gross draw and returned energy. The two per-completion figures of §24.8 are both published. Energy is descriptive in version 1 of the fixture and takes no part in the rule.
The analytic layer. These rows are known answers, each a limiting rig that runs before the full cart. Symbols: r
wheel radius, h lip height, x = √(2rh − h²) the horizontal distance from axle to lip at lift-off, tan θ = x/(r − h),
F₀ the static load on the climbing axle on level ground, W the total weight. The rigid, quasi-static derivations are
on file under docs/derivations/fixture-cart-sill-dock/.
| Row | Statement | Value |
|---|---|---|
| Mass roll-up | Total mass; front-wheel mass and spin-inertia ratio of B to A | 8.000 kg and 8.225 kg; 1.5625 and 625/256 |
| Tread force cap | Two wheels at the current limit: 2·k_t·I_max/r | 40 N for A, 32 N for B |
| Force slope | Unsaturated flat rolling at settled current: 2·k_t·K_ω/r² per unit speed error | 222.22 and 142.22 N/(m/s). A gain of 3.125 A·s/rad for B matches the slopes |
| Lip lever | x for a sharp lip of 12 mm | 36.000 mm at r = 60 mm; 40.694 mm at r = 75 mm |
| Front lift-off, creep limit | Driven axle at the lip, trailing axle free on the floor. The lip force is vertical, so the friction demand is μ ≥ tan θ and the total drive torque is F₀·x/(1 + x/L) | μ ≥ 0.7500 for A and μ ≥ 0.6459 for B. With 2 mm fillets, 0.7332 and 0.6351. Torque for A at δ = 0: 1.270 N·m of 2.4 N·m available |
| Rear pull-over, creep limit | Free axle at the lip, driven axle on the floor. The lip force passes through the free axle, so its horizontal part is V·tan θ with V = W·d/(L − x − h·tan θ), d the horizontal distance from the driven contact to the mass centre. The driven axle must supply that traction | For A, V/W = 32/55 − δ with δ in metres, and the friction demand is ¾·(32/55 − δ)/(23/55 + δ): 24/23 = 1.043 at δ = 0, 0.887 at δ = +0.04 m. For B at δ = 0 the demand of 34.2 N exceeds the 32 N force cap |
| Creep-feasible set for A | Both axles | μ ≥ ¾·(32/55 − δ)/(23/55 + δ), which lies inside the admitted friction range only for δ ≥ 23.0 mm: 1.13% of the development distribution below |
| Energy | Raising the front axle of A by h takes F₀·h; the cart's kinetic energy at 0.30 m/s is ½·m·v² plus wheel spin | 0.471 J against 0.360 J + 0.018 J. Momentum alone does not carry even one axle over |
| Wheel-loop stability | The 75 mm wheel with 0.0002 kg·m² added rotor inertia (0.00107890625 kg·m² in all), slipping against a constant torque, rate sampled and current target held for 10 ms: spectral radius of the one-tick map | 1.4820 with the feedforward term R·i_ref; 0.9207 with R·i as specified above. An instantaneous-current approximation gives 2.6917 for both (2.7075 if the shaft damping is left out) |
| Power ledger | A blocked winding at 3 A and 3 V; a drive at 50 rad/s and −2 A | 9 W dissipated, 0.375 A from the bus; v = 8 V, 16 W returned, 20 W from the shaft, 4 W in copper |
The creep limit is run as a rig at a commanded speed of 0.02 m/s. For the front-axle rig the outcome must flip between
the friction grid points that bracket the threshold, on a grid of spacing 0.01 that excludes points within 0.005 of
the threshold on the referee and within 0.01 on the fast tiers (TARGET). A run exactly at the threshold must carry the
boundary flag. The wheel-loop row is also a mutation test for the actuator stage: at a 0.5 ms step, explicit Euler
that updates the current before the wheel speed gives 1.854 and 1.497, which turns the stable case into an unstable
one.
What the analytic layer says about the mission: in the creep limit design A docks in about one condition in ninety, design B in fewer still because its 32 N force cap binds at the rear lip (the derivation file carries B's pitched geometry), and for both the rear axle binds, not the front. Whatever difference the batch shows between the designs is therefore produced by momentum, impact and the transient of the drive, which is the reason a static screen cannot settle this question. FACT (planning archive, planar sill pilot; Appendix H): in a planar pilot of 156 design-condition rows, 62 passed the dynamic scorer and failed a quasi-static screen, 7 passed both and 87 failed both; no row passed the static screen alone. Changing only the feedforward term from R·i_ref to R·i changed 24 of the 156 labels, taking the 60 mm design from 8 to 18 successes of 78 and the 75 mm design from 61 to 67. The pilot is exploratory and keeps its own law and scorer.
The development batch. 64 independent conditions from master seed 17, on counter-based streams keyed by the world address (17, k, 0) for k = 0 … 63 and by stream name:
| Stream | Draw |
|---|---|
fixture1/mu |
Friction, uniform on [0.35, 0.95] |
fixture1/payload_dx |
Payload fore-aft offset δ, uniform on [−0.04, 0.04] m |
fixture1/heading |
Initial heading, uniform on [−2°, 2°] |
fixture1/edge |
Sharp or filleted lip, a fair Bernoulli draw |
Both designs receive every condition. The 64-row input table, the generator pin, the mapping from generator output to each draw, and the hash of this section's constants are written to the ledger before either design runs (INV-ARENA-04). The nominal analytic case is separate from the batch. These are synthetic challenge distributions and claim nothing about the frequency of sills.
The rule. The study is fixed-n (§24.12). Let n be the number of complete pairs, b the pairs in which only B docks and a the pairs in which only A docks. The effect estimate is (b − a)/n. Form two-sided 97.5% exact Clopper–Pearson intervals [L_b, U_b] and [L_a, U_a] for the two discordance probabilities; the interval [L_b − U_a, U_b − L_a] then covers their difference with probability at least 95%.
| Verdict | Condition |
|---|---|
| Prefer B | L_b − U_a > +0.05, and B violates no hard limit in a condition where A does not |
| B inferior | U_b − L_a < −0.05 |
| Practical tie | The interval lies inside [−0.05, +0.05] |
| Inconclusive | Otherwise |
Both-succeed and neither-succeed counts, invalid pairs and the worst-case range for missing pairs are reported, with unresolved counts given per arm and per action regime, because a policy can enter states where the solver fails preferentially: a candidate with 2 successes in 2 valid trials and 98 unresolved has a complete-case rate of 100% and a success rate anywhere in [2%, 100%], against a baseline at 60 of 100, and its unresolved worlds are re-run on the referee before a ranking that depends on them is promoted. A preference needs complete pairs. If an apparent improvement reverses under a halved step, a refined contact feature or the alternative contact law, the result is reported as fidelity-dependent and both runs are kept.
The pre-analysis. The rule is enumerated before any pair runs. Every count pair (a, b) with a + b ≤ n is
classified; the table is regenerated by eval.preanalyze and compared in CI.
| n | Count pairs | Prefer B | B inferior | Practical tie | Inconclusive |
|---|---|---|---|---|---|
| 64 | 2,145 | 521 | 521 | 0 | 1,103 |
| 86 | 3,828 | 1,031 | 1,031 | 1 | 1,765 |
| 128 | 8,385 | 2,517 | 2,517 | 4 | 3,347 |
| 256 | 33,153 | 11,384 | 11,384 | 39 | 10,346 |
Three consequences follow. First, no outcome at n = 64 can establish a practical tie: perfect concordance gives [−0.0662, +0.0662]. A tie first becomes reachable at n = 86 and only for (a, b) = (0, 0); with 5% discordance each way the chance of that outcome is 0.9⁸⁶ = 0.0116%. Second, with no A-only pair a preference for B needs at least 15 B-only pairs of 64, and 17, 19, 21 and 23 with one to four A-only pairs. Third, the batch has little power for a modest effect. Each alternative below is a pair of probabilities, of a B-only pair and of an A-only pair:
| n | P(prefer B) at discordance (0.17, 0) | at (0.25, 0.02) | at (0.40, 0.02) | P(tie) at (0.02, 0.02) | P(tie) at (0, 0) |
|---|---|---|---|---|---|
| 64 | 0.117 | 0.406 | 0.962 | 0 | 0 |
| 128 | 0.775 | 0.906 | above 0.999 | 0.072 | 1 |
| 256 | 0.998 | above 0.999 | above 0.999 | 0.427 | 1 |
The pair (0.17, 0) is the share of the friction range in which only the 75 mm wheel meets the front lift-off threshold
of the analytic layer; it is a reference effect size, not a prediction. The 64 conditions are therefore a development
batch: they can prefer a design or call it inferior, they cannot declare a tie, and they are not used to tune anything
that the holdout then judges. The holdout is sized from this table for the claim that is wanted, runs through
eval.qualify on sealed conditions (§24.13), and uses the level of its request. A claim of equivalence may instead
register the standalone test that bounds the difference by the total discordance probability: zero discordant pairs in
64 give an exact one-sided 95% bound of 1 − 0.05^(1/64) = 0.0457, inside the five-point band. The inference is
registered before the holdout is opened, and the development batch keeps its original analysis.
Mechanism controls. Two registered follow-ups separate the mechanisms. One sets B's gain to 3.125 A·s/rad to match the force slopes, leaving the caps at 40 N and 32 N. The other is a 2 × 2 of rotor inertia (zero or 0.0002 kg·m²) and control cadence (100 Hz or 1 kHz). FACT (planning archive; Appendix H): in an exploratory planar sweep the two designs' success fractions were 0.093 and 0.780 at 100 Hz with zero rotor inertia and 1.000 and 1.000 at 1 kHz or with added inertia. A comparison that leaves rotor inertia and controller rate unpinned is a comparison of loop stability.
24.11 Fixture 2: a quadruped on a split-grip landing S
Question. Does a wider sole help a quadruped arrive on a landing whose surface has a low-grip strip under one
side? On a uniform Coulomb surface a wider foot has no more translational traction at the same normal load; the
proposed advantage is a change in the share of the sole that rests on the high-grip side of a material boundary. The
fixture, registered as fixture:quad-split-grip, is built so that the limiting model predicts a reversal, which a
count of contact area cannot produce.
| Item | Value (DESIGN) |
|---|---|
| Body | The reference 12-actuator quadruped template of §14 at its pinned DesignId, with identical actuator limits, equal limb masses and a symmetric nominal stance |
| Controller | One crawl controller from Reflex (§22), frozen before either sole is tested |
| Sole A, sole B | Rectangular plates 80 × 40 mm and 80 × 70 mm, the 80 mm side along the direction of travel, on all four feet; the same sole law, sole height and ankle origin; mass and spatial inertia matched by a declared insert. Mounting and leg interference are checked for each, and each design is settled independently |
| Course | A ramp 1 m long rising at 6°, then a level landing 1.2 m long |
| Surfaces | Friction 0.8 everywhere except a longitudinal strip of friction 0.25 on the landing, 0.20 m wide, whose inner boundary runs through the centres of the left soles in the nominal straight gait and whose outer edge lies outside both footprints throughout |
| Command | 0.25 m/s along the ramp from rest; stand once all feet have cleared the crest by 0.40 m |
| Pass | Standing reached within 12 s and held for 1 s; no body strike; roll and pitch below 30°; less than 50 mm of slip per foot over its loaded stance, by the slip metric of §24.8 with a 1 N floor |
| Cases | All-dry, split-grip, and split-grip reflected to the right side |
| Development set | Nominal strip position and offsets of ±5 mm, centred payload |
| Holdout | Ramp at 8°; strip offsets ±12 mm; initial yaw ±2°; lateral payload shift ±20 mm; the full factorial of eight cells with its eight reflected counterparts, and the four dry cells |
| Contact laws | Distributed rigid Coulomb contact as the baseline; then a finer distributed compliant sole with a declared pressure and shear law, with the step halved and the footprint quadrature refined until the classification stops changing |
The limiting model. For uniform normal pressure, the share of a sole of width w that rests on the high-grip side when its centre is displaced by z into that side is f = clip(½ + z/w, 0, 1). Its longitudinal friction capacity per unit normal load is c(z, w) = 0.8·f + 0.25·(1 − f) = 0.525 + 0.55·clip(z/w, −½, ½), and at the slip limit the plate carries a yaw moment of 0.275·N·(w/4 − z²/w) about its centre for |z| ≤ w/2.
| Offset z | Capacity, 40 mm sole | Capacity, 70 mm sole | Yaw moment per newton, 40 mm ; 70 mm |
|---|---|---|---|
| 0 | 0.525 | 0.525 | 2.75 mm ; 4.81 mm |
| +12 mm | 0.690 | 0.619 | 1.76 mm ; 4.25 mm |
| −12 mm | 0.360 | 0.431 | 1.76 mm ; 4.25 mm |
The narrow sole has more capacity when the foot lands toward the high-grip side and less when it lands toward the strip. The two held-out offsets therefore carry opposite predictions, and the wide sole always carries the larger yaw moment. Area alone cannot predict traction when pressure redistributes; the actual load split and moment are measured under each contact law.
Executable known answers. A plate-drag rig realizes the limiting model: one sole plate on a compliant layer under a centred normal load, guided against yaw, pulled along the boundary until it slips. The pull force at slip must equal c(z, w)·N and the guide moment the value above. The slip metric is checked for invariance under duplication of a quadrature point. The reflected cases are a metamorphic relation: the mirrored robot on the mirrored course returns the mirrored result, inside its determinism class.
Decision rule. Results are counts on a finite suite. Reflected pairs are related and are reported as pairs. A sole is preferred only if it completes more held-out cases under both contact laws with no new hard-limit failure. A reversal between the laws is reported as a fidelity-dependent ranking, which is a finding about the tiers and not about the soles. Distance, work, worst slip, peak torque and falls are published for every design and condition. A comparison with a learned residual on top of the frozen controller is a later study with equal training resources.
24.12 Two study types S
A study is a registered plan to answer one comparative or absolute question. It declares its type when it is created and keeps it (INV-ARENA-05).
| Anytime-valid study | Fixed-n exact study | |
|---|---|---|
| Inference | A betting e-process or a confidence sequence (C-15), valid at every stopping time | An exact interval or test at a sample size fixed in advance |
| Stopping | When the evidence crosses its threshold or the budget ends; unfinished episodes are cancelled | At n complete units, never earlier |
| Before the first episode | Null hypothesis, effect margin, bound on outcomes, betting rule, level, budget | n, the rule, the level, and a pre-analysis: which verdicts are reachable at that n, and the power at declared alternatives |
| Reports | The e-value, the confidence sequence, pairs used, the stopping reason | Counts, the interval, the verdict, and the pre-analysis hash |
| Interim information | Any; validity does not depend on it | Progress only. Outcomes may be looked at, and the verdict is computed once, at n |
| Typical use | Screening, racing, promotion between tiers, bisection | Registered A/B questions, equivalence claims, qualification of a finalist |
The types are never mixed inside one study. A fixed-n interval computed at the moment an anytime-valid study stops has no coverage guarantee, and a fixed-n study that stops when its running p-value first looks good loses its level. Counterexample for the second: an exact one-sided sign test at nominal level 5% on pairs that favour either design with probability ½, looked at after every pair. After 8 looks the chance of a false rejection is 13/256 = 5.08%; after 16 it is 8.76%, after 32, 12.9%, and after 64, 17.4%. A request for either mixture is refused (FROB-REF-552).
A fixed-n study without a pre-analysis is not admitted, and a study that names a verdict its rule cannot reach at the declared n and level is refused (FROB-REF-532). Fixture 1 shows why: at 64 pairs its rule cannot return one of its four verdicts.
Four rules hold for both types. Units are paired by world address, and when several episodes share a cluster such as one building, the cluster is the unit of inference. Paired statistics are refused under an uncontrolled determinism class (§35). An N-way race spends α/(N − 1) per comparison, so that a null race keeps its best candidate with probability at least 1 − α. And a comparison of designs declares its controller arm: a shared controller isolates the change of body, controllers retuned with equal resources compare the best systems, and a candidate that inherits weights declares its warm start and receives a matched transfer baseline.
24.13 The blinded qualification lane S
An e-process makes one comparison valid under optional stopping. It does nothing about an agent that submits fifty
candidates to the same held-out scenes until one passes. With independent attempts at level 5% each, fourteen null
submissions give a 51% chance of at least one false promotion, and five hundred give certainty for practical purposes.
The qualification lane closes that hole. Its verb is eval.qualify.
Lanes. The discovery lane is everything an agent may read while it designs: training scenes, the development
splits, batteries, the hard-world set, its own runs and telemetry. The confirmation lane is a service inside frobd
that owns a sealed family of worlds and runs registered requests against it. A campaign is a set of confirmation
requests that share a claim family, meaning a task, a baseline class and a held-out world family, and a total error
budget α. All agents working on the claim family share the campaign and its index sequence. Opening a campaign needs
the capability qual:admin; an agent cannot reset its index by opening another.
The protocol for request i of a campaign.
- Commit. The request names candidate, baseline, task, claim, study type and rule, tier plan, number of clusters and budget. It is hashed into the ledger with an idempotency key before any confirmation world exists (INV-QUAL-04). For a fixed-n study the pre-analysis is computed at this request's level and attached.
- Spend. The request receives the level α_i = α / [i·(i + 1)] and the ledger records it. The index advances for every committed request, including ones that end inconclusive or are refused after commitment (INV-QUAL-03).
- Draw. The service derives fresh world clusters for this request from a sealed key, the campaign identifier, the index i and a cluster counter. A cluster is the unit that shares a generator draw, such as one building with its materials. No cluster serves two requests and none has appeared in any discovery dataset (INV-QUAL-02).
- Run. Candidate and baseline run paired on the clusters at the tier plan, with promotion to a higher tier by the rule of §16. The study runs at level α_i with clusters as units.
- Answer. The agent receives a receipt and nothing else: the decision, the index, the level spent, the number of units used and a hash.
- Close. When the campaign closes, the key is revealed, the clusters and their episodes become readable, and the family is retired into discovery. It never serves as a confirmation family again.
{ "schema": "frob.qualify.request/1", "campaign": "q:threshold-delivery/3", "index": 7,
"alpha_total": "0.05", "alpha_i": "0.05/56",
"candidate": {"design": "d:7c03…", "controller": "c:91ab…"},
"baseline": {"design": "d:41aa…", "controller": "c:91ab…"},
"task": "t:cart-sill-dock@6be1…", "claim": "candidate docks more often than baseline by more than 0.05",
"study": {"type": "fixed-n", "n": 256, "rule": "paired-clopper-pearson/1", "preanalysis": "b3:5d10…"},
"tier_plan": {"run": "batch-cpu", "promote_to": "referee"}, "clusters": 256, "budget": {"cpu_core_s": 90000},
"commit": {"ledger_seq": "18812", "hash": "b3:e41f…", "idempotency_key": "k:2c9a…"} }
{ "schema": "frob.qualify.receipt/1", "campaign": "q:threshold-delivery/3", "index": 7,
"alpha_i": "0.05/56", "alpha_spent": "0.05*7/8", "decision": "not-promoted", "units_used": 256,
"sealed_until_close": ["world addresses", "per-unit outcomes", "telemetry", "stills"],
"receipt_hash": "b3:90c2…", "authorization": {"execution_authorized": false,
"hardware_safety_signoff": false, "sim_to_real_certified": false} }
Why the error rate holds. Because 1/[i·(i + 1)] = 1/i − 1/(i + 1), the levels sum to α·m/(m + 1) after m requests and to α in the limit (PO-ARENA-02). Request i tests a candidate that the agent chose with everything it had seen before, on clusters that are independent of everything it had seen before, so conditional on the past the chance of a false promotion at request i is at most α_i. A union bound over requests gives a campaign-wide chance of any false promotion of at most α, for any number of retries and any adaptive strategy. The argument needs three things and the lane enforces exactly those: the request is fixed before the draw, the clusters are fresh, and the level is spent whether or not the agent likes the answer.
What blinding adds. Freshness alone controls the error rate. Blinding protects the claim's population. If an agent
could read confirmation worlds, they would inform its next candidate, the held-out family would stop being held out,
and a promotion would describe worlds the design was fitted to. So discovery cannot read confirmation worlds
(INV-QUAL-01): no verb returns a sealed address, outcome, trace or still to a session without qual:admin before the
campaign closes, and an attempt is refused and logged (FROB-REF-840). If a reversal on a sealed family is ever used to
inform training or selection, that family is retired at once.
What a retry costs. The sample size of a test grows with ln(1/α_i). Relative to a single test at α = 0.05, request 1 needs about 1.23 times the units, request 10 about 2.57 times, request 100 about 4.08 times and request 500 about 5.15 times. A fixed-n rule changes with the level too: for the rule of fixture 1 a practical tie first becomes reachable at n = 99 for request 1, n = 178 for request 10 and n = 328 for request 500, and at 64 pairs request 1 has 478 preferring outcomes of 2,145 where the development level had 521. Retrying is allowed and is never free.
Standing. A result is qualified only with a receipt whose decision is promoted. Reports cite the campaign, the
index and the level. Citing a discovery-lane comparison as a qualification is a protocol refusal (FROB-REF-845). The
receipts, the ledger of levels and, after close, the cluster derivation are part of the .frobcert bundle and are
replayed by frob-verify (§12).
24.14 Verbs and refusals
Every verb of kind run is a long tool in the sense of §32: it accepts a bounded synchronous wait and otherwise
returns a job handle.
| Verb | Kind | First milestone | Notes |
|---|---|---|---|
task.new, task.show, task.list |
mutate, read | M1 | Tasks are content-addressed |
task.check |
read | M1 | Units, clocks, windows, reach against horizon, labelled examples, mutation check |
task.verdict |
read | M1 | Evaluates the requirements of a task on a recorded run and returns frob.verdict/1 |
task.settle |
run | M1 | Builds or refreshes the settled-pose cache for a model and task |
task.fixture |
run | M1 | Runs a registered fixture's analytic layer or batch and reports every row against its known answer |
eval.run, eval.scorecard |
run, read | M2, M3 | Scorecards with the outcome columns of §24.8 |
eval.compare, eval.promote |
run | M3 | Defined in §28 and §16; each belongs to a study of §24.12 |
eval.preanalyze |
read | M1 | Enumerates the reachable verdicts and the power of a fixed-n rule at a given n and level |
eval.qualify |
run | M3 | Submits a committed request to a campaign and returns a receipt |
eval.ledger |
read | M3 | The campaign's index, levels spent and receipts; nothing sealed |
| Code | Refusal |
|---|---|
| FROB-REF-502 | An evaluation reuses a scene from the policy's training manifest (carried from §19) |
| FROB-REF-520 | An atom has no scale and is not declared exact |
| FROB-REF-535 | Dimension mismatch inside an atom |
| FROB-REF-536 | A temporal operator has no finite window |
| FROB-REF-523 | A future-time operator appears in an online signal |
| FROB-REF-538 | A window bound is not a multiple of the clock period and no rounding is declared |
| FROB-REF-525 | A signal is read across clocks without a declared reduction, or its channel kind does not admit the reduction |
| FROB-REF-526 | The episode horizon is shorter than the reach of the verdict |
| FROB-REF-539 | A requirement has no labelled examples, or the monitor disagrees with a label |
| FROB-REF-528 | An online window exceeds its capacity |
| FROB-REF-529 | No settled-pose cache for this model and task |
| FROB-REF-530 | A run offered as a fixture run does not match the fixture's recorded hash |
| FROB-REF-552 | A request mixes study types |
| FROB-REF-532 | A fixed-n study has no pre-analysis, or names a verdict it cannot reach |
| FROB-REF-533 | Paired statistics requested under an uncontrolled determinism class |
| FROB-REF-840 | A session without qual:admin asks for sealed confirmation data before close |
| FROB-REF-841 | A confirmation request was not committed first, or differs from its commitment |
| FROB-REF-842 | The units the request needs at its level exceed its budget; the reachable verdicts are attached |
| FROB-REF-843 | The candidate's lineage contains data from the sealed family |
| FROB-REF-844 | A cluster would be used twice |
| FROB-REF-845 | A result is cited as qualified without a promoting receipt |
24.15 Invariants and proof obligations
- INV-ARENA-01. A verdict is a pure function of the trace and the
TaskId. It never depends on a reward term. - INV-ARENA-02. No requirement value is computed from a truncated or incomplete window.
- INV-ARENA-03. Every scenario of a battery appears in exactly one outcome column, and the columns sum to the battery size.
- INV-ARENA-04. A fixture's constants, seeds, input table and rule are hashed into the ledger before its first run.
- INV-ARENA-05. A study has one type for its whole life.
- INV-QUAL-01. Discovery cannot read confirmation worlds.
- INV-QUAL-02. Confirmation clusters are fresh: no cluster serves two requests, and none is in any discovery dataset before the campaign closes.
- INV-QUAL-03. The ledger of levels is append-only, the index advances by one for every committed request, and the total spent never exceeds α.
- INV-QUAL-04. A request's commitment precedes the derivation of its clusters in ledger order.
| Obligation | Statement | Class sought |
|---|---|---|
| PO-ARENA-01 | The online block kernels, the deque monitor and the definitional evaluator agree on every past-time operator | Tested: differential, bit for bit |
| PO-ARENA-02 | Σ α/[i·(i + 1)] over i ≥ 1 equals α, and the campaign error is bounded by the sum | A candidate theorem card for §12; Tested by exact rational summation until a checker accepts it |
| PO-ARENA-03 | The pre-analysis enumerator is exact: counts are integers, and interval endpoints are rounded outward | Tested against an independent enumeration |
| PO-ARENA-04 | A positive margin implies the formula holds and a negative margin that it fails | Tested as a law on random traces |
24.16 Known answers
| Known answer | Expected | Negative control |
|---|---|---|
| Fixture 1, analytic layer | Every row of the table in §24.10 within its tolerance | The wheel-loop row under explicit current-then-wheel Euler at 0.5 ms; a creep rig with the rear lip removed |
| Fixture 1, rule enumeration | 521, 521, 0 and 1,103 at n = 64; first tie at n = 86 for (0, 0) only; 15 B-only pairs for a preference with no A-only pair | A request for a tie verdict at n = 64 is refused |
| Fixture 1, energy accounting | 10 J against 12 J and 30 J against 12 J for the two-trial example | Dropping failed attempts from the total |
| Fixture 2, limiting model | The table of §24.11 on the plate-drag rig, within 1% on the referee | A pyramidal friction cone; a sole with one quadrature point |
| Slip metric | Unchanged when a quadrature point is duplicated with halved weights | A metric that sums unweighted point speeds |
| Normalization | The binding atom of the two-atom example is the same in metres and millimetres | Composition of raw margins |
| Boundary comparator | At U = 3/4 and ε = 5/16 the strict docking requirement fails; the 65,536-case screen has no final error with the guard | A non-strict comparator admits the equality case; unguarded single-precision comparison misclassifies the screen |
| Online and offline agreement | A past-time signal sampled at the end of the trace equals the offline evaluation of the same formula | A monitor that pads a short window |
| Peeking | The sign-test example gives 13/256 after 8 looks | The same data under a confidence sequence stay below their level |
| Joint stage accounting | 18 or 19 of 20, never 18.05 | Multiplying stage rates |
| Level series | Exact partial sums α·m/(m + 1) for m up to 500 | A constant level per request |
24.17 Gates, kill gate and pilots
| Gate | Claim | Threshold (TARGET) | Earliest |
|---|---|---|---|
| GATE-ARENA-01 | Known answer | Every task is reproducible inside its determinism class from its TaskId, SceneId, ModelId and seed; reference-robot baseline scorecards are recorded |
M2 |
| GATE-ARENA-02 | Known answer | The scene-split check rejects an evaluation that reuses a training scene | M3 |
| GATE-ARENA-03 | Known answer | Monitors: the three evaluators agree bit for bit on 10⁶ random traces per operator, including multi-rate signals and windows with rounding; every requirement-language refusal of §24.14 (FROB-REF-520, -535, -536, -523, -538, -525, -526, -539 and -528) is triggered by its planted task file | M1 |
| GATE-ARENA-04 | Known answer | Fixture 1: every analytic row within tolerance on the referee and inside the fidelity envelope on the batch tiers; the creep rigs flip at the stated thresholds; both negative controls are caught | M1 on the referee; M2 on the batch tiers |
| GATE-ARENA-05 | Known answer | eval.preanalyze reproduces the tables of §24.10 exactly and refuses the unreachable verdict |
M1 |
| GATE-ARENA-06 | Known answer | Fixture 2: limiting-model rows within 1% on the referee; slip invariance to 10⁻¹² relative in double precision; reflected cases mirror inside the determinism class | M2 |
| GATE-ARENA-07 | Known answer | Scorecard accounting: a planted infeasible scenario appears in its column and in the all-scenario denominator; columns sum to the battery size on 100 random batteries; a silently shortened battery changes identity | M3 |
| GATE-ARENA-08 | Known answer | Reset cache: penetration at most 0.5 mm and first impulse within 5% of static support on all 64 states for each reference robot; a design edit invalidates the cache | M2 |
| GATE-ARENA-09 | Design comparison | Requirement writing: on 50 scripted mission statements, each with a reference formula and 20 labelled trajectories the agent does not see, at most 20% of first-draft requirements mislabel a trajectory, and task.check rejects every draft that does |
M3 |
| GATE-QUAL-01 | Known answer | Over 2,000 simulated campaigns on no-signal worlds at α = 0.05, each with 500 adaptive retries by a scripted adversary that sees every receipt and all discovery data, a false promotion occurs in at most 123 campaigns (an exact one-sided test of a rate at or below 0.05 rejects at the 1% level from 124). A planted true gain of the declared size is promoted within 10 requests in at least 80% of campaigns. Negative control: the same adversary against reused clusters at a constant level obtains a false promotion in more than 99% of campaigns | M3 |
| GATE-QUAL-02 | Known answer | Firewall: 10⁴ fuzzed read attempts across every verb return no sealed field; a planted leak path is found by the audit; receipts contain only the listed fields | M3 |
| GATE-QUAL-03 | Known answer | frob-verify replays the levels, the commitments and, after close, the cluster derivation of a campaign; a reordered index, a changed request and a reused cluster are each rejected |
M5 |
| ✖ KILL-SPEC-REWARD | — | Trigger: in the training pilot below, the arm with the added term does not improve success over the shaped reward across five seeds. Fallback: requirements stay acceptance metrics and search scores only, and the added-term option is removed from the task schema. The margin-only arm is run to record whether it reaches 80% of the shaped arm's success; below that, margin-only training is not offered | M3 |
Pilots before funding. The requirement-writing pilot is GATE-ARENA-09 run on paper: one day, no GPU. The training pilot for the Frontier reward term is one quadruped velocity-tracking task under three arms, shaped reward, smoothed margin only, and both, with five seeds each, about ten GPU-hours on a development oracle outside the product (§38). The first fixture needs no pilot; its analytic layer is the pilot.
24.18 Open questions
- OPEN-ARENA-1. Which public task definitions to mirror for comparability. Default: mirror the reward terms of the public velocity-tracking tasks and make no claim of score parity.
- OPEN-ARENA-2. A permissively licensed hexapod reference model. Default: none; the template stands alone.
- OPEN-ARENA-3. The size and composition of the sealed world families, and who holds
qual:admin. Owner: project lead. Blocks GATE-QUAL-01. Default: the owner alone opens and closes campaigns; one sealed family per task family, sized for 20 requests at 256 clusters; no result is called qualified until a family exists. - OPEN-ARENA-4. Whether fixture 2's quadruped is the reference template or a dedicated minimal body. Owner: Arena
lead. Blocks GATE-ARENA-06. Default: the reference template at a pinned
DesignId. - OPEN-ARENA-5. The guard band for boundary atoms on each backend. Owner: Spark lead. Blocks the device half of GATE-ARENA-03. Default: 2⁻²¹ with CPU re-evaluation of every flagged atom.
24.19 Traction capacity is a slice of the wrench set at the commanded wrench
What it is. S A metric, and two analytic rows for Fixture 2 (§24.11), used in every comparison of soles, tyres and pads. The scalar friction capacity Σ μᵢ·Nᵢ is the largest longitudinal force with the yaw moment left free. A robot commands a wrench. The capacity that ranks two designs is the largest force along the commanded direction among contact forces that also deliver the commanded moments.
Counterexample. A two-point sole carries 50 N on each point, at lateral offsets ±20 mm, one point on μ = 0.25 and the other on μ = 0.8. The largest forward force is 12.5 + 40 = 52.5 N, and it comes with a yaw moment of (40 − 12.5) N × 0.020 m = 0.55 N·m. If the command is a pure forward force, the two longitudinal forces must be equal, and the largest is 2 × 12.5 = 25 N. A sole on uniform μ = 0.4 supplies 40 N at zero yaw. By scalar capacity the split sole wins, 52.5 N against 40 N. At the commanded wrench it loses, 25 N against 40 N.
Mechanism. DESIGN:
traction_at_wrench(direction, moment command)is a cone program over the contact wrench set of C-06: maximize the force along the direction, subject to the per-point friction cones, the measured normal loads and the moment equalities. It is the program of §22.5 with another objective, so it needs no new solver. Owner:frob-controlfor the program,frob-arenafor the metric,frob-analyzefor the static margin of A-08.- Scorecards (§24.8) report the scalar capacity under the name
capacity, moment free, and the slice under the name of the commanded wrench. A comparison of contact geometries cites the slice. - Fixture 2 gains two rows. The plate-drag rig runs once guided against yaw, as now, and once free to yaw, which enforces zero moment physically. For the uniform-pressure strip model with the boundary through the centre and longitudinal traction only, the zero-yaw capacity per unit normal load is (μ_L + √(μ_L·μ_H))/2, which is 0.3486 for μ_L = 0.25 and μ_H = 0.8, against 0.525 with the moment free, and it does not depend on the sole width. The closed form is a lower bound when lateral traction may also carry moment; the cone program gives the exact value.
Known answers. Two-point rig: 52.5 N and 0.55 N·m guided; 25 N free; 40 N both ways on uniform μ = 0.4. Strip model at zero offset: 0.525·N guided for both widths; at least 0.3486·N free for both widths.
| GATE-ARENA-10 | Known answer. The cone program reproduces 52.5 N, 0.55 N·m, 25 N and 40 N to 10⁻⁹ in double precision; the two-point rig slips within 2% of each value on the referee and inside the fidelity envelope on the batch tiers; a scorecard that ranks the two soles by moment-free capacity when the task commands zero yaw is rejected by task.check. M2. |
24.20 Auditing a task before any design runs on it: foil bodies and intent invariants
Task audit F; intent invariants S. The detectors of §23.16 ask whether a good score is an artifact of the
solver. A behaviour that is physically real and unintended passes every one of them: sliding on the belly where
walking was meant, tumbling down a flight to reach the goal region, riding a wall, satisfying the predicate only at
the instant it is checked. Faster search finds the cheapest such loophole sooner, and one loophole invalidates every
evaluation of the campaign that ran under it. task.audit attacks the task itself before any candidate is scored.
Foil bodies. A library assets/foils/ of bodies that are deliberately over-capable or degenerate, written in the
design language and marked foil: true. Each runs in the task's own scene and is scored by the task's own
requirement and cost. A foil is never a candidate: offering one to opt.*, eval.compare or eval.qualify is
refused.
| Foil | What it is | What it probes |
|---|---|---|
| Free centroidal body | One rigid body with the mass and inertia of the design class, applying bounded wrenches at the contact sites the task designates, inside friction cones; no limbs, no actuator limits | Whether the predicate can be met by forces alone, without the intended mechanism |
| Relaxed family member | The current design with card limits multiplied by 3 (DESIGN) and collision enabled on every link | Behaviours the real design is only barely kept from |
| Degenerate shapes | A sled, a ball, a tumbling box and a single-leg hopper of the class's mass | Sliding, rolling, tumbling and bouncing routes to a satisfied verdict |
Search. The sampling controller of §22.8 and rank-based search over open-loop parameters, filling a quality-diversity archive (§28.8) over behaviour descriptors: the fraction of time each site class touches the ground, the range of base attitude ‖log R‖, the airborne fraction, and the sliding fraction (tangential speed at stance contacts above a card threshold). The objective is many distinct high-scoring behaviours, not one optimum.
Intent invariants. Past-time predicates in the requirement language of §24.5 over channels Chronicle already records. A task family declares which it adopts.
| Invariant | Signal |
|---|---|
| Contact only at designated sites (feet, wheels, tracks) | Contact provenance per site |
| Attitude excursion inside a bound | ‖log R‖ of the base |
| Airborne and sliding fractions inside their bounds | Contact state; tangential speed at stance contacts |
| No net positive work by a contact | The contact-active monitor of §29.9 |
| The body stays inside the route corridor | The scene's route graph (§19) |
| Dwell: success is held for a declared time, and is not first reached in the last 5% of the horizon (DESIGN) | The verdict's own online signal |
Findings and their resolution. A finding is (foil, behaviour cell, score, violated invariant, replay address,
proposed clause), the clause being a typed edit of the requirements section. The author accepts the clause, which
yields a new TaskId because a task is content-addressed, and the foil's trace is added as (example … violated),
so the amended requirement is tested against the loophole from then on. Or the author marks the behaviour intended,
and the trace is added as (example … satisfied). A high-scoring degenerate behaviour that violates no adopted
invariant is reported unclassified. Campaign results cite the TaskId they ran under. eval.qualify on a task
with an unresolved finding, or with no audit record for its TaskId, is refused. The audit runs on discovery worlds
only and reads no confirmation world.
In-campaign check S. The winning episodes of the top-k designs are evaluated against the adopted invariants. A
design whose wins violate one is flagged wins-outside-intent and cannot be promoted until the author resolves it.
Hand case. Goal region x ∈ [4.9, 5.1] m and requirement (F (0 s 10 s) in_goal). A sled foil moving at a constant
1 m/s is inside the region for t ∈ [4.9, 5.1] s and never stops: the verdict is satisfied with a dwell of 0.2 s. The
proposed clause is (F (0 s 10 s) (H (0 s 2 s) in_goal)). The sled's longest dwell is 0.2 s < 2 s, so the amended
requirement is violated by the foil; a body at rest at x = 5 m from t = 6 s satisfies it from t = 8 s.
Owners and verb. frob-arena owns the audit, the foil library and the invariants; frob-control and frob-opt
supply the search; findings are stored by frob-chronicle keyed by TaskId. Verb: task.audit (run; budgeted; a
long call returns a job handle).
Ceiling, pilot and gate. Ceiling: the compute of campaigns that a loophole would have invalidated. How often
authored tasks contain one is unmeasured. FACT (§2.2, source register): in one published agent design loop 15.8% of
trained candidates were degenerate after a full training run. HYPOTHESIS: at least one accepted finding in a fifth of
authored tasks; test: the count over the task families of §24.3 at M3. Pilot before funding: task.audit on ten
authored tasks with a budget of 5% of a reference campaign each. If no finding is accepted by an author, only the
in-campaign check is built.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-ARENA-11 | Known answer; Design comparison | The hand case exactly. On 30 tasks with one planted loophole each (off-site contact, tumbling, off-route travel, last-instant success, energy-creating contact) and 30 clean tasks: at least 80% of planted loopholes found before any design campaign, at no more than 5% of a reference campaign's compute per task, with findings on at most 20% of the clean tasks | M3 |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-FOIL | Detection below 60%; or findings on more than 30% of clean tasks; or the sampling controller on the actual design finds the same loopholes at the same cost, so foil bodies add nothing | The in-campaign check on winning episodes only; task.audit runs the actual design alone |
24.21 The item bank: importance sampling of battery items for close comparisons
Item bank F. The cost of a population decision scales with designs × battery items × replicates. Racing and the ladder work on the first factor and the tiers on the cost of an episode. Nothing so far models the items, although most items say nothing about two close designs: scenes both pass and scenes both fail. The item bank keeps a persistent model of items and uses it only to choose which items a comparison runs. The estimand never changes.
Item, law and bank. An item is (scene instance, task, disturbance law, family of initial conditions), identified by content hash. A battery version fixes a law p(i) over its items; p is the estimand's weight and is never edited by the bank. The bank stores, per morphology class and battery version, a response model fitted by regularized maximum likelihood on every (design, item, outcome) row of discovery data in the archive, in canonical row order:
success: P(Y = 1 | d, i) = σ(a_iᵀ·θ_d − b_i), θ_d ∈ ℝᵏ, k ≤ 5 (DESIGN)
continuous: Y | d, i ~ N(μ_i + a_iᵀ·θ_d, s_i²)
The fit has a version, and the version is part of the identity of every comparison that used it.
Proposal and estimator. For a contrast of designs A and B with paired differences D_i = Y_B,i − Y_A,i,
q*(i) ∝ p(i)·√(E_model[D_i²]), q(i) = (1 − ε)·q*(i) + ε·p(i), ε = 0.1 (DESIGN),
Ĉ = (1/n)·Σ_j w(i_j)·D_{i_j}, i_j drawn from q by an addressed stream, w(i) = p(i)/q(i) ≤ 1/ε.
E_q[w·D] = Σ_i q(i)·(p(i)/q(i))·E[D_i] = Σ_i p(i)·E[D_i]: the estimate is unbiased for the contrast under the frozen law whether or not the response model is right. The model buys efficiency and nothing else. Items that do not discriminate are down-weighted in q and never removed from p.
Arithmetic, exact. Let D be ±1 on a fraction f of items and 0 elsewhere, with mean μ. Uniform sampling has variance f − μ². Under q* the weight is f on the discriminating items and the variance is f² − μ². At f = 0.1 and μ = 0.05 these are 0.0975 and 0.0075, a ratio of 13. With the defensive mixture at ε = 0.1 the weight on a discriminating item is 1/(0.9/f + 0.1) = 0.1099 and the variance is 0.1 × 0.1099 − 0.0025 = 0.0085, a ratio of 11.5.
Counterexample: a wrong model costs up to 1/ε. Let the model place q* on a set of items disjoint from the items that discriminate. Each discriminating item then has q = ε·p and weight 10, the second moment is f × 10 = 1.0, and the variance is 0.9975 against 0.0975 under uniform sampling: 10.2 times worse, with the estimate still unbiased. The mixture bounds the damage at 1/ε. It does not remove it, and the gate therefore measures realized interval width and not the model's own forecast.
Rules.
- Anytime studies on discovery worlds only. The weighted increment w·D lies in a span 1/ε times the span declared for D, so a range-driven sequence would lose by (1/ε)² in sample size. Item-bank sampling is admitted only with the variance-adaptive betting sequence, and the span it declares is the widened one.
- Refused for
fixed-nstudies and inside the blinded qualification lane. The exact rule of §28.6 counts discordant pairs under the frozen law, and confirmation draws from p on fresh sealed clusters. The fit never reads a confirmation world. - Not composed with the tier-correction estimator. Promotion pairs are drawn from p; multiplying the two weights would allow 10 × 20 = 200.
- A rare-failure bound refuses item-bank episodes, as it refuses any episode not drawn nominally (§26).
- Replicate noise is untouched. Variance between seeds inside an item is not reduced by choosing items.
What an agent reads. Beside the verdict: the episodes used and the uniform-equivalent count, and the separating items ranked by discrimination ‖a_i‖ with their difficulty b_i, which says in which capability the two designs differ.
Owners and verbs. frob-arena owns the battery law and the bank; frob-opt owns the proposal and the estimator;
rows come from frob-chronicle. eval.compare --sampling item-bank runs a comparison; eval.items (read) returns
the bank rows for a battery and class with their sample counts and fit version.
Ceiling and expectation. The idealized ratio is 1/f when a fraction f of items discriminates. With model error, the mixture and replicate noise the expectation is 4 to 8 (HYPOTHESIS; test: the replay study below). The saving multiplies with the tier correction of §29.11, which lowers the cost of an episode instead of their number.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-ARENA-12 | Known answer | The two-value fixture reproduces 0.0975, 0.0075 and 0.0085 in exact arithmetic; under the disjoint wrong model the mean of Ĉ over 10⁴ replays equals the true contrast within its Monte Carlo error and the variance ratio stays at or below 1/ε; a fixed-n request with item-bank sampling is refused |
M3 |
| GATE-ARENA-13 | Design comparison | Replay on a recorded full matrix of about 40 designs × 150 items × 4 replicates (24,000 episodes): for the 20 closest pairs, episodes to equal interval width lower by a factor of at least 4; coverage within 2 points of nominal; item parameters fitted on one family predict held-out designs better than per-item means | M3 |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-ITEMBANK | At GATE-ARENA-13 the factor is under 2 for the 20 closest pairs; or coverage falls more than 2 points below nominal; or the held-out prediction is no better than per-item means | Stratified sampling by interior archetype with proportional allocation, which needs no model |
24.22 Exposure domains and the coupling test
Exposure domains S; chosen couplings F. This extends the randomization section of the task file (§24.4),
pairing rule 2 of §28.6 and the keyed streams of §35.9. Those rules say that a draw is keyed by the physical entity
it perturbs. What is still unstated is which kind of entity each draw belongs to, what happens to a disturbance that
a mission event triggers, and how anyone checks that a coupling left each arm's own distribution alone. Every draw
in randomization now declares one of four exposure domains, and the task compiler in frob-arena lowers the
section to a versioned exposure table that is part of the task's identity.
| Domain | Key, beside WorldAddr and stream_id |
Meaning | Example |
|---|---|---|---|
field |
An anchored coordinate cell or a fixture identifier | Sampled once per world at stable coordinates; a robot reads it where it is | Friction of a floor tile; the mass of a cart in the scene |
process |
A node of the dyadic tree over physical time (§16.5) | An exogenous process in physical time; a change of step or control rate reads the same path | A gust; a push scheduled at 3 s |
event |
A mission role and an occurrence index | Exists only where the task defines the exposure by a declared mission event; its physical time may differ between arms by definition | A push when the base first crosses the docking gate |
private |
The hash of a port name | Noise with no counterpart in the other arm | Noise of a sensor that only one design carries |
Roles are the typed ports and task regions of §14 and §19, so they survive edits. An event draw is legal only when
the exposure is defined by a mission event in the task; keying by the order of contacts, footfalls or draws remains
refused (§28.6).
The coupling test. task.check --coupling runs four checks on a task and a pair of controllers or designs. A
coupling that fails one is refused.
- Marginal invariance. For each arm, outcomes under the coupled protocol and under independent sampling agree in distribution within a tolerance fixed in advance. A coupling may change the joint law of the two arms and nothing about either arm alone.
- Insertion invariance. Adding unused draws, a sensor or a limb changes no exposure of another domain entry (the sensor-set property of §20.17, applied to exposures).
- Rate invariance. Halving the physics step changes no
fieldvalue, and aprocessincrement over a coarse interval equals the sum of its fine increments. - Retiming. Under a deliberately retimed controller, a
fieldstays at its coordinates, aprocesspush occurs at the same physical time, and aneventpush occurs at each arm's own crossing.
Counterexamples, hand-checkable. A push keyed by control tick 1,500 occurs at 3.0 s with a 2 ms step and at 1.5 s with a 1 ms step. A push keyed by "the tenth footfall" occurs at 5.0 s under a gait period of 0.5 s and at 4.0 s under 0.4 s: a disturbance meant to be timed now follows gait phase, and the two arms were pushed in different states. The first keying fails check 3 and the second fails check 4.
Choosing among legal couplings F. Where more than one legal keying exists (friction by tile or by fixture), the choice is made on pilot addresses by the measured variance of the contrast, the table version is then locked, and confirmation addresses are drawn only afterwards. Both marginals and the contrast variance are recorded. For equal marginal variances σ², Var(D) = 2σ²·(1 − ρ): moving the paired correlation from 0.50 to 0.95 lowers the variance of a contrast ten times at a fixed count. That is an illustration of what is at stake, not a prediction.
Owners and verb. frob-arena owns the compiler and the test; frob-scene caches field samples per world; the
stream registry of §35.9 holds the domains. Verb: task.check --coupling. A paired study on a task whose exposure
table has not passed the test is refused.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-ARENA-14 | Known answer; Design comparison | Fixture: a slick strip over x ∈ [2.0, 2.5] m (field, per 0.25 m tile), a push at t = 3 s (process), a push at the first crossing of a gate at x = 5 m (event), a baseline controller and one retimed from a 0.5 s to a 0.4 s gait period. With 1,000 unused draws inserted, the step halved and a limb added: tile values bit-identical; the timed push at 3 s in every arm; the gate push at each arm's crossing; per-arm outcomes indistinguishable from independent sampling in a two-sided test at level 0.01 on 4,096 worlds. The tick-keyed and footfall-keyed controls are refused. On a held-out slip-and-dock battery the variance of the contrast is at most half that of named streams keyed per stream and tick, at equal count |
M2 for the fixture; M3 for the battery |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-EXPODOMAIN | Under a factor of 2 in contrast variance on the held-out battery; or the automatic choice among couplings picks one that fails the test in any pilot | The keyed streams of §35.9 as they stand, with explicit per-stream adapters; the four checks are kept as a test |
24.23 The idle arm of the task audit
What it is. S One more arm for task.audit (§24.20). The foil bodies probe ways of satisfying a task without
the intended mechanism. The idle arm probes the cheapest of them: doing nothing. A requirement or a reward built from
safety atoms alone is satisfied in full by a robot that never leaves its start. Owner: frob-arena. Verb:
task.audit.
Hand case. A delivery task uses the past-time requirement (H (0 s 2 s) (and clear-margin thermal-ok spill-ok))
as its online reward over 60 s. A parked robot keeps all three margins positive for the whole episode and collects the
full reward. A robot that carries the tray through the pass and loses part of its spill margin for one second collects
less. Search prefers the parked robot.
Mechanism. DESIGN.
- The audit runs the candidate design, and each foil, under three idle laws: zero command with the brakes as carded; a hold-posture command from the baseline controller of §22.4; and the backup controller of §22.12.3.
- A task is flagged
rewards-idlenesswhen an idle run's requirement verdict issatisfied, or when its return is at least a declared fraction of the best audited return (DESIGN default 0.5). - The finding carries a proposed clause, like every finding of §24.20: a progress atom with its scale (for the hand
case, approach speed above 0.15 m/s on a scale of 0.50 m/s, which gives the parked robot a margin of −0.3), or a
completion requirement with a deadline. The author accepts it, which yields a new
TaskId, or marks idleness intended, as a station-keeping task would. eval.qualifyon a task with an unresolved idleness finding is refused, under the existing rule for unresolved findings.
| GATE-ARENA-15 | Known answer. On ten authored tasks with a planted safety-only reward and ten clean tasks, the idle arm flags ten of ten and none of ten (TARGET: at least nine, at most one). The hand case is flagged, and after the progress clause is accepted the parked robot's verdict is violated. M3. |