Cortex: the gym, learning, family policies, inheritance
12,910 words29 subsections21 tables3 listingsPlan version 1.1
On this page29
23.1 What it owns
Cortex owns the vectorized environment and its semantics, the learning algorithms and their correctness rules, the networks, policy checkpoints with their manifests, distillation, family policies, inheritance along design edits, and the detectors that keep a learned result from resting on a simulator artifact (GOAL-14, GOAL-16).
| Crate | Layer | Owns |
|---|---|---|
frob-learn |
4 | VecEnv, tensor views, rollout storage, PPO and the other algorithms, networks, on-device learning kernels' host side, the FrankenTorch bridge, checkpoints, distillation, family policies, inheritance, exploitation detectors |
bindings/frob-gymnasium |
annex | The optional Gymnasium-compatible adapter |
Task terms are written in .ftask and compiled by frob-spec (§24) into a TaskProgram of kernel stages,
declared beside the step plan in layer 3, which frob-learn consumes.
What learning is for here. Training is the dominant cost of published design loops (FACT, Appendix H). The plan's lever is to train fewer designs: screen with Reflex, reuse with family policies and inheritance, and spend specialist training on finalists. Engine throughput is a parity requirement: an infinitely fast environment step speeds training by at most about 7.7 times on one incumbent stack and about 1.4 times on another (FACT, Appendix H).
23.2 Decisions
| ADR | Decision | Reason |
|---|---|---|
| ADR-CORTEX-01 | Explicit reset is the default, and the final observation of every ended world is preserved in every mode. | A masked automatic reset that returns post-reset observations loses the observation a truncation bootstrap needs. |
| ADR-CORTEX-02 | The Rust VecEnv is the stepping authority. An optional Gymnasium-compatible adapter lives in an annex. |
Rival learning stacks must be able to drive this environment in the ten-fold test (§39) without Python entering the core. |
| ADR-CORTEX-03 | PPO is specified by rules with tests. No PPO implementation, ours or a sibling's, is admitted without passing them. | The suite's only PPO has three errors in its update and a wrong joint law in its action sampler, none of which plumbing tests found (§23.6, §23.25; the count of §33.11 and §33.17). |
| ADR-CORTEX-04 | The policy, the backward pass and the optimizer run on the device as kernels in the shared dialect. FrankenTorch is the CPU trainer, the gradient oracle and the interchange format. | FrankenTorch tensors execute on the CPU only (FACT, §23.8). |
| ADR-CORTEX-05 | Family policies are per morphology class, and one is used as an evaluation rung only after its rank agreement with specialists has been measured. | Within a class the evidence supports 70 to 85% of specialist return with no adaptation; across topology-diverse designs 10 to 40%. No published rank agreement exists (FACT, Appendix H). |
| ADR-CORTEX-06 | Inheritance is the default for parameter edits once GATE-CORTEX-13 has passed for the edit class, and an experiment for topology edits; resets follow a measured regression, with a depth schedule as fallback. | Transfers from a class prior cost about 2 to 25% of an early-stopped from-scratch run, and the one parent-to-child figure is about 50%; single-limb edits 33 to 100%; chained inheritance loses plasticity (FACT, Appendix H). |
| ADR-CORTEX-07 | Weights are tied only under Γ_obs. Augmentation with a symmetry loss is the Solid alternative and the control arm. | §22.13. Augmentation recovers most of the gain in published comparisons (FACT, Appendix H). |
| ADR-CORTEX-08 | A training gate tests learned behaviour against negative controls, never plumbing. | The exported checkpoint of the suite's PPO has an all-zero policy head, and its own bead says it proves plumbing (FACT, §23.8). |
| ADR-CORTEX-09 | A nondeterministic training profile is allowed only as class D4, and its results are re-evaluated on a stricter profile before they are cited. | Speed in the learner must not weaken the evidence (§35). |
23.3 The environment API and its semantics S
VecEnv exposes reset(mask), step(actions), snapshot(worlds) and restore(snapshot) over device-resident
buffers. Host code receives views; nothing is copied unless asked. snapshot and restore are what make replay
and teleport possible (§29). The semantics below replace the common convention of masked automatic reset with
post-reset observations, which cannot support a correct bootstrap.
/// DESIGN sketch. The vectorized environment. Buffers stay on the device. Explicit reset is the default.
pub trait VecEnv {
fn spec(&self) -> &TaskProgram;
fn reset(&mut self, mask: &WorldMask, seeds: &SeedPlan, cx: &Cx<'_>) -> Result<ResetView<'_>, Refusal>;
fn step(&mut self, actions: ActionView<'_>, cx: &Cx<'_>) -> Result<StepView<'_>, Refusal>;
fn snapshot(&self, worlds: &[WorldId]) -> Result<Snapshot, Refusal>;
fn restore(&mut self, snapshot: &Snapshot) -> Result<(), Refusal>;
}
pub struct StepView<'a> {
pub obs: TensorView<'a>, // after the transition; for an ended world, its final observation
pub reward: TensorView<'a>, // integrated over the control interval
pub dt_sim: TensorView<'a>, // simulated duration of the interval, per world
pub terminated: MaskView<'a>, // success, failure, or an intrinsic mission deadline
pub truncated: MaskView<'a>, // an external collection limit on a continuing task
pub invalid: MaskView<'a>, // numerical failure: a study error, not a transition
pub needs_reset: MaskView<'a>, // terminated | truncated | invalid
pub applied_action: TensorView<'a>, // after the limiter and any shield
pub final_obs: Option<TensorView<'a>>, // masked-autoreset mode only; `obs` then holds post-reset values
pub info: InfoView<'a>, // sample ages, validity masks, control layer, shield events
}
Rules, each with a fixture in GATE-CORTEX-06:
- E-1 Explicit reset. A world with
needs_resetset stays ended until the caller resets it. Stepping it is refused (FROB-REF-521). - E-2 The final observation is preserved. In explicit mode the
obsof an ended world is its final observation. In the labeled modeautoreset: maskedthe world is reset inside the step,final_obscarries the final observation andobsthe post-reset one. The mode is in the run manifest and in every result (INV-CORTEX-01). - E-3 Termination and truncation are different facts. Termination is success, failure or an intrinsic mission deadline. Truncation is an external collection limit on a task that would have continued. If both are set, termination controls the bootstrap.
- E-4 Invalid physics is a study error. A world that is
Faulted, orDegradedbeyond the task's policy (§16), setsinvalid. It has weight zero in policy updates and is kept, with its trace, in benchmark and failure counts. A fall is a valid transition. - E-5 Actions are checked before state changes. A non-finite action is refused before any state is mutated (FROB-REF-522). For bounded actions the out-of-range rule is declared, and proposed and applied values are both logged.
- E-6 Spaces are typed. Each channel has a shape, a dtype, a channel id, a unit, a frame, a representation, a validity mask and a sample age from the clock contract of §20. Normalization is versioned and invertible, and clipping is disclosed.
- E-7 Identity and order. Worlds have logical ids and per-world seeds. A batch split across two devices returns in the caller's order. Recurrent hidden state is reset only for worlds that ended, and it is part of every checkpoint and snapshot.
- E-8 Access classes. Every observation channel is
actor,critic-onlyorevaluation-only. The actor schema is frozen and is run with the other channels absent; a read outside it is refused (FROB-REF-524). - E-9 Causality. Histories with identical delivered observations, past actions and policy random draws up to a decision instant yield the same command. The test perturbs future terrain, hidden friction and measurements not yet delivered, one at a time.
Bootstrap targets. For a transition with reward rₜ, next observation o′ₜ (the final observation if the world ended) and value function V:
δₜ = rₜ + γₜ·(1 − terminatedₜ)·V(o′ₜ) − V(oₜ)
Aₜ = δₜ + γₜ·λₜ·(1 − needs_resetₜ)·Aₜ₊₁
The value bootstrap is cut by termination only. The advantage trace is cut at every reset. Known answer: a true
8 s deadline with terminal reward 0 gives a bootstrap target of 0; an external cut at the same instant on a
continuing task with V = 10 and γ = 0.99 gives 9.9. A single done flag cannot produce both. A fully observed
finite-horizon task gives the actor the time remaining, or declares the clock hidden.
Physical discounting. γ(Δt) = exp(−β·Δt) with β in 1/s declared by the task, and trace decay λ(Δt) = exp(−κ·Δt). The control interval may then differ between designs, tasks or rungs without changing what is optimized. With β = 0.5025 /s, γ is 0.99 at 20 ms and 0.99499 at 10 ms. A constant reward rate r is then worth r·Δt/(1 − e^(−β·Δt)): 2.000·r at 20 ms and 1.995·r at 10 ms, both near r/β = 1.990·r. With one per-step γ = 0.99 at both rates the same behaviour is worth 2.0·r and 1.0·r.
Time-aware rewards. A running term is a rate, integrated over simulated time across the physics substeps of the interval. An event term is counted once per event identity. A terminal term is counted once. Counterexample: a penalty of 1 per control step over a 10 s episode totals 500 at 50 Hz and 1,000 at 100 Hz for the same motion. Evaluation metrics stay in physical units (§24), so a change of reward weights does not change what success means.
Two units of throughput. An environment step is one control interval; a world-step is one physics step. At ten physics steps per interval, 400 k environment steps/s is 4.0 M world-steps/s. Every figure in this chapter and in §36 names its unit.
23.4 Device-resident tensor views S
A TensorView is a typed, strided view into a device buffer that rollout and learning kernels share. It carries
the device, the allocation id, the dtype, the extents, a logical origin and signed strides in elements, an
ownership epoch and the intended access. Rollout storage (observations, sampled and applied actions,
log-probabilities, values, rewards, flags, final observations) stays on the device for the whole collection.
Observations never leave it during a rollout.
- One stride convention, converted at one boundary. FACT (Appendix H; re-run at admission, §33): the pinned
FrankenNumPy uses signed byte strides and the pinned FrankenTorch uses unsigned element strides. A contiguous
2 × 2
f32view has strides (8, 4) in the first and (2, 1) in the second. Copying the numbers verbatim into a view on a 16-element buffer reads element 12 at index [1, 1] where element 3 belongs, and both reads are in bounds. A view with byte origin 4 and strides (8, −4) indexes elements (1, 0, 3, 2) and has no unsigned form. - Unrepresentable views. A view the receiver cannot represent is materialized, with the copy charged to the receipt, or it is refused under a zero-copy contract (FROB-REF-547).
- Leases. A host read of a device buffer is valid only after the command buffer of its epoch has completed. A view carries its epoch, and a read of a stale epoch is refused. Diagnostics read double-buffered ring slots, so they never wait on the step.
- Masks apply everywhere. Padding worlds and padded degrees of freedom (§17) are masked in kernels, rewards and gradient accumulation. A padded element never contributes to a mean (INV-CORTEX-07).
23.5 Declarative task specifications with requirement terms S
A task is data with a TaskId: observation terms (each with unit, noise, delay, and the representation under
which it transforms, C-09), action terms (joint position targets through the driver loop by default; torque or
velocity as alternatives), reward terms with units and explicit normalization, termination conditions,
randomization and curriculum. Declaring the representation of each term is what lets the framework build
equivariant networks, generate mirrored data, and test declared invariances mechanically. The format is .ftask
(§24). What Cortex requires of each kind of term (DESIGN):
| Term | Required beyond the above | Rule |
|---|---|---|
| Observation | Access class; rate as a rational period; sample age exposed | E-6, E-8 |
| Action | Bounds; the registered action law; the entry in the mapping table of §23.7 | P-3 |
| Reward | Kind: rate (per second), event or terminal |
Time-aware integration |
| Termination, truncation | Which conditions belong to the mission and which are collection limits | E-3 |
| Requirement | A predicate ψ(x) ≥ 0 with a unit and a scale s; a temporal operator over a bounded window | Below |
| Randomization | The card field each range comes from | §23.15 |
| Curriculum | A schedule over training distributions only | Evaluation episodes are fixed and never chosen by a curriculum |
| Reset | A settled-pose cache of precomputed quasi-static poses (TARGET: 64 per design, penetration at most 0.5 mm) | No spawn impulse in the first transitions |
Requirement terms. A requirement is the acceptance metric of a mission (§24) and may also shape training.
- Unit-normalized margins. Each predicate declares a scale, and its robustness is ρ = ψ/s, a pure number, before any minimum or maximum combines predicates. Counterexample: the minimum of a clearance of 0.05 m and a thermal margin of 12 K is 0.05 in metres and 12 if the clearance is written as 50 mm. With scales of 0.10 m and 10 K the margins are 0.5 and 1.2, and the clearance binds in every unit system. A temperature margin is a difference, so its scale is written in kelvin.
- Past-time online, future-time offline. "Historically", "once" and "since" over a bounded past window compile to streaming kernels with a monotonic deque, and may be used as a reward or a monitor. "Always", "eventually" and "until" over a future window give a verdict only on a completed trace. A future-time term offered as an online reward is refused (FROB-REF-554), never truncated: a 600 s window at 50 Hz is 30,000 samples.
- Requirements do not replace rewards. Bounded-time requirements are the acceptance metric and a falsification score; training keeps its reward terms (FACT, Appendix H). A policy is accepted on the requirement verdict, whatever its return.
23.6 Algorithms and the PPO rules
- PPO with generalized advantage estimation, value normalization and an adaptive step size S.
- SAC, and teacher-student distillation from privileged to proprioceptive observations F.
- Behaviour cloning from the baseline controller as a warm start F: Reflex supplies demonstrations for free.
- TD3 and evolution strategies F; the latter use FrankenSim's derivative-free optimizers through the airlock for small policies and for the gains of rung R1 (§22.7).
Verbs: train.start (algorithm, network, devices, tying and profile are arguments), train.status,
train.stop, train.export. The environment has no verb of its own; train.*, eval.* and sim.run reach it.
The PPO rules. Each is a rule with a test. Together they are the admission battery of GATE-CORTEX-07 for any PPO implementation.
| Rule | Statement | Test |
|---|---|---|
| P-1 Two masks | The value bootstrap is cut by termination only; the advantage trace is cut at every reset (§23.3). | The 0 and 9.9 fixture; both flags set gives 0 |
| P-2 Frozen behaviour | Policy weights and the observation normalizer do not change during collection. | An update attempted mid-collection is refused; two collections from one snapshot are identical |
| P-3 Same random variable | The likelihood ratio is computed for the random variable that was sampled. Either the sampled latent command is stored with its density and a deterministic limiter follows (latent-then-limit), or the policy is defined on the applied clipped action and has point masses at the bounds (clipped-with-endpoint-mass). |
Unit-variance Gaussians clipped at +1 with means 0 and 1: the endpoint ratio is 0.5/0.158655 = 3.15149; ordinary densities at +1 give e^0.5 = 1.64872. Each law returns its own number; a mixture is refused (FROB-REF-527) |
| P-4 Clipped surrogate, both signs | With L = min(ρ·A, clip(ρ, 1 − ε, 1 + ε)·A), the unclipped branch is active for A ≥ 0 when ρ ≤ 1 + ε, and for A < 0 when ρ ≥ 1 − ε. | Finite differences of the loss at ρ = 0.5, 1 and 2 with ε = 0.2, for both signs of A and at both clip boundaries |
| P-5 Critic gradient from the current output | The value loss differentiates the current forward value, not the value stored at collection. | Critic output 2 with stored value and return 0: the gradient of c_V·(V − R)² into the value head is 4·c_V, not 0 |
| P-6 Episode boundaries in sequences | A sequence model evaluates each transition under its own episode history. Packed episodes do not see across a reset, and a minibatch that starts mid-episode carries the right position and prior history. | Two independently reset one-step episodes packed together: a frozen policy's ratio is 1 on both |
| P-7 Physical discount | γ and λ come from β, κ and the interval's simulated duration. | The 2.000 and 1.995 fixture across control rates |
| P-8 Time in the input | A finite-horizon task gives the actor the time remaining or declares the clock hidden. | With the clock, the value at the deadline equals the terminal reward; without it the result is labeled |
| P-9 Valid samples only | Invalid worlds and padded elements have weight zero in every loss and statistic, and the advantage trace of the valid samples before an invalid one ends there, bootstrapping from the last valid value as at a truncation. | A planted Faulted world changes no gradient and appears in the failure count |
| P-10 Global, versioned, invertible normalization | Advantage and value normalization use statistics over all valid samples of the update (§23.9); the normalizer is stored with the checkpoint. | The normalizer round-trips; the shard fixtures of §23.9 |
| P-11 Entropy follows the action law | Entropy and multi-dimensional saturation use the registered law of P-3. | The entropy of the clipped law at a bound matches a quadrature reference and differs from the Gaussian's |
FACT (read at FrankenSim commit 483d09c682f2, crates/fs-g1-train/src/ppo.rs; the numbers come from
executable fixtures and are re-run at admission, §33). The suite's only PPO violates P-4, P-5 and P-6:
- For a negative advantage its branch test is reversed: it follows the ratio below 0.8 and clips it above 1.2. At ratios 0.5, 1 and 2, finite differences of the fixture loss in log σ give 0, −0.5 and −1; the code gives −0.25, −0.5 and 0.
- Its value gradient uses the value stored at collection. For a critic output of 2 with stored value and return 0, the gradient into the shared critic bias is 0 where the squared-error objective gives 2.
- It evaluates a concatenated trajectory as one sequence. Two independently reset one-step episodes give the frozen policy a second likelihood ratio of 1.582459, where replaying each episode gives 1.
It also drives its advantage recursion with a single done flag, which P-1 forbids, and its action sampler has
correct marginals and a wrong joint law, which rule P-12 forbids (§23.25). These are admission tests and upstream
work items (§33, Appendix F), and the reason for ADR-CORTEX-03.
23.7 Networks, checkpoints and the mapping table
- MLP and GRU S. A small observation MLP with explicit normalization is the first network trained against every baseline.
- Asymmetric actor-critic S: the critic may read
critic-onlychannels and the actor may not (E-8). - A residual head over a Reflex baseline F, with a zero-initialized output layer (§22.7).
- An equivariant MLP under Γ_obs F, with augmentation and a symmetry loss as the Solid alternative (§23.13).
- A morphology transformer with one token per link F (§23.11).
- A geometric-algebra tree policy M (§23.14).
Every checkpoint carries a manifest and a mapping table, schema frob.policy/1, stored in the safetensors
metadata and in the content store. It is not a new file format (DESIGN sketch):
{
"schema": "frob.policy/1",
"network": { "kind": "mlp", "widths": [256, 256, 128], "recurrent": false },
"action_law": "latent-then-limit",
"discount": { "beta_per_s": "0.5025", "kappa_per_s": "2.565" },
"channels": [
{ "id": "fl_hip_target", "role": "leg[0].hip", "unit": "rad", "sign": 1,
"bounds": ["-0.8", "0.8"], "rate_hz": "50/1" }
],
"observation_schema": { "actor": "digest", "critic_only": "digest", "normalizer": "digest" },
"trained_on": { "task": "TaskId", "scenes": "split-digest", "randomization": "card-digests",
"tier": "gpu-cuda", "determinism": "D2", "reset_mode": "explicit", "donor": null },
"group": { "mech": "C2xC2", "obs": "C2", "tying": "augmentation" }
}
A policy reaches a design only through its mapping table. Channel roles, dimensions, signs, units and rates are matched by role, never by position or by name order, and a mismatch is refused (FROB-REF-531, INV-CORTEX-04).
23.8 Where learning runs, and what FrankenTorch is for
FACT about FrankenTorch today (frankentorch/crates/ft-core, ft-nn, ft-optim, ft-serialize, read at
commit 8165d6a18028): tensors execute on the CPU only; Device::Cuda and Device::Metal are metadata;
it has a reverse-mode tape with higher-order gradients, Linear, GRU, LSTM and attention modules, Adam and
gradient clipping, a passing end-to-end MLP-with-Adam training test, safetensors and a limited ONNX export; it
has no PPO and no vmap. FACT about the suite's only PPO: frankensim/crates/fs-g1-train hand-rolls PPO with
manual backpropagation and depends only on libm; its exported checkpoint has an all-zero policy head, and its
bead says it "proves model/training/export plumbing, not learned humanoid locomotion".
DESIGN:
- On device, the policy forward pass, the backward pass and the optimizer are kernels in the shared dialect (matrix products, activations, Adam). Observations never leave the device during a rollout, which is the main source of GPU-simulator throughput.
- FrankenTorch has three jobs: the CPU trainer for small runs and tests; the gradient oracle against which the on-device backward kernels are differentially tested; and policy interchange through safetensors.
- An upstream work item (FD-FT-02) adds real device execution to FrankenTorch. When it lands, the on-device trainer switches to it under the rule in Appendix F.
- The lesson from
fs-g1-trainis a gate rule: a training gate tests learned behaviour against a negative control, never plumbing (GATE-CORTEX-01). - Which machine. The two RTX 4090 train. The M5 Ultra holds the world bank, large scenes, replay, the viewer and large-memory evaluation (§21). Only seeds, weights and receipts cross between them. A policy trained on CUDA is evaluated on Metal inside a declared determinism class, and the report names the class (NS-10).
- Precision. Networks run in single precision. Half-precision and tensor-core paths are rejected for contact (§21) and are not adopted for policies of a few hundred units either: the arithmetic is too small to repay the risk.
- Profiles. Training may run on a labeled D4 profile, with nondeterministic reductions in the learner only and never in physics. A D4 result is re-evaluated on a D2 or D3 profile before it is cited; citing it earlier is refused (FROB-REF-867).
- The learner is a parity item too. FACT (Appendix H): on one incumbent stack the learner is about 13% of training wall-clock. Its share on ours is recorded in the phase ledger of §39.
23.9 Multi-device PPO: the normalization rule S
Each device is one shard that holds complete worlds, replicated immutable packs and frozen behaviour weights.
- Each shard returns sums, not means: the sum of per-sample loss gradients and its count of valid samples.
- Sums are reduced across shards in a fixed order and divided by the global valid count. An empty shard contributes zero and still takes part. If the global count is zero the update is skipped.
- Advantage and observation statistics are merged from per-shard triples (n, mean μ, sum of squared deviations S) by the exact pairwise formula: n = n_a + n_b, d = μ_b − μ_a, μ = μ_a + d·n_b/n, S = S_a + S_b + d²·n_a·n_b/n. No shard normalizes with its own statistics.
- Weights are published only after a complete update. Minibatch order is frozen for equivalence checks. An asynchronous variant needs a stated bound on version lag and is F.
Known answers. One sample with gradient 1 and three with gradient 0 give 0.25 on one device and on two shards of sizes 1 and 3; averaging the shard means gives 0.5. Shards {0, 0} and {4} have global mean 4/3 and variance 32/9; each shard's own variance is 0.
Determinism: with a fixed shard layout and reduction order the update is class D2 per backend. Across layouts it is D3, with the envelope recorded (PO-CORTEX-03). Scaling TARGET: at least 1.25 times (tolerance; the ambition is 1.6, §36.3 row 5) from one RTX 4090 to two at constant global workload and equal task quality, measured also at constant load per device. Below that, further sharding is not funded, and the second card runs independent seeds or a second design.
23.10 Distillation F
One mechanism, four teachers. The student is trained by supervised learning on states the student itself visits, relabeled by the teacher (dataset aggregation).
| Teacher | What it gives | Cost and caveat |
|---|---|---|
| The baseline controller (§22.4) | A warm start for any class that has a baseline | Seconds. The student inherits the baseline's limits. |
| The sampling controller (§22.8) | Demonstrations for a class that has a cost and no shaped reward; pooled seeds for a family prior | About 67 GPU-minutes per 4,000 simulated seconds at the cheapest configuration and the throughput target. The teacher is model-privileged. |
| Specialists | A family policy distilled from many specialists | The specialists must exist; it pays when they were trained anyway for finalists |
| A privileged teacher | A proprioceptive student from a teacher that reads critic-only channels |
The student's schema is run with those channels absent |
Rules: the student is evaluated on its own rollouts with its own observation schema, and teacher states are never used at evaluation. The distillation gap is reported as the student's share of the teacher's return, with the requirement verdicts of both. A student trained on shuffled teacher labels must fail the threshold. A distilled policy's manifest lists its teachers, and it takes the weakest standing among them.
Verb: train.distill. Gate: GATE-CORTEX-14. Kill gate: KILL-DISTILL.
23.11 Family policies per morphology class F
What it is. One set of weights per morphology class (quadruped-like legged, multi-legged, biped and humanoid, wheeled, tracked, wheel-leg hybrid), conditioned on the design. It is not one universal policy.
Architecture (DESIGN). Tokens come from the typed design term, one per joint and one per contact site.
- Static features: the screw basis; the joint placement as log(base_from_joint); scaled subtree inertia; an actuator-card summary; tree depth and parent index; the symmetry-orbit label.
- Dynamic features: the unit-complex joint angle; velocity; the last action; the contact flag.
- A shared per-token encoder, a plain transformer over the tokens with a tree-distance bias, masked pooling and a per-token decoder, with 1 to 5 million parameters. Padding tokens are masked (§17).
- Canonical inputs. Frames are heading-normalized, and the design term is in the canonical form of §14, so two descriptions of one robot give one input. Training also augments over equivalent descriptions (axis conventions, joint order). FACT (Appendix H): a published multi-embodiment policy kept 3.6% of its return on equivalently re-described robots, and 80.6% when trained across conventions.
Pretraining distribution. Per class, designs are drawn from the grammar of §14: the class templates; parameter perturbations of them (limb lengths and masses within ±20 to ±30%, actuator cards within one family, payloads within the task range); and single-limb edits as a separate, labeled stratum. Three held-out sets are fixed before training: parametric designs inside the box, single-limb edits, and out-of-class designs that the policy must decline. Membership is decided by a stated rule on the parameter box and the topology signature. Outside it the policy is refused (FROB-REF-534) and the fallback of §22.8 applies.
The compute envelope question. FACT (Appendix H): results at 16 to 100 training embodiments cost about 36 to 60 GPU-hours; the results that reach 85 to 97% of specialists used far more, about 19,200 GPU-hours in one case. Pretraining at 10⁹ to 10¹⁰ steps costs about 1 to 30 GPU-hours, and at 10¹² steps over a thousand. The machine's standing job limit is one device for one hour (OPEN-SPARK-2). Pretraining therefore does not start without an owner-approved envelope (OPEN-CORTEX-1). The fail-closed default is no family policy: the baseline screens, and finalists get specialists with inheritance.
The numbers the evidence supports (FACT, Appendix H; all are simulation returns):
| Setting | Return with no adaptation (zero-shot), as a share of a specialist's | Fine-tuning cost |
|---|---|---|
| Within one class, in a parametric neighbourhood of the trained templates | 70 to 85% | About 2 to 25% of a from-scratch run stopped at matched return (1 to 10% against the fixed budget one source used; 20% in a second) |
| Across topology-diverse designs at affordable pretraining | 10 to 40% | A gain of 2 to 3 times at best, and none on flat terrain |
| Outside the training distribution | Fails outright | Not applicable |
Every figure in the first row is for flat or mildly varied ground with fixed actuation. No source measures a class prior on stairs, slopes or wet floors, with payload or with actuator-card changes; the held-out evaluation of GATE-CORTEX-11 therefore runs on the task family's interior terrain set.
When it pays (DESIGN arithmetic on those FACTs). With pretraining cost P, specialist cost s per design and fine-tuning fraction f, the family policy pays after N > P/(s·(1 − f)) designs. At P = 60 GPU-hours and f = 0.1 that is about 590 quadruped designs (s = 410 s) and about 70 to 130 humanoid designs (s = 0.5 to 1 GPU-hour). The ceiling on the training phase is 1/f, between 4 and 50 times for f between 0.25 and 0.02, and on the whole cycle it is whatever share training has in the phase ledger of §39.
The rule for using it as an evaluation rung. A family policy's score, with no adaptation or after a brief fine-tune, may rank designs only after its rank agreement with specialists has been measured for that class and task family (INV-CORTEX-06). This is the decisive early experiment of the program:
- A calibration set of 40 designs in one class. Each is scored three ways: by the family policy with no adaptation on the fast tier; after a fine-tune at 5% of specialist steps; and by a specialist on the referee.
- Reported: Kendall τ, top-10 recall and bootstrap intervals, and the per-episode correlation across tiers on 256 paired episodes per design.
- Consequences, fixed in advance. A lower 95% bound on τ of at least 0.7 near the top of the ranking, together with the winner check of §23.28, lets the rung select finalists. Between 0.5 and 0.7 it discards only, and at least the top quarter goes up to the fine-tune rung. Below 0.5 it is not a rung (FROB-REF-537). Allocation between rungs is the ladder rule of §28.
- What is known. FACT (Appendix H): no published rank agreement between a shared policy and specialists was found. The nearest figure is a Spearman correlation of 0.40 overall, and 0.50, 0.60 and 0.83 on three robots, between a critic and rollouts of the same shared policy. Weight sharing in architecture search gave Kendall τ of 0.236, 0.740 and 0.159 on three benchmarks.
- Bias, not only noise. FACT (Appendix H): shared controllers pull a search toward designs that are easy to steer, and early evaluation favours fast learners. Rank agreement is therefore reported per region of design space, and the audit lane of §28 samples what the rung discarded and plants a known winner.
Verbs: train.family (pretrain, after the envelope check), policy.apply (a design, with or without
fine-tuning; refused out of distribution). Gates: GATE-CORTEX-11, -12. Kill gate: KILL-FAMILY.
23.12 Inheritance along design edits F
What it is. A child design's policy starts from its parent's, through the typed edit that joins them (§14).
| Edit | Effect on the policy | Standing |
|---|---|---|
| Parameter edits: set a parameter, regear, reshape, swap an actuator within a family | Token features change for a family policy; a specialist's channels are unchanged | The default once its gate has passed for the edit class. FACT (Appendix H): about 2 to 25% of an early-stopped from-scratch run for transfer from a prior to a similar body; about 50% in the one parent-to-child study; no figure for a small typed edit |
| Single-limb topology edits: add or remove a limb, mirror a pair, swap a wheel for a foot | Tokens are added or dropped; new specialist channels start at zero; the edit's declared correspondence of frames, actuator roles, sensors, contact roles and phase clocks maps the rest; unmatched recurrent state is reset | An experiment behind GATE-CORTEX-13. FACT (Appendix H): 33 to 100% of from-scratch steps, and no gain on flat terrain |
Rules:
- Transport by the mapping table. Twist-like quantities are carried by the adjoint and wrench-like ones by its dual. Limbs related by an element of Γ_obs reuse blocks with the sign and frame changes of §22.13. A mapping that cannot be built is a refusal, not a guess (FROB-REF-531).
- Disclosure and a matched arm. A candidate that inherits weights names its donor in its manifest. A comparison of an inherited child with a fresh one is run both ways of §22.7: with a shared controller, and with equal-resource retuning.
- Scheduled resets against loss of plasticity. FACT (Appendix H): PPO degrades under continual learning, on a changing task and, after long training, on a stationary one. A lineage therefore resets when its audit twin falls more than 5% below the fresh control, and in any case at depth k (DESIGN default k = 5, OPEN-CORTEX-3), when the child is trained from the class prior, or from scratch where there is none. One child in m (default m = 8) is also trained fresh as an audit, and its inherited twin must be within 5% of its return.
- Inheritance biases the search. FACT (Appendix H): an inherited controller favours bodies near the parent, and morphological diversity declines faster under inheritance. A diversity-seeking search (§28) therefore counts inherited and fresh evaluations separately and does not compare them on one scale.
Verb: train.inherit (child, donor, edit path), which refuses without a mapping table and reports the steps to
the parent's requirement verdict. Gate: GATE-CORTEX-13. Kill gate: KILL-INHERIT.
23.13 Equivariance, only where the closed loop has it
Which group. Weights are tied only under Γ_obs of §22.13: the subgroup that preserves sensors, payload, task and terrain symmetry as well as the mechanism. A quadruped with a front-only lidar is tied under C₂, not under its mechanical C₂ × C₂. Tying under Γ_mech constrains the policy to something false.
Two mechanisms, two claims.
- Exact reduction F. An equivariant network whose layers commute with the representations that the task terms declare. Hidden layers carry copies of the regular representation, so pointwise nonlinearities commute with the group. Linear layers are parameterized by isotypic blocks: scalars for real type, and 2 × 2 blocks [[a, −b], [b, a]] for complex type, which the cyclic groups Cₙ with n ≥ 3 have and radial robots without a mirror need. Reynolds averaging, W ↦ |Γ|⁻¹·Σ ρ_out(g)·W·ρ_in(g)⁻¹, gives the same space exactly and serves as the test oracle; it is only more expensive. Quaternionic-type blocks are not built, because no point group or leg-permutation group in scope has them. Wreath products, for limbs that are themselves symmetric, sit behind the same gate.
- Augmentation S. Transformed copies of each transition, with a symmetry loss. It needs only a symmetry of the distribution of worlds, not invariance of each world.
In law, not pathwise. A stochastic policy is equivariant when π(ρ_U(g)·u | ρ_O(g)·o) = π(u | o). At an observation fixed by g, a deterministic equivariant policy must emit an action fixed by g, which the two-latch fixture of §22.13 shows can be wrong. The policy class therefore keeps its randomness at fixed points, and the failure of a tied deterministic policy excludes only that class.
The experiment for BET-07. Three arms on the same seeds: an equivariant policy conditioned on the transformed terrain; an ordinary policy with augmentation; an unaugmented baseline. The arms are ranked separately on sample efficiency, on compute and on performance outside the training distribution. A one-sided slippery patch is in the battery, so a policy that cannot act asymmetrically is caught. FACT (Appendix H): augmentation made a baseline transformer as data-efficient as an equivariant model in one study; in another, hard constraints were more vulnerable to distribution shift and gains were a few per cent. The expected benefit is modest, which is why KILL-EQUIV exists.
23.14 A geometric-algebra tree policy M
What it is. Link tokens that carry multivector features (motors from fs-ga), layers equivariant to rigid
motions and to relabelling, and message passing along the kinematic tree.
What the evidence supports. FACT (Appendix H): invariance to heading and to description conventions is necessary, and canonical inputs with augmentation recover most of it. No published geometric-algebra policy for multi-embodiment locomotion was found. Such layers ran 1.4 to 7.5 times slower per epoch than plain ones; a pure geometric-algebra network failed as a control network through slow convergence; and in one study removing morphological message passing did no harm. Gains near two times are published only for supervised dynamics prediction and for imitation.
The experiment. The same embodiment set and the same wall-clock budget (not equal parameters), three seeds per arm. Arm A, the control, is the plain morphology transformer of §23.11 with heading normalization and convention augmentation. Arm B is A with a geometric-algebra encoder and decoder only. Arm C is full geometric-algebra tree message passing. Measured: time to a return threshold, return with no adaptation on held-out designs, and retention under re-description. The ceiling is 0 to 1.3 times on an amortized pretraining run in reinforcement learning, so it is not a source of the result the ten-fold test measures. It is off the critical path. Gate: GATE-CORTEX-17. Kill gate: KILL-GATREE.
23.15 Randomization from cards S
Every randomized parameter traces to an uncertainty on a card (actuator, interface, material, sensor). There are no free-floating ranges. Objectives may be the mean or a tail measure (conditional value at risk) over the randomization.
DESIGN additions:
- Resolved values are stored. Each episode's drawn values are in its record, so a failure can be replayed and attributed (§27).
- Physical and numerical parameters are separate. A smaller time step must not silently stiffen a fitted foot spring. Randomization touches physical parameters only; numerical ones are varied by the detectors of §23.16.
- Adaptive randomization F may reweight inside a card's support. A range beyond the card is labeled
beyond-card, and its results cannot be cited for the carded hardware. - Held out. Delay, gains, sensor corruption and layout are randomized through their own cards and the scene splits of §19. Complete buildings and material conditions are held out.
23.16 Detecting exploitation (GOAL-16), and rank robustness of finalists S
Before a policy result is reported it is re-evaluated under perturbations that should not matter to a physically meaningful behaviour. A collapse outside the envelope sets a flag on the result.
| Perturbation | Flag |
|---|---|
| Half the time step | dt-sensitive |
| The next tier up | tier-sensitive |
| Friction and stiffness moved within their card uncertainty | parameter-sensitive |
| The alternative contact law, including the hard-contact batch law of §18 where it is built | law-sensitive |
| Solver tolerance divided by 10, and Newton iterations doubled | tolerance-sensitive |
| Contact stiffness halved and doubled | stiffness-sensitive |
| Controller rate changed; rotor inertia restored to its card value where a model omitted it | rate-sensitive |
| The mirrored policy on a symmetric robot, which must score the same within noise | mirror-asymmetric |
| The causal test of E-9 | leak-sensitive |
| A per-step work-energy audit from the ledger of §29: ΔE_mech − W_act ≤ ε | energy-created |
The first four rows and the mirror test speak for themselves. The others have these reasons. FACT (Appendix H): in an executed planar study, success fractions of 0.093 and 0.780 became 1.000 and 1.000 with a 1 kHz loop or with added rotor inertia, so controller rate and rotor inertia are pinned before any benchmark. The energy audit catches penetration recovery that pumps energy into a hopping body. The planted exploit for the law check is a friction cone linearized as a box, which grants √2·μ along its diagonals, 41% more traction.
Rank robustness of finalists S. A flag on one policy is not enough when designs are compared; what matters is whether the ranking survives. In each final round the top k designs and the runners-up are re-evaluated under the model-form variants above (contact law, cone, stiffness, step, tolerance), with the energy audit. The result is typed:
model-robust { variants, top_k_overlap, energy_audit }when the top-k set, and its order within noise, survive every variant; ormodel-conditionalwhen a variant flips the ranking. A model-conditional result cannot be promoted toestablished(§16) and cannot spend build or prototype budget in §31; an attempt is refused (FROB-REF-830).
TARGET cost: 5 to 10% of a campaign. Demotion rule: if over 20 campaigns the top-8 overlap is at least 7 of 8 in 95% of them with no energy flag, the check becomes a periodic audit.
Verbs: eval.detect (the detectors, on one result), fidelity.modelform (rank robustness of a finalist set; defined
in §16.9). Gates: GATE-CORTEX-04, -15.
23.17 Evaluation protocol S
Held-out scenes (§19); at least five seeds; interquartile mean with a stratified bootstrap interval; pairwise comparisons through the e-process of C-15.
DESIGN additions:
- One study type per study. A comparison is declared as an anytime-valid e-process study, or as a fixed-n exact study with its reachable verdicts stated. The two are never mixed in one study (§24).
- Pairing needs determinism. Paired statistics on common random numbers are refused under an uncontrolled determinism class (§35).
- Matched compute. Results report time to a target success rate and performance at matched compute. More rollouts with an easier sensor model do not establish a win.
- Estimated state. Every reported return has its
--state estimatedtwin, and the gap is part of the result. - Fixed evaluation episodes. No curriculum, adaptive randomization or search chooses evaluation episodes.
- Holdout retirement. A reversal on a sealed holdout is reported against the frozen system. If it then informs training or selection, that holdout is retired and a new one is drawn.
- Acceptance is elsewhere. A return is not an acceptance. Acceptance is the requirement verdict in the blinded qualification lane of §24, on worlds that the discovery side never read.
- Method gates use ten paired seeds and a 95% paired lower bound, with competing arms and a potency control.
23.18 The Gymnasium-compatible adapter F
What it is. bindings/frob-gymnasium, an optional annex outside the core workspace. It presents the
five-result step contract (obs, reward, terminated, truncated, info) and a vector interface to Python learning
stacks. Its purpose is that rival stacks can drive this environment in the ten-fold test (§39), so that
comparison does not depend on our learner. The exclusion of Python from the core stands: the adapter contains no
physics, the native worker is the stepping authority, and the core never links an interpreter (rule L-5).
- Autoreset is declared. The adapter states which of the host library's vector autoreset conventions it
presents, maps it onto
explicitorautoreset: masked, and always supplies the final observation ininfo. - Spaces carry units, bounds, channel ids and sample ages as metadata. Normalization is the versioned one of the checkpoint.
- Transport. Tensors cross as device-resident views where the host stack shares the device and the stride conversion of §23.4 applies. Otherwise one copy per step is made and charged to the receipt (OPEN-CORTEX-4).
- Parity. An episode driven through the adapter and the same episode driven natively have equal digests in class D2.
- Provenance. Results obtained through the adapter record the external learner's identity and versions as untrusted provenance. They are admissible to the rival arm of the ten-fold test and not to our method gates.
Gate: GATE-CORTEX-16. Kill gate: KILL-GYMADAPT.
23.19 Refusals
Cortex uses the 5xx family, and two codes in the evidence family.
| Code | Refused | Ranked repairs |
|---|---|---|
| FROB-REF-521 | step on a world that needs reset, in explicit mode |
Reset the world; or run in the labeled autoreset: masked mode |
| FROB-REF-522 | A non-finite action, before any state changes | Fix the policy output; the world and channel are named |
| FROB-REF-524 | The actor schema reads a critic-only or evaluation-only channel |
Move the channel to the critic; or declare a privileged teacher and distill |
| FROB-REF-527 | The action law at learning differs from the law at collection | Recollect; or relabel under the registered law |
| FROB-REF-531 | A policy's mapping table does not match the design, or cannot be built for an edit | Supply the correspondence in the typed edit; train fresh |
| FROB-REF-534 | A family policy applied outside its class or its pretraining distribution | Screen with the baseline or the sampling controller; train a specialist |
| FROB-REF-537 | A rung used to select or discard without a rank-agreement record that meets its threshold | Run the rank-agreement experiment; use the rung as advisory |
| FROB-REF-544 | A training gate record without its negative controls | Add the zero-weight, random and shuffled-label controls |
| FROB-REF-547 | A tensor view the receiver cannot represent, under a zero-copy contract | Allow one charged copy; change the layout |
| FROB-REF-554 | A future-time requirement term offered as an online reward | Use the past-time form online; keep the future-time verdict offline |
| FROB-REF-830 | A model-conditional result offered for promotion, build or prototype budget |
Resolve the variant that flips the ranking; report both rankings |
| FROB-REF-867 | A D4 training result cited without re-evaluation on a stricter profile | Re-evaluate on D2 or D3 |
23.20 Invariants and proof obligations
| ID | Invariant |
|---|---|
| INV-CORTEX-01 | The final observation of every ended world is retrievable in every reset mode. |
| INV-CORTEX-02 | Behaviour weights and the observation normalizer are frozen during collection. |
| INV-CORTEX-03 | A likelihood ratio is computed for the random variable that was sampled. |
| INV-CORTEX-04 | A checkpoint is inseparable from its manifest and mapping table, and a policy reaches a design only through that table. |
| INV-CORTEX-05 | Every training gate record contains its negative controls. |
| INV-CORTEX-06 | No rung selects or discards designs without its rank-agreement record. |
| INV-CORTEX-07 | Invalid and padded samples have weight zero in every loss and statistic, and are counted in every failure report. |
| ID | Obligation | Verification |
|---|---|---|
| PO-CORTEX-01 | The value bootstrap is cut by termination only and the trace by any reset. | All eight flag combinations on a three-step fixture against a hand table |
| PO-CORTEX-02 | A frozen policy's likelihood ratio is exactly 1 on any replay of its own collection, under any packing or minibatch order. | Property test over 10⁴ random packings |
| PO-CORTEX-03 | The update is invariant to the partition into shards inside the D3 envelope, empty shards included. | 10³ random partitions |
| PO-CORTEX-04 | The actor's output does not depend on any channel outside its schema, nor on anything not yet delivered. | The causal test; inference with the other channels absent |
| PO-CORTEX-05 | Conversion between stride conventions round-trips or refuses. | Exhaustive over two- and three-dimensional views with element strides in −2 to 2 |
| PO-CORTEX-06 | On-device backward kernels equal the FrankenTorch oracle. | GATE-CORTEX-02 |
23.21 Known answers and negative controls
| Fixture | Expected answer | Negative control |
|---|---|---|
| Bootstrap | 0 for a true deadline; 9.9 for a cut with V = 10 and γ = 0.99; termination wins when both are set | A single done flag gives 0 for both |
| Final observation | The same values in explicit mode and in final_obs under masked autoreset |
A mode that returns only post-reset observations fails the truncation fixture |
| Reward rate | A rate term totals the same at 50 Hz and 100 Hz | A per-step penalty totals 500 and 1,000 |
| Physical discount | 2.000·r and 1.995·r | A per-step γ gives 2.0·r and 1.0·r |
| Action law | 3.15149 for the clipped law; 1.64872 for the latent law | A mixture is refused |
| Clipped surrogate | Finite differences 0, −0.5, −1 at ratios 0.5, 1, 2 for a negative advantage | The reversed branch gives −0.25, −0.5, 0 |
| Critic gradient | 4·c_V at output 2 and target 0 | The stored-value form gives 0 |
| Packed episodes | Ratios 1 and 1 | Reading across the reset gives 1.582459 |
| Shard gradient | 0.25 | The mean of shard means gives 0.5 |
| Shard statistics | Mean 4/3, variance 32/9 | Per-shard variance 0 |
| Stride conversion | Element 3; the order (1, 0, 3, 2) | A verbatim copy reads element 12 |
| Mixed-unit requirement | The clearance binds, with margins 0.5 and 1.2 | The unscaled minimum changes with the unit |
| Training gate controls | Zero-weight, random and shuffled-label policies fail | An exported all-zero policy head passes a plumbing test and fails this one |
| Box-cone exploit | Flagged law-sensitive; the ranking is typed model-conditional |
An honest policy on the circular cone is not flagged |
| Family break-even | About 590 quadruped designs at P = 60 GPU-hours and f = 0.1 | — |
23.22 Pilots and ceilings for the Frontier and Moonshot items
Each pilot is paper-scale, runs with existing tools outside the workspace as a development-time experiment (§38), and must run before its work package is funded.
| Item | Tier | Pilot | Falsifier | Ceiling of the benefit | Cost (DESIGN estimate) |
|---|---|---|---|---|---|
| Family policy | F | A prior trained on 200 generated designs of one class; 20 parametric and 20 single-limb-edit designs held out, a specialist for each; the nearest parent's policy with no adaptation as the rival | Median return with no adaptation below 40% of specialists on parametric held-outs, or fine-tuning above 50% of steps | 1/f on the training phase (5 to 100 times), after about 590 quadruped or 70 to 130 humanoid designs | About 60 GPU-hours of pretraining and 40 for specialists |
| Rank agreement between rungs | F | The 40-design calibration of §23.11, reusing the pilot's specialists | Top-of-ranking τ below 0.5, or per-episode correlation across tiers below 0.7 | A cheap rung can discard; a control variate saves about 2 times unless the correlation exceeds 0.9 | About 5 GPU-hours with reuse, 45 without |
| Inheritance | F | Three base robots; 10 parameter edits of 10 to 30% and 5 single-limb edits; three arms (scratch, from parent, from prior), three seeds; one chain of 10 edits | Median parameter-edit cost above 60% of steps; or the inherited arm more than 5% worse in over 20% of edits | 5 to 100 times fewer steps on parameter edits only | About 70 GPU-hours |
| Distillation and behaviour cloning | F | Inside the sampling-controller pilot of §22.17: students for 10 designs from that teacher and from a hand-written baseline | Students below 70% of the teacher's requirement pass rate | A warm start from the baseline is nearly free; the sampling teacher pays only when pooled over a family or where no shaped reward exists (§22.8) | About 10 GPU-hours |
| Other algorithms (SAC, TD3, evolution strategies, privileged teacher and student, adaptive randomization, asynchronous sharding) | F | Each against PPO on the reference quadruped task at equal wall-clock on an incumbent stack, three seeds | The arm beats PPO neither in time to the threshold nor in final requirement pass rate | Bounded by training's share of the phase ledger of §39; none is on the critical path | About 10 GPU-hours each |
| Equivariant networks | F | The three-arm experiment of §23.13 on one symmetric quadruped on an incumbent stack, five seeds | The equivariant arm needs more than 0.7 times the steps of the unaugmented arm, or loses to augmentation at equal compute | At most the order of Γ_obs on data (2 to 4 for the robots in scope); published gains are smaller | About 10 GPU-hours |
| Gymnasium adapter | F | The host library's environment checker against a stub that implements E-1 to E-5 | Any semantic mismatch that cannot be declared as a mode | None on speed; it removes one confound from the ten-fold test | 2 engineer-days |
| Geometric-algebra tree policy | M | The three arms of §23.14 at about 10 GPU-hours each | Neither B nor C is 1.3 times faster than A, nor better with no adaptation beyond seed spread | 0 to 1.3 times on an amortized pretraining run | About 90 GPU-hours |
23.23 Gates and kill gates
The behaviour-not-plumbing rule. A training gate is a claim about learned behaviour. Its record must hold a zero-weight policy and a random policy that both fail the same threshold, a shuffled-label student where distillation is involved, and a replay of the exported checkpoint that reproduces the behaviour. A record without them is refused (FROB-REF-544, INV-CORTEX-05). A test that a training loop runs, exports and reloads is a plumbing test and satisfies no gate below.
| Gate | Claim | Threshold (TARGET) | At |
|---|---|---|---|
| GATE-CORTEX-01 | Known answer | PPO on the reference quadruped's flat velocity task reaches the return threshold on batch-cpu and on each GPU backend; a zero-weight policy and a random policy both fail the same threshold. The exported checkpoint, reloaded on batch-cpu, reproduces the return inside the D3 envelope, and its policy head is not zero. |
M2 |
| GATE-CORTEX-02 | Parity | On-device backward kernels match FrankenTorch autograd within 10⁻⁵ relative on 10³ random batches. | M2 |
| GATE-CORTEX-03 | Design comparison | BET-07 as the three-arm experiment of §23.13, ten paired seeds with intervals: the equivariant arm reaches the return in at most 0.7 times the steps of the unaugmented arm with no loss of final return; the one-sided patch is passed; the comparison with the augmented arm is reported, whichever wins. | M4 |
| GATE-CORTEX-04 | Known answer | Each detector of §23.16 flags its planted exploit and passes an honest policy. | M3 |
| GATE-CORTEX-05 | Performance | Wall-clock to the return threshold of GATE-CORTEX-01 on one RTX 4090 is at most 1.25 times an incumbent's wall-clock for the comparable task, re-measured on the same card under the disclosure block of §36. Published context: about 410 s. This is parity, not advantage. | M2 |
| GATE-CORTEX-06 | Known answer | Rules E-1 to E-9 each pass their fixture; the final observation is retrievable in both reset modes; the 0 and 9.9 bootstrap fixture; the 500 against 1,000 reward fixture is invariant under rate terms. | M2 |
| GATE-CORTEX-07 | Known answer | The PPO battery P-1 to P-11 passes for our implementation, and three planted defects that reproduce the sibling's errors each fail it. | M2 |
| GATE-CORTEX-08 | Parity and Performance | Two-shard and one-device updates agree inside the D3 envelope on 10³ random batches, an empty shard included; the 0.25 and 32/9 fixtures; strong scaling of at least 1.25 times at constant global workload and equal task quality (§36.3 row 5; 1.6 is the ambition and the funding threshold of KILL-SHARD). | M2 |
| GATE-CORTEX-09 | Known answer | The stride fixture reads element 3; the negative-stride view is materialized or refused; a stale-epoch read is refused. | M2 |
| GATE-CORTEX-10 | Known answer | Streaming past-time robustness equals offline evaluation on 10⁴ random traces; the mixed-unit fixture binds on the clearance in both unit systems; a future-time online reward is refused. | M3 |
| GATE-CORTEX-11 | Design comparison | Family policy in one class, on held-out parametric designs: median return with no adaptation at least 70% of a specialist's; fine-tuning reaches specialist return in at most 20% of the steps of a from-scratch run stopped at 95% of its final return, on the task family's terrain set including stairs, slopes and wet patches; out-of-class designs are declined; re-described robots keep at least 80% of return. | M4 |
| GATE-CORTEX-12 | Design comparison | The rank-agreement experiment of §23.11 is run and its consequence applied: τ, top-10 recall and intervals are recorded per class and task family, and a rung without a record is refused. The gate passes when the record exists and the rule is enforced, whatever τ is. | M3 as a pilot on incumbent tools; M4 on ours |
| GATE-CORTEX-13 | Design comparison | Inheritance on 12 parent-child edit pairs, 4 of them blind: median adaptation samples for parameter edits at most 20% of a from-scratch run stopped at 95% of its final return, and fewer than a raw-checkpoint warm start needs; the inherited child more than 5% worse in at most 20% of edits; a chain of 10 edits shows no decline under scheduled resets, with the audit twin within 5%. Topology edits are reported, not required. | M4 |
| GATE-CORTEX-14 | Known answer and Design comparison | The shuffled-label control fails; a student of a baseline or specialist teacher reaches at least 90% of the teacher's requirement pass rate; the figure for a sampling-controller teacher is recorded with its cost. | M4 |
| GATE-CORTEX-15 | Known answer | Planted box-cone and penetration-energy exploits flip a planted ranking and are typed model-conditional; a genuine winner stays model-robust; the cost is at most 10% of the campaign; FROB-REF-830 fires. |
M3 |
| GATE-CORTEX-16 | Parity | 10³ episodes driven through the adapter and natively have equal digests (D2); the final observation is in info in every autoreset mode; an external PPO reaches the threshold of GATE-CORTEX-01 through it. |
M3 |
| GATE-CORTEX-17 | Design comparison | Geometric-algebra tree policy M: arm B or C is at least 1.3 times faster than arm A to the return threshold at equal wall-clock, or better with no adaptation beyond seed spread. | M8 |
| ✖ Kill gate | Trigger | Fallback, named in advance |
|---|---|---|
| ✖ KILL-EQUIV | GATE-CORTEX-03 fails (BET-07) | Symmetry is used for augmentation and metamorphic testing only |
| ✖ KILL-LEARNKERN | GATE-CORTEX-02 fails after one repair cycle, or GATE-CORTEX-05 is missed by more than two times after one optimization pass | The forward pass stays on the device; rollout storage is read back once per rollout and the learner runs on FrankenTorch on the CPU, with the lost throughput disclosed; the trainer moves to FrankenTorch device execution if FD-FT-02 lands |
| ✖ KILL-FAMILY | At GATE-CORTEX-11, return with no adaptation below 70% of specialists on more than half of the held-out set and fine-tuning above 20% of specialist steps; or the pilot's falsifier holds | No family policy: specialists with inheritance from the nearest parent, and the baseline screens |
| ✖ KILL-INHERIT | Median parameter-edit cost above 60% of from-scratch steps, or the inherited arm more than 5% worse in over 20% of edits | Every child trains from the class prior or from scratch; the mapping table is kept for loading policies |
| ✖ KILL-DISTILL | Students stay below 70% of their teachers' requirement pass rate after one repair cycle | Distillation is limited to the privileged-teacher case; warm starts come from behaviour cloning only |
| ✖ KILL-ALTALGO | For any one of the other algorithms of §23.22, its pilot's falsifier holds | That algorithm is not built; PPO under the rules of §23.6 remains the trainer |
| ✖ KILL-GATREE | GATE-CORTEX-17 fails | The plain morphology transformer remains; fs-ga features are not used in policies |
| ✖ KILL-GYMADAPT | GATE-CORTEX-16 fails | Rival stacks drive the environment through the job interface of §32 in whole rollouts, and the ten-fold test records the added latency in its phase ledger |
23.24 Open questions
- OPEN-CORTEX-1. The compute envelope for family-policy pretraining. Owner: the project owner. Blocks
GATE-CORTEX-11 and every use of a family policy. Default: no pretraining. The proposal awaiting approval is one
class (quadruped), 200 designs, at most 60 GPU-hours of pretraining and 40 for specialists, on
trj. - OPEN-CORTEX-2. The default action law. Owner: Cortex lead. Blocks GATE-CORTEX-07. Default:
latent-then-limit. - OPEN-CORTEX-3. The reset schedule for lineages (depth k and audit rate m). Owner: Cortex lead. Blocks GATE-CORTEX-13. Default: k = 5 and m = 8.
- OPEN-CORTEX-4. The adapter's transport: shared device views, or one copy per step across a process boundary. Owner: Tether lead. Blocks GATE-CORTEX-16. Default: one charged copy per step.
- OPEN-CORTEX-5. Default β and κ per task family. Owner: Arena lead. Blocks nothing. Default: β = 0.5025 /s and κ = 2.565 /s, which give γ = 0.99 and λ = 0.95 at 20 ms.
- OPEN-CORTEX-6. Whether the incumbent wall-clock of GATE-CORTEX-05 is re-measured on our card before M2.
Owner: Spark lead. Blocks citing GATE-CORTEX-05 as passed. Default: the gate is recorded against the published
figure with the label
published-contextuntil the re-measurement exists.
23.25 The joint law of sampled actions
What it is. S A rule for the action sampler, added to the PPO battery of §23.6 as P-12: the joint law of the sampled action equals the law that the likelihood computation assumes, across the channels of one decision, across consecutive decisions of one world, and across neighbouring worlds. Marginal checks do not establish it.
Why it is a rule. FACT (frankensim/crates/fs-g1-train/src/ppo.rs:172–189 at 483d09c682f2; to be re-run at
admission, §33.11): gaussian_action draws each channel from gaussian_sample, a Box–Muller cosine over a counter
advanced by the constant 0x9E3779B97F4A7C15, with the phase uniform taken from the top 53 bits of the counter times
0x94D049BB133111EB. The phase is therefore a linear function of the counter modulo 1, and consecutive draws differ in
phase by the fixed fraction 0.167245 of a turn, an angle of 0.334490·π. For a uniform angle θ, cos θ and cos(θ + φ)
have the same sign with probability 1 − φ/π, so two adjacent channels agree in sign with probability 0.665510 where
independence gives 0.5. Reported counts in 100,000 two-channel calls per seed are 66,552, 66,552 and 66,551, while
each channel alone is positive half the time. The trainer scores these samples with a product of marginal densities,
so its likelihood ratio is computed for a random variable that was not the one sampled, which is the failure rule
P-3 exists to prevent.
Mechanism. DESIGN:
- Every noise draw is a Philox output keyed by
(WorldAddr, decision index, stream id, channel)(§35.9). A Box–Muller pair consumes two uniforms from distinct counters, and its second output is used or discarded by a declared rule. No draw is a function of a neighbouring draw's counter. - The sampler has a manifest: the declared joint law (a product Gaussian, or a full covariance when the policy has one), the transform and bounds of P-3, and the stream layout.
frob-conformancequalifies a sampler against its manifest with four statistics over N decisions: the sign-agreement frequency of every pair of channels; the same for one channel at consecutive decisions; the same for one channel in adjacent worlds; and the empirical covariance of the standardized noise. Under a product law every agreement frequency has mean ½ and standard deviation ½/√N.train.startrefuses a sampler that has no qualification receipt for the pinned kernel (refusal: action sampler not qualified against its likelihood).
Owner: frob-learn; kernels in frob-kernels; fixtures in frob-conformance.
Known answer and negative control. The recurrence above, re-implemented as a planted sampler, must be rejected: its adjacent-channel agreement is 0.6655, more than 100 standard deviations from ½ at N = 10⁵. The qualified sampler must show every agreement inside ½ ± 0.005 at N = 10⁵ (TARGET: 3.2 standard deviations; the number of pairs tested is recorded, so the family-wise rate is stated).
| GATE-CORTEX-18 | Known answer. For action dimensions 2, 12 and 29, N = 10⁵ decisions and 64 worlds: every channel-pair, consecutive-decision and adjacent-world sign agreement lies in ½ ± 0.005; every off-diagonal entry of the standardized covariance is below 4/√N in magnitude; the planted linear-phase sampler is rejected with an agreement of 0.6655 ± 0.005. M2. |
23.26 The actor's information contract covers every field it can read
What it is. S Rule E-10, extending E-8 and E-9 of §23.3. The actor's information contract lists every value an
online actor can read at a decision instant, with its delivery instant under the clock contract of §20: observation
channels, earlier rewards, termination and truncation flags, action masks, and each field of info. Outputs of the
trainer and of the scorer are outside the actor's view unless the mission declares them delivered.
Why. A reward is a function of state, and delivered to the actor it is a sensor. Smallest case, on the docking model of §20.16 (three one-second ticks, hidden gain g ∈ {1, 2}, acceleration g·u): with first command u₀ = ¼ the position after one tick is x₁ = g/8, so a shaped reward −|1 − x₁| is −7/8 under g = 1 and −3/4 under g = 2. When the velocity packet arrives late, no causal policy docks in both worlds (§25.15). If the wrapper hands that reward to the actor before the second command, the two worlds are distinguished and the commands (¼, ½, −¾) and (¼, 0, −¼) dock both. An exclusion computed without reward visibility in its contract is then false for the environment that was run.
Mechanism. DESIGN:
ActorViewis a typed projection ofStepViewthat contains only fields of access classactor.reward,terminated,truncated, masks and everyinfofield carry an access class and a delivery instant. The default class ofrewardand of everyinfofield istrainer-only. A recurrent policy that takes the previous reward as an input requires the task to declare reward as anactorchannel.- The digest of the contract (channels, classes, instants) is part of the
TaskId, of every memo key and of every certificate's assumption set (C-18), so a cached answer is not reused under a different contract. sense.causal-test(§20.15) gains four planted leaks that must each fail: the previous reward wired into the actor; aninfofield that carries a hidden parameter; an action mask computed from hidden state; a termination flag delivered before its instant.- The Gymnasium-compatible adapter (§23.18) returns rewards to the learner, as that interface requires, and records whether the driven policy consumed them. A run whose policy read an undeclared field is labeled and cannot be cited for a mission answer (refusal: actor read a field outside its information contract).
Owner: frob-learn (types), frob-sense (test), frob-spec (declaration).
Known answer. The late-packet docking exclusion, run through VecEnv and through the adapter. With the reward
hidden the verdict is excluded. With the reward declared and delivered before the second command the verdict
changes to feasible, with the witness above. Both runs are required: a harness that reports the same verdict in
both is defective.
| GATE-CORTEX-19 | Known answer. Each of the four planted leaks fails sense.causal-test in 100 of 100 seeds; the docking exclusion holds with reward hidden and flips with reward delivered, on VecEnv and on the adapter; a contract change alters the TaskId and invalidates the cached exclusion. M2; adapter rows at its release. |
23.27 A threshold crossing is not attainment
What it is. S A rule for every figure of the form "time to a target", which includes the training-time parity rows of §36.3 and GATE-CORTEX-05. A recorded score crossing is a descriptive event. Attainment is a claim about the policy's population performance, and it needs error control over every evaluation look that could have produced the crossing.
Counterexample. A policy with true success rate 0.89 is evaluated against a target of 0.90 with 30 episodes per look. A look with 30 successes of 30 has an exact one-sided 95% lower bound of 0.05^(1/30) = 0.90497, above the target. One look does this with probability 0.89³⁰ = 0.0303. Across 50 independent looks at the unchanged policy the probability of at least one such crossing is 1 − (1 − 0.0303)⁵⁰ = 0.78548. A rule that stops the clock at the first crossing reports attainment, about four times in five, for a policy that never attains.
Mechanism. DESIGN:
- Evaluation during training runs on a fixed development stream, disjoint from training worlds, at a cadence registered before the run. The run record holds every look, not only the best.
train.statusreports two fields.crossingis the first look whose point score met the target, and is descriptive.attainedis the first look at which a procedure valid for the registered cadence rejects "at or below target". The default procedure allocates error across looks as αₖ = α/[k·(k + 1)], the series of §24.13, and applies an exact one-sided binomial bound at level αₖ for success targets, or a confidence sequence on bounded returns (C-15) for return targets. With all-success looks against a 0.90 target at α = 0.05, look k needs at least ln(1/αₖ)/ln(1/0.9) episodes: 36 at k = 1, 46 at k = 2, 74 at k = 10, 103 at k = 50.- Alternatively the checkpoint chosen on the development stream is evaluated once on a final holdout. That qualifies the chosen checkpoint and leaves its first attainment time unknown; the reported time is then an upper bound and is labeled as one.
- Non-attainment within the training budget is right-censored at the budget and summarized by a restricted mean.
- In a parity comparison both arms are scored by the same rule on the same stream sizes. An incumbent's published time to threshold is context; the figure re-measured under this rule is the comparator.
eval.preanalyze prints, for a registered cadence, episode count and target, the probability that an unchanged
policy at stated true rates produces a crossing. Owner: frob-learn, with the study machinery of frob-arena.
| GATE-CORTEX-20 | Known answer. In 2,000 simulated trainings of an unchanged policy at true success 0.89, with 50 looks of 30 episodes, the first-crossing rule reports attainment in 78.5% ± 3 points and the spending rule in at most 5% (exact one-sided test at the 1% level); the four episode counts above are reproduced exactly. M2. |
23.28 The winner check for a selecting rung
A selecting rung must pass a winner check S. This extends the consequences fixed in §23.11 and the rank-agreement row of §28.10. A lower bound of 0.7 on the top-band Kendall τ admits a rung to selection. But τ counts inverted pairs and weighs every pair equally; it does not say that the best design survives a cut.
Counterexample. Forty designs. The cheap rung reproduces the true order exactly, except that the true best is ranked third. Among the true top ten, 2 of 45 pairs are inverted, so τ = (43 − 2)/45 = 41/45 ≈ 0.911, far above 0.7. A cut that keeps the top 5%, two designs, discards the winner.
Rule. For a rung admitted to selection with m finalists wanted, the number of designs advanced is the largest of
three numbers: m; the survivor count of the conformal margin rule (§28.9, layer 2); and the containment rank
k_c(m), the smallest k such that, in the calibration experiment and in every audit-lane draw to date for the class
and task family, the higher rung's top m all lie inside the lower rung's top k. The ledger row of §28.10 gains two
fields: containment_rank(m), and the regret of a cut, which is the higher-rung score of the best design the
cut discards minus that of the best design it keeps. Positive regret means the winner was lost. A selection whose
retention is below k_c(m) is refused, with k_c(m) as the repair. Where a class has no calibration, the rung
discards by the fixed table and does not select.
Owner and verbs. frob-opt. eval.rank-agreement reports τ, top-k recall, the containment rank and the regret
of the requested cut; opt.cascade reads k_c(m) when it sizes a rung transition.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-CORTEX-21 | Known answer | On the forty-design fixture eval.rank-agreement returns τ = 41/45 on the true top ten, top-1 recall 0 at k = 2, containment rank 3 for m = 1 and positive regret at k = 2. A request to select with k = 2 is refused and the same request with k = 3 is accepted |
M3 |
23.29 Short-horizon gradient training with design parameters
Short-horizon gradient training with design parameters M. This is the learning arm that would consume the step adjoints of §16; the gradient mechanism, its step labels and its kill gate are specified there. It is off the critical path and is not funded before the step-adjoint gate has passed.
What it is. Policy parameters θ and a continuous design vector φ (up to about 20 entries of the parameter row of §16.6) are updated together from analytic gradients of a short rollout. For a window of K control steps, K ∈ {16, 32}, the loss is the negated sum of rewards in the window plus a learned value V_ψ at its end. Its gradient in θ and φ is one reverse sweep through the window: policy network, step adjoints, reward terms. The critic V_ψ is fitted to temporal-difference targets as in on-policy actor-critic training. Three guards make it usable where contact changes.
- A cosine monitor. On a fixed share of windows (DESIGN default 1 in 16) the analytic design gradient is
compared with the paired central difference of §17.13 on the same world addresses. The running median cosine is
logged. Below 0.5 the window halves; at K = 8 and still below 0.5 the design update switches to the paired
stencil and the run is labelled
gradient: stencil. - Step labels. A window that contains a step the adjoint labels anything other than
smoothcontributes to the policy update and not to the design update, because its design gradient is one-sided or fixed-mode. - Design updates are proposals. φ moves inside the family's box. Every design the run reports is a typed edit sequence, recompiled and scored by the ordinary ladder (§28.9). No verdict cites a gradient.
Training with analytic gradients is a labelled D4 training profile (§35); every score that enters the ladder is an evaluation on a D2 profile.
Why the window is short. At the closed-loop exponent quoted in §27.9, 0.16 per control step, a perturbation grows by e^(0.16·K): 13 at K = 16, 167 at K = 32 and 7.8 × 10⁸ at K = 128. A gradient through 128 steps is dominated by its variance, and its mean tells the optimizer nothing. The exponent is a published figure for other systems; the monitor measures ours.
Rival check. FACT (Appendix H): several incumbents differentiate their step, and short-horizon actor-critic training on a differentiable simulator is published. This is not a differentiator, and a composed incumbent can host it. It is here because joint refinement of a selected design and its controller is the one stage where population methods are known to be slow.
Ceiling, before any speed test. The finalist refinement stage only. Its share of the phase ledger (§39) bounds the end-to-end gain, and nothing here helps screening.
Pilot before funding. One quadruped on rough ground and stairs with 20 design parameters, three seeds. Arms at equal GPU-hours: this method; PPO with a derivative-free outer loop on φ; PPO with the paired stencil. Falsifier: the method does not reach the matched, referee-promoted return in at most 0.35 of the derivative-free arm's GPU-hours, or the median cosine is below 0.5 on more than 40% of monitored windows.
Owner and verb. frob-learn; train.start --algo short-horizon-ac --design-params <list>.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-CORTEX-22 | Design comparison | The pilot's comparison on both tasks: matched referee-promoted return in at most 0.35 of the derivative-free arm's GPU-hours; cosine at least 0.5 on at least 60% of monitored windows; every reported design re-scored by the ladder, none promoted on a gradient | M6 |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-SHORTGRAD | The pilot's falsifier fires, or the gate fails after one repair cycle | PPO with the paired stencil of §17.13 where that passes its own gate; otherwise derivative-free search over φ |