FrankenRobotics

Phylum: families of bodies in one batch

9,052 words21 subsections14 tables2 listingsPlan version 1.1

On this page21
17.1 What Phylum owns17.2 Decisions17.3 Three levels of variation17.4 Shape classes and model tables17.5 Size buckets, and when bucketing recovers the padding overhead17.6 Design-family lanes and the speedup bound17.7 Memory: which population shapes fit which machine17.8 Admission, and edit-to-first-step latency as a target17.9 Specialization of survivors, a gated optimization17.10 The morphology generator17.11 Campaign quotienting17.12 Quality-diversity maps per dispatch17.13 Paired finite-difference design gradients17.14 The head-to-head experiment17.15 Verbs17.16 Refusals17.17 Invariants and proof obligations17.18 Known answers17.19 Gates and kill gates17.20 Open questions17.21 Limb-archetype kernels

Crate: frob-phylum (layer 3). Phylum calls frob-model for lowering (§14), frob-algebra for groups (§13), frob-dyn and frob-kernels for the step (§16) and, through the airlock, FrankenSim's fs-archive, fs-race and fs-recompute.

17.1 What Phylum owns

A design studio's workload is many different robots with few worlds each. Phylum owns what makes a population, and not one model, the unit of work:

  • cohorts: shape classes, model tables and slots, size buckets and the bucket planner (§17.3 to §17.5);
  • design-family lanes that share structure and split at ambiguous guards (§17.6);
  • memory admission for population shapes on each machine (§17.7);
  • admission of a design into a running cohort, with edit-to-first-step latency as a target (§17.8);
  • specialization of survivors as a gated optimization (§17.9);
  • the morphology generator, which defines the distribution over designs (§17.10);
  • campaign quotienting: orbit representatives, the mission-observable digest and the invalidation contract (§17.11);
  • quality-diversity maps per dispatch and paired finite-difference design gradients (§17.12, §17.13);
  • the head-to-head experiment that decides whether any of this stays (§17.14).

Phylum does not own the step or its kernels (§16, §21), the design term or the model compiler (§14), the computation of symmetry groups (§13), family policies and inheritance (§23), racing, the archive and the memo store (§28), or the scheduling policy of frobd (§32).

The claim, stated at the size the evidence supports F. FACT (Appendix H): batching parameter values per world is incumbent practice. Batching different trees is not available in the leading reduced-coordinate incumbent, whose structural arrays carry no world dimension and whose documentation says heterogeneous degrees of freedom are not available; a second incumbent requires all variants to share links, joints and names. Mixed topologies in one batch do exist in one manipulation simulator and in two solvers of another engine, and one incumbent solver publishes a mixed batch of three body plans at 86 k samples/s, 0.86 of the harmonic mean of its homogeneous rates; that solver is maximal-coordinate, is documented as not deterministic, and is reported as slower than the reduced-coordinate solvers of its framework. What is left to claim is therefore narrow: mixed-topology populations with few worlds per design, in size buckets, in a reduced-coordinate solver, with any one world replayable outside its cohort. The differentiators that remain are two measured numbers, throughput and the time from an edit to the first GPU step, and one property, run-to-run replay and forks inside the mixed batch. All three can be lost (§17.14).

17.2 Decisions

ADR Decision Reason
ADR-PHYLUM-01 The unit of batching is a cohort of distinct designs that share one step plan, not one model. A population of 512 designs with 16 worlds each fills 8,192 slots; run design by design it uses 0.2% of a device per launch and pays launch overhead 512 times (§17.5).
ADR-PHYLUM-02 Morphology is table data inside padded capacities, in size buckets. No interpreter, no compile per design. Kernels then compile per shape class and kernel version. The step is memory-bandwidth-bound at these state sizes (Appendix H), so a table read costs bytes, not branches, and buckets bound the bytes.
ADR-PHYLUM-03 Results are keyed by world address and ModelId, never by slot, cohort or arrival order. INV-KIN-04: a world is replayed on batch-cpu with its own model alone, and the CPU never needs the cohort.
ADR-PHYLUM-04 Deduplication and weight sharing use only the observation-preserving symmetry subgroup (§13). The mechanical group ignores sensors, payload and task. A quadruped with a front-only lidar has a four-element mechanical group and a two-element closed-loop group; merging under the larger one merges designs the mission can tell apart.
ADR-PHYLUM-05 A per-family-template arm competes in the head-to-head, and the plan commits in advance to adopt it if it wins. Templates are the incumbent regime plus per-design rows. If they cover the mutations agents make and run faster, the table-driven cohort is the wrong engine and should become a preview path.
ADR-PHYLUM-06 Exact sharing only. A quantity is shared between designs when all of its inputs and rounding rules agree; a nearby trajectory is never reused. A smooth distance in design space gives no right to reuse contact history. Reuse with an error certificate over contact is a research problem (§25), not a batching feature.

17.3 Three levels of variation

Level What differs between worlds of one batch Mechanism Tier
P-A parameter variants Values on one tree: lengths, inertias, gear ratios, card parameters, radii, friction One structure table; one parameter table per model slot; no masks, no padding S (§16.6)
P-B model variants Trees, joint classes, sample lists, actuator counts One model table per slot inside the padded capacities of a shape class; masks F
P-C design families As P-B, plus grouping by shared structure and current contact mode Family lanes with shared constants and static row layouts (§17.6) F

17.4 Shape classes and model tables

A shape class is the set of capacities one static step plan is built for. Its hash, the ShapeClassId, is part of the StepPlanId (§16.6). A cohort is a set of model slots and worlds running under one step plan.

// DESIGN: fixes intent and layout, not signatures.
pub struct ShapeClass {
    pub n_v: u16,                 // degree-of-freedom capacity: the bucket size
    pub links: u16,               // link capacity
    pub depth: u8,                // maximum tree depth
    pub samples: u16,             // collision samples tested per world
    pub contacts: u16,            // C of §16.6
    pub rows: u16,                // R of §16.6
    pub actuators: u16,
    pub sensor_layout: SensorLayoutId,
    pub single_structure: bool,   // P-A: one tree, unmasked path
}

pub struct ModelTable {           // one per model slot; read-only in a rollout; about 192 bytes per link (§21.11)
    pub model: ModelId,
    pub level_start: Vec<u16>,    // links sorted by depth; ranges per level
    pub parent: Vec<u16>,         // parent index, less than the link's own; padding links are roots of nothing
    pub placement: Vec<Se3Packed>,            // parent_from_joint
    pub screw: Vec<[f32; 18]>,    // screw basis S in R^(6x3)
    pub screw_cols: Vec<u8>,      // k = 0 (padding), 1, 2 or 3 active columns
    pub inertia: Vec<[f32; 10]>,  // zero for padding links
    pub limits: Vec<JointLimits>,
    pub actuator_slot: Vec<u16>,
    pub sample_range: Vec<(u16, u16)>,
}

pub struct WorldSlots { pub model: u32, pub scene: u32, pub task: u32, pub job: u32, pub addr: WorldAddr }
  • Joint classes are data. A joint is its screw basis with a column count: k = 1 for revolute, prismatic and helical joints, 2 for cylindrical and universal, 3 for spherical and planar (C-01). The joint map τ(θ₁S₁)·τ(θ₂S₂)·τ(θ₃S₃), with masked columns as the identity, is one derived kernel with no branch on joint kind. This is where the Lie-typed core pays in this chapter: the joint is its screw basis.
  • Padding is exact. Padding links are identity transforms with zero inertia, appended after the real links and masked out of every sum; padded degrees of freedom get identity rows. By PO-KIN-02 and PO-KIN-03 the real part of every result is bit-identical to the unpadded computation.
  • Runs. Worlds of one model are contiguous in runs of 16 or 32, so one warp or SIMD group holds one model and the level loop does not diverge inside it. Contact counts still differ inside a run, which is why stages 5 to 8 of §16.6 are compacted and masked per world: homogeneity by morphology is not homogeneity by work.
  • Capacities are read, not compiled. Kernels read capacities from the step-plan header. A new bucket size needs new buffers and a new static launch graph, and no kernel compile.
  • Loops and rolling constraints are rows of the coupled solve (§16.6 stage 7), not table entries.

17.5 Size buckets, and when bucketing recovers the padding overhead

The bound. Let work per world in a stage scale as Lᵖ in the size L of the robot (p = 1 for kinematics, about 2 for Jacobian and Hessian assembly, 3 for a dense factor). A cohort padded to L_max spends L_maxᵖ per world, so its overhead against unpadded work is Ω_p = L_maxᵖ / E[Lᵖ]. With r = E[L]/L_max, the mean-to-maximum ratio,

1/r ≤ Ω_p ≤ r⁻ᵖ ≤ r⁻³ for 1 ≤ p ≤ 3.

The lower bound holds because Lᵖ ≤ L_maxᵖ⁻¹·L, so E[Lᵖ] ≤ L_maxᵖ⁻¹·E[L]; the upper bound is Jensen's inequality, E[Lᵖ] ≥ E[L]ᵖ. The largest robot, not the mean, sets the cost.

Population (degrees of freedom) r Ω₁ Ω₂ Ω₃ Bounds 1/r and r⁻³
Half 18, half 35 0.76 1.32 1.58 1.76 1.32 and 2.30
90% at 18, 10% at 35 0.56 1.78 2.96 4.50 1.78 and 5.61
89% at 18, 10% at 35, 1% at 66 0.31 3.27 9.59 23.3 3.27 and 35.0

One large hybrid in a hundred designs makes a single padded batch cost three to twenty-three times the unpadded work. A single batch padded to 64 breaks any factor-of-four budget for small robots on every super-linear stage.

Buckets. A bucket set with ratio q between successive sizes caps the overhead inside a bucket at q for linear stages and q³ for cubic ones: 1.25 and 1.95 at q = 1.25, which takes eight buckets to span 12 to 66 degrees of freedom. DESIGN: the planner chooses bucket boundaries for the population in hand by an exact dynamic program over the sorted sizes, under a cap of eight buckets, with a registered default set {12, 18, 24, 36, 48, 64} that keeps launch graphs warm. If contact capacity, not degrees of freedom, dominates padding (rim-sampled wheels beside point feet), buckets become two-dimensional and may fragment; the planner reports that as FROB-REF-741.

When a bucket pays for itself. Each bucket is its own static step plan and pays its own launches. Let τ be the unpadded cost of one world-step, c_L the launch overhead of one step plan per step, N the world count, Ω the overhead of the coarser plan and Ω_b that of the finer one with B buckets instead of one. The finer plan wins when

N·τ·(Ω − Ω_b) > (B − 1)·c_L.

HYPOTHESIS for c_L: about 60 launches at 5 to 10 µs, 0.3 to 0.6 ms per step, equal to 600 to 1,200 world-steps at 2 M world-steps/s. A warm launch graph may make it nearly constant; the test is a measurement of launch overhead per plan at M2. With those figures a further split that removes 0.05 to 0.1 of overhead needs about 10⁴ worlds in the bucket to pay, and below about 10³ worlds per morphology two or three padded buckets are cheaper than eight.

Worked card (padding and buckets). Illustrative values.

  • Population. 8,192 worlds: 90% quadrupeds at 18 degrees of freedom, 10% humanoids at 35.
  • One bucket at 35. With stage shares of one half cubic, three tenths quadratic and one fifth linear (HYPOTHESIS, replaced by the M2 profile), Ω ≈ 0.5·4.50 + 0.3·2.96 + 0.2·1.78 = 3.5. At τ = 0.5 µs the step costs 8,192 × 0.5 µs × 3.5 + 0.45 ms = 14.8 ms.
  • Two buckets, 18 and 35. Ω_b = 1: 4.1 ms + 2 × 0.45 ms = 5.0 ms. Bucketing wins by about 3.
  • The same mix at 512 worlds. One bucket 1.35 ms; two buckets 1.16 ms: nearly equal. At 128 worlds the single padded bucket wins, 0.67 ms against 0.96 ms. The crossover is N = c_L/(τ·(Ω − Ω_b)) = 360 worlds.
  • Flip. A launch overhead ten times smaller moves the crossover to 36 worlds and makes fine buckets nearly free; a tree-sparse factor that skips masked slots (§16.6) removes most of the cubic term and makes one bucket viable at larger N.

What mixing can gain, stated before any speed test. Against a comparator that runs D designs one after another with W worlds each, the per-step ratio is D·(c_L + W·τ) / (c_L + Ω·D·W·τ). At D = 512, Ω = 1.3 and the figures above it is about 40 at W = 16, 11 at W = 64, 3.5 at W = 256, 1.4 at W = 1,024 and 1.1 at W = 2,048. The benefit is occupancy at few worlds per design, and it vanishes near a thousand worlds per design. That comparator is the weakest fair one. A comparator that batches parameter variants of one tree, compiles in parallel, or keeps one template per family (§17.14) closes most of the gap, and the gap against it is what the experiment measures.

17.6 Design-family lanes and the speedup bound

A family lane groups worlds that share more than a shape class: the same tree, the same sensor graph and, for a stretch of time, the same active contact mode. Inside a lane three things become shared constants instead of per-world data: structure tables and motion subspaces; the constraint-row layout, so stages 5 to 8 need no compaction and no masks; and the sparsity of the factor. Continuous parameters stay in per-slot columns.

A lane is valid only while every world in it has the same sign on every guard. A lane is segmented at the first guard whose sign is ambiguous across its members. At the next control-interval boundary the worlds on each side move, by index indirection, to the lane for their mode or to the general masked lane. Every lane has a fixed capacity, so shapes never change; a lane that is full sends the overflow to the general lane. Around grazing contacts, redundant contacts and uncertain mode sequences a world stays in the general lane.

The bound. If a fraction f of matched wall time is improved by a factor r and the method adds a relative overhead h (guard tests, regrouping, extra launches), then under a serial cost model

speedup ≤ 1 / (1 − f + f/r + h).

"Matched" means the same completed work on the strongest baseline's critical path, which also caches graphs, factors and compiled kernels. Three consequences. Improving 85% of the work without limit cannot reach ten (the bound is 6.7). With f = 0.97, r = 40 and h = 0.02 the bound is about 13.5. And a pruning ratio is never multiplied by a speedup that was measured on work the pruning already removed.

For the family lane itself: FACT, constraint assembly is about 15% of an incumbent's humanoid step (38 of 262 ns per world-step in a published trace, Appendix H). HYPOTHESIS: with compaction and mask overhead in the solve, f is 0.15 to 0.35 on our step, r is at most 4 and h is about 0.02, so the ceiling is 1.10 to 1.32. The lane is a modest optimization. Pilot before funding: read f from the M2 profile of the mixed workload; falsifier, a predicted bound below 1.15. ✖ KILL-FAMILYLANE: measured whole-step gain below 1.15 on the mixed workload, or h ≥ f·(1 − 1/r); fallback, the general masked lane only, with P-A sharing of structure tables kept.

17.7 Memory: which population shapes fit which machine

Bytes per padded world, physics only, from the formula of §16.6 plus a few kilobytes of state: about 35 kB in an 18-DoF bucket (C = 32, R = 160), 85 kB at 36 (C = 48, R = 256) and 223 kB at 64 (C = 96, R = 448), all with a dense Jacobian; the path-table Jacobian brings the 64 bucket to about 125 kB. Model tables add about 192 bytes per link (§21.11): about 12 kB for a 64-link variant, 3 MB for 256 such variants and at most about 50 MB for 4,096 designs; collision sample lists are counted separately. Training adds 23 to 100 kB of rollout storage per world (assumption, Appendix H). Planning limits, FACT (Appendix H): 16 to 19 GB usable of 24 GB on an RTX 4090; on the M5 Ultra a measured Metal working-set limit of about 222 GiB and a largest single buffer of about 167 GiB. The two RTX 4090 have no link, so a population split across them is two independent shards and each must fit.

Population shape Bucket Worlds Physics bytes One RTX 4090 Two cards, split M5 Ultra
256 designs × 16 worlds 18 4,096 0.14 GB Fits — Fits
4,096 designs × 16 worlds 18 65,536 2.3 GB Fits; 8.9 GB when training — Fits
512 designs × 64 worlds 36 32,768 2.8 GB Fits; 6.1 GB when training — Fits
256 designs × 16 worlds 64 4,096 0.9 GB Fits — Fits
5,000 designs × 16 worlds, pretraining a family policy 36 80,000 6.8 GB At the edge: 14.8 GB with rollout storage Fits Fits
8,192 designs × 32 worlds 36 262,144 22.3 GB Does not fit Fits, 11.1 GB each Fits
256 designs × 2,048 rollouts (sampling control inside the batch) 18 524,288 18.4 GB Does not fit Fits, 9.2 GB each Fits
The same 36 524,288 44.6 GB Does not fit Does not fit Fits
The same 64 524,288 117 GB Does not fit Does not fit Fits in one buffer under 167 GiB
1,024 designs × 256 worlds 64 262,144 58.5 GB Does not fit Does not fit Fits
16,384 designs × 64 worlds 18 1,048,576 36.7 GB Does not fit Does not fit Fits; 142 GB when training

Three readings. First, memory admits ten to a hundred times more worlds than the throughput targets of §36 use; row capacity dominates the bytes, and throughput and latency, not memory, bound ordinary populations. Second, "256 designs per dispatch" holds for policy rollouts on one card and fails for per-design sampling control, which multiplies worlds by the sample count: those shapes belong on the 256 GB machine or are refused. Third, scenes are shared and never one per world: a card holds 160 to 210 homes or two to three warehouses at 2 cm (FACT, Appendix H), so a population that needs more distinct large scenes than that also goes to the M5 Ultra (§19, §21). phylum.plan prints this table for the population in hand, and a shape that fits nowhere is FROB-REF-740 with repairs: fewer worlds per design, a split across cards, the path-table Jacobian, or the Mac. The M5 Ultra's throughput at these shapes is unmeasured; "fits" is a statement about memory only.

17.8 Admission, and edit-to-first-step latency as a target

Edit-to-first-step latency is the wall time from an accepted design.edit (a new DesignId) to the first committed world-step of the new design on a GPU. It is a first-class target with its own gate, because it is one of the two differentiators left (§17.1) and because an agent's turn is bounded by it.

Path Phases TARGET, 50th / 95th / 99th percentile
Warm: the design fits a shape class of a running cohort Compile with memoized subterms (§14); lower to a model table; copy tens of kilobytes into a free slot at the next reset under masks; first committed step 0.1 s / 0.5 s / 2 s
Cool: a new shape class, kernels cached The above, plus buffers and one launch-graph capture 1 s / 5 s / 10 s
Cold: empty kernel cache The above, plus kernel compilation Reported separately; never averaged with the others

FACT, for context (Appendix H): an incumbent's cold kernel compile has been reported at about 96 s and its cached start at about 0.9 s; a model edit there needs a host compile, a model upload and a new graph capture; and no incumbent publishes an edit-to-first-step latency. The comparison number therefore does not exist until it is measured on the pinned incumbents on trj, which happens before this plan claims anything about it (§39).

Slot-level dispatch F. One persistent environment exists per device and shape class. A slot that ends an episode takes its next episode from any admitted job at its reset; model, scene, task and job are per-world slots. A 96-pair comparison then occupies 192 of 8,192 slots beside other work instead of owning a launch. Results are keyed by world address, dispatch is a pure function of queue state, and by INV-KIN-04 digests are the same under any co-tenancy. Budgeted mode (DESIGN): at most 64 jobs per environment and 5% of slots reserved for short jobs; no shape change enters a running environment. Fallback, triggered by any invariance failure or a 99th percentile step time more than 10% over solo: one environment per job. The scheduling policy is §32's.

17.9 Specialization of survivors, a gated optimization F

A kernel specialized to one design (constants folded, loops unrolled, padding removed) may replace the table-driven kernel for designs that survive several rounds of a race and will run for a long time.

Ceiling, stated first. FACT (Appendix H): in an incumbent's published step trace for a humanoid at 8,192 worlds, everything a design-specialized recursion could touch (kinematics, centre-of-mass terms, composite inertia, bias forces, forward acceleration) sums to 53 of 262 ns, 20% of the step. An infinitely fast specialized recursion gives 1.25 and one twice as fast gives 1.11; on a manipulation scene the share is 16% and the ceiling 1.19. Generated code does not reliably beat a good generic library on CPUs either. The expected gain is therefore 1.05 to 1.15 and the cap is near 1.2.

Gate to fund. Specialization is built only if our own M2 profile shows a specializable share s of at least one third, so that 1/(1 − s) ≥ 1.5. Pilot before funding, two engineer-days: three arms on one 12-joint quadruped (padded table-driven kernel, unpadded table-driven kernel, a fully unrolled constant-folded kernel written by script) measured on the whole step.

If built. A survivor is promoted when T_remaining·(1 − 1/g) exceeds the compile cost, g being the measured gain; compile time is capped at 60 s per design on CPU workers; the swap happens at a control-interval boundary with a versioned state map. A specialized kernel is a new KernelId. It either reproduces the table-driven digests on the pinned corpus, in which case results are interchangeable, or it is a new kernel version that owes the fidelity battery (INV-PHYLUM-05). ✖ KILL-SPECIALIZE: measured whole-step gain below 1.2, or compile above 60 s per design; fallback, the table-driven kernel only. Derived derivative kernels for CPU solvers are a separate matter and stay (§21, §22).

17.10 The morphology generator F

A population search and a family policy both need a stated distribution over designs. The generator is that distribution, as an object with an identity.

  • What it is. A probabilistic attribute grammar over the design term of §14: archetype, then body plan (limb count and arrangement through mirror and radial), then limb type (serial leg, wheel-leg, wheel, track unit, arm), then parameters with distributions inside their declared bounds, actuator cards drawn from the catalogue, and typed edits with probabilities as mutation operators. One generator per robot family: FACT, the evidence supports policies per morphology class, with zero-shot transfer of 70 to 85% of specialist return inside a class and 10 to 40% across topology-diverse designs (Appendix H).
  • Identity and addressing. A generator is content-addressed (GeneratorId). Design i of seed s is a pure function of (GeneratorId, s, i) through the counter-based streams of §35, and comes with its derivation and its exact log-probability, so importance weights between generators are exact.
  • Validity. Samples that fail design check are rejected with their refusal codes; the rejection rate per code is part of the generator's report. DESIGN: a generator above 50% rejection is flagged as badly specified, because its stated distribution is then mostly not the distribution of what runs.
  • Splits. A DesignSplit separates training designs from held-out designs by seed and by structural stratum, and groups designs in one orbit of the observation-preserving group (§17.11) on the same side, so a mirrored copy of a training design cannot appear in the test set. A family-policy score is admitted only for designs absent from the policy's training manifest.
  • Support. phylum.generate --locate returns the derivation of a given design under a generator, or reports that it lies outside the support. A family policy applied to a design outside the support of its training generator is labeled out-of-distribution and its score is not a rung of the ladder (FROB-REF-541).
  • Shape awareness. The generator reports the bucket histogram and padding overhead of what it emits, and can be stratified by shape class so that a pretraining population does not fragment into small cohorts.
  • Cost of use. FACT (Appendix H): pretraining at 10⁹ to 10¹⁰ steps costs about 1 to 30 GPU-hours and at 10¹² steps over a thousand. The first needs an owner-approved compute envelope (§21); the second is out of scope.

Pilot before funding: sample 1,000 designs per family on the CPU and report rejection rate, bucket histogram and padding overhead under the bucket set; falsifier, more than eight buckets needed to hold padding waste at or below 25%. Ceiling: none on speed; the generator is what makes a family policy's claim well defined.

17.11 Campaign quotienting

A campaign should not simulate the same closed-loop system twice. Three mechanisms remove duplicates, each exact and each with a stated limit. None is a source of large factors except on fixtures built to have them.

Orbit representatives under the observation-preserving subgroup S. Before a combinatorial sweep of k module types over n slots, Phylum takes the group Γ_obs of §13 (the subgroup of the mechanical symmetry group that also preserves every sensor pose and field of view, the payload, the task's goal orientation and the terrain's symmetry), computes its action on the slots, and enumerates one canonical representative per orbit: the lexicographically least labeling in the orbit. The count is known in advance from the cycle index, which is the known answer. Worked, exact: ten leg types on the six slots of a radial hexapod under the dihedral group of order 12 give (10⁶ + 2·10 + 2·10² + 10³ + 3·10⁴ + 3·10³)/12 = 86,185 orbits, a reduction of 11.6. If sensors or the task leave only the sagittal mirror, the count is (10⁶ + 10³)/2 = 500,500, a reduction of 2.0. The honest range is two to twelve, and it shrinks as sensors break symmetry. A request to quotient under a group that is not observation-preserving is refused (FROB-REF-161).

Members of an orbit have equal outcome distributions, not equal outcomes at equal addresses: the design g·d in world w behaves as d does in world g⁻¹·w. Distributional results (scorecards, verdicts over a world distribution that the group preserves) are therefore shared across an orbit. A result tied to an address is shared only after the address is mapped through the group action, where the scene generator can do that; a comparison paired by address against a third design is re-run.

The mission-observable digest F. Two designs that lower to the same model tables, under the same task, scenes, policy, kernels, tier and profile, are the same experiment. The digest is the hash of exactly what the step plan and the task read: the lowered tables in canonical link order, the sensor poses and the task-visible fields. Results are memoized on it in the memo graph of §28. Two rules make it sound and useful:

  • Structural, not proved. "Equal digests imply identical traces" is a statement about the simulator and is not dischargeable in a proof kernel. It holds by construction, because the kernels read nothing but the tables, and it is checked by replay: sampled pairs inside a class must produce identical digests (class Tested, D2).
  • Exact lowering. Welded sub-assemblies are rolled up in exact rational arithmetic from the design's rational parameters and rounded once (§14), so designs that are equal in exact arithmetic lower to equal bits.

Fixture, exact: eight identical ballast slugs in 24 collinear slots give C(24, 8) = 735,471 placements, which enter rigid-body dynamics only through the count, Σx and Σx². There are 24,546 distinct (Σx, Σx²) pairs, so 24,546 classes, a reduction of 30.0. The negative control: two placements with equal Σx and different Σx² must fall in different classes. The factor is a property of this fixture. Adding a placement-dependent hot spot, a harness length or a collision shape to what the mission reads enlarges the digest and splits classes, down to singletons.

The invalidation contract S. Every memoized result records the digest of the fields it read. After an edit a result is reused exactly when that digest is unchanged, and otherwise recomputed.

  • What survives. Replacing a leaf module keeps its siblings' local analyses, derivations and kernels. An edit to a field the mission does not read (a label, a visual mesh) keeps everything.
  • What does not. An edit that touches a shared battery bus, a shared thermal sink, a kinematic loop across branches or any mass or geometry of a robot in contact invalidates every episode-level result of that design, because the floating body couples all of it. FACT (Appendix H): exact reuse after a dynamics-affecting edit is at most about a tenth of compute. The contract is hygiene and an evidence rule, not a speed lever.
  • Positive control. A mirror swap under an element of Γ_obs must reuse every distributional result, and every per-address result after the address mapping of the orbit rule above.
  • Negative controls. A shared-bus edit must invalidate. A reversed port sign must be refused. And the coupled bus: two fingers with identical local summaries on a bus limited to one unit of current have no feasible one-tick schedule and two feasible two-tick schedules, so a contract that adds peak demands prunes a feasible design. A local summary without time correlation is not a sound key.
  • Calibration. DESIGN: one hit in 64, chosen by key hash, is recomputed in shadow. One mismatch stops serving that kind of node until it is explained. Counterexample on file: a dependency omitted from a key returned a stale promotion in an executed test; keys are therefore derived from declared inputs, never written by hand.

17.12 Quality-diversity maps per dispatch F

A population dispatch returns a map, not a winner. Each phylum.sweep updates a MAP-Elites grid (or a centroidal tessellation) over behaviour descriptors such as mass, speed, energy per metre and footprint (fs-archive, FACT), and the envelope carries the cells that changed, their elites with provisional scores, the count per cell and the coverage of descriptor space. An agent's turn then yields many different good designs instead of one optimum, which is how fewer serial rounds are bought.

  • Merging. A cell keeps the maximum under a total order (score, then DesignId), so merging maps is commutative, associative and idempotent, and maps from two cards or two agents combine in any order.
  • Bias. A population scored under one shared policy drifts toward designs that policy steers easily. The envelope therefore reports descriptor coverage beside quality, and elites stay provisional until the rank-agreement record of §16.9 or §28 permits selection in their stratum.
  • Ceiling. FACT (Appendix H): published population co-design reports end-to-end compute gains of about three to four, and batched black-box search cuts rounds almost linearly in batch size up to about 128. HYPOTHESIS: campaigns on the benchmark of §39 need at most half the serial rounds of one-design-per-turn search. For parametric families this is not engine-specific; an incumbent batches the same sweep.

17.13 Paired finite-difference design gradients F

For a design with P continuous parameters θ and a smooth metric J, Phylum estimates ∂J/∂θ in one dispatch. Worlds are laid out as a stencil: for each seed s, one world at θ and two at θ ⊕ δ·eᵢ and θ ⊖ δ·eᵢ for each parameter, moved along the parameter's own retraction (§14) so that an inertia stays on its cone. Every world of a seed shares every random draw: scene, initial state, disturbances, sensor noise and randomization come from the same addresses, and only the design differs. The estimate is the mean over seeds of central differences (J₊ − J₋)/(2δ) with a confidence interval. A stencil of 20 parameters and 64 seeds is 41 × 64 = 2,624 worlds.

Limits, all enforced:

  • Short horizons only. FACT (Appendix H): in contact-rich simulation a perturbation of 5 × 10⁻⁴ degree has been reported to produce significant divergence after 50 steps. Pairing buys variance reduction only while the paired worlds stay correlated. DESIGN: the default horizon cap is 50 control intervals; a longer horizon is admitted only where the measured variance ratio of paired to independent differences is at least 2 at that horizon for that class.
  • The divergence caveat. Each pair records the first step at which the active contact sets of its two worlds differ. After that step the pair is two nearly independent samples and its difference is noise of variance about σ²/(2δ²). The estimator truncates each pair at its divergence step and labels the result truncated-horizon, or refuses if more than half the pairs diverge before a quarter of the horizon.
  • Smooth metrics only. Energy, tracking error and margins are admitted. A binary outcome (a fall, a mode switch) has zero gradient almost everywhere and a finite-difference variance that grows as 1/δ; it is refused (FROB-REF-540).
  • Determinism. Pairing requires a controlled determinism class (§35); under an uncontrolled class the verb refuses.
  • Bias. Under a shared policy the gradient points toward designs that policy handles. The result is classed Estimated and a step taken along it is checked by a specialist or a fine-tuned policy at its end point.

Pilot before funding, about four GPU-hours on an existing engine or on batch-cpu: one quadruped, 12 design parameters, a fixed policy; variance ratio at horizons of 25, 50, 100, 250 and 500 steps and perturbations of 1%, 3% and 10%; sign agreement over 20 replicate batches; 20 ascent steps against one generation of CMA-ES at equal budget. Falsifier: variance ratio below 2 at every horizon of 50 or more, or sign agreement below 70%. Ceiling: at most the ratio of evaluations, about 2P + 1 worlds per gradient against a few hundred evaluations of a derivative-free generation, and only on the smooth, short-horizon part of the objective. ✖ KILL-FDGRAD: the falsifier fires, or no end point is confirmed by a specialist; fallback, rank-based search (CMA-ES, racing) with the stencil kept as a sensitivity report for A-16.

17.14 The head-to-head experiment

This experiment decides whether the table-driven cohort is the engine, and the plan commits to its outcome in advance.

Arm What runs
A Phylum mixed cohorts: model tables, size buckets, slot dispatch
B (control) Per-family templates: at most nine padded templates, one per family of the eighteen archetypes of §14.10 that share a maximal tree (the grouping is frozen with the arm), each a fixed maximal tree with lockable joints and per-design parameter tables (level P-A only)
C An incumbent's mixed-topology batch, out of process, at its pinned version
D The leading incumbent run design by design with warm caches, parallel compilation and per-world parameter batching wherever the tree is shared
  • Workload. 512 topologies with realistic contact tails (wheels with rim samples and tracks beside point feet), drawn from the generators of §17.10 plus a frozen corpus of structural edits; K = 1, 8, 32 and 128 episodes per design; arms A and B on both machines, arms C and D on trj.
  • Measures. (1) Edit to first accepted result: wall time from the release of unseen structural edits to a completed, accepted cohort. (2) World-steps/s relative to a homogeneous batch of each bucket's largest design. (3) Padding overhead, bucket count and the share of the mutation corpus each arm can express. (4) Digest invariance (GATE-PHYLUM-02). (5) Run-to-run digests of the mixed batch for arm C under each of its solver's deterministic modes that accepts a heterogeneous batch; if none does, that is recorded as the finding.
  • Statistics. A fixed-n study (§24): at least ten timed repetitions per cell after warm-up, cold runs kept apart, runs interleaved, a 95% lower bound on each paired ratio, a thermal soak on the Mac, and the disclosure block of §36 on every number.
  • Pre-declared outcomes. If B expresses at least 95% of the mutation corpus and wins measure (1) at K ≤ 128, the templates become the engine and the table-driven cohort becomes an admission and preview path (✖ KILL-PHYLUM). If C ties or beats A on measures (1) and (2), the plan says so in its status table, withdraws the two performance differentiators of §17.1, and keeps the cohort verbs for what they still give: replay of one world outside its batch, deterministic forks and a Metal tier. If A falls below 0.5 of the harmonic mean of its homogeneous rates on 512 designs or needs more than eight buckets, KILL-PHYLUM fires as well.
  • Pilot before funding. At M1 with no GPU code: padding overhead and bucket count of the design corpus (§24) and of 1,000 generated designs under the bucket set. Then two to three engineer-days and under five GPU-hours on existing engines: 64 topologies with 64 worlds each for 1,000 steps, timing the incumbent's switch overhead design by design, a 64-DoF maximal template with locked joints against the specialized model, and an incumbent's mixed batch. Falsifiers: sequential bucketed runs within a factor of two of the best mixed wall-clock, or the maximal template above four times the specialized model.

17.15 Verbs

Verb Kind First milestone What it returns
phylum.plan read M2 For a population: buckets, padding overhead per stage exponent, bytes per world and in total, the machine that fits it, expected throughput and expected edit-to-first-step. Never runs physics.
phylum.sweep run M2 Runs a population (--designs, --worlds-per-design, --task, --tier); one envelope with a provisional scorecard per design, cohort count, padding overhead, the map delta of §17.12 and the coverage figures of §16.11.
phylum.generate read M3 Samples designs from a generator with derivations and log-probabilities; --locate tests support; --report gives rejection rates and the bucket histogram.
phylum.quotient read M3 Orbit representatives under Γ_obs with the orbit count, and mission-digest classes for a design set, with the fields that split them.
phylum.reuse read M3 For an edit: which cached results survive, which are invalidated, and the field that forced each invalidation.
phylum.gradient run M4 The paired finite-difference gradient with intervals, per-pair divergence steps and the truncated-horizon label where it applies.
phylum.map read M4 The quality-diversity map of a campaign or one dispatch; merge of maps.

Seven verbs, the limit for one cluster (§32). Specialization of survivors is an option of phylum.sweep, not a verb, and stays off until its gate to fund passes.

frob phylum sweep --designs @archive:gen-14 --task stairs-0.15 --worlds-per-design 16 --tier gpu-cuda --json
  # illustrative: 512 designs in 3 cohorts (buckets 18, 24, 36); padding overhead 1.2; 8,192 worlds
  # one envelope: a provisional scorecard per design, 37 map cells changed, referee share 0.4%

17.16 Refusals

Code Name Raised when Ranked repairs
FROB-REF-160 No shape class A design exceeds the largest admitted bucket and the tree-operator route is off Enable the route; run on batch-cpu; simplify
FROB-REF-161 Group not observation-preserving A quotient or a weight tie is requested under a group that moves a sensor, the payload or the task Use Γ_obs; declare the invariance and let the metamorphic test check it
FROB-REF-162 Reuse without a dependency digest A cached result has no recorded dependency digest, or the shadow recompute disagreed Recompute; quarantine the node kind
FROB-REF-540 Gradient unreliable Non-smooth metric, horizon above the admitted cap, pairs diverged early, or an uncontrolled determinism class Shorten the horizon; change the metric; use rank-based search
FROB-REF-541 Outside generator support A family-policy score is requested for a design outside its training generator's support Fine-tune; train a specialist; extend the generator
FROB-REF-740 Population exceeds device The population's bytes exceed every admitted placement Fewer worlds per design; split across cards; path-table Jacobian; the M5 Ultra
FROB-REF-741 Bucket fragmentation More than eight buckets are needed to hold padding waste at or below the limit Stratify the generator; raise the waste limit knowingly; split the campaign

17.17 Invariants and proof obligations

ID Statement How it is checked
INV-PHYLUM-01 Every result of a cohort is keyed by world address and ModelId. Slot, cohort and arrival order appear in no result. Permute slots and co-tenants; envelopes compare equal after sorting by address
INV-PHYLUM-02 A world of a mixed cohort has the same digests as the same world in a homogeneous batch of its own model. GATE-PHYLUM-02
INV-PHYLUM-03 Quotients use Γ_obs only, and equal mission digests mean equal lowered tables. The lidar counterexample; replay inside sampled classes
INV-PHYLUM-04 A memoized result is served only when its dependency digest is unchanged. Positive and negative controls of §17.11; shadow recompute
INV-PHYLUM-05 A specialized kernel reproduces table-driven digests or is a new kernel version. Digest comparison on the pinned corpus
INV-PHYLUM-06 A generated design is a pure function of (GeneratorId, seed, index) and its log-probability is exact. Regeneration on both platforms; probabilities sum to one on enumerable sub-grammars
INV-PHYLUM-07 The worlds of one stencil seed share every random draw. Stream addresses compared; a planted unshared stream is caught
ID Obligation Class and route
PO-PHYLUM-01 The number of canonical representatives enumerated equals the cycle-index count. D0, exact; checked for every sweep
PO-PHYLUM-02 1/r ≤ Ω_p ≤ r⁻ᵖ for p ≥ 1. Proved (two lines, §17.5); property-tested on random populations
PO-PHYLUM-03 Map merge is commutative, associative and idempotent. Proved from the total order; property-tested

17.18 Known answers

Case Truth Negative control
Hexapod sweep, ten leg types, dihedral group of order 12 86,185 representatives Counting under the rotation subgroup only
The same with only the sagittal mirror 500,500 representatives Quotienting under the mechanical group when a front-only sensor is mounted
Ballast fixture 24,546 classes from 735,471 placements Two placements with equal Σx and different Σx² merged
Padding table The Ω values of §17.5 from the three populations —
Mixed cohort against homogeneous Identical digests per world A reduction tree shaped by capacity
Mirror swap under Γ_obs Every distributional result reused; per-address results reused after address mapping A shared-bus edit reused; an unmapped per-address result reused
Coupled bus Zero one-tick schedules and two two-tick schedules Pruning by summed peak demand
Paired gradient on a ballistic body ∂R/∂v = 2·v·sin 2α / g for range R = v²·sin 2α / g, recovered to O(δ²) A binary success metric admitted
Bouncing body near grazing Pairs flagged at their divergence step Divergence ignored and the full horizon averaged
Generator The same (GeneratorId, seed, index) gives the same DesignId on Linux and macOS A generator reading the wall clock or a hash-map order

17.19 Gates and kill gates

Gate Claim Threshold (TARGET) Earliest
GATE-PHYLUM-01 Known answer Orbit and ballast rows of §17.18 exact; zero false quotients in 10⁴ sampled class pairs by replay; the lidar and equal-Σx controls caught. M3
GATE-PHYLUM-02 Known answer (determinism) Each of 512 designs in a mixed cohort has the same digests as in a homogeneous batch, on batch-cpu and per GPU back end, at three bucket sizes and under slot dispatch with 100 random co-tenancies. M2
GATE-PHYLUM-03 Performance The head-to-head of §17.14 run and published with all four arms. For the cohort to remain the engine: at least 0.70 of the harmonic mean of the homogeneous rates (§36.3 row 10) at padding waste of 25% or less on cohorts of at least 256 designs, a lower bound above 1 on the ratio against arm B for edit to first accepted result at K ≤ 128, and at most eight buckets. M2
GATE-PHYLUM-04 Performance Edit-to-first-step latency on the warm and cool paths inside the targets of §17.8 over 1,000 edits from the mutation corpus, cold reported separately, with the same quantity measured on the pinned incumbents. M2
GATE-PHYLUM-05 Known answer Invalidation contract: the positive control reuses, each negative control invalidates or refuses, and 10⁴ random edit sequences give results equal to full recomputation. M3
GATE-PHYLUM-06 Known answer and method Paired gradient: the ballistic row within 1%; binary metrics refused; on the pilot task a variance ratio of at least 2 at the admitted horizon and sign agreement of at least 70% over 20 replicates. M4
GATE-PHYLUM-07 Known answer Generator: reproducible across platforms; exact probabilities on enumerable sub-grammars; rejection rates reported; no orbit straddles a DesignSplit. M3
GATE-PHYLUM-08 Design comparison Maps per dispatch: on the benchmark of §39, campaigns reach the acceptance target in at most half the serial rounds of one-design-per-turn search at equal compute, with compute reported beside rounds. M4
Kill Trigger Fallback
✖ KILL-PHYLUM GATE-PHYLUM-03 fails: below 0.5 of the harmonic mean of the homogeneous rates on 512 designs, more than eight buckets, or arm B covers at least 95% of the mutation corpus and is faster. This is the one decision on the mixed batch; KILL-BUCKET (§21.11) is its device-side statement and defers to it Per-family templates become the engine; cohorts are formed by exact tree with per-design parameter tables (level P-A); the table-driven path remains for admission and preview on batch-cpu
✖ KILL-FAMILYLANE §17.6 The general masked lane only
✖ KILL-SPECIALIZE §17.9 The table-driven kernel only
✖ KILL-FDGRAD §17.13 Rank-based search; the stencil as a sensitivity report
✖ KILL-QUOTIENT One false quotient survives a repair cycle, or digest classes save less than 5% of dispatches on the campaign corpus Orbit representatives only; the digest is kept as a cache key with no claim

17.20 Open questions

  • OPEN-PHYLUM-1. The registered default bucket set and the padding-waste limit. Default: {12, 18, 24, 36, 48, 64} and 25%. Decided by the M1 pilot of §17.14. Fail-closed: the planner refuses with FROB-REF-741.
  • OPEN-PHYLUM-2. Run length per model, 16 or 32 worlds. Default: 32 on CUDA, the device's reported execution width on Metal. Decided by the M2 profile.
  • OPEN-PHYLUM-3. The compute envelope for the head-to-head and for pretraining populations. Owner: project lead. Default: the one-device, one-hour rule of §21 until approved, which admits the pilot and not the full experiment.
  • OPEN-PHYLUM-4. Which incumbents and versions fill arms C and D. Frozen with the rival audit (§39) before any run. Default: none, and the arms report not run.
  • OPEN-PHYLUM-5. Whether contact capacity needs its own bucket axis. Default: no, one axis on degrees of freedom with capacities taken from the largest member. Decided by the overflow and padding statistics at M2.

17.21 Limb-archetype kernels

What it is. F. This subsection extends §17.9 and uses the limb port interface of §14.6. Between the table-driven kernel, which serves every design of a shape class, and a kernel specialized to one design, which §17.9 reserves for survivors, there is a third level: a kernel specialized to a limb archetype. An archetype fixes a limb's joint-class sequence and depth (DESIGN examples: a serial three-revolute leg, a four-joint wheel-leg, a seven-revolute arm, a track flipper) and leaves every value in the model table: placements, screw components, inertias, limits, card slots. Archetype kernels are compiled once per kernel version and never per design, so a new design assembled from registered archetypes runs on them with no compile. Owner: frob-kernels for the kernels; frob-phylum for decomposing a model table into a base and archetype instances. Exposure: the option --limb-kernels of phylum.sweep; phylum.plan reports the archetype coverage of a population.

Mechanism (DESIGN).

  • Registry. At most 16 archetypes, drawn from the limbs of the archetypes of §14.10. A limb is an instance when its joint-class sequence and depth match after the canonical ordering of §14. A design with any limb outside the registry stays whole on the table-driven kernel.
  • Per-limb stages. Stages 1 and 6 of §16.6 run per limb instance, with loops unrolled over the archetype's fixed depth and the joint-class column masks resolved at compile time. Worlds are grouped so that one execution group holds instances of one archetype.
  • The port is the only interface. On the dense route a limb with k joints returns its k × k block of the mass matrix, the 6 × k coupling block expressed at its mechanical port, its composite inertia at the port (10 scalars) and its bias terms, and the base kernel assembles the full matrix. On the tree-operator route it returns the LimbPortPacket of §14.6 (27 scalars).
  • Contact is untouched. Stages 5, 7 and 8 run for the whole robot. Contact rows couple every limb through the base. An archetype kernel is a faster way to compute terms that the coupled solve needs, and nothing more.
  • Same bits. The operation order inside an archetype kernel equals the table-driven kernel's order for the same limb, so results are bit-identical (INV-PHYLUM-05) and a design moves between the two paths without a new StepPlanId.

Counterexample: a port packet is not the limb when its foot is down. A point mass m on a massless leg stands on a floor under gravity g. Treated as a free tree limb, its packet reports the body's inertia and a bias equal to the weight m·g, and the base equation closed from the packet gives a downward acceleration g. With the contact row the normal force is m·g and the acceleration is zero. Any path that closes the base equation from port packets without the contact rows is wrong by exactly g here. The second control is the packet of §14.6 with value diag(0, 1, 1, 1, 1, 1): it is accepted as a packet and refused as an invertible matrix.

Ceiling, stated first. An archetype kernel touches the stages that per-design specialization touches, and cannot beat it. FACT (Appendix H): those stages are 53 of 262 ns of an incumbent's humanoid step, so an infinitely fast recursion gives 262/209 = 1.25 and one twice as fast gives 1.11. HYPOTHESIS: the gain is 1.05 to 1.15. What this level adds over §17.9 is reach: with zero compile per design it applies inside the edit loop and to every design, where per-design kernels apply to survivors only. Two things are not claimed: a fused kernel for all topologies at three quarters of homogeneous throughput, which rested on an occupancy miscount, and a lossless limb summary under contact, which the counterexample above refutes.

Pilot before funding. A fourth arm in the pilot of §17.9, at no added calendar time: the 12-joint quadruped as a base and four instances of the serial three-revolute archetype, measured on the whole step beside the padded table-driven, unpadded table-driven and fully unrolled arms. Falsifier: the archetype arm gains less than 1.10 on the whole step, or less than half of what the fully unrolled arm gains. A paper measurement at M1 accompanies it: the share of limbs in the design corpus (§24) and in 1,000 generated designs (§17.10) that match a registry of 16.

Gate and kill gate.

Gate Claim Threshold (TARGET) Earliest
GATE-PHYLUM-09 Performance; known answer (determinism) Every registered archetype reproduces table-driven digests bit for bit on the pinned corpus; at most 16 archetypes cover at least 90% of the limbs of the design corpus; zero kernel compiles while 1,000 corpus edits are admitted; the 95% lower bound of the paired whole-step ratio against the table-driven kernel is at least 1.15 at 8,192 worlds; both hand controls reproduce. M2

✖ KILL-LIMBKERNEL. Trigger: the pilot's falsifier holds; or coverage is below 80% with 16 archetypes; or the gate's ratio fails after one repair cycle. Fallback: the table-driven kernel for every design, per-design specialization for survivors under §17.9's own gate, and the limb port interface kept for the tree-operator route and the per-limb caches of §14.6.

In the head-to-head of §17.14 the level is reported as a variant of arm A on measures (1) and (2).