Phylum: families of bodies in one batch
9,052 words21 subsections14 tables2 listingsPlan version 1.1
On this page21
Crate: frob-phylum (layer 3). Phylum calls frob-model for lowering (§14), frob-algebra for groups (§13),
frob-dyn and frob-kernels for the step (§16) and, through the airlock, FrankenSim's fs-archive, fs-race
and fs-recompute.
17.1 What Phylum owns
A design studio's workload is many different robots with few worlds each. Phylum owns what makes a population, and not one model, the unit of work:
- cohorts: shape classes, model tables and slots, size buckets and the bucket planner (§17.3 to §17.5);
- design-family lanes that share structure and split at ambiguous guards (§17.6);
- memory admission for population shapes on each machine (§17.7);
- admission of a design into a running cohort, with edit-to-first-step latency as a target (§17.8);
- specialization of survivors as a gated optimization (§17.9);
- the morphology generator, which defines the distribution over designs (§17.10);
- campaign quotienting: orbit representatives, the mission-observable digest and the invalidation contract (§17.11);
- quality-diversity maps per dispatch and paired finite-difference design gradients (§17.12, §17.13);
- the head-to-head experiment that decides whether any of this stays (§17.14).
Phylum does not own the step or its kernels (§16, §21), the design term or the model compiler (§14), the
computation of symmetry groups (§13), family policies and inheritance (§23), racing, the archive and the memo
store (§28), or the scheduling policy of frobd (§32).
The claim, stated at the size the evidence supports F. FACT (Appendix H): batching parameter values per world is incumbent practice. Batching different trees is not available in the leading reduced-coordinate incumbent, whose structural arrays carry no world dimension and whose documentation says heterogeneous degrees of freedom are not available; a second incumbent requires all variants to share links, joints and names. Mixed topologies in one batch do exist in one manipulation simulator and in two solvers of another engine, and one incumbent solver publishes a mixed batch of three body plans at 86 k samples/s, 0.86 of the harmonic mean of its homogeneous rates; that solver is maximal-coordinate, is documented as not deterministic, and is reported as slower than the reduced-coordinate solvers of its framework. What is left to claim is therefore narrow: mixed-topology populations with few worlds per design, in size buckets, in a reduced-coordinate solver, with any one world replayable outside its cohort. The differentiators that remain are two measured numbers, throughput and the time from an edit to the first GPU step, and one property, run-to-run replay and forks inside the mixed batch. All three can be lost (§17.14).
17.2 Decisions
| ADR | Decision | Reason |
|---|---|---|
| ADR-PHYLUM-01 | The unit of batching is a cohort of distinct designs that share one step plan, not one model. | A population of 512 designs with 16 worlds each fills 8,192 slots; run design by design it uses 0.2% of a device per launch and pays launch overhead 512 times (§17.5). |
| ADR-PHYLUM-02 | Morphology is table data inside padded capacities, in size buckets. No interpreter, no compile per design. | Kernels then compile per shape class and kernel version. The step is memory-bandwidth-bound at these state sizes (Appendix H), so a table read costs bytes, not branches, and buckets bound the bytes. |
| ADR-PHYLUM-03 | Results are keyed by world address and ModelId, never by slot, cohort or arrival order. |
INV-KIN-04: a world is replayed on batch-cpu with its own model alone, and the CPU never needs the cohort. |
| ADR-PHYLUM-04 | Deduplication and weight sharing use only the observation-preserving symmetry subgroup (§13). | The mechanical group ignores sensors, payload and task. A quadruped with a front-only lidar has a four-element mechanical group and a two-element closed-loop group; merging under the larger one merges designs the mission can tell apart. |
| ADR-PHYLUM-05 | A per-family-template arm competes in the head-to-head, and the plan commits in advance to adopt it if it wins. | Templates are the incumbent regime plus per-design rows. If they cover the mutations agents make and run faster, the table-driven cohort is the wrong engine and should become a preview path. |
| ADR-PHYLUM-06 | Exact sharing only. A quantity is shared between designs when all of its inputs and rounding rules agree; a nearby trajectory is never reused. | A smooth distance in design space gives no right to reuse contact history. Reuse with an error certificate over contact is a research problem (§25), not a batching feature. |
17.3 Three levels of variation
| Level | What differs between worlds of one batch | Mechanism | Tier |
|---|---|---|---|
| P-A parameter variants | Values on one tree: lengths, inertias, gear ratios, card parameters, radii, friction | One structure table; one parameter table per model slot; no masks, no padding | S (§16.6) |
| P-B model variants | Trees, joint classes, sample lists, actuator counts | One model table per slot inside the padded capacities of a shape class; masks | F |
| P-C design families | As P-B, plus grouping by shared structure and current contact mode | Family lanes with shared constants and static row layouts (§17.6) | F |
17.4 Shape classes and model tables
A shape class is the set of capacities one static step plan is built for. Its hash, the ShapeClassId, is
part of the StepPlanId (§16.6). A cohort is a set of model slots and worlds running under one step plan.
// DESIGN: fixes intent and layout, not signatures.
pub struct ShapeClass {
pub n_v: u16, // degree-of-freedom capacity: the bucket size
pub links: u16, // link capacity
pub depth: u8, // maximum tree depth
pub samples: u16, // collision samples tested per world
pub contacts: u16, // C of §16.6
pub rows: u16, // R of §16.6
pub actuators: u16,
pub sensor_layout: SensorLayoutId,
pub single_structure: bool, // P-A: one tree, unmasked path
}
pub struct ModelTable { // one per model slot; read-only in a rollout; about 192 bytes per link (§21.11)
pub model: ModelId,
pub level_start: Vec<u16>, // links sorted by depth; ranges per level
pub parent: Vec<u16>, // parent index, less than the link's own; padding links are roots of nothing
pub placement: Vec<Se3Packed>, // parent_from_joint
pub screw: Vec<[f32; 18]>, // screw basis S in R^(6x3)
pub screw_cols: Vec<u8>, // k = 0 (padding), 1, 2 or 3 active columns
pub inertia: Vec<[f32; 10]>, // zero for padding links
pub limits: Vec<JointLimits>,
pub actuator_slot: Vec<u16>,
pub sample_range: Vec<(u16, u16)>,
}
pub struct WorldSlots { pub model: u32, pub scene: u32, pub task: u32, pub job: u32, pub addr: WorldAddr }
- Joint classes are data. A joint is its screw basis with a column count: k = 1 for revolute, prismatic and helical joints, 2 for cylindrical and universal, 3 for spherical and planar (C-01). The joint map τ(θ₁S₁)·τ(θ₂S₂)·τ(θ₃S₃), with masked columns as the identity, is one derived kernel with no branch on joint kind. This is where the Lie-typed core pays in this chapter: the joint is its screw basis.
- Padding is exact. Padding links are identity transforms with zero inertia, appended after the real links and masked out of every sum; padded degrees of freedom get identity rows. By PO-KIN-02 and PO-KIN-03 the real part of every result is bit-identical to the unpadded computation.
- Runs. Worlds of one model are contiguous in runs of 16 or 32, so one warp or SIMD group holds one model and the level loop does not diverge inside it. Contact counts still differ inside a run, which is why stages 5 to 8 of §16.6 are compacted and masked per world: homogeneity by morphology is not homogeneity by work.
- Capacities are read, not compiled. Kernels read capacities from the step-plan header. A new bucket size needs new buffers and a new static launch graph, and no kernel compile.
- Loops and rolling constraints are rows of the coupled solve (§16.6 stage 7), not table entries.
17.5 Size buckets, and when bucketing recovers the padding overhead
The bound. Let work per world in a stage scale as Lᵖ in the size L of the robot (p = 1 for kinematics, about 2 for Jacobian and Hessian assembly, 3 for a dense factor). A cohort padded to L_max spends L_maxᵖ per world, so its overhead against unpadded work is Ω_p = L_maxᵖ / E[Lᵖ]. With r = E[L]/L_max, the mean-to-maximum ratio,
1/r ≤ Ω_p ≤ r⁻ᵖ ≤ r⁻³ for 1 ≤ p ≤ 3.
The lower bound holds because Lᵖ ≤ L_maxᵖ⁻¹·L, so E[Lᵖ] ≤ L_maxᵖ⁻¹·E[L]; the upper bound is Jensen's inequality, E[Lᵖ] ≥ E[L]ᵖ. The largest robot, not the mean, sets the cost.
| Population (degrees of freedom) | r | Ω₁ | Ω₂ | Ω₃ | Bounds 1/r and r⁻³ |
|---|---|---|---|---|---|
| Half 18, half 35 | 0.76 | 1.32 | 1.58 | 1.76 | 1.32 and 2.30 |
| 90% at 18, 10% at 35 | 0.56 | 1.78 | 2.96 | 4.50 | 1.78 and 5.61 |
| 89% at 18, 10% at 35, 1% at 66 | 0.31 | 3.27 | 9.59 | 23.3 | 3.27 and 35.0 |
One large hybrid in a hundred designs makes a single padded batch cost three to twenty-three times the unpadded work. A single batch padded to 64 breaks any factor-of-four budget for small robots on every super-linear stage.
Buckets. A bucket set with ratio q between successive sizes caps the overhead inside a bucket at q for linear
stages and q³ for cubic ones: 1.25 and 1.95 at q = 1.25, which takes eight buckets to span 12 to 66 degrees of
freedom. DESIGN: the planner chooses bucket boundaries for the population in hand by an exact dynamic program
over the sorted sizes, under a cap of eight buckets, with a registered default set {12, 18, 24, 36, 48, 64} that
keeps launch graphs warm. If contact capacity, not degrees of freedom, dominates padding (rim-sampled wheels
beside point feet), buckets become two-dimensional and may fragment; the planner reports that as
FROB-REF-741.
When a bucket pays for itself. Each bucket is its own static step plan and pays its own launches. Let τ be the unpadded cost of one world-step, c_L the launch overhead of one step plan per step, N the world count, Ω the overhead of the coarser plan and Ω_b that of the finer one with B buckets instead of one. The finer plan wins when
N·τ·(Ω − Ω_b) > (B − 1)·c_L.
HYPOTHESIS for c_L: about 60 launches at 5 to 10 µs, 0.3 to 0.6 ms per step, equal to 600 to 1,200 world-steps at 2 M world-steps/s. A warm launch graph may make it nearly constant; the test is a measurement of launch overhead per plan at M2. With those figures a further split that removes 0.05 to 0.1 of overhead needs about 10⁴ worlds in the bucket to pay, and below about 10³ worlds per morphology two or three padded buckets are cheaper than eight.
Worked card (padding and buckets). Illustrative values.
- Population. 8,192 worlds: 90% quadrupeds at 18 degrees of freedom, 10% humanoids at 35.
- One bucket at 35. With stage shares of one half cubic, three tenths quadratic and one fifth linear (HYPOTHESIS, replaced by the M2 profile), Ω ≈ 0.5·4.50 + 0.3·2.96 + 0.2·1.78 = 3.5. At τ = 0.5 µs the step costs 8,192 × 0.5 µs × 3.5 + 0.45 ms = 14.8 ms.
- Two buckets, 18 and 35. Ω_b = 1: 4.1 ms + 2 × 0.45 ms = 5.0 ms. Bucketing wins by about 3.
- The same mix at 512 worlds. One bucket 1.35 ms; two buckets 1.16 ms: nearly equal. At 128 worlds the single padded bucket wins, 0.67 ms against 0.96 ms. The crossover is N = c_L/(τ·(Ω − Ω_b)) = 360 worlds.
- Flip. A launch overhead ten times smaller moves the crossover to 36 worlds and makes fine buckets nearly free; a tree-sparse factor that skips masked slots (§16.6) removes most of the cubic term and makes one bucket viable at larger N.
What mixing can gain, stated before any speed test. Against a comparator that runs D designs one after another with W worlds each, the per-step ratio is D·(c_L + W·τ) / (c_L + Ω·D·W·τ). At D = 512, Ω = 1.3 and the figures above it is about 40 at W = 16, 11 at W = 64, 3.5 at W = 256, 1.4 at W = 1,024 and 1.1 at W = 2,048. The benefit is occupancy at few worlds per design, and it vanishes near a thousand worlds per design. That comparator is the weakest fair one. A comparator that batches parameter variants of one tree, compiles in parallel, or keeps one template per family (§17.14) closes most of the gap, and the gap against it is what the experiment measures.
17.6 Design-family lanes and the speedup bound
A family lane groups worlds that share more than a shape class: the same tree, the same sensor graph and, for a stretch of time, the same active contact mode. Inside a lane three things become shared constants instead of per-world data: structure tables and motion subspaces; the constraint-row layout, so stages 5 to 8 need no compaction and no masks; and the sparsity of the factor. Continuous parameters stay in per-slot columns.
A lane is valid only while every world in it has the same sign on every guard. A lane is segmented at the first guard whose sign is ambiguous across its members. At the next control-interval boundary the worlds on each side move, by index indirection, to the lane for their mode or to the general masked lane. Every lane has a fixed capacity, so shapes never change; a lane that is full sends the overflow to the general lane. Around grazing contacts, redundant contacts and uncertain mode sequences a world stays in the general lane.
The bound. If a fraction f of matched wall time is improved by a factor r and the method adds a relative overhead h (guard tests, regrouping, extra launches), then under a serial cost model
speedup ≤ 1 / (1 − f + f/r + h).
"Matched" means the same completed work on the strongest baseline's critical path, which also caches graphs, factors and compiled kernels. Three consequences. Improving 85% of the work without limit cannot reach ten (the bound is 6.7). With f = 0.97, r = 40 and h = 0.02 the bound is about 13.5. And a pruning ratio is never multiplied by a speedup that was measured on work the pruning already removed.
For the family lane itself: FACT, constraint assembly is about 15% of an incumbent's humanoid step (38 of 262 ns per world-step in a published trace, Appendix H). HYPOTHESIS: with compaction and mask overhead in the solve, f is 0.15 to 0.35 on our step, r is at most 4 and h is about 0.02, so the ceiling is 1.10 to 1.32. The lane is a modest optimization. Pilot before funding: read f from the M2 profile of the mixed workload; falsifier, a predicted bound below 1.15. ✖ KILL-FAMILYLANE: measured whole-step gain below 1.15 on the mixed workload, or h ≥ f·(1 − 1/r); fallback, the general masked lane only, with P-A sharing of structure tables kept.
17.7 Memory: which population shapes fit which machine
Bytes per padded world, physics only, from the formula of §16.6 plus a few kilobytes of state: about 35 kB in an 18-DoF bucket (C = 32, R = 160), 85 kB at 36 (C = 48, R = 256) and 223 kB at 64 (C = 96, R = 448), all with a dense Jacobian; the path-table Jacobian brings the 64 bucket to about 125 kB. Model tables add about 192 bytes per link (§21.11): about 12 kB for a 64-link variant, 3 MB for 256 such variants and at most about 50 MB for 4,096 designs; collision sample lists are counted separately. Training adds 23 to 100 kB of rollout storage per world (assumption, Appendix H). Planning limits, FACT (Appendix H): 16 to 19 GB usable of 24 GB on an RTX 4090; on the M5 Ultra a measured Metal working-set limit of about 222 GiB and a largest single buffer of about 167 GiB. The two RTX 4090 have no link, so a population split across them is two independent shards and each must fit.
| Population shape | Bucket | Worlds | Physics bytes | One RTX 4090 | Two cards, split | M5 Ultra |
|---|---|---|---|---|---|---|
| 256 designs × 16 worlds | 18 | 4,096 | 0.14 GB | Fits | — | Fits |
| 4,096 designs × 16 worlds | 18 | 65,536 | 2.3 GB | Fits; 8.9 GB when training | — | Fits |
| 512 designs × 64 worlds | 36 | 32,768 | 2.8 GB | Fits; 6.1 GB when training | — | Fits |
| 256 designs × 16 worlds | 64 | 4,096 | 0.9 GB | Fits | — | Fits |
| 5,000 designs × 16 worlds, pretraining a family policy | 36 | 80,000 | 6.8 GB | At the edge: 14.8 GB with rollout storage | Fits | Fits |
| 8,192 designs × 32 worlds | 36 | 262,144 | 22.3 GB | Does not fit | Fits, 11.1 GB each | Fits |
| 256 designs × 2,048 rollouts (sampling control inside the batch) | 18 | 524,288 | 18.4 GB | Does not fit | Fits, 9.2 GB each | Fits |
| The same | 36 | 524,288 | 44.6 GB | Does not fit | Does not fit | Fits |
| The same | 64 | 524,288 | 117 GB | Does not fit | Does not fit | Fits in one buffer under 167 GiB |
| 1,024 designs × 256 worlds | 64 | 262,144 | 58.5 GB | Does not fit | Does not fit | Fits |
| 16,384 designs × 64 worlds | 18 | 1,048,576 | 36.7 GB | Does not fit | Does not fit | Fits; 142 GB when training |
Three readings. First, memory admits ten to a hundred times more worlds than the throughput targets of §36 use;
row capacity dominates the bytes, and throughput and latency, not memory, bound ordinary populations. Second,
"256 designs per dispatch" holds for policy rollouts on one card and fails for per-design sampling control, which
multiplies worlds by the sample count: those shapes belong on the 256 GB machine or are refused. Third, scenes
are shared and never one per world: a card holds 160 to 210 homes or two to three warehouses at 2 cm (FACT,
Appendix H), so a population that needs more distinct large scenes than that also goes to the M5 Ultra (§19,
§21). phylum.plan prints this table for the population in hand, and a shape that fits nowhere is
FROB-REF-740 with repairs: fewer worlds per design, a split across cards, the path-table Jacobian, or the Mac.
The M5 Ultra's throughput at these shapes is unmeasured; "fits" is a statement about memory only.
17.8 Admission, and edit-to-first-step latency as a target
Edit-to-first-step latency is the wall time from an accepted design.edit (a new DesignId) to the first
committed world-step of the new design on a GPU. It is a first-class target with its own gate, because it is one
of the two differentiators left (§17.1) and because an agent's turn is bounded by it.
| Path | Phases | TARGET, 50th / 95th / 99th percentile |
|---|---|---|
| Warm: the design fits a shape class of a running cohort | Compile with memoized subterms (§14); lower to a model table; copy tens of kilobytes into a free slot at the next reset under masks; first committed step | 0.1 s / 0.5 s / 2 s |
| Cool: a new shape class, kernels cached | The above, plus buffers and one launch-graph capture | 1 s / 5 s / 10 s |
| Cold: empty kernel cache | The above, plus kernel compilation | Reported separately; never averaged with the others |
FACT, for context (Appendix H): an incumbent's cold kernel compile has been reported at about 96 s and its cached
start at about 0.9 s; a model edit there needs a host compile, a model upload and a new graph capture; and no
incumbent publishes an edit-to-first-step latency. The comparison number therefore does not exist until it is
measured on the pinned incumbents on trj, which happens before this plan claims anything about it (§39).
Slot-level dispatch F. One persistent environment exists per device and shape class. A slot that ends an episode takes its next episode from any admitted job at its reset; model, scene, task and job are per-world slots. A 96-pair comparison then occupies 192 of 8,192 slots beside other work instead of owning a launch. Results are keyed by world address, dispatch is a pure function of queue state, and by INV-KIN-04 digests are the same under any co-tenancy. Budgeted mode (DESIGN): at most 64 jobs per environment and 5% of slots reserved for short jobs; no shape change enters a running environment. Fallback, triggered by any invariance failure or a 99th percentile step time more than 10% over solo: one environment per job. The scheduling policy is §32's.
17.9 Specialization of survivors, a gated optimization F
A kernel specialized to one design (constants folded, loops unrolled, padding removed) may replace the table-driven kernel for designs that survive several rounds of a race and will run for a long time.
Ceiling, stated first. FACT (Appendix H): in an incumbent's published step trace for a humanoid at 8,192 worlds, everything a design-specialized recursion could touch (kinematics, centre-of-mass terms, composite inertia, bias forces, forward acceleration) sums to 53 of 262 ns, 20% of the step. An infinitely fast specialized recursion gives 1.25 and one twice as fast gives 1.11; on a manipulation scene the share is 16% and the ceiling 1.19. Generated code does not reliably beat a good generic library on CPUs either. The expected gain is therefore 1.05 to 1.15 and the cap is near 1.2.
Gate to fund. Specialization is built only if our own M2 profile shows a specializable share s of at least one third, so that 1/(1 − s) ≥ 1.5. Pilot before funding, two engineer-days: three arms on one 12-joint quadruped (padded table-driven kernel, unpadded table-driven kernel, a fully unrolled constant-folded kernel written by script) measured on the whole step.
If built. A survivor is promoted when T_remaining·(1 − 1/g) exceeds the compile cost, g being the measured
gain; compile time is capped at 60 s per design on CPU workers; the swap happens at a control-interval boundary
with a versioned state map. A specialized kernel is a new KernelId. It either reproduces the table-driven
digests on the pinned corpus, in which case results are interchangeable, or it is a new kernel version that owes
the fidelity battery (INV-PHYLUM-05). ✖ KILL-SPECIALIZE: measured whole-step gain below 1.2, or compile above
60 s per design; fallback, the table-driven kernel only. Derived derivative kernels for CPU solvers are a
separate matter and stay (§21, §22).
17.10 The morphology generator F
A population search and a family policy both need a stated distribution over designs. The generator is that distribution, as an object with an identity.
- What it is. A probabilistic attribute grammar over the design term of §14: archetype, then body plan (limb
count and arrangement through
mirrorandradial), then limb type (serial leg, wheel-leg, wheel, track unit, arm), then parameters with distributions inside their declared bounds, actuator cards drawn from the catalogue, and typed edits with probabilities as mutation operators. One generator per robot family: FACT, the evidence supports policies per morphology class, with zero-shot transfer of 70 to 85% of specialist return inside a class and 10 to 40% across topology-diverse designs (Appendix H). - Identity and addressing. A generator is content-addressed (
GeneratorId). Design i of seed s is a pure function of(GeneratorId, s, i)through the counter-based streams of §35, and comes with its derivation and its exact log-probability, so importance weights between generators are exact. - Validity. Samples that fail
design checkare rejected with their refusal codes; the rejection rate per code is part of the generator's report. DESIGN: a generator above 50% rejection is flagged as badly specified, because its stated distribution is then mostly not the distribution of what runs. - Splits. A
DesignSplitseparates training designs from held-out designs by seed and by structural stratum, and groups designs in one orbit of the observation-preserving group (§17.11) on the same side, so a mirrored copy of a training design cannot appear in the test set. A family-policy score is admitted only for designs absent from the policy's training manifest. - Support.
phylum.generate --locatereturns the derivation of a given design under a generator, or reports that it lies outside the support. A family policy applied to a design outside the support of its training generator is labeled out-of-distribution and its score is not a rung of the ladder (FROB-REF-541). - Shape awareness. The generator reports the bucket histogram and padding overhead of what it emits, and can be stratified by shape class so that a pretraining population does not fragment into small cohorts.
- Cost of use. FACT (Appendix H): pretraining at 10⁹ to 10¹⁰ steps costs about 1 to 30 GPU-hours and at 10¹² steps over a thousand. The first needs an owner-approved compute envelope (§21); the second is out of scope.
Pilot before funding: sample 1,000 designs per family on the CPU and report rejection rate, bucket histogram and padding overhead under the bucket set; falsifier, more than eight buckets needed to hold padding waste at or below 25%. Ceiling: none on speed; the generator is what makes a family policy's claim well defined.
17.11 Campaign quotienting
A campaign should not simulate the same closed-loop system twice. Three mechanisms remove duplicates, each exact and each with a stated limit. None is a source of large factors except on fixtures built to have them.
Orbit representatives under the observation-preserving subgroup S. Before a combinatorial sweep of k module
types over n slots, Phylum takes the group Γ_obs of §13 (the subgroup of the mechanical symmetry group that also
preserves every sensor pose and field of view, the payload, the task's goal orientation and the terrain's
symmetry), computes its action on the slots, and enumerates one canonical representative per orbit: the
lexicographically least labeling in the orbit. The count is known in advance from the cycle index, which is the
known answer. Worked, exact: ten leg types on the six slots of a radial hexapod under the dihedral group of
order 12 give (10⁶ + 2·10 + 2·10² + 10³ + 3·10⁴ + 3·10³)/12 = 86,185 orbits, a reduction of 11.6. If sensors or
the task leave only the sagittal mirror, the count is (10⁶ + 10³)/2 = 500,500, a reduction of 2.0. The honest
range is two to twelve, and it shrinks as sensors break symmetry. A request to quotient under a group that is not
observation-preserving is refused (FROB-REF-161).
Members of an orbit have equal outcome distributions, not equal outcomes at equal addresses: the design g·d in world w behaves as d does in world g⁻¹·w. Distributional results (scorecards, verdicts over a world distribution that the group preserves) are therefore shared across an orbit. A result tied to an address is shared only after the address is mapped through the group action, where the scene generator can do that; a comparison paired by address against a third design is re-run.
The mission-observable digest F. Two designs that lower to the same model tables, under the same task, scenes, policy, kernels, tier and profile, are the same experiment. The digest is the hash of exactly what the step plan and the task read: the lowered tables in canonical link order, the sensor poses and the task-visible fields. Results are memoized on it in the memo graph of §28. Two rules make it sound and useful:
- Structural, not proved. "Equal digests imply identical traces" is a statement about the simulator and is not dischargeable in a proof kernel. It holds by construction, because the kernels read nothing but the tables, and it is checked by replay: sampled pairs inside a class must produce identical digests (class Tested, D2).
- Exact lowering. Welded sub-assemblies are rolled up in exact rational arithmetic from the design's rational parameters and rounded once (§14), so designs that are equal in exact arithmetic lower to equal bits.
Fixture, exact: eight identical ballast slugs in 24 collinear slots give C(24, 8) = 735,471 placements, which enter rigid-body dynamics only through the count, Σx and Σx². There are 24,546 distinct (Σx, Σx²) pairs, so 24,546 classes, a reduction of 30.0. The negative control: two placements with equal Σx and different Σx² must fall in different classes. The factor is a property of this fixture. Adding a placement-dependent hot spot, a harness length or a collision shape to what the mission reads enlarges the digest and splits classes, down to singletons.
The invalidation contract S. Every memoized result records the digest of the fields it read. After an edit a result is reused exactly when that digest is unchanged, and otherwise recomputed.
- What survives. Replacing a leaf module keeps its siblings' local analyses, derivations and kernels. An edit to a field the mission does not read (a label, a visual mesh) keeps everything.
- What does not. An edit that touches a shared battery bus, a shared thermal sink, a kinematic loop across branches or any mass or geometry of a robot in contact invalidates every episode-level result of that design, because the floating body couples all of it. FACT (Appendix H): exact reuse after a dynamics-affecting edit is at most about a tenth of compute. The contract is hygiene and an evidence rule, not a speed lever.
- Positive control. A mirror swap under an element of Γ_obs must reuse every distributional result, and every per-address result after the address mapping of the orbit rule above.
- Negative controls. A shared-bus edit must invalidate. A reversed port sign must be refused. And the coupled bus: two fingers with identical local summaries on a bus limited to one unit of current have no feasible one-tick schedule and two feasible two-tick schedules, so a contract that adds peak demands prunes a feasible design. A local summary without time correlation is not a sound key.
- Calibration. DESIGN: one hit in 64, chosen by key hash, is recomputed in shadow. One mismatch stops serving that kind of node until it is explained. Counterexample on file: a dependency omitted from a key returned a stale promotion in an executed test; keys are therefore derived from declared inputs, never written by hand.
17.12 Quality-diversity maps per dispatch F
A population dispatch returns a map, not a winner. Each phylum.sweep updates a MAP-Elites grid (or a centroidal
tessellation) over behaviour descriptors such as mass, speed, energy per metre and footprint (fs-archive,
FACT), and the envelope carries the cells that changed, their elites with provisional scores, the count per
cell and the coverage of descriptor space. An agent's turn then yields many different good designs instead of
one optimum, which is how fewer serial rounds are bought.
- Merging. A cell keeps the maximum under a total order (score, then
DesignId), so merging maps is commutative, associative and idempotent, and maps from two cards or two agents combine in any order. - Bias. A population scored under one shared policy drifts toward designs that policy steers easily. The
envelope therefore reports descriptor coverage beside quality, and elites stay
provisionaluntil the rank-agreement record of §16.9 or §28 permits selection in their stratum. - Ceiling. FACT (Appendix H): published population co-design reports end-to-end compute gains of about three to four, and batched black-box search cuts rounds almost linearly in batch size up to about 128. HYPOTHESIS: campaigns on the benchmark of §39 need at most half the serial rounds of one-design-per-turn search. For parametric families this is not engine-specific; an incumbent batches the same sweep.
17.13 Paired finite-difference design gradients F
For a design with P continuous parameters θ and a smooth metric J, Phylum estimates ∂J/∂θ in one dispatch. Worlds are laid out as a stencil: for each seed s, one world at θ and two at θ ⊕ δ·eᵢ and θ ⊖ δ·eᵢ for each parameter, moved along the parameter's own retraction (§14) so that an inertia stays on its cone. Every world of a seed shares every random draw: scene, initial state, disturbances, sensor noise and randomization come from the same addresses, and only the design differs. The estimate is the mean over seeds of central differences (J₊ − J₋)/(2δ) with a confidence interval. A stencil of 20 parameters and 64 seeds is 41 × 64 = 2,624 worlds.
Limits, all enforced:
- Short horizons only. FACT (Appendix H): in contact-rich simulation a perturbation of 5 × 10⁻⁴ degree has been reported to produce significant divergence after 50 steps. Pairing buys variance reduction only while the paired worlds stay correlated. DESIGN: the default horizon cap is 50 control intervals; a longer horizon is admitted only where the measured variance ratio of paired to independent differences is at least 2 at that horizon for that class.
- The divergence caveat. Each pair records the first step at which the active contact sets of its two worlds
differ. After that step the pair is two nearly independent samples and its difference is noise of variance
about σ²/(2δ²). The estimator truncates each pair at its divergence step and labels the result
truncated-horizon, or refuses if more than half the pairs diverge before a quarter of the horizon. - Smooth metrics only. Energy, tracking error and margins are admitted. A binary outcome (a fall, a mode
switch) has zero gradient almost everywhere and a finite-difference variance that grows as 1/δ; it is refused
(
FROB-REF-540). - Determinism. Pairing requires a controlled determinism class (§35); under an uncontrolled class the verb refuses.
- Bias. Under a shared policy the gradient points toward designs that policy handles. The result is classed
Estimatedand a step taken along it is checked by a specialist or a fine-tuned policy at its end point.
Pilot before funding, about four GPU-hours on an existing engine or on batch-cpu: one quadruped, 12 design
parameters, a fixed policy; variance ratio at horizons of 25, 50, 100, 250 and 500 steps and perturbations of
1%, 3% and 10%; sign agreement over 20 replicate batches; 20 ascent steps against one generation of CMA-ES at
equal budget. Falsifier: variance ratio below 2 at every horizon of 50 or more, or sign agreement below 70%.
Ceiling: at most the ratio of evaluations, about 2P + 1 worlds per gradient against a few hundred evaluations
of a derivative-free generation, and only on the smooth, short-horizon part of the objective. ✖ KILL-FDGRAD:
the falsifier fires, or no end point is confirmed by a specialist; fallback, rank-based search (CMA-ES, racing)
with the stencil kept as a sensitivity report for A-16.
17.14 The head-to-head experiment
This experiment decides whether the table-driven cohort is the engine, and the plan commits to its outcome in advance.
| Arm | What runs |
|---|---|
| A | Phylum mixed cohorts: model tables, size buckets, slot dispatch |
| B (control) | Per-family templates: at most nine padded templates, one per family of the eighteen archetypes of §14.10 that share a maximal tree (the grouping is frozen with the arm), each a fixed maximal tree with lockable joints and per-design parameter tables (level P-A only) |
| C | An incumbent's mixed-topology batch, out of process, at its pinned version |
| D | The leading incumbent run design by design with warm caches, parallel compilation and per-world parameter batching wherever the tree is shared |
- Workload. 512 topologies with realistic contact tails (wheels with rim samples and tracks beside point
feet), drawn from the generators of §17.10 plus a frozen corpus of structural edits; K = 1, 8, 32 and 128
episodes per design; arms A and B on both machines, arms C and D on
trj. - Measures. (1) Edit to first accepted result: wall time from the release of unseen structural edits to a completed, accepted cohort. (2) World-steps/s relative to a homogeneous batch of each bucket's largest design. (3) Padding overhead, bucket count and the share of the mutation corpus each arm can express. (4) Digest invariance (GATE-PHYLUM-02). (5) Run-to-run digests of the mixed batch for arm C under each of its solver's deterministic modes that accepts a heterogeneous batch; if none does, that is recorded as the finding.
- Statistics. A fixed-n study (§24): at least ten timed repetitions per cell after warm-up, cold runs kept apart, runs interleaved, a 95% lower bound on each paired ratio, a thermal soak on the Mac, and the disclosure block of §36 on every number.
- Pre-declared outcomes. If B expresses at least 95% of the mutation corpus and wins measure (1) at K ≤ 128, the templates become the engine and the table-driven cohort becomes an admission and preview path (✖ KILL-PHYLUM). If C ties or beats A on measures (1) and (2), the plan says so in its status table, withdraws the two performance differentiators of §17.1, and keeps the cohort verbs for what they still give: replay of one world outside its batch, deterministic forks and a Metal tier. If A falls below 0.5 of the harmonic mean of its homogeneous rates on 512 designs or needs more than eight buckets, KILL-PHYLUM fires as well.
- Pilot before funding. At M1 with no GPU code: padding overhead and bucket count of the design corpus (§24) and of 1,000 generated designs under the bucket set. Then two to three engineer-days and under five GPU-hours on existing engines: 64 topologies with 64 worlds each for 1,000 steps, timing the incumbent's switch overhead design by design, a 64-DoF maximal template with locked joints against the specialized model, and an incumbent's mixed batch. Falsifiers: sequential bucketed runs within a factor of two of the best mixed wall-clock, or the maximal template above four times the specialized model.
17.15 Verbs
| Verb | Kind | First milestone | What it returns |
|---|---|---|---|
phylum.plan |
read | M2 | For a population: buckets, padding overhead per stage exponent, bytes per world and in total, the machine that fits it, expected throughput and expected edit-to-first-step. Never runs physics. |
phylum.sweep |
run | M2 | Runs a population (--designs, --worlds-per-design, --task, --tier); one envelope with a provisional scorecard per design, cohort count, padding overhead, the map delta of §17.12 and the coverage figures of §16.11. |
phylum.generate |
read | M3 | Samples designs from a generator with derivations and log-probabilities; --locate tests support; --report gives rejection rates and the bucket histogram. |
phylum.quotient |
read | M3 | Orbit representatives under Γ_obs with the orbit count, and mission-digest classes for a design set, with the fields that split them. |
phylum.reuse |
read | M3 | For an edit: which cached results survive, which are invalidated, and the field that forced each invalidation. |
phylum.gradient |
run | M4 | The paired finite-difference gradient with intervals, per-pair divergence steps and the truncated-horizon label where it applies. |
phylum.map |
read | M4 | The quality-diversity map of a campaign or one dispatch; merge of maps. |
Seven verbs, the limit for one cluster (§32). Specialization of survivors is an option of phylum.sweep, not a
verb, and stays off until its gate to fund passes.
frob phylum sweep --designs @archive:gen-14 --task stairs-0.15 --worlds-per-design 16 --tier gpu-cuda --json
# illustrative: 512 designs in 3 cohorts (buckets 18, 24, 36); padding overhead 1.2; 8,192 worlds
# one envelope: a provisional scorecard per design, 37 map cells changed, referee share 0.4%
17.16 Refusals
| Code | Name | Raised when | Ranked repairs |
|---|---|---|---|
FROB-REF-160 |
No shape class | A design exceeds the largest admitted bucket and the tree-operator route is off | Enable the route; run on batch-cpu; simplify |
FROB-REF-161 |
Group not observation-preserving | A quotient or a weight tie is requested under a group that moves a sensor, the payload or the task | Use Γ_obs; declare the invariance and let the metamorphic test check it |
FROB-REF-162 |
Reuse without a dependency digest | A cached result has no recorded dependency digest, or the shadow recompute disagreed | Recompute; quarantine the node kind |
FROB-REF-540 |
Gradient unreliable | Non-smooth metric, horizon above the admitted cap, pairs diverged early, or an uncontrolled determinism class | Shorten the horizon; change the metric; use rank-based search |
FROB-REF-541 |
Outside generator support | A family-policy score is requested for a design outside its training generator's support | Fine-tune; train a specialist; extend the generator |
FROB-REF-740 |
Population exceeds device | The population's bytes exceed every admitted placement | Fewer worlds per design; split across cards; path-table Jacobian; the M5 Ultra |
FROB-REF-741 |
Bucket fragmentation | More than eight buckets are needed to hold padding waste at or below the limit | Stratify the generator; raise the waste limit knowingly; split the campaign |
17.17 Invariants and proof obligations
| ID | Statement | How it is checked |
|---|---|---|
| INV-PHYLUM-01 | Every result of a cohort is keyed by world address and ModelId. Slot, cohort and arrival order appear in no result. |
Permute slots and co-tenants; envelopes compare equal after sorting by address |
| INV-PHYLUM-02 | A world of a mixed cohort has the same digests as the same world in a homogeneous batch of its own model. | GATE-PHYLUM-02 |
| INV-PHYLUM-03 | Quotients use Γ_obs only, and equal mission digests mean equal lowered tables. | The lidar counterexample; replay inside sampled classes |
| INV-PHYLUM-04 | A memoized result is served only when its dependency digest is unchanged. | Positive and negative controls of §17.11; shadow recompute |
| INV-PHYLUM-05 | A specialized kernel reproduces table-driven digests or is a new kernel version. | Digest comparison on the pinned corpus |
| INV-PHYLUM-06 | A generated design is a pure function of (GeneratorId, seed, index) and its log-probability is exact. |
Regeneration on both platforms; probabilities sum to one on enumerable sub-grammars |
| INV-PHYLUM-07 | The worlds of one stencil seed share every random draw. | Stream addresses compared; a planted unshared stream is caught |
| ID | Obligation | Class and route |
|---|---|---|
| PO-PHYLUM-01 | The number of canonical representatives enumerated equals the cycle-index count. | D0, exact; checked for every sweep |
| PO-PHYLUM-02 | 1/r ≤ Ω_p ≤ r⁻ᵖ for p ≥ 1. | Proved (two lines, §17.5); property-tested on random populations |
| PO-PHYLUM-03 | Map merge is commutative, associative and idempotent. | Proved from the total order; property-tested |
17.18 Known answers
| Case | Truth | Negative control |
|---|---|---|
| Hexapod sweep, ten leg types, dihedral group of order 12 | 86,185 representatives | Counting under the rotation subgroup only |
| The same with only the sagittal mirror | 500,500 representatives | Quotienting under the mechanical group when a front-only sensor is mounted |
| Ballast fixture | 24,546 classes from 735,471 placements | Two placements with equal Σx and different Σx² merged |
| Padding table | The Ω values of §17.5 from the three populations | — |
| Mixed cohort against homogeneous | Identical digests per world | A reduction tree shaped by capacity |
| Mirror swap under Γ_obs | Every distributional result reused; per-address results reused after address mapping | A shared-bus edit reused; an unmapped per-address result reused |
| Coupled bus | Zero one-tick schedules and two two-tick schedules | Pruning by summed peak demand |
| Paired gradient on a ballistic body | ∂R/∂v = 2·v·sin 2α / g for range R = v²·sin 2α / g, recovered to O(δ²) | A binary success metric admitted |
| Bouncing body near grazing | Pairs flagged at their divergence step | Divergence ignored and the full horizon averaged |
| Generator | The same (GeneratorId, seed, index) gives the same DesignId on Linux and macOS |
A generator reading the wall clock or a hash-map order |
17.19 Gates and kill gates
| Gate | Claim | Threshold (TARGET) | Earliest |
|---|---|---|---|
| GATE-PHYLUM-01 | Known answer | Orbit and ballast rows of §17.18 exact; zero false quotients in 10⁴ sampled class pairs by replay; the lidar and equal-Σx controls caught. | M3 |
| GATE-PHYLUM-02 | Known answer (determinism) | Each of 512 designs in a mixed cohort has the same digests as in a homogeneous batch, on batch-cpu and per GPU back end, at three bucket sizes and under slot dispatch with 100 random co-tenancies. |
M2 |
| GATE-PHYLUM-03 | Performance | The head-to-head of §17.14 run and published with all four arms. For the cohort to remain the engine: at least 0.70 of the harmonic mean of the homogeneous rates (§36.3 row 10) at padding waste of 25% or less on cohorts of at least 256 designs, a lower bound above 1 on the ratio against arm B for edit to first accepted result at K ≤ 128, and at most eight buckets. | M2 |
| GATE-PHYLUM-04 | Performance | Edit-to-first-step latency on the warm and cool paths inside the targets of §17.8 over 1,000 edits from the mutation corpus, cold reported separately, with the same quantity measured on the pinned incumbents. | M2 |
| GATE-PHYLUM-05 | Known answer | Invalidation contract: the positive control reuses, each negative control invalidates or refuses, and 10⁴ random edit sequences give results equal to full recomputation. | M3 |
| GATE-PHYLUM-06 | Known answer and method | Paired gradient: the ballistic row within 1%; binary metrics refused; on the pilot task a variance ratio of at least 2 at the admitted horizon and sign agreement of at least 70% over 20 replicates. | M4 |
| GATE-PHYLUM-07 | Known answer | Generator: reproducible across platforms; exact probabilities on enumerable sub-grammars; rejection rates reported; no orbit straddles a DesignSplit. |
M3 |
| GATE-PHYLUM-08 | Design comparison | Maps per dispatch: on the benchmark of §39, campaigns reach the acceptance target in at most half the serial rounds of one-design-per-turn search at equal compute, with compute reported beside rounds. | M4 |
| Kill | Trigger | Fallback |
|---|---|---|
| ✖ KILL-PHYLUM | GATE-PHYLUM-03 fails: below 0.5 of the harmonic mean of the homogeneous rates on 512 designs, more than eight buckets, or arm B covers at least 95% of the mutation corpus and is faster. This is the one decision on the mixed batch; KILL-BUCKET (§21.11) is its device-side statement and defers to it | Per-family templates become the engine; cohorts are formed by exact tree with per-design parameter tables (level P-A); the table-driven path remains for admission and preview on batch-cpu |
| ✖ KILL-FAMILYLANE | §17.6 | The general masked lane only |
| ✖ KILL-SPECIALIZE | §17.9 | The table-driven kernel only |
| ✖ KILL-FDGRAD | §17.13 | Rank-based search; the stencil as a sensitivity report |
| ✖ KILL-QUOTIENT | One false quotient survives a repair cycle, or digest classes save less than 5% of dispatches on the campaign corpus | Orbit representatives only; the digest is kept as a cache key with no claim |
17.20 Open questions
- OPEN-PHYLUM-1. The registered default bucket set and the padding-waste limit. Default: {12, 18, 24, 36, 48,
64} and 25%. Decided by the M1 pilot of §17.14. Fail-closed: the planner refuses with
FROB-REF-741. - OPEN-PHYLUM-2. Run length per model, 16 or 32 worlds. Default: 32 on CUDA, the device's reported execution width on Metal. Decided by the M2 profile.
- OPEN-PHYLUM-3. The compute envelope for the head-to-head and for pretraining populations. Owner: project lead. Default: the one-device, one-hour rule of §21 until approved, which admits the pilot and not the full experiment.
- OPEN-PHYLUM-4. Which incumbents and versions fill arms C and D. Frozen with the rival audit (§39) before
any run. Default: none, and the arms report
not run. - OPEN-PHYLUM-5. Whether contact capacity needs its own bucket axis. Default: no, one axis on degrees of freedom with capacities taken from the largest member. Decided by the overflow and padding statistics at M2.
17.21 Limb-archetype kernels
What it is. F. This subsection extends §17.9 and uses the limb port interface of §14.6. Between the
table-driven kernel, which serves every design of a shape class, and a kernel specialized to one design, which
§17.9 reserves for survivors, there is a third level: a kernel specialized to a limb archetype. An archetype
fixes a limb's joint-class sequence and depth (DESIGN examples: a serial three-revolute leg, a four-joint
wheel-leg, a seven-revolute arm, a track flipper) and leaves every value in the model table: placements, screw
components, inertias, limits, card slots. Archetype kernels are compiled once per kernel version and never per
design, so a new design assembled from registered archetypes runs on them with no compile. Owner:
frob-kernels for the kernels; frob-phylum for decomposing a model table into a base and archetype
instances. Exposure: the option --limb-kernels of phylum.sweep; phylum.plan reports the archetype
coverage of a population.
Mechanism (DESIGN).
- Registry. At most 16 archetypes, drawn from the limbs of the archetypes of §14.10. A limb is an instance when its joint-class sequence and depth match after the canonical ordering of §14. A design with any limb outside the registry stays whole on the table-driven kernel.
- Per-limb stages. Stages 1 and 6 of §16.6 run per limb instance, with loops unrolled over the archetype's fixed depth and the joint-class column masks resolved at compile time. Worlds are grouped so that one execution group holds instances of one archetype.
- The port is the only interface. On the dense route a limb with k joints returns its k × k block of the mass
matrix, the 6 × k coupling block expressed at its mechanical port, its composite inertia at the port (10
scalars) and its bias terms, and the base kernel assembles the full matrix. On the tree-operator route it
returns the
LimbPortPacketof §14.6 (27 scalars). - Contact is untouched. Stages 5, 7 and 8 run for the whole robot. Contact rows couple every limb through the base. An archetype kernel is a faster way to compute terms that the coupled solve needs, and nothing more.
- Same bits. The operation order inside an archetype kernel equals the table-driven kernel's order for the
same limb, so results are bit-identical (INV-PHYLUM-05) and a design moves between the two paths without a
new
StepPlanId.
Counterexample: a port packet is not the limb when its foot is down. A point mass m on a massless leg stands on a floor under gravity g. Treated as a free tree limb, its packet reports the body's inertia and a bias equal to the weight m·g, and the base equation closed from the packet gives a downward acceleration g. With the contact row the normal force is m·g and the acceleration is zero. Any path that closes the base equation from port packets without the contact rows is wrong by exactly g here. The second control is the packet of §14.6 with value diag(0, 1, 1, 1, 1, 1): it is accepted as a packet and refused as an invertible matrix.
Ceiling, stated first. An archetype kernel touches the stages that per-design specialization touches, and cannot beat it. FACT (Appendix H): those stages are 53 of 262 ns of an incumbent's humanoid step, so an infinitely fast recursion gives 262/209 = 1.25 and one twice as fast gives 1.11. HYPOTHESIS: the gain is 1.05 to 1.15. What this level adds over §17.9 is reach: with zero compile per design it applies inside the edit loop and to every design, where per-design kernels apply to survivors only. Two things are not claimed: a fused kernel for all topologies at three quarters of homogeneous throughput, which rested on an occupancy miscount, and a lossless limb summary under contact, which the counterexample above refutes.
Pilot before funding. A fourth arm in the pilot of §17.9, at no added calendar time: the 12-joint quadruped as a base and four instances of the serial three-revolute archetype, measured on the whole step beside the padded table-driven, unpadded table-driven and fully unrolled arms. Falsifier: the archetype arm gains less than 1.10 on the whole step, or less than half of what the fully unrolled arm gains. A paper measurement at M1 accompanies it: the share of limbs in the design corpus (§24) and in 1,000 generated designs (§17.10) that match a registry of 16.
Gate and kill gate.
| Gate | Claim | Threshold (TARGET) | Earliest |
|---|---|---|---|
| GATE-PHYLUM-09 | Performance; known answer (determinism) | Every registered archetype reproduces table-driven digests bit for bit on the pinned corpus; at most 16 archetypes cover at least 90% of the limbs of the design corpus; zero kernel compiles while 1,000 corpus edits are admitted; the 95% lower bound of the paired whole-step ratio against the table-driven kernel is at least 1.15 at 8,192 worlds; both hand controls reproduce. | M2 |
✖ KILL-LIMBKERNEL. Trigger: the pilot's falsifier holds; or coverage is below 80% with 16 archetypes; or the gate's ratio fails after one repair cycle. Fallback: the table-driven kernel for every design, per-design specialization for survivors under §17.9's own gate, and the limb port interface kept for the tree-operator route and the per-limb caches of §14.6.
In the head-to-head of §17.14 the level is reported as a variant of arm A on measures (1) and (2).