FrankenRobotics

The performance program

7,173 words16 subsections12 tables1 listingPlan version 1.1

On this page16
36.1 Position36.2 Units36.3 Targets36.4 Amdahl limits36.5 The speedup bound for family batches, and the ceiling-first rule36.6 Admission by paired lower bound; weak and strong scaling36.7 The disclosure block36.8 Method36.9 Where time goes36.10 Memory arithmetic per world and per scene36.11 The bandwidth-bound observation36.12 Camera memory arithmetic36.13 The distributed PPO normalization rule36.14 The compute envelope question36.15 End-to-end rows36.16 Known answers, gates, kill gate and open questions

36.1 Position

Engine throughput is a parity requirement. It is not the source of the ten-fold (§2, §39), and §36.4 shows by arithmetic why it cannot be. The performance program therefore has three jobs: keep the engine inside a stated tolerance of the strongest incumbent measured on the same hardware; measure in a way that cannot flatter us; and supply the phase ledger from which the ten-fold test is computed. The measurement rules are S. The numbers are TARGETs. Two-device strong scaling beyond the incumbent's own is F under KILL-SHARD.

ADR Decision Reason
ADR-PERF-01 Engine targets are ratios to incumbents measured on our own hardware; published figures give the planning values Published incumbent figures exist for the reference tasks; an absolute target without a comparator hides a gap
ADR-PERF-02 Two units, world-steps and env-steps, each written with its decimation An env-step count means nothing without the number of physics steps inside it (§36.2)
ADR-PERF-03 Every target is a tolerance or an ambition and says which A reader must know which numbers the plan depends on
ADR-PERF-04 A ratio is admitted on a paired lower bound, never on a point estimate §36.6
ADR-PERF-05 Ceiling first: no speed experiment without its computed ceiling §36.5

36.2 Units

Term Definition
World-step One physics step of length h of one world: stages 1 to 9 of the batched step (§16). Stage 10 runs on control ticks
Env-step One control tick of one world: d world-steps, then observation, reward and reset logic. A row says whether policy inference and learning are included
Decimation d Control period divided by h. An integer for every reference workload. Where a task's rational clocks (§20) make it non-integer, the mean number of world-steps per control tick is reported as a rational
World-second One second of simulated time of one world: 1/h world-steps
Device-second One second of one device's wall-clock occupancy, including compilation, admission, transfer and synchronization
Rate Completed work over wall-clock from submission to completion. Steps of worlds marked Degraded or Faulted are counted separately and are not in the numerator

Conversion: for one run, world-steps/s = d × env-steps/s. Ordering (INV-PERF-01): in env-steps/s, training is at most the full environment step, which is at most physics only. The smallest contradiction: 400 k env-steps/s of training at d = 10 is 4.0 M world-steps/s inside training, and cannot stand beside a physics-only figure of 2.0 M world-steps/s on the same card. The harness checks the ordering on every report (GATE-SPARK-19).

FACT (Appendix H): incumbents do not share a decimation. One publishes its quadruped task at a control period of 20 ms over a physics step of 4 ms (d = 5); another reports 1.20 M env-steps/s of physics as 4.8 M physics steps per second (d = 4). The reference workloads here use h = 2 ms and d = 10. Parity is therefore measured twice: at matched settings (the incumbent run at our h, d and iteration count where its solver allows) and at native settings (each stack at its own recommended step for the task), the second in env-steps/s and in time to a return threshold.

36.3 Targets

All figures are TARGETs. None is a measurement. A tolerance is a parity band: the plan's argument holds anywhere inside it, and leaving it has a named consequence (§36.16). An ambition is a better number that we intend to reach and on which nothing else in the plan depends. Where an incumbent can be run on our hardware, the tolerance is a ratio to that measurement (GATE-SPARK-21) and the absolute figure beside it is the planning value implied by the published number. Incumbent references are FACTs from Appendix H, each to be re-read from its primary source before a gate cites it.

The tolerance ratio for physics rows is 0.67, a step at most 1.5 times slower than the incumbent's. Reason: the environment step is between 0.29 and 0.87 of training wall-clock on incumbent stacks (§36.4), so such a step lengthens training by a factor between 1.15 and 1.44, which the plan's other levers must absorb and which the ten-fold test charges in full. Beyond that the engine is a liability (KILL-PARITY).

Reference workloads (DESIGN; capacities frozen at M2, OPEN-SPARK-3).

Workload Robot Degrees of freedom Bodies Contact capacity C Row capacity R Step h Newton iterations Decimation d
Q Reference quadruped, 12 actuated joints, 16 foot samples, flat and rough ground 18 13 32 160 2 ms 4, warm-started 10 (50 Hz control)
H Reference humanoid, 29 actuated joints 35 30 64 320 2 ms 4, warm-started 10 (50 Hz control)

Row capacity is four rows per contact (normal, two tangential, spin) plus one limit row and one stiction or backlash row per actuated joint, rounded up: 128 + 12 + 12 for Q and 256 + 29 + 29 for H. TARGET: fewer than one world-step in 10⁴ is marked Degraded for capacity on the standard battery (GATE-KIN-07); otherwise the capacity is raised and the rows below are measured again.

# Workload, tier and hardware Unit and decimation Tolerance (binding) Ambition Incumbent reference (FACT)
1 Q, physics only, gpu-cuda, one RTX 4090, 8,192 worlds World-steps/s at d = 10; env-steps/s is one tenth At least 0.67 of the incumbent at matched settings on trj. Until that is measured, an interim floor of 2.0 M (200 k env-steps/s), which is half the 4.0 M planning value and is not the 0.67 tolerance (2.7 M) At least 1.0 of the incumbent; planning value 4.0 M (400 k env-steps/s) mjlab's nightly benchmark on MuJoCo Warp: quadruped velocity task, RTX 5090, 4,096 worlds, 4.8 M world-steps/s of physics at d = 4
2 Q, full environment step without policy, same hardware Env-steps/s at d = 10 At least 0.75 of row 1 as measured (non-physics overhead at most 25%); floor 150 k At least 0.85 of row 1; planning value 340 k The same benchmark: 347 k env-steps/s for the full step against 1.20 M for physics, 71% overhead
3 Q, PPO training with policy forward, backward and optimizer, same hardware Env-steps/s at d = 10 At least 0.60 of row 1 as measured; floor 120 k At least 0.70 of row 1; planning value 280 k Isaac Lab on one RTX 4090, humanoid rough task, 4,096 worlds: 94 k, 88 k and 82 k env-steps/s for step, step with inference, and step with inference and training. MuJoCo Playground reports 417 k env-steps/s of quadruped training on an A100 with MJX at d = 5
4 Q, time to the return threshold of the reference flat velocity task, cold start, one RTX 4090 Seconds of wall-clock, compilation and admission included; d = 10 At most 1.25 times the incumbent reproduced on trj (GATE-CORTEX-05); planning value 513 s (1.25 × 410 s) At most the incumbent's time; planning value 410 s MuJoCo Playground: about 410 s on one RTX 4090, read by eye from a published figure (± 20 s), of which about 45 s is compilation
5 Row 4 on two RTX 4090: strong scaling Ratio of row 4's time to this time; d = 10 At least 1.25 At least 1.6 (the funding gate of §36.6) MuJoCo Playground: about 325 s on two cards against about 410 s on one, 1.26
6 Q, physics only, two RTX 4090, 8,192 worlds on each: weak scaling Ratio to row 1 as measured; d = 10 At least 1.8 At least 1.95 Independent shards; no published figure
7 H, physics only, gpu-cuda, one RTX 4090, 8,192 worlds, flat ground World-steps/s at d = 10 At least 0.67 of the incumbent at matched settings on trj; planning value 1.7 M At least 1.0; planning value 2.55 M MuJoCo Warp's nightly benchmark: 2.55 M humanoid (G1) world-steps/s at 8,192 worlds on an RTX 6000 Ada
8 H, physics only on a heightfield, same hardware World-steps/s at d = 10 At least 0.67; planning value 1.1 M At least 1.0; planning value 1.68 M The same benchmark: 1.68 M on a heightfield
9 H, time to the return threshold of a first-stage flat-ground task, two RTX 4090 Minutes of wall-clock; d = 10 At most 1.5 times the incumbent reproduced on trj; planning value 45 min At most 30 min MuJoCo Playground: under 30 minutes on two RTX 4090 for two humanoids. Isaac Lab's default rough-terrain humanoid run is about 1 h on one RTX 4090 (derived from its published rate)
10 Mixed batch of three body plans in one bucket, one RTX 4090, 8,192 worlds Mixed rate over the harmonic mean of the homogeneous rates; d = 10 At least 0.70 At least 0.86 Newton's Kamino solver, a mixed batch of three-, four- and five-legged bodies: 0.86 (86 k samples/s at 2,048 environments)
11 Q with per-world parameter variants, 8,192 worlds Ratio to identical worlds; d = 10 At least 0.95 At least 0.98 An incumbent capability (MuJoCo Warp's batched model fields); no published cost
12 Q, physics only, gpu-metal, this M5 Ultra Ratio to row 1 as measured on one RTX 4090; d = 10 At least 0.25 (BET-09) At least 0.60 No published physics benchmark on this chip was found; Genesis, which has a Metal path, is measured on this machine and reported beside ours
13 H, physics only, batch-cpu in f32, this M5 Ultra, 24 threads World-steps/s at d = 10 At least 0.5 of CPU MuJoCo on the same machine and threads; planning value 275 k At least 1.0; planning value 550 k CPU MuJoCo in f64 on this machine under load, a 35-degree-of-freedom humanoid at h = 2 ms: 24 k world-steps/s on one thread, 550 k on 24, 576 k on 36. Published: 650 k and 1.8 M for a lighter humanoid on a laptop chip and a 64-core workstation
14 Q, physics only, batch-cpu in f32, same machine and threads World-steps/s at d = 10 At least 0.5 of CPU MuJoCo measured on the same machine; until measured, row 13's floor, since a quadruped step is cheaper At least 1.0 No published quadruped CPU figure was found
15 Strict batch-cpu profile (transactional, event-located) World-steps/s at d = 10 None; the rate is reported with every strict result — —
16 Referee, humanoid, one world, one CPU core Multiple of real time; no decimation applies At least 0.2 At least 1.0 —
17 Referee sampler on a device run (§16) Share of device world-seconds shadowed; throughput lost; d = 10 The share is reported; loss at most 5% A share of at least 1% on H, which needs row 16's ambition on all 36 cores Arithmetic of §36.9: about 0.1 to 2.4% of GPU world-seconds at a referee speed of 0.2 to 1.0 times real time per core on 24 cores (HYPOTHESIS; measured at M2)
18 Height scan, 187 rays per world at 50 Hz, any device Added cost per env-step at d = 10 At most 15% At most 8% —
19 design check; control synth; interior scene generate and compile; CPU Seconds; no decimation applies Under 2 s at the 95th percentile; under 5 s; under 60 s design check after a one-parameter edit, with the memo graph: 0.2, 0.5 and 1.0 s at the 50th, 95th and 99th percentiles —

Rows 20 to 22, the end-to-end rows, are in §36.15.

Three notes on the table. The planning values of rows 1 to 3 are mutually consistent in one unit by construction: 400 k, 340 k and 280 k env-steps/s. Row 13's planning value is half of an incumbent CPU rate that was measured on a loaded machine and is therefore a lower bound on the incumbent. And every ratio in the table is a paired lower bound (§36.6), not a mean.

36.4 Amdahl limits

Each statement below is arithmetic on a FACT from Appendix H.

  1. An infinitely fast environment step. On Isaac Lab with one RTX 4090 the environment step is 87% of training wall-clock for a humanoid rough-terrain task (94 k, 88 k and 82 k env-steps/s nested), so removing it entirely speeds training by at most 1/(1 − 0.87) ≈ 7.7; for a dexterous hand 6.7; for a cart-pole 1.9.
  2. An infinitely fast physics kernel. On mjlab with MuJoCo Warp physics is about 29% of the environment step (347 k against 1.20 M env-steps/s), so a faster physics kernel alone gains at most 1.4.
  3. Inference and learning. On Isaac Lab they are about 13% of training wall-clock, so removing both gains at most 1.15.
  4. The design cycle. A cycle of 120 s with no training step, against an incumbent quadruped cycle of 360 to 420 s (the 300 s of item 6 plus an agent turn of 60 to 120 s), is 3.0 to 3.5. Against a 30-minute humanoid run it is about 15. Both hold only if a baseline controller can stand in for a trained policy, which is untested (HYPOTHESIS; §22, §39).
  5. Compute per decision. With an incumbent cost of about 410 device-seconds per trained candidate, a screening cost c_s per design, a fraction f of designs that go on to training and a training cost c_t of our own, the ratio is 410/(c_s + f·c_t). Ten-fold needs c_s + f·c_t ≤ 41 device-seconds. At c_t = 410: f ≤ 7.6% if c_s = 10, f ≤ 0.7% if c_s = 38, and it is impossible if c_s ≥ 41.
  6. Agent latency. If an agent takes 60 to 120 s per turn and the incumbent trains for about 300 s per turn (a round planning value just under the published 325 s on two RTX 4090), deciding one design per turn gains at most (300 + L)/L, between 3.5 and 6, with zero compute on our side.
  7. Deleting training. For the composite quadruped iteration of about 5.9 minutes (354 s; §2.2), with compilation inside the published training time, deleting training and compilation altogether gives 6.6.

Conclusion. On quadruped briefs no single phase reaches ten-fold: not the step, not the cycle, not compute per decision unless few designs are trained and screening is cheap. The factors the plan bets on beyond these (designs decided per agent turn, fewer turns to a qualified design, a cheaper training rung through family policies and inheritance) are defined and tested in §2, §23 and §39. This chapter's duty to them is parity and honest measurement.

36.5 The speedup bound for family batches, and the ceiling-first rule

The bound. If a fraction f of the strong baseline's matched wall time is improved by a factor r, and the method adds overhead h as a fraction of that baseline time, then

speedup ≤ 1 / (1 − f + f/r + h).

Worked values: infinite acceleration of 85% of the time gives at most 6.7; f = 0.97, r = 40 and h = 0.02 give about 13.5; with h = 0.02 no r reaches ten-fold unless f ≥ 0.92.

Applied to a family batch. A generation of 256 designs at 16 worlds and 10 s each is 20.5 M world-steps, about 10 s of simulation at 2.0 M world-steps/s. Run design by design, it also pays 256 switches (compile or upload, graph capture). What a mixed batch removes is the switching; what it adds is padding, here taken as 30% of the simulation time. If a switch costs 1 s, f = 256/266 = 0.96 and the bound is 266/13 ≈ 20. If a switch costs 0.1 s, f = 0.72 and the bound is 35.6/13 ≈ 2.7. The gain of family batching therefore depends almost entirely on the incumbent's cost of switching designs, which is unpublished (HYPOTHESIS for both values) and is measured at GATE-SPARK-21 before §17's experiment is funded.

Three rules for using the bound (DESIGN).

  1. f, r and h are measured on the strong baseline's critical path: a comparator that also caches graphs, factors and compiled kernels.
  2. A pruning ratio is never multiplied by a device speedup that was measured on work already removed. The smallest case: 100 candidates of 10 s each; pruning removes 90; the ten survivors are the hard ones and accelerate by 2, not by the 5 measured on the full set. The result is 1,000/50 = 20, not 10 × 5 = 50.
  3. Queue time for proofs and transfers counts toward decision time.

Ceiling first. Before any optimization or speed experiment is funded, its ceiling is computed from the current profile with this bound, and an experiment whose ceiling is below its goal is refused (FROB-REF-866). FACT, Appendix K: a specialized solver missed its speed gate at 1.345 times warm and 1.088 times cold, and a later measurement left a residual of 575.7 µs that the solver does not touch (parsing, assembly, checking, serialization and harness, not separately timed), so a solver of zero cost would have reached about 1.46 (842.8 ÷ 575.7). The ceiling was computable before the experiment ran.

36.6 Admission by paired lower bound; weak and strong scaling

The rule (DESIGN). A performance ratio (ours against an incumbent, after against before, two devices against one) is admitted only as the one-sided 95% lower confidence bound of a paired ratio:

  • at least ten pairs; the two arms of a pair run back to back in randomized order on the same host state, on the same workload and addresses;
  • with ℓᵢ = ln(aᵢ/bᵢ), mean ℓ̄ and standard deviation s, the bound is exp(ℓ̄ − t·s/√n), t being the 95th percentile of Student's distribution with n − 1 degrees of freedom (1.833 for n = 10);
  • the incumbent arm is its strongest warm or cached configuration, frozen after tuning on development fixtures only (§39);
  • compilation, admission, synchronization, transfer, fallback, refusal and verification are all in the denominator;
  • on this Mac a 20-minute sustained load precedes measurement; on trj the host load is recorded.

A gate threshold applies to the bound, never to the mean. A ratio stated without one is refused (FROB-REF-863).

Optimizations. An optimization lands only with its ceiling computed beforehand, with identical digests (classes D1 and D2) or an unchanged fidelity battery, and with a paired lower bound of at least 1.03 on the row it claims to improve.

Two devices. Scaling from one RTX 4090 to two is measured both ways, because they answer different questions.

  • Weak scaling holds the load per device constant (row 6). Shards are independent, so the tolerance is 1.8.
  • Strong scaling holds the job constant: the same training to the same threshold at equal task quality (row 5). FACT: MuJoCo Playground's own two-card run gains 1.26. Fixed costs explain it: about 45 s of its 410 s is compilation, the learner runs on one stream, and sample efficiency falls as the batch grows.
  • Each shard must pass the memory estimator on its own, and peer access, bandwidth and free memory are read from the fingerprint at admission.
✖ KILL-SHARD Trigger: after one optimization pass, the paired lower bound of strong scaling at equal task quality is below 1.6. Fallback: no further sharding work is funded (asynchronous collection, the fast-training profile); the two cards run as two independent job lanes, which the division of labour of §21.17 uses anyway. The tolerance of 1.25 still applies to synchronous sharding where it is used.

36.7 The disclosure block

Any Performance claim carries all of: hardware and driver; backend, compiler and kernel versions; world count; time step, substeps and control decimation; solver iterations and tolerance; contact capacity and mean active contacts per world; sensors enabled; whether the policy and learning are included; scene; determinism class; warm-up and duration; whether launch graphs were captured; median and 99th-percentile step time; peak memory. A public dispute over one simulator's headline number, later re-measured two orders of magnitude lower under corrected settings (FACT, Appendix H), is the reason this block is mandatory.

Added fields (DESIGN):

  • the unit of every rate and the decimation beside it;
  • row capacity, and the counts of worlds marked Degraded and Faulted;
  • the profile and precision profile; the fingerprint digest; the placement receipt with requested and actual backend per stage, fallback count, transfer bytes and synchronization count;
  • cold or warm, with compile and admission times and the cache state;
  • for a mixed batch: buckets, variants per bucket and the padding fraction;
  • thermal state and soak duration; host load;
  • for a ratio: the number of pairs, the lower bound, and the comparator's revision and configuration;
  • rows that did not fit in memory, kept in the table and marked;
  • energy where the platform exposes credible data, and a named gap where it does not, never a zero.

A figure without the complete block is refused as a Performance claim (FROB-REF-862).

36.8 Method

  1. Profile before optimizing. Kernel microbenchmarks precede physics tuning. The first profile at M2 decides where effort goes.
  2. Account for the whole workflow. Transfers, synchronization, compilation, resets and telemetry stay in the denominator (the rule in FrankenSim's accelerator doctrine, FACT).
  3. One lever at a time, with a behaviour proof. An optimization lands only with identical digests (classes D1 and D2) or an unchanged fidelity battery.
  4. Paired, interleaved runs on a quiet host, with a thermal soak on the Mac. trj is often heavily loaded (FACT), so its measurements record the load.
  5. Regressions are test failures against a tolerance band, on the smoke battery.
  6. Ceiling first (§36.5), and the doctrine's own screen: effort goes to a kernel only if the top three kernels hold at least 50% of end-to-end wall time and the chosen one at least 15% (FACT: thresholds of frankensim/docs/ACCELERATOR_DOCTRINE.md).
  7. Measure the incumbent first (GATE-SPARK-21, M0): on trj and on this Mac, by phase, starting with what needs no agent: cold start, a parameter edit and a topology edit to first step, physics and full step at matched and native settings, training to threshold on one and two cards. The harness is an out-of-process tool (§38); the comparator revisions are frozen by §39.
  8. Calibrate across sizes. Every row is measured at 1, 8, 64 and 256 worlds and upward to capacity, giving a throughput-against-worlds curve. Rows that do not fit in memory stay in the table, marked.
  9. Timers cover completion, never enqueue alone (INV-PERF-02). A run reports its latency distribution, env-steps/s, world-seconds per wall-second, completed episodes per second, valid steps, peak memory, and energy where available.
  10. Every estimate is an interval. sim.estimate returns a calibrated interval for time and an exact figure for memory, never a point for time; its calibration is per fingerprint and is redone when the digest changes.

36.9 Where time goes

FACT (MuJoCo Warp's event trace of a humanoid at 8,192 worlds, Appendix H): of 262 ns per world-step, the solve is 47%, constraint construction 15%, kinematics and smooth dynamics 20%, collision 4%, integration 4%. A manipulation scene at 513 ns shifts to 44% solve and 32% collision. On a full environment step of another stack physics is under a third (§36.4). HYPOTHESIS for our step, to be replaced by the M2 profile: the constraint solve 40 to 60%, collision 15 to 25% once worlds sample a scene distance field, dynamics terms 10 to 20%.

Referee coverage. A device produces T·h world-seconds per second; a referee on c cores at a real-time factor ρ per core covers c·ρ of them. For H at the tolerance of row 7, T·h = 1.7 M × 2 ms = 3,400; 24 cores at 0.2 to 1.0 cover 4.8 to 24, that is 0.14 to 0.7%. For Q at row 1's floor it is 0.12 to 0.6%, and at the ambitions they fall to between a half and two thirds of those. So the referee is a sampler by necessity: about 0.1 to 2.4% of GPU world-seconds across device rates from 1,000 world-seconds per second upward, at a referee speed of 0.2 to 1.0 times real time per core on 24 cores (HYPOTHESIS; measured at M2), and row 17 asks only that its share be reported. HYPOTHESIS: the batch-cpu tier, not the referee, is the tier that can shadow a tenth or more of humanoid worlds, since row 13's tolerance of 275 k world-steps/s is 550 world-seconds per second, 16% of 3,400.

36.10 Memory arithmetic per world and per scene

Per world (DESIGN byte model, f32).

Term Bytes Q: 18 degrees of freedom, 13 bodies, C = 32, R = 160 H: 35, 30, C = 64, R = 320
Kinematic and dynamic cache 4·(61·bodies + 17·n) 4,396 9,700
Mass matrix, Hessian and factor 12·n² 3,888 14,700
Dense constraint Jacobian 4·R·n 11,520 44,800
Row records 48·R 7,680 15,360
Contact records 96·C 3,072 6,144
State, warm starts, solver vectors, actuator state estimate about 4,000 about 6,000
Total about 35 kB about 97 kB

These sit inside the ranges of the envelope study, 20 to 53 kB and 56 to 180 kB, and beside MuJoCo Warp's measured 88.4 kB per humanoid world at 8,192 worlds (FACT, Appendix H). Consequences, all arithmetic:

  • 8,192 worlds of Q are 0.28 GB and 8,192 of H are 0.79 GB: 1 to 3% of a 24 GB card.
  • PPO rollout storage is 4·T·(observations + privileged observations + actions + 4) bytes per world. At a horizon of 24 steps with 48 observations, a 187-ray height scan and 12 actions it is 24 kB, comparable to the physics state.
  • One RTX 4090 with 16 to 19 GB usable (assumption) holds about 0.46 to 0.55 M worlds of Q or 0.17 to 0.20 M of H before rollout storage. Memory never binds the world count; throughput and iteration latency do.
  • The optional hard-contact law (§18) forms an R × R operator: 102 kB per world of Q and 410 kB per world of H. It is budgeted separately and by the largest island.

The Mac's limits (FACT, measured on this machine's Metal device, re-read by the fingerprint). The working-set limit is about 222.7 GiB and the largest single buffer about 167.0 GiB; ft-kernel-metal exposes both (recommended_working_set_bytes, max_buffer_length) and the bytes currently allocated. Consequences (DESIGN): the arena splits its slabs so that none exceeds the largest buffer, which matters because the argument limit pushes toward few, large buffers; the device budget is the working-set limit less a host reserve, by default 85% of it (about 189 GiB, or 203 GB; the admission cap of §19.15), since on unified memory the CPU's allocations compete for the same pool; by the byte model that budget holds about 5.8 M worlds of Q or 2.1 M of H.

Per scene. Dense storage of a 16-bit field is 2·V/s³ bytes for volume V and spacing s. A narrow band keeps a fraction of about (A/V)·t·s for surface area A and a band of t voxels (8 to 20 with brick rounding).

Scene (volume) 1 cm dense 1 cm band 2 cm dense 2 cm band 4 cm dense
Home (300 m³) 0.6 GB 0.16 to 0.4 GB 75 MB 40 to 75 MB 9 MB
Warehouse (19,200 m³) 38 GB 6 to 14 GB 4.8 GB 1.4 to 3.5 GB 0.6 GB
Six-storey building (57,600 m³) 115 GB 10 to 26 GB 14.4 GB 2.6 to 6.5 GB 1.8 GB

Dense values are arithmetic; band values rest on assumed surface areas and are HYPOTHESES until archetypes are compiled (§19). The band columns exclude the per-brick labels, which add 12.5%; §19.15 prints the 2 cm band with labels. Consequences:

  • One scene per world is impossible: 8,192 homes at 75 MB are 614 GB. Scenes are shared: a device holds K scene variants over one brick pool and every world indexes one (§19, §21.11).
  • With 12 to 16 GB of a card for scenes, a card holds 160 to 210 homes at 2 cm, 20 to 27 at 1 cm, two or three warehouses at 2 cm and none at 1 cm. With 203 GB, this Mac holds about 2,700 homes at 2 cm, 340 at 1 cm, 42 warehouses at 2 cm, five at 1 cm, and fourteen buildings at 2 cm.
  • Hence the role of §21.17: 1 cm warehouses and whole buildings are Mac-only, and the Mac is the world bank.

Estimation. sim.estimate computes memory from the byte model and the plan's live intervals (the largest set of slabs alive at once), and is then checked against measured high-water marks (GATE-SPARK-22). It refuses a job that does not fit before anything is allocated (FROB-REF-602). Its time estimate is a conformal interval fitted to recorded runs per fingerprint.

36.11 The bandwidth-bound observation

Observation (a derived estimate; a HYPOTHESIS until GATE-SPARK-22 measures bytes touched per world-step): at these state sizes an RTX 4090's step is bound by memory bandwidth, not by arithmetic.

  • FACT (Appendix H): an RTX 4090 has about 1,008 GB/s of memory bandwidth and 72 MB of L2 cache; this M5 Ultra has 1.2 TB/s, shared with its CPU.
  • A batch outgrows the L2 cache above about 2,100 worlds of Q or 750 of H, so at the batch sizes of §36.3 every kernel streams from device memory.
  • If a world-step touches B bytes, the ceiling is η·(bandwidth)/B with an efficiency η of 0.5 to 0.8 (assumption). With an assumed traffic of 100 to 500 kB for Q and 0.2 to 1.5 MB for H, the ceilings are 1 to 8 M and 0.3 to 4 M world-steps/s. Arithmetic is not the bound: 1 to 3·10⁵ operations per step of Q at 5 to 20% of the card's peak rate would allow 14 to 165 M.
  • The estimate is contradicted in its middle: its central humanoid ceiling, 0.7 to 1.1 M, is below the 2.55 M that MuJoCo Warp measures on the same class of card. Either a step touches fewer bytes than assumed or η is higher. That is why the quantity is measured and not assumed.

Consequences (DESIGN):

  1. Bytes touched per world-step is a first-class figure, measured per launch and reported in the disclosure block. The ambition of row 1, 4.0 M world-steps/s, allows at most 126 to 202 kB per world-step of Q.
  2. The dense Jacobian is the first suspect. At 11.5 kB, three passes in each of four Newton iterations are 138 kB, the whole budget. Profile-gated remedies, each with a behaviour proof: a fused assemble, factor and solve chain; storing a contact row only over the columns it touches (the base and one leg); a line search that does not re-read the Jacobian. They are tested at sizes 18, 30 and 60.
  3. No order of magnitude is available per step. MuJoCo Warp already measures 2.55 M humanoid world-steps/s on this class of card.
  4. On Metal the bound is elsewhere. BET-09's threshold, a quarter of 2.0 M world-steps/s of Q at an assumed 240 kB per step, needs 120 GB/s, a tenth of the chip's bandwidth. Command-buffer overhead and arithmetic rate are what the fingerprint must measure.

36.12 Camera memory arithmetic

A 640 × 480 frame with 4 bytes of colour and 4 bytes of depth per pixel is 2,457,600 bytes. Two such cameras on 256 worlds are 1.17 GiB per frame set; on 8,192 worlds, 37.5 GiB, more than a card holds. With 16 GB usable a card holds at most about 3,200 two-camera worlds before any physics state. A 64 × 64 depth image in f32 is 16,384 bytes: two per world on 8,192 worlds are 0.27 GB. Exposure-time and rolling-shutter bins multiply the ray work, not the stored frame.

Rules (DESIGN): frame buffers are slabs of the arena like any other and enter sim.estimate; a camera's resolution and cadence are part of the task identity, because changing the cadence changes the control problem, so neither is a performance knob; camera-heavy evaluation belongs on the Mac (§21.17).

36.13 The distributed PPO normalization rule

For synchronous training on two shards (DESIGN):

  1. Behaviour weights and normalizers are frozen during collection.
  2. Each shard reduces its loss-gradient sums and its count of valid samples. The host adds the sums in device order and divides by the global valid count.
  3. An empty shard contributes zero and still participates. If the global count is zero the update is skipped.
  4. Minibatch order is fixed by address-keyed streams whenever equivalence between shard counts is being tested.
  5. Weights are published only after a complete update. An asynchronous variant needs a declared bound on version lag and runs only under the fast-training profile (§35.2).

Known answer: one sample with gradient 1 and three with gradient 0 give 0.25, on one shard and on two, however the four are split. The shortcut that fails: averaging the shard means. Split as {1} and {0, 0, 0} it gives (1 + 0)/2 = 0.5; with an empty shard it divides by zero or halves the answer.

36.14 The compute envelope question

FACT: FrankenSim's machine rule caps any run at 16 GiB of memory, 8 cores or one hour of wall time without written approval, and builds and tests count (frankensim/AGENTS.md, rule 1.1). The fail-closed default (OPEN-SPARK-2) is that FrankenRobotics inherits that rule for CPU work and limits a GPU job to one device and one hour until the owner approves an envelope.

What the default cannot run, by arithmetic:

  • One device-hour at 50 to 500 k env-steps/s is 0.18 to 1.8·10⁹ env-steps.
  • The performance gates themselves: row 13 uses 24 threads, rows 5 and 6 use two devices, and the world bank holds far more than 16 GiB.
  • The rank-agreement experiment between evaluation rungs (§28, §39) needs about forty per-design trainings: about 45 device-hours, or, where specialists are reused, 5 to 7.5 device-hours.
  • Pretraining of a family policy at published sizes (FACT, Appendix H): 1.6·10⁹ steps are 0.9 to 9 device-hours; 5·10⁹ are 2.8 to 28; 2·10¹² are 1,100 to 11,000, which is 23 to 230 days on two cards and is not planned in any envelope.

A proposal for the owner (OPEN; nothing above E0 runs until it is ratified or amended):

Class Scope Proposed limits
E0 Default 16 GiB, 8 cores, 1 hour, one device
E1 A named gate measurement Both machines, all cores, memory up to the device budget of §36.10, 2 hours per run, 12 device-hours per gate
E2 A design campaign 48 device-hours
E3 Pretraining of one family policy 500 device-hours, approved per run with its ceiling and kill rule written first

sim.estimate prints the class a job needs, and a job above the class in force is refused before it starts (FROB-REF-761).

36.15 End-to-end rows

These rows are defined and gated in §39. They are listed here because the performance program measures their engine-side inputs and owns none of their thresholds except where stated.

# Quantity Definition What this chapter measures TARGET Gated in
20 Edit to first step Wall-clock from acceptance of a typed design edit to completion of the first device world-step of the edited design, at the 50th and 95th percentiles Compilation or cache hit, variant admission, graph capture, first chunk Tolerance at the 95th percentile, with the ambition after it. (a) A parameter edit inside a shape class: 0.5 s; 0.1 s; no compile and no capture. (b) A topology edit inside an existing bucket: 2 s; 0.5 s. (c) An edit that needs a new shape class: 5 s with a warm cache; the cold time is reported, not targeted §39; (a) and (b) also at GATE-SPARK-10
21 Designs decided per hour §39 Distinct designs screened per device-second, derived from rows 1 and 10: at 16 worlds and 10 s per design (80,000 world-steps), 17 at the tolerances (2.0 M × 0.70) and 43 at the ambitions (4.0 M × 0.86) None of its own; a derived figure §39
22 Time to a qualified design §39: restricted mean time to an independently accepted answer The phase ledger: compilation, admission, screening, training, evaluation, promotion, qualification and waiting, per device and per agent turn None here §39

FACT for row 20 (Appendix H): no incumbent publishes this latency. An incumbent model edit needs a host compile, a model upload and a new graph capture; MuJoCo Warp's cold kernel compilation takes about 96 s on an RTX 6000 Ada; an idle-host compile of one kernel in the suite's renderer takes about 5 s. The incumbent's figure is measured at GATE-SPARK-21.

36.16 Known answers, gates, kill gate and open questions

Known answer Expected Negative control that must be caught
A report with 400 k env-steps/s of training at d = 10 beside 2.0 M world-steps/s of physics Rejected by the ordering check The same report with the decimation omitted must also be rejected
Ten pairs, each with ratio exactly 1.5 Lower bound 1.5 An A/A campaign must not clear 1.02
Padding overhead for degrees of freedom 18 and 35 mixed evenly 1.32, 1.58, 1.76 for p = 1, 2, 3 —
Family-batch bound with f = 0.97, r = 40, h = 0.02 13.5 A pruning ratio multiplied by a full-set speedup must be flagged
Gradient of one sample at 1 and three at 0 0.25 on one and two shards and with an empty shard The mean of shard means, 0.5
Byte model for Q and H 34.6 kB and 96.7 kB before rounding A capacity change must change the estimate by 4·n + 48 bytes per row
Camera frame set, 2 cameras, 256 worlds, 640 × 480 1,258,291,200 bytes —
Gate Claim Threshold (TARGET) Earliest
GATE-SPARK-19 Known answer The harness rejects a report that violates the unit ordering or omits a decimation; the paired-bound known answer holds; of 100 A/A campaigns at most 9 clear a threshold of 1.02; a figure without a complete disclosure block is refused M2
GATE-SPARK-20 Performance Weak scaling at least 1.8 and strong scaling at least 1.25 at equal task quality, both as paired lower bounds; the gradient known answer of §36.13 holds and its control is caught M2
GATE-SPARK-21 Parity baseline Incumbents are measured on trj and on this Mac under the disclosure block: physics and full step for Q and H at matched and native settings; time to threshold on one and two cards; cold start; edit to first step for a parameter edit and a topology edit; CPU MuJoCo on this Mac at 1, 24 and 36 threads; an incumbent with a Metal path on this Mac. Each published figure of §36.3 is reproduced within 20% or the difference is explained in writing M0
GATE-SPARK-22 Known answer sim.estimate memory is within 5% of measured high-water marks at 1, 8, 64 and 256 worlds and at capacity for Q and H on each device; a job that does not fit is refused before allocation; time estimates fall inside their intervals on at least 90% of held-out runs; bytes touched per world-step are measured per launch M2

GATE-SPARK-03 and GATE-SPARK-04 (§21.24) are the throughput gates against the tolerances of §36.3; GATE-CORTEX-05 (§23) is restated against row 4.

✖ KILL-PARITY Trigger: at M2 exit, after one optimization pass guided by the profile, the paired lower bound of a physics row (1, 7 or 8) is below half its tolerance, that is, below 0.33 of the incumbent on the same card. Fallback: feature work on the device tier stops and effort goes to the top three profile entries until the row is inside tolerance; the arithmetic of §2 and §39 is redone with measured rates before the partial ten-fold gate at M3; and whether to ship the layer over an incumbent engine is decided at that gate under its own rules, not here. A measured ratio between 0.33 and 0.67 is a recorded defect: one further optimization pass, and a disclosure on every Performance claim until it closes.
  • OPEN-SPARK-2. The compute envelope (§36.14). Owner: the project owner. Default: E0. Blocks GATE-SPARK-03, -04, -20 and -21 as specified.
  • OPEN-PERF-1. The tolerance ratio of 0.67 and the band below it. Default: as stated. Blocks GATE-SPARK-03.
  • OPEN-PERF-2. Which incumbent revisions and configurations are the comparators. Default: the revisions in the source register, frozen by §39 before any paired run. Blocks GATE-SPARK-21.
  • OPEN-PERF-3. A credible energy source on each machine. Default: a named gap in every disclosure block. Blocks nothing.
  • OPEN-PERF-4. The default physics step for the reference workloads is OPEN-KIN-1's to decide; if it moves from 2 ms, every row of §36.3 is restated with its new decimation before any gate cites it.