The ten-fold thesis
4,272 words7 subsections3 tablesPlan version 1.1
On this page7
The owner's ambition for this project is comparative: an order of magnitude better than every existing system at getting from a robot-design question to an answer that can be trusted. This chapter states what that means as a measurement, where the time in today's loops goes, which ceilings follow from that, which mechanisms the program bets on, what will not deliver the result, and what range the evidence supports. The protocol is in §39. Nothing here is a result.
2.1 The metric
Time to an independently accepted answer. DESIGN: the unit of success is an answer, not a step and not a score. The reason is that every published bound on step speed, and every pilot behind this version, says the same thing: the time between a question and a trusted answer is dominated by what happens around the step (§2.2).
- An answer is what the mission asks for. It is one of: a design and controller that pass a frozen acceptance
battery; a family answer with inner and outer design sets and witnesses (§25); a correct withdrawal or confirmation
of earlier verdicts after an amendment (§28); or a truthful
unknownwith the required witness, where the question is whether the evidence identifies the choice at all. - Independently accepted means that a checker which is not the proposing system accepted it: an acceptance battery
on worlds and in an evaluator that neither arm trained in, the offline auditor
frob-verifyfor every submitted certificate, and a neutral adjudicator who releases receipts only after both arms close (§39). - The clock T starts when the agent or engineer receives the mission and the permitted inputs. It stops when the checker accepts the first correct answer with all requested scope and precision. Unsuccessful runs, training, tuning, compilation, device transfer, waiting, investigation of unknowns, certificate checking, repair and review are all inside it.
- The restricted mean time at a common deadline H is RMT = E[min(T, H)], taken over the sealed missions of a stratum. A mission that is unresolved at H costs H.
- The ratio for a stratum is R = E[min(T_rival, H)] ÷ E[min(T_frob, H)], where the rival is the strongest composed rival: leading simulators, an optimizer, analytic bounds, a verifier and the same AI agent, assembled and tuned by a competent baseline engineer on development fixtures and then frozen (§39).
Four properties of the metric are fixed here because they shape the plan.
- Missed deadlines cap the ratio. If the candidate misses a fraction q of deadlines, then R ≤ 1/q. With 12% unfinished the ratio cannot exceed 25/3 ≈ 8.33. Coverage is therefore a first-order design goal, and a refusal that arrives late is as costly as a failure.
- Correctness is a gate, not a term. One false certificate disqualifies an arm from the scoped claim. A dangerously wrong accepted answer fails the mission. The time score is never improved by lowering the bar for what counts as accepted.
- Cold and warm runs are separate results. A cold run includes authoring the environment and the design domain, for both arms. A warm campaign runs a declared sequence of amendments with identical reusable inputs and cache budgets, keeps failed amendments in the sequence, and reports the crossover reuse count: the number of amendments after which the candidate's one-time authoring has paid for itself. A fair rival also caches graphs, factors and compiled kernels.
- The clock is not the only column. GPU-hours, CPU-hours, energy, agent tokens, human minutes and money are reported beside wall time under identical resource caps. Two derived units are reported for the design loop: designs decided per hour, and compute per established decision at a matched error rate.
The decision rule that turns measured ratios into the word "ten-fold", with its power analysis, its censoring rule and its multiplicity control, is in §39. One finding binds it from here: a draft rule that required a point estimate of at least 10 on every brief confirmed a true ratio of 10 in at most 18% of simulated campaigns (FACT, Appendix K). The rule of §39 is therefore specified on a pooled ratio, and its power against a true ratio of 10 is computed before the sealed suite is opened. The sealed-suite result is a finite-suite result: it describes the missions of that suite. A claim about the population of missions needs at least 32 independent mission families, and the suite size, the unit of independence and the estimator are fixed in §39 (§39.7, §39.20). A measured ratio is always reported with its interval, whatever it is.
2.2 The Amdahl ledger: where the time goes today
The table composes published figures for the phases of an agent-driven design loop on incumbent stacks. Every row is a FACT about its source (Appendix H) unless marked. It is not a measurement of a loop: no end-to-end phase ledger of a design loop on the late-2026 software stack was found, so the denominator of any ratio must be measured by this program at M0 and M1 before a mechanism is funded against it (§39).
| Phase | Published figure | Note |
|---|---|---|
| Propose and diagnose (the agent's turn) | One published agent co-design loop reports about 1 h per round of 16 candidates on eight GPUs, "dominated by RL evaluation". Its estimate of 4 minutes of language-model time per round appears in one version of the paper only and is modelled, not measured. | Agent latency per serial turn is taken as 60–120 s in the bounds below (HYPOTHESIS; the M0 ledger measures it). It is common to both arms. |
| Realize the design | Text to mesh in 1–2 minutes and mesh to robot description in about 30 s in one published pipeline. | A parametric edit is assumed to take 0.5 minute in an incumbent (HYPOTHESIS). |
| Compile and first step | One stack reports that just-in-time compilation "can be slow (1-3 minutes)"; a second compiles kernels at first launch, "can take a minute", and caches them on disk; a third starts in 4.4–6.3 s with cached shaders and 263 s without, and loads a warehouse scene in 27 s. | No latency from a topology edit to the first GPU step is published. The compile region of the quadruped run below is about 45 s and is inside its 410 s. |
| Train, quadruped | About 410 s on one RTX 4090 and about 325 s on two for a flat-ground joystick policy; under 4 minutes on flat ground and 20 minutes on rough terrain on one GPU in an earlier result. | The dominant phase for quadruped briefs: about 85% of a composite iteration of 5.9 minutes. Deleting training and compilation outright would give 6.6×. The composite joins figures from different stacks and is not a measurement. |
| Train, humanoid | Under 15 minutes for one humanoid and under 30 minutes for two others on two RTX 4090 (first-stage flat-ground training); about 1 h for a default rough-terrain run on one RTX 4090 (derived). | The dominant phase for humanoid briefs: over 90% of a composite iteration of about 31 minutes. |
| Environment step as a share of training | 87% for a rough-terrain humanoid, 85% for a dexterous hand and 46% for a cart-pole on one stack; on another, physics is about 29% of the full environment step. | Sets the bound on what a faster step can do (§2.3). |
| Evaluate | 128 rollouts of 1,000 steps per design in one campaign: seconds at 10⁵ steps/s (derived). | Not a bottleneck in incumbents. |
| Confirm | Five retrainings of the winner in two published campaigns. | Multiplies the training phase for finalists. |
| Trainings per campaign | 80, 80, 125 and 1,600 trained candidates in four published campaigns; 100 new robots per generation for 20 to 50 generations in a fifth. | 7.5 to 30 GPU-minutes per trained candidate. |
| Compute per campaign | About 40 GPU-hours per five-round run on eight GPUs; about 200 GPU-hours per seed on eight A100. | — |
| Candidates that fail trivially | Under 2% of morphology proposals fail validation before training in one loop; 15.8% of trained candidates are degenerate after a full training run with a strong language model, and 35–49% with smaller ones. | If 16% is the ceiling, pruning trivial failures alone cuts trainings by at most 1.19×. |
| Human or agent diagnosis | Not published. | Measured in the phase ledger. |
2.3 The ceilings
Each ceiling is arithmetic on the FACT rows above. Each is the reason one tempting strategy is not the plan.
- Engine speed alone. At a fixed sample count, an infinitely fast environment step speeds training by at most 1/(1 − s), where s is the step's share: about 7.7× (1/(1 − 0.87)) for the rough-terrain humanoid, 6.7× for the hand and 1.9× for the cart-pole on one incumbent stack, and about 1.4× from physics alone on another. An incumbent already measures about 2.55 M humanoid world-steps/s. There is no order of magnitude left in the step, and a faster simulator alone cannot deliver ten-fold end to end.
- The design cycle alone. A design-check-simulate-explain-edit cycle of 120 s with no training in it (NS-06) is 3.0–3.5× faster than an incumbent quadruped cycle of 360–420 s (300 s of training plus an agent turn of 60–120 s), and about 15× faster than a 30-minute humanoid run. The comparison is valid only if a controller that needs no per-design training can stand in for a trained policy when designs are ranked. That is untested (BET-17).
- One design per agent turn. With the same agent latency t in both arms and about 300 s of incumbent training (a round planning value just under the published 325 s on two RTX 4090), deciding one design per turn gives at most (300 + t)/t, which is 3.5–6× for t between 60 and 120 s, even if FrankenRobotics spent no compute at all. Against the one-card figure of 410 s the same cap is 4.4–7.8×. It is below ten either way.
- Compute per decision. Let C be the incumbent's cost per trained candidate, c the cost of the cheap rung per design, c_T the cost of the training rung, and f the fraction of simulated designs that are trained. The compute ratio is C ÷ (c + c_T·f). With C = 410 GPU-s (one RTX 4090), an illustrative c of 38 GPU-s and c_T = 410 GPU-s (training at parity), the ceiling at f = 0 is about 10.8×, a ratio of ten needs f ≤ 0.7%, and no f reaches ten if the cheap rung costs more than 41 GPU-s. A cheaper cheap rung, many designs per rung, or a cheaper training rung is required; pruning alone is not enough.
- The general bound. If a fraction f of the strong baseline's matched wall time is improved by a factor r and the method adds overhead h, the speedup is at most 1/(1 − f + f/r + h). Infinite acceleration of 85% of the time cannot reach ten. With f = 0.97, r = 40 and h = 0.02 the ceiling is about 13.5×. The three terms are measured on the rival's critical path, and a pruning ratio is never multiplied by a GPU speedup measured on work the pruning already removed.
- Unremovable cost. If the cost that no mechanism removes (authoring the mission, checking certificates, serializing, reviewing) already exceeds one tenth of the strongest baseline's time, ten-fold requires changing another stage or changing the claim. In one pilot a residual of 575.7 µs that the solver does not touch (parsing, assembly, checking, serialization and harness, not separately timed) capped the measurable ratio at about 1.46 (842.8 ÷ 575.7; FACT, Appendix K).
- Completion. R ≤ 1/q (§2.1). An arm that leaves one mission in eight unresolved cannot show ten-fold however fast it is on the rest.
Three smaller ceilings close other doors: per-design specialization of the recursions is capped near 1.2× on an incumbent's step trace (H-04); whether structure-exploiting constraint assembly and solve give more than parity is open and is measured at M2 (H-04b, §16); exact reuse of rollouts after a mass or geometry edit is at most about 10% of compute (H-09); and stopping rules give 2–5× when candidates are well separated and about 1× when they are not (H-10).
2.4 The levers
No single mechanism delivers ten-fold, and no conjunction has been shown to (§0.8). The candidate thesis is a conjunction of eight levers, each of which attacks a different row of the ledger. Seven of the eight are methods a composed rival can also host; §2.6 says what that implies. The "surviving number" column gives the range the evidence supports; every entry is a HYPOTHESIS about this system, and published figures behind it are FACTs in Appendix H. Gates are named by the section that owns them; their identifiers and thresholds are in §40.
| # | Lever | Mechanism and section | Surviving number | Tier | Gate |
|---|---|---|---|---|---|
| 1 | Decide many designs per turn | A design family per dispatch: per-world model variants inside padded capacities and size buckets, many different robots in one batch (Phylum, §17); population verbs and slot-level dispatch so one agent turn returns a ranked family and not one score (§28, §32). | One incumbent solver's mixed batch runs at 0.86 of the harmonic mean of its homogeneous rates; that solver is not deterministic and is slower than its framework's reduced-coordinate solvers. Published population methods report compute gains of about 3–4× (H-03, H-23). Removes the (300 + t)/t cap of §2.3. | S for parameter variants within a topology; F for mixed topologies | The head-to-head of §17 with a per-family-template control arm; two numbers against the incumbent mixed batch (BET-18) |
| 2 | Train few | Family policies per morphology class, conditioned on the design; inheritance along lineage edges; both admitted only where measured rank agreement with per-design specialists is high enough (Cortex, §23). | Zero-shot at 70–85% of specialist return within a class on flat or mildly varied ground; fine-tuning from a class prior at about 2–25% of a from-scratch run stopped at matched return; the one published parent-to-child figure is about 50%; single-limb topology edits 33–100% (H-02, H-21). Nothing is published for interior terrain within a class. Rank agreement is unmeasured; the nearest published figure, 0.40, is for an easier comparison (H-22a). | F | The rank-agreement gate of §23 and §28 (BET-17), then BET-15 and BET-16. The decisive early experiment |
| 3 | Spend where it matters | A ladder of rungs from static check to referee; allocation by measured rank agreement; anytime-valid stopping and elimination; paired tier correction that reports a referee-level estimate from fast-tier episodes (Atelier, §28; Kinesis, §16; Arena, §24). | 2–5× from stopping and elimination when candidates are separated, about 1× otherwise (H-10). A paired correction gains 1/(1 − ρ²) in the limit of free GPU samples: 2× at a tier correlation of 0.7, 3× at 0.82 and 10× at 0.95; the correlation is unmeasured and may be near zero for fall events. | S for stopping; F for ladder allocation | The cascade gates of §28; the fidelity gates of §16 |
| 4 | Find failures instead of waiting for them | Adversarial search inside legality bounds with a solvability witness; reduction to minimal counterexample worlds; bounded-weight rare-event estimation, with exploration kept apart from certification (Nemesis, §26). | Published measured gains in finding failures are 2–20×; 10–100× at failure probabilities of 10⁻⁴–10⁻⁵ is a TARGET. The gain is in finding failures and in refuting a failure-rate target; certifying absence is never cheaper than crude sampling (H-07). | S for bounded-weight estimation; F for minimal counterexample worlds | The planted-trap gates of §26 (BET-22, BET-32) |
| 5 | Answer over families instead of points | Mission queries with inner and outer design sets, witnesses and impossibility witnesses; per-constraint region certificates; observation-alias and clearance impossibility cuts that stop a search before it starts (Augur, §25; Lemma, §12; Habitat, §19). | Certificates per constraint, with bisection confined to a split set of about six parameters and the rest enclosed at full range (H-06); refinement with up to 17 design parameters in 4–10 minutes (H-17). On small plants, standard exact methods tie a specialized solver, so the gain is in scope and reuse, not in solve speed. | F | The family strata of §39; the certificate gates of §12 and §25 (BET-19, BET-20) |
| 6 | Do not recompute or re-decide | A content-addressed memo graph for analyses, step plans, kernels and scene compiles; dependency-directed revalidation that says which verdicts survive an edit; campaign quotienting of designs that are equivalent under symmetry and mission observables (Atelier, §28; Skeleton, §14). | Exact rollout reuse is at most about 10% of compute (H-09). Training avoided by inheritance is lever 2 and is counted there, once. Deduplication by symmetry gives 2× with mirror symmetry alone and more on special fixtures. A correctly keyed dependency graph ties any more elaborate reuse rule. | S for the memo graph and revalidation; F for quotienting | The shadow-recompute and missed-withdrawal gates of §28 (BET-31, BET-33) |
| 7 | Explain by intervention | Autopsy: fork a snapshot of a failed episode under typed interventions on identical random streams, with a placebo fork, and report which change flips the outcome and how stable that is; repairs come back as typed edits (§27). | No published figure for agent repair turns. TARGET: at least 3× fewer repair turns than an agent given logs, on planted causes. | S | The planted-cause gate of §27 (BET-23) |
| 8 | Short setup | Kernels derived at build time and cached as binaries; topology as data, so a new design recompiles nothing; an arena lease so a job starts on warm device state; model variants without recompilation (Spark, §21; Phylum, §17). | Incumbent compile and first-step times of about 45 s to 3 minutes per cold start, cached thereafter. Worth most where the rival recompiles per structural edit. | S | The edit-to-first-GPU-step measurement of §17 and §21 |
The levers multiply only where they act on different time. Lever 1 and lever 2 act on the two largest rows of the ledger and carry most of the thesis. Levers 3, 4 and 7 reduce the number of episodes and of agent turns. Lever 5 changes what an answer is. Levers 6 and 8 are hygiene that keeps the others from being eaten by overhead.
2.5 What will not deliver ten-fold
Stated plainly, so that no work package is funded on a hope the evidence has already closed.
- A faster step. Engine throughput is a parity requirement (§36). The plan must match incumbents closely enough that the levers above are not cancelled; it does not look for the result there. One engine-internal throughput question is open: structure-exploiting constraint assembly and solve (§16). Its ceiling on an incumbent's trace is 2.6×. It is measured at M2 and is not counted in the thesis until then.
- Per-design kernel specialization. Specialization of the recursions is capped near 1.2×. It is an optimization for survivors, and only if our own profile shows the recursion share justifies it.
- Half precision or tensor cores for contact. Refuted (H-15). Not built.
- Sampling control as a stand-in evaluator. Even the most sample-efficient published controller costs about 4 × 10³ times a policy rollout per simulated second, so a battery of 256 twenty-second episodes takes about 86 minutes of a device against about 1.3 s for a policy. It is a screening rung, a fallback and a source of demonstrations. Its episodes are acceptance evidence only for itself, where it is the delivered controller (§22.8, H-01).
- One universal policy. Out of reach at an affordable pretraining budget (H-02).
- Static checks alone. Pruning trivial failures caps near 1.19× on trainings. Their value is feedback in seconds and typed repairs, which is lever 1 and lever 7, not pruning.
- Reusing rollouts across edits. Exact reuse is at most about 10% (H-09).
- A faster solver for family queries. In the pilot, a specialized solver measured 1.345× warm and 1.088× cold against a composed one, with a ceiling of about 1.46× set by a residual the solver does not touch (FACT, Appendix K).
- Certificates, full-room multiphysics and generative structure. Certificates change what can be trusted, coupled multiphysics changes which failures are visible, and structure optimization improves the built object. None shortens the time to a decision (H-12, H-16), and none is counted in the thesis.
- Any published ten-fold. None was found for time to a qualified legged-robot design.
2.6 The expected range, by mission class
HYPOTHESIS, to be replaced by the partial gate at M3 and the tournament at M7. The ranges are what the ceilings of §2.3 and the surviving numbers of §2.4 allow. They are stated against the incumbent as it is commonly run, with per-design training and a standard outer loop. The metric of §2.1 is against the strongest composed rival, which may host every portable lever; against that rival the expected ratio on fixed-topology CUDA missions is near one unless an engine-internal property binds, and no range is offered. §39 reports both, with the recovery ratio between them. They are not forecasts with probabilities.
| Mission class | Incumbent denominator (FACT) | Binding ceiling | Expected ratio against per-design training | What must be true |
|---|---|---|---|---|
| Quadruped locomotion briefs with parametric and single-limb edits | Training in about 410 s on one RTX 4090; a cycle of about 360–420 s | One design per turn caps at 3.5–7.8×; compute per decision caps near 10.8× at f = 0 | 3 to 10 | Many designs per turn; under about 1% of simulated designs trained; rank agreement above threshold for the class |
| Humanoid locomotion briefs | Training in under 30 minutes on two RTX 4090; about 1 h for a default rough-terrain run | Phase arithmetic does not bind, since training is over 90% of the iteration; the cycle arithmetic of §2.3 gives 15–30×, and the completion ceiling 1/q applies | 5 to 20 | A baseline or family policy stands in for training when designs are ranked (BET-06, BET-15, BET-17). Otherwise near parity |
| Wheeled, tracked and hybrid briefs in interiors | No published loop; scene build time at thousands of worlds not published | Unmeasured | 3 to 10, with low confidence | Clearance and static analysis remove candidates before any training |
| Family queries with amendments, warm | An expert with a linear-programming, SMT or reachability tool over an incumbent | The residual a faster solver does not touch (parsing, assembly, checking, serialization and harness); the crossover reuse count | 1 to 3 per query, more only through reuse across amendments; cold may be below 1 | Deriving the abstraction from a rich plant is cheaper than an expert doing it by hand |
| Failure discovery and repair | Uniform randomization and logs | The proposal must cover the failure mode; certifying absence gains nothing | 2 to 20 in episodes to a planted failure (published range), up to 100 as a TARGET; about 3 in repair turns | A good distress score; the solvability witness |
| Facility and multi-robot missions | No composed incumbent workflow was found | Unmeasured | Not estimated | M scope; a ratio is claimed only for strata that were run |
On quadruped missions the range against per-design training tops out at ten, and reaching the top needs every lever to work at once, including the two that are least measured (family policies and rank agreement). The denominator is an accepted decision and not a training run: a 1% failure bound at 95% confidence costs 299 failure-free episodes per accepted design in every arm. On humanoid missions the range is wider because the incumbent's training phase is longer. If, on any stratum, a short robust program over an incumbent engine reaches the same accepted answers in comparable time, the report says so, the mechanisms ship as a layer over that engine for that stratum, and engine work continues only for the gaps that are internal to an engine (BET-14).
2.7 What the program does first because of this
- Measure the rival before building against it. At M0 and M1 the incumbent loop is timed by phase on
trj, starting with what needs no agent: cold start, a topology edit to the first step, and training to threshold on one and two cards. Versions and warm configurations are pinned and frozen first (§39). - Run the rank-agreement experiment early. Whether cheap rungs order designs as the trained, referee-promoted rung does decides lever 2 and half of lever 3. About forty per-design trainings (about 5 GPU-hours where the specialists of §23 are reused, about 45 otherwise; §28.10) need the owner's approval because they exceed the default compute envelope (§23, §42).
- Put the first mission slice on the CPU at M1. The docking fixture of NS-22 exercises the mission query, the clock contract, the certificate bundle and the auditor before any GPU work exists.
- Hold a partial gate at M3 on legged and wheeled briefs, with a written rule for each outcome (BET-13), and the tournament at M7.