FrankenRobotics

The seven claims and the claim product

1,825 words3 subsections2 tables1 listingPlan version 1.1

On this page3
8.1 The seven claims8.2 The claim product8.3 Rules

Every result this system produces makes exactly one kind of claim, and each kind is established by a different kind of evidence. Gates, reports and documentation keep them apart. Version 0.2 had six claims; this version adds Design comparison, because "this design is better" and "this design meets the mission" are what the studio most often says, and neither is a fidelity statement or a known answer.

8.1 The seven claims

Claim Meaning Evidence that establishes it What it does not establish
Proof A stated proposition holds in the declared model. A checker accepted it: a Lean kernel for a general theorem (class Proved); or, for an instance-level statement, FrankenSymPy's proof kernel, an interval or exact-rational certificate, or the offline auditor frob-verify on a family certificate bundle (class Certified) (§12). Anything about floating-point results; the model's agreement with the world; any quantifier order other than the one stated.
Known answer The implementation recovers an analytic or synthetic truth. A known-answer test with a derivation on file (§38, Appendix M). Correctness away from the tested cases.
Parity The implementation reproduces a reference tool or a published number. A fixture captured from a pinned oracle that was never regenerated from the candidate, or a comparison with a published figure under stated conditions (§38). That either side is right. Two programs that share a solver have correlated error. Agreement with a published real-robot figure is still Parity.
Fidelity A faster tier agrees with a slower, more trusted tier. The fidelity ledger: per-quantity disagreement distributions on the standard battery, and agreement of the task decisions the two tiers lead to (§16, §35). That the trusted tier is right. Anything outside the battery.
Design comparison One design is better than another, or a design meets a mission, on a stated family of worlds. A declared study, anytime-valid or fixed-n exact, on common world addresses; for a qualified result, the blinded qualification lane (§24, §28). That either design meets a hard limit when the verdict is a paired one; anything outside the stated world family; anything about reality.
Reality The simulation agrees with physical measurement. Imported real-robot measurements with their own provenance, predictions frozen before comparison, and a reality band for a calibrated family (§30). Everywhere else this claim is no-claim, and reports say so. Anything about a design that was never built or a family that was never calibrated.
Performance A speed or scale figure on named hardware. A recorded run with the disclosure block (§36); for a ratio against a rival, the protocol of §39. Anything about another workload, world count or machine.

Three clarifications fix the edges (DESIGN).

  • A claim has a kind and a class. The kind is one of the seven. The class is one of the four evidence classes of §12, ordered Proved, Certified, Tested, Estimated from strongest to weakest. Only a Proof claim can be Proved or Certified. The other six are Tested or Estimated however much evidence they rest on: no amount of testing becomes a proof.
  • A family certificate is a Proof claim of class Certified. It says that, in the declared model and under the declared assumptions, every design in an inner cell satisfies the mission, or none in an excluded cell does. The assumptions are inputs (rule 6 below).
  • A comparison of methods is a Design comparison. The "method" gates of version 0.2, which test whether a method of this plan beats a competing arm on a corpus of designs, name this claim. The unit compared changes; the evidence required does not.

Each claim has a permitted wording, and reports use no other.

Claim Permitted wording in a report
Proof "proved: ⟨statement⟩; checker ⟨name and build⟩; certificate ⟨hash⟩" or "certified: ⟨statement⟩; auditor frob-verify ⟨build⟩; bundle ⟨hash⟩"
Known answer "recovers ⟨KA identifier⟩ within ⟨tolerance⟩"
Parity "matches ⟨oracle⟩ at ⟨pin⟩ within ⟨tolerance⟩ on ⟨fixture⟩"
Fidelity "tracks ⟨higher tier⟩ within ⟨envelope⟩ on ⟨battery⟩; decisions agree on ⟨share⟩"
Design comparison "⟨B⟩ beats ⟨A⟩ on ⟨metric⟩, evidence ⟨e-value or exact interval⟩, study ⟨identifier⟩" or "qualified at level α_i on sealed clusters ⟨identifier⟩"
Reality "inside the calibrated band for ⟨family⟩, nominal coverage ⟨level⟩ at n = ⟨count⟩" or "reality: no-claim"
Performance "measured ⟨figure⟩ on ⟨hardware⟩; disclosure block ⟨identifier⟩"

8.2 The claim product

In version 0.2 the rule "a result inherits the weakest claim among its inputs" was prose. Here it is a type (DESIGN). The reason is that an optimizing agent will find any path by which a statistical result can be reported as a proof, and a rule that lives in a style guide is such a path. frob-core owns the type; every verb's output envelope carries one (§32); frob-verify reads one from every bundle (§12).

// DESIGN sketch: fixes intent, not signatures. Owned by frob-core.

/// The four evidence classes of §12. Combination takes the minimum.
pub enum EvidenceClass { Estimated, Tested, Certified, Proved }   // Ord: Estimated < Tested < Certified < Proved

/// Exactly one of the seven claims.
pub enum ClaimKind { Proof, KnownAnswer, Parity, Fidelity, DesignComparison, Reality, Performance }

/// What a result asserts, how strongly, about what, and that it authorizes nothing.
pub struct ClaimProduct {
    kind: ClaimKind,
    class: EvidenceClass,           // minimum over inputs; no setter
    weakest: InputRef,              // the input that set the class: content hash and field path
    scope: Scope,                   // design family, world family, quantifier order, horizon, validity region
    tier: Tier,                     // lowest physics tier among the inputs
    determinism: DetClass,          // weakest of D0..D4 among the inputs
    reality: RealityScope,          // NoClaim, or the calibrated family a band was issued for
    conditions: FlagSet,            // union over inputs: tier-sensitive, controller-sensitive, model-conditional
    inputs: Vec<InputRef>,          // every consumed artifact, each with its own claim product
    authorization: NoAuthorization, // three constants, all false
}

impl ClaimProduct {
    /// The only way to build a product from parts. Class, tier and determinism take the weakest; scope takes
    /// the intersection; conditions take the union; a mismatch of quantifier order is a refusal.
    pub fn combine(kind: ClaimKind, parts: &[ClaimProduct]) -> Result<ClaimProduct, Refusal>;
}

/// `False` is a unit type that serializes as the JSON literal false. No constructor produces `true`.
pub struct NoAuthorization {
    pub execution_authorized: False,
    pub hardware_safety_signoff: False,
    pub sim_to_real_certified: False,
}
// A build packet adds a fourth constant, `fabrication_released: False`, in the packet's own sealed type (§31.12).

// Three things do not exist, and a compile-fail test pins each: a conversion from ClaimProduct into a
// Proved value; a setter for `class`; a public constructor that takes a class as an argument.

A leaf product is created only by the component that produced the evidence: a checker creates Proved or Certified, the Gauntlet harness and the study runner create Tested, and everything else creates Estimated. An actuator, interface or material card creates one product per field from that field's provenance, so that a single assumed number is visible as the weakest input of everything built on it.

A worked case. A comparison of two designs consumes: a kernel whose group-membership identity is Certified; a friction coefficient from an interface card field marked assumed, which is Estimated; and a paired anytime-valid study on the GPU tier under determinism class D2, which is Tested. The product is a Design comparison of class Estimated, its weakest input is the friction field of the card, its tier is the GPU tier and so it is provisional until promoted (§16), and its reality scope is no-claim. The report prints all of that in one line and offers, as its first next action, the measurement that would raise the weakest input.

8.3 Rules

  1. A gate names exactly one claim.
  2. A failure is a failed cell, never an excluded cell. Partial coverage never rounds up, and every failed episode occupies a row.
  3. Parity is not Reality. Agreeing with another simulator establishes that two programs agree. A comparison with a published real-robot figure, made before any calibration data is imported, is labelled Parity.
  4. Fidelity is directional. It says the fast tier tracks the referee on a battery, not that either is right.
  5. The weakest input wins, by construction. A proved kernel fed an assumed friction coefficient produces an estimate, and the type leaves no other outcome.
  6. Assumptions are inputs. A certificate over a model with an assumed friction interval is Certified as a statement about that model. Any statement about the robot inherits the class of the assumption. A bundle lists its assumptions as inputs with their own claim products, and frob-verify refuses a bundle whose assumptions are missing.
  7. A comparison is relative. A paired verdict that B beats A is emitted together with each design's own verdict on the mission's hard limits. A relative win with no regression does not establish that either design meets the mission.
  8. Quantifiers are part of the claim. "For every world, some controller works" and "one controller works for every world" are different claims with different evidence. Evidence for the easier order is rejected, and the auditor rejects an unknown proof kind or a changed quantifier even when every hash is consistent. The smallest case that separates the two orders is in Appendix N: a plant in which every build has a command that works and no single command works for all builds.
  9. unknown is a verdict. A solver failure, a timeout or a refusal leaves a verdict unknown. It is never recorded as a failure of the design and never as a success. Where contact is set-valued, a fork with any failing admissible branch invalidates a universal success claim.
  10. A claim does not outlive its inputs. After an edit, a claim is carried forward only if revalidation shows that each input is unchanged or is implied by what changed (§28). A hash identifies an artifact; only an implication licenses reuse.
  11. Conditions travel. Flags such as tier-sensitive, controller-sensitive and model-conditional are joined by union and cannot be dropped by a consumer. A model-conditional result may not spend prototype budget.
  12. Composition needs inclusion. Two claims compose into a claim about a sequence only when the guaranteed end set of the first lies inside the entry set of the second, with matching clocks, uncertainty dependencies and state memory. Two individually successful episodes prove no such inclusion.
  13. Nothing authorizes. Every exported artifact carries execution_authorized: false, hardware_safety_signoff: false and sim_to_real_certified: false. No verb, flag or configuration produces any other value.
  14. The word "verified" is not used. Reports, documentation and tool output use the permitted wordings of §8.1.