Sensorium: sensors, clocks, observability
12,681 words32 subsections18 tables3 listingsPlan version 1.1
On this page32
Owner crate: frob-sense (layer 3). Ray and sensor kernels live in the kernel library (§21) and are authored here.
Labels in this chapter: a worked example is a derivation from its stated premises and is not a measurement; default
policies and capacities are DESIGN choices; every sensor and optical parameter value is assumed until a card records
a datasheet or a measurement; thresholds in gate tables are TARGETs; statements about sibling code or executed
experiments are marked FACT with their source.
20.1 What it owns
Sensorium owns what a robot is allowed to know and when it is allowed to know it. That is: the sensor models; the observation service that turns physical history into typed, timestamped, masked packets; the clock contract under which sensors, links, estimators, controllers and actuators run; the queues that hold packets and commands in flight; stills and identifier buffers for agents; the snapshot feed for the viewer; and the analyses that follow from treating sensing as part of the design: the observability Gramian, the observation-alias witness, the observation-preserving symmetry subgroup, extrinsic calibration, the causal-policy test and noninterference.
It does not own estimators and controllers (§22), policy training (§23), mission queries (§25), calibration of the simulator against measurement (§30), or the finite-group machinery it calls (§13).
20.2 Decisions
- ADR-SENS-01: observations come only from a deterministic observation service that is part of the model identity. They never come from a renderer, a viewer, presentation timestamps or level-of-detail state. Reason: a result must not depend on whether someone was watching.
- ADR-SENS-02: time is exact. Every clock has a rational period and an integer phase accumulator. The earlier rule that sub-schedules are integer ratios of the physics tick is withdrawn. Reason: real rates are not integer ratios of one another (§20.4.6), and rounding a rate changes the task.
- ADR-SENS-03: seven instants are distinct fields. Sample, exposure, timestamp, arrival, decision, command and actuation are stored separately on every packet and command, and none is inferred from another. Reason: each pair that is conflated hides a class of real failure (stale data, mis-stamped data, a command applied before it could have been computed).
- ADR-SENS-04: one boundary ordering rule on every tier (§20.4.3). Reason: without it two backends can disagree about whether a packet that arrives exactly at a decision instant is visible.
- ADR-SENS-05: clocks, open exposures, packet queues, command queues and noise-process states are part of every checkpoint. Reason: a restore that drops a packet in flight is a different world.
- ADR-SENS-06: sensors read the collision field, the overlays and the optical map, never the visual mesh (with ADR-HAB-04).
- ADR-SENS-07: a missing measurement is a masked value with a reason, never a number. A numeric zero is a measurement only where the schema says so.
- ADR-SENS-08: a failure with a physical cause is a function of the ray-and-scene interaction and of a latent drawn per surface region and episode, not an independent coin per sample. Reason: independent coins let repetition average away an error that in a real sensor persists (§20.6).
- ADR-SENS-09: truth, measurements, belief and policy observations are four namespaces with declared access. The actor's input schema is frozen and is run with every privileged channel absent. Reason: the commonest silent defect in learned control is a leak of simulator truth.
- ADR-SENS-10: noise streams are keyed by logical identity (world address, sensor, sample index, component), never by draw order. Reason: adding a sensor must not change what the others read.
- ADR-SENS-11: a symmetry consumer receives only the observation-preserving subgroup (§20.13).
- ADR-SENS-12: a camera that feeds a policy has an exposure and a rolling shutter from M2. An instantaneous global-shutter camera is a labeled idealization, not the default.
20.3 The rule
A sensor is a map from a history of physical states and a calibrated stochastic process to a timestamped observation with a frame, units, a noise specification, a validity mask and a quality flag. It is a Component: it reads ports and fields and emits a typed signal.
Observations come from the observation service, which is part of the model identity. They never come from a renderer, a viewer, presentation timestamps or level-of-detail state (the same rule the Wright Flyer plan sets for its modeled pilot, FACT). The viewer follows a wall-clock presentation schedule; a sensor follows its exposure and transport schedule on simulation time. The gym cannot implement a camera by asking the viewer for its latest frame.
20.4 The clock contract S
20.4.1 Exact time
A run declares its clocks: the physics step h and, for every sensor, link, estimator, controller and actuator channel, a period and a phase. All are rational numbers of seconds. A clock may also declare a rate error as a rational (50 parts per million is 1/20,000), which gives it the period T·(1 + ε), still rational. A drifting clock is therefore still an exact clock.
Let Q be the least common multiple of all the denominators. Simulation time is an unsigned 64-bit count of quanta of
1/Q seconds. Each clock keeps one integer, the time of its next nominal event; at a boundary at time t, every clock
whose next event is at or before t fires and advances by its period. This is a phase accumulator: integer addition and
comparison only, so the schedule is in determinism class D0 and cannot drift. Admission refuses a lattice with Q above
2⁴⁰ (TARGET; a run of 10⁶ s then stays below 2⁶⁰ quanta) with FROB-REF-440 and the repair "round the offending rate
to a declared rational".
20.4.2 Seven instants
| # | Instant | Definition | Set by |
|---|---|---|---|
| 1 | Sample | The sensor's clock fires: the nominal trigger and the start of its exposure. | The sensor's clock |
| 2 | Exposure | The close of the half-open window [open, close) over which the instrument integrates. Zero length for an instantaneous sensor; one window per row for a rolling shutter. | Exposure and readout fields of the sensor card |
| 3 | Timestamp | The time the device writes on the packet, in its own clock with its own offset and drift, at a declared point of the exposure (start, middle or end). | The device-clock model |
| 4 | Arrival | The packet becomes readable by its consumer, after transport latency, jitter, loss, duplication and reordering. | The link model |
| 5 | Decision | An estimator or controller tick reads its inputs. | The consumer's clock |
| 6 | Command | The tick's output is issued, after compute latency. | The controller card |
| 7 | Actuation | The command takes effect at the actuator, after bus and driver latency, and holds over the following open interval. | The actuator card (§15) |
Order constraints: sample ≤ exposure ≤ arrival; a packet is used by a decision at t only if its arrival is at or before t; decision ≤ command ≤ actuation. The timestamp is deliberately unconstrained. It is what the device believes, and an estimator that trusts timestamps over arrival order is exposed to the errors a real one has.
Every observation exposes its age, decision minus sample, as an integer count of quanta. Telemetry always records it; a policy receives it if its observation schema declares it (OPEN-SENS-3). A held value is never presented as fresh.
20.4.3 The boundary ordering rule
At a boundary at time t, in this order and on every tier:
- Commit the physical state at t.
- Close the exposures whose window ends at t.
- Stamp and enqueue the samples closed in step 2, computing each arrival instant from the link model.
- Deliver every packet whose arrival is at or before t, in the total order (arrival, sensor identifier, sequence number).
- Run the estimator and controller ticks whose decision instant is t. They see exactly the packets delivered so far.
- Latch the commands whose actuation instant is t. A latched command acts over the open interval that follows.
- Open the exposures that start at t, then advance physics.
Consequences that the tests of §20.20 pin down:
- A zero-latency loop is legal: a sample taken at t can inform a command that acts just after t.
- A packet that arrives one quantum after t cannot influence the command decided at t.
- Integrated quantities use one half-open window convention. An impact of 0.2 N·s inside one 10 ms exposure of a force sensor contributes 20 N to that exposure's average and nothing to the adjacent one; an impulse exactly at t belongs to the window that opens at t.
- At an episode reset the clocks take a fresh origin, queues are emptied, open exposures are discarded and phases are redrawn from the world-address stream if the task randomizes them. The final observation of the ended episode is kept for the semantics of §23.
20.4.4 Events inside a step
A clock event that falls strictly inside a physics step is handled by a policy declared per clock and recorded in the model identity:
| Policy | Behaviour | Where |
|---|---|---|
split |
The step ends at the event instant, so the event lands on a boundary. Lateness is zero. | The referee and the strict batch-cpu profile, whose steps are event-located (§16) |
interpolate |
The sample is taken at the nominal instant from the state at the previous boundary advanced along that step's own twist by the integrator's retraction, with a declared error bound. | Any tier, per sensor |
hold |
The event is realized at the first boundary at or after its nominal instant. Lateness, always less than h, is recorded on the packet. | Default on throughput tiers (OPEN-SENS-1) |
Interpolation never crosses a boundary at which an impact reset a velocity. A camera that straddles an impact samples the segmented pose history; a linear blend across the reset would fabricate motion blur and an inertial acceleration that never occurred.
20.4.5 Packets and queues, inside every checkpoint
A packet carries its sensor identifier, sequence number, the first four instants, a validity mask with reasons, and a
payload reference. Packets wait in a fixed-capacity ring per world and sensor. The capacity is computed at admission
from the cards: the longest latency plus the longest exposure, divided by the period, plus the reordering window. If
the history buffers cannot cover the declared exposure window and latency bound of a task, admission refuses
(FROB-REF-441); a queue never overflows silently.
The link model draws latency, loss, duplication and reordering from counter-based streams keyed by world address, sensor, sequence number and component. A late or out-of-order packet may be folded into a buffered estimator and may change its future outputs. Decisions already taken, and the observations they were taken on, are immutable history and are what telemetry records.
A checkpoint (§29) contains, per world: every clock's next-event time; every open exposure with its partial integral and, for a rolling shutter, its row position; every packet and command in flight; the states of correlated noise processes (bias random walks); and the latent values of ADR-SENS-08. Estimator and controller histories are in the same checkpoint under their owner (§22). Three adversarial instants are tested: just after a shutter opens, just after a packet is dropped, and just before a controller tick.
// DESIGN. Fixes intent, not signatures. Every time is an integer count of 1/Q seconds.
pub struct ClockSpec { period: Quanta, phase: Quanta, rate_error: Ratio, intra_step: IntraStep }
pub enum IntraStep { Split, Interpolate { bound: Bound }, Hold }
pub struct Packet {
sensor: SensorId, seq: u32,
t_sample: Quanta, t_exposure: Quanta, t_stamp: DeviceTime, t_arrival: Quanta,
lateness: Quanta, mask: MaskRef, payload: BufRef,
}
pub struct Command { t_decision: Quanta, t_command: Quanta, t_actuation: Quanta, payload: BufRef }
20.4.6 A worked schedule, and the shortcut it replaces
Physics at 400 Hz and a policy at 30 Hz: h = 1/400 s, T = 1/30 s, Q = 1,200, so a step is 3 quanta and a policy period
is 40. Nominal decisions fall at 0, 40, 80 and 120 quanta. Under hold they are realized at boundaries 0, 14, 27 and
40 (42, 81 and 120 quanta), with lateness 0, 2, 1 and 0 quanta, that is 0, 1.67, 0.83 and 0 ms. The pattern repeats
every 0.1 s with decision intervals of 14, 13 and 13 steps, and the long-run rate is exactly 30 Hz.
The shortcut "run the policy every thirteenth physics step" runs it at 400/13 = 30.77 Hz. Over a 20 s episode, counted on the half-open window as the 600 are, that is 616 decisions instead of 600 (steps 0, 13, …, 7,995 of 8,000), and a reward counted per decision is overpaid by 2.7%. Under the earlier integer-ratio rule the only alternatives were that error or a change of the physics step to suit the policy.
20.4.7 Schedule races F
Several events can share one instant: a sample closes, a packet arrives, an estimator ticks, a watchdog fires, a
command latches. The ordering rule fixes the order between stages. Within a stage it fixes a stable logical order, and
the result must not depend on that order where the events are independent. The race check enumerates one
representative of each equivalence class of orderings (two orderings are equivalent when they differ only by swapping
independent events) and compares digests. Hand-checkable size: six events with three independent conflicting pairs
have 720 orderings in 2³ = 8 classes. FACT: asupersync has a deterministic laboratory runtime
(asupersync/src/lab/runtime.rs) in which such schedules can be driven. Owner: frob-conformance. Pilot and
falsifier: plant one stale read and one watchdog-against-latch race in a fixture; the check is funded beyond the
fixture only if it finds both. Ceiling: a correctness guard with no speed benefit. ✖ KILL-SCHED: one planted race
missed; fallback, the fixed total order of §20.4.3 with no enumeration.
20.5 Proprioception S
- Joint encoders with the card's resolution and noise, quantization, index offset and wraparound; backlash as seen through the transmission (§15); velocity by the estimator the real driver uses, with the delay its window implies. First check: a prescribed rotation against a known tick count.
- Motor current and estimated torque; winding and housing temperature; battery voltage and current, read through a calibrated readout with clipping and lag. First checks: the step response of a resistor-inductor circuit and a thermal cool-down (§15).
- IMU: specific force and angular rate in the sensor frame with bias random walks, noise densities, scale and
misalignment, saturation, sample rate and latency. For a sensor rigidly mounted at offset r from a body point O, the
point acceleration is a_P = a_O + α × r + ω × (ω × r) and the reading is f = R_WSᵀ·(a_P − g). FACT:
fs-mbdexposesprepare_point_motion,point_motionandspecific_acceleration_worldfor rigid and articulated bodies; Sensorium uses them on the referee and derives the batched kernel from the same expression. First checks: at rest the reading is (0, 0, +g) with zero rate; in free fall without rotation it is zero; in free fall spinning at 5 rad/s with a 0.10 m perpendicular offset it is 2.5 m/s² toward the axis; gravity enters exactly once for prescribed and for floating bases. - Foot contact switches with threshold and hysteresis. Force-torque sensors are a port reading: the continuous wrench is accumulated over the exposure, each impact impulse is added once, and the sum is divided by the exposure duration. Wrenches are moved to the sensor frame and reference point by the coadjoint action of §11 at the time of application. First checks: dead weight; a known offset torque; the 0.2 N·s impulse of §20.4.3.
- Wheel odometry: encoder displacement, which equals travel only while the wheel rolls. First checks: a circular arc on a dry floor; a deliberate slip on a film patch, where the odometry error must equal the slip distance in the contact telemetry.
- Tactile arrays F: finite taxel footprints with normal and shear saturation. First check: the resultant load against the sum over taxels. No task before M6 needs them.
20.6 Exteroception S
Depth cameras, spinning and solid-state lidar patterns, height scans around the base, and semantic and instance labels are all ray queries: sphere tracing against the scene distance field, exact tests against overlays, and analytic ray tests against the moving primitives. Noise models cover range-dependent noise, grazing-angle dropout and range limits. Because the ray kernels are ordinary kernels (§21), sensor outputs have the same determinism class as the physics.
A lidar beam has its own pose and its own instant inside the scan; a height scan is a lidar whose pattern is a grid under the base. What a ray reports at its first hit depends on the optical card of the region it hits (§19.3.1):
| Optical class | Ray rule |
|---|---|
diffuse |
A return at the hit range, with strength from albedo, incidence and range; masked weak below the detection threshold |
absorber |
No return; masked absorbed |
glass |
Reflectance by the polynomial R(θ) = r₀ + (1 − r₀)·(1 − cos θ)⁵. A direct return only within the card's acceptance angle of normal incidence; otherwise the ray continues through the pane to whatever lies behind, or follows the mirror direction, whichever return is stronger. A sensor with several echoes reports both. |
mirror |
The ray follows the mirror direction for one bounce. The reported point lies on the original ray at the unfolded path length, which is the mirror image of the true surface: a phantom behind the mirror. |
specular-floor |
As mirror with weight R(θ), as diffuse otherwise; R rises with the wetness latent. Phantoms appear below the floor at grazing angles. |
retroreflector |
A return with gain; saturation and bloom are F |
emitter |
No range; raises the ambient level seen by pattern-projecting depth cameras |
| Unknown region | invalid: unknown-region (§19.13) |
DESIGN: one bounce at most, so the launch shape is fixed and the cost of a ray on a specular class is at most doubled. The reflectance formula uses only multiplication, so no elementary function enters the kernel.
Optical volumes. A ray that crosses a volume of extinction coefficient σ over a length ℓ keeps the fraction
exp(−σ·ℓ) of its power one way and exp(−2σ·ℓ) for an active sensor, computed with the deterministic exponential of the
kernel dialect (§35). Below the detection threshold the return is masked attenuated. Worked case: with σ = 2 m⁻¹
(assumed) and a 5% threshold, an active sensor loses every return behind more than ln(20)/4 = 0.75 m of steam.
Pattern-projecting depth cameras. A pixel is valid only if the projector ray and the camera ray reach the same diffuse point with pattern contrast above ambient. It drops out on glass (the pattern passes through), on mirrors and polished metal (it is reflected away), on absorbers, on floors lit above the pattern's irradiance, beyond range, at grazing incidence, and in the shadow between projector and camera. Each cause has its own mask reason.
Why causes are not coins. Two payload classes need different grips, and a sensor reads the class correctly three times in four. With independent errors, a majority of three reads is right 27/32 of the time, of five 459/512, of seven 3,807/4,096 (93%). If the error is common to all reads, as it is when the same pane of glass or the same glare is in every frame, the answer stays right three times in four however many reads are taken. A model that draws dropout independently per sample overstates what repetition buys. Hence ADR-SENS-08: the dropout of a ray through a given region is determined by geometry, the optical card and one latent per region and episode, and repeated reads through the same pane fail together.
20.7 Cameras with exposure and rolling shutter [S from M2]
A camera row r is exposed over [t_s + r·t_row, t_s + r·t_row + t_exp). Each row's rays are cast from the sensor pose at that row's instant. The pose inside a step is the pose at the previous boundary advanced along that step's twist by the integrator's own retraction (§13, §16), so it is exactly the trajectory the integrator implies, and it is segmented at every boundary. The exposure duration comes from the card and, for cameras with automatic exposure, from the lighting level of the zone, so a dim hallway lengthens exposure and increases skew and blur.
The cost is one retraction per row and no additional rays. TARGET: at most 5% added to the ray stage.
| Tier | What is modeled |
|---|---|
| S, M2 | Per-row sample instants (rolling shutter); exposure represented by its midpoint; per-beam instants for lidar |
| F | Exposure integration with two or four sub-instants per row, with a pixel masked mixed when its sub-samples disagree by more than a threshold (flying pixels at depth edges) |
| F | Radiometric RGB: shot and read noise, glare, lens distortion; rasterized textured meshes for policies. Not needed by any task before M5 |
| F | Event cameras: asynchronous streams of log-brightness changes |
Hand-checkable known answers: a camera with a focal length of 600 pixels and a 16 ms readout that yaws at 2 rad/s images a vertical edge near the optical axis with a skew of 600 × 2 × 0.016 = 19.2 pixels from the first row to the last. A lidar spinning at 10 Hz on a robot yawing at 1 rad/s turns 0.1 rad during a scan, so a wall 5 m away is displaced by 0.5 m across the scan seam.
For the F rows: benefit ceiling, none on speed; they change what a perception policy must tolerate. Pilot and falsifier: on the analytic moving-bar fixture, compare midpoint sampling with four-sample integration; if depth error and mask statistics differ by less than the sensor's noise over the hazard list's speed range, integration is not built. Kill (KILL-EXPOSURE): more than 25% added ray cost; fallback, midpoint sampling.
The other Frontier sensor models named in this chapter (tactile arrays, event cameras, radiometric RGB, retroreflector bloom, backscatter from optical volumes) are on no task's path before M5. Each is funded only when a registered task needs it, names one analytic fixture as its pilot when it is, has no speed benefit to claim, and falls back to the mask-only model under the same cost rule.
20.8 Sensor failure modes
A sensor model is useful to a designer in proportion to the failures it shares with the instrument. Each mode below is
a parameter on the sensor card with provenance, a reason code in the validity mask or a flag on the packet, and a
first independent check. Parameter values are assumed until a card records a datasheet or a measurement.
| Sensor | Failure mode | Model | First check |
|---|---|---|---|
| Encoder | Quantization, index offset, wraparound, missed counts | Integer counts with a declared loss process | Tick count over a prescribed rotation |
| Encoder with transmission | Backlash hides small load-side motion | Dead zone from the transmission card (§15) | Reversal test |
| IMU | Bias random walk, scale, misalignment | Correlated noise process in the checkpoint | Allan-deviation slope of the simulated stream |
| IMU | Saturation on impact | Clipping at the card range, flag saturated |
A drop test that exceeds range |
| IMU | Mount offset | Lever-arm terms of §20.5 | The spinning free-fall case |
| Current, temperature, voltage | Clipping, lag | First-order readout with limits | Circuit step response |
| Contact switch | Chatter, hysteresis | Threshold pair | A slow load ramp |
| Force-torque | Drift, bandwidth, impulse aliasing | Exposure averaging with a drift process | The 0.2 N·s impulse |
| Wheel odometry | Slip | None added: the encoder is honest and the wheel is not | Slip on a film patch |
| Depth, projected pattern | Dropout on glass, mirror, absorber, sunlit floor; projector shadow; flying pixels | §20.6, §20.7 | Plane at a known angle; glass pane; edge fixture |
| Depth, any | Near-plane blind zone, range limit, grazing dropout | Range and incidence masks | Plane swept through the near limit |
| Lidar | Glass invisibility and multipath phantoms | §20.6 | Mirror phantom at the reflected point |
| Lidar | Motion distortion | Per-beam instants | Moving plane |
| Lidar | Attenuation in steam and dust | Optical volumes | Slab threshold |
| Lidar | Range quantization, blind zone | Card fields | Analytic sphere |
| Camera | Rolling-shutter skew, blur, long exposure in dim light | §20.7 | Moving bar |
| Any | Latency, jitter, loss, duplication, reordering | Link model | A packet is never delivered before its latency |
| Any | Clock offset and drift, wrong timestamp point | Device-clock model | Two sensors drift apart by the declared rate |
| Any | Extrinsic bump after an impact | A step change of the mount transform | §20.14 |
| Any | Stuck value, dead sensor | Fault injection available to Nemesis (§26) | The causal-policy harness sees the mask |
20.9 Images
- Stills for agents S.
frob telemetry stillsdraws contact sheets (several views at key frames) with the same ray kernels, shaded simply, together with an identifier buffer.frob telemetry inspect-pixelmaps a pixel to a link, a fixture, a region or a contact. A vision-capable agent can look and then ask (the loop FrankenTextiles uses for garments, FACT). Stills are drawn from the collision field, so they show what physics has (§19.5.3), and they are private when the scene is (§37). - RGB cameras for policies F. Rasterized textured meshes with the radiometric model of §20.7. Not needed by any task before M5.
- Hero renders F. FrankenSim's spectral path tracer, offline only (its
tracerfeature is Frontier-gated and off by default, FACT).
20.10 The viewer
The viewer, its packaging route and its overlay table are specified in §32.15. The self-contained replay is exported by
telemetry.export-html, a verb of the telemetry cluster of §29 (§29.12); its packet is §32.24. This section states
only what Sensorium supplies to the viewer.
The viewer draws physical overlays from the same snapshots: contact forces and friction cones, film thickness, sensor frusta and individual rays coloured by mask reason, unknown regions, the collision-to-visual deviation map, and the bottleneck of a clearance cut. The viewer never feeds anything back to physics: the simulator owns state and the viewer reads immutable snapshots.
20.11 The observability Gramian S
What it is. An analysis that says which directions of state error a sensor suite cannot see over a window of motion. For an estimator error state δx (in the invariant error coordinates of §22), a window of N steps, state-transition matrices Φ_k from the window start, measurement Jacobians H_k and noise covariances R_k,
W = Σ_k Φ_kᵀ·H_kᵀ·R_k⁻¹·H_k·Φ_k, summed over k = 0, …, N − 1.
Its eigenvalues are information per unit error along its eigenvectors. Two forms are provided. The linearized form
uses the Jacobians of the sensor models and of the estimator's error dynamics. The empirical form perturbs the
initial state by ±ε along each error coordinate, replays the window as 2n worlds of one batch, and forms W from
whitened output differences; it needs no Jacobian and works on any tier. Owner: frob-sense for H, R and the
empirical driver; §22 supplies Φ. Verb: sense.observability. Atelier lists it among its analyses beside sensor
coverage (§28).
What it reports. The dimension of the null space against the declared gauge; the smallest eigenvalue outside the gauge, after a declared scaling of the error coordinates; and the corresponding eigenvector in named coordinates ("translation along the corridor axis"). It is computed along a route, so the result is a profile: where on the route the estimate degrades, and in which direction.
Gauge. With an IMU, encoders, contact switches and gravity, global position and yaw cannot be observed. The null space has dimension four, and the analysis must find exactly that, analytically and by finite perturbation (PO-SENS-09). A larger null space is a finding; a smaller one is a defect in a sensor model.
Hand-checkable case. A robot at (x, y) in a corridor whose walls are the lines y = ±a measures its distance a − y to one wall with variance s². Then H = (0, −1) and, standing still for N samples, W = (N/s²)·diag(0, 1). The null vector is (1, 0): position along the corridor is unobservable. A second beam at 45° reads (a − y)·√2 and adds nothing along x. A beam that reaches an end wall at x = b reads b − x and makes W full rank. In an infinite corridor no beam direction observes x, and one end wall, door frame or fiducial within range restores it.
Its limit, which is the reason for §20.12. The Gramian is local. Two aisles that are exact copies of each other give a full-rank Gramian in each, yet nothing distinguishes one aisle from the other. Infinitesimal ambiguity is the Gramian's subject; finite ambiguity is the alias witness's.
Uses. A warning in the design check when a route has a stretch whose smallest eigenvalue falls below a threshold; an input to the assumption monitors of runtime shields (§22); and a score for sensor placement F. For the placement use: benefit ceiling, the share of estimator failures that come from sensing geometry and not from calibration or filter dynamics, which is unknown. Pilot and falsifier: on the M3 battery, classify estimator divergences by cause; if under a fifth are geometric, placement by Gramian is not built. Kill: no measured drop in drift after a recommended move; fallback, the Gramian remains a report.
20.12 The observation-alias impossibility witness
The question. Can more training on the present sensors ever satisfy this mission? Sometimes the answer is no for a reason that has nothing to do with the body, the controller or the amount of compute: two situations that the robot cannot tell apart require different actions.
The object. For a design with observation interface Y (sensors, cards, clocks, links) and a mission whose worlds form a declared set, an alias witness is:
- two admitted worlds w₁ and w₂;
- a commitment time t* and a set of permitted probe behaviours before t*;
- evidence that the delivered observation transcripts up to t* are identical in the two worlds under every permitted behaviour, or have total-variation distance δ under a fixed behaviour;
- evidence that the sets of action continuations at t* that can still satisfy the mission, S₁ and S₂, are disjoint.
The statement (PO-SENS-07). A causal policy maps delivered observations, its past actions and its own random draws to an action. If the transcripts are identical, it takes the same action in both worlds at t*, and that action lies outside S₁ or outside S₂. No causal policy satisfies the mission in both worlds, and private randomization does not help. In the stochastic case with equal prior weight on the two worlds and a fixed experiment with observation laws P and Q, the probability of choosing correctly is at most (1 + TV(P, Q))/2, which bounds mission success.
Hand-checkable example: a mission that is unsolvable for want of a sensor. A robot stands at a T-junction. A charging plate lies at the end of the left branch in world L and at the end of the right branch in world R; each branch is 10 m long. The two branches are mirror images. The robot's only exteroceptive sensor is a planar lidar; the plate lies below the scan plane and is detected only by contact or within 0.5 m. The battery allows 12 m of travel.
- Each world alone is solvable: turn the correct way and drive 10 m.
- In both worlds the lidar returns the same scan at the junction and along either branch until 0.5 m from its end, whatever the robot does. The transcripts are identical up to the turn.
- S_L is "turn left" and S_R is "turn right". A wrong turn is discovered after 9.5 m and costs 9.5 + 9.5 + 10 = 29 m in all, against a budget of 12 m.
No policy succeeds in both worlds, and with equal priors none succeeds more than half the time. Training cannot change that. Four changes can, and they are of four different kinds: a camera that sees a marker on the plate from the junction (a sensor); a fiducial at the junction (a retrofit, §19.12); a battery good for 29 m (a body); or a label given to the robot before it starts (information by commitment). The studio's answer is the witness and these alternatives with their prices.
The same structure with noisy sensing. Two payload classes need disjoint final grip forces. A probe reads the class correctly with probability 3/4. By exhaustive enumeration: with no read, success is 1/2; with three independent reads 27/32; with five 459/512; with seven 3,807/4,096, which first exceeds 90%. If a common fault affects every read, success stays at 3/4 after any number. And a deadline changes the answer: a rule that stops when one class leads by two reads is right 90% of the time with 3.2 reads on average, but it is still undecided after six reads with probability 27/512, and forcing a guess then yields 225/256, below 90%. Every sensing policy is therefore compared under the mission's deadline, with delay priced.
Where witnesses come from.
| Source | Method | Tier | Class of the result |
|---|---|---|---|
| Structural | A hidden parameter has no information path to any delivered packet before t*: no ray can intersect geometry that depends on it, and no contact with it can occur before a computed earliest time. Uses the dependency analysis behind noninterference (§20.17). | S | Certified relative to the model, for every policy |
| Symmetry and repetition | Candidate world pairs from the scene's own symmetries and repeated modules (a mirrored layout, a translated aisle) under which the sensor suite is invariant and the goal is not. | S to propose; checked as below | As checked |
| Enumerated | On a finite abstraction (a lattice of actions and readings in exact arithmetic), exhaust every causal policy tree. | S for small models | Certified for the abstraction |
| Searched | Nemesis (§26) searches world pairs for minimal transcript distance subject to disjoint success sets. | F | Tested, and labeled experiment-specific unless one of the rows above also applies |
Checking. A claimed witness is checked independently of how it was found: both worlds are replayed on the referee
under each permitted behaviour, the bytes of every delivered packet up to t* are compared, and the disjointness of the
success sets is established by the method the witness names. A witness that covers only a declared set of probe
behaviours says so; it is not presented as a bound over all policies. Owner: frob-sense. Verb: sense.alias, which
returns a witness or "none found", never "none exists" unless a structural or enumerated argument supports it.
Three engineering answers. A perceptual ambiguity can be resolved by extracting the information (a better sensor, a faster or earlier packet), by moving to acquire it (a permitted probe, a different mount), or by changing the mechanics so that the information is unnecessary (a passive compliance whose response suits both cases). Mission queries (§25) price all three, together with retrofits, against one another. The timing variant, in which the right sensor exists but its packet arrives too late, is the docking fixture of §20.16.
Effect on other verbs. A training or optimization run requested on a mission with a standing witness for its
interface is refused (FROB-REF-445) with the witness attached, unless the caller overrides for diagnosis.
Ceiling of the benefit, stated before any speed test. The witness saves the training compute that would have been spent on missions that are unsolvable under their interface. The share of such missions is unknown; it is measured on the M3 mission corpus before the search lane is funded. Paper-scale pilot and falsifier: a suite of 64 layouts in which the aliased ones are planted by construction; the lane must find every planted alias and report none on the rest.
✖ KILL-ALIAS. Trigger: one false witness, that is, a policy exhibited that satisfies the mission in both worlds of an accepted witness. Fallback: the searched lane is withdrawn; structural and enumerated witnesses, and pairs declared by a mission author, remain.
20.13 The observation-preserving symmetry subgroup S
The mechanical symmetry group Γ_mech of a design (C-09) is computed from masses, inertias, joints and actuator cards. It ignores what the robot sees and what it is asked to do. A quadruped built symmetric fore-aft and left-right has Γ_mech = C₂ × C₂, of order four. Mount a depth camera facing forward on its sagittal plane and the fore-aft reflection is no longer a symmetry of the closed loop: an obstacle ahead is seen, an obstacle behind is not. A policy constrained to be equivariant under all of Γ_mech would be required to respond to the unseen obstacle as the mirror image of its response to the seen one.
Definition. Γ_obs is the set of g in Γ_mech that also preserve:
- every sensor: g maps each mount frame to the mount frame of a sensor with an identical card, and maps its pattern (beam set, pixel grid) to itself including its timing;
- every clock, link and latency of the interface;
- the payload's inertia and any declared asymmetry of the load;
- the task: its goal set, requirement predicates and reward terms (declared by Arena, §24);
- the law of the scenes: the scene family is invariant under the corresponding reflection or rotation of the world.
The elements that preserve a structure form a subgroup (PO-SENS-05), so Γ_obs ≤ Γ_mech. It is stored with the tuple
(design, sensor profile, task, scene family). Owner: frob-sense for the predicates, frob-algebra for the group
operations. Verb: sense.symmetry.
Two cases worth checking by hand.
- The quadruped above: the left-right reflection maps the camera to itself and each image row to itself, so it survives if rows are read out top to bottom. The fore-aft reflection and the half-turn do not. Γ_obs has order two, and data augmentation doubles the data; it does not quadruple it.
- A spinning lidar on the same plane of symmetry: a reflection reverses the order in which beams fire. With per-beam instants (§20.6) the mirrored robot would need a lidar that spins the other way, so the reflection is broken at first order in the product of body rate and scan time. It is restored only if the scan is treated as instantaneous, and the report says which was assumed.
An IMU mounted off the plane of symmetry breaks the reflection of raw observations too. The symmetry analysis (§28) reports it as a near-symmetry with its repair: move the IMU by the stated offset, or declare the lever-arm correction part of the observation map.
Rule (INV-SENS-08). Equivariant weight sharing, augmentation, deduplication of closed-loop results and quotients
of gait orbits use Γ_obs. A request that names an element outside it is refused (FROB-REF-443). Open-loop mechanics
(mass-matrix structure, static analyses) may still use all of Γ_mech.
Test. For each generator of Γ_obs: run a world and its image under g, with noise stream identifiers permuted by g, and require the observation streams to correspond exactly up to the signed permutation. For an element of Γ_mech outside Γ_obs the same test must fail; that is the negative control. Equivariant policies built on Γ_obs are Frontier and belong to §23.
20.14 Calibration as a least-squares pose graph S
What it is. Extrinsic calibration of a sensor rig is weighted least squares on a graph, and is named as such. Nodes are sensor frames and the base. The unknowns are corrections δ_v in 𝔰𝔢(3) to each sensor's mount transform, with the base anchored at zero to remove the six-dimensional gauge. Edges are pairwise relative-pose measurements (camera to IMU, lidar to odometry, hand to eye), each with a residual r_e in 𝔰𝔢(3) and a 6 × 6 weight W_e, the inverse of its covariance. Linearized, the stacked residual is r = B·δ, where B is the block incidence matrix of the graph with adjoint blocks on the tail side: with corrections applied on the right of each mount transform (§11), an edge from sensor 1 to sensor 2 has r₁₂ = log(T̂₁₂⁻¹·T₁₂) ≈ δ₂ − Ad(T̂₂₁)·δ₁, where T̂₁₂ is the predicted and T₁₂ the measured pose of sensor 2 in the frame of sensor 1, and T̂₂₁ = T̂₁₂⁻¹. Using Ad(T̂₁₂) on the tail is the planted frame mutant. The Gauss–Newton step solves the normal equations
(Bᵀ·W·B)·δ = Bᵀ·W·r,
and the update is applied on the group by the retraction of §13. Owner: frob-sense. Verb: sense.calibrate.
Consumers: the simulated robot's own self-check after an impact; fault injection ("extrinsic bump"); and Ground (§30),
which runs the same solver on logs from a real rig.
The consistency residual. The part of r that no assignment of extrinsics can explain is r_⊥ = (I − B·(Bᵀ·W·B)⁻¹·Bᵀ·W)·r. The projector is orthogonal in the metric W, not in the Euclidean metric; rotations in radians and translations in metres have different scales, and the Euclidean projector gives a different and wrong decomposition whenever W is not a multiple of the identity (the negative control). Two different faults look different:
- A bumped sensor is a change of extrinsics. The solve absorbs it: δ̂ at that node moves by the bump and r_⊥ stays at noise level. It is detected as a change of δ̂ beyond its covariance.
- A faulty edge (one bad pairwise estimate, a time offset on one pair) is inconsistent round every cycle through it: r_⊥ is not small.
What the graph must be for this to work. On a tree, B is square after anchoring and r_⊥ is identically zero: there
is no redundancy, a bump and a bad edge are indistinguishable, and a request for fault localization is refused
(FROB-REF-446). Detecting a bad edge needs that edge to lie on a cycle. Identifying which edge is bad needs more,
and the smallest cases show it. Scalar offsets, unit weights, one edge biased by β:
- Three sensors, three pairwise edges (one cycle). The least-squares residual is β/3 on every edge. The loop is known to be inconsistent; the bad edge is not identified.
- Four sensors, six pairwise edges. The residual is β/2 on the bad edge, β/4 on the four edges that share a node with it and zero on the opposite edge. The bad edge has the unique largest residual, by a factor of two.
Known answer for the bump: on a four-sensor rig with all six pairwise measurements, a planted lidar bump of 2° and 15 mm is recovered within 0.1° and 1 mm (TARGET), attributed to the lidar node, with r_⊥ at noise level.
20.15 The causal-policy test S
A policy is causal if its command at a decision instant depends only on the observations delivered to it by then, its past actions and its own random draws. The simulator can violate this on the policy's behalf in many small ways, and the test catches them without reading the policy's code.
Procedure. Run a world and a twin that differs only in something the policy cannot yet know: terrain beyond every
sensor's reach or after the decision time; the friction of a surface not yet touched; a measurement whose packet is
delayed past the decision; a randomization parameter of a later phase. Record, at every decision instant, a digest of
the bytes delivered so far and a digest of the command. Wherever the delivered-observation digests of the two runs
agree, the command digests must agree. Verb: sense.causal-test. Owner: frob-sense, with fixtures in
frob-conformance. It runs on every policy before its first evaluation is recorded, and on the baseline controllers
of §22.
Planted leaks that must each fail (the negative controls): a privileged state channel wired into the actor's input; a reset mask that lets history cross an episode boundary; a sensor read at its current value instead of from its delivered packet; a curriculum parameter passed as an observation; a noise stream keyed so that a hidden parameter shifts the draws of a visible sensor.
An asymmetric critic may read privileged state during training under a schema marked training-only. The actor's inference schema is frozen, and the test runs with the privileged channels physically absent, not zeroed.
20.16 The docking fixture: a known answer with a closed-form boundary
One small model exercises the clock contract, strict comparison, the alias witness and exact arithmetic together. Position is in metres and time in seconds.
Model. Three ticks of one second. x₀ = v₀ = 0. During tick k the acceleration is g·u_k, with a hidden gain g ∈ {1, 2} fixed for the episode and a command |u_k| ≤ U, for U in [½, 1]. The mission is (x₃, v₃) = (1, 0) exactly. The only informative packet is a reading of v₁ with additive error in [−ε, +ε].
If the packet arrives before the second command. Write p = u₀. The terminal conditions force the second command to be u₁ = 1/g − 2p and the third to be u₂ = p − 1/g (from v₃ = g·(p + u₁ + u₂) and x₃ = g·(2p + u₁)). The command limits in both worlds leave 1 − U ≤ p ≤ (U + ½)/2. The reading lies in [p − ε, p + ε] if g = 1 and in [2p − ε, 2p + ε] if g = 2; the two required values of u₁ differ by ½, so the intervals must be disjoint, which is ε < p/2. Choosing the largest admissible p gives the exact boundary
ε < (U + ½)/4, equality excluded.
With U = ¾ and p = ¼ the commands are (¼, ½, −¾) for g = 1 and (¼, 0, −¼) for g = 2, both reaching (1, 0), and the tolerable error is below 1/8. With p = 5/8 it is below 5/16: the first command is a probe, and a bolder probe buys a sensor two and a half times worse. At ε = 5/16 exactly, both worlds can report 15/16 while requiring u₁ = −¼ and u₁ = −¾: the strict predicate fails at equality.
If the packet arrives after the third command is committed. A causal controller issues one command vector in both worlds. The final position of a common vector is g·(2p + u₁), which cannot equal 1 for both gains. Every case is excluded. This is an alias witness (§20.12) whose cause is timing, not the absence of a sensor.
Grid known answer. On U = ½ + i/32 for i = 0…16 and ε = j/64 for j = 0…32, the boundary admits a cell exactly when 2j < 32 + i: 344 of 561 cells are admitted, 217 excluded, and nine lie exactly on the boundary (the even i). All 561 cells are excluded when the packet is late.
A hypothesis set widened by a hair. Admit a third gain, 33/32. At U = ¾ and ε = 1/64 the reading (65/64)·p is possible for g = 1 and for g = 33/32 under every first command, so both worlds receive the same remaining commands; their final positions are c and 33c/32, and the smallest achievable worst error is 1/65 m, at c = 64/65. Every cell admitted under the two-gain mission is wrong for the three-gain mission. The rule this fixture enforces everywhere: a verdict is bound to the declared support of the hidden parameters, the delivery deadline, the error units and the target tolerance, and a change to any of them invalidates it.
Arithmetic. The fixture is evaluated in exact integer arithmetic (class D0) and, as a control, in single precision. FACT (an executed boundary-screen experiment on this machine's Metal device and on both RTX 4090 cards, listed in the source register and to be re-read before a gate cites it): on the dense grid of 65,536 cases, one of the two grids of the boundary corpus of §21.8, unguarded single precision gave 452 false admissions and 23,438 false exclusions; a guard band with exact fallback gave no final errors; an exact integer kernel gave none and produced identical bytes on every device. The fixture is therefore also the first test of the exact-integer kernel class and of guarded single precision (§35).
Owner: frob-sense with frob-conformance. Nothing about a real docking device follows from it.
20.17 Noninterference S
Five properties, each tested by changing one thing and comparing digests.
| Property | Change | Must be unchanged |
|---|---|---|
| Presentation | Turn the viewer, stills or the telemetry level on or off | Every physics digest and every delivered observation |
| Truth | Change simulator truth behind a sensor that is disconnected, masked or not yet delivered | The actor's inputs and commands (§20.15) |
| Sensor set | Add, remove or reorder sensors that no controller consumes | The outputs of every other sensor, and physics |
| Batch | Run the same world address alone, in a batch of 8,192, and beside worlds with different sensor suites | That world's delivered observations (with the batch-independence test of §35) |
| Optics against contact | Change an optical card, or an interface card | Physics digests, or ray outputs, respectively (§19.5.4) |
The sensor-set property is what ADR-SENS-10 buys. A generator that hands out draws in call order fails it the first time a sensor is added.
20.18 Verbs and refusals
| Verb | Kind | First milestone | Returns |
|---|---|---|---|
sense.profile |
read | M2 | The sensor suite, cards with provenance, clocks, link models and queue capacities of a design and task |
sense.schedule |
read | M2 | The exact boundary schedule over one repeat of the clock lattice, with lateness per event |
sense.causal-test |
run | M2 | Pass, or the first decision instant and channel at which causality fails |
sense.observability |
run | M3 | The Gramian spectrum along a route, with named weak directions |
sense.alias |
run | M3 | An alias witness with its class and the priced separating changes, or "none found" |
sense.symmetry |
read | M3 | Γ_obs, the elements of Γ_mech it excludes and the sensor or task that excludes each |
sense.calibrate |
run | M3 in simulation, M5 on imported logs | Extrinsic corrections with covariance, the consistency residual per edge, and a fault attribution where the graph supports one |
telemetry.stills, telemetry.inspect-pixel |
read | M2 | As in §20.9. telemetry.export-html belongs to the telemetry cluster of §29 (§29.12); the viewer it packages is §32.15 |
| Code | Refusal | Ranked repairs |
|---|---|---|
FROB-REF-440 |
The clock lattice is not admissible: a period is zero or negative, or the common denominator exceeds the limit | Round the offending rate to a declared rational |
FROB-REF-441 |
History buffers cannot cover the declared latency and exposure window | Raise the queue capacity; shorten the latency bound; lower the rate |
FROB-REF-442 |
An observation is requested from presentation state, or an actor schema names a privileged channel | Use the observation service; move the channel to the training-only schema |
FROB-REF-443 |
A symmetry consumer names a group element outside Γ_obs | Use Γ_obs; change the sensor suite; restrict to open-loop use |
FROB-REF-445 |
A training or optimization run is requested on a mission with a standing alias witness for its interface | Read the witness; apply a separating change; override for diagnosis |
FROB-REF-446 |
A calibration request exceeds what the graph supports: no anchor, or fault localization on a graph without the needed cycles | Anchor the base; add pairwise measurements |
FROB-REF-447 |
A Reality claim rests on a sensor or optical card field of class assumed |
Record a datasheet or a measurement; drop the claim |
20.19 Invariants and proof obligations
| ID | Invariant |
|---|---|
| INV-SENS-01 | No observation depends on presentation state: viewer, stills, telemetry level or wall clock. |
| INV-SENS-02 | A packet is never delivered before its arrival instant; a decision at t depends only on packets that arrived at or before t, past actions and the policy's own draws. |
| INV-SENS-03 | A restored checkpoint reproduces every subsequently delivered observation byte for byte. |
| INV-SENS-04 | The k-th nominal event of every clock is at phase + k·period exactly; lateness is zero under split and less than h under hold. |
| INV-SENS-05 | A masked value never appears as a number, and every mask has a reason. |
| INV-SENS-06 | Adding, removing or reordering sensors does not change what the others read. |
| INV-SENS-07 | The actor's inference input contains no privileged channel. |
| INV-SENS-08 | Only elements of Γ_obs are used for weight sharing, augmentation and deduplication of closed-loop results. |
| INV-SENS-09 | A world's delivered observations do not depend on the batch it ran in. |
| ID | Statement | Verification |
|---|---|---|
| PO-SENS-01 | The integer event schedule equals the rational schedule for all time. | Induction on file; a run of 10⁹ ticks against exact rational arithmetic. |
| PO-SENS-02 | The ordering key (stage, arrival, sensor identifier, sequence number) is a total order with no ties. | Uniqueness of the key by construction; property test. |
| PO-SENS-03 | The pose used for a row or beam instant equals the integrator's continuous extension and agrees with the committed pose at both ends of the step. | Symbolic identity for the Cayley and exponential retractions (FrankenSymPy, Certified). |
| PO-SENS-04 | The reported point of a one-bounce mirror return is the reflection of the true hit point through the mirror plane. | Geometric identity card; analytic fixture. |
| PO-SENS-05 | The elements of Γ_mech that preserve the sensor suite, the task and the scene law form a subgroup. | Closure card; exhaustive check on the template designs. |
| PO-SENS-06 | The docking boundary is ε < (U + ½)/4, with equality excluded. | Derivation on file; exact rational evaluation on the 561-cell grid. |
| PO-SENS-07 | Identical delivered transcripts and disjoint success sets exclude every causal policy; the stochastic bound is (1 + TV)/2. | Two-line proofs as theorem cards (§12). |
| PO-SENS-08 | The consistency projector is orthogonal in the metric W, and r_⊥ vanishes on a tree. | Algebraic card; numeric check with anisotropic weights. |
| PO-SENS-09 | The observation model of IMU, encoders and contacts with gravity has a four-dimensional unobservable subspace: position and yaw. | Analytic rank computation; finite-perturbation Gramian. |
20.20 Known answers
| Case | Truth | Negative control |
|---|---|---|
| Depth against analytic scenes | Within 1 mm plus the brick bound | Ray distance swapped for optical depth |
| IMU at rest, in free fall, spinning with offset | (0, 0, +g); zero; 2.5 m/s² toward the axis | Gravity applied twice |
| IMU on a turntable | Reads the table rate | — |
| Lidar pattern | Beam geometry and beam instants match the pattern definition | — |
| 400 Hz physics, 30 Hz policy | Boundaries 0, 14, 27, 40; lateness 0, 2, 1, 0 quanta of 1/1,200 s; exactly 600 decisions in 20 s | Every-thirteenth-step scheduling gives 616 |
| Drifting clock | Two sensors at 50 parts per million apart separate by 1 ms after 20 s, exactly | A floating-point accumulator drifts from this |
| Impulse in an exposure | 0.2 N·s in a 10 ms window reads 20 N; the adjacent window reads 0 | A closed-closed window counts it twice |
| Arrival at the decision instant | Visible at t; arrival one quantum later is not | Evaluating the controller before delivery |
| Checkpoint at three adversarial instants | Byte-equal delivered observations after restore | A checkpoint without queues |
| Rolling shutter | Skew 19.2 pixels for 600 px, 16 ms, 2 rad/s | A global-shutter build shows none |
| Lidar motion distortion | 0.5 m displacement at the seam for 10 Hz, 1 rad/s, 5 m | — |
| Camera across an impact | No blur beyond the segmented motion | Linear pose blend across the reset |
| Mirror | Phantom at the reflected point: a target at (1, 2, 0) and a mirror plane x = 3 give (5, 2, 0) | — |
| Glass pane | Direct return near normal incidence per the card; the surface behind otherwise | Treating glass as diffuse |
| Steam slab | Return lost beyond 0.75 m at σ = 2 m⁻¹ and a 5% threshold | One-way attenuation for an active sensor |
| Repeated reads | 27/32, 459/512, 3,807/4,096 with independent errors; 3/4 with a common fault | Independent coins through one pane |
| Corridor Gramian | Null vector (1, 0); full rank with an end wall | — |
| Gauge | Null space of dimension four | — |
| T-junction | Witness accepted; success at most 1/2 | With a 29 m budget the witness is rejected |
| Sequential reads under a deadline | 90% with 3.2 reads on average; 27/512 undecided after six; 225/256 if forced | — |
| Quadruped with a forward camera | Γ_obs of order two inside Γ_mech of order four | The fore-aft reflection fails the equivariance test |
| Spinning lidar under reflection | Broken with per-beam instants | — |
| Pose graph, one biased edge | β/3 on all three edges of a triangle; β/2, β/4, 0 on six edges among four sensors | Euclidean projector with anisotropic weights |
| Pose graph, bumped sensor | 2° and 15 mm recovered and attributed | A tree refuses localization |
| Causal-policy test | The reference actor passes | Each of five planted leaks fails |
| Docking | ε < (U + ½)/4; 344, 217 and 9 on the grid; all late cases excluded; 1/65 m with the third gain | Unguarded single precision misclassifies near the boundary |
| Noninterference | The five rows of §20.17 | A sensor added to a draw-order generator |
20.21 Gates and kill gates
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-SENS-01 | Known answer | Depth against analytic scenes within 1 mm plus the brick bound; the three IMU cases of §20.5 and the turntable case; lidar beam geometry and instants match the pattern definition. | M2 |
| GATE-SENS-02 | Fidelity | Sensor outputs, masks and delivered-observation digests agree across backends inside their determinism class. | M2 |
| GATE-SENS-03 | Known answer | Noninterference: all five properties of §20.17 hold on the standard battery; the draw-order control fails. | M2 |
| GATE-SENS-04 | Known answer | Clock contract: the 400/30 schedule exactly; zero drift against rational arithmetic over 10⁹ ticks; the impulse and arrival cases; identical schedules on every tier. | M2 |
| GATE-SENS-05 | Known answer | Checkpoints: restore at the three adversarial instants gives byte-equal delivered observations and physics digests over the next 10⁴ ticks on every tier. | M2 |
| GATE-SENS-06 | Known answer | Rolling shutter and beam instants: skew within 0.5 pixel of 19.2; seam displacement within 1 cm of 0.5 m; no fabricated blur across an impact; at most 5% added ray-stage cost. | M2 |
| GATE-SENS-07 | Known answer | Optical failure modes: the mirror, glass, steam and pattern-dropout cases of §20.20; every masked sample carries a reason; repeated reads through one pane fail together. | M3 |
| GATE-SENS-08 | Known answer | Observability: the corridor case and the four-dimensional gauge, by the linearized and the empirical form, agreeing to within 5% in the non-zero eigenvalues. | M3 |
| GATE-SENS-09 | Known answer | Alias witness: the T-junction and the late-packet docking witnesses are found and pass the checker; on 64 planted layouts every planted alias is found and no false witness is reported. | M3 |
| GATE-SENS-10 | Known answer | Γ_obs: the two cases of §20.13 on every template; the equivariance test passes for members and fails for non-members. | M3 |
| GATE-SENS-11 | Known answer | Calibration: the triangle and four-sensor residual patterns; the planted bump recovered within 0.1° and 1 mm and attributed to the right node in 100 of 100 seeds. | M3 |
| GATE-SENS-12 | Known answer | Causal-policy test: the reference actor and every baseline controller pass; each of the five planted leaks fails. | M2 |
| GATE-SENS-13 | Known answer | Docking fixture: 344, 217 and 9 in exact arithmetic on batch-cpu, CUDA and Metal with identical bytes; guarded single precision with fallback gives zero final errors; the third-gain amendment reverses every admitted cell. |
M2 |
| GATE-SENS-14 | Known answer | Sensor cards: every failure mode of §20.8 marked S has its first check passing, and every card field reports its provenance. | M3 |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-ALIAS | One false witness from the searched lane (§20.12) | Structural, enumerated and declared witnesses only |
| ✖ KILL-SCHED | The race check misses a planted race (§20.4.7) | The fixed total order, with no enumeration |
| ✖ KILL-EXPOSURE | Exposure integration costs more than 25% of the ray stage or changes no mask statistic beyond noise (§20.7) | Midpoint sampling |
20.22 Open questions
- OPEN-SENS-1. The default policy for clock events inside a step on throughput tiers. Default:
holdwith recorded lateness. Owner: Kinesis and Sensorium leads. Decided at M2 from the measured effect of lateness on the fidelity ledger; blocks GATE-SENS-04 only for the choice of default. - OPEN-SENS-2. The limit on the clock lattice and the menu of declared rationals for common device rates. Default: Q at most 2⁴⁰. Decided at M2.
- OPEN-SENS-3. Whether observation age is a default policy input. Default: recorded in telemetry always, given to a policy only when its schema declares it. Decided at M3 with the first tasks that randomize latency.
- OPEN-SENS-4. Which sensor and optical cards ship with datasheet or measured fields. Default: all
assumed, Realityno-claim(FROB-REF-447). Owner: project lead. Blocks any Reality claim that depends on sensing. - OPEN-SENS-5. Whether glass reports the stronger return or draws between returns. Default: the stronger return, deterministic. Decided when a measured card for one depth camera and one lidar exists.
20.23 Force sensors act on a force history: filter, clip and integrate in a declared order
What it is. S A sensor-chain rule for force, torque, current and pressure readouts that saturate, and a refusal for peak-load claims on tiers whose contacts are impulses. §20.5 averages the wrench over the exposure and adds each impact impulse once. That is correct for a linear sensor. A transducer that overloads acts on force as a function of time, and an impulse has no duration.
Counterexample. A rectangular pulse of 1 N·s lasts 1 ms or 10 ms, with peaks of 1,000 N and 100 N. Both average 50 N over a 20 ms exposure. A transducer that clips at 200 N before integration reports 200 N × 1 ms ÷ 20 ms = 10 N for the short pulse and 50 N for the long one. Clipping after averaging reports 50 N for both.
Mechanism. DESIGN:
- A sensor card declares its chain as an ordered list of stages: a transfer function (bandwidth, with its state in
every checkpoint), an overload law (a clip level, or a recovery model), exposure integration and quantization.
The kernel
K-sense-ftapplies the stages in the declared order, at the substep at which force is resolved. - A contact force history is available in three grades.
resolved: the compliant laws compute force as a function of state at every physics step, and the chain is applied to that history.modeled: on an impulse-level step (the referee's event step, the hard-contact batch law) the interface card supplies a pulse model with provenance, a duration τ_c and a shape, for example a half-sine with τ_c = π·√(m_eff/k) and peak (π/2)·J/τ_c for an impulse J.none: there is no pulse model; the linear reading is still produced, itsoverloadfield isunknown, and any requirement atom on peak load or on sensor overload evaluates toundetermined(§24.5). sense.profilelists, per sensor, the chain, the grade available on each tier, and the atoms that the grade cannot decide (refusal: peak-load claim without a resolved or modeled force history).
Owner: frob-sense; pulse models live on interface cards (§18).
Known answers. The rectangular rows: 10 N and 50 N in the declared order; 50 N twice in the swapped order, which is the mutation that must fail. A half-sine with J = 1 N·s: τ_c = 10 ms gives a peak of 157 N, no clipping and a reading of 50 N; τ_c = 1 ms gives a peak of 1,571 N and a clipped reading of 9.59 N.
| GATE-SENS-15 | Known answer. The rectangular rows are exact; the swapped-order mutant is rejected; on the referee an impulse with grade none yields overload: unknown and an undetermined peak-load atom in 100 of 100 cases; on a compliant tier whose step resolves the pulse with at least 10 samples the clipped reading is within 2% of the closed form. M2. |
20.24 Visibility under self-occlusion, and sensor mounts as design variables
Visibility along a motion S; mounts in the optimizer F. This extends the sensor-coverage analysis A-14 and the placement use of the Gramian in §20.11. Coverage by ray geometry against the scene misses the occluder that changes most between designs: the robot's own limbs and whatever it carries. A torso camera is blinded by a tote just before a step; a low body cannot see over a lip; a swinging leg crosses the field of view once per stride.
The query. sense.visibility takes a design, a payload envelope, a scene and a nominal motion (a route with a gait
or drive profile). At every sample instant of every ranging sensor it casts the sensor's pattern against three
occluders: the scene field, the robot's own collision primitives at the pose of that instant, and the declared
payload. A target is visible at an instant when at least the card's minimum number of valid returns land on it after
the optical masks of §20.6. Targets are the contact sites the motion will use (footholds, wheel paths), the
thresholds and steps of the route, and the hazard regions of the scene's hazard list (§19).
Two numbers per target. With t_c the instant the body first touches or crosses the target:
- lead = t_c − (first instant the target is visible);
- blind interval = t_c − (last instant the target is visible).
The lead must cover the sense-decide-actuate chain of A-21 plus the controller's planning horizon; the blind interval must stay under the bound the task declares. A third figure, the blind dwell, is the floor area inside the stopping distance ahead of the body that no sensor has seen within a declared age. Each is reported as a profile along the route with the occluder that caused each gap (scene, named link, or payload).
Hand case. A depth camera at the body's front edge at height H looks down and forward. A carried box has its top front corner 0.4 m ahead of the camera and 0.3 m above the floor. The ray grazing that corner meets the floor at x = 0.4·H/(H − 0.3), and floor nearer than that is hidden. At H = 0.5 m, x = 1.0 m; with the next foothold 0.2 m ahead of the camera at 1 m/s the blind interval is 0.8 s. At H = 0.9 m, x = 0.6 m and the blind interval is 0.4 s. The price is in another analysis: a 1 kg mast head raised by 0.4 m on a 20 kg robot lifts the centre of mass by 0.02 m, and with a half-track of 0.2 m the static tip-over angle falls from atan(0.2/0.30) = 33.7° to atan(0.2/0.32) = 32.0°. With the box removed the hidden region ends at the body's own front edge.
Mounts as design variables F. A sensor mount is already a model field with a typed edit (move-sensor, §14.7).
The optimizer of §28.8 may hold mount transforms and payload-bay geometry as continuous parameters, with lead, blind
interval and the Gramian index of §20.11 as objectives or constraints beside the static margins of A-08 and the
clearance of A-18. Every move of a mount recomputes Γ_obs (§20.13). Nothing here claims an estimator bound: the
Gramian remains a local index, the alias witness of §20.12 remains the test for finite ambiguity, and bias states
remove the state-independence that the invariant error enjoys without them.
Owners and verb. frob-sense owns the query and reuses the ray kernels of §20.6 with the model's collision
primitives as moving occluders; Atelier lists the result under A-14; frob-opt consumes it. Verb: sense.visibility
(run). At stage 7 of design.check a target with a lead shorter than the declared latency chain is a refusal with
ranked repairs as typed edits (raise or tilt the mount, move the payload, shorten the latency).
Ceiling, pilot and gates. Ceiling: the share of estimated-state failures caused by visibility and not by calibration, filter dynamics or control, which is unknown; it is the same unknown as in §20.11 and the same pilot settles both. Pilot before funding the optimizer use: on the M3 battery classify failures under estimated state by cause; under a fifth from visibility or geometry and the optimizer use is not built.
| Gate | Claim | Threshold (TARGET) | First milestone |
|---|---|---|---|
| GATE-SENS-16 | Known answer | The box-and-mast case: hidden floor boundary at 1.000 m and 0.600 m within 1 cm; no hidden region beyond the body's front edge with the payload removed; a planted occlusion by a swinging limb is found and attributed to the link; lead and blind interval identical on every tier inside their determinism class | M3 |
| GATE-SENS-17 | Design comparison | On 20 designs carrying payloads across stairs, a doorway and wet tiles under estimated state: mounts recommended by the optimizer cut the gap between privileged-state and estimated-state success by at least half; lead and blind interval correlate with the estimated-state failure rate at 0.65 or more | M4 |
| Kill gate | Trigger | Fallback |
|---|---|---|
| ✖ KILL-VISIBILITY | GATE-SENS-17 fails on both counts, or the pilot attributes under a fifth of estimated-state failures to visibility or geometry | sense.visibility stays as a report and a stage-7 check; mounts are not optimizer variables |
20.25 Sampled readings: impacts in an exposure, encoder unwrapping, the load-cell cut
What it is. S Three additions to the proprioceptive models of §20.5. Each fixes what a sampled reading means in
a case where the instantaneous formula gives a wrong or ambiguous number. Owner: frob-sense. Verbs: sense.profile,
sense.alias.
An integrating accelerometer and an impact. The reading over an exposure window of length T is the mean specific force, and on a step that resolves contact as an impulse the velocity of the mounted point jumps. DESIGN: the window's reading is
f̄ = (1/T)·[ ∫ R_WSᵀ·(a_P − g) dt + Σ R_WSᵀ·(v_P⁺ − v_P⁻) ],
with the sum over the impacts inside the window. With continuous pose, a rigid offset r from the body point O contributes (ω⁺ − ω⁻) × r to the jump of the point's velocity. Each impact belongs to exactly one window by the half-open convention of §20.4.3. On the compliant tiers the force history is resolved and the integral already contains it; the jump term applies to impulse-level steps (the referee's event step and the hard-contact batch law). Bandwidth and clipping follow the ordered chain of §20.23, with the same three grades of force history.
Known answer: a body in ballistic flight, not rotating, rebounds from −1 to +1 m/s inside a 10 ms window. The window reads +200 m/s² upward. Every smooth specific-force sample in the window is zero. The mutant without the jump term reads zero through a bounce.
Unwrapping a sampled encoder. A modulo-one-turn position sampled at 100 Hz delivers identical samples at +80 and at −20 turns per second, with no noise at all: 0.8 turn and −0.2 turn per sample are the same position. A controller cannot recover the direction from those samples. DESIGN: nearest-increment unwrapping is valid when
Ω·Δt + 2·ε < 1/2 (in turns),
with Ω a bound on the absolute speed, Δt the actual gap between delivered samples, and ε the error of one reading. At
equality with ε = 0 the two half-turns coincide. sense.profile reports each encoder's largest unambiguous speed,
(1/2 − 2·ε)/Δt, against the joint's speed limit seen through the transmission, and sense.alias returns the two-speed
pair as a witness when the limit exceeds it. After a lost packet Δt is the gap that occurred, and a velocity estimate
whose gap violates the condition carries the flag ambiguous until a hardware turn counter or an independent velocity
channel resolves it. The estimator never picks the nearer branch in silence. This is the sampling counterpart of the
static turn count of §13.11.
The load-cell cut. A six-axis force-torque sensor measures the wrench across a cut in the load path. Its card states the cut, the side (distal on proximal, or the reverse), the sign convention, the moment origin, the frame and the tare state. DESIGN: the reading is computed from the momentum balance of everything distal to the cut: gravity on the distal bodies, every external load on them, and their rate of change of momentum, with moments about the stated origin. A sum of the sole's contact forces is a different quantity.
Known answer: a 2 kg foot below the cut, upward positive, g = 9.81 m/s², untared, reporting distal on proximal. Stationary under a 100 N ground reaction it reads +80.38 N. In swing with no contact and an upward acceleration of 2 m/s² it reads −23.62 N. The sole's contact sum reads 100 N and 0 N in the two cases.
| GATE-SENS-18 | Known answer. The rebound window reads 200 m/s² on the referee to 10⁻⁹ relative, and the adjacent windows read zero; the mutant without the jump term and the closed-closed window mutant both fail. The encoder fixture: +80 and −20 turns per second deliver bit-equal samples, sense.alias returns the pair, and the largest unambiguous speed is reported as 50 turns per second at ε = 0; after a planted packet loss the estimate is flagged ambiguous in 100 of 100 cases. The load-cell rows read +80.38 N and −23.62 N to 10⁻⁹, and each of the four card mutants (side, sign, origin, tare) changes the reading and is caught. M2. |