The missing sequence

Imagine a humanoid standing beside a transfer station, one hand still closed around nothing. A component lies on the floor. A worker has stepped back. The machine has stopped, the safety indicator is steady, and the production dashboard reports a completed intervention. Nobody is injured. The supervisor wants to know whether the cell can restart.

The control engineer opens the logs. There is a torque deviation, a communications warning and a protective stop. The perception service says that the handling area was clear. The task planner says that it requested a placement. The joint controller says that it followed the command it received. Each subsystem offers a plausible account. None establishes the order in which the physical event unfolded.

Did the grip loosen before contact? Did a stale position estimate reach the controller? Did the supply voltage fall before the communications warning, or because the stop changed the electrical load? The difference is operationally decisive. One sequence points toward a mechanical problem. Another points toward integration. A third might implicate the conditions under which the task was authorized.

A maintenance manager can replace a gripper. A software team can change a timeout. Neither intervention is persuasive when the evidence cannot distinguish between their explanations. Restarting would become a wager on whichever specialist sounds most certain.

Physical AI needs an engineered event memory: a bounded, protected record of what the machine received, accepted, commanded and measured around a consequential event. Its value is the ability to preserve disagreements among those observations. A reliable black box should make an unsupported explanation harder to sustain, including an explanation offered by the company that built the robot.

The machine does not need a humanlike recollection. It needs an evidence architecture that can outlive the electrical, mechanical or software failure under investigation. That requirement belongs in the platform definition, alongside power distribution and control interfaces. By the time somebody asks what happened, the most useful evidence may already have disappeared.

Borrow the discipline, respect the boundary

Automotive engineering offers a concrete precedent. The US National Highway Traffic Safety Administration describes event data recorders as preserving technical information around a crash, including vehicle dynamics, system state and inputs. Its overview distinguishes this recording from audio or video capture and cautions that reconstruction also needs other evidence. A recording is one witness to an event, with a defined field of view. NHTSA: Event Data Recorder.

European legislation makes several architectural priorities explicit. Article 6 of Regulation (EU) 2019/2144 addresses data from before, during and after a collision, survivability, protection against manipulation and misuse, system configuration identification and access through a standardized interface. These are requirements within the regulation’s automotive scope. They provide a useful engineering analogy for robots, not a shortcut to a humanoid compliance claim. Regulation (EU) 2019/2144.

The distinction matters even within automotive regulation. Delegated Regulation (EU) 2024/2220 concerns heavy-duty vehicle categories. Among its provisions are protected software updates, retrieval through a diagnostic connector and direct access when that connector no longer works after a collision. Its specificity is instructive: preserving bits is insufficient if the only route to them has been destroyed. Delegated Regulation (EU) 2024/2220.

NHTSA’s engineering study, Light-Vehicle Event Data Recorder Technologies, separately examines recording capabilities, survivability and tamper resistance. It is historical engineering evidence, published in 2020, rather than a characterization of every current recorder. The useful lesson is methodological: data selection, physical survival and trustworthy retrieval require separate investigation. NHTSA engineering study.

Robotics adds its own difficulty. A mobile machine can redistribute its weight, adjust a grip and revise a movement while a high-level planner continues pursuing the same task. The action request and the electrical command are separated by several decisions. A humanoid recorder must preserve enough of those boundaries to expose where the behavior changed.

ISO 10218-1:2025 covers industrial robots, while ISO 10218-2:2025 covers industrial robot applications and cells. Their published scope descriptions are not a basis for presenting the recorder proposed here as a mandated humanoid architecture. Safety applicability still depends on the actual machine and application. Event reconstruction supports investigation; it does not replace risk reduction or the safety functions themselves. ISO 10218-1:2025; ISO 10218-2:2025.

Follow the command until it meets matter

Return to the component on the floor. The planner’s placement request could have been entirely reasonable when issued. The world model may then have aged while a lower-level controller waited for a resource. A motion constraint may have shortened the approach. A grip controller may have reached a current limit. The original request alone cannot resolve any of these possibilities.

The proposed recorder should preserve command provenance across explicit interfaces. An AI action request needs an identifier, its issuing component, the state reference on which it depended and its validity conditions. Controller acceptance should be a separate record. A rejection, modification, expiry or replacement needs its own reason and a link to the request it affected.

The next boundary is equally important. A command transmitted to a joint is not proof that the joint applied it. Where the interface allows, the evidence should distinguish transmission, receipt, acceptance and application. Measured position, velocity, current and force then describe the observed response. A missing acknowledgement remains a missing acknowledgement, even when later motion makes successful delivery seem likely.

Commanded torque deserves careful labeling. It is not measured contact force. Motor current can support a torque estimate, but that estimate depends on motor characteristics and is affected by the drive and transmission. Friction, compliance, temperature, gearing and saturation can separate an electrical command from the force at the object. An investigator needs units, calibration identity and the distinction between direct measurement and derived quantity.

Perception records require the same discipline. A low localization confidence can explain why a controller slowed down, but a confidence value is not automatically a calibrated probability of being wrong. Preserve its definition and estimator version. A state snapshot should include relevant uncertainty and freshness information, not merely the position that downstream software happened to consume.

This evidence does not require access to an AI model’s private internal reasoning. An operational record should capture observable inputs, outputs and decision boundaries. A fluent explanation generated afterward would be another output to assess, not a replacement for the contemporaneous trace. The most important sentence may be a machine-readable rejection code.

Two accurate clocks can tell the wrong story

The investigator aligns the three warnings. The communications fault appears first. That seems to settle the matter until another engineer asks whether the timestamps describe acquisition, publication or arrival at the logging server. The plots are beautifully aligned. The underlying events may not be.

A sensor can sample a value, queue it, transmit it and have it recorded at four different moments. A network delay shifts the last two without changing the first. If a central collector timestamps only arrival, congestion can masquerade as a late physical response. The recorder should preserve the source timestamp and, where useful, a receiving timestamp with an explicit description of each.

A shared time service helps, but it does not abolish uncertainty. The proposed record should include clock identity, synchronization state, estimated timing error and discontinuities. A restarted controller needs a new boot identity so that a counter beginning again at zero cannot be mistaken for an earlier moment in the same run. Monotonic local counters can preserve local order while a mapping relates them to the wider event.

Consider an illustrative case in which two records are separated by one millisecond, but each clock may differ from the reference by two milliseconds. Their relative order cannot be established from those timestamps alone. Printing more decimal places would only improve the appearance of the uncertainty. Sequence numbers, acknowledgements and request identifiers may establish a causal relationship that the clocks cannot.

Distributed buffers should therefore preserve both time and relationships. The event assembler can align intervals, follow command identifiers and mark where the available evidence supports only a partial ordering. It should retain the original timestamps when producing a normalized view. Otherwise the convenience of a common timeline may conceal the transformations that created it.

A robot does not need universal timing precision for every signal. A slowly changing enclosure temperature and a brief contact impulse have different evidentiary demands. Sampling rate, sensor bandwidth, timestamp accuracy and transport delay must be chosen together for the questions the investigation needs to answer. Faster logging cannot recover dynamics that the sensor never measured.

The useful memory begins before the alarm

When a protective stop appears, the triggering condition has already developed. Starting a recording at that instant may preserve the response while losing the cause. A circular buffer solves part of the problem by retaining a moving window of recent data and replacing older entries during normal operation.

This pattern already exists in robotics tooling. The rosbag2 project documents snapshot mode, which maintains buffered messages and writes a snapshot when requested. That is a useful building block for development. Its presence does not demonstrate crash survivability, protected provenance or a complete pre-event and post-event evidence chain for a deployed machine. Those properties require additional design and verification. rosbag2 project documentation.

For the proposed recorder, triggers should come from defined operational conditions: unexpected contact, a fall indicator, object loss, a safety-state transition, serious command disagreement, communications failure or a power warning. Human operators also need a controlled way to preserve an event that the automatic rules missed. An uncomfortable near miss may leave no convenient fault code.

Trigger rules are part of the evidence. Preserve their version, thresholds, source and the time at which they fired. Otherwise a fleet update can change what counts as an event without making that change visible to analysts. A quieter dashboard could then mean fewer incidents, fewer sensitive triggers or more missing data.

Freezing evidence must not freeze control. One buffer can be sealed while another continues accepting new records, with a defined post-event window and bounded transfer work. Local domains should preserve essential data when the central trigger message cannot reach them. A communications failure is precisely the moment when relying solely on a communications-dependent trigger becomes fragile.

Repeated triggers require policy. A stumble, contact and supply interruption might describe one evolving event; merging their windows may preserve the sequence. Unrelated events need separate identities. A storm of minor triggers must not overwrite an earlier serious record without an explicit, auditable rule. Reserved capacity and priority classes can help, but priorities themselves need review.

A delayed discovery creates another boundary. A component may be found damaged minutes after a handling task that produced no trigger. The short high-rate buffer may already have been overwritten. A compact task history can help locate the relevant operation, but it cannot recreate deleted sensor data. The retention design should distinguish immediately detectable events from slowly discovered defects and state which questions each tier can answer. Extending every high-rate buffer indefinitely would merely postpone the same decision at greater cost.

The recorder should declare its coverage. A record may contain the requested pre-event window for joint state but only a shorter perception history. Another domain may have restarted. These are findings, not housekeeping details. An absent record cannot be interpreted as an absent event unless the recording path was known to be operating.

Spend bandwidth on the disputed seconds

Recording everything sounds prudent until the design review reaches memory, bandwidth and power. Continuous raw imagery can dominate the budget while still omitting the controller decision that matters. Selecting evidence begins with plausible incident questions, then works backward to signals and retention windows.

Take an explicitly illustrative calculation: forty joints, six four-byte quantities per joint, sampled one thousand times per second. The payload is 960,000 bytes per second. A thirty-second window requires 28.8 megabytes in decimal units, before timestamps, identifiers, framing, other domains or redundancy. These figures describe an assumed configuration, not a measured humanoid or a recommended universal sampling rate.

The calculation is useful because it makes the trade-off inspectable. Halving the rate halves that payload, but it may obscure a brief transition. Removing one quantity saves space, but may eliminate the observation needed to distinguish a command limit from a physical obstruction. A budget decision has an investigative consequence.

A practical proposal is to separate a compact essential record from richer optional evidence. The essential set would preserve command relationships, selected joint and contact state, safety transitions, power conditions, timing quality and configuration identity. Larger perception fragments could be retained selectively where the application justifies them. Their omission should be explicit, especially when it prevents checking what an object detector missed.

Different rates and windows can coexist. High-rate data can cover a short mechanical transient, while lower-rate task and configuration history extends further back. Preallocated storage and bounded recorder work reduce interference with control. Compression can save capacity, but the design must account for its worst-case processing time and poor compression on difficult data.

A protected record also needs interpretation metadata: units, coordinate frames, schema versions, scaling and calibration references. A perfectly preserved number without its meaning is expensive debris. Long-term retrieval should not depend on finding the one engineer who remembers why a particular firmware release reversed an axis.

The last write must survive the failure

The most awkward evidence loss occurs when the recorder shares the failure under investigation. A main computer crashes before forwarding its buffer. A supply collapses while storage is updating metadata. A harness tears and isolates the joint whose behavior mattered most. The architecture must identify which failures it promises to survive.

There is no single enclosure or memory component that resolves all three. Local buffers can preserve data despite a network break, but volatile buffers still depend on local power. A central protected recorder can simplify retrieval, but cannot collect data that never arrived. A layered arrangement may therefore combine local capture, a bounded central evidence set and persistent event storage.

Power-fail completion starts before the voltage becomes unusable. A detector must provide enough warning for a bounded sequence: stop nonessential work, preserve the highest-priority evidence, commit metadata and mark completion. The stored format should remain recoverable after interruption, using independently verifiable chunks and an explicit incomplete-record state. A successfully returned software write is not necessarily proof that the bytes reached durable media.

Hold-up energy belongs in the same budget. If an isolated recorder domain consumed five watts for two hundred milliseconds after input power vanished, it would need at least one joule delivered to its load. That is an illustrative energy balance. Conversion losses, voltage limits, capacitor tolerance, temperature, ageing and write-time variation would increase the required reserve.

The reserve must not inadvertently keep actuators energized after a safety action. Recorder power and motion power need an intentional relationship, with no unintended backfeed path. Evidence preservation cannot be allowed to delay a required stop. Likewise, write traffic must not block a safety controller’s critical execution path.

Semiconductor selection follows these obligations. A microcontroller unit (MCU) or system on chip (SoC) may provide timestamping, trace and bounded data movement. Nonvolatile memory (NVM) supplies persistence, with endurance and interruption behavior that must match the workload. Secure elements or hardware security module (HSM) functions can protect signing keys. Power monitors and an isolated reserve can support final writes.

The battery management system (BMS) should contribute supply status and relevant switching events, while an inertial measurement unit (IMU) can contribute measured acceleration and angular rate. Their records must retain sensor range and saturation indicators; a clipped impact peak is not its true maximum. A real-time clock (RTC) can help relate the incident to a calendar time, but it should not substitute for the monotonic, quality-qualified timing needed to order fast events. Each function answers a different question.

No part number, memory density or security label proves that the assembled recorder works. Mounting, connectors, reset behavior, software scheduling and the extraction path remain system properties. The qualification target should name the intended impacts, supply interruptions and environmental exposures, then demonstrate retrieval after them. Surviving the enclosure test is useful only if the record still makes sense.

A valid signature can protect a wrong measurement

The security engineer verifies the event package. Its signature is valid. Nothing in the protected payload appears to have changed since sealing. That removes one uncertainty. It does not establish that the force sensor was calibrated, that the clock was synchronized or that the signing software received an honest account from the controller.

Evidence integrity must be described precisely. A checksum can help detect accidental corruption. A cryptographic authentication mechanism can detect unauthorized changes under its assumptions about keys and trusted code. Neither turns a faulty observation into physical truth. The record needs provenance and quality indicators as well as protection.

The proposed package should bind its data chunks to an event identity, configuration identity and recording policy. Sequence information can expose missing chunks; authenticated manifests can make later additions or removals detectable within the protected set. Encryption can limit disclosure, but encryption alone does not establish who produced the data or whether the set is complete.

Deletion is especially difficult. A signed file cannot complain after the entire file has disappeared. A protected event counter or independently retained receipt can help reveal a gap, subject to the survivability and trust limits of that mechanism. A counter increment without a recoverable record should produce an explicit missing-event condition, not a clean history.

Key management lasts longer than the incident. Authorized investigators may need to verify a record after a service replacement, certificate expiry or supplier exit. Retain the information needed to evaluate the signature at the relevant time, together with the extraction history. Recovery access should be controlled and documented without becoming a universal bypass.

NIST’s log-management guidance treats logging as an organizational process involving infrastructure and ongoing management. The same discipline is relevant here: collection, access, review and disposal cannot be left to a storage driver. The robot-specific protection scheme remains an engineering proposal, with its own threat model and assurance needs. NIST SP 800-92.

Authenticated extraction should produce a preserved original and clearly identified working copies. Transformations, redactions and interpretation software need traceability. An analyst should be able to distinguish the machine’s record from the investigator’s reconstruction and the manufacturer’s commentary. Merging them into one attractive report would erase exactly the boundaries that make the evidence useful.

The worker is not a telemetry channel

The worker at the transfer station has a different question. Was the robot recording her all morning? The engineering team may answer that it only needed the seconds around the incident. That answer is credible only if the system was actually designed to forget the rest.

Event-centered recording can reduce collection and retention, but a circular buffer is still processing data. Camera frames may identify people. Derived trajectories, task records or location histories can also become identifying when combined with workplace schedules. Replacing faces with object identifiers does not automatically make a dataset anonymous.

For personal data within its scope, the General Data Protection Regulation establishes principles including purpose limitation, data minimization, storage limitation and security. Article 6 separately addresses lawful bases for processing. A bounded buffer is an engineering measure, not a complete legal justification; the deployment needs an assessment of its actual processing and obligations. GDPR, Articles 5 and 6.

The proposed policy should distinguish transient buffering, sealed event retention and subsequent uses. An incident investigation does not automatically justify indefinite reuse for productivity scoring or model training. Each expansion changes who is exposed and what access is appropriate. A company cannot resolve that governance decision by calling the data technical.

Cameras and audio should therefore require a specific reconstruction need. Selected geometry, contact state and control traces may answer some questions with less exposure. Other incidents may genuinely require a bounded image sequence. Record the choice and its limits: deleting imagery may protect privacy while leaving a perception failure unresolved. Both consequences deserve an honest explanation.

Access should follow purpose. A maintenance technician may need electrical traces; an authorized investigator may require a broader sealed record. Defined retention periods, documented preservation holds and verifiable deletion processes can prevent event storage from becoming a permanent workplace archive. Workers need intelligible notice and a route to question how the system is used.

The trade-off cannot be eliminated by technology. A narrowly collected record may omit exculpatory context. A broad record may expose people who had nothing to do with the event. Good governance makes those decisions before an incident creates pressure to retain everything and before the strongest party gets to define what matters.

Replay can test an explanation, not certify it

With the surviving traces aligned, the investigation returns to the gripper. The accepted placement command preceded a current limit. The measured opening then changed before the protective stop. That sequence would narrow the investigation, but it would not by itself explain whether the object slipped, the mechanism bound or the estimated payload was wrong.

A replay environment can help compare those hypotheses. It needs the relevant software and model versions, controller parameters, calibration, coordinate transforms and initial state. A record that identifies only the application release may miss a drive firmware change or an updated payload parameter. Configuration identity should reference the actual deployed combination.

Deterministic replay is not guaranteed. Concurrency, asynchronous inputs, accelerator behavior and unrecorded state can change execution. Physical contact adds uncertainty about friction, compliance and geometry. Preserve random seeds where meaningful, but do not promise that a seed restores everything. The replay tool should state which conditions were reproduced and which were approximated.

Analysis should separate observed facts from derived estimates and counterfactuals. A recorded current limit is an observation about the drive. A calculated contact force is an estimate. A simulation in which a different grip succeeds is a counterfactual outcome under that model. Treating all three as equally direct evidence would give a visually persuasive reconstruction more authority than it deserves.

The same caution applies to fleet learning. Records from successful recoveries and near misses can reveal patterns, but trigger-based datasets are selected samples. If the recorder captures only falls, it cannot alone establish the frequency of unstable movements across all operating hours. Exposure information and recording-health data are needed before drawing rate comparisons.

An event recorder complements a digital shadow, a maintenance history or a robot passport by contributing a bounded, traceable incident record. It does not need to absorb those systems. The interfaces should let investigators relate the event to configuration and service history while keeping the sealed observations distinct from later annotations.

Buy a demonstrable capability to reconstruct events

A procurement team can ask a supplier whether its robot has logs and receive an immediate yes. A more useful request is to demonstrate what survives a specified failure. Start with an unexpected contact under controlled conditions, interrupt a communication path, remove power and ask an independent engineer to reconstruct the sequence.

The acceptance criteria should describe recovered evidence, not merely generated files. Did the pre-event window survive? Are gaps and timing uncertainty visible? Can the accepted command be linked to its measured response? Does the record identify its software and calibration? Can an authorized tool retrieve it without restoring the failed application?

Fault injection should exercise the recorder’s own weaknesses: storage near capacity, a second trigger during transfer, lost synchronization, a controller reset, interrupted writes and attempts to modify an extracted copy. Compare the recovered trace with independent test instrumentation. A self-consistent record can still omit the very interval in which its capture path failed.

Platform leadership also has to define degraded operation. If the recorder is unavailable, should a particular task continue, continue with restrictions or remain disabled? The answer depends on the application and its risk assessment. Recording loss is not automatically a universal stop condition, but silently carrying on would conceal the reduced investigative capability.

Responsibility also crosses commercial boundaries. The integrator should own the assembled evidence contract, including which supplier supplies each signal and who maintains its decoder. A controller vendor can provide a correct local trace while the complete robot still lacks the link to the originating task. Contracts should identify who can authorize extraction, how records move between organizations and how access continues after support ends. These agreements are easier to negotiate before an incident turns cooperation into a dispute.

One useful acceptance exercise is to give two teams the same preserved package and the same interpretation specification. They need not reach identical causal conclusions, but they should agree on units, configuration, missing intervals and the sequence the evidence actually supports. If that basic agreement depends on private guidance from the developer, the extraction interface is incomplete. A portable file format becomes useful evidence only when its semantics are portable too.

There is an uncomfortable authority question here. A system may restrict work because evidence preservation is unavailable, even though its motion controls appear healthy. That decision should come from a reviewed operating policy with accountable human ownership. The robot should not acquire an open-ended right to withhold service by treating every uncertain condition as grounds for intervention.

The business case is strongest where the evidence can change a decision: isolate an affected configuration, avoid replacing sound hardware, distinguish misuse from design weakness or identify a shared integration defect. These are potential benefits, not guaranteed savings. They depend on usable records, competent investigation and a willingness to act on inconvenient findings.

The right to challenge the machine

The component is still on the floor when the supervisor asks again about restarting. A trustworthy record may support a targeted intervention. It may also reveal that the decisive interval did not survive. The second answer is disappointing, but it is more responsible than assigning a cause the evidence cannot establish.

A robot’s black box should preserve that distinction. What was requested, what was accepted, what was commanded and what was measured belong in separate, connected records. So do missing data, uncertain timing and the conditions under which integrity was protected. The goal is a reconstruction that another competent party can challenge.

Physical AI will be judged partly by how it behaves when people cannot agree about its behavior. Manufacturers, operators, workers and investigators need access appropriate to their responsibilities, with protections against both tampering and unnecessary exposure. The company operating the machine should not hold an unquestionable monopoly over its account of an incident.

The robot may have done exactly what its systems permitted. The permissions may have been wrong. Only a preserved chain of evidence gives that distinction a fair hearing.