PIPELINE
Deployment data pipeline

From live failure to training-ready bundle.

This demo shows the part most labs still build themselves: synchronized capture, failure isolation, recovery segmentation, packaging, and export in formats the training stack can actually ingest.

Interactive walk-through

Four stages. One deployment session.

Use the controls to step through what happens after the robot starts operating. The right-hand panels stay anchored on the same example session and update as the pipeline advances.

Stage 01

Capture

Continuous synchronized session logging.

Stage 02

Failure slice

Pin the exact failure window and state.

Stage 03

Recovery cut

Isolate the corrective trajectory cleanly.

Stage 04

Export

Bundle it for retraining and eval.

Capture

Every rollout is logged as a continuous, synchronized session.

Before anything fails, the pipeline has already aligned the streams that matter: camera feeds, robot state, task ID, environment metadata, and operator notes. That makes later segmentation possible without manual reconstruction.

  • Wall-clock and robot-clock alignment on ingest
  • Per-site metadata attached at session start
  • Task-level markers inserted before rollout begins
Captured streams cam_front.mp4 · cam_wrist.mp4 · joints.parquet · session.json

The raw layer is still retained, but the pipeline immediately organizes it around the rollout rather than leaving it as an unlabeled dump.

Why it matters

If time sync is wrong, recovery data is noisy. The value of the final correction depends on capturing the pre-failure context, the actual error event, and the human intervention in one coherent timeline.

Failure slice

The pipeline narrows a full rollout into the exact failure interval.

Once the robot slips, stalls, collides, or diverges, the system clips around the event and records a failure code. This is where a long session stops being generic runtime footage and starts becoming usable supervision.

  • Failure type tagged against a stable taxonomy
  • Pre-failure buffer preserved for causal context
  • Operator handoff timestamp attached to the event
Window extracted -8.0s to +5.2s around GRASP_SLIP

The clip includes the lead-in, not only the bad frame, because labs need to see how the policy approached failure as much as the failure itself.

Eval benefit

The same failure slice becomes evaluation infrastructure if it is tagged consistently across sites. Ashmyr can compare how policies fail before and after retraining, not just whether they fail.

Recovery cut

The operator correction is isolated as its own trajectory.

This is the highest-signal unit in the demo. Instead of handing back “robot failed here,” the pipeline extracts the expert intervention that got the task back on track and pairs it with the failure state that triggered it.

  • Recovery start and stop markers inserted explicitly
  • Corrective actions preserved separately from background runtime
  • Session context retained so the cutout is not decontextualized
Recovery package recovery_clip.mp4 · recovery_joints.parquet · failure_state.json

Labs can train on the corrective snippet without losing the session context that explains why the intervention was needed.

Ashmyr role

The value is not only the intervention itself. It is the discipline of cutting it cleanly, tagging it consistently, and delivering it in a reusable format across many deployments and many sites.

Export

The final output is a training and evaluation bundle, not an ops artifact.

At export time, the session has already been structured enough to hand directly into a lab’s post-training or evaluation workflow. The point is to reduce the gap between “the robot failed” and “the training stack can use this.”

  • Manifest summarizes session, failure, and recovery metadata
  • Files are grouped by rollout and recovery unit
  • Output can map to RLDS-like or lab-specific schemas
Bundle status Ready for post-training and field eval replay

The same package can support retraining, failure analysis, and cross-site robustness tracking if the export contract stays stable.

What labs avoid

No manual log stitching, no ad hoc file naming, no guessing which frames align with the human correction. The packaging work is already done.

Current bundle

Deployment session `ASM-052`

Raw synchronized session with site metadata, task markers, and aligned streams ready for failure slicing.

Session file

session_manifest.json

Task, site, shift, operator, and synchronized timestamp map.

Video

cam_front.mp4 / cam_wrist.mp4

Continuous RGB capture of the live rollout.

Robot state

joints.parquet

Joint positions and task markers over the full session.

Live session log

Example deployment feed

DEPLOYMENT PIPELINE
Rollouts 0
Failures 0
Exports 0
Why this matters

Pipeline outputs for model teams

These metrics are illustrative, but they show what changes when the deployment loop is structured instead of bespoke.

Segmentation 13.2s

Failure window isolated from a 51-second rollout.

Recovery cut 8.2s

Corrective intervention preserved as its own unit.

Artifacts 7 files

Exported in one bundle instead of scattered raw logs.

Schema Stable

Consistent enough to compare runs across sites.

This is what your deployment data looks like.
Ready to get started?

Schedule a conversation abenezer@ashmyr.com