cmvr-es/docs/teleoperation/ume_cmvr_validation.md

119 lines
6.3 KiB
Markdown
Raw Permalink Normal View History

# UME / CMVR-ES validation gates
This checklist is part of the first UME migration. Passing a software gate
does not authorize a physical power-on. The checked-in UME device entries and
their `hardware_enabled` fields remain `false`.
The current revision has no production setpoint producer for
`UmeTeleopTask::submitSetpoint()` and no haptic consumer for returned follower
effort. M9 therefore validates the framework, protocol and migrated original
algorithm separately; it is not an end-to-end motion or force-feedback test.
## M9: software and network validation
Run these gates on both target CPU architectures before deploying:
1. Build the complete `cmvr_es` target with tests enabled.
2. Run the UME legacy controller and Pinocchio model-adapter golden tests.
3. Run the Damiao codec, CAN-FD chain, and `UmeRobotArm` lifecycle tests.
4. Run the process-wide control-authority tests.
5. Run the gRPC client, `UmeTeleopTask`, and `ArmTeleopService` tests.
6. Repeat the concurrent client/task/service tests to screen for shutdown and
reconnect races.
7. Start each checked-in edge profile without UME hardware and verify that it
never opens `can4`/`can5` or issues actuator enable frames.
The communication tests must demonstrate all of the following:
- an `OpenSession` manifest mismatch is rejected before backend activation;
- a second controller cannot acquire the same robot control resource;
- sequence numbers are non-zero and strictly increasing per session;
- the sender and receiver use capacity-one, latest-only command storage;
- a setpoint whose local validity has expired is never dispatched;
- heartbeat loss, lease loss, stream cancellation, and backend failure call
the robot safe-stop boundary;
- reconnect clears pending motion intent and starts a new sequence space;
- `StopSession` is attempted before client cancellation;
- the legacy unary ArmService cannot issue motion, enable, calibration, or
fault-reset commands while the teleoperation lease is active;
- `torqueOff` and `stopMotion` remain available as safety overrides.
Before a real follower backend can be enabled, control authority must also be
extended to every local arm task and to the underlying MotorService resources.
The current process-wide lease covers ArmTeleop and unary ArmService only.
For a wired two-computer run, record at least:
- one-way command age at the robot ingress;
- command mailbox overwrite and rejection counters;
- heartbeat and control-lease remaining time;
- servo apply duration and deadline misses;
- disconnect detection-to-safe-stop time;
- packet loss, reordering, and delay from an explicit network impairment
profile rather than an assumed LAN condition.
No end-to-end stability or transparency claim is supported until those logs
are tied to a specified controller rate, robot servo period, payload, motion
envelope, and network impairment profile.
## M10: staged physical commissioning
Every row is a separate sign-off. Do not combine first power-on with a human
wearing the UME.
- [ ] Independent physical emergency stop is installed and verified.
- [ ] CAN arbitration/data bitrates and CAN-FD+BRS MTU are verified for each
interface.
- [ ] Motor product, firmware, command ID, feedback ID, and reported motor ID
are read back and matched to the configuration.
- [ ] The four-bit Damiao feedback status meanings and raw temperature limits
are verified for the exact product/firmware and entered as an explicit
per-joint whitelist/threshold contract.
- [ ] Joint direction and zero reference are verified one joint at a time with
the mechanism unloaded.
- [ ] Mechanical position, velocity, and torque limits replace the checked-in
placeholders and receive an independent review.
- [ ] Motor feedback timestamps and the 800 Hz cycle are measured on the UME
target computer under load.
- [ ] The exact Damiao firmware's Disable acknowledgement semantics are
documented and verified. Until then, SocketCAN send success is only
evidence that the local kernel accepted the complete frame batch.
- [ ] Because MIT feedback has no sequence field, stale request/reply
correlation is resolved by a reviewed firmware marker or by measured,
enforced bus timing; draining only the frames already queued is not
sufficient evidence.
- [ ] Every UME control-thread I/O operation is shown to be deadline-bounded
and interruptible on the target kernel. The configured shutdown timeout
is currently a diagnostic failure threshold, not a C++ timed-join
primitive.
- [ ] With torque disabled, both edge profiles run for at least 30 minutes
without sequence, deadline, lease, or reconnect anomalies.
- [ ] With the UME fixed to a stand, each joint is enabled independently at a
low torque limit and its watchdog disable path is measured.
- [ ] Both UME arms are tested together on stands; CAN and CPU deadline margins
are recorded.
- [ ] The robot backend's group `servoJ` semantics and worst-case call duration
are measured. Sequential per-joint dispatch is not accepted as an
atomic group backend without a documented skew bound.
- [ ] gRPC writer backpressure cannot stall the independent robot watchdog,
and an expired lease cannot be regranted until safe stop is confirmed.
- [ ] TouchScreenTask and direct MotorService commands are either disabled by
the deployment profile or participate in the same resource authority.
- [ ] Robot-only low-speed setpoint execution is validated before connecting
the leader-side algorithm.
- [ ] Wired-network cable removal, peer process kill, delayed packets, stale
commands, duplicate sequences, lease theft, and robot fault injection
all lead to a bounded safe stop.
- [ ] The original UME gravity/friction/feedback controller is commissioned on
a stand with force feedback initially clamped to zero, then increased in
reviewed steps.
- [ ] Only after all previous evidence is archived may a supervised human test
be considered under a separate risk assessment.
## Evidence record
For each completed physical gate, archive the exact Git revision, installed
`output/` checksum, configuration files, model and calibration SHA-256 values,
test operator, hardware serial numbers, raw logs, and pass/fail decision. A
successful build or simulator run must not be recorded as physical validation.