SmartBench/exoskeleton
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Archive calibration audit and freeze decisions

Browse Source
This commit is contained in:
xtkuang 2026-07-27 13:25:58 +08:00
parent 4503a12bf1
commit ea896b1012
5 changed files with 343 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@ -21,6 +21,8 @@ completed.
generation.
- `code/config/experiments/`: smoke, calibration, and locked-template study
specifications.
- `docs/calibration/`: calibration policy, traceable audit, and machine-readable
freeze decisions.
- `paper/exoskeleton/IEEEtran/main2.tex`: canonical manuscript source.
## Reproducible environment
@ -65,6 +67,12 @@ XDG_CACHE_HOME=/tmp/exoskeleton-xdg-cache \
Equivalent executor/config pairs are documented in
`code/experiments/README.md`.
The current calibration decision is recorded in
`docs/calibration/CALIBRATION_AUDIT_2026-07-27.md`. It deliberately leaves H1
and the bilateral gain/energy settings unfrozen; calibration values are not
manuscript Results. The companion formula-linked workbook is
`outputs/calibration-20260727/calibration_audit_2026-07-27.xlsx`.
## Manuscript build
Compile from `paper/exoskeleton` so the `assets/` paths resolve:

31
code/experiments/README.md
View File

@ -27,6 +27,20 @@ execute_h2_synthetic h2_smoke.json / h2_calibration.json
execute_bilateral_simulation smoke.json / bilateral_calibration.json
```
Auditable second-stage calibration specifications are:
```text
execute_h1_retargeting h1_calibration_v2.json
execute_h2_synthetic h2_calibration_v2.json
execute_bilateral_simulation bilateral_calibration_v2_energy.json
execute_bilateral_simulation bilateral_calibration_v2_network.json
```
The bilateral network specification is a gated Stage B template. Its
`requires_stage_a_selection` flag means the haptic parameters are placeholders;
do not execute it as a locked study until the energy/gain Stage A acceptance
gate has passed.
An executor callable receives one immutable trial mapping and returns:
```python
@ -56,10 +70,19 @@ python -m analysis.make_paper_artifacts \
--metric-config code/config/experiments/metrics_h1_calibration.json
```
Use `metrics_h2.json` for H2 batches and `metrics_bilateral.json` for bilateral
batches. The bilateral configuration derives H3 for every mapping/supervisor
condition, but its H4 table contains only the three tank-supervised methods;
PO/PC and bypass conditions cannot be silently mixed into a tank audit.
Use the matching independent metric configuration:
```text
h1_calibration.json metrics_h1_calibration.json
h1_calibration_v2.json metrics_h1_calibration_v2.json
h2_calibration*.json metrics_h2.json
bilateral_calibration.json metrics_bilateral.json
bilateral_calibration_v2_*.json metrics_bilateral_v2.json
```
The bilateral configurations derive H3 for every mapping/supervisor condition,
but their H4 tables contain only tank-supervised methods; PO/PC and bypass
conditions cannot be silently mixed into a tank audit.
The declared minimal storage contract is JSON for manifests/plans, NPZ for
numeric sample arrays, JSON Lines for events/trial metrics, and CSV for paper

233
docs/calibration/CALIBRATION_AUDIT_2026-07-27.md Normal file
View File

@ -0,0 +1,233 @@
# Calibration Audit — 2026-07-27
## Scope and decision rule
This audit covers pre-prototype numerical and rigid-body simulation evidence
only. It does not contain physical prototype, independent F/T, fixture, or
human-subject results, and none of its values may be copied into the manuscript
Results section.
The calibration code was first committed as
`4503a12bf10886902e69d4be3874a76d3be577d6`. All three reported v2 raw batches
record that commit with `dirty=false`. The independent metric code and configs
were tested with 98 passing tests and one explicitly unsupported legacy MuJoCo
demo skipped.
The decision terms **verified contract**, **provisional numerical setting**,
**frozen setting**, and **not freeze-ready** follow
`docs/calibration/CALIBRATION_POLICY.md`.
The companion formula-linked audit workbook is
`outputs/calibration-20260727/calibration_audit_2026-07-27.xlsx`. It contains
13 visually inspected worksheets and has SHA-256
`24df38f67ba1aa428928348561b452db181f963ed30ae18a570aabc4bd8ad920`.
## Traceable batches
| Study | Completed | Plan hash | Metric-config hash | Row hash | Source-data SHA-256 |
|---|---:|---|---|---|---|
| H1 calibration v2 | 60/60 | `e86b4aec90912954fdee97834dee08dd630647a89000a24402fa56f25b3aea83` | `2be0e67feda3f862c3510b3318af47437977c2b87a17ab810ce68a49212d5215` | `9e274b01db38c97112ecd1e7b50050d5c342cf316a22ea9b2d2c0ea4acf347d4` | `85110ad917d6b933656571b15c764b26d6f8215bea6d6baa338d759ed51fc0ee` |
| H2 calibration v2 | 576/576 | `87f797edb5d2d6378a42b95616fca8ed733d0c81de5e22c422b006ead7071b12` | `caaf8c3b1c8180eb19afb98d986e8a54b8a897bd1ab706f78e39c41441932a31` | `fcac031716e967d3832d8eb6f770f580bd68bb6c8f715d55d04b5878f86f0fbf` | `e3a9bc29003193b559fbfc595ce2595b19a29a7135b9f14057128a8c5f504710` |
| Bilateral energy/gain v2 Stage A | 36/36 | `c10fa2cd84392f9dbf62809b0960bfb38d1e7aac4791ef0345a7353c45d8e062` | `37ca1e72ae55eb2a4594ef8e6395302d76ebfea87c4bca43615e86b3eeb89ec2` | `3f3f7f09387f02fe88c1b070815bd108f7e6bf63a091861ae62c1fac1dcdf421` | H3 `baa82701ea4c3e116657b9ef4d12aa795849957934f9dc5d58d5811405936da7`; H4 `47a9cc05f22724edf078a525e31ace6332b6048ab36ed72a005a50e8ecb79453`; diagnostics `a765cb6cc87c8fc89849c4d0a5d2560a768769169caa625baf41ddfd18f7af63` |
All batches validated with zero failed trials and no validator warnings.
## H1 — SEW retargeting
### Design corrections verified
- Five trajectory strata are now separate: nominal, wider valid reach,
valid near-singularity, intentional upper-reach clipping, and master
joint-limit stress.
- Each stratum has three genuinely different seeded trajectory instances.
All four methods receive a bit-identical master trajectory within a pair.
The audit found one trajectory hash per pair and 15 unique hashes across the
15 pairs.
- Reach clipping, active limits, geometry degeneracy, low manipulability, pose
failure, and differential invalidity are distinct machine-readable fields.
No invalid sample in v2 had an unexplained reason.
### Calibration outcome
| Method | Trials with \(C_r=1\) | Main observation |
|---|---:|---|
| SEW | 6/15 | Passed nominal, wider valid reach, and valid near-singularity; failed the intentionally invalid reach-clip and joint-limit strata |
| Task-priority IK | 6/15 | Same trajectory-level composite outcomes as SEW in every pair |
| Bounded DLS IK | 6/15 | Same trajectory-level composite outcomes as SEW in every pair |
| Scaled joint-space map | 15/15 | Mapping branch remained numerically valid in several strata, but task-space error exceeded the locked calibration tolerances |
The paired difference \(C_{r,\mathrm{SEW}}-C_{r,\mathrm{taskIK}}\) was zero in
all 15 pairs. Every method had \(D_r=0\), including all valid trajectories.
Consequently, this calibration does not support the H1 superiority hypothesis
and cannot calibrate or freeze the discontinuity threshold.
The valid near-singularity stratum is no longer mixed with reach clipping:
SEW, task-priority IK, and bounded DLS had 100% valid samples and zero reach
clips; the SEW slave-output low-manipulability fraction was approximately
0.502. The intentional upper-reach-clip stratum had a clip fraction of 1.0
and `reach_clipped_upper` as the primary invalid reason. The joint-limit
stratum had no reach clipping and correctly reported `joint_limit_active`.
Runtime tails over all v2 samples were:
| Method | P50 | P95 | P99 | Maximum |
|---|---:|---:|---:|---:|
| Scaled joint-space | 0.140 ms | 0.159 ms | 0.235 ms | 0.475 ms |
| Bounded DLS IK | 0.191 ms | 1.693 ms | 1.760 ms | 5.156 ms |
| SEW | 0.845 ms | 10.290 ms | 12.007 ms | 15.643 ms |
| Task-priority IK | 0.322 ms | 10.139 ms | 10.423 ms | 10.713 ms |
These are workstation numerical timings, not a hardware real-time claim.
### H1 decision
**Not freeze-ready.** Before a locked H1 study:
1. add valid branch-transition and near-degeneracy trajectories that can
actually exercise \(D_r\);
2. declare a target control period and separate target construction,
recovery, and differential-map timing;
3. tune both SEW and the primary task-priority baseline only on a new
calibration split;
4. require a nonzero number of eligible increments in every continuity
stratum; and
5. rerun with enough independent trajectories for a precision-based paired
confidence interval.
## H2 — residual-wrench inversion
### Pairing and conditioning audit
The v2 plan contains 16 physical-data groups. Each group has 36
\(\ell_c\times\lambda\times\)method candidates and one unique seed hash.
Independent array hashes confirmed that the wrench reference, joint velocity,
truth Jacobian, estimator Jacobian, and sensor noise are identical across all
36 candidates within each group.
Numerical rank and operational ill-conditioning are now separate. No sample was
numerically rank deficient, but the minimum scaled singular value reached
approximately \(9.83\times10^{-4}\), and 39.9% of samples at the provisional
candidate were below the preregistered operational threshold.
### Parameter scan
For scaled DLS, trial-mean force/moment RMSE at the strongest candidates was:
| \(\ell_c\) (m) | \(\lambda\) | Force mean / max (N) | Moment mean / max (Nm) |
|---:|---:|---:|---:|
| 0.2 | 0.02 | 3.051 / 9.336 | 0.722 / 1.478 |
| 0.2 | 0.03 | 2.848 / 8.229 | 0.704 / 1.269 |
| 0.2 | 0.05 | **2.763 / 7.322** | 0.705 / **1.126** |
| 0.3 | 0.03 | 3.033 / 8.058 | 0.772 / 1.283 |
| 0.4 | 0.02 | 3.184 / 8.448 | 0.812 / 1.362 |
At \(\ell_c=0.2\) m and \(\lambda=0.05\):
| Method | Force mean / max (N) | Moment mean / max (Nm) |
|---|---:|---:|
| Scaled DLS | 2.763 / 7.322 | 0.705 / 1.126 |
| Undamped SVD | 6.064 / 25.502 | 1.248 / 4.281 |
| No bias correction | 3.132 / 7.512 | 0.786 / 1.140 |
| No friction correction | 2.965 / 7.091 | 0.759 / 1.153 |
The paired DLS-minus-undamped mean differences were \(-3.301\) N and
\(-0.544\) Nm, but DLS was better in only 10 of 16 physical-data groups. It
was deliberately worse in several well-modelled, low-noise cells and much
better in the low-singular/model-error tail. This is a bias--variance result,
not a universal accuracy result. The largest undamped retained outlier was
25.502 N / 4.281 Nm; it was not removed.
### H2 decision
**Provisional numerical setting:** use scaled DLS with
\(\ell_c=0.2\) m and \(\lambda=0.05\) only for the next synthetic pilot.
Keep \(\ell_c=0.2,\lambda=0.03\) and undamped SVD as sensitivity conditions.
**Not a frozen physical calibration.** The length scale must ultimately be
anchored to robot geometry and physical calibration, and H2 cannot support a
paper wrench-accuracy claim before independent six-axis F/T truth, TCP/F/T
transforms, torque conversion, payload/friction calibration, timestamp
alignment, and causal acceleration estimation are available.
## H3/H4 — bilateral mapping and final-port energy supervision
### Contracts already verified
The v1 stored-array audit verified:
- fixed-branch \(A\)-port virtual-work error at numerical precision;
- same-input proposed-versus-matched-wrench behavior, with their contact
normalized-power-mismatch difference approximately zero;
- 96/96 tank trials passing the independent final-applied-port audit;
- maximum preclip floor deficit of zero;
- maximum accounting and software-preclip error of
\(6.94\times10^{-18}\) J; and
- zero downstream torque modification after the audited projection.
These are verified implementation contracts, not H3/H4 performance
acceptance.
The v1 free-space power normalizer was only about \(2\times10^{-4}\) J, so its
normalized mismatch was not suitable as a primary endpoint. Metric schema v2
therefore always reports absolute mismatch and marks normalized H3 values
invalid below a 1 mJ activity gate.
### Stage A outcome
Stage A scanned feedback strength 0.2/0.35/0.5 and initial headroom
0.005/0.020/0.040 J at two wall stiffness levels.
- 36/36 independent H4 audits passed.
- Maximum accounting and software-preclip error was
\(1.39\times10^{-17}\) J.
- Downstream modification and projected floor deficit were both zero.
- \(D_{\mathrm{proj}}\) remained 0.756--0.905, with a mean of 0.837.
- Projection intervention occupied 0.328--0.531 of samples.
- Every trial reached the configured 80 N contact-force limit.
- Slave tracking RMSE was 0.807--1.035 rad.
The audit therefore verifies the final-port implementation, while the tested
contact envelope is unsuitable for selecting a transparency setting. Choosing
the numerically smallest \(D_{\mathrm{proj}}\) cell would merely freeze a
force-limited, poorly tracked trajectory.
### H3/H4 decision
- **Verified contract:** differential-dual virtual work and final-applied-port
energy accounting.
- **Not freeze-ready:** feedback strength, energy bounds, H4 distortion
threshold, contact envelope, network envelope, and H3 superiority margin.
- **Stage B not executed:** its configuration explicitly requires an accepted
Stage A profile. Running it with the placeholder middle cell would violate
the calibration policy.
The next bilateral calibration must first establish a stable fixture envelope
without force limiting or large tracking error, then introduce a separate,
controlled energy-challenging excitation. Contact stability and energy-budget
excitation must not be forced by the same aggressive trajectory. Only after
that gate passes should the directional network screen cover forward/return
delay, jitter, and loss.
## Freeze record
| Item | Decision | Permitted use now |
|---|---|---|
| H1 thresholds and SEW superiority margin | Not freeze-ready | trajectory/debug development only |
| H2 scaled DLS \(\ell_c=0.2\) m, \(\lambda=0.05\) | Provisional | next synthetic pilot only |
| \(A^\top\) virtual-work implementation | Verified contract | deterministic regression gate |
| Final-output energy accounting | Verified contract | deterministic regression gate |
| H3 normalized endpoint | Definition repaired; margin not frozen | use gated value plus absolute mismatch in calibration |
| Feedback gain and tank energy window | Not freeze-ready | continue calibration |
| Bilateral Stage B network matrix | Blocked by Stage A selection gate | do not execute as a locked or confirmatory study |
## Required next execution order
1. Redesign and rerun H1 continuity trajectories.
2. Run the H2 provisional setting on a larger held-out synthetic pilot while
preserving the 16-group pairing contract.
3. Establish a stable, unsaturated fixture/contact envelope.
4. Design a distinct bounded energy-challenging excitation and rerun Stage A.
5. Replace Stage B placeholder haptic values only after Stage A passes.
6. Build the prototype measurement chain and complete physical calibration.
7. Freeze new `locked` JSON plans, metric configs, margins, exclusions, and a
clean source commit before collecting confirmatory data.

75
docs/calibration/calibration_decision_2026-07-27.json Normal file
View File

@ -0,0 +1,75 @@
{
"kind": "calibration_decision",
"decision_date": "2026-07-27",
"scope": "pre-prototype numerical and rigid-body simulation only",
"raw_source_commit": "4503a12bf10886902e69d4be3874a76d3be577d6",
"paper_results_authorized": false,
"audit_artifact": {
"path": "outputs/calibration-20260727/calibration_audit_2026-07-27.xlsx",
"sha256": "24df38f67ba1aa428928348561b452db181f963ed30ae18a570aabc4bd8ad920",
"worksheets": 13,
"formula_error_count": 0,
"visual_inspection_completed": true
},
"batches": {
"h1_v2": {
"path": "output/experiments/h1-calibration-v2-clean",
"completed_trials": 60,
"failed_trials": 0,
"plan_hash": "e86b4aec90912954fdee97834dee08dd630647a89000a24402fa56f25b3aea83",
"metric_configuration_hash": "2be0e67feda3f862c3510b3318af47437977c2b87a17ab810ce68a49212d5215",
"row_hash": "9e274b01db38c97112ecd1e7b50050d5c342cf316a22ea9b2d2c0ea4acf347d4",
"source_data_sha256": "85110ad917d6b933656571b15c764b26d6f8215bea6d6baa338d759ed51fc0ee"
},
"h2_v2": {
"path": "output/experiments/h2-calibration-v2-clean",
"completed_trials": 576,
"failed_trials": 0,
"plan_hash": "87f797edb5d2d6378a42b95616fca8ed733d0c81de5e22c422b006ead7071b12",
"metric_configuration_hash": "caaf8c3b1c8180eb19afb98d986e8a54b8a897bd1ab706f78e39c41441932a31",
"row_hash": "fcac031716e967d3832d8eb6f770f580bd68bb6c8f715d55d04b5878f86f0fbf",
"source_data_sha256": "e3a9bc29003193b559fbfc595ce2595b19a29a7135b9f14057128a8c5f504710"
},
"bilateral_stage_a_v2": {
"path": "output/experiments/bilateral-calibration-v2-energy-clean",
"completed_trials": 36,
"failed_trials": 0,
"plan_hash": "c10fa2cd84392f9dbf62809b0960bfb38d1e7aac4791ef0345a7353c45d8e062",
"metric_configuration_hash": "37ca1e72ae55eb2a4594ef8e6395302d76ebfea87c4bca43615e86b3eeb89ec2",
"row_hash": "3f3f7f09387f02fe88c1b070815bd108f7e6bf63a091861ae62c1fac1dcdf421",
"source_data_sha256": {
"h3": "baa82701ea4c3e116657b9ef4d12aa795849957934f9dc5d58d5811405936da7",
"h4": "47a9cc05f22724edf078a525e31ace6332b6048ab36ed72a005a50e8ecb79453",
"bilateral_diagnostics": "a765cb6cc87c8fc89849c4d0a5d2560a768769169caa625baf41ddfd18f7af63"
}
}
},
"decisions": {
"h1": {
"status": "not_freeze_ready",
"reason": "SEW and primary task-priority IK tied on C_r in all 15 pairs and no valid trajectory produced D_r=1",
"next_gate": "valid branch-transition trajectories plus a declared timing budget"
},
"h2": {
"status": "provisional_numerical_setting",
"method": "scaled_dls",
"characteristic_length_m": 0.2,
"damping": 0.05,
"permitted_use": "next synthetic pilot only",
"physical_claim_authorized": false
},
"h3_virtual_work": {
"status": "verified_contract",
"performance_margin_frozen": false
},
"h4_final_port_accounting": {
"status": "verified_contract",
"gain_and_energy_window_frozen": false
},
"bilateral_stage_a": {
"status": "not_freeze_ready",
"reason": "all cells reached the 80 N force limit and had 0.807-1.035 rad slave tracking RMSE",
"stage_b_authorized": false
}
}
}

BIN
outputs/calibration-20260727/calibration_audit_2026-07-27.xlsx Normal file
View File

Binary file not shown.