add detect phone

This commit is contained in:
xtkuang 2026-07-21 16:23:33 +08:00
parent ab42c6d21a
commit 76189302a9
27 changed files with 1264 additions and 130 deletions

View File

@ -10,17 +10,18 @@
- 版本化插件注册表、检测模型注册表和 Python entry point 扩展机制;
- cmvr-es RGB 相机 gRPC Source带退避上限的指数重连、AGV gRPC Sink
- PyAV H264/H265 有状态解码、通用检测模型节点和重复命中规则节点;
- Construction PPE YOLOv8 与六类 PPE YOLOv8n 模型注册,以及按标签、置信度和最大 FPS 的部署配置;
- Construction PPE、六类 PPE YOLOv8n、People-Talking YOLOv8x 与 YOLOv8n Mobile Phone 模型注册,以及按标签、置信度和最大 FPS 的部署配置;
- 平台 HTTP JSON Sink支持有限重试、`raise/log_and_drop` 失败策略,并使用告警
`event_id` 作为幂等键;
- `RobotCommand -> ApprovedRobotCommand` 安全门和无重试的类型化 AGV 命令映射;
- 文本/JSON 日志、共享 gRPC Channel 与 HTTP Client
- 可直接执行的最小测试 fixture以及包含 PPE 检测和对话占位链路的统一部署配置。
- 可直接执行的最小测试 fixture以及包含 PPE/电话使用检测和对话占位链路的统一部署配置。
`configs/edge_ai.yaml` 是统一部署配置,其中同时定义 `detection``talk` 两个
Pipeline。`detection` 当前只启用 Construction PPE 模型,并经过时间窗口规则向 8081
上报告警。六类 PPE 模型及第二 HTTP 平台的实现仍保留在注册表和连接器中,但不在当前
Pipeline 图中实例化,因此不会加载第二份权重、执行第二次推理或访问 8082。
Pipeline。`detection` 从同一个相机 decoder 并行运行 Construction PPE 与
People-Talking 两个 detector各自经过独立时间窗口规则后向同一个告警平台上报
`model_id``rule_id` 用于区分事件。六类 PPE 模型仍保留在注册表中,但不在当前
Pipeline 图中实例化。
VAD/ASR/LLM/TTS 尚未内置;`talk` 仍使用模拟音频数据,等待 cmvr-es 音频双向流
proto 落地。
@ -49,14 +50,16 @@ cmvr_edge_ai/
├── configs/
│ ├── edge_ai.yaml # detection + talk 统一部署配置
│ └── debug/
│ └── detection_viewer.yaml # 远端相机 -> PPE 检测 -> 本地画框窗口
│ └── detection_viewer.yaml # 远端相机 -> PPE+电话检测 -> 合并画框窗口
├── detect_server/
│ ├── README.md # PPE 检测链路与 Viewer 使用说明
│ ├── README.md # PPE/电话检测链路与 Viewer 使用说明
│ └── show_detections.py # OpenCV 实时检测结果 Demo
├── models/
│ └── detection/ # 按模型 ID/版本组织的检测模型制品库
│ ├── construction-ppe-yolov8/v1/ # best.pt + 独立 model card
│ └── ppe-6classes-yolov8n/v1/ # best.pt + 独立 model card
│ ├── ppe-6classes-yolov8n/v1/ # best.pt + 独立 model card
│ ├── people-talking-yolov8x/v1/ # best.pt + 独立 model card
│ └── yolov8n-mobile-phone/ # Viewer 手机目标检测权重 + model card
├── talk_server/
│ └── nodes/ # 对话插件预留目录
├── scripts/
@ -160,7 +163,7 @@ CLI 的四个子命令如下:
链路;需要同进程运行两条链路时,可以重复传两个 `--pipeline`。如果完全省略
`--pipeline`,运行时会启动配置中所有 `enabled: true` 的 Pipeline。
## 运行 PPE 检测链路
## 运行视觉检测链路
默认 bootstrap 就是当前 YAML 使用的 CPU 检测环境。它会从相邻的
`../cmvr-es` 读取 proto、用锁定的 `grpcio-tools` 生成 bindings然后安装完整
@ -221,6 +224,20 @@ pipelines:
alert_image:
enabled: true
jpeg_quality: 85
phone_detector:
with:
model: people-talking-yolov8x@1
detect_labels: [talking on phone]
max_fps: 5
attach_frame: true
model_options:
weights: models/detection/people-talking-yolov8x/v1/best.pt
device: cpu
phone_repeat_gate:
with:
alert_image:
enabled: true
jpeg_quality: 85
alert_platform:
with:
endpoint: ppe_alert_platform
@ -240,16 +257,18 @@ uv run --no-sync cmvr-edge-ai run --config configs/edge_ai.yaml \
--log-format json
```
启动后detector 会先输出一条 `detection model loaded`,表示对应权重已经成功
加载。收到解码帧并完成真实 `predict` 后,会立即输出第一条 `detection inference`,之后
启动后,`detector` 和 `phone_detector` 会分别输出一条 `detection model loaded`,表示
两份权重已经成功加载。收到解码帧并完成真实 `predict` 后,会分别输出
`detection inference`,之后
`inference_log_interval_s` 聚合输出一次;其中 `window_frames` 是本周期推理帧数,
`window_detections` 是检测框总数,`hit_labels` 是各标签的检测框累计数。持续只有 loaded 而没有
inference说明相机或 decoder 尚未把帧送到模型inference 中 detection 为 0 只表示
当前阈值下没有命中。短时调试可设为 `1` 秒,生产环境可设为 `30``60` 秒,省略则关闭
周期推理日志。这里使用标准日志而不是裸 `print`,因此与 `--log-format json` 兼容。
如果需要直接观察每次推理对应的画框图像,使用独立的 OpenCV Demo。它连接同一个
cmvr-es gRPC CameraService但不经过重复触发规则也不会访问 HTTP 平台:
如果需要直接观察两个模型在同一帧上的画框结果,使用独立的 OpenCV Demo。它连接同一个
cmvr-es gRPC CameraService共享解码帧并行执行 PPE 与 YOLOv8n Mobile Phone再把两个结果
合并到一个窗口;它不经过重复触发规则,也不会访问 HTTP 平台:
```bash
uv run --no-sync python detect_server/show_detections.py \
@ -258,8 +277,10 @@ uv run --no-sync python detect_server/show_detections.py \
--log-level INFO
```
运行前在 `configs/debug/detection_viewer.yaml` 中配置远端地址、`device_id` 和模型权重;按 `q``Esc`
退出。详细说明见 [detect_server/README.md](detect_server/README.md#实时画框-demo)。
运行前在 `configs/debug/detection_viewer.yaml` 中配置远端地址、`device_id` 和两份模型
权重;按 `q``Esc` 退出。框标签使用 `PPE:`/`PHONE:` 前缀,手机框标签为
`PHONE:mobile_phone`,窗口刷新率由较慢的模型分支决定。详细说明见
[detect_server/README.md](detect_server/README.md#实时画框-demo)。
相机连接器会在每次首次连接或重连时先发 `CameraService.StartCamera`,收到成功反馈后
才建立 `GetRGBImageStream`。终端会依次出现 `camera start requested/succeeded`
@ -267,24 +288,28 @@ uv run --no-sync python detect_server/show_detections.py \
如果只有 opening 而没有 first frameprogress 中会持续显示
`first_frame_received=false window_frames=0`用于区分“RPC 已建立但相机没有出帧”。
运行前需要确认 Construction PPE 权重存在、cmvr-es 已启用 `right_hand_cam`8081 平台的
运行前需要确认 Construction PPE 与 People-Talking 权重存在、cmvr-es 已启用 `right_hand_cam`8081 平台的
`/v1/detection-alerts` 可访问。默认 `yolo-cpu` profile 将 PyTorch 2.7.0 和
torchvision 0.22.0 绑定到官方 CPU wheel并固定 checkpoint 记录的 Ultralytics
8.4.31。`model_options.device` 在该环境中应保持 `cpu``half` 应保持 `false`
8.4.31;同时显式安装 People-Talking checkpoint 反序列化所需的 `dill`
`model_options.device` 在该环境中应保持 `cpu``half` 应保持 `false`
x86 CUDA 和 Jetson/JetPack 的 PyTorch wheel 与驱动强绑定,不能复用这个 CPU
profile。项目保留了不绑定 CPU index 的 `yolo` extra 作为设备专用环境的基础,但
GPU 部署前应为目标设备建立单独的 uv source/lock或使用 NVIDIA 容器),再把 YAML
中的 `device` 改为 `cuda:0`;不要只改 YAML 就认为 CUDA 环境已经就绪。
`detection.model@1` 根据 `model``DetectionModelRegistry` 解析模型。`detect_labels` 只选择需要检测的标签,省略时检测注册模型的全部标签;`confidence` 是全局阈值,也可以用 `label_confidence` 为个别标签覆盖。当前内置 `construction-ppe-yolov8@1` 的 19 个标签和 `ppe-6classes-yolov8n@1` 的 6 个标签都可以通过 `cmvr-edge-ai models` 查看。两个模型制品和独立 model card 位于仓库内:
`detection.model@1` 根据 `model``DetectionModelRegistry` 解析模型。`detect_labels` 只选择需要检测的标签,省略时检测注册模型的全部标签;`confidence` 是全局阈值,也可以用 `label_confidence` 为个别标签覆盖。当前四个内置模型都可以通过 `cmvr-edge-ai models` 查看,模型制品和独立 model card 位于仓库内:
- [Construction PPE YOLOv8 v1](models/detection/construction-ppe-yolov8/v1/README.md):包含正向 PPE、`No-*` 违规类和施工现场设备类;
- [PPE YOLOv8n 6 Classes v1](models/detection/ppe-6classes-yolov8n/v1/README.md):轻量的六类正向装备检测模型。
- [People Talking YOLOv8x v1](models/detection/people-talking-yolov8x/v1/README.md)电话使用行为检测Pipeline 只选择 `talking on phone` 类。
- [YOLOv8n Mobile Phone v1](models/detection/yolov8n-mobile-phone/README.md)单类手机目标检测Debug Viewer 使用 `mobile_phone` 类。
注册 ID 中的 `@1` 与制品目录的 `v1` 对应;这是项目的版本组织约定,实际
`weights` 路径仍由部署 YAML 显式指定。标签顺序、训练指标、局限和许可声明以上述
model card 为准。`attach_frame: true` 让检测结果临时携带对应的解码帧,供后续告警节点使用;因此原 detector 到 repeat gate 的队列应保持较小,避免堆积未压缩图像。
注册 ID 中的 `@1` 表示版本 1多数制品使用 `v1` 子目录,当前 YOLOv8n Mobile
Phone 保留导入时的扁平目录和上游文件名。实际 `weights` 路径仍由部署 YAML 显式
指定。标签顺序、训练指标、局限和许可声明以上述 model card 为准。
`attach_frame: true` 让检测结果临时携带对应的解码帧,供后续告警节点使用;因此原 detector 到 repeat gate 的队列应保持较小,避免堆积未压缩图像。
六类模型的标签是 `Gloves`、`Vest`、`goggles`、`helmet`、`mask` 和 `safety_shoe`
语义是“画面中检测到了该装备”,不是“人员缺少该装备”。它没有 `Person``No-*`
@ -292,8 +317,9 @@ model card 为准。`attach_frame: true` 让检测结果临时携带对应的解
注册、未被 `configs/edge_ai.yaml``detection` Pipeline 引用;需要恢复第二分支时,应同时配置 detector、
8082 endpoint、HTTP Sink 和两条关联 edge。
若以后恢复双模型配置,应从 decoder 输出端口 fan-out让两个 detector 共享同一个相机
订阅和 PyAV decoder两个模型仍会分别加载和推理并共享应用的有界线程池。
当前 Construction PPE 和 People-Talking 分支从 decoder 输出端口 fan-out共享同一个
相机订阅和 PyAV decoder两个模型仍会分别加载和推理并共享应用的有界线程池。
People-Talking 是视觉行为判断,不能证明电话已接通,也不能区分来电和去电。
`detection.repeat_gate@1` 只在一个规则的 `window_ms` 内看到至少 `min_hits` 个不同帧后生成 `DetectionAlert/v1`。同一帧有多个同类框仍只算一次;触发后进入 `cooldown_ms`,冷却期间不累计,结束后必须重新满足次数。`scope: source` 按相机统计;`scope: track` 按 `track_id` 统计,但当前 YOLO adapter 只做逐帧检测,不产生 `track_id`,因此使用 track 规则前必须增加跟踪/关联节点。

View File

@ -1,8 +1,8 @@
api_version: cmvr.edge.ai/v1
runtime:
# Decoder, YOLO and viewer drawing share this bounded application pool.
thread_workers: 3
# Decoder, two YOLO models and viewer drawing share this bounded pool.
thread_workers: 4
shutdown_timeout_s: 8
endpoints:
@ -33,6 +33,11 @@ pipelines:
decoder:
uses: media.video_decoder.pyav@1
frame_fanout:
# One shared fan-out node chooses the latest decoded frame before both
# blocking detector queues. This keeps the two model inputs identical.
uses: core.passthrough@1
detector:
uses: detection.model@1
with:
@ -46,8 +51,6 @@ pipelines:
- No-Mask
- No-Vest
confidence: 0.50
# A 30 FPS stream starts at most 10 model inferences each second.
max_fps: 10
inference_log_interval_s: 5
# Required so the viewer receives the exact decoded inference frame.
attach_frame: true
@ -60,16 +63,36 @@ pipelines:
half: false
max_det: 100
phone_detector:
uses: detection.model@1
with:
model: yolov8n-mobile-phone@1
detect_labels:
- mobile_phone
confidence: 0.50
inference_log_interval_s: 5
attach_frame: true
model_options:
weights: models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt
device: cpu
imgsz: 640
iou: 0.70
half: false
max_det: 100
viewer:
uses: demo.opencv_detection_viewer@1
with:
window_name: CMVR PPE Detection
window_name: CMVR PPE + Phone Detection
window_width: 1280
window_height: 720
wait_key_ms: 1
box_thickness: 2
font_scale: 0.6
show_stats: true
# Render only after both results for the exact same ImageFrame arrive.
expected_inputs: [ppe, phone]
max_pending_frames: 8
edges:
# H264/H265 packets must remain contiguous until decoding.
@ -80,18 +103,42 @@ pipelines:
capacity: 8
overflow: block
# Keep only the latest decoded frame while YOLO is busy.
# Drop only before the shared fan-out, so both models see the same
# surviving decoded frame rather than independently dropping frames.
- from: decoder.frames
to: detector.frames
to: frame_fanout.input
qos:
profile: realtime_latest
capacity: 1
overflow: drop_oldest
# A slow GUI must not accumulate raw frames or stale detection results.
- from: detector.detections
to: viewer.input
# These two blocking edges preserve the same frame set and ordering for
# both models. Do not add independent detector max_fps limits here.
- from: frame_fanout.output
to: detector.frames
qos:
profile: realtime_latest
profile: telemetry
capacity: 1
overflow: drop_oldest
overflow: block
- from: frame_fanout.output
to: phone_detector.frames
qos:
profile: telemetry
capacity: 1
overflow: block
# The viewer joins both result streams by frame identity before drawing.
- from: detector.detections
to: viewer.ppe
qos:
profile: telemetry
capacity: 2
overflow: block
- from: phone_detector.detections
to: viewer.phone
qos:
profile: telemetry
capacity: 2
overflow: block

View File

@ -1,7 +1,8 @@
api_version: cmvr.edge.ai/v1
runtime:
thread_workers: 3
# PyAV plus the PPE and phone-use YOLO branches share this bounded pool.
thread_workers: 4
shutdown_timeout_s: 8
endpoints:
@ -16,7 +17,8 @@ endpoints:
ppe_alert_platform:
transport: http
# Violation-alert platform HTTP base URL.
# Detection-alert platform HTTP base URL. The alert payload model_id and
# rule_id distinguish PPE violations from phone-use violations.
base_url: http://192.168.0.222:13080
timeout_s: 3
@ -134,6 +136,45 @@ pipelines:
min_confidence: 0.50
scope: source
# phone_detector:
# uses: detection.model@1
# with:
# model: people-talking-yolov8x@1
# # The checkpoint also contains a generic class named "label". It is
# # retained in model registration for class-ID safety but is not an
# # actionable phone-use event, so this branch selects only class 1.
# detect_labels:
# - talking on phone
# confidence: 0.50
# # YOLOv8x is substantially heavier than the PPE model. Start with a
# # conservative CPU rate and tune only after measuring target hardware.
# max_fps: 5
# inference_log_interval_s: 5
# attach_frame: true
# model_options:
# weights: models/detection/people-talking-yolov8x/v1/best.pt
# device: cpu
# imgsz: 640
# iou: 0.70
# half: false
# max_det: 100
# phone_repeat_gate:
# uses: detection.repeat_gate@1
# with:
# time_source: received
# alert_image:
# enabled: true
# jpeg_quality: 85
# rules:
# - id: talking-on-phone
# labels: [talking on phone]
# min_hits: 3
# window_ms: 2000
# cooldown_ms: 30000
# min_confidence: 0.50
# scope: source
alert_platform:
uses: platform.http_json_sink@1
with:
@ -163,6 +204,15 @@ pipelines:
capacity: 1
overflow: drop_oldest
# Fan out the already decoded image; do not open a second camera stream
# or decode the same H264/H265 packet twice.
- from: decoder.frames
to: phone_detector.frames
qos:
profile: realtime_latest
capacity: 1
overflow: drop_oldest
- from: detector.detections
to: repeat_gate.detections
qos:
@ -173,6 +223,14 @@ pipelines:
capacity: 2
overflow: drop_oldest
# - from: phone_detector.detections
# to: phone_repeat_gate.detections
# qos:
# profile: telemetry
# # attach_frame carries the decoded image until the rule is evaluated.
# capacity: 2
# overflow: drop_oldest
- from: repeat_gate.alerts
to: alert_platform.input
qos:
@ -180,6 +238,13 @@ pipelines:
capacity: 64
overflow: block
# - from: phone_repeat_gate.alerts
# to: alert_platform.input
# qos:
# profile: telemetry
# capacity: 64
# overflow: block
talk:
enabled: true
nodes:

View File

@ -1,17 +1,18 @@
# PPE 检测流水线
# PPE 与电话使用检测流水线
`configs/edge_ai.yaml` 中的 `detection` Pipeline 是当前可运行的园区施工安全装备检测
链路。它与 `talk` Pipeline 共用一个部署 YAML当前只启用 Construction PPE 模型和
8081 告警平台;六类模型与 8082 模拟平台的实现保留但不实例化:
`configs/edge_ai.yaml` 中的 `detection` Pipeline 是当前可运行的园区视觉告警链路。
它与 `talk` Pipeline 共用一个部署 YAML并从同一个 decoder 并行运行 Construction
PPE 与 People-Talking 两个模型。两个 detector 使用独立 repeat gate再把带有不同
`model_id`/`rule_id` 的告警交给同一个平台 Sink
```text
cmvr-es CameraService
-> H264/H265 ImageFrame/v1
-> media.video_decoder.pyav@1
-> BGR8 ImageFrame/v1
-> construction-ppe-yolov8@1 -> DetectionResult/v1
-> detection.repeat_gate@1 -> DetectionAlert/v1
-> POST 127.0.0.1:8081/v1/detection-alerts
|-> construction-ppe-yolov8@1 -> PPE repeat gate --|
`-> people-talking-yolov8x@1 -> phone repeat gate --+-> DetectionAlert/v1
`-> POST /v1/detection-alerts
```
## 安装与启动
@ -32,8 +33,9 @@ bash scripts/bootstrap.sh \
```
环境固定 Python 3.10、Ultralytics 8.4.31、PyTorch 2.7.0 CPU 和 torchvision
0.22.0,并通过 `image` extra 安装 Pillow 以生成告警图片;所有具体包版本记录在
仓库的 `uv.lock` 中。GPU/Jetson 需要单独适配与驱动或 JetPack 匹配的 PyTorch
0.22.0,通过 `image` extra 安装 Pillow 以生成告警图片,并显式安装 People-Talking
checkpoint 反序列化所需的 `dill`;所有具体包版本记录在仓库的 `uv.lock` 中。
GPU/Jetson 需要单独适配与驱动或 JetPack 匹配的 PyTorch
wheel不能直接复用 `detection-cpu` profile。手动组合依赖时必须显式增加
`--extra image`,不能只依赖 YOLO 间接安装 Pillow。
@ -65,6 +67,20 @@ pipelines:
alert_image:
enabled: true
jpeg_quality: 85
phone_detector:
with:
model: people-talking-yolov8x@1
detect_labels: [talking on phone]
max_fps: 5
attach_frame: true
model_options:
weights: models/detection/people-talking-yolov8x/v1/best.pt
device: cpu
phone_repeat_gate:
with:
alert_image:
enabled: true
jpeg_quality: 85
alert_platform:
with:
endpoint: ppe_alert_platform
@ -105,7 +121,7 @@ uv run --no-sync cmvr-edge-ai run -c configs/edge_ai.yaml \
- 短时观察可把 `inference_log_interval_s` 改为 `1` 秒,长期运行建议 `30``60` 秒,
省略该字段会关闭周期推理日志。
运行前确认 cmvr-es 已启用 `right_hand_cam`Construction PPE 权重存在,并且 8081
运行前确认 cmvr-es 已启用 `right_hand_cam`两份生产权重存在,并且 8081
平台接受 `POST /v1/detection-alerts`。默认 profile 下 `model_options.device` 应设为
`cpu`;只有完成设备专用的 CUDA/Jetson PyTorch 环境适配后,才能改为 `cuda:0` 等值。
@ -115,7 +131,10 @@ uv run --no-sync cmvr-edge-ai run -c configs/edge_ai.yaml \
独立调试链路:
```text
cmvr-es CameraService -> PyAV decoder -> YOLO detector -> OpenCV window
cmvr-es CameraService -> PyAV decoder -> latest decoded frame
|-> Construction PPE --|
`-> YOLOv8n Mobile ----+-> same-frame join
-> OpenCV window
```
先编辑 `configs/debug/detection_viewer.yaml` 中的远端 cmvr-es 地址、相机 ID 和权重路径:
@ -133,15 +152,22 @@ pipelines:
device_id: wrist_cam
detector:
with:
max_fps: 10
model_options:
weights: /absolute/path/to/best.pt
weights: models/detection/construction-ppe-yolov8/v1/best.pt
device: cpu
phone_detector:
with:
model: yolov8n-mobile-phone@1
detect_labels: [mobile_phone]
model_options:
weights: models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt
device: cpu
```
`max_fps: 10` 表示最多每秒执行 10 次推理;视频流更快时,中间的已解码帧通过
`realtime_latest + drop_oldest` 丢弃,以保持低延迟。`attach_frame: true` 已在 Demo 配置
中启用viewer 因而能拿到与本次推理严格对应的原图并绘制 bounding box。
Demo 只在共享 fan-out 之前通过 `realtime_latest + drop_oldest` 选择最新解码帧,之后用
两条 `block` 边把同一个 `ImageFrame` 交给两个 detector。两个 detector 都不设置独立
`max_fps`避免各自跳过不同帧。Viewer 按 frame identity 等待两个结果配齐,再在同一
张图上绘制;因此预览吞吐由较慢的模型分支决定。
只检查配置和插件连线,不连接相机、不加载模型、也不创建窗口:
@ -161,10 +187,11 @@ uv run --no-sync python detect_server/show_detections.py \
--log-format json
```
相机 Source 会先调用 `StartCamera`,成功后再建立 gRPC 视频流。窗口显示每个实际推理
结果,即使本帧没有检测框也会刷新;按 `q`、`Q`、`Esc` 或关闭窗口可安全退出。这个
Demo 直接订阅 detector 输出,刻意绕过 `repeat_gate` 和 HTTP Sink因此只用于观察模型
效果,不代表某条告警规则已满足。
相机 Source 会先调用 `StartCamera`,成功后再建立 gRPC 视频流。窗口只在同一帧的两个
推理结果都到达后刷新PPE 框显示为 `PPE:<label>`,手机框显示为
`PHONE:mobile_phone`,顶部同时显示两模型的框数和推理耗时。即使两模型都没有框,
配对完成后也会刷新。按 `q`、`Q`、`Esc` 或关闭窗口可安全退出。这个 Demo 绕过
`repeat_gate` 和 HTTP Sink因此只用于观察模型效果不代表某条告警规则已满足。
OpenCV 窗口出现在运行命令的机器上。无桌面的边缘设备不能直接显示;通过 SSH 运行时
需要启用 X11 转发并确保 `DISPLAY` 可用,否则程序会给出明确错误并退出。依赖缺失时先
@ -174,23 +201,38 @@ OpenCV 窗口出现在运行命令的机器上。无桌面的边缘设备不能
`detection.model@1` 不绑定某一个框架;它通过 `DetectionModelRegistry` 查找配置中的
`model`。每个 `DetectionModelSpec` 注册版本化模型 ID、模型名称、backend 和有序
`supported_labels`。注册表当前包含两个内置模型,但 Pipeline 只引用第一个:
`supported_labels`。注册表当前包含四个内置模型:生产 Pipeline 引用 Construction PPE
与 People-TalkingDebug Viewer 则并行运行 Construction PPE 与 YOLOv8n Mobile Phone
- [`construction-ppe-yolov8@1`](../models/detection/construction-ppe-yolov8/v1/README.md)
对应制品目录 `v1`19 类,包含用于违规告警的 `No-*` 标签;
- [`ppe-6classes-yolov8n@1`](../models/detection/ppe-6classes-yolov8n/v1/README.md)
对应制品目录 `v1`,包含 `Gloves`、`Vest`、`goggles`、`helmet`、`mask`、
`safety_shoe` 六个正向装备标签。
- [`people-talking-yolov8x@1`](../models/detection/people-talking-yolov8x/v1/README.md)
对应制品目录 `v1`checkpoint 标签为 `label`、`talking on phone`;生产配置只选择
语义明确的 `talking on phone`
- [`yolov8n-mobile-phone@1`](../models/detection/yolov8n-mobile-phone/README.md)
checkpoint 单类标签为 `mobile_phone`Debug Viewer 用它显示画面中的手机物体。
模型 ID 尾部的 `@1` 与制品目录的 `v1` 对应;运行时不会根据 ID 自动拼接文件
路径,仍由 YAML 中的 `model_options.weights` 显式指定。训练信息、完整标签
顺序、性能、限制和许可信息请查看各自的 model card。
模型 ID 尾部的 `@1` 表示注册版本 1多数制品使用 `v1` 子目录,当前
YOLOv8n Mobile Phone 保留导入时的扁平目录和上游文件名。运行时不会根据 ID 自动
拼接路径,仍由 YAML 中的 `model_options.weights` 显式指定。训练信息、完整标签顺序、
性能、限制和许可信息请查看各自的 model card。
可用 `cmvr-edge-ai models` 核对 ID、名称、backend 和标签顺序。六类模型只表达
“检测到某件装备”,不包含 `Person``No-*` 类,也不执行人员/PPE 关联;所以它
不能直接判断某个人缺少装备。需要这种语义时,仍应增加人员检测、空间关联和缺失
判定节点,不能把“没有检测到 helmet”直接当成“人员未戴安全帽”。
People-Talking 是纯视觉检测:它只能判断画面是否像“正在用手机通话”,不能读取真实
通话状态,不能区分接听/拨出,也不能证明电话已经接通。上游 checkpoint 中含义不清的
`label` 类为了保持 class ID 对齐仍在注册表中,但不会进入当前告警规则。
YOLOv8n Mobile Phone 只检测手机物体,不能据此判断人员正在接打电话。合成全黑图的
本地 smoke test 也出现过高置信度误报,因此 Viewer 默认阈值只是起点,应使用真实相机
画面校准 `confidence`
检测节点的关键参数:
| 参数 | 语义 |
@ -324,8 +366,9 @@ H264/H265 解码;这不会复用模型计算,两个 YOLO 实例仍会分别
`HTTP report dropped ... action=drop` WARNING丢弃当前告警并继续处理后续视频帧。
`CancelledError`、序列化错误和未知本地程序异常不会被吞掉。
当前唯一 HTTP 输出将告警发送到
`http://127.0.0.1:8081/v1/detection-alerts`。8082 六类逐推理结果 endpoint 当前未配置。
当前唯一 HTTP Sink 将 PPE 和电话使用告警发送到
`http://127.0.0.1:8081/v1/detection-alerts`payload 中的 `model_id``rule_id`
用于区分来源。8082 六类逐推理结果 endpoint 当前未配置。
启用告警图片后POST JSON 的 `payload.image` 结构如下:

View File

@ -46,6 +46,7 @@ _DEFAULT_CONFIG = (
/ "detection_viewer.yaml"
)
_VIEWER_PLUGIN_ID = "demo.opencv_detection_viewer@1"
_VIEWER_INPUT_PORTS = ("input", "ppe", "phone")
class DetectionViewerError(RuntimeError):
@ -53,7 +54,7 @@ class DetectionViewerError(RuntimeError):
class OpenCvDetectionViewerSink(Sink):
"""Render ``DetectionResult`` boxes and show the corresponding source frame."""
"""Pair same-frame model results and render their boxes in one window."""
_PARAM_KEYS = frozenset(
{
@ -64,6 +65,8 @@ class OpenCvDetectionViewerSink(Sink):
"box_thickness",
"font_scale",
"show_stats",
"expected_inputs",
"max_pending_frames",
}
)
@ -123,6 +126,16 @@ class OpenCvDetectionViewerSink(Sink):
params.get("show_stats", True),
"show_stats",
)
self._expected_inputs = _input_ports(
params.get("expected_inputs", ["input"])
)
self._expected_input_set = frozenset(self._expected_inputs)
self._max_pending_frames = _bounded_int(
params.get("max_pending_frames", 8),
"max_pending_frames",
minimum=1,
maximum=256,
)
self._request_stop = request_stop
self._module_loader = module_loader
self._executor: Executor | None = None
@ -135,6 +148,11 @@ class OpenCvDetectionViewerSink(Sink):
self._frames_shown = 0
self._last_frame_at: float | None = None
self._display_fps: float | None = None
self._pending_results: dict[
tuple[str, str, str | None, int, int],
dict[str, DetectionResult],
] = {}
self._pending_evicted = 0
async def setup(self, context: ComponentContext) -> None:
executor = context.metadata.get("thread_executor")
@ -208,11 +226,16 @@ class OpenCvDetectionViewerSink(Sink):
envelope: Envelope[Any],
input_port: str = "input",
) -> None:
del input_port
if self._stop_requested:
return
if not self._window_open or self._cv2 is None or self._numpy is None:
raise RuntimeError("detection viewer has not been set up")
if input_port not in self._expected_input_set:
expected = ", ".join(self._expected_inputs)
raise DetectionViewerError(
f"{self._node_id} received unexpected input port "
f"{input_port!r}; configured expected_inputs: {expected}"
)
result = envelope.payload
if not isinstance(result, DetectionResult):
raise TypeError(
@ -226,6 +249,58 @@ class OpenCvDetectionViewerSink(Sink):
"attach_frame: true in the demo config"
)
if len(self._expected_inputs) == 1:
await self._display_results(((input_port, result),))
return
frame_key = (
envelope.trace_id,
envelope.source_id,
envelope.session_id,
envelope.sequence,
id(frame),
)
pending = self._pending_results.setdefault(frame_key, {})
if input_port in pending:
raise DetectionViewerError(
f"duplicate result for input port {input_port!r} and frame "
f"source={envelope.source_id!r} sequence={envelope.sequence}"
)
pending[input_port] = result
if self._expected_input_set.issubset(pending):
self._pending_results.pop(frame_key, None)
await self._display_results(
tuple((port, pending[port]) for port in self._expected_inputs)
)
return
while len(self._pending_results) > self._max_pending_frames:
oldest_key = next(iter(self._pending_results))
dropped = self._pending_results.pop(oldest_key)
self._pending_evicted += 1
_LOGGER.warning(
"detection viewer evicted unmatched frame node=%s "
"source=%s sequence=%s received_inputs=%s evicted_total=%s",
self._node_id,
oldest_key[1],
oldest_key[3],
",".join(sorted(dropped)),
self._pending_evicted,
)
async def _display_results(
self,
results: tuple[tuple[str, DetectionResult], ...],
) -> None:
frame = results[0][1].source_frame
if frame is None: # guarded in consume; retained for type narrowing
raise DetectionViewerError("DetectionResult has no source_frame")
if any(result.source_frame is not frame for _, result in results[1:]):
raise DetectionViewerError(
"paired detection results do not reference the same source "
"frame; keep the shared decoded-frame fan-out in the demo graph"
)
now = monotonic()
if self._last_frame_at is not None and now > self._last_frame_at:
instantaneous_fps = 1.0 / (now - self._last_frame_at)
@ -238,16 +313,45 @@ class OpenCvDetectionViewerSink(Sink):
header = None
if self._show_stats:
display_fps = self._display_fps or 0.0
model_name = result.model_name or result.model_id
header = (
f"{model_name} | boxes={len(result.detections)} | "
f"inference={result.inference_ms:.1f} ms | display={display_fps:.1f} FPS"
)
if len(results) == 1:
result = results[0][1]
model_name = result.model_name or result.model_id
header = (
f"{model_name} | boxes={len(result.detections)} | "
f"inference={result.inference_ms:.1f} ms | "
f"display={display_fps:.1f} FPS"
)
else:
model_stats = " | ".join(
f"{port.upper()} boxes={len(result.detections)} "
f"{result.inference_ms:.1f}ms"
for port, result in results
)
header = f"{model_stats} | display={display_fps:.1f} FPS"
detections: list[Detection] = []
for port, result in results:
for detection in result.detections:
if not isinstance(detection, Detection):
raise DetectionViewerError(
"DetectionResult.detections must contain Detection instances"
)
if len(results) == 1:
detections.append(detection)
else:
detections.append(
Detection(
label=f"{port.upper()}:{detection.label}",
confidence=detection.confidence,
box=detection.box,
track_id=detection.track_id,
)
)
image = await run_blocking(
_render_detection_frame,
frame,
result.detections,
tuple(detections),
cv2_module=self._cv2,
numpy_module=self._numpy,
box_thickness=self._box_thickness,
@ -259,6 +363,7 @@ class OpenCvDetectionViewerSink(Sink):
self._frames_shown += 1
async def stop(self) -> None:
self._pending_results.clear()
if not self._window_open:
return
self._window_open = False
@ -276,9 +381,11 @@ class OpenCvDetectionViewerSink(Sink):
exc_info=True,
)
_LOGGER.info(
"detection viewer stopped node=%s frames_shown=%s",
"detection viewer stopped node=%s frames_shown=%s "
"unmatched_evicted=%s",
self._node_id,
self._frames_shown,
self._pending_evicted,
)
async def _pump_window_events(self) -> None:
@ -541,6 +648,23 @@ def _strict_bool(value: Any, name: str) -> bool:
return value
def _input_ports(value: Any) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)):
raise ValueError("expected_inputs must be a list of input port names")
ports = tuple(_non_empty_string(item, "expected_inputs item") for item in value)
if not ports:
raise ValueError("expected_inputs must contain at least one input port")
if len(set(ports)) != len(ports):
raise ValueError("expected_inputs must not contain duplicate ports")
unknown = sorted(set(ports) - set(_VIEWER_INPUT_PORTS))
if unknown:
raise ValueError(
"expected_inputs contains unsupported viewer port(s): "
+ ", ".join(unknown)
)
return ports
def _build_registry(request_stop: Callable[[], None]) -> PluginRegistry:
model_registry = create_default_model_registry()
registry = create_default_registry(model_registry=model_registry)
@ -553,8 +677,14 @@ def _build_registry(request_stop: Callable[[], None]) -> PluginRegistry:
params,
request_stop=request_stop,
),
inputs={"input": "DetectionResult/v1"},
description="Show detection source frames with bounding boxes in OpenCV",
inputs={
"input": "DetectionResult/v1",
"ppe": "DetectionResult/v1",
"phone": "DetectionResult/v1",
},
description=(
"Pair same-frame model results and show their boxes in OpenCV"
),
tags=frozenset({"demo", "visualization"}),
)
)

View File

@ -342,7 +342,7 @@ with:
| `detection.model@1` | `ImageFrame/v1 -> DetectionResult/v1` | 加载一个已注册模型,在线程池中推理,按部署标签和阈值二次过滤;可附带对应解码帧 |
| `detection.repeat_gate@1` | `DetectionResult/v1 -> DetectionAlert/v1` | 按不同帧、时间窗口、scope 和 cooldown 把逐帧检测转换为平台告警,仅在规则触发时按需画框并编码 JPEG |
核心包不会强制安装大型视觉运行时。使用解码器安装 `video` extra告警图的画框和 JPEG 编码由 Pillow 提供,安装独立的 `image` extra内置 YOLO adapter 提供通用 `yolo` 和固定 CPU wheel 的 `yolo-cpu` 两个互斥 extra。当前 PPE CPU 链路通过 `bash scripts/bootstrap.sh` 一键生成 cmvr-es bindings并按 `uv.lock` 安装 `grpc + http + video + image + yolo-cpu`。开发环境使用 `bash scripts/bootstrap.sh --profile dev`。PyAV、Pillow 和 Ultralytics 都延迟导入,因此不运行对应检测能力的对话服务不会加载它们。不要依赖 Ultralytics 间接带入 Pillow手动组合告警图片环境时必须显式选择 `--extra image`。CUDA/Jetson 必须按目标驱动或 JetPack 建立独立依赖源和锁文件。
核心包不会强制安装大型视觉运行时。使用解码器安装 `video` extra告警图的画框和 JPEG 编码由 Pillow 提供,安装独立的 `image` extra内置 YOLO adapter 提供通用 `yolo` 和固定 CPU wheel 的 `yolo-cpu` 两个互斥 extra。当前 PPE/电话使用检测 CPU 链路通过 `bash scripts/bootstrap.sh` 一键生成 cmvr-es bindings并按 `uv.lock` 安装 `grpc + http + video + image + yolo-cpu`。`yolo` 与 `yolo-cpu` 都显式包含 People-Talking checkpoint 反序列化所需的 `dill`。开发环境使用 `bash scripts/bootstrap.sh --profile dev`。PyAV、Pillow 和 Ultralytics 都延迟导入,因此不运行对应检测能力的对话服务不会加载它们。不要依赖 Ultralytics 间接带入 Pillow手动组合告警图片环境时必须显式选择 `--extra image`。CUDA/Jetson 必须按目标驱动或 JetPack 建立独立依赖源和锁文件。
具体模型不再各自注册一套 DAG 插件。`DetectionModelRegistry` 保存 `DetectionModelSpec`
@ -452,7 +452,7 @@ with:
规则达到阈值并准备发出告警时绘制 bounding boxes、标签和置信度并编码 JPEG
不会为每个 `DetectionResult` 产生图片。
`scope: source` 按相机和规则隔离状态;`scope: track` 还按 `track_id` 隔离,同一 track ID 在不同相机之间不会混合。没有 track ID 的检测会被 track 规则忽略。当前 Construction PPE 告警分支必须使用 `source`;要判断同一个人,需要先增加 tracker 和人员/PPE 关联节点。
`scope: source` 按相机和规则隔离状态;`scope: track` 还按 `track_id` 隔离,同一 track ID 在不同相机之间不会混合。没有 track ID 的检测会被 track 规则忽略。当前 Construction PPE 和 People-Talking 告警分支都使用 `source`;要判断同一个人,需要先增加 tracker 和人员/检测框关联节点。
每次触发创建 `DetectionAlert.event_id`。告警的 `detections` 和图片中的 bounding boxes 只来自达到 `min_hits` 的阈值帧,以控制内存和 HTTP payload窗口内更早帧只参与 `labels/max_confidence/first_seen/last_seen/hit_count` 的聚合。内部可选图片为 JPEG bytesHTTP wire 中 `payload.image` 被序列化为扁平对象:

View File

@ -37,12 +37,9 @@ construction-ppe-yolov8@1
## 大文件管理
Detection 的 `.pt` 权重使用 Git LFS。克隆仓库后若权重尚未下载执行
```bash
git lfs install
git lfs pull
```
Detection 的 `.pt` 权重直接作为普通 Git blob 保存在仓库中,不使用 Git LFS。克隆
仓库后即可获得完整权重,不需要执行 `git lfs pull`。新增大权重前应同时确认 Git
服务及反向代理允许对应的 push 大小和上传时长。
将新权重加入仓库前,先核对模型卡中的 SHA256

View File

@ -10,6 +10,8 @@
|---|---|---|---:|---|
| `construction-ppe-yolov8@1` | [construction-ppe-yolov8/v1](construction-ppe-yolov8/v1/README.md) | `ultralytics-yolo` | 19 | PPE、PPE 缺失违规及部分现场设备 |
| `ppe-6classes-yolov8n@1` | [ppe-6classes-yolov8n/v1](ppe-6classes-yolov8n/v1/README.md) | `ultralytics-yolo` | 6 | 画面中实际出现的六类 PPE |
| `people-talking-yolov8x@1` | [people-talking-yolov8x/v1](people-talking-yolov8x/v1/README.md) | `ultralytics-yolo` | 2 | 上游通用 `label` 类及 `talking on phone` 行为类 |
| `yolov8n-mobile-phone@1` | [yolov8n-mobile-phone](yolov8n-mobile-phone/README.md) | `ultralytics-yolo` | 1 | 画面中的 `mobile_phone` 手机物体 |
可通过以下命令查看运行时注册信息及有序标签:
@ -34,12 +36,21 @@ nodes:
模型加载时会严格比较 checkpoint 的类别名称和顺序与注册信息。权重不匹配时节点会停止
启动,不能通过只修改 `detect_labels` 绕过类别校验。
当前生产 Pipeline 会从同一个 decoder 并行运行 Construction PPE 与 People-Talking
两个 detector。People-Talking checkpoint 的 class 0 名称是含义不清的 `label`;注册
表为保证 ID 对齐而保留它,但部署配置只选择 `talking on phone`
`configs/debug/detection_viewer.yaml` 不改变生产告警链路,它使用
`yolov8n-mobile-phone@1` 替代 People-Talking与 Construction PPE 同帧显示。该模型
只说明画面中检测到手机物体,不能直接判断人员正在通话。
## 新增版本
新增 Detection 模型或模型版本时:
1. 新建独立的 `<model-name>/v<version>/` 目录;
2. 放入 Git LFS 管理的权重并编写完整模型卡;
1. 新建独立的 `<model-name>/v<version>/` 目录;已有的
`yolov8n-mobile-phone@1` 保留上游文件名和扁平目录,后续版本应恢复这一约定;
2. 把完整权重作为普通 Git 文件提交,并编写完整模型卡;
3. 使用 SHA256 校验权重来源和复制结果;
4. 在 `DetectionModelRegistry` 中注册唯一的 `<model-name>@<version>`
5. 保证 `supported_labels` 与 checkpoint 类别编号严格同序;

View File

@ -0,0 +1,101 @@
# People Talking YOLOv8x v1
## 注册信息
| 字段 | 值 |
|---|---|
| Model ID | `people-talking-yolov8x@1` |
| 注册名称 | `People Talking YOLOv8x` |
| Backend | `ultralytics-yolo` |
| 任务 | 画面中人员正在打电话的行为检测 |
| 权重 | `models/detection/people-talking-yolov8x/v1/best.pt` |
| 权重格式 | PyTorch/Ultralytics `.pt` checkpoint |
| 权重大小 | `136694313` bytes |
| SHA256 | `86cd63926de1c69f70dd7f8755e4de9e382bbdeb0668821b7c3846b826471526` |
| 基础模型 | YOLOv8x约 6815 万参数) |
| checkpoint Ultralytics | `8.0.196` |
| 当前部署 Ultralytics | `8.4.31` |
Model ID 的 `@1` 与本目录的 `v1` 对应。权重从用户提供的
`/home/xtkuang/Projects/cmvr/changan_robot/call_phone/best.pt` 复制;复制前后的大小和
SHA256 必须一致。不兼容的新权重应使用新版本目录和注册 ID不能覆盖本文件记录的
`v1`
## 有序标签
checkpoint 的类别编号必须与下表严格一致:
| ID | 标签 | 部署用途 |
|---:|---|---|
| 0 | `label` | 上游 checkpoint 中含义不清的通用类别;当前 Pipeline 不选择 |
| 1 | `talking on phone` | 电话使用行为告警 |
`label` 不是背景类,不能从注册信息中删除,否则类别编号会错位并导致模型加载失败。
生产配置通过 `detect_labels: [talking on phone]` 只请求 class 1。
## 输入与运行参数
- 输入必须是已经解码的 `BGR8``RGB8` 三通道图像;
- checkpoint 训练参数记录的 `imgsz``640`
- 当前 CPU 部署从 `max_fps: 5`、`confidence: 0.50` 开始;
- 该权重由带 `dill` 的环境序列化,项目的 `yolo`/`yolo-cpu` extra 已显式包含
`dill`,不能依赖 Ultralytics 运行时临时联网安装;
- `.pt` 由 PyTorch pickle loader 读取,只应部署经过校验的可信制品。
配置示例:
```yaml
phone_detector:
uses: detection.model@1
with:
model: people-talking-yolov8x@1
detect_labels:
- talking on phone
confidence: 0.50
max_fps: 5
attach_frame: true
model_options:
weights: models/detection/people-talking-yolov8x/v1/best.pt
device: cpu
imgsz: 640
iou: 0.70
half: false
max_det: 100
```
## 来源与许可
- 上游项目:[`Anidipta/People-Talking`](https://github.com/Anidipta/People-Talking)
- 上游仓库根 `LICENSE` 标记为 Apache-2.0,但 README 的许可段落写为 MIT
- 上游没有在模型卡中单独说明训练数据和权重的再分发许可。
因此在外部发布或商业部署前,需要进一步确认代码、权重和训练数据各自的授权范围,
不能把仓库 README 的一句许可描述直接当成全部资产的授权结论。仓库中的 `.pt` 作为
普通 Git blob 保存,克隆后会直接获得完整权重。
## 限制
- 这是视觉行为检测,不读取手机屏幕、通话音频或运营商状态;它不能判断来电/去电、
是否真正接通,也不能区分普通手持手机与真实通话的全部边界情况;
- 上游只提供曲线图片,未给出可复核的逐类 AP/precision/recall 数值;当前没有把
未知测试集上的指标写成部署承诺;
- 手部和手机被遮挡、远距离小目标、侧后视角、弱光以及相似手势都可能导致漏检或误检;
- 当前 adapter 是逐帧检测,不跟踪具体人员。`repeat_gate` 的 `scope: source` 只能表示
同一相机连续命中,不能证明连续命中来自同一个人;
- YOLOv8x 在 CPU 上较重,必须在目标设备测量推理延迟、内存和实际 FPS 后再调高
`max_fps`
## 验证
从仓库根目录执行:
```bash
sha256sum models/detection/people-talking-yolov8x/v1/best.pt
uv run --no-sync cmvr-edge-ai models
uv run --no-sync cmvr-edge-ai validate \
--config configs/edge_ai.yaml \
--pipeline detection
```
`validate` 只检查配置和 DAG不加载权重。发布前还必须执行真实模型加载和单帧推理
smoke test确认日志或测试结果中模型 ID、标签顺序和推理输出均正确。

Binary file not shown.

View File

@ -0,0 +1,70 @@
# YOLOv8n Mobile Phone v1
这是 Debug OpenCV Viewer 使用的单类手机目标检测模型。它来自
[`IndUSV/yolov8n-mobile-phone`](https://huggingface.co/IndUSV/yolov8n-mobile-phone)
用于检测画面中可见的手机物体。
| 字段 | 值 |
|---|---|
| Model ID | `yolov8n-mobile-phone@1` |
| 注册名称 | `YOLOv8n Mobile Phone` |
| Backend | `ultralytics-yolo` |
| 基础模型 | YOLOv8n |
| 输入尺寸 | 640 × 640 |
| 权重 | `models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt` |
| 文件大小 | 6,234,666 bytes |
| SHA256 | `9230e4bfa7cba7134215c4c7f228b5e58760b02138788ccaff0149258c2d2e19` |
| Checkpoint Ultralytics 版本 | 8.4.12 |
| 上游模型卡许可 | MIT |
## 标签
checkpoint 只有一个类别,注册顺序必须与 class ID 完全一致:
| Class ID | 标签 | 语义 |
|---:|---|---|
| 0 | `mobile_phone` | 画面中可见的手机物体 |
## Viewer 配置
```yaml
phone_detector:
uses: detection.model@1
with:
model: yolov8n-mobile-phone@1
detect_labels: [mobile_phone]
confidence: 0.50
attach_frame: true
model_options:
weights: models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt
device: cpu
imgsz: 640
iou: 0.70
half: false
max_det: 100
```
## 限制
- 这是手机物体检测器,不是接打电话行为识别器。检测到手机不能证明人员正在通话,
也不能区分接听、拨出或电话是否接通。
- 本地 smoke test 中640 × 640 全黑合成图在 `confidence: 0.50` 下仍出现过误报;
该结果不代表真实画面精度,但说明上线前必须用目标相机数据校准阈值。
- 遮挡、小目标、反光、屏幕和外形相近物体都可能造成漏检或误检。
- 上游模型卡未给出可直接用于本项目验收的量化指标;部署效果应以现场数据回放为准。
## 来源与校验
上游模型卡说明该模型基于 YOLOv8n在 Datacluster Labs Mobile Phone Image Dataset
上训练,单类标签为 `mobile_phone`,输入尺寸为 640 × 640并标注 MIT 许可。运行时及
其依赖仍需分别遵守各自许可。
```bash
sha256sum models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt
uv run --no-sync cmvr-edge-ai models
uv run --no-sync python detect_server/show_detections.py \
--config configs/debug/detection_viewer.yaml \
--validate-only
```

View File

@ -28,10 +28,12 @@ image = [
"Pillow>=10,<13",
]
yolo = [
"dill>=0.3.8,<1",
"numpy>=1.24,<3",
"ultralytics==8.4.31",
]
yolo-cpu = [
"dill>=0.3.8,<1",
"numpy>=1.24,<3",
"torch==2.7.0",
"torchvision==0.22.0",

View File

@ -23,8 +23,8 @@ verify_model_artifact() {
fi
if LC_ALL=C grep -q -F "version https://git-lfs.github.com/spec/v1" "${artifact_path}"; then
echo "error: model artifact is only a Git LFS pointer: ${relative_path}" >&2
echo "run 'git lfs pull' to fetch the real model weights" >&2
echo "error: model artifact is an unexpected Git LFS pointer: ${relative_path}" >&2
echo "recommit the complete weight as a regular Git file" >&2
exit 1
fi
@ -197,11 +197,21 @@ if [[ "${needs_cmvr_bindings}" == true ]]; then
"models/detection/ppe-6classes-yolov8n/v1/best.pt" \
"5625014" \
"07172ef3ae9e256c40a1fb0ce3eefe5547d90170645aa73dded0fffc382cdb31"
verify_model_artifact \
"people-talking-yolov8x@1" \
"models/detection/people-talking-yolov8x/v1/best.pt" \
"136694313" \
"86cd63926de1c69f70dd7f8755e4de9e382bbdeb0668821b7c3846b826471526"
verify_model_artifact \
"yolov8n-mobile-phone@1" \
"models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt" \
"6234666" \
"9230e4bfa7cba7134215c4c7f228b5e58760b02138788ccaff0149258c2d2e19"
echo "==> checking detection runtime imports"
.venv/bin/python -c \
"import av, grpc, httpx, PIL, torch, ultralytics; import cmvr.api.camera_service_pb2_grpc; print(f'torch={torch.__version__} cuda={torch.cuda.is_available()} ultralytics={ultralytics.__version__} pillow={PIL.__version__}')"
echo "==> validating the PPE detection pipeline"
"import av, dill, grpc, httpx, PIL, torch, ultralytics; import cmvr.api.camera_service_pb2_grpc; print(f'torch={torch.__version__} cuda={torch.cuda.is_available()} ultralytics={ultralytics.__version__} dill={dill.__version__} pillow={PIL.__version__}')"
echo "==> validating the PPE and phone-use detection pipeline"
.venv/bin/cmvr-edge-ai validate \
--config configs/edge_ai.yaml \
--pipeline detection

View File

@ -16,9 +16,11 @@ Requires-Dist: av<17,>=12; extra == "video"
Provides-Extra: image
Requires-Dist: Pillow<13,>=10; extra == "image"
Provides-Extra: yolo
Requires-Dist: dill<1,>=0.3.8; extra == "yolo"
Requires-Dist: numpy<3,>=1.24; extra == "yolo"
Requires-Dist: ultralytics==8.4.31; extra == "yolo"
Provides-Extra: yolo-cpu
Requires-Dist: dill<1,>=0.3.8; extra == "yolo-cpu"
Requires-Dist: numpy<3,>=1.24; extra == "yolo-cpu"
Requires-Dist: torch==2.7.0; extra == "yolo-cpu"
Requires-Dist: torchvision==0.22.0; extra == "yolo-cpu"
@ -36,13 +38,20 @@ Requires-Dist: ultralytics==8.4.31; extra == "yolo-cpu"
- 版本化插件注册表、检测模型注册表和 Python entry point 扩展机制;
- cmvr-es RGB 相机 gRPC Source带退避上限的指数重连、AGV gRPC Sink
- PyAV H264/H265 有状态解码、通用检测模型节点和重复命中规则节点;
- Construction PPE YOLOv8 模型注册,以及按标签、置信度和最大 FPS 的部署配置;
- 平台 HTTP JSON Sink支持有限重试并使用告警 `event_id` 作为幂等键;
- Construction PPE、六类 PPE YOLOv8n、People-Talking YOLOv8x 与 YOLOv8n Mobile Phone 模型注册,以及按标签、置信度和最大 FPS 的部署配置;
- 平台 HTTP JSON Sink支持有限重试、`raise/log_and_drop` 失败策略,并使用告警
`event_id` 作为幂等键;
- `RobotCommand -> ApprovedRobotCommand` 安全门和无重试的类型化 AGV 命令映射;
- 文本/JSON 日志、共享 gRPC Channel 与 HTTP Client
- 可直接执行的 smoke、PPE 检测和对话占位配置。
- 可直接执行的最小测试 fixture以及包含 PPE/电话使用检测和对话占位链路的统一部署配置。
`detect_server/pipeline.yaml` 已实现“相机编码流 -> PyAV 解码 -> 注册模型推理 -> 时间窗口规则 -> HTTP 告警”。VAD/ASR/LLM/TTS 尚未内置;`talk_server/pipeline.yaml` 仍使用模拟音频数据,等待 cmvr-es 音频双向流 proto 落地。
`configs/edge_ai.yaml` 是统一部署配置,其中同时定义 `detection` 和 `talk` 两个
Pipeline。`detection` 从同一个相机 decoder 并行运行 Construction PPE 与
People-Talking 两个 detector各自经过独立时间窗口规则后向同一个告警平台上报
`model_id` 和 `rule_id` 用于区分事件。六类 PPE 模型仍保留在注册表中,但不在当前
Pipeline 图中实例化。
VAD/ASR/LLM/TTS 尚未内置;`talk` 仍使用模拟音频数据,等待 cmvr-es 音频双向流
proto 落地。
## 架构概览
@ -67,12 +76,20 @@ cmvr_edge_ai/
├── .python-version # uv 默认 Python 3.10
├── uv.lock # 所有 profile 的可复现依赖锁
├── configs/
│ └── smoke.yaml # 不依赖外部服务的最小运行验证
│ ├── edge_ai.yaml # detection + talk 统一部署配置
│ └── debug/
│ └── detection_viewer.yaml # 远端相机 -> PPE+电话检测 -> 合并画框窗口
├── detect_server/
│ └── pipeline.yaml # 相机 -> 解码 -> PPE 检测 -> 重复规则 -> HTTP
│ ├── README.md # PPE/电话检测链路与 Viewer 使用说明
│ └── show_detections.py # OpenCV 实时检测结果 Demo
├── models/
│ └── detection/ # 按模型 ID/版本组织的检测模型制品库
│ ├── construction-ppe-yolov8/v1/ # best.pt + 独立 model card
│ ├── ppe-6classes-yolov8n/v1/ # best.pt + 独立 model card
│ ├── people-talking-yolov8x/v1/ # best.pt + 独立 model card
│ └── yolov8n-mobile-phone/ # Viewer 手机目标检测权重 + model card
├── talk_server/
│ ├── nodes/ # 对话插件预留目录
│ └── pipeline.yaml # 模拟音频 -> 对话占位 -> 日志
│ └── nodes/ # 对话插件预留目录
├── scripts/
│ ├── bootstrap.sh # 一键创建 uv 环境、生成 bindings 并自检
│ └── generate_cmvr_stubs.py # 从 cmvr-es proto 生成 Python bindings
@ -90,6 +107,8 @@ cmvr_edge_ai/
│ ├── compiler.py # 配置到可执行 DAG 的编译器
│ └── cli.py # validate/run/plugins/models
└── tests/
└── fixtures/
└── minimal_pipeline.yaml # 不依赖外部服务的框架/CLI 自检配置
```
## 快速开始
@ -119,11 +138,17 @@ bash scripts/bootstrap.sh --profile core
机器上重新选择依赖版本。无需 `source .venv/bin/activate`,统一通过
`uv run --no-sync` 使用已经安装好的环境:
配置中的相对文件路径按进程启动时的当前工作目录(`cwd`)解析,不是按 YAML
文件所在目录解析。因此本文的 bootstrap、validate、run 和 Viewer 命令都应从仓库根目录
`/home/xtkuang/Projects/cmvr/cmvr_edge_ai` 执行;从其他目录启动时,必须把
配置中的模型等文件路径改为正确的绝对路径。
```bash
uv run --no-sync cmvr-edge-ai validate --config configs/smoke.yaml
uv run --no-sync cmvr-edge-ai validate \
--config tests/fixtures/minimal_pipeline.yaml
uv run --no-sync cmvr-edge-ai plugins
uv run --no-sync cmvr-edge-ai run \
--config configs/smoke.yaml \
--config tests/fixtures/minimal_pipeline.yaml \
--log-level INFO \
--log-format text
```
@ -134,7 +159,7 @@ bootstrap 支持以下环境:
| Profile | 安装内容 | 命令 |
|---|---|---|
| `core` | 框架核心和模拟 smoke/talk 链路 | `bash scripts/bootstrap.sh --profile core` |
| `core` | 框架核心、最小测试 fixture 和模拟 talk 链路 | `bash scripts/bootstrap.sh --profile core` |
| `detection-cpu` | gRPC、HTTP、PyAV、Pillow 告警图片和固定版本 CPU YOLO默认值 | `bash scripts/bootstrap.sh` |
| `dev` | `detection-cpu` 加测试和 protobuf codegen 工具,并运行完整测试 | `bash scripts/bootstrap.sh --profile dev` |
@ -161,11 +186,16 @@ CLI 的四个子命令如下:
`--pipeline` 可以重复传入。`run` 还支持 `--log-level` 和 `--log-format text|json`。配置错误退出码为 `2`,运行错误为 `1`,键盘中断为 `130`。
## 运行 PPE 检测链路
生产配置 `configs/edge_ai.yaml` 在一个 YAML 中同时定义 `detection` 和 `talk`。部署时
建议显式传 `--pipeline detection` 或 `--pipeline talk`,这样进程只加载并运行选中的
链路;需要同进程运行两条链路时,可以重复传两个 `--pipeline`。如果完全省略
`--pipeline`,运行时会启动配置中所有 `enabled: true` 的 Pipeline。
## 运行视觉检测链路
默认 bootstrap 就是当前 YAML 使用的 CPU 检测环境。它会从相邻的
`../cmvr-es` 读取 proto、用锁定的 `grpcio-tools` 生成 bindings然后安装完整
检测依赖并校验 smoke 和 PPE 配置:
检测依赖并校验最小测试 fixture 和统一配置中的 PPE 链路
```bash
cd /home/xtkuang/Projects/cmvr/cmvr_edge_ai
@ -194,29 +224,52 @@ uv sync --locked --only-group codegen
生成后应再次执行目标 profile 的 `uv sync --locked`让可编辑安装识别新包bootstrap
已经按这个顺序处理。
在 `detect_server/pipeline.yaml` 中配置部署参数:
在 `configs/edge_ai.yaml` 的 `detection` Pipeline 中配置部署参数:
```yaml
endpoints:
cmvr_es:
target: 127.0.0.1:50052
platform:
ppe_alert_platform:
base_url: http://127.0.0.1:8081
pipelines:
detection:
nodes:
camera:
with:
device_id: right_hand_cam
stream_log_interval_s: 5
detector:
with:
attach_frame: true
inference_log_interval_s: 5
model_options:
weights: /home/xtkuang/Projects/cmvr/changan_robot/construction-ppe-yolov8/best.pt
weights: models/detection/construction-ppe-yolov8/v1/best.pt
device: cpu
repeat_gate:
with:
alert_image:
enabled: true
jpeg_quality: 85
phone_detector:
with:
model: people-talking-yolov8x@1
detect_labels: [talking on phone]
max_fps: 5
attach_frame: true
model_options:
weights: models/detection/people-talking-yolov8x/v1/best.pt
device: cpu
phone_repeat_gate:
with:
alert_image:
enabled: true
jpeg_quality: 85
alert_platform:
with:
endpoint: ppe_alert_platform
failure_mode: log_and_drop
```
然后验证并运行检测链路,不再要求预先导出环境变量:
@ -224,22 +277,77 @@ pipelines:
```bash
uv run --no-sync cmvr-edge-ai models
uv run --no-sync cmvr-edge-ai validate \
--config detect_server/pipeline.yaml \
--config configs/edge_ai.yaml \
--pipeline detection
uv run --no-sync cmvr-edge-ai run --config detect_server/pipeline.yaml \
uv run --no-sync cmvr-edge-ai run --config configs/edge_ai.yaml \
--pipeline detection \
--log-level INFO \
--log-format json
```
运行前需要确认权重文件存在、cmvr-es 已启用 `right_hand_cam`,并且平台的 `/v1/detection-alerts` 可访问。默认 `yolo-cpu` profile 将 PyTorch 2.7.0 和 torchvision 0.22.0 绑定到官方 CPU wheel并固定 checkpoint 记录的 Ultralytics 8.4.31。`model_options.device` 在该环境中应保持 `cpu``half` 应保持 `false`。
启动后,`detector` 和 `phone_detector` 会分别输出一条 `detection model loaded`,表示
两份权重已经成功加载。收到解码帧并完成真实 `predict` 后,会分别输出
`detection inference`,之后
按 `inference_log_interval_s` 聚合输出一次;其中 `window_frames` 是本周期推理帧数,
`window_detections` 是检测框总数,`hit_labels` 是各标签的检测框累计数。持续只有 loaded 而没有
inference说明相机或 decoder 尚未把帧送到模型inference 中 detection 为 0 只表示
当前阈值下没有命中。短时调试可设为 `1` 秒,生产环境可设为 `30``60` 秒,省略则关闭
周期推理日志。这里使用标准日志而不是裸 `print`,因此与 `--log-format json` 兼容。
如果需要直接观察两个模型在同一帧上的画框结果,使用独立的 OpenCV Demo。它连接同一个
cmvr-es gRPC CameraService共享解码帧并行执行 PPE 与 YOLOv8n Mobile Phone再把两个结果
合并到一个窗口;它不经过重复触发规则,也不会访问 HTTP 平台:
```bash
uv run --no-sync python detect_server/show_detections.py \
--config configs/debug/detection_viewer.yaml \
--pipeline detection_show \
--log-level INFO
```
运行前在 `configs/debug/detection_viewer.yaml` 中配置远端地址、`device_id` 和两份模型
权重;按 `q` 或 `Esc` 退出。框标签使用 `PPE:`/`PHONE:` 前缀,手机框标签为
`PHONE:mobile_phone`,窗口刷新率由较慢的模型分支决定。详细说明见
[detect_server/README.md](detect_server/README.md#实时画框-demo)。
相机连接器会在每次首次连接或重连时先发 `CameraService.StartCamera`,收到成功反馈后
才建立 `GetRGBImageStream`。终端会依次出现 `camera start requested/succeeded`、
`camera stream opening`、`camera stream first frame` 和周期性的 `camera stream progress`。
如果只有 opening 而没有 first frameprogress 中会持续显示
`first_frame_received=false window_frames=0`用于区分“RPC 已建立但相机没有出帧”。
运行前需要确认 Construction PPE 与 People-Talking 权重存在、cmvr-es 已启用 `right_hand_cam`8081 平台的
`/v1/detection-alerts` 可访问。默认 `yolo-cpu` profile 将 PyTorch 2.7.0 和
torchvision 0.22.0 绑定到官方 CPU wheel并固定 checkpoint 记录的 Ultralytics
8.4.31;同时显式安装 People-Talking checkpoint 反序列化所需的 `dill`。
`model_options.device` 在该环境中应保持 `cpu``half` 应保持 `false`。
x86 CUDA 和 Jetson/JetPack 的 PyTorch wheel 与驱动强绑定,不能复用这个 CPU
profile。项目保留了不绑定 CPU index 的 `yolo` extra 作为设备专用环境的基础,但
GPU 部署前应为目标设备建立单独的 uv source/lock或使用 NVIDIA 容器),再把 YAML
中的 `device` 改为 `cuda:0`;不要只改 YAML 就认为 CUDA 环境已经就绪。
`detection.model@1` 根据 `model` 从 `DetectionModelRegistry` 解析模型。`detect_labels` 只选择需要检测的标签,省略时检测注册模型的全部标签;`confidence` 是全局阈值,也可以用 `label_confidence` 为个别标签覆盖。内置模型 `construction-ppe-yolov8@1` 的名称和 19 个标签可以通过 `cmvr-edge-ai models` 查看。`attach_frame: true` 让检测结果临时携带对应的解码帧,供后续告警节点使用;因此 detector 到 repeat gate 的队列应保持较小,避免堆积未压缩图像。
`detection.model@1` 根据 `model` 从 `DetectionModelRegistry` 解析模型。`detect_labels` 只选择需要检测的标签,省略时检测注册模型的全部标签;`confidence` 是全局阈值,也可以用 `label_confidence` 为个别标签覆盖。当前四个内置模型都可以通过 `cmvr-edge-ai models` 查看,模型制品和独立 model card 位于仓库内:
- [Construction PPE YOLOv8 v1](models/detection/construction-ppe-yolov8/v1/README.md):包含正向 PPE、`No-*` 违规类和施工现场设备类;
- [PPE YOLOv8n 6 Classes v1](models/detection/ppe-6classes-yolov8n/v1/README.md):轻量的六类正向装备检测模型。
- [People Talking YOLOv8x v1](models/detection/people-talking-yolov8x/v1/README.md)电话使用行为检测Pipeline 只选择 `talking on phone` 类。
- [YOLOv8n Mobile Phone v1](models/detection/yolov8n-mobile-phone/README.md)单类手机目标检测Debug Viewer 使用 `mobile_phone` 类。
注册 ID 中的 `@1` 表示版本 1多数制品使用 `v1` 子目录,当前 YOLOv8n Mobile
Phone 保留导入时的扁平目录和上游文件名。实际 `weights` 路径仍由部署 YAML 显式
指定。标签顺序、训练指标、局限和许可声明以上述 model card 为准。
`attach_frame: true` 让检测结果临时携带对应的解码帧,供后续告警节点使用;因此原 detector 到 repeat gate 的队列应保持较小,避免堆积未压缩图像。
六类模型的标签是 `Gloves`、`Vest`、`goggles`、`helmet`、`mask` 和 `safety_shoe`
语义是“画面中检测到了该装备”,不是“人员缺少该装备”。它没有 `Person` 或 `No-*`
类,也没有人员与装备关联能力,因此不能只靠配置推断某个人未佩戴 PPE。该模型当前仅
注册、未被 `configs/edge_ai.yaml` 的 `detection` Pipeline 引用;需要恢复第二分支时,应同时配置 detector、
8082 endpoint、HTTP Sink 和两条关联 edge。
当前 Construction PPE 和 People-Talking 分支从 decoder 输出端口 fan-out共享同一个
相机订阅和 PyAV decoder两个模型仍会分别加载和推理并共享应用的有界线程池。
People-Talking 是视觉行为判断,不能证明电话已接通,也不能区分来电和去电。
`detection.repeat_gate@1` 只在一个规则的 `window_ms` 内看到至少 `min_hits` 个不同帧后生成 `DetectionAlert/v1`。同一帧有多个同类框仍只算一次;触发后进入 `cooldown_ms`,冷却期间不累计,结束后必须重新满足次数。`scope: source` 按相机统计;`scope: track` 按 `track_id` 统计,但当前 YOLO adapter 只做逐帧检测,不产生 `track_id`,因此使用 track 规则前必须增加跟踪/关联节点。
@ -266,8 +374,12 @@ GPU 部署前应为目标设备建立单独的 uv source/lock或使用 NVIDIA
对话占位链路不依赖音频 proto
```bash
uv run --no-sync cmvr-edge-ai validate --config talk_server/pipeline.yaml
uv run --no-sync cmvr-edge-ai run --config talk_server/pipeline.yaml
uv run --no-sync cmvr-edge-ai validate \
--config configs/edge_ai.yaml \
--pipeline talk
uv run --no-sync cmvr-edge-ai run \
--config configs/edge_ai.yaml \
--pipeline talk
```
## 配置最小示例
@ -311,6 +423,8 @@ v1 支持五个 `qos.profile`,并在编译期约束其溢出策略:编码 H2
## 注册新的检测模型
检测模型和 DAG 插件是两层注册:流水线固定使用通用的 `detection.model@1`,具体模型通过 `DetectionModelRegistry` 注册 `DetectionModelSpec`。每个 spec 必须给出版本化 `model_id`、面向运维的 `name`、有序且唯一的 `supported_labels`、`backend` 和 factory。factory 返回实现 `load/predict/close` 的 `DetectionModel`;部署 YAML 中的 `model_options` 原样交给它。第三方模型包可以使用 `cmvr_edge_ai.detection_models` entry point 发布 spec 或注册回调。安装后先执行 `cmvr-edge-ai models`,再让配置引用其中的精确模型 ID。
仓库自带制品统一放在 `models/detection/<model-name>/vN/`,并在每个版本目录保存
`README.md` model card。新的 `model_id` 尾部 `@N` 应与制品目录 `vN` 保持一致。
模型实际输出的标签仍会在通用 Operator 边界二次校验和过滤;模型返回未注册标签会让节点失败。直接相连的重复规则若引用了 detector 没有选择的标签,也会在 `validate` 阶段被编译器拒绝。
@ -395,7 +509,10 @@ AGV Sink 必须绑定固定的非空 `device_id`。`set_velocity` 默认禁用
- UDP 与 QUIC 目录目前是扩展占位,没有可用传输实现。
- 运行时 v1 只支持 `execution.mode: async|inline`,两者当前都是单 task、单并发执行。`thread`、`process`、`model_worker` 是保留值;`concurrency != 1`、非空 `max_in_flight/timeout_s`、`ordered: false` 也是保留配置,都会在编译期被拒绝。插件仍可在组件内部显式使用 `workers.run_blocking()`、`PersistentProcessWorker` 或自有模型 Worker但不能把 `execution` 声明误当作自动调度。
- v1 对未实现的声明采取 fail-closed显式设置 `runtime.max_processes/process_start_method/health_bind/reserved_memory_mb`、非默认 pipeline `priority` 或任何非空 `resources` 都会在编译期被拒绝。健康状态和队列统计目前只能通过 Python API 获取。
- 当前没有配置热更新、配置 overlay、持久化 outbox/spool 或共享内存图像池。HTTP Sink 只有当前进程内的有限重试,重试耗尽会终止 Pipeline进程退出或断电仍可能丢失未上报告警。
- 当前没有配置热更新、配置 overlay、持久化 outbox/spool 或共享内存图像池。HTTP Sink
只有当前进程内的有限重试;默认 `failure_mode: raise` 会终止 Pipeline当前检测配置的
`log_and_drop` 则在重试耗尽后记录 WARNING 并丢弃该告警。两种模式在进程退出或断电时
都可能丢失未上报数据。
- 相机 Source 可以重连,但 cmvr-es 当前服务端通过 `getLatestEncodedFrame` 获取最新编码数据;如果上游在 edge-ai 收到之前已跳过 H264/H265 参考包,`video_contiguous` 无法补回数据,解码器只能在错误后重置并等待关键帧。正式部署应验证 cmvr-es 输出的是连续 access unit 流,或改为对 AI 友好的原始/JPEG/可检测不连续性的接口。
- 内置 YOLO adapter 不运行 tracker所有检测的 `track_id` 都为空;当前示例因此使用 `scope: source`。
- PPE 权重仓库对权重的许可说明与 Ultralytics runtime/checkpoint 中的 AGPL 信息需要在商业交付前核对,并同时确认训练数据和权重分发许可。

View File

@ -186,6 +186,8 @@ src/cmvr_edge_ai/detection/registry.py
src/cmvr_edge_ai/detection/rules.py
src/cmvr_edge_ai/detection/video.py
src/cmvr_edge_ai/detection/models/__init__.py
src/cmvr_edge_ai/detection/models/mobile_phone.py
src/cmvr_edge_ai/detection/models/people_talking.py
src/cmvr_edge_ai/detection/models/ppe_yolo.py
src/cmvr_edge_ai/detection/models/yolo.py
src/cmvr_edge_ai/observability/__init__.py

View File

@ -15,10 +15,12 @@ Pillow<13,>=10
av<17,>=12
[yolo]
dill<1,>=0.3.8
numpy<3,>=1.24
ultralytics==8.4.31
[yolo-cpu]
dill<1,>=0.3.8
numpy<3,>=1.24
torch==2.7.0
torchvision==0.22.0

View File

@ -3,6 +3,8 @@
from .base import DetectionModel
from .models import (
CONSTRUCTION_PPE_LABELS,
MOBILE_PHONE_LABELS,
PEOPLE_TALKING_LABELS,
PPE_6CLASS_LABELS,
register_builtin_detection_models,
)
@ -21,6 +23,8 @@ from .video import VideoDecoderOperator
__all__ = [
"CONSTRUCTION_PPE_LABELS",
"MOBILE_PHONE_LABELS",
"PEOPLE_TALKING_LABELS",
"PPE_6CLASS_LABELS",
"DetectionModel",
"DetectionModelFactory",

View File

@ -1,17 +1,37 @@
"""Built-in detection model adapters and artifact registrations."""
from ..registry import DetectionModelRegistry
from .mobile_phone import (
MOBILE_PHONE_LABELS,
register_mobile_phone_detection_model,
)
from .people_talking import (
PEOPLE_TALKING_LABELS,
register_people_talking_detection_model,
)
from .ppe_yolo import (
CONSTRUCTION_PPE_LABELS,
PPE_6CLASS_LABELS,
register_builtin_detection_models,
register_builtin_detection_models as register_builtin_ppe_models,
)
from .yolo import UltralyticsYoloModel, YoloDependencyError, YoloModelOptions
def register_builtin_detection_models(registry: DetectionModelRegistry) -> None:
register_builtin_ppe_models(registry)
register_people_talking_detection_model(registry)
register_mobile_phone_detection_model(registry)
__all__ = [
"CONSTRUCTION_PPE_LABELS",
"MOBILE_PHONE_LABELS",
"PEOPLE_TALKING_LABELS",
"PPE_6CLASS_LABELS",
"UltralyticsYoloModel",
"YoloDependencyError",
"YoloModelOptions",
"register_builtin_detection_models",
"register_mobile_phone_detection_model",
"register_people_talking_detection_model",
]

View File

@ -0,0 +1,44 @@
"""Registration metadata for the YOLOv8n mobile-phone checkpoint."""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from ..base import DetectionModel
from ..registry import DetectionModelRegistry, DetectionModelSpec
from .yolo import UltralyticsYoloModel
# Keep this tuple in the exact class-ID order stored in the checkpoint.
MOBILE_PHONE_LABELS = ("mobile_phone",)
def _create_mobile_phone_model(
params: Mapping[str, Any],
) -> DetectionModel:
return UltralyticsYoloModel(
params,
expected_labels=MOBILE_PHONE_LABELS,
)
def register_mobile_phone_detection_model(
registry: DetectionModelRegistry,
) -> None:
registry.register(
DetectionModelSpec(
model_id="yolov8n-mobile-phone@1",
name="YOLOv8n Mobile Phone",
supported_labels=MOBILE_PHONE_LABELS,
factory=_create_mobile_phone_model,
backend="ultralytics-yolo",
description="Detect visible mobile-phone objects in images",
)
)
__all__ = [
"MOBILE_PHONE_LABELS",
"register_mobile_phone_detection_model",
]

View File

@ -0,0 +1,53 @@
"""Registration metadata for the People-Talking YOLOv8x checkpoint."""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from ..base import DetectionModel
from ..registry import DetectionModelRegistry, DetectionModelSpec
from .yolo import UltralyticsYoloModel
# These labels must remain in the exact class-ID order stored in best.pt. The
# upstream checkpoint contains a generic class named ``label`` in addition to
# the actionable ``talking on phone`` class. Deployments select only the latter
# through detect_labels, but load-time validation still covers both IDs.
PEOPLE_TALKING_LABELS = (
"label",
"talking on phone",
)
def _create_people_talking_model(
params: Mapping[str, Any],
) -> DetectionModel:
return UltralyticsYoloModel(
params,
expected_labels=PEOPLE_TALKING_LABELS,
)
def register_people_talking_detection_model(
registry: DetectionModelRegistry,
) -> None:
registry.register(
DetectionModelSpec(
model_id="people-talking-yolov8x@1",
name="People Talking YOLOv8x",
supported_labels=PEOPLE_TALKING_LABELS,
factory=_create_people_talking_model,
backend="ultralytics-yolo",
description=(
"Detect people talking on a mobile phone; the deployment "
"pipeline selects the actionable 'talking on phone' class"
),
)
)
__all__ = [
"PEOPLE_TALKING_LABELS",
"register_people_talking_detection_model",
]

View File

@ -84,8 +84,8 @@ class UltralyticsYoloModel(DetectionModel):
except ImportError as exc:
raise YoloDependencyError(
"YOLO support is not installed; install "
"cmvr-edge-ai[yolo]. The PPE checkpoint was created with "
"Ultralytics 8.4.31, so validate the deployed 8.x runtime."
"cmvr-edge-ai[yolo]. Built-in checkpoints require a validated "
"Ultralytics 8.x runtime; this project pins 8.4.31."
) from exc
# Ultralytics .pt files use the PyTorch checkpoint loader. Deploy only

View File

@ -104,6 +104,14 @@ def test_cli_lists_detection_model_metadata(capsys) -> None: # type: ignore[no-
"ultralytics-yolo"
) in output
assert "Gloves,Vest,goggles,helmet,mask,safety_shoe" in output
assert (
"people-talking-yolov8x@1\tPeople Talking YOLOv8x\t"
"ultralytics-yolo\tlabel,talking on phone"
) in output
assert (
"yolov8n-mobile-phone@1\tYOLOv8n Mobile Phone\t"
"ultralytics-yolo\tmobile_phone"
) in output
def test_merged_config_compiles_each_pipeline_without_loading_optional_runtimes() -> None:
@ -133,10 +141,17 @@ def test_merged_config_compiles_each_pipeline_without_loading_optional_runtimes(
.params["failure_mode"]
== "log_and_drop"
)
phone_detector = config.pipelines["detection"].nodes["phone_detector"]
assert phone_detector.params["model"] == "people-talking-yolov8x@1"
assert phone_detector.params["detect_labels"] == ["talking on phone"]
phone_gate = config.pipelines["detection"].nodes["phone_repeat_gate"]
assert phone_gate.params["rules"][0]["labels"] == ["talking on phone"]
assert set(compiled[0].plugin_specs) == {
"camera",
"decoder",
"detector",
"repeat_gate",
"phone_detector",
"phone_repeat_gate",
"alert_platform",
}

View File

@ -10,6 +10,10 @@ from cmvr_edge_ai.config import load_config
PROJECT_ROOT = Path(__file__).resolve().parents[2]
CONSTRUCTION_WEIGHTS = "models/detection/construction-ppe-yolov8/v1/best.pt"
PEOPLE_TALKING_WEIGHTS = "models/detection/people-talking-yolov8x/v1/best.pt"
MOBILE_PHONE_WEIGHTS = (
"models/detection/yolov8n-mobile-phone/yolov8n-mobile-phone.pt"
)
MODEL_ARTIFACTS = (
(
"construction-ppe-yolov8",
@ -21,6 +25,11 @@ MODEL_ARTIFACTS = (
5_625_014,
"07172ef3ae9e256c40a1fb0ce3eefe5547d90170645aa73dded0fffc382cdb31",
),
(
"people-talking-yolov8x",
136_694_313,
"86cd63926de1c69f70dd7f8755e4de9e382bbdeb0668821b7c3846b826471526",
),
)
@ -50,6 +59,19 @@ def test_model_artifact_is_complete(
assert _sha256(weights) == expected_sha256
def test_mobile_phone_model_artifact_is_complete() -> None:
weights = PROJECT_ROOT / MOBILE_PHONE_WEIGHTS
model_card = weights.parent / "README.md"
assert model_card.is_file()
assert weights.is_file()
assert weights.stat().st_size == 6_234_666
assert (
_sha256(weights)
== "9230e4bfa7cba7134215c4c7f228b5e58760b02138788ccaff0149258c2d2e19"
)
@pytest.mark.parametrize(
("config_path", "pipeline_id"),
(
@ -70,3 +92,60 @@ def test_detection_configs_use_repository_construction_weights(
assert detector.params["model"] == "construction-ppe-yolov8@1"
assert detector.params["model_options"]["weights"] == CONSTRUCTION_WEIGHTS
assert (PROJECT_ROOT / CONSTRUCTION_WEIGHTS).is_file()
def test_production_pipeline_uses_repository_people_talking_weights() -> None:
config = load_config(PROJECT_ROOT / "configs" / "edge_ai.yaml")
pipeline = config.pipelines["detection"]
detector = pipeline.nodes["phone_detector"]
gate = pipeline.nodes["phone_repeat_gate"]
assert detector.params["model"] == "people-talking-yolov8x@1"
assert detector.params["detect_labels"] == ["talking on phone"]
assert detector.params["model_options"]["weights"] == PEOPLE_TALKING_WEIGHTS
assert (PROJECT_ROOT / PEOPLE_TALKING_WEIGHTS).is_file()
assert gate.params["rules"][0]["labels"] == ["talking on phone"]
assert any(
edge.source == "decoder.frames" and edge.target == "phone_detector.frames"
for edge in pipeline.edges
)
assert any(
edge.source == "phone_detector.detections"
and edge.target == "phone_repeat_gate.detections"
for edge in pipeline.edges
)
assert any(
edge.source == "phone_repeat_gate.alerts"
and edge.target == "alert_platform.input"
for edge in pipeline.edges
)
def test_debug_viewer_uses_repository_mobile_phone_weights() -> None:
config = load_config(PROJECT_ROOT / "configs" / "debug" / "detection_viewer.yaml")
pipeline = config.pipelines["detection_show"]
ppe_detector = pipeline.nodes["detector"]
detector = pipeline.nodes["phone_detector"]
viewer = pipeline.nodes["viewer"]
assert detector.params["model"] == "yolov8n-mobile-phone@1"
assert detector.params["detect_labels"] == ["mobile_phone"]
assert detector.params["model_options"]["weights"] == MOBILE_PHONE_WEIGHTS
assert ppe_detector.params["attach_frame"] is True
assert detector.params["attach_frame"] is True
assert viewer.params["expected_inputs"] == ["ppe", "phone"]
assert (PROJECT_ROOT / MOBILE_PHONE_WEIGHTS).is_file()
assert any(
edge.source == "frame_fanout.output"
and edge.target == "phone_detector.frames"
for edge in pipeline.edges
)
assert any(
edge.source == "phone_detector.detections"
and edge.target == "viewer.phone"
for edge in pipeline.edges
)
assert any(
edge.source == "detector.detections" and edge.target == "viewer.ppe"
for edge in pipeline.edges
)

View File

@ -94,29 +94,43 @@ def _frame(
)
def _result(*, source_frame: ImageFrame | None = None) -> DetectionResult:
def _result(
*,
source_frame: ImageFrame | None = None,
label: str = "No-Helmet",
model_id: str = "construction-ppe-yolov8@1",
model_name: str = "Construction PPE YOLOv8s",
inference_ms: float = 12.5,
) -> DetectionResult:
return DetectionResult(
detections=(
Detection(
"No-Helmet",
label,
0.91,
BoundingBox(0.0, 0.0, 1.0, 1.0),
),
),
model_id="construction-ppe-yolov8@1",
model_name="Construction PPE YOLOv8s",
inference_ms=12.5,
model_id=model_id,
model_name=model_name,
inference_ms=inference_ms,
source_frame=_frame() if source_frame is None else source_frame,
)
def _envelope(payload: Any) -> Envelope[Any]:
def _envelope(
payload: Any,
*,
sequence: int = 7,
trace_id: str = "viewer-test-frame",
) -> Envelope[Any]:
return Envelope(
payload,
schema_name="DetectionResult",
schema_version=1,
source_id="wrist_cam",
sequence=7,
sequence=sequence,
trace_id=trace_id,
session_id="viewer-test-session",
)
@ -129,7 +143,7 @@ def _context() -> ComponentContext:
)
def test_demo_config_compiles_with_only_camera_decoder_detector_and_viewer() -> None:
def test_demo_config_compiles_with_shared_fanout_and_two_detectors() -> None:
config = load_config(DEMO_CONFIG)
compiled = validate_application(
config,
@ -141,21 +155,41 @@ def test_demo_config_compiles_with_only_camera_decoder_detector_and_viewer() ->
assert set(compiled[0].plugin_specs) == {
"camera",
"decoder",
"frame_fanout",
"detector",
"phone_detector",
"viewer",
}
assert config.runtime.thread_workers == 4
detector = config.pipelines["detection_show"].nodes["detector"]
assert detector.params["max_fps"] == 10
assert detector.params["attach_frame"] is True
viewer_edge = next(
edge
assert "max_fps" not in detector.params
phone_detector = config.pipelines["detection_show"].nodes["phone_detector"]
assert phone_detector.params["model"] == "yolov8n-mobile-phone@1"
assert phone_detector.params["detect_labels"] == ["mobile_phone"]
assert phone_detector.params["attach_frame"] is True
assert "max_fps" not in phone_detector.params
viewer = config.pipelines["detection_show"].nodes["viewer"]
assert viewer.params["expected_inputs"] == ["ppe", "phone"]
edges = {
(edge.source, edge.target): edge
for edge in config.pipelines["detection_show"].edges
if edge.target == "viewer.input"
}
latest = edges[("decoder.frames", "frame_fanout.input")]
assert latest.qos.profile == "realtime_latest"
assert latest.qos.capacity == 1
assert latest.qos.overflow == "drop_oldest"
for target in ("detector.frames", "phone_detector.frames"):
fanout = edges[("frame_fanout.output", target)]
assert fanout.qos.profile == "telemetry"
assert fanout.qos.capacity == 1
assert fanout.qos.overflow == "block"
assert edges[("detector.detections", "viewer.ppe")].qos.overflow == "block"
assert (
edges[("phone_detector.detections", "viewer.phone")].qos.overflow
== "block"
)
assert viewer_edge.source == "detector.detections"
assert viewer_edge.qos.profile == "realtime_latest"
assert viewer_edge.qos.capacity == 1
assert viewer_edge.qos.overflow == "drop_oldest"
@pytest.mark.parametrize(
@ -170,6 +204,11 @@ def test_demo_config_compiles_with_only_camera_decoder_detector_and_viewer() ->
({"box_thickness": 21}, "box_thickness must be between"),
({"font_scale": float("nan")}, "font_scale must be between"),
({"show_stats": "true"}, "show_stats must be a boolean"),
({"expected_inputs": "ppe"}, "expected_inputs must be a list"),
({"expected_inputs": []}, "must contain at least one"),
({"expected_inputs": ["ppe", "ppe"]}, "must not contain duplicate"),
({"expected_inputs": ["other"]}, "unsupported viewer port"),
({"max_pending_frames": 0}, "max_pending_frames must be between"),
],
)
def test_viewer_parameters_are_strict(params: dict[str, Any], message: str) -> None:
@ -377,6 +416,167 @@ def test_consume_displays_rendered_detection_frame(
assert fake_cv2.shown == [("CMVR PPE Detection", "frame")]
def test_viewer_pairs_two_models_and_renders_one_combined_frame(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fake_cv2 = _FakeCv2()
render_calls: list[tuple[ImageFrame, tuple[Detection, ...], str | None]] = []
def render(
frame: ImageFrame,
detections: tuple[Detection, ...],
**kwargs: Any,
) -> str:
render_calls.append((frame, detections, kwargs["header"]))
return "combined-frame"
monkeypatch.setattr(demo, "_render_detection_frame", render)
monkeypatch.setenv("DISPLAY", ":99")
frame = _frame()
ppe = _result(source_frame=frame)
phone = _result(
source_frame=frame,
label="mobile_phone",
model_id="yolov8n-mobile-phone@1",
model_name="YOLOv8n Mobile Phone",
inference_ms=87.5,
)
async def scenario() -> None:
viewer = demo.OpenCvDetectionViewerSink(
"viewer",
{"expected_inputs": ["ppe", "phone"]},
request_stop=lambda: None,
module_loader=lambda name: fake_cv2 if name == "cv2" else object(),
)
await viewer.setup(_context())
await viewer.start()
# Arrival order must not change the configured overlay/header order.
await viewer.consume(_envelope(phone), input_port="phone")
assert fake_cv2.shown == []
await viewer.consume(_envelope(ppe), input_port="ppe")
await viewer.stop()
asyncio.run(scenario())
assert fake_cv2.shown == [
("CMVR PPE Detection", "combined-frame"),
]
assert len(render_calls) == 1
rendered_frame, detections, header = render_calls[0]
assert rendered_frame is frame
assert tuple(item.label for item in detections) == (
"PPE:No-Helmet",
"PHONE:mobile_phone",
)
assert header is not None
assert "PPE boxes=1 12.5ms" in header
assert "PHONE boxes=1 87.5ms" in header
def test_viewer_never_pairs_different_source_frame_objects(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fake_cv2 = _FakeCv2()
monkeypatch.setattr(demo, "_render_detection_frame", lambda *args, **kwargs: None)
monkeypatch.setenv("DISPLAY", ":99")
async def scenario() -> None:
viewer = demo.OpenCvDetectionViewerSink(
"viewer",
{"expected_inputs": ["ppe", "phone"]},
request_stop=lambda: None,
module_loader=lambda name: fake_cv2 if name == "cv2" else object(),
)
await viewer.setup(_context())
await viewer.consume(_envelope(_result()), input_port="ppe")
await viewer.consume(
_envelope(
_result(
label="mobile_phone",
model_id="yolov8n-mobile-phone@1",
)
),
input_port="phone",
)
assert len(viewer._pending_results) == 2
await viewer.stop()
asyncio.run(scenario())
assert fake_cv2.shown == []
def test_viewer_never_pairs_different_traces_for_same_source_frame(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fake_cv2 = _FakeCv2()
monkeypatch.setattr(demo, "_render_detection_frame", lambda *args, **kwargs: None)
monkeypatch.setenv("DISPLAY", ":99")
frame = _frame()
async def scenario() -> None:
viewer = demo.OpenCvDetectionViewerSink(
"viewer",
{"expected_inputs": ["ppe", "phone"]},
request_stop=lambda: None,
module_loader=lambda name: fake_cv2 if name == "cv2" else object(),
)
await viewer.setup(_context())
await viewer.consume(
_envelope(_result(source_frame=frame), trace_id="ppe-trace"),
input_port="ppe",
)
await viewer.consume(
_envelope(
_result(
source_frame=frame,
label="mobile_phone",
model_id="yolov8n-mobile-phone@1",
),
trace_id="phone-trace",
),
input_port="phone",
)
assert len(viewer._pending_results) == 2
await viewer.stop()
asyncio.run(scenario())
assert fake_cv2.shown == []
def test_viewer_pending_frame_cache_is_bounded(
monkeypatch: pytest.MonkeyPatch,
) -> None:
fake_cv2 = _FakeCv2()
monkeypatch.setenv("DISPLAY", ":99")
async def scenario() -> None:
viewer = demo.OpenCvDetectionViewerSink(
"viewer",
{
"expected_inputs": ["ppe", "phone"],
"max_pending_frames": 2,
},
request_stop=lambda: None,
module_loader=lambda name: fake_cv2 if name == "cv2" else object(),
)
await viewer.setup(_context())
for sequence in range(3):
await viewer.consume(
_envelope(
_result(),
sequence=sequence,
trace_id=f"frame-{sequence}",
),
input_port="ppe",
)
assert len(viewer._pending_results) == 2
assert viewer._pending_evicted == 1
await viewer.stop()
asyncio.run(scenario())
def test_linux_headless_session_fails_before_loading_opencv(
monkeypatch: pytest.MonkeyPatch,
) -> None:

View File

@ -8,6 +8,8 @@ import pytest
from cmvr_edge_ai.contracts import BoundingBox, Detection, ImageFrame
from cmvr_edge_ai.detection.models import (
MOBILE_PHONE_LABELS,
PEOPLE_TALKING_LABELS,
PPE_6CLASS_LABELS,
register_builtin_detection_models,
)
@ -246,6 +248,87 @@ def test_ppe_6classes_factory_enforces_checkpoint_label_order(
assert created[0].task == "detect"
def test_people_talking_builtin_registration_preserves_checkpoint_label_order() -> None:
registry = DetectionModelRegistry()
register_builtin_detection_models(registry)
spec = registry.resolve("people-talking-yolov8x@1")
assert spec.name == "People Talking YOLOv8x"
assert spec.backend == "ultralytics-yolo"
assert spec.supported_labels == ("label", "talking on phone")
assert spec.supported_labels == PEOPLE_TALKING_LABELS
def test_people_talking_factory_enforces_generic_checkpoint_label(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
registry = DetectionModelRegistry()
register_builtin_detection_models(registry)
spec = registry.resolve("people-talking-yolov8x@1")
weights = _weights(tmp_path)
model = spec.factory({"weights": str(weights)})
_install_fake_runtime(
monkeypatch,
names={0: "talking on phone"},
)
assert isinstance(model, UltralyticsYoloModel)
with pytest.raises(ValueError, match="checkpoint labels do not match"):
model.load()
_, created = _install_fake_runtime(
monkeypatch,
names=dict(enumerate(PEOPLE_TALKING_LABELS)),
)
matching = spec.factory({"weights": str(weights)})
matching.load()
assert isinstance(matching, UltralyticsYoloModel)
assert len(created) == 1
assert created[0].weights == str(weights)
assert created[0].task == "detect"
def test_mobile_phone_builtin_registration_preserves_checkpoint_label_order() -> None:
registry = DetectionModelRegistry()
register_builtin_detection_models(registry)
spec = registry.resolve("yolov8n-mobile-phone@1")
assert spec.name == "YOLOv8n Mobile Phone"
assert spec.backend == "ultralytics-yolo"
assert spec.supported_labels == ("mobile_phone",)
assert spec.supported_labels == MOBILE_PHONE_LABELS
def test_mobile_phone_factory_enforces_checkpoint_label(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
registry = DetectionModelRegistry()
register_builtin_detection_models(registry)
spec = registry.resolve("yolov8n-mobile-phone@1")
weights = _weights(tmp_path)
model = spec.factory({"weights": str(weights)})
_install_fake_runtime(monkeypatch, names={0: "cell_phone"})
assert isinstance(model, UltralyticsYoloModel)
with pytest.raises(ValueError, match="checkpoint labels do not match"):
model.load()
_, created = _install_fake_runtime(
monkeypatch,
names=dict(enumerate(MOBILE_PHONE_LABELS)),
)
matching = spec.factory({"weights": str(weights)})
matching.load()
assert isinstance(matching, UltralyticsYoloModel)
assert len(created) == 1
assert created[0].weights == str(weights)
assert created[0].task == "detect"
@pytest.mark.parametrize(
("frame", "message"),
[

13
uv.lock generated
View File

@ -160,12 +160,14 @@ video = [
{ name = "av" },
]
yolo = [
{ name = "dill" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and extra == 'extra-12-cmvr-edge-ai-yolo') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version == '3.11.*' and extra == 'extra-12-cmvr-edge-ai-yolo') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.12' and extra == 'extra-12-cmvr-edge-ai-yolo') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
{ name = "ultralytics" },
]
yolo-cpu = [
{ name = "dill" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version == '3.11.*' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.12' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu') or (extra == 'extra-12-cmvr-edge-ai-yolo' and extra == 'extra-12-cmvr-edge-ai-yolo-cpu')" },
@ -193,6 +195,8 @@ test = [
[package.metadata]
requires-dist = [
{ name = "av", marker = "extra == 'video'", specifier = ">=12,<17" },
{ name = "dill", marker = "extra == 'yolo'", specifier = ">=0.3.8,<1" },
{ name = "dill", marker = "extra == 'yolo-cpu'", specifier = ">=0.3.8,<1" },
{ name = "grpcio", marker = "extra == 'grpc'", specifier = ">=1.76,<2" },
{ name = "httpx", marker = "extra == 'http'", specifier = ">=0.27,<1" },
{ name = "numpy", marker = "extra == 'yolo'", specifier = ">=1.24,<3" },
@ -345,6 +349,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321, upload-time = "2023-10-07T05:32:16.783Z" },
]
[[package]]
name = "dill"
version = "0.4.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/81/e1/56027a71e31b02ddc53c7d65b01e68edf64dea2932122fe7746a516f75d5/dill-0.4.1.tar.gz", hash = "sha256:423092df4182177d4d8ba8290c8a5b640c66ab35ec7da59ccfa00f6fa3eea5fa", size = 187315, upload-time = "2026-01-19T02:36:56.85Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/77/dc8c558f7593132cf8fefec57c4f60c83b16941c574ac5f619abb3ae7933/dill-0.4.1-py3-none-any.whl", hash = "sha256:1e1ce33e978ae97fcfcff5638477032b801c46c7c65cf717f95fbc2248f79a9d", size = 120019, upload-time = "2026-01-19T02:36:55.663Z" },
]
[[package]]
name = "exceptiongroup"
version = "1.3.1"