tirtc-device-builder 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +1 -1
- package/CHANGELOG.md +9 -0
- package/README.md +625 -427
- package/package.json +1 -1
- package/skills/tirtc-esp32-builder/SKILL.md +5 -3
- package/skills/tirtc-esp32-builder/USAGE.md +14 -11
- package/skills/tirtc-esp32-builder/assets/developer-intake-prompt.md +50 -0
- package/skills/tirtc-esp32-builder/assets/hardware-ir-v2.example.json +122 -0
- package/skills/tirtc-esp32-builder/assets/report-template.md +15 -0
- package/skills/tirtc-esp32-builder/references/capability-rules.md +41 -16
- package/skills/tirtc-esp32-builder/references/hardware-ir.md +44 -24
- package/skills/tirtc-esp32-builder/references/porting-risks.md +49 -0
- package/skills/tirtc-esp32-builder/references/reporting.md +5 -1
- package/skills/tirtc-esp32-builder/references/workflow.md +10 -9
- package/skills/tirtc-esp32-builder/scripts/hardware_ir.py +670 -40
package/package.json
CHANGED
|
@@ -13,14 +13,16 @@ Turn board evidence into an evidence-backed ESP-IDF project. Treat the Hardware
|
|
|
13
13
|
2. Locate the versioned ESP32 Device Kit root containing `device-sim/` using the explicit input, `TIRTC_THING_CONNECT_ROOT`, the managed setup configuration, or workspace discovery. Treat its manifest and packaged protocol documents as the generation facts. If the user explicitly supplies a full ThingConnect source workspace, also read its applicable `AGENTS.md`.
|
|
14
14
|
3. Run the Doctor through the managed environment helper when one exists; otherwise run `python3 <skill-dir>/scripts/doctor.py --expected-idf 5.5 --target esp32s3`. Add `--require-workspace` when generation or repository reference documents are needed; a self-contained generated project can instead resolve its bundled SDK through `--project`. Resolve every required failure before claiming build readiness.
|
|
15
15
|
4. Read [workflow.md](references/workflow.md). Select the registered-board, new-board intake, or existing-project branch. The branch is selected when every supplied artifact has been accounted for and the exact board revision is known or explicitly unresolved.
|
|
16
|
-
5. Read [hardware-ir.md](references/hardware-ir.md) when a Hardware IR must be created or updated. Record a source and verification level for every hardware fact that affects a requested feature.
|
|
16
|
+
5. Read [hardware-ir.md](references/hardware-ir.md) when a Hardware IR must be created or updated. New intake uses schema v2; schema v1 remains readable for existing H.264 projects. Record a source and verification level for every hardware fact that affects a requested feature.
|
|
17
17
|
6. Run `python3 <skill-dir>/scripts/hardware_ir.py validate <hardware-ir.json>` and then `assess --strict`. Generation may proceed for a requested feature only when it is `READY_TO_PORT` or `HIL_VERIFIED`; otherwise report the exact missing evidence and continue with safe discovery or scaffolding only.
|
|
18
18
|
|
|
19
19
|
## Build the project
|
|
20
20
|
|
|
21
21
|
Run `<device-kit-root>/device-sim/scripts/create_esp32_project.py` for the current ESP32-S3 H5/AI starter. Keep ThingConnect onboarding, H5, AI, TiRTC lifecycle, callback, stream, and generation behavior in the existing deep modules. Put board-specific camera, microphone, encoder, codec, amplifier, GPIO, DMA, and task behavior behind the `starter_media` seam or a board media adapter owned by it.
|
|
22
22
|
|
|
23
|
-
Before changing media code, read [capability-rules.md](references/capability-rules.md) and the repository documents
|
|
23
|
+
Before changing media code, read [capability-rules.md](references/capability-rules.md) and [porting-risks.md](references/porting-risks.md), then read the repository documents they route to. A camera sensor alone does not establish H5 video support; validate the user-selected MJPEG, H.264, or H.265 profile end to end. Choose half duplex for AI when the supplied hardware and BSP do not establish a usable full-duplex/AEC path.
|
|
24
|
+
|
|
25
|
+
Keep the Skill board-agnostic. The prompt supplies product intent and artifact locations; Hardware IR stores evidence; the generated board adapter owns concrete sensors, codecs, pins, clocks, slots, DMA, and task allocation. Never add one board's values to Skill defaults to make an assessment pass.
|
|
24
26
|
|
|
25
27
|
Run focused tests before ESP-IDF build. Resolve the TiRTC SDK target and `manifest/build-contract.env` against the generated `sdkconfig`; a mismatched precompiled SDK is a blocked build, not a code-generation problem.
|
|
26
28
|
|
|
@@ -28,7 +30,7 @@ Run focused tests before ESP-IDF build. Resolve the TiRTC SDK target and `manife
|
|
|
28
30
|
|
|
29
31
|
Flash only when the user requested hardware mutation and the exact serial port and chip have been resolved. When more than one candidate device exists, obtain the target choice before writing. Keep credentials outside generated files and redact device keys, Wi-Fi passwords, MQTT/WHIP tokens, and user media from logs and reports.
|
|
30
32
|
|
|
31
|
-
Read [reporting.md](references/reporting.md) before end-to-end verification. Report every acceptance level as `PASS`, `FAIL`, or `SKIP`, with commands and evidence. A build-only result is not H5 or AI completion; missing hardware, browser, account, service, or network evidence remains an explicit `SKIP` or blocker.
|
|
33
|
+
Read [reporting.md](references/reporting.md) before end-to-end verification. Report every acceptance level as `PASS`, `FAIL`, or `SKIP`, with commands and evidence. Bind HIL observations to the exact firmware SHA-256 with `assess --artifact-sha256`; an older artifact cannot verify a newer build. A build-only result is not H5 or AI completion; missing hardware, browser, account, service, or network evidence remains an explicit `SKIP` or blocker.
|
|
32
34
|
|
|
33
35
|
## Finish
|
|
34
36
|
|
|
@@ -14,20 +14,23 @@ npx tirtc-device-builder@latest setup esp32 --install
|
|
|
14
14
|
```text
|
|
15
15
|
$tirtc-esp32-builder
|
|
16
16
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
- H5 按住说话
|
|
20
|
-
- AI 双向对讲
|
|
17
|
+
开发板:<厂商、完整型号、PCB 版本>
|
|
18
|
+
资料与实物对应:<是/否/未知>
|
|
21
19
|
|
|
22
20
|
资料:
|
|
23
|
-
-
|
|
24
|
-
- 原理图:/absolute/path/board-schematic.pdf
|
|
25
|
-
- BSP 或示例工程:/absolute/path/vendor-bsp
|
|
26
|
-
- 输出目录:/absolute/path/my-tirtc-device
|
|
21
|
+
- <原理图、BSP/厂商示例、数据手册和产品页;一行一个>
|
|
27
22
|
|
|
28
|
-
|
|
23
|
+
目标:<H5 实时音视频、H5 对讲、AI 双向语音等>
|
|
24
|
+
视频:<MJPEG/H264/H265/根据证据选择>
|
|
25
|
+
Wi-Fi 与设备绑定:<指定方案/根据 BSP 和平台合同选择>
|
|
26
|
+
工程:<输出目录或现有工程的绝对路径>
|
|
27
|
+
|
|
28
|
+
先运行 Doctor,再生成 Hardware IR v2。达到 READY_TO_PORT 后完成板级适配和编译,输出 TIRTC_PORTING_REPORT.md。
|
|
29
|
+
本轮不访问串口、不烧录、不擦除 NVS,也不把凭证写入源码或报告。
|
|
29
30
|
```
|
|
30
31
|
|
|
32
|
+
可直接复制的版本见[开发板接入提示词](assets/developer-intake-prompt.md)。SoftAP 只是可选方案;没有 AP 配网时,Skill 会根据 BSP 和产品要求评估其他可重配路径。生产 SSID 和密码不能进入源码或报告。
|
|
33
|
+
|
|
31
34
|
## ESP32 Device Kit
|
|
32
35
|
|
|
33
36
|
一键安装会下载固定版本的最小资源包并校验 SHA-256,不需要克隆 ThingConnect 服务端仓库。安装路径由下面的文件记录:
|
|
@@ -92,8 +95,8 @@ python3 <skill-dir>/scripts/hardware_ir.py validate /tmp/hardware-ir.json
|
|
|
92
95
|
python3 <skill-dir>/scripts/hardware_ir.py assess --strict /tmp/hardware-ir.json
|
|
93
96
|
```
|
|
94
97
|
|
|
95
|
-
`BLOCKED`
|
|
98
|
+
`init` 默认创建 schema v2;v1 仅用于兼容已有 H.264 IR。`BLOCKED` 表示资料已确认硬件/合同/资源或凭证策略不满足;`NEEDS_CONFIRMATION` 表示仍有未知项或只有单一来源;`READY_TO_PORT` 表示可以生成并实现板级适配;`HIL_VERIFIED` 还要求 `--artifact-sha256` 匹配该固件的 L5/L6 运行证据。
|
|
96
99
|
|
|
97
100
|
## 当前边界
|
|
98
101
|
|
|
99
|
-
ThingConnect 仓库提供 ESP32-S3 H5/AI
|
|
102
|
+
ThingConnect 仓库提供 ESP32-S3 H5/AI 模板和生成器,但默认媒体适配器不包含特定开发板的摄像头、麦克风、选定视频路径、Wi-Fi 凭证方法和扬声器驱动。模板生成和编译成功只证明工程与协议骨架可用,不代表 Web 已经出图或 AI 音频已经通过实机验收。
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# TiRTC ESP32 开发板接入提示词
|
|
2
|
+
|
|
3
|
+
不用先把所有硬件参数查齐。你只要说明是哪块板、资料放在哪里、想实现什么,以及工程输出到哪里。其余内容由 Skill 从原理图、BSP、Device Kit 和数据手册中提取;确实不知道的地方写“未知”。
|
|
4
|
+
|
|
5
|
+
## 精简模板
|
|
6
|
+
|
|
7
|
+
```text
|
|
8
|
+
$tirtc-esp32-builder
|
|
9
|
+
|
|
10
|
+
请为下面这块开发板完成 ThingConnect TiRTC ESP32 接入。
|
|
11
|
+
|
|
12
|
+
开发板:
|
|
13
|
+
- 厂商、完整型号、PCB/硬件版本:<填写>
|
|
14
|
+
- 资料与手中实物是否对应:<是/否/未知>
|
|
15
|
+
|
|
16
|
+
资料:
|
|
17
|
+
- <原理图、BSP/厂商示例、数据手册、产品页等本地绝对路径或固定链接;一行一个>
|
|
18
|
+
|
|
19
|
+
目标:
|
|
20
|
+
- <例如:H5 实时视频和声音、H5 语音对讲、AI 双向语音对讲>
|
|
21
|
+
- 视频选择:<MJPEG/H264/H265/根据合同和硬件证据选择>
|
|
22
|
+
- 双工或 AEC 的硬要求:<无,初版可半双工/必须全双工或 AEC/未知>
|
|
23
|
+
|
|
24
|
+
接入方式:
|
|
25
|
+
- Wi-Fi:<选择一种,或写“根据 BSP 选择”>
|
|
26
|
+
- 设备绑定:<选择一种,或写“根据平台合同选择”>
|
|
27
|
+
|
|
28
|
+
工程:
|
|
29
|
+
- 输出目录或现有工程:<绝对路径>
|
|
30
|
+
|
|
31
|
+
执行要求:
|
|
32
|
+
1. 先运行 Device Kit Doctor,读完全部资料,再生成 Hardware IR v2。没有证据的器件、GPIO 和媒体能力保持未知。
|
|
33
|
+
2. 资料不足时,列出矛盾、缺失项和最小补充动作;达到 READY_TO_PORT 后再生成板级 adapter、编译并记录 BIN/ELF SHA-256。
|
|
34
|
+
3. 移植前核对媒体合同、板级资源、内存预算、配网和绑定状态。具体板卡参数只写入该板的 Hardware IR 和 adapter。
|
|
35
|
+
4. 输出 TIRTC_PORTING_REPORT.md,并把编译结果与 L2-L7 实机验收分开记录。
|
|
36
|
+
5. 本轮不访问串口、不烧录、不擦除 NVS。Wi-Fi 密码、设备密钥、token、私钥和用户音视频也不能写入工程或报告。
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## 只有已知时才补充
|
|
40
|
+
|
|
41
|
+
这些信息如果手头已有,也可以附上;没有就交给 Skill 查资料:
|
|
42
|
+
|
|
43
|
+
- 主控/模组完整型号、Flash 和 PSRAM 容量及总线模式;
|
|
44
|
+
- BSP 的 commit、tag 或 release,以及已验证的摄像头、录音和播放示例;
|
|
45
|
+
- H5/AI 媒体合同链接、选定 profile、stream ID 和帧或 access unit 边界;
|
|
46
|
+
- 凭证保存与重配方式、已有绑定处理和独立清除入口;
|
|
47
|
+
- 已知资料矛盾、实机 PID、串口日志或已知良好固件 SHA-256;
|
|
48
|
+
- 非默认 ESP-IDF、TiRTC SDK、Device Kit、服务发现地址或 HTTP/HTTPS 分阶段要求。
|
|
49
|
+
|
|
50
|
+
直接把它们追加在提示词末尾即可,不必逐项填表。
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
{
|
|
2
|
+
"schema_version": 2,
|
|
3
|
+
"board": {
|
|
4
|
+
"id": "example_esp32s3_media_board",
|
|
5
|
+
"vendor": "Example Vendor",
|
|
6
|
+
"model": "ESP32-S3 Media Board",
|
|
7
|
+
"hardware_revision": "unspecified"
|
|
8
|
+
},
|
|
9
|
+
"sources": [
|
|
10
|
+
{
|
|
11
|
+
"id": "board-materials",
|
|
12
|
+
"kind": "user-supplied-materials",
|
|
13
|
+
"location": "user-supplied://board-materials",
|
|
14
|
+
"revision": "unspecified"
|
|
15
|
+
}
|
|
16
|
+
],
|
|
17
|
+
"soc": {
|
|
18
|
+
"target": "esp32s3",
|
|
19
|
+
"module": "ESP32-S3-WROOM-1-N16R8",
|
|
20
|
+
"flash_mb": 16,
|
|
21
|
+
"psram_mb": 8,
|
|
22
|
+
"source_refs": [
|
|
23
|
+
"board-materials"
|
|
24
|
+
]
|
|
25
|
+
},
|
|
26
|
+
"toolchain": {
|
|
27
|
+
"framework": "esp-idf",
|
|
28
|
+
"framework_version": "5.5.x",
|
|
29
|
+
"verification": "extracted",
|
|
30
|
+
"tirtc": {
|
|
31
|
+
"platform": "espressif-esp32s3",
|
|
32
|
+
"version": "2.3.0",
|
|
33
|
+
"sdk_path": "device-sim/sdk/espressif-esp32s3/2.3.0",
|
|
34
|
+
"build_contract": "manifest/build-contract.env"
|
|
35
|
+
},
|
|
36
|
+
"source_refs": [
|
|
37
|
+
"board-materials"
|
|
38
|
+
]
|
|
39
|
+
},
|
|
40
|
+
"camera": {
|
|
41
|
+
"present": null,
|
|
42
|
+
"sensor": null,
|
|
43
|
+
"interface": null,
|
|
44
|
+
"selected_video_profile": null,
|
|
45
|
+
"video_profiles": [],
|
|
46
|
+
"source_refs": [
|
|
47
|
+
"board-materials"
|
|
48
|
+
]
|
|
49
|
+
},
|
|
50
|
+
"audio_input": {
|
|
51
|
+
"present": null,
|
|
52
|
+
"interface": null,
|
|
53
|
+
"codecs": [],
|
|
54
|
+
"source_refs": [
|
|
55
|
+
"board-materials"
|
|
56
|
+
]
|
|
57
|
+
},
|
|
58
|
+
"audio_output": {
|
|
59
|
+
"present": null,
|
|
60
|
+
"interface": null,
|
|
61
|
+
"codecs": [],
|
|
62
|
+
"source_refs": [
|
|
63
|
+
"board-materials"
|
|
64
|
+
]
|
|
65
|
+
},
|
|
66
|
+
"hardware_resources": {
|
|
67
|
+
"i2c": {
|
|
68
|
+
"used": null,
|
|
69
|
+
"driver_family": null,
|
|
70
|
+
"single_driver_family": null,
|
|
71
|
+
"verification": "extracted"
|
|
72
|
+
},
|
|
73
|
+
"i2s": {
|
|
74
|
+
"used": null,
|
|
75
|
+
"controller_and_gpio_ownership_resolved": null,
|
|
76
|
+
"verification": "extracted"
|
|
77
|
+
},
|
|
78
|
+
"audio_channel_mapping": {
|
|
79
|
+
"required": null,
|
|
80
|
+
"resolved": null,
|
|
81
|
+
"verification": "extracted"
|
|
82
|
+
},
|
|
83
|
+
"camera_realtime": {
|
|
84
|
+
"pipeline_safe": null,
|
|
85
|
+
"verification": "extracted"
|
|
86
|
+
},
|
|
87
|
+
"memory": {
|
|
88
|
+
"startup_and_media_budgeted": null,
|
|
89
|
+
"verification": "extracted"
|
|
90
|
+
},
|
|
91
|
+
"source_refs": [
|
|
92
|
+
"board-materials"
|
|
93
|
+
]
|
|
94
|
+
},
|
|
95
|
+
"onboarding": {
|
|
96
|
+
"wifi_credentials": {
|
|
97
|
+
"selected_method": null,
|
|
98
|
+
"methods": [],
|
|
99
|
+
"credentials_committed_to_source": null,
|
|
100
|
+
"reprovisioning_defined": null
|
|
101
|
+
},
|
|
102
|
+
"device_binding": {
|
|
103
|
+
"selected_method": null,
|
|
104
|
+
"methods": [],
|
|
105
|
+
"credentials_committed_to_source": null,
|
|
106
|
+
"stored_credential_state_handled": null,
|
|
107
|
+
"clear_binding_control": null
|
|
108
|
+
},
|
|
109
|
+
"source_refs": [
|
|
110
|
+
"board-materials"
|
|
111
|
+
]
|
|
112
|
+
},
|
|
113
|
+
"runtime_evidence": [],
|
|
114
|
+
"features": {
|
|
115
|
+
"requested": [
|
|
116
|
+
"h5_live_audio",
|
|
117
|
+
"h5_live_video",
|
|
118
|
+
"h5_talkback",
|
|
119
|
+
"ai_talk"
|
|
120
|
+
]
|
|
121
|
+
}
|
|
122
|
+
}
|
|
@@ -5,6 +5,9 @@
|
|
|
5
5
|
- Board ID: `{{BOARD_ID}}`
|
|
6
6
|
- Model/revision: `{{BOARD_MODEL_REVISION}}`
|
|
7
7
|
- Requested features: `{{REQUESTED_FEATURES}}`
|
|
8
|
+
- Selected media profile: `{{SELECTED_MEDIA_PROFILE}}`
|
|
9
|
+
- Selected Wi-Fi credential method: `{{SELECTED_WIFI_METHOD}}`
|
|
10
|
+
- Selected device-binding method: `{{SELECTED_BINDING_METHOD}}`
|
|
8
11
|
- Output project: `{{PROJECT_PATH}}`
|
|
9
12
|
|
|
10
13
|
## Locked inputs
|
|
@@ -46,6 +49,18 @@
|
|
|
46
49
|
- Firmware SHA-256: `{{FIRMWARE_SHA256}}`
|
|
47
50
|
- Flash command/result: `{{FLASH_RESULT}}`
|
|
48
51
|
|
|
52
|
+
## Artifact-bound runtime evidence
|
|
53
|
+
|
|
54
|
+
| Artifact SHA-256 | Acceptance levels | Observations | Current/superseded |
|
|
55
|
+
|---|---|---|---|
|
|
56
|
+
| | | | |
|
|
57
|
+
|
|
58
|
+
## Runtime metrics
|
|
59
|
+
|
|
60
|
+
- Wi-Fi BSSID/channel/RSSI and roaming: `{{WIFI_RUNTIME_METRICS}}`
|
|
61
|
+
- Media tx/rx/drop/error and queue watermarks: `{{MEDIA_RUNTIME_METRICS}}`
|
|
62
|
+
- Camera overflow, internal heap/largest block, PSRAM: `{{RESOURCE_RUNTIME_METRICS}}`
|
|
63
|
+
|
|
49
64
|
## Remaining work and risks
|
|
50
65
|
|
|
51
66
|
`{{REMAINING_WORK}}`
|
|
@@ -1,32 +1,57 @@
|
|
|
1
1
|
# Capability rules
|
|
2
2
|
|
|
3
|
-
Run `hardware_ir.py assess --strict` before generation.
|
|
3
|
+
Run `hardware_ir.py assess --strict` before generation. Hardware IR v2 validates the selected product contract rather than assuming one video codec or provisioning method. Schema v1 remains readable for existing H.264 projects.
|
|
4
4
|
|
|
5
5
|
## Current starter contract
|
|
6
6
|
|
|
7
7
|
| Feature | Required hardware/media path |
|
|
8
8
|
|---|---|
|
|
9
|
-
| `h5_live_audio` | Microphone path
|
|
10
|
-
| `h5_live_video` | Camera plus H.264
|
|
11
|
-
| `h5_talkback` | G.711 A-law, 8 kHz downlink decode and speaker path for stream 14 |
|
|
12
|
-
| `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session
|
|
9
|
+
| `h5_live_audio` | Microphone path producing G.711 A-law, 8 kHz, mono for stream 10; audio controller/channel ownership and memory budget resolved |
|
|
10
|
+
| `h5_live_video` | Camera plus the selected MJPEG, H.264, or H.265 profile for stream 11; refresh/key-frame control, realtime pipeline, and memory budget resolved |
|
|
11
|
+
| `h5_talkback` | G.711 A-law, 8 kHz downlink decode and speaker path for stream 14; audio controller/GPIO ownership and memory budget resolved |
|
|
12
|
+
| `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session`; audio ownership/channel mapping and memory budget resolved |
|
|
13
13
|
|
|
14
|
-
The complete media path must be at least `corroborated` to become `READY_TO_PORT`. A path with unknown facts is `NEEDS_CONFIRMATION`; a confirmed missing or incompatible resource is `BLOCKED`.
|
|
14
|
+
The complete media path must be at least `corroborated` to become `READY_TO_PORT`. A path with unknown facts is `NEEDS_CONFIRMATION`; a confirmed missing or incompatible resource is `BLOCKED`.
|
|
15
15
|
|
|
16
|
-
##
|
|
16
|
+
## Selected video profiles
|
|
17
|
+
|
|
18
|
+
Hardware IR v2 stores one or more `camera.video_profiles` and exactly one selected profile for H5 video:
|
|
19
|
+
|
|
20
|
+
| Codec | Required output contract |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `mjpeg` | `jpeg_complete_frames`: one complete JPEG per send; each frame independently refreshable |
|
|
23
|
+
| `h264` | `h264_annex_b_access_units`: Annex-B access units with SPS/PPS and IDR request behavior |
|
|
24
|
+
| `h265` | `h265_annex_b_access_units`: Annex-B access units with parameter-set and refresh behavior defined by the coordinated H5 contract |
|
|
25
|
+
|
|
26
|
+
Available but unselected profiles do not satisfy or block the selected contract. Stream IDs and codec support must come from the applicable ThingConnect/H5 contract, not this table alone.
|
|
27
|
+
|
|
28
|
+
## Project gates
|
|
29
|
+
|
|
30
|
+
Schema v2 also requires:
|
|
31
|
+
|
|
32
|
+
- one evidenced I2C driver family when I2C is used;
|
|
33
|
+
- a selected Wi-Fi credential method that is available, reprovisionable, and keeps credentials outside source control;
|
|
34
|
+
- one evidenced binding method—verification code, factory-bound identity, development credentials, or documented custom flow—plus stored-binding behavior and reset control;
|
|
35
|
+
- feature-specific I2S/GPIO ownership, channel/TDM mapping, realtime camera policy, and startup/media memory budget.
|
|
36
|
+
|
|
37
|
+
SoftAP is one Wi-Fi option, not a universal requirement. BLE, SmartConfig, factory NVS, development configuration, or a documented custom method can satisfy intake when the selected path is evidenced. Committed plaintext credentials are always `BLOCKED`.
|
|
38
|
+
|
|
39
|
+
## Non-negotiable runtime checks
|
|
17
40
|
|
|
18
41
|
- Match the TiRTC precompiled SDK platform to the ESP-IDF target and its `manifest/build-contract.env` to the generated configuration.
|
|
19
|
-
- Keep H5 stream IDs and formats stable unless the user explicitly authorizes a coordinated
|
|
20
|
-
- Start AI media only after the successful `start_session` response
|
|
21
|
-
- Copy SDK callback payloads into bounded queues before returning. Perform decoding, playback, HTTP, and lifecycle changes outside
|
|
42
|
+
- Keep H5 stream IDs and formats stable unless the user explicitly authorizes a coordinated contract change across the server and consumers.
|
|
43
|
+
- Start AI media only after the successful `start_session` response; stop and flush media before disconnecting.
|
|
44
|
+
- Copy SDK callback payloads into bounded queues before returning. Perform decoding, playback, HTTP, and lifecycle changes outside callbacks.
|
|
22
45
|
- Use monotonic timestamps and session generation to reject stale frames and delayed callbacks.
|
|
46
|
+
- Record runtime evidence with the exact BIN/ELF SHA-256. Use `assess --artifact-sha256 <sha>`; documentation verification alone never becomes v2 `HIL_VERIFIED`.
|
|
23
47
|
|
|
24
48
|
## Typical blocked cases
|
|
25
49
|
|
|
26
|
-
-
|
|
27
|
-
-
|
|
28
|
-
-
|
|
29
|
-
- A generic ESP32-S3 module is named without the carrier board
|
|
30
|
-
-
|
|
50
|
+
- The selected MJPEG/H.264/H.265 profile lacks its required output or refresh semantics.
|
|
51
|
+
- Audio hardware exists, but the A-law path, physical channel/TDM slot, clock, amplifier, or resource ownership is unresolved.
|
|
52
|
+
- Third-party components introduce both legacy and new ESP-IDF I2C drivers.
|
|
53
|
+
- A generic ESP32-S3 module is named without the carrier board wiring and hardware revision.
|
|
54
|
+
- Wi-Fi credentials are embedded in tracked source, or no available/reprovisionable credential path is selected.
|
|
55
|
+
- The TiRTC archive targets another chip, ESP-IDF ABI, or FreeRTOS configuration.
|
|
31
56
|
|
|
32
|
-
When a blocked case
|
|
57
|
+
When a blocked case requires changing a public contract, replacing hardware, obtaining a new SDK, or choosing a provisioning policy, report alternatives instead of silently changing the project.
|
|
@@ -1,49 +1,69 @@
|
|
|
1
1
|
# Hardware IR
|
|
2
2
|
|
|
3
|
-
Hardware IR
|
|
3
|
+
Hardware IR describes one exact board revision and product contract. It is the deterministic handoff from board evidence to capability assessment, generation, build, and HIL reporting.
|
|
4
|
+
|
|
5
|
+
Create the current schema:
|
|
4
6
|
|
|
5
7
|
```bash
|
|
6
8
|
python3 <skill-dir>/scripts/hardware_ir.py init <output>/hardware-ir.json
|
|
7
9
|
```
|
|
8
10
|
|
|
11
|
+
The initializer creates schema v2. The validator still accepts schema v1 for existing H.264-only projects; update new or materially changed boards to v2.
|
|
12
|
+
|
|
9
13
|
## Evidence rules
|
|
10
14
|
|
|
11
15
|
- Give every source a stable `id`, `kind`, `location`, and revision when available.
|
|
12
|
-
- Reference
|
|
13
|
-
- Use `null` for unknown
|
|
14
|
-
- Hardware revision `unspecified` is
|
|
15
|
-
- Keep
|
|
16
|
-
-
|
|
16
|
+
- Reference source IDs from board facts, selected profiles, onboarding methods, resources, and runtime evidence.
|
|
17
|
+
- Use `null` for unknown facts. Retain contradictory values as explicit issues instead of choosing silently.
|
|
18
|
+
- Hardware revision `unspecified` is valid during intake but blocks reusable-board readiness.
|
|
19
|
+
- Keep requested product features under `features.requested`; desired behavior is not hardware evidence.
|
|
20
|
+
- Store concrete sensors, pins, clocks, codecs, slots, and task allocation in this board IR/adapter, never in generic Skill defaults.
|
|
17
21
|
|
|
18
22
|
Verification levels are ordered:
|
|
19
23
|
|
|
20
|
-
1. `extracted`:
|
|
21
|
-
2. `corroborated`:
|
|
22
|
-
3. `build_verified`:
|
|
23
|
-
4. `hardware_verified`: the peripheral works on the exact
|
|
24
|
-
5. `hil_verified`:
|
|
24
|
+
1. `extracted`: one source.
|
|
25
|
+
2. `corroborated`: another authoritative artifact confirms it.
|
|
26
|
+
3. `build_verified`: matching implementation builds with the locked toolchain.
|
|
27
|
+
4. `hardware_verified`: the local peripheral works on the exact board revision.
|
|
28
|
+
5. `hil_verified`: retained for legacy facts; schema v2 feature HIL additionally requires matching artifact evidence.
|
|
25
29
|
|
|
26
|
-
##
|
|
30
|
+
## Schema v2 contracts
|
|
27
31
|
|
|
28
32
|
The IR contains:
|
|
29
33
|
|
|
30
|
-
- exact board identity and
|
|
31
|
-
-
|
|
32
|
-
-
|
|
33
|
-
-
|
|
34
|
-
-
|
|
35
|
-
-
|
|
34
|
+
- exact board identity, module, Flash/PSRAM, ESP-IDF, TiRTC SDK and build contract;
|
|
35
|
+
- camera identity evidence plus `video_profiles[]` and `selected_video_profile`;
|
|
36
|
+
- audio input/output paths;
|
|
37
|
+
- `hardware_resources` for I2C, I2S/GPIO ownership, audio channel mapping, camera realtime policy, and memory budget;
|
|
38
|
+
- `onboarding.wifi_credentials` with selectable SoftAP/BLE/SmartConfig/factory/development/custom methods;
|
|
39
|
+
- selectable ThingConnect binding methods plus stored-binding states and reset control;
|
|
40
|
+
- requested features;
|
|
41
|
+
- optional `runtime_evidence[]`, each bound to a full firmware SHA-256.
|
|
42
|
+
|
|
43
|
+
The selected Wi-Fi method must be available and corroborated, credentials must remain outside tracked source, and reprovisioning must be defined. A board without SoftAP is valid when another selected method meets those conditions.
|
|
44
|
+
|
|
45
|
+
The selected video profile controls assessment. An unselected H.264 fallback cannot make an MJPEG target pass, and missing H.264 cannot block an evidenced MJPEG target.
|
|
46
|
+
|
|
47
|
+
## Artifact-bound HIL
|
|
48
|
+
|
|
49
|
+
Run:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
python3 <skill-dir>/scripts/hardware_ir.py assess hardware-ir.json \
|
|
53
|
+
--artifact-sha256 <64-character-sha256> --strict
|
|
54
|
+
```
|
|
36
55
|
|
|
37
|
-
|
|
56
|
+
H5 features require matching L5 evidence; AI requires matching L6 evidence. Evidence from an older firmware remains provenance but does not verify the current artifact.
|
|
38
57
|
|
|
39
58
|
## Intake quality
|
|
40
59
|
|
|
41
|
-
Preferred
|
|
60
|
+
Preferred evidence order:
|
|
42
61
|
|
|
43
62
|
1. exact schematic/netlist and BOM for the physical revision;
|
|
44
63
|
2. official BSP pinned to a commit or release;
|
|
45
|
-
3. sensor, codec, amplifier
|
|
46
|
-
4.
|
|
47
|
-
5. product pages,
|
|
64
|
+
3. sensor, codec, amplifier and module datasheets;
|
|
65
|
+
4. minimal peripheral projects built for that board;
|
|
66
|
+
5. product pages, photographs and community material;
|
|
67
|
+
6. artifact-bound boot, media and browser observations.
|
|
48
68
|
|
|
49
|
-
A schematic establishes
|
|
69
|
+
A schematic establishes connectivity, not driver family compatibility, encoding throughput, acoustic behavior, provisioning usability, network performance, or end-to-end TiRTC operation. Preserve each as a separate fact and verification level.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Board-porting risk gates
|
|
2
|
+
|
|
3
|
+
Read this reference for every new board or failing media bring-up. It defines generic evidence gates; concrete values belong in the board's Hardware IR and adapter.
|
|
4
|
+
|
|
5
|
+
## Freeze the contract before code
|
|
6
|
+
|
|
7
|
+
Record the selected video profile, audio formats and stream IDs, duplex/AEC policy, onboarding method, platform-discovery transport, output path, and mutation authorization. Keep unknown values explicit. A server supporting several codecs does not select one for the device.
|
|
8
|
+
|
|
9
|
+
## Hardware identity and ownership
|
|
10
|
+
|
|
11
|
+
- Sensor and codec identity: reconcile schematic/BOM names with BSP probes or ID registers. Store every evidenced variant in an allowlist; retain contradictions as issues.
|
|
12
|
+
- I2C: select one ESP-IDF driver family for the final image. Audit the linked ELF when third-party components can introduce another family; a conflict bypass is not a resolution.
|
|
13
|
+
- I2S/audio: record controller, GPIO, master/slave mode, clocks, DMA, channel/TDM slot to physical-signal mapping, and shared-signal handoff. A codec name or I2C address does not prove the audio path.
|
|
14
|
+
- Realtime camera path: record DMA/event queues, task core/priority, and competing Wi-Fi work. Treat queue overflows as scheduling or throughput evidence, then change one variable per HIL comparison.
|
|
15
|
+
|
|
16
|
+
Turn every discovered invariant that can regress into a focused test or post-link gate. Generate board-specific values with the project; keep the Skill generic.
|
|
17
|
+
|
|
18
|
+
## Wi-Fi credentials and device binding
|
|
19
|
+
|
|
20
|
+
Wi-Fi provisioning and ThingConnect binding are separate state machines.
|
|
21
|
+
|
|
22
|
+
Select one evidenced Wi-Fi credential method from SoftAP, BLE, SmartConfig, secure factory/NVS provisioning, development configuration, or a documented custom path. SoftAP is not mandatory. A board without AP provisioning can reach `READY_TO_PORT` through another available method when credentials remain outside source control and a reprovisioning path is defined.
|
|
23
|
+
|
|
24
|
+
Development configuration may inject credentials through an untracked sdkconfig, environment, or provisioning artifact. Treat committed plaintext credentials as `BLOCKED`. Never copy SSIDs/passwords into Hardware IR, reports, examples, or source.
|
|
25
|
+
|
|
26
|
+
Select the binding method from the applicable platform/product contract: verification code, factory-bound identity, development credentials, or a documented custom flow. When verification-code binding is selected, distinguish at least:
|
|
27
|
+
|
|
28
|
+
- no Wi-Fi credentials;
|
|
29
|
+
- Wi-Fi present but no device binding, so the verification-code flow runs;
|
|
30
|
+
- stored binding present, so verification is skipped intentionally;
|
|
31
|
+
- binding-only reset and Wi-Fi reset.
|
|
32
|
+
|
|
33
|
+
For every method, keep device credentials outside source control, handle an already stored identity explicitly, and define binding reset/replacement behavior.
|
|
34
|
+
|
|
35
|
+
## Startup memory, TLS, and transport
|
|
36
|
+
|
|
37
|
+
Platform service discovery and the TiRTC SDK service endpoint are different settings. Record both. HTTPS requires valid time, DNS, certificate validation, TLS client/TLS 1.2, and enough contiguous internal memory.
|
|
38
|
+
|
|
39
|
+
Before sizing TiRTC buffers, measure internal free/largest blocks, PSRAM, frame size distribution, queue watermarks, and send/drop rates. A larger queue can prevent drops, exhaust startup memory, or add buffer latency. If an authorized HTTP baseline exists, stage transport changes separately from media changes and retain the HTTPS requirements as a pending acceptance item.
|
|
40
|
+
|
|
41
|
+
## Network and media evidence
|
|
42
|
+
|
|
43
|
+
Disable Wi-Fi power saving for the realtime baseline unless the product contract says otherwise. Record BSSID, channel, RSSI, reconnect/roaming events, media counters, queue watermarks, camera overflows, internal heap, and largest block. Establish a strong, stable AP baseline before controlled weak-network testing.
|
|
44
|
+
|
|
45
|
+
Confirm SDK send return semantics and callback payload lifetimes from the selected SDK version. Keep SDK callbacks bounded and copy payloads before returning.
|
|
46
|
+
|
|
47
|
+
## Artifact discipline
|
|
48
|
+
|
|
49
|
+
Record BIN/ELF SHA-256 for every build used in HIL. Runtime evidence applies only to the exact artifact. Diagnose one failing invariant, make the smallest correction, rerun that layer, and preserve the comparison in the report.
|
|
@@ -18,6 +18,10 @@ Use [the report template](../assets/report-template.md) and preserve separate `P
|
|
|
18
18
|
|
|
19
19
|
## Evidence
|
|
20
20
|
|
|
21
|
-
Record exact board revision, toolchain and SDK versions, source/adapter revisions, commands, return codes, firmware size and SHA-256, serial port/chip, sanitized log paths, browser or platform observations, and every unavailable dependency.
|
|
21
|
+
Record exact board revision, selected media, Wi-Fi, and binding profiles, toolchain and SDK versions, source/adapter revisions, commands, return codes, firmware size and SHA-256, serial port/chip, sanitized log paths, browser or platform observations, and every unavailable dependency.
|
|
22
|
+
|
|
23
|
+
Runtime evidence belongs to one artifact. Label superseded artifacts and keep their observations as history; do not promote them to the current BIN/ELF. For HIL assessment, add the full SHA-256 to `runtime_evidence` and run `hardware_ir.py assess --artifact-sha256 <sha> --strict`.
|
|
24
|
+
|
|
25
|
+
For L3-L7 media runs, capture the signals that distinguish software regressions from environment changes: onboarding/binding state, BSSID/channel/RSSI, reconnect or roaming events, audio/video send/receive/drop/error counters, queue watermarks, camera overflow count, internal heap/largest block, PSRAM, and measured latency when available. Never record credential values.
|
|
22
26
|
|
|
23
27
|
Reports describe observed current behavior. A `SKIP` caused by missing hardware, account, service, browser, or external network does not become a pass. If the user's requested completion level includes a skipped critical case, the final outcome remains incomplete.
|
|
@@ -19,11 +19,12 @@ The branch is complete when the new run has its own build and verification evide
|
|
|
19
19
|
Use this branch when the user supplies a board model, vendor URL, schematic, BOM, pin map, BSP, datasheets, photographs, or peripheral example projects without a verified adapter.
|
|
20
20
|
|
|
21
21
|
1. Resolve the full model, module, PCB marking, and hardware revision. Treat different revisions as different boards.
|
|
22
|
-
2.
|
|
23
|
-
3.
|
|
24
|
-
4.
|
|
25
|
-
5.
|
|
26
|
-
6.
|
|
22
|
+
2. Freeze the user-supplied product contract: selected video profile, audio/stream formats, duplex/AEC policy, supported Wi-Fi credential methods, selected onboarding method, transport staging, output path, and mutation boundary. Use `unknown` where the prompt lacks an answer.
|
|
23
|
+
3. Prefer official schematic/BOM and BSP facts. For a PDF schematic, inspect page labels and net names; prefer an exported netlist, pin CSV, or vendor board definition when available.
|
|
24
|
+
4. Cross-check critical pins, clocks, power enables, reset lines, sensor/codec variants, ESP-IDF version, resource ownership, and onboarding behavior across at least two independent artifacts when possible.
|
|
25
|
+
5. Create the Hardware IR v2. Use `null` for unknown facts and retain contradictory values as an explicit issue instead of selecting one silently. Store concrete board values in the IR/adapter rather than Skill files.
|
|
26
|
+
6. Validate and assess requested features. Ask only for unresolved facts that block the next safe step. SoftAP is optional when another evidenced Wi-Fi credential method is available, keeps credentials outside source, and defines reprovisioning.
|
|
27
|
+
7. When all requirements reach `READY_TO_PORT`, generate the starter and implement the board adapter. When requirements remain blocked, generate only the IR, capability report, and an optional compile-safe skeleton if requested.
|
|
27
28
|
|
|
28
29
|
The branch is complete when every supplied artifact maps to an IR fact, provenance entry, contradiction, or declared irrelevant item.
|
|
29
30
|
|
|
@@ -55,16 +56,16 @@ The runtime-facing `starter_media` interface stays stable. A reusable board inte
|
|
|
55
56
|
|
|
56
57
|
The adapter owns:
|
|
57
58
|
|
|
58
|
-
- camera capture and H.264
|
|
59
|
+
- camera capture and the selected MJPEG/H.264/H.265 media path;
|
|
59
60
|
- microphone capture and audio encoding;
|
|
60
61
|
- downlink audio decode, buffering, codec, amplifier, and I2S playback;
|
|
61
|
-
- DMA buffers, hardware clocks, power, reset, GPIO,
|
|
62
|
+
- DMA buffers, hardware clocks, power, reset, GPIO, refresh/key-frame requests, and realtime task allocation;
|
|
62
63
|
- bounded stop, resource release, and generation-aware flushing.
|
|
63
64
|
|
|
64
65
|
The stable modules own stream IDs, negotiated/contracted formats, TiRTC callback copying, connection handles, session generation, and H5/AI sequencing.
|
|
65
66
|
|
|
66
67
|
## Verification loop
|
|
67
68
|
|
|
68
|
-
Use a bounded loop per layer: diagnose one failing invariant, make the smallest correction, and rerun that layer before moving forward. Stop and report when the remaining failure requires unavailable hardware, credentials, a new SDK binary, a public protocol change, or a user choice.
|
|
69
|
+
Use a bounded loop per layer: diagnose one failing invariant, make the smallest correction, and rerun that layer before moving forward. Change one high-risk variable per HIL comparison. Turn reusable invariants into tests or post-link gates. Stop and report when the remaining failure requires unavailable hardware, credentials, a new SDK binary, a public protocol change, or a user choice.
|
|
69
70
|
|
|
70
|
-
Do not use successful compilation as evidence for camera frames, speaker output, Web rendering, AI audio, or long-run stability.
|
|
71
|
+
Do not use successful compilation as evidence for camera frames, speaker output, Web rendering, AI audio, or long-run stability. Bind every runtime conclusion to the tested firmware SHA-256.
|