tirtc-device-builder 0.9.4 → 0.9.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +1 -1
- package/CHANGELOG.md +18 -0
- package/README.md +20 -19
- package/bin/esp32-kit-metadata.js +8 -7
- package/bin/install-esp32-kit.js +13 -4
- package/bin/setup-esp32.js +11 -11
- package/package.json +1 -1
- package/skills/tirtc-esp32-builder/SKILL.md +9 -5
- package/skills/tirtc-esp32-builder/USAGE.md +2 -2
- package/skills/tirtc-esp32-builder/VERSION +1 -1
- package/skills/tirtc-esp32-builder/assets/hardware-ir-v2.example.json +2 -2
- package/skills/tirtc-esp32-builder/assets/hardware-ir.example.json +2 -2
- package/skills/tirtc-esp32-builder/assets/lckfb-szpi-esp32s3-portable-prompt.md +4 -4
- package/skills/tirtc-esp32-builder/knowledge/waveshare-esp32-p4-wifi6-touch-lcd-3.5/knowledge.md +65 -3
- package/skills/tirtc-esp32-builder/references/environment.md +9 -6
- package/skills/tirtc-esp32-builder/references/runtime-resource-review.md +9 -0
- package/skills/tirtc-esp32-builder/references/tirtc-platform.md +13 -4
- package/skills/tirtc-esp32-builder/references/video-orientation.md +117 -0
- package/skills/tirtc-esp32-builder/scripts/capture_uplink.py +220 -0
- package/skills/tirtc-esp32-builder/scripts/doctor.py +16 -5
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,24 @@
|
|
|
2
2
|
|
|
3
3
|
This project follows Semantic Versioning.
|
|
4
4
|
|
|
5
|
+
## 0.9.6
|
|
6
|
+
|
|
7
|
+
- Pin ESP32-S3 Device Kit 1.1.6 with TiRTC C SDK 2.5.0 and verify the
|
|
8
|
+
GitHub release archive checksum before npm publication.
|
|
9
|
+
- Build the Kit from the builder-owned ESP32 source and validate the SDK,
|
|
10
|
+
installer, project generator, and standalone reference firmware.
|
|
11
|
+
- Keep board-specific media adapters and hardware evidence separate from the
|
|
12
|
+
reusable Skill; hardware verification remains board-specific.
|
|
13
|
+
|
|
14
|
+
## 0.9.5
|
|
15
|
+
|
|
16
|
+
- Add focused video-orientation and WeChat UI-profile guidance distinguishing
|
|
17
|
+
encoded pixels, display size, rotation conventions and endpoint verification.
|
|
18
|
+
- Package the bounded XiaoTai serial capture/export helper with offline tests
|
|
19
|
+
for ANSI queries, evidence integrity and retained/empty capture recovery.
|
|
20
|
+
- Pin Waveshare product knowledge to a95f368 and document the watchpoint-proven
|
|
21
|
+
LVGL callback-count overwrite; preserve pending HIL and non-default UI angles.
|
|
22
|
+
|
|
5
23
|
## 0.9.4
|
|
6
24
|
|
|
7
25
|
- Add a knowledge-only Waveshare ESP32-P4-WIFI6-Touch-LCD-3.5 package with
|
package/README.md
CHANGED
|
@@ -8,11 +8,11 @@ TiRTC Device Builder 用于把 ESP32-S3/ESP32-P4 开发板接入 TiRTC。输入
|
|
|
8
8
|
|---|---|---|
|
|
9
9
|
| `tirtc-esp32-builder` | ESP32-S3 / ESP32-P4、ESP-IDF 5.5.x | 板型识别、Hardware IR、工程生成/移植、H5/AI/设备互呼/微信 VoIP、AEC 门禁、编译烧录和分层验收 |
|
|
10
10
|
|
|
11
|
-
H5/AI 的 ESP32-S3 托管模板、协议文档和 TiRTC SDK 已打包在独立的 ESP32 Device Kit
|
|
11
|
+
H5/AI 的 ESP32-S3 托管模板、协议文档和 TiRTC SDK 已打包在独立的 ESP32 Device Kit 中,安装时会自动下载并校验。ESP32 示例源码由本仓库的 `kit-src/` 维护;设备互呼或微信 VoIP 的移植可参考其中的独立示例。ESP32-P4 必须使用匹配的 P4 SDK、BSP 与网络方案,不能复用 S3 预编译库。
|
|
12
12
|
|
|
13
13
|
- npm 包:[tirtc-device-builder](https://www.npmjs.com/package/tirtc-device-builder)
|
|
14
14
|
- GitHub 仓库:[tangeai/tirtc-device-builder](https://github.com/tangeai/tirtc-device-builder)
|
|
15
|
-
- ESP32 Device Kit:[kit-esp32s3-v1.1.
|
|
15
|
+
- ESP32 Device Kit:[kit-esp32s3-v1.1.6](https://github.com/tangeai/tirtc-device-builder/releases/tag/kit-esp32s3-v1.1.6)
|
|
16
16
|
|
|
17
17
|
文档导航:
|
|
18
18
|
|
|
@@ -282,14 +282,16 @@ board-materials/
|
|
|
282
282
|
| 参考模组 | ESP32-S3-WROOM-1-N16R8,或资源和配置经过确认的兼容板 |
|
|
283
283
|
| ESP-IDF | 5.5.x |
|
|
284
284
|
| 自动安装版本 | ESP-IDF v5.5.4 |
|
|
285
|
-
| TiRTC SDK | `espressif-esp32s3/2.
|
|
286
|
-
| ESP32 Device Kit | 1.1.
|
|
285
|
+
| 本仓库源码与本地打包使用的 TiRTC SDK | `espressif-esp32s3/2.5.0` |
|
|
286
|
+
| ESP32 Device Kit | 1.1.6(内含 TiRTC SDK 2.5.0) |
|
|
287
287
|
| Node.js | 18 或更高版本 |
|
|
288
288
|
| 支持自动安装的系统 | Linux、WSL、macOS |
|
|
289
289
|
| 原生 Windows | 使用 Espressif 官方安装器准备 ESP-IDF,再重新运行检查 |
|
|
290
290
|
|
|
291
291
|
当前托管自动生成器只提供 ESP32-S3 模板。ESP32-P4 可由 Skill 在已有、证据完整的 P4 BSP/工程上移植,但必须使用 `espressif-esp32p4` SDK、匹配的构建合同和明确的 ESP-Hosted 或以太网方案。Flash 或 PSRAM 容量变化时,需要重新评估 `sdkconfig.defaults`、分区表、DMA 和媒体缓存预算。
|
|
292
292
|
|
|
293
|
+
从本仓库 `kit-src/` 直接生成或打包时使用 TiRTC 2.5.0;`setup esp32` 安装固定的 Kit 版本,其 SDK 版本以该 Kit 的 `manifest.json` 为准。
|
|
294
|
+
|
|
293
295
|
### 本机软件
|
|
294
296
|
|
|
295
297
|
`setup esp32 --install` 会检查以下基础命令:
|
|
@@ -392,7 +394,7 @@ Cline 当前还需要在 `Settings → Features → Enable Skills` 中启用实
|
|
|
392
394
|
|---|---|
|
|
393
395
|
| Agent Skill | 上表中所选目录下的 `tirtc-esp32-builder` |
|
|
394
396
|
| 托管根目录 | `~/.tirtc-device-builder` |
|
|
395
|
-
| Device Kit | `~/.tirtc-device-builder/kits/esp32s3/1.1.
|
|
397
|
+
| Device Kit | `~/.tirtc-device-builder/kits/esp32s3/1.1.6` |
|
|
396
398
|
| ESP-IDF | `~/.tirtc-device-builder/esp-idf-v5.5.4` |
|
|
397
399
|
| Espressif 工具 | `~/.tirtc-device-builder/espressif` |
|
|
398
400
|
| 安装记录 | `~/.tirtc-device-builder/config.json` |
|
|
@@ -437,7 +439,7 @@ npx --yes tirtc-device-builder@latest setup esp32 --install \
|
|
|
437
439
|
|
|
438
440
|
```bash
|
|
439
441
|
npx --yes tirtc-device-builder@latest setup esp32 --install \
|
|
440
|
-
--kit-archive /absolute/path/tirtc-esp32s3-kit-1.1.
|
|
442
|
+
--kit-archive /absolute/path/tirtc-esp32s3-kit-1.1.6.tar.gz
|
|
441
443
|
```
|
|
442
444
|
|
|
443
445
|
安装器仍会核对固定的 SHA-256、目录结构、清单和每个资源文件,不接受未经验证的同名压缩包。
|
|
@@ -820,7 +822,7 @@ npx --yes tirtc-device-builder@latest setup esp32 --install --force-skill
|
|
|
820
822
|
|
|
821
823
|
```bash
|
|
822
824
|
npx --yes tirtc-device-builder@latest setup esp32 --install \
|
|
823
|
-
--kit-archive /absolute/path/tirtc-esp32s3-kit-1.1.
|
|
825
|
+
--kit-archive /absolute/path/tirtc-esp32s3-kit-1.1.6.tar.gz
|
|
824
826
|
```
|
|
825
827
|
|
|
826
828
|
安装器会校验 SHA-256 和内部文件清单。如果校验不一致,请重新获取官方 Release 附件,不要跳过校验。
|
|
@@ -880,7 +882,7 @@ npx --yes tirtc-device-builder@latest setup esp32
|
|
|
880
882
|
|
|
881
883
|
不需要。普通开发所需的生成器、模板、协议文档、TiRTC 头文件和 `libTiRTC.a` 都在版本化的 ESP32 Device Kit 中。一键安装会下载公开 Release,并把路径写进 `env.sh`。
|
|
882
884
|
|
|
883
|
-
`--thing-connect-root`
|
|
885
|
+
维护者可用 `--thing-connect-root` 指向本仓库的 `kit-src/`;普通使用者使用安装好的 Device Kit 路径。
|
|
884
886
|
|
|
885
887
|
### 为什么推荐 `npx`,不是安装一个 npm 包就结束?
|
|
886
888
|
|
|
@@ -931,29 +933,28 @@ python3 ~/.codex/skills/.system/plugin-creator/scripts/validate_plugin.py .
|
|
|
931
933
|
|
|
932
934
|
```bash
|
|
933
935
|
npm run pack:esp32-kit -- \
|
|
934
|
-
--
|
|
935
|
-
--kit-version 1.1.4
|
|
936
|
+
--kit-version 1.1.6
|
|
936
937
|
```
|
|
937
938
|
|
|
938
939
|
输出位于 `dist/`:
|
|
939
940
|
|
|
940
941
|
```text
|
|
941
|
-
tirtc-esp32s3-kit-1.1.
|
|
942
|
-
tirtc-esp32s3-kit-1.1.
|
|
942
|
+
tirtc-esp32s3-kit-1.1.6.tar.gz
|
|
943
|
+
tirtc-esp32s3-kit-1.1.6.tar.gz.sha256
|
|
943
944
|
```
|
|
944
945
|
|
|
945
|
-
校验后推送独立的 `kit-esp32s3-v<version>` 标签。`publish-kit.yml` 会从 metadata
|
|
946
|
+
校验后推送独立的 `kit-esp32s3-v<version>` 标签。`publish-kit.yml` 会从 metadata 固定的本仓库源码 commit 重建压缩包、核对 SHA-256,并使用 GitHub Actions token 创建 Release:
|
|
946
947
|
|
|
947
948
|
```bash
|
|
948
949
|
cd dist
|
|
949
|
-
sha256sum -c tirtc-esp32s3-kit-1.1.
|
|
950
|
+
sha256sum -c tirtc-esp32s3-kit-1.1.6.tar.gz.sha256
|
|
950
951
|
cd ..
|
|
951
952
|
|
|
952
|
-
git tag -a kit-esp32s3-v1.1.
|
|
953
|
-
git push origin kit-esp32s3-v1.1.
|
|
953
|
+
git tag -a kit-esp32s3-v1.1.6 -m "TiRTC ESP32-S3 Device Kit 1.1.6"
|
|
954
|
+
git push origin kit-esp32s3-v1.1.6
|
|
954
955
|
```
|
|
955
956
|
|
|
956
|
-
metadata
|
|
957
|
+
metadata 中的版本、标签、本仓库源码 commit 和期望 SHA-256 必须与本地可复现打包结果一致;工作流不会从浮动的 `main` 取发布内容。
|
|
957
958
|
|
|
958
959
|
### 发布 npm
|
|
959
960
|
|
|
@@ -961,8 +962,8 @@ metadata 中的版本、标签、上游 commit 和期望 SHA-256 必须与本地
|
|
|
961
962
|
|
|
962
963
|
```bash
|
|
963
964
|
npm test
|
|
964
|
-
git tag -a v0.9.
|
|
965
|
-
git push origin v0.9.
|
|
965
|
+
git tag -a v0.9.6 -m "v0.9.6"
|
|
966
|
+
git push origin v0.9.6
|
|
966
967
|
```
|
|
967
968
|
|
|
968
969
|
不要重复发布已经存在的 npm 版本。版本变化同步更新 `package.json`、`.codex-plugin/plugin.json` 和发布说明。
|
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
export const ESP32_KIT = Object.freeze({
|
|
2
|
-
archiveName: "tirtc-esp32s3-kit-1.1.
|
|
3
|
-
archiveRoot: "tirtc-esp32s3-kit-1.1.
|
|
4
|
-
releaseTag: "kit-esp32s3-v1.1.
|
|
5
|
-
sha256: "
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
2
|
+
archiveName: "tirtc-esp32s3-kit-1.1.6.tar.gz",
|
|
3
|
+
archiveRoot: "tirtc-esp32s3-kit-1.1.6",
|
|
4
|
+
releaseTag: "kit-esp32s3-v1.1.6",
|
|
5
|
+
sha256: "61243703bdee3e1b492fcc6f5245143f419efbd2310127d180875f0708e91d8b",
|
|
6
|
+
sdkVersion: "2.5.0",
|
|
7
|
+
sourceCommit: "68a3b34054ea933d5ea5bfd8acf71e5ff07a7a61",
|
|
8
|
+
url: "https://github.com/tangeai/tirtc-device-builder/releases/download/kit-esp32s3-v1.1.6/tirtc-esp32s3-kit-1.1.6.tar.gz",
|
|
9
|
+
version: "1.1.6",
|
|
9
10
|
});
|
package/bin/install-esp32-kit.js
CHANGED
|
@@ -24,9 +24,6 @@ const REQUIRED_FILES = [
|
|
|
24
24
|
"device-sim/templates/esp32-h5-ai/CMakeLists.txt",
|
|
25
25
|
"device-sim/templates/esp32-h5-ai/platform-media-contract.json",
|
|
26
26
|
"device-sim/templates/esp32-h5-ai/tirtc-runtime-contract.json",
|
|
27
|
-
"device-sim/sdk/espressif-esp32s3/2.3.0/include/tirtc/tiRTC.h",
|
|
28
|
-
"device-sim/sdk/espressif-esp32s3/2.3.0/lib/libTiRTC.a",
|
|
29
|
-
"device-sim/sdk/espressif-esp32s3/2.3.0/manifest/build-contract.env",
|
|
30
27
|
];
|
|
31
28
|
|
|
32
29
|
function hashFile(path) {
|
|
@@ -98,9 +95,21 @@ function validateExtractedKit(root, metadata) {
|
|
|
98
95
|
`Device Kit version mismatch: expected ${metadata.version}, got ${manifest.kit_version}`,
|
|
99
96
|
);
|
|
100
97
|
}
|
|
101
|
-
if (manifest.target !== "esp32s3" ||
|
|
98
|
+
if (manifest.target !== "esp32s3" ||
|
|
99
|
+
!/^(?:2\.3\.0|2\.5\.0)$/.test(manifest.tirtc_sdk_version) ||
|
|
100
|
+
(metadata.sdkVersion && manifest.tirtc_sdk_version !== metadata.sdkVersion)) {
|
|
102
101
|
throw new Error("Device Kit target or TiRTC SDK version is incompatible");
|
|
103
102
|
}
|
|
103
|
+
const sdkRoot = `device-sim/sdk/espressif-esp32s3/${manifest.tirtc_sdk_version}`;
|
|
104
|
+
const sdkFiles = [
|
|
105
|
+
`${sdkRoot}/include/tirtc/tiRTC.h`,
|
|
106
|
+
`${sdkRoot}/lib/libTiRTC.a`,
|
|
107
|
+
`${sdkRoot}/manifest/build-contract.env`,
|
|
108
|
+
];
|
|
109
|
+
const missingSdk = sdkFiles.filter((path) => !existsSync(join(root, path)));
|
|
110
|
+
if (missingSdk.length > 0) {
|
|
111
|
+
throw new Error(`Device Kit SDK is incomplete: ${missingSdk.join(", ")}`);
|
|
112
|
+
}
|
|
104
113
|
if (!manifest.files || typeof manifest.files !== "object") {
|
|
105
114
|
throw new Error("Device Kit manifest has no file checksums");
|
|
106
115
|
}
|
package/bin/setup-esp32.js
CHANGED
|
@@ -29,12 +29,6 @@ const GENERATOR_PATH = join(
|
|
|
29
29
|
"scripts",
|
|
30
30
|
"create_esp32_project.py",
|
|
31
31
|
);
|
|
32
|
-
const SDK_PATH = join(
|
|
33
|
-
"device-sim",
|
|
34
|
-
"sdk",
|
|
35
|
-
"espressif-esp32s3",
|
|
36
|
-
"2.3.0",
|
|
37
|
-
);
|
|
38
32
|
const REQUIRED_SDK_FILES = [
|
|
39
33
|
join("include", "tirtc", "tiRTC.h"),
|
|
40
34
|
join("lib", "libTiRTC.a"),
|
|
@@ -233,6 +227,7 @@ export function inspectDeviceKit(root) {
|
|
|
233
227
|
const manifestPath = root ? join(root, DEVICE_KIT_MANIFEST) : null;
|
|
234
228
|
let manifestPresent = false;
|
|
235
229
|
let version = null;
|
|
230
|
+
let sdkVersion = null;
|
|
236
231
|
let manifestError = null;
|
|
237
232
|
if (manifestPath && existsSync(manifestPath)) {
|
|
238
233
|
manifestPresent = true;
|
|
@@ -245,6 +240,9 @@ export function inspectDeviceKit(root) {
|
|
|
245
240
|
manifest.kit_version.trim()
|
|
246
241
|
) {
|
|
247
242
|
version = manifest.kit_version.trim();
|
|
243
|
+
if (typeof manifest.tirtc_sdk_version === "string") {
|
|
244
|
+
sdkVersion = manifest.tirtc_sdk_version;
|
|
245
|
+
}
|
|
248
246
|
} else {
|
|
249
247
|
manifestError = "manifest.json does not declare kit_version";
|
|
250
248
|
}
|
|
@@ -258,11 +256,13 @@ export function inspectDeviceKit(root) {
|
|
|
258
256
|
const generatorReady = Boolean(
|
|
259
257
|
root && existsSync(join(root, GENERATOR_PATH)),
|
|
260
258
|
);
|
|
261
|
-
const
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
259
|
+
const sdkVersions = sdkVersion
|
|
260
|
+
? [sdkVersion]
|
|
261
|
+
: [...new Set(["2.5.0", ESP32_KIT.sdkVersion])];
|
|
262
|
+
const sdkReady = Boolean(root && sdkVersions.some((candidate) => {
|
|
263
|
+
const sdk = join(root, "device-sim", "sdk", "espressif-esp32s3", candidate);
|
|
264
|
+
return REQUIRED_SDK_FILES.every((relative) => existsSync(join(sdk, relative)));
|
|
265
|
+
}));
|
|
266
266
|
const structureReady = generatorReady && sdkReady;
|
|
267
267
|
const versionCompatible =
|
|
268
268
|
!manifestPresent ||
|
package/package.json
CHANGED
|
@@ -20,11 +20,12 @@ past build as proof for a new artifact.
|
|
|
20
20
|
talkback, AI intercom, device-to-device calling, and WeChat VoIP. Do not
|
|
21
21
|
silently remove a requested feature because the current starter lacks it.
|
|
22
22
|
3. Resolve a versioned source of truth. The managed ESP32-S3 Device Kit is the
|
|
23
|
-
default for its packaged capabilities
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
when the
|
|
27
|
-
|
|
23
|
+
default for its packaged capabilities; this repository's `kit-src/` owns the
|
|
24
|
+
ESP32 generator, SDK and standalone reference implementation. For
|
|
25
|
+
device-call/WeChat protocol simulation use a pinned `tirtc-server-example`
|
|
26
|
+
checkout when the Kit lacks the Linux C simulator and protocol documents.
|
|
27
|
+
Clone or download it only when the user authorizes that external write.
|
|
28
|
+
Read the exact business documents routed by `workflow.md`.
|
|
28
29
|
4. Run Doctor for the exact target and SDK package. Managed generation currently
|
|
29
30
|
automates ESP32-S3. ESP32-P4 is valid only with an evidenced P4 BSP/network
|
|
30
31
|
path and the matching `espressif_esp32p4` SDK/build contract; never link the
|
|
@@ -48,6 +49,9 @@ are source maps and evidence, not permission to install a candidate adapter.
|
|
|
48
49
|
For an existing project's media, memory or scheduling bug, read
|
|
49
50
|
[runtime-resource-review.md](references/runtime-resource-review.md); preserve
|
|
50
51
|
the user's business baseline separately from the hardware reference.
|
|
52
|
+
For video orientation, unexpected cropping, or WeChat display-profile changes,
|
|
53
|
+
read [video-orientation.md](references/video-orientation.md) before changing
|
|
54
|
+
pixel transforms or UI angles.
|
|
51
55
|
|
|
52
56
|
## 3. Prove the business flows before board porting
|
|
53
57
|
|
|
@@ -74,7 +74,7 @@ printf '%s\n' "$TIRTC_THING_CONNECT_ROOT"
|
|
|
74
74
|
python3 <skill-dir>/scripts/doctor.py \
|
|
75
75
|
--expected-idf 5.5 \
|
|
76
76
|
--target esp32s3 \
|
|
77
|
-
--thing-connect-root ~/.tirtc-device-builder/kits/esp32s3/1.1.
|
|
77
|
+
--thing-connect-root ~/.tirtc-device-builder/kits/esp32s3/1.1.6 \
|
|
78
78
|
--require-workspace
|
|
79
79
|
```
|
|
80
80
|
|
|
@@ -119,4 +119,4 @@ python3 <skill-dir>/scripts/firmware_identity.py build/<app>.bin \
|
|
|
119
119
|
|
|
120
120
|
## 当前边界
|
|
121
121
|
|
|
122
|
-
|
|
122
|
+
Device Kit 提供 ESP32-S3 H5/AI 模板和生成器,但默认媒体适配器不包含特定开发板的摄像头、麦克风、选定视频路径和扬声器驱动。模板生成和编译成功只证明工程与协议骨架可用,不代表 Web 已经出图或 AI 音频已经通过实机验收。
|
|
@@ -1 +1 @@
|
|
|
1
|
-
0.9.
|
|
1
|
+
0.9.6
|
|
@@ -29,8 +29,8 @@
|
|
|
29
29
|
"verification": "extracted",
|
|
30
30
|
"tirtc": {
|
|
31
31
|
"platform": "espressif-esp32s3",
|
|
32
|
-
"version": "2.
|
|
33
|
-
"sdk_path": "device-sim/sdk/espressif-esp32s3/2.
|
|
32
|
+
"version": "2.5.0",
|
|
33
|
+
"sdk_path": "device-sim/sdk/espressif-esp32s3/2.5.0",
|
|
34
34
|
"build_contract": "manifest/build-contract.env"
|
|
35
35
|
},
|
|
36
36
|
"source_refs": [
|
|
@@ -29,8 +29,8 @@
|
|
|
29
29
|
"verification": "extracted",
|
|
30
30
|
"tirtc": {
|
|
31
31
|
"platform": "espressif-esp32s3",
|
|
32
|
-
"version": "2.
|
|
33
|
-
"sdk_path": "device-sim/sdk/espressif-esp32s3/2.
|
|
32
|
+
"version": "2.5.0",
|
|
33
|
+
"sdk_path": "device-sim/sdk/espressif-esp32s3/2.5.0",
|
|
34
34
|
"build_contract": "manifest/build-contract.env"
|
|
35
35
|
},
|
|
36
36
|
"source_refs": [
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
10
|
export TIRTC_AGENT_CLIENT=codex
|
|
11
|
-
npx --yes tirtc-device-builder@0.9.
|
|
11
|
+
npx --yes tirtc-device-builder@0.9.6 setup esp32 --install \
|
|
12
12
|
--client "$TIRTC_AGENT_CLIENT" --force-skill
|
|
13
13
|
```
|
|
14
14
|
|
|
@@ -17,11 +17,11 @@ npx --yes tirtc-device-builder@0.9.4 setup esp32 --install \
|
|
|
17
17
|
本轮第一步先只读运行:
|
|
18
18
|
|
|
19
19
|
```bash
|
|
20
|
-
npx --yes tirtc-device-builder@0.9.
|
|
21
|
-
npx --yes tirtc-device-builder@0.9.
|
|
20
|
+
npx --yes tirtc-device-builder@0.9.6 --version
|
|
21
|
+
npx --yes tirtc-device-builder@0.9.6 setup esp32 --client "$TIRTC_AGENT_CLIENT"
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
必须根据命令的实际输出和本机文件确认:npm 包为 0.9.
|
|
24
|
+
必须根据命令的实际输出和本机文件确认:npm 包为 0.9.6、已安装 Skill 的 `VERSION` 为 0.9.6、所选 Device Kit 的 `manifest.json` 中 `kit_version` 为 1.1.6、`tirtc_sdk_version` 为 2.5.0,并且 Doctor 对 `--expected-kit 1.1.6` 输出 `OVERALL: PASS`。Plugin manifest 不属于这种 npm 安装方式的运行时前置条件,不得把不可访问的 Plugin 版本当作阻塞项。如果版本不一致、Skill 是在当前会话启动后才安装,或环境检查未通过,停止并报告前置条件不成立;不要在当前会话中替换 Skill 后继续生成工程。
|
|
25
25
|
|
|
26
26
|
工作区与 clean-room 边界:
|
|
27
27
|
- 将启动 Agent 客户端时的当前目录定义为 `WORKSPACE_ROOT`。
|
package/skills/tirtc-esp32-builder/knowledge/waveshare-esp32-p4-wifi6-touch-lcd-3.5/knowledge.md
CHANGED
|
@@ -9,7 +9,7 @@ authorize pin reuse. No credentials, raw call logs or user media are retained.
|
|
|
9
9
|
|
|
10
10
|
| Baseline | Pinned source | Locate next |
|
|
11
11
|
| --- | --- | --- |
|
|
12
|
-
| Product business | xiaotai
|
|
12
|
+
| Product business | xiaotai `a95f368a037eb3ebe88d8564e1cb2f85e369f525` | `lckfb-szpi-esp32s3-tirtc/components/starter_runtime/src/starter_runtime.c`, shared `starter_product`, `starter_tirtc` |
|
|
13
13
|
| Hardware/media | tirtc-device-example `cad4cbe58c3ff451906322930ceede19e6bf6a07`, app 1.5.3 | `complete-applications/esp32-p4/device-monitor/main/` |
|
|
14
14
|
| Vendor | waveshareteam/ESP32-P4-WIFI6-Touch-LCD-3.5 `870588e62daaf723f3c52f0060d8dc053665ed30` | `examples/esp-idf/06_I2SCodec`, `docs/revisions.md`, schematic |
|
|
15
15
|
| Cross-check | xiaozhi-esp32 `1ce658bcb9ce9aac69d3f87af9894a715b7175f6` | `main/boards/waveshare/esp32-p4-wifi6-touch-lcd-3.5/` |
|
|
@@ -59,7 +59,7 @@ is in `waveshare-esp32p4-xiaotai/PORT_STATUS.md` and `BUILD_IDENTITY.md`.
|
|
|
59
59
|
| UI slow / video not smooth | Product UI tick100ms limits presentation near10fps; renderer input24x256KiB plus output20x480x320x2 dominate pools. Retained pools are intentional, not proven optimal for this product. | `call_video_renderer_config.h`, `p4_video_ui_tick`; measure before tuning |
|
|
60
60
|
|
|
61
61
|
Product-specific media: device CALL H264 both directions; WeChat VOIP H264 up /
|
|
62
|
-
MJPEG down, profile video enabled with
|
|
62
|
+
MJPEG down, profile video enabled with actual screen 480x320, 8k mono A-law. Preserve explicit
|
|
63
63
|
voice calls. Camera privacy disables local capture/send while keeping remote
|
|
64
64
|
decode/audio; microphone is independent. Profile declaration is not proof of
|
|
65
65
|
actual packets or display. P4 H264 encode is hardware; downstream H264 here is
|
|
@@ -78,7 +78,69 @@ These identify historical local artifacts, not bundled binaries or a reproducibl
|
|
|
78
78
|
release. User reports video visible after the stream-filter correction, without
|
|
79
79
|
a complete artifact-bound capture. Startup logs prove one successful media
|
|
80
80
|
start, not long-run stability. Earlier HMAC signing -> invalid LVGL event crash
|
|
81
|
-
|
|
81
|
+
was subsequently traced to an LVGL overwrite, as detailed below; task cleanup
|
|
82
|
+
was a separate fix. AEC/double-talk, weak
|
|
82
83
|
network, repeated calls, PMIC controls and full product parity remain pending.
|
|
83
84
|
Promote only after exact identity, portable adapter/IR/contracts and per-flow
|
|
84
85
|
artifact-bound acceptance are retained.
|
|
86
|
+
|
|
87
|
+
## Later findings at the committed product baseline
|
|
88
|
+
|
|
89
|
+
### LVGL callback counter overwrites an unrelated allocator hook
|
|
90
|
+
|
|
91
|
+
The diagnostic `diag.9` hardware watchpoint caught `lv_obj_add_event_cb` writing
|
|
92
|
+
the mbedTLS calloc function pointer while `render_page` registered the screen
|
|
93
|
+
event again. `lv_obj_clean(screen)` removed children, not screen callbacks.
|
|
94
|
+
The selected LVGL descriptor count was six bits: registration 64 wrapped it,
|
|
95
|
+
zero-sized realloc returned the allocator's sentinel, and descriptor index -1
|
|
96
|
+
wrote 12 bytes before that sentinel. An ensuing HMAC allocation jumped into
|
|
97
|
+
`on_screen_event(e=1)`. The apparent crypto/LVGL call stack was a consequence,
|
|
98
|
+
not an allocator ABI diagnosis or proof of stack exhaustion.
|
|
99
|
+
|
|
100
|
+
Correction: register once when the persistent screen is initialized, not on
|
|
101
|
+
every page rebuild; remove temporary watchpoints after capturing the writer.
|
|
102
|
+
`tools/test_screen_event_lifetime.py` in the S3 project runs 512 actual setup
|
|
103
|
+
paths against the relevant LVGL counter logic. Source/build regression passed;
|
|
104
|
+
post-fix sustained HIL is not established by that test. Watchpoint addresses
|
|
105
|
+
belonged to that ELF and must never become hardcoded protection logic.
|
|
106
|
+
|
|
107
|
+
### Orientation, aspect ratio and profile experiments
|
|
108
|
+
|
|
109
|
+
Read [video-orientation.md](../../references/video-orientation.md) for the
|
|
110
|
+
general procedure and protocol fields. The capture.13 pre-SDK PNG showed the
|
|
111
|
+
upright test card clockwise90; thus the device's encoded pixels were already
|
|
112
|
+
sideways, not merely H5 CSS. The original H264 was not retained at inspection,
|
|
113
|
+
so 1280x960 is corroborated by PNG/capture metadata, not a fresh SPS inspection.
|
|
114
|
+
|
|
115
|
+
The upright.14 / voipdir.15 implementation captures native1280x960 and uses
|
|
116
|
+
PPA CCW90 before H264 encode, producing960x1280@15 / target2Mbps without crop
|
|
117
|
+
or spatial scale. It adds about1,843,200 bytes of rotation-output PSRAM and one
|
|
118
|
+
PPA transaction. The source geometry and submission tests cover exchanged axes,
|
|
119
|
+
zero crop offsets, 1:1 scale and rotation-aware resource matching. CALL's
|
|
120
|
+
decoder-limited profile remains separate; these values are not maximum ratings.
|
|
121
|
+
|
|
122
|
+
The user reported H5 orientation correct and P4's MJPEG CW90 display correct
|
|
123
|
+
with voipdir.15, but mini-program rendering still wrong. These are user-reported
|
|
124
|
+
per-direction observations, not a complete artifact-bound acceptance bundle.
|
|
125
|
+
voipui.16 added UI fields with rotation0 / aspect0.75 / mirrorsfalse / contain.
|
|
126
|
+
The subsequent request selected **additional UI rotation270** in voipui.17;
|
|
127
|
+
that latest setting compiled but has **no confirming post-change HIL**. Do not
|
|
128
|
+
promote270 into the board registry's default or claim it fixes every endpoint.
|
|
129
|
+
|
|
130
|
+
- Latest compiled BIN SHA256: `8de6f919acd120fe283db17ffd653d347a5850df715e1f3879e7910b3b3df29c`
|
|
131
|
+
- Latest compiled ELF SHA256: `2aecb8fb9e81347ef9aed76371c5f0404364b687f393df8fb4ae2099edfddc1c`
|
|
132
|
+
- Source: `waveshare-esp32p4-xiaotai/main/media/{camera_pipeline,video_yuv420_scaler}.c`,
|
|
133
|
+
`main/services/call_video_renderer.c`; shared `request_voip_profile`.
|
|
134
|
+
- Tests: P4 `tools/test_full_frame_uplink.py`, `test_uplink_rotation.py`,
|
|
135
|
+
`test_voip_profile.py`. Local SDK PPA enums are CCW, UI angles CW.
|
|
136
|
+
|
|
137
|
+
### Reusable capture tool boundary
|
|
138
|
+
|
|
139
|
+
The packaged `scripts/capture_uplink.py` is derived from the committed P4
|
|
140
|
+
`tools/capture_uplink.py`. Its device counterpart is
|
|
141
|
+
`components/p4_hardware/p4_video_capture.c` plus the pre-SDK hook in `p4_video.c`.
|
|
142
|
+
Device firmware is not shipped by this knowledge-only package. Read product
|
|
143
|
+
`VIDEO_CAPTURE.md` and confirm compatible commands before running the helper.
|
|
144
|
+
The helper's actual dump parsing, fragmented ANSI query handling, stopped/empty
|
|
145
|
+
recovery and disconnect classification have packaged host regressions. They do
|
|
146
|
+
not establish unattended operation on an untested USB/terminal combination.
|
|
@@ -16,7 +16,7 @@ The automatic branch never runs `sudo` or modifies a persistent shell profile. W
|
|
|
16
16
|
When `<setup-root>/env.sh` exists, use it only as an activation prefix for the current command:
|
|
17
17
|
|
|
18
18
|
```bash
|
|
19
|
-
bash -lc '. "<setup-root>/env.sh" && python3 "<skill-dir>/scripts/doctor.py" --expected-idf 5.5 --expected-kit 1.1.
|
|
19
|
+
bash -lc '. "<setup-root>/env.sh" && python3 "<skill-dir>/scripts/doctor.py" --expected-idf 5.5 --expected-kit 1.1.6 --target esp32s3 --require-workspace'
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
The helper contains paths, not device or network credentials. Read `<setup-root>/config.json` when exact managed paths are needed; the environment helper does not authorize unrelated downloads, shell-profile changes, flashing, or credential writes.
|
|
@@ -26,7 +26,7 @@ Run the doctor before generation, build, flash, or monitor:
|
|
|
26
26
|
```bash
|
|
27
27
|
python3 <skill-dir>/scripts/doctor.py \
|
|
28
28
|
--expected-idf 5.5 \
|
|
29
|
-
--expected-kit 1.1.
|
|
29
|
+
--expected-kit 1.1.6 \
|
|
30
30
|
--target esp32s3 \
|
|
31
31
|
--require-workspace
|
|
32
32
|
```
|
|
@@ -44,15 +44,18 @@ For an explicitly selected legacy workspace, omit `--expected-kit`; otherwise re
|
|
|
44
44
|
1. an explicit `--thing-connect-root <path>`;
|
|
45
45
|
2. `TIRTC_THING_CONNECT_ROOT`;
|
|
46
46
|
3. an ancestor of the project or current directory containing `device-sim/scripts/create_esp32_project.py`;
|
|
47
|
-
4. an ancestor whose `thing-connect/` child contains that generator.
|
|
47
|
+
4. an ancestor whose `kit-src/` or `thing-connect/` child contains that generator.
|
|
48
48
|
|
|
49
|
-
The default managed root is `<setup-root>/kits/esp32s3/<kit-version>`. The
|
|
49
|
+
The default managed root is `<setup-root>/kits/esp32s3/<kit-version>`. The `tirtc-device-builder/kit-src` directory is the ESP32 development source; the doctor also accepts an installed Device Kit root. Older `thing-connect/` layouts remain accepted as legacy inputs.
|
|
50
|
+
Resolve the SDK from the selected Kit manifest or generated project. Kit 1.1.6
|
|
51
|
+
and this repository's source generator use the ESP32-S3 SDK 2.5.0.
|
|
50
52
|
|
|
51
53
|
The managed Kit currently contains the ESP32-S3 H5/AI starter. When the requested
|
|
52
54
|
portfolio includes device-to-device calling or WeChat VoIP and the selected Kit
|
|
53
55
|
does not contain `device-sim-c`, `device-call.md`, `device-voip.md`, and the API
|
|
54
|
-
reference, use
|
|
55
|
-
|
|
56
|
+
reference, use this repository's `kit-src/device-sim/device-sim-esp32` for the
|
|
57
|
+
ESP32 implementation and a pinned `tirtc-server-example` commit for server
|
|
58
|
+
protocols and non-ESP32 simulators. Cloning is an external write and requires
|
|
56
59
|
user authorization. Do not track a moving default branch as build evidence.
|
|
57
60
|
|
|
58
61
|
ESP32-P4 uses a separate `espressif_esp32p4` SDK archive and RISC-V toolchain.
|
|
@@ -29,6 +29,15 @@ specific, not generic magic constants.
|
|
|
29
29
|
|
|
30
30
|
## Resource lifecycle
|
|
31
31
|
|
|
32
|
+
- A callback reached from an unrelated allocator/crypto call can be the victim
|
|
33
|
+
of an earlier overwrite. Validate the exact ELF, inspect the indirect-call
|
|
34
|
+
target and use a narrowly scoped watchpoint to catch the writer. In LVGL,
|
|
35
|
+
persistent screen callbacks belong to screen creation, not every child-page
|
|
36
|
+
rebuild: cleaning children does not remove callbacks on their parent. Check
|
|
37
|
+
the actual version's descriptor counter width. See the Waveshare knowledge
|
|
38
|
+
package for the captured six-bit overflow case; numeric addresses are not
|
|
39
|
+
portable fixes.
|
|
40
|
+
|
|
32
41
|
- Inventory task creation and deletion pairs. ESP-IDF `xTaskCreate*WithCaps`
|
|
33
42
|
requires `vTaskDeleteWithCaps`; ordinary deletion retains its statically
|
|
34
43
|
registered stack/TCB. Confirm the installed IDF implementation. Self-deletion
|
|
@@ -20,7 +20,7 @@ used by a build.
|
|
|
20
20
|
## SDK lifecycle
|
|
21
21
|
|
|
22
22
|
TiRTC is one process-wide runtime. The exact selected header governs API and
|
|
23
|
-
option availability. For SDK 2.
|
|
23
|
+
option availability. For SDK 2.5.0 the important order is:
|
|
24
24
|
|
|
25
25
|
1. Set `TIRTC_OPT_MAX_SEND_BUFFER` before `TiRtcInit()` when overriding it.
|
|
26
26
|
2. Call `TiRtcInit()` once.
|
|
@@ -33,7 +33,12 @@ option availability. For SDK 2.3.0 the important order is:
|
|
|
33
33
|
6. Stop sessions and connections through deferred lifecycle work before the one
|
|
34
34
|
final `TiRtcStop()` / `TiRtcUninit()` sequence.
|
|
35
35
|
|
|
36
|
-
The callback table and its context outlive the SDK runtime.
|
|
36
|
+
The callback table and its context outlive the SDK runtime. SDK 2.5.0 adds
|
|
37
|
+
`on_sleep_wakeup_info` at the end of `TIRTCCALLBACKS`; zero-initialize the full
|
|
38
|
+
struct and keep it `NULL` when sleep wakeup is unused. If supplied, this callback
|
|
39
|
+
can run synchronously inside `TiRtcStart()`. Set a custom
|
|
40
|
+
`TIRTC_OPT_TGTRP_POLL_TIMEOUT` after `TiRtcInit()`, which resets it to 10 ms.
|
|
41
|
+
Callback payloads
|
|
37
42
|
are borrowed; copy required data into bounded application-owned storage before
|
|
38
43
|
returning.
|
|
39
44
|
|
|
@@ -68,6 +73,10 @@ authorized contact list, route WeChat MQTT events, and use `POST
|
|
|
68
73
|
selects an authorized contact; it is not a direct SDK API. Mini-program
|
|
69
74
|
authorization and plugin behavior are separate platform acceptance evidence.
|
|
70
75
|
|
|
76
|
+
For video profile fields and orientation, use
|
|
77
|
+
[video-orientation.md](video-orientation.md). Screen dimensions, encoded-frame
|
|
78
|
+
aspect ratio and mini-program display transforms describe different boundaries.
|
|
79
|
+
|
|
71
80
|
## Simulator before hardware
|
|
72
81
|
|
|
73
82
|
For the four-feature portfolio, use the pinned Linux C reference implementation
|
|
@@ -81,8 +90,8 @@ drop a requested feature or call it verified.
|
|
|
81
90
|
|
|
82
91
|
## ESP32 targets
|
|
83
92
|
|
|
84
|
-
The current managed generator and Device Kit automate ESP32-S3.
|
|
85
|
-
|
|
93
|
+
The current managed generator and Device Kit automate ESP32-S3. ESP32-P4 requires
|
|
94
|
+
a distinct target-specific SDK package. P4 needs ESP-IDF 5.5.4, matching
|
|
86
95
|
RISC-V toolchain/build contract, PSRAM, and an evidenced network path such as
|
|
87
96
|
ESP-Hosted with C6/C61 or Ethernet. P4 and S3 archives are not interchangeable.
|
|
88
97
|
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Video geometry, orientation and WeChat UI
|
|
2
|
+
|
|
3
|
+
Use when a video appears sideways, mirrored, cropped/zoomed, or differs between
|
|
4
|
+
H5 and WeChat. Keep four independent facts: sensor output, encoded pixels,
|
|
5
|
+
device display viewport, and remote player's UI transform. An absent rotate
|
|
6
|
+
call does not prove upright pixels: camera mounting can already introduce an
|
|
7
|
+
offset. H5 working does not prove the VoIP-mode encoded pixels are identical.
|
|
8
|
+
|
|
9
|
+
## Locate the transform
|
|
10
|
+
|
|
11
|
+
Keep the board and an asymmetric upright test card stationary. Use readable
|
|
12
|
+
text plus top/left/right marks; rotation alone and rotation plus reflection
|
|
13
|
+
are distinct. State only what the screenshot establishes, not where the
|
|
14
|
+
transform happened. Avoid repeatedly asking for placement once confirmed.
|
|
15
|
+
|
|
16
|
+
Compare short **pre-SDK** H264 clips from each affected mode, with the exact
|
|
17
|
+
firmware identity and active generation. Start at SPS/PPS plus IDR, decode
|
|
18
|
+
without automatic rotation, inspect pixels, and read coded width/height from
|
|
19
|
+
SPS (for example ffprobe). Callback dimensions are useful corroboration, not
|
|
20
|
+
a substitute for SPS. A truncated printed ELF hash is not a full identity.
|
|
21
|
+
|
|
22
|
+
- Matching source clips but different displays: inspect SDK/bridge and player
|
|
23
|
+
UI settings rather than globally rotating the camera again.
|
|
24
|
+
- Different source clips: inspect mode-specific capture, crop, encoder input
|
|
25
|
+
and profile transitions first.
|
|
26
|
+
- One correctly oriented receive direction says nothing about the opposite
|
|
27
|
+
direction. Apply and test corrections at the intended boundary only.
|
|
28
|
+
|
|
29
|
+
Retain separate claims for observed pixels, verified source configuration,
|
|
30
|
+
successful compilation, and post-change hardware observations.
|
|
31
|
+
|
|
32
|
+
## Geometry and resources
|
|
33
|
+
|
|
34
|
+
For a full-frame 90-degree rotation, exchange width/height. A 1280x960 sensor
|
|
35
|
+
frame becomes 960x1280; preserving complete pixels is not the same as forcing
|
|
36
|
+
a landscape aspect ratio. Validate encoder width AND height limits, alignment,
|
|
37
|
+
crop coordinates in input axes, scale factors, clamp logic, direct-input
|
|
38
|
+
bypasses, and reserved-resource matching including rotation. Budget an extra
|
|
39
|
+
output surface and conversion time. PPA angles in the ESP-IDF P4 driver are
|
|
40
|
+
counterclockwise; check the selected header instead of sharing enum values
|
|
41
|
+
with a clockwise UI API.
|
|
42
|
+
|
|
43
|
+
Treat a conservative reference profile as policy, not silicon capacity. Verify
|
|
44
|
+
the installed sensor's native modes and actual output; selecting an 800x640
|
|
45
|
+
mode and taking its central 640x480 narrows field of view even without a
|
|
46
|
+
digital zoom setting. Disable or expose automatic resolution fallback when
|
|
47
|
+
the product requires an exact output. Keep H5/phone encode capability separate
|
|
48
|
+
from a peer P4's software H264 decode limits.
|
|
49
|
+
|
|
50
|
+
## WeChat profile contract
|
|
51
|
+
|
|
52
|
+
Checked 2026-09-07 against the primary
|
|
53
|
+
[profile API](https://github.com/tangeai/tirtc-server-example/blob/main/thing-connect/api-reference.md#post-v1voipdeviceprofile)
|
|
54
|
+
and [VoIP guide](https://github.com/tangeai/tirtc-server-example/blob/main/thing-connect/device-voip.md).
|
|
55
|
+
These URLs track main: pin/recheck the selected platform revision before a
|
|
56
|
+
new integration. Do not infer protocol absence from an older board example.
|
|
57
|
+
|
|
58
|
+
| Field | Boundary / constraint |
|
|
59
|
+
| --- | --- |
|
|
60
|
+
| `screen_width`, `screen_height` | Actual device display region, not uplink coded dimensions |
|
|
61
|
+
| `camera_rotation` | Additional clockwise mini-program UI angle, one of 0/90/180/270; default 0 |
|
|
62
|
+
| `aspect_ratio` | Device video's width/height, positive; default 4/3 can be wrong for portrait output |
|
|
63
|
+
| `hor_mirror`, `vert_mirror` | Mini-program horizontal/vertical reflection; defaults false |
|
|
64
|
+
| `object_fit` | Mini-program `fill` or `contain`; default fill |
|
|
65
|
+
| `video_res_mode` | Downlink `auto`, `fit_screen`, or `fill_screen`; does not rotate |
|
|
66
|
+
|
|
67
|
+
The five UI fields do not configure TiRTC encoding. `fit_screen` bounds MJPEG
|
|
68
|
+
inside the screen without cropping/upscaling; `fill_screen` can upscale and
|
|
69
|
+
center-crop. Both need valid screen sizes; fill requires even dimensions.
|
|
70
|
+
Do not silently change a working downlink mode while investigating uplink UI.
|
|
71
|
+
The profile JSON must fit 512 bytes. Test actual serialized fields, types and
|
|
72
|
+
size, not only log text. Re-report after changes and start a fresh call so the
|
|
73
|
+
mini-program consumes updated configuration.
|
|
74
|
+
|
|
75
|
+
Choose UI angle relative to the already-transformed encoded pixels; avoid
|
|
76
|
+
blindly reporting a physical mounting angle. If pixel correction is already
|
|
77
|
+
upright, zero additional UI rotation is a hypothesis to test, not an absolute
|
|
78
|
+
rule. A observed need for another angle is endpoint-specific evidence: retain
|
|
79
|
+
H5/VoIP captures and test the new call before calling it verified. In particular,
|
|
80
|
+
the Waveshare experiment's requested 270 is not a universal board default.
|
|
81
|
+
|
|
82
|
+
## Bounded capture and serial recovery
|
|
83
|
+
|
|
84
|
+
The packaged `scripts/capture_uplink.py` supports the **XiaoTai video-capture
|
|
85
|
+
console protocol**, not arbitrary TiRTC firmware. It does not flash or install
|
|
86
|
+
firmware. Source and device-side integration are pinned in the Waveshare
|
|
87
|
+
knowledge package; verify `video-capture` exists before use. Host requirements:
|
|
88
|
+
Python plus pyserial on POSIX; ffprobe/ffmpeg are optional for decoded PNG.
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
python3 <skill-dir>/scripts/capture_uplink.py --port <exact-port> --output <new-path>.h264
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Use test-card media only, with explicit serial/capture authorization. The
|
|
95
|
+
firmware allocates at most 512 KiB PSRAM on demand and copies complete frames
|
|
96
|
+
without blocking for a lock or doing file/serial I/O in the encoder callback.
|
|
97
|
+
It stops on capacity, generation/size change, or contention. Approximate
|
|
98
|
+
two-second duration is not guaranteed; missing SPS/PPS/IDR can leave it empty.
|
|
99
|
+
Export is console-task work after stop and disconnect. No network upload.
|
|
100
|
+
|
|
101
|
+
The helper answers ANSI status/cursor queries (including queries without a
|
|
102
|
+
newline), preserves partial serial lines, uses CR and clears stale command
|
|
103
|
+
input. It opens the port exclusively where supported, but a read interruption
|
|
104
|
+
alone does not distinguish another reader from USB disconnect or reboot.
|
|
105
|
+
Check process ownership and USB logs without killing unrelated processes.
|
|
106
|
+
Display safe panic/reboot markers rather than dumping credential-bearing logs.
|
|
107
|
+
|
|
108
|
+
Recovery is explicit: use `--dump-only` to stop then inspect retained capture;
|
|
109
|
+
keep nonempty media, release only an inactive empty buffer, and ask the user
|
|
110
|
+
to close the H5 page or hang up before exporting. Terminal Enter is confirmation,
|
|
111
|
+
not a remote hangup command. Check offsets, length and checksum before writing
|
|
112
|
+
new files. Existing H264/PNG/TXT files can indicate a successful prior export:
|
|
113
|
+
inspect them or select a new basename instead of deleting them blindly.
|
|
114
|
+
|
|
115
|
+
Completion: accepted serialized profile, identified coded dimensions and
|
|
116
|
+
pre-SDK pixels, plus separate H5 and WeChat display checks. Tool/host tests alone
|
|
117
|
+
do not prove camera orientation, serial HIL or call stability.
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Opt-in serial capture of pre-SDK H264. Close idf.py monitor before use."""
|
|
3
|
+
import argparse
|
|
4
|
+
import re
|
|
5
|
+
import shutil
|
|
6
|
+
import subprocess
|
|
7
|
+
import time
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
class Terminal:
|
|
12
|
+
"""Minimal ANSI responder; retain partial lines across serial read timeouts."""
|
|
13
|
+
def __init__(self, port):
|
|
14
|
+
self.port = port
|
|
15
|
+
self.pending = bytearray()
|
|
16
|
+
self.tail = bytearray()
|
|
17
|
+
self.received = 0
|
|
18
|
+
self.signals = []
|
|
19
|
+
|
|
20
|
+
def write(self, data):
|
|
21
|
+
return self.port.write(data)
|
|
22
|
+
|
|
23
|
+
def readline(self):
|
|
24
|
+
deadline = time.monotonic() + 0.2
|
|
25
|
+
while time.monotonic() < deadline:
|
|
26
|
+
if b"\n" in self.pending:
|
|
27
|
+
line, _, rest = self.pending.partition(b"\n")
|
|
28
|
+
self.pending = bytearray(rest)
|
|
29
|
+
for signature in (b"Guru Meditation", b"panic'ed", b"assert failed",
|
|
30
|
+
b"CORRUPT HEAP", b"Stack smashing", b"Brownout",
|
|
31
|
+
b"Rebooting", b"ESP-ROM:"):
|
|
32
|
+
if signature in line and signature.decode() not in self.signals:
|
|
33
|
+
self.signals.append(signature.decode())
|
|
34
|
+
# Do not print unrelated runtime logs or credentials.
|
|
35
|
+
print(f"设备诊断标记:{signature.decode()}", flush=True)
|
|
36
|
+
return bytes(line) + b"\n"
|
|
37
|
+
try:
|
|
38
|
+
chunk = self.port.read(min(self.port.in_waiting or 1, 4096))
|
|
39
|
+
except OSError as error:
|
|
40
|
+
raise RuntimeError(
|
|
41
|
+
f"串口读取中断(已收到{self.received}字节;"
|
|
42
|
+
f"已观测异常标记={','.join(self.signals) or '无,不能排除重启'})。"
|
|
43
|
+
"请检查串口占用及USB断连/设备重启;未自动重连或清除设备抓流。"
|
|
44
|
+
) from error
|
|
45
|
+
self.received += len(chunk)
|
|
46
|
+
for byte in chunk:
|
|
47
|
+
self.tail.append(byte)
|
|
48
|
+
self.tail = self.tail[-8:]
|
|
49
|
+
if self.tail.endswith(b"\x1b[6n"):
|
|
50
|
+
self.write(b"\x1b[1;80R")
|
|
51
|
+
elif self.tail.endswith(b"\x1b[5n"):
|
|
52
|
+
self.write(b"\x1b[0n")
|
|
53
|
+
self.pending.extend(chunk)
|
|
54
|
+
if len(self.pending) > 16384 and b"\n" not in self.pending:
|
|
55
|
+
raise RuntimeError("console line too long; check serial port/baud")
|
|
56
|
+
return b""
|
|
57
|
+
|
|
58
|
+
def settle(self):
|
|
59
|
+
deadline = time.monotonic() + 0.6
|
|
60
|
+
while time.monotonic() < deadline:
|
|
61
|
+
self.readline()
|
|
62
|
+
|
|
63
|
+
def command(self, text):
|
|
64
|
+
self.settle()
|
|
65
|
+
# Ctrl-U clears a stale partially typed command; REPL expects CR.
|
|
66
|
+
self.write(b"\x15" + text.encode("ascii") + b"\r")
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def firmware_identity(port):
|
|
70
|
+
port.settle()
|
|
71
|
+
port.command("version")
|
|
72
|
+
try:
|
|
73
|
+
return until(port, "Firmware:", timeout=5)
|
|
74
|
+
except TimeoutError:
|
|
75
|
+
# An earlier monitor may have consumed the cursor query without replying.
|
|
76
|
+
# Complete that outstanding read, then clear the input and retry once.
|
|
77
|
+
port.write(b"\x1b[1;80R")
|
|
78
|
+
port.settle()
|
|
79
|
+
port.command("version")
|
|
80
|
+
return until(port, "Firmware:", timeout=10)
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
def fnv(data):
|
|
84
|
+
value = 2166136261
|
|
85
|
+
for byte in data:
|
|
86
|
+
value = ((value ^ byte) * 16777619) & 0xffffffff
|
|
87
|
+
return value
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
class Dump:
|
|
91
|
+
def __init__(self):
|
|
92
|
+
self.data = bytearray()
|
|
93
|
+
self.meta = None
|
|
94
|
+
|
|
95
|
+
def feed(self, line):
|
|
96
|
+
# Ignore unrelated logs, but reject holes, duplicates and corrupted data.
|
|
97
|
+
start = re.search(r"VCAP BEGIN bytes=(\d+) fnv=([0-9a-f]{8}) mode=(\d+) "
|
|
98
|
+
r"generation=(\d+) width=(\d+) height=(\d+) frames=(\d+)", line)
|
|
99
|
+
if start:
|
|
100
|
+
if self.meta is not None:
|
|
101
|
+
raise ValueError("duplicate dump header")
|
|
102
|
+
self.meta = [int(start[1]), int(start[2], 16)] + [int(start[i]) for i in range(3, 8)]
|
|
103
|
+
if not 0 < self.meta[0] <= 512 * 1024:
|
|
104
|
+
raise ValueError("invalid dump size")
|
|
105
|
+
elif "VCAP DATA " in line:
|
|
106
|
+
match = re.search(r"VCAP DATA ([0-9a-f]{8}) ([0-9a-f]+)\s*$", line)
|
|
107
|
+
if self.meta is None or not match or int(match[1], 16) != len(self.data):
|
|
108
|
+
raise ValueError("missing/corrupted/out-of-order dump line; retry dump")
|
|
109
|
+
chunk = bytes.fromhex(match[2])
|
|
110
|
+
if not 0 < len(chunk) <= 64 or len(self.data) + len(chunk) > self.meta[0]:
|
|
111
|
+
raise ValueError("invalid chunk size")
|
|
112
|
+
self.data.extend(chunk)
|
|
113
|
+
elif "VCAP END" in line:
|
|
114
|
+
if self.meta is None or len(self.data) != self.meta[0] or fnv(self.data) != self.meta[1]:
|
|
115
|
+
raise ValueError("incomplete dump or checksum mismatch; retry dump")
|
|
116
|
+
return True
|
|
117
|
+
elif "VCAP ERROR" in line:
|
|
118
|
+
raise RuntimeError(line[line.index("VCAP ERROR"):].strip())
|
|
119
|
+
return False
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
def until(port, marker, timeout=15):
|
|
123
|
+
deadline = time.monotonic() + timeout
|
|
124
|
+
while time.monotonic() < deadline:
|
|
125
|
+
line = port.readline().decode("ascii", errors="replace")
|
|
126
|
+
if "VCAP ERROR" in line:
|
|
127
|
+
raise RuntimeError(line[line.index("VCAP ERROR"):].strip())
|
|
128
|
+
if marker in line:
|
|
129
|
+
return line[line.index(marker):].strip()
|
|
130
|
+
raise TimeoutError(f"did not receive {marker}; check port/firmware")
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
def capture_status(port, command="status"):
|
|
134
|
+
port.command("video-capture " + command)
|
|
135
|
+
line = until(port, "VCAP STATUS")
|
|
136
|
+
print(line, flush=True)
|
|
137
|
+
match = re.search(r"active=(\d+) bytes=(\d+) frames=(\d+) width=(\d+) height=(\d+) reason=(\S+)", line)
|
|
138
|
+
if not match:
|
|
139
|
+
raise RuntimeError("抓流状态不完整,请保留上述状态输出")
|
|
140
|
+
return dict(zip(("active", "bytes", "frames", "width", "height", "reason"),
|
|
141
|
+
[int(match[i]) for i in range(1, 6)] + [match[6]]))
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
def stop_for_export(port):
|
|
145
|
+
status = capture_status(port, "stop")
|
|
146
|
+
if status["active"]:
|
|
147
|
+
raise RuntimeError("设备尚未停止抓流,未尝试导出")
|
|
148
|
+
if not status["bytes"]:
|
|
149
|
+
# No media to preserve. Release the allocation that otherwise blocks start.
|
|
150
|
+
capture_status(port, "clear")
|
|
151
|
+
raise RuntimeError(
|
|
152
|
+
f"缓冲为0字节,没有可导出的H264(reason={status['reason']});"
|
|
153
|
+
"已释放空缓冲。请保留上面的STATUS输出,重新建立视频后运行不带--dump-only的命令。"
|
|
154
|
+
)
|
|
155
|
+
return status
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
def main():
|
|
159
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
160
|
+
parser.add_argument("--port", required=True)
|
|
161
|
+
parser.add_argument("--output", required=True, type=Path)
|
|
162
|
+
parser.add_argument("--dump-only", action="store_true", help="retry exporting retained capture after hangup")
|
|
163
|
+
args = parser.parse_args()
|
|
164
|
+
if args.output.exists() or args.output.with_suffix(".png").exists() or args.output.with_suffix(".txt").exists():
|
|
165
|
+
parser.error("output or companion file exists; choose a new name")
|
|
166
|
+
import serial
|
|
167
|
+
# POSIX advisory exclusive access: prevents cooperating serial tools from
|
|
168
|
+
# taking the same port; cannot evict/detect every already-open reader.
|
|
169
|
+
with serial.Serial(args.port, 115200, timeout=0.1, write_timeout=5, exclusive=True) as raw_port:
|
|
170
|
+
port = Terminal(raw_port)
|
|
171
|
+
if not args.dump_only:
|
|
172
|
+
input("保持标识纸不动,进入 H5 实时查看或微信视频通话,出图后按 Enter:")
|
|
173
|
+
identity = firmware_identity(port)
|
|
174
|
+
print(identity)
|
|
175
|
+
status = capture_status(port)
|
|
176
|
+
if status["active"] or status["bytes"]:
|
|
177
|
+
raise RuntimeError("存在正在进行或已保存的抓流,未覆盖。请退出查看后用--dump-only导出")
|
|
178
|
+
if status["reason"] != "empty":
|
|
179
|
+
capture_status(port, "clear") # inactive, zero-byte allocation only
|
|
180
|
+
port.command("video-capture start")
|
|
181
|
+
print(until(port, "VCAP ARMED"))
|
|
182
|
+
# Drain unrelated logs while allowing up to 5s for IDR + 2s capture.
|
|
183
|
+
deadline = time.monotonic() + 7
|
|
184
|
+
while time.monotonic() < deadline:
|
|
185
|
+
port.readline()
|
|
186
|
+
else:
|
|
187
|
+
identity = firmware_identity(port)
|
|
188
|
+
print(identity)
|
|
189
|
+
stop_for_export(port)
|
|
190
|
+
input("现在退出所有 H5 查看页面/挂断通话;确认连接断开后按 Enter 导出(不要只最小化页面):")
|
|
191
|
+
port.command("video-capture dump")
|
|
192
|
+
dump = Dump()
|
|
193
|
+
deadline = time.monotonic() + 240
|
|
194
|
+
while time.monotonic() < deadline:
|
|
195
|
+
if dump.feed(port.readline().decode("ascii", errors="replace")):
|
|
196
|
+
break
|
|
197
|
+
else:
|
|
198
|
+
raise TimeoutError("dump timed out; retained on device, retry with --dump-only")
|
|
199
|
+
with args.output.open("xb") as file:
|
|
200
|
+
file.write(dump.data)
|
|
201
|
+
with args.output.with_suffix(".txt").open("x") as file:
|
|
202
|
+
file.write(identity + "\n")
|
|
203
|
+
file.write(f"bytes,fnv,mode,generation,width,height,frames={dump.meta}\n")
|
|
204
|
+
port.command("video-capture clear")
|
|
205
|
+
until(port, "VCAP STATUS")
|
|
206
|
+
print(f"校验通过,已保存 {args.output};设备抓流缓冲已释放。")
|
|
207
|
+
if shutil.which("ffprobe"):
|
|
208
|
+
subprocess.run(["ffprobe", "-v", "error", "-f", "h264", "-show_entries",
|
|
209
|
+
"stream=codec_name,width,height", "-of", "default=nw=1", str(args.output)], check=True)
|
|
210
|
+
if shutil.which("ffmpeg"):
|
|
211
|
+
subprocess.run(["ffmpeg", "-v", "error", "-n", "-noautorotate", "-f", "h264", "-i",
|
|
212
|
+
str(args.output), "-frames:v", "1", str(args.output.with_suffix(".png"))], check=True)
|
|
213
|
+
print(f"发送前画面(未自动旋转):{args.output.with_suffix('.png')}")
|
|
214
|
+
|
|
215
|
+
|
|
216
|
+
if __name__ == "__main__":
|
|
217
|
+
try:
|
|
218
|
+
main()
|
|
219
|
+
except (TimeoutError, RuntimeError, OSError, ValueError) as error:
|
|
220
|
+
raise SystemExit(f"抓流未完成:{error}") from None
|
|
@@ -24,9 +24,7 @@ CONTRACT_KEYS = {
|
|
|
24
24
|
}
|
|
25
25
|
THING_CONNECT_ENV = "TIRTC_THING_CONNECT_ROOT"
|
|
26
26
|
GENERATOR_RELATIVE_PATH = Path("device-sim/scripts/create_esp32_project.py")
|
|
27
|
-
|
|
28
|
-
"device-sim/sdk/espressif-esp32s3/2.3.0"
|
|
29
|
-
)
|
|
27
|
+
SDK_RELATIVE_ROOT = Path("device-sim/sdk/espressif-esp32s3")
|
|
30
28
|
DEVICE_KIT_MANIFEST = "manifest.json"
|
|
31
29
|
|
|
32
30
|
|
|
@@ -161,7 +159,7 @@ def discover_serial_ports() -> list[str]:
|
|
|
161
159
|
def normalize_thing_connect_root(candidate: Path) -> Path | None:
|
|
162
160
|
"""Accept either the ThingConnect directory or its parent repository."""
|
|
163
161
|
resolved = candidate.expanduser().resolve()
|
|
164
|
-
for root in (resolved, resolved / "thing-connect"):
|
|
162
|
+
for root in (resolved, resolved / "kit-src", resolved / "thing-connect"):
|
|
165
163
|
if (root / GENERATOR_RELATIVE_PATH).is_file():
|
|
166
164
|
return root
|
|
167
165
|
return None
|
|
@@ -248,7 +246,20 @@ def resolve_sdk_dir(
|
|
|
248
246
|
if bundled.is_dir():
|
|
249
247
|
return bundled, "generated project"
|
|
250
248
|
if thing_connect_root is not None:
|
|
251
|
-
|
|
249
|
+
manifest_path = thing_connect_root / DEVICE_KIT_MANIFEST
|
|
250
|
+
if manifest_path.is_file():
|
|
251
|
+
try:
|
|
252
|
+
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
|
253
|
+
version = manifest.get("tirtc_sdk_version")
|
|
254
|
+
if isinstance(version, str) and re.fullmatch(r"\d+\.\d+\.\d+", version):
|
|
255
|
+
return thing_connect_root / SDK_RELATIVE_ROOT / version, "Device Kit"
|
|
256
|
+
except (OSError, json.JSONDecodeError):
|
|
257
|
+
pass
|
|
258
|
+
for version in ("2.5.0", "2.3.0"):
|
|
259
|
+
candidate = thing_connect_root / SDK_RELATIVE_ROOT / version
|
|
260
|
+
if candidate.is_dir():
|
|
261
|
+
return candidate, "ESP32 source workspace"
|
|
262
|
+
return thing_connect_root / SDK_RELATIVE_ROOT / "2.5.0", "ESP32 source workspace"
|
|
252
263
|
return None, "not found"
|
|
253
264
|
|
|
254
265
|
|