tirtc-device-builder 0.9.3 → 0.9.5

package/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tirtc-device-builder",
-  "version": "0.9.3",
+  "version": "0.9.5",
   "description": "Codex workflows for building and validating TiRTC device firmware across supported chip platforms.",
   "author": {
     "name": "TangeAI",

package/CHANGELOG.md

@@ -2,6 +2,25 @@
 
 This project follows Semantic Versioning.
 
+## 0.9.5
+
+- Add focused video-orientation and WeChat UI-profile guidance distinguishing
+  encoded pixels, display size, rotation conventions and endpoint verification.
+- Package the bounded XiaoTai serial capture/export helper with offline tests
+  for ANSI queries, evidence integrity and retained/empty capture recovery.
+- Pin Waveshare product knowledge to a95f368 and document the watchpoint-proven
+  LVGL callback-count overwrite; preserve pending HIL and non-default UI angles.
+
+## 0.9.4
+
+- Add a knowledge-only Waveshare ESP32-P4-WIFI6-Touch-LCD-3.5 package with
+  pinned source maps, media/resource lessons and explicit hardware/HIL limits.
+- Expose validated knowledge references for model candidates without granting
+  adapter reuse; allow unknown PCB revisions only in knowledge-only packages.
+- Add focused media/resource diagnosis guidance covering directional stream
+  contracts, WithCaps task cleanup, reserved codec ownership and UI cadence.
+- Test candidate discovery, unknown revisions and reference path boundaries.
+
 ## 0.9.3
 
 - Pin ESP32 Device Kit 1.1.4 and use wildcard DNS plus HTTP probe redirects for

package/README.md

@@ -961,8 +961,8 @@ metadata 中的版本、标签、上游 commit 和期望 SHA-256 必须与本地
 
 ```bash
 npm test
-git tag -a v0.9.3 -m "v0.9.3"
-git push origin v0.9.3
+git tag -a v0.9.5 -m "v0.9.5"
+git push origin v0.9.5
 ```
 
 不要重复发布已经存在的 npm 版本。版本变化同步更新 `package.json`、`.codex-plugin/plugin.json` 和发布说明。

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tirtc-device-builder",
-  "version": "0.9.3",
+  "version": "0.9.5",
   "description": "Install and run TiRTC device-development Agent Skills across popular coding clients.",
   "license": "MIT",
   "author": {

package/skills/tirtc-esp32-builder/SKILL.md

@@ -43,6 +43,15 @@ A probable match supplies hypotheses; a component match supplies only component
 lessons. Any identity conflict creates a new variant and keeps concrete GPIO,
 clock, DMA and task values unresolved.
 
+Read a match's `knowledge_refs` before scanning reference repositories. These
+are source maps and evidence, not permission to install a candidate adapter.
+For an existing project's media, memory or scheduling bug, read
+[runtime-resource-review.md](references/runtime-resource-review.md); preserve
+the user's business baseline separately from the hardware reference.
+For video orientation, unexpected cropping, or WeChat display-profile changes,
+read [video-orientation.md](references/video-orientation.md) before changing
+pixel transforms or UI angles.
+
 ## 3. Prove the business flows before board porting
 
 When device-call or WeChat VoIP is requested, first build and run the pinned

package/skills/tirtc-esp32-builder/VERSION

@@ -1 +1 @@
-0.9.3
+0.9.5

package/skills/tirtc-esp32-builder/assets/lckfb-szpi-esp32s3-portable-prompt.md

@@ -8,7 +8,7 @@
 
 ```bash
 export TIRTC_AGENT_CLIENT=codex
-npx --yes tirtc-device-builder@0.9.3 setup esp32 --install \
+npx --yes tirtc-device-builder@0.9.5 setup esp32 --install \
   --client "$TIRTC_AGENT_CLIENT" --force-skill
 ```
 
@@ -17,11 +17,11 @@ npx --yes tirtc-device-builder@0.9.3 setup esp32 --install \
 本轮第一步先只读运行：
 
 ```bash
-npx --yes tirtc-device-builder@0.9.3 --version
-npx --yes tirtc-device-builder@0.9.3 setup esp32 --client "$TIRTC_AGENT_CLIENT"
+npx --yes tirtc-device-builder@0.9.5 --version
+npx --yes tirtc-device-builder@0.9.5 setup esp32 --client "$TIRTC_AGENT_CLIENT"
 ```
 
-必须根据命令的实际输出和本机文件确认：npm 包为 0.9.3、已安装 Skill 的 `VERSION` 为 0.9.3、所选 Device Kit 的 `manifest.json` 中 `kit_version` 为 1.1.4，并且 Doctor 对 `--expected-kit 1.1.4` 输出 `OVERALL: PASS`。Plugin manifest 不属于这种 npm 安装方式的运行时前置条件，不得把不可访问的 Plugin 版本当作阻塞项。如果版本不一致、Skill 是在当前会话启动后才安装，或环境检查未通过，停止并报告前置条件不成立；不要在当前会话中替换 Skill 后继续生成工程。
+必须根据命令的实际输出和本机文件确认：npm 包为 0.9.5、已安装 Skill 的 `VERSION` 为 0.9.5、所选 Device Kit 的 `manifest.json` 中 `kit_version` 为 1.1.4，并且 Doctor 对 `--expected-kit 1.1.4` 输出 `OVERALL: PASS`。Plugin manifest 不属于这种 npm 安装方式的运行时前置条件，不得把不可访问的 Plugin 版本当作阻塞项。如果版本不一致、Skill 是在当前会话启动后才安装，或环境检查未通过，停止并报告前置条件不成立；不要在当前会话中替换 Skill 后继续生成工程。
 
 工作区与 clean-room 边界：
 - 将启动 Agent 客户端时的当前目录定义为 `WORKSPACE_ROOT`。

package/skills/tirtc-esp32-builder/knowledge/board-registry.json

@@ -1,6 +1,43 @@
 {
   "schema_version": 1,
   "boards": [
+    {
+      "package_id": "waveshare_esp32p4_wifi6_touch_lcd_35_unknown_rev",
+      "status": "knowledge_only",
+      "identity": {
+        "vendor": "Waveshare",
+        "model": "ESP32-P4-WIFI6-Touch-LCD-3.5",
+        "aliases": ["微雪ESP32-P4-WIFI6-Touch-LCD-3.5", "Waveshare ESP32-P4-WIFI6-Touch-LCD-3.5"],
+        "hardware_revision": null,
+        "target": "esp32p4",
+        "module": null,
+        "flash_mb": 16,
+        "psram_mb": 32,
+        "components": [
+          {"kind": "camera", "model": "OV5647"},
+          {"kind": "audio_codec", "model": "ES8311"},
+          {"kind": "display", "model": "ST7796"},
+          {"kind": "touch", "model": "FT6336"},
+          {"kind": "network", "model": "ESP32-C6"}
+        ],
+        "probes": []
+      },
+      "compatibility": {"idf": "5.5.4", "tirtc_sdk": "2.3.0"},
+      "artifacts": {},
+      "knowledge_refs": ["waveshare-esp32-p4-wifi6-touch-lcd-3.5/knowledge.md"],
+      "lessons": [
+        {
+          "id": "waveshare_p4_media_receive_identity",
+          "scope": "board",
+          "subject": "VoIP receive admission",
+          "guidance": "Observed WeChat JPEG stream1 differs from subscribe/uplink stream11. Check active connection, generation and codec; reproduce with actual ingress before editing filters. See knowledge_refs for evidence and limitations.",
+          "action": "warn",
+          "verification": "corroborated",
+          "applies_to": {"package_id": "waveshare_esp32p4_wifi6_touch_lcd_35_unknown_rev"},
+          "evidence": [{"board_package_id": "waveshare_esp32p4_wifi6_touch_lcd_35_unknown_rev", "source_ref": "waveshare-esp32-p4-wifi6-touch-lcd-3.5/knowledge.md#media-and-failures-worth-remembering"}]
+        }
+      ]
+    },
     {"package_id":"lckfb_szpi_esp32s3_v1_0_1","status":"knowledge_only","identity":{"vendor":"深圳嘉立创科技集团股份有限公司（立创开发板）","model":"立创·实战派 ESP32-S3 开发板","aliases":["LCKFB SZPI ESP32-S3"],"hardware_revision":"V1.0.1","target":"esp32s3","module":"ESP32-S3 module","flash_mb":16,"psram_mb":8,"components":[{"kind":"camera","model":"GC2145"},{"kind":"aec_reference","model":"ES7210 MIC3 hardware reference"}],"probes":[]},"compatibility":{"idf":"5.5.4","tirtc_sdk":"2.3.0"},"artifacts":{},"lessons":[]},
     {"package_id":"alientek_atk_dnesp32s3_v1","status":"knowledge_only","identity":{"vendor":"正点原子 (ALIENTEK)","model":"ATK-DNESP32S3","aliases":[],"hardware_revision":"V1.0","target":"esp32s3","module":"ATK-MWS3S","flash_mb":16,"psram_mb":8,"components":[{"kind":"camera","model":"OV5640 PID 0x5640"},{"kind":"audio_codec","model":"ES8388"}],"probes":[]},"compatibility":{"idf":"5.5.4","tirtc_sdk":"2.3.0"},"artifacts":{},"lessons":[]}
   ]

package/skills/tirtc-esp32-builder/knowledge/waveshare-esp32-p4-wifi6-touch-lcd-3.5/knowledge.md

@@ -0,0 +1,146 @@
+# Waveshare ESP32-P4-WIFI6-Touch-LCD-3.5
+
+2026-09-07. Knowledge only; PCB and silicon revisions unresolved. This package
+contains source-correlated observations and targeted regressions, not an
+installable adapter or complete Hardware IR. A matching sales name does not
+authorize pin reuse. No credentials, raw call logs or user media are retained.
+
+## Read only the needed source
+
+| Baseline | Pinned source | Locate next |
+| --- | --- | --- |
+| Product business | xiaotai `a95f368a037eb3ebe88d8564e1cb2f85e369f525` | `lckfb-szpi-esp32s3-tirtc/components/starter_runtime/src/starter_runtime.c`, shared `starter_product`, `starter_tirtc` |
+| Hardware/media | tirtc-device-example `cad4cbe58c3ff451906322930ceede19e6bf6a07`, app 1.5.3 | `complete-applications/esp32-p4/device-monitor/main/` |
+| Vendor | waveshareteam/ESP32-P4-WIFI6-Touch-LCD-3.5 `870588e62daaf723f3c52f0060d8dc053665ed30` | `examples/esp-idf/06_I2SCodec`, `docs/revisions.md`, schematic |
+| Cross-check | xiaozhi-esp32 `1ce658bcb9ce9aac69d3f87af9894a715b7175f6` | `main/boards/waveshare/esp32-p4-wifi6-touch-lcd-3.5/` |
+
+Source paths are repository-relative, not prerequisites for a particular home
+directory. Product HEAD alone does not identify dirty source; retain the local
+diff and BIN/ELF identity when promoting this package. Project source maps live
+under `docs/boards/waveshare-esp32-p4-wifi6-touch-lcd-3.5/`; current build status
+is in `waveshare-esp32p4-xiaotai/PORT_STATUS.md` and `BUILD_IDENTITY.md`.
+
+## Hardware candidates (source-correlated, re-probe on new hardware)
+
+- P4, 32 MiB PSRAM, 16 MiB flash; ESP32-C6 over Hosted SDIO. P4 requires its own
+  RISC-V SDK; the tested IDF is 5.5.4-dirty, TiRTC 2.3.0.
+- ST7796 **SPI**, physical 320x480 / landscape 480x320; FT6336 uses the FT5x06
+  driver family. CSI camera does not imply a DSI display.
+- No onboard IMU is documented for this model: vendor hardware inventory omits
+  one, pinned monitor BSP declares `BSP_CAPS_IMU=0`, and vendor BSP 2.0.1 lists
+  IMU unavailable. External sensing is needed for motion-based rotation; BSP
+  support flags alone are not proof of hardware absence. Sources checked
+  2026-09-07: https://docs.waveshare.com/ESP32-P4-WIFI6-Touch-LCD-3.5 and
+  https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_3_5/versions/2.0.1/readme
+- Shared I2C1 SDA7/SCL8: pass the existing bus to camera SCCB (`init_sccb=false`),
+  codec and touch; an unused I2C0 macro is not evidence for a second owner.
+- ES8311 I2S1 MCLK13/BCLK12/WS10/DOUT9/DIN11; PA53 high. Shared TX/RX,
+  16 kHz stereo 16-bit, left MIC/right DAC reference. Monitor uses
+  `.no_dac_ref=false`, codec register 0x44=0x58. Validate reference energy and
+  double-talk; a board-level reference macro in Xiaozhi does not prove its
+  actual codec object enables reference input.
+- LCD MOSI20/CLK21/CS23/DC26/RST27/BL28; touch RST29/INT50; BOOT35.
+  Verify combined rotation/mirror and all four touch corners. BOOT is distinct
+  from AXP2101 power controls; PMIC power sequencing remains unverified.
+- C6 SDIO CLK18/CMD19/D0..3=14..17/RST54. Read host/slave versions; STA working
+  does not verify SoftAP/captive portal. Read the remote STA MAC after got-IP.
+- OV5647 source: `main/drivers/camera/camera_driver.c`. Sensor mode may run
+  faster than requested output: observed 800x640@50 sensor paced to 15 fps.
+  Verify PID and actual silicon revision before selecting rev<3 defaults.
+
+## Media and failures worth remembering
+
+| Symptom | Evidence and correction | Regression / next source |
+| --- | --- | --- |
+| VoIP audio but no remote image | Callback metadata stream=1/media=65 (JPEG), rx=37 and decoded=0; adapter only allowed stream11. Receive by active connection/generation/codec, not uplink ID. User later reported image visible. | P4 `tools/test_video_ingress.py`, actual `p4_video_submit`/`drain_video`; monitor `tirtc_session_on_video` |
+| Remote producer not requested | Monitor subscribes then sends its version-specific video-enable command `(SN<<16)\u007c0x1105`, byte1. Added missing request, but this alone was not proven to fix image. | `tools/test_p4_video.py`; monitor `tirtc_session.c`, `tirtc_commands.c` |
+| H264 open fails with >8 MiB free | Singleton output workspace remained reserved for old profile. Stop worker, apply profile, reconcile reservation, restart. | `tools/test_p4_profile_switch.py`; `camera_pipeline_on_rtc_video_config_changed` |
+| Repeated WHIP attempts leak | Shared worker used WithCaps creation and ordinary deletion. Fixed to paired deletion; 24 KiB stack plus TCB per completed attempt was retained. | S3 `tools/test_voip_task_cleanup.py`; installed IDF `esp_additions/idf_additions.c` |
+| UI slow / video not smooth | Product UI tick100ms limits presentation near10fps; renderer input24x256KiB plus output20x480x320x2 dominate pools. Retained pools are intentional, not proven optimal for this product. | `call_video_renderer_config.h`, `p4_video_ui_tick`; measure before tuning |
+
+Product-specific media: device CALL H264 both directions; WeChat VOIP H264 up /
+MJPEG down, profile video enabled with actual screen 480x320, 8k mono A-law. Preserve explicit
+voice calls. Camera privacy disables local capture/send while keeping remote
+decode/audio; microphone is independent. Profile declaration is not proof of
+actual packets or display. P4 H264 encode is hardware; downstream H264 here is
+software, MJPEG uses hardware JPEG plus scaling.
+
+## Evidence boundary
+
+P4 `xiaotai-p4-voip.8` compiled with host regressions including 100 simulated
+worker exits; no repeated-call heap HIL yet.
+
+- BIN SHA256: `8bb2e46f3ce68318ebdd544c6cff979e6885e1c40b70ff0ee1731342b07c55e1`
+- ELF SHA256: `6a138dede2fe20e7980937801c48a0206b0a78a3634a49a9ef3e31f9bce811aa`
+- P4 SDK archive SHA256: `a7a01ffd496a55364c7e4d665ff3884d078147bba96752a965d97befca12e451`
+
+These identify historical local artifacts, not bundled binaries or a reproducible
+release. User reports video visible after the stream-filter correction, without
+a complete artifact-bound capture. Startup logs prove one successful media
+start, not long-run stability. Earlier HMAC signing -> invalid LVGL event crash
+was subsequently traced to an LVGL overwrite, as detailed below; task cleanup
+was a separate fix. AEC/double-talk, weak
+network, repeated calls, PMIC controls and full product parity remain pending.
+Promote only after exact identity, portable adapter/IR/contracts and per-flow
+artifact-bound acceptance are retained.
+
+## Later findings at the committed product baseline
+
+### LVGL callback counter overwrites an unrelated allocator hook
+
+The diagnostic `diag.9` hardware watchpoint caught `lv_obj_add_event_cb` writing
+the mbedTLS calloc function pointer while `render_page` registered the screen
+event again. `lv_obj_clean(screen)` removed children, not screen callbacks.
+The selected LVGL descriptor count was six bits: registration 64 wrapped it,
+zero-sized realloc returned the allocator's sentinel, and descriptor index -1
+wrote 12 bytes before that sentinel. An ensuing HMAC allocation jumped into
+`on_screen_event(e=1)`. The apparent crypto/LVGL call stack was a consequence,
+not an allocator ABI diagnosis or proof of stack exhaustion.
+
+Correction: register once when the persistent screen is initialized, not on
+every page rebuild; remove temporary watchpoints after capturing the writer.
+`tools/test_screen_event_lifetime.py` in the S3 project runs 512 actual setup
+paths against the relevant LVGL counter logic. Source/build regression passed;
+post-fix sustained HIL is not established by that test. Watchpoint addresses
+belonged to that ELF and must never become hardcoded protection logic.
+
+### Orientation, aspect ratio and profile experiments
+
+Read [video-orientation.md](../../references/video-orientation.md) for the
+general procedure and protocol fields. The capture.13 pre-SDK PNG showed the
+upright test card clockwise90; thus the device's encoded pixels were already
+sideways, not merely H5 CSS. The original H264 was not retained at inspection,
+so 1280x960 is corroborated by PNG/capture metadata, not a fresh SPS inspection.
+
+The upright.14 / voipdir.15 implementation captures native1280x960 and uses
+PPA CCW90 before H264 encode, producing960x1280@15 / target2Mbps without crop
+or spatial scale. It adds about1,843,200 bytes of rotation-output PSRAM and one
+PPA transaction. The source geometry and submission tests cover exchanged axes,
+zero crop offsets, 1:1 scale and rotation-aware resource matching. CALL's
+decoder-limited profile remains separate; these values are not maximum ratings.
+
+The user reported H5 orientation correct and P4's MJPEG CW90 display correct
+with voipdir.15, but mini-program rendering still wrong. These are user-reported
+per-direction observations, not a complete artifact-bound acceptance bundle.
+voipui.16 added UI fields with rotation0 / aspect0.75 / mirrorsfalse / contain.
+The subsequent request selected **additional UI rotation270** in voipui.17;
+that latest setting compiled but has **no confirming post-change HIL**. Do not
+promote270 into the board registry's default or claim it fixes every endpoint.
+
+- Latest compiled BIN SHA256: `8de6f919acd120fe283db17ffd653d347a5850df715e1f3879e7910b3b3df29c`
+- Latest compiled ELF SHA256: `2aecb8fb9e81347ef9aed76371c5f0404364b687f393df8fb4ae2099edfddc1c`
+- Source: `waveshare-esp32p4-xiaotai/main/media/{camera_pipeline,video_yuv420_scaler}.c`,
+  `main/services/call_video_renderer.c`; shared `request_voip_profile`.
+- Tests: P4 `tools/test_full_frame_uplink.py`, `test_uplink_rotation.py`,
+  `test_voip_profile.py`. Local SDK PPA enums are CCW, UI angles CW.
+
+### Reusable capture tool boundary
+
+The packaged `scripts/capture_uplink.py` is derived from the committed P4
+`tools/capture_uplink.py`. Its device counterpart is
+`components/p4_hardware/p4_video_capture.c` plus the pre-SDK hook in `p4_video.c`.
+Device firmware is not shipped by this knowledge-only package. Read product
+`VIDEO_CAPTURE.md` and confirm compatible commands before running the helper.
+The helper's actual dump parsing, fragmented ANSI query handling, stopped/empty
+recovery and disconnect classification have packaged host regressions. They do
+not establish unattended operation on an untested USB/terminal combination.

package/skills/tirtc-esp32-builder/references/board-knowledge.md

@@ -51,6 +51,13 @@ Registry package status controls reuse:
 
 An older HIL result is provenance, not proof for a newly built artifact.
 
+Knowledge-only packages may record an unknown PCB revision as `null`; verified
+packages still require an explicit revision. `match` returns registry-relative
+`knowledge_refs` for exact/model candidates, including knowledge-only packages.
+Read that source map first and inspect only relevant source deltas. A probable
+match provides investigation leads, not validated pins or an installable adapter.
+The Waveshare ESP32-P4-WIFI6-Touch-LCD-3.5 package follows this boundary.
+
 ## Learning loop
 
 After the final assessment, create a project-local candidate:

package/skills/tirtc-esp32-builder/references/runtime-resource-review.md

@@ -0,0 +1,67 @@
+# Existing-project media and resource review
+
+Use for missing remote video, allocation failures, task leaks, slow UI or call
+switching. This is a focused diagnostic route, not a requirement to redo board
+intake, replay every simulator flow or regenerate a working product.
+
+## Establish a narrow baseline
+
+Record business baseline and hardware reference separately. Keep the user's
+contacts, call arbitration, onboarding and UI semantics in the former; borrow
+drivers, capture and codec resource ownership from the latter. Read the matched
+knowledge package's source map, compare pinned commits and relevant dirty files,
+then inspect only the path under investigation. Reuse prior build evidence only
+when artifact identity matches; otherwise label it historical.
+
+## Receive-to-display contract
+
+For each active mode record uplink codec, subscribe ID, downlink codec and
+observed receive metadata independently. H5 video-up stream 11 remains valid;
+it does not establish a universal receive stream ID. Authenticate the active
+connection, validate generation and negotiated media, and test actual ingress
+with captured non-sensitive frame metadata before changing filters.
+
+Use bounded counters at callback, admission, queue, decoder, converter and
+presentation boundaries. Count admission rejection separately from overflow.
+`rx>0, decoded=0` is not automatically a codec failure. Test enabling remote
+production separately from subscription; protocol commands are SDK/reference
+specific, not generic magic constants.
+
+## Resource lifecycle
+
+- A callback reached from an unrelated allocator/crypto call can be the victim
+  of an earlier overwrite. Validate the exact ELF, inspect the indirect-call
+  target and use a narrowly scoped watchpoint to catch the writer. In LVGL,
+  persistent screen callbacks belong to screen creation, not every child-page
+  rebuild: cleaning children does not remove callbacks on their parent. Check
+  the actual version's descriptor counter width. See the Waveshare knowledge
+  package for the captured six-bit overflow case; numeric addresses are not
+  portable fixes.
+
+- Inventory task creation and deletion pairs. ESP-IDF `xTaskCreate*WithCaps`
+  requires `vTaskDeleteWithCaps`; ordinary deletion retains its statically
+  registered stack/TCB. Confirm the installed IDF implementation. Self-deletion
+  may allocate a helper task; under tight internal RAM prefer owner-driven
+  cleanup after the worker has stopped. A mock API-pair test is not heap HIL.
+- Distinguish free capacity, largest contiguous block, DMA capability and
+  reserved-workspace ownership. Stop the old codec owner before reconfiguring
+  or acquiring a singleton workspace. Retry cannot repair stale ownership.
+- Budget internal RAM and PSRAM independently for idle, handshake, duplex media
+  and stop/reconnect overlap. Include codec pools, ingress copies, UI surfaces,
+  worker stacks and SDK queues; a retained pool is not a per-call leak.
+- Measure receive, decode and presentation cadence independently. A 100 ms UI
+  timer caps presentation near 10 fps even if decode runs at 15 fps. Evaluate
+  dedicated video presentation and smaller MJPEG latest-frame pools against
+  audio deadlines; H264 recovery buffers have different dependencies.
+
+## Completion evidence
+
+Run an actual-code regression for the failing seam. Report build/host tests
+separately from hardware. For resource changes, compare post-warmup baselines
+over repeated connect/cancel/hangup cycles and a sustained duplex call: heap
+free/largest/minimum by capability, task stack low-water marks, queue drops,
+decode/present rates and audio starvation. Sample cheaply outside realtime
+callbacks; choose intervals for the experiment, not permanent noisy polling.
+Unreproduced crashes remain unresolved even if a later call succeeds. Change
+one high-risk variable per hardware comparison and retain redacted artifact-
+bound results; user-reported video is evidence of video, not AEC or stability.

package/skills/tirtc-esp32-builder/references/tirtc-platform.md

@@ -68,6 +68,10 @@ authorized contact list, route WeChat MQTT events, and use `POST
 selects an authorized contact; it is not a direct SDK API. Mini-program
 authorization and plugin behavior are separate platform acceptance evidence.
 
+For video profile fields and orientation, use
+[video-orientation.md](video-orientation.md). Screen dimensions, encoded-frame
+aspect ratio and mini-program display transforms describe different boundaries.
+
 ## Simulator before hardware
 
 For the four-feature portfolio, use the pinned Linux C reference implementation

package/skills/tirtc-esp32-builder/references/video-orientation.md

@@ -0,0 +1,117 @@
+# Video geometry, orientation and WeChat UI
+
+Use when a video appears sideways, mirrored, cropped/zoomed, or differs between
+H5 and WeChat. Keep four independent facts: sensor output, encoded pixels,
+device display viewport, and remote player's UI transform. An absent rotate
+call does not prove upright pixels: camera mounting can already introduce an
+offset. H5 working does not prove the VoIP-mode encoded pixels are identical.
+
+## Locate the transform
+
+Keep the board and an asymmetric upright test card stationary. Use readable
+text plus top/left/right marks; rotation alone and rotation plus reflection
+are distinct. State only what the screenshot establishes, not where the
+transform happened. Avoid repeatedly asking for placement once confirmed.
+
+Compare short **pre-SDK** H264 clips from each affected mode, with the exact
+firmware identity and active generation. Start at SPS/PPS plus IDR, decode
+without automatic rotation, inspect pixels, and read coded width/height from
+SPS (for example ffprobe). Callback dimensions are useful corroboration, not
+a substitute for SPS. A truncated printed ELF hash is not a full identity.
+
+- Matching source clips but different displays: inspect SDK/bridge and player
+  UI settings rather than globally rotating the camera again.
+- Different source clips: inspect mode-specific capture, crop, encoder input
+  and profile transitions first.
+- One correctly oriented receive direction says nothing about the opposite
+  direction. Apply and test corrections at the intended boundary only.
+
+Retain separate claims for observed pixels, verified source configuration,
+successful compilation, and post-change hardware observations.
+
+## Geometry and resources
+
+For a full-frame 90-degree rotation, exchange width/height. A 1280x960 sensor
+frame becomes 960x1280; preserving complete pixels is not the same as forcing
+a landscape aspect ratio. Validate encoder width AND height limits, alignment,
+crop coordinates in input axes, scale factors, clamp logic, direct-input
+bypasses, and reserved-resource matching including rotation. Budget an extra
+output surface and conversion time. PPA angles in the ESP-IDF P4 driver are
+counterclockwise; check the selected header instead of sharing enum values
+with a clockwise UI API.
+
+Treat a conservative reference profile as policy, not silicon capacity. Verify
+the installed sensor's native modes and actual output; selecting an 800x640
+mode and taking its central 640x480 narrows field of view even without a
+digital zoom setting. Disable or expose automatic resolution fallback when
+the product requires an exact output. Keep H5/phone encode capability separate
+from a peer P4's software H264 decode limits.
+
+## WeChat profile contract
+
+Checked 2026-09-07 against the primary
+[profile API](https://github.com/tangeai/tirtc-server-example/blob/main/thing-connect/api-reference.md#post-v1voipdeviceprofile)
+and [VoIP guide](https://github.com/tangeai/tirtc-server-example/blob/main/thing-connect/device-voip.md).
+These URLs track main: pin/recheck the selected platform revision before a
+new integration. Do not infer protocol absence from an older board example.
+
+| Field | Boundary / constraint |
+| --- | --- |
+| `screen_width`, `screen_height` | Actual device display region, not uplink coded dimensions |
+| `camera_rotation` | Additional clockwise mini-program UI angle, one of 0/90/180/270; default 0 |
+| `aspect_ratio` | Device video's width/height, positive; default 4/3 can be wrong for portrait output |
+| `hor_mirror`, `vert_mirror` | Mini-program horizontal/vertical reflection; defaults false |
+| `object_fit` | Mini-program `fill` or `contain`; default fill |
+| `video_res_mode` | Downlink `auto`, `fit_screen`, or `fill_screen`; does not rotate |
+
+The five UI fields do not configure TiRTC encoding. `fit_screen` bounds MJPEG
+inside the screen without cropping/upscaling; `fill_screen` can upscale and
+center-crop. Both need valid screen sizes; fill requires even dimensions.
+Do not silently change a working downlink mode while investigating uplink UI.
+The profile JSON must fit 512 bytes. Test actual serialized fields, types and
+size, not only log text. Re-report after changes and start a fresh call so the
+mini-program consumes updated configuration.
+
+Choose UI angle relative to the already-transformed encoded pixels; avoid
+blindly reporting a physical mounting angle. If pixel correction is already
+upright, zero additional UI rotation is a hypothesis to test, not an absolute
+rule. A observed need for another angle is endpoint-specific evidence: retain
+H5/VoIP captures and test the new call before calling it verified. In particular,
+the Waveshare experiment's requested 270 is not a universal board default.
+
+## Bounded capture and serial recovery
+
+The packaged `scripts/capture_uplink.py` supports the **XiaoTai video-capture
+console protocol**, not arbitrary TiRTC firmware. It does not flash or install
+firmware. Source and device-side integration are pinned in the Waveshare
+knowledge package; verify `video-capture` exists before use. Host requirements:
+Python plus pyserial on POSIX; ffprobe/ffmpeg are optional for decoded PNG.
+
+```bash
+python3 <skill-dir>/scripts/capture_uplink.py --port <exact-port> --output <new-path>.h264
+```
+
+Use test-card media only, with explicit serial/capture authorization. The
+firmware allocates at most 512 KiB PSRAM on demand and copies complete frames
+without blocking for a lock or doing file/serial I/O in the encoder callback.
+It stops on capacity, generation/size change, or contention. Approximate
+two-second duration is not guaranteed; missing SPS/PPS/IDR can leave it empty.
+Export is console-task work after stop and disconnect. No network upload.
+
+The helper answers ANSI status/cursor queries (including queries without a
+newline), preserves partial serial lines, uses CR and clears stale command
+input. It opens the port exclusively where supported, but a read interruption
+alone does not distinguish another reader from USB disconnect or reboot.
+Check process ownership and USB logs without killing unrelated processes.
+Display safe panic/reboot markers rather than dumping credential-bearing logs.
+
+Recovery is explicit: use `--dump-only` to stop then inspect retained capture;
+keep nonempty media, release only an inactive empty buffer, and ask the user
+to close the H5 page or hang up before exporting. Terminal Enter is confirmation,
+not a remote hangup command. Check offsets, length and checksum before writing
+new files. Existing H264/PNG/TXT files can indicate a successful prior export:
+inspect them or select a new basename instead of deleting them blindly.
+
+Completion: accepted serialized profile, identified coded dimensions and
+pre-SDK pixels, plus separate H5 and WeChat display checks. Tool/host tests alone
+do not prove camera orientation, serial HIL or call stability.

package/skills/tirtc-esp32-builder/scripts/board_registry.py

@@ -95,12 +95,16 @@ def validate_probe(item: Any, path: str, errors: list[str]) -> None:
         errors.append(f"{path}.required_for_exact must be true or false")
 
 
-def validate_identity(identity: Any, path: str, errors: list[str]) -> None:
+def validate_identity(identity: Any, path: str, errors: list[str], *, allow_unknown_revision: bool = False) -> None:
     if not isinstance(identity, dict):
         errors.append(f"{path} must be an object")
         return
-    for field in ("vendor", "model", "hardware_revision", "target"):
+    for field in ("vendor", "model", "target"):
         nonempty(identity.get(field), f"{path}.{field}", errors)
+    if allow_unknown_revision:
+        nullable_nonempty(identity.get("hardware_revision"), f"{path}.hardware_revision", errors)
+    else:
+        nonempty(identity.get("hardware_revision"), f"{path}.hardware_revision", errors)
     aliases = string_array(identity.get("aliases", []), f"{path}.aliases", errors)
     names = [identity.get("model"), *aliases]
     normalized_names = [normalized(value) for value in names if normalized(value)]
@@ -244,7 +248,15 @@ def validate_registry(data: dict[str, Any], registry_path: Path) -> list[str]:
             errors.append(
                 f"{prefix}.status must be one of {', '.join(sorted(PACKAGE_STATUSES))}"
             )
-        validate_identity(board.get("identity"), f"{prefix}.identity", errors)
+        validate_identity(board.get("identity"), f"{prefix}.identity", errors,
+                          allow_unknown_revision=status == "knowledge_only")
+        for ref_index, value in enumerate(string_array(
+            board.get("knowledge_refs", []), f"{prefix}.knowledge_refs", errors
+        )):
+            resolved = safe_registry_path(registry_path, value,
+                f"{prefix}.knowledge_refs[{ref_index}]", errors)
+            if resolved is not None and not resolved.is_file():
+                errors.append(f"{prefix}.knowledge_refs[{ref_index}] does not exist")
         compatibility = board.get("compatibility")
         if not isinstance(compatibility, dict):
             errors.append(f"{prefix}.compatibility must be an object")
@@ -453,6 +465,10 @@ def match_board(board: dict[str, Any], query: dict[str, Any]) -> dict[str, Any]
             {"kind": kind, "model": model} for kind, model in shared_components
         ],
         "applicable_lessons": applicable_lessons,
+        # A model candidate can read its evidence without acquiring GPIO/adapter
+        # authority. Component-only matches must not inherit board-specific docs.
+        "knowledge_refs": board.get("knowledge_refs", [])
+            if classification in {"exact", "probable"} else [],
         "artifacts": board.get("artifacts", {}) if reuse == "registered_board" else {},
     }
 

package/skills/tirtc-esp32-builder/scripts/capture_uplink.py

@@ -0,0 +1,220 @@
+#!/usr/bin/env python3
+"""Opt-in serial capture of pre-SDK H264. Close idf.py monitor before use."""
+import argparse
+import re
+import shutil
+import subprocess
+import time
+from pathlib import Path
+
+
+class Terminal:
+    """Minimal ANSI responder; retain partial lines across serial read timeouts."""
+    def __init__(self, port):
+        self.port = port
+        self.pending = bytearray()
+        self.tail = bytearray()
+        self.received = 0
+        self.signals = []
+
+    def write(self, data):
+        return self.port.write(data)
+
+    def readline(self):
+        deadline = time.monotonic() + 0.2
+        while time.monotonic() < deadline:
+            if b"\n" in self.pending:
+                line, _, rest = self.pending.partition(b"\n")
+                self.pending = bytearray(rest)
+                for signature in (b"Guru Meditation", b"panic'ed", b"assert failed",
+                                  b"CORRUPT HEAP", b"Stack smashing", b"Brownout",
+                                  b"Rebooting", b"ESP-ROM:"):
+                    if signature in line and signature.decode() not in self.signals:
+                        self.signals.append(signature.decode())
+                        # Do not print unrelated runtime logs or credentials.
+                        print(f"设备诊断标记：{signature.decode()}", flush=True)
+                return bytes(line) + b"\n"
+            try:
+                chunk = self.port.read(min(self.port.in_waiting or 1, 4096))
+            except OSError as error:
+                raise RuntimeError(
+                    f"串口读取中断（已收到{self.received}字节；"
+                    f"已观测异常标记={','.join(self.signals) or '无，不能排除重启'}）。"
+                    "请检查串口占用及USB断连/设备重启；未自动重连或清除设备抓流。"
+                ) from error
+            self.received += len(chunk)
+            for byte in chunk:
+                self.tail.append(byte)
+                self.tail = self.tail[-8:]
+                if self.tail.endswith(b"\x1b[6n"):
+                    self.write(b"\x1b[1;80R")
+                elif self.tail.endswith(b"\x1b[5n"):
+                    self.write(b"\x1b[0n")
+            self.pending.extend(chunk)
+            if len(self.pending) > 16384 and b"\n" not in self.pending:
+                raise RuntimeError("console line too long; check serial port/baud")
+        return b""
+
+    def settle(self):
+        deadline = time.monotonic() + 0.6
+        while time.monotonic() < deadline:
+            self.readline()
+
+    def command(self, text):
+        self.settle()
+        # Ctrl-U clears a stale partially typed command; REPL expects CR.
+        self.write(b"\x15" + text.encode("ascii") + b"\r")
+
+
+def firmware_identity(port):
+    port.settle()
+    port.command("version")
+    try:
+        return until(port, "Firmware:", timeout=5)
+    except TimeoutError:
+        # An earlier monitor may have consumed the cursor query without replying.
+        # Complete that outstanding read, then clear the input and retry once.
+        port.write(b"\x1b[1;80R")
+        port.settle()
+        port.command("version")
+        return until(port, "Firmware:", timeout=10)
+
+
+def fnv(data):
+    value = 2166136261
+    for byte in data:
+        value = ((value ^ byte) * 16777619) & 0xffffffff
+    return value
+
+
+class Dump:
+    def __init__(self):
+        self.data = bytearray()
+        self.meta = None
+
+    def feed(self, line):
+        # Ignore unrelated logs, but reject holes, duplicates and corrupted data.
+        start = re.search(r"VCAP BEGIN bytes=(\d+) fnv=([0-9a-f]{8}) mode=(\d+) "
+                          r"generation=(\d+) width=(\d+) height=(\d+) frames=(\d+)", line)
+        if start:
+            if self.meta is not None:
+                raise ValueError("duplicate dump header")
+            self.meta = [int(start[1]), int(start[2], 16)] + [int(start[i]) for i in range(3, 8)]
+            if not 0 < self.meta[0] <= 512 * 1024:
+                raise ValueError("invalid dump size")
+        elif "VCAP DATA " in line:
+            match = re.search(r"VCAP DATA ([0-9a-f]{8}) ([0-9a-f]+)\s*$", line)
+            if self.meta is None or not match or int(match[1], 16) != len(self.data):
+                raise ValueError("missing/corrupted/out-of-order dump line; retry dump")
+            chunk = bytes.fromhex(match[2])
+            if not 0 < len(chunk) <= 64 or len(self.data) + len(chunk) > self.meta[0]:
+                raise ValueError("invalid chunk size")
+            self.data.extend(chunk)
+        elif "VCAP END" in line:
+            if self.meta is None or len(self.data) != self.meta[0] or fnv(self.data) != self.meta[1]:
+                raise ValueError("incomplete dump or checksum mismatch; retry dump")
+            return True
+        elif "VCAP ERROR" in line:
+            raise RuntimeError(line[line.index("VCAP ERROR"):].strip())
+        return False
+
+
+def until(port, marker, timeout=15):
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        line = port.readline().decode("ascii", errors="replace")
+        if "VCAP ERROR" in line:
+            raise RuntimeError(line[line.index("VCAP ERROR"):].strip())
+        if marker in line:
+            return line[line.index(marker):].strip()
+    raise TimeoutError(f"did not receive {marker}; check port/firmware")
+
+
+def capture_status(port, command="status"):
+    port.command("video-capture " + command)
+    line = until(port, "VCAP STATUS")
+    print(line, flush=True)
+    match = re.search(r"active=(\d+) bytes=(\d+) frames=(\d+) width=(\d+) height=(\d+) reason=(\S+)", line)
+    if not match:
+        raise RuntimeError("抓流状态不完整，请保留上述状态输出")
+    return dict(zip(("active", "bytes", "frames", "width", "height", "reason"),
+                    [int(match[i]) for i in range(1, 6)] + [match[6]]))
+
+
+def stop_for_export(port):
+    status = capture_status(port, "stop")
+    if status["active"]:
+        raise RuntimeError("设备尚未停止抓流，未尝试导出")
+    if not status["bytes"]:
+        # No media to preserve. Release the allocation that otherwise blocks start.
+        capture_status(port, "clear")
+        raise RuntimeError(
+            f"缓冲为0字节，没有可导出的H264（reason={status['reason']}）；"
+            "已释放空缓冲。请保留上面的STATUS输出，重新建立视频后运行不带--dump-only的命令。"
+        )
+    return status
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--port", required=True)
+    parser.add_argument("--output", required=True, type=Path)
+    parser.add_argument("--dump-only", action="store_true", help="retry exporting retained capture after hangup")
+    args = parser.parse_args()
+    if args.output.exists() or args.output.with_suffix(".png").exists() or args.output.with_suffix(".txt").exists():
+        parser.error("output or companion file exists; choose a new name")
+    import serial
+    # POSIX advisory exclusive access: prevents cooperating serial tools from
+    # taking the same port; cannot evict/detect every already-open reader.
+    with serial.Serial(args.port, 115200, timeout=0.1, write_timeout=5, exclusive=True) as raw_port:
+        port = Terminal(raw_port)
+        if not args.dump_only:
+            input("保持标识纸不动，进入 H5 实时查看或微信视频通话，出图后按 Enter：")
+            identity = firmware_identity(port)
+            print(identity)
+            status = capture_status(port)
+            if status["active"] or status["bytes"]:
+                raise RuntimeError("存在正在进行或已保存的抓流，未覆盖。请退出查看后用--dump-only导出")
+            if status["reason"] != "empty":
+                capture_status(port, "clear")  # inactive, zero-byte allocation only
+            port.command("video-capture start")
+            print(until(port, "VCAP ARMED"))
+            # Drain unrelated logs while allowing up to 5s for IDR + 2s capture.
+            deadline = time.monotonic() + 7
+            while time.monotonic() < deadline:
+                port.readline()
+        else:
+            identity = firmware_identity(port)
+            print(identity)
+        stop_for_export(port)
+        input("现在退出所有 H5 查看页面/挂断通话；确认连接断开后按 Enter 导出（不要只最小化页面）：")
+        port.command("video-capture dump")
+        dump = Dump()
+        deadline = time.monotonic() + 240
+        while time.monotonic() < deadline:
+            if dump.feed(port.readline().decode("ascii", errors="replace")):
+                break
+        else:
+            raise TimeoutError("dump timed out; retained on device, retry with --dump-only")
+        with args.output.open("xb") as file:
+            file.write(dump.data)
+        with args.output.with_suffix(".txt").open("x") as file:
+            file.write(identity + "\n")
+            file.write(f"bytes,fnv,mode,generation,width,height,frames={dump.meta}\n")
+        port.command("video-capture clear")
+        until(port, "VCAP STATUS")
+    print(f"校验通过，已保存 {args.output}；设备抓流缓冲已释放。")
+    if shutil.which("ffprobe"):
+        subprocess.run(["ffprobe", "-v", "error", "-f", "h264", "-show_entries",
+                        "stream=codec_name,width,height", "-of", "default=nw=1", str(args.output)], check=True)
+    if shutil.which("ffmpeg"):
+        subprocess.run(["ffmpeg", "-v", "error", "-n", "-noautorotate", "-f", "h264", "-i",
+                        str(args.output), "-frames:v", "1", str(args.output.with_suffix(".png"))], check=True)
+        print(f"发送前画面（未自动旋转）：{args.output.with_suffix('.png')}")
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except (TimeoutError, RuntimeError, OSError, ValueError) as error:
+        raise SystemExit(f"抓流未完成：{error}") from None