tirtc-device-builder 0.7.3 → 0.8.1

package/skills/tirtc-esp32-builder/references/tirtc-platform.md

@@ -0,0 +1,92 @@
+# TiRTC and ThingConnect platform contract
+
+Read this reference when integrating the C SDK, selecting an ESP32 target, or
+implementing H5, AI, device-call or WeChat VoIP.
+
+## Authoritative sources
+
+- TiRTC documentation: <https://docs.tange.ai/products/tirtc/>
+- C API: <https://docs.tange.ai/products/tirtc/api-reference/c.html>
+- C SDK integration: <https://docs.tange.ai/products/tirtc/guides/sdk-integration/c.html>
+- ESP32-S3 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-s3.html>
+- ESP32-P4 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-p4.html>
+- ThingConnect reference platform: <https://github.com/tangeai/tirtc-server-example>
+- Demo platform: <https://demo-open.tange-ai.com/>
+
+Use the locked SDK header/build contract and a pinned ThingConnect commit during
+generation. Public pages explain the contract but do not replace the exact files
+used by a build.
+
+## SDK lifecycle
+
+TiRTC is one process-wide runtime. The exact selected header governs API and
+option availability. For SDK 2.3.0 the important order is:
+
+1. Set `TIRTC_OPT_MAX_SEND_BUFFER` before `TiRtcInit()` when overriding it.
+2. Call `TiRtcInit()` once.
+3. Set `TIRTC_OPT_DEVICE_SECRET_KEY`, `TIRTC_OPT_CLIENT_ID`, the discovered
+   service endpoint, and applicable network options before `TiRtcStart()`.
+4. Call `TiRtcStart(device_id, &callbacks)` once and wait for
+   `TIRTC_EVENT_SYS_STARTED`; return value zero only accepts the request.
+5. Establish or accept connections, then use `TiRtcSendVideoStream()` and
+   `TiRtcSendAudioStream()` with the selected complete-frame contract.
+6. Stop sessions and connections through deferred lifecycle work before the one
+   final `TiRtcStop()` / `TiRtcUninit()` sequence.
+
+The callback table and its context outlive the SDK runtime. Callback payloads
+are borrowed; copy required data into bounded application-owned storage before
+returning.
+
+## Product features
+
+### H5 live view and talkback
+
+The device starts as a listener, accepts the H5 connection, sends the selected
+audio/video streams, receives talkback, and handles refresh/key-frame requests.
+Browser rendering and audible talkback are HIL evidence.
+
+### AI intercom
+
+Obtain the current AI connection parameters, connect, send `start_session`,
+validate the authoritative session ID and input/output formats, then start
+media. Remote or local `end_session` converges through the runtime task and
+restores the H5 baseline.
+
+### Device-to-device call
+
+Follow the pinned `device-call.md`: contact authorization, `POST
+/v1/call/request`, incoming MQTT routing, accept through `POST
+/v1/call/device/info`, reject/cancel/hangup, room recovery and `TiRtcConnect`.
+Expose `call <device_id> [video|audio]` only after those paths and conflict rules
+are implemented.
+
+### WeChat VoIP
+
+Follow the pinned `device-voip.md`: report the device media profile, maintain the
+authorized contact list, route WeChat MQTT events, and use `POST
+/v1/voip/device/call` for device-originated calls. `wxcall [N] [video|audio]`
+selects an authorized contact; it is not a direct SDK API. Mini-program
+authorization and plugin behavior are separate platform acceptance evidence.
+
+## Simulator before hardware
+
+For the four-feature portfolio, use the pinned Linux C reference implementation
+with file-backed media to prove platform credentials, HTTP/MQTT fields,
+connection tokens, commands and the session arbiter before replacing media with
+board drivers. Record this separately from ESP32 L1-L7 evidence.
+
+The managed ESP32-S3 starter may expose fewer businesses than the full reference
+repository. Missing generated modules are implementation work, not permission to
+drop a requested feature or call it verified.
+
+## ESP32 targets
+
+The current managed generator and Device Kit automate ESP32-S3. Official SDK
+2.3.0 also provides a distinct ESP32-P4 package. P4 needs ESP-IDF 5.5.4, matching
+RISC-V toolchain/build contract, PSRAM, and an evidenced network path such as
+ESP-Hosted with C6/C61 or Ethernet. P4 and S3 archives are not interchangeable.
+
+Use FreeRTOS tasks, bounded queues and explicit memory capabilities. PSRAM is
+suitable for large media, fixed pools and eligible background stacks; internal
+RAM remains necessary for DMA descriptors/buffers, ISR-visible state, control
+objects and allocations required while flash cache is unavailable.

package/skills/tirtc-esp32-builder/references/workflow.md

@@ -4,13 +4,18 @@
 
 ### Registered board
 
-Use this branch when a board and exact hardware revision already have a Hardware IR plus a matching board media adapter.
-
-1. Validate and assess the saved Hardware IR against the requested features.
-2. Confirm that its BSP, ESP-IDF, TiRTC SDK, and adapter revisions are still resolvable.
-3. Generate a new starter project without overwriting an existing path.
-4. Install the matching board adapter and configuration overlay.
-5. Build, optionally flash, execute the requested acceptance levels, and issue a fresh report.
+Use this branch only when `board_registry.py match` returns `exact` with
+`safe_registered_reuse=true` and the package has a Hardware IR plus matching
+board adapter.
+
+1. Recheck every required runtime probe and reject a changed sensor/codec as a
+   new variant.
+2. Validate and assess the saved Hardware IR against the requested features.
+3. Confirm that its BSP, ESP-IDF, TiRTC SDK, business protocol, and adapter
+   revisions are still resolvable.
+4. Generate a new starter project without overwriting an existing path.
+5. Install the matching board adapter and configuration overlay.
+6. Build, optionally flash, execute the requested acceptance levels, and issue a fresh report.
 
 The branch is complete when the new run has its own build and verification evidence; an older board report is provenance, not proof of the new artifact.
 
@@ -18,10 +23,16 @@ The branch is complete when the new run has its own build and verification evide
 
 Use this branch when the user supplies a board model, vendor URL, schematic, BOM, pin map, BSP, datasheets, photographs, or peripheral example projects without a verified adapter.
 
-1. Resolve the full model, module, PCB marking, and hardware revision. Treat different revisions as different boards.
+1. Create `board-identity.json`, combine the developer's full model/PCB marking
+   with safe SoC/component probes, and query the board registry. Treat different
+   revisions or conflicting required probes as different variants. A probable
+   or component match supplies hypotheses only.
 2. Freeze the user-supplied product contract: selected video profile, audio/stream formats, duplex/AEC policy, supported Wi-Fi credential methods, selected onboarding method, transport staging, output path, and mutation boundary. Use `unknown` where the prompt lacks an answer.
 3. Prefer official schematic/BOM and BSP facts. For a PDF schematic, inspect page labels and net names; prefer an exported netlist, pin CSV, or vendor board definition when available.
 4. Cross-check critical pins, clocks, power enables, reset lines, sensor/codec variants, ESP-IDF version, resource ownership, and onboarding behavior across at least two independent artifacts when possible.
+   For buttons, power keys and enclosure labels, identify the full physical
+   product including carrier/PMIC layers; a working control missing from the
+   baseboard schematic is a variant or evidence gap, not proof that it is absent.
 5. Create the Hardware IR v2. Use `null` for unknown facts and retain contradictory values as an explicit issue instead of selecting one silently. Store concrete board values in the IR/adapter rather than Skill files.
 6. Validate and run the intake assessment. Classify unresolved facts by their next evidence source: source, implementation, build, HIL, or user input. Ask only for `user_blocked` facts that prevent a safe design. SoftAP is optional when another evidenced Wi-Fi credential method is available, keeps credentials outside source, and defines reprovisioning.
 7. When hardware identity, wiring, product contracts, and an evidenced resource plan reach `READY_TO_PORT`, generate the starter and implement the board adapter. Generate a compile-safe adapter by default when remaining uncertainty is implementation-, build-, or HIL-resolvable. Stop at the IR/report only when missing user evidence or an incompatible dependency makes a safe implementation impossible.
@@ -46,13 +57,16 @@ Read only the documents for the active branch, but read each selected document c
 - ESP32 starter or adapter work: `device-sim/device-sim-esp32/README.md`, `device-sim/ESP32_STARTER.md`, the selected SDK package README, and the generated template README.
 - H5 live view or talkback: `device-h5-live.md`.
 - AI intercom: `device-ai.md`.
+- Device-to-device call: `device-call.md` and the applicable API reference.
+- WeChat VoIP: `device-voip.md`, the mini-program integration document, and the
+  applicable API reference.
 - H5/AI switching, delayed callbacks, ownership, or timeouts: `device-session-model.md` and `device-session-arbiter.md`.
 - Onboarding, binding, MQTT, token, or identity: `device-integration.md`.
 - Public HTTP field or error changes: `api-reference.md` and `error-response-policy.md`.
 
 ## Generation and board seam
 
-The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, `starter_runtime`, or `starter_tirtc` for each board.
+The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, CALL, VOIP, `starter_runtime`, or `starter_tirtc` for each board.
 
 The adapter owns:
 
@@ -62,13 +76,30 @@ The adapter owns:
 - DMA buffers, hardware clocks, power, reset, GPIO, refresh/key-frame requests, and realtime task allocation;
 - bounded stop, resource release, and generation-aware flushing.
 
-The stable modules own the discovered TiRTC service endpoint, stream IDs, negotiated/contracted formats, TiRTC callback copying, connection handles, session generation, and H5/AI sequencing. SDK lifecycle changes such as disconnect run in a worker/state-machine context, never directly inside an SDK callback.
+The stable modules own the discovered TiRTC service endpoint, stream IDs,
+negotiated/contracted formats, HTTP/MQTT business fields, TiRTC callback copying,
+connection handles, pending calls, monotonic deadlines, session generation, and
+H5/AI/CALL/VOIP sequencing. SDK lifecycle changes such as disconnect run in a
+worker/state-machine context, never directly inside an SDK callback.
+
+A separate product-controls adapter owns physical input setup, debounce,
+boot-held suppression and gestures. It queues bounded intents into the stable
+runtime. It never manipulates TiRTC/session state directly or keeps a competing
+session Boolean. Prefer an atomic runtime-owned toggle intent; when only explicit
+start/stop intents exist, follow
+[product-controls.md](product-controls.md) for snapshot mapping and authoritative
+runtime revalidation.
 
 ## Verification loop
 
 Use a bounded loop per layer: diagnose one failing invariant, make the smallest correction, and rerun that layer before moving forward. Change one high-risk variable per HIL comparison. Turn reusable invariants into tests or post-link gates. Stop and report when the remaining failure requires unavailable hardware, credentials, a new SDK binary, a public protocol change, or a user choice.
 
-For every generated H5/AI project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
+For every generated H5/AI/call/VoIP project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. AI/call/VoIP also require the full-duplex AEC result from the audio gate. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
+
+When CALL or VOIP is requested, the runtime contract must also bind the project
+to the pinned business protocol and unified session arbiter. First prove those
+flows with the Linux C simulator. Simulator success establishes the protocol
+baseline but cannot replace the ESP32 build or HIL levels.
 
 Run the assessor once per layer:
 

package/skills/tirtc-esp32-builder/scripts/audio_contract.py

@@ -383,6 +383,8 @@ def verify_contract(contract_path: Path, project_path: Path) -> dict[str, Any]:
             f"simultaneous={shared.get('directions_simultaneous')} "
             f"aec={aec.get('enabled', False)}"
         ),
+        "simultaneous_capture_playback": shared.get("directions_simultaneous") is True,
+        "echo_cancellation_enabled": aec.get("enabled") is True,
         "inputs": inputs,
         "errors": errors,
     }