tirtc-device-builder 0.7.2 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/.codex-plugin/plugin.json +1 -1
  2. package/CHANGELOG.md +23 -0
  3. package/README.md +44 -14
  4. package/bin/tirtc-device-builder.js +33 -1
  5. package/package.json +1 -1
  6. package/skills/tirtc-esp32-builder/SKILL.md +139 -39
  7. package/skills/tirtc-esp32-builder/USAGE.md +3 -2
  8. package/skills/tirtc-esp32-builder/VERSION +1 -1
  9. package/skills/tirtc-esp32-builder/agents/openai.yaml +1 -1
  10. package/skills/tirtc-esp32-builder/assets/board-audio-contract.example.json +4 -0
  11. package/skills/tirtc-esp32-builder/assets/board-identity.example.json +16 -0
  12. package/skills/tirtc-esp32-builder/assets/developer-intake-prompt.md +3 -3
  13. package/skills/tirtc-esp32-builder/assets/hardware-ir-v2.example.json +9 -1
  14. package/skills/tirtc-esp32-builder/assets/lckfb-szpi-esp32s3-portable-prompt.md +5 -5
  15. package/skills/tirtc-esp32-builder/assets/report-template.md +6 -1
  16. package/skills/tirtc-esp32-builder/assets/tirtc-runtime-contract.example.json +13 -0
  17. package/skills/tirtc-esp32-builder/knowledge/board-registry.json +4 -0
  18. package/skills/tirtc-esp32-builder/references/audio-contract.md +33 -3
  19. package/skills/tirtc-esp32-builder/references/board-knowledge.md +72 -0
  20. package/skills/tirtc-esp32-builder/references/capability-rules.md +11 -1
  21. package/skills/tirtc-esp32-builder/references/environment.md +20 -0
  22. package/skills/tirtc-esp32-builder/references/hardware-ir.md +17 -3
  23. package/skills/tirtc-esp32-builder/references/porting-risks.md +25 -0
  24. package/skills/tirtc-esp32-builder/references/reporting.md +1 -1
  25. package/skills/tirtc-esp32-builder/references/runtime-contract.md +12 -2
  26. package/skills/tirtc-esp32-builder/references/tirtc-platform.md +92 -0
  27. package/skills/tirtc-esp32-builder/references/workflow.md +31 -11
  28. package/skills/tirtc-esp32-builder/scripts/audio_contract.py +105 -3
  29. package/skills/tirtc-esp32-builder/scripts/board_registry.py +677 -0
  30. package/skills/tirtc-esp32-builder/scripts/hardware_ir.py +147 -8
  31. package/skills/tirtc-esp32-builder/scripts/project_portability.py +178 -3
  32. package/skills/tirtc-esp32-builder/scripts/runtime_contract.py +122 -1
@@ -27,6 +27,9 @@
27
27
  | H5 live video | | |
28
28
  | H5 talkback | | |
29
29
  | AI intercom | | |
30
+ | Device-to-device call | | |
31
+ | WeChat VoIP | | |
32
+ | Full-duplex AEC/reference path | | |
30
33
 
31
34
  ## Semantic build gates
32
35
 
@@ -37,6 +40,8 @@
37
40
  - Audio contract path/SHA-256: `{{AUDIO_CONTRACT}}`
38
41
  - Codec clock-table result: `{{AUDIO_CLOCK_GATE}}`
39
42
  - I2S mode/controller/slot/handoff result: `{{AUDIO_TOPOLOGY_GATE}}`
43
+ - Simultaneous capture/playback and AEC result: `{{AEC_GATE}}`
44
+ - Device-call/WeChat protocol and arbiter result: `{{BUSINESS_RUNTIME_GATE}}`
40
45
  - Video contract path/SHA-256: `{{VIDEO_CONTRACT}}`
41
46
  - Camera lock/PID/CPU/frame/backpressure result: `{{VIDEO_GATE}}`
42
47
  - Final ELF I2C driver-family result: `{{I2C_ELF_GATE}}`
@@ -54,7 +59,7 @@
54
59
  | L3 Online | | |
55
60
  | L4 Media | | |
56
61
  | L5 H5 | | |
57
- | L6 AI | | |
62
+ | L6 AI/CALL/VOIP/AEC | | |
58
63
  | L7 Stability | | |
59
64
 
60
65
  ## Firmware and flash record
@@ -6,5 +6,18 @@
6
6
  "app_main": "main/app_main.c",
7
7
  "starter_tirtc": "components/starter_tirtc/src/starter_tirtc.c",
8
8
  "starter_runtime": "components/starter_runtime/src/starter_runtime.c"
9
+ },
10
+ "business": {
11
+ "features": [],
12
+ "protocol_revision": null,
13
+ "session_arbiter": {
14
+ "single_foreground_owner": null,
15
+ "pending_capacity": null,
16
+ "generation_guard": null,
17
+ "monotonic_deadlines": null,
18
+ "deferred_lifecycle": null,
19
+ "restore_h5_after_call": null
20
+ },
21
+ "implementation_assertions": []
9
22
  }
10
23
  }
@@ -0,0 +1,4 @@
1
+ {
2
+ "schema_version": 1,
3
+ "boards": []
4
+ }
@@ -10,12 +10,40 @@ The contract must describe:
10
10
 
11
11
  - the PCM sample rate, MCLK ratio, resulting MCLK, and every clocked codec driver table;
12
12
  - capture/playback controller, role, and standard/TDM/DSP/PCM mode;
13
+ - per-direction slot count and slot bit width when simultaneous directions use
14
+ different framing modes;
13
15
  - TDM enable, slot count/order, physical signal at each slot, selected slot, and mapping evidence when TDM is used;
14
16
  - shared clock GPIOs, whether directions are simultaneous, and the release/recreate handoff;
17
+ - paired-channel ownership and equal BCLKs per frame when one controller runs
18
+ simultaneous standard TX and TDM RX;
19
+ - when AEC is selected, the hardware sample rate plus distinct microphone and
20
+ playback-reference slots/signals;
15
21
  - source assertions tying the normalized contract to the actual adapter implementation.
16
22
 
17
23
  A generic header comment such as “typically 256” is not coefficient evidence. After `idf.py reconfigure` resolves managed components, the selected `(MCLK, sample rate)` pair must exist in every locked codec table named by the contract.
18
24
 
25
+ Simultaneous directions do not have to use the same ESP-IDF mode name. A valid
26
+ mixed-mode topology must set `shared_clock.paired_channels=true`, use the same
27
+ controller, and provide `slot_count` plus `slot_bit_width` for both directions;
28
+ their products must match. For example, four 16-bit TDM RX slots and two 32-bit
29
+ standard TX slots both consume 64 BCLKs per frame. This exception does not allow
30
+ two independent masters to drive shared clock GPIOs.
31
+
32
+ For hardware-reference AEC, add `echo_cancellation` with `enabled`,
33
+ `sample_rate_hz`, `microphone_slot`/`microphone_signal`, and
34
+ `reference_slot`/`reference_signal`. The gate verifies that AEC runs while both
35
+ directions are active, uses the hardware clock rate, and maps two distinct TDM
36
+ signals. Implementation assertions must still bind the contract to the paired
37
+ channel setup, AEC library configuration, and exact slot extraction code.
38
+
39
+ AEC is not optional when Hardware IR requests `ai_talk`, `device_call`, or
40
+ `wechat_voip`. For those features, `echo_cancellation.enabled=false`, missing
41
+ playback-reference wiring, or non-simultaneous capture/playback is a hard
42
+ `BLOCKED` result. Do not silently downgrade the product to push-to-talk. The
43
+ contract verifier exports structured `simultaneous_capture_playback` and
44
+ `echo_cancellation_enabled` results so the Hardware IR assessor enforces this
45
+ rule rather than trusting summary text.
46
+
19
47
  ## Mandatory gate
20
48
 
21
49
  Run the gate before claiming an audio-capable build:
@@ -44,8 +72,10 @@ Audio reaches `BUILD_VERIFIED` only when all of these are true:
44
72
  1. the contract has at least two authoritative source IDs;
45
73
  2. its arithmetic and codec table lookups pass;
46
74
  3. its topology, TDM mapping, and handoff are internally consistent;
47
- 4. its adapter assertions pass;
48
- 5. the gate is part of the ordinary build;
49
- 6. the exact BIN/ELF is hashed after that build.
75
+ 4. simultaneous mixed framing has equal BCLKs per frame and any AEC reference is
76
+ mapped to a distinct evidenced slot;
77
+ 5. its adapter assertions pass;
78
+ 6. the gate is part of the ordinary build;
79
+ 7. the exact BIN/ELF is hashed after that build.
50
80
 
51
81
  Compilation without this gate is `COMPILE_PASS`, not audio `BUILD_VERIFIED`.
@@ -0,0 +1,72 @@
1
+ # Board knowledge and identity
2
+
3
+ Use this reference when identifying a board, reusing an existing adapter, or
4
+ capturing lessons after bring-up. The registry is curated, versioned input. It
5
+ does not train a model or mutate an installed Skill from conversation history.
6
+
7
+ ## Identity boundary
8
+
9
+ An ESP32 can report its SoC target, revision, Flash/PSRAM, MAC and sometimes its
10
+ module. Board firmware can probe camera PIDs, codec chip IDs and other bus
11
+ devices. These observations identify components, not necessarily the carrier
12
+ board sales model, PCB marking, hardware revision, wiring, power topology or
13
+ acoustic path. Obtain those facts from the developer and board documents.
14
+
15
+ Create a project-local identity file, merge the developer declaration with safe
16
+ read-only observations, then query the registry:
17
+
18
+ ```bash
19
+ python3 <skill-dir>/scripts/board_registry.py init-identity \
20
+ --output <project>/board-identity.json
21
+ python3 <skill-dir>/scripts/board_registry.py match \
22
+ --identity <project>/board-identity.json
23
+ ```
24
+
25
+ Use only an `exact` result with `safe_registered_reuse=true` to install a saved
26
+ adapter. `probable` can supply hypotheses while missing revision or probe facts
27
+ are resolved. `component` can supply component-level risks and driver patterns,
28
+ but never GPIO, clock, DMA or adapter values. A conflict creates a new variant.
29
+
30
+ Exact matching requires vendor, model or alias, hardware revision, SoC/resource
31
+ compatibility, and every probe marked `required_for_exact`. Marketing names are
32
+ not enough because vendors may substitute sensors or codecs without renaming a
33
+ product.
34
+
35
+ ## Knowledge scopes
36
+
37
+ - `generic`: an ESP32/TiRTC invariant. Promote only after two independent board
38
+ packages support it and a focused regression test enforces it.
39
+ - `component`: a sensor, codec, amplifier or library observation. Reuse it on a
40
+ different carrier only as a hypothesis until that board supplies evidence.
41
+ - `board`: a fact for one exact package identity. Apply it only to an exact
42
+ match.
43
+
44
+ Registry package status controls reuse:
45
+
46
+ - `knowledge_only`: lessons are usable, but the normal new-board intake remains
47
+ mandatory.
48
+ - `adapter_verified`: the Hardware IR, adapter, configuration and semantic
49
+ contracts are complete enough for the registered-board workflow.
50
+ - `hil_verified`: the reusable package also retains artifact-bound HIL evidence.
51
+
52
+ An older HIL result is provenance, not proof for a newly built artifact.
53
+
54
+ ## Learning loop
55
+
56
+ After the final assessment, create a project-local candidate:
57
+
58
+ ```bash
59
+ python3 <skill-dir>/scripts/board_registry.py candidate \
60
+ --hardware-ir <project>/hardware-ir.json \
61
+ --output <project>/board-knowledge-candidate.json
62
+ ```
63
+
64
+ Review the candidate before promotion. Add exact runtime probes, copy only
65
+ portable project-relative adapter/config/contract inputs, classify each lesson's
66
+ scope, attach the tested artifact SHA-256 for hardware or HIL claims, and add a
67
+ regression test for every generic invariant. Then commit it to a maintained
68
+ registry and publish a new Skill version. An installed Skill never edits itself
69
+ or promotes an observation automatically.
70
+
71
+ Keep credentials, MAC-derived identity, device keys, Wi-Fi secrets, tokens and
72
+ user media out of identity files, candidates and registries.
@@ -9,7 +9,16 @@ Run `hardware_ir.py assess --phase intake --strict` before generation. Hardware
9
9
  | `h5_live_audio` | Microphone path producing G.711 A-law, 8 kHz, mono for stream 10; audio controller/channel ownership and memory budget resolved |
10
10
  | `h5_live_video` | Camera plus the selected MJPEG, H.264, or H.265 profile for stream 11; refresh/key-frame control, realtime pipeline, and memory budget resolved |
11
11
  | `h5_talkback` | G.711 A-law, 8 kHz downlink decode and speaker path for stream 14; audio controller/GPIO ownership and memory budget resolved |
12
- | `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session`; audio ownership/channel mapping and memory budget resolved |
12
+ | `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session`; simultaneous capture/playback, physical reference, enabled AEC, audio ownership/channel mapping and memory budget resolved |
13
+ | `device_call` | Device-call protocol and arbiter contract plus simultaneous A-law capture/playback, physical reference and enabled AEC |
14
+ | `wechat_voip` | WeChat VoIP protocol and arbiter contract plus simultaneous A-law capture/playback, physical reference and enabled AEC |
15
+
16
+ `ai_talk`, `device_call`, and `wechat_voip` are AEC-required features. They are
17
+ `BLOCKED` when the board cannot capture a playback reference, cannot keep RX/TX
18
+ active together, or has no evidenced AEC implementation. At build time the audio
19
+ contract must explicitly report both simultaneous directions and enabled AEC;
20
+ generic full-duplex labels, a software flag without reference routing, or a
21
+ half-duplex fallback do not satisfy the gate.
13
22
 
14
23
  Hardware identity, wiring, the selected media contract, and the adapter/resource design must be at least `corroborated` to become `READY_TO_PORT`. At intake, `available=true` means a pinned source path can implement the selected profile; it does not claim that the final adapter or physical media path has run. Implementation composition, final ELF policy, runtime memory margin, browser media, and stability belong to the build or HIL phases.
15
24
 
@@ -45,6 +54,7 @@ SoftAP is one Wi-Fi option, not a universal requirement. BLE, SmartConfig, facto
45
54
  - Match the TiRTC precompiled SDK platform to the ESP-IDF target and its `manifest/build-contract.env` to the generated configuration.
46
55
  - Keep H5 stream IDs and formats stable unless the user explicitly authorizes a coordinated contract change across the server and consumers.
47
56
  - Start AI media only after the successful `start_session` response; stop and flush media before disconnecting.
57
+ - Route AI, device-call and WeChat VoIP through the same foreground-session arbiter and keep AEC active for the entire simultaneous capture/playback interval.
48
58
  - Copy SDK callback payloads into bounded queues before returning. Perform decoding, playback, HTTP, and lifecycle changes outside callbacks.
49
59
  - Use monotonic timestamps and session generation to reject stale frames and delayed callbacks.
50
60
  - Record runtime evidence with the exact BIN/ELF SHA-256. Use `assess --phase hil --artifact-sha256 <sha>`; documentation verification alone never becomes v2 `HIL_VERIFIED`.
@@ -48,6 +48,19 @@ For an explicitly selected legacy workspace, omit `--expected-kit`; otherwise re
48
48
 
49
49
  The default managed root is `<setup-root>/kits/esp32s3/<kit-version>`. The public ThingConnect workspace remains an optional legacy/development input; the doctor accepts either a Device Kit root, a repository root, or its `thing-connect/` child.
50
50
 
51
+ The managed Kit currently contains the ESP32-S3 H5/AI starter. When the requested
52
+ portfolio includes device-to-device calling or WeChat VoIP and the selected Kit
53
+ does not contain `device-sim-c`, `device-call.md`, `device-voip.md`, and the API
54
+ reference, use a full `tirtc-server-example` checkout pinned to a recorded commit
55
+ for the simulator and porting source. Cloning is an external write and requires
56
+ user authorization. Do not track a moving default branch as build evidence.
57
+
58
+ ESP32-P4 uses a separate `espressif_esp32p4` SDK archive and RISC-V toolchain.
59
+ The managed setup does not install that archive or generate a P4 starter. A P4
60
+ task therefore needs an explicit BSP/network project plus the exact P4 SDK and
61
+ `manifest/build-contract.env`; run Doctor with `--target esp32p4` and never accept
62
+ the packaged S3 archive as a compatible fallback.
63
+
51
64
  SDK resolution is independent after generation: an explicit `--sdk-dir` wins, followed by `<project>/third_party/tirtc`, then the SDK packaged in the resolved Device Kit or legacy workspace. The generated project remains diagnosable after it is moved away from the Kit.
52
65
 
53
66
  Before copying a generated project to another machine, run:
@@ -58,6 +71,13 @@ python3 <skill-dir>/scripts/project_portability.py <generated-project> --export
58
71
 
59
72
  Copy source inputs only. Never export `build/`: CMake caches absolute source, toolchain, and Python paths from the originating machine. `managed_components/` may be regenerated from the committed `dependencies.lock`; the bundled `third_party/tirtc` SDK and its build contract must remain in the source package. CMake must invoke shell gates through `bash <script>` so the build does not depend on archive- or filesystem-specific executable bits.
60
73
 
74
+ The export check also requires Hardware IR, every requested-feature semantic
75
+ contract, `sdkconfig.defaults`, a referenced custom partition table, and each
76
+ artifact retained in `build_evidence.artifacts[]`. Inside a Git worktree these
77
+ inputs must not be untracked files hidden by `.gitignore`. Ignore local build
78
+ trees and intermediate firmware snapshots, then explicitly include the exact
79
+ validated release bundle used by retained evidence.
80
+
61
81
  ## Required checks
62
82
 
63
83
  - `python3`, `git`, `idf.py`, and the target compiler are available in the active shell;
@@ -10,6 +10,11 @@ python3 <skill-dir>/scripts/hardware_ir.py init <output>/hardware-ir.json
10
10
 
11
11
  The initializer creates schema v2. The validator still accepts schema v1 for existing H.264-only projects; update new or materially changed boards to v2.
12
12
 
13
+ Schema v1 cannot represent an MJPEG/H.265 selection, semantic-contract paths,
14
+ or artifact-bound v2 HIL. If an existing v1 project requests H5 video without
15
+ an available H.264 path, migrate it to v2 rather than interpreting the legacy
16
+ H.264 assessment failure as a board or platform codec failure.
17
+
13
18
  ## Evidence rules
14
19
 
15
20
  - Give every source a stable `id`, `kind`, `location`, and revision when available.
@@ -37,16 +42,23 @@ The IR contains:
37
42
  - exact board identity, module, Flash/PSRAM, ESP-IDF, TiRTC SDK and build contract;
38
43
  - camera identity evidence plus `video_profiles[]` and `selected_video_profile`;
39
44
  - audio input/output paths;
40
- - `hardware_resources` for I2C, I2S/GPIO ownership, audio channel mapping, camera realtime policy, and memory budget;
45
+ - `hardware_resources` for I2C, I2S/GPIO ownership, audio channel mapping,
46
+ full-duplex/AEC capability, camera realtime policy, and memory budget;
41
47
  - project-relative `hardware_resources.audio_semantic_contract` for every project requesting audio;
42
48
  - project-relative `hardware_resources.video_semantic_contract` for every project requesting video;
43
- - project-relative `hardware_resources.runtime_semantic_contract` for every generated H5/AI project;
49
+ - project-relative `hardware_resources.runtime_semantic_contract` for every generated H5/AI/call/VoIP project;
44
50
  - `onboarding.wifi_credentials` with selectable SoftAP/BLE/SmartConfig/factory/development/custom methods;
45
51
  - selectable ThingConnect binding methods plus stored-binding states and reset control;
46
52
  - requested features;
47
53
  - optional `runtime_evidence[]`, each bound to a full firmware SHA-256.
48
54
  - `build_evidence.artifacts[]` containing each accepted BIN/ELF path, byte size, and full SHA-256.
49
55
 
56
+ When `ai_talk`, `device_call`, or `wechat_voip` is requested,
57
+ `hardware_resources.duplex_audio` must establish simultaneous capture/playback,
58
+ an actual playback-reference signal, and an AEC implementation. Unknown values
59
+ remain `NEEDS_CONFIRMATION`; a confirmed missing value is `BLOCKED`. Build/HIL
60
+ assessment also requires the project audio semantic gate to prove enabled AEC.
61
+
50
62
  The selected Wi-Fi method must be available and corroborated, credentials must remain outside tracked source, and reprovisioning must be defined. A board without SoftAP is valid when another selected method meets those conditions.
51
63
 
52
64
  The selected video profile controls assessment. An unselected H.264 fallback cannot make an MJPEG target pass, and missing H.264 cannot block an evidenced MJPEG target.
@@ -72,7 +84,9 @@ python3 <skill-dir>/scripts/hardware_ir.py assess hardware-ir.json \
72
84
  --phase hil --artifact-sha256 <64-character-sha256> --strict
73
85
  ```
74
86
 
75
- H5 features require matching L5 evidence; AI requires matching L6 evidence. Evidence from an older firmware remains provenance but does not verify the current artifact.
87
+ H5 features require matching L5 evidence; AI, device-call and WeChat VoIP
88
+ require matching L6 evidence. Evidence from an older firmware remains
89
+ provenance but does not verify the current artifact.
76
90
 
77
91
  ## Intake quality
78
92
 
@@ -13,6 +13,17 @@ Record the selected video profile, audio formats and stream IDs, duplex/AEC poli
13
13
  - I2S/audio: record controller, GPIO, master/slave mode, clocks, DMA, channel/TDM slot to physical-signal mapping, and shared-signal handoff. A codec name or I2C address does not prove the audio path.
14
14
  - Realtime camera path: record DMA/event queues, task core/priority, and competing Wi-Fi work. Treat queue overflows as scheduling or throughput evidence, then change one variable per HIL comparison.
15
15
 
16
+ Distinguish the camera driver's event task from an adapter-owned software
17
+ JPEG/H.26x task. Core isolation of the driver does not prove that a CPU-bound
18
+ encoder is isolated. On ESP32-S3, code using floating point can cause an
19
+ unpinned task to become pinned to the first core where it uses the FPU; AEC and
20
+ other DSP tasks therefore need intentional affinity when they would otherwise
21
+ land on the Wi-Fi core. A DMA-fed loop or software encoder that can remain
22
+ continuously runnable must block on a queue/semaphore or yield at a bounded
23
+ frame boundary. Keep the idle-task watchdog enabled, and expose maximum
24
+ processing time plus deadline misses instead of hiding starvation by widening
25
+ or disabling the watchdog.
26
+
16
27
  Turn every discovered invariant that can regress into a focused test or post-link gate. Generate board-specific values with the project; keep the Skill generic.
17
28
 
18
29
  ## Wi-Fi credentials and device binding
@@ -38,6 +49,13 @@ Platform service discovery and the TiRTC SDK service endpoint are different sett
38
49
 
39
50
  Define a conservative static budget before implementation using the locked SDK contract, framebuffer geometry, DMA/queue bounds, task stacks, and an internal-memory reserve. Before claiming runtime margin or tuning TiRTC buffers, measure internal free/largest blocks, PSRAM, frame size distribution, queue watermarks, and send/drop rates on the exact artifact. A larger queue can prevent drops, exhaust startup memory, or add buffer latency. If an authorized HTTP baseline exists, stage transport changes separately from media changes and retain the HTTPS requirements as a pending acceptance item.
40
51
 
52
+ Treat PSRAM framebuffer placement, direct peripheral DMA into PSRAM, and an
53
+ internal-DMA staging buffer as three separate facts. PSRAM capacity does not
54
+ prove DMA compatibility or frame integrity for a sensor/pixel format. Promote
55
+ direct PSRAM DMA only after artifact-bound HIL checks image boundaries and line
56
+ integrity; otherwise retain the evidenced staging path and budget its internal
57
+ DMA reserve explicitly.
58
+
41
59
  ## Network and media evidence
42
60
 
43
61
  Disable Wi-Fi power saving for the realtime baseline unless the product contract says otherwise. Record BSSID, channel, RSSI, reconnect/roaming events, media counters, queue watermarks, camera overflows, internal heap, and largest block. Establish a strong, stable AP baseline before controlled weak-network testing.
@@ -47,3 +65,10 @@ Confirm SDK send return semantics and callback payload lifetimes from the select
47
65
  ## Artifact discipline
48
66
 
49
67
  Record BIN/ELF SHA-256 for every build used in HIL. Runtime evidence applies only to the exact artifact. Diagnose one failing invariant, make the smallest correction, rerun that layer, and preserve the comparison in the report.
68
+
69
+ Local intermediate snapshots may be ignored, but the Hardware IR, semantic
70
+ contracts, `dependencies.lock`, custom partition input, and the artifact named
71
+ by retained build evidence must survive the chosen export mechanism. Broad
72
+ `*.json`, `*.csv`, or `artifacts/` ignore rules are invalid when they hide an
73
+ untracked required input; `project_portability.py --export` checks this when the
74
+ project is inside a Git worktree.
@@ -13,7 +13,7 @@ Use [the report template](../assets/report-template.md) and preserve separate `P
13
13
  | L3 Online | Wi-Fi provisioning, binding, MQTT, and TiRTC reach ready state |
14
14
  | L4 Media | Camera/microphone/speaker local paths work and counters/measurements are captured |
15
15
  | L5 H5 | Browser receives the declared video/audio and talkback reaches the device |
16
- | L6 AI | Token, WHIP, `start_session`, bidirectional audio, stop, and H5 recovery work |
16
+ | L6 Sessions | AI, device-call and WeChat VoIP requested flows work; simultaneous capture/playback, AEC/double-talk, stop, timeout and H5 recovery are observed separately |
17
17
  | L7 Stability | Requested weak-network, repeated-session, resource, and soak criteria pass |
18
18
 
19
19
  Run and record the intake assessment before L0, the build assessment with `--project` and the exact artifact SHA-256 at L1, and the HIL assessment only when matching runtime evidence exists. Before assessment, copy final deliverable BIN/ELF files to project-relative `artifacts/` paths and record their actual byte size and SHA-256 in `build_evidence.artifacts[]`; the assessor reopens the file and rejects stale metadata. Missing serial or browser access is a `SKIP` for the affected L2-L7 levels, not an L0/L1 failure.
@@ -1,6 +1,6 @@
1
1
  # TiRTC runtime protocol contract
2
2
 
3
- Use this gate for every generated H5/AI project. It verifies protocol behavior that is
3
+ Use this gate for every generated H5/AI/call/VoIP project. It verifies protocol behavior that is
4
4
  neither a board clock fact nor a camera fact: service discovery wiring, SDK callback
5
5
  lifecycle, stream/media metadata, and AI session negotiation.
6
6
 
@@ -30,5 +30,15 @@ The gate requires all of the following:
30
30
  - AI media starts only after a matching response provides a non-empty session ID and
31
31
  authoritative input/output audio formats matching the implemented codec;
32
32
  - remote `end_session` converges through the runtime control task.
33
+ - requested device-call/WeChat VoIP endpoints, MQTT event names, `call`/`wxcall`
34
+ commands and their pinned `tirtc-server-example` protocol revision are present
35
+ in implementation files;
36
+ - one foreground-session arbiter owns STREAM/AI/CALL/VOIP state, allows one
37
+ pending request, rejects stale generations, uses monotonic deadlines, defers
38
+ lifecycle work outside callbacks and restores H5 after foreground calls.
33
39
 
34
- Compilation without this gate is not an H5/AI `BUILD_VERIFIED` result.
40
+ For device-call or WeChat VoIP, populate the example contract's `business`
41
+ section. An empty `business.features` list remains valid for H5/AI-only projects,
42
+ but cannot satisfy Hardware IR that requests `device_call` or `wechat_voip`.
43
+
44
+ Compilation without this gate is not an H5/AI/call/VoIP `BUILD_VERIFIED` result.
@@ -0,0 +1,92 @@
1
+ # TiRTC and ThingConnect platform contract
2
+
3
+ Read this reference when integrating the C SDK, selecting an ESP32 target, or
4
+ implementing H5, AI, device-call or WeChat VoIP.
5
+
6
+ ## Authoritative sources
7
+
8
+ - TiRTC documentation: <https://docs.tange.ai/products/tirtc/>
9
+ - C API: <https://docs.tange.ai/products/tirtc/api-reference/c.html>
10
+ - C SDK integration: <https://docs.tange.ai/products/tirtc/guides/sdk-integration/c.html>
11
+ - ESP32-S3 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-s3.html>
12
+ - ESP32-P4 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-p4.html>
13
+ - ThingConnect reference platform: <https://github.com/tangeai/tirtc-server-example>
14
+ - Demo platform: <https://demo-open.tange-ai.com/>
15
+
16
+ Use the locked SDK header/build contract and a pinned ThingConnect commit during
17
+ generation. Public pages explain the contract but do not replace the exact files
18
+ used by a build.
19
+
20
+ ## SDK lifecycle
21
+
22
+ TiRTC is one process-wide runtime. The exact selected header governs API and
23
+ option availability. For SDK 2.3.0 the important order is:
24
+
25
+ 1. Set `TIRTC_OPT_MAX_SEND_BUFFER` before `TiRtcInit()` when overriding it.
26
+ 2. Call `TiRtcInit()` once.
27
+ 3. Set `TIRTC_OPT_DEVICE_SECRET_KEY`, `TIRTC_OPT_CLIENT_ID`, the discovered
28
+ service endpoint, and applicable network options before `TiRtcStart()`.
29
+ 4. Call `TiRtcStart(device_id, &callbacks)` once and wait for
30
+ `TIRTC_EVENT_SYS_STARTED`; return value zero only accepts the request.
31
+ 5. Establish or accept connections, then use `TiRtcSendVideoStream()` and
32
+ `TiRtcSendAudioStream()` with the selected complete-frame contract.
33
+ 6. Stop sessions and connections through deferred lifecycle work before the one
34
+ final `TiRtcStop()` / `TiRtcUninit()` sequence.
35
+
36
+ The callback table and its context outlive the SDK runtime. Callback payloads
37
+ are borrowed; copy required data into bounded application-owned storage before
38
+ returning.
39
+
40
+ ## Product features
41
+
42
+ ### H5 live view and talkback
43
+
44
+ The device starts as a listener, accepts the H5 connection, sends the selected
45
+ audio/video streams, receives talkback, and handles refresh/key-frame requests.
46
+ Browser rendering and audible talkback are HIL evidence.
47
+
48
+ ### AI intercom
49
+
50
+ Obtain the current AI connection parameters, connect, send `start_session`,
51
+ validate the authoritative session ID and input/output formats, then start
52
+ media. Remote or local `end_session` converges through the runtime task and
53
+ restores the H5 baseline.
54
+
55
+ ### Device-to-device call
56
+
57
+ Follow the pinned `device-call.md`: contact authorization, `POST
58
+ /v1/call/request`, incoming MQTT routing, accept through `POST
59
+ /v1/call/device/info`, reject/cancel/hangup, room recovery and `TiRtcConnect`.
60
+ Expose `call <device_id> [video|audio]` only after those paths and conflict rules
61
+ are implemented.
62
+
63
+ ### WeChat VoIP
64
+
65
+ Follow the pinned `device-voip.md`: report the device media profile, maintain the
66
+ authorized contact list, route WeChat MQTT events, and use `POST
67
+ /v1/voip/device/call` for device-originated calls. `wxcall [N] [video|audio]`
68
+ selects an authorized contact; it is not a direct SDK API. Mini-program
69
+ authorization and plugin behavior are separate platform acceptance evidence.
70
+
71
+ ## Simulator before hardware
72
+
73
+ For the four-feature portfolio, use the pinned Linux C reference implementation
74
+ with file-backed media to prove platform credentials, HTTP/MQTT fields,
75
+ connection tokens, commands and the session arbiter before replacing media with
76
+ board drivers. Record this separately from ESP32 L1-L7 evidence.
77
+
78
+ The managed ESP32-S3 starter may expose fewer businesses than the full reference
79
+ repository. Missing generated modules are implementation work, not permission to
80
+ drop a requested feature or call it verified.
81
+
82
+ ## ESP32 targets
83
+
84
+ The current managed generator and Device Kit automate ESP32-S3. Official SDK
85
+ 2.3.0 also provides a distinct ESP32-P4 package. P4 needs ESP-IDF 5.5.4, matching
86
+ RISC-V toolchain/build contract, PSRAM, and an evidenced network path such as
87
+ ESP-Hosted with C6/C61 or Ethernet. P4 and S3 archives are not interchangeable.
88
+
89
+ Use FreeRTOS tasks, bounded queues and explicit memory capabilities. PSRAM is
90
+ suitable for large media, fixed pools and eligible background stacks; internal
91
+ RAM remains necessary for DMA descriptors/buffers, ISR-visible state, control
92
+ objects and allocations required while flash cache is unavailable.
@@ -4,13 +4,18 @@
4
4
 
5
5
  ### Registered board
6
6
 
7
- Use this branch when a board and exact hardware revision already have a Hardware IR plus a matching board media adapter.
8
-
9
- 1. Validate and assess the saved Hardware IR against the requested features.
10
- 2. Confirm that its BSP, ESP-IDF, TiRTC SDK, and adapter revisions are still resolvable.
11
- 3. Generate a new starter project without overwriting an existing path.
12
- 4. Install the matching board adapter and configuration overlay.
13
- 5. Build, optionally flash, execute the requested acceptance levels, and issue a fresh report.
7
+ Use this branch only when `board_registry.py match` returns `exact` with
8
+ `safe_registered_reuse=true` and the package has a Hardware IR plus matching
9
+ board adapter.
10
+
11
+ 1. Recheck every required runtime probe and reject a changed sensor/codec as a
12
+ new variant.
13
+ 2. Validate and assess the saved Hardware IR against the requested features.
14
+ 3. Confirm that its BSP, ESP-IDF, TiRTC SDK, business protocol, and adapter
15
+ revisions are still resolvable.
16
+ 4. Generate a new starter project without overwriting an existing path.
17
+ 5. Install the matching board adapter and configuration overlay.
18
+ 6. Build, optionally flash, execute the requested acceptance levels, and issue a fresh report.
14
19
 
15
20
  The branch is complete when the new run has its own build and verification evidence; an older board report is provenance, not proof of the new artifact.
16
21
 
@@ -18,7 +23,10 @@ The branch is complete when the new run has its own build and verification evide
18
23
 
19
24
  Use this branch when the user supplies a board model, vendor URL, schematic, BOM, pin map, BSP, datasheets, photographs, or peripheral example projects without a verified adapter.
20
25
 
21
- 1. Resolve the full model, module, PCB marking, and hardware revision. Treat different revisions as different boards.
26
+ 1. Create `board-identity.json`, combine the developer's full model/PCB marking
27
+ with safe SoC/component probes, and query the board registry. Treat different
28
+ revisions or conflicting required probes as different variants. A probable
29
+ or component match supplies hypotheses only.
22
30
  2. Freeze the user-supplied product contract: selected video profile, audio/stream formats, duplex/AEC policy, supported Wi-Fi credential methods, selected onboarding method, transport staging, output path, and mutation boundary. Use `unknown` where the prompt lacks an answer.
23
31
  3. Prefer official schematic/BOM and BSP facts. For a PDF schematic, inspect page labels and net names; prefer an exported netlist, pin CSV, or vendor board definition when available.
24
32
  4. Cross-check critical pins, clocks, power enables, reset lines, sensor/codec variants, ESP-IDF version, resource ownership, and onboarding behavior across at least two independent artifacts when possible.
@@ -46,13 +54,16 @@ Read only the documents for the active branch, but read each selected document c
46
54
  - ESP32 starter or adapter work: `device-sim/device-sim-esp32/README.md`, `device-sim/ESP32_STARTER.md`, the selected SDK package README, and the generated template README.
47
55
  - H5 live view or talkback: `device-h5-live.md`.
48
56
  - AI intercom: `device-ai.md`.
57
+ - Device-to-device call: `device-call.md` and the applicable API reference.
58
+ - WeChat VoIP: `device-voip.md`, the mini-program integration document, and the
59
+ applicable API reference.
49
60
  - H5/AI switching, delayed callbacks, ownership, or timeouts: `device-session-model.md` and `device-session-arbiter.md`.
50
61
  - Onboarding, binding, MQTT, token, or identity: `device-integration.md`.
51
62
  - Public HTTP field or error changes: `api-reference.md` and `error-response-policy.md`.
52
63
 
53
64
  ## Generation and board seam
54
65
 
55
- The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, `starter_runtime`, or `starter_tirtc` for each board.
66
+ The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, CALL, VOIP, `starter_runtime`, or `starter_tirtc` for each board.
56
67
 
57
68
  The adapter owns:
58
69
 
@@ -62,13 +73,22 @@ The adapter owns:
62
73
  - DMA buffers, hardware clocks, power, reset, GPIO, refresh/key-frame requests, and realtime task allocation;
63
74
  - bounded stop, resource release, and generation-aware flushing.
64
75
 
65
- The stable modules own the discovered TiRTC service endpoint, stream IDs, negotiated/contracted formats, TiRTC callback copying, connection handles, session generation, and H5/AI sequencing. SDK lifecycle changes such as disconnect run in a worker/state-machine context, never directly inside an SDK callback.
76
+ The stable modules own the discovered TiRTC service endpoint, stream IDs,
77
+ negotiated/contracted formats, HTTP/MQTT business fields, TiRTC callback copying,
78
+ connection handles, pending calls, monotonic deadlines, session generation, and
79
+ H5/AI/CALL/VOIP sequencing. SDK lifecycle changes such as disconnect run in a
80
+ worker/state-machine context, never directly inside an SDK callback.
66
81
 
67
82
  ## Verification loop
68
83
 
69
84
  Use a bounded loop per layer: diagnose one failing invariant, make the smallest correction, and rerun that layer before moving forward. Change one high-risk variable per HIL comparison. Turn reusable invariants into tests or post-link gates. Stop and report when the remaining failure requires unavailable hardware, credentials, a new SDK binary, a public protocol change, or a user choice.
70
85
 
71
- For every generated H5/AI project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
86
+ For every generated H5/AI/call/VoIP project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. AI/call/VoIP also require the full-duplex AEC result from the audio gate. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
87
+
88
+ When CALL or VOIP is requested, the runtime contract must also bind the project
89
+ to the pinned business protocol and unified session arbiter. First prove those
90
+ flows with the Linux C simulator. Simulator success establishes the protocol
91
+ baseline but cannot replace the ESP32 build or HIL levels.
72
92
 
73
93
  Run the assessor once per layer:
74
94