tirtc-device-builder 0.7.2 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +1 -1
- package/CHANGELOG.md +23 -0
- package/README.md +44 -14
- package/bin/tirtc-device-builder.js +33 -1
- package/package.json +1 -1
- package/skills/tirtc-esp32-builder/SKILL.md +139 -39
- package/skills/tirtc-esp32-builder/USAGE.md +3 -2
- package/skills/tirtc-esp32-builder/VERSION +1 -1
- package/skills/tirtc-esp32-builder/agents/openai.yaml +1 -1
- package/skills/tirtc-esp32-builder/assets/board-audio-contract.example.json +4 -0
- package/skills/tirtc-esp32-builder/assets/board-identity.example.json +16 -0
- package/skills/tirtc-esp32-builder/assets/developer-intake-prompt.md +3 -3
- package/skills/tirtc-esp32-builder/assets/hardware-ir-v2.example.json +9 -1
- package/skills/tirtc-esp32-builder/assets/lckfb-szpi-esp32s3-portable-prompt.md +5 -5
- package/skills/tirtc-esp32-builder/assets/report-template.md +6 -1
- package/skills/tirtc-esp32-builder/assets/tirtc-runtime-contract.example.json +13 -0
- package/skills/tirtc-esp32-builder/knowledge/board-registry.json +4 -0
- package/skills/tirtc-esp32-builder/references/audio-contract.md +33 -3
- package/skills/tirtc-esp32-builder/references/board-knowledge.md +72 -0
- package/skills/tirtc-esp32-builder/references/capability-rules.md +11 -1
- package/skills/tirtc-esp32-builder/references/environment.md +20 -0
- package/skills/tirtc-esp32-builder/references/hardware-ir.md +17 -3
- package/skills/tirtc-esp32-builder/references/porting-risks.md +25 -0
- package/skills/tirtc-esp32-builder/references/reporting.md +1 -1
- package/skills/tirtc-esp32-builder/references/runtime-contract.md +12 -2
- package/skills/tirtc-esp32-builder/references/tirtc-platform.md +92 -0
- package/skills/tirtc-esp32-builder/references/workflow.md +31 -11
- package/skills/tirtc-esp32-builder/scripts/audio_contract.py +105 -3
- package/skills/tirtc-esp32-builder/scripts/board_registry.py +677 -0
- package/skills/tirtc-esp32-builder/scripts/hardware_ir.py +147 -8
- package/skills/tirtc-esp32-builder/scripts/project_portability.py +178 -3
- package/skills/tirtc-esp32-builder/scripts/runtime_contract.py +122 -1
|
@@ -27,6 +27,9 @@
|
|
|
27
27
|
| H5 live video | | |
|
|
28
28
|
| H5 talkback | | |
|
|
29
29
|
| AI intercom | | |
|
|
30
|
+
| Device-to-device call | | |
|
|
31
|
+
| WeChat VoIP | | |
|
|
32
|
+
| Full-duplex AEC/reference path | | |
|
|
30
33
|
|
|
31
34
|
## Semantic build gates
|
|
32
35
|
|
|
@@ -37,6 +40,8 @@
|
|
|
37
40
|
- Audio contract path/SHA-256: `{{AUDIO_CONTRACT}}`
|
|
38
41
|
- Codec clock-table result: `{{AUDIO_CLOCK_GATE}}`
|
|
39
42
|
- I2S mode/controller/slot/handoff result: `{{AUDIO_TOPOLOGY_GATE}}`
|
|
43
|
+
- Simultaneous capture/playback and AEC result: `{{AEC_GATE}}`
|
|
44
|
+
- Device-call/WeChat protocol and arbiter result: `{{BUSINESS_RUNTIME_GATE}}`
|
|
40
45
|
- Video contract path/SHA-256: `{{VIDEO_CONTRACT}}`
|
|
41
46
|
- Camera lock/PID/CPU/frame/backpressure result: `{{VIDEO_GATE}}`
|
|
42
47
|
- Final ELF I2C driver-family result: `{{I2C_ELF_GATE}}`
|
|
@@ -54,7 +59,7 @@
|
|
|
54
59
|
| L3 Online | | |
|
|
55
60
|
| L4 Media | | |
|
|
56
61
|
| L5 H5 | | |
|
|
57
|
-
| L6 AI | | |
|
|
62
|
+
| L6 AI/CALL/VOIP/AEC | | |
|
|
58
63
|
| L7 Stability | | |
|
|
59
64
|
|
|
60
65
|
## Firmware and flash record
|
|
@@ -6,5 +6,18 @@
|
|
|
6
6
|
"app_main": "main/app_main.c",
|
|
7
7
|
"starter_tirtc": "components/starter_tirtc/src/starter_tirtc.c",
|
|
8
8
|
"starter_runtime": "components/starter_runtime/src/starter_runtime.c"
|
|
9
|
+
},
|
|
10
|
+
"business": {
|
|
11
|
+
"features": [],
|
|
12
|
+
"protocol_revision": null,
|
|
13
|
+
"session_arbiter": {
|
|
14
|
+
"single_foreground_owner": null,
|
|
15
|
+
"pending_capacity": null,
|
|
16
|
+
"generation_guard": null,
|
|
17
|
+
"monotonic_deadlines": null,
|
|
18
|
+
"deferred_lifecycle": null,
|
|
19
|
+
"restore_h5_after_call": null
|
|
20
|
+
},
|
|
21
|
+
"implementation_assertions": []
|
|
9
22
|
}
|
|
10
23
|
}
|
|
@@ -10,12 +10,40 @@ The contract must describe:
|
|
|
10
10
|
|
|
11
11
|
- the PCM sample rate, MCLK ratio, resulting MCLK, and every clocked codec driver table;
|
|
12
12
|
- capture/playback controller, role, and standard/TDM/DSP/PCM mode;
|
|
13
|
+
- per-direction slot count and slot bit width when simultaneous directions use
|
|
14
|
+
different framing modes;
|
|
13
15
|
- TDM enable, slot count/order, physical signal at each slot, selected slot, and mapping evidence when TDM is used;
|
|
14
16
|
- shared clock GPIOs, whether directions are simultaneous, and the release/recreate handoff;
|
|
17
|
+
- paired-channel ownership and equal BCLKs per frame when one controller runs
|
|
18
|
+
simultaneous standard TX and TDM RX;
|
|
19
|
+
- when AEC is selected, the hardware sample rate plus distinct microphone and
|
|
20
|
+
playback-reference slots/signals;
|
|
15
21
|
- source assertions tying the normalized contract to the actual adapter implementation.
|
|
16
22
|
|
|
17
23
|
A generic header comment such as “typically 256” is not coefficient evidence. After `idf.py reconfigure` resolves managed components, the selected `(MCLK, sample rate)` pair must exist in every locked codec table named by the contract.
|
|
18
24
|
|
|
25
|
+
Simultaneous directions do not have to use the same ESP-IDF mode name. A valid
|
|
26
|
+
mixed-mode topology must set `shared_clock.paired_channels=true`, use the same
|
|
27
|
+
controller, and provide `slot_count` plus `slot_bit_width` for both directions;
|
|
28
|
+
their products must match. For example, four 16-bit TDM RX slots and two 32-bit
|
|
29
|
+
standard TX slots both consume 64 BCLKs per frame. This exception does not allow
|
|
30
|
+
two independent masters to drive shared clock GPIOs.
|
|
31
|
+
|
|
32
|
+
For hardware-reference AEC, add `echo_cancellation` with `enabled`,
|
|
33
|
+
`sample_rate_hz`, `microphone_slot`/`microphone_signal`, and
|
|
34
|
+
`reference_slot`/`reference_signal`. The gate verifies that AEC runs while both
|
|
35
|
+
directions are active, uses the hardware clock rate, and maps two distinct TDM
|
|
36
|
+
signals. Implementation assertions must still bind the contract to the paired
|
|
37
|
+
channel setup, AEC library configuration, and exact slot extraction code.
|
|
38
|
+
|
|
39
|
+
AEC is not optional when Hardware IR requests `ai_talk`, `device_call`, or
|
|
40
|
+
`wechat_voip`. For those features, `echo_cancellation.enabled=false`, missing
|
|
41
|
+
playback-reference wiring, or non-simultaneous capture/playback is a hard
|
|
42
|
+
`BLOCKED` result. Do not silently downgrade the product to push-to-talk. The
|
|
43
|
+
contract verifier exports structured `simultaneous_capture_playback` and
|
|
44
|
+
`echo_cancellation_enabled` results so the Hardware IR assessor enforces this
|
|
45
|
+
rule rather than trusting summary text.
|
|
46
|
+
|
|
19
47
|
## Mandatory gate
|
|
20
48
|
|
|
21
49
|
Run the gate before claiming an audio-capable build:
|
|
@@ -44,8 +72,10 @@ Audio reaches `BUILD_VERIFIED` only when all of these are true:
|
|
|
44
72
|
1. the contract has at least two authoritative source IDs;
|
|
45
73
|
2. its arithmetic and codec table lookups pass;
|
|
46
74
|
3. its topology, TDM mapping, and handoff are internally consistent;
|
|
47
|
-
4.
|
|
48
|
-
|
|
49
|
-
|
|
75
|
+
4. simultaneous mixed framing has equal BCLKs per frame and any AEC reference is
|
|
76
|
+
mapped to a distinct evidenced slot;
|
|
77
|
+
5. its adapter assertions pass;
|
|
78
|
+
6. the gate is part of the ordinary build;
|
|
79
|
+
7. the exact BIN/ELF is hashed after that build.
|
|
50
80
|
|
|
51
81
|
Compilation without this gate is `COMPILE_PASS`, not audio `BUILD_VERIFIED`.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Board knowledge and identity
|
|
2
|
+
|
|
3
|
+
Use this reference when identifying a board, reusing an existing adapter, or
|
|
4
|
+
capturing lessons after bring-up. The registry is curated, versioned input. It
|
|
5
|
+
does not train a model or mutate an installed Skill from conversation history.
|
|
6
|
+
|
|
7
|
+
## Identity boundary
|
|
8
|
+
|
|
9
|
+
An ESP32 can report its SoC target, revision, Flash/PSRAM, MAC and sometimes its
|
|
10
|
+
module. Board firmware can probe camera PIDs, codec chip IDs and other bus
|
|
11
|
+
devices. These observations identify components, not necessarily the carrier
|
|
12
|
+
board sales model, PCB marking, hardware revision, wiring, power topology or
|
|
13
|
+
acoustic path. Obtain those facts from the developer and board documents.
|
|
14
|
+
|
|
15
|
+
Create a project-local identity file, merge the developer declaration with safe
|
|
16
|
+
read-only observations, then query the registry:
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
python3 <skill-dir>/scripts/board_registry.py init-identity \
|
|
20
|
+
--output <project>/board-identity.json
|
|
21
|
+
python3 <skill-dir>/scripts/board_registry.py match \
|
|
22
|
+
--identity <project>/board-identity.json
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Use only an `exact` result with `safe_registered_reuse=true` to install a saved
|
|
26
|
+
adapter. `probable` can supply hypotheses while missing revision or probe facts
|
|
27
|
+
are resolved. `component` can supply component-level risks and driver patterns,
|
|
28
|
+
but never GPIO, clock, DMA or adapter values. A conflict creates a new variant.
|
|
29
|
+
|
|
30
|
+
Exact matching requires vendor, model or alias, hardware revision, SoC/resource
|
|
31
|
+
compatibility, and every probe marked `required_for_exact`. Marketing names are
|
|
32
|
+
not enough because vendors may substitute sensors or codecs without renaming a
|
|
33
|
+
product.
|
|
34
|
+
|
|
35
|
+
## Knowledge scopes
|
|
36
|
+
|
|
37
|
+
- `generic`: an ESP32/TiRTC invariant. Promote only after two independent board
|
|
38
|
+
packages support it and a focused regression test enforces it.
|
|
39
|
+
- `component`: a sensor, codec, amplifier or library observation. Reuse it on a
|
|
40
|
+
different carrier only as a hypothesis until that board supplies evidence.
|
|
41
|
+
- `board`: a fact for one exact package identity. Apply it only to an exact
|
|
42
|
+
match.
|
|
43
|
+
|
|
44
|
+
Registry package status controls reuse:
|
|
45
|
+
|
|
46
|
+
- `knowledge_only`: lessons are usable, but the normal new-board intake remains
|
|
47
|
+
mandatory.
|
|
48
|
+
- `adapter_verified`: the Hardware IR, adapter, configuration and semantic
|
|
49
|
+
contracts are complete enough for the registered-board workflow.
|
|
50
|
+
- `hil_verified`: the reusable package also retains artifact-bound HIL evidence.
|
|
51
|
+
|
|
52
|
+
An older HIL result is provenance, not proof for a newly built artifact.
|
|
53
|
+
|
|
54
|
+
## Learning loop
|
|
55
|
+
|
|
56
|
+
After the final assessment, create a project-local candidate:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
python3 <skill-dir>/scripts/board_registry.py candidate \
|
|
60
|
+
--hardware-ir <project>/hardware-ir.json \
|
|
61
|
+
--output <project>/board-knowledge-candidate.json
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Review the candidate before promotion. Add exact runtime probes, copy only
|
|
65
|
+
portable project-relative adapter/config/contract inputs, classify each lesson's
|
|
66
|
+
scope, attach the tested artifact SHA-256 for hardware or HIL claims, and add a
|
|
67
|
+
regression test for every generic invariant. Then commit it to a maintained
|
|
68
|
+
registry and publish a new Skill version. An installed Skill never edits itself
|
|
69
|
+
or promotes an observation automatically.
|
|
70
|
+
|
|
71
|
+
Keep credentials, MAC-derived identity, device keys, Wi-Fi secrets, tokens and
|
|
72
|
+
user media out of identity files, candidates and registries.
|
|
@@ -9,7 +9,16 @@ Run `hardware_ir.py assess --phase intake --strict` before generation. Hardware
|
|
|
9
9
|
| `h5_live_audio` | Microphone path producing G.711 A-law, 8 kHz, mono for stream 10; audio controller/channel ownership and memory budget resolved |
|
|
10
10
|
| `h5_live_video` | Camera plus the selected MJPEG, H.264, or H.265 profile for stream 11; refresh/key-frame control, realtime pipeline, and memory budget resolved |
|
|
11
11
|
| `h5_talkback` | G.711 A-law, 8 kHz downlink decode and speaker path for stream 14; audio controller/GPIO ownership and memory budget resolved |
|
|
12
|
-
| `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session`; audio ownership/channel mapping and memory budget resolved |
|
|
12
|
+
| `ai_talk` | A-law 8 kHz microphone and speaker paths for AI stream 1, started only after `start_session`; simultaneous capture/playback, physical reference, enabled AEC, audio ownership/channel mapping and memory budget resolved |
|
|
13
|
+
| `device_call` | Device-call protocol and arbiter contract plus simultaneous A-law capture/playback, physical reference and enabled AEC |
|
|
14
|
+
| `wechat_voip` | WeChat VoIP protocol and arbiter contract plus simultaneous A-law capture/playback, physical reference and enabled AEC |
|
|
15
|
+
|
|
16
|
+
`ai_talk`, `device_call`, and `wechat_voip` are AEC-required features. They are
|
|
17
|
+
`BLOCKED` when the board cannot capture a playback reference, cannot keep RX/TX
|
|
18
|
+
active together, or has no evidenced AEC implementation. At build time the audio
|
|
19
|
+
contract must explicitly report both simultaneous directions and enabled AEC;
|
|
20
|
+
generic full-duplex labels, a software flag without reference routing, or a
|
|
21
|
+
half-duplex fallback do not satisfy the gate.
|
|
13
22
|
|
|
14
23
|
Hardware identity, wiring, the selected media contract, and the adapter/resource design must be at least `corroborated` to become `READY_TO_PORT`. At intake, `available=true` means a pinned source path can implement the selected profile; it does not claim that the final adapter or physical media path has run. Implementation composition, final ELF policy, runtime memory margin, browser media, and stability belong to the build or HIL phases.
|
|
15
24
|
|
|
@@ -45,6 +54,7 @@ SoftAP is one Wi-Fi option, not a universal requirement. BLE, SmartConfig, facto
|
|
|
45
54
|
- Match the TiRTC precompiled SDK platform to the ESP-IDF target and its `manifest/build-contract.env` to the generated configuration.
|
|
46
55
|
- Keep H5 stream IDs and formats stable unless the user explicitly authorizes a coordinated contract change across the server and consumers.
|
|
47
56
|
- Start AI media only after the successful `start_session` response; stop and flush media before disconnecting.
|
|
57
|
+
- Route AI, device-call and WeChat VoIP through the same foreground-session arbiter and keep AEC active for the entire simultaneous capture/playback interval.
|
|
48
58
|
- Copy SDK callback payloads into bounded queues before returning. Perform decoding, playback, HTTP, and lifecycle changes outside callbacks.
|
|
49
59
|
- Use monotonic timestamps and session generation to reject stale frames and delayed callbacks.
|
|
50
60
|
- Record runtime evidence with the exact BIN/ELF SHA-256. Use `assess --phase hil --artifact-sha256 <sha>`; documentation verification alone never becomes v2 `HIL_VERIFIED`.
|
|
@@ -48,6 +48,19 @@ For an explicitly selected legacy workspace, omit `--expected-kit`; otherwise re
|
|
|
48
48
|
|
|
49
49
|
The default managed root is `<setup-root>/kits/esp32s3/<kit-version>`. The public ThingConnect workspace remains an optional legacy/development input; the doctor accepts either a Device Kit root, a repository root, or its `thing-connect/` child.
|
|
50
50
|
|
|
51
|
+
The managed Kit currently contains the ESP32-S3 H5/AI starter. When the requested
|
|
52
|
+
portfolio includes device-to-device calling or WeChat VoIP and the selected Kit
|
|
53
|
+
does not contain `device-sim-c`, `device-call.md`, `device-voip.md`, and the API
|
|
54
|
+
reference, use a full `tirtc-server-example` checkout pinned to a recorded commit
|
|
55
|
+
for the simulator and porting source. Cloning is an external write and requires
|
|
56
|
+
user authorization. Do not track a moving default branch as build evidence.
|
|
57
|
+
|
|
58
|
+
ESP32-P4 uses a separate `espressif_esp32p4` SDK archive and RISC-V toolchain.
|
|
59
|
+
The managed setup does not install that archive or generate a P4 starter. A P4
|
|
60
|
+
task therefore needs an explicit BSP/network project plus the exact P4 SDK and
|
|
61
|
+
`manifest/build-contract.env`; run Doctor with `--target esp32p4` and never accept
|
|
62
|
+
the packaged S3 archive as a compatible fallback.
|
|
63
|
+
|
|
51
64
|
SDK resolution is independent after generation: an explicit `--sdk-dir` wins, followed by `<project>/third_party/tirtc`, then the SDK packaged in the resolved Device Kit or legacy workspace. The generated project remains diagnosable after it is moved away from the Kit.
|
|
52
65
|
|
|
53
66
|
Before copying a generated project to another machine, run:
|
|
@@ -58,6 +71,13 @@ python3 <skill-dir>/scripts/project_portability.py <generated-project> --export
|
|
|
58
71
|
|
|
59
72
|
Copy source inputs only. Never export `build/`: CMake caches absolute source, toolchain, and Python paths from the originating machine. `managed_components/` may be regenerated from the committed `dependencies.lock`; the bundled `third_party/tirtc` SDK and its build contract must remain in the source package. CMake must invoke shell gates through `bash <script>` so the build does not depend on archive- or filesystem-specific executable bits.
|
|
60
73
|
|
|
74
|
+
The export check also requires Hardware IR, every requested-feature semantic
|
|
75
|
+
contract, `sdkconfig.defaults`, a referenced custom partition table, and each
|
|
76
|
+
artifact retained in `build_evidence.artifacts[]`. Inside a Git worktree these
|
|
77
|
+
inputs must not be untracked files hidden by `.gitignore`. Ignore local build
|
|
78
|
+
trees and intermediate firmware snapshots, then explicitly include the exact
|
|
79
|
+
validated release bundle used by retained evidence.
|
|
80
|
+
|
|
61
81
|
## Required checks
|
|
62
82
|
|
|
63
83
|
- `python3`, `git`, `idf.py`, and the target compiler are available in the active shell;
|
|
@@ -10,6 +10,11 @@ python3 <skill-dir>/scripts/hardware_ir.py init <output>/hardware-ir.json
|
|
|
10
10
|
|
|
11
11
|
The initializer creates schema v2. The validator still accepts schema v1 for existing H.264-only projects; update new or materially changed boards to v2.
|
|
12
12
|
|
|
13
|
+
Schema v1 cannot represent an MJPEG/H.265 selection, semantic-contract paths,
|
|
14
|
+
or artifact-bound v2 HIL. If an existing v1 project requests H5 video without
|
|
15
|
+
an available H.264 path, migrate it to v2 rather than interpreting the legacy
|
|
16
|
+
H.264 assessment failure as a board or platform codec failure.
|
|
17
|
+
|
|
13
18
|
## Evidence rules
|
|
14
19
|
|
|
15
20
|
- Give every source a stable `id`, `kind`, `location`, and revision when available.
|
|
@@ -37,16 +42,23 @@ The IR contains:
|
|
|
37
42
|
- exact board identity, module, Flash/PSRAM, ESP-IDF, TiRTC SDK and build contract;
|
|
38
43
|
- camera identity evidence plus `video_profiles[]` and `selected_video_profile`;
|
|
39
44
|
- audio input/output paths;
|
|
40
|
-
- `hardware_resources` for I2C, I2S/GPIO ownership, audio channel mapping,
|
|
45
|
+
- `hardware_resources` for I2C, I2S/GPIO ownership, audio channel mapping,
|
|
46
|
+
full-duplex/AEC capability, camera realtime policy, and memory budget;
|
|
41
47
|
- project-relative `hardware_resources.audio_semantic_contract` for every project requesting audio;
|
|
42
48
|
- project-relative `hardware_resources.video_semantic_contract` for every project requesting video;
|
|
43
|
-
- project-relative `hardware_resources.runtime_semantic_contract` for every generated H5/AI project;
|
|
49
|
+
- project-relative `hardware_resources.runtime_semantic_contract` for every generated H5/AI/call/VoIP project;
|
|
44
50
|
- `onboarding.wifi_credentials` with selectable SoftAP/BLE/SmartConfig/factory/development/custom methods;
|
|
45
51
|
- selectable ThingConnect binding methods plus stored-binding states and reset control;
|
|
46
52
|
- requested features;
|
|
47
53
|
- optional `runtime_evidence[]`, each bound to a full firmware SHA-256.
|
|
48
54
|
- `build_evidence.artifacts[]` containing each accepted BIN/ELF path, byte size, and full SHA-256.
|
|
49
55
|
|
|
56
|
+
When `ai_talk`, `device_call`, or `wechat_voip` is requested,
|
|
57
|
+
`hardware_resources.duplex_audio` must establish simultaneous capture/playback,
|
|
58
|
+
an actual playback-reference signal, and an AEC implementation. Unknown values
|
|
59
|
+
remain `NEEDS_CONFIRMATION`; a confirmed missing value is `BLOCKED`. Build/HIL
|
|
60
|
+
assessment also requires the project audio semantic gate to prove enabled AEC.
|
|
61
|
+
|
|
50
62
|
The selected Wi-Fi method must be available and corroborated, credentials must remain outside tracked source, and reprovisioning must be defined. A board without SoftAP is valid when another selected method meets those conditions.
|
|
51
63
|
|
|
52
64
|
The selected video profile controls assessment. An unselected H.264 fallback cannot make an MJPEG target pass, and missing H.264 cannot block an evidenced MJPEG target.
|
|
@@ -72,7 +84,9 @@ python3 <skill-dir>/scripts/hardware_ir.py assess hardware-ir.json \
|
|
|
72
84
|
--phase hil --artifact-sha256 <64-character-sha256> --strict
|
|
73
85
|
```
|
|
74
86
|
|
|
75
|
-
H5 features require matching L5 evidence; AI
|
|
87
|
+
H5 features require matching L5 evidence; AI, device-call and WeChat VoIP
|
|
88
|
+
require matching L6 evidence. Evidence from an older firmware remains
|
|
89
|
+
provenance but does not verify the current artifact.
|
|
76
90
|
|
|
77
91
|
## Intake quality
|
|
78
92
|
|
|
@@ -13,6 +13,17 @@ Record the selected video profile, audio formats and stream IDs, duplex/AEC poli
|
|
|
13
13
|
- I2S/audio: record controller, GPIO, master/slave mode, clocks, DMA, channel/TDM slot to physical-signal mapping, and shared-signal handoff. A codec name or I2C address does not prove the audio path.
|
|
14
14
|
- Realtime camera path: record DMA/event queues, task core/priority, and competing Wi-Fi work. Treat queue overflows as scheduling or throughput evidence, then change one variable per HIL comparison.
|
|
15
15
|
|
|
16
|
+
Distinguish the camera driver's event task from an adapter-owned software
|
|
17
|
+
JPEG/H.26x task. Core isolation of the driver does not prove that a CPU-bound
|
|
18
|
+
encoder is isolated. On ESP32-S3, code using floating point can cause an
|
|
19
|
+
unpinned task to become pinned to the first core where it uses the FPU; AEC and
|
|
20
|
+
other DSP tasks therefore need intentional affinity when they would otherwise
|
|
21
|
+
land on the Wi-Fi core. A DMA-fed loop or software encoder that can remain
|
|
22
|
+
continuously runnable must block on a queue/semaphore or yield at a bounded
|
|
23
|
+
frame boundary. Keep the idle-task watchdog enabled, and expose maximum
|
|
24
|
+
processing time plus deadline misses instead of hiding starvation by widening
|
|
25
|
+
or disabling the watchdog.
|
|
26
|
+
|
|
16
27
|
Turn every discovered invariant that can regress into a focused test or post-link gate. Generate board-specific values with the project; keep the Skill generic.
|
|
17
28
|
|
|
18
29
|
## Wi-Fi credentials and device binding
|
|
@@ -38,6 +49,13 @@ Platform service discovery and the TiRTC SDK service endpoint are different sett
|
|
|
38
49
|
|
|
39
50
|
Define a conservative static budget before implementation using the locked SDK contract, framebuffer geometry, DMA/queue bounds, task stacks, and an internal-memory reserve. Before claiming runtime margin or tuning TiRTC buffers, measure internal free/largest blocks, PSRAM, frame size distribution, queue watermarks, and send/drop rates on the exact artifact. A larger queue can prevent drops, exhaust startup memory, or add buffer latency. If an authorized HTTP baseline exists, stage transport changes separately from media changes and retain the HTTPS requirements as a pending acceptance item.
|
|
40
51
|
|
|
52
|
+
Treat PSRAM framebuffer placement, direct peripheral DMA into PSRAM, and an
|
|
53
|
+
internal-DMA staging buffer as three separate facts. PSRAM capacity does not
|
|
54
|
+
prove DMA compatibility or frame integrity for a sensor/pixel format. Promote
|
|
55
|
+
direct PSRAM DMA only after artifact-bound HIL checks image boundaries and line
|
|
56
|
+
integrity; otherwise retain the evidenced staging path and budget its internal
|
|
57
|
+
DMA reserve explicitly.
|
|
58
|
+
|
|
41
59
|
## Network and media evidence
|
|
42
60
|
|
|
43
61
|
Disable Wi-Fi power saving for the realtime baseline unless the product contract says otherwise. Record BSSID, channel, RSSI, reconnect/roaming events, media counters, queue watermarks, camera overflows, internal heap, and largest block. Establish a strong, stable AP baseline before controlled weak-network testing.
|
|
@@ -47,3 +65,10 @@ Confirm SDK send return semantics and callback payload lifetimes from the select
|
|
|
47
65
|
## Artifact discipline
|
|
48
66
|
|
|
49
67
|
Record BIN/ELF SHA-256 for every build used in HIL. Runtime evidence applies only to the exact artifact. Diagnose one failing invariant, make the smallest correction, rerun that layer, and preserve the comparison in the report.
|
|
68
|
+
|
|
69
|
+
Local intermediate snapshots may be ignored, but the Hardware IR, semantic
|
|
70
|
+
contracts, `dependencies.lock`, custom partition input, and the artifact named
|
|
71
|
+
by retained build evidence must survive the chosen export mechanism. Broad
|
|
72
|
+
`*.json`, `*.csv`, or `artifacts/` ignore rules are invalid when they hide an
|
|
73
|
+
untracked required input; `project_portability.py --export` checks this when the
|
|
74
|
+
project is inside a Git worktree.
|
|
@@ -13,7 +13,7 @@ Use [the report template](../assets/report-template.md) and preserve separate `P
|
|
|
13
13
|
| L3 Online | Wi-Fi provisioning, binding, MQTT, and TiRTC reach ready state |
|
|
14
14
|
| L4 Media | Camera/microphone/speaker local paths work and counters/measurements are captured |
|
|
15
15
|
| L5 H5 | Browser receives the declared video/audio and talkback reaches the device |
|
|
16
|
-
| L6
|
|
16
|
+
| L6 Sessions | AI, device-call and WeChat VoIP requested flows work; simultaneous capture/playback, AEC/double-talk, stop, timeout and H5 recovery are observed separately |
|
|
17
17
|
| L7 Stability | Requested weak-network, repeated-session, resource, and soak criteria pass |
|
|
18
18
|
|
|
19
19
|
Run and record the intake assessment before L0, the build assessment with `--project` and the exact artifact SHA-256 at L1, and the HIL assessment only when matching runtime evidence exists. Before assessment, copy final deliverable BIN/ELF files to project-relative `artifacts/` paths and record their actual byte size and SHA-256 in `build_evidence.artifacts[]`; the assessor reopens the file and rejects stale metadata. Missing serial or browser access is a `SKIP` for the affected L2-L7 levels, not an L0/L1 failure.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# TiRTC runtime protocol contract
|
|
2
2
|
|
|
3
|
-
Use this gate for every generated H5/AI project. It verifies protocol behavior that is
|
|
3
|
+
Use this gate for every generated H5/AI/call/VoIP project. It verifies protocol behavior that is
|
|
4
4
|
neither a board clock fact nor a camera fact: service discovery wiring, SDK callback
|
|
5
5
|
lifecycle, stream/media metadata, and AI session negotiation.
|
|
6
6
|
|
|
@@ -30,5 +30,15 @@ The gate requires all of the following:
|
|
|
30
30
|
- AI media starts only after a matching response provides a non-empty session ID and
|
|
31
31
|
authoritative input/output audio formats matching the implemented codec;
|
|
32
32
|
- remote `end_session` converges through the runtime control task.
|
|
33
|
+
- requested device-call/WeChat VoIP endpoints, MQTT event names, `call`/`wxcall`
|
|
34
|
+
commands and their pinned `tirtc-server-example` protocol revision are present
|
|
35
|
+
in implementation files;
|
|
36
|
+
- one foreground-session arbiter owns STREAM/AI/CALL/VOIP state, allows one
|
|
37
|
+
pending request, rejects stale generations, uses monotonic deadlines, defers
|
|
38
|
+
lifecycle work outside callbacks and restores H5 after foreground calls.
|
|
33
39
|
|
|
34
|
-
|
|
40
|
+
For device-call or WeChat VoIP, populate the example contract's `business`
|
|
41
|
+
section. An empty `business.features` list remains valid for H5/AI-only projects,
|
|
42
|
+
but cannot satisfy Hardware IR that requests `device_call` or `wechat_voip`.
|
|
43
|
+
|
|
44
|
+
Compilation without this gate is not an H5/AI/call/VoIP `BUILD_VERIFIED` result.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# TiRTC and ThingConnect platform contract
|
|
2
|
+
|
|
3
|
+
Read this reference when integrating the C SDK, selecting an ESP32 target, or
|
|
4
|
+
implementing H5, AI, device-call or WeChat VoIP.
|
|
5
|
+
|
|
6
|
+
## Authoritative sources
|
|
7
|
+
|
|
8
|
+
- TiRTC documentation: <https://docs.tange.ai/products/tirtc/>
|
|
9
|
+
- C API: <https://docs.tange.ai/products/tirtc/api-reference/c.html>
|
|
10
|
+
- C SDK integration: <https://docs.tange.ai/products/tirtc/guides/sdk-integration/c.html>
|
|
11
|
+
- ESP32-S3 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-s3.html>
|
|
12
|
+
- ESP32-P4 notes: <https://docs.tange.ai/products/tirtc/guides/integration-notes/espressif/esp32-p4.html>
|
|
13
|
+
- ThingConnect reference platform: <https://github.com/tangeai/tirtc-server-example>
|
|
14
|
+
- Demo platform: <https://demo-open.tange-ai.com/>
|
|
15
|
+
|
|
16
|
+
Use the locked SDK header/build contract and a pinned ThingConnect commit during
|
|
17
|
+
generation. Public pages explain the contract but do not replace the exact files
|
|
18
|
+
used by a build.
|
|
19
|
+
|
|
20
|
+
## SDK lifecycle
|
|
21
|
+
|
|
22
|
+
TiRTC is one process-wide runtime. The exact selected header governs API and
|
|
23
|
+
option availability. For SDK 2.3.0 the important order is:
|
|
24
|
+
|
|
25
|
+
1. Set `TIRTC_OPT_MAX_SEND_BUFFER` before `TiRtcInit()` when overriding it.
|
|
26
|
+
2. Call `TiRtcInit()` once.
|
|
27
|
+
3. Set `TIRTC_OPT_DEVICE_SECRET_KEY`, `TIRTC_OPT_CLIENT_ID`, the discovered
|
|
28
|
+
service endpoint, and applicable network options before `TiRtcStart()`.
|
|
29
|
+
4. Call `TiRtcStart(device_id, &callbacks)` once and wait for
|
|
30
|
+
`TIRTC_EVENT_SYS_STARTED`; return value zero only accepts the request.
|
|
31
|
+
5. Establish or accept connections, then use `TiRtcSendVideoStream()` and
|
|
32
|
+
`TiRtcSendAudioStream()` with the selected complete-frame contract.
|
|
33
|
+
6. Stop sessions and connections through deferred lifecycle work before the one
|
|
34
|
+
final `TiRtcStop()` / `TiRtcUninit()` sequence.
|
|
35
|
+
|
|
36
|
+
The callback table and its context outlive the SDK runtime. Callback payloads
|
|
37
|
+
are borrowed; copy required data into bounded application-owned storage before
|
|
38
|
+
returning.
|
|
39
|
+
|
|
40
|
+
## Product features
|
|
41
|
+
|
|
42
|
+
### H5 live view and talkback
|
|
43
|
+
|
|
44
|
+
The device starts as a listener, accepts the H5 connection, sends the selected
|
|
45
|
+
audio/video streams, receives talkback, and handles refresh/key-frame requests.
|
|
46
|
+
Browser rendering and audible talkback are HIL evidence.
|
|
47
|
+
|
|
48
|
+
### AI intercom
|
|
49
|
+
|
|
50
|
+
Obtain the current AI connection parameters, connect, send `start_session`,
|
|
51
|
+
validate the authoritative session ID and input/output formats, then start
|
|
52
|
+
media. Remote or local `end_session` converges through the runtime task and
|
|
53
|
+
restores the H5 baseline.
|
|
54
|
+
|
|
55
|
+
### Device-to-device call
|
|
56
|
+
|
|
57
|
+
Follow the pinned `device-call.md`: contact authorization, `POST
|
|
58
|
+
/v1/call/request`, incoming MQTT routing, accept through `POST
|
|
59
|
+
/v1/call/device/info`, reject/cancel/hangup, room recovery and `TiRtcConnect`.
|
|
60
|
+
Expose `call <device_id> [video|audio]` only after those paths and conflict rules
|
|
61
|
+
are implemented.
|
|
62
|
+
|
|
63
|
+
### WeChat VoIP
|
|
64
|
+
|
|
65
|
+
Follow the pinned `device-voip.md`: report the device media profile, maintain the
|
|
66
|
+
authorized contact list, route WeChat MQTT events, and use `POST
|
|
67
|
+
/v1/voip/device/call` for device-originated calls. `wxcall [N] [video|audio]`
|
|
68
|
+
selects an authorized contact; it is not a direct SDK API. Mini-program
|
|
69
|
+
authorization and plugin behavior are separate platform acceptance evidence.
|
|
70
|
+
|
|
71
|
+
## Simulator before hardware
|
|
72
|
+
|
|
73
|
+
For the four-feature portfolio, use the pinned Linux C reference implementation
|
|
74
|
+
with file-backed media to prove platform credentials, HTTP/MQTT fields,
|
|
75
|
+
connection tokens, commands and the session arbiter before replacing media with
|
|
76
|
+
board drivers. Record this separately from ESP32 L1-L7 evidence.
|
|
77
|
+
|
|
78
|
+
The managed ESP32-S3 starter may expose fewer businesses than the full reference
|
|
79
|
+
repository. Missing generated modules are implementation work, not permission to
|
|
80
|
+
drop a requested feature or call it verified.
|
|
81
|
+
|
|
82
|
+
## ESP32 targets
|
|
83
|
+
|
|
84
|
+
The current managed generator and Device Kit automate ESP32-S3. Official SDK
|
|
85
|
+
2.3.0 also provides a distinct ESP32-P4 package. P4 needs ESP-IDF 5.5.4, matching
|
|
86
|
+
RISC-V toolchain/build contract, PSRAM, and an evidenced network path such as
|
|
87
|
+
ESP-Hosted with C6/C61 or Ethernet. P4 and S3 archives are not interchangeable.
|
|
88
|
+
|
|
89
|
+
Use FreeRTOS tasks, bounded queues and explicit memory capabilities. PSRAM is
|
|
90
|
+
suitable for large media, fixed pools and eligible background stacks; internal
|
|
91
|
+
RAM remains necessary for DMA descriptors/buffers, ISR-visible state, control
|
|
92
|
+
objects and allocations required while flash cache is unavailable.
|
|
@@ -4,13 +4,18 @@
|
|
|
4
4
|
|
|
5
5
|
### Registered board
|
|
6
6
|
|
|
7
|
-
Use this branch when
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
7
|
+
Use this branch only when `board_registry.py match` returns `exact` with
|
|
8
|
+
`safe_registered_reuse=true` and the package has a Hardware IR plus matching
|
|
9
|
+
board adapter.
|
|
10
|
+
|
|
11
|
+
1. Recheck every required runtime probe and reject a changed sensor/codec as a
|
|
12
|
+
new variant.
|
|
13
|
+
2. Validate and assess the saved Hardware IR against the requested features.
|
|
14
|
+
3. Confirm that its BSP, ESP-IDF, TiRTC SDK, business protocol, and adapter
|
|
15
|
+
revisions are still resolvable.
|
|
16
|
+
4. Generate a new starter project without overwriting an existing path.
|
|
17
|
+
5. Install the matching board adapter and configuration overlay.
|
|
18
|
+
6. Build, optionally flash, execute the requested acceptance levels, and issue a fresh report.
|
|
14
19
|
|
|
15
20
|
The branch is complete when the new run has its own build and verification evidence; an older board report is provenance, not proof of the new artifact.
|
|
16
21
|
|
|
@@ -18,7 +23,10 @@ The branch is complete when the new run has its own build and verification evide
|
|
|
18
23
|
|
|
19
24
|
Use this branch when the user supplies a board model, vendor URL, schematic, BOM, pin map, BSP, datasheets, photographs, or peripheral example projects without a verified adapter.
|
|
20
25
|
|
|
21
|
-
1.
|
|
26
|
+
1. Create `board-identity.json`, combine the developer's full model/PCB marking
|
|
27
|
+
with safe SoC/component probes, and query the board registry. Treat different
|
|
28
|
+
revisions or conflicting required probes as different variants. A probable
|
|
29
|
+
or component match supplies hypotheses only.
|
|
22
30
|
2. Freeze the user-supplied product contract: selected video profile, audio/stream formats, duplex/AEC policy, supported Wi-Fi credential methods, selected onboarding method, transport staging, output path, and mutation boundary. Use `unknown` where the prompt lacks an answer.
|
|
23
31
|
3. Prefer official schematic/BOM and BSP facts. For a PDF schematic, inspect page labels and net names; prefer an exported netlist, pin CSV, or vendor board definition when available.
|
|
24
32
|
4. Cross-check critical pins, clocks, power enables, reset lines, sensor/codec variants, ESP-IDF version, resource ownership, and onboarding behavior across at least two independent artifacts when possible.
|
|
@@ -46,13 +54,16 @@ Read only the documents for the active branch, but read each selected document c
|
|
|
46
54
|
- ESP32 starter or adapter work: `device-sim/device-sim-esp32/README.md`, `device-sim/ESP32_STARTER.md`, the selected SDK package README, and the generated template README.
|
|
47
55
|
- H5 live view or talkback: `device-h5-live.md`.
|
|
48
56
|
- AI intercom: `device-ai.md`.
|
|
57
|
+
- Device-to-device call: `device-call.md` and the applicable API reference.
|
|
58
|
+
- WeChat VoIP: `device-voip.md`, the mini-program integration document, and the
|
|
59
|
+
applicable API reference.
|
|
49
60
|
- H5/AI switching, delayed callbacks, ownership, or timeouts: `device-session-model.md` and `device-session-arbiter.md`.
|
|
50
61
|
- Onboarding, binding, MQTT, token, or identity: `device-integration.md`.
|
|
51
62
|
- Public HTTP field or error changes: `api-reference.md` and `error-response-policy.md`.
|
|
52
63
|
|
|
53
64
|
## Generation and board seam
|
|
54
65
|
|
|
55
|
-
The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, `starter_runtime`, or `starter_tirtc` for each board.
|
|
66
|
+
The runtime-facing `starter_media` interface stays stable. A reusable board integration should implement an internal `BoardMediaAdapterV1`-style adapter owned by `starter_media` rather than editing H5, AI, CALL, VOIP, `starter_runtime`, or `starter_tirtc` for each board.
|
|
56
67
|
|
|
57
68
|
The adapter owns:
|
|
58
69
|
|
|
@@ -62,13 +73,22 @@ The adapter owns:
|
|
|
62
73
|
- DMA buffers, hardware clocks, power, reset, GPIO, refresh/key-frame requests, and realtime task allocation;
|
|
63
74
|
- bounded stop, resource release, and generation-aware flushing.
|
|
64
75
|
|
|
65
|
-
The stable modules own the discovered TiRTC service endpoint, stream IDs,
|
|
76
|
+
The stable modules own the discovered TiRTC service endpoint, stream IDs,
|
|
77
|
+
negotiated/contracted formats, HTTP/MQTT business fields, TiRTC callback copying,
|
|
78
|
+
connection handles, pending calls, monotonic deadlines, session generation, and
|
|
79
|
+
H5/AI/CALL/VOIP sequencing. SDK lifecycle changes such as disconnect run in a
|
|
80
|
+
worker/state-machine context, never directly inside an SDK callback.
|
|
66
81
|
|
|
67
82
|
## Verification loop
|
|
68
83
|
|
|
69
84
|
Use a bounded loop per layer: diagnose one failing invariant, make the smallest correction, and rerun that layer before moving forward. Change one high-risk variable per HIL comparison. Turn reusable invariants into tests or post-link gates. Stop and report when the remaining failure requires unavailable hardware, credentials, a new SDK binary, a public protocol change, or a user choice.
|
|
70
85
|
|
|
71
|
-
For every generated H5/AI project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
|
|
86
|
+
For every generated H5/AI/call/VoIP project, validate `tirtc-runtime-contract.json` and run `install_runtime_gate.py <project>` before the ordinary build. Audio and video projects additionally install their media gates. AI/call/VoIP also require the full-duplex AEC result from the audio gate. A build that bypasses any applicable gate is not `BUILD_VERIFIED`.
|
|
87
|
+
|
|
88
|
+
When CALL or VOIP is requested, the runtime contract must also bind the project
|
|
89
|
+
to the pinned business protocol and unified session arbiter. First prove those
|
|
90
|
+
flows with the Linux C simulator. Simulator success establishes the protocol
|
|
91
|
+
baseline but cannot replace the ESP32 build or HIL levels.
|
|
72
92
|
|
|
73
93
|
Run the assessor once per layer:
|
|
74
94
|
|