hci-atrium 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GUIDE.md +1182 -0
- package/PARITY.md +385 -0
- package/README.md +21 -0
- package/bin/atrium.mjs +35 -0
- package/package.json +8 -2
package/GUIDE.md
ADDED
|
@@ -0,0 +1,1182 @@
|
|
|
1
|
+
# hci-atrium — Student & AI-assistant guide
|
|
2
|
+
|
|
3
|
+
A complete, task-oriented reference for the `hci_atrium` Python SDK — the client for the Atrium
|
|
4
|
+
multimodal asset library. It's written to be pasted into a coding assistant verbatim: every
|
|
5
|
+
signature below is checked against `clients/python/src/hci_atrium/`. If you're new to the SDK,
|
|
6
|
+
start with [`README.md`](README.md) for the 60-second overview; this file is the reference you
|
|
7
|
+
come back to while building.
|
|
8
|
+
|
|
9
|
+
Everything here works with either client:
|
|
10
|
+
|
|
11
|
+
```python
|
|
12
|
+
from hci_atrium import Atrium # sync — the default for scripts
|
|
13
|
+
from hci_atrium import AsyncAtrium # asyncio — same surface, `await` everything
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
The two mirror each other method-for-method. Examples below use the sync client unless a feature
|
|
17
|
+
(rooms driven by `on`/`serve`, the Home Assistant bridge) needs asyncio — those are marked **async**.
|
|
18
|
+
|
|
19
|
+
## Reading this from TypeScript
|
|
20
|
+
|
|
21
|
+
The same SDK is published on npm as `hci-atrium` (`pnpm add hci-atrium`), at parity with the
|
|
22
|
+
Python surface below apart from the deliberate gaps noted at the end of this section. This guide
|
|
23
|
+
stays the reference for both — translate it mechanically:
|
|
24
|
+
|
|
25
|
+
- **Names**: `snake_case` → `camelCase`. `library.search_by_examples` → `library.searchByExamples`,
|
|
26
|
+
`add_direct_model` → `addDirectModel`, `content_kind` → `contentKind`.
|
|
27
|
+
- **Keyword arguments** become one trailing `params` object:
|
|
28
|
+
`client.library.search("rain", modality="audio", limit=10)` →
|
|
29
|
+
`client.library.search("rain", { modality: "audio", limit: 10 })`.
|
|
30
|
+
- **`None` → `null`**, `True`/`False` → `true`/`false`, dicts → plain objects.
|
|
31
|
+
- **Async everywhere.** There is no sync/async split — every call returns a `Promise`, so the
|
|
32
|
+
`AsyncAtrium` examples are the ones to follow (`await` the sync ones too).
|
|
33
|
+
|
|
34
|
+
Deliberate gaps: the full `atrium` CLI (`login`, `whoami`, `search`, …) and the Home Assistant
|
|
35
|
+
bridge are **Python-only**. The npm package ships exactly one command, `atrium guide`, which prints
|
|
36
|
+
this file (`pnpm exec atrium guide`). Everything else is ported.
|
|
37
|
+
|
|
38
|
+
For the per-surface specifics — the parity ledger, the browser-side differences (API keys, capture),
|
|
39
|
+
and the React bindings — see `PARITY.md` and `README.md`, which ship alongside this file in the npm
|
|
40
|
+
package (`clients/typescript/` in the repo).
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## 1. Setup + auth
|
|
45
|
+
|
|
46
|
+
### Install
|
|
47
|
+
|
|
48
|
+
**Fresh project with uv (recommended):**
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
uv init my-study-app
|
|
52
|
+
cd my-study-app
|
|
53
|
+
uv add hci-atrium
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Run scripts with `uv run python main.py` (or `uv run <script.py>`) — uv creates the venv and syncs
|
|
57
|
+
automatically, no activation. The CLI comes along: `uv run atrium --help`. The SDK needs Python
|
|
58
|
+
>=3.10; if the pinned version is older, `uv python pin 3.12`.
|
|
59
|
+
|
|
60
|
+
**Existing project:** `uv add hci-atrium` in a uv-managed project; for a plain venv/pip project:
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
pip install hci-atrium
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
**Throwaway try-out:** `uv run --with hci-atrium python` opens a REPL with the SDK importable, no
|
|
67
|
+
project needed.
|
|
68
|
+
|
|
69
|
+
SDK developers working against a checkout: `uv add --editable path/to/clients/python` overrides the
|
|
70
|
+
published package with your local source.
|
|
71
|
+
|
|
72
|
+
### Connect
|
|
73
|
+
|
|
74
|
+
```python
|
|
75
|
+
from hci_atrium import Atrium
|
|
76
|
+
|
|
77
|
+
client = Atrium() # defaults to the deployed instance (or ATRIUM_BASE_URL)
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
| What | Argument | Environment variable | Default |
|
|
81
|
+
|---|---|---|---|
|
|
82
|
+
| Instance URL | `Atrium(base_url=...)` | `ATRIUM_BASE_URL` | the deployed instance |
|
|
83
|
+
| Credential | `Atrium(api_key="atr_...")` | `ATRIUM_API_KEY` | cached OAuth → browser sign-in |
|
|
84
|
+
|
|
85
|
+
**Zero-config sign-in.** With no `api_key`, the first authenticated call opens a browser
|
|
86
|
+
(Authorization Code + PKCE, out-of-band code). Tokens are cached in `~/.atrium/credentials.json`
|
|
87
|
+
(mode `0600`) and refreshed silently — later runs and later scripts don't prompt again.
|
|
88
|
+
`client.login()` forces this now instead of lazily; `client.logout()` forgets the cached session;
|
|
89
|
+
`client.token()` returns a live bearer token (for pointing another OpenAI-compatible tool at the
|
|
90
|
+
same instance).
|
|
91
|
+
|
|
92
|
+
**API keys (CI / headless / a fixed identity).** A key looks like `atr_...` and is minted in the
|
|
93
|
+
Atrium web app under a project's *Settings → API keys*. Two shapes:
|
|
94
|
+
|
|
95
|
+
- **Project-scoped key** — acts as that project; every call is scoped to it automatically, no
|
|
96
|
+
`project=` needed. The right shape for a study app that only ever writes to one project.
|
|
97
|
+
- **Account-wide key** — acts as your user across every project you can see.
|
|
98
|
+
|
|
99
|
+
Pass it as `Atrium(api_key="atr_...")` or set `ATRIUM_API_KEY` — either skips the browser flow
|
|
100
|
+
entirely.
|
|
101
|
+
|
|
102
|
+
```python
|
|
103
|
+
me = client.me() # GET /v1/users/me
|
|
104
|
+
me.email, me.role
|
|
105
|
+
me.key_project_id # set when a project-scoped key is in use
|
|
106
|
+
me.effective_project_id # the project data ops target (scoped key, else your Personal project)
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Projects & scope
|
|
110
|
+
|
|
111
|
+
A **project** is identified by a `p_...` short id (e.g. `p_4mD2qX`) — a base58, URL-safe id that
|
|
112
|
+
doubles as the API "connect target" for that project. Scope any call to a project (and optionally
|
|
113
|
+
a collection within it) three ways:
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
client.library.search("rain", project="p_4mD2qX") # per call
|
|
117
|
+
client = Atrium(project="p_4mD2qX") # default for a whole client
|
|
118
|
+
photos = client.collection("c_7Bq3Xk") # a reusable scoped view
|
|
119
|
+
photos.library.search("rain") # every call on `photos` is scoped
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`client.project(id)` / `client.collection(id)` return a scoped view that shares the same
|
|
123
|
+
authenticated session (cheap — no re-auth). A project-scoped API key scopes everything
|
|
124
|
+
automatically; you never need `project=` with one.
|
|
125
|
+
|
|
126
|
+
### The CLI
|
|
127
|
+
|
|
128
|
+
Installing the package puts an `atrium` command on `PATH`. From inside the monorepo:
|
|
129
|
+
|
|
130
|
+
```sh
|
|
131
|
+
uv run --project clients/python atrium login
|
|
132
|
+
uv run --project clients/python atrium whoami
|
|
133
|
+
uv run --project clients/python atrium search "calm rainy afternoon" --modality audio
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Once installed into your own project, just `atrium ...` (or `python -m hci_atrium ...`). Full
|
|
137
|
+
command list: `login`, `logout`, `whoami`, `search`, `assets`, `upload`, `enrich`, `chat`,
|
|
138
|
+
`ha-bridge` — every command takes `--base-url`; `search`/`assets`/`upload` also take `--project`/
|
|
139
|
+
`--collection`. Run `atrium <command> --help` for flags.
|
|
140
|
+
|
|
141
|
+
### Errors
|
|
142
|
+
|
|
143
|
+
Every exception the SDK raises derives from `hci_atrium.AtriumError`:
|
|
144
|
+
|
|
145
|
+
```python
|
|
146
|
+
from hci_atrium import AtriumError, AuthError, APIError
|
|
147
|
+
|
|
148
|
+
try:
|
|
149
|
+
client.library.search("rain")
|
|
150
|
+
except APIError as exc: # a non-2xx HTTP response
|
|
151
|
+
exc.status_code, exc.body # int, str (truncated)
|
|
152
|
+
except AuthError: # sign-in / token exchange / refresh failed
|
|
153
|
+
...
|
|
154
|
+
except AtriumError: # catch-all for everything the SDK raises
|
|
155
|
+
...
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## 2. Core concepts, in one screen
|
|
161
|
+
|
|
162
|
+
- **Project** — the top-level scope for data (`p_...` short id). Every asset, room, and session
|
|
163
|
+
lives in exactly one project. A study is usually one project.
|
|
164
|
+
- **Collection** — a node in a project's recursive collection tree (`c_...` short id); `Library` =
|
|
165
|
+
the project's root. Used to scope search/listing to a subtree. The SDK only *scopes to* an
|
|
166
|
+
existing collection id (`client.collection(id)`) — creating/managing the tree is a web-app task,
|
|
167
|
+
not exposed here.
|
|
168
|
+
- **Asset** — one library item. Three independent things it can carry:
|
|
169
|
+
- **bytes** (`sha256` set) — decodable content, served at `asset.content_url`.
|
|
170
|
+
- **link** (`source_url` set) — an external/attribution URL (e.g. a Spotify playlist).
|
|
171
|
+
An asset can have *both* (uploaded audio with a source link) or *neither* (a description-only
|
|
172
|
+
asset, e.g. "the smell of rain").
|
|
173
|
+
- **description** — always present as metadata; it *is* the content when there are no bytes.
|
|
174
|
+
- `asset.facts` — a server-owned JSON bag (analysis results, the `link` discriminator, etc.);
|
|
175
|
+
never write to it yourself.
|
|
176
|
+
- `asset.data` — an app-owned JSON bag the server never touches — your study's own structured
|
|
177
|
+
metadata on the asset.
|
|
178
|
+
- **Rooms vs sessions** — deliberately decoupled:
|
|
179
|
+
- A **room** is pure live transport: `room.send(...)` fans a message to everyone connected,
|
|
180
|
+
nothing is stored. Anyone can join by a short **connect code**, no account needed
|
|
181
|
+
(`join_room(code)`). No replay — if you weren't listening, you missed it.
|
|
182
|
+
- A **session** is a durable, append-only, gapless log (`session.event(...)`). No running/ended
|
|
183
|
+
lifecycle — it's simply the record, always appendable. A session doesn't need a room.
|
|
184
|
+
- To keep what happens live, a host **bridges** the two: `room.record(into=session)`.
|
|
185
|
+
- **Surveys** — a durable *data object* on a session, guarded by a one-time capability token
|
|
186
|
+
(`session.token`); a project-scoped, versioned template drives what it asks.
|
|
187
|
+
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## 3. Library
|
|
191
|
+
|
|
192
|
+
### Search
|
|
193
|
+
|
|
194
|
+
```python
|
|
195
|
+
client.library.search(
|
|
196
|
+
query: str = "",
|
|
197
|
+
*,
|
|
198
|
+
like=None, # rank "more like this" — an Asset, SearchHit, or id
|
|
199
|
+
like_by: str | None = None, # which of the liked asset's signals seeds it (default: native)
|
|
200
|
+
modality: str | None = None, # desired result kind, e.g. "audio", "image"
|
|
201
|
+
space: str | None = None, # force a specific embedding space (id or name)
|
|
202
|
+
limit: int = 20,
|
|
203
|
+
tags: list[str] | None = None, # results must carry ALL of these tags
|
|
204
|
+
any_tags: list[str] | None = None, # …and at least one of these
|
|
205
|
+
exclude_tags: list[str] | None = None, # …and none of these
|
|
206
|
+
provider: str | list[str] | None = None, # who provides the content — see below
|
|
207
|
+
filters: Filters | None = None, # the long tail: content_kind, statuses, ranges, exclude_ids, …
|
|
208
|
+
image=None, audio=None, video=None, # search BY a media file instead of text (cross-modal)
|
|
209
|
+
vector: list[float] | None = None, # rank by a raw vector (pass space= with it)
|
|
210
|
+
include_vectors: bool | str = False, # inline one vector per hit onto hit.vector
|
|
211
|
+
project: str | None = None,
|
|
212
|
+
collection: str | None = None,
|
|
213
|
+
) -> list[SearchHit]
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
```python
|
|
217
|
+
for hit in client.library.search("calm rainy afternoon", modality="audio", limit=5):
|
|
218
|
+
print(round(hit.score, 3), hit.asset.title or hit.asset.filename)
|
|
219
|
+
print(" ", hit.asset.content_url)
|
|
220
|
+
|
|
221
|
+
playlists = client.library.search("lofi beats", modality="audio", provider="spotify:playlist")
|
|
222
|
+
|
|
223
|
+
# "More like this" — pass an asset (or a SearchHit straight from a previous search) as like=
|
|
224
|
+
similar = client.library.search(like=hit, modality="audio")
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
- Pass **exactly one** query input: a text `query`, an example asset (`like=`), a media source
|
|
228
|
+
(`image=`/`audio=`/`video=`), or a raw `vector=`. `modality` means "what kind of result I want":
|
|
229
|
+
results OF that kind, **provided by the library's own storage and matched by their native
|
|
230
|
+
content** (the server's media-intent defaults — text-about-content and byteless links don't
|
|
231
|
+
compete unless you opt in). For a text query it also picks that modality's space; for a media
|
|
232
|
+
query the query embeds in the cross-modal joint space.
|
|
233
|
+
- `like=` ranks "more like this": an `Asset`, a `SearchHit`, or a bare id. `modality` still filters
|
|
234
|
+
the results (cross-modal: an image can fetch its fitting audio, ranked in the cross-modal default
|
|
235
|
+
space); the source asset must be embedded in the ranking `space`. `like_by=` picks which of the
|
|
236
|
+
asset's stored signals seeds the query (`"native"`, `"manual"`, `"bridge:<modality>"`,
|
|
237
|
+
`"view:N"`/`"seg:<id>"`) — default is the native-first auto pick.
|
|
238
|
+
- `provider` selects where results come from: `"library"` (stored bytes — the implied default
|
|
239
|
+
for a media `modality`), a streaming provider (`"spotify"`, entity-qualified
|
|
240
|
+
`"spotify:playlist"` / `"spotify:track"` / `"spotify:album"`), `"link"` (anything with a URL,
|
|
241
|
+
recognized or not), `"none"` (neither bytes nor URL — e.g. a fileless asset you hand-tagged
|
|
242
|
+
"audio"), `"any"` (everything), or a list to mix. Non-library classes match by their
|
|
243
|
+
description text — mixing classes ranks text-matches against content-matches, which is only
|
|
244
|
+
score-comparable on a fully calibrated space; and a provider link with no description holds no
|
|
245
|
+
vectors at all, so ranked search can't surface it (keyword search can). Unknown values → 422
|
|
246
|
+
listing the vocabulary.
|
|
247
|
+
- `hit.asset` is an `Asset`; `hit.score`/`hit.raw_score`/`hit.calibrated`/`hit.matched_by`/
|
|
248
|
+
`hit.match_label` describe the match.
|
|
249
|
+
- `include_vectors=True` inlines onto `hit.vector` the one vector that scored each hit (its identity
|
|
250
|
+
is `hit.matched_by`); a space id/name instead returns that space's vector per hit, which can
|
|
251
|
+
differ from the one that ranked it. Off by default — vectors are large and never leave Postgres in
|
|
252
|
+
the search path; `hit.vector` is `None` unless you asked.
|
|
253
|
+
- `filters=Filters(...)` carries full library-filter power: `content_kind` (the immutable kind
|
|
254
|
+
sniffed from the file's bytes — `"image"`, `"audio"`, …, unlike `modality` which matches the
|
|
255
|
+
editable label, so it finds a JPEG even after it's retagged `"product"`), `statuses`, numeric
|
|
256
|
+
fact `ranges` (`{"bpm": (90, 120), "duration_s": (None, 30)}`), `uploader`, extra kind/label
|
|
257
|
+
facets (`all_kinds`/`any_kinds`/`none_kinds`/`all_labels`/`any_labels`/`none_labels`),
|
|
258
|
+
`max_seconds`, `match_by` (the Match-by override: an explicit list — e.g. `["native", "manual"]`,
|
|
259
|
+
or `[]` for every signal — applies globally, replacing the per-class default that comes with a
|
|
260
|
+
`provider` filter), `data_match={"kind": "field"}` (a JSONB containment filter on the app-owned
|
|
261
|
+
`asset.data`, `data @> data_match` — retrieval of what you wrote, not derivation), and
|
|
262
|
+
`exclude_ids` (drop these asset ids — dedup for a prefetch loop; capped at 512).
|
|
263
|
+
|
|
264
|
+
Other `Library` methods, same scoping conventions (`project=`/`collection=`, `space=`):
|
|
265
|
+
|
|
266
|
+
| Method | Purpose |
|
|
267
|
+
|---|---|
|
|
268
|
+
| `library.add(file=None, *, modality=, description=, collection=)` | add an asset to the library — a file, or (omit `file`) a fileless asset whose `description` *is* its content. See [Add to the library](#add-to-the-library) |
|
|
269
|
+
| `library.compare(a, b, *, space=, breakdown=False)` | pairwise similarity of two operands — `Asset`, `str`, `list[float]`, or `by.file(...)`/`by.asset(id)`/`by.concept(name_or_id, collection=)`; `breakdown=True` adds the per-feature terms (see below) |
|
|
270
|
+
| `library.search_by_examples(positives, negatives=, *, method="cav"/"centroid"/"probe", match_by=, filters=, ...)` | "mood board" — rank by a set of example assets (default `method="cav"`). `match_by=["native"]` picks the signals by class — same meaning as `search`'s `filters.match_by`. `match_by={"native": 1.0, "manual": 0.3}` also weights them (only a set query can: search picks one best signal, this blends every one). An allowlist either way; no space ids. `probe` fits its own weights, so pass `method="cav"` to set them by hand. `filters=` narrows by kind/label facets + `content_kind`; a Filters field with no set-query slot (`statuses`/`ranges`/`uploader`/`max_seconds`/`data_match`/`exclude_ids`) raises |
|
|
271
|
+
| `library.enrich(query, modality, *, tags=, total=, preview=, wait=)` | pull external candidates (Freesound/Openverse/…) into the library |
|
|
272
|
+
| `library.vectors(ids, *, space, match_by=None, strict=False)` | bulk-read stored vectors for many assets — see [Reading an asset back](#reading-an-asset-back) |
|
|
273
|
+
| `library.spaces()` | list the searchable embedding spaces |
|
|
274
|
+
| `library.default_space(modality)` | the deployment's default space for that modality (a `spaces()` dict, or `None`) |
|
|
275
|
+
|
|
276
|
+
`by.concept(...)` can only be one side of a `compare` call — it scores the *other* operand's vector
|
|
277
|
+
against that tag's materialized concept snapshot (calibrated to the concept's own boundary, not the
|
|
278
|
+
usual search calibration), 404 if the concept doesn't resolve, 422 if it has none. `breakdown=True`
|
|
279
|
+
then fills `result.features` with the per-`(space, matched_by)` terms behind that score (it stays
|
|
280
|
+
`None` for a plain asset/vector compare, which has no per-feature structure):
|
|
281
|
+
|
|
282
|
+
```python
|
|
283
|
+
result = client.library.compare(asset, by.concept("cozy"), breakdown=True)
|
|
284
|
+
for f in result.features or []:
|
|
285
|
+
print(f.space, f.matched_by, f.weight, f.raw_score, f.score) # which signals carried the match
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
Read those as **raw per-feature terms, not additive contributions** — they cover only the scored
|
|
289
|
+
space's features and do not sum to `result.score`.
|
|
290
|
+
|
|
291
|
+
`default_space` keeps deployment space ids out of your code — the deployment declares its defaults,
|
|
292
|
+
the app just asks:
|
|
293
|
+
|
|
294
|
+
```python
|
|
295
|
+
space = client.library.default_space("audio") # e.g. {"id": "clap", "name": "CLAP", ...} or None
|
|
296
|
+
rows = client.library.vectors(ids, space=space["id"])
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
### Asset views
|
|
300
|
+
|
|
301
|
+
```python
|
|
302
|
+
asset = client.assets.get("a_123")
|
|
303
|
+
|
|
304
|
+
asset.capabilities # set[str] ⊆ {"bytes", "link"} — independent flags, both can hold
|
|
305
|
+
asset.link # AssetLink(provider, entity, uri) | None — present iff "link" in capabilities
|
|
306
|
+
asset.playback # PlaybackIntent — ALWAYS present
|
|
307
|
+
asset.content_url # public URL for the bytes (fingerprinted ?v=sha when known)
|
|
308
|
+
asset.thumbnail_url # public URL for an image/video poster (404 if none)
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
`asset.playback` normalizes the asset into one play intent, mirroring the room `play` signal's
|
|
312
|
+
`{source, ref, mode?}` shape 1:1 — a host can forward it as-is:
|
|
313
|
+
|
|
314
|
+
```python
|
|
315
|
+
intent = asset.playback # PlaybackIntent(source=..., ref=..., mode=...)
|
|
316
|
+
intent.as_signal() # {"source": ..., "ref": ..., "mode": ...} — mode omitted if unset
|
|
317
|
+
room.send("play", {"playback": intent.as_signal()})
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
Priority: `source="bytes"` whenever the asset has decodable content (even if it also has a link);
|
|
321
|
+
else the recognized link provider's name (currently only `"spotify"`) with `ref` a
|
|
322
|
+
`provider:entity:id` URI, `mode="shuffle"` for a playlist; else `source="link"` with the raw URL
|
|
323
|
+
for an unrecognized provider; else `source="description"` with `ref` = the asset id (nothing to
|
|
324
|
+
fetch, but a host can still look it up).
|
|
325
|
+
|
|
326
|
+
<a name="reading-an-asset-back"></a>
|
|
327
|
+
### Reading an asset back: vectors, segments, index
|
|
328
|
+
|
|
329
|
+
What the library stored *for* an asset — its embeddings, its cut sound events, and the processing
|
|
330
|
+
state of each signal. All three are bound on the asset itself:
|
|
331
|
+
|
|
332
|
+
```python
|
|
333
|
+
asset = client.assets.get("a_123")
|
|
334
|
+
|
|
335
|
+
for v in asset.vectors(): # every stored vector; list[AssetVector]
|
|
336
|
+
print(v.space, v.matched_by, len(v.vector)) # matched_by: "native" / "manual" / "bridge:text" / "seg:<id>"
|
|
337
|
+
asset.vectors(space="CLAP", match_by="native") # narrow: space (id or name, one or a list) + signal
|
|
338
|
+
|
|
339
|
+
for seg in asset.segments(): # the sound events cut out of an audio asset
|
|
340
|
+
print(seg.ordinal, seg.start_s, seg.end_s, seg.content_url) # just that slice's pre-cut bytes
|
|
341
|
+
|
|
342
|
+
for entry in asset.index(): # one row per stored signal — the processing state
|
|
343
|
+
print(entry.space, entry.matched_by, entry.active, entry.status, entry.error)
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
`client.assets.vectors(asset, space=, match_by=)` / `.segments(asset)` / `.index(asset)` are the
|
|
347
|
+
unbound forms — each takes an `Asset`, a `SearchHit`, or a bare id. Vectors and segments are public
|
|
348
|
+
by id (like `content_url`); the index map is authenticated. On the async client the bound methods are
|
|
349
|
+
awaitable (`await asset.vectors()`).
|
|
350
|
+
|
|
351
|
+
For many assets at once, `library.vectors(ids, *, space, match_by=None, strict=False)` is the bulk
|
|
352
|
+
read — `space` is **required** here (one id/name, or a list):
|
|
353
|
+
|
|
354
|
+
```python
|
|
355
|
+
rows = client.library.vectors([h.asset for h in hits], space="CLAP")
|
|
356
|
+
{r.asset: r.vector for r in rows} # each AssetVector carries its owning asset id
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
With `match_by=None` the signal is resolved per asset, so one call can return **mixed** `matched_by`
|
|
360
|
+
values — read `r.matched_by` to see which each row is. Ids that don't exist or aren't visible to your
|
|
361
|
+
key land in the server's `missing` list: `strict=False` (default) omits them, `strict=True` raises
|
|
362
|
+
listing them.
|
|
363
|
+
|
|
364
|
+
### Browse, add to the library
|
|
365
|
+
|
|
366
|
+
```python
|
|
367
|
+
page = client.assets.list(
|
|
368
|
+
modality="audio", # editable label
|
|
369
|
+
content_kind=None, # or ["image", "audio"] — the bytes-sniffed kind instead
|
|
370
|
+
limit=48, cursor=None, project=None, collection=None,
|
|
371
|
+
)
|
|
372
|
+
page.items # list[Asset]
|
|
373
|
+
page.next_cursor # pass back as cursor= for the next page, or None
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
<a name="add-to-the-library"></a>
|
|
377
|
+
`library.add(...)` is the front door for putting an asset into the library — with a file, or fileless:
|
|
378
|
+
|
|
379
|
+
```python
|
|
380
|
+
asset = client.library.add(
|
|
381
|
+
"path/to/rain.wav", # path, Path, bytes, open file, or an http(s) URL
|
|
382
|
+
modality=None, # override the modality *label* (a search/display facet;
|
|
383
|
+
# embedding follows the bytes, never this)
|
|
384
|
+
description="rain on a tin roof",
|
|
385
|
+
collection=None, # home it in a collection, else the default scope
|
|
386
|
+
)
|
|
387
|
+
|
|
388
|
+
# Fileless — no file, the description *is* the content (the web UI's "Add other"):
|
|
389
|
+
note = client.library.add(description="a warm, relaxed evening mood")
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
A positional string is always a file path/URL, never a description — pass `description=` as a
|
|
393
|
+
keyword; a fileless add requires it. `client.assets.upload(file, ...)` is the same call for a file.
|
|
394
|
+
|
|
395
|
+
Every media argument across the SDK (`image=`, `audio=`, `video=`, `library.add(file)`) accepts a
|
|
396
|
+
path, `Path`, raw `bytes`, an open binary file, or an `http(s)` URL — normalized by
|
|
397
|
+
`hci_atrium._files`.
|
|
398
|
+
|
|
399
|
+
### Bulk import
|
|
400
|
+
|
|
401
|
+
For importing many assets at once from a manifest (JSON/CSV) instead of one-by-one uploads,
|
|
402
|
+
`POST /v1/library/import` is a raw multipart endpoint, not (yet) wrapped by the SDK; call it with
|
|
403
|
+
`httpx` directly or `client._request("POST", "/v1/library/import", ...)`.
|
|
404
|
+
|
|
405
|
+
### Searching external sources
|
|
406
|
+
|
|
407
|
+
Search external providers (Freesound, Openverse, …) for media the library doesn't have yet, then
|
|
408
|
+
import the ones you want. The search is **read-only** — nothing is imported until you call
|
|
409
|
+
`import_`, and every result carries the licence + attribution you need to honour.
|
|
410
|
+
|
|
411
|
+
```python
|
|
412
|
+
sources = client.sources.list() # which providers are registered
|
|
413
|
+
|
|
414
|
+
result = client.sources.search(
|
|
415
|
+
"rain on a tin roof",
|
|
416
|
+
providers=None, # None = every source; or "freesound" / ["freesound", …]
|
|
417
|
+
k=12, # per-provider cap
|
|
418
|
+
modality="audio", # narrow to sources serving this kind
|
|
419
|
+
)
|
|
420
|
+
|
|
421
|
+
for c in result.results: # provider-tagged Candidates (not yet imported)
|
|
422
|
+
print(c.provider, c.title, c.license, c.preview_url) # c.duration / c.size / c.content_type too
|
|
423
|
+
|
|
424
|
+
for skip in result.skipped: # a source that couldn't answer, and why
|
|
425
|
+
print("skipped", skip.provider, "—", skip.reason) # e.g. missing credentials
|
|
426
|
+
|
|
427
|
+
asset = client.sources.import_(result.results[0]) # pull that candidate into the library
|
|
428
|
+
# or name it yourself: client.sources.import_(source_id="freesound", external_id="12345")
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
- `search` fans out over every registered source by default; a source that's missing credentials,
|
|
432
|
+
is down, or doesn't serve the ask lands in `result.skipped` (with a reason) instead of failing the
|
|
433
|
+
whole call — so you always get the sources that *did* answer.
|
|
434
|
+
- Each `Candidate` is external-only until imported: `provider` + `external_id` are what
|
|
435
|
+
`import_(candidate)` re-resolves to fetch the bytes. `import_` returns the library `Asset`;
|
|
436
|
+
re-importing the same candidate is a no-op that returns the existing one.
|
|
437
|
+
- `import_` takes an optional `modality=` / `description=` to override what the source reports.
|
|
438
|
+
|
|
439
|
+
### Chat & agents
|
|
440
|
+
|
|
441
|
+
```python
|
|
442
|
+
reply = client.chat("what's a good rainy-day soundscape?", model="gpt-4o-mini", temperature=0.7)
|
|
443
|
+
print(reply) # a str with .model / .raw
|
|
444
|
+
for chunk in client.chat("hi", stream=True):
|
|
445
|
+
print(chunk, end="")
|
|
446
|
+
|
|
447
|
+
reply = client.agent("find a calm track and tell me its title", tools=[my_search_tool])
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
`chat(messages, *, model=, stream=False, strict_params=False, **params)` posts to
|
|
451
|
+
`/v1/chat/completions`; `**params` are extra OpenAI parameters (`temperature`, `max_tokens`,
|
|
452
|
+
`tools`, …) forwarded verbatim. `agent(prompt, *, tools, model=, max_turns=6, **params)` runs a
|
|
453
|
+
tool-using loop, forwarding `**params` to every `chat` call it makes.
|
|
454
|
+
|
|
455
|
+
> **Provider-unsupported params are silently dropped by default.** Atrium routes chat through
|
|
456
|
+
> LiteLLM, which drops a sampling param a model doesn't support (e.g. a non-default `temperature`
|
|
457
|
+
> on an OpenAI "gpt-5"-family model) rather than erroring — matched by **model-name pattern**, so a
|
|
458
|
+
> self-hosted model named like a known family inherits that family's quirks. Pass
|
|
459
|
+
> `strict_params=True` to error instead, with the provider's own message. This only applies to
|
|
460
|
+
> models routed through Atrium/LiteLLM — a model registered via `add_direct_model` bypasses all of
|
|
461
|
+
> it: its request body goes verbatim to the endpoint.
|
|
462
|
+
|
|
463
|
+
---
|
|
464
|
+
|
|
465
|
+
## 4. Rooms & sessions
|
|
466
|
+
|
|
467
|
+
**Two kinds of traffic flow through a room.** *Messages* are the named things on the wire — react to
|
|
468
|
+
them by name with `room.on("play")`, `room.on("presence.join")`, `room.on("mood.switch")`; the
|
|
469
|
+
vocabulary is open, anyone can `room.send` any type. *Audio* is the live sound in the room — process
|
|
470
|
+
it with `async for` over the `audio_*` family (`audio_vectors` / `audio_scores` / `audio_transcripts`
|
|
471
|
+
/ `audio_clips`), streams the SDK assembles for you from the room's raw `audio.snippet` windows, with
|
|
472
|
+
any refinement (embedding, transcription) running on *your* credentials. Transcripts show the duality
|
|
473
|
+
directly: `room.on("transcript.final")` **consumes** a caption someone else is already broadcasting
|
|
474
|
+
(free — you read the wire), while `room.audio_transcripts()` **computes** your own from the audio.
|
|
475
|
+
|
|
476
|
+
### Sync: host a room
|
|
477
|
+
|
|
478
|
+
```python
|
|
479
|
+
from hci_atrium import Atrium
|
|
480
|
+
|
|
481
|
+
client = Atrium()
|
|
482
|
+
room = client.rooms.open(name="living room") # named → stable connect code across runs
|
|
483
|
+
print(room.connect_code)
|
|
484
|
+
|
|
485
|
+
with client, room: # on exit: delete anonymous / leave a named room standing
|
|
486
|
+
for event in room.receive(): # blocks; SSE stream
|
|
487
|
+
if event.source != "host":
|
|
488
|
+
print(event.type, event.payload)
|
|
489
|
+
```
|
|
490
|
+
|
|
491
|
+
`rooms.open(*, name=None, project=None, embedding_space=None)` — give it a `name` to reuse the same
|
|
492
|
+
room (and connect code) across runs; the `name → room id` mapping is cached in
|
|
493
|
+
`~/.atrium/rooms.json`. Anonymous (`name=None`) opens a fresh room every call. `rooms.get(id)` /
|
|
494
|
+
`rooms.list()` fetch existing rooms.
|
|
495
|
+
|
|
496
|
+
`room.send(type, payload=None, *, to=None, include_self=False, source=None, ephemeral=False,
|
|
497
|
+
ack_id=None, reply_to=None)` — fire-and-forget, never stored. `to` (a connection id, an
|
|
498
|
+
`identity`, or a list) unicasts/multicasts; default fan-out is everyone else. `ephemeral=True`
|
|
499
|
+
marks throwaway noise (beats, cursors) that a recorder skips.
|
|
500
|
+
|
|
501
|
+
`room.receive(*, identity=None, announce=None)` — blocking SSE iterator of `Event`s; sets
|
|
502
|
+
`room.connection_id` from the first `connection.ready` frame. Reconnect yourself on drop (rooms
|
|
503
|
+
have no replay).
|
|
504
|
+
|
|
505
|
+
### Join without an account
|
|
506
|
+
|
|
507
|
+
```python
|
|
508
|
+
from hci_atrium import join_room
|
|
509
|
+
|
|
510
|
+
with join_room("ABC123") as room: # connect code, no account, no full client
|
|
511
|
+
room.send("message", {"text": "hi"})
|
|
512
|
+
for event in room.receive():
|
|
513
|
+
...
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
### Sessions (durable log)
|
|
517
|
+
|
|
518
|
+
```python
|
|
519
|
+
session = client.sessions.start(name="P07 · pilot", participant_ref="P07")
|
|
520
|
+
session.event("note", {"text": "participant arrived"})
|
|
521
|
+
|
|
522
|
+
session = client.sessions.resume(session_id) # continue an existing session by id — raises if unknown
|
|
523
|
+
print(session.last_seq) # last_seq reflects the session as of the fetch — re-`resume()` to refresh it
|
|
524
|
+
```
|
|
525
|
+
|
|
526
|
+
### Recording a room into a session
|
|
527
|
+
|
|
528
|
+
```python
|
|
529
|
+
session = client.sessions.start(name="studio take")
|
|
530
|
+
room = client.rooms.open(name="studio")
|
|
531
|
+
|
|
532
|
+
with room.record(into=session): # starts a background capture thread immediately
|
|
533
|
+
room.send("note", {"text": "session started"})
|
|
534
|
+
room.send("beat", {"bpm": 120}, ephemeral=True) # skipped — ephemeral
|
|
535
|
+
...
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
`ephemeral=True` events, presence, and non-`signal` frames are skipped; everything else is
|
|
539
|
+
appended with `via={"room_id", "room_name", "from", "to"}`. A dropped stream reconnects with
|
|
540
|
+
backoff (gaps are lost — rooms have no replay); check `rec.alive` / `rec.error`.
|
|
541
|
+
|
|
542
|
+
### Reading sessions back (analysis)
|
|
543
|
+
|
|
544
|
+
The write side (`sessions.start`/`resume`, `session.event`) has a read-model twin for analysis:
|
|
545
|
+
|
|
546
|
+
```python
|
|
547
|
+
for info in client.sessions.list(phase="pilot"): # newest first; phase: a string or a list
|
|
548
|
+
print(info.id, info.participant_ref, info.condition, info.last_seq)
|
|
549
|
+
|
|
550
|
+
info = client.sessions.get(session_id) # one session — has_transcript populated here
|
|
551
|
+
for row in client.sessions.events(session_id, after=0, limit=500):
|
|
552
|
+
print(row.seq, row.type, row.payload, row.source, row.via)
|
|
553
|
+
|
|
554
|
+
client.sessions.export(session_id, "csv", path="out/") # "jsonl"|"csv"|"transcript"|"bundle"
|
|
555
|
+
client.sessions.export_bundle([id_a, id_b], path="out/study.zip") # several sessions, one zip
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
- `sessions.list(*, project=None, phase=None)` returns `SessionInfo` rows for your default project
|
|
559
|
+
(a scoped key's project, else Personal); `""` as a phase matches sessions with no phase set.
|
|
560
|
+
`has_transcript` is `None` in list results — only `sessions.get(id)` populates it. There is **no
|
|
561
|
+
participant filter server-side**: filter the rows on `participant_ref` yourself.
|
|
562
|
+
- `sessions.get(id)` is the read model; `sessions.resume(id)` hits the same endpoint but returns the
|
|
563
|
+
live `Session` handle for continued logging.
|
|
564
|
+
- `sessions.events(id, *, after=None, limit=None)` pages the durable log as flat, pandas-friendly
|
|
565
|
+
`SessionEvent` rows. `after` is a **sequence number**, not a timestamp (there is no time filter
|
|
566
|
+
server-side); `limit` is 1..500 (server default 100).
|
|
567
|
+
- `export(id, format="jsonl", *, path=None)` returns the raw `bytes` — or, with `path=` (a file, or a
|
|
568
|
+
directory to derive the filename in), writes them and returns the written `Path`. Each format is a
|
|
569
|
+
distinct server route, not a `?format=` param; `"transcript"` 404s when the session has none.
|
|
570
|
+
- `export_bundle(ids, *, path=None)` zips several sessions. `ids` is required (no "all sessions"
|
|
571
|
+
mode) and capped at 100 **distinct** ids — more raises `ValueError` rather than truncating.
|
|
572
|
+
|
|
573
|
+
### Mirror your app's own activity: `room.trace()`
|
|
574
|
+
|
|
575
|
+
One call mirrors what the SDK is doing on this client into the room as `trace.*` signals — so an
|
|
576
|
+
observation dashboard, or any code-joined peer, can watch the app think:
|
|
577
|
+
|
|
578
|
+
```python
|
|
579
|
+
with room.trace(): # or: t = room.trace(); …; t.stop()
|
|
580
|
+
client.library.search("calm rain", modality="audio") # -> trace.library.search
|
|
581
|
+
room.play(hit, layer="music", mode="queue") # -> trace.play
|
|
582
|
+
```
|
|
583
|
+
|
|
584
|
+
- A traced call sends `trace.<label>` (`trace.library.search` / `trace.chat` / `trace.play`) carrying
|
|
585
|
+
a one-line summary plus a capped detail; a recorder starting/stopping sends `trace.state.<label>`.
|
|
586
|
+
- **Default off** — it broadcasts app internals to anyone holding the connect code. Tracing stops on
|
|
587
|
+
`t.stop()` or when the room's `with` exits, and a trace failure never breaks the traced call.
|
|
588
|
+
- Ephemerality is per-signal: `trace.library.*` / `trace.chat` are **durable** (they happen outside
|
|
589
|
+
the room, so the trace is their only record — a recorder captures them into the session), while
|
|
590
|
+
`trace.play` and `trace.state.*` are ephemeral (the underlying `play` already rides the room).
|
|
591
|
+
- Async: `room.trace()` is still a plain call, but scope it with `async with room.trace(): …`.
|
|
592
|
+
|
|
593
|
+
### Async: `on`/`serve`, presence, targeted request/reply
|
|
594
|
+
|
|
595
|
+
The async room adds an event-dispatch style on top of the same transport — register a handler per
|
|
596
|
+
signal type, then `serve()` to run until cancelled:
|
|
597
|
+
|
|
598
|
+
```python
|
|
599
|
+
import asyncio
|
|
600
|
+
from hci_atrium import AsyncAtrium
|
|
601
|
+
|
|
602
|
+
async def main() -> None:
|
|
603
|
+
async with AsyncAtrium() as client:
|
|
604
|
+
room = await client.rooms.open(name="lab")
|
|
605
|
+
print(room.connect_code)
|
|
606
|
+
|
|
607
|
+
@room.on("presence.join")
|
|
608
|
+
def on_join(event):
|
|
609
|
+
print("joined:", event.payload.get("identity"))
|
|
610
|
+
|
|
611
|
+
@room.on("ping")
|
|
612
|
+
async def on_ping(event):
|
|
613
|
+
return {"pong": True} # only sent back if the ping was a `request(...)`
|
|
614
|
+
|
|
615
|
+
await room.connect(identity="host")
|
|
616
|
+
await room.serve() # blocks; reconnects transient drops with backoff
|
|
617
|
+
|
|
618
|
+
asyncio.run(main())
|
|
619
|
+
```
|
|
620
|
+
|
|
621
|
+
- `await room.connect(*, identity=None, announce=None)` joins live and starts the dispatcher,
|
|
622
|
+
returning the connection id. `presence.join`/`presence.leave` arrive as ordinary signals — just
|
|
623
|
+
register handlers for them.
|
|
624
|
+
- `await room.request(type, payload=None, *, to, timeout=None)` sends to one peer and awaits its
|
|
625
|
+
single reply (the peer's handler return value) — `to` is required. Raises `TimeoutError` on no
|
|
626
|
+
reply within `timeout` (default: the room's request timeout).
|
|
627
|
+
- A handler's return value becomes the auto-reply when the *incoming* signal itself was a
|
|
628
|
+
`request(...)` (carried an `ack_id`) — a responder never touches correlation plumbing.
|
|
629
|
+
- Don't mix `on`/`serve` with `receive()` on the same room — each opens its own stream (a second
|
|
630
|
+
connection, a second presence join).
|
|
631
|
+
|
|
632
|
+
### Live audio vectors (async)
|
|
633
|
+
|
|
634
|
+
If a Player in the room has its **"Share audio clips"** toggle on (default off — it shares actual
|
|
635
|
+
audio, so consent lives on the device making sound), you can get a live semantic embedding of what
|
|
636
|
+
the room currently sounds like — ~10 s windows, embedded in the deployment's default audio search
|
|
637
|
+
space, so each vector is directly comparable against library-derived vectors:
|
|
638
|
+
|
|
639
|
+
```python
|
|
640
|
+
async for v in room.audio_vectors(): # zero-args: default audio space, that space's model
|
|
641
|
+
score = cosine(v.vector, tense_anchor) # v: AudioVector(vector, space, sequence,
|
|
642
|
+
await room.send("mood.update", {"score": score}) # window_seconds, timestamp)
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
- Subscribing **is** the demand signal: while no consumer is subscribed, an armed Player captures
|
|
646
|
+
nothing. Break out of the loop to end the subscription.
|
|
647
|
+
- **The loop blocks until a Player shares audio.** If no participant is already announcing the
|
|
648
|
+
snippet-producer role at subscribe time, one warning is logged and the loop keeps waiting (no
|
|
649
|
+
timeout) — nothing yields until someone turns their Player's "Share audio clips" toggle on.
|
|
650
|
+
- `space=` pins a different space; `model=` overrides the embedder (rarely needed — by default the
|
|
651
|
+
space's own model is resolved once at subscribe). Embedding runs on *your* credentials/compute.
|
|
652
|
+
- Vectors are ephemeral room signals, never stored. Record the room into a session if a study
|
|
653
|
+
needs a durable trace.
|
|
654
|
+
- `relay=True` also relays each vector into the room as an `audio.vector` signal
|
|
655
|
+
(`ephemeral=True`, recorder-skipped) — so a loginless, code-joined peer (e.g. a Unity scene) can
|
|
656
|
+
read the room's live mood vector without credentials of its own
|
|
657
|
+
(`docs/reference/rooms-sessions-protocol.md`).
|
|
658
|
+
|
|
659
|
+
`room.audio_scores(anchors, *, space=None, model=None, top_n=1, relay=False)` fuses
|
|
660
|
+
`audio_vectors()` with per-anchor `library.compare` scoring — a live "which anchor does the room
|
|
661
|
+
sound like right now" signal, no manual cosine math:
|
|
662
|
+
|
|
663
|
+
```python
|
|
664
|
+
async for frame in room.audio_scores({"tense": "a tense, anxious atmosphere", "calm": "calm and peaceful"}):
|
|
665
|
+
print(frame.scores) # {"tense": 0.71, "calm": 0.22} — calibrated per-anchor compare scores
|
|
666
|
+
print(frame.mix) # scores normalized to sum 1 (uniform when every score is 0 — gate on
|
|
667
|
+
# max(frame.scores) to tell "no match" from "anchors tied")
|
|
668
|
+
```
|
|
669
|
+
|
|
670
|
+
- `anchors` maps a name to anything the `by.*` grammar accepts (bare `str`/`Asset`/`list[float]`,
|
|
671
|
+
`by.file(...)`/`by.asset(...)`/`by.concept(...)`) — **or a list** of those, a few-shot category
|
|
672
|
+
(e.g. `"tense": ["a tense, anxious atmosphere", "a horror movie soundtrack"]`) scored as the mean
|
|
673
|
+
of its **top-`top_n`** members (default `top_n=1` — the nearest member wins, so spread-out
|
|
674
|
+
examples don't blur into an average; pass a larger `top_n` to widen back toward the plain mean —
|
|
675
|
+
`top_n >= len(members)` averages every member).
|
|
676
|
+
- Every window scores its anchors concurrently (`asyncio.gather`); a handful of `compare` calls per
|
|
677
|
+
~10 s window is cheap.
|
|
678
|
+
- Every anchor is validated once, before waiting on any frame: a typo'd/unresolvable `by.concept(…)`
|
|
679
|
+
anchor raises `AtriumError` naming it immediately, rather than only failing mid-stream.
|
|
680
|
+
- `relay=True` forwards straight to `audio_vectors` — it relays each window's raw
|
|
681
|
+
`audio.vector` signal (see above), **not** the computed `scores`/`mix` (peers compare via the
|
|
682
|
+
embed broker instead).
|
|
683
|
+
- No smoothing: every `ScoredAudio` frame is independent. Ease/debounce the stream yourself —
|
|
684
|
+
`hci_atrium.reactive.Decider` is a small, framework-free helper for turning a live score dict into
|
|
685
|
+
a stable "current winner":
|
|
686
|
+
|
|
687
|
+
```python
|
|
688
|
+
from hci_atrium import Decider
|
|
689
|
+
|
|
690
|
+
decider = Decider(margin=0.1, dwell=2.0) # a new winner must beat the current one by 0.1,
|
|
691
|
+
# and stay switched for at least 2s
|
|
692
|
+
async for frame in room.audio_scores(anchors):
|
|
693
|
+
winner = decider.update(frame.scores) # the new winner's name, or None if unchanged
|
|
694
|
+
if winner is not None:
|
|
695
|
+
await room.send("mood.switch", {"to": winner})
|
|
696
|
+
```
|
|
697
|
+
|
|
698
|
+
`decider.current` holds the standing winner between updates; with two anchors this is a plain
|
|
699
|
+
binary hysteresis switch.
|
|
700
|
+
|
|
701
|
+
### Live transcripts (async)
|
|
702
|
+
|
|
703
|
+
`room.audio_transcripts()` is the speech-to-text counterpart of `audio_vectors()` — it consumes the
|
|
704
|
+
same Player-shared `audio.snippet` stream, but transcribes the audio into **live captions** instead
|
|
705
|
+
of embedding it, yielding a `Transcript` (a `str` subclass) per **finalized sentence**:
|
|
706
|
+
|
|
707
|
+
```python
|
|
708
|
+
async for t in room.audio_transcripts(): # zero-args: deployment default model
|
|
709
|
+
print(str(t)) # a settled sentence; t.raw has start/end
|
|
710
|
+
```
|
|
711
|
+
|
|
712
|
+
- Same demand gate as `audio_vectors`: subscribing announces the `audio-snippet-consumer` role, so
|
|
713
|
+
an armed Player only captures while a consumer is subscribed; break out of the loop to end it (the
|
|
714
|
+
stream tears down deterministically on `break`). The same one-time "no Player is sharing audio"
|
|
715
|
+
warning fires if nobody is producing yet.
|
|
716
|
+
- **Streaming by default.** When the deployment offers the live transcription route, snippets flow
|
|
717
|
+
through it and you get per-sentence finals as the speech settles — each with `.final == True` and
|
|
718
|
+
`t.raw` carrying the sentence's `start`/`end`. Pass `partials=True` to *also* get the revisable
|
|
719
|
+
live hypothesis (`.final == False`); a partial **replaces** the previous one wholesale, so render
|
|
720
|
+
the latest, don't append.
|
|
721
|
+
- `model=` / `language=` pick the transcription model and language hint (omit `language` to
|
|
722
|
+
autodetect; omit `model` for the deployment default). Transcription runs on *your*
|
|
723
|
+
credentials/compute. An empty-text sentence is dropped unless `skip_empty=False`; a snippet the
|
|
724
|
+
server can't transcribe is skipped so one bad frame can't kill the stream, but a rejected
|
|
725
|
+
credential (401/4401) raises `AuthError`.
|
|
726
|
+
- **Degradation ladder.** If the streaming route can't be reached — an old deploy, a proxy that
|
|
727
|
+
strips the WebSocket upgrade, a refused connection — this degrades to per-clip batch
|
|
728
|
+
transcription: one independent transcript per ~10 s window, **no partials**, the clip's format
|
|
729
|
+
taken from each frame's own MIME type (the Player ships Opus-in-WebM). On *this* fallback path
|
|
730
|
+
each transcript carries its source frame's `sequence` and `window_seconds` under
|
|
731
|
+
`t.raw["snippet"]` — a cheap ordering handle that has no equivalent on the streaming path (there's
|
|
732
|
+
no 1:1 snippet↔sentence correlation). The fallback is **connect-time only**: once streaming is
|
|
733
|
+
live a mid-session drop *raises* rather than silently degrading.
|
|
734
|
+
|
|
735
|
+
**Rooms-free — `client.transcribe_live(chunks, ...)`.** Not everything lives in a room. This runs
|
|
736
|
+
the same streaming engine over any async iterable of audio byte-chunks (a mic, a file, a socket),
|
|
737
|
+
yielding the same finals (and `partials=True` partials):
|
|
738
|
+
|
|
739
|
+
```python
|
|
740
|
+
async for t in client.transcribe_live(mic_chunks(), format="pcm16", sample_rate=16000):
|
|
741
|
+
print(str(t))
|
|
742
|
+
```
|
|
743
|
+
|
|
744
|
+
- **Container clips** (`format="wav"` default, or `"webm"`/`"ogg"`/…): each chunk is a
|
|
745
|
+
self-contained, decodable clip — the browser `MediaRecorder` / room-snippet shape.
|
|
746
|
+
- **Raw PCM** (`format="pcm16"`): a continuous PCM16LE **mono** byte stream, the shape native mic
|
|
747
|
+
libraries (`sounddevice`/`pyaudio`) yield. This **requires** `sample_rate=` — raw PCM is
|
|
748
|
+
headerless, so the SDK can't sniff the capture rate. Passing `sample_rate=` with a container
|
|
749
|
+
format is an error (containers carry their own rate).
|
|
750
|
+
|
|
751
|
+
**In-room broker — `room.transcribe()`.** The broker form consumes the room's snippet stream on
|
|
752
|
+
**your** credentials and publishes the transcript back into the room as signals, so any consumer —
|
|
753
|
+
including a loginless, code-joined tool — reads captions without transcription access of its own:
|
|
754
|
+
|
|
755
|
+
```python
|
|
756
|
+
async with room.transcribe(): # a side effect: starts immediately, stops on room exit
|
|
757
|
+
...
|
|
758
|
+
|
|
759
|
+
# any consumer, even a code-joined peer with no credentials of its own:
|
|
760
|
+
@room.on("transcript.final")
|
|
761
|
+
def on_line(event):
|
|
762
|
+
print(event.payload["seq"], event.payload["text"])
|
|
763
|
+
```
|
|
764
|
+
|
|
765
|
+
- `transcript.partial` — `{"text": …}`, **ephemeral** (a live caption the recorder skips).
|
|
766
|
+
Wholesale-replaced — render the latest, don't append.
|
|
767
|
+
- `transcript.final` — `{"text": …, "start": …, "end": …, "seq": n}`, durable. `seq` is a per-handle
|
|
768
|
+
counter from 0; the signal's `source` is `"transcriber"`. Bridge the room into a session
|
|
769
|
+
(`room.record(into=session)`) and each final is persisted — no new recording API, the `ephemeral`
|
|
770
|
+
flag alone drives the recorder split.
|
|
771
|
+
- Like `record`, it's a side effect: started in the background, stopped when the room's `async with`
|
|
772
|
+
exits — or `await handle.stop()` / scope it with `async with room.transcribe(): …`. See
|
|
773
|
+
`examples/live_transcript.py` for the manual live-test recipe.
|
|
774
|
+
|
|
775
|
+
### Live audio clips (async)
|
|
776
|
+
|
|
777
|
+
`room.audio_clips()` is the rawest member of the family — it hands you each ~10 s window decoded and
|
|
778
|
+
nothing else (no embedding, no transcription — cost: none), for custom processing:
|
|
779
|
+
|
|
780
|
+
```python
|
|
781
|
+
async for clip in room.audio_clips(): # zero-args
|
|
782
|
+
do_something(clip.data, clip.mime) # AudioClip(data, mime, sequence,
|
|
783
|
+
# window_seconds, source, source_name)
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
Same demand gate and one-time warning as `audio_vectors`. Producer-agnostic: a Player (v1 frames,
|
|
787
|
+
`clip.source is None`) or a Capture tool (v2 frames — `clip.source` is the sender's connection id,
|
|
788
|
+
`clip.source_name` its announced name, resolved from the subscribe-time roster).
|
|
789
|
+
|
|
790
|
+
### Share your embedding / chat access: `offer_embeds` / `offer_chat` (async)
|
|
791
|
+
|
|
792
|
+
Let other room participants use your embedding/chat access — they ask, your client answers. A
|
|
793
|
+
loginless, code-joined peer (e.g. a Unity scene) has no Atrium credentials of its own. Register one
|
|
794
|
+
of these on an already-authed room **before** `connect()`/`serve()` — the offer is only advertised
|
|
795
|
+
when the connection opens, so calling either after connecting raises `AtriumError`. Once registered,
|
|
796
|
+
it answers that peer's requests over the room — riding the ordinary `on`/`request` machinery, no new
|
|
797
|
+
transport:
|
|
798
|
+
|
|
799
|
+
```python
|
|
800
|
+
room.offer_embeds(spaces=["clip"]) # spaces=None (default): unrestricted
|
|
801
|
+
room.offer_chat(model="gpt-4o-mini", max_tokens=200) # model=None: the deployment/client default
|
|
802
|
+
await room.connect(identity="broker")
|
|
803
|
+
await room.serve()
|
|
804
|
+
```
|
|
805
|
+
|
|
806
|
+
- `offer_embeds` handles `embed.resolve`: `{anchors: {name: <descriptor>}, space?}` →
|
|
807
|
+
`{space, vectors: {name: [floats]}, errors?}`. A descriptor is `{"text": "…"}` /
|
|
808
|
+
`{"concept": "name", "collection"?: "…"}` / `{"vector": [...]}` — the same shapes the `by.*`
|
|
809
|
+
operand grammar builds for `compare`. **The requester chooses `space`** (omitted → the
|
|
810
|
+
deployment's default text space); `spaces=` optionally allowlists what this broker serves. A
|
|
811
|
+
failing anchor (unknown concept, an unsupported `asset` descriptor) lands in `errors` without
|
|
812
|
+
failing the rest; a malformed/oversized request or a disallowed space gets a whole-request
|
|
813
|
+
`{"error": "…"}` reply. Text is embedded with *this* client's own credentials; repeated
|
|
814
|
+
text/concept anchors are cached.
|
|
815
|
+
- `offer_chat` handles `chat.generate`: `{messages: [{role, content}, …]}` → `{text, model}`.
|
|
816
|
+
Stateless (no history kept). The **broker** pins `model` (cost control — the requester can't
|
|
817
|
+
choose it) and caps `max_tokens`; malformed/empty/over-sized `messages` get `{"error": "…"}`.
|
|
818
|
+
**Billing**: any peer holding the connect code can trigger a reply on your model billing (there's
|
|
819
|
+
a per-call `max_tokens` cap, but no aggregate limit across calls) — rotate the code for an open
|
|
820
|
+
study, or prefer a local model, if that's a concern.
|
|
821
|
+
- Both announce a discoverable role in the presence roster (`announce.roles`) —
|
|
822
|
+
`"embed-broker"` / `"chat-broker"` — so a code-joined peer can find them. The full wire dialect
|
|
823
|
+
(including the `audio.vector` info signal from `relay=True` above) is in
|
|
824
|
+
`docs/reference/rooms-sessions-protocol.md`.
|
|
825
|
+
|
|
826
|
+
### Offer live captions + recording to a Capture tool: `offer_transcription` / `offer_audio_capture` (async)
|
|
827
|
+
|
|
828
|
+
The Capture tool (`/tools/capture/?code=…`) is a loginless voice surface. The host publishes two
|
|
829
|
+
standing-service offers; the tool's toggles turn them on/off over the room (`<service>.start` /
|
|
830
|
+
`<service>.stop`, each replied `{"ok": True}` / `{"error": …}`). Register both **before**
|
|
831
|
+
`connect()`/`serve()` (like the brokers above):
|
|
832
|
+
|
|
833
|
+
```python
|
|
834
|
+
async with room.record(into=session): # the single durable binding — see below
|
|
835
|
+
room.offer_transcription() # "Live captions" toggle → transcript.* signals
|
|
836
|
+
room.offer_audio_capture() # "Save recording" toggle → audio into the session
|
|
837
|
+
await room.serve()
|
|
838
|
+
```
|
|
839
|
+
|
|
840
|
+
- `offer_transcription(model=None, language=None)` runs the same engine as `room.transcribe()` on
|
|
841
|
+
*your* credentials while on, publishing `transcript.partial` (ephemeral) / `transcript.final`
|
|
842
|
+
(durable) — each stamped with the producer's `source` (connection id) and roster-resolved
|
|
843
|
+
`source_name`. Bare call is the happy path (deployment default model, language autodetect).
|
|
844
|
+
- `offer_audio_capture()` is **session-less**: `room.record(into=session)` is the single durable
|
|
845
|
+
binding, and the capture bridge writes recordings into **the active recorder's** session. So the
|
|
846
|
+
"Save recording" toggle is meaningful only while the room is being recorded, and availability
|
|
847
|
+
rides the wire so the tool can show it accordingly:
|
|
848
|
+
- a `capture.status` request replies `{"ok": True, "available": <bool>}` — available ⇔ ≥1 recorder
|
|
849
|
+
is active on this room handle;
|
|
850
|
+
- every recorder start/stop broadcasts an ephemeral `capture.available` `{"available": <bool>}`;
|
|
851
|
+
- `capture.start` with **no** active recorder replies
|
|
852
|
+
`{"error": "no active recorder — call room.record(into=session) first"}`.
|
|
853
|
+
On `capture.start` it binds the **first** active recorder's session and bridges each `audio.snippet`
|
|
854
|
+
frame into one recording per source (keyed by announced `name`, else connection id — a same-name
|
|
855
|
+
reconnect continues the same recording). When that recorder stops (explicit stop, the room's
|
|
856
|
+
`async with` exit, or room close) every open recording is finalized, the bridge disengages, and
|
|
857
|
+
`capture.available: false` is broadcast. **Exactly one recorder feeds audio** — a second concurrent
|
|
858
|
+
recorder warns once and is not double-written.
|
|
859
|
+
- See `examples/capture_session.py` for the whole shape (recording off until you flip the toggle).
|
|
860
|
+
|
|
861
|
+
### Driving the Player's audio layers
|
|
862
|
+
|
|
863
|
+
The hosted Player (`/tools/player/?code=…`) plays sustained, mixable **layers** — background music,
|
|
864
|
+
ambient beds, SFX. A layer is just a named, independently-mixed channel (its own volume + effects);
|
|
865
|
+
every layer can hold both a single-track **queue** and overlapping **one-shots** at once — the
|
|
866
|
+
difference is `mode`, not the layer. `Room` and `RoomParticipant` carry the app side of the
|
|
867
|
+
`play` / `layer.*` dialect (`docs/reference/audio-layers-dialect.md`):
|
|
868
|
+
|
|
869
|
+
```python
|
|
870
|
+
room.configure_layers([ # optional — layers auto-vivify on first use
|
|
871
|
+
{"id": "music", "label": "Music", "volume": 0.5},
|
|
872
|
+
{"id": "ambient", "crossfade_ms": 3000},
|
|
873
|
+
{"id": "sfx"},
|
|
874
|
+
])
|
|
875
|
+
# play() takes an Asset / SearchHit, a content-URL or asset-id string, a track dict, or a list:
|
|
876
|
+
room.play(tavern_song, layer="music", mode="queue", transition="now") # crossfade a song in
|
|
877
|
+
room.play({"asset_id": rain.id, "loop": True}, layer="ambient", mode="queue") # looping bed
|
|
878
|
+
room.play(beds, layer="ambient", mode="queue", repeat=True, shuffle=True) # endless shuffled ambience
|
|
879
|
+
room.play(clang, layer="sfx") # one-shot (mode defaults to oneshot)
|
|
880
|
+
room.play([hit_a, hit_b], layer="music", mode="queue", transition="queued") # queue several
|
|
881
|
+
room.play({"asset_id": clang.id, "delay_seconds": 0.2, "spatial": {"x": -1, "y": 0, "z": 0}}, layer="sfx")
|
|
882
|
+
|
|
883
|
+
room.clear_layer(layer="ambient", fade_out_ms=2000) # one layer (queue + one-shots)
|
|
884
|
+
room.clear_all(fade_out_ms=1000) # stop all audio
|
|
885
|
+
room.set_layer_effects("music", {"lowpass": {"hz": 700}}, fade_ms=600) # backroom muffle
|
|
886
|
+
room.set_layer_effects("ambient", {"reverb": {"mix": 0.4, "ir": "cave"}, # wet convolution send
|
|
887
|
+
"echo": {"time_ms": 330, "feedback": 0.35}}) # + a slapback
|
|
888
|
+
room.set_layer_effects("*", {}, fade_ms=800) # every existing layer back to neutral
|
|
889
|
+
room.set_layer_position("music", 0, 0, -2, fade_ms=800) # place the whole layer 2 m in front (HRTF)
|
|
890
|
+
state = room.query_layers(timeout=10.0) # {"layers": {"music": {...}, ...}} — blocking pull
|
|
891
|
+
|
|
892
|
+
for snap in room.watch_layers(): # subscribe: a fresh snapshot on every change
|
|
893
|
+
music = snap["layers"].get("music", {})
|
|
894
|
+
if music.get("remaining_ms", 0) < 60_000 and music.get("queued_count", 0) < 2:
|
|
895
|
+
room.play(next_track, layer="music", mode="queue", transition="queued") # refill ahead of silence
|
|
896
|
+
```
|
|
897
|
+
|
|
898
|
+
`play(source, *, layer=None, mode="oneshot", transition="now", repeat=False, shuffle=False)`.
|
|
899
|
+
`repeat`/`shuffle` are queue-mode modes — `repeat` loops the whole queue, `shuffle` randomizes its
|
|
900
|
+
order (combinable). A track dict carries per-sound options — `loop` (a single sustained track,
|
|
901
|
+
distinct from queue `repeat`), `segment` (a stored sound-event id),
|
|
902
|
+
`clip_start_seconds`/`clip_end_seconds`, `gain`, `spatial`, and (one-shot)
|
|
903
|
+
`delay_seconds`/`max_duration_seconds`/`fade_in_ms`/`fade_out_ms`.
|
|
904
|
+
|
|
905
|
+
`set_layer_effects(layer, effects, *, fade_ms=None)` sets the layer's whole effect bus at once —
|
|
906
|
+
`lowpass`/`highpass` (filters), `reverb` (a **wet send**: `{"mix": 0.4, "ir": "hall"}`, `ir` one of
|
|
907
|
+
`room`/`hall`/`cave`/`outdoor`, or `ir_asset` an asset id to convolve with your own impulse
|
|
908
|
+
response), and `echo` (`{"time_ms": 330, "feedback": 0.35, "mix": 0.25}`). It's the state, not a
|
|
909
|
+
patch: an **absent** implemented key ramps that effect back to neutral, so `{}` clears the bus.
|
|
910
|
+
`layer="*"` applies the same state to **every existing** layer (it doesn't auto-vivify new ones), the
|
|
911
|
+
same wildcard `clear_all` uses.
|
|
912
|
+
|
|
913
|
+
`set_layer_position(layer, x, y, z=0.0, *, fade_ms=None, distance=None)` places a whole layer in 3D
|
|
914
|
+
(HRTF panning of everything on it) — the listener sits at the origin, `+x` is right, `+y` up, and
|
|
915
|
+
**negative `z` is in front**. The panner is created lazily on the first call, so a layer you never
|
|
916
|
+
position keeps the plain, unpanned graph. `distance=` opts into distance attenuation
|
|
917
|
+
(`{"model": "inverse", "ref": 1, "max": 40, "rolloff": 1}`; `model` one of
|
|
918
|
+
`linear`/`inverse`/`exponential`) and is **sticky** — a later move that omits it keeps the model, and
|
|
919
|
+
`{"rolloff": 0}` goes back to direction-only. There is no `"*"` wildcard: position is per-layer. A
|
|
920
|
+
per-sound `spatial` (`{"x", "y", "z", "distance"?}` on a track dict) composes *within* a positioned
|
|
921
|
+
layer — a placed sound inside a placed group — and a per-sound `effects` dict stacks in front of the
|
|
922
|
+
layer's bus rather than exempting the sound from it (for a sound that must stay dry, play it on its
|
|
923
|
+
own clean layer). Full dialect: `docs/reference/audio-layers-dialect.md`.
|
|
924
|
+
|
|
925
|
+
The Player **loudness-normalizes** library audio to a consistent level by default (a per-track gain
|
|
926
|
+
from the asset's measured loudness toward −18 LUFS, plus a master limiter). Playing an `Asset` /
|
|
927
|
+
`SearchHit` attaches the measured loudness automatically, so it just works; the listener can turn it
|
|
928
|
+
off (Settings / `?normalize=off`).
|
|
929
|
+
`query_layers` opens a short-lived room connection (peers see a presence join/leave) and returns the
|
|
930
|
+
Player's real snapshot — current track, queue, one-shot count, elapsed/remaining per layer.
|
|
931
|
+
`watch_layers` subscribes and yields that same snapshot on every change (plus a ~1 s heartbeat while
|
|
932
|
+
playing) until you break out — so a soundscape loop refills a low queue before it runs dry. The
|
|
933
|
+
low-water threshold is yours to pick off the snapshot; the Player pushes raw state. The **async**
|
|
934
|
+
client has identical signatures — `await room.play(...)`, `await room.clear_all()`,
|
|
935
|
+
`await room.query_layers()` (a broadcast request, no `to=` needed; first Player to reply wins), and
|
|
936
|
+
`async for snap in room.watch_layers():`.
|
|
937
|
+
|
|
938
|
+
Need a Player to drive? Launch the hosted one and only open a tab if none is connected:
|
|
939
|
+
|
|
940
|
+
```python
|
|
941
|
+
if not room.has(identity_prefix="player:"): # per-worker roster check (best-effort)
|
|
942
|
+
room.open_player() # opens the hosted Player tab
|
|
943
|
+
print(room.player_url()) # the plain launch URL — print / hand out / QR-encode
|
|
944
|
+
room.open_player(analysis=False, clips=True) # bake the room-sharing opt-ins into the URL
|
|
945
|
+
url = room.player_url(clips=True) # …or just build the URL with those params
|
|
946
|
+
```
|
|
947
|
+
|
|
948
|
+
`room.participants()` reads the room's current presence roster (a short observe stream) as typed
|
|
949
|
+
`Participant`s — `connection_id`, `identity`, `name`, `roles` (the announce's `role`/`roles` shapes
|
|
950
|
+
merged), `connected_at`, and the original `raw` dict. `room.has(role=None, *, name=None,
|
|
951
|
+
identity_prefix=None)` is the boolean sugar over it: give at least one predicate (several are ANDed)
|
|
952
|
+
— `role` matches the merged roles, `name` the announced name, `identity_prefix` the identity's
|
|
953
|
+
leading segment. A hosted Player announces identity `player:web` (no role), so
|
|
954
|
+
`room.has(identity_prefix="player:")` is the launch-once guard. Both are best-effort per web worker.
|
|
955
|
+
|
|
956
|
+
`player_url(*, analysis=None, clips=None)` and `open_player(*, analysis=None, clips=None)` take the
|
|
957
|
+
Player's URL settings as typed kwargs (`bool` → `on`/`off`; `None` omits, keeping the Player default).
|
|
958
|
+
(async: `await room.participants()` / `if not await room.has(identity_prefix="player:"):
|
|
959
|
+
room.open_player()` — `open_player`/`player_url` stay sync.)
|
|
960
|
+
|
|
961
|
+
The observation dashboard has the symmetric pair: `room.dashboard_url()` builds
|
|
962
|
+
`<base>/tools/dashboard?code=<code>` (the dashboard takes only the connect code) and
|
|
963
|
+
`room.open_dashboard()` opens it in a tab — both sync, on the sync and async clients alike.
|
|
964
|
+
|
|
965
|
+
The hosted Viewer is the visual sibling — put an image (or video) on a shared screen over the room:
|
|
966
|
+
|
|
967
|
+
```python
|
|
968
|
+
room.show(asset_id="…", caption="Rainy market") # or url="https://…", projection="360"
|
|
969
|
+
room.clear_view() # blank the stage
|
|
970
|
+
room.open_viewer() # launch a viewer tab (or print room.viewer_url())
|
|
971
|
+
```
|
|
972
|
+
|
|
973
|
+
`show` takes exactly one of `asset_id` / `url`; `viewer_url()` / `open_viewer()` mirror the Player
|
|
974
|
+
pair (both sync).
|
|
975
|
+
|
|
976
|
+
---
|
|
977
|
+
|
|
978
|
+
## 5. Surveys
|
|
979
|
+
|
|
980
|
+
A survey is a durable data object on a session, guarded by a one-time capability token:
|
|
981
|
+
|
|
982
|
+
```python
|
|
983
|
+
survey = client.surveys.start(into=session) # uses the project's DEFAULT template
|
|
984
|
+
# or: client.surveys.start("onboarding-v2", into=session, name="intro survey", return_to="https://...")
|
|
985
|
+
|
|
986
|
+
survey.token # once-only capability: read+write exactly this object
|
|
987
|
+
survey.url # hosted survey tool URL, token in the fragment — hand to the participant
|
|
988
|
+
survey.open() # open `survey.url` in the local browser (same-machine delivery)
|
|
989
|
+
```
|
|
990
|
+
|
|
991
|
+
`Surveys.start(template=None, *, into, name=None, return_to=None)` — `template=None` pins the
|
|
992
|
+
project's default template; naming one pins *that* template's current version, so later edits to
|
|
993
|
+
the template never change an already-started survey. `return_to` seeds an `app`-controlled return
|
|
994
|
+
button on the survey's end page.
|
|
995
|
+
|
|
996
|
+
Manage templates (project-scoped, immutable numbered versions) — safe to rerun every script start:
|
|
997
|
+
|
|
998
|
+
```python
|
|
999
|
+
client.surveys.template(
|
|
1000
|
+
"onboarding-v2",
|
|
1001
|
+
{"pages": [...]}, # dialect: docs/reference/survey-template-dialect.md
|
|
1002
|
+
project="p_4mD2qX", # or rely on the client's default project scope
|
|
1003
|
+
default=False,
|
|
1004
|
+
) # -> {"name", "version", "is_default"} — idempotent, appends a new
|
|
1005
|
+
# version only if the body actually changed
|
|
1006
|
+
|
|
1007
|
+
client.surveys.templates(project="p_4mD2qX") # list: name, current_version, is_default
|
|
1008
|
+
client.surveys.upload_file("logo.png", project="p_4mD2qX", role="attachment")
|
|
1009
|
+
# -> {"id", "url", "filename", "content_type"} — reference `id` from
|
|
1010
|
+
# a template body's `image` field
|
|
1011
|
+
```
|
|
1012
|
+
|
|
1013
|
+
Read the answers back for analysis, by template **name** (not a started `Survey`), across the
|
|
1014
|
+
project's sessions:
|
|
1015
|
+
|
|
1016
|
+
```python
|
|
1017
|
+
rows = client.surveys.responses("onboarding-v2", phase="post", submitted_only=True, limit=500)
|
|
1018
|
+
for r in rows:
|
|
1019
|
+
print(r.participant_ref, r.session_id, r.status, r.version, r.contents) # contents = raw answers
|
|
1020
|
+
|
|
1021
|
+
csv = client.surveys.responses("onboarding-v2", format="csv") # bytes: one row per response
|
|
1022
|
+
```
|
|
1023
|
+
|
|
1024
|
+
`responses(template, *, project=None, phase=None, submitted_only=False, limit=200, format="rows")` —
|
|
1025
|
+
newest first, resolved with `project=` exactly like `template`/`templates`. `phase` (a string or a
|
|
1026
|
+
list; `""` matches unset) filters by the response's session phase; `submitted_only` drops drafts;
|
|
1027
|
+
`limit`'s server ceiling is 500. `format="rows"` gives `SurveyResponse` rows (`contents` stays a free
|
|
1028
|
+
dict — the survey dialect is your app's); `format="csv"` gives the CSV export as `bytes` with
|
|
1029
|
+
question-id columns.
|
|
1030
|
+
|
|
1031
|
+
`start_survey(template=None, *, into, name=None, return_to=None)` (module-level) is sugar for
|
|
1032
|
+
`into._client.surveys.start(...)` — handy when you only have the session in hand.
|
|
1033
|
+
|
|
1034
|
+
**Async** mirrors every method 1:1 (`await client.surveys.start(...)`, `await async_start_survey(...)`).
|
|
1035
|
+
|
|
1036
|
+
---
|
|
1037
|
+
|
|
1038
|
+
## 6. Home Assistant bridge
|
|
1039
|
+
|
|
1040
|
+
`hci_atrium.ha.HomeAssistantBridge` runs near your HA instance, joins an already-open room, and
|
|
1041
|
+
declaratively wires room signals to HA services (and HA state back into the room). It's
|
|
1042
|
+
**async-only**. `websockets` ships as a core dependency — nothing extra to install.
|
|
1043
|
+
|
|
1044
|
+
```python
|
|
1045
|
+
import asyncio
|
|
1046
|
+
from hci_atrium import async_join_room
|
|
1047
|
+
from hci_atrium.ha import HomeAssistantBridge
|
|
1048
|
+
|
|
1049
|
+
async def main() -> None:
|
|
1050
|
+
room = await async_join_room("ABC123")
|
|
1051
|
+
bridge = (
|
|
1052
|
+
HomeAssistantBridge("ws://homeassistant.local:8123/api/websocket", "***token***")
|
|
1053
|
+
.media_sink("media_player.living_room", "media_player.kitchen") # play `play` intents
|
|
1054
|
+
.on_signal( # any other signal -> any HA service
|
|
1055
|
+
"plate.pulse", domain="number", service="set_value",
|
|
1056
|
+
entity_id="number.plate_intensity", data=lambda p: {"value": p["strength"]},
|
|
1057
|
+
)
|
|
1058
|
+
)
|
|
1059
|
+
try:
|
|
1060
|
+
await bridge.run(room) # blocks until cancelled
|
|
1061
|
+
finally:
|
|
1062
|
+
await room.aclose()
|
|
1063
|
+
|
|
1064
|
+
asyncio.run(main())
|
|
1065
|
+
```
|
|
1066
|
+
|
|
1067
|
+
- **`media_sink(*entity_ids, forward_state=True)`** — plays the room's `play` intent (the typed
|
|
1068
|
+
`{playback: {source, ref, mode?}}` shape, or the legacy `content_url`/`asset_id` shapes) via
|
|
1069
|
+
`media_player.play_media` (+ `.shuffle_set` for a shuffled playlist) on each entity. Also
|
|
1070
|
+
forwards those entities' state back as `media.state` unless `forward_state=False`.
|
|
1071
|
+
- **`on_signal(type, *, domain, service, entity_id=None, data=None)`** — maps one room signal type
|
|
1072
|
+
to one HA service call; `data` is a static dict or `callable(payload) -> dict`. Several
|
|
1073
|
+
registrations for the same `type` all fire.
|
|
1074
|
+
- **`forward_state(entity_ids, *, as_type="media.state", debounce=True)`** — forwards HA
|
|
1075
|
+
`state_changed` on these entities into the room. Call it directly for entities outside
|
|
1076
|
+
`media_sink`, or to change `as_type`/`debounce`.
|
|
1077
|
+
- **`await bridge.run(room)`** — wires everything, announces the bridge's identity, and runs both
|
|
1078
|
+
legs (room dispatcher + HA connection) until cancelled. Raises only on a terminal room refusal
|
|
1079
|
+
(a 4xx); the HA leg reconnects with backoff on its own.
|
|
1080
|
+
|
|
1081
|
+
**Zero-code CLI**, once the room exists:
|
|
1082
|
+
|
|
1083
|
+
```sh
|
|
1084
|
+
atrium ha-bridge --code ABC123 \
|
|
1085
|
+
--ha-url ws://homeassistant.local:8123/api/websocket --ha-token *** \
|
|
1086
|
+
--media-player media_player.living_room media_player.kitchen
|
|
1087
|
+
```
|
|
1088
|
+
|
|
1089
|
+
Every flag also reads an env var fallback: `ATRIUM_ROOM_CODE`, `HA_URL`, `HA_TOKEN`,
|
|
1090
|
+
`HA_MEDIA_PLAYERS` (comma-separated).
|
|
1091
|
+
|
|
1092
|
+
---
|
|
1093
|
+
|
|
1094
|
+
## 7. Recipes
|
|
1095
|
+
|
|
1096
|
+
### Search the library and play the top hit into a room (sync)
|
|
1097
|
+
|
|
1098
|
+
```python
|
|
1099
|
+
from hci_atrium import Atrium
|
|
1100
|
+
|
|
1101
|
+
client = Atrium()
|
|
1102
|
+
|
|
1103
|
+
with client, client.rooms.open(name="living room") as room:
|
|
1104
|
+
print(f"connect code: {room.connect_code}")
|
|
1105
|
+
hits = client.library.search("calm rainy afternoon", modality="audio", limit=1)
|
|
1106
|
+
if hits:
|
|
1107
|
+
asset = hits[0].asset
|
|
1108
|
+
room.send("play", {"playback": asset.playback.as_signal()})
|
|
1109
|
+
print(f"playing: {asset.title or asset.filename}")
|
|
1110
|
+
# on exit: leaves the named room standing (code kept for next run), then closes the client
|
|
1111
|
+
```
|
|
1112
|
+
|
|
1113
|
+
### Record a study session with a survey
|
|
1114
|
+
|
|
1115
|
+
```python
|
|
1116
|
+
from hci_atrium import Atrium
|
|
1117
|
+
|
|
1118
|
+
client = Atrium()
|
|
1119
|
+
session = client.sessions.start(name="P07 · pilot", participant_ref="P07")
|
|
1120
|
+
session.event("phase", {"name": "intro"})
|
|
1121
|
+
|
|
1122
|
+
survey = client.surveys.start(into=session) # the project's default template
|
|
1123
|
+
print(f"send this to the participant:\n {survey.url}")
|
|
1124
|
+
survey.open() # or: hand survey.token / survey.url to a participant-facing device
|
|
1125
|
+
|
|
1126
|
+
session.event("phase", {"name": "survey_sent"})
|
|
1127
|
+
print(f"session {session.id} — {client.sessions.resume(session.id).last_seq} events so far")
|
|
1128
|
+
```
|
|
1129
|
+
|
|
1130
|
+
### A room consumer that reacts to signals and replies to requests (async)
|
|
1131
|
+
|
|
1132
|
+
```python
|
|
1133
|
+
import asyncio
|
|
1134
|
+
from hci_atrium import async_join_room
|
|
1135
|
+
|
|
1136
|
+
async def main() -> None:
|
|
1137
|
+
async with await async_join_room("ABC123") as room:
|
|
1138
|
+
|
|
1139
|
+
@room.on("play")
|
|
1140
|
+
async def on_play(event):
|
|
1141
|
+
intent = event.payload.get("playback")
|
|
1142
|
+
print("play:", intent)
|
|
1143
|
+
|
|
1144
|
+
@room.on("ping")
|
|
1145
|
+
async def on_ping(event):
|
|
1146
|
+
return {"pong": True} # answers a room.request("ping", to=...)
|
|
1147
|
+
|
|
1148
|
+
await room.connect(identity="visuals")
|
|
1149
|
+
await room.serve() # blocks; reconnects transient drops
|
|
1150
|
+
|
|
1151
|
+
asyncio.run(main())
|
|
1152
|
+
```
|
|
1153
|
+
|
|
1154
|
+
### Record a room into a session, skipping the noise
|
|
1155
|
+
|
|
1156
|
+
```python
|
|
1157
|
+
import time
|
|
1158
|
+
from hci_atrium import Atrium
|
|
1159
|
+
|
|
1160
|
+
client = Atrium()
|
|
1161
|
+
room = client.rooms.open(name="studio")
|
|
1162
|
+
session = client.sessions.start(name="studio take")
|
|
1163
|
+
print(f"connect code: {room.connect_code} session: {session.id}")
|
|
1164
|
+
|
|
1165
|
+
with room.record(into=session):
|
|
1166
|
+
room.send("note", {"text": "session started"})
|
|
1167
|
+
room.send("beat", {"bpm": 120}, ephemeral=True) # not recorded — ephemeral
|
|
1168
|
+
time.sleep(30) # capture whatever participants send
|
|
1169
|
+
|
|
1170
|
+
print(f"recorded {client.sessions.resume(session.id).last_seq} events")
|
|
1171
|
+
```
|
|
1172
|
+
|
|
1173
|
+
---
|
|
1174
|
+
|
|
1175
|
+
## Left out / not verified here
|
|
1176
|
+
|
|
1177
|
+
- Bulk import (`POST /v1/library/import`) has no SDK wrapper; only the raw endpoint is documented.
|
|
1178
|
+
- Collection *creation/management* (the tree itself) is not an SDK operation — only scoping to an
|
|
1179
|
+
existing collection id.
|
|
1180
|
+
- `hci_atrium.dash.Dashboard` (the observation-dashboard helper) and the SFX/agent-tooling pieces
|
|
1181
|
+
(`client.tool`, `client.agent`, direct-model routing) exist in the SDK but are out of scope for
|
|
1182
|
+
this guide — see the source docstrings and `README.md` for those.
|