hci-atrium 0.3.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/GUIDE.md +1182 -0
  2. package/PARITY.md +385 -0
  3. package/README.md +96 -24
  4. package/bin/atrium.mjs +35 -0
  5. package/dist/atrium.d.ts +126 -42
  6. package/dist/atrium.d.ts.map +1 -1
  7. package/dist/atrium.js +192 -87
  8. package/dist/atrium.js.map +1 -1
  9. package/dist/dash.d.ts +3 -3
  10. package/dist/dash.d.ts.map +1 -1
  11. package/dist/dash.js +1 -2
  12. package/dist/dash.js.map +1 -1
  13. package/dist/env.d.ts +14 -0
  14. package/dist/env.d.ts.map +1 -1
  15. package/dist/env.js +22 -0
  16. package/dist/env.js.map +1 -1
  17. package/dist/index.d.ts +4 -4
  18. package/dist/index.d.ts.map +1 -1
  19. package/dist/index.js +11 -3
  20. package/dist/index.js.map +1 -1
  21. package/dist/models.d.ts +84 -31
  22. package/dist/models.d.ts.map +1 -1
  23. package/dist/models.js +97 -25
  24. package/dist/models.js.map +1 -1
  25. package/dist/player/play-command.d.ts +6 -6
  26. package/dist/player/play-command.d.ts.map +1 -1
  27. package/dist/player/play-intent.d.ts.map +1 -1
  28. package/dist/player/play-intent.js +2 -1
  29. package/dist/player/play-intent.js.map +1 -1
  30. package/dist/player/runtime.d.ts +7 -3
  31. package/dist/player/runtime.d.ts.map +1 -1
  32. package/dist/player/runtime.js.map +1 -1
  33. package/dist/react/use-room-connection.d.ts +4 -1
  34. package/dist/react/use-room-connection.d.ts.map +1 -1
  35. package/dist/react/use-room-connection.js +5 -1
  36. package/dist/react/use-room-connection.js.map +1 -1
  37. package/dist/rooms/index.d.ts +3 -1
  38. package/dist/rooms/index.d.ts.map +1 -1
  39. package/dist/rooms/index.js +4 -3
  40. package/dist/rooms/index.js.map +1 -1
  41. package/dist/rooms/offer-transcription.d.ts +1 -1
  42. package/dist/rooms/offer-transcription.js +1 -1
  43. package/dist/rooms/owned-room.d.ts +251 -0
  44. package/dist/rooms/owned-room.d.ts.map +1 -0
  45. package/dist/rooms/owned-room.js +844 -0
  46. package/dist/rooms/owned-room.js.map +1 -0
  47. package/dist/rooms/recorder.d.ts +1 -1
  48. package/dist/rooms/recorder.js +1 -1
  49. package/dist/rooms/room.d.ts +82 -278
  50. package/dist/rooms/room.d.ts.map +1 -1
  51. package/dist/rooms/room.js +60 -854
  52. package/dist/rooms/room.js.map +1 -1
  53. package/dist/rooms/standing-source.d.ts +1 -1
  54. package/dist/rooms/standing-source.js +1 -1
  55. package/dist/rooms/transcriber.d.ts +3 -3
  56. package/dist/rooms/transcriber.d.ts.map +1 -1
  57. package/dist/rooms/transcriber.js +2 -2
  58. package/dist/rooms/transport.d.ts +1 -1
  59. package/dist/rooms/transport.js +2 -2
  60. package/dist/shaping/assets.d.ts +13 -0
  61. package/dist/shaping/assets.d.ts.map +1 -1
  62. package/dist/shaping/assets.js +27 -0
  63. package/dist/shaping/assets.js.map +1 -1
  64. package/dist/shaping/chat.d.ts +3 -1
  65. package/dist/shaping/chat.d.ts.map +1 -1
  66. package/dist/shaping/chat.js +4 -2
  67. package/dist/shaping/chat.js.map +1 -1
  68. package/dist/shaping/rooms.d.ts +78 -1
  69. package/dist/shaping/rooms.d.ts.map +1 -1
  70. package/dist/shaping/rooms.js +58 -6
  71. package/dist/shaping/rooms.js.map +1 -1
  72. package/dist/shaping/sources.d.ts +7 -3
  73. package/dist/shaping/sources.d.ts.map +1 -1
  74. package/dist/shaping/sources.js +9 -10
  75. package/dist/shaping/sources.js.map +1 -1
  76. package/dist/transcribe-stream.d.ts +1 -1
  77. package/dist/transcribe-stream.js +1 -1
  78. package/dist/transcribe.d.ts +2 -2
  79. package/dist/transcribe.js +2 -2
  80. package/package.json +18 -3
package/GUIDE.md ADDED
@@ -0,0 +1,1182 @@
1
+ # hci-atrium — Student & AI-assistant guide
2
+
3
+ A complete, task-oriented reference for the `hci_atrium` Python SDK — the client for the Atrium
4
+ multimodal asset library. It's written to be pasted into a coding assistant verbatim: every
5
+ signature below is checked against `clients/python/src/hci_atrium/`. If you're new to the SDK,
6
+ start with [`README.md`](README.md) for the 60-second overview; this file is the reference you
7
+ come back to while building.
8
+
9
+ Everything here works with either client:
10
+
11
+ ```python
12
+ from hci_atrium import Atrium # sync — the default for scripts
13
+ from hci_atrium import AsyncAtrium # asyncio — same surface, `await` everything
14
+ ```
15
+
16
+ The two mirror each other method-for-method. Examples below use the sync client unless a feature
17
+ (rooms driven by `on`/`serve`, the Home Assistant bridge) needs asyncio — those are marked **async**.
18
+
19
+ ## Reading this from TypeScript
20
+
21
+ The same SDK is published on npm as `hci-atrium` (`pnpm add hci-atrium`), at parity with the
22
+ Python surface below apart from the deliberate gaps noted at the end of this section. This guide
23
+ stays the reference for both — translate it mechanically:
24
+
25
+ - **Names**: `snake_case` → `camelCase`. `library.search_by_examples` → `library.searchByExamples`,
26
+ `add_direct_model` → `addDirectModel`, `content_kind` → `contentKind`.
27
+ - **Keyword arguments** become one trailing `params` object:
28
+ `client.library.search("rain", modality="audio", limit=10)` →
29
+ `client.library.search("rain", { modality: "audio", limit: 10 })`.
30
+ - **`None` → `null`**, `True`/`False` → `true`/`false`, dicts → plain objects.
31
+ - **Async everywhere.** There is no sync/async split — every call returns a `Promise`, so the
32
+ `AsyncAtrium` examples are the ones to follow (`await` the sync ones too).
33
+
34
+ Deliberate gaps: the full `atrium` CLI (`login`, `whoami`, `search`, …) and the Home Assistant
35
+ bridge are **Python-only**. The npm package ships exactly one command, `atrium guide`, which prints
36
+ this file (`pnpm exec atrium guide`). Everything else is ported.
37
+
38
+ For the per-surface specifics — the parity ledger, the browser-side differences (API keys, capture),
39
+ and the React bindings — see `PARITY.md` and `README.md`, which ship alongside this file in the npm
40
+ package (`clients/typescript/` in the repo).
41
+
42
+ ---
43
+
44
+ ## 1. Setup + auth
45
+
46
+ ### Install
47
+
48
+ **Fresh project with uv (recommended):**
49
+
50
+ ```sh
51
+ uv init my-study-app
52
+ cd my-study-app
53
+ uv add hci-atrium
54
+ ```
55
+
56
+ Run scripts with `uv run python main.py` (or `uv run <script.py>`) — uv creates the venv and syncs
57
+ automatically, no activation. The CLI comes along: `uv run atrium --help`. The SDK needs Python
58
+ >=3.10; if the pinned version is older, `uv python pin 3.12`.
59
+
60
+ **Existing project:** `uv add hci-atrium` in a uv-managed project; for a plain venv/pip project:
61
+
62
+ ```sh
63
+ pip install hci-atrium
64
+ ```
65
+
66
+ **Throwaway try-out:** `uv run --with hci-atrium python` opens a REPL with the SDK importable, no
67
+ project needed.
68
+
69
+ SDK developers working against a checkout: `uv add --editable path/to/clients/python` overrides the
70
+ published package with your local source.
71
+
72
+ ### Connect
73
+
74
+ ```python
75
+ from hci_atrium import Atrium
76
+
77
+ client = Atrium() # defaults to the deployed instance (or ATRIUM_BASE_URL)
78
+ ```
79
+
80
+ | What | Argument | Environment variable | Default |
81
+ |---|---|---|---|
82
+ | Instance URL | `Atrium(base_url=...)` | `ATRIUM_BASE_URL` | the deployed instance |
83
+ | Credential | `Atrium(api_key="atr_...")` | `ATRIUM_API_KEY` | cached OAuth → browser sign-in |
84
+
85
+ **Zero-config sign-in.** With no `api_key`, the first authenticated call opens a browser
86
+ (Authorization Code + PKCE, out-of-band code). Tokens are cached in `~/.atrium/credentials.json`
87
+ (mode `0600`) and refreshed silently — later runs and later scripts don't prompt again.
88
+ `client.login()` forces this now instead of lazily; `client.logout()` forgets the cached session;
89
+ `client.token()` returns a live bearer token (for pointing another OpenAI-compatible tool at the
90
+ same instance).
91
+
92
+ **API keys (CI / headless / a fixed identity).** A key looks like `atr_...` and is minted in the
93
+ Atrium web app under a project's *Settings → API keys*. Two shapes:
94
+
95
+ - **Project-scoped key** — acts as that project; every call is scoped to it automatically, no
96
+ `project=` needed. The right shape for a study app that only ever writes to one project.
97
+ - **Account-wide key** — acts as your user across every project you can see.
98
+
99
+ Pass it as `Atrium(api_key="atr_...")` or set `ATRIUM_API_KEY` — either skips the browser flow
100
+ entirely.
101
+
102
+ ```python
103
+ me = client.me() # GET /v1/users/me
104
+ me.email, me.role
105
+ me.key_project_id # set when a project-scoped key is in use
106
+ me.effective_project_id # the project data ops target (scoped key, else your Personal project)
107
+ ```
108
+
109
+ ### Projects & scope
110
+
111
+ A **project** is identified by a `p_...` short id (e.g. `p_4mD2qX`) — a base58, URL-safe id that
112
+ doubles as the API "connect target" for that project. Scope any call to a project (and optionally
113
+ a collection within it) three ways:
114
+
115
+ ```python
116
+ client.library.search("rain", project="p_4mD2qX") # per call
117
+ client = Atrium(project="p_4mD2qX") # default for a whole client
118
+ photos = client.collection("c_7Bq3Xk") # a reusable scoped view
119
+ photos.library.search("rain") # every call on `photos` is scoped
120
+ ```
121
+
122
+ `client.project(id)` / `client.collection(id)` return a scoped view that shares the same
123
+ authenticated session (cheap — no re-auth). A project-scoped API key scopes everything
124
+ automatically; you never need `project=` with one.
125
+
126
+ ### The CLI
127
+
128
+ Installing the package puts an `atrium` command on `PATH`. From inside the monorepo:
129
+
130
+ ```sh
131
+ uv run --project clients/python atrium login
132
+ uv run --project clients/python atrium whoami
133
+ uv run --project clients/python atrium search "calm rainy afternoon" --modality audio
134
+ ```
135
+
136
+ Once installed into your own project, just `atrium ...` (or `python -m hci_atrium ...`). Full
137
+ command list: `login`, `logout`, `whoami`, `search`, `assets`, `upload`, `enrich`, `chat`,
138
+ `ha-bridge` — every command takes `--base-url`; `search`/`assets`/`upload` also take `--project`/
139
+ `--collection`. Run `atrium <command> --help` for flags.
140
+
141
+ ### Errors
142
+
143
+ Every exception the SDK raises derives from `hci_atrium.AtriumError`:
144
+
145
+ ```python
146
+ from hci_atrium import AtriumError, AuthError, APIError
147
+
148
+ try:
149
+ client.library.search("rain")
150
+ except APIError as exc: # a non-2xx HTTP response
151
+ exc.status_code, exc.body # int, str (truncated)
152
+ except AuthError: # sign-in / token exchange / refresh failed
153
+ ...
154
+ except AtriumError: # catch-all for everything the SDK raises
155
+ ...
156
+ ```
157
+
158
+ ---
159
+
160
+ ## 2. Core concepts, in one screen
161
+
162
+ - **Project** — the top-level scope for data (`p_...` short id). Every asset, room, and session
163
+ lives in exactly one project. A study is usually one project.
164
+ - **Collection** — a node in a project's recursive collection tree (`c_...` short id); `Library` =
165
+ the project's root. Used to scope search/listing to a subtree. The SDK only *scopes to* an
166
+ existing collection id (`client.collection(id)`) — creating/managing the tree is a web-app task,
167
+ not exposed here.
168
+ - **Asset** — one library item. Three independent things it can carry:
169
+ - **bytes** (`sha256` set) — decodable content, served at `asset.content_url`.
170
+ - **link** (`source_url` set) — an external/attribution URL (e.g. a Spotify playlist).
171
+ An asset can have *both* (uploaded audio with a source link) or *neither* (a description-only
172
+ asset, e.g. "the smell of rain").
173
+ - **description** — always present as metadata; it *is* the content when there are no bytes.
174
+ - `asset.facts` — a server-owned JSON bag (analysis results, the `link` discriminator, etc.);
175
+ never write to it yourself.
176
+ - `asset.data` — an app-owned JSON bag the server never touches — your study's own structured
177
+ metadata on the asset.
178
+ - **Rooms vs sessions** — deliberately decoupled:
179
+ - A **room** is pure live transport: `room.send(...)` fans a message to everyone connected,
180
+ nothing is stored. Anyone can join by a short **connect code**, no account needed
181
+ (`join_room(code)`). No replay — if you weren't listening, you missed it.
182
+ - A **session** is a durable, append-only, gapless log (`session.event(...)`). No running/ended
183
+ lifecycle — it's simply the record, always appendable. A session doesn't need a room.
184
+ - To keep what happens live, a host **bridges** the two: `room.record(into=session)`.
185
+ - **Surveys** — a durable *data object* on a session, guarded by a one-time capability token
186
+ (`session.token`); a project-scoped, versioned template drives what it asks.
187
+
188
+ ---
189
+
190
+ ## 3. Library
191
+
192
+ ### Search
193
+
194
+ ```python
195
+ client.library.search(
196
+ query: str = "",
197
+ *,
198
+ like=None, # rank "more like this" — an Asset, SearchHit, or id
199
+ like_by: str | None = None, # which of the liked asset's signals seeds it (default: native)
200
+ modality: str | None = None, # desired result kind, e.g. "audio", "image"
201
+ space: str | None = None, # force a specific embedding space (id or name)
202
+ limit: int = 20,
203
+ tags: list[str] | None = None, # results must carry ALL of these tags
204
+ any_tags: list[str] | None = None, # …and at least one of these
205
+ exclude_tags: list[str] | None = None, # …and none of these
206
+ provider: str | list[str] | None = None, # who provides the content — see below
207
+ filters: Filters | None = None, # the long tail: content_kind, statuses, ranges, exclude_ids, …
208
+ image=None, audio=None, video=None, # search BY a media file instead of text (cross-modal)
209
+ vector: list[float] | None = None, # rank by a raw vector (pass space= with it)
210
+ include_vectors: bool | str = False, # inline one vector per hit onto hit.vector
211
+ project: str | None = None,
212
+ collection: str | None = None,
213
+ ) -> list[SearchHit]
214
+ ```
215
+
216
+ ```python
217
+ for hit in client.library.search("calm rainy afternoon", modality="audio", limit=5):
218
+ print(round(hit.score, 3), hit.asset.title or hit.asset.filename)
219
+ print(" ", hit.asset.content_url)
220
+
221
+ playlists = client.library.search("lofi beats", modality="audio", provider="spotify:playlist")
222
+
223
+ # "More like this" — pass an asset (or a SearchHit straight from a previous search) as like=
224
+ similar = client.library.search(like=hit, modality="audio")
225
+ ```
226
+
227
+ - Pass **exactly one** query input: a text `query`, an example asset (`like=`), a media source
228
+ (`image=`/`audio=`/`video=`), or a raw `vector=`. `modality` means "what kind of result I want":
229
+ results OF that kind, **provided by the library's own storage and matched by their native
230
+ content** (the server's media-intent defaults — text-about-content and byteless links don't
231
+ compete unless you opt in). For a text query it also picks that modality's space; for a media
232
+ query the query embeds in the cross-modal joint space.
233
+ - `like=` ranks "more like this": an `Asset`, a `SearchHit`, or a bare id. `modality` still filters
234
+ the results (cross-modal: an image can fetch its fitting audio, ranked in the cross-modal default
235
+ space); the source asset must be embedded in the ranking `space`. `like_by=` picks which of the
236
+ asset's stored signals seeds the query (`"native"`, `"manual"`, `"bridge:<modality>"`,
237
+ `"view:N"`/`"seg:<id>"`) — default is the native-first auto pick.
238
+ - `provider` selects where results come from: `"library"` (stored bytes — the implied default
239
+ for a media `modality`), a streaming provider (`"spotify"`, entity-qualified
240
+ `"spotify:playlist"` / `"spotify:track"` / `"spotify:album"`), `"link"` (anything with a URL,
241
+ recognized or not), `"none"` (neither bytes nor URL — e.g. a fileless asset you hand-tagged
242
+ "audio"), `"any"` (everything), or a list to mix. Non-library classes match by their
243
+ description text — mixing classes ranks text-matches against content-matches, which is only
244
+ score-comparable on a fully calibrated space; and a provider link with no description holds no
245
+ vectors at all, so ranked search can't surface it (keyword search can). Unknown values → 422
246
+ listing the vocabulary.
247
+ - `hit.asset` is an `Asset`; `hit.score`/`hit.raw_score`/`hit.calibrated`/`hit.matched_by`/
248
+ `hit.match_label` describe the match.
249
+ - `include_vectors=True` inlines onto `hit.vector` the one vector that scored each hit (its identity
250
+ is `hit.matched_by`); a space id/name instead returns that space's vector per hit, which can
251
+ differ from the one that ranked it. Off by default — vectors are large and never leave Postgres in
252
+ the search path; `hit.vector` is `None` unless you asked.
253
+ - `filters=Filters(...)` carries full library-filter power: `content_kind` (the immutable kind
254
+ sniffed from the file's bytes — `"image"`, `"audio"`, …, unlike `modality` which matches the
255
+ editable label, so it finds a JPEG even after it's retagged `"product"`), `statuses`, numeric
256
+ fact `ranges` (`{"bpm": (90, 120), "duration_s": (None, 30)}`), `uploader`, extra kind/label
257
+ facets (`all_kinds`/`any_kinds`/`none_kinds`/`all_labels`/`any_labels`/`none_labels`),
258
+ `max_seconds`, `match_by` (the Match-by override: an explicit list — e.g. `["native", "manual"]`,
259
+ or `[]` for every signal — applies globally, replacing the per-class default that comes with a
260
+ `provider` filter), `data_match={"kind": "field"}` (a JSONB containment filter on the app-owned
261
+ `asset.data`, `data @> data_match` — retrieval of what you wrote, not derivation), and
262
+ `exclude_ids` (drop these asset ids — dedup for a prefetch loop; capped at 512).
263
+
264
+ Other `Library` methods, same scoping conventions (`project=`/`collection=`, `space=`):
265
+
266
+ | Method | Purpose |
267
+ |---|---|
268
+ | `library.add(file=None, *, modality=, description=, collection=)` | add an asset to the library — a file, or (omit `file`) a fileless asset whose `description` *is* its content. See [Add to the library](#add-to-the-library) |
269
+ | `library.compare(a, b, *, space=, breakdown=False)` | pairwise similarity of two operands — `Asset`, `str`, `list[float]`, or `by.file(...)`/`by.asset(id)`/`by.concept(name_or_id, collection=)`; `breakdown=True` adds the per-feature terms (see below) |
270
+ | `library.search_by_examples(positives, negatives=, *, method="cav"/"centroid"/"probe", match_by=, filters=, ...)` | "mood board" — rank by a set of example assets (default `method="cav"`). `match_by=["native"]` picks the signals by class — same meaning as `search`'s `filters.match_by`. `match_by={"native": 1.0, "manual": 0.3}` also weights them (only a set query can: search picks one best signal, this blends every one). An allowlist either way; no space ids. `probe` fits its own weights, so pass `method="cav"` to set them by hand. `filters=` narrows by kind/label facets + `content_kind`; a Filters field with no set-query slot (`statuses`/`ranges`/`uploader`/`max_seconds`/`data_match`/`exclude_ids`) raises |
271
+ | `library.enrich(query, modality, *, tags=, total=, preview=, wait=)` | pull external candidates (Freesound/Openverse/…) into the library |
272
+ | `library.vectors(ids, *, space, match_by=None, strict=False)` | bulk-read stored vectors for many assets — see [Reading an asset back](#reading-an-asset-back) |
273
+ | `library.spaces()` | list the searchable embedding spaces |
274
+ | `library.default_space(modality)` | the deployment's default space for that modality (a `spaces()` dict, or `None`) |
275
+
276
+ `by.concept(...)` can only be one side of a `compare` call — it scores the *other* operand's vector
277
+ against that tag's materialized concept snapshot (calibrated to the concept's own boundary, not the
278
+ usual search calibration), 404 if the concept doesn't resolve, 422 if it has none. `breakdown=True`
279
+ then fills `result.features` with the per-`(space, matched_by)` terms behind that score (it stays
280
+ `None` for a plain asset/vector compare, which has no per-feature structure):
281
+
282
+ ```python
283
+ result = client.library.compare(asset, by.concept("cozy"), breakdown=True)
284
+ for f in result.features or []:
285
+ print(f.space, f.matched_by, f.weight, f.raw_score, f.score) # which signals carried the match
286
+ ```
287
+
288
+ Read those as **raw per-feature terms, not additive contributions** — they cover only the scored
289
+ space's features and do not sum to `result.score`.
290
+
291
+ `default_space` keeps deployment space ids out of your code — the deployment declares its defaults,
292
+ the app just asks:
293
+
294
+ ```python
295
+ space = client.library.default_space("audio") # e.g. {"id": "clap", "name": "CLAP", ...} or None
296
+ rows = client.library.vectors(ids, space=space["id"])
297
+ ```
298
+
299
+ ### Asset views
300
+
301
+ ```python
302
+ asset = client.assets.get("a_123")
303
+
304
+ asset.capabilities # set[str] ⊆ {"bytes", "link"} — independent flags, both can hold
305
+ asset.link # AssetLink(provider, entity, uri) | None — present iff "link" in capabilities
306
+ asset.playback # PlaybackIntent — ALWAYS present
307
+ asset.content_url # public URL for the bytes (fingerprinted ?v=sha when known)
308
+ asset.thumbnail_url # public URL for an image/video poster (404 if none)
309
+ ```
310
+
311
+ `asset.playback` normalizes the asset into one play intent, mirroring the room `play` signal's
312
+ `{source, ref, mode?}` shape 1:1 — a host can forward it as-is:
313
+
314
+ ```python
315
+ intent = asset.playback # PlaybackIntent(source=..., ref=..., mode=...)
316
+ intent.as_signal() # {"source": ..., "ref": ..., "mode": ...} — mode omitted if unset
317
+ room.send("play", {"playback": intent.as_signal()})
318
+ ```
319
+
320
+ Priority: `source="bytes"` whenever the asset has decodable content (even if it also has a link);
321
+ else the recognized link provider's name (currently only `"spotify"`) with `ref` a
322
+ `provider:entity:id` URI, `mode="shuffle"` for a playlist; else `source="link"` with the raw URL
323
+ for an unrecognized provider; else `source="description"` with `ref` = the asset id (nothing to
324
+ fetch, but a host can still look it up).
325
+
326
+ <a name="reading-an-asset-back"></a>
327
+ ### Reading an asset back: vectors, segments, index
328
+
329
+ What the library stored *for* an asset — its embeddings, its cut sound events, and the processing
330
+ state of each signal. All three are bound on the asset itself:
331
+
332
+ ```python
333
+ asset = client.assets.get("a_123")
334
+
335
+ for v in asset.vectors(): # every stored vector; list[AssetVector]
336
+ print(v.space, v.matched_by, len(v.vector)) # matched_by: "native" / "manual" / "bridge:text" / "seg:<id>"
337
+ asset.vectors(space="CLAP", match_by="native") # narrow: space (id or name, one or a list) + signal
338
+
339
+ for seg in asset.segments(): # the sound events cut out of an audio asset
340
+ print(seg.ordinal, seg.start_s, seg.end_s, seg.content_url) # just that slice's pre-cut bytes
341
+
342
+ for entry in asset.index(): # one row per stored signal — the processing state
343
+ print(entry.space, entry.matched_by, entry.active, entry.status, entry.error)
344
+ ```
345
+
346
+ `client.assets.vectors(asset, space=, match_by=)` / `.segments(asset)` / `.index(asset)` are the
347
+ unbound forms — each takes an `Asset`, a `SearchHit`, or a bare id. Vectors and segments are public
348
+ by id (like `content_url`); the index map is authenticated. On the async client the bound methods are
349
+ awaitable (`await asset.vectors()`).
350
+
351
+ For many assets at once, `library.vectors(ids, *, space, match_by=None, strict=False)` is the bulk
352
+ read — `space` is **required** here (one id/name, or a list):
353
+
354
+ ```python
355
+ rows = client.library.vectors([h.asset for h in hits], space="CLAP")
356
+ {r.asset: r.vector for r in rows} # each AssetVector carries its owning asset id
357
+ ```
358
+
359
+ With `match_by=None` the signal is resolved per asset, so one call can return **mixed** `matched_by`
360
+ values — read `r.matched_by` to see which each row is. Ids that don't exist or aren't visible to your
361
+ key land in the server's `missing` list: `strict=False` (default) omits them, `strict=True` raises
362
+ listing them.
363
+
364
+ ### Browse, add to the library
365
+
366
+ ```python
367
+ page = client.assets.list(
368
+ modality="audio", # editable label
369
+ content_kind=None, # or ["image", "audio"] — the bytes-sniffed kind instead
370
+ limit=48, cursor=None, project=None, collection=None,
371
+ )
372
+ page.items # list[Asset]
373
+ page.next_cursor # pass back as cursor= for the next page, or None
374
+ ```
375
+
376
+ <a name="add-to-the-library"></a>
377
+ `library.add(...)` is the front door for putting an asset into the library — with a file, or fileless:
378
+
379
+ ```python
380
+ asset = client.library.add(
381
+ "path/to/rain.wav", # path, Path, bytes, open file, or an http(s) URL
382
+ modality=None, # override the modality *label* (a search/display facet;
383
+ # embedding follows the bytes, never this)
384
+ description="rain on a tin roof",
385
+ collection=None, # home it in a collection, else the default scope
386
+ )
387
+
388
+ # Fileless — no file, the description *is* the content (the web UI's "Add other"):
389
+ note = client.library.add(description="a warm, relaxed evening mood")
390
+ ```
391
+
392
+ A positional string is always a file path/URL, never a description — pass `description=` as a
393
+ keyword; a fileless add requires it. `client.assets.upload(file, ...)` is the same call for a file.
394
+
395
+ Every media argument across the SDK (`image=`, `audio=`, `video=`, `library.add(file)`) accepts a
396
+ path, `Path`, raw `bytes`, an open binary file, or an `http(s)` URL — normalized by
397
+ `hci_atrium._files`.
398
+
399
+ ### Bulk import
400
+
401
+ For importing many assets at once from a manifest (JSON/CSV) instead of one-by-one uploads,
402
+ `POST /v1/library/import` is a raw multipart endpoint, not (yet) wrapped by the SDK; call it with
403
+ `httpx` directly or `client._request("POST", "/v1/library/import", ...)`.
404
+
405
+ ### Searching external sources
406
+
407
+ Search external providers (Freesound, Openverse, …) for media the library doesn't have yet, then
408
+ import the ones you want. The search is **read-only** — nothing is imported until you call
409
+ `import_`, and every result carries the licence + attribution you need to honour.
410
+
411
+ ```python
412
+ sources = client.sources.list() # which providers are registered
413
+
414
+ result = client.sources.search(
415
+ "rain on a tin roof",
416
+ providers=None, # None = every source; or "freesound" / ["freesound", …]
417
+ k=12, # per-provider cap
418
+ modality="audio", # narrow to sources serving this kind
419
+ )
420
+
421
+ for c in result.results: # provider-tagged Candidates (not yet imported)
422
+ print(c.provider, c.title, c.license, c.preview_url) # c.duration / c.size / c.content_type too
423
+
424
+ for skip in result.skipped: # a source that couldn't answer, and why
425
+ print("skipped", skip.provider, "—", skip.reason) # e.g. missing credentials
426
+
427
+ asset = client.sources.import_(result.results[0]) # pull that candidate into the library
428
+ # or name it yourself: client.sources.import_(source_id="freesound", external_id="12345")
429
+ ```
430
+
431
+ - `search` fans out over every registered source by default; a source that's missing credentials,
432
+ is down, or doesn't serve the ask lands in `result.skipped` (with a reason) instead of failing the
433
+ whole call — so you always get the sources that *did* answer.
434
+ - Each `Candidate` is external-only until imported: `provider` + `external_id` are what
435
+ `import_(candidate)` re-resolves to fetch the bytes. `import_` returns the library `Asset`;
436
+ re-importing the same candidate is a no-op that returns the existing one.
437
+ - `import_` takes an optional `modality=` / `description=` to override what the source reports.
438
+
439
+ ### Chat & agents
440
+
441
+ ```python
442
+ reply = client.chat("what's a good rainy-day soundscape?", model="gpt-4o-mini", temperature=0.7)
443
+ print(reply) # a str with .model / .raw
444
+ for chunk in client.chat("hi", stream=True):
445
+ print(chunk, end="")
446
+
447
+ reply = client.agent("find a calm track and tell me its title", tools=[my_search_tool])
448
+ ```
449
+
450
+ `chat(messages, *, model=, stream=False, strict_params=False, **params)` posts to
451
+ `/v1/chat/completions`; `**params` are extra OpenAI parameters (`temperature`, `max_tokens`,
452
+ `tools`, …) forwarded verbatim. `agent(prompt, *, tools, model=, max_turns=6, **params)` runs a
453
+ tool-using loop, forwarding `**params` to every `chat` call it makes.
454
+
455
+ > **Provider-unsupported params are silently dropped by default.** Atrium routes chat through
456
+ > LiteLLM, which drops a sampling param a model doesn't support (e.g. a non-default `temperature`
457
+ > on an OpenAI "gpt-5"-family model) rather than erroring — matched by **model-name pattern**, so a
458
+ > self-hosted model named like a known family inherits that family's quirks. Pass
459
+ > `strict_params=True` to error instead, with the provider's own message. This only applies to
460
+ > models routed through Atrium/LiteLLM — a model registered via `add_direct_model` bypasses all of
461
+ > it: its request body goes verbatim to the endpoint.
462
+
463
+ ---
464
+
465
+ ## 4. Rooms & sessions
466
+
467
+ **Two kinds of traffic flow through a room.** *Messages* are the named things on the wire — react to
468
+ them by name with `room.on("play")`, `room.on("presence.join")`, `room.on("mood.switch")`; the
469
+ vocabulary is open, anyone can `room.send` any type. *Audio* is the live sound in the room — process
470
+ it with `async for` over the `audio_*` family (`audio_vectors` / `audio_scores` / `audio_transcripts`
471
+ / `audio_clips`), streams the SDK assembles for you from the room's raw `audio.snippet` windows, with
472
+ any refinement (embedding, transcription) running on *your* credentials. Transcripts show the duality
473
+ directly: `room.on("transcript.final")` **consumes** a caption someone else is already broadcasting
474
+ (free — you read the wire), while `room.audio_transcripts()` **computes** your own from the audio.
475
+
476
+ ### Sync: host a room
477
+
478
+ ```python
479
+ from hci_atrium import Atrium
480
+
481
+ client = Atrium()
482
+ room = client.rooms.open(name="living room") # named → stable connect code across runs
483
+ print(room.connect_code)
484
+
485
+ with client, room: # on exit: delete anonymous / leave a named room standing
486
+ for event in room.receive(): # blocks; SSE stream
487
+ if event.source != "host":
488
+ print(event.type, event.payload)
489
+ ```
490
+
491
+ `rooms.open(*, name=None, project=None, embedding_space=None)` — give it a `name` to reuse the same
492
+ room (and connect code) across runs; the `name → room id` mapping is cached in
493
+ `~/.atrium/rooms.json`. Anonymous (`name=None`) opens a fresh room every call. `rooms.get(id)` /
494
+ `rooms.list()` fetch existing rooms.
495
+
496
+ `room.send(type, payload=None, *, to=None, include_self=False, source=None, ephemeral=False,
497
+ ack_id=None, reply_to=None)` — fire-and-forget, never stored. `to` (a connection id, an
498
+ `identity`, or a list) unicasts/multicasts; default fan-out is everyone else. `ephemeral=True`
499
+ marks throwaway noise (beats, cursors) that a recorder skips.
500
+
501
+ `room.receive(*, identity=None, announce=None)` — blocking SSE iterator of `Event`s; sets
502
+ `room.connection_id` from the first `connection.ready` frame. Reconnect yourself on drop (rooms
503
+ have no replay).
504
+
505
+ ### Join without an account
506
+
507
+ ```python
508
+ from hci_atrium import join_room
509
+
510
+ with join_room("ABC123") as room: # connect code, no account, no full client
511
+ room.send("message", {"text": "hi"})
512
+ for event in room.receive():
513
+ ...
514
+ ```
515
+
516
+ ### Sessions (durable log)
517
+
518
+ ```python
519
+ session = client.sessions.start(name="P07 · pilot", participant_ref="P07")
520
+ session.event("note", {"text": "participant arrived"})
521
+
522
+ session = client.sessions.resume(session_id) # continue an existing session by id — raises if unknown
523
+ print(session.last_seq) # last_seq reflects the session as of the fetch — re-`resume()` to refresh it
524
+ ```
525
+
526
+ ### Recording a room into a session
527
+
528
+ ```python
529
+ session = client.sessions.start(name="studio take")
530
+ room = client.rooms.open(name="studio")
531
+
532
+ with room.record(into=session): # starts a background capture thread immediately
533
+ room.send("note", {"text": "session started"})
534
+ room.send("beat", {"bpm": 120}, ephemeral=True) # skipped — ephemeral
535
+ ...
536
+ ```
537
+
538
+ `ephemeral=True` events, presence, and non-`signal` frames are skipped; everything else is
539
+ appended with `via={"room_id", "room_name", "from", "to"}`. A dropped stream reconnects with
540
+ backoff (gaps are lost — rooms have no replay); check `rec.alive` / `rec.error`.
541
+
542
+ ### Reading sessions back (analysis)
543
+
544
+ The write side (`sessions.start`/`resume`, `session.event`) has a read-model twin for analysis:
545
+
546
+ ```python
547
+ for info in client.sessions.list(phase="pilot"): # newest first; phase: a string or a list
548
+ print(info.id, info.participant_ref, info.condition, info.last_seq)
549
+
550
+ info = client.sessions.get(session_id) # one session — has_transcript populated here
551
+ for row in client.sessions.events(session_id, after=0, limit=500):
552
+ print(row.seq, row.type, row.payload, row.source, row.via)
553
+
554
+ client.sessions.export(session_id, "csv", path="out/") # "jsonl"|"csv"|"transcript"|"bundle"
555
+ client.sessions.export_bundle([id_a, id_b], path="out/study.zip") # several sessions, one zip
556
+ ```
557
+
558
+ - `sessions.list(*, project=None, phase=None)` returns `SessionInfo` rows for your default project
559
+ (a scoped key's project, else Personal); `""` as a phase matches sessions with no phase set.
560
+ `has_transcript` is `None` in list results — only `sessions.get(id)` populates it. There is **no
561
+ participant filter server-side**: filter the rows on `participant_ref` yourself.
562
+ - `sessions.get(id)` is the read model; `sessions.resume(id)` hits the same endpoint but returns the
563
+ live `Session` handle for continued logging.
564
+ - `sessions.events(id, *, after=None, limit=None)` pages the durable log as flat, pandas-friendly
565
+ `SessionEvent` rows. `after` is a **sequence number**, not a timestamp (there is no time filter
566
+ server-side); `limit` is 1..500 (server default 100).
567
+ - `export(id, format="jsonl", *, path=None)` returns the raw `bytes` — or, with `path=` (a file, or a
568
+ directory to derive the filename in), writes them and returns the written `Path`. Each format is a
569
+ distinct server route, not a `?format=` param; `"transcript"` 404s when the session has none.
570
+ - `export_bundle(ids, *, path=None)` zips several sessions. `ids` is required (no "all sessions"
571
+ mode) and capped at 100 **distinct** ids — more raises `ValueError` rather than truncating.
572
+
573
+ ### Mirror your app's own activity: `room.trace()`
574
+
575
+ One call mirrors what the SDK is doing on this client into the room as `trace.*` signals — so an
576
+ observation dashboard, or any code-joined peer, can watch the app think:
577
+
578
+ ```python
579
+ with room.trace(): # or: t = room.trace(); …; t.stop()
580
+ client.library.search("calm rain", modality="audio") # -> trace.library.search
581
+ room.play(hit, layer="music", mode="queue") # -> trace.play
582
+ ```
583
+
584
+ - A traced call sends `trace.<label>` (`trace.library.search` / `trace.chat` / `trace.play`) carrying
585
+ a one-line summary plus a capped detail; a recorder starting/stopping sends `trace.state.<label>`.
586
+ - **Default off** — it broadcasts app internals to anyone holding the connect code. Tracing stops on
587
+ `t.stop()` or when the room's `with` exits, and a trace failure never breaks the traced call.
588
+ - Ephemerality is per-signal: `trace.library.*` / `trace.chat` are **durable** (they happen outside
589
+ the room, so the trace is their only record — a recorder captures them into the session), while
590
+ `trace.play` and `trace.state.*` are ephemeral (the underlying `play` already rides the room).
591
+ - Async: `room.trace()` is still a plain call, but scope it with `async with room.trace(): …`.
592
+
593
+ ### Async: `on`/`serve`, presence, targeted request/reply
594
+
595
+ The async room adds an event-dispatch style on top of the same transport — register a handler per
596
+ signal type, then `serve()` to run until cancelled:
597
+
598
+ ```python
599
+ import asyncio
600
+ from hci_atrium import AsyncAtrium
601
+
602
+ async def main() -> None:
603
+ async with AsyncAtrium() as client:
604
+ room = await client.rooms.open(name="lab")
605
+ print(room.connect_code)
606
+
607
+ @room.on("presence.join")
608
+ def on_join(event):
609
+ print("joined:", event.payload.get("identity"))
610
+
611
+ @room.on("ping")
612
+ async def on_ping(event):
613
+ return {"pong": True} # only sent back if the ping was a `request(...)`
614
+
615
+ await room.connect(identity="host")
616
+ await room.serve() # blocks; reconnects transient drops with backoff
617
+
618
+ asyncio.run(main())
619
+ ```
620
+
621
+ - `await room.connect(*, identity=None, announce=None)` joins live and starts the dispatcher,
622
+ returning the connection id. `presence.join`/`presence.leave` arrive as ordinary signals — just
623
+ register handlers for them.
624
+ - `await room.request(type, payload=None, *, to, timeout=None)` sends to one peer and awaits its
625
+ single reply (the peer's handler return value) — `to` is required. Raises `TimeoutError` on no
626
+ reply within `timeout` (default: the room's request timeout).
627
+ - A handler's return value becomes the auto-reply when the *incoming* signal itself was a
628
+ `request(...)` (carried an `ack_id`) — a responder never touches correlation plumbing.
629
+ - Don't mix `on`/`serve` with `receive()` on the same room — each opens its own stream (a second
630
+ connection, a second presence join).
631
+
632
+ ### Live audio vectors (async)
633
+
634
+ If a Player in the room has its **"Share audio clips"** toggle on (default off — it shares actual
635
+ audio, so consent lives on the device making sound), you can get a live semantic embedding of what
636
+ the room currently sounds like — ~10 s windows, embedded in the deployment's default audio search
637
+ space, so each vector is directly comparable against library-derived vectors:
638
+
639
+ ```python
640
+ async for v in room.audio_vectors(): # zero-args: default audio space, that space's model
641
+ score = cosine(v.vector, tense_anchor) # v: AudioVector(vector, space, sequence,
642
+ await room.send("mood.update", {"score": score}) # window_seconds, timestamp)
643
+ ```
644
+
645
+ - Subscribing **is** the demand signal: while no consumer is subscribed, an armed Player captures
646
+ nothing. Break out of the loop to end the subscription.
647
+ - **The loop blocks until a Player shares audio.** If no participant is already announcing the
648
+ snippet-producer role at subscribe time, one warning is logged and the loop keeps waiting (no
649
+ timeout) — nothing yields until someone turns their Player's "Share audio clips" toggle on.
650
+ - `space=` pins a different space; `model=` overrides the embedder (rarely needed — by default the
651
+ space's own model is resolved once at subscribe). Embedding runs on *your* credentials/compute.
652
+ - Vectors are ephemeral room signals, never stored. Record the room into a session if a study
653
+ needs a durable trace.
654
+ - `relay=True` also relays each vector into the room as an `audio.vector` signal
655
+ (`ephemeral=True`, recorder-skipped) — so a loginless, code-joined peer (e.g. a Unity scene) can
656
+ read the room's live mood vector without credentials of its own
657
+ (`docs/reference/rooms-sessions-protocol.md`).
658
+
659
+ `room.audio_scores(anchors, *, space=None, model=None, top_n=1, relay=False)` fuses
660
+ `audio_vectors()` with per-anchor `library.compare` scoring — a live "which anchor does the room
661
+ sound like right now" signal, no manual cosine math:
662
+
663
+ ```python
664
+ async for frame in room.audio_scores({"tense": "a tense, anxious atmosphere", "calm": "calm and peaceful"}):
665
+ print(frame.scores) # {"tense": 0.71, "calm": 0.22} — calibrated per-anchor compare scores
666
+ print(frame.mix) # scores normalized to sum 1 (uniform when every score is 0 — gate on
667
+ # max(frame.scores) to tell "no match" from "anchors tied")
668
+ ```
669
+
670
+ - `anchors` maps a name to anything the `by.*` grammar accepts (bare `str`/`Asset`/`list[float]`,
671
+ `by.file(...)`/`by.asset(...)`/`by.concept(...)`) — **or a list** of those, a few-shot category
672
+ (e.g. `"tense": ["a tense, anxious atmosphere", "a horror movie soundtrack"]`) scored as the mean
673
+ of its **top-`top_n`** members (default `top_n=1` — the nearest member wins, so spread-out
674
+ examples don't blur into an average; pass a larger `top_n` to widen back toward the plain mean —
675
+ `top_n >= len(members)` averages every member).
676
+ - Every window scores its anchors concurrently (`asyncio.gather`); a handful of `compare` calls per
677
+ ~10 s window is cheap.
678
+ - Every anchor is validated once, before waiting on any frame: a typo'd/unresolvable `by.concept(…)`
679
+ anchor raises `AtriumError` naming it immediately, rather than only failing mid-stream.
680
+ - `relay=True` forwards straight to `audio_vectors` — it relays each window's raw
681
+ `audio.vector` signal (see above), **not** the computed `scores`/`mix` (peers compare via the
682
+ embed broker instead).
683
+ - No smoothing: every `ScoredAudio` frame is independent. Ease/debounce the stream yourself —
684
+ `hci_atrium.reactive.Decider` is a small, framework-free helper for turning a live score dict into
685
+ a stable "current winner":
686
+
687
+ ```python
688
+ from hci_atrium import Decider
689
+
690
+ decider = Decider(margin=0.1, dwell=2.0) # a new winner must beat the current one by 0.1,
691
+ # and stay switched for at least 2s
692
+ async for frame in room.audio_scores(anchors):
693
+ winner = decider.update(frame.scores) # the new winner's name, or None if unchanged
694
+ if winner is not None:
695
+ await room.send("mood.switch", {"to": winner})
696
+ ```
697
+
698
+ `decider.current` holds the standing winner between updates; with two anchors this is a plain
699
+ binary hysteresis switch.
700
+
701
+ ### Live transcripts (async)
702
+
703
+ `room.audio_transcripts()` is the speech-to-text counterpart of `audio_vectors()` — it consumes the
704
+ same Player-shared `audio.snippet` stream, but transcribes the audio into **live captions** instead
705
+ of embedding it, yielding a `Transcript` (a `str` subclass) per **finalized sentence**:
706
+
707
+ ```python
708
+ async for t in room.audio_transcripts(): # zero-args: deployment default model
709
+ print(str(t)) # a settled sentence; t.raw has start/end
710
+ ```
711
+
712
+ - Same demand gate as `audio_vectors`: subscribing announces the `audio-snippet-consumer` role, so
713
+ an armed Player only captures while a consumer is subscribed; break out of the loop to end it (the
714
+ stream tears down deterministically on `break`). The same one-time "no Player is sharing audio"
715
+ warning fires if nobody is producing yet.
716
+ - **Streaming by default.** When the deployment offers the live transcription route, snippets flow
717
+ through it and you get per-sentence finals as the speech settles — each with `.final == True` and
718
+ `t.raw` carrying the sentence's `start`/`end`. Pass `partials=True` to *also* get the revisable
719
+ live hypothesis (`.final == False`); a partial **replaces** the previous one wholesale, so render
720
+ the latest, don't append.
721
+ - `model=` / `language=` pick the transcription model and language hint (omit `language` to
722
+ autodetect; omit `model` for the deployment default). Transcription runs on *your*
723
+ credentials/compute. An empty-text sentence is dropped unless `skip_empty=False`; a snippet the
724
+ server can't transcribe is skipped so one bad frame can't kill the stream, but a rejected
725
+ credential (401/4401) raises `AuthError`.
726
+ - **Degradation ladder.** If the streaming route can't be reached — an old deploy, a proxy that
727
+ strips the WebSocket upgrade, a refused connection — this degrades to per-clip batch
728
+ transcription: one independent transcript per ~10 s window, **no partials**, the clip's format
729
+ taken from each frame's own MIME type (the Player ships Opus-in-WebM). On *this* fallback path
730
+ each transcript carries its source frame's `sequence` and `window_seconds` under
731
+ `t.raw["snippet"]` — a cheap ordering handle that has no equivalent on the streaming path (there's
732
+ no 1:1 snippet↔sentence correlation). The fallback is **connect-time only**: once streaming is
733
+ live a mid-session drop *raises* rather than silently degrading.
734
+
735
+ **Rooms-free — `client.transcribe_live(chunks, ...)`.** Not everything lives in a room. This runs
736
+ the same streaming engine over any async iterable of audio byte-chunks (a mic, a file, a socket),
737
+ yielding the same finals (and `partials=True` partials):
738
+
739
+ ```python
740
+ async for t in client.transcribe_live(mic_chunks(), format="pcm16", sample_rate=16000):
741
+ print(str(t))
742
+ ```
743
+
744
+ - **Container clips** (`format="wav"` default, or `"webm"`/`"ogg"`/…): each chunk is a
745
+ self-contained, decodable clip — the browser `MediaRecorder` / room-snippet shape.
746
+ - **Raw PCM** (`format="pcm16"`): a continuous PCM16LE **mono** byte stream, the shape native mic
747
+ libraries (`sounddevice`/`pyaudio`) yield. This **requires** `sample_rate=` — raw PCM is
748
+ headerless, so the SDK can't sniff the capture rate. Passing `sample_rate=` with a container
749
+ format is an error (containers carry their own rate).
750
+
751
+ **In-room broker — `room.transcribe()`.** The broker form consumes the room's snippet stream on
752
+ **your** credentials and publishes the transcript back into the room as signals, so any consumer —
753
+ including a loginless, code-joined tool — reads captions without transcription access of its own:
754
+
755
+ ```python
756
+ async with room.transcribe(): # a side effect: starts immediately, stops on room exit
757
+ ...
758
+
759
+ # any consumer, even a code-joined peer with no credentials of its own:
760
+ @room.on("transcript.final")
761
+ def on_line(event):
762
+ print(event.payload["seq"], event.payload["text"])
763
+ ```
764
+
765
+ - `transcript.partial` — `{"text": …}`, **ephemeral** (a live caption the recorder skips).
766
+ Wholesale-replaced — render the latest, don't append.
767
+ - `transcript.final` — `{"text": …, "start": …, "end": …, "seq": n}`, durable. `seq` is a per-handle
768
+ counter from 0; the signal's `source` is `"transcriber"`. Bridge the room into a session
769
+ (`room.record(into=session)`) and each final is persisted — no new recording API, the `ephemeral`
770
+ flag alone drives the recorder split.
771
+ - Like `record`, it's a side effect: started in the background, stopped when the room's `async with`
772
+ exits — or `await handle.stop()` / scope it with `async with room.transcribe(): …`. See
773
+ `examples/live_transcript.py` for the manual live-test recipe.
774
+
775
+ ### Live audio clips (async)
776
+
777
+ `room.audio_clips()` is the rawest member of the family — it hands you each ~10 s window decoded and
778
+ nothing else (no embedding, no transcription — cost: none), for custom processing:
779
+
780
+ ```python
781
+ async for clip in room.audio_clips(): # zero-args
782
+ do_something(clip.data, clip.mime) # AudioClip(data, mime, sequence,
783
+ # window_seconds, source, source_name)
784
+ ```
785
+
786
+ Same demand gate and one-time warning as `audio_vectors`. Producer-agnostic: a Player (v1 frames,
787
+ `clip.source is None`) or a Capture tool (v2 frames — `clip.source` is the sender's connection id,
788
+ `clip.source_name` its announced name, resolved from the subscribe-time roster).
789
+
790
+ ### Share your embedding / chat access: `offer_embeds` / `offer_chat` (async)
791
+
792
+ Let other room participants use your embedding/chat access — they ask, your client answers. A
793
+ loginless, code-joined peer (e.g. a Unity scene) has no Atrium credentials of its own. Register one
794
+ of these on an already-authed room **before** `connect()`/`serve()` — the offer is only advertised
795
+ when the connection opens, so calling either after connecting raises `AtriumError`. Once registered,
796
+ it answers that peer's requests over the room — riding the ordinary `on`/`request` machinery, no new
797
+ transport:
798
+
799
+ ```python
800
+ room.offer_embeds(spaces=["clip"]) # spaces=None (default): unrestricted
801
+ room.offer_chat(model="gpt-4o-mini", max_tokens=200) # model=None: the deployment/client default
802
+ await room.connect(identity="broker")
803
+ await room.serve()
804
+ ```
805
+
806
+ - `offer_embeds` handles `embed.resolve`: `{anchors: {name: <descriptor>}, space?}` →
807
+ `{space, vectors: {name: [floats]}, errors?}`. A descriptor is `{"text": "…"}` /
808
+ `{"concept": "name", "collection"?: "…"}` / `{"vector": [...]}` — the same shapes the `by.*`
809
+ operand grammar builds for `compare`. **The requester chooses `space`** (omitted → the
810
+ deployment's default text space); `spaces=` optionally allowlists what this broker serves. A
811
+ failing anchor (unknown concept, an unsupported `asset` descriptor) lands in `errors` without
812
+ failing the rest; a malformed/oversized request or a disallowed space gets a whole-request
813
+ `{"error": "…"}` reply. Text is embedded with *this* client's own credentials; repeated
814
+ text/concept anchors are cached.
815
+ - `offer_chat` handles `chat.generate`: `{messages: [{role, content}, …]}` → `{text, model}`.
816
+ Stateless (no history kept). The **broker** pins `model` (cost control — the requester can't
817
+ choose it) and caps `max_tokens`; malformed/empty/over-sized `messages` get `{"error": "…"}`.
818
+ **Billing**: any peer holding the connect code can trigger a reply on your model billing (there's
819
+ a per-call `max_tokens` cap, but no aggregate limit across calls) — rotate the code for an open
820
+ study, or prefer a local model, if that's a concern.
821
+ - Both announce a discoverable role in the presence roster (`announce.roles`) —
822
+ `"embed-broker"` / `"chat-broker"` — so a code-joined peer can find them. The full wire dialect
823
+ (including the `audio.vector` info signal from `relay=True` above) is in
824
+ `docs/reference/rooms-sessions-protocol.md`.
825
+
826
+ ### Offer live captions + recording to a Capture tool: `offer_transcription` / `offer_audio_capture` (async)
827
+
828
+ The Capture tool (`/tools/capture/?code=…`) is a loginless voice surface. The host publishes two
829
+ standing-service offers; the tool's toggles turn them on/off over the room (`<service>.start` /
830
+ `<service>.stop`, each replied `{"ok": True}` / `{"error": …}`). Register both **before**
831
+ `connect()`/`serve()` (like the brokers above):
832
+
833
+ ```python
834
+ async with room.record(into=session): # the single durable binding — see below
835
+ room.offer_transcription() # "Live captions" toggle → transcript.* signals
836
+ room.offer_audio_capture() # "Save recording" toggle → audio into the session
837
+ await room.serve()
838
+ ```
839
+
840
+ - `offer_transcription(model=None, language=None)` runs the same engine as `room.transcribe()` on
841
+ *your* credentials while on, publishing `transcript.partial` (ephemeral) / `transcript.final`
842
+ (durable) — each stamped with the producer's `source` (connection id) and roster-resolved
843
+ `source_name`. Bare call is the happy path (deployment default model, language autodetect).
844
+ - `offer_audio_capture()` is **session-less**: `room.record(into=session)` is the single durable
845
+ binding, and the capture bridge writes recordings into **the active recorder's** session. So the
846
+ "Save recording" toggle is meaningful only while the room is being recorded, and availability
847
+ rides the wire so the tool can show it accordingly:
848
+ - a `capture.status` request replies `{"ok": True, "available": <bool>}` — available ⇔ ≥1 recorder
849
+ is active on this room handle;
850
+ - every recorder start/stop broadcasts an ephemeral `capture.available` `{"available": <bool>}`;
851
+ - `capture.start` with **no** active recorder replies
852
+ `{"error": "no active recorder — call room.record(into=session) first"}`.
853
+ On `capture.start` it binds the **first** active recorder's session and bridges each `audio.snippet`
854
+ frame into one recording per source (keyed by announced `name`, else connection id — a same-name
855
+ reconnect continues the same recording). When that recorder stops (explicit stop, the room's
856
+ `async with` exit, or room close) every open recording is finalized, the bridge disengages, and
857
+ `capture.available: false` is broadcast. **Exactly one recorder feeds audio** — a second concurrent
858
+ recorder warns once and is not double-written.
859
+ - See `examples/capture_session.py` for the whole shape (recording off until you flip the toggle).
860
+
861
+ ### Driving the Player's audio layers
862
+
863
+ The hosted Player (`/tools/player/?code=…`) plays sustained, mixable **layers** — background music,
864
+ ambient beds, SFX. A layer is just a named, independently-mixed channel (its own volume + effects);
865
+ every layer can hold both a single-track **queue** and overlapping **one-shots** at once — the
866
+ difference is `mode`, not the layer. `Room` and `RoomParticipant` carry the app side of the
867
+ `play` / `layer.*` dialect (`docs/reference/audio-layers-dialect.md`):
868
+
869
+ ```python
870
+ room.configure_layers([ # optional — layers auto-vivify on first use
871
+ {"id": "music", "label": "Music", "volume": 0.5},
872
+ {"id": "ambient", "crossfade_ms": 3000},
873
+ {"id": "sfx"},
874
+ ])
875
+ # play() takes an Asset / SearchHit, a content-URL or asset-id string, a track dict, or a list:
876
+ room.play(tavern_song, layer="music", mode="queue", transition="now") # crossfade a song in
877
+ room.play({"asset_id": rain.id, "loop": True}, layer="ambient", mode="queue") # looping bed
878
+ room.play(beds, layer="ambient", mode="queue", repeat=True, shuffle=True) # endless shuffled ambience
879
+ room.play(clang, layer="sfx") # one-shot (mode defaults to oneshot)
880
+ room.play([hit_a, hit_b], layer="music", mode="queue", transition="queued") # queue several
881
+ room.play({"asset_id": clang.id, "delay_seconds": 0.2, "spatial": {"x": -1, "y": 0, "z": 0}}, layer="sfx")
882
+
883
+ room.clear_layer(layer="ambient", fade_out_ms=2000) # one layer (queue + one-shots)
884
+ room.clear_all(fade_out_ms=1000) # stop all audio
885
+ room.set_layer_effects("music", {"lowpass": {"hz": 700}}, fade_ms=600) # backroom muffle
886
+ room.set_layer_effects("ambient", {"reverb": {"mix": 0.4, "ir": "cave"}, # wet convolution send
887
+ "echo": {"time_ms": 330, "feedback": 0.35}}) # + a slapback
888
+ room.set_layer_effects("*", {}, fade_ms=800) # every existing layer back to neutral
889
+ room.set_layer_position("music", 0, 0, -2, fade_ms=800) # place the whole layer 2 m in front (HRTF)
890
+ state = room.query_layers(timeout=10.0) # {"layers": {"music": {...}, ...}} — blocking pull
891
+
892
+ for snap in room.watch_layers(): # subscribe: a fresh snapshot on every change
893
+ music = snap["layers"].get("music", {})
894
+ if music.get("remaining_ms", 0) < 60_000 and music.get("queued_count", 0) < 2:
895
+ room.play(next_track, layer="music", mode="queue", transition="queued") # refill ahead of silence
896
+ ```
897
+
898
+ `play(source, *, layer=None, mode="oneshot", transition="now", repeat=False, shuffle=False)`.
899
+ `repeat`/`shuffle` are queue-mode modes — `repeat` loops the whole queue, `shuffle` randomizes its
900
+ order (combinable). A track dict carries per-sound options — `loop` (a single sustained track,
901
+ distinct from queue `repeat`), `segment` (a stored sound-event id),
902
+ `clip_start_seconds`/`clip_end_seconds`, `gain`, `spatial`, and (one-shot)
903
+ `delay_seconds`/`max_duration_seconds`/`fade_in_ms`/`fade_out_ms`.
904
+
905
+ `set_layer_effects(layer, effects, *, fade_ms=None)` sets the layer's whole effect bus at once —
906
+ `lowpass`/`highpass` (filters), `reverb` (a **wet send**: `{"mix": 0.4, "ir": "hall"}`, `ir` one of
907
+ `room`/`hall`/`cave`/`outdoor`, or `ir_asset` an asset id to convolve with your own impulse
908
+ response), and `echo` (`{"time_ms": 330, "feedback": 0.35, "mix": 0.25}`). It's the state, not a
909
+ patch: an **absent** implemented key ramps that effect back to neutral, so `{}` clears the bus.
910
+ `layer="*"` applies the same state to **every existing** layer (it doesn't auto-vivify new ones), the
911
+ same wildcard `clear_all` uses.
912
+
913
+ `set_layer_position(layer, x, y, z=0.0, *, fade_ms=None, distance=None)` places a whole layer in 3D
914
+ (HRTF panning of everything on it) — the listener sits at the origin, `+x` is right, `+y` up, and
915
+ **negative `z` is in front**. The panner is created lazily on the first call, so a layer you never
916
+ position keeps the plain, unpanned graph. `distance=` opts into distance attenuation
917
+ (`{"model": "inverse", "ref": 1, "max": 40, "rolloff": 1}`; `model` one of
918
+ `linear`/`inverse`/`exponential`) and is **sticky** — a later move that omits it keeps the model, and
919
+ `{"rolloff": 0}` goes back to direction-only. There is no `"*"` wildcard: position is per-layer. A
920
+ per-sound `spatial` (`{"x", "y", "z", "distance"?}` on a track dict) composes *within* a positioned
921
+ layer — a placed sound inside a placed group — and a per-sound `effects` dict stacks in front of the
922
+ layer's bus rather than exempting the sound from it (for a sound that must stay dry, play it on its
923
+ own clean layer). Full dialect: `docs/reference/audio-layers-dialect.md`.
924
+
925
+ The Player **loudness-normalizes** library audio to a consistent level by default (a per-track gain
926
+ from the asset's measured loudness toward −18 LUFS, plus a master limiter). Playing an `Asset` /
927
+ `SearchHit` attaches the measured loudness automatically, so it just works; the listener can turn it
928
+ off (Settings / `?normalize=off`).
929
+ `query_layers` opens a short-lived room connection (peers see a presence join/leave) and returns the
930
+ Player's real snapshot — current track, queue, one-shot count, elapsed/remaining per layer.
931
+ `watch_layers` subscribes and yields that same snapshot on every change (plus a ~1 s heartbeat while
932
+ playing) until you break out — so a soundscape loop refills a low queue before it runs dry. The
933
+ low-water threshold is yours to pick off the snapshot; the Player pushes raw state. The **async**
934
+ client has identical signatures — `await room.play(...)`, `await room.clear_all()`,
935
+ `await room.query_layers()` (a broadcast request, no `to=` needed; first Player to reply wins), and
936
+ `async for snap in room.watch_layers():`.
937
+
938
+ Need a Player to drive? Launch the hosted one and only open a tab if none is connected:
939
+
940
+ ```python
941
+ if not room.has(identity_prefix="player:"): # per-worker roster check (best-effort)
942
+ room.open_player() # opens the hosted Player tab
943
+ print(room.player_url()) # the plain launch URL — print / hand out / QR-encode
944
+ room.open_player(analysis=False, clips=True) # bake the room-sharing opt-ins into the URL
945
+ url = room.player_url(clips=True) # …or just build the URL with those params
946
+ ```
947
+
948
+ `room.participants()` reads the room's current presence roster (a short observe stream) as typed
949
+ `Participant`s — `connection_id`, `identity`, `name`, `roles` (the announce's `role`/`roles` shapes
950
+ merged), `connected_at`, and the original `raw` dict. `room.has(role=None, *, name=None,
951
+ identity_prefix=None)` is the boolean sugar over it: give at least one predicate (several are ANDed)
952
+ — `role` matches the merged roles, `name` the announced name, `identity_prefix` the identity's
953
+ leading segment. A hosted Player announces identity `player:web` (no role), so
954
+ `room.has(identity_prefix="player:")` is the launch-once guard. Both are best-effort per web worker.
955
+
956
+ `player_url(*, analysis=None, clips=None)` and `open_player(*, analysis=None, clips=None)` take the
957
+ Player's URL settings as typed kwargs (`bool` → `on`/`off`; `None` omits, keeping the Player default).
958
+ (async: `await room.participants()` / `if not await room.has(identity_prefix="player:"):
959
+ room.open_player()` — `open_player`/`player_url` stay sync.)
960
+
961
+ The observation dashboard has the symmetric pair: `room.dashboard_url()` builds
962
+ `<base>/tools/dashboard?code=<code>` (the dashboard takes only the connect code) and
963
+ `room.open_dashboard()` opens it in a tab — both sync, on the sync and async clients alike.
964
+
965
+ The hosted Viewer is the visual sibling — put an image (or video) on a shared screen over the room:
966
+
967
+ ```python
968
+ room.show(asset_id="…", caption="Rainy market") # or url="https://…", projection="360"
969
+ room.clear_view() # blank the stage
970
+ room.open_viewer() # launch a viewer tab (or print room.viewer_url())
971
+ ```
972
+
973
+ `show` takes exactly one of `asset_id` / `url`; `viewer_url()` / `open_viewer()` mirror the Player
974
+ pair (both sync).
975
+
976
+ ---
977
+
978
+ ## 5. Surveys
979
+
980
+ A survey is a durable data object on a session, guarded by a one-time capability token:
981
+
982
+ ```python
983
+ survey = client.surveys.start(into=session) # uses the project's DEFAULT template
984
+ # or: client.surveys.start("onboarding-v2", into=session, name="intro survey", return_to="https://...")
985
+
986
+ survey.token # once-only capability: read+write exactly this object
987
+ survey.url # hosted survey tool URL, token in the fragment — hand to the participant
988
+ survey.open() # open `survey.url` in the local browser (same-machine delivery)
989
+ ```
990
+
991
+ `Surveys.start(template=None, *, into, name=None, return_to=None)` — `template=None` pins the
992
+ project's default template; naming one pins *that* template's current version, so later edits to
993
+ the template never change an already-started survey. `return_to` seeds an `app`-controlled return
994
+ button on the survey's end page.
995
+
996
+ Manage templates (project-scoped, immutable numbered versions) — safe to rerun every script start:
997
+
998
+ ```python
999
+ client.surveys.template(
1000
+ "onboarding-v2",
1001
+ {"pages": [...]}, # dialect: docs/reference/survey-template-dialect.md
1002
+ project="p_4mD2qX", # or rely on the client's default project scope
1003
+ default=False,
1004
+ ) # -> {"name", "version", "is_default"} — idempotent, appends a new
1005
+ # version only if the body actually changed
1006
+
1007
+ client.surveys.templates(project="p_4mD2qX") # list: name, current_version, is_default
1008
+ client.surveys.upload_file("logo.png", project="p_4mD2qX", role="attachment")
1009
+ # -> {"id", "url", "filename", "content_type"} — reference `id` from
1010
+ # a template body's `image` field
1011
+ ```
1012
+
1013
+ Read the answers back for analysis, by template **name** (not a started `Survey`), across the
1014
+ project's sessions:
1015
+
1016
+ ```python
1017
+ rows = client.surveys.responses("onboarding-v2", phase="post", submitted_only=True, limit=500)
1018
+ for r in rows:
1019
+ print(r.participant_ref, r.session_id, r.status, r.version, r.contents) # contents = raw answers
1020
+
1021
+ csv = client.surveys.responses("onboarding-v2", format="csv") # bytes: one row per response
1022
+ ```
1023
+
1024
+ `responses(template, *, project=None, phase=None, submitted_only=False, limit=200, format="rows")` —
1025
+ newest first, resolved with `project=` exactly like `template`/`templates`. `phase` (a string or a
1026
+ list; `""` matches unset) filters by the response's session phase; `submitted_only` drops drafts;
1027
+ `limit`'s server ceiling is 500. `format="rows"` gives `SurveyResponse` rows (`contents` stays a free
1028
+ dict — the survey dialect is your app's); `format="csv"` gives the CSV export as `bytes` with
1029
+ question-id columns.
1030
+
1031
+ `start_survey(template=None, *, into, name=None, return_to=None)` (module-level) is sugar for
1032
+ `into._client.surveys.start(...)` — handy when you only have the session in hand.
1033
+
1034
+ **Async** mirrors every method 1:1 (`await client.surveys.start(...)`, `await async_start_survey(...)`).
1035
+
1036
+ ---
1037
+
1038
+ ## 6. Home Assistant bridge
1039
+
1040
+ `hci_atrium.ha.HomeAssistantBridge` runs near your HA instance, joins an already-open room, and
1041
+ declaratively wires room signals to HA services (and HA state back into the room). It's
1042
+ **async-only**. `websockets` ships as a core dependency — nothing extra to install.
1043
+
1044
+ ```python
1045
+ import asyncio
1046
+ from hci_atrium import async_join_room
1047
+ from hci_atrium.ha import HomeAssistantBridge
1048
+
1049
+ async def main() -> None:
1050
+ room = await async_join_room("ABC123")
1051
+ bridge = (
1052
+ HomeAssistantBridge("ws://homeassistant.local:8123/api/websocket", "***token***")
1053
+ .media_sink("media_player.living_room", "media_player.kitchen") # play `play` intents
1054
+ .on_signal( # any other signal -> any HA service
1055
+ "plate.pulse", domain="number", service="set_value",
1056
+ entity_id="number.plate_intensity", data=lambda p: {"value": p["strength"]},
1057
+ )
1058
+ )
1059
+ try:
1060
+ await bridge.run(room) # blocks until cancelled
1061
+ finally:
1062
+ await room.aclose()
1063
+
1064
+ asyncio.run(main())
1065
+ ```
1066
+
1067
+ - **`media_sink(*entity_ids, forward_state=True)`** — plays the room's `play` intent (the typed
1068
+ `{playback: {source, ref, mode?}}` shape, or the legacy `content_url`/`asset_id` shapes) via
1069
+ `media_player.play_media` (+ `.shuffle_set` for a shuffled playlist) on each entity. Also
1070
+ forwards those entities' state back as `media.state` unless `forward_state=False`.
1071
+ - **`on_signal(type, *, domain, service, entity_id=None, data=None)`** — maps one room signal type
1072
+ to one HA service call; `data` is a static dict or `callable(payload) -> dict`. Several
1073
+ registrations for the same `type` all fire.
1074
+ - **`forward_state(entity_ids, *, as_type="media.state", debounce=True)`** — forwards HA
1075
+ `state_changed` on these entities into the room. Call it directly for entities outside
1076
+ `media_sink`, or to change `as_type`/`debounce`.
1077
+ - **`await bridge.run(room)`** — wires everything, announces the bridge's identity, and runs both
1078
+ legs (room dispatcher + HA connection) until cancelled. Raises only on a terminal room refusal
1079
+ (a 4xx); the HA leg reconnects with backoff on its own.
1080
+
1081
+ **Zero-code CLI**, once the room exists:
1082
+
1083
+ ```sh
1084
+ atrium ha-bridge --code ABC123 \
1085
+ --ha-url ws://homeassistant.local:8123/api/websocket --ha-token *** \
1086
+ --media-player media_player.living_room media_player.kitchen
1087
+ ```
1088
+
1089
+ Every flag also reads an env var fallback: `ATRIUM_ROOM_CODE`, `HA_URL`, `HA_TOKEN`,
1090
+ `HA_MEDIA_PLAYERS` (comma-separated).
1091
+
1092
+ ---
1093
+
1094
+ ## 7. Recipes
1095
+
1096
+ ### Search the library and play the top hit into a room (sync)
1097
+
1098
+ ```python
1099
+ from hci_atrium import Atrium
1100
+
1101
+ client = Atrium()
1102
+
1103
+ with client, client.rooms.open(name="living room") as room:
1104
+ print(f"connect code: {room.connect_code}")
1105
+ hits = client.library.search("calm rainy afternoon", modality="audio", limit=1)
1106
+ if hits:
1107
+ asset = hits[0].asset
1108
+ room.send("play", {"playback": asset.playback.as_signal()})
1109
+ print(f"playing: {asset.title or asset.filename}")
1110
+ # on exit: leaves the named room standing (code kept for next run), then closes the client
1111
+ ```
1112
+
1113
+ ### Record a study session with a survey
1114
+
1115
+ ```python
1116
+ from hci_atrium import Atrium
1117
+
1118
+ client = Atrium()
1119
+ session = client.sessions.start(name="P07 · pilot", participant_ref="P07")
1120
+ session.event("phase", {"name": "intro"})
1121
+
1122
+ survey = client.surveys.start(into=session) # the project's default template
1123
+ print(f"send this to the participant:\n {survey.url}")
1124
+ survey.open() # or: hand survey.token / survey.url to a participant-facing device
1125
+
1126
+ session.event("phase", {"name": "survey_sent"})
1127
+ print(f"session {session.id} — {client.sessions.resume(session.id).last_seq} events so far")
1128
+ ```
1129
+
1130
+ ### A room consumer that reacts to signals and replies to requests (async)
1131
+
1132
+ ```python
1133
+ import asyncio
1134
+ from hci_atrium import async_join_room
1135
+
1136
+ async def main() -> None:
1137
+ async with await async_join_room("ABC123") as room:
1138
+
1139
+ @room.on("play")
1140
+ async def on_play(event):
1141
+ intent = event.payload.get("playback")
1142
+ print("play:", intent)
1143
+
1144
+ @room.on("ping")
1145
+ async def on_ping(event):
1146
+ return {"pong": True} # answers a room.request("ping", to=...)
1147
+
1148
+ await room.connect(identity="visuals")
1149
+ await room.serve() # blocks; reconnects transient drops
1150
+
1151
+ asyncio.run(main())
1152
+ ```
1153
+
1154
+ ### Record a room into a session, skipping the noise
1155
+
1156
+ ```python
1157
+ import time
1158
+ from hci_atrium import Atrium
1159
+
1160
+ client = Atrium()
1161
+ room = client.rooms.open(name="studio")
1162
+ session = client.sessions.start(name="studio take")
1163
+ print(f"connect code: {room.connect_code} session: {session.id}")
1164
+
1165
+ with room.record(into=session):
1166
+ room.send("note", {"text": "session started"})
1167
+ room.send("beat", {"bpm": 120}, ephemeral=True) # not recorded — ephemeral
1168
+ time.sleep(30) # capture whatever participants send
1169
+
1170
+ print(f"recorded {client.sessions.resume(session.id).last_seq} events")
1171
+ ```
1172
+
1173
+ ---
1174
+
1175
+ ## Left out / not verified here
1176
+
1177
+ - Bulk import (`POST /v1/library/import`) has no SDK wrapper; only the raw endpoint is documented.
1178
+ - Collection *creation/management* (the tree itself) is not an SDK operation — only scoping to an
1179
+ existing collection id.
1180
+ - `hci_atrium.dash.Dashboard` (the observation-dashboard helper) and the SFX/agent-tooling pieces
1181
+ (`client.tool`, `client.agent`, direct-model routing) exist in the SDK but are out of scope for
1182
+ this guide — see the source docstrings and `README.md` for those.