@ateam-ai-mobile/plugin-runtime 0.1.5 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/chunk-PKDQLNBL.mjs +2873 -0
- package/dist/chunk-PKDQLNBL.mjs.map +1 -0
- package/dist/index.d.mts +1759 -8
- package/dist/index.d.ts +1759 -8
- package/dist/index.js +2467 -1321
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +335 -2139
- package/dist/index.mjs.map +1 -1
- package/dist/loader-5LUVXUY2.mjs +9 -0
- package/dist/loader-5LUVXUY2.mjs.map +1 -0
- package/package.json +2 -2
- package/src/bundles/bundleManager.ts +368 -27
- package/src/bundles/integrity.ts +29 -0
- package/src/bundles/storage.ts +40 -0
- package/src/deviceStateHost.ts +197 -0
- package/src/hooks/usePlugin.ts +1 -1
- package/src/host.ts +80 -8
- package/src/index.ts +90 -1
- package/src/pluginRuntime.ts +97 -0
- package/src/sdk/CameraPreview.tsx +94 -0
- package/src/sdk/device-state.ts +1225 -0
- package/src/sdk/index.ts +35 -3
- package/src/sdk/registration.ts +39 -0
- package/src/sdk/types.ts +345 -8
- package/src/sdk/voice.ts +148 -0
- package/src/tools/loader.ts +167 -0
- package/src/tools/platformTools.ts +41 -0
- package/src/tools/registry.ts +183 -0
- package/src/tools/serverCalls.ts +78 -0
- package/src/tools/sessionSync.ts +107 -0
- package/src/voiceHost.ts +126 -0
|
@@ -0,0 +1,1225 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @adas/device-state — Device Bridge v2, the SEMANTIC layer.
|
|
3
|
+
*
|
|
4
|
+
* Intended repo path:
|
|
5
|
+
* packages/plugin-runtime/src/sdk/device-state.ts
|
|
6
|
+
* Intended module specifier for plugin bundles:
|
|
7
|
+
* '@adas/device-state' (NOT '@adas/plugin-sdk')
|
|
8
|
+
*
|
|
9
|
+
* ─────────────────────────────────────────────────────────────────────────
|
|
10
|
+
* WHICH LAYER AM I IN?
|
|
11
|
+
*
|
|
12
|
+
* props.device.* MECHANICAL. "Do this to the hardware, now."
|
|
13
|
+
* VERB names: takePhoto, getCurrent, scanBarcode,
|
|
14
|
+
* readOnce, authenticate, pickImage.
|
|
15
|
+
* Returns a BARE value. THROWS when it cannot.
|
|
16
|
+
* Declared in the manifest as `native: { camera: true }`.
|
|
17
|
+
* Lives in '@adas/plugin-sdk'.
|
|
18
|
+
*
|
|
19
|
+
* props.deviceState.* SEMANTIC. "What is true right now, and how well do
|
|
20
|
+
* we know it?"
|
|
21
|
+
* STATE names: getSnapshot, subscribe, start, stop.
|
|
22
|
+
* Returns a SNAPSHOT carrying freshness + confidence,
|
|
23
|
+
* and able to say `unknown` in every field.
|
|
24
|
+
* A read NEVER throws for absence.
|
|
25
|
+
* Declared in the manifest as `stateDomains: ['visual']`.
|
|
26
|
+
* Lives in '@adas/device-state' (this file).
|
|
27
|
+
*
|
|
28
|
+
* The two containers are the discriminator. Every semantic call in a plugin
|
|
29
|
+
* begins with the eight characters `deviceState.`; every mechanical call
|
|
30
|
+
* begins with `native.`. There is no line of code where the layer is
|
|
31
|
+
* ambiguous, and no object on which both appear.
|
|
32
|
+
*
|
|
33
|
+
* This is a deliberate departure from
|
|
34
|
+
* Docs/device-bridge-v2/DEVICE_BRIDGE_V2_IMPLEMENTATION_GROUNDING_2026-08-07.md
|
|
35
|
+
* §"Migration strategy", which proposed adding `visual`, `spatial`,
|
|
36
|
+
* `connectivityState`… as fields ON `PluginNative`. See the design note at the
|
|
37
|
+
* bottom of this file for why that layout is rejected and what it costs.
|
|
38
|
+
* ─────────────────────────────────────────────────────────────────────────
|
|
39
|
+
*
|
|
40
|
+
* PROVENANCE MARKERS used throughout. They are not decoration — the freeze
|
|
41
|
+
* really does fix some of this and really does leave the rest open, and an
|
|
42
|
+
* implementor needs to know which is which before changing a line:
|
|
43
|
+
*
|
|
44
|
+
* FROZEN — API_FREEZE 2026-08-07 §15 lists it as a frozen candidate.
|
|
45
|
+
* V0.1 — "v0.1 freeze candidate; may change only when implementation
|
|
46
|
+
* grounding proves a field impossible/non-portable."
|
|
47
|
+
* OPEN — the docs explicitly leave it undecided. Do not quietly close it.
|
|
48
|
+
* DECISION — not in the docs at all; decided here, and reversible.
|
|
49
|
+
*/
|
|
50
|
+
|
|
51
|
+
/* eslint-disable @typescript-eslint/no-empty-interface */
|
|
52
|
+
|
|
53
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
54
|
+
// 1. Common metadata model
|
|
55
|
+
// FROZEN (API_FREEZE §1, §15 "Common metadata model: frozen candidate").
|
|
56
|
+
// Shared verbatim by all eight domains. Do not fork a per-domain variant.
|
|
57
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
58
|
+
|
|
59
|
+
/** FROZEN (API_FREEZE §1). The eight domain keys. */
|
|
60
|
+
export type DeviceBridgeDomain =
|
|
61
|
+
| 'spatial'
|
|
62
|
+
| 'visual'
|
|
63
|
+
| 'activity'
|
|
64
|
+
| 'deviceContext'
|
|
65
|
+
| 'connectivity'
|
|
66
|
+
| 'capability'
|
|
67
|
+
| 'environment'
|
|
68
|
+
| 'interaction';
|
|
69
|
+
|
|
70
|
+
export type FreshnessState = 'fresh' | 'aging' | 'stale' | 'unknown';
|
|
71
|
+
|
|
72
|
+
/**
|
|
73
|
+
* FROZEN. Both fields are nullable on purpose: "we have no observation at all"
|
|
74
|
+
* has to be expressible without inventing a timestamp.
|
|
75
|
+
*
|
|
76
|
+
* `aging` is a real third state, not a rounding of fresh/stale — a consumer
|
|
77
|
+
* that only branches on fresh-vs-stale is misreading the contract.
|
|
78
|
+
*
|
|
79
|
+
* API_FREEZE §1 "Why observedAt + ageMs": the host may compute `ageMs` at read
|
|
80
|
+
* time, so it can be more accurate than `Date.now() - observedAt` measured on
|
|
81
|
+
* the JS side after a bridge hop. Read `ageMs` when you have it.
|
|
82
|
+
*/
|
|
83
|
+
export interface Freshness {
|
|
84
|
+
state: FreshnessState;
|
|
85
|
+
/** Epoch ms of the observation itself. `null` = never observed. */
|
|
86
|
+
observedAt: number | null;
|
|
87
|
+
/** Age at read time. `null` = never observed. */
|
|
88
|
+
ageMs: number | null;
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
export type ConfidenceLevel = 'high' | 'medium' | 'low' | 'unknown';
|
|
92
|
+
|
|
93
|
+
/**
|
|
94
|
+
* FROZEN. `level` is mandatory and ordinal; `score` is optional and exists
|
|
95
|
+
* only "when a stable numeric estimate exists" (API_FREEZE §1).
|
|
96
|
+
*
|
|
97
|
+
* The asymmetry is the point: native must not manufacture 0.5 to fill a field.
|
|
98
|
+
* A missing `score` is information — it says the estimator has no calibrated
|
|
99
|
+
* number, only a bucket.
|
|
100
|
+
*/
|
|
101
|
+
export interface Confidence {
|
|
102
|
+
level: ConfidenceLevel;
|
|
103
|
+
/** 0..1. Omitted when no stable numeric estimate exists. */
|
|
104
|
+
score?: number;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/**
|
|
108
|
+
* FROZEN. Every snapshot in v2 carries exactly one of these.
|
|
109
|
+
*
|
|
110
|
+
* `freshness` required, `confidence` optional — because some domains report
|
|
111
|
+
* exact OS facts (battery level, audio route) where a confidence is
|
|
112
|
+
* meaningless, and others report inference (activity, indoor/outdoor) where it
|
|
113
|
+
* is mandatory. Cross-domain review: "Freshness is mandatory for dynamic
|
|
114
|
+
* state. Confidence is mandatory for inferred/estimated state; optional for
|
|
115
|
+
* exact OS facts."
|
|
116
|
+
*
|
|
117
|
+
* NOTE (OPEN): freshness is per-SNAPSHOT, never per-field. So a snapshot
|
|
118
|
+
* cannot today say "lighting is fresh but acoustics is stale". The
|
|
119
|
+
* architecture doc raised this for Spatial and Environment and never resolved
|
|
120
|
+
* it. Do not work around it by inventing a second freshness field.
|
|
121
|
+
*/
|
|
122
|
+
export interface StateQuality {
|
|
123
|
+
freshness: Freshness;
|
|
124
|
+
confidence?: Confidence;
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
// ── Support / permission / availability / restriction ──────────────────────
|
|
128
|
+
// FROZEN (API_FREEZE §1). Four INDEPENDENT axes. The capability doc is blunt:
|
|
129
|
+
// "Do not collapse all failure modes into a single boolean." A camera can be
|
|
130
|
+
// supported + granted + busy, and each of those three answers leads the caller
|
|
131
|
+
// somewhere different.
|
|
132
|
+
|
|
133
|
+
export type SupportState = 'supported' | 'unsupported' | 'unknown';
|
|
134
|
+
|
|
135
|
+
export type PermissionState =
|
|
136
|
+
| 'granted'
|
|
137
|
+
| 'denied'
|
|
138
|
+
| 'restricted'
|
|
139
|
+
| 'notDetermined'
|
|
140
|
+
| 'limited'
|
|
141
|
+
| 'unavailable'
|
|
142
|
+
| 'unknown';
|
|
143
|
+
|
|
144
|
+
export type AvailabilityState =
|
|
145
|
+
| 'available'
|
|
146
|
+
| 'busy'
|
|
147
|
+
| 'temporarilyUnavailable'
|
|
148
|
+
| 'unavailable'
|
|
149
|
+
| 'unknown';
|
|
150
|
+
|
|
151
|
+
export type RestrictionState =
|
|
152
|
+
| 'none'
|
|
153
|
+
| 'backgroundRestricted'
|
|
154
|
+
| 'systemRestricted'
|
|
155
|
+
| 'policyRestricted'
|
|
156
|
+
| 'lowPowerRestricted'
|
|
157
|
+
| 'unknown';
|
|
158
|
+
|
|
159
|
+
// ── Events ────────────────────────────────────────────────────────────────
|
|
160
|
+
|
|
161
|
+
/**
|
|
162
|
+
* FROZEN (API_FREEZE core rule 4): "Events are meaningful semantic changes,
|
|
163
|
+
* never raw sensor streams." If you find yourself wanting an event per frame
|
|
164
|
+
* or per GPS fix, you are in the wrong layer.
|
|
165
|
+
*/
|
|
166
|
+
export interface StateEvent<TName extends string = string> {
|
|
167
|
+
domain: DeviceBridgeDomain;
|
|
168
|
+
name: TName;
|
|
169
|
+
/** Epoch ms. */
|
|
170
|
+
timestamp: number;
|
|
171
|
+
/**
|
|
172
|
+
* OPEN: per-event payload schemas are not specified anywhere in the three
|
|
173
|
+
* design docs. Typed `unknown` rather than `any` so a consumer is forced to
|
|
174
|
+
* narrow it rather than read a field that may not exist.
|
|
175
|
+
*/
|
|
176
|
+
payload?: unknown;
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
export interface Subscription {
|
|
180
|
+
unsubscribe(): void;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
export type StateListener<TName extends string = string> = (
|
|
184
|
+
event: StateEvent<TName>,
|
|
185
|
+
) => void;
|
|
186
|
+
|
|
187
|
+
/**
|
|
188
|
+
* VOCABULARY §"Event Naming Rule", expressed as a type.
|
|
189
|
+
*
|
|
190
|
+
* Four domains (visual, activity, spatial, deviceContext) have NO frozen event
|
|
191
|
+
* vocabulary — the vocabulary doc publishes "Initial … Events" lists for
|
|
192
|
+
* connectivity, capability, interaction and environment only. Rather than mint
|
|
193
|
+
* names those docs deliberately did not mint, their events are typed against
|
|
194
|
+
* the naming RULE. A wrong-shaped name is a compile error; the specific
|
|
195
|
+
* vocabulary stays open, as the docs left it.
|
|
196
|
+
*/
|
|
197
|
+
export type ConformingEventName =
|
|
198
|
+
| `${string}Changed`
|
|
199
|
+
| `${string}Started`
|
|
200
|
+
| `${string}Stopped`
|
|
201
|
+
| `${string}Available`
|
|
202
|
+
| `${string}Unavailable`
|
|
203
|
+
| `${string}StateStale`;
|
|
204
|
+
|
|
205
|
+
// ── Errors ────────────────────────────────────────────────────────────────
|
|
206
|
+
|
|
207
|
+
/** FROZEN (API_FREEZE §12). */
|
|
208
|
+
export type DeviceBridgeErrorCode =
|
|
209
|
+
| 'UNSUPPORTED'
|
|
210
|
+
| 'PERMISSION_DENIED'
|
|
211
|
+
| 'RESTRICTED'
|
|
212
|
+
| 'BUSY'
|
|
213
|
+
| 'TEMPORARILY_UNAVAILABLE'
|
|
214
|
+
| 'STALE'
|
|
215
|
+
| 'TIMEOUT'
|
|
216
|
+
| 'CANCELLED'
|
|
217
|
+
| 'INTERNAL';
|
|
218
|
+
|
|
219
|
+
/**
|
|
220
|
+
* FROZEN shape (API_FREEZE §12).
|
|
221
|
+
*
|
|
222
|
+
* READ THE SCOPE LINE CAREFULLY. This is for FAILED OPERATIONS —
|
|
223
|
+
* `visual.capture()`, `visual.start()`. It is NOT how the surface reports that
|
|
224
|
+
* a camera is denied or a sensor is missing: "Expected state should still be
|
|
225
|
+
* exposed in snapshots."
|
|
226
|
+
*
|
|
227
|
+
* So: `getSnapshot()` does not throw this. `capture()` and `start()` do.
|
|
228
|
+
* That split is the whole no-fabrication contract in one sentence — a read
|
|
229
|
+
* always returns something honest, an action either happens or explains
|
|
230
|
+
* itself in a typed code.
|
|
231
|
+
*/
|
|
232
|
+
export class DeviceBridgeError extends Error {
|
|
233
|
+
readonly code: DeviceBridgeErrorCode;
|
|
234
|
+
readonly domain?: DeviceBridgeDomain;
|
|
235
|
+
/** Whether retrying could plausibly succeed (BUSY yes, UNSUPPORTED no). */
|
|
236
|
+
readonly recoverable?: boolean;
|
|
237
|
+
|
|
238
|
+
constructor(
|
|
239
|
+
code: DeviceBridgeErrorCode,
|
|
240
|
+
message: string,
|
|
241
|
+
opts?: { domain?: DeviceBridgeDomain; recoverable?: boolean },
|
|
242
|
+
) {
|
|
243
|
+
super(message);
|
|
244
|
+
this.name = 'DeviceBridgeError';
|
|
245
|
+
this.code = code;
|
|
246
|
+
this.domain = opts?.domain;
|
|
247
|
+
this.recoverable = opts?.recoverable;
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
// ── Sessions and the lifecycle pattern ────────────────────────────────────
|
|
252
|
+
|
|
253
|
+
export interface StateSession {
|
|
254
|
+
id: string;
|
|
255
|
+
/** The intent this session was started with. */
|
|
256
|
+
intent: string;
|
|
257
|
+
startedAt: number;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
export interface StartOptions<TIntent extends string> {
|
|
261
|
+
/**
|
|
262
|
+
* MANDATORY, and that is the single most important line in this file.
|
|
263
|
+
*
|
|
264
|
+
* You cannot open a v2 session without naming what it is FOR. Native then
|
|
265
|
+
* owns sampling rate, resolution, buffering, fusion and power — API_FREEZE
|
|
266
|
+
* core rule 6. `start()` means "maintain state according to this intent",
|
|
267
|
+
* not "open a particular native API" (§2).
|
|
268
|
+
*
|
|
269
|
+
* Contrast the mechanical layer, where `camera.takePhoto()` is a legal,
|
|
270
|
+
* complete, intent-free call. That difference is visible at every call site.
|
|
271
|
+
*/
|
|
272
|
+
intent: TIntent;
|
|
273
|
+
/**
|
|
274
|
+
* DECISION (not frozen). §13 requires bounded timeout/cancellation for
|
|
275
|
+
* session starts and says chain STOP must propagate to native, but leaves
|
|
276
|
+
* the plumbing "implementation-specific". `AbortSignal` is what
|
|
277
|
+
* `DeviceToolContext.signal` already carries in this SDK, so reuse it rather
|
|
278
|
+
* than invent a second cancellation idiom.
|
|
279
|
+
*/
|
|
280
|
+
signal?: AbortSignal;
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
/**
|
|
284
|
+
* FROZEN pattern (API_FREEZE §2), reproduced as documentation. The eight
|
|
285
|
+
* concrete domains below do NOT extend this — each spells out exactly the
|
|
286
|
+
* methods it has, because §11 warns "domains should not be forced to implement
|
|
287
|
+
* meaningless lifecycle methods" and three domains genuinely have no intent
|
|
288
|
+
* vocabulary at all.
|
|
289
|
+
*
|
|
290
|
+
* Two invariants apply to every domain:
|
|
291
|
+
* - `getSnapshot()` reads CACHED state. It must not trigger expensive raw
|
|
292
|
+
* sensing (no camera open, no network probe, no GPS cold start).
|
|
293
|
+
* - "Multiple plugins may eventually share one native engine; session
|
|
294
|
+
* ownership must not imply duplicate sensor pipelines." (OPEN: the
|
|
295
|
+
* arbitration mechanism is undefined. See DESIGN NOTE 3.)
|
|
296
|
+
*/
|
|
297
|
+
export interface StatefulDomain<
|
|
298
|
+
TSnapshot,
|
|
299
|
+
TIntent extends string = never,
|
|
300
|
+
TEvent extends string = string,
|
|
301
|
+
> {
|
|
302
|
+
getSnapshot(): Promise<TSnapshot>;
|
|
303
|
+
start?(options: StartOptions<TIntent>): Promise<StateSession>;
|
|
304
|
+
stop?(sessionId?: string): Promise<void>;
|
|
305
|
+
subscribe?(listener: StateListener<TEvent>): Subscription;
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
/**
|
|
309
|
+
* Why every read is `Promise<T>` even though it is a cached read.
|
|
310
|
+
*
|
|
311
|
+
* Not because it samples — it must not. Because it crosses the RN bridge, and
|
|
312
|
+
* this SDK has no synchronous JSI module today (verified: everything in
|
|
313
|
+
* PluginNative except haptics/connectivity.isOnline is Promise-based). The
|
|
314
|
+
* grounding doc's own target for a warm cached snapshot is p50 < 10 ms /
|
|
315
|
+
* p95 < 30 ms, which is an async-bridge budget, not a sampling budget.
|
|
316
|
+
*/
|
|
317
|
+
|
|
318
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
319
|
+
// 2. Domain 1 — Spatial State
|
|
320
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
321
|
+
|
|
322
|
+
/**
|
|
323
|
+
* FROZEN (API_FREEZE §3). Copied verbatim, not designed here.
|
|
324
|
+
*
|
|
325
|
+
* This landed as a placeholder first, on the belief that §3's field schema had
|
|
326
|
+
* never been written. It had — the placeholder was a gap in what got read, not
|
|
327
|
+
* a gap in the freeze. Worth recording, because "the docs leave this open" and
|
|
328
|
+
* "I did not find it" produce identical-looking comments and only one of them
|
|
329
|
+
* is a reason to invent a shape.
|
|
330
|
+
*/
|
|
331
|
+
export type SpatialIntent =
|
|
332
|
+
| 'navigation'
|
|
333
|
+
| 'fitness'
|
|
334
|
+
| 'geofence'
|
|
335
|
+
| 'lowPowerTracking'
|
|
336
|
+
| 'preciseLocate';
|
|
337
|
+
|
|
338
|
+
/** FROZEN (API_FREEZE §3). */
|
|
339
|
+
export interface SpatialPoint {
|
|
340
|
+
latitude: number;
|
|
341
|
+
longitude: number;
|
|
342
|
+
altitude?: number | null;
|
|
343
|
+
/** Metres. `null` when the platform gives no estimate — never 0, which
|
|
344
|
+
* would read as a perfect fix. */
|
|
345
|
+
accuracyM?: number | null;
|
|
346
|
+
timestamp: number;
|
|
347
|
+
}
|
|
348
|
+
|
|
349
|
+
/** FROZEN (API_FREEZE §3). */
|
|
350
|
+
export interface SpatialSnapshot {
|
|
351
|
+
/** `null` when there is no usable fix. Not a zero coordinate — (0,0) is a
|
|
352
|
+
* real place in the Gulf of Guinea. */
|
|
353
|
+
position: SpatialPoint | null;
|
|
354
|
+
movement: {
|
|
355
|
+
speedMps: number | null;
|
|
356
|
+
bearingDeg: number | null;
|
|
357
|
+
distanceRecentM: number | null;
|
|
358
|
+
state?: 'stationary' | 'moving' | 'unknown';
|
|
359
|
+
};
|
|
360
|
+
orientation: {
|
|
361
|
+
headingDeg: number | null;
|
|
362
|
+
headingAccuracyDeg?: number | null;
|
|
363
|
+
pitchDeg?: number | null;
|
|
364
|
+
rollDeg?: number | null;
|
|
365
|
+
};
|
|
366
|
+
/** Newest last. Empty when nothing has been recorded — the ring is a native
|
|
367
|
+
* concern, and its window is chosen by intent. */
|
|
368
|
+
recentPath: SpatialPoint[];
|
|
369
|
+
changes?: {
|
|
370
|
+
lastMeaningfulTurnDeg?: number | null;
|
|
371
|
+
timestamp?: number | null;
|
|
372
|
+
};
|
|
373
|
+
quality: StateQuality;
|
|
374
|
+
}
|
|
375
|
+
|
|
376
|
+
export interface SpatialDomain {
|
|
377
|
+
getSnapshot(): Promise<SpatialSnapshot>;
|
|
378
|
+
start(options: StartOptions<SpatialIntent>): Promise<StateSession>;
|
|
379
|
+
stop(sessionId?: string): Promise<void>;
|
|
380
|
+
/** OPEN: no frozen spatial event vocabulary; rule-conforming names only. */
|
|
381
|
+
subscribe(listener: StateListener<ConformingEventName>): Subscription;
|
|
382
|
+
}
|
|
383
|
+
|
|
384
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
385
|
+
// 3. Domain 2 — Visual State
|
|
386
|
+
//
|
|
387
|
+
// Visual is NOT "the camera API". It is a native-maintained visual timeline
|
|
388
|
+
// plus camera/capture state. Everything interpretive — what is in the scene,
|
|
389
|
+
// what the sign says, which corridor to take — sits ABOVE Device Bridge and is
|
|
390
|
+
// not modelled here.
|
|
391
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
392
|
+
|
|
393
|
+
/** FROZEN (API_FREEZE §4) + VOCABULARY §"Visual Intents". Nine values. */
|
|
394
|
+
export type VisualIntent =
|
|
395
|
+
| 'navigation'
|
|
396
|
+
| 'document'
|
|
397
|
+
| 'ocr'
|
|
398
|
+
| 'shopping'
|
|
399
|
+
| 'inspection'
|
|
400
|
+
| 'accessibility'
|
|
401
|
+
| 'stream'
|
|
402
|
+
| 'highQuality'
|
|
403
|
+
| 'lowPower';
|
|
404
|
+
|
|
405
|
+
/**
|
|
406
|
+
* Media crosses by REFERENCE, never as inline bytes (core rule 3).
|
|
407
|
+
*
|
|
408
|
+
* AN ID AND NOTHING ELSE. The `uri` field is gone (2026-08-16), and its
|
|
409
|
+
* removal answers the question this comment used to record as OPEN — "how does
|
|
410
|
+
* a plugin turn a handle into pixels?" It doesn't. Ever.
|
|
411
|
+
*
|
|
412
|
+
* Frames now live in a native ring keyed by id and are never written to disk.
|
|
413
|
+
* `native.vision.readText/describe/assessQuality` take this id and resolve it
|
|
414
|
+
* INSIDE the trusted layer; the pixels do not cross into plugin code, so a
|
|
415
|
+
* solution's bundle cannot copy, upload or retain them. That is what makes
|
|
416
|
+
* "frames never leave the phone" enforceable rather than a promise.
|
|
417
|
+
*
|
|
418
|
+
* Do not add `uri` or `bytes` back. A plugin that believes it needs the pixels
|
|
419
|
+
* needs a new method on `native.vision` instead — one whose implementation
|
|
420
|
+
* stays on this side of the boundary.
|
|
421
|
+
*/
|
|
422
|
+
export interface MediaHandle {
|
|
423
|
+
id: string;
|
|
424
|
+
mimeType?: string;
|
|
425
|
+
/**
|
|
426
|
+
* Epoch ms after which this id no longer resolves — the frame's real
|
|
427
|
+
* eviction time from the native ring, not a guess.
|
|
428
|
+
*
|
|
429
|
+
* Past it, `native.vision` throws rather than returning an empty result:
|
|
430
|
+
* "I cannot look at that frame" and "I looked and found nothing" are
|
|
431
|
+
* different answers and must never be confused by a caller about to speak.
|
|
432
|
+
* In practice a caller rarely hits this, because the timeline is clamped to
|
|
433
|
+
* native retention — see `visualEngine.effectiveWindowMs()`.
|
|
434
|
+
*/
|
|
435
|
+
expiresAt?: number | null;
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
/**
|
|
439
|
+
* V0.1. Per-frame usability is CATEGORICAL, not numeric — 'degraded' rather
|
|
440
|
+
* than a blur score. Coarse honest classes beat false precision; the same
|
|
441
|
+
* principle the connectivity doc states outright for connection quality.
|
|
442
|
+
*
|
|
443
|
+
* Every member carries `'unknown'` so a platform that cannot assess blur says
|
|
444
|
+
* so instead of defaulting to 'good'.
|
|
445
|
+
*/
|
|
446
|
+
export interface VisualFrameQuality {
|
|
447
|
+
blur?: 'good' | 'degraded' | 'unknown';
|
|
448
|
+
exposure?: 'good' | 'dark' | 'bright' | 'unknown';
|
|
449
|
+
/** `null` = cannot tell. Distinct from `false` = checked, not obstructed. */
|
|
450
|
+
obstructed?: boolean | null;
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
export interface VisualFrameRef {
|
|
454
|
+
handle: MediaHandle;
|
|
455
|
+
timestamp: number;
|
|
456
|
+
orientation?: { headingDeg?: number; pitchDeg?: number; rollDeg?: number };
|
|
457
|
+
quality?: VisualFrameQuality;
|
|
458
|
+
}
|
|
459
|
+
|
|
460
|
+
export interface VisualSnapshot {
|
|
461
|
+
camera: {
|
|
462
|
+
available: boolean;
|
|
463
|
+
/**
|
|
464
|
+
* Independent of `available` on purpose. A camera can be present and
|
|
465
|
+
* permitted (`available: true`) while no session is running
|
|
466
|
+
* (`active: false`) — that is a different situation from a camera the app
|
|
467
|
+
* cannot use at all, and Guide has to say different things about them.
|
|
468
|
+
*
|
|
469
|
+
* DRIFT (flagged, not resolved): the architecture doc's conceptual block
|
|
470
|
+
* also carried `camera.orientation`; the frozen §4 block does not. Not
|
|
471
|
+
* added here.
|
|
472
|
+
*/
|
|
473
|
+
active: boolean;
|
|
474
|
+
facing?: 'front' | 'back' | 'unknown';
|
|
475
|
+
};
|
|
476
|
+
/**
|
|
477
|
+
* `null` when there is no usable current frame.
|
|
478
|
+
*
|
|
479
|
+
* This is the anti-fabrication field. The alternative implementation —
|
|
480
|
+
* returning the last frame you happen to still hold — is precisely how a
|
|
481
|
+
* guidance loop ends up describing a corridor the user left thirty seconds
|
|
482
|
+
* ago. If it is not current, it is `null`, and the timeline below is where
|
|
483
|
+
* you go looking for history, knowingly.
|
|
484
|
+
*/
|
|
485
|
+
latest: VisualFrameRef | null;
|
|
486
|
+
/**
|
|
487
|
+
* TIME-based, not frame-count-based. Default window ~5s; documented bounded
|
|
488
|
+
* range roughly 1–15s, selected by profile/intent. Native decides how
|
|
489
|
+
* densely to retain frames inside the window and evicts everything outside
|
|
490
|
+
* it automatically.
|
|
491
|
+
*
|
|
492
|
+
* `windowMs` is exposed; FPS knobs are NOT, and must never be added
|
|
493
|
+
* (API_FREEZE §4). If a plugin wants "more frames", it declares a different
|
|
494
|
+
* intent — it does not turn a dial.
|
|
495
|
+
*/
|
|
496
|
+
recentTimeline: { windowMs: number; frames: VisualFrameRef[] };
|
|
497
|
+
quality: StateQuality;
|
|
498
|
+
}
|
|
499
|
+
|
|
500
|
+
export interface VisualCaptureResult {
|
|
501
|
+
frame: VisualFrameRef;
|
|
502
|
+
}
|
|
503
|
+
|
|
504
|
+
/**
|
|
505
|
+
* OPEN, worth knowing before you use it: whether `capture({intent})` may pass
|
|
506
|
+
* an intent DIFFERENT from the running session's intent is not addressed
|
|
507
|
+
* anywhere in the three docs.
|
|
508
|
+
*/
|
|
509
|
+
export interface VisualCaptureOptions {
|
|
510
|
+
intent: VisualIntent;
|
|
511
|
+
/** §13: capture must support bounded timeout/cancellation. */
|
|
512
|
+
signal?: AbortSignal;
|
|
513
|
+
timeoutMs?: number;
|
|
514
|
+
}
|
|
515
|
+
|
|
516
|
+
export interface VisualDomain {
|
|
517
|
+
/**
|
|
518
|
+
* Cached read. Does NOT open the camera and does NOT take a picture.
|
|
519
|
+
*
|
|
520
|
+
* This is the call a guidance loop makes on every turn. The hot path is:
|
|
521
|
+
* native camera pipeline → rolling native timeline → cached visual state →
|
|
522
|
+
* one snapshot read from RN. "Avoid repeatedly starting the camera or
|
|
523
|
+
* capturing a new image for every plugin-tool call during an interactive
|
|
524
|
+
* session."
|
|
525
|
+
*/
|
|
526
|
+
getSnapshot(): Promise<VisualSnapshot>;
|
|
527
|
+
|
|
528
|
+
/** Maintain visual state according to this intent. Not "open the camera". */
|
|
529
|
+
start(options: StartOptions<VisualIntent>): Promise<StateSession>;
|
|
530
|
+
|
|
531
|
+
/**
|
|
532
|
+
* An intentional, one-shot, high-quality-ish acquisition. THROWS
|
|
533
|
+
* DeviceBridgeError on failure — it is an operation, not a read.
|
|
534
|
+
*
|
|
535
|
+
* If you are about to call this once per conversational turn, stop: that is
|
|
536
|
+
* `getSnapshot()`'s job, and doing it here is the frame-timer pattern this
|
|
537
|
+
* whole layer exists to replace.
|
|
538
|
+
*/
|
|
539
|
+
capture(options: VisualCaptureOptions): Promise<VisualCaptureResult>;
|
|
540
|
+
|
|
541
|
+
/**
|
|
542
|
+
* OPEN (doc conflict): the architecture doc shows `visual.subscribe(events)`
|
|
543
|
+
* — a filter — while API_FREEZE shows `visual.subscribe(listener)`. The
|
|
544
|
+
* freeze is followed here. Unresolved.
|
|
545
|
+
*
|
|
546
|
+
* OPEN: there is no "Initial Visual Events" list anywhere. The candidate set
|
|
547
|
+
* in prose (camera became available/unavailable, frame quality degraded/
|
|
548
|
+
* recovered, camera appears obstructed, significant scene change, visual
|
|
549
|
+
* session suspended/resumed, visual state became stale) is NOT a minted
|
|
550
|
+
* vocabulary. Typed against the naming rule only.
|
|
551
|
+
*/
|
|
552
|
+
subscribe(listener: StateListener<ConformingEventName>): Subscription;
|
|
553
|
+
|
|
554
|
+
stop(sessionId?: string): Promise<void>;
|
|
555
|
+
}
|
|
556
|
+
|
|
557
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
558
|
+
// 4. Domain 3 — Human Activity State
|
|
559
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
560
|
+
|
|
561
|
+
/** FROZEN (API_FREEZE §5, VOCABULARY §"Human Activity Values"). */
|
|
562
|
+
export type HumanActivity =
|
|
563
|
+
| 'stationary'
|
|
564
|
+
| 'walking'
|
|
565
|
+
| 'running'
|
|
566
|
+
| 'cycling'
|
|
567
|
+
| 'driving'
|
|
568
|
+
| 'unknown';
|
|
569
|
+
|
|
570
|
+
/** FROZEN. Independent of HumanActivity — walking+pocket, driving+mounted. */
|
|
571
|
+
export type DevicePlacement =
|
|
572
|
+
| 'handheld'
|
|
573
|
+
| 'pocket'
|
|
574
|
+
| 'table'
|
|
575
|
+
| 'mounted'
|
|
576
|
+
| 'unknown';
|
|
577
|
+
|
|
578
|
+
/**
|
|
579
|
+
* FROZEN. Note `driving` is BOTH an intent here and a HumanActivity value
|
|
580
|
+
* above. The docs define no relationship between the two and do not flag the
|
|
581
|
+
* collision: declaring intent `driving` does not assert the user is driving.
|
|
582
|
+
*/
|
|
583
|
+
export type ActivityIntent =
|
|
584
|
+
| 'navigation'
|
|
585
|
+
| 'fitness'
|
|
586
|
+
| 'driving'
|
|
587
|
+
| 'safety'
|
|
588
|
+
| 'accessibility'
|
|
589
|
+
| 'lowPower';
|
|
590
|
+
|
|
591
|
+
export interface ActivitySnapshot {
|
|
592
|
+
human: {
|
|
593
|
+
state: HumanActivity;
|
|
594
|
+
/** `null` when the start of the current activity is not known. */
|
|
595
|
+
since: number | null;
|
|
596
|
+
/** REQUIRED, unlike StateQuality.confidence — an inferred activity may
|
|
597
|
+
* never be handed over without saying how sure the inference is. */
|
|
598
|
+
confidence: Confidence;
|
|
599
|
+
};
|
|
600
|
+
devicePose: {
|
|
601
|
+
placement: DevicePlacement;
|
|
602
|
+
/** V0.1. These four values appear only in the freeze TS, not in the
|
|
603
|
+
* vocabulary doc — unlike the placement values. Mild drift, flagged. */
|
|
604
|
+
orientation?: 'portrait' | 'landscape' | 'flat' | 'unknown';
|
|
605
|
+
confidence: Confidence;
|
|
606
|
+
};
|
|
607
|
+
/**
|
|
608
|
+
* Advisory only. `likelyNext` is a hint and never a fact.
|
|
609
|
+
* OPEN: transition names are bare `string` — no controlled vocabulary exists
|
|
610
|
+
* anywhere, which sits awkwardly beside "Intents are not arbitrary strings".
|
|
611
|
+
*/
|
|
612
|
+
transition?: {
|
|
613
|
+
current?: string | null;
|
|
614
|
+
likelyNext?: string | null;
|
|
615
|
+
confidence?: Confidence;
|
|
616
|
+
};
|
|
617
|
+
/** OPEN: no window length or cap is specified for Activity (Spatial gets
|
|
618
|
+
* ~10–20s, Visual ~5s). Required array — an empty one is a real answer. */
|
|
619
|
+
recentTransitions: Array<{ name: string; timestamp: number }>;
|
|
620
|
+
quality: StateQuality;
|
|
621
|
+
}
|
|
622
|
+
|
|
623
|
+
export interface ActivityDomain {
|
|
624
|
+
getSnapshot(): Promise<ActivitySnapshot>;
|
|
625
|
+
/**
|
|
626
|
+
* OPEN: §11's naming list shows only `activity.getSnapshot()`, and §2 makes
|
|
627
|
+
* start/stop optional. Whether Activity is session-based or a passive
|
|
628
|
+
* always-on domain is genuinely unstated. Included here because the six
|
|
629
|
+
* frozen ActivityIntent values would otherwise have no seam to arrive
|
|
630
|
+
* through — but this is a DECISION, not a reading of the freeze.
|
|
631
|
+
*/
|
|
632
|
+
start(options: StartOptions<ActivityIntent>): Promise<StateSession>;
|
|
633
|
+
stop(sessionId?: string): Promise<void>;
|
|
634
|
+
/** OPEN: no "Initial Activity Events" list exists. */
|
|
635
|
+
subscribe(listener: StateListener<ConformingEventName>): Subscription;
|
|
636
|
+
}
|
|
637
|
+
|
|
638
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
639
|
+
// 5. Domain 4 — Device Context
|
|
640
|
+
//
|
|
641
|
+
// SCOPE WARNING. The early architecture draft put lifecycle, audio route and
|
|
642
|
+
// hardware availability here. The freeze REMOVED all three: app foreground/
|
|
643
|
+
// background and active channels are canonicalized under Interaction; hardware
|
|
644
|
+
// support/permission moved to Capability. Frozen Device Context is power +
|
|
645
|
+
// thermal + resources, and nothing else.
|
|
646
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
647
|
+
|
|
648
|
+
export interface DeviceContextSnapshot {
|
|
649
|
+
power: {
|
|
650
|
+
/** 0..1 FRACTION, not a percentage. `null` = not known. */
|
|
651
|
+
batteryLevel: number | null;
|
|
652
|
+
charging: boolean | null;
|
|
653
|
+
lowPowerMode: boolean | null;
|
|
654
|
+
};
|
|
655
|
+
thermal: {
|
|
656
|
+
state: 'nominal' | 'fair' | 'serious' | 'critical' | 'unknown';
|
|
657
|
+
};
|
|
658
|
+
resources: {
|
|
659
|
+
memoryPressure?: 'normal' | 'warning' | 'critical' | 'unknown';
|
|
660
|
+
storagePressure?: 'normal' | 'warning' | 'critical' | 'unknown';
|
|
661
|
+
};
|
|
662
|
+
/** Confidence is legitimately omitted here — these are exact OS facts, not
|
|
663
|
+
* estimates. Freshness still matters: a populated batteryLevel says nothing
|
|
664
|
+
* about whether it was sampled a second or a minute ago. */
|
|
665
|
+
quality: StateQuality;
|
|
666
|
+
}
|
|
667
|
+
|
|
668
|
+
/**
|
|
669
|
+
* NO INTENT TYPE, and that is deliberate — not an oversight, and not a gap to
|
|
670
|
+
* fill in later. There is no `DeviceContextIntent` in the freeze and no
|
|
671
|
+
* "Device Context Intents" section in the vocabulary, because
|
|
672
|
+
* (cross-domain review) "A domain may expose an intent only if it changes
|
|
673
|
+
* native implementation strategy in a meaningful way." Reading battery and
|
|
674
|
+
* thermal does not.
|
|
675
|
+
*
|
|
676
|
+
* The direction of flow is the other way round: Device Context is an INPUT to
|
|
677
|
+
* how OTHER domains satisfy their intents. Visual intent `navigation` +
|
|
678
|
+
* device context `battery 5% + low-power mode` → native picks a lower-power
|
|
679
|
+
* camera strategy while still honouring the Visual contract. The plugin does
|
|
680
|
+
* not retune anything, and must not be asked to.
|
|
681
|
+
*
|
|
682
|
+
* Do not add `deviceContext.start({intent:'lowPower'})` because five other
|
|
683
|
+
* domains have a lowPower intent.
|
|
684
|
+
*/
|
|
685
|
+
export interface DeviceContextDomain {
|
|
686
|
+
getSnapshot(): Promise<DeviceContextSnapshot>;
|
|
687
|
+
/**
|
|
688
|
+
* OPEN: `DeviceContextDomain`'s interface body is never written in the
|
|
689
|
+
* freeze, and no Device Context event names are frozen (the vocabulary has
|
|
690
|
+
* "Initial … Events" for connectivity/capability/interaction/environment
|
|
691
|
+
* only). DECISION: expose subscribe, because the candidate events —
|
|
692
|
+
* low-power mode changed, charging changed, thermal pressure changed — are
|
|
693
|
+
* exactly what an adaptive session needs and polling for them is wasteful.
|
|
694
|
+
*
|
|
695
|
+
* OPEN: "battery crossed significant threshold" and "changed materially"
|
|
696
|
+
* have no numbers anywhere, and Device Context has no profiles to
|
|
697
|
+
* parameterize them.
|
|
698
|
+
*/
|
|
699
|
+
subscribe(listener: StateListener<ConformingEventName>): Subscription;
|
|
700
|
+
}
|
|
701
|
+
|
|
702
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
703
|
+
// 6. Domain 5 — Connectivity State
|
|
704
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
705
|
+
|
|
706
|
+
/** FROZEN (API_FREEZE §7, VOCABULARY §"Connectivity Intents"). */
|
|
707
|
+
export type ConnectivityIntent =
|
|
708
|
+
| 'lowLatencyConversation'
|
|
709
|
+
| 'interactive'
|
|
710
|
+
| 'bulkTransfer'
|
|
711
|
+
| 'backgroundSync'
|
|
712
|
+
| 'offlineResilient'
|
|
713
|
+
| 'lowPower';
|
|
714
|
+
|
|
715
|
+
/**
|
|
716
|
+
* V0.1, and note it is FLAT — the connectivity design doc sketched a nested
|
|
717
|
+
* shape (internet{…}, quality{…}, recent_history{…}) and the freeze flattened
|
|
718
|
+
* it. Two unreconciled deltas, carried here as-is rather than merged:
|
|
719
|
+
* - `bandwidthClass` appears in the freeze but in neither the design doc nor
|
|
720
|
+
* the vocabulary's Connectivity Values.
|
|
721
|
+
* - there is NO recent-history field, although the design doc requires native
|
|
722
|
+
* to maintain recent transitions and leaves the default window OPEN.
|
|
723
|
+
*/
|
|
724
|
+
export interface ConnectivitySnapshot {
|
|
725
|
+
/** NOTE: the design doc says "online / offline / uncertain"; the frozen
|
|
726
|
+
* third value is `unknown`. The two words are never reconciled. */
|
|
727
|
+
reachability: 'online' | 'offline' | 'unknown';
|
|
728
|
+
transport: 'wifi' | 'cellular' | 'ethernet' | 'other' | 'none' | 'unknown';
|
|
729
|
+
qualityClass:
|
|
730
|
+
| 'excellent'
|
|
731
|
+
| 'good'
|
|
732
|
+
| 'degraded'
|
|
733
|
+
| 'poor'
|
|
734
|
+
| 'unusable'
|
|
735
|
+
| 'unknown';
|
|
736
|
+
stability: 'stable' | 'changing' | 'flapping' | 'unknown';
|
|
737
|
+
/** `null` when no recent measurement exists. Explicitly preferred over
|
|
738
|
+
* reporting a stale number: "return unknown rather than stale precision." */
|
|
739
|
+
latencyMs?: number | null;
|
|
740
|
+
bandwidthClass?: 'high' | 'medium' | 'low' | 'unknown';
|
|
741
|
+
metered?: boolean | null;
|
|
742
|
+
constrained?: boolean | null;
|
|
743
|
+
quality: StateQuality;
|
|
744
|
+
}
|
|
745
|
+
|
|
746
|
+
/** FROZEN v0.1 (VOCABULARY §"Initial Connectivity Events"). All ten. */
|
|
747
|
+
export type ConnectivityEvent =
|
|
748
|
+
| 'connectivityOnline'
|
|
749
|
+
| 'connectivityOffline'
|
|
750
|
+
| 'transportChanged'
|
|
751
|
+
| 'connectivityDegraded'
|
|
752
|
+
| 'connectivityRecovered'
|
|
753
|
+
| 'connectivityUnstable'
|
|
754
|
+
| 'connectivityStable'
|
|
755
|
+
| 'meteredStateChanged'
|
|
756
|
+
| 'constrainedStateChanged'
|
|
757
|
+
| 'connectivityStateStale';
|
|
758
|
+
|
|
759
|
+
export interface ConnectivityDomain {
|
|
760
|
+
/**
|
|
761
|
+
* A fast local cached read. It must NOT ping, and must NOT run a bandwidth
|
|
762
|
+
* or speed test. Latency is sampled on native's own schedule, chosen by
|
|
763
|
+
* intent, and cached with a timestamp.
|
|
764
|
+
*/
|
|
765
|
+
getSnapshot(): Promise<ConnectivitySnapshot>;
|
|
766
|
+
/**
|
|
767
|
+
* OPEN: `ConnectivityDomain` is named in §11 but never defined, so whether
|
|
768
|
+
* connectivity is session-based is unstated. DECISION: yes — six frozen
|
|
769
|
+
* intents with documented behavioural differences (lowLatencyConversation
|
|
770
|
+
* "prioritize fresh latency knowledge"; lowPower "rely primarily on passive
|
|
771
|
+
* OS signals and minimize active probes") only mean something if there is a
|
|
772
|
+
* way to declare one.
|
|
773
|
+
*/
|
|
774
|
+
start(options: StartOptions<ConnectivityIntent>): Promise<StateSession>;
|
|
775
|
+
stop(sessionId?: string): Promise<void>;
|
|
776
|
+
subscribe(listener: StateListener<ConnectivityEvent>): Subscription;
|
|
777
|
+
}
|
|
778
|
+
|
|
779
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
780
|
+
// 7. Domain 6 — Capability / Permission State
|
|
781
|
+
//
|
|
782
|
+
// The "why" domain. When another snapshot says `unknown`, this is where the
|
|
783
|
+
// reason lives. Descriptive only — it never triggers a permission dialog.
|
|
784
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
785
|
+
|
|
786
|
+
/**
|
|
787
|
+
* FROZEN. Four independent axes, deliberately not collapsed. The canonical
|
|
788
|
+
* worked example from the capability doc: supported + granted + busy + none —
|
|
789
|
+
* a camera you own, are allowed to use, and cannot have right now.
|
|
790
|
+
*/
|
|
791
|
+
export interface CapabilityStatus {
|
|
792
|
+
support: SupportState;
|
|
793
|
+
permission?: PermissionState;
|
|
794
|
+
availability: AvailabilityState;
|
|
795
|
+
restriction?: RestrictionState;
|
|
796
|
+
}
|
|
797
|
+
|
|
798
|
+
export interface CapabilitySnapshot {
|
|
799
|
+
camera: CapabilityStatus;
|
|
800
|
+
microphone: CapabilityStatus;
|
|
801
|
+
/** The only capability the freeze gives scoped detail to. Health and
|
|
802
|
+
* photos/media do NOT get one — see the gap note below. */
|
|
803
|
+
location: CapabilityStatus & {
|
|
804
|
+
precision?: 'precise' | 'approximate' | 'unknown';
|
|
805
|
+
backgroundPermission?: PermissionState;
|
|
806
|
+
};
|
|
807
|
+
motion: CapabilityStatus;
|
|
808
|
+
bluetooth: CapabilityStatus;
|
|
809
|
+
uwb: CapabilityStatus;
|
|
810
|
+
nfc: CapabilityStatus;
|
|
811
|
+
notifications: CapabilityStatus;
|
|
812
|
+
/**
|
|
813
|
+
* GAP (flagged): the capability doc notes health "authorization is often
|
|
814
|
+
* record-type specific rather than a single global permission" and its
|
|
815
|
+
* candidate shape had a `permission_summary`. The frozen §9 gives health a
|
|
816
|
+
* plain CapabilityStatus, so scoped health authorization is NOT
|
|
817
|
+
* representable in v0.1. Its own exit criterion ("scoped permissions are
|
|
818
|
+
* representable") is therefore only partly met.
|
|
819
|
+
*/
|
|
820
|
+
health: CapabilityStatus;
|
|
821
|
+
backgroundExecution: CapabilityStatus;
|
|
822
|
+
/** GAP (flagged): photos/media appears in the design doc's native inputs and
|
|
823
|
+
* its full-vs-limited scope discussion, but there is NO photos key in the
|
|
824
|
+
* frozen CapabilitySnapshot. Not added here. */
|
|
825
|
+
quality: StateQuality;
|
|
826
|
+
}
|
|
827
|
+
|
|
828
|
+
/** FROZEN v0.1 (VOCABULARY §"Initial Capability Events"). All ten. */
|
|
829
|
+
export type CapabilityEvent =
|
|
830
|
+
| 'capabilityChanged'
|
|
831
|
+
| 'permissionChanged'
|
|
832
|
+
| 'capabilityAvailable'
|
|
833
|
+
| 'capabilityUnavailable'
|
|
834
|
+
/** NOTE: this pair does not match the documented naming patterns. The docs
|
|
835
|
+
* neither flag nor reconcile it; carried verbatim rather than "fixed". */
|
|
836
|
+
| 'capabilityBusy'
|
|
837
|
+
| 'capabilityFree'
|
|
838
|
+
| 'restrictionChanged'
|
|
839
|
+
| 'locationPrecisionChanged'
|
|
840
|
+
| 'backgroundPermissionChanged'
|
|
841
|
+
| 'capabilityStateStale';
|
|
842
|
+
|
|
843
|
+
/**
|
|
844
|
+
* NO INTENT TYPE — the one domain that is intent-less by explicit reasoning
|
|
845
|
+
* rather than by omission: "Capability State itself should generally be cheap
|
|
846
|
+
* and mostly event-driven, so it does not need many sensing profiles."
|
|
847
|
+
* A candidate `interactive/background/minimal` vocabulary is floated in the
|
|
848
|
+
* design doc and explicitly conditional ("only be kept if implementation
|
|
849
|
+
* proves they add value"). Nothing frozen. Not added.
|
|
850
|
+
*
|
|
851
|
+
* Also NOT here: any `requestPermission()`. Permission-request UX is outside
|
|
852
|
+
* this state engine by design, and a floated generic primitive is future work.
|
|
853
|
+
*/
|
|
854
|
+
export interface CapabilityDomain {
|
|
855
|
+
getSnapshot(): Promise<CapabilitySnapshot>;
|
|
856
|
+
subscribe(listener: StateListener<CapabilityEvent>): Subscription;
|
|
857
|
+
}
|
|
858
|
+
|
|
859
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
860
|
+
// 8. Domain 7 — Environment State
|
|
861
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
862
|
+
|
|
863
|
+
/** FROZEN (VOCABULARY §"Environment Intents"). */
|
|
864
|
+
export type EnvironmentIntent =
|
|
865
|
+
| 'ambient'
|
|
866
|
+
| 'navigation'
|
|
867
|
+
| 'accessibility'
|
|
868
|
+
| 'proximity'
|
|
869
|
+
| 'safety'
|
|
870
|
+
| 'lowPower';
|
|
871
|
+
|
|
872
|
+
export interface EnvironmentSnapshot {
|
|
873
|
+
/** Privacy is part of the SHAPE, not advice: this says `dim`, never a camera
|
|
874
|
+
* frame; `loud`, never microphone audio; a beacon count, never a BLE
|
|
875
|
+
* payload. Raw audio/images/BLE payloads are not part of this domain. */
|
|
876
|
+
lighting: {
|
|
877
|
+
level: 'dark' | 'dim' | 'normal' | 'bright' | 'veryBright' | 'unknown';
|
|
878
|
+
lux?: number | null;
|
|
879
|
+
confidence?: Confidence;
|
|
880
|
+
};
|
|
881
|
+
acoustics: {
|
|
882
|
+
level: 'quiet' | 'moderate' | 'loud' | 'veryLoud' | 'unknown';
|
|
883
|
+
approximateDb?: number | null;
|
|
884
|
+
confidence?: Confidence;
|
|
885
|
+
};
|
|
886
|
+
/** `uncertain` and `unknown` are different: uncertain = we looked and cannot
|
|
887
|
+
* decide; unknown = we have no basis to decide. */
|
|
888
|
+
enclosure: {
|
|
889
|
+
state: 'indoor' | 'outdoor' | 'uncertain' | 'unknown';
|
|
890
|
+
confidence?: Confidence;
|
|
891
|
+
};
|
|
892
|
+
proximity?: {
|
|
893
|
+
distanceClass?: 'immediate' | 'near' | 'far' | 'unknown';
|
|
894
|
+
confidence?: Confidence;
|
|
895
|
+
};
|
|
896
|
+
/** The only sub-block with no confidence — these are presence booleans, not
|
|
897
|
+
* estimates. Note they cannot distinguish unsupported / denied / switched
|
|
898
|
+
* off; that distinction lives in `capability`. */
|
|
899
|
+
nearbySignals?: {
|
|
900
|
+
bleAvailable?: boolean;
|
|
901
|
+
beaconCount?: number | null;
|
|
902
|
+
uwbAvailable?: boolean;
|
|
903
|
+
nfcAvailable?: boolean;
|
|
904
|
+
};
|
|
905
|
+
quality: StateQuality;
|
|
906
|
+
}
|
|
907
|
+
|
|
908
|
+
/** FROZEN v0.1 (VOCABULARY §"Initial Environment Events"). All ten. */
|
|
909
|
+
export type EnvironmentEvent =
|
|
910
|
+
| 'lightingChanged'
|
|
911
|
+
| 'environmentBecameDark'
|
|
912
|
+
| 'environmentBecameBright'
|
|
913
|
+
| 'noiseLevelChanged'
|
|
914
|
+
| 'environmentBecameLoud'
|
|
915
|
+
| 'indoorOutdoorChanged'
|
|
916
|
+
| 'proximityChanged'
|
|
917
|
+
| 'nearbySignalAppeared'
|
|
918
|
+
| 'nearbySignalDisappeared'
|
|
919
|
+
| 'environmentStateStale';
|
|
920
|
+
|
|
921
|
+
export interface EnvironmentDomain {
|
|
922
|
+
getSnapshot(): Promise<EnvironmentSnapshot>;
|
|
923
|
+
start(options: StartOptions<EnvironmentIntent>): Promise<StateSession>;
|
|
924
|
+
stop(sessionId?: string): Promise<void>;
|
|
925
|
+
subscribe(listener: StateListener<EnvironmentEvent>): Subscription;
|
|
926
|
+
}
|
|
927
|
+
|
|
928
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
929
|
+
// 9. Domain 8 — Interaction State
|
|
930
|
+
//
|
|
931
|
+
// Reports interaction FACTS. It never decides "pause the guide" or "speak now"
|
|
932
|
+
// — that is orchestration and belongs to the Plugin Tool / Skill / AI.
|
|
933
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
934
|
+
|
|
935
|
+
export interface InteractionSnapshot {
|
|
936
|
+
lifecycle: {
|
|
937
|
+
appState: 'foreground' | 'background' | 'inactive' | 'unknown';
|
|
938
|
+
screenOn: boolean | null;
|
|
939
|
+
userActive?: boolean | null;
|
|
940
|
+
idleDurationMs?: number | null;
|
|
941
|
+
};
|
|
942
|
+
audio: {
|
|
943
|
+
captureActive: boolean | null;
|
|
944
|
+
playbackActive: boolean | null;
|
|
945
|
+
route:
|
|
946
|
+
| 'speaker'
|
|
947
|
+
| 'earpiece'
|
|
948
|
+
| 'wiredHeadset'
|
|
949
|
+
| 'bluetooth'
|
|
950
|
+
| 'carAudio'
|
|
951
|
+
| 'other'
|
|
952
|
+
| 'unknown';
|
|
953
|
+
/** `none` (checked, nothing interrupting) and `unknown` (no knowledge) are
|
|
954
|
+
* deliberately different values. */
|
|
955
|
+
interruption: 'none' | 'temporary' | 'system' | 'unknown';
|
|
956
|
+
};
|
|
957
|
+
/** Optional GROUP: a platform that cannot report voice session state omits
|
|
958
|
+
* the whole block rather than guessing `false` for each field. */
|
|
959
|
+
voice?: {
|
|
960
|
+
sessionActive: boolean | null;
|
|
961
|
+
userSpeaking?: boolean | null;
|
|
962
|
+
assistantSpeaking?: boolean | null;
|
|
963
|
+
};
|
|
964
|
+
/** May report that a camera session is running; must NOT duplicate frame
|
|
965
|
+
* state. Frames belong to `visual`. */
|
|
966
|
+
visual?: {
|
|
967
|
+
cameraSessionActive: boolean | null;
|
|
968
|
+
};
|
|
969
|
+
quality: StateQuality;
|
|
970
|
+
}
|
|
971
|
+
|
|
972
|
+
/** FROZEN v0.1 (VOCABULARY §"Initial Interaction Events"). All twelve. */
|
|
973
|
+
export type InteractionEvent =
|
|
974
|
+
| 'appStateChanged'
|
|
975
|
+
| 'screenStateChanged'
|
|
976
|
+
| 'audioRouteChanged'
|
|
977
|
+
| 'audioInterruptionStarted'
|
|
978
|
+
| 'audioInterruptionEnded'
|
|
979
|
+
| 'voiceSessionStarted'
|
|
980
|
+
| 'voiceSessionStopped'
|
|
981
|
+
| 'cameraSessionStarted'
|
|
982
|
+
| 'cameraSessionStopped'
|
|
983
|
+
| 'userBecameActive'
|
|
984
|
+
| 'userBecameIdle'
|
|
985
|
+
| 'interactionStateStale';
|
|
986
|
+
|
|
987
|
+
/**
|
|
988
|
+
* NO INTENT TYPE. §10 declares only `InteractionSnapshot`, and the vocabulary
|
|
989
|
+
* — which does publish intents for spatial, visual, activity, connectivity and
|
|
990
|
+
* environment — has no "Interaction Intents" section. Five profiles are
|
|
991
|
+
* floated in the design doc, hedged twice ("Likely profiles, if needed") with
|
|
992
|
+
* no obligation attached to any of them. Not added.
|
|
993
|
+
*/
|
|
994
|
+
export interface InteractionDomain {
|
|
995
|
+
getSnapshot(): Promise<InteractionSnapshot>;
|
|
996
|
+
subscribe(listener: StateListener<InteractionEvent>): Subscription;
|
|
997
|
+
}
|
|
998
|
+
|
|
999
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1000
|
+
// 10. Root surface
|
|
1001
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1002
|
+
|
|
1003
|
+
/**
|
|
1004
|
+
* FROZEN name (API_FREEZE §11) and FROZEN member names.
|
|
1005
|
+
*
|
|
1006
|
+
* §11's "Recommended naming rule" says to call it `visual.getSnapshot()`, not
|
|
1007
|
+
* `device.visual.*`. Read here as a rule about the WIRE/tool-name convention
|
|
1008
|
+
* (`device.*` stays a human/backend routing convention for callable plugin
|
|
1009
|
+
* tools) rather than a ban on a plugin-prop container. That reading is a
|
|
1010
|
+
* judgement call and is flagged as such — if it is wrong, the fix is to rename
|
|
1011
|
+
* the PROP, never to merge these eight members into `PluginNative`.
|
|
1012
|
+
*/
|
|
1013
|
+
export interface DeviceBridgeV2 {
|
|
1014
|
+
readonly spatial: SpatialDomain;
|
|
1015
|
+
readonly visual: VisualDomain;
|
|
1016
|
+
readonly activity: ActivityDomain;
|
|
1017
|
+
readonly deviceContext: DeviceContextDomain;
|
|
1018
|
+
readonly connectivity: ConnectivityDomain;
|
|
1019
|
+
readonly capability: CapabilityDomain;
|
|
1020
|
+
readonly environment: EnvironmentDomain;
|
|
1021
|
+
readonly interaction: InteractionDomain;
|
|
1022
|
+
}
|
|
1023
|
+
|
|
1024
|
+
/** The name plugin authors see. See DESIGN NOTE 1 for why not `deviceBridge`. */
|
|
1025
|
+
export type DeviceState = DeviceBridgeV2;
|
|
1026
|
+
|
|
1027
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1028
|
+
// 11. Manifest gate — a SEPARATE declaration from `native`
|
|
1029
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1030
|
+
|
|
1031
|
+
/**
|
|
1032
|
+
* DECISION (not in the docs). A second, differently-named manifest field, so a
|
|
1033
|
+
* plugin cannot half-declare its way across the boundary:
|
|
1034
|
+
*
|
|
1035
|
+
* native: { camera: true } → props.device.camera.takePhoto()
|
|
1036
|
+
* stateDomains: ['visual', 'spatial'] → props.deviceState.visual.getSnapshot()
|
|
1037
|
+
*
|
|
1038
|
+
* Declaring `native.camera` grants NOTHING in the semantic layer, and vice
|
|
1039
|
+
* versa. There is no field whose meaning spans both.
|
|
1040
|
+
*
|
|
1041
|
+
* How an UNDECLARED domain must behave — and this is the load-bearing part:
|
|
1042
|
+
* it returns a normal snapshot in which every enum is `unknown`, every
|
|
1043
|
+
* nullable is `null`, and `quality.freshness` is
|
|
1044
|
+
* `{ state:'unknown', observedAt:null, ageMs:null }`. It does NOT throw and it
|
|
1045
|
+
* does NOT return undefined.
|
|
1046
|
+
*
|
|
1047
|
+
* Why that matters here specifically: today's `gatePluginNative` gives an
|
|
1048
|
+
* undeclared capability a proxy whose methods `console.warn` and return
|
|
1049
|
+
* `undefined` (packages/plugin-runtime/src/host.ts:67-76). So
|
|
1050
|
+
* `await native.location.getCurrent()` on an undeclared plugin resolves to
|
|
1051
|
+
* `undefined`, and `pos?.latitude` is `undefined` — which then flows into an
|
|
1052
|
+
* LLM answer as a confident nothing. A silent undefined is a fabrication
|
|
1053
|
+
* vector; a snapshot of explicit unknowns is not.
|
|
1054
|
+
*
|
|
1055
|
+
* The "why" is readable from `capability.getSnapshot()` using an existing
|
|
1056
|
+
* frozen enum member: a manifest-gate denial maps to
|
|
1057
|
+
* `restriction: 'policyRestricted'` — the plugin manifest IS a policy. No new
|
|
1058
|
+
* vocabulary needed. (The docs never consider a manifest gate, so this
|
|
1059
|
+
* mapping is a DECISION, but it uses only frozen values.)
|
|
1060
|
+
*/
|
|
1061
|
+
export type StateDomainDeclaration = readonly DeviceBridgeDomain[];
|
|
1062
|
+
|
|
1063
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1064
|
+
// 12. Small helpers — the "present does not mean fresh" guard
|
|
1065
|
+
// DECISION. Not frozen, not in any doc. Included because the single most
|
|
1066
|
+
// likely misuse of this API is reading a value and ignoring `quality`, and a
|
|
1067
|
+
// one-line guard is easier to reach for than a correct hand-rolled check.
|
|
1068
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1069
|
+
|
|
1070
|
+
const CONFIDENCE_ORDER: Record<ConfidenceLevel, number> = {
|
|
1071
|
+
unknown: 0,
|
|
1072
|
+
low: 1,
|
|
1073
|
+
medium: 2,
|
|
1074
|
+
high: 3,
|
|
1075
|
+
};
|
|
1076
|
+
|
|
1077
|
+
/**
|
|
1078
|
+
* Is this snapshot good enough to act on?
|
|
1079
|
+
*
|
|
1080
|
+
* Deliberately requires the caller to state a budget: there is no universal
|
|
1081
|
+
* "fresh enough", and the docs give no numeric thresholds for any domain
|
|
1082
|
+
* (native chooses sampling behaviour). A guidance loop wants ~1500 ms; a
|
|
1083
|
+
* battery-aware background job is happy with 60 s.
|
|
1084
|
+
*
|
|
1085
|
+
* `unknown` freshness fails: it means nothing was ever observed, which is not
|
|
1086
|
+
* the same as an old-but-real reading and must not be treated as one.
|
|
1087
|
+
*/
|
|
1088
|
+
export function isActionable(
|
|
1089
|
+
quality: StateQuality,
|
|
1090
|
+
budget: { maxAgeMs: number; minConfidence?: ConfidenceLevel },
|
|
1091
|
+
): boolean {
|
|
1092
|
+
const { freshness, confidence } = quality;
|
|
1093
|
+
if (freshness.state === 'unknown' || freshness.state === 'stale') return false;
|
|
1094
|
+
if (freshness.ageMs === null || freshness.ageMs > budget.maxAgeMs) return false;
|
|
1095
|
+
if (budget.minConfidence) {
|
|
1096
|
+
const have = CONFIDENCE_ORDER[confidence?.level ?? 'unknown'];
|
|
1097
|
+
if (have < CONFIDENCE_ORDER[budget.minConfidence]) return false;
|
|
1098
|
+
}
|
|
1099
|
+
return true;
|
|
1100
|
+
}
|
|
1101
|
+
|
|
1102
|
+
/** The honest "we have never observed this" value. Use it instead of
|
|
1103
|
+
* inventing a timestamp when constructing a degraded snapshot. This is the
|
|
1104
|
+
* canonical source of the `unknown` freshness shape — the engine's
|
|
1105
|
+
* `qualityUnknown()` derives from it so the two never drift. */
|
|
1106
|
+
export function neverObserved(): Freshness {
|
|
1107
|
+
return { state: 'unknown', observedAt: null, ageMs: null };
|
|
1108
|
+
}
|
|
1109
|
+
|
|
1110
|
+
/**
|
|
1111
|
+
* One human-readable sentence for why a capability cannot answer, built ONLY
|
|
1112
|
+
* from what the four axes actually say. Returns null when the capability is
|
|
1113
|
+
* fine — so a caller cannot accidentally print a reason that does not exist.
|
|
1114
|
+
*/
|
|
1115
|
+
export function explainCapability(status: CapabilityStatus): string | null {
|
|
1116
|
+
if (status.support === 'unsupported') return 'this device does not have it';
|
|
1117
|
+
if (status.permission === 'denied') return 'permission was denied';
|
|
1118
|
+
if (status.permission === 'restricted') return 'permission is restricted';
|
|
1119
|
+
if (status.permission === 'notDetermined') return 'permission has not been asked for yet';
|
|
1120
|
+
if (status.permission === 'limited') return 'only limited access was granted';
|
|
1121
|
+
if (status.restriction === 'policyRestricted') return 'this plugin did not declare it';
|
|
1122
|
+
if (status.restriction === 'backgroundRestricted') return 'the app is in the background';
|
|
1123
|
+
if (status.restriction === 'lowPowerRestricted') return 'the device is in low-power mode';
|
|
1124
|
+
if (status.availability === 'busy') return 'it is in use by something else';
|
|
1125
|
+
if (status.availability === 'temporarilyUnavailable') return 'it is temporarily unavailable';
|
|
1126
|
+
if (status.availability === 'unavailable') return 'it is unavailable';
|
|
1127
|
+
if (status.support === 'unknown' || status.availability === 'unknown') {
|
|
1128
|
+
return 'its state could not be determined';
|
|
1129
|
+
}
|
|
1130
|
+
return null;
|
|
1131
|
+
}
|
|
1132
|
+
|
|
1133
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1134
|
+
// 13. Integration seams into the existing SDK
|
|
1135
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1136
|
+
|
|
1137
|
+
/**
|
|
1138
|
+
* Additive extension of `PluginProps` (sdk/types.ts:259).
|
|
1139
|
+
*
|
|
1140
|
+
* `deviceState` is OPTIONAL because a host that has not installed a v2
|
|
1141
|
+
* provider must not pretend to have one. `undefined` means "this host has no
|
|
1142
|
+
* semantic layer", which a plugin can check once; it does NOT mean "the
|
|
1143
|
+
* sensors said nothing". Those are different, and collapsing them would
|
|
1144
|
+
* reintroduce exactly the ambiguity this file exists to remove.
|
|
1145
|
+
*/
|
|
1146
|
+
export interface PluginPropsV2Extension {
|
|
1147
|
+
readonly deviceState?: DeviceState;
|
|
1148
|
+
}
|
|
1149
|
+
|
|
1150
|
+
/**
|
|
1151
|
+
* Additive extension of `DeviceToolContext` (sdk/types.ts:320-327).
|
|
1152
|
+
*
|
|
1153
|
+
* This is the seam that matters most: device tools are what the AI calls, and
|
|
1154
|
+
* device tools are where a fabricated value reaches a user. The lint rule
|
|
1155
|
+
* proposed alongside this file (`no-mechanical-sensing-in-device-tools`) bans
|
|
1156
|
+
* `ctx.native.camera`, `ctx.native.location` and `ctx.native.connectivity`
|
|
1157
|
+
* inside a `DeviceToolDefinition.run` body — a device tool must reach sensing
|
|
1158
|
+
* through `ctx.deviceState`. UI code is unaffected.
|
|
1159
|
+
*/
|
|
1160
|
+
export interface DeviceToolContextV2Extension {
|
|
1161
|
+
readonly deviceState?: DeviceState;
|
|
1162
|
+
}
|
|
1163
|
+
|
|
1164
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1165
|
+
// DESIGN NOTES
|
|
1166
|
+
// ═════════════════════════════════════════════════════════════════════════
|
|
1167
|
+
|
|
1168
|
+
/**
|
|
1169
|
+
* DESIGN NOTE 1 — why the prop is `deviceState`, not `deviceBridge`.
|
|
1170
|
+
*
|
|
1171
|
+
* "Device Bridge" already names three different things in this codebase:
|
|
1172
|
+
* - `DeviceBridge`, the telemetry sync class that polls collectors and
|
|
1173
|
+
* pushes to the relay (packages/expo/src/device-bridge/index.ts:60);
|
|
1174
|
+
* - `PluginNative`, which sdk/types.ts:323 documents verbatim as
|
|
1175
|
+
* "Device Bridge — camera, location, sensors, storage";
|
|
1176
|
+
* - the v2 semantic layer in this file.
|
|
1177
|
+
* A fourth use of the word would be the opposite of unmistakable. `deviceState`
|
|
1178
|
+
* says what the object is (state, not actions) and collides with nothing.
|
|
1179
|
+
*
|
|
1180
|
+
*
|
|
1181
|
+
* DESIGN NOTE 2 — why NOT `PluginNative.visual` / `PluginNative.connectivityState`.
|
|
1182
|
+
*
|
|
1183
|
+
* The implementation-grounding doc proposes adding the eight domains as fields
|
|
1184
|
+
* on `PluginNative`, with `connectivityState` to dodge the collision with the
|
|
1185
|
+
* existing `connectivity`. Rejected, for three reasons:
|
|
1186
|
+
*
|
|
1187
|
+
* (a) It puts `native.camera.takePhoto()` and `native.visual.getSnapshot()`
|
|
1188
|
+
* on one object, one keystroke apart in autocomplete. The layer becomes
|
|
1189
|
+
* invisible at the call site, which is the failure being designed out.
|
|
1190
|
+
* (b) `native.connectivity` vs `native.connectivityState` is a confusable
|
|
1191
|
+
* pair by construction — two fields, one object, near-identical names,
|
|
1192
|
+
* different layers, different return shapes. `deviceState.connectivity`
|
|
1193
|
+
* vs `native.connectivity` is disambiguated by the first token instead.
|
|
1194
|
+
* (c) The gate model would have to span both meanings of one manifest field.
|
|
1195
|
+
*
|
|
1196
|
+
* The grounding doc's actual constraint is §14's "Do not create a second
|
|
1197
|
+
* plugin runtime" — and this design does not. Same runtime, same registration,
|
|
1198
|
+
* same bundle pipeline, same props object; one additional namespace on it.
|
|
1199
|
+
* Everything else in that doc's file-by-file plan (resident engines, warm
|
|
1200
|
+
* sessions, reference counting, no pipeline per tool call, the p50<10ms /
|
|
1201
|
+
* p95<30ms snapshot budget) stands unchanged.
|
|
1202
|
+
*
|
|
1203
|
+
*
|
|
1204
|
+
* DESIGN NOTE 3 — what this file does NOT decide, because the docs did not.
|
|
1205
|
+
*
|
|
1206
|
+
* - Per-intent behavioural contracts. The architecture doc MANDATES that each
|
|
1207
|
+
* intent document latency goal, quality target, continuous-vs-one-shot,
|
|
1208
|
+
* rolling-history behaviour, power posture, background behaviour, required
|
|
1209
|
+
* permissions and output shape. No document supplies these for ANY of the
|
|
1210
|
+
* nine visual intents, and there is no equivalent table for activity. The
|
|
1211
|
+
* vocabulary is minted; the semantics behind each name are not. This is the
|
|
1212
|
+
* largest hole in the whole design and no amount of TypeScript closes it.
|
|
1213
|
+
* - Multi-consumer session ownership / reference counting.
|
|
1214
|
+
* - MediaHandle → bytes resolution, and expiry behaviour.
|
|
1215
|
+
* - Whether `visual.capture()` may use a different intent from the session.
|
|
1216
|
+
* - Event vocabularies for visual, activity, spatial, deviceContext.
|
|
1217
|
+
* - Per-event payload schemas for every domain.
|
|
1218
|
+
* - Background/foreground semantics for activity and connectivity (specified
|
|
1219
|
+
* only for visual).
|
|
1220
|
+
* - Numeric thresholds for fresh/aging/stale, and for "materially changed".
|
|
1221
|
+
* - Cross-domain conflict resolution (activity says walking, spatial speed
|
|
1222
|
+
* says vehicle) — explicitly deferred as future State Engine Fusion and
|
|
1223
|
+
* out of v2 scope. A v2 snapshot must be internally coherent; it is NOT
|
|
1224
|
+
* obliged to reconcile itself with another domain.
|
|
1225
|
+
*/
|