arcane-os 0.84.0 → 0.85.0

package/docs/reference/model-observation.md

@@ -0,0 +1,220 @@
+# Existing model observation
+
+Model observation reads the existing renderer and Core owners selected by the
+application. It exposes their current lifecycle, progress, retained errors and
+actual operation identities to an explicitly selected developer observer.
+Model loading, inference, selection, cancellation and saved application state
+remain with those owners. Observation creates no model, accessor or Core runtime.
+
+## Compose the existing Core
+
+```js
+import {createModelObservationService} from 'arcane-os/core/model-observation';
+
+const observation = createModelObservationService({
+    runtime: host.runtime,
+    localAI: existingLocalAIService,
+    image: existingImageService,
+    decisions: existingDecisionService
+});
+host.runtime.registerService(observation);
+```
+
+`host` and the three services in this example are the application's existing
+instances. `localAI`, `image` and `decisions` are optional and default to `null`.
+Supply their actual service definitions, including `current()` and `methods`.
+The constructor neither looks up nor starts a service. Registration can follow
+`startCoreHost()` on its returned runtime, before opening an additional listener;
+it requires no model readiness barrier.
+
+The service is named `model-observation` and exposes `current()`, `dispose()` and
+the fixed Core methods below. `dispose()` retires only its observation requests
+and subscriptions. The host continues to own the Core and model services.
+
+An external local caller can connect through the explicitly selected
+[`startCoreListener`](core-shared-host.md) endpoint on that same runtime.
+For a packaged window, `native.launchContext.coreListener: {endpoint}` selects
+that same-runtime listener. Before binding it, the generated Core entry registers
+`model-observation` with the actual existing `services` definitions that expose
+`localai.status`, `image.status` and `decisions.status`; absent owners are `null`.
+The observer reuses those model-service owners and creates no accessor or model.
+The application supplies its renderer attachment separately.
+The observation module does not discover endpoints, launch a host, reconnect,
+change a profile, or create a second runtime. Native DOM inspection and actions
+retain their separate [app-control contract](native-app-control.md).
+
+## Attach existing renderer owners
+
+```js
+import * as aiRuntimeState from 'arcane-os/ai-runtime-state';
+import {attachModelObservation} from 'arcane-os/ai/model-observation';
+
+const attachment = attachModelObservation({
+    client: existingCoreClient,
+    aiRuntimeState,
+    imageRuntime: existingImageAccessor,
+    decisionModel: existingBrowserDecisionModel,
+    modelController: existingModelController,
+    signal: pageLifetime.signal,
+    onError: function reportObservationFailure(error) {
+        developerDiagnostics.report(error);
+    }
+});
+await attachment.ready;
+```
+
+Each optional owner defaults to `null`. Pass the instances already used by the
+application. Creating another image accessor would describe that new accessor's
+requests rather than the existing work. The supported owner methods are:
+
+| Input | Read | Subscribe |
+|---|---|---|
+| `aiRuntimeState` | `getAIRuntimeState()` | `subscribeAIRuntimeState()` |
+| `imageRuntime` | `current()` | `subscribe()` |
+| `decisionModel` | `status()` | `subscribe()` |
+| `modelController` | `status()` | `on('statechange')`, `on('progress')` |
+
+The attachment returns `{ready, closed, current, close}`. `ready` resolves with
+`{attachmentRequestId, coreRequestId}` after Core acknowledges the attachment.
+`current()` returns the latest complete renderer snapshot read by this
+attachment, or `null` when it was cancelled before reading. Subscriptions begin
+after acknowledgement; the attachment then reads and publishes current state
+again to cover changes during that handshake.
+
+Caller abort, `pagehide`, retirement of the exact installed Core client,
+transport failure, or `close()` removes the observation subscriptions and
+cancels only this attachment's observation RPCs. A supplied client remains
+caller-owned. The attachment never follows a replacement client or automatically
+reattaches. Native document retirement also cancels its long-lived Core request.
+Core then removes the renderer record; late publication to that retired
+attachment fails honestly. Detachment does not unload or cancel a model.
+
+`close()` returns `closed`, which settles after the attachment's observation
+RPC promises settle locally. It does not claim that a remote model or host has
+closed. Transport and observation errors remain available through `onError`
+and the rejected public promises. With `onError` omitted they reach the developer
+console. Normal cancellation, including cancellation initiated by the existing
+client or native document owner, closes observation without a model error. When
+cancellation precedes acknowledgement, `ready` rejects with the cancellation
+while `closed` resolves after the observation RPCs settle.
+
+## Read or subscribe from an existing client
+
+```js
+import {subscribeModelObservation} from 'arcane-os/ai/model-observation';
+
+const current = await client.invoke('model.observation.current', {}, {signal});
+const observation = subscribeModelObservation({
+    client,
+    signal,
+    onState: function showRetainedOwners(snapshot) {
+        developerDiagnostics.show(snapshot);
+    },
+    onDiagnostic: function showModelFailure(record) {
+        developerDiagnostics.report(record);
+    },
+    onError: function showObservationFailure(error) {
+        developerDiagnostics.report(error);
+    }
+});
+await observation.ready;
+```
+
+`onState` is required. The returned `{ready, closed, current, close}` has the same
+observation-only lifecycle as the renderer attachment. `ready` resolves with
+the first current-state replay; `current()` holds the latest complete snapshot,
+initially `null`. `onDiagnostic` explicitly selects live correlated Core model
+failure frames. Omitting it requests state observation only. Callback failures
+close that observation and reach `onError`; callback promises remain
+caller-owned and may await `close()` without becoming its completion barrier.
+
+Every snapshot is:
+
+```js
+{
+    core: existingCoreRuntime.current(),
+    native: {localAI, image, decisions},
+    renderers: [{attachmentRequestId, coreRequestId, owners: {
+        aiRuntimeState, imageRuntime, decisionModel, modelController
+    }}]
+}
+```
+
+The values represent complete supplied-owner snapshots. Absent optional owners
+are `null`; absent renderer attachments produce an empty array. Core state also
+contains its real active observation requests, identified by their actual
+methods, alongside other active requests. Observation does not relabel them as
+model work. Snapshots are current observations of independent owners, rather
+than an atomic transaction across all owners.
+
+## Fixed transport and correlation
+
+| Method/event | Contract |
+|---|---|
+| `model.observation.current` | Empty parameters; returns the current snapshot without model refresh or readiness waits. |
+| `model.observation.watch` | `{diagnostics:false}` by default; remains pending until cancellation. |
+| `model.observation.state` | Targeted watch event `{watchRequestId,snapshot}`; the first event acknowledges the watch and replays current state. |
+| `model.observation.error` | Targeted diagnostic `{watchRequestId,source:'core',method,frame}` preserving the complete original failed response frame. |
+| `model.observation.renderer.attach` | `{snapshot}`; remains pending to own the renderer publication lifetime. |
+| `model.observation.renderer.attached` | Targeted acknowledgement `{attachmentRequestId,coreRequestId}`. |
+| `model.observation.renderer.publish` | `{attachmentRequestId,coreRequestId,snapshot}`; replaces that attachment's current snapshot and returns both attachment identities. |
+
+Both helpers register listeners before invoking. Existing
+[`client.invoke`](core-client.md) supplies `onRequest({requestId})` before send;
+long-lived observation requests use `timeoutMs:0`. Their acknowledgement is a
+targeted request event, while the invocation promise remains pending.
+
+`attachmentRequestId` and `watchRequestId` are the actual client-visible Core
+request identities supplied by service context `clientRequestId`.
+`coreRequestId` is the actual internal Core frame identity;
+renderer attachments are keyed by that value so clients with identical local
+request IDs cannot collide. The renderer echoes both values when publishing.
+`snapshot.core.activeRequests[].id` and diagnostic `frame.id` use internal Core
+identities. They can differ from client-visible IDs under connection routing.
+None of these observation/transport IDs is a model-operation ID.
+
+Core supplies both metadata fields for direct runtime calls, stdio connections,
+`startCoreListener()` connections and the existing `startSharedCoreHost()` path.
+Each connection preserves the original incoming request ID as `clientRequestId`
+independently of its established `requestId` mapping. Direct calls use the
+incoming frame ID for both identities. Observation requires these actual
+context fields; it does not infer them from `requestId` or rewrite event
+payloads to manufacture correlation.
+
+The exact `methods` keys on the supplied native service definitions select the
+Core operations whose live failures can be observed. The observer tracks their
+actual active request IDs while a watch exists. Core publishes request removal
+before emitting its response; the observer retains that correlation until the
+response arrives, then removes it. Removing the final watcher releases frame
+observation and remaining transient correlation. It retains no failure history.
+
+## Evidence and lifecycle limits
+
+- AIRuntimeState exposes its existing llm/stt/tts records, including the reported
+  operation ID. Its retained errors contain only code/message. Concurrent role
+  work is not a complete request enumeration in this snapshot.
+- The existing image accessor exposes its own active method, stream ID, status
+  and progress records. Its retained error is preserved; that error has no
+  reliable explicit stream association. Constructor-only diagnostic callbacks
+  cannot be retroactively attached through this API.
+- Browser and native decision snapshots retain their existing errors and active
+  request counts. A count does not supply missing per-evaluation IDs.
+- ModelController status exposes its existing state/progress and code/message
+  error. Its load/unload event IDs are not retained by `status()`, and are not
+  presented as inference IDs or reconstructed history here.
+- Core supplies its real active request identities and complete live failed
+  response frames. A watch cannot recover responses that settled before it
+  observed them. Retained owner errors remain available in current snapshots.
+
+Renderer `Error` values in owner error fields use the existing Core error
+serializer at the JSON boundary; other owner fields remain unchanged. A value
+the selected transport cannot encode fails through the observation error path
+rather than being replaced with shortened or invented content. Observation
+does not alter model inputs, outputs, prompts or their persistence policy.
+
+The shared JavaScript contract is portable wherever the existing Core client
+and supplied owners are available. Endpoint and native-host availability belong
+to their documented adapters. Developer observation remains separate from
+ordinary user status, app-owned stage/avatar association and actual model
+execution verification. This capability does not establish that an inference
+failure has been corrected.

package/docs/reference/sdk-api.md

@@ -55,6 +55,8 @@ runtime layouts.
 | `arcane-os/core/contracts` | Shared Core protocol, frame, method and diagnostic error contracts. |
 | `arcane-os/core/classic-source` | Node generator for the canonical classic-script Core client projection. |
 | `arcane-os/core/runtime` | App-neutral native Core dispatcher and service lifecycle. |
+| `arcane-os/core/model-observation` | Passive observation of an existing Core and its actual native model service owners. |
+| `arcane-os/ai/model-observation` | Attach existing renderer model owners and subscribe through an existing Core client. |
 | `arcane-os/core/development` | App-service composition and document-owned Core transport during source development, independently of local AI. |
 | `arcane-os/core/stdio` | Framed Node stdio transport with graceful runtime drain. |
 | `arcane-os/core/host` | Node-only reusable Core host lifecycle, app-selected native launch defaults and shared native location resolution. |
@@ -420,6 +422,9 @@ The remaining data-only subpaths are eight JSON Schemas and package metadata.
 | `McpProtocolError` | class | `arcane-os/mcp/stdio` | MCP STDIO | Node |
 | `createCoreClassicSource()` | function | `arcane-os/core/classic-source` | Core classic client projection | Node; generated source runs in browser and native WebView renderers |
 | `createCoreRuntime()` | function | `arcane-os/core/runtime` | Native Core runtime | Node on Windows, Linux and macOS native hosts; Android requires host adaptation |
+| `createModelObservationService()` | function | `arcane-os/core/model-observation` | Existing model observation | Node with an existing Core runtime and optional existing native model services |
+| `attachModelObservation()` | function | `arcane-os/ai/model-observation` | Existing model observation | JavaScript with an existing Core client and optional existing renderer owners |
+| `subscribeModelObservation()` | function | `arcane-os/ai/model-observation` | Existing model observation | JavaScript with an existing Core client and composed observation service |
 | `encodeCoreFrame()` | function | `arcane-os/core/stdio` | Native Core stdio transport | Node |
 | `createCoreFrameDecoder()` | function | `arcane-os/core/stdio` | Native Core stdio transport | Node |
 | `startCoreStdio()` | function | `arcane-os/core/stdio` | Native Core stdio transport | Node |
@@ -7395,13 +7400,17 @@ Import it from `arcane-os/core/runtime`. The runtime exposes `current()`,
 `subscribe()`, `onFrame()`, `emit()`, `registerService()`, `getService()`, `start()`, `handle()`
 and idempotent `close()`.
 
-`runtime.handle(frame, {contextRequestId})` optionally preserves a transport's
-original client ID in handler `context.requestId`, while `context.coreRequestId`
-always exposes the actual internal `frame.id`. Active requests, cancellation,
-response IDs and request-event envelopes retain that internal ID. Ordinary
-`handle(frame)` exposes both context IDs equal to `frame.id`. This metadata
-does not change parameters or event data; transports own client-facing protocol
-correlation. See [the runtime contract](core-runtime.md#state-and-frames).
+`runtime.handle(frame, {contextRequestId, contextClientRequestId})` accepts
+optional nonempty transport metadata strings. Handler `requestId` remains
+`contextRequestId ?? frame.id`; additive `clientRequestId` is
+`contextClientRequestId ?? contextRequestId ?? frame.id`; `coreRequestId` always
+exposes the actual internal `frame.id`. Direct `handle(frame)` exposes all three
+equal. Stdio and both IPC paths supply the actual client identity separately,
+preserving the owning shared-host adapter's established internal `requestId`.
+Active requests, cancellation, response IDs and request-event envelopes retain
+the internal ID. Parameters, results and event data stay unchanged; transports
+own client-facing protocol correlation. See
+[the runtime contract](core-runtime.md#state-and-frames).
 
 `await runtime.getService(name)` and service `context.getService(name)` return
 the actual registered service after its shared startup. They wait for that
@@ -7588,6 +7597,80 @@ const transport = startCoreStdio({runtime});
 await transport.closed;
 ```
 
+## createModelObservationService()
+
+```text
+createModelObservationService({runtime,localAI=null,image=null,decisions=null}={})
+```
+
+Import from `arcane-os/core/model-observation`. Supply the existing Core runtime
+and actual native service definitions, including their `current()` and `methods`.
+Returns `{name:'model-observation',current,dispose,methods}`. Register it on that
+runtime after ordinary host creation and before opening an additional listener.
+Construction starts no service, model, worker or readiness wait.
+
+The four methods are `model.observation.current`, `model.observation.watch`,
+`model.observation.renderer.attach` and `model.observation.renderer.publish`.
+`current()` returns `{core,native:{localAI,image,decisions},renderers}` from the
+supplied owners. Absent model owners are `null`; absent attachments produce `[]`.
+The watch replays current state and remains pending until cancellation. Its
+`diagnostics:false` default excludes live failure frames; explicit `true`
+observes complete failed Core responses for the supplied owners' method keys.
+It retains no failure history and invents no model-operation identities.
+
+Renderer attachment and watch acknowledgements use actual `clientRequestId`;
+attachments are keyed by the actual `coreRequestId` to distinguish clients with
+equal local IDs. `dispose()` retires observation requests/subscriptions only.
+Core still owns model services and shutdown. See
+[complete methods, events and evidence limits](model-observation.md).
+
+## attachModelObservation()
+
+```text
+attachModelObservation({client,aiRuntimeState=null,imageRuntime=null,decisionModel=null,modelController=null,signal,onError}={})
+```
+
+Import from `arcane-os/ai/model-observation`. Pass the existing Core client and
+the application-owned instances already doing the work. Supported owner methods
+are AIRuntimeState's `getAIRuntimeState/subscribeAIRuntimeState`, image runtime's
+`current/subscribe`, decision model's `status/subscribe`, and ModelController's
+`status/on`. It creates no replacement owner or transport.
+
+Returns `{ready,closed,current,close}`. `ready` resolves with the Core-supplied
+`{attachmentRequestId,coreRequestId}` acknowledgement. Subscriptions then publish
+complete current owner snapshots, including a fresh read after acknowledgement.
+`current()` returns the latest renderer snapshot, initially `null`. The existing
+Core error serializer represents owner `Error` fields at the JSON boundary;
+other fields remain unchanged.
+
+Caller abort, page retirement, retirement of the exact installed client,
+transport failure or `close()` retires only observation RPCs/subscriptions.
+`close()` returns `closed`; it never closes the supplied client or unloads models.
+Normal cancellation closes cleanly; cancellation before acknowledgement rejects
+`ready`. Other errors reject public promises and reach `onError`, defaulting to
+complete console diagnostics. A replacement client requires a new explicit
+attachment. Keep attachment readiness independent of page rendering. See
+[renderer ownership and lifecycle](model-observation.md#attach-existing-renderer-owners).
+
+## subscribeModelObservation()
+
+```text
+subscribeModelObservation({client,signal,onState,onDiagnostic,onError}={})
+```
+
+Import from `arcane-os/ai/model-observation`. Requires an existing client and
+`onState`; returns `{ready,closed,current,close}`. `ready` resolves with the first
+current-state snapshot and `current()` retains the latest snapshot, initially
+`null`. Providing `onDiagnostic` explicitly selects live complete correlated Core
+model failure frames; omission selects state only. These are developer
+diagnostics, not ordinary user status or saved conversation content.
+
+The observer uses the same observation-only cancellation/error lifecycle as
+the attachment. Callback failures retire it and reach `onError`; callback
+promises may await `close()` without becoming its completion barrier. It starts
+no model, refresh, polling, endpoint discovery, host or reconnect. See
+[observation records and limits](model-observation.md#read-or-subscribe-from-an-existing-client).
+
 ## createPortableNativeProvider()
 
 ### Overview
@@ -7621,6 +7704,13 @@ published SDK/runtime dependencies and the canonical Core entry/client into a
 fresh output directory. Service factories are imported only when a real native
 host later launches the generated Core entry, never during packaging.
 
+Explicit `native.launchContext.coreListener:{endpoint}` also composes one passive
+model observation service after `startCoreHost()` and before listener binding,
+using that runtime and existing services exposing `localai.status`, `image.status`
+and `decisions.status`. Missing owners remain `null`; renderer attachment stays
+application-owned. Ordinary stdio and shared/headless branches gain no model
+startup or readiness barrier. See [native observation composition](core-native-packaging.md#launch-time-locations).
+
 An optional `appDescriptor.native.window` record carries `width`, `height` and
 `resizable` into `manifest.window`. Dimensions are positive integral logical
 client units; `resizable` is an optional boolean. Omitted fields remain omitted.
@@ -8848,7 +8938,7 @@ borrowed listener implements no `core.host.shutdown` operation; use client
 `close()` for disconnection and the existing runtime's owner for host shutdown.
 
 Each connection owns response/event correlation and request cancellation.
-Handler `requestId` preserves the client ID; `coreRequestId` is the distinct
+Handler `requestId` and `clientRequestId` preserve the client ID; `coreRequestId` is the distinct
 internal dispatch ID. Parameters, results and event data, including
 `data.requestId`, remain unchanged. Runtime-state snapshots retain internal
 active IDs; no service event is invented or retained for later replay.
@@ -8867,6 +8957,9 @@ try {
 `host` is the application's existing stdio host. Generated native entries
 select this attachment with `native.launchContext.coreListener:{endpoint}`,
 preserving their actual runtime, service set, profile, origin and state root.
+That generated opt-in registers the passive observation service from existing
+native model owners before binding; direct `startCoreListener()` itself never
+registers services. Renderer owners are attached separately by the application.
 Omit `sharedHost`; the two options describe alternative lifetimes. Full
 contract: [existing-runtime listener](core-shared-host.md#attach-to-an-existing-window-owned-core).
 
@@ -8894,6 +8987,11 @@ service-owned. The supplied signal begins the same shutdown. `closed` rejects
 on shutdown failure; `onError` observes transport/background failures, with
 complete console diagnostics by default.
 
+The owning adapter preserves its existing internal handler `requestId`, equal
+to `coreRequestId`, and its matching `data.requestId` compatibility mapping.
+Additive `clientRequestId` exposes the original client ID independently; other
+parameters, results and event data remain unchanged.
+
 ### Availability and example
 
 Node supports app-selected named pipes on Windows and Unix-domain sockets on

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "arcane-os",
-  "version": "0.84.0",
+  "version": "0.85.0",
   "description": "Arcane OS JavaScript SDK, project-local CLI, browser runtime, and repository-portable application packager.",
   "type": "module",
   "main": "./src/index.mjs",
@@ -43,6 +43,7 @@
     "./ai/core-onnx": "./browser-runtime/ai/core-onnx.mjs",
     "./ai/core-image": "./browser-runtime/ai/core-image.mjs",
     "./ai/core-model-assets": "./browser-runtime/ai/core-model-assets.mjs",
+    "./ai/model-observation": "./browser-runtime/ai/model-observation.mjs",
     "./ai/browser-decisions": "./browser-runtime/ai/browser-decisions.mjs",
     "./ai/tool-text-stream": "./browser-runtime/ai/tool-text-stream.mjs",
     "./ai/twin-cloud": "./browser-runtime/ai/twin-cloud.mjs",
@@ -75,6 +76,7 @@
     "./core/image": "./src/core/services/image.mjs",
     "./core/speech": "./src/core/services/speech.mjs",
     "./core/model-assets": "./src/core/services/model-assets.mjs",
+    "./core/model-observation": "./src/core/services/model-observation.mjs",
     "./diarization": "./src/diarization/index.mjs",
     "./diarization/build": "./src/diarization/build.mjs",
     "./local-ai": "./src/local-ai/install.mjs",

package/src/core/runtime-connection.mjs

@@ -137,7 +137,10 @@ export function createCoreRuntimeConnection({
         try {
             return await runtime.handle(
                 {...frame, id: record.internalId},
-                preserveContextRequestId ? {contextRequestId: record.id} : undefined
+                {
+                    contextRequestId: preserveContextRequestId ? record.id : undefined,
+                    contextClientRequestId: record.id
+                }
             );
         } finally {
             releaseRequest();

package/src/core/runtime.mjs

@@ -218,7 +218,7 @@ class CoreRuntime {
         return this.current();
     }
 
-    async handle(frame, {contextRequestId} = {}) {
+    async handle(frame, {contextRequestId, contextClientRequestId} = {}) {
         if (frame?.protocol !== CORE_PROTOCOL) throw failure('INVALID_RPC_REQUEST', 'Unknown Core protocol.');
         if (frame.type === 'control') {
             if (frame.control === 'runtime.replay') {
@@ -238,7 +238,10 @@ class CoreRuntime {
         if (contextRequestId !== undefined && (!is.string(contextRequestId) || !contextRequestId)) {
             throw failure('INVALID_RPC_REQUEST', 'A transport context request id must be a nonempty string.');
         }
-        const task = this.#answer(frame, contextRequestId);
+        if (contextClientRequestId !== undefined && (!is.string(contextClientRequestId) || !contextClientRequestId)) {
+            throw failure('INVALID_RPC_REQUEST', 'A transport client request id must be a nonempty string.');
+        }
+        const task = this.#answer(frame, contextRequestId, contextClientRequestId);
         this.#responses.add(task);
         try {
             return await task;
@@ -247,7 +250,7 @@ class CoreRuntime {
         }
     }
 
-    async #answer(frame, contextRequestId) {
+    async #answer(frame, contextRequestId, contextClientRequestId) {
         let result;
         let error;
         let ok = true;
@@ -257,7 +260,7 @@ class CoreRuntime {
             if (frame.method === 'system.ping') result = {ok: true};
             else if (frame.method === 'version.current') result = this.#version;
             else if (frame.method === 'app.current') result = this.#application;
-            else result = await this.#request(frame, contextRequestId);
+            else result = await this.#request(frame, contextRequestId, contextClientRequestId);
         } catch (caught) {
             ok = false;
             error = caught;
@@ -273,7 +276,7 @@ class CoreRuntime {
         return response;
     }
 
-    #request(frame, contextRequestId) {
+    #request(frame, contextRequestId, contextClientRequestId) {
         const registration = this.#methods.get(frame.method);
         if (!registration) {
             const namespace = frame.method.split('.')[0];
@@ -303,8 +306,13 @@ class CoreRuntime {
                     const result = await handle.call(
                         service.definition,
                         frame.parameters,
-                        {...runtime.#serviceContext(service, frame.id),
-                            requestId: contextRequestId ?? frame.id, coreRequestId: frame.id, signal: controller.signal}
+                        {
+                            ...runtime.#serviceContext(service, frame.id),
+                            requestId: contextRequestId ?? frame.id,
+                            clientRequestId: contextClientRequestId ?? contextRequestId ?? frame.id,
+                            coreRequestId: frame.id,
+                            signal: controller.signal
+                        }
                     );
                     aborted(controller);
                     return result;