arcane-os 0.92.0 → 0.94.0

package/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 0.94.0
+
+- Recover all TWiN HTTP 429 quota responses through the shared chat/System One
+  transport. Honor Retry-After or exponential backoff, share a transient
+  same-endpoint/credential cooldown, and pace concurrent recovery dispatches.
+  Fetch rejection, 429 and 529 share three retries; cancellation stops waits.
+  Preserve exact requests and complete terminal errors, including provider
+  status_code and nested error records. Do not replay 400/413 errors, consumed
+  streams, callbacks, or image submissions.
+- Include the completed native Kokoro synthesis capability: retained voices,
+  explicit model preparation, owned cancellation and audio output through the
+  shared native speech contract. Reuse its retained compatible runtime and
+  source archives alongside the existing native host, Whisper and ONNX assets.
+  Package inclusion is distinct from application or native-runtime acceptance.
+
+## 0.93.0
+
+- Add native desktop notification status, submission, state and removal through
+  `Arcane.notifications`, with ordered notification state events and an owned
+  Windows adapter. The host reports actual submission and OS errors; submission
+  does not establish that a person saw or activated the notification.
+- Retain complete native document errors in the Windows host session, including
+  `window.error`, `unhandledrejection` and explicit `reportNativeError()` reports.
+  Retrieve non-consuming records through `connectAppControl().diagnostics()`
+  without foregrounding the app or waiting for Core/model readiness. Records
+  survive document navigation within that host session and end with the host.
+- Publish the matching rebuilt Windows host archive with its document-start
+  diagnostic helper. Keep ordinary console output separate from retained error
+  history; console interception and caught console arguments are outside this
+  release. Native notification behavior and application acceptance remain
+  consumer-owned verification, distinct from the selected compile and package.
+
 ## 0.92.0
 
 - Add opt-in `allowUnborn` Git text snapshots and `initialBranch` repository

package/README.md

@@ -25,7 +25,7 @@ selected SDK projection or an explicit live SDK source mount. Both profiles
 share the theme, packaging, event, cancellation,
 and browser run contracts while retaining their selected application layout.
 
-This checkout defines the `0.92.0` SDK contract. Development applications declare
+This checkout defines the `0.94.0` SDK contract. Development applications declare
 `arcane-os` as `latest` and retain their lockfile. Exact SDK pins begin when Roshi
 designates a production build. Follow [Consumer updates](docs/consumer-updates.md)
 to refresh and compare the installed SDK and lockfile with the published channel;

package/browser-runtime/ai/twin-cloud.mjs

@@ -4,6 +4,8 @@ import {arcaneLogging} from '../logging.mjs';
 const is = new Is(false);
 const twinChatURL = 'https://inference.do-ai.run/v1/chat/completions';
 const twinSystemOneURL = 'https://inference.do-ai.run/v1/systemone';
+const activeHTTPRequests = new Map();
+const httpRetryDelayMs = 3000;
 
 /** A stateless TWiN request; browser AI shares the HTTP and format owners below. */
 export async function fetchRequest({
@@ -450,10 +452,24 @@ export function openAIResponseFormat(format){
 /** Shared with browser streaming; consume no successful body at this boundary. */
 export async function fetchHTTPResponse(url, options, {onRetry = null} = {}) {
     const {signal} = options;
-    const retryDelayMs = 3000;
     const maxRecoveryRetries = 3;
-    let recoveryRetries = 0;
     let attempt = 0;
+    let retry = null;
+    const destination = url?.url ?? String(url);
+    const headers = new Headers(options.headers ?? url?.headers);
+    const requestOptions = {...options, headers};
+    const authorization = headers.get('authorization') ?? '';
+    let accounts = activeHTTPRequests.get(destination);
+    if (!accounts) {
+        accounts = new Map();
+        activeHTTPRequests.set(destination, accounts);
+    }
+    let pacing = accounts.get(authorization);
+    if (!pacing) {
+        pacing = {active: 0, rateLimited: false, resumeAt: 0, nextDispatchAt: 0, error: null};
+        accounts.set(authorization, pacing);
+    }
+    pacing.active += 1;
     try {
         while (true) {
             if (signal?.aborted) {
@@ -462,7 +478,7 @@ export async function fetchHTTPResponse(url, options, {onRetry = null} = {}) {
             let response;
             let error;
             try {
-                response = await fetch(url, options);
+                response = await dispatchHTTPRequest(url, requestOptions, pacing, onRetry, retry, attempt);
             } catch (fetchError) {
                 if (isAIRequestAbort(fetchError, signal)) {
                     throw normalizeAIRequestAbort(fetchError);
@@ -483,7 +499,7 @@ export async function fetchHTTPResponse(url, options, {onRetry = null} = {}) {
                         ? await response.json()
                         : await response.text();
                 } catch (bodyError) {
-                    if (status !== 529 || isAIRequestAbort(bodyError, signal)) {
+                    if ((status !== 429 && status !== 529) || isAIRequestAbort(bodyError, signal)) {
                         throw bodyError;
                     }
                     error = bodyError;
@@ -495,61 +511,108 @@ export async function fetchHTTPResponse(url, options, {onRetry = null} = {}) {
             const message = is.string(error)
                 ? error
                 : error?.error?.message ?? error?.message;
-            const overload = status === 429
-                && is.string(message)
-                && message.toLowerCase().includes('overload');
-            if (!overload) {
-                if ((status !== null && status !== 529)
-                    || recoveryRetries >= maxRecoveryRetries) {
-                    throw error;
-                }
-                recoveryRetries += 1;
+            const retryDelayMs = status === 429
+                ? httpRateLimitDelay(response, httpRetryDelayMs * (2 ** Math.min(attempt, maxRecoveryRetries - 1)))
+                : httpRetryDelayMs;
+            if (status === 429) {
+                pacing.rateLimited = true;
+                pacing.resumeAt = Math.max(pacing.resumeAt, Date.now() + retryDelayMs);
+                pacing.error = error;
+            }
+            if ((status !== null && status !== 429 && status !== 529)
+                || attempt >= maxRecoveryRetries) {
+                throw error;
             }
             attempt += 1;
             arcaneLogging.warn(
                 `${message ?? 'AI request transport failed.'}\nRetrying in ${retryDelayMs / 1000} seconds`,
                 error
             );
-            observeRequestRetry(
-                onRetry,
-                {phase: 'waiting', attempt, delayMs: retryDelayMs, status, error}
-            );
-            await new Promise(
-                function waitForRequestRetry(resolve, reject) {
-                    function finishRetryDelay() {
-                        signal?.removeEventListener('abort', cancelRetryDelay);
-                        resolve();
-                    }
-                    function cancelRetryDelay() {
-                        clearTimeout(timer);
-                        signal.removeEventListener('abort', cancelRetryDelay);
-                        reject(normalizeAIRequestAbort(signal.reason));
-                    }
-                    const timer = setTimeout(finishRetryDelay, retryDelayMs);
-                    signal?.addEventListener(
-                        'abort',
-                        cancelRetryDelay,
-                        {once: true}
-                    );
-                    if (signal?.aborted) {
-                        cancelRetryDelay();
-                    }
-                }
-            );
-            if (signal?.aborted) {
-                throw normalizeAIRequestAbort(signal.reason);
-            }
-            observeRequestRetry(
-                onRetry,
-                {phase: 'requesting', attempt, delayMs: retryDelayMs, status, error}
-            );
+            retry = {phase: 'waiting', attempt, delayMs: retryDelayMs, status, error};
+            observeRequestRetry(onRetry, retry);
+            await waitForHTTPDelay(retryDelayMs, signal);
         }
     } catch (error) {
         if (isAIRequestAbort(error, signal)) {
             throw normalizeAIRequestAbort(error);
         }
         throw error;
+    } finally {
+        pacing.active -= 1;
+        if (pacing.active === 0) {
+            // Retain neither credentials nor quota state beyond the active request group.
+            accounts.delete(authorization);
+            if (accounts.size === 0) {
+                activeHTTPRequests.delete(destination);
+            }
+        }
+    }
+}
+
+function httpRateLimitDelay(response, fallback) {
+    const value = response.headers.get('retry-after')?.trim();
+    if (!value) return fallback;
+    const seconds = Number(value);
+    if (is.finite(seconds)) {
+        return seconds >= 0 && is.finite(seconds * 1000) ? seconds * 1000 : fallback;
     }
+    // Accept HTTP weekday date forms, not an arbitrary numeric string interpreted as a date.
+    const date = /^(Mon|Tue|Wed|Thu|Fri|Sat|Sun)/i.test(value) ? Date.parse(value) : NaN;
+    return is.finite(date) ? Math.max(0, date - Date.now()) : fallback;
+}
+
+async function dispatchHTTPRequest(url, options, pacing, onRetry, retry, attempt) {
+    const {signal} = options;
+    let observation = null;
+    // Only an observed quota response coordinates otherwise independent requests.
+    while (pacing.rateLimited) {
+        if (signal?.aborted) throw normalizeAIRequestAbort(signal.reason);
+        const delayMs = Math.max(pacing.resumeAt, pacing.nextDispatchAt) - Date.now();
+        if (delayMs <= 0) {
+            // Claim and initiate Fetch in this continuation, with no intervening asynchronous wait.
+            pacing.nextDispatchAt = Date.now() + httpRetryDelayMs;
+            break;
+        }
+        if (!retry && !observation) {
+            observation = {phase: 'waiting', attempt, delayMs, status: 429, error: pacing.error};
+            observeRequestRetry(onRetry, observation);
+        }
+        await waitForHTTPDelay(delayMs, signal);
+    }
+    if (observation && !signal?.aborted) {
+        observeRequestRetry(onRetry, {...observation, phase: 'requesting'});
+    }
+    if (retry) {
+        observeRequestRetry(onRetry, {...retry, phase: 'requesting'});
+    }
+    if (signal?.aborted) throw normalizeAIRequestAbort(signal.reason);
+    return fetch(url, options);
+}
+
+async function waitForHTTPDelay(delayMs, signal) {
+    let remaining = delayMs;
+    do {
+        if (signal?.aborted) throw normalizeAIRequestAbort(signal.reason);
+        // Only split at the host timer range; never shorten the provider's requested wait.
+        const milliseconds = Math.min(remaining, 2147483647);
+        await new Promise(
+            function waitForRequestRetry(resolve, reject) {
+                function finishRetryDelay() {
+                    signal?.removeEventListener('abort', cancelRetryDelay);
+                    resolve();
+                }
+                function cancelRetryDelay() {
+                    clearTimeout(timer);
+                    signal.removeEventListener('abort', cancelRetryDelay);
+                    reject(normalizeAIRequestAbort(signal.reason));
+                }
+                const timer = setTimeout(finishRetryDelay, milliseconds);
+                signal?.addEventListener('abort', cancelRetryDelay, {once: true});
+                if (signal?.aborted) cancelRetryDelay();
+            }
+        );
+        remaining -= milliseconds;
+    } while (remaining > 0);
 }
 
 function observeRequestRetry(onRetry, state) {

package/browser-runtime/core/client.mjs

@@ -598,6 +598,8 @@ export function createCoreFacade(client){
             remove:name=>invoke('environment.delete',{name})},
         mail:{send:request=>invoke('mail.send',request??{},{timeoutMs:450000})},
         speech:{status:()=>invoke('speech.status',{},{timeoutMs:10000}),
+            load:function loadSpeech(request,options){return invoke('speech.load',request??{},{timeoutMs:0,signal:options?.signal,onRequest:options?.onRequest});},
+            unload:function unloadSpeech(request,options){return invoke('speech.unload',request??{},{timeoutMs:0,signal:options?.signal,onRequest:options?.onRequest});},
             synthesize:(request,options)=>invoke('speech.synthesize',request??{},{timeoutMs:180000,signal:options?.signal}),
             transcribe:(request,options)=>invoke('speech.transcribe',request??{},{timeoutMs:180000,signal:options?.signal,onRequest:options?.onRequest})},
         localAI:{status:()=>invoke('localai.status',{},{timeoutMs:15000}),
@@ -646,6 +648,24 @@ export function createCoreFacade(client){
         storage:{list:()=>invoke('storage.list'),get:key=>invoke('storage.get',{key}),set:(key,value)=>invoke('storage.set',{key,value}),delete:key=>invoke('storage.delete',{key})},
         preferences:{list:()=>invoke('preferences.list'),get:key=>invoke('preferences.get',{key}),set:(key,value)=>invoke('preferences.set',{key,value}),
             setMany:entries=>invoke('preferences.setMany',{entries}),delete:key=>invoke('preferences.delete',{key})},
+        notifications: {
+            status(options) {
+                return invoke('notifications.status', {}, options).catch(function notificationStatusUnavailable(error) {
+                    if (error?.code !== 'METHOD_NOT_ALLOWED' || error?.reason !== 'core-namespace-unavailable') throw error;
+                    return {supported: false, available: false, permissionDisabled: null,
+                        reason: 'host-notifications-unavailable', error};
+                });
+            },
+            show(request, options) {
+                return invoke('notifications.show', request, options);
+            },
+            state(selection = {}, options) {
+                return invoke('notifications.state', selection, options);
+            },
+            close(selection, options) {
+                return invoke('notifications.close', selection, options);
+            }
+        },
         window: {
             state(options) {
                 return invoke('window.state', {}, {signal: options?.signal});

package/browser-runtime/core/contracts.mjs

@@ -43,6 +43,26 @@ export const CORE_METHOD_CONTRACTS={
         output: '{requestedTarget,resolvedDevice,resolution,reason}',
         meaning: 'Resolve a saved physical identity. Explicit null selects engine-owned automatic resolution; it does not claim a configured or observed device.'
     },
+    'notifications.status': {
+        input: '{}',
+        output: '{supported,available,permissionDisabled,reason?,error?}',
+        meaning: 'Actual native host support and OS permission state; no preference changes or browser/audio substitution.'
+    },
+    'notifications.show': {
+        input: '{id,title,body,data}',
+        output: 'notification-record-v1',
+        meaning: 'Submit one exact application notification. Submitted means native Show returned, not display or human receipt. Caller IDs cannot be reused during this host lifetime.'
+    },
+    'notifications.state': {
+        input: '{id?}',
+        output: '{revision,notifications:notification-record-v1[]}',
+        meaning: 'Retained complete records for this application host lifetime, including actual terminal outcomes; accepted notices survive renderer navigation.'
+    },
+    'notifications.close': {
+        input: '{id}',
+        output: 'notification-record-v1',
+        meaning: 'Remove only the exact owned notice and retain its record. Cancellation after submission does not reverse human interaction.'
+    },
     'window.state': {
         input: '{}',
         output: '{platform,supported,state}',

package/browser-runtime/core/native-diagnostics.mjs

@@ -0,0 +1,217 @@
+const NATIVE_DIAGNOSTICS_KEY = Symbol.for('arcane-os.native-diagnostics');
+const NATIVE_DOCUMENT_KEY = Symbol.for('arcane-os.app-control.document');
+
+/** Report an application-caught error to this document's native host ledger. */
+export async function reportNativeError(error, options = {}) {
+    const owner = globalThis[NATIVE_DIAGNOSTICS_KEY];
+    if (!owner) throw nativeDiagnosticsUnavailable();
+    return owner.report(error, options);
+}
+
+/** The native host installs this once, at document creation, before app scripts. */
+export function installNativeDiagnostics(global = globalThis) {
+    if (global[NATIVE_DIAGNOSTICS_KEY]) return global[NATIVE_DIAGNOSTICS_KEY];
+
+    const documentId = global[NATIVE_DOCUMENT_KEY] ??= global.crypto.randomUUID();
+    let delivery = Promise.resolve();
+    const owner = {
+        documentId,
+        ready: null,
+        report(error, options = {}) {
+            return capture('application', error, options);
+        }
+    };
+    global[NATIVE_DIAGNOSTICS_KEY] = owner;
+    global.addEventListener('error', captureWindowError);
+    global.addEventListener('unhandledrejection', captureUnhandledRejection);
+    owner.ready = transmit(envelope('document'));
+    owner.ready.catch(reportCaptureFailure);
+    return owner;
+
+    function envelope(type, record) {
+        return {
+            protocol: 'arcane.native-diagnostics/1',
+            type,
+            documentId,
+            documentUrl: global.location.href,
+            time: new Date().toISOString(),
+            ...(record === undefined ? {} : {record})
+        };
+    }
+
+    async function capture(kind, error, options = {}, observation) {
+        // Snapshot synchronously at the reported failure, before any bridge wait.
+        const record = {kind, ...encodeDiagnosticValues(error, options)};
+        if (observation !== undefined) {
+            record.location = observation.location;
+            record.message = observation.message;
+        }
+        return transmit(envelope('error', record));
+    }
+
+    function transmit(frame) {
+        // This document has one ordered ingress. A failed write is observed by
+        // its caller and does not prevent subsequent independent reports.
+        const text = JSON.stringify(frame);
+        const pending = delivery.then(async function sendDiagnostic() {
+            const windows = global.chrome?.webview?.hostObjects?.arcaneDiagnostics;
+            const webkit = global.webkit?.messageHandlers?.arcaneDiagnostics;
+            let reply;
+            if (windows) reply = await windows.Send(text);
+            else if (webkit) reply = await webkit.postMessage(text);
+            else throw nativeDiagnosticsUnavailable();
+            const result = typeof reply === 'string' ? JSON.parse(reply) : reply;
+            if (result?.accepted !== true) {
+                const error = new Error('The native host did not retain the diagnostic.', {cause: result?.error});
+                error.code = 'ARCANE_NATIVE_DIAGNOSTIC_REJECTED';
+                error.details = result;
+                throw error;
+            }
+            return result;
+        });
+        delivery = pending.catch(function observeOrderedFailure() {});
+        return pending;
+    }
+
+    function captureWindowError(event) {
+        // Resource-element load failures are a different observation boundary;
+        // this listener does not inspect their DOM or request payloads.
+        if (event.target !== global) return;
+        capture('window.error', event.error, {},
+            {message: event.message, location: {file: event.filename, line: event.lineno, column: event.colno}}
+        ).catch(function failedWindowCapture(failure) {
+            reportCaptureFailure(failure, event.error);
+        });
+    }
+
+    function captureUnhandledRejection(event) {
+        capture('unhandledrejection', event.reason).catch(function failedRejectionCapture(failure) {
+            reportCaptureFailure(failure, event.reason);
+        });
+    }
+
+    function reportCaptureFailure(failure, original) {
+        // Observe our own asynchronous failure without manufacturing another
+        // unhandled rejection. The browser's ordinary error behavior continues.
+        global.console.error('Arcane native diagnostic capture failed.', failure, original);
+    }
+}
+
+function nativeDiagnosticsUnavailable() {
+    const error = new Error('This document has no native diagnostic capture host.');
+    error.code = 'ARCANE_NATIVE_DIAGNOSTICS_UNAVAILABLE';
+    return error;
+}
+
+/**
+ * Transport-local property graph, not JSON.stringify(error). Descriptor values
+ * preserve non-enumerable Error fields, symbols, cycles and shared references.
+ * Accessors remain descriptors; inspection never calls their get/set functions.
+ * An object that rejects reflection fails the report with that original error.
+ */
+function encodeDiagnosticValues(error, options) {
+    const references = new Map();
+    const pending = [];
+    const values = [];
+    const record = {valueFormat: 'arcane.diagnostic-value/1', error: encode(error), values};
+    const details = Object.getOwnPropertyDescriptor(options, 'details');
+    if (details) {
+        record.details = Object.hasOwn(details, 'value') ? encode(details.value)
+            : {type: 'accessor', get: encode(details.get), set: encode(details.set)};
+    }
+
+    for (let index = 0; index < pending.length; index += 1) {
+        const value = pending[index];
+        const node = values[index];
+        if (typeof value === 'symbol') {
+            node.description = encode(value.description);
+            node.globalKey = encode(Symbol.keyFor(value));
+            continue;
+        }
+        const descriptors = Object.getOwnPropertyDescriptors(value);
+        node.properties = Reflect.ownKeys(descriptors).map(function describeProperty(key) {
+            return {key: encode(key), ...encodeDescriptor(descriptors[key])};
+        });
+        const prototype = Object.getPrototypeOf(value);
+        const names = prototypeNames(prototype);
+        if (names.length) node.constructorName = names[0];
+
+        // Error subclasses may supply these on a prototype, including custom
+        // accessors. Preserve the descriptor without reading through it.
+        node.inherited = [];
+        for (const key of ['name', 'message', 'stack', 'cause', 'errors', 'code', 'details']) {
+            if (Object.hasOwn(descriptors, key)) continue;
+            const descriptor = findDescriptor(prototype, key);
+            if (descriptor) node.inherited.push({key, ...encodeDescriptor(descriptor)});
+        }
+        if (names.includes('Map')) {
+            node.entries = Array.from(Map.prototype.entries.call(value), function encodeEntry([key, item]) {
+                return [encode(key), encode(item)];
+            });
+        } else if (names.includes('Set')) {
+            node.entries = Array.from(Set.prototype.values.call(value), encode);
+        } else if (names.includes('Date')) {
+            node.time = encode(Date.prototype.getTime.call(value));
+        } else if (names.includes('RegExp')) {
+            node.source = Object.getOwnPropertyDescriptor(RegExp.prototype, 'source').get.call(value);
+            node.flags = {};
+            for (const key of ['hasIndices', 'global', 'ignoreCase', 'multiline', 'dotAll', 'unicode', 'unicodeSets', 'sticky']) {
+                const getter = Object.getOwnPropertyDescriptor(RegExp.prototype, key)?.get;
+                if (getter) node.flags[key] = getter.call(value);
+            }
+        }
+    }
+    return record;
+
+    function encode(value) {
+        const type = typeof value;
+        if (value === null || type === 'string' || type === 'boolean') return value;
+        if (type === 'number') {
+            return Number.isFinite(value) && !Object.is(value, -0)
+                ? value : {type, value: Object.is(value, -0) ? '-0' : String(value)};
+        }
+        if (type === 'undefined') return {type};
+        if (type === 'bigint') return {type, value: String(value)};
+        if (!references.has(value)) {
+            const id = values.length + 1;
+            references.set(value, id);
+            pending.push(value);
+            values.push({id, type: Array.isArray(value) ? 'array' : type});
+        }
+        return {ref: references.get(value)};
+    }
+
+    function encodeDescriptor(descriptor) {
+        const result = {enumerable: descriptor.enumerable, configurable: descriptor.configurable};
+        if (Object.hasOwn(descriptor, 'value')) {
+            return {...result, value: encode(descriptor.value), writable: descriptor.writable};
+        }
+        return {...result, get: encode(descriptor.get), set: encode(descriptor.set)};
+    }
+
+    function findDescriptor(prototype, key) {
+        const visited = new Set();
+        while (prototype && !visited.has(prototype)) {
+            visited.add(prototype);
+            const descriptor = Object.getOwnPropertyDescriptor(prototype, key);
+            if (descriptor) return descriptor;
+            prototype = Object.getPrototypeOf(prototype);
+        }
+        return undefined;
+    }
+
+    function prototypeNames(prototype) {
+        const names = [];
+        const visited = new Set();
+        while (prototype && !visited.has(prototype)) {
+            visited.add(prototype);
+            const constructor = Object.getOwnPropertyDescriptor(prototype, 'constructor')?.value;
+            if (typeof constructor === 'function') {
+                const name = Object.getOwnPropertyDescriptor(constructor, 'name')?.value;
+                if (typeof name === 'string') names.push(name);
+            }
+            prototype = Object.getPrototypeOf(prototype);
+        }
+        return names;
+    }
+}

package/docs/reference/ai/twin-cloud.md

@@ -94,17 +94,34 @@ The key is
 transport authentication, not part of either callback's message payload. Keep
 credentials out of application logging as well.
 
-Before a successful response is consumed, a rejected Fetch or HTTP `529` can
-retry up to three times, waiting `3000` milliseconds before each retry. These
-two failure types share that three-retry budget. HTTP `429` with a message
-containing `overload` (case-insensitive) retains its unlimited three-second
-retry behavior and does not consume that budget. Each attempt reuses the exact
-destination, Fetch options, serialized request, and signal. Other HTTP errors,
-successful-response decoding failures, and application callback failures do not
-retry. A known HTTP `529` also retries when reading or parsing its diagnostic
-body fails; `onRetry.error` retains that exact failure, which is thrown unchanged
-if recovery is exhausted. Diagnostic-body failures for other HTTP statuses do
-not retry. The last complete failure is thrown unchanged when recovery is exhausted.
+Before a successful response is consumed, rejected Fetch calls and HTTP `429`
+or `529` share a budget of three retries (four total attempts). Every `429`
+qualifies, including request-per-minute and token-per-minute quota responses;
+the error message does not select recovery. A readable `Retry-After` header
+supplies the `429` delay as seconds or an HTTP date. Otherwise the upcoming
+retry number selects a `3000`, `6000`, or `12000` millisecond delay. Rejected
+Fetch and `529` retain their `3000` millisecond delay. A provider's longer wait
+is preserved, split only at the host timer's range when necessary.
+
+After an observed `429`, overlapping requests through this module with the
+same exact endpoint and Authorization value share the latest cooldown and
+dispatch at least three seconds apart. A later quota response extends that
+cooldown, including a terminal quota response while sibling calls remain active;
+its fallback remains at most `12000` milliseconds. Different endpoints and credentials remain independent, and requests
+before any quota response remain concurrent. This is active-operation pacing,
+not a persistent global rate limiter: the module releases the group, including
+its credential key and error, when its last HTTP operation settles. It does not
+coordinate separate tabs, processes, or independently loaded module instances.
+
+Each attempt reuses the exact destination, Fetch option values, serialized request,
+and signal. Headers are normalized once through standard Fetch `Headers` and
+retained for retries, including when the caller supplied a one-shot iterable.
+Other HTTP errors (including `400` and `413`), successful-response
+decoding failures, and application callback failures do not retry. A known
+`429` or `529` also retries when reading its diagnostic body fails;
+`onRetry.error` retains that exact failure. The last complete error is thrown
+unchanged on exhaustion, preserving provider fields such as `status_code` and
+nested `error.type`/`error.message`. No error envelope is rebuilt or sanitized.
 Pass a fresh
 `AbortController`'s `signal` and call `abort()` to cancel. Cancellation during
 the request, response-body read, retry wait, or callback settlement prevents
@@ -114,12 +131,16 @@ A missing key uses `AI_PROVIDER_NOT_CONFIGURED`, a missing explicit model
 throws `TypeError`, and an unsupported `structuredOutput` input uses
 `AI_STRUCTURED_OUTPUT_INVALID`.
 
-Optional `onRetry({phase,attempt,delayMs,status,error})` observes each retry.
+Optional `onRetry({phase,attempt,delayMs,status,error})` observes recovery.
 `phase:'waiting'` arrives before the abortable delay; `phase:'requesting'`
 arrives immediately before the next Fetch. `attempt` is the one-based upcoming
-retry number across both retry policies, `delayMs` is `3000`, `status` is the
+retry number across the shared budget, `delayMs` is the selected wait, `status` is the
 HTTP status or `null` for a rejected Fetch, and `error` is the complete original
-failure value. This observer runs synchronously without awaiting its returned
+failure value. An initial request waiting behind an already observed quota
+reports `attempt:0`, `status:429`, and that complete quota error. Shared pacing
+can extend the actual wait beyond its initial observation; `requesting` is
+emitted only after that wait, immediately before dispatch. This observer runs
+synchronously without awaiting its returned
 promise. Synchronous throws and rejected promises are reported through the
 shared console logger and cannot change the request outcome or trigger another
 retry. `onRequest` still runs once per logical request. Retry observations stay

package/docs/reference/availability-and-normalization.md

@@ -99,7 +99,7 @@ where listed. See [native output and host requirements](core-native-packaging.md
 | Label speakers in caller-fed mono audio | [`arcane-os/diarization`, `arcane-os/core/diarization`](diarization.md) | **Node** with built helper, selected official NeMo libraries and Nemotron model; official CPU archives cover Windows x64 and Linux/macOS x64/arm64 | Independent retained streams, complete timed segments and optional probability frames. Generic labels are not personal identities or speech transcripts; cancellation and close observe native cleanup. Android needs host adaptation. |
 | Use local AI without coupling app code to Ollama HTTP | `Arcane.localAI`, `Arcane.ai`, or `/arcane/modules/Ollama.js` | Primarily **Native**; Android exposes a narrower admitted inference projection | Admission, errors, and managed-operation events are normalized. Direct Ollama response envelopes remain **Provider-native**. |
 | Use TWiN Cloud from the renderer profile | `/arcane/modules/AI.js` | **Cloud** from an allowed browser/native renderer | High-level chat behavior is normalized by the module. The TWiN access key authenticates remote LLM chat; raw provider diagnostics remain provider-specific. No automatic cloud fallback is inferred from local failure. |
-| Send one TWiN Cloud request with an explicit key and model | `fetchRequest` from `arcane-os/ai/twin-cloud` | **Node** and **Browser**, using standard Fetch and a remote HTTPS provider | Keeps complete messages and returns the full parsed provider JSON. Rejected Fetch and HTTP 529 share three retries after 3000 ms; HTTP 429 overload retains unlimited three-second retries. Optional `onRetry` reports waiting/requesting phases without blocking recovery. Shared structured-output mapping and cancellation match browser TWiN transport. No browser profile, AI/user singleton, DBOPFS, or retained request history is created. |
+| Send one TWiN Cloud request with an explicit key and model | `fetchRequest` from `arcane-os/ai/twin-cloud` | **Node** and **Browser**, using standard Fetch and a remote HTTPS provider | Keeps complete messages and returns the full parsed provider JSON. Rejected Fetch and all HTTP 429/529 share three retries. Quota recovery honors Retry-After or exponential delay; active same-endpoint/credential requests share cooldown and paced dispatch. Optional `onRetry` reports waiting/requesting without blocking recovery. Complete errors and cancellation match browser TWiN transport. No browser profile, AI/user singleton, DBOPFS, or retained request history is created. |
 | Evaluate caller-owned state and questions with TWiN System One | `fetchSystemOneRequest` from `arcane-os/ai/twin-cloud` | **Node** and **Browser**, using standard Fetch and a remote HTTPS provider | Posts the explicit model and complete state/questions through the existing JSON transport. Returns the entire parsed provider result and shares its retry, diagnostic, and cancellation behavior. Model choice, question definitions, answer interpretation, and scoring policy stay with the caller; no conversation state is retained. |
 | Generate images from a complete caller-owned prompt | `generateImages` from `arcane-os/ai/twin-cloud` | **Node** and **Browser**, using standard Fetch, Blob, and a remote HTTPS provider | Caller selects `fal-ai/flux/schnell` or `stable-diffusion-3.5-large`, credentials, and provider parameters. Returns every image as a Blob with media metadata in provider order; owns progress, job polling, and cancellation. Submission is not automatically replayed after an ambiguous failure. Display and storage stay application-owned. |
 | Use speech through one application helper | `/arcane/modules/AI.js` and `Arcane.speech` | **Browser** or **Native** | Whisper and Kokoro keep their on-device paths; explicitly selected Web Speech STT/TTS and TWiN Cloud TTS declare their non-local behavior. `prepareTTSPlayback` supports real audio and silent native descriptors; `fetchTTS` and durable `prepareTTS` require real audio. Provider choice, model, voice, and credentials stay application-owned. |