arcane-os 0.60.0 → 0.62.0

package/CHANGELOG.md

@@ -1,5 +1,45 @@
 # Changelog
 
+## 0.62.0
+
+- Add whole-image PNG editing through the existing local image runtime, Core
+  service and browser accessor: `edit({model, image, prompt, strength, ...})`
+  and `image.edit` share the retained generation context and lifecycle.
+- Preserve complete original PNG input and prompts through lossless transport,
+  copy queued native input, and pass explicit finite strength without clamping.
+  Return a separate complete PNG result; applications own persistence.
+- Preserve synchronous Thinking, progress, ordered context work, cancellation,
+  unload and close. SD1.4 now advertises `img2img` beside `txt2img`.
+- Support static non-interlaced 8-bit RGB/RGBA PNG input using the existing
+  worker decoder. Unsupported formats report their actual limitation; model
+  cropping, resizing and alpha behavior remain explicit in the editing guide.
+- Reuse the approved runtime and model workflow with no dependency change.
+  This increment does not add masks, reference editing, new models or GPU
+  installation, and does not claim exact visual instruction adherence.
+
+## 0.61.0
+
+- Add retained local image generation through Core, with public native runtime,
+  Core service and browser accessor entrypoints. Applications keep ownership of
+  prompts, styles, saved images and associations.
+- Select the approved stable-diffusion.cpp runtime and Stable Diffusion 1.4
+  checkpoint through existing local-AI configuration. Runtime installation and
+  model storage/preparation remain separate; model setup uses the existing
+  DBOPFS model store and a retained Core working-file projection.
+- Preserve complete prompts and PNG results, synchronous transient Thinking
+  acknowledgement, observable model state and progress, actual cancellation
+  joining, and native context/file cleanup. Preparation does not block page
+  rendering or unrelated Core services.
+- Expose model-assets preparation and service entrypoints alongside image
+  generation. Native contexts retain working files until actual unload; the
+  browser accessor closes only its own requests and subscriptions.
+- Built-in distributions select CPU on Windows and Linux and the same upstream
+  Metal/CPU archive on macOS. Windows CUDA preparation remains separate pending
+  work; publication does not claim GPU-first Windows or PM execution.
+- Keep optional engine/binding acquisition in the selected application's
+  existing installer workflow. No engine binaries or model checkpoint are added
+  to the npm package, and its declared dependency tree is unchanged.
+
 ## 0.60.0
 
 - Add `Arcane.window.setTheme(presentation, {signal})` for the current native

package/README.md

@@ -20,7 +20,7 @@ selected SDK projection or an explicit live SDK source mount. Both profiles
 share the theme, packaging, event, cancellation,
 and browser run contracts while retaining their selected application layout.
 
-This checkout defines the `0.60.0` SDK contract. Development applications declare
+This checkout defines the `0.62.0` SDK contract. Development applications declare
 `arcane-os` as `latest` and retain their lockfile. Exact SDK pins begin when Roshi
 designates a production build. Follow [Consumer updates](docs/consumer-updates.md)
 to refresh and compare the installed SDK and lockfile with the published channel;

package/browser-runtime/ai/core-image.mjs

@@ -0,0 +1,392 @@
+import Is from '../dependencies/strong-type/index.js';
+import {getInstalledCoreClient} from '../core/client.mjs';
+import {CoreError, serializeCoreError} from '../core/contracts.mjs';
+import {createArcaneEventSource} from '../event-manager.mjs';
+
+const is = new Is(false);
+
+function unavailable(message) {
+    return new CoreError(
+        {code: 'ARCANE_IMAGE_CORE_UNAVAILABLE', message}
+    );
+}
+
+/** Decode the complete JSON transport representation without retaining it. */
+function decodeImage(image) {
+    const {data, encoding, ...metadata} = image;
+    if (encoding !== 'base64') {
+        throw new TypeError('Core image results require base64 transport encoding.');
+    }
+    const decoded = Uint8Array.from(
+        globalThis.atob(data),
+        function decodeImageCharacter(character) {
+            return character.charCodeAt(0);
+        }
+    );
+    const blob = new Blob(
+        [decoded],
+        {type: metadata.mediaType}
+    );
+    return {...metadata, blob};
+}
+
+/** Preserve the complete encoded input at the JSON transport boundary. */
+async function encodeImage(image, signal) {
+    if (!(image instanceof Blob)) {
+        throw new TypeError('Image editing requires the original PNG Blob or File.');
+    }
+    const content = new Uint8Array(await image.arrayBuffer());
+    signal.throwIfAborted();
+    let data;
+    if (is.function(content.toBase64)) {
+        data = content.toBase64();
+    } else {
+        let binary = '';
+        for (const value of content) binary += String.fromCharCode(value);
+        data = globalThis.btoa(binary);
+    }
+    return {data, encoding: 'base64', mediaType: image.type};
+}
+
+/** Browser access to the selected native image service; no result persistence. */
+export function createCoreImageRuntime(
+    {client = getInstalledCoreClient(), signal, onEvent} = {}
+) {
+    const owner = {};
+    const events = createArcaneEventSource(
+        owner,
+        {source: 'core-image', eventTypes: ['image.state']}
+    );
+    const lifetime = new AbortController();
+    const lifetimeSignal = signal ? AbortSignal.any([signal, lifetime.signal]) : lifetime.signal;
+    const pending = new Set();
+    let state = {
+        id: 'stable-diffusion.cpp',
+        installed: false,
+        available: false,
+        selectedModel: null,
+        loaded: false,
+        state: client ? 'unknown' : 'unavailable',
+        backend: null,
+        devices: [],
+        resources: {},
+        progress: null,
+        error: null,
+        models: []
+    };
+    let revision = 0;
+    let closed = false;
+    let closing;
+
+    function current() {
+        const requests = [...pending].map(
+            function requestState(request) {
+                return {
+                    method: request.method,
+                    streamId: request.streamId,
+                    status: request.status,
+                    progress: request.progress
+                };
+            }
+        );
+        return {
+            ...state,
+            requests,
+            status: !closed && requests.some(
+                function generationPending(request) {
+                    return request.status === 'Thinking';
+                }
+            ) ? 'Thinking' : null,
+            busy: pending.size > 0,
+            closed
+        };
+    }
+
+    function reportObserverFailure(error) {
+        globalThis.console?.error('Core image observer failed.', error);
+    }
+
+    function notifyListener(listener, value) {
+        const result = listener(value);
+        if (result && is.function(result.then)) {
+            // Observers own their callback lifetimes and may await close().
+            Promise.resolve(result).catch(reportObserverFailure);
+        }
+    }
+
+    function observe(callback, value) {
+        if (!is.function(callback)) {
+            return;
+        }
+        try {
+            notifyListener(callback, value);
+        } catch (error) {
+            reportObserverFailure(error);
+        }
+    }
+
+    function publish() {
+        const snapshot = current();
+        events.dispatch('image.state', snapshot);
+        if (snapshot.closed === closed) {
+            observe(
+                onEvent,
+                {type: 'image.state', data: snapshot}
+            );
+        }
+        return snapshot;
+    }
+
+    function accept(value) {
+        if (closed) {
+            return;
+        }
+        state = value;
+        revision += 1;
+        publish();
+    }
+
+    const unsubscribe = client?.events.on('image.state', accept)
+        ?? function noCoreSubscription() {};
+
+    function invoke(method, parameters, {signal: requestSignal, onProgress} = {}) {
+        if (closed) {
+            throw unavailable('The Core image accessor is closed.');
+        }
+        if (!client) {
+            throw unavailable('Local image generation requires an available Core connection.');
+        }
+        const operationSignal = requestSignal
+            ? AbortSignal.any([requestSignal, lifetimeSignal]) : lifetimeSignal;
+        operationSignal.throwIfAborted();
+        const request = {
+            method,
+            revision,
+            streamId: parameters.streamId,
+            status: method === 'image.generate' || method === 'image.edit' ? 'Thinking' : null,
+            progress: null,
+            task: null
+        };
+        let stopProgress;
+        if (request.streamId) {
+            stopProgress = client.events.on(
+                'image.progress',
+                function imageProgress(value) {
+                    if (closed || operationSignal.aborted || value.streamId !== request.streamId) {
+                        return;
+                    }
+                    request.progress = value.progress;
+                    publish();
+                    if (closed || operationSignal.aborted) {
+                        return;
+                    }
+                    observe(onProgress, value.progress);
+                    if (closed || operationSignal.aborted) {
+                        return;
+                    }
+                    observe(
+                        onEvent,
+                        {type: 'image.progress', data: value}
+                    );
+                }
+            );
+        }
+        request.task = Promise.resolve().then(
+            async function executeImageRequest() {
+                operationSignal.throwIfAborted();
+                const requestParameters = method === 'image.edit'
+                    ? {...parameters, image: await encodeImage(parameters.image, operationSignal)} : parameters;
+                operationSignal.throwIfAborted();
+                observe(
+                    onEvent,
+                    {type: 'image.request', data: {method, parameters: requestParameters}}
+                );
+                operationSignal.throwIfAborted();
+                const result = await client.invoke(
+                    method,
+                    requestParameters,
+                    {signal: operationSignal, timeoutMs: 0}
+                );
+                operationSignal.throwIfAborted();
+                if (method === 'image.generate' || method === 'image.edit') {
+                    const images = result.images.map(decodeImage);
+                    operationSignal.throwIfAborted();
+                    const decoded = {...result, images};
+                    observe(
+                        onEvent,
+                        {type: 'image.result', data: decoded}
+                    );
+                    operationSignal.throwIfAborted();
+                    return decoded;
+                }
+                return result;
+            }
+        ).catch(
+            function imageRequestFailed(error) {
+                if (!closed && !operationSignal.aborted) {
+                    if (method !== 'image.status' || revision === request.revision) {
+                        state = {...state, error: serializeCoreError(error)};
+                    }
+                    observe(
+                        onEvent,
+                        {type: 'image.error', data: {method, error: serializeCoreError(error)}}
+                    );
+                }
+                throw error;
+            }
+        ).finally(
+            function releaseImageRequest() {
+                stopProgress?.();
+                pending.delete(request);
+                if (!closed) {
+                    publish();
+                }
+            }
+        );
+        pending.add(request);
+        // Acknowledgement precedes transport, preparation, model loading and inference.
+        publish();
+        return request.task;
+    }
+
+    async function inspect({signal: requestSignal} = {}) {
+        const before = revision;
+        try {
+            const result = await invoke(
+                'image.status',
+                {},
+                {signal: requestSignal}
+            );
+            if (!closed && revision === before) {
+                accept(result);
+            }
+            return current();
+        } catch (error) {
+            if (!closed && !requestSignal?.aborted && revision === before) {
+                accept(
+                    {...state, available: false, state: 'unavailable', error: serializeCoreError(error)}
+                );
+            }
+            throw error;
+        }
+    }
+
+    async function load({model, context, assetProjectionId, resourcePaths, signal: requestSignal} = {}) {
+        const before = revision;
+        const result = await invoke(
+            'image.load',
+            {model, context, assetProjectionId, resourcePaths},
+            {signal: requestSignal}
+        );
+        if (!closed && revision === before) {
+            accept(result);
+        }
+        return result;
+    }
+
+    function generate(
+        {model, prompt, parameters, assetProjectionId, resourcePaths, signal: requestSignal, onProgress} = {}
+    ) {
+        const streamId = client?.uuid?.() ?? globalThis.crypto.randomUUID();
+        return invoke(
+            'image.generate',
+            {model, prompt, parameters, streamId, assetProjectionId, resourcePaths},
+            {signal: requestSignal, onProgress}
+        );
+    }
+
+    function edit(
+        {model, image, prompt, strength, parameters, assetProjectionId, resourcePaths, signal: requestSignal, onProgress} = {}
+    ) {
+        const streamId = client?.uuid?.() ?? globalThis.crypto.randomUUID();
+        return invoke(
+            'image.edit',
+            {model, image, prompt, strength, parameters, streamId, assetProjectionId, resourcePaths},
+            {signal: requestSignal, onProgress}
+        );
+    }
+
+    async function unload({signal: requestSignal} = {}) {
+        const before = revision;
+        const result = await invoke(
+            'image.unload',
+            {},
+            {signal: requestSignal}
+        );
+        if (!closed && revision === before) {
+            accept(result);
+        }
+        return result;
+    }
+
+    function subscribe(listener, {replay = true, signal: subscriptionSignal} = {}) {
+        if (!is.function(listener)) {
+            throw new TypeError('An image state listener must be a function.');
+        }
+        if (closed) {
+            if (replay && !subscriptionSignal?.aborted) {
+                notifyListener(listener, current());
+            }
+            return function closedSubscription() {};
+        }
+        const stop = events.on(
+            'image.state',
+            function imageStateChanged(event) {
+                notifyListener(listener, event.detail);
+            },
+            {signal: subscriptionSignal}
+        );
+        try {
+            if (replay && !subscriptionSignal?.aborted) {
+                notifyListener(listener, current());
+            }
+        } catch (error) {
+            stop();
+            throw error;
+        }
+        return stop;
+    }
+
+    function close() {
+        if (closing) {
+            return closing;
+        }
+        closed = true;
+        closing = Promise.resolve().then(
+            async function closeImageAccessor() {
+                await Promise.allSettled(
+                    [...pending].map(
+                        function requestCompletion(request) {
+                            return request.task;
+                        }
+                    )
+                );
+            }
+        );
+        lifetime.abort();
+        lifetimeSignal.removeEventListener('abort', close);
+        unsubscribe();
+        state = {...state, available: false, state: 'closed'};
+        publish();
+        events.dispose();
+        return closing;
+    }
+
+    lifetimeSignal.addEventListener(
+        'abort',
+        close,
+        {once: true}
+    );
+    if (lifetimeSignal.aborted) {
+        close();
+    } else if (client) {
+        // Subscribe before the snapshot request so a later lifecycle event wins.
+        inspect().catch(
+            function initialImageStateUnavailable(error) {
+                globalThis.console?.error('Core image initial state unavailable.', error);
+            }
+        );
+    }
+
+    return {load, generate, edit, unload, inspect, current, subscribe, close};
+}

package/docs/reference/local-image-editing.md

@@ -1,10 +1,97 @@
-# Local image edit inputs
+# Local image editing through Core
 
-The SDK input decoder prepares a PNG raster for the shared native image
-runtime. Decoding does not load a model or perform inference. The runtime owns
-model selection, supported editing operations, native calls and their complete
-results. Applications retain the original image and decide whether to save a
-newly generated result.
+`createCoreImageRuntime()` exposes whole-image img2img through `edit()`, using
+the same retained native context as generation. The selected SD1.4 model
+advertises `txt2img` and `img2img`. Applications supply the original PNG, the
+complete prompt and explicit strength, retain the original, and decide whether
+to save the separate generated result.
+
+## Browser accessor
+
+Prepare and load the selected model using the
+[shared generation and model-asset lifecycle](local-image-generation.md).
+Then pass a generated PNG Blob or a supported PNG File directly:
+
+```js
+import {createCoreImageRuntime} from 'arcane-os/ai/core-image';
+
+const images = createCoreImageRuntime({signal: page.signal});
+images.subscribe(function showImageStatus(snapshot) {
+    statusElement.textContent = snapshot.status ?? snapshot.state;
+});
+
+const result = await images.edit({
+    model: 'sd14',
+    image: originalPNG,
+    prompt: 'A dignified raccoon astronaut wearing a purple knitted helmet, holding a tiny moon cactus.',
+    strength: 0.6,
+    parameters: {
+        width: 512,
+        height: 512,
+        sample_params: {sample_steps: 20}
+    },
+    signal: page.signal,
+    onProgress(progress) {
+        console.log(progress);
+    }
+});
+
+const editedPNG = result.images[0].blob;
+```
+
+The example uses the application's existing `page` AbortController,
+`statusElement` and original `originalPNG` Blob or File. It assumes that the
+selected model has already been loaded. `edit()` can also prepare a selected
+model on demand using the same `assetProjectionId` and `resourcePaths` fields
+as `generate()`.
+
+| Member or field | Contract |
+| --- | --- |
+| `edit({model, image, prompt, strength, parameters?, assetProjectionId?, resourcePaths?, signal?, onProgress?})` | Returns the complete result with `images: [{blob, mediaType, width, height}]`, preserving additional runtime result metadata. |
+| `image` | Original complete PNG `Blob` or `File`, with `image/png` or an empty declared media type. Supported PNG encodings are listed below. |
+| `prompt` | Complete caller-authored string, passed unchanged. Native C-string incompatibilities retain the generation error contract. |
+| `strength` | Explicit finite number from `0` through `1`, mapped to native img2img strength. It takes precedence over a model default or `parameters.strength`; it is never clamped. |
+| `parameters` | Native generation options, including canvas dimensions, seed, sampling and `image_preprocess`. The top-level image supplies native `init_image`; the top-level strength supplies native `strength`. Other options retain the generation contract. |
+
+`Thinking` is published synchronously when the browser accepts the edit, before
+reading the Blob, sending the request, preparing the runtime or loading the
+model. Native sampling progress uses the existing progress surface. Complete
+PNG results arrive after native completion; the SDK fabricates no image chunks
+or assistant turns. Closing the accessor cancels and joins its requests while
+preserving the shared Core client and model ownership.
+
+The original encoded input is preserved through lossless Core JSON framing.
+The browser does not decode, draw, re-encode, resize, crop, composite or save it.
+The request signal reaches input preparation, worker decoding and cooperative
+native cancellation. Cancellation during `Blob.arrayBuffer()` is observed
+after that platform operation completes. An aborted edit produces no late
+success or progress callback. Request invocation retains `timeoutMs: 0`.
+
+## Core and native request
+
+The `arcane-os/core/image` service adds `image.edit` with
+`{model, image, prompt, strength, parameters?, streamId?, assetProjectionId?, resourcePaths?}`.
+At this JSON boundary only, `image` is
+`{data: completeBase64PNG, encoding: 'base64', mediaType}`. The service emits
+the existing accepted `image.progress` event before preparation, then forwards
+the decoded transport content to the shared image worker. Results use the
+same complete PNG transport records as `image.generate`.
+
+The native runtime's `edit()` takes the same operation fields, with
+`image: {data: Uint8Array, mediaType}` and no Core projection arguments.
+It copies the supplied data view at acceptance so caller reuse cannot change
+an input waiting behind another operation.
+The worker decodes once, loads or reuses the exact selected context, and passes
+the raster as native `init_image` with explicit `strength`. The existing native
+binding retains input memory through completion and copies complete output
+rasters before native cleanup. The same queue, cancellation, unload and close
+owners serve generation and editing; no second engine or model store is created.
+
+Unreadable or unsupported input rejects the edit without substituting an
+image. Errors retain their complete details on rejected promises and diagnostic
+events. Applications keep those engineering diagnostics outside ordinary status
+surfaces and durable chat history. No prompt, input or result is persisted by
+this operation.
 
 ## Decoder contract
 
@@ -56,22 +143,21 @@ failure never returns partial imagery or substitutes a different image.
 
 ## Editing semantics at the native owner
 
-Whole-image img2img uses an initial image, prompt and strength. Masked editing
-adds a separate mask; instruction/reference editing uses a model that supports
-reference conditioning. These are distinct operations and must be advertised
-only when the selected model and native path implement them. Ordinary img2img
-does not require LoRA training. The candidate engine documents this directly
-with its base SD1.4 model in the
+Whole-image img2img uses an initial image, prompt and strength. This `edit()`
+contract selects that operation. Masked and reference editing require their
+own model-specific contracts and are outside this PNG img2img operation.
+Ordinary img2img does not require LoRA training. The selected engine documents
+this directly with its base SD1.4 model in the
 [tagged img2img example](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/sd.md#img2img-example).
 
 Source dimensions are decoder output, not an implicit request to change the
-generation dimensions. The model owner documents its actual canvas adaptation
-and reports actual output dimensions. In the candidate engine, temporary
-initial images use model preprocessing while caller images remain unchanged;
+generation dimensions. The engine center-crops and resizes temporary initial
+images to the selected canvas by default and reports actual output dimensions.
+Caller images remain unchanged;
 even `mode=none` does not suppress all later model adaptation. See the
 [native preprocessing contract](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/image_preprocessing.md).
 
-An RGB-only model such as the selected SD1.4 candidate consumes the stored RGB
+An RGB-only model such as the selected SD1.4 consumes the stored RGB
 and does not preserve alpha. The SDK must not silently choose a compositing
 background or claim that passing RGBA preserves its displayed appearance.
 Strength zero is also not an exact-copy operation: native img2img still uses