arcane-os 0.34.2 → 0.35.0

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 0.35.0
+
+- Add `DBOPFS.createCompressedPNG({tableNames, additionalTables, signal})` to
+  return a PNG Blob without downloading or writing storage. Omitted table names
+  select all saved tables; an empty list selects none. Caller-supplied tables
+  replace matching saved tables in the export only.
+- Preserve `downloadCompressedPNG(name)` and accept the same optional export
+  options. Cancellation stops further preparation and prevents a later download;
+  browser file reads and canvas encoding already underway finish normally.
+- Add optional async `restoreFromPNG(file, {selectTables})` selection over the
+  complete decoded table/file map before any writes. The callback owns selection
+  and projection; existing one-argument restore remains supported.
+- Report rejected restore writes after all selected table batches settle through
+  `DBOPFS_RESTORE_WRITE_FAILED`, retaining original errors and table/file details.
+  Successful writes remain saved; restore is not an atomic transaction. Release
+  decoded image resources on both success and extraction failure.
+
 ## 0.34.2
 
 - Remove temporary `persist:false` Chat cards on completion, failure,

package/README.md

@@ -19,7 +19,7 @@ version-locked SDK runtime, while an integrated Arcane checkout uses its live
 `arcane/` runtime. Both profiles share the theme, packaging, event, cancellation,
 and browser run contracts while retaining their selected application layout.
 
-This checkout defines the `0.34.2` SDK contract. Applications pin one exact npm
+This checkout defines the `0.35.0` SDK contract. Applications pin one exact npm
 version and lockfile; registry state is deliberately not baked into application
 artifacts.
 

package/docs/reference/inventory/runtime-modules.json

@@ -568,7 +568,7 @@
       "availability": "Browser / native WebView",
       "protocol": "OPFS, DBOPFSWorker, Compression Streams, and per-realm globalThis.arcaneEvents authority",
       "normalization": "App scope and recognized file parsing are normalized; nonblank unreadable JSONL rows remain complete raw strings, empty-table removal reports removed, absent, or not-empty after a create:false directory lookup and without scanning or recursively clearing a directory, same-named files are preserved, aliases and cached handles are invalidated only when the directory is gone, and unexpected DOM/storage errors are preserved.",
-      "surface": "default `DBOPFS`; installs `window.dbopfs`, emits `dbopfs-ready`; table/file/backup APIs including removeEmptyTable(tableName)."
+      "surface": "default `DBOPFS`; installs `window.dbopfs`, emits `dbopfs-ready`; table/file/backup APIs including removeEmptyTable(tableName). createCompressedPNG({tableNames,additionalTables,signal}) returns a PNG Blob without download or record writes; omitted names select all, [] none, and supplied tables replace matching saved tables in the export. downloadCompressedPNG(name,options) reuses the same preparation and initiates one download; cancellation prevents it. restoreFromPNG(file,{selectTables}) closes the decoded image after pixel extraction, awaits the optional caller selection/projection before any writes, and rejects DBOPFS_RESTORE_WRITE_FAILED AggregateError after all selected table batches settle when records fail, preserving original errors and failures {tableName,fileName,reason}; successful writes remain. No-options restore and setMany retain their contracts."
     },
     {
       "file": "runtime/arcane/modules/DBOPFSDocumentLibrary.js",

package/docs/reference/runtime-modules.md

@@ -1986,6 +1986,39 @@ default `DBOPFS`; installs `window.dbopfs`, emits `dbopfs-ready`; table/file/bac
 
 Exact exports: `DBOPFS_EVENT_TYPES`, `DBOPFS_REASONS`, `default`.
 
+`createCompressedPNG({tableNames, additionalTables, signal})` returns an
+`image/png` Blob without downloading or writing records. Omitted `tableNames`
+selects all discovered saved tables; `[]` selects none. Named absent tables
+are encoded as empty tables without creating OPFS directories. The existing
+`memories`/`memory` alias is retained. `additionalTables` is an application-owned
+`{tableName:{fileName:value}}` map. Each supplied table replaces the matching
+saved table in this export only, rather than inheriting records the caller did
+not select. Values retain the existing JSON serialization contract. Applications
+own recipient policy, profile projections, and any product format marker.
+
+`downloadCompressedPNG(name, options)` uses that same method and its options,
+then initiates one timestamp-named download. Existing `downloadCompressedPNG(name)`
+calls retain full-database behavior. An optional `AbortSignal` stops further
+record preparation and prevents a later download. File reads and browser canvas
+encoding already in progress cannot be interrupted; their result is discarded
+on cancellation. No object URL or download is created by `createCompressedPNG`.
+
+`restoreFromPNG(file, {selectTables})` preserves the existing deflate/RGB backup format and
+restores through `setMany()` without changing its settled-result API or per-key
+write ordering. The optional callback receives the complete decoded table/file
+map exactly once and may return a map or promise of a map. Only its returned
+tables and records are written, allowing application-owned selection and
+projection of older full backups without a second decoder or temporary storage
+writes. Callback failure rejects before any writes. With no callback, all
+decoded tables are restored as before.
+
+Restore resolves only when every selected record write succeeds. It
+continues the table batches after individual rejected writes, then rejects an
+`AggregateError` with code `DBOPFS_RESTORE_WRITE_FAILED`, original reasons in
+`errors`, and `failures` entries `{tableName, fileName, reason}`. Successful
+writes remain saved; restore is not an all-or-nothing transaction. The decoded
+image is closed after pixel extraction, including canvas failures.
+
 ### Availability and normalization
 
 **Browser / native WebView.** App scope and recognized JSON/JSONL file parsing

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "arcane-os",
-  "version": "0.34.2",
+  "version": "0.35.0",
   "description": "Arcane OS JavaScript SDK, project-local CLI, browser runtime, and repository-portable application packager.",
   "type": "module",
   "main": "./src/index.mjs",
@@ -86,7 +86,7 @@
     "test": "npm run test:unit && npm run test:functional && npm run test:integration && npm run test:regression",
     "test:release": "node ./bin/arcane-test.mjs test/npm-release.test.mjs",
     "test:unit": "node ./bin/arcane-test.mjs test/app-descriptor.test.mjs test/app-schema.test.mjs test/app-selection.test.mjs test/contracts.test.mjs test/doctor.test.mjs test/mail-credentials.test.mjs test/mail-outbox.test.mjs test/mail-public-api.test.mjs test/mail-send.test.mjs test/mail-transport.test.mjs test/targets.test.mjs test/workspace-operation-lock.test.mjs",
-    "test:functional": "node ./bin/arcane-test.mjs test/ai-request-model.test.mjs test/ai-request-retry.test.mjs test/ai-stream-cancellation.test.mjs test/browser-speech-providers.test.mjs test/browser-wasm-gpu-notice.test.mjs test/browser-wasm-download-resume.test.mjs test/chat-stream-speech.test.mjs test/chat-transcript-scroll.test.mjs test/chat-transient-retention.test.mjs test/cli.test.mjs test/dbopfs-document-library.test.mjs test/dbopfs.test.mjs test/dev-server.test.mjs test/dev-pwa.test.mjs test/dom-event-instrumentation.test.mjs test/event-manager.test.mjs test/events.test.mjs test/import-map.test.mjs test/mail-cli.test.mjs test/mail-runtime.test.mjs test/mail-server.test.mjs test/modal.test.mjs test/packaging.test.mjs test/pwa-packaging.test.mjs test/pwa-client.test.mjs test/pwa-install.test.mjs test/pwa-worker.test.mjs test/persistent-ai-chat-session.test.mjs test/reference-completeness.test.mjs test/runtime-api-behavior.test.mjs test/runtime.test.mjs test/scaffold.test.mjs test/speech-playback.test.mjs test/site.test.mjs test/twin-cloud.test.mjs test/update-check.test.mjs test/user-explicit-updates.test.mjs",
+    "test:functional": "node ./bin/arcane-test.mjs test/ai-request-model.test.mjs test/ai-request-retry.test.mjs test/ai-stream-cancellation.test.mjs test/browser-speech-providers.test.mjs test/browser-wasm-gpu-notice.test.mjs test/browser-wasm-download-resume.test.mjs test/chat-stream-speech.test.mjs test/chat-transcript-scroll.test.mjs test/chat-transient-retention.test.mjs test/cli.test.mjs test/dbopfs-document-library.test.mjs test/dbopfs-png.test.mjs test/dbopfs.test.mjs test/dev-server.test.mjs test/dev-pwa.test.mjs test/dom-event-instrumentation.test.mjs test/event-manager.test.mjs test/events.test.mjs test/import-map.test.mjs test/mail-cli.test.mjs test/mail-runtime.test.mjs test/mail-server.test.mjs test/modal.test.mjs test/packaging.test.mjs test/pwa-packaging.test.mjs test/pwa-client.test.mjs test/pwa-install.test.mjs test/pwa-worker.test.mjs test/persistent-ai-chat-session.test.mjs test/reference-completeness.test.mjs test/runtime-api-behavior.test.mjs test/runtime.test.mjs test/scaffold.test.mjs test/speech-playback.test.mjs test/site.test.mjs test/twin-cloud.test.mjs test/update-check.test.mjs test/user-explicit-updates.test.mjs",
     "test:integration": "node ./bin/arcane-test.mjs test/installed-package-runtime.test.mjs test/root-app-layout.test.mjs test/integrated-shared.test.mjs test/integrated-workspace.test.mjs test/mail-browser.test.mjs test/native-plan.test.mjs test/native-provider-loader.test.mjs test/npm-release.test.mjs test/release-bundle.test.mjs test/release-capability-smoke.test.mjs test/shared-payload-batch.test.mjs test/tarball.test.mjs test/browser-wasm-cpu.test.mjs test/wllama-webgpu-runtime.test.mjs",
     "test:regression": "node ./bin/arcane-test.mjs test/channel-workflows.test.mjs test/html-import-registration.test.mjs test/logging-regression.test.mjs test/markdown-speech.test.mjs test/prepared-speech.test.mjs test/native-provider-generation.test.mjs test/speech-queue-regression.test.mjs test/testing.test.mjs test/test-sets.test.mjs",
     "check": "node tools/check-source.mjs && npm test",

package/runtime/arcane/modules/DBOPFS.js

@@ -1049,165 +1049,194 @@ class DBOPFS {
     }
 
     /**
-     * Downloads the entire database as a compressed PNG backup.
+     * Creates a PNG backup without downloading or writing database records.
+     * JSON is deflated and framed with the existing little-endian payload
+     * length, then encoded in RGB channels with opaque alpha for restoration.
      *
-     * The database is streamed as JSON, compressed using gzip via
-     * CompressionStream, and encoded into RGBA pixels inside a PNG.
-     *
-     * Each pixel stores 4 bytes of payload data.
-     * The first 4 bytes store the gzip byte length header.
-     *
-     * Output example:
-     *
-     *     DBOPFS-backup-2026-03-10-13-45-12.png
-     *
-     * @param {string} name
-     * Base name used for the generated backup filename.
-     *
-     * @returns {Promise<void>}
+     * @param {Object} options
+     * @param {string[]} [options.tableNames] Saved tables; omitted selects all.
+     * @param {Object<string,Object<string,*>>} [options.additionalTables]
+     * Caller-owned tables. Each replaces the matching saved table in this
+     * export only. Values must be JSON-serializable; no storage write occurs.
+     * @param {AbortSignal} [options.signal]
+     * Stops further preparation and prevents a subsequent download. Browser
+     * file reads and canvas encoding already in progress cannot be interrupted.
+     * @returns {Promise<Blob>} An image/png Blob in the existing backup format.
      */
-    async downloadCompressedPNG(name='DBOPFS-backup'){
-        if(!('CompressionStream' in window)){
-            throw new Error('CompressionStream not supported.')
+    async createCompressedPNG({tableNames, additionalTables = {}, signal} = {}) {
+        signal?.throwIfAborted();
+        if (!('CompressionStream' in window)) {
+            throw new Error('CompressionStream not supported.');
         }
 
-        const encoder=new TextEncoder()
-        await this.getTableNames(true)
-
-        const tableNames=Object.keys(this.#tableHandles)
-
-        const jsonStream=new ReadableStream({
-            start:async controller=>{
-                controller.enqueue(encoder.encode('{'))
+        await this.readyPromise;
+        signal?.throwIfAborted();
+        await this.getTableNames(true);
+        signal?.throwIfAborted();
 
-                for(let t=0;t<tableNames.length;t++){
-                    const tableName=tableNames[t]
-                    const table=await this.getTableHandle(tableName)
-
-                    if(t>0){
-                        controller.enqueue(encoder.encode(','))
-                    }
-
-                    controller.enqueue(
-                        encoder.encode(`${JSON.stringify(tableName)}:{`)
-                    )
-
-                    let first=true
+        function logicalTableName(name) {
+            return tableNameForDirectory(directoryNameForTable(name));
+        }
 
-                    for await(const [fileName]of table.entries()){
-                        const value=await this.get(tableName,fileName)
+        const selectedNames = new Set(
+            (tableNames === undefined ? Object.keys(this.#tableHandles) : tableNames)
+                .map(logicalTableName)
+        );
+        const suppliedTables = new Map();
+        for (const [name, records] of Object.entries(additionalTables)) {
+            const tableName = logicalTableName(name);
+            selectedNames.add(tableName);
+            suppliedTables.set(tableName, records);
+        }
 
-                        if(!first){
-                            controller.enqueue(encoder.encode(','))
-                        }
+        const database = this;
+        const encoder = new TextEncoder();
 
-                        first=false
+        async function* tableRecords(tableName) {
+            if (suppliedTables.has(tableName)) {
+                yield* Object.entries(suppliedTables.get(tableName));
+                return;
+            }
 
-                        const json=`${JSON.stringify(fileName)}:${JSON.stringify(value)}`
+            const table = database.#tableHandles[tableName];
+            if (!table) return;
+            for await (const [fileName] of table.entries()) {
+                signal?.throwIfAborted();
+                const value = await database.get(tableName, fileName);
+                signal?.throwIfAborted();
+                yield [fileName, value];
+            }
+        }
 
-                        controller.enqueue(encoder.encode(json))
+        async function* encodeDatabase() {
+            yield encoder.encode('{');
+            let firstTable = true;
+            for (const tableName of selectedNames) {
+                signal?.throwIfAborted();
+                if (!firstTable) yield encoder.encode(',');
+                firstTable = false;
+                yield encoder.encode(`${JSON.stringify(tableName)}:{`);
+
+                let firstRecord = true;
+                for await (const [fileName, value] of tableRecords(tableName)) {
+                    signal?.throwIfAborted();
+                    if (!firstRecord) yield encoder.encode(',');
+                    firstRecord = false;
+                    const serialized = JSON.stringify(value);
+                    if (serialized === undefined) {
+                        throw new TypeError(`PNG record ${tableName}/${fileName} is not JSON-serializable.`);
                     }
-
-                    controller.enqueue(encoder.encode('}'))
+                    yield encoder.encode(`${JSON.stringify(fileName)}:${serialized}`);
                 }
-
-                controller.enqueue(encoder.encode('}'))
-                controller.close()
-            }
-        })
-
-        const gzipStream=jsonStream.pipeThrough(
-            new CompressionStream('deflate')
-        )
-
-        const reader=gzipStream.getReader()
-
-        const chunks=[]
-        let totalLength=0
-
-        while(true){
-            const {done,value}=await reader.read()
-
-            if(done){
-                break
+                yield encoder.encode('}');
             }
-
-            chunks.push(value)
-            totalLength+=value.length
+            yield encoder.encode('}');
         }
 
-        const gzipBytes=new Uint8Array(totalLength)
-
-        let offset=0
-        for(const chunk of chunks){
-            gzipBytes.set(chunk,offset)
-            offset+=chunk.length
+        const encoded = encodeDatabase();
+        const jsonStream = new ReadableStream(
+            {
+                async pull(controller) {
+                    signal?.throwIfAborted();
+                    const {done, value} = await encoded.next();
+                    if (done) controller.close();
+                    else controller.enqueue(value);
+                },
+                async cancel() {
+                    await encoded.return();
+                }
+            }
+        );
+        const compressedStream = jsonStream.pipeThrough(
+            new CompressionStream('deflate'),
+            {signal}
+        );
+        const reader = compressedStream.getReader();
+        const chunks = [];
+        let totalLength = 0;
+        try {
+            while (true) {
+                const {done, value} = await reader.read();
+                signal?.throwIfAborted();
+                if (done) break;
+                chunks.push(value);
+                totalLength += value.length;
+            }
+        } finally {
+            reader.releaseLock();
         }
 
-        const payload=new Uint8Array(totalLength+4)
-
-        const view=new DataView(payload.buffer)
-        view.setUint32(0,totalLength,true)
-
-        payload.set(gzipBytes,4)
-
-        const size=Math.ceil(Math.sqrt(payload.length/3))
-
-        const totalPixels=size*size
-
-        if(totalPixels*3 < payload.length){
-            throw new Error('PNG canvas too small for payload.')
+        // Length and channel offsets belong only to the existing PNG format.
+        const payload = new Uint8Array(totalLength + 4);
+        const view = new DataView(payload.buffer);
+        view.setUint32(0, totalLength, true);
+        let offset = 4;
+        for (const chunk of chunks) {
+            payload.set(chunk, offset);
+            offset += chunk.length;
         }
 
-        const canvas=document.createElement('canvas')
-        canvas.width=size
-        canvas.height=size
-
-        const ctx=canvas.getContext('2d')
-
-        if(!ctx){
-            throw new Error('Canvas 2D context unavailable.')
+        const size = Math.ceil(Math.sqrt(payload.length / 3));
+        const canvas = document.createElement('canvas');
+        canvas.width = size;
+        canvas.height = size;
+        const ctx = canvas.getContext('2d');
+        if (!ctx) {
+            throw new Error('Canvas 2D context unavailable.');
         }
 
-        const imgData=ctx.createImageData(size,size)
-        imgData.data.fill(0);
-        
-        let p=0;
-
-        for(let i=0;i<imgData.data.length;i+=4){
-
-            imgData.data[i]   = payload[p]   ?? 0;
-            imgData.data[i+1] = payload[p+1] ?? 0;
-            imgData.data[i+2] = payload[p+2] ?? 0;
-            imgData.data[i+3] = 255;
-
-            p+=3;
+        const imgData = ctx.createImageData(size, size);
+        let position = 0;
+        for (let index = 0; index < imgData.data.length; index += 4) {
+            imgData.data[index] = payload[position] ?? 0;
+            imgData.data[index + 1] = payload[position + 1] ?? 0;
+            imgData.data[index + 2] = payload[position + 2] ?? 0;
+            imgData.data[index + 3] = 255;
+            position += 3;
         }
+        ctx.putImageData(imgData, 0, 0);
 
-        ctx.putImageData(imgData,0,0)
-
-        const blob=await new Promise(resolve=>canvas.toBlob(resolve,'image/png'))
-
-        if(!blob){
-            throw new Error('PNG export failed.')
+        const blob = await new Promise(
+            function encodeBackupPNG(resolve) {
+                canvas.toBlob(resolve, 'image/png');
+            }
+        );
+        signal?.throwIfAborted();
+        if (!blob) {
+            throw new Error('PNG export failed.');
         }
+        return blob;
+    }
 
-        const stamp=new Date()
+    /**
+     * Downloads one PNG backup using the same options as createCompressedPNG.
+     * Existing one-argument callers still export the whole database.
+     * @param {string} name Base filename; an ISO timestamp is appended.
+     * @param {Object} options PNG table selection and optional signal.
+     * @returns {Promise<void>}
+     */
+    async downloadCompressedPNG(name = 'DBOPFS-backup', options = {}) {
+        const blob = await this.createCompressedPNG(options);
+        options.signal?.throwIfAborted();
+        const stamp = new Date()
             .toISOString()
-            .slice(0,19)
-            .replace(/[:T]/g,'-')
-
-        const url=URL.createObjectURL(blob)
-
-        const a=document.createElement('a')
-        a.href=url
-        a.download=`${name}-${stamp}.png`
-
-        document.body.appendChild(a)
-        a.click()
-        a.remove()
-
-        URL.revokeObjectURL(url)
+            .slice(0, 19)
+            .replace(/[:T]/g, '-');
+
+        const url = URL.createObjectURL(blob);
+
+        const a = document.createElement('a');
+        a.href = url;
+        a.download = `${name}-${stamp}.png`;
+
+        try {
+            document.body.appendChild(a);
+            options.signal?.throwIfAborted();
+            a.click();
+        } finally {
+            a.remove();
+            URL.revokeObjectURL(url);
+        }
     }
 
     /**
@@ -1229,30 +1258,39 @@ class DBOPFS {
      *
      * @param {File|Blob} file
      * PNG backup file generated by DBOPFS.
+     * @param {Object} options
+     * @param {function(Object): Object|Promise<Object>} [options.selectTables]
+     * Receives the complete decoded table/file map once, before writes. Only
+     * its returned map is restored; the application owns selection/projection.
      *
      * @returns {Promise<void>}
-     * Resolves once the database restore process is complete.
+     * Resolves after every write succeeds. Rejected writes are collected after
+     * all table batches settle and reported in an AggregateError with the
+     * original reasons and table/file associations; successful writes remain.
      */
-    async restoreFromPNG(file){
+    async restoreFromPNG(file, {selectTables} = {}) {
         const img=await createImageBitmap(
             file,
             {premultiplyAlpha:'none'}
         )
 
-        const canvas=document.createElement('canvas')
-        canvas.width=img.width
-        canvas.height=img.height
+        let data;
+        try {
+            const canvas = document.createElement('canvas');
+            canvas.width = img.width;
+            canvas.height = img.height;
 
-        const ctx=canvas.getContext('2d')
+            const ctx = canvas.getContext('2d');
+            if (!ctx) {
+                throw new Error('Canvas 2D context unavailable.');
+            }
 
-        if(!ctx){
-            throw new Error('Canvas 2D context unavailable.')
+            ctx.drawImage(img, 0, 0);
+            data = ctx.getImageData(0, 0, img.width, img.height).data;
+        } finally {
+            img.close();
         }
 
-        ctx.drawImage(img,0,0)
-
-        const data=ctx.getImageData(0,0,img.width,img.height).data
-
         if(data.length < 4){
             throw new Error('Invalid PNG backup.')
         }
@@ -1283,12 +1321,38 @@ class DBOPFS {
 
         const json=await new Response(stream).text()
 
-        const db=JSON.parse(json)
+        const decodedTables = JSON.parse(json);
+        const db = selectTables === undefined
+            ? decodedTables
+            : await selectTables(decodedTables);
 
         const tables=Object.keys(db)
 
-        for(const table of tables){
-            await this.setMany(table,db[table])
+        const failures = [];
+        for (const tableName of tables) {
+            const fileNames = Object.keys(db[tableName]);
+            const results = await this.setMany(tableName, db[tableName]);
+            for (const [index, result] of results.entries()) {
+                if (result.status === 'rejected') {
+                    failures.push(
+                        {tableName, fileName: fileNames[index], reason: result.reason}
+                    );
+                }
+            }
+        }
+
+        if (failures.length > 0) {
+            const error = new AggregateError(
+                failures.map(
+                    function restoreWriteReason(failure) {
+                        return failure.reason;
+                    }
+                ),
+                'PNG restore could not save every record.'
+            );
+            error.code = 'DBOPFS_RESTORE_WRITE_FAILED';
+            error.failures = failures;
+            throw error;
         }
     }
 }