@augmentd-labs/canvas-edge 0.0.0-stage → 0.4.0

package/node_modules/canvas-stored/NOTICE

@@ -0,0 +1,85 @@
+Canvas StoreD
+Copyright (C) 2025-2026 Jozef Melich
+
+This program is free software: you can redistribute it and/or modify it under
+the terms of the GNU Affero General Public License as published by the Free
+Software Foundation, either version 3 of the License, or (at your option) any
+later version.
+
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License along
+with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Contact:
+    lic@augmentd.eu       licensing
+    contrib@augmentd.eu   contributions
+    security@augmentd.eu   security
+
+PART OF CANVAS
+--------------
+
+StoreD is the blob storage layer behind Canvas. It provides
+content-addressed object storage across local and remote backends.
+
+This repository is one component of the Canvas project. It is developed and
+released as part of Canvas, but it lives in its own repository:
+
+    this component   https://github.com/canvas-ui/canvas-stored
+    the project      https://github.com/canvas-ui/canvas-server
+
+Canvas components are not all licensed the same way. The server runtime and the
+services it embeds are dual-licensed. The standalone clients are AGPL-only. The
+COMPONENTS section of the project NOTICE lists which is which.
+
+NETWORK USE
+-----------
+
+This component is a library embedded in canvas-server rather than a network
+service of its own, but it forms part of a program that is made available to
+users over a network. Section 13 of the AGPL therefore applies to the combined
+work. If you run a modified Canvas and let users interact with it over a
+network, those users must be able to obtain the corresponding source of your
+modifications, including any you make to this component.
+
+canvas-server implements its side of that obligation. Every HTTP response
+carries an `X-Source-Code` header, and `GET /rest/v2/ping` (no authentication
+required) reports the running version, revision and source URL:
+
+    curl -sI https://your-host/ | grep -i x-source-code
+    curl -s  https://your-host/rest/v2/ping
+
+If you deploy a modified version, repoint these at the repository that publishes
+your changes. Set `CANVAS_SOURCE_URL`, and set `CANVAS_SOURCE_COMMIT` where the
+build has no git metadata. Removing the notice instead of repointing it does not
+satisfy the licence.
+
+COPYRIGHT AND COMMERCIAL LICENSING
+----------------------------------
+
+Copyright in this codebase is held by Jozef Melich. Augmentd s.r.o. holds an
+exclusive licence to it, including the right to grant sublicences, and is the
+entity authorised to issue commercial licences for Canvas.
+
+The AGPL is not the only option for this component. If its terms do not work for
+you, most often because you want to build a commercial or hosted product on
+Canvas without publishing your own changes, a separate commercial licence is
+available and no AGPL obligation applies to it. See COMMERCIAL.md.
+
+Enquiries: lic@augmentd.eu
+
+    Augmentd s.r.o.
+    DUETT Business Center
+    Námestie Osloboditeľov 3/A
+    040 01 Košice, Slovakia
+    Company ID: 45331936
+    VAT ID: SK2022968717
+    https://augmentd.eu
+
+THIRD-PARTY CODE
+----------------
+
+Third-party dependencies keep their own licences. See the dependency manifests
+and lockfiles for their terms. This notice covers Canvas code only.

package/node_modules/canvas-stored/README.md

@@ -0,0 +1,472 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/canvas-ai/.github/main/banners/canvas-banner_1200x480.jpg" alt="Canvas" width="100%" />
+</p>
+
+# StoreD
+
+Cache-first blob storage with a content-addressable local cache, LMDB metadata index and pluggable backends. Local backends write immediately; remote backends are fed from the cache by a sync queue (disk copies on a worker thread, network drivers such as Google Drive committed in-process).
+
+Used with canvas-server/synapsd to build virtual context trees over indexed data: checksum-based identity, multi-location replication, and `stored://` URLs as the canonical fetch form.
+
+```
+put(blob)  → stream hash → local backends (hardlink/rename) + remote (cacache → SyncQueue → backend.commit)
+get(id)    → cacache → synced backend location → cache on read (buffers only)
+```
+
+Paths and streams are ingested without buffering the whole blob in memory (safe for 10GB+ files). Local-only puts never touch the cache; the cache + sync queue are used only when a remote backend is targeted.
+
+## Quick start
+
+### Simple example
+```js
+import Stored from './src/index.js';
+
+const stored = new Stored({ root: './.stored' });
+stored.addBackend('fs:data', { driver: 'file', root: stored.dataPath });
+```
+
+### Backend configuration
+```js
+const stored = new Stored({
+  root: './.stored',                    // → ./.stored/{index,cache,data}
+  checksums: ['sha256'],                // sha256 is the default and the id/cache algorithm
+  primaryChecksum: 'sha256',
+});
+
+stored.addBackend('fs:home', {
+  driver: 'file',
+  root: './home',                       // any path - not tied to .stored/data
+  watch: true,
+  ignored: /node_modules/,
+});
+
+stored.on('object:add', ({ kind, id, key }) => { /* index in synapsd */ });
+stored.on('synced', ({ id, results }) => { /* remote write finished */ });
+
+// `backends` is required - there is no implicit fan-out.
+const res = await stored.put(Buffer.from('hello'), { key: 'docs/hello.txt', backends: ['fs:home'] });
+const data = await stored.get(res.id);
+await stored.getByUrl('stored://fs:home/docs/hello.txt');
+
+await stored.stop();
+```
+
+### Drivers
+
+| driver | type | notes |
+|---|---|---|
+| `file` | local | Any directory; chokidar watch, inode-based rename matching, container (folder) ops |
+| `cacache` | local | Content-addressable blob store (managed; opaque keys) |
+| `gdrive` | remote | Google Drive folder subtree over OAuth refresh token — see below |
+| `canvas` | remote | One backend of a workspace on a canvas hub, over the objects/sync protocol — see *Sync engine* below |
+| `s3`, `http` | remote | Skeletons: URL parsing only (`http` has a read-only `get`) |
+
+Register additional drivers at runtime with `BackendManager.register(name, Class)` (`import BackendManager from 'canvas-stored/src/backends/BackendManager.js'`).
+
+### Google Drive backend
+
+```js
+stored.addBackend('gdrive:work', {
+  driver: 'gdrive',
+  clientId: '…apps.googleusercontent.com',   // OAuth client (Desktop/Web app) with the drive scope
+  clientSecret: '…',
+  refreshToken: '1//…',                      // offline-access grant (access_type=offline, prompt=consent)
+  folderId: 'root',                          // subtree root; default = My Drive
+  watch: true,                               // poll changes.list (pollInterval, default 60000 ms)
+  permanentDelete: false,                    // default: delete() trashes instead of purging
+});
+```
+
+- Keys are `/`-joined Drive *names* below the root folder. Duplicate siblings get a ` (<id-prefix>)` suffix, `/` in a name becomes `∕`. Native Google Docs/Sheets/Slides (no bytes) are skipped.
+- `scan()` takes sha256/sha1/md5 straight from file metadata — no downloads unless you ask for an algorithm Drive doesn't serve.
+- Writes go cache → SyncQueue → `commit(key, path)` (resumable upload, 8 MiB chunks, in-place update when the key exists). Reads (`get`, `getRange`) stream from the API.
+- `watch()` emits `file:add|change|unlink` from the changes feed; renames/moves carry `ino = fileId`, so Stored's rename window rewrites the location in place instead of orphaning the document.
+- `verifyRoot()` → `{ ok, reason: 'unauthorized' | 'root-missing' | 'unreachable' }` is the liveness gate (credentials + root folder).
+
+### Canvas hub backend
+
+```js
+stored.addBackend('hub', {
+  driver: 'canvas',
+  url: 'https://canvas.example',            // hub base URL
+  workspaceId: 'ws-id',
+  backend: 'workspace:home',                // hub backend address (default)
+  token: '<device or api token>',
+  deviceId: 'dev-…',                        // stamped as X-Canvas-Origin on every mutation
+  deviceName: 'laptop',
+  instanceId: '…',                          // optional: refuse a hub whose /ping instanceId differs
+  prefixes: ['Docs'],                       // optional selective scope (prefixes or globs)
+  pollInterval: 30000,                      // change-feed poll (watch)
+  cursor: null,                             // feed position to resume from
+});
+```
+
+Speaks `canvas-server/docs/sync-protocol.md`: `stat` = `HEAD`, `get`/`getRange` = `GET` (Range), `list`/`scan` page the listing, `putStream(key, source, { ifMatch, ifNoneMatch, sha256, mtime, origin, conflictOf, conflictMode, baseSha256, deviceName, mimeType })` is a streaming `PUT`, `delete(key, { ifMatch })`, `rename(from, to, { ifMatch })`, `changes(since, limit)`, `poll()`, `watch()`. Rows carry `checksums.sha256`, `ino = sha256`, `dev = canvas:<instanceId>`, so a hub-side rename (`unlink(from) + add(to)`, same ino) is paired by Stored into an in-place location rewrite. Own-origin feed entries are applied but not emitted; a `410` clears the cursor and emits `backend:state { reason:'cursor-too-old' }`. Errors are typed on `err.code`: `PRECONDITION_FAILED` (`err.current`), `CURSOR_TOO_OLD`, `TARGET_EXISTS`, `NOT_FOUND`, `UNAUTHORIZED`, `REFUSED`, `RETRYABLE` (429/5xx, retried in-call first), `OFFLINE`.
+
+`copy()`/`move()` onto a `canvas` target stream straight to the hub (no temp file, cache or SyncQueue) and forward `ifMatch`/`ifNoneMatch`/`origin`/`mtime`/conflict headers; a `412` comes back as `{ ok:false, reason:'precondition-failed', current }`. A same-backend `move()`/`renameObject()` on the hub is one server-side rename.
+
+---
+
+## Sync engine (device mirror)
+
+```js
+import Stored, { Mirror } from 'canvas-stored';
+
+const stored = new Stored({ root: '~/Workspaces/notes/.workspace/db/stored' });
+stored.addBackend('local',     { driver: 'file', root: '~/Workspaces/notes', watch: true, ignored: ['.workspace/**'] });
+stored.addBackend('trash',     { driver: 'file', root: '~/Workspaces/notes/.workspace/trash' });
+stored.addBackend('conflicts', { driver: 'file', root: '~/Workspaces/notes/.workspace/conflicts' });
+stored.addBackend('hub',       { driver: 'canvas', url, workspaceId, token, deviceId, deviceName });
+
+const mirror = new Mirror(stored, {
+  id: 'notes', local: 'local', remote: 'hub', trash: 'trash', conflicts: 'conflicts',
+  deviceId, deviceName,
+  prefixes: [],                 // selective sync (prefixes / globs), applied on both sides
+  ignore: [],                   // on top of MIRROR_IGNORE_DEFAULTS (+ the hub's exclusions you pass here)
+  deletes: 'propagate',         // | 'keep'  — local deletes reach the hub, or only drop the base
+  conflictMode: 'prompt',       // | 'rename' — inbox upload, or Dropbox-style conflict copy on the hub
+  debounceMs: 1500, fullReconcileEvery: 6 * 3600e3,
+});
+await mirror.start();           // scan local → list/poll hub → reconcile → watch → worker
+mirror.on('conflict', (c) => …); mirror.on('skip', ({ key, reason }) => …);
+mirror.on('offline', …); mirror.on('online', …); mirror.on('status', (s) => …);
+await mirror.nudge();           // "poll now" (socket nudge)
+mirror.status();                // { state, cursor, head, pending, running, failed, conflicts, lastSyncAt, lastError, skipped, skips }
+await mirror.stop();            // drains running jobs
+```
+
+- **View**: Stored's index holds both sides — local keys through the file watcher/scan, hub keys through the `canvas` driver's feed poller. **Base ledger** (`Ledger`, LMDB sub-db `mirror`): `base/<mirror>/<key>` → `{ sha256, size, mtime, remoteSeq }`, `cursor/<mirror>`, `skip/<mirror>/<key>`, `state/<mirror>`. Keys + digests are the identity; hub document ids are never stored.
+- **Decision**: per dirty key, the protocol's three-way table (`decide(L, B, R)`, exported from `src/sync/Mirror.js`); R is confirmed with `HEAD` before a push or a conflict.
+- **Jobs** (`JobQueue`, LMDB sub-dbs `jobs` + `jobs.dedupe`): `push`, `pull`, `delete-remote`, `trash-local`, `rename-remote`, `rename-local`, `conflict`; dedupe per `<kind>|<key>`, backoff `min(60s·2^n, 1h)`, running→pending recovery on open, one job per key in flight, concurrency 4 (a brand-new key skips the pre-push HEAD; the PUT's If-None-Match:* guards it). Hub requests abort after `requestTimeoutMs` (60 s) plus an upload allowance (`minUploadBytesPerSec`, 50 kB/s) so a dropped connection never pins a worker. Every byte move goes through `stored.copy`/`move`/`renameObject`/`removeObject`, so locations, succession and echo suppression are the ordinary ones. Base is written only after the operation succeeded; the feed cursor after a whole batch reconciled.
+- **Conflicts** (`prompt`): the device version is copied to the `conflicts` backend as `<stem> (conflict from <device> <YYYY-MM-DD HHmm>).<ext>`, uploaded with `X-Canvas-Conflict-Of` (inbox), the hub version is pulled into place, base := R. `rename`: the upload is an ordinary object at the conflict-copy key (the hub tags it); the copy then arrives on every device like any other object.
+- **Renames**: a local `mv` (paired by Stored's inode window) → `POST objects/rename`; a hub rename → local `rename(2)` when the local copy is clean. **Hub delete** → local copy to the `trash` backend. **Offline** is a state: jobs stay queued, the engine retries with backoff and catches up (`poll`, or a full listing after `410`).
+- **Keys**: `src/sync/keys.js` — `normalizeKey` (NFC, `/`), `validateKey` (traversal, NUL, Windows reserved names, `<>:"|?*`, trailing dot/space, length) → skipped visibly (`skip` event, `status().skips`), `isIgnored(key, patterns)` (picomatch, `dot:true`), `matchesPrefixes`, `conflictKey`, `MIRROR_IGNORE_DEFAULTS`.
+
+`JobQueue`, `Ledger`, `Mirror`, `BackendManager` and `Index` are named exports next to the default `Stored`.
+
+---
+
+## Configuration options
+
+All paths hang off a single **`root`** (default `./.stored`):
+
+```
+.stored/
+  index/     LMDB metadata + path index
+  cache/     cacache (content-addressable)
+  data/      default root for an optional local file backend
+```
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `root` | `./.stored` | Base directory for index, cache, and data |
+| `index.path` | `<root>/index` | Override index location |
+| `cache.path` | `<root>/cache` | Override cacache location |
+| `data.path` | `<root>/data` | Override local data directory |
+| `checksums` | `['sha256']` | Algorithms computed on `put` / `scan`. sha256 is mandatory (id + cache); extra algorithms are optional |
+| `primaryChecksum` | `'sha256'` | Algorithm used for content `id` (`<algo>:<hex>`) |
+
+The constructor extends **eventemitter2** with `wildcard: true` and `delimiter: ':'`, so listeners can bind patterns like `object:*` or `scan:*`.
+
+### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `root` | `string` | Resolved store root |
+| `paths` | `{ root, index, cache, data }` | All resolved paths |
+| `dataPath` | `string` | `<root>/data` (convenience) |
+| `cache` | `Cache` | cacache wrapper |
+| `index` | `Index` | LMDB metadata store |
+| `backends` | `BackendManager` | Registered backend instances |
+
+The `cache` wrapper exposes cacache maintenance helpers directly: `stats()` (entry count + total size, streamed so large stores are never held in memory), `clear()`, `verify()`, and `list()` / `listStream()`.
+
+---
+
+## Metadata
+
+`put()`, `scan()`, and file watchers populate the index. `stat()` / `has()` / `get()` accept either a content **id** or a path key **`{backend}:{key}`**.
+
+```js
+{
+  id: 'sha256:abc...',           // primaryChecksum:hex
+  checksums: { sha256: '...', md5: '...' },
+  size: 1234,
+  mimeType: 'text/plain',
+  created: 1710000000000,
+  modified: 1710000000000,
+  custom: { /* from put(..., { metadata }) */ },
+  locations: [
+    {
+      backend: 'fs:home',
+      driver: 'file',
+      key: 'docs/hello.txt',
+      synced: true,              // false until remote SyncQueue completes
+      source: {                  // provenance descriptor for synapsd / UI
+        provider: 'fs',
+        account: 'home',
+        container: 'home',         // basename of root, or bucket/share/folder from config
+        path: 'docs/hello.txt',
+      },
+    },
+  ],
+}
+```
+
+Auto-generated keys (when `put` has no `key`) use a sharded hash path: `ab/cd/<full-hex>`.
+
+---
+
+## `stored://` URLs
+
+Canonical fetch/delete form: `stored://<backend>/<key>`. Backend names may contain colons (`fs:data:email`); only the **first** `/` after the scheme separates backend from key.
+
+| Method | Description |
+|--------|-------------|
+| `getByUrl(url)` | `backend.get(key)` - bypasses the content index. Returns `Buffer \| null` (never throws on bad input). |
+| `getStreamByUrl(url)` | Same as `getByUrl` but returns a `Readable \| null`. |
+| `deleteByUrl(url)` | Deletes bytes on the backend and drops that location from the index (the consumer's document index is still the caller's to trim). Emits `object:location:remove` when the content survives elsewhere. Returns `{ ok: boolean, reason?: string }` where `reason` is `malformed-url`, `unknown-backend`, or `read-only-backend`. |
+
+---
+
+## API
+
+### Backend management
+
+| Method | Returns | Description |
+|--------|---------|-------------|
+| `addBackend(name, config)` | `StorageBackend` | Register driver; wire events; start `watch()` if `config.watch` |
+| `removeBackend(name)` | `Promise<boolean>` | `stop()` backend and unregister |
+| `listBackends()` | `string[]` | Registered backend names |
+| `getBackend(name)` | `StorageBackend \| undefined` | Instance by name |
+
+**`addBackend` config** (common):
+
+| Field | Description |
+|-------|-------------|
+| `driver` | `'file'` \| `'cacache'` \| `'s3'` \| `'http'` |
+| `watch` | Start backend watcher when true (file: chokidar) |
+| `root` | File backend root directory (required for `file`) |
+| `ignored` | chokidar ignore pattern (file backend) |
+| `provider`, `account`, `container`, `bucket`, `share`, `folder` | Override `location.source` fields (else derived from backend name + config) |
+
+Throws if the name exists or the driver is unknown.
+
+### Core
+
+#### `put(blob, options?) → Promise<{ ok, id, key, size, mimeType, checksums, locations, … }>`
+
+Accepts `Buffer`, `string` (content), filesystem path (`string` path readable as file), or `Readable` stream. Paths and streams are hashed and written in a single streaming pass - the whole blob is never held in memory.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `key` | auto from checksum | Storage key on backends |
+| `backends` | **required** | Target backend names. Empty/missing → `{ ok:false, reason:'no-targets' }` |
+| `metadata` | `{}` | Stored as `custom` on the index entry |
+| `mimeType` | sniffed | Override the detected mime type |
+
+Flow: stream → hash (+ head peek for mime) → local backends (hardlink/rename from a same-fs temp) + remote placeholders (streamed into cacache) → index → enqueue remote sync. Returns `{ ok:false, reason }` on failure (`no-targets`, `unknown-backend`).
+
+Emits `put` with `{ id, key, metadata }`.
+
+#### `get(idOrKey) → Promise<Buffer \| null>` / `getStream(idOrKey) → Promise<Readable \| null>`
+
+Lookup order: path index (`backend:key`) or id → cacache → first **synced** location on a backend. On a backend buffer hit, bytes are written back to cache asynchronously.
+
+#### `delete(idOrKey, options?) → Promise<{ ok, deleted, kept } \| { ok:false, reason:'not-found' }>`
+
+Removes from cache and backends. The index entry is removed only when no locations remain.
+
+| Option | Description |
+|--------|-------------|
+| `backends` | Only delete locations on these backends |
+| `urls` | Only delete the listed `stored://<backend>/<key>` locations (precise multi-name targeting) |
+
+Emits `delete` with `{ id, backends }` (names deleted).
+
+#### `copy(idOrKey, options) → Promise<{ ok, id, added, locations } | { ok:false, reason }>`
+
+Copy bytes onto other backends. Content identity is unchanged — the same index entry gains locations.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `to` | **required** | Target backend name, or an array of them |
+| `key` | source key | Key on the targets |
+| `from` | cheapest readable location | Source backend name or `stored://` URL |
+| `onConflict` | `error` | Other content already on the destination key: `error` (refuse), `rename` (next free `name-1.ext`), `overwrite` |
+
+Bytes stream through one hashing pass and the result is checked against the indexed id **before** anything is committed, so a source that was rewritten behind stored's back fails with `checksum-mismatch` instead of propagating wrong bytes under the old id. Targets that already hold the key are skipped (`unchanged:true`, not an error).
+
+Destination keys are normalized (`Fotky//2019` → `Fotky/2019`) so the index always spells a key the way the filesystem reports it back.
+
+Reasons: `not-found`, `no-targets`, `no-source`, `unknown-backend`, `read-only-target`, `source-offline`, `target-offline`, `source-unreadable`, `checksum-mismatch`, `target-exists`, `invalid-key`, `transfer-failed`.
+
+Emits `object:location:add` per new location.
+
+#### `move(idOrKey, options) → Promise<{ ok, id, state, from, to, locations } | { ok:false, reason }>`
+
+Same options as `copy`, but exactly one target (`move-single-target` otherwise) and the source location is dropped afterwards.
+
+- Within one filesystem the transfer is a single `rename(2)` — no bytes cross userspace and no re-hash (the inode carries the identity).
+- Across filesystems/drivers it is a verified stream copy, then a source delete.
+- Onto a `type:'remote'` backend it returns `state:'pending'` and completes on the `synced` event: **the source is never released before the destination write is durable.** A failed sync leaves the object as a copy, never lost.
+- If the destination lands but the source delete fails, the result is `{ ok:false, reason:'source-delete-failed', degradedToCopy:true }` — a cleanup problem, not data loss.
+
+Emits `object:move` on completion. Consumers must patch `locations[]` **in place**; treating it as unlink + add loses the document's identity and curated placements.
+
+#### `stat(idOrKey) → Promise<Metadata | null>`
+
+#### `has(idOrKey) → Promise<boolean>`
+
+#### `locations(idOrKey) → Promise<Location[]>`
+
+Where the content actually lives: `{ url, nativeUrl, backend, key, driver, synced, size, source }` per location.
+
+- `url` - canonical `stored://<backend>/<key>`, the fetch form (`getByUrl`). Single source of truth for the grammar.
+- `nativeUrl` - the real protocol URL for provenance/UI: `https://…`, `s3://bucket/…`, `imap://account/…`, `file://…` (local), or `null` when the backend has none.
+
+Returns `[]` when unknown. Backend drivers render `nativeUrl(key)`; remote backends are the interesting case (a blob can live under several names across backends).
+
+#### `list() → AsyncIterable<Metadata>`
+
+Yields all index metadata entries.
+
+#### `listBackend(name, options?) → AsyncIterable`
+
+Yields `{ key, size, … }` from that backend’s native `list()`. `options` (`prefix`, `recursive`) are passed through.
+
+#### `scan(backendName?, options?) → Promise<{ ok, backend, count, files }>`
+
+Index existing objects from one backend or all. `options` is forwarded verbatim to each backend's `scan()` (per-driver knobs). File backends skip re-hashing files whose size+mtime are unchanged. Updates locations, prunes index entries for keys removed on disk; `files` are per-file scan rows `{ key, size, checksums, mimeType, backend, … }`.
+
+### Lifecycle
+
+#### `stop() → Promise<void>`
+
+Stops sync queue, all backend watchers, and closes the LMDB index.
+
+---
+
+## Backends
+
+Registered in `BackendManager` via `DRIVERS`:
+
+| Driver | Type | Status | Notes |
+|--------|------|--------|-------|
+| `file` | `local` | complete | CRUD, chokidar `watch`, `scan` |
+| `cacache` | `local` | complete | Content-addressable blob store (cacache); CRUD, no `watch`/`scan` |
+| `http` | `remote` | partial | Read-only `get` / `stat` via `fetch`; `config.baseUrl`, `headers` |
+| `s3` | `remote` | skeleton | Registered for URL parsing; CRUD not implemented |
+
+Each backend implements `StorageBackend`: `put`, `get`, `delete`, `stat`, `list`; optional `commit` (local, for streaming put), `watch`, `scan`, `stop`.
+
+> StoreD abstracts **blob** backends only. Non-blob connectors (mail/IMAP, git, …) are not drivers here - they live in separate consumer services and use StoreD only to persist the blobs they produce. The generic `object:*` / `backend:state` event surface still lets such a connector emit through a host that wraps StoreD, but the IMAP protocol itself is no longer bundled.
+
+| Capability | Meaning |
+|------------|---------|
+| `capabilities.delete === false` | `deleteByUrl` returns `read-only-backend` (e.g. HTTP) |
+| `type === 'local'` | Written synchronously on `put` |
+| `type === 'remote'` | Written via `SyncQueue` after cache write |
+
+### `file` driver config
+
+```js
+{ driver: 'file', root: '/path/to/root', watch: true, ignored: /pattern/, algorithms: ['sha256'] }
+```
+
+#### Network mounts (`remote`)
+
+Remote shares are consumed as **OS-level mounts** (kernel cifs/nfs, `rclone mount`, …) rather than protocol drivers: the file driver already carries the whole vocabulary a share needs (`createRoot:false`, `verifyRoot()` fsid liveness, partial-scan semantics, ranged reads, `stat.dev/ino` for rename matching and skip-hash), and the kernel client caches far better than a userspace one.
+
+```js
+{ driver: 'file', root: '/mnt/nas/photos', remote: true, transport: 'cifs' }
+```
+
+Detected from the kernel mount table (`/proc/mounts`, Linux) at registration; explicit config always wins. `remote` is a **separate axis from `type`**: `type: 'local'|'remote'` decides the write path (synchronous put vs cache + SyncQueue), and a CIFS mount is still a synchronous POSIX write target — `type:'local'`, `remote:true`.
+
+What it changes:
+
+- **`watch()` refuses to start** without `usePolling: true`. inotify only reports changes made through *this* client, so a watched share silently misses everything another machine writes — worse than not watching, because it looks live. A `backend:state` event with `reason: 'remote-watch-requires-polling'` is emitted instead; use `scan()`/resync. With `usePolling` the watcher polls at `pollInterval` (default 30s).
+- **Reads prefer local locations.** `get()`/`getStream()` order candidates cache → local disk → network mount → protocol backend, so a LAN round-trip never wins over a local copy that was merely indexed later.
+- **`locations()` carries `remote` + `transport`**, and the location's `source` descriptor records them too (sparsely — only when remote), so a location stays self-describing after its backend is unmounted or dropped from config.
+- Liveness is unchanged and load-bearing: an unmounted share either vanishes or exposes a different `dev`, and `verifyRoot()` fails the scan with `complete:false` and **zero rows**. An absent mount is never read as an empty one.
+
+### `cacache` driver config
+
+```js
+{ driver: 'cacache', root: '/path/to/blobstore', algorithms: ['sha256'] }
+```
+
+Content-addressable local blob store backed by [cacache](https://www.npmjs.com/package/cacache): bytes are sha-keyed, deduped, and integrity-checked. Same key→value CRUD surface as `file` (`put`/`commit`/`get`/`delete`/`stat`/`list`), so it is a drop-in `type: 'local'` write target - bytes land synchronously on `put`, no `SyncQueue`. `config.root` is the only required field ("the data route"); `nativeUrl` is `null` (the store is internal - `stored://<backend>/<key>` is the sole address) and the store is **not** watched/scanned (a managed write target, not an external source). canvas-server wires it as the opt-in `workspace:data` backend (disabled by default).
+
+### `http` driver config
+
+```js
+{ driver: 'http', baseUrl: 'https://cdn.example/', account: 'cdn', headers: {} }
+```
+
+---
+
+## Events
+
+Stored re-emits backend events and adds its own. File watcher events are duplicated as **`file:*`** and **`object:*`** (`kind: 'file'`) so consumers can bind either `file:add` or `object:add`.
+
+### File / object (from file backend + Stored index updates)
+
+| Event | Payload (typical) |
+|-------|-------------------|
+| `file:add` | `{ backend, key, path, checksums, mimeType, size, id, locations }` |
+| `file:change` | Same shape on modify (emits `object:unlink` for old content, then `object:add` for new) |
+| `file:unlink` | `{ backend, key, path?, id?, checksums?, locations? }` |
+| `object:add` | `{ kind: 'file', … }` (or `{ kind, backend, key, … }` from a non-file connector emitting through stored) |
+| `object:change` | Forwarded from backends that emit it |
+| `object:unlink` | Forwarded or synthesized on file delete |
+
+### Location events (identity-preserving)
+
+Same content, different places. Consumers **patch `locations[]` in place** — routing these through unlink + add rebuilds the document and loses its identity and curated placements. Every payload carries the full post-mutation `locations` array.
+
+| Event | Payload | Meaning |
+|-------|---------|---------|
+| `object:move` | `{ kind, id, checksums, from:{backend,key,url}, to:{…}, location, locations }` | One location replaced another (`move()`), destination already durable |
+| `object:location:add` | `{ kind, id, checksums, location, locations }` | Content gained a location (`copy()`), keeping the existing ones |
+| `object:location:remove` | `{ …, reason }` | One location dropped while others survive. `reason`: `deleted` (today) — `evicted` / `backend-removed` land with the cache policy |
+
+Note the three-segment names: an `object:*` wildcard binding does **not** match them (eventemitter2 `*` is single-level); bind them explicitly or use `object:**`.
+
+Watcher echoes from stored's own transfers are suppressed, so a move never also arrives as an `unlink` of the source key.
+
+Wildcard: `object:*`, `file:*`, `scan:*`.
+
+### API / sync / scan
+
+| Event | Payload |
+|-------|---------|
+| `put` | `{ id, key, metadata }` |
+| `delete` | `{ id, backends }` |
+| `synced` | `{ id, results }` - `results[]` has `{ backend, success, … }`; successful rows set `location.synced = true` |
+| `scan:start` | `{ backend }` |
+| `scan:complete` | `{ backend, count }` |
+| `backend:state` | Backend-specific cursor/state `{ backend, … }` (forwarded for any connector that emits it) |
+| `error` | From backends or sync queue |
+
+---
+
+## Workspace integration
+
+canvas-server owns a `Stored` instance per workspace at `{WORKSPACE_ROOT}/.stored` (index + cacache). Workspace blobs still live under `data/`, `home/`, etc. as separate file backends. `WorkspaceStoredIndex` listens on `object:*`, builds synapsd documents with `stored://` locations, and resolves blobs via `getByUrl`. See `STORAGE-URL-SCHEME.md` at the repo root for the full URL grammar.
+
+## Licence
+
+Copyright (C) 2025-2026 Jozef Melich. Canvas StoreD is dual-licensed:
+
+- **[AGPL-3.0-or-later](LICENSE)**, free for everyone. Run it, modify it, build
+  on it. If you distribute a modified version, or expose one to users over a
+  network, they are entitled to your changes (AGPL section 13).
+- **[Commercial licence](COMMERCIAL.md)**, the same code without the copyleft
+  obligations, for hosted products and proprietary distribution. Issued by
+  Augmentd s.r.o., lic@augmentd.eu.
+
+Same software either way. There is no cut-down community edition. See
+[NOTICE](NOTICE) for the full position, and [CONTRIBUTING.md](CONTRIBUTING.md)
+before opening a pull request.

package/node_modules/canvas-stored/package.json

@@ -0,0 +1,26 @@
+{
+  "name": "canvas-stored",
+  "version": "1.9.1",
+  "description": "",
+  "main": "./src/index.js",
+  "type": "module",
+  "author": "Jozef Melich <me@idnc.sk>",
+  "license": "AGPL-3.0-or-later",
+  "dependencies": {
+    "cacache": "^20.0.4",
+    "chokidar": "^5.0.0",
+    "debug": "^4.4.3",
+    "eventemitter2": "^6.4.9",
+    "exifr": "^7.1.3",
+    "file-type": "^21.3.3",
+    "fs-extra": "^11.3.4",
+    "image-size": "^2.0.2",
+    "imap": "^0.8.19",
+    "lmdb": "^3.5.2",
+    "music-metadata": "^11.9.0",
+    "picomatch": "^2.3.1"
+  },
+  "engines": {
+    "node": ">=20.0.0"
+  }
+}

package/node_modules/canvas-stored/src/backends/BackendManager.js

@@ -0,0 +1,60 @@
+import Debug from 'debug';
+import FileBackend from './file/index.js';
+import CacacheBackend from './cacache/index.js';
+import S3Backend from './s3/index.js';
+import HttpBackend from './http/index.js';
+import GdriveBackend from './gdrive/index.js';
+import CanvasBackend from './canvas/index.js';
+
+const debug = Debug('stored:backends');
+
+// `s3`/`http` are skeletons (see their index.js). `smb`/`webdav` are reserved
+// scheme names with no driver yet — register a driver class here (or at
+// runtime via BackendManager.register) when implemented. `gdrive` is the first
+// real remote driver; `canvas` mirrors one backend of a workspace on another
+// canvas-server over the objects protocol (docs/sync-protocol.md). Non-blob connectors (mail/git/…) are NOT stored drivers;
+// they live in separate workspace services and only use stored to persist blobs.
+const DRIVERS = { file: FileBackend, cacache: CacacheBackend, s3: S3Backend, http: HttpBackend, gdrive: GdriveBackend, canvas: CanvasBackend };
+
+export default class BackendManager {
+    #backends = new Map();
+
+    /** Register (or override) a driver class under `driver` — plugin/test seam. */
+    static register(driver, Driver) {
+        if (!driver || typeof Driver !== 'function') throw new Error('register(driver, DriverClass) required');
+        DRIVERS[driver] = Driver;
+    }
+
+    static drivers() { return Object.keys(DRIVERS); }
+
+    get(name) { return this.#backends.get(name); }
+    has(name) { return this.#backends.has(name); }
+    list() { return [...this.#backends.keys()]; }
+    all() { return [...this.#backends.values()]; }
+
+    add(name, config) {
+        if (this.#backends.has(name)) throw new Error(`Backend "${name}" already exists`);
+        const Driver = DRIVERS[config.driver];
+        if (!Driver) throw new Error(`Unknown driver: ${config.driver}`);
+
+        const backend = new Driver(name, config);
+        this.#backends.set(name, backend);
+        debug(`Added backend "${name}" (${config.driver})`);
+        return backend;
+    }
+
+    async remove(name) {
+        const backend = this.#backends.get(name);
+        if (!backend) return false;
+        await backend.stop();
+        this.#backends.delete(name);
+        debug(`Removed backend "${name}"`);
+        return true;
+    }
+
+    async stopAll() {
+        for (const backend of this.#backends.values()) {
+            await backend.stop();
+        }
+    }
+}