mikser-io-drive 11.3.0 → 11.4.0

package/README.md

@@ -137,12 +137,31 @@ Three protocol facts worth knowing, all asserted in `test/protocol.test.js`:
 | --- | --- |
 | default (`emulate`) | `DAV: 1, 3, 2` — **class 2**, which macOS requires for a read-write mount |
 | `locks: 'disallow'` | drops class 2 and `LOCK` from `Allow`. **Finder will refuse a read-write mount.** A trap, because it is invisible until someone tries |
-| `emulate` vs `meta-files` | both return a valid `Lock-Token` header; `emulate` returns an empty `<lockdiscovery/>` where `meta-files` returns the full `<activelock>` |
-
-The last is the real cost of the default: clients read the header, so this is
-survivable, but a client that parses the body for the token finds nothing.
-Choose `meta-files` if you need real dead properties or real locking — with
-persisted locks a second `LOCK` on a held resource correctly answers `423`.
+| `emulate` vs `meta-files` | both are real locking now (11.4.0): the `<activelock>` is in the LOCK body and in a later PROPFIND, and a held resource answers `423`. They differ in where the lock lives — memory, or a sidecar |
+
+**Until 11.4.0 `emulate` stored no lock at all** — a token in the header and an
+empty `<lockdiscovery/>`, which this table used to describe as survivable
+because "clients read the header". The Windows WebDAV redirector does not: it
+needs the `<activelock>` RFC 4918 §9.10.1 requires, and without it a mapped
+drive lists and reads perfectly and **cannot be written to**. Every save fails
+with "The parameter is incorrect" and leaves a zero-byte file, because Windows
+PUTs an empty file, LOCKs it, PROPFINDs it, finds no lock, and loops until it
+gives up. It reads like a permissions problem and is not.
+
+`emulate` now keeps locks in memory, so the folder stays free of sidecars and
+the locks are real. They do not survive a restart — the same observable as
+every lock timing out at once, which is why locks carry timeouts — and there is
+no second process to share them with, since one mikser instance owns a working
+folder.
+
+Choose `meta-files` if you need **dead properties** to persist. That is the one
+thing `emulate` still does not do, deliberately: a PROPPATCH answers `207` and
+the property is gone by the next PROPFIND (measured). Properties are meant to
+last, and an in-memory store would lose them on restart while a same-session
+read made them look durable — not storing is the honest answer. In practice
+what Windows PROPPATCHes is its Win32 timestamps, so on `emulate` a file keeps
+the server's mtime rather than the client's. It does not affect the write: the
+PUT has already succeeded by then.
 
 One more, because it surprises people: **`LOCK` on a path that does not exist
 creates an empty file** (RFC 4918 §9.10.4). A client that locks before writing
@@ -353,6 +372,45 @@ class of problem from Nephele's four known gaps, and it carries 65 open issues
 against Nephele's zero. Nephele is also Apache-2.0 and load-bearing for its
 author's own product, which is the maintenance signal that matters most.
 
+**Writing it here was considered and is not close.** RFC 4918 is about 4,650
+lines in Nephele — 2,994 of method handlers and 1,662 of machinery — and the
+weight is not where it looks:
+
+| | lines | |
+| --- | --- | --- |
+| `Method.js` | 930 | the base class every method inherits |
+| — `checkIfHeader` | 234 | the `If:` header alone |
+| — `checkConditionalHeaders` | 77 | |
+| `LOCK` + `UNLOCK` | 442 | plus 81 lock references threaded through `Method.js` |
+| `MOVE` + `COPY` | 438 | `Depth` and `Overwrite` semantics |
+| `PROPFIND` | 276 | |
+| `GET`/`HEAD` | 286 | ranges, etags |
+
+Listing files is the cheap part. `If:` is a grammar rather than a value —
+tagged and untagged lists, `Not`, lock tokens and etags mixed, evaluated
+against resources other than the request target — and getting it wrong means
+Finder's Save As either walks over somebody's lock or refuses forever. Locking
+is not isolated in `LOCK.js` either: provisional locks, timeouts, per-user lock
+permission and token extraction run on every write method.
+
+And the litmus column above is what would be given up: 95 of 104 on
+`meta-files`, with four known gaps named and pinned. A hand-written server
+starts at zero and earns each point by a client breaking in someone's hands.
+
+The read-only case is genuinely different — `OPTIONS` + `PROPFIND` + `GET` with
+no locking and no conditionals is a few hundred lines, and the drive root in
+`lib/root-listing.js` is already that shape. It is not the case anyone is
+asking for.
+
+**The sharper version of the question is the ADAPTER, not the engine.** Three
+things have been worked around here, and all three are in
+`@nephele/adapter-file-system` rather than in the protocol: `displayname`
+commented out of its live-property list with a TODO, a flat `max-age=604800` on
+the OPTIONS response, and a read-only mount advertising `PUT` in `Allow`. The
+adapter answers "what is at this path and what are its properties", which is
+the thin half. If it keeps costing, replace it and keep the 4,650 lines that
+matter.
+
 Staying on Nephele. If its gaps ever become the binding constraint, the
 serious alternative is not another npm package — it is **Apache `mod_dav`**,
 the implementation litmus was written to test, which reads `htpasswd` and

package/index.js

@@ -6,6 +6,7 @@ import { registerFileTools } from './lib/files.js'
 import { withStagedWrites, stageWrites } from './lib/staged-writes.js'
 import { withDisplayName } from './lib/display-name.js'
 import { rootListing } from './lib/root-listing.js'
+import { withEmulatedLockStore } from './lib/emulated-locks.js'
 
 export { MikserAuthenticator, withStagedWrites, stageWrites }
 
@@ -47,9 +48,12 @@ export const writeCapability = (name) => `drive:${name}:write`
  *             data:    { folder: 'data', readOnly: true },
  *             Reports: { folder: 'reports' },            // shows as "Reports"
  *             q3:      { folder: 'q3', displayName: 'Q3 reports' },
- *             share:   { folder: 'share', host: 'drive.example.com' },
  *         },
  *         auth: identity,
+ *         // A domain of its own, where `/` lists the endpoints the caller
+ *         // may read. Belongs to the DRIVE, not to one endpoint: the whole
+ *         // point is that several endpoints share it.
+ *         host: 'drive.example.com',
  *     })
  *
  * One Nephele server per endpoint, mounted at `<base>/<name>` — the same
@@ -90,12 +94,29 @@ export function drive(options = {}) {
         // browse and commit should not fill up with page.md.nephelemeta
         // files.
         //
-        // The cost is measurable and small: 'emulate' returns a valid
-        // Lock-Token header but an EMPTY <lockdiscovery/> body, where
-        // 'meta-files' returns the full <activelock>. Clients read the
-        // header; one that parses the body for the token would not find it.
-        // Choose 'meta-files' if you need real dead properties or real
-        // locks, and accept the sidecars.
+        // `locks: 'emulate'` is a real locking implementation now — see
+        // lib/emulated-locks.js. It stored nothing until 11.4.0, answering an
+        // empty <lockdiscovery/>, and the comment here called that cost
+        // "measurable and small", affecting "a client that parses the body for
+        // the token". That client is the Windows WebDAV redirector, and the
+        // cost was that the drive could not be WRITTEN to from Explorer at
+        // all: every save failed with "The parameter is incorrect" and left a
+        // zero-byte file. Listing and reading worked, which is what made it
+        // look like a permissions problem for weeks.
+        //
+        // `properties: 'emulate'` still reports success for a PROPPATCH and
+        // stores nothing — measured: 207 back, and the property is gone on the
+        // next PROPFIND. That is deliberate where the lock case was not.
+        // Locks carry timeouts, so keeping them in memory loses nothing a
+        // client does not already handle; dead properties are meant to
+        // persist, and an in-memory store would lose them on restart while a
+        // same-session read made them look durable. Not storing is the honest
+        // answer; 'meta-files' is the one that stores them.
+        //
+        // What Windows PROPPATCHes after a write is its Win32 timestamps, so
+        // on 'emulate' a file keeps the server's mtime rather than the
+        // client's. It does not block the write — the PUT is already done by
+        // then.
         //
         // Do NOT choose 'disallow' if macOS clients matter: it drops DAV
         // compliance class 2 from the OPTIONS response, and Finder refuses a
@@ -145,6 +166,18 @@ export function drive(options = {}) {
                 if (!ep.folder) {
                     throw new Error(`drive: endpoint ${JSON.stringify(name)} declares no folder`)
                 }
+                // `host` belongs to the drive, not to an endpoint — several
+                // endpoints share one domain, and `/` on it lists them. Said
+                // out loud because the alternative is a config that reads as
+                // though it works and does nothing: this option was
+                // per-endpoint while it was being designed, and the example in
+                // this file's own JSDoc showed it that way after it moved.
+                if (ep.host) {
+                    logger.warn(
+                        'drive: endpoint %j sets `host`, which is a plugin-level option and is ignored here. '
+                        + 'Move it beside `endpoints`: drive({ host: %j, endpoints: { … } }).',
+                        name, ep.host)
+                }
                 const root = resolve(ep.folder)
                 const mountPath = `${base}/${name}`
 
@@ -252,13 +285,14 @@ export function drive(options = {}) {
                     // badly as a folder label. `displayName` overrides it for
                     // a key that is not presentable.
                     adapter: withDisplayName(
-                        (ep.atomicWrites === false)
-                            ? fsAdapter
-                            : withStagedWrites(fsAdapter, {
-                                onFailure: (err, file) => logger.warn(
-                                    'drive: upload of %s failed, original left intact — %s',
-                                    path.basename(file), err.message),
-                            }),
+                        withEmulatedLockStore(
+                            (ep.atomicWrites === false)
+                                ? fsAdapter
+                                : withStagedWrites(fsAdapter, {
+                                    onFailure: (err, file) => logger.warn(
+                                        'drive: upload of %s failed, original left intact — %s',
+                                        path.basename(file), err.message),
+                                })),
                         ep.displayName ?? name,
                     ),
                     authenticator,

package/lib/emulated-locks.js

@@ -0,0 +1,147 @@
+// Real locks, without sidecar files.
+//
+// `locks: 'emulate'` kept the content folder clean by not storing locks at
+// all: the adapter's readMetadataFile returns `{}` unless locks are
+// 'meta-files', so `Lock.save()` writes nowhere and `getLocks()` answers `[]`
+// the moment the response is sent. The token in the `Lock-Token` header is the
+// only trace the lock ever existed.
+//
+// That is not a cosmetic gap. The Microsoft WebDAV redirector needs the
+// `<activelock>` that RFC 4918 §9.10.1 requires in the LOCK body, and without
+// it Windows cannot WRITE to the drive at all — it PUTs an empty file, LOCKs
+// it, PROPFINDs it, sees no lock, and loops until it gives up with
+// ERROR_INVALID_PARAMETER. Listing and reading work, so the share looks
+// mounted and healthy while every save fails with "The parameter is
+// incorrect". Captured on the wire against
+// Microsoft-WebDAV-MiniRedir/10.0.22631.
+//
+// So locks are kept in memory instead. The whole lock lifecycle — save,
+// delete, getLocks, and the lockdiscovery PROPFIND reads — goes through those
+// two methods on the RESOURCE, which makes them the seam: route the `locks`
+// half of the metadata through a Map and leave the `props` half exactly as it
+// was.
+//
+// In memory is the right scope here rather than a compromise. One mikser
+// instance owns a working folder — the engine enforces it, forwarding any
+// second invocation over a unix socket rather than opening the folder twice —
+// so there is no other process to share a lock with. What it does mean is that
+// locks do not survive a restart, which is the same observable as every lock
+// timing out at once, and is why they carry timeouts.
+
+// path -> { token -> entry }, shared by every resource from one adapter.
+// Keyed by absolute path because that is what identifies a resource across the
+// several Resource objects a single request can produce.
+const storeFor = new WeakMap()
+
+function locksFor(adapter) {
+    if (!storeFor.has(adapter)) storeFor.set(adapter, new Map())
+    return storeFor.get(adapter)
+}
+
+// The lock methods themselves, not the metadata underneath them.
+//
+// Intercepting readMetadataFile looks like the smaller seam and does nothing:
+// a Proxy binds methods to the TARGET, so when `getLocks()` runs it is the raw
+// resource's `this.readMetadataFile()` that answers, and the interception is
+// never reached. The same shape cost an afternoon on `displayname`, where
+// PROPFIND reads through the by-user variants and those delegate to the plain
+// ones on `this`.
+//
+// So the three methods that make up the lock lifecycle are replaced outright,
+// and the metadata file is left alone.
+function withEmulatedLocks(resource, locks) {
+    const key = resource.absolutePath
+    const held = () => locks.get(key) ?? {}
+
+    // A Lock that saves to the store instead of to a sidecar. Built from the
+    // adapter's own, so it is the adapter's Lock class with the adapter's
+    // defaults — only the two persistence methods differ.
+    const stored = (lock) => new Proxy(lock, {
+        get(target, prop, receiver) {
+            if (prop === 'save') {
+                return async () => {
+                    locks.set(key, {
+                        ...held(),
+                        [target.token]: {
+                            username:    target.username,
+                            date:        target.date.getTime(),
+                            timeout:     target.timeout,
+                            scope:       target.scope,
+                            depth:       target.depth,
+                            provisional: target.provisional,
+                            owner:       target.owner,
+                        },
+                    })
+                }
+            }
+            if (prop === 'delete') {
+                return async () => {
+                    const rest = { ...held() }
+                    delete rest[target.token]
+                    if (Object.keys(rest).length) locks.set(key, rest)
+                    else locks.delete(key)
+                }
+            }
+            const value = Reflect.get(target, prop, receiver)
+            return typeof value === 'function' ? value.bind(target) : value
+        },
+    })
+
+    const revive = async (token, entry) => {
+        const lock = await resource.createLockForUser({ username: entry.username })
+        lock.token       = token
+        lock.date        = new Date(entry.date)
+        lock.timeout     = entry.timeout
+        lock.scope       = entry.scope
+        lock.depth       = entry.depth
+        lock.provisional = entry.provisional
+        lock.owner       = entry.owner
+        return stored(lock)
+    }
+
+    return new Proxy(resource, {
+        get(target, prop, receiver) {
+            if (prop === 'getLocks') {
+                return async () => Promise.all(
+                    Object.entries(held()).map(([token, entry]) => revive(token, entry)))
+            }
+            if (prop === 'getLocksByUser') {
+                return async (user) => Promise.all(
+                    Object.entries(held())
+                        .filter(([, entry]) => entry.username === user?.username)
+                        .map(([token, entry]) => revive(token, entry)))
+            }
+            if (prop === 'createLockForUser') {
+                return async (user) => stored(await target.createLockForUser(user))
+            }
+            const value = Reflect.get(target, prop, receiver)
+            return typeof value === 'function' ? value.bind(target) : value
+        },
+    })
+}
+
+/**
+ * Give an adapter's resources a lock store that outlives the response.
+ *
+ * Same shape as withStagedWrites and withDisplayName, and composes with both:
+ * each wraps the three methods that hand back a resource and passes the rest
+ * through.
+ *
+ * @param {object} adapter a nephele adapter
+ */
+export function withEmulatedLockStore(adapter) {
+    const locks = locksFor(adapter)
+    const wrap = (method) => async (...args) => {
+        const resource = await adapter[method](...args)
+        return resource ? withEmulatedLocks(resource, locks) : resource
+    }
+    return new Proxy(adapter, {
+        get(target, prop, receiver) {
+            if (prop === 'getResource' || prop === 'newResource' || prop === 'newCollection') {
+                return wrap(prop)
+            }
+            const value = Reflect.get(target, prop, receiver)
+            return typeof value === 'function' ? value.bind(target) : value
+        },
+    })
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mikser-io-drive",
-  "version": "11.3.0",
+  "version": "11.4.0",
   "description": "WebDAV for mikser-io. Exposes working-folder directories as WebDAV endpoints via Nephele, authenticated through mikser-io-auth's htpasswd identity — so content can be edited from Finder, Explorer or any DAV client and the build picks the change up.",
   "main": "index.js",
   "type": "module",

package/test/protocol.test.js

@@ -142,19 +142,61 @@ describe('DAV compliance class — what decides whether Finder will mount', () =
         }
     })
 
-    it("but 'emulate' returns an EMPTY lockdiscovery body where 'meta-files' returns the lock", async () => {
-        // RFC 4918 puts the activelock in the body. Clients read the header,
-        // so this is survivable — but a client that parses the body for the
-        // token finds nothing, and that is the cost of a clean folder.
-        const emulated = await (await lock('emulated', 'body-emulated.md')).text()
-        assert.match(emulated, /<lockdiscovery\s*\/>/)
-
-        const meta = await (await lock('meta', 'body-meta.md')).text()
-        assert.match(meta, /<activelock>/)
+    it('every mode that advertises class 2 returns a real activelock, and reports it again', async () => {
+        // RFC 4918 §9.10.1 puts the activelock in the LOCK body, and the
+        // Microsoft WebDAV redirector needs it. Without it Windows cannot
+        // WRITE to the drive at all: it PUTs an empty file, LOCKs it,
+        // PROPFINDs it, finds no lock, and loops until it gives up with
+        // ERROR_INVALID_PARAMETER — so the share lists and reads perfectly
+        // while every save fails with "The parameter is incorrect" and leaves
+        // a zero-byte file behind.
+        //
+        // `emulate` used to answer an empty <lockdiscovery/>, because it
+        // stored no lock at all — the token in the header was the only trace
+        // the lock had ever existed. It keeps them in memory now.
+        //
+        // Asserted for EVERY mode that claims class 2 in OPTIONS, because
+        // claiming it is the promise this checks. A status-only assertion is
+        // what let the gap ship.
+        for (const ep of ['emulated', 'meta']) {
+            const res = await lock(ep, `activelock-${ep}.md`)
+            const body = await res.text()
+            const token = res.headers.get('lock-token')
+
+            assert.match(body, /<activelock>/, `${ep}: no activelock in the LOCK body\n${body}`)
+            assert.match(body, /<locktoken>/, `${ep}: no locktoken\n${body}`)
+            // The token in the body must BE the token in the header. A body
+            // carrying a different one is worse than an empty body.
+            const inBody = body.match(/<locktoken>\s*<href>([^<]+)<\/href>/)?.[1]
+            assert.equal(`<${inBody}>`, token, `${ep}: body token and header disagree\n${body}`)
+
+            // And it has to be discoverable afterwards, which is the half a
+            // client uses to find out whether its own lock still holds.
+            const found = await fetch(`http://127.0.0.1:${port}/drive/${ep}/activelock-${ep}.md`, {
+                method: 'PROPFIND',
+                headers: { authorization: AUTH, 'content-type': 'text/xml', depth: '0' },
+                body: '<?xml version="1.0"?><propfind xmlns="DAV:"><prop><lockdiscovery/></prop></propfind>',
+            })
+            const discovered = await found.text()
+            assert.match(discovered, /<activelock>/,
+                `${ep}: PROPFIND does not report the lock\n${discovered}`)
+            assert.ok(discovered.includes(inBody),
+                `${ep}: PROPFIND reports a different token\n${discovered}`)
+        }
     })
 })
 
 describe('a persisted lock is honoured', () => {
+    it("'emulate' refuses a second LOCK too, which is what makes it real", async () => {
+        // The other half of storing locks. A mode that hands out a token to
+        // anyone who asks is a polite fiction; refusing the second caller is
+        // what a client is relying on when it takes one.
+        const first = await lock('emulated', 'contended-emulated.md')
+        assert.ok([200, 201].includes(first.status), `first lock: ${first.status}`)
+        const second = await lock('emulated', 'contended-emulated.md')
+        assert.equal(second.status, 423, `a held resource must answer 423 Locked, got ${second.status}`)
+    })
+
     it('a second LOCK on an already-locked resource is refused', async () => {
         // Only meaningful where locks are persisted. Worth asserting because
         // it is what makes 'meta-files' a real locking implementation rather

package/test/root-listing.test.js

@@ -113,6 +113,32 @@ after(async () => {
     await rm(dir, { recursive: true, force: true })
 })
 
+describe('host in the wrong place', () => {
+    // It was per-endpoint while this was being designed, and moved to the
+    // plugin when the root listing arrived — several endpoints share one
+    // domain, and `/` on it lists them. This file's own JSDoc went on showing
+    // the old shape, which is a config that reads as though it works and does
+    // nothing at all. So it says so.
+    it('warns instead of silently ignoring it', async () => {
+        const said = []
+        const core = {
+            runtime,
+            onLoad:    () => {},
+            onLoaded:  (cb) => said.push(cb),
+            useLogger: () => ({
+                info(){}, error(){}, debug(){}, trace(){},
+                warn: (...args) => said.push(args.join(' ')),
+            }),
+        }
+        drive({ endpoints: { Share: { folder: 'documents', host: 'drive.example.test' } } })(core)
+        for (const entry of [...said]) if (typeof entry === 'function') await entry()
+
+        const warning = said.find(entry =>
+            typeof entry === 'string' && entry.includes('plugin-level option'))
+        assert.ok(warning, `no warning about a misplaced host:\n${said.filter(e => typeof e === 'string').join('\n')}`)
+    })
+})
+
 describe('the drive root on its own host', () => {
     it('announces itself as WebDAV, which is what the redirector asks first', async () => {
         // Answered without credentials on purpose: this is discovery, and it