@mieweb/pulsevault 0.4.2 → 0.6.0

package/CHANGELOG.md

@@ -7,6 +7,219 @@ breaking changes, called out explicitly below.
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-10-12
+
+Protocol 2.5 (`PROTOCOL.md` §7.4): 2.4 adds the https pairing link form (#83), 2.5 the conform
+step's `webReady` result. `webReady` becomes a **conform** step: whatever video a
+person uploads, PulseVault serves it in the one format the Pulse app records, and a Pulse upload
+is left untouched (#84).
+
+### Added
+
+- **https pairing links** (#83): `buildUploadLink({ …, form: 'https' })`
+  returns `https://mieweb.github.io/pulse/open#v=1&artifactId=…&server=…&token=…`,
+  the same parameters as the `pulsecam://` form, in the fragment. It is a
+  Universal Link / App Link: a phone camera opens it, Pulse opens straight
+  from it when installed, and the page sends the person to the store when it
+  isn't. Browsers never send the fragment, so the link host never receives the
+  token. `PULSE_OPEN_URL` is exported. Without `form` the output is
+  unchanged, `pulsecam://`, because Pulse releases from before the link host
+  was claimed open the https form only through the page's button.
+- **Conform target** (`CONFORM_TARGET`): faststart MP4, H.264 8-bit `yuv420p` (limited range)
+  with even dimensions, SDR (HDR tone-mapped to BT.709; SDR colour tags kept), the longest edge at most `webReady.maxEdge` (default 1920; aspect ratio and
+  orientation kept), rotation applied, AAC or no audio. `ensureWebReady` leaves a file already
+  in it byte-for-byte (`none`), remuxes losslessly when only the container or `moov` position is
+  off, and otherwise runs ffmpeg once, re-encoding only the off-target streams: HEVC and other
+  codecs to H.264, 10-bit HDR (PQ/HLG) tone-mapped to SDR (FFmpeg 8's `scale`, or `zscale` on
+  older builds), full-range video (iPhone screen recordings) to limited range, oversized video
+  scaled down, Opus/PCM and other audio to AAC.
+- **Container change**: a `.mov`, `.m4v`, `.webm`, `.mkv`, `.3gp` or `.avi` becomes a new
+  `<id>.mp4` (`WebReadyAction` `conformed`). The artifact's stored file, `ext` and served
+  `Content-Type` follow it; the artifact id and its URLs don't change. The sidecar switches
+  first, then the original is deleted (`sourceExt` remembers it so `remove` cleans up after a
+  crash). New patch fields `ext` and `webReady` (local adapter; the S3 adapter refuses them).
+  `describeArtifact` reports the uploaded extension as `sourceExt` once it differs, and
+  `uploadIdOf(meta)` names tus's upload by it, so a replayed hook's `ext` and `uploadId` still
+  match `filename`.
+- `webReady.timeoutSeconds` (default `60 + 10 ×` the duration in 1080p30 seconds, so 4K or high
+  frame rates get proportionally longer, at most an hour since it comes from the upload's own
+  metadata; set it explicitly for longer runs): a longer run is killed, the original
+  kept, and `skipped` recorded with the reason.
+- The conversion's result is recorded on the artifact: `GET /artifacts/:id/status` and
+  `getStatus` report `webReady: { action, reason }` (protocol 2.5), and `describeArtifact`
+  returns it.
+- `core.conformAvailable()` (also on `fastify.pulseVaultCore`): `false` when `webReady` is off or
+  ffmpeg/ffprobe are missing, for a host's health check.
+- `createVideoValidator({ maxDurationSeconds? })`: a `validatePayload` hook that checks a video
+  by its content with ffprobe (a video stream with a duration above zero, including a browser
+  WebM without one in its header; a picture — PNG, JPEG, GIF, HEIC, a one-frame clip, counted
+  from the first two packets when the container has no frame count — is not a video), refusing with `422 That file isn't a video.` or `That video is
+  longer than the limit of …` (the configured limit exactly: "1 minute 1 second"); falls back to `sniffVideo` (`ftyp`, EBML or RIFF AVI) without
+  ffprobe. Also exported: `probeVideo`, `webReadyAvailable`, `CONFORM_VIDEO_EXTENSIONS`.
+- An upload over `maxUploadSize` is refused with `413 That file is larger than 500 MB.` instead
+  of tus's "Maximum size exceeded".
+- `webReady.probeTimeoutSeconds` and `createVideoValidator({ probeTimeoutSeconds })` (default 60):
+  every ffprobe run over an upload is bounded; the validator refuses with `422 That video
+  couldn't be checked in time.`
+- Content types for `.webm`, `.mkv`, `.3gp` and `.avi`.
+- While a video is being converted, `GET /artifacts/:id` is served with `max-age=0` instead of
+  the configured `cache` (which may be `immutable`), so no client keeps the original bytes once
+  the converted file replaces them.
+
+### Changed
+
+- The default video `allowedExtensions` are `.mp4`, `.mov`, `.m4v`, `.webm`, `.mkv`, `.3gp` and
+  `.avi` (was `.mp4`). A host's own `allowedExtensions` still overrides them.
+- A failed conversion is now recorded (`webReady: { action: "skipped", reason }`) as well as
+  logged, and still not retried; a conversion that never recorded a result is resumed by the
+  replay, as before.
+- The local adapter's `remove` reads the sidecar itself (not its cache) and deletes the
+  converted `<id>.mp4` and the uploaded file as well as the stored one. A converted video's bytes
+  are always at `<id>.mp4`, and `resolve`/`getLocalPath` fall back to it when the sidecar's `ext`
+  file is gone: across instances sharing a workspace, where the last sidecar write wins (as
+  before; there is no cross-instance lock), a write from a copy older than a conversion still
+  serves the converted file.
+- A client that re-sends the final `PATCH` of a finished upload whose file a conversion replaced
+  (a `.webm` conformed to `<id>.mp4`) gets `204`, as without a conversion: the local adapter's tus
+  store reads such an upload as complete and takes its empty tail without a write.
+- A `webReady` conversion needs the adapter's `generation`: an adapter with `getLocalPath` but no
+  generation has its videos recorded `skipped` (`the storage adapter keeps no generation for this
+  upload`). The local adapter gives one at `markReady` to an upload reserved before this release.
+- Reservations run under the in-process sidecar lock, and `remove` deletes the sidecar last, so a new upload
+  of a removed id never has its bytes deleted by the old removal. Each reservation gets a
+  `generation` (`describeArtifact`), new every time the id is reserved.
+- A conversion is installed in one patch under the sidecar lock: the runner converts into a temporary
+  `.mp4` (`ensureWebReady(…, { install: false })`), then `patchArtifact` with `file` renames it
+  onto the stored file, switches `ext`, records the result and deletes the original — only for
+  the reservation the conversion began on (`generation`) and only while no other pass has
+  recorded one (`unlessConverted`). When a condition doesn't hold, `patchArtifact` resolves
+  `false`, changing nothing, and the runner drops its file.
+- Completion work belongs to one reservation: a queued conversion checks, when its turn comes,
+  that the id still names the upload it was queued for (and that it's finished); the
+  acknowledgement is recorded only on that reservation; and a new upload of a removed id is never
+  joined to the old one's work in progress.
+- `GET /artifacts/:id` opens the file before sending headers, checks it's the file `send`
+  described (inode, size, mtime), and resolves once more when it's gone or another file was
+  renamed onto the path (a conversion just replaced it), instead of a 404, a cut-off response or
+  headers that don't match the bytes; bytes this
+  instance resolved under an extension storage no longer names are served with `max-age=0`.
+- A failed run's recorded reason is `ffmpeg exited with code N`, never ffmpeg's output (which
+  names server paths); an HDR video on an ffmpeg without tone mapping is left as uploaded
+  (`skipped`, saying what's missing) rather than converted with wrong colours.
+- The local adapter's `remove` decides from the sidecar alone, never its cache: an artifact
+  another reader had cached is reported removed once, not twice.
+- `webReady` needs an adapter with `describeArtifact` too (both built-in adapters have it).
+- The local adapter's `getLocalPath` reads the sidecar, not its cache, so it names the current
+  file after another instance removed the id and reserved it again.
+- ffmpeg's capability queries (`-version`, `-h filter=scale`, `-filters`) are bounded (30 s).
+- An `onArtifactEvent` observer that throws on `processed` is logged as its own failure; the
+  recorded result still reaches `completeAfter`'s hook as `ctx.webReady`.
+- A replayed `onUploadComplete` (the hook threw, or the process stopped) gets the recorded
+  conversion result as `ctx.webReady` too, so a host naming the file from it (an `.mp4` once a
+  WebM was conformed) gets it right on the retry.
+- `web-ready-migrate.mjs` rerun after an interruption deletes an original the sidecar no longer
+  names.
+- `scripts/web-ready-migrate.mjs` conforms existing artifacts the same way, recording the result
+  on each sidecar and switching a changed container to its `.mp4`. Run it with the server
+  stopped.
+- `transcode: false` records a file it leaves off-target (HEVC, oversized, non-AAC audio) as
+  `skipped`, not `none`: `none` now means only that the file already conforms. (Was `none`.)
+- A browser WebM's missing duration is found by streaming ffprobe's packet list and keeping
+  only the latest end, never buffering it.
+- An upload of deferred length that grows past `maxUploadSize` (a chunk with a length, or a
+  chunked body cut off mid-stream) gets the same `413 That file is larger than …` message.
+- An MP4-family `.mov`/`.m4v` that already conformed used to be served as is (`video/quicktime`,
+  a type not every browser takes); it is now remuxed into an `.mp4`.
+
+## [0.5.0] - 2026-10-05
+
+Protocol 2.3 (`PROTOCOL.md` §7.4). What every host rebuilt around PulseVault
+(#80), in the core: the shape of a pulse, a completion that survives a throw
+or a restart, a lock on finished pulses, the token's own context, a status
+route, background web-ready, a poster route and `getPulse`, and the content
+types phones send. Two defaults change; both have an escape hatch.
+
+### Changed (breaking)
+
+- **A create must have the shape of a pulse** (`pulseShape`, on by default):
+  a video with no `relatedTo`; a thumbnail, beat manifest or captions under
+  its own id, `relatedTo` the video. With `createCapabilityAuthorize` the ids
+  are also tied to the token: the video only under the token's own
+  `artifactId`, the rest only `relatedTo` it. Before, one token could create
+  any number of artifacts under ids the client picked, or take the video's
+  own id with a thumbnail so the real video answered `409` for good. The
+  Pulse app has always uploaded in this shape. For uploads that aren't
+  pulses, pass `pulseShape: false` to the plugin or core *and* to
+  `createCapabilityAuthorize`.
+- **A create reclaims an idle unfinished upload of the same artifactId**
+  (`reclaim`, `{ idleSeconds: 300 }` by default): an app killed mid-upload
+  sends no TUS `DELETE`, so re-scanning the same link answered `409` until
+  `retention` ran. Only an upload of the same kind and `relatedTo` is taken
+  over (the same token authorized both), and only once the local adapter has
+  seen no bytes for `idleSeconds`. `reclaim: false` keeps the old `409`.
+- **`onUploadComplete` is at-least-once.** PulseVault records that the hook
+  returned (`acknowledged` in the sidecar) and fires it again, with
+  `ctx.replay: true`, for a finished artifact it never recorded: the hook
+  threw, or the process stopped first. Every 300 seconds, with the first pass
+  shortly after start (`replayCompletions`; `false` turns it off, and
+  `core.replayCompletions()` / `fastify.pulseVaultCore.replayCompletions()`
+  runs a pass on demand). Hosts must check their own record before writing
+  it again. Finished sidecars written before this release read as acknowledged, so an
+  upgrade replays nothing; one still uploading at the upgrade completes
+  normally. A retried final `PATCH` no longer fires the hook a
+  second time.
+- **`authorize` gains the `status` phase** (`GET /artifacts/:id/status`).
+  An exhaustive `switch` on the phase needs a case for it.
+- **Content types**: the `.pulse` beat manifest is served as
+  `application/json`, and `.mov`, `.m4v` and `.srt` — when a host allows them
+  in `allowedExtensions`; the defaults stay what the Pulse app sends — as
+  `video/quicktime`, `video/x-m4v` and `application/x-subrip`, instead of
+  `application/octet-stream`, which `<video>` downloads instead of playing.
+
+### Added
+
+- **Hook context**: `authorize` and `onUploadComplete` receive `relatedTo`,
+  `name`, `appVersion`, `filename`, `ext` and the token's `context`, so a
+  host maps a finished file to its own record without reading storage back.
+- **`context` in the capability token** (`issueCapabilityToken(id, secret,
+  { context })`, the `ctx` claim, at most 1 KiB of JSON): an owner, a
+  destination — signed, stored with the artifact at create (`authorize` may
+  return `{ context }`), and handed to every hook, `getStatus` and `getPulse`.
+  Most hosts need no table keyed by artifactId any more.
+- **`GET /artifacts/:id/status`** and `core.getStatus(id)`: `unknown`,
+  `uploading` (with `bytesReceived` against `size`), `processing` or
+  `ready`, plus `acknowledged` and whatever the host recorded with
+  `recordOutcome(id, outcome)`. Authorized as `status`;
+  `createCapabilityAuthorize` grants it to the pairing token and a view token.
+- **`webReady`**: the conversion `ensureWebReady` does, run by the core in
+  the background after the final `PATCH` is answered, `concurrency` at a
+  time, with a `processing` status meanwhile, a `processed` event when done,
+  and `completeAfter: true` to hold `onUploadComplete` until the conversion
+  has finished (`ctx.webReady` says what it did). Whether it finished is
+  recorded on the sidecar (`converted`), so a conversion a restart
+  interrupted is redone by the replay. Local storage only.
+- **`lockWhenReady`**: a finished artifact, and anything `relatedTo` a
+  finished video, can't be deleted through the routes (`403`); the host
+  removes through `storage.remove` on its own terms.
+- **`core.getPulse(videoId)`** → `{ video, thumbnail, captions, manifest }`
+  and **`GET /artifacts/:videoId/poster`**, both from a relation index the
+  adapters keep at reserve time (`storage.listRelated`), so neither scans
+  storage and nothing is backfilled.
+- **Storage adapter contract**: optional `describeArtifact`, `patchArtifact`
+  and `listRelated`, and `context` in `reserveUpload` (both built-in
+  adapters implement them). The local adapter's relation index lives at
+  `.pulsevault/related/<videoId>/<artifactId>`; the S3 adapter's under the
+  same key prefix.
+- **`onArtifactEvent`**: `phase: "processed"` (with `webReady`), and
+  `reason: "reclaimed"` on `remove`.
+- **Fastify**: `fastify.pulseVaultCore` (`coreDecoratorName`) exposes
+  `getStatus`, `getPulse`, `recordOutcome` and `replayCompletions`.
+- **`examples/meteor-demo`** is now the reference host integration, shaped
+  like a team app: people, destinations, a reserve that signs the owner and
+  destination into the token, delivery from `onUploadComplete`, status
+  polling, and video cards with posters — with no table of uploads, no shape
+  check, no stale-upload cleanup and no content-type fix-up of its own.
+
 ## [0.4.2] - 2026-10-02
 
 ### Added
@@ -411,7 +624,9 @@ if you've evaluated against an intermediate build.
   lock, so this is now at least surfaced: a one-time `console.warn` fires
   per process the first time a deployment falls into this degraded mode.
 
-[Unreleased]: https://github.com/mieweb/pulsevault/compare/v0.4.2...HEAD
+[Unreleased]: https://github.com/mieweb/pulsevault/compare/v0.6.0...HEAD
+[0.6.0]: https://github.com/mieweb/pulsevault/compare/v0.5.0...v0.6.0
+[0.5.0]: https://github.com/mieweb/pulsevault/compare/v0.4.2...v0.5.0
 [0.4.2]: https://github.com/mieweb/pulsevault/compare/v0.4.1...v0.4.2
 [0.4.1]: https://github.com/mieweb/pulsevault/compare/v0.4.0...v0.4.1
 [0.4.0]: https://github.com/mieweb/pulsevault/compare/v0.3.0...v0.4.0

package/OPERATIONS.md

@@ -54,7 +54,15 @@ Two ways to actually support multiple instances:
 2. **Shared filesystem**: point every instance's `workspaceDir` at the same
    NFS/EFS mount. Verify your mount's durability/consistency guarantees
    under concurrent writes from multiple instances before relying on this in
-   production.
+   production. Each instance changes an artifact's metadata (`.pulsevault/
+   <id>.json`) one write at a time, but no lock spans instances: when two
+   write the same artifact's metadata at once, the last write wins. The
+   completion bookkeeping recovers: the replay redoes a lost acknowledgement
+   or conversion record, and a converted video's bytes are always at
+   `<id>.mp4`, which is served even when the metadata that won still names
+   the uploaded file. A host's `recordOutcome` lost that way is not redone. A
+   removal racing another instance's write to the same artifact can leave its
+   metadata behind; for strict guarantees across instances, use S3.
 
 **The S3/R2 adapter (`createS3Storage`) has no such requirement** — every
 instance talks to the same bucket, so it scales horizontally with zero
@@ -160,52 +168,124 @@ backup time will resume correctly via the normal TUS `HEAD`-then-resume path
 once the client retries, or will sit as abandoned partial uploads until
 `retention` removes them (see "Retention" below) if the client never retries.
 
-## Web-ready playback (faststart remux + H.264 transcode)
-
-Browsers streaming an MP4 over progressive HTTP stall (or fail outright) on
-two things mobile capture pipelines routinely upload:
-
-- **moov atom at the end of the file** — the player must fetch the file's tail
-  before rendering frame one: seconds of startup delay behind Range requests,
-  or a full download in naive players.
-- **HEVC video** — Firefox never decodes it; Chrome usually can't without
-  hardware support.
-
-The `ensureWebReady` helper (exported from the package root) fixes both, in
-place and atomically (tmp file + rename): a lossless sub-second
-`-c copy -movflags +faststart` remux when only the moov position is wrong, and
-a one-time `libx264` transcode (audio stream-copied) when the codec is
-hostile. It is fail-open by design — without `ffmpeg`/`ffprobe` on `PATH` it
-logs one warning and the original bytes keep serving exactly as before.
-
-**Prerequisite:** install ffmpeg on the serving host — `apt install ffmpeg`
-(Debian/Ubuntu), `dnf install ffmpeg` (Fedora/EL + RPM Fusion), or
-`brew install ffmpeg` (macOS). Nothing else changes; the hook detects it at
-first use.
-
-**New uploads** — wire it into `onUploadComplete` (the fastify-demo ships with
-this enabled):
+## Web-ready playback (conform to one format)
+
+Videos reach PulseVault from the Pulse app, which already uploads the format
+below, and from a host's own file picker: a screen recording, an iPhone HEVC
+`.mov` (often 10-bit HDR), a WebM from a browser recorder, a 4K clip. Browsers
+streaming over progressive HTTP stall on a `moov` atom at the end of the file,
+Firefox never decodes HEVC, Chrome usually can't without hardware support,
+10-bit HDR shown as SDR looks washed out, and a 4K file is far more than a
+phone needs.
+
+With the `webReady` option on, every finished video is **conformed** to one
+target (`CONFORM_TARGET`, exported from the package root):
+
+| Property | Target |
+|---|---|
+| Container | MP4, `moov` at the front (faststart) |
+| Video | H.264, 8-bit `yuv420p` (limited range), even width and height |
+| Colour | SDR: HDR (PQ or HLG) is tone-mapped to BT.709; an SDR source keeps its own colour tags (BT.709 or BT.601), which browsers honour |
+| Size | longest edge at most 1920 (`maxEdge`); aspect ratio and orientation kept — no crop, pad or stretch |
+| Rotation | applied to the pixels, so every player shows it the way it was held |
+| Audio | AAC; a video with no audio stays silent |
+
+`ensureWebReady` does the least work that gets a file there:
+
+| The upload | What happens | `action` |
+|---|---|---|
+| Already in the target (a Pulse upload) | nothing; bytes untouched | `none` |
+| MP4 with the `moov` at the end | lossless `-c copy` faststart remux | `remuxed` |
+| MP4 with an off-target stream | one ffmpeg run, in place | `transcoded` |
+| Another container (`.mov`, `.m4v`, `.webm`, `.mkv`, `.3gp`, `.avi`) | one ffmpeg run into a new `<id>.mp4`: a stream copy when the streams already conform, otherwise only the off-target streams re-encoded | `conformed` |
+| No ffmpeg, unreadable, failed or timed out | nothing; the original serves | `skipped` |
+
+A re-encode scales by the longest edge, applies the rotation, tone-maps HDR
+(PQ or HLG) to SDR, and converts non-AAC audio (Opus, PCM, …) to AAC. After a
+container change the artifact's stored file, extension and served
+`Content-Type` (`video/mp4`) follow the new file; the artifact id, and so every
+URL a host stored, stay the same. The new file is written beside the original,
+the artifact's sidecar switches to it, and only then is the original deleted:
+a crash at any point leaves either the original serving (the conversion is
+redone by the replay) or an unused original that is deleted with the
+artifact.
+
+While a video is being converted, its URL serves the original bytes with
+`max-age=0` instead of the configured `cache`, so a browser that fetched it
+early revalidates and gets the converted file (with a new ETag) afterwards.
+
+A failed run is recorded as `conversion failed: ffmpeg exited with code N`;
+ffmpeg's own output, which names server paths, goes only to the server log.
+
+The result is recorded on the artifact: the status route reports it as
+`webReady: { action, reason }` (a host can say "Your video is ready", or "We
+couldn't convert this video, so it may not play in every browser"),
+`onArtifactEvent` fires `processed` with it, and `completeAfter` hands it to
+`onUploadComplete` as `ctx.webReady`.
+
+**Prerequisite:** install ffmpeg (which includes ffprobe) on the serving host
+— `apt install ffmpeg` (Debian/Ubuntu), `dnf install ffmpeg` (Fedora/EL + RPM
+Fusion), or `brew install ffmpeg` (macOS). It is detected at first use; without
+it every upload is still accepted and served exactly as uploaded, one warning
+is logged, and `core.conformAvailable()` (also `fastify.pulseVaultCore`)
+resolves `false` for a health check. HDR tone mapping uses the `scale` filter
+on FFmpeg 8 and later, or `zscale` (libzimg, included in the Debian and Ubuntu
+packages) on older builds. A build with neither leaves an HDR video as
+uploaded and records `skipped` with what's missing: converting it without tone
+mapping would give wrong colours.
 
 ```js
-const storage = createLocalStorage({ workspaceDir: dataDir });
 await app.register(pulseVault, {
-  storage,
-  onUploadComplete: async (_request, { artifactId, kind }) => {
-    if (kind !== "video") return;
-    const localPath = await storage.getLocalPath(artifactId);
-    if (localPath) await ensureWebReady(localPath, { logger: app.log });
-  },
+  storage: createLocalStorage({ workspaceDir: dataDir }),
+  validatePayload: createVideoValidator({ maxDurationSeconds: 600 }),
+  webReady: { concurrency: 1, completeAfter: true },
 });
 ```
 
-Options: `{ transcode: false }` restricts it to the lossless remux (no CPU
-cost beyond a file rewrite); `crf`/`preset` tune the transcode
-(defaults `23`/`veryfast`); `ffmpegPath`/`ffprobePath` point at binaries off
-`PATH`.
-
-**Existing artifacts** — uploads that landed before the hook existed are fixed
-once with the bundled migration script (idempotent; interrupt and rerun
-freely, already-fixed files are skipped for free):
+Settings:
+
+- `concurrency` (default `1`): conversions run one at a time per process; the
+  rest wait in a queue. A conversion a restart interrupted is resumed by the
+  completion replay (`replayCompletions`): an artifact isn't marked converted
+  until its result is recorded.
+- `timeoutSeconds` (default `60 + 10 ×` the video's duration counted in
+  1080p30 seconds — a second of 4K at 120 fps counts 16 — at most an hour,
+  and an hour when the duration is unknown; the default comes from the
+  upload's own metadata, so set this for longer runs): a run past it is
+  killed, the original kept, and
+  `skipped` recorded with the reason.
+- `probeTimeoutSeconds` (default `60`): the same guard for each ffprobe run
+  over the upload, so a malformed file can't hold the queue.
+- `maxEdge` (default `1920`): the longest edge of the served video.
+- `transcode: false`: never re-encode; only lossless remuxes run (a `moov`
+  moved to the front, or another container whose streams already conform copied
+  into an `.mp4`), so no CPU cost beyond a file rewrite. A file that would need
+  a re-encode is left as uploaded and recorded as `skipped` (`none` means only
+  "already conforms").
+- `crf`/`preset` (defaults `23`/`veryfast`) tune the H.264 encode;
+  `ffmpegPath`/`ffprobePath` point at binaries off `PATH`.
+
+**Accepted containers.** The default video `allowedExtensions` are `.mp4`,
+`.mov`, `.m4v`, `.webm`, `.mkv`, `.3gp` and `.avi` (`CONFORM_VIDEO_EXTENSIONS`);
+a host's own `allowedExtensions` still overrides them. The extension only
+decides what's accepted at `create`; `createVideoValidator()` checks the
+received bytes by what they are: ffprobe must find a video stream with a
+duration above zero that isn't a picture (a PNG, JPEG, GIF, HEIC photo or
+one-frame clip is refused), and, with `maxDurationSeconds`, not longer, or the
+upload is refused with `422 That file isn't a video.` / `That video is longer
+than the limit of 10 minutes.` Each ffprobe run is bounded by
+`probeTimeoutSeconds` (default `60`), so a malformed file can't hold the final
+`PATCH` open: past it the upload is refused with `422 That video couldn't be
+checked in time.` Without ffprobe it falls back to sniffing the
+container's first bytes (`ftyp`, EBML or RIFF AVI). An upload over
+`maxUploadSize` is refused at `create` with `413 That file is larger than
+500 MB.` Without `webReady`, a WebM or MKV is served as uploaded, with its own
+content type.
+
+**Existing artifacts** — uploads that landed before `webReady` was on are
+conformed once with the bundled migration script. Stop the server while it
+runs (the server caches each artifact's extension, which a container change
+rewrites). It is idempotent; interrupt and rerun freely:
 
 ```sh
 node node_modules/@mieweb/pulsevault/scripts/web-ready-migrate.mjs /path/to/workspaceDir
@@ -215,11 +295,12 @@ node node_modules/@mieweb/pulsevault/scripts/web-ready-migrate.mjs /path/to/work
 node node_modules/@mieweb/pulsevault/scripts/web-ready-migrate.mjs /path/to/workspaceDir --no-transcode
 ```
 
-A transcode re-encodes the video stream (one-time, quality-preserving at the
-default CRF but not bit-identical). Take a backup first if that matters to
+A re-encode is one-time, quality-preserving at the default CRF but not
+bit-identical, and a conformed file replaces the upload (the original is not
+kept). Take a backup first if that matters to
 your deployment (see "Backup and restore" above).
 
-Note on checksums: a remux or transcode changes the artifact's bytes, so an
+Note on checksums: a conversion changes the artifact's bytes, so an
 upload-time checksum recorded for it (the sidecar `checksum` from
 `Upload-Metadata`) describes the original upload, not the rewritten file.
 Treat `getChecksum` as upload-time provenance rather than current-file
@@ -231,6 +312,16 @@ integrity for artifacts this feature has touched.
 captions, manifest or thumbnail of a video that never finished — are cleaned
 up by the opt-in `retention` option (or `sweepAbandonedUploads` from your own
 scheduler); see the README's `retention` option. Both adapters support it.
+Separately, a create that arrives for an unfinished upload of the same id
+(the person scanned the same link again after the app died) takes it over
+once it has been idle for `reclaim.idleSeconds` (300 by default), so nobody
+waits for the sweep.
+
+**Completions the host never recorded** — an `onUploadComplete` that threw,
+or a restart between the final byte and the hook — are fired again by the
+completion replay (`replayCompletions`, every 300 seconds by default, and
+`pulseVaultCore.replayCompletions()` on demand). Nothing to schedule; make the
+hook idempotent.
 
 How long to keep **finished** content is a policy decision `pulsevault`
 leaves to the operator (see `PROTOCOL.md` §1 on lifecycle ownership). A

package/PROTOCOL.md

@@ -3,7 +3,7 @@
 <!-- BEGIN GENERATED: protocol-version (from package.json pulseProtocol — edit that, then run `npm run protocol`) -->
 | | |
 |---|---|
-| Spec revision | `2.2` |
+| Spec revision | `2.5` |
 | Protocol majors accepted | 2 |
 <!-- END GENERATED: protocol-version -->
 
@@ -69,12 +69,30 @@ the integer from `protocolVersion`.
 
 ## 3. Pairing (deep link)
 
-A server presents a pairing link or QR code of the form:
+A server presents a pairing link or QR code in one of two equivalent forms
+(since 2.4), carrying the same parameters:
 
 ```
 pulsecam://?v=1&artifactId=<uuid>&server=<origin>&token=<opaque>
+https://mieweb.github.io/pulse/open#v=1&artifactId=<uuid>&server=<origin>&token=<opaque>
 ```
 
+- **`pulsecam://`**: the parameters are the query string. Every Pulse release
+  opens it, but phone cameras and browsers don't open custom schemes reliably.
+- **`https://`**: a Universal Link (iOS) / App Link (Android) on the link host
+  the Pulse app claims. When Pulse is installed the OS opens it straight from
+  the link; when it isn't, the link opens a static page that sends the person
+  to the store and offers a `pulsecam://` fallback. The parameters are the
+  **fragment**, which is never sent in an HTTP request, so the link host never
+  receives the token. The link host is fixed (an app can only claim domains it
+  was built with); `server` still points at the deployment's own server, so
+  uploads never touch the link host.
+
+A client that accepts the `https://` form MUST treat it exactly like the
+`pulsecam://` form with the same parameters. A server that wants every Pulse
+release to open its links directly keeps the `pulsecam://` form until the
+releases that claim the link host are installed.
+
 - `v` (REQUIRED): the deep-link schema version. A client MUST refuse and
   explain (not silently misparse) a `v` it doesn't recognize.
 - `artifactId` (REQUIRED): a UUID minted by the server. The client uses this
@@ -95,7 +113,7 @@ pulsecam://?v=1&artifactId=<uuid>&server=<origin>&token=<opaque>
   credential — see §5.4 for the recommended (but not required) capability-
   token shape.
 
-A client SHOULD ignore query params it does not recognize, so links minted by
+A client SHOULD ignore parameters it does not recognize, so links minted by
 an older or newer server stay valid. The parameters are defined by
 [`protocol/schemas/deep-link.schema.json`](protocol/schemas/deep-link.schema.json).
 
@@ -246,6 +264,18 @@ the token's `artifactId` as its `relatedTo` (§8) — this lets one token cover
 an entire upload session (a pulse's video plus its captions, beat manifest and
 thumbnail) rather than requiring one token per artifact.
 
+A token MAY also carry an opaque `ctx` claim (protocol 2.3): host data signed
+with the rest — who the upload is for, where it goes — that the server stores
+with the artifact the token creates and hands back to its own hooks, so the
+host needs no table keyed by artifactId. A client never reads it.
+
+When a token names a pulse's video, a server SHOULD hold the token to the
+shape of a pulse (§8): the video only under the token's own `artifactId`, with
+no `relatedTo`, and a thumbnail, beat manifest or captions only under another
+id, `relatedTo` the token's. Otherwise one token can create any number of
+artifacts under ids of the client's choosing, or take the video's own id with a
+thumbnail so the video itself can never land.
+
 This shape is exactly what `@mieweb/pulsevault`'s `issueCapabilityToken`/
 `verifyCapabilityToken`/`createCapabilityAuthorize` implement, but any server
 is free to use its own scheme entirely — only §5.1–§5.3 are normative.
@@ -312,6 +342,31 @@ right after the upload finishes, while its pairing token is still valid; a
 server without `viewLinks` offers no shareable link, and the client MUST NOT
 share the pairing token in its place.
 
+### 6.5 Status and poster (optional, protocol 2.3)
+
+A server MAY expose `GET {prefix}/artifacts/<artifactId>/status`, authorized
+like opening the artifact (§5.1; a view token, §5.5, is enough), reporting
+where an upload is so a page waiting on it needs nothing from the host. Its
+body is defined by
+[`protocol/schemas/artifact-status.schema.json`](protocol/schemas/artifact-status.schema.json):
+`state` is `unknown`, `uploading`, `processing` (the server is converting the
+bytes for the web) or `ready`, with the bytes received against the declared
+length while uploading, and whatever outcome the host recorded once finished.
+A server that converts videos for the web also reports what the conversion did
+(`webReady`, protocol 2.5): nothing, a remux, a transcode, a new container, or
+why it was skipped (the original then serves). The response MUST NOT be cached.
+
+A server MAY also expose `GET {prefix}/artifacts/<videoId>/poster`: the finished
+thumbnail `relatedTo` the video (§8), served exactly as §6.2 serves an artifact
+and authorized as opening the video, so whoever may watch it may see its poster.
+It returns `404` until the thumbnail has landed.
+
+An unfinished upload whose client died (an app killed mid-upload sends no
+TUS `DELETE`) would otherwise answer `409` to every later create of the same
+`artifactId`. A server SHOULD let a create authorized by the same token take
+over such an upload once it has been idle for a while, instead of refusing it
+until cleanup runs.
+
 ## 7. Versioning
 
 The protocol has a version `major.minor`, written once in `@mieweb/pulsevault`'s
@@ -371,10 +426,11 @@ a server it paired with earlier: a server can be upgraded in between.
 | File | Defines |
 |---|---|
 | [`openapi.json`](protocol/openapi.json) | Every HTTP route, generated from the plugin |
+| [`artifact-status.schema.json`](protocol/schemas/artifact-status.schema.json) | GET /artifacts/<id>/status response |
 | [`beat-manifest.schema.json`](protocol/schemas/beat-manifest.schema.json) | Beat manifest |
 | [`capabilities.schema.json`](protocol/schemas/capabilities.schema.json) | GET /capabilities response |
 | [`capability-token.schema.json`](protocol/schemas/capability-token.schema.json) | Capability token claims |
-| [`deep-link.schema.json`](protocol/schemas/deep-link.schema.json) | Pairing link query parameters |
+| [`deep-link.schema.json`](protocol/schemas/deep-link.schema.json) | Pairing link parameters |
 | [`pulse-client.schema.json`](protocol/schemas/pulse-client.schema.json) | Pulse-Client request header |
 | [`upload-metadata.schema.json`](protocol/schemas/upload-metadata.schema.json) | TUS Upload-Metadata keys |
 | [`view-link.schema.json`](protocol/schemas/view-link.schema.json) | POST /artifacts/<id>/view-link response |
@@ -388,6 +444,9 @@ a server it paired with earlier: a server can be upgraded in between.
 | 2.0 | **Breaking:** `uploadUnit` removed from `/capabilities` and pairing links — a pulse always uploads as one video (§8). Clients built for 1.0 required it. |
 | 2.1 | Added `protocolRevision` to `/capabilities`, the `Pulse-Client` header with `426 Upgrade Required` (§7.2), and `Upload-Metadata.appVersion` (§4.1). |
 | 2.2 | Added read-only view links: `viewLinks` in `/capabilities`, `POST {prefix}/artifacts/<id>/view-link` (§6.4), and the view token's `use` claim (§5.5). |
+| 2.3 | Added `GET {prefix}/artifacts/<id>/status` and `GET {prefix}/artifacts/<id>/poster` (§6.5), the capability token's `ctx` claim (§5.4), and the pulse-shape rule for creates (§5.4). |
+| 2.4 | Added the `https://mieweb.github.io/pulse/open#…` pairing link form (§3), with the parameters in the fragment. |
+| 2.5 | Added `webReady` to the status response (§6.5): what the server's conversion did. |
 
 ## 8. Artifact relationships (`relatedTo`)