@mieweb/pulsevault 0.0.2 → 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,184 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
+semantic versioning once it reaches `1.0.0` — pre-1.0, minor bumps may include
+breaking changes, called out explicitly below.
+
+## [Unreleased]
+
+This release reworks the upload contract for genuine multi-tenant use — any
+third party can implement a compatible server from `PROTOCOL.md` alone, not
+just by reading this package's source. It bundles several breaking changes
+into one release rather than spreading them across several; see "Upgrading"
+in `OPERATIONS.md` for the migration path.
+
+### Security
+
+The `videoid` → `artifactId` / generic-route rework above introduced two
+related authorization bugs in `core.ts`'s `PATCH`/`HEAD` handling. Both are
+fixed in this same release; upgrade before this reaches a published version
+if you've evaluated against an intermediate build.
+
+- **`authorize` was never actually invoked for `PATCH`/`HEAD`/in-flight-`DELETE`
+  under `{prefix}/upload/<id>`**, regardless of configuration — including the
+  built-in `createCapabilityAuthorize`. The helper that recovered an
+  artifactId from the tus resource URL for the `authorize` context assumed
+  the wrong shape for the current `<kind>/<artifactId><ext>` tus id and never
+  successfully extracted a UUID, so the "no artifactId, so allow" fallback
+  silently took over on every such request. Since an artifactId is not a
+  secret (it's carried in the pairing deep link/QR code by design, per
+  `PROTOCOL.md` §3), this let anyone who had seen a pairing link write bytes
+  into that upload with no token at all. Fixed by resolving the artifactId
+  through the same parser the actual upload-completion path uses, and by
+  making authorization-context resolution failure a hard reject rather than
+  a silent allow (`PROTOCOL.md` §5.2).
+- **A crafted multi-segment `PATCH` URL could write bytes to a different
+  artifact than the one `authorize()` checked.** The `authorize`-context URL
+  parser took the first path segment after `/upload/`, while `@tus/server`'s
+  own request routing (which decides what's actually read/written) takes the
+  URL's last segment — a divergence exploitable via extra path segments
+  (accepted by the `/upload/*` wildcard route). A party holding a valid
+  token for their own artifact could append a second, victim artifactId as a
+  trailing segment: `authorize()` saw and approved their own id, while the
+  request body landed on the victim's file. Fixed by resolving the
+  authorization-context artifactId via the exact same last-segment
+  extraction `@tus/server` itself uses (`PROTOCOL.md` §4.4, a new normative
+  requirement on server implementations generally, not just this package).
+- Both are covered by new regression tests in `test/plugin.test.mjs` that
+  fail against the pre-fix code and pass against the fix.
+
+### Breaking
+
+- **Renamed `videoid` → `artifactId`** across the storage interface
+  (`ReserveUploadParams`, `PulseVaultStorage` methods), the `authorize`
+  context (`PulseVaultAuthorizeContext`), the `validatePayload`/
+  `onUploadComplete` hook contexts, the `request.pulseVault` TypeScript
+  augmentation (`./augment.ts` — a separate breaking change from the wire
+  rename, since it's the published type surface, not the HTTP contract), and
+  the deep-link query parameter built by `buildUploadLink`. `videoid`/
+  `projectid` remain accepted as legacy aliases in `Upload-Metadata` — only
+  the *primary* name changed, not backward compatibility for existing
+  clients' requests.
+- **Collapsed the per-kind routes into one generic route.** `GET/DELETE
+  /:videoid` and `GET/DELETE /project/:projectid` are gone; replaced by
+  `GET/DELETE /artifacts/:artifactId`, which resolves the kind from storage
+  rather than the URL. This also means `kind=captions` artifacts (new, see
+  below) don't need a third route pair.
+- **`validateProjectPayload`/`onProjectUploadComplete` are deprecated** (not
+  yet removed) in favor of the now-generic `validatePayload`/
+  `onUploadComplete`, which receive `ctx.kind` and run for every artifact
+  kind. Passing either deprecated option now emits a one-time
+  `DeprecationWarning` at plugin registration.
+- **`allowedExtensions` requires a `captions` default to be considered** if
+  you were relying on the exact shape of the normalized object internally
+  (public consumers passing `allowedExtensions` as documented are unaffected
+  — the new key just has a default like `video`/`project` already did).
+
+### Added
+
+- **`kind: "captions"`** artifact type (default extension `.srt`), running
+  through the same generic `validatePayload`/`onUploadComplete` hooks as
+  every other kind.
+- **`relatedTo`** — an optional `Upload-Metadata` key (and matching
+  `ReserveUploadParams`/storage field) linking one artifact to another (e.g.
+  a video's captions, or a beat belonging to a pulse manifest's session).
+  Storage adapters expose it via the new optional `getRelatedTo` method.
+- **`checksum`** — an optional `Upload-Metadata` key (`<algorithm>:<hex
+  digest>`) verified post-upload via the new `createChecksumValidator`
+  (local storage) / `createS3ChecksumValidator` (S3/R2) helpers, chainable
+  with `createMp4Sniffer`. This is at-rest integrity verification on the
+  finished file, not a per-chunk check — `@tus/server`'s installed version
+  doesn't implement the TUS Checksum extension, which would have covered
+  chunks in flight.
+- **`src/lib/capability-token.ts`**: `issueCapabilityToken`,
+  `verifyCapabilityToken`, `createCapabilityAuthorize` — a stateless,
+  HMAC-signed capability-token scheme with `kid` (key rotation with an
+  overlap window), `iat` (clock-skew tolerance), and `issuer` (cross-deployment
+  replay protection) claims. Fully optional — bring your own `authorize` if
+  you have existing auth.
+- **`GET {prefix}/capabilities`** — unauthenticated discovery route reporting
+  `protocolVersion`, `minSupportedVersion`/`maxSupportedVersion`,
+  `uploadUnit`, `kinds`, `allowedExtensions`, `maxUploadSize`, and supported
+  checksum algorithms. Lets independently-versioned deployments and clients
+  detect compatibility before pairing.
+- **`uploadUnit` plugin option** (`"beat" | "merged"`, default `"beat"`) —
+  purely advertised via `/capabilities`; the operator declares which upload
+  strategy this deployment expects, the client branches on it. Not enforced
+  by the plugin.
+- **`buildUploadLink({ uploadUnit })`** (PROTOCOL.md §3, §8) — optional
+  per-session override of the deployment-wide `uploadUnit`, carried on the
+  pairing link itself instead of only `/capabilities`. Lets an operator run
+  `"beat"` and `"merged"` sessions concurrently (staged rollout, A/B test,
+  per-tenant policy) without racing a client's separate `/capabilities` fetch
+  against a single, deployment-wide value. Fully backward compatible: omit it
+  and a client falls back to `/capabilities` exactly as before.
+- **`onArtifactEvent` plugin option** — one low-frequency hook (authorize
+  rejection, completion, validation rejection — never per chunk) covering
+  both ops metrics and a compliance audit trail from a single integration
+  point.
+- **`Protocol-Version` response header** on every route this plugin mounts.
+- **`@mieweb/pulsevault/core`** — a framework-agnostic entry point with no
+  Fastify dependency. `createPulseVaultCore(...)` returns a connect-style
+  `handler(req, res, next?)` usable directly as Express middleware
+  (`app.use(prefix, handler)`), Meteor middleware
+  (`WebApp.connectHandlers.use(prefix, handler)`), or a bare
+  `http.createServer` callback — same options, hooks, and storage adapters as
+  the Fastify plugin (which is now itself a thin adapter over this core, so
+  behavior can't drift between the two). See "Non-Fastify hosts" in
+  `README.md`, and `examples/express-demo`/`examples/meteor-demo` for full
+  runnable servers verified against the real frameworks.
+  - As part of this, `PulseVaultAuthorize`/`PulseVaultValidatePayload`/
+    `PulseVaultOnUploadComplete`'s `request` parameter is now typed as the
+    generic `PulseVaultRequest` (just `{ headers }`) instead of
+    `FastifyRequest` — `FastifyRequest` still satisfies it structurally, so
+    untyped/inferred hook callbacks (the documented pattern) are unaffected;
+    only an explicit `(request: FastifyRequest, ctx) => ...` annotation on
+    one of these three hooks would need loosening.
+- Bounded, insertion-order-evicting in-memory metadata cache in both storage
+  adapters (`metaCacheLimit` option, default 10,000 entries) — previously
+  unbounded for the life of the process.
+- `S3Storage.readAll` — full-object read, used by `createS3ChecksumValidator`.
+- `S3Storage.digestAll` — streams a finalized object through a hash digest
+  without buffering the whole thing into memory first; the streaming
+  counterpart to `readAll`, now used internally by
+  `createS3ChecksumValidator` (see "Fixed" below).
+
+### Fixed
+
+- `maxUploadSize` enforcement and the in-progress-upload 404 behavior now
+  have explicit test coverage proving bytes are rejected/hidden at the right
+  point, not just implied by the implementation.
+- **Meteor compatibility.** Meteor's bundler doesn't resolve `package.json`
+  `"exports"` subpath maps, which broke both this package's own `"./core"`
+  entry and, transitively, `@tus/server`'s dependency on `srvx` (which as of
+  `@tus/server@2.1.0` ships *only* subpath exports, no legacy `main`
+  fallback). Fixed two ways: `@tus/server`/`@tus/file-store` are pinned to
+  `2.0.0` (the last release before the `srvx` migration — everything added
+  in `2.1.0`–`2.4.1` was either the `srvx` migration itself, follow-up
+  fixes for regressions it introduced, or one unused `exposedHeaders`
+  option, so nothing this package relies on is lost); and plain root-level
+  `core.js`/`core.d.ts`/`augment.js`/`augment.d.ts` files now ship alongside
+  the `"exports"` map, so resolvers that ignore `"exports"` (Meteor's
+  included) still find these entry points via ordinary relative-path
+  resolution. Verified against a real Meteor 3.4 app: `WebApp.connectHandlers`
+  resolves `@mieweb/pulsevault/core` cleanly and a full TUS
+  create → PATCH → GET round-trip works.
+- **Unbounded memory use during checksum validation.** `createChecksumValidator`
+  read the whole finalized file into a single `Buffer` via `fs.readFile`
+  before hashing; combined with the documented-supported `maxUploadSize:
+  Infinity`, a large upload could exhaust process memory just to verify its
+  checksum. Now streams the file through the hash via `createReadStream` +
+  a stream pipeline. `createS3ChecksumValidator` had the same issue via
+  `S3Storage.readAll`'s full-object buffer — now uses the new streaming
+  `S3Storage.digestAll` instead.
+- **Silent TOCTOU on S3-compatible backends without `IfNoneMatch` support.**
+  `reserveUpload`'s conditional-write collision guard falls back to a
+  weaker check-then-write on backends that reject `IfNoneMatch` (some
+  S3-compatible stores, and the `s3rver` mock this repo's own test suite
+  runs against) — this reopens the race between concurrent/retried creates
+  for the same artifactId. There's no way to close it without an external
+  lock, so this is now at least surfaced: a one-time `console.warn` fires
+  per process the first time a deployment falls into this degraded mode.
+
+[Unreleased]: https://github.com/mieweb/pulsevault/compare/v0.0.1...HEAD

package/LICENSE

@@ -1,45 +1,21 @@
-@mieweb/pulsevault - Source Available License
+MIT License
 
-Copyright (c) 2026 Medical Informatics Engineering, LLC. All rights reserved.
+Copyright (c) 2026 Medical Informatics Engineering, LLC
 
-This software and associated documentation files (the "Software") are the
-proprietary property of Medical Informatics Engineering, LLC. ("MIE").
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
 
-OPEN SOURCE / NON-COMMERCIAL USE
-
-Permission is hereby granted, free of charge, to any person or organization
-obtaining a copy of this Software, to use, copy, modify, merge, and distribute
-the Software for non-commercial purposes, subject to the following conditions:
-
-1. Attribution Required: The above copyright notice, this permission notice,
-   and clear attribution to Medical Informatics Engineering, LLC. shall be
-   included in all copies or substantial portions of the Software.
-
-2. Open Source Requirement: If you distribute the Software or any derivative
-   works, you must do so under an open source license approved by the Open
-   Source Initiative (OSI) and make the source code publicly available.
-
-3. Non-Commercial Use Only: This license does not grant permission to use the
-   Software for commercial purposes. "Commercial purposes" means any use
-   intended for or directed toward commercial advantage or monetary
-   compensation.
-
-COMMERCIAL USE
-
-For commercial use, including but not limited to:
-- Use in proprietary/closed-source products
-- Use in products or services sold for profit
-- Use by for-profit organizations for internal business operations
-
-You must obtain a separate commercial license from Medical Informatics
-Engineering, LLC. Contact: https://www.mieweb.com or helpdesk@mieweb.com
-
-NO WARRANTY
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-MEDICAL INFORMATICS ENGINEERING, LLC. OR ITS AFFILIATES BE LIABLE FOR ANY
-CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
-OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
-OR OTHER DEALINGS IN THE SOFTWARE.
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/OPERATIONS.md

@@ -0,0 +1,223 @@
+# Operations
+
+Deployment, scaling, and maintenance guidance for running `@mieweb/pulsevault`
+in production. See `README.md` for the quick-start and plugin options, and
+`PROTOCOL.md` for the wire contract.
+
+## Upgrading to this release
+
+This release renames `videoid` → `artifactId` and collapses the per-kind
+routes into one generic route (see `CHANGELOG.md`). **No on-disk data
+migration is required**: the local and S3 storage layouts never stored the
+id as a field inside the sidecar JSON — the id is the filename — so existing
+uploaded artifacts are immediately readable through the new
+`GET /artifacts/:artifactId` route with zero changes to the files on disk.
+
+What *does* need updating:
+
+- Any of your own code calling the old routes directly (`GET/DELETE
+  /:videoid`, `GET/DELETE /project/:projectid`) must move to `GET/DELETE
+  /artifacts/:artifactId` — the old routes return `404`, not a redirect.
+- Any code reading `request.pulseVault.videoid` (if you imported
+  `@mieweb/pulsevault/augment`) must read `.artifactId` instead.
+- If you use `validateProjectPayload`/`onProjectUploadComplete`, you'll see a
+  one-time `DeprecationWarning` at startup. They still work this release —
+  migrate to `validatePayload`/`onUploadComplete` with a `ctx.kind ===
+  "project"` branch when convenient, before a future major version removes
+  the deprecated options.
+- Clients still sending `Upload-Metadata: videoid ...` (instead of
+  `artifactId`) keep working — the alias is accepted indefinitely for
+  protocol version 1 (see `PROTOCOL.md` §4.1) — but new client code should
+  send `artifactId`.
+
+There is no automated migration script because there is nothing on disk to
+migrate. If a future release ever does change the storage layout, that
+release's `OPERATIONS.md` entry will include one.
+
+## Horizontal scaling
+
+**The local storage adapter (`createLocalStorage`) requires either
+sticky-session load-balancer routing or a shared filesystem (NFS/EFS/etc.)
+across instances.** TUS resumable uploads need subsequent `PATCH`/`HEAD`
+requests to reach an instance that can see the partial upload's bytes and
+`@tus/file-store` offset metadata. If you run multiple instances behind a
+load balancer without either of these, a retried `PATCH` that lands on a
+different instance will silently fail to resume — the client's `HEAD` will
+see no offset and effectively restart from byte zero, with no error
+surfaced anywhere.
+
+Two ways to actually support multiple instances:
+
+1. **Sticky sessions**: configure your load balancer to route by client IP
+   or a session cookie so all requests for one upload land on the same
+   instance.
+2. **Shared filesystem**: point every instance's `workspaceDir` at the same
+   NFS/EFS mount. Verify your mount's durability/consistency guarantees
+   under concurrent writes from multiple instances before relying on this in
+   production.
+
+**The S3/R2 adapter (`createS3Storage`) has no such requirement** — every
+instance talks to the same bucket, so it scales horizontally with zero
+additional configuration. Prefer it for any multi-instance deployment unless
+you have a specific reason to use local disk.
+
+### S3-compatible backend collision-guard fallback
+
+`reserveUpload`'s collision guard normally uses `PutObjectCommand`'s
+`IfNoneMatch: "*"` to atomically reject a second create for an artifactId
+that already has an upload. Some S3-compatible backends (older/less-complete
+implementations; the `s3rver` mock this package's own test suite runs
+against) don't support conditional writes and reject that header outright.
+On those backends, `reserveUpload` falls back to a weaker check-then-write —
+functionally the same guard, but with the original race reopened: two
+truly concurrent (or retried) creates for the same artifactId can both pass
+the check before either writes, and the second silently clobbers the
+first's metadata. There's no way to close this without an external lock,
+since it's a limitation of the backend, not this package. If your bucket
+provider doesn't support `IfNoneMatch`, this fallback logs a one-time
+`console.warn` per process the first time it's used — treat that warning as
+a signal to either move to a backend that supports conditional writes, or
+serialize artifactId creation yourself (e.g. in your own `/reserve`
+endpoint) if concurrent creates for the same id are a real possibility in
+your deployment.
+
+## Resource limits and abuse prevention
+
+`pulsevault` does not implement rate limiting or a concurrent-upload cap
+itself — that's left to the operator, consistent with this project's general
+principle that policy decisions belong to the deployment, not the library.
+A minimal example using `@fastify/rate-limit`, scoped to just the upload
+routes:
+
+```ts
+import rateLimit from "@fastify/rate-limit";
+
+await app.register(rateLimit, {
+  max: 20,
+  timeWindow: "1 minute",
+  // Scope to the pulsevault prefix only — don't rate-limit your whole app
+  // with upload-sized limits.
+  allowList: (req) => !req.url.startsWith("/pulsevault/upload"),
+});
+```
+
+Under `@mieweb/pulsevault/core` (Express, Meteor, plain `http`), use the equivalent middleware for your host — e.g. `express-rate-limit` mounted ahead of `pulseVault.handler`, scoped the same way.
+
+## Monitoring and audit logging
+
+Wire `onArtifactEvent` once to get both ops metrics and a compliance audit
+trail from the same hook — it fires at low-frequency, audit-worthy moments
+only (never per chunk): authorize rejection (on `create`/`delete`/`resolve`,
+not `patch`), successful completion, and validation rejection.
+
+**Plain structured logging** (zero new dependencies):
+
+```ts
+onArtifactEvent: (event) => {
+  app.log.info(event, "pulsevault artifact event");
+},
+```
+
+**Prometheus counters**:
+
+```ts
+import { Counter } from "prom-client";
+const artifactEvents = new Counter({
+  name: "pulsevault_artifact_events_total",
+  help: "pulsevault artifact lifecycle events",
+  labelNames: ["phase", "kind"],
+});
+
+onArtifactEvent: (event) => {
+  artifactEvents.inc({ phase: event.phase, kind: event.kind });
+  if (event.phase === "reject" || event.phase === "authorize") {
+    app.log.warn(event, "pulsevault artifact event");
+  }
+},
+```
+
+`onArtifactEvent` isn't request-scoped (no Fastify instance to hang a logger
+off of either way), so the examples above read the same under
+`@mieweb/pulsevault/core` — just close over whatever logger (or `console`)
+your app already uses instead of `app.log`. The core's own internal
+diagnostics (authorize-rejection/error logging separate from
+`onArtifactEvent`) accept an explicit `logger` option on
+`createPulseVaultCore(...)` for the same reason — see "Non-Fastify hosts" in
+`README.md`.
+
+What to alert on: a sustained rise in `reject`/`authorize`-rejection events
+(misconfigured auth, or an attacker probing), disk usage on the local
+adapter's `workspaceDir` approaching capacity, and any `5xx` rate on the
+upload routes.
+
+## Backup and restore (local storage)
+
+The entire state of the local adapter lives under `workspaceDir`:
+`.pulsevault/` (sidecars), `video/`, `project/`, `captions/` (bytes). Back it
+up as a normal filesystem tree — there's no separate database to keep in
+sync. To restore, copy the tree back and restart; in-progress uploads at
+backup time will resume correctly via the normal TUS `HEAD`-then-resume path
+once the client retries, or will simply sit as abandoned partial uploads
+(see "Retention" below) if the client never retries.
+
+## Retention
+
+`pulsevault` has no built-in retention/expiry feature — this is intentionally
+left to the operator (see `PROTOCOL.md` §1 on lifecycle ownership). A sample
+cron script for the local adapter, deleting artifacts whose sidecar has been
+`"uploading"` for longer than a cutoff (abandoned uploads) or that are older
+than a retention window (compliance-driven deletion):
+
+```ts
+import { readdir, readFile, stat } from "node:fs/promises";
+import path from "node:path";
+import { createLocalStorage } from "@mieweb/pulsevault";
+
+const storage = createLocalStorage({ workspaceDir: "./data" });
+const sidecarDir = path.join(storage.workspaceRoot, ".pulsevault");
+const ABANDONED_AFTER_MS = 24 * 60 * 60 * 1000; // 24h stuck "uploading"
+const RETAIN_FOR_MS = 90 * 24 * 60 * 60 * 1000; // 90 days
+
+for (const file of await readdir(sidecarDir)) {
+  if (!file.endsWith(".json")) continue;
+  const artifactId = file.slice(0, -".json".length);
+  const sidecarPath = path.join(sidecarDir, file);
+  const [sidecar, stats] = await Promise.all([
+    readFile(sidecarPath, "utf8").then(JSON.parse),
+    stat(sidecarPath),
+  ]);
+  const ageMs = Date.now() - stats.mtimeMs;
+  const stuckUploading = sidecar.status === "uploading" && ageMs > ABANDONED_AFTER_MS;
+  const pastRetention = sidecar.status === "ready" && ageMs > RETAIN_FOR_MS;
+  if (stuckUploading || pastRetention) {
+    await storage.remove(artifactId);
+    console.log(`removed ${artifactId} (${stuckUploading ? "abandoned" : "retention"})`);
+  }
+}
+```
+
+Run this on whatever schedule your retention policy requires. For S3/R2
+storage, prefer your bucket provider's native lifecycle-policy feature (S3
+Lifecycle Rules, R2 Object Lifecycle) over a custom script where available.
+
+## Secrets management
+
+If you use `createCapabilityAuthorize`, you supply the HMAC secret(s)
+yourself via `lookupSecret`. For local development, reading from an
+environment variable is fine. For production, read from your organization's
+actual secrets manager instead of a raw env var — e.g.:
+
+```ts
+import { SecretsManagerClient, GetSecretValueCommand } from "@aws-sdk/client-secrets-manager";
+
+const client = new SecretsManagerClient({});
+const keys: Record<string, string> = {};
+async function loadKey(kid: string) {
+  const res = await client.send(new GetSecretValueCommand({ SecretId: `pulsevault/${kid}` }));
+  keys[kid] = res.SecretString!;
+}
+```
+
+Rotate by adding the new `kid` to your lookup table alongside the old one,
+switching issuance to the new `kid`, and removing the old entry only after
+its longest-lived outstanding token has expired.