plumb-fhir 0.20.0 → 0.22.0

package/CHANGELOG.md

@@ -1,5 +1,39 @@
 # Changelog
 
+## 0.22.0 (2026-10-23)
+
+A profile output's OperationDefinition changes, so the next `push` updates it.
+
+- **A passing `plumb validate` records a receipt:** one `Basic` in the
+  project, coded `validation` as the migration ledger is coded `migration`,
+  naming each profile's URL, version and a SHA-256 of its closure, the
+  checker's build, and the second the run began by the server's clock. Each
+  pass replaces the last. A failing or shadowed run records none.
+- **`push`'s gate and re-check read from the receipt.** When it covers every
+  profile the gate checks, at the same version, content and checker, the
+  gate reads only what was updated from the second validate began
+  (`_lastUpdated=ge…`), and the re-check only what was updated from the
+  gate's start: `gate  incremental since 2026-10-23T09:00:00.000Z (receipt
+  Basic/…): 4 Patient stamped my-patient checked, nothing would fail`. With
+  no matching receipt the gate reads everything as before, and names why:
+  `Full scan: no receipt from plumb validate.` So a large project validates
+  in a step of its own, with a long time limit, and deploys fast (#291).
+- **A profile output's parameter names its profile in `documentation`**
+  (`Conforms to <url>`) instead of `targetProfile`, which FHIR's `opd-3`
+  allows only on a Reference or canonical parameter (#291).
+
+## 0.21.0 (2026-10-23)
+
+- **`push` gates only the changed profiles.** The gate and the re-check read
+  what is stamped with a profile this push creates or updates, or whose
+  parent or extension it does, not every selected profile's records, and
+  say what they read: `489 Patient stamped my-patient checked, nothing would
+  fail; 53 unchanged profiles skipped`. With strict mode on, the server
+  refused any write failing an already-loaded profile, so its records need
+  no second read. With strict mode off, or unknown, every profile is still
+  checked. A push changing only terminology runs no check, as the checker
+  validates no binding. `plumb validate` remains a full pass.
+
 ## 0.20.0 (2026-10-18)
 
 The checker's input changes, so the next `push` redeploys it.

package/README.md

@@ -557,8 +557,15 @@ Failed in 40.8s
 - **Exit codes:** 0 when nothing fails; 1 when something would fail, a type
   could not be read, or a profile is shadowed; 2 for usage, config and
   connection errors, and when the checker is missing or from another Plumb
-  release (run `plumb push` to install it). `validate` never installs or
-  changes anything.
+  release (run `plumb push` to install it). `validate` installs and changes
+  nothing; a passing run records only its receipt, below.
+- **A passing run records a receipt:** one `Basic` in the project, coded
+  `validation` in Plumb's system, as the [migration ledger](#migrate-stored-data)
+  is, holding each profile's URL, version and a SHA-256 of what it was
+  checked with (it, its parents, its extensions), the checker's build, and
+  the second the run began by the server's clock. Each pass replaces the
+  last; the `Basic`'s history keeps them. A failing or shadowed run records
+  none.
 
 **4. Fix or [migrate](#migrate-stored-data)** the failing records, and
 validate again until nothing fails.
@@ -575,9 +582,10 @@ plumb push --env prod
 ✔ connect   https://api.medplum.com/ (strict mode off)   320ms
 ✔ checker   plumb-checker 0.6.0 unchanged   90ms
 ✔ plan      load us-core-patient 9.0.0 (+7 dependencies)   210ms
-✔ gate      nothing stored would fail   36.1s
+✔ gate      312 Patient stamped us-core-patient checked, nothing would fail   36.1s
+    Strict mode is off, so every profile is checked.
 ✔ apply     8 created, 0 updated   1.4s
-✔ recheck   nothing stored fails   35.7s
+✔ recheck   312 Patient stamped us-core-patient checked, nothing fails   35.7s
     Strict mode is off, and only a super admin can turn it on; every stamped resource checked passes its profiles.
 Done in 76.4s
 ```
@@ -598,11 +606,29 @@ Done in 76.4s
   written.
 - **gate** runs the checker against the planned versions. If any stored
   resource would fail, `push` refuses and loads nothing. `--dry-run` stops
-  here, so it is a safe way to install the checker and preview a push.
+  here, so it is a safe way to install the checker and preview a push. It
+  reads only what is stamped with a profile this push changes: one created
+  or updated, or whose parent or extension is. With strict mode on, the
+  server already refused any write failing a loaded profile, so an unchanged
+  profile's records are skipped, and the step says how many profiles were.
+  With strict mode off it checks every selected profile, as `validate` does.
+  A push that changes only terminology has nothing for the checker to read,
+  since it validates no binding.
+- **A receipt makes the gate incremental.** When the project's `validate`
+  receipt covers every profile the gate checks, at the same version and
+  content and under the same checker, the gate reads only what was updated
+  from the second that run began (`_lastUpdated=ge…`, by the server's
+  clock): everything older passed those very profiles. It says so:
+  `gate  incremental since 2026-10-23T09:00:00.000Z (receipt Basic/…): 4
+  Patient stamped …`. Otherwise it reads everything and names why, such as
+  `Full scan: no receipt from plumb validate.` or `Full scan: receipt
+  Basic/… covers my-patient 2.1.0, not 2.2.0.`, so no receipt is never less
+  safe than none existing.
 - **apply** creates or updates the definitions, terminology first.
-- **recheck** checks again at once, to catch a failing write made between the
-  gate and loading. A failure there fails the push (exit 1), and the profiles
-  stay loaded, as Postgres keeps a `NOT VALID` constraint.
+- **recheck** checks the same profiles again at once, to catch a failing
+  write made between the gate and loading. A failure there fails the push (exit 1), and the profiles
+  stay loaded, as Postgres keeps a `NOT VALID` constraint. After an
+  incremental gate it reads only what was updated from the gate's start.
 
 Exit codes are `validate`'s: a write that fails, or a plan blocked by an
 untagged or shadowed resource, exits 1. `--json` and `--quiet` work as for
@@ -615,15 +641,29 @@ resource that fails a profile it names; a clean `push` re-check says the
 stored data is ready.
 
 In CI, run `push` where the deploy runs, with the secrets as environment
-variables:
+variables. On a large project, validate first, in a step of its own with a
+long time limit, then push: the slow full read is the validate, and the
+deploy's gate reads only what changed since it began, as Postgres adds a
+constraint `NOT VALID` and runs `VALIDATE CONSTRAINT` apart:
 
 ```yaml
-- run: npx plumb push --env prod
+- run: npx plumb validate --env prod --concurrency 8
+  timeout-minutes: 240
+  env:
+    MEDPLUM_PROD_CLIENT_ID: ${{ secrets.MEDPLUM_PROD_CLIENT_ID }}
+    MEDPLUM_PROD_CLIENT_SECRET: ${{ secrets.MEDPLUM_PROD_CLIENT_SECRET }}
+- run: npx plumb push --env prod   # gate and recheck read only what changed since
   env:
     MEDPLUM_PROD_CLIENT_ID: ${{ secrets.MEDPLUM_PROD_CLIENT_ID }}
     MEDPLUM_PROD_CLIENT_SECRET: ${{ secrets.MEDPLUM_PROD_CLIENT_SECRET }}
 ```
 
+Validate with the same `plumb-fhir` and `plumb.config.ts` as the push, or
+the receipt does not match and the gate reads everything. `validate` needs
+the checker this `plumb-fhir` installs, and exits 2 (`checker-outdated`)
+until a push has installed it: land a `plumb-fhir` upgrade in a deploy of its
+own, before the profile change.
+
 ## Migrate stored data
 
 `validate` says which stored records a stricter profile would refuse; a
@@ -634,9 +674,15 @@ database's **check, backfill, then tighten**:
 npx plumb validate --env prod          # 312 Patients fail us-core-patient: birthDate missing
 npx plumb migrate --env prod           # dry run: 312 to change, and a forecast
 npx plumb migrate --env prod --write   # the backfill, resumable
-npx plumb push --env prod              # the gate passes; the stricter profile loads
+npx plumb validate --env prod          # nothing fails: the receipt is recorded
+npx plumb push --env prod              # the gate reads only what changed since; the stricter profile loads
 ```
 
+The validate is the slow step: it reads every stamped record. Run it apart,
+with a long time limit, and the push that follows stays fast, as the gate and
+re-check read only what was updated after the validate began. Without a
+receipt for the same profile versions, `push` reads everything, as before.
+
 **Declare a migration** in a module the config lists. `plumb migrate new
 patient-birthdate` scaffolds `<date>-patient-birthdate.ts` in the first
 `modules` folder:

package/dist/cjs/testing.cjs

@@ -2163,9 +2163,36 @@ async function loadAndConnect(options, result2, step) {
 }
 
 // src/conformance.ts
+var RECEIPT = `${PLUMB_SYSTEM}#validation`;
+function covered(gated) {
+  return gated.profiles.map((p) => ({
+    url: p.url,
+    ...p.sd.version ? { version: p.sd.version } : {},
+    hash: (0, import_node_crypto5.createHash)("sha256").update(JSON.stringify(closure([p.url], gated))).digest("hex")
+  }));
+}
+var findReceipt = (medplum) => medplum.searchOne("Basic", owned(medplum, { code: `${PLUMB_SYSTEM}|validation` }));
+async function receiptFor(medplum, gated, checker) {
+  const held = await findReceipt(medplum);
+  const value = held?.extension?.find((e) => e.url === RECEIPT)?.valueString;
+  if (!held || !value) return { reason: "no receipt from plumb validate" };
+  const receipt = JSON.parse(value);
+  const id = `receipt Basic/${held.id}`;
+  if (receipt.checker !== checker) return { reason: `${id} is from another checker build` };
+  for (const p of covered(gated)) {
+    const name = p.url.slice(p.url.lastIndexOf("/") + 1);
+    const found = receipt.profiles.find((r) => r.url === p.url);
+    if (!found) return { reason: `${id} does not cover ${name}` };
+    if (found.version !== p.version) {
+      return { reason: `${id} covers ${name} ${found.version}, not ${p.version}` };
+    }
+    if (found.hash !== p.hash) return { reason: `${id} covers other content of ${name}` };
+  }
+  return { id: held.id, since: receipt.began };
+}
 async function checkStored(medplum, loaded, botId, options) {
   const profiles = loaded.profiles.map((p) => [p.url, p.sd.version]).sort();
-  const modes = [options.full ?? false, options.forecast !== void 0];
+  const modes = [options.full ?? false, options.forecast !== void 0, options.since ?? null];
   const key = (0, import_node_crypto5.createHash)("sha256").update(JSON.stringify([options.filename, profiles, ...modes])).digest("hex");
   const { saved, warnings: warnings2 } = await start(medplum, options, key);
   const resumed = saved.pages;
@@ -2194,6 +2221,7 @@ async function checkStored(medplum, loaded, botId, options) {
     );
   }
   const checked = {
+    began: new Date(Date.parse(saved.before) - 1e3).toISOString(),
     types: {},
     profiles: {},
     resumed,
@@ -2301,10 +2329,12 @@ async function checkType(medplum, saved, options, botId, input, onCore) {
   let type = saved.types[resourceType];
   if (type?.done) return;
   if (!type) {
-    const count2 = await medplum.search(resourceType, {
-      _summary: "count",
-      _total: "accurate"
-    });
+    const since = options.since ? { slice: { from: options.since } } : void 0;
+    const count2 = await medplum.search(resourceType, [
+      ...since ? readQuery({ ...input, ...since }) : [],
+      ["_summary", "count"],
+      ["_total", "accurate"]
+    ]);
     type = {
       exists: count2.total ?? 0,
       read: 0,
@@ -2314,7 +2344,7 @@ async function checkType(medplum, saved, options, botId, input, onCore) {
       silent: { unknown: 0, versioned: 0, empty: 0 },
       otherProfiles: {},
       stampedOther: 0,
-      slices: await split(medplum, input, options.concurrency),
+      slices: await split(medplum, input, options.concurrency, options.since),
       done: false
     };
     saved.types[resourceType] = type;
@@ -2329,7 +2359,7 @@ async function checkType(medplum, saved, options, botId, input, onCore) {
     merge(saved, current, page, resourceType);
     saved.pages++;
     write2(options.reportPath, saved);
-    const progress = input.full ? { read: current.read, of: current.exists } : { read: current.stamped, of: current.of ?? 0 };
+    const progress = input.full ? { read: current.read, of: current.exists } : { read: current.stamped, of: current.of ?? (options.since ? current.exists : 0) };
     return options.onPage?.(resourceType, saved.pages, progress);
   });
 }
@@ -2363,16 +2393,18 @@ async function drive(run, input, type, save) {
 var SLICES_PER_WORKER = 4;
 var MIN_SLICE = 1e3;
 var MIN_SPAN_MS = 1e3;
-async function split(medplum, input, concurrency = 1) {
-  if (concurrency <= 1 || !input.before) return [{ done: false }];
+async function split(medplum, input, concurrency = 1, since) {
+  if (concurrency <= 1 || !input.before)
+    return [{ ...since ? { from: since } : {}, done: false }];
   const count2 = async (from, to) => {
     const slice = { ...from === void 0 ? {} : { from: iso(from) }, to: iso(to) };
     const query = [...readQuery({ ...input, slice }), ["_summary", "count"]];
     return (await medplum.search(input.resourceType, query)).total ?? 0;
   };
   const end = Date.parse(input.before);
+  const first = since ? Date.parse(since) : void 0;
   const ranges = [
-    { to: end, n: await count2(void 0, end) }
+    { ...first === void 0 ? {} : { from: first }, to: end, n: await count2(first, end) }
   ];
   const target = Math.max(
     MIN_SLICE,
@@ -2388,8 +2420,9 @@ async function split(medplum, input, concurrency = 1) {
     ranges.splice(ranges.indexOf(largest), 1, { ...largest, to: mid, n: left }, right);
   }
   const kept = ranges.filter((r, i) => r.n > 0 || i === ranges.length - 1);
+  const lower = (i) => i === 0 ? first : kept[i - 1]?.to;
   return kept.map((r, i) => ({
-    ...i === 0 ? {} : { from: iso(kept[i - 1]?.to ?? 0) },
+    ...lower(i) === void 0 ? {} : { from: iso(lower(i)) },
     to: iso(r.to),
     done: false
   }));
@@ -3395,7 +3428,8 @@ function operationDefinition(contract, botId, typeOf) {
         min: 1,
         max: "1",
         type,
-        ...!json && output.includes(":") ? { targetProfile: [output] } : {}
+        // FHIR's opd-3 allows targetProfile on Reference and canonical parameters only.
+        ...!json && output.includes(":") ? { documentation: `Conforms to ${output}` } : {}
       }
     ],
     ...botId ? { extension: [{ url: IMPLEMENTATION, valueReference: { reference: `Bot/${botId}` } }] } : {},
@@ -3758,22 +3792,15 @@ async function push(unscoped2) {
   if (result2.plan.some((p) => p.action === "shadowed" || p.action === "linked")) return step.done();
   if (changes.length === 0) return finish(true);
   const filename = checkerFilename(options.checker.code, options.checker.version);
-  result2.reportPath = options.reportPath;
-  const gated = gatedProfiles(loaded, changes);
-  const check = (name) => checkStored(medplum, gated, botId, { ...options, filename }).catch((err) => {
-    step.fail(name, [{ code: "checker-failed", message: (0, import_core13.normalizeErrorString)(err) }]);
-    return void 0;
-  });
-  result2.gate = await check("gate");
-  if (!result2.gate) return result2;
-  const gate = judge(result2.gate, gated.profiles.length);
-  step.finish(
-    "gate",
-    gate.failed ? failingSummary(result2.gate) : "nothing stored would fail",
-    gate.failed ? await refusal(medplum, result2.gate, options) : options.dryRun ? ["Dry run: nothing loaded."] : [],
-    gate.failed
+  const gated = gatedProfiles(loaded, changes, result2.strictMode === true);
+  const check = (name, since2) => checkStored(medplum, gated, botId, { ...options, filename, ...since2 ? { since: since2 } : {} }).catch(
+    (err) => {
+      step.fail(name, [{ code: "checker-failed", message: (0, import_core13.normalizeErrorString)(err) }]);
+      return void 0;
+    }
   );
-  if (gate.failed) return step.done();
+  const gate = { gated, filename, check };
+  if (!await gateStep(medplum, gate, options, result2, step)) return step.done();
   if (options.dryRun) return finish(true);
   try {
     await apply(medplum, changes, planned, held);
@@ -3782,17 +3809,51 @@ async function push(unscoped2) {
   }
   const created = changes.filter((p) => p.action === "create").length;
   step.finish("apply", `${created} created, ${changes.length - created} updated`);
-  result2.recheck = await check("recheck");
+  if (gated.profiles.length === 0) return finish(true);
+  const since = result2.receipt && result2.gate?.began;
+  result2.recheck = await check("recheck", since);
   if (!result2.recheck) return result2;
   const recheck = judge(result2.recheck, gated.profiles.length);
   step.finish(
     "recheck",
-    recheck.failed ? failingSummary(result2.recheck, true) : "nothing stored fails",
+    `${since ? `incremental since ${since}: ` : ""}${recheck.failed ? failingSummary(result2.recheck, true) : scopedSummary(gated, "nothing fails", result2.recheck)}`,
     recheckNotes(recheck.failed, result2.strictMode === true),
     recheck.failed
   );
   return finish(!recheck.failed);
 }
+async function gateStep(medplum, {
+  gated,
+  filename,
+  check
+}, options, result2, step) {
+  const dryRun = options.dryRun ? ["Dry run: nothing loaded."] : [];
+  if (gated.profiles.length === 0) {
+    step.finish("gate", scopedSummary(gated, "no profile changed, nothing to check"), dryRun);
+    return true;
+  }
+  const found = await receiptFor(medplum, gated, filename).catch((err) => ({
+    reason: `the receipt was not readable: ${(0, import_core13.normalizeErrorString)(err)}`
+  }));
+  if ("since" in found) result2.receipt = found;
+  result2.reportPath = options.reportPath;
+  result2.gate = await check("gate", result2.receipt?.since);
+  if (!result2.gate) return false;
+  const gate = judge(result2.gate, gated.profiles.length);
+  const mode = "since" in found ? `incremental since ${found.since} (receipt Basic/${found.id}): ` : "";
+  const scan = "reason" in found ? [`Full scan: ${found.reason}.`] : [];
+  const full = result2.strictMode ? [] : ["Strict mode is off, so every profile is checked."];
+  step.finish(
+    "gate",
+    `${mode}${gate.failed ? failingSummary(result2.gate) : scopedSummary(gated, "nothing would fail", result2.gate)}`,
+    [
+      ...scan,
+      ...gate.failed ? await refusal(medplum, result2.gate, options) : [...full, ...dryRun]
+    ],
+    gate.failed
+  );
+  return !gate.failed;
+}
 async function prepare(options, out, result2, step) {
   const ready = await loadAndConnect(options, result2, step);
   if (!ready) return void 0;
@@ -4107,13 +4168,28 @@ async function heldCopies(medplum, planned) {
   }
   return { held, server, ours: (d) => d.meta?.project === project.id };
 }
-function gatedProfiles(loaded, changes) {
+function gatedProfiles(loaded, changes, strictMode) {
   const selected = new Set(loaded.profiles.map((p) => p.url));
   const others = changes.flatMap((c) => {
     const sd = loaded.definitions.get(c.url)?.resource;
     return !selected.has(c.url) && sd?.resourceType === "StructureDefinition" && sd.kind === "resource" && sd.derivation === "constraint" ? [{ url: c.url, sd }] : [];
   });
-  return { profiles: [...loaded.profiles, ...others], definitions: loaded.definitions };
+  const all = [...loaded.profiles, ...others];
+  const changed = new Set(changes.map((c) => c.url));
+  const profiles = strictMode ? all.filter((p) => closure([p.url], loaded).some((sd) => changed.has(sd.url))) : all;
+  return { profiles, definitions: loaded.definitions, skipped: all.length - profiles.length };
+}
+function scopedSummary(gated, verdict, checked) {
+  const types = [...new Set(gated.profiles.map((p) => p.sd.type))].sort();
+  const read2 = types.map((type) => {
+    const names = gated.profiles.filter((p) => p.sd.type === type).map((p) => p.url.slice(p.url.lastIndexOf("/") + 1));
+    const which = names.length > 3 ? `${names.length} profiles` : names.join(", ");
+    return `${checked?.types[type]?.stamped ?? 0} ${type} stamped ${which}`;
+  });
+  return [
+    checked ? `${read2.join(", ")} checked, ${verdict}` : verdict,
+    gated.skipped > 0 && `${gated.skipped} unchanged profile${gated.skipped === 1 ? "" : "s"} skipped`
+  ].filter(Boolean).join("; ");
 }
 async function apply(medplum, changes, planned, held) {
   for (const change of changes) {