@scalequality/check 0.5.5 → 0.5.6

package/README.md

@@ -19,13 +19,26 @@ and **Turn on continuous measurement**. Store the token as a CI secret named
 | `--token` | `SQ_TOKEN` | — | Your SQ token (required) |
 | `--project` | `SQ_PROJECT` | — | ScaleQuality project id (required) |
 | `--host` | `SQ_HOST` | `https://app.scalequality.io` | API host |
-| `--timeout` | — | `600` | Max seconds to wait for the measurement |
+| `--timeout` | — | `600` | Max seconds to wait for the measurement, and separately for a measurement of the same project already running |
 | `--json` | — | off | Print the raw result as JSON |
 | `--exclude <path>` | — | — | Exclude a generated file/directory by literal checkout-relative path; repeatable |
+| `--strict` | `SQ_STRICT` | off | Fail the step (exit `2`) when the run could not be measured |
 | `--dry-run` | — | off | Measure locally and validate the package without uploading; no token needed |
 | `--artifact <path>` | `SQ_ARTIFACTS` | — | Scan a binary this job built for its components, licenses and CVEs (needs trivy); repeatable |
 
-Exit codes: `0` passed (or track-only), `1` below the cutoff or a license blocked by your policy, `2` error.
+Exit codes:
+
+| Code | When |
+|------|------|
+| `0` | Measured and passed, measured in track-only mode, or **NOT MEASURED** (see below) |
+| `1` | Measured: below your cutoff, or a new dependency with a license your organization blocks |
+| `2` | NOT MEASURED with `--strict`, or `--dry-run` could not prepare the package |
+
+Only a measurement fails the build. When a run cannot be measured, the step says
+**NOT MEASURED** in the log, in a warning on the run (GitHub Actions, Azure
+Pipelines), in the job summary and in `--json` (`"measured": false`, with
+`origin` and `reason`), never says the gate passed, and exits `0`. Pass `--strict`
+to fail the step instead.
 
 ## CI examples
 
@@ -225,3 +238,33 @@ licenses and their CVEs join the measurement. Needs trivy on PATH, like
 - run: npx @scalequality/check --artifact target/app.war
   env: { SQ_TOKEN: ${{ secrets.SQ_TOKEN }}, SQ_PROJECT: ${{ vars.SQ_PROJECT }} }
 ```
+
+## 0.5.6: only a measurement fails the build
+
+Until 0.5.5 every run that could not be measured exited `2`, and a pipeline that
+runs the step as is failed on it: ScaleQuality unreachable or answering 5xx, a
+timeout, a failed measurement, an expired token, a lapsed plan, continuous turned
+off, or a pull request from a fork or Dependabot that gets no secrets. Now only a
+measured verdict fails the step (`1`). Everything else exits `0` and says
+**NOT MEASURED**, with who has to act:
+
+| `origin` | `reason` | What happened |
+|----------|----------|---------------|
+| `scalequality` | `unreachable`, `server_error`, `auth_unavailable`, `server_rejected` | The upload did not reach a working ScaleQuality (after three short retries for network errors, 429 and 5xx) |
+| `scalequality` | `measurement_already_running` | A measurement of the same project kept running for longer than `--timeout` (the check waits for it first) |
+| `scalequality` | `timeout`, `status_unavailable`, `run_not_found`, `no_run_id` | The measurement did not report back in time |
+| `scalequality` | `measurement_failed` | The engine failed; `failureReason` says why |
+| `scalequality` | `no_verdict` | The measurement finished without a score to hold the cutoff against (exited `1` before) |
+| `scalequality` | `protocol_unconfirmed`, `durability_unconfirmed` | The server did not confirm the report this checkout sent |
+| `configuration` | `missing_credentials` | No `--token`/`--project` (secrets do not reach fork or Dependabot pull requests) |
+| `configuration` | `invalid_token`, `token_expired` | The token was refused or expired (tokens default to 90 days) |
+| `configuration` | `subscription_required` | No active plan with continuous measurement |
+| `configuration` | `continuous_not_enabled` | Continuous measurement is turned off for the project |
+| `configuration` | `project_not_found`, `invalid_option` | The project id or an `--exclude` is wrong |
+| `checkout` | `durability_not_collected`, `checkout_not_packaged`, `head_changed` | This job's checkout could not be measured or packaged (a shallow clone, for example); nothing was uploaded |
+
+Each request has its own deadline, so a host that accepts the connection and never
+answers no longer holds the job until the CI kills it. `--strict` (or
+`SQ_STRICT=true`) restores exit `2` for all of the above, for teams that prefer to
+fail closed. A license your organization blocks and a score below the cutoff still
+exit `1`, with or without it.

package/bin/check.mjs

@@ -1,8 +1,11 @@
 #!/usr/bin/env node
 // ScaleQuality continuous check. Uploads THIS CI checkout to ScaleQuality (re-measures
-// the code the runner already has, with a local 90-day durability report) and exits
-// non-zero when maturity is below the configured cutoff, so a CI step can fail the
-// build. Pure Node 18+ (global fetch/FormData/Blob), no npm dependencies.
+// the code the runner already has, with a local 90-day durability report) and exits 1
+// when the measurement says the gate did not pass (below the configured cutoff, or a
+// license the organization blocks), so a CI step can fail the build. Only a measured
+// verdict fails it: a run ScaleQuality could not measure exits 0 with a NOT MEASURED
+// warning, unless --strict asks to fail closed. Pure Node 18+ (global
+// fetch/FormData/Blob), no npm dependencies.
 
 import { spawnSync } from "node:child_process"
 import { readFileSync, existsSync, appendFileSync } from "node:fs"
@@ -35,8 +38,11 @@ Options:
                    binary, .NET, wheel) for its components, their licenses and
                    CVEs; repeatable. Needs trivy on PATH; skipped with a note if
                    it is missing, never fails the build for it.
-  --timeout <s>    Max seconds to wait for the measurement (default 600)
+  --timeout <s>    Max seconds to wait for the measurement, and separately for a
+                   measurement of this project that is already running (default 600)
   --exclude <p>    Exclude a generated path relative to the checkout; repeatable
+  --strict         Fail the step (exit 2) when the run could not be measured.
+                   Off by default (or env SQ_STRICT=true).
   --dry-run        Measure locally and check the package; do not upload (no token needed)
   --json           Print the raw result as JSON
   -h, --help       Show this help
@@ -45,7 +51,17 @@ Pull requests that add a dependency whose license your company policy forbids
 are listed in the log and the job summary; the check fails for it only where
 your organization turned blocking on.
 
-Exit codes: 0 passed (or track-only), 1 below the cutoff or a blocked license, 2 error.`)
+Only a measurement fails the build. When ScaleQuality cannot measure this run
+(unreachable, a server error, a timeout, a failed measurement) or the platform
+refuses it (an invalid or expired token, no active plan, continuous turned off
+for the project), the step reports NOT MEASURED in the log, the job summary and
+--json, never says the gate passed, and exits 0 so your build is not failed by
+it. Pass --strict to fail the step instead.
+
+Exit codes:
+  0  passed, track-only, or NOT MEASURED
+  1  measured: below the cutoff, or a license your organization blocks
+  2  NOT MEASURED with --strict, or --dry-run could not prepare the package`)
   process.exit(0)
 }
 
@@ -59,15 +75,151 @@ const artifacts = [
   ...args.flatMap((arg, i) => (arg === "--artifact" && args[i + 1] ? [args[i + 1]] : [])),
   ...(process.env.SQ_ARTIFACTS ? process.env.SQ_ARTIFACTS.split(",").map((s) => s.trim()).filter(Boolean) : []),
 ]
-const timeoutMs = Math.max(30, Number(flag("--timeout") || 600)) * 1000
+const strict = has("--strict") || /^(1|true|yes)$/i.test(process.env.SQ_STRICT || "")
+// Test-only: shrinks every wait (retries, poll ladder, timeouts) so the suite can walk
+// the slow paths in milliseconds. A pipeline never sets it.
+const timeScale = Number(process.env.SQ_CHECK_TIME_SCALE) > 0 ? Number(process.env.SQ_CHECK_TIME_SCALE) : 1
+const scaled = (ms) => Math.max(1, Math.round(ms * timeScale))
+const timeoutSeconds = Math.max(30, Number(flag("--timeout")) || 600)
+const timeoutMs = scaled(timeoutSeconds * 1000)
 
 const base = `${host}/api/insights/continuous`
 const log = (m) => { if (!jsonOut) console.log(m) }
-const fail = (m) => { console.error(`ScaleQuality: ${m}`); process.exit(2) }
+// Progress that must not land in the --json stdout.
+const note = (m) => (jsonOut ? console.error : console.log)(m)
 const sleep = (ms) => new Promise((r) => setTimeout(r, ms))
 
+/**
+ * A run that ended without a measured verdict. Only a measurement fails the build
+ * (exit 1); everything that kept it from happening exits 0 with a NOT MEASURED
+ * warning, or 2 with --strict, whoever caused it. The origin says who, so the log
+ * points at the right fix:
+ *  - scalequality: our side. Unreachable, a 5xx, a timeout, a failed measurement,
+ *    no score to hold the cutoff against, a protocol the server did not confirm, a
+ *    measurement of the same project that kept running.
+ *  - configuration: what the platform holds for this pipeline (the token, the plan,
+ *    continuous turned on, the project id), or a secret the CI did not hand to the job.
+ *  - checkout: this job's own checkout could not be packaged (Git history, the
+ *    archive limit, a file that changed while it was read).
+ * A token that expired after its 90 days, a plan that lapsed or a measurement
+ * turned off in the platform must not turn every build of the customer red, and a
+ * pull request from a fork or from Dependabot gets no secrets at all.
+ */
+class NotMeasured extends Error {
+  constructor(origin, reason, message, details = {}) {
+    super(message)
+    this.origin = origin
+    this.reason = reason
+    this.details = details
+  }
+}
+const notMeasured = (origin, reason, message, details) => { throw new NotMeasured(origin, reason, message, details) }
+
+/**
+ * GitHub Actions shows it on the run and on the pull request; Azure Pipelines on the
+ * run summary. Both read these commands from stdout, which --json keeps for the result.
+ */
+function annotate(level, title, text) {
+  const write = jsonOut ? console.error : console.log
+  const data = (s) => String(s).replace(/%/g, "%25").replace(/\r/g, "%0D").replace(/\n/g, "%0A")
+  const property = (s) => data(s).replace(/:/g, "%3A").replace(/,/g, "%2C")
+  if (process.env.GITHUB_ACTIONS === "true") write(`::${level} title=${property(title)}::${data(text)}`)
+  else if (process.env.TF_BUILD) write(`##vso[task.logissue type=${level}]${String(text).replace(/[\r\n]+/g, " ")}`)
+}
+
+function finishNotMeasured({ origin, reason, message, details = {} }) {
+  if (dryRun) {
+    console.error(`ScaleQuality: ${message}`)
+    process.exit(2)
+  }
+  const exitCode = strict ? 2 : 0
+  const outcome = strict
+    ? "The gate was not evaluated for this commit. Failing the step because --strict is set."
+    : "The gate was not evaluated for this commit, so this step exits 0: a run ScaleQuality could not measure does not fail your build. Pass --strict to fail it instead."
+  console.error(`ScaleQuality: NOT MEASURED. ${message}`)
+  console.error(`ScaleQuality: ${outcome}`)
+  annotate(strict ? "error" : "warning", "ScaleQuality: not measured", `${message} ${outcome}`)
+  if (process.env.GITHUB_STEP_SUMMARY) {
+    const md = ["### ScaleQuality: not measured", "", message, "", outcome, ""].join("\n")
+    try { appendFileSync(process.env.GITHUB_STEP_SUMMARY, md) } catch { /* the log already says it */ }
+  }
+  if (jsonOut) console.log(JSON.stringify({ measured: false, outcome: "not_measured", origin, reason, message, strict, exitCode, ...details }))
+  process.exit(exitCode)
+}
+
 if (!dryRun && (!token || !project)) {
-  fail("--token and --project are required (or env SQ_TOKEN / SQ_PROJECT). See --help.")
+  finishNotMeasured(new NotMeasured("configuration", "missing_credentials",
+    "--token and --project are required (or env SQ_TOKEN / SQ_PROJECT). On a pull request from a fork or from Dependabot the CI does not hand repository secrets to the job. See --help."))
+}
+
+/** The `error` code ScaleQuality puts in a JSON error body, when there is one. */
+function errorCode(text) {
+  try { return JSON.parse(text)?.error } catch { return typeof text === "string" && text.includes("measurement_already_running") ? "measurement_already_running" : undefined }
+}
+const detail = (text) => {
+  const s = String(text || "").replace(/\s+/g, " ").trim().slice(0, 200)
+  return s ? `: ${s}` : ""
+}
+
+/** The token refusals that are the pipeline's to fix, as opposed to our auth store failing (auth_failed). */
+function tokenRefused(code) {
+  return code === "token_expired"
+    ? ["token_expired", "The SQ token expired. Issue a new one in ScaleQuality, update the SQ_TOKEN secret, then revoke the old token."]
+    : ["invalid_token", "The SQ token was refused (invalid or revoked). Issue a new one in ScaleQuality and update the SQ_TOKEN secret."]
+}
+
+/**
+ * One request with its own deadline: a host that accepts the connection and then
+ * never answers must not hold the job until the CI kills it (a red build again).
+ */
+async function request(url, init, limitMs) {
+  try {
+    const res = await fetch(url, { ...init, signal: AbortSignal.timeout(Math.max(1, limitMs)) })
+    return { res, text: res.ok ? undefined : await res.text().catch(() => "") }
+  } catch (error) {
+    return { error }
+  }
+}
+
+/** What an upload that was not accepted means, and whether waiting can change it. */
+function uploadProblem(status, text) {
+  const code = errorCode(text)
+  if (status === 401 && code === "auth_failed") return { origin: "scalequality", reason: "auth_unavailable", message: "ScaleQuality could not check the token (401 auth_failed).", retry: "transient" }
+  if (status === 401) { const [reason, message] = tokenRefused(code); return { origin: "configuration", reason, message } }
+  if (status === 402) return { origin: "configuration", reason: "subscription_required", message: "Continuous measurement needs an active subscription for this organization." }
+  if (status === 404 && code === "project_not_found") return { origin: "configuration", reason: "project_not_found", message: `Project ${project} was not found in this token's organization. Compare --project with the id in the URL of the repository's diagnosis.` }
+  if (status === 409 && code === "measurement_already_running") return { origin: "scalequality", reason: "measurement_already_running", message: `Another measurement of this project was still running after ${timeoutSeconds}s of waiting for it.`, retry: "busy" }
+  if (status === 409) return { origin: "configuration", reason: "continuous_not_enabled", message: "Continuous measurement is turned off for this project in ScaleQuality. Turn it on there, or remove this step." }
+  if (status === 429 || status >= 500) return { origin: "scalequality", reason: "server_error", message: `ScaleQuality answered ${status}${detail(text)}.`, retry: "transient" }
+  return { origin: "scalequality", reason: "server_rejected", message: `ScaleQuality did not accept the report (${status}${detail(text)}).` }
+}
+
+// A deploy rolling over or a dropped connection lasts seconds; three short retries
+// cover it without hiding a real outage behind minutes of waiting.
+const RETRY_MS = [2000, 5000, 10000]
+// A measurement of the same project already running (another push, a scheduled
+// re-measurement) usually ends in seconds: wait for it instead of skipping this one.
+const BUSY_MS = 10000
+
+async function upload(form, auth) {
+  const busyUntil = Date.now() + timeoutMs
+  let retries = 0
+  for (;;) {
+    const { res, text, error } = await request(`${base}/report/upload`, { method: "POST", headers: auth, body: form }, timeoutMs)
+    if (res?.ok) return res
+    const problem = error
+      ? { origin: "scalequality", reason: "unreachable", message: `Could not reach ${host} (${error.message}).`, retry: "transient" }
+      : uploadProblem(res.status, text)
+    const wait = problem.retry === "busy" ? scaled(BUSY_MS)
+      : problem.retry === "transient" && retries < RETRY_MS.length ? scaled(RETRY_MS[retries++]) : null
+    if (wait == null || (problem.retry === "busy" && Date.now() + wait > busyUntil)) {
+      notMeasured(problem.origin, problem.reason, problem.message, res ? { httpStatus: res.status } : {})
+    }
+    note(problem.retry === "busy"
+      ? "ScaleQuality: another measurement of this project is running; waiting for it to finish before sending this one."
+      : `ScaleQuality: ${problem.message} Trying again in ${Math.round(wait / 1000)}s.`)
+    await sleep(wait)
+  }
 }
 
 /**
@@ -216,21 +368,41 @@ function reportLicenses(license) {
 
 async function main() {
   const auth = { Authorization: `Bearer ${token}` }
-  const excludes = normalizeExcludes(args.flatMap((arg, i) => arg === "--exclude" ? [args[i + 1]] : []))
-  const durabilityReport = await measureDurability({ root: process.cwd(), log: jsonOut ? (message) => console.error(message) : log })
-  if (!durabilityReport) fail("Git durability collection failed. No upload was sent and no new score was submitted. Resolve the Git error above before rerunning this step.")
-  const coverage = await stageCoverageReports({ log })
-  let packed, buf
+  let excludes
+  try {
+    excludes = normalizeExcludes(args.flatMap((arg, i) => arg === "--exclude" ? [args[i + 1]] : []))
+  } catch (error) {
+    notMeasured("configuration", "invalid_option", error?.message || String(error))
+  }
+  // The collector logs why it stopped; the warning and the job summary carry it too,
+  // since an annotation is read without the log line above it.
+  let durabilityProblem
+  const durabilityLog = (message) => {
+    const stopped = /^ScaleQuality: Git durability not measured \u2014 (.+?)\. No Git history was uploaded\.$/.exec(message)
+    if (stopped) durabilityProblem = stopped[1]
+    ;(jsonOut ? console.error : log)(message)
+  }
+  const durabilityReport = await measureDurability({ root: process.cwd(), log: durabilityLog })
+  if (!durabilityReport) {
+    const why = durabilityProblem ? ` (${durabilityProblem})` : ""
+    const shallow = /shallow/i.test(durabilityProblem || "") ? " Check the repository out with full history: fetch-depth 0, GIT_DEPTH 0, clone depth full or fetchDepth 0, depending on the CI." : ""
+    notMeasured("checkout", "durability_not_collected", `Git durability collection failed${why}. No upload was sent and no new score was submitted.${shallow || " Resolve the Git error before rerunning this step."}`)
+  }
+  let coverage, packed, buf
   try {
+    coverage = await stageCoverageReports({ log })
     packed = prepareCheckoutArchive({ root: process.cwd(), coverageRoot: coverage.root, durabilityReport, excludes, log })
     if (durabilityReport) {
       const head = spawnSync("git", ["--no-optional-locks", "-C", process.cwd(), "rev-parse", "--verify", "HEAD"], { encoding: "utf8", timeout: 10000 })
-      if (head.error || head.status !== 0 || head.stdout.trim() !== durabilityReport.headSha) throw new Error("Git HEAD changed during measurement or packaging. No upload was sent; rerun after the checkout is stable.")
+      if (head.error || head.status !== 0 || head.stdout.trim() !== durabilityReport.headSha) notMeasured("checkout", "head_changed", "Git HEAD changed during measurement or packaging. No upload was sent; rerun after the checkout is stable.")
     }
     if (!dryRun) buf = readFileSync(packed.archivePath)
+  } catch (error) {
+    if (error instanceof NotMeasured) throw error
+    notMeasured("checkout", "checkout_not_packaged", error?.message || String(error))
   } finally {
     packed?.cleanup()
-    coverage.cleanup()
+    coverage?.cleanup()
   }
   const localCheckout = { version: 1, history: packed.history, archiveBytes: packed.bytes, durability: durabilityReport ? { source: "ci", windowDays: 90, settleDays: 14, headSha: durabilityReport.headSha, measuredAt: durabilityReport.measuredAt, engineCommit: durabilityReport.engineCommit } : undefined }
   const limitation = "ScaleQuality: historical secret scanning and commit-convention signals are not included in this checkout report."
@@ -259,20 +431,11 @@ async function main() {
   // Which files the change touched: the other modules of a monorepo it reached are measured again too.
   const touched = changedFiles()
   if (touched) form.append("ciChangedFiles", touched.join("\n"))
-  const r = await fetch(`${base}/report/upload`, {
-    method: "POST",
-    headers: auth,
-    body: form,
-  }).catch((e) => fail(`could not reach ${host} (${e.message})`))
-  if (!r.ok) {
-    const body = await r.text().catch(() => "")
-    if (r.status === 401) fail("invalid token.")
-    if (r.status === 402) fail("continuous measurement needs an active subscription.")
-    if (r.status === 409) fail(body.includes("measurement_already_running") ? "another measurement is already running for this project. Wait for it to finish and rerun this step." : "continuous is not enabled for this project. Turn it on in ScaleQuality.")
-    fail(`report failed (${r.status}) ${body}`)
-  }
-  const { runId, modules } = await r.json()
-  if (!runId) fail("no run id returned.")
+  const r = await upload(form, auth)
+  const accepted = await r.json().catch(() => null)
+  const runId = accepted?.runId
+  const modules = accepted?.modules
+  if (!runId) notMeasured("scalequality", "no_run_id", "ScaleQuality accepted the report but returned no measurement id.")
   log(`ScaleQuality: measuring ${project} …`)
   if (Array.isArray(modules) && modules.length) log(`ScaleQuality: also measuring the ${modules.length === 1 ? "module" : `${modules.length} modules`} this change touched: ${modules.map((m) => m.path || "whole repository").join(", ")}`)
 
@@ -290,39 +453,68 @@ async function main() {
   const POLL_MS = [1200, 1200, 900, 700, 700, 700, 700, 700, 1000, 1500, 2000, 3000]
   const deadline = Date.now() + timeoutMs
   let last
+  // A status read that fails is retried on the next tick until the deadline: one
+  // dropped read must not cost a measurement that is about to finish.
+  let lastProblem
   let poll = 0
   while (Date.now() < deadline) {
-    await sleep(POLL_MS[Math.min(poll++, POLL_MS.length - 1)])
-    const sr = await fetch(`${base}/report/${runId}`, { headers: auth }).catch((e) => fail(`status check failed (${e.message})`))
-    if (!sr.ok) { const b = await sr.text().catch(() => ""); fail(`status failed (${sr.status}) ${b}`) }
-    last = await sr.json()
+    await sleep(scaled(POLL_MS[Math.min(poll++, POLL_MS.length - 1)]))
+    const { res, text, error } = await request(`${base}/report/${runId}`, { headers: auth }, scaled(30000))
+    if (error) { lastProblem = `could not reach ${host} (${error.message})`; continue }
+    if (!res.ok) {
+      const code = errorCode(text)
+      if (res.status === 401 && code !== "auth_failed") { const [reason, message] = tokenRefused(code); notMeasured("configuration", reason, message, { runId, httpStatus: 401 }) }
+      if (res.status === 404) notMeasured("scalequality", "run_not_found", `ScaleQuality no longer finds measurement ${runId}.`, { runId, httpStatus: 404 })
+      lastProblem = `ScaleQuality answered ${res.status}${detail(text)}`
+      continue
+    }
+    const read = await res.json().catch(() => null)
+    if (!read || typeof read !== "object") { lastProblem = "ScaleQuality answered with an unreadable status"; continue }
+    last = read
+    lastProblem = undefined
     if (last.status === "COMPLETED" || last.status === "FAILED") break
     if (!jsonOut) process.stdout.write(".")
   }
   if (!jsonOut) process.stdout.write("\n")
 
-  if (!last || (last.status !== "COMPLETED" && last.status !== "FAILED")) fail("timed out waiting for the measurement.")
-  if (last.status === "FAILED") fail(`measurement failed: ${last.failureReason || "unknown"}`)
-  if (last.checkout?.version !== 1) fail("the server did not confirm the checkout report protocol. No passing gate is claimed; update to the compatible continuous-measurement server release.")
+  if (!last || (last.status !== "COMPLETED" && last.status !== "FAILED")) {
+    if (lastProblem) notMeasured("scalequality", "status_unavailable", `Could not read the measurement status before the ${timeoutSeconds}s timeout (last attempt: ${lastProblem}).`, { runId })
+    notMeasured("scalequality", "timeout", `Timed out after ${timeoutSeconds}s waiting for measurement ${runId}. Raise --timeout if this repository needs longer.`, { runId, status: last?.status ?? null })
+  }
+  if (last.status === "FAILED") notMeasured("scalequality", "measurement_failed", `The measurement failed on ScaleQuality's side (${last.failureReason || "unknown"}).`, { runId, failureReason: last.failureReason ?? null })
+  if (last.checkout?.version !== 1) notMeasured("scalequality", "protocol_unconfirmed", "The server did not confirm the checkout report protocol. No passing gate is claimed; the server needs the compatible continuous-measurement release.", { runId })
   if (durabilityReport && (last.checkout?.durability?.source !== "ci" || last.checkout.durability.windowDays !== 90 || last.checkout.durability.settleDays !== 14 || last.checkout.durability.headSha !== durabilityReport.headSha || last.checkout.durability.measuredAt !== durabilityReport.measuredAt || last.checkout.durability.engineCommit !== durabilityReport.engineCommit)) {
-    fail("the server did not confirm this checkout's 90-day durability report. No passing gate is claimed; the server needs the compatible continuous-measurement release.")
+    notMeasured("scalequality", "durability_unconfirmed", "The server did not confirm this checkout's 90-day durability report. No passing gate is claimed; the server needs the compatible continuous-measurement release.", { runId })
+  }
+
+  // Exit 1 only for what the measurement said: below the cutoff, or a license the
+  // organization blocks. A completed run with no score to hold the cutoff against
+  // (gateResult UNKNOWN) or with no verdict at all was not measured, and says so.
+  const licenseBlocked = last.license?.result === "BLOCKED"
+  const belowCutoff = last.gateResult === "FAILED"
+    || (typeof last.maturity === "number" && typeof last.cutoff === "number" && last.maturity < last.cutoff)
+  const gateFailed = last.passed === false && (belowCutoff || licenseBlocked)
+  if (typeof last.passed !== "boolean" || (last.passed === false && !gateFailed)) {
+    if (!jsonOut) reportLicenses(last.license)
+    notMeasured("scalequality", "no_verdict", last.cutoff != null && last.maturity == null
+      ? `The measurement finished without a maturity score, so the cutoff ${last.cutoff} could not be evaluated.`
+      : `The measurement finished without a gate verdict (${last.gateResult ?? "UNKNOWN"}).`, { runId, gateResult: last.gateResult ?? null })
   }
 
   if (jsonOut) {
-    console.log(JSON.stringify(last))
+    console.log(JSON.stringify({ ...last, measured: true, outcome: gateFailed ? "gate_failed" : last.cutoff == null ? "track_only" : "passed" }))
   } else {
     const cut = last.cutoff != null ? ` (cutoff ${last.cutoff})` : " (track only)"
     log(`ScaleQuality: maturity ${last.maturity ?? "—"}${cut}`)
     reportLicenses(last.license)
-    const licenseBlocked = last.license?.result === "BLOCKED"
-    const maturityFailed = last.gateResult === "FAILED"
-    if (last.passed === false && maturityFailed) console.error("ScaleQuality: below the line — failing the build.")
-    else if (last.passed === false && licenseBlocked) console.error("ScaleQuality: a forbidden license, failing the build.")
-    else if (last.passed === false) console.error("ScaleQuality: the gate did not pass, failing the build.")
+    if (gateFailed && belowCutoff) console.error("ScaleQuality: below the line, failing the build.")
+    else if (gateFailed) console.error("ScaleQuality: a forbidden license, failing the build.")
     else if (last.cutoff == null) log("ScaleQuality: measurement completed (track only; no cutoff configured).")
     else log("ScaleQuality: passed.")
   }
-  process.exit(last.passed === false ? 1 : 0)
+  process.exit(gateFailed ? 1 : 0)
 }
 
-main().catch((e) => fail(e?.message || String(e)))
+main().catch((error) => finishNotMeasured(error instanceof NotMeasured
+  ? error
+  : new NotMeasured("scalequality", "check_error", `The check stopped unexpectedly (${error?.message || String(error)}).`)))

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@scalequality/check",
-  "version": "0.5.5",
-  "description": "ScaleQuality continuous code-maturity check for CI. Re-measures your repo on every build and fails the pipeline below your maturity line.",
+  "version": "0.5.6",
+  "description": "ScaleQuality continuous code-maturity check for CI. Re-measures your repo on every build and fails the pipeline below your maturity line, never for a run it could not measure.",
   "type": "module",
   "bin": {
     "check": "bin/check.mjs",