@scalequality/check 0.5.0 → 0.5.2

package/README.md

@@ -87,15 +87,15 @@ source, coverage, and a bounded durability report (at most 1 MiB); it never uplo
 the raw `.git` object database. Years of old binary history therefore do not add
 to the archive size. It does not truncate or change your repository.
 
-Use a complete checkout for that measurement. Shallow history is reported as
-unavailable; it is not treated as a full 90-day measurement.
+Use a complete checkout for that measurement. A shallow or incomplete checkout
+is rejected locally before upload; it is not treated as a full 90-day measurement.
 
 ```yaml
 # GitHub Actions
 - uses: actions/checkout@v4
   with:
     fetch-depth: 0
-- run: npx @scalequality/check@0.5.0 --project <projectId>
+- run: npx @scalequality/check@0.5.2 --project <projectId>
   env:
     SQ_TOKEN: ${{ secrets.SQ_TOKEN }}
 ```
@@ -109,8 +109,8 @@ clone:
 
 The current-code scanners still run on the server. Historical secret scanning and
 commit-convention analysis are not included in this report; the result shows those
-limits and identifies durability as reported by CI. CPU, time, malformed Git or
-report-size limits yield unavailable history, never a fabricated clean result.
+limits and identifies durability as reported by CI. A failed Git collection or
+invalid report stops the CLI before upload, without submitting a replacement score.
 
 Version 0.5.0 needs the compatible ScaleQuality server release. The CLI checks the
 server's acknowledgement of this exact report before declaring a passing gate.
@@ -126,9 +126,68 @@ that you expect to measure.
 
 ```bash
 # Local verification; does not need a token and sends nothing to ScaleQuality.
-npx @scalequality/check@0.5.0 --dry-run
+npx @scalequality/check@0.5.2 --dry-run
 ```
 
 The CLI checks compressed size before allocating the upload buffer or sending a
 request. If current source alone cannot fit, it lists the largest included files
 and exits with an actionable error. No oversized upload is sent.
+
+
+## 0.5.1: Composer dependencies and large Git blame output
+
+The CLI recognizes installed Composer dependencies using `composer.json` and a
+valid `composer.lock`. It excludes the default `vendor/`, or a safe relative
+`config.vendor-dir`, while preserving both files for dependency analysis. Nested
+Composer projects are recognized too. A directory named `vendor` alone is not
+sufficient evidence to exclude it. Ambiguous configuration, source paths that
+overlap the dependency directory, and unavailable local package origins keep the
+directory in the archive. The CLI reports which directories it excluded or kept.
+It reads JSON only; it does not run Composer, plugins or scripts.
+
+`node_modules/` and the existing generated-output directories remain excluded.
+Coverage reports from application tests are staged separately before packaging.
+Excluding installed dependencies does not certify that their locked versions are
+free of vulnerabilities.
+
+Git durability reads incremental blame ranges as a stream instead of buffering
+all per-line metadata and source content. This preserves the native calculation,
+90-day window, 14-day settlement and tracked-file selection. It handles large
+blame output without increasing the worker's memory or time budgets.
+
+Archive exclusions, including `--exclude`, do not change Git durability's tracked
+file scope. A tracked dependency can still contribute to the historical metric.
+Changing that methodology requires a separate coordinated version; this patch
+does not silently rewrite prior scores. Coverage may remain absent when the job
+does not generate it; Git collection failures stop the check before upload.
+
+Track-only completion now says `measurement completed (track only; no cutoff
+configured)`. Its exit code remains zero; completion does not certify that every
+dimension was measured or that a maturity threshold was passed.
+
+The 60 MiB compressed-source limit still applies. A checkout with more than that
+amount of first-party source or ambiguous generated assets is rejected locally;
+this release does not truncate source to force a successful upload.
+
+
+## 0.5.2: complete Git collection before submitting a score
+
+The complete HEAD object inventory now streams through the same bounded reader
+as blame. Large old histories no longer have to fit into a 16 MiB output buffer
+for that check. Missing objects remain errors. The 90-day window, 14-day settlement,
+scoring calculations and archive limit are unchanged.
+
+Unexpected Git command failures stop collection; they cannot become zero surviving
+lines or an upload with durability omitted. A temporary OS `EAGAIN` failure to
+start Git gets one automatic retry within the existing measurement deadline.
+Timeouts, missing or corrupt history, and resource-limit failures do not trigger
+unbounded retries. A technical failure exits with code 2 and sends no new score.
+
+Tracked blob names are preserved exactly, including Git-quoted characters.
+Submodule gitlinks are not source blobs and are not sent to blame; ordinary files,
+symlinks and binary blobs retain their existing measurement treatment.
+
+A valid report with zero settled or AI-attributed lines still proceeds. It does
+not invent 100% AI survival. Missing coverage alone does not block submission.
+A completed check below a configured maturity cutoff still exits with code 1;
+track-only completion still exits with code 0.

package/bin/archive.mjs

@@ -2,6 +2,7 @@ import { spawnSync } from "node:child_process"
 import { mkdtempSync, mkdirSync, readdirSync, lstatSync, existsSync, statSync, writeFileSync, rmSync } from "node:fs"
 import { tmpdir } from "node:os"
 import { join, resolve, posix } from "node:path"
+import { planComposerExclusions } from "./composer-vendor.mjs"
 
 export const ARCHIVE_LIMIT_BYTES = 60 * 1024 * 1024
 const MAX_DURABILITY_REPORT_BYTES = 1024 * 1024
@@ -23,8 +24,12 @@ export function normalizeExcludes(excludes = []) {
   })
 }
 
-function inventory(root, { reports = false, excludes = [] } = {}) {
+function inventory(root, { reports = false, excludes = [], log = () => {} } = {}) {
   const paths = [], largest = []
+  const composerExcluded = reports ? [] : planComposerExclusions(root, excludes, SKIP, log)
+  for (const candidate of composerExcluded) {
+    log(`ScaleQuality: excluding installed Composer dependencies at ${JSON.stringify(candidate.path)}; retaining ${JSON.stringify(candidate.manifest)} and ${JSON.stringify(candidate.lockfile)} for dependency analysis. Git durability scope is unchanged.`)
+  }
   const walk = (rel) => {
     const abs = join(root, rel)
     const st = lstatSync(abs)
@@ -36,12 +41,12 @@ function inventory(root, { reports = false, excludes = [] } = {}) {
     }
   }
   const visit = (rel, name) => {
-    if (!reports && (name === ".git" || rel === ".sqreports" || excludes.some((p) => rel === p || rel.startsWith(`${p}/`)))) return
+    if (!reports && (name === ".git" || rel === ".sqreports" || [...excludes, ...composerExcluded.map(item => item.path)].some((p) => rel === p || rel.startsWith(`${p}/`)))) return
     if (!reports && SKIP.has(name) && lstatSync(join(root, rel)).isDirectory()) return
     walk(rel)
   }
   for (const entry of readdirSync(root).sort()) visit(entry, entry)
-  return { paths, largest: largest.sort((a, b) => b.bytes - a.bytes).slice(0, 5) }
+  return { paths, largest: largest.sort((a, b) => b.bytes - a.bytes).slice(0, 5), composerExcluded }
 }
 
 /**
@@ -58,7 +63,7 @@ export function prepareCheckoutArchive({ root, coverageRoot, durabilityReport, l
   process.once("exit", onExit)
   const cleanup = () => { process.removeListener("exit", onExit); rmSync(temp, { recursive: true, force: true }) }
   try {
-    const source = inventory(root, { excludes })
+    const source = inventory(root, { excludes, log })
     const sourceList = join(temp, "source.list")
     writeFileSync(sourceList, source.paths.length ? `${source.paths.join("\0")}\0` : "")
     const metadataRoot = join(temp, "metadata")
@@ -66,6 +71,7 @@ export function prepareCheckoutArchive({ root, coverageRoot, durabilityReport, l
     const hasGit = existsSync(join(root, ".git"))
     const history = { mode: hasGit ? "omitted" : "unavailable", reason: durabilityReport ? "ci_report" : hasGit ? "invalid_git" : "missing_git" }
     writeFileSync(join(metadataRoot, ".sqreports/checkout.json"), JSON.stringify({ version: 1, history, archiveLimitBytes: ARCHIVE_LIMIT_BYTES }))
+    if (source.composerExcluded.length) writeFileSync(join(metadataRoot, ".sqreports/source.json"), JSON.stringify({ version: 1, composerDependencyExclusions: source.composerExcluded }))
     if (durabilityReport) {
       const json = JSON.stringify(durabilityReport)
       if (Buffer.byteLength(json) > MAX_DURABILITY_REPORT_BYTES) throw new Error("Durability report exceeds the 1 MiB report limit. No upload was sent.")

package/bin/check.mjs

@@ -216,6 +216,7 @@ async function main() {
   const auth = { Authorization: `Bearer ${token}` }
   const excludes = normalizeExcludes(args.flatMap((arg, i) => arg === "--exclude" ? [args[i + 1]] : []))
   const durabilityReport = await measureDurability({ root: process.cwd(), log: jsonOut ? (message) => console.error(message) : log })
+  if (!durabilityReport) fail("Git durability collection failed. No upload was sent and no new score was submitted. Resolve the Git error above before rerunning this step.")
   const coverage = stageCoverageReports()
   let packed, buf
   try {
@@ -298,6 +299,7 @@ async function main() {
     const cut = last.cutoff != null ? ` (cutoff ${last.cutoff})` : " (track only)"
     log(`ScaleQuality: maturity ${last.maturity ?? "—"}${cut}`)
     if (last.passed === false) console.error("ScaleQuality: below the line — failing the build.")
+    else if (last.cutoff == null) log("ScaleQuality: measurement completed (track only; no cutoff configured).")
     else log("ScaleQuality: passed.")
   }
   process.exit(last.passed === false ? 1 : 0)

package/bin/composer-vendor.mjs

@@ -0,0 +1,198 @@
+import { constants, closeSync, fstatSync, lstatSync, openSync, readSync, readdirSync } from 'node:fs'
+import { join, posix } from 'node:path'
+
+const MAX_JSON_BYTES = 4 * 1024 * 1024
+const object = value => value !== null && typeof value === 'object' && !Array.isArray(value)
+const overlaps = (a, b) => a === b || a.startsWith(`${b}/`) || b.startsWith(`${a}/`) || a === '.' || b === '.'
+
+// Inspect data only. Composer plugins/scripts and customer symlinks are never run/followed.
+function readJson(path) {
+  let fd
+  try {
+    fd = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK)
+    const stat = fstatSync(fd)
+    if (!stat.isFile() || stat.size > MAX_JSON_BYTES) return null
+    const raw = Buffer.alloc(stat.size + 1)
+    let bytes = 0
+    while (bytes < raw.length) {
+      const count = readSync(fd, raw, bytes, raw.length - bytes, null)
+      if (!count) break
+      bytes += count
+    }
+    if (bytes !== stat.size) return null
+    const value = JSON.parse(raw.subarray(0, bytes).toString('utf8'))
+    return object(value) ? value : null
+  } catch { return null }
+  finally { if (fd !== undefined) closeSync(fd) }
+}
+
+function safeRelative(value) {
+  if (typeof value !== 'string' || !value || /[\\\0*?\[\]$~]/.test(value) || value.startsWith('/') || /^[A-Za-z]:/.test(value)) return null
+  if (value.split('/').includes('..')) return null
+  const path = posix.normalize(value).replace(/\/$/, '')
+  return path === '.' || path.split('/').some(part => ['.git', '.sqreports'].includes(part)) ? null : path
+}
+
+function sourcePath(module, value) {
+  if (typeof value !== 'string' || /[\\\0$~]/.test(value) || value.startsWith('/') || /^[A-Za-z]:/.test(value)) return null
+  // A glob's fixed directory prefix bounds every possible source it might select.
+  const glob = value.search(/[*?\[]/)
+  const prefix = glob < 0 ? value : value.slice(0, glob).replace(/[^/]*$/, '')
+  const path = posix.normalize(posix.join(module || '.', prefix || '.')).replace(/\/$/, '')
+  return path === '..' || path.startsWith('../') ? null : path
+}
+
+function autoloadPaths(value) {
+  if (value === undefined) return []
+  if (!object(value)) return [null]
+  const paths = []
+  for (const key of ['psr-4', 'psr-0', 'classmap', 'files']) {
+    const entry = value[key]
+    if (entry === undefined) continue
+    const values = ['psr-4', 'psr-0'].includes(key) ? object(entry) ? Object.values(entry) : [null] : Array.isArray(entry) ? entry : [null]
+    for (const item of values) paths.push(...(Array.isArray(item) ? item : [item]))
+  }
+  return paths
+}
+
+function regularDirectory(root, path, consume = () => {}) {
+  let current = root
+  for (const part of path.split('/')) {
+    consume()
+    current = join(current, part)
+    try { if (!lstatSync(current).isDirectory()) return false }
+    catch { return false }
+  }
+  return true
+}
+
+/** A transport exclusion, never a change to the native Git durability scope. */
+export function composerVendorDirectory(root, module = '', excludes = [], ignoredDirectories = new Set(), consume = () => {}) {
+  const manifest = readJson(join(root, module, 'composer.json'))
+  if (!manifest) return null
+  const label = posix.join(module, 'composer.json')
+  const keep = reason => ({ kept: true, manifest: label, reason })
+  const lock = readJson(join(root, module, 'composer.lock'))
+  if (!lock || !Array.isArray(lock.packages) || (lock['packages-dev'] !== undefined && !Array.isArray(lock['packages-dev']))) return keep('a valid composer.lock is required')
+  const packages = [...lock.packages, ...(lock['packages-dev'] || [])]
+  if (!packages.length || packages.some(pkg => !object(pkg) || typeof pkg.name !== 'string' || !pkg.name || typeof pkg.version !== 'string' || !pkg.version)) return keep('locked dependency versions could not be confirmed')
+  if (manifest.config !== undefined && !object(manifest.config)) return keep('Composer config is ambiguous')
+  const vendor = safeRelative(manifest.config && Object.hasOwn(manifest.config, 'vendor-dir') ? manifest.config['vendor-dir'] : 'vendor')
+  if (!vendor) return keep('vendor-dir must be a literal directory within the Composer project')
+  const path = posix.join(module, vendor)
+  // If either evidence file is explicitly excluded, do not infer a safe replacement for installed dependencies.
+  if (['composer.json', 'composer.lock'].some(file => excludes.some(ex => overlaps(ex, posix.join(module, file))))) return keep('the manifest or lockfile was explicitly excluded')
+
+  const repositories = manifest.repositories === undefined ? [] : Array.isArray(manifest.repositories) ? manifest.repositories : object(manifest.repositories) ? Object.values(manifest.repositories) : [null]
+  if (repositories.some(repo => repo !== false && !object(repo))) return keep('Composer repositories are ambiguous')
+  const localOrigins = [
+    ...repositories.filter(repo => repo && repo.type === 'path').map(repo => repo.url),
+    ...packages.filter(pkg => pkg.dist?.type === 'path').map(pkg => pkg.dist.url),
+  ]
+  // Resolving Composer's glob rules and each symlink target is outside this transport hotfix.
+  if (localOrigins.some(value => typeof value !== 'string' || /[*?\[]/.test(value))) return keep('local package origins use ambiguous paths or globs')
+  const sources = [...autoloadPaths(manifest.autoload), ...autoloadPaths(manifest['autoload-dev']), ...localOrigins]
+  if (sources.some(value => { consume(); const source = sourcePath(module, value); return source === null || overlaps(path, source) })) return keep('local source paths may overlap installed dependencies or lie outside the checkout')
+  if (localOrigins.some(value => {
+    const source = sourcePath(module, value)
+    consume()
+    return !regularDirectory(root, source, consume) || source.split('/').some(part => ignoredDirectories.has(part) || part === 'vendor') || excludes.some(ex => overlaps(ex, source))
+  })) return keep('a local package origin is missing, linked or excluded from the archive')
+  // A custom vendor-dir can have several components: reject a symlink in ANY component.
+  if (!regularDirectory(root, path, consume)) return null
+  return { path, manifest: label, lockfile: posix.join(module, 'composer.lock') }
+}
+
+function declaredSources(root, module, consume) {
+  const manifest = readJson(join(root, module, 'composer.json'))
+  if (!manifest) return [null]
+  const repositories = manifest.repositories === undefined ? [] : Array.isArray(manifest.repositories) ? manifest.repositories : object(manifest.repositories) ? Object.values(manifest.repositories) : [null]
+  if (repositories.some(repo => repo !== false && !object(repo))) return [null]
+  const lock = readJson(join(root, module, 'composer.lock'))
+  let packages = []
+  try {
+    lstatSync(join(root, module, 'composer.lock'))
+    if (!lock || !Array.isArray(lock.packages) || (lock['packages-dev'] !== undefined && !Array.isArray(lock['packages-dev']))) return [null]
+    packages = [...lock.packages, ...(lock['packages-dev'] || [])]
+    if (packages.some(pkg => !object(pkg))) return [null]
+  } catch (error) { if (error.code !== 'ENOENT') return [null] }
+  return [
+    ...autoloadPaths(manifest.autoload), ...autoloadPaths(manifest['autoload-dev']),
+    ...repositories.filter(repo => repo && repo.type === 'path').map(repo => repo.url),
+    ...packages.filter(pkg => pkg.dist?.type === 'path').map(pkg => pkg.dist.url),
+  ].map(value => {
+    consume()
+    // Resolving a glob or symlink could reveal another project's only source copy.
+    if (typeof value !== 'string' || /[*?\[]/.test(value)) return null
+    const path = sourcePath(module, value)
+    if (path === null) return null
+    let current = root
+    for (const part of path.split('/')) {
+      consume()
+      current = join(current, part)
+      try { if (lstatSync(current).isSymbolicLink()) return null }
+      catch (error) { if (error.code === 'ENOENT') break; return null }
+    }
+    return path
+  })
+}
+
+/** Discover all projects before applying any Composer exclusion. References can cross modules. */
+export function planComposerExclusions(root, excludes = [], ignoredDirectories = new Set(), log = () => {}) {
+  const limit = new Error('Composer discovery resource limit')
+  const deadline = Date.now() + 5_000
+  let work = 0
+  const consume = () => { if (++work > 1_000_000 || Date.now() > deadline) throw limit }
+  try {
+    const projects = []
+    const pending = ['']
+    let visited = 0
+    while (pending.length) {
+      consume()
+      const module = pending.pop()
+      const entries = readdirSync(join(root, module), { withFileTypes: true })
+      visited += entries.length
+      if (visited > 100_000 || projects.length > 5_000) throw limit
+      if (entries.some(entry => entry.name === 'composer.json')) projects.push({
+        module,
+        candidate: composerVendorDirectory(root, module, excludes, ignoredDirectories, consume),
+        sources: declaredSources(root, module, consume),
+      })
+      for (const entry of entries.sort((a, b) => b.name.localeCompare(a.name))) {
+        consume()
+        const path = posix.join(module, entry.name)
+        if (!entry.isDirectory() || entry.name === '.git' || path === '.sqreports' || ignoredDirectories.has(entry.name) || excludes.some(ex => path === ex || path.startsWith(`${ex}/`))) continue
+        pending.push(path)
+      }
+    }
+    const candidates = projects.flatMap(project => project.candidate?.path ? [project.candidate] : [])
+    const protectedPaths = new Set()
+    const isActive = project => !candidates.some(candidate => { consume(); return !protectedPaths.has(candidate.path) && (project.module === candidate.path || project.module.startsWith(`${candidate.path}/`)) })
+    // Dependency package manifests do not veto their containing vendor directory.
+    // Once an external project references it, it is retained and its declarations
+    // become relevant too. Iterate until no newly retained project protects another.
+    let changed
+    do {
+      changed = false
+      const sources = projects.filter(isActive).flatMap(project => project.sources)
+      for (const candidate of candidates) {
+        if (!protectedPaths.has(candidate.path) && sources.some(source => { consume(); return source === null || overlaps(candidate.path, source) })) {
+          protectedPaths.add(candidate.path)
+          changed = true
+        }
+      }
+    } while (changed)
+    for (const project of projects.filter(isActive)) {
+      if (project.candidate?.kept) log(`ScaleQuality: retaining possible Composer dependencies near ${JSON.stringify(project.candidate.manifest)}: ${project.candidate.reason}.`)
+    }
+    for (const candidate of candidates) {
+      if (protectedPaths.has(candidate.path)) log(`ScaleQuality: retaining ${JSON.stringify(candidate.path)}: a project declares source there or its source paths could not be resolved safely.`)
+    }
+    // Nested installed package manifests cannot create independent upload exclusions.
+    return projects.filter(isActive).flatMap(project => project.candidate?.path && !protectedPaths.has(project.candidate.path) ? [project.candidate] : [])
+  } catch (error) {
+    if (error !== limit) throw error
+    log('ScaleQuality: retaining Composer dependency directories: project discovery exceeded its resource limit.')
+    return []
+  }
+}

package/bin/durability/GitReader.mjs

@@ -1,7 +1,27 @@
-import { runGit as git, isFatalGitError } from "./runtime.mjs";
+import { runGit as git, runGitLines, GitMeasurementError } from "./runtime.mjs";
 const US = "\x1f"; // unit separator (field)
 const RS = "\x1e"; // record separator (commit)
 const CO_AUTHOR_RE = /co-authored-by:\s*(.*?)\s*<([^>]+)>/gi;
+/** `git log --numstat` C-quotes path bytes, independently of the UTF-8 source.
+ * Decode only this transport representation; leave rename/statistics semantics
+ * unchanged. `ls-tree -z` and blame both use the resulting literal pathname. */
+function decodeGitPath(path) {
+    if (!path.startsWith('"') || !path.endsWith('"')) return path;
+    const parts = [];
+    let literal = '';
+    const escapes = { a: 7, b: 8, t: 9, n: 10, v: 11, f: 12, r: 13, '\\': 92, '"': 34 };
+    for (let i = 1; i < path.length - 1; i++) {
+        if (path[i] !== '\\') { literal += path[i]; continue; }
+        parts.push(Buffer.from(literal)); literal = '';
+        const escaped = path[++i];
+        if (Object.hasOwn(escapes, escaped)) parts.push(Buffer.from([escapes[escaped]]));
+        else if (/^[0-7]{3}$/.test(path.slice(i, i + 3))) {
+            parts.push(Buffer.from([parseInt(path.slice(i, i + 3), 8)])); i += 2;
+        } else throw new GitMeasurementError('Git file path could not be parsed');
+    }
+    parts.push(Buffer.from(literal));
+    return Buffer.concat(parts).toString('utf8');
+}
 function parseCoAuthors(body) {
     const out = [];
     let m;
@@ -77,42 +97,48 @@ export async function readCommits(repoPath, sinceDays, untilDays = 0) {
             const del = Number.parseInt(cols[1], 10);
             rec.additions += Number.isFinite(add) ? add : 0;
             rec.deletions += Number.isFinite(del) ? del : 0;
-            rec.files.push(cols.slice(2).join("\t"));
+            rec.files.push(decodeGitPath(cols.slice(2).join("\t")));
         }
     }
     return [...bySha.values()];
 }
-/** Files that currently exist at HEAD (tracked). */
+/** Blameable entries currently at HEAD. Gitlinks point to another repository;
+ * native blame already contributed zero for them. Keep all blobs, including
+ * symlinks and binaries, without changing the commit/additions denominator. */
 export async function listHeadFiles(repoPath) {
-    const out = await git(repoPath, ["ls-tree", "-r", "--name-only", "HEAD"]);
-    return new Set(out.split("\n").map((l) => l.trim()).filter(Boolean));
+    const out = await git(repoPath, ["ls-tree", "-r", "-z", "HEAD"]);
+    const files = new Set();
+    for (const line of out.split("\0")) {
+        if (!line) continue;
+        const entry = /^[0-7]{6} (blob|tree|commit) [0-9a-f]{40}\t([\s\S]+)$/.exec(line);
+        if (!entry) throw new GitMeasurementError('Git tree output could not be parsed');
+        if (entry[1] === 'blob') files.add(entry[2]);
+    }
+    return files;
 }
-const SHA_LINE = /^([0-9a-f]{40}) /;
+const BLAME_GROUP = /^([0-9a-f]{40}) [1-9][0-9]* [1-9][0-9]* ([1-9][0-9]*)$/;
 /**
  * Blames a file at HEAD and returns, per introducing commit sha, how many
- * CURRENT lines it owns (--line-porcelain → one sha header per line). Used
- * to compute "lines introduced in window that still survive". Returns an
- * empty map on any blame failure (deleted/binary/huge file).
+ * CURRENT lines it owns. Incremental output describes identical attribution
+ * in groups, without source lines or repeated per-line commit metadata.
+ * Stream those groups so large output does not require a large buffer.
+ * Callers select files present at HEAD. A failed command is incomplete evidence,
+ * never proof that this file has no surviving lines.
  */
 export async function blameFile(repoPath, file) {
     const counts = new Map();
-    try {
-        const out = await git(repoPath, [
-            "blame",
-            "--line-porcelain",
-            "HEAD",
-            "--",
-            file,
-        ]);
-        for (const line of out.split("\n")) {
-            const m = SHA_LINE.exec(line);
-            if (m)
-                counts.set(m[1], (counts.get(m[1]) ?? 0) + 1);
-        }
-    }
-    catch (error) {
-        if (isFatalGitError(error)) throw error;
-        // deleted at HEAD / binary / unreadable → no surviving lines counted
-    }
+    await runGitLines(repoPath, [
+        "blame",
+        "--incremental",
+        "HEAD",
+        "--",
+        file,
+    ], line => {
+        const m = BLAME_GROUP.exec(line);
+        if (!m) return;
+        const total = (counts.get(m[1]) ?? 0) + Number(m[2]);
+        if (!Number.isSafeInteger(total)) throw new GitMeasurementError('Git blame line count exceeded the numeric limit');
+        counts.set(m[1], total);
+    });
     return counts;
 }

package/bin/durability/PROVENANCE.md

@@ -12,9 +12,18 @@ moduleResolution node, strict true, esModuleInterop true, skipLibCheck true, and
 Node type declarations. No dependencies are needed by the resulting runtime.
 Rename the generated files to `.mjs` and give their relative imports `.mjs`
 extensions. The engine's calculations and classifier remain unchanged. In the
-reader, replace its promisified `execFile` wrapper with `runtime.mjs`'s `runGit`,
-and rethrow `isFatalGitError(error)` in its per-file blame catch. Retain native
-handling for other per-file errors. Recompute provenance hashes and run the
+reader, replace its promisified `execFile` wrapper with `runtime.mjs`'s `runGit`.
+For blame, use `runGitLines` with `--incremental` and sum each group's line count
+instead of counting individual `--line-porcelain` headers. Attribution is unchanged;
+source text and repeated per-line metadata are not emitted or retained. Any failed
+blame aborts the report instead of returning empty or partial counts. Read the
+object type from `ls-tree -rz` and select only blobs, preserving literal names
+without trimming. Decode Git's C-quoted path bytes in `log --numstat` so tabs,
+quotes, Unicode and leading/trailing whitespace match these same blob names.
+This skips only submodule gitlinks,
+whose native blame contributed zero; binary and symlink blobs remain eligible.
+Commit statistics and the introduced-lines denominator remain unchanged.
+Recompute provenance hashes and run the
 parity and cancellation tests. The reference modules under
 `test/native-durability-reference/` preserve the original reader for small fixture
 comparisons only; they are excluded from the published package.
@@ -27,10 +36,16 @@ part of native Git measurement even if archive exclusions omit those directories
 No clone, fetch, checkout or Git write is performed. Shallow or partial histories,
 missing objects, changing HEAD and invalid reports produce no measurement.
 The separate worker has a 120-second total budget and 512 MiB V8 heap ceiling;
-each Git command has at most 30 seconds and 16 MiB of output. The complete HEAD
-object inventory uses this same limit, so sufficiently large object inventories
-can be unavailable even when the 90-day change window is small. Limit failures
-are reported as unavailable, never converted into successful zero measurements.
+each Git command has at most 30 seconds. Buffered commands retain at most 16 MiB
+of output. Incremental blame and the complete HEAD object inventory stream stdout,
+retaining one line (at most 16 MiB) and at most 16 MiB of stderr, rather than
+limiting total streamed output. One failed OS spawn with `EAGAIN` can recover after
+100 ms, within the original total deadline; the recovery allowance is shared by
+all Git commands. Command exits, signals, timeouts, missing/incomplete history and
+deterministic resource failures are never automatically retried.
+Collection failures abort the CLI before packaging or upload and cannot submit a
+replacement score. A validated report with zero settled or AI-eligible lines is
+still a successful collection; an absent optional coverage report is permitted.
 Only the measurement worker's own process group and Git children are cleaned up
 on cancellation or parent IPC disconnection. Git command output, author details,
 credentials and customer environment tokens are not logged or sent to the worker.

package/bin/durability/provenance.json

@@ -29,7 +29,11 @@
   "runtimeChanges": [
     "ESM relative import extensions only in engine/classifier",
     "GitReader process wrapper replaced by bounded local-only runner",
-    "GitReader blame catch rethrows resource/incomplete-history failures; ordinary per-file errors retain native handling"
+    "GitReader blame aborts the report on any failed command; partial counts never become a successful measurement",
+    "GitReader selects literal blob paths from NUL-delimited ls-tree and decodes C-quoted numstat filenames; submodule gitlinks retain zero native contribution without changing introduced-line counts",
+    "GitReader blame streams incremental groups and sums their line counts with exact native attribution; bounded lines and stderr preserve resource limits without capping total streamed stdout",
+    "Worker streams the complete HEAD object inventory and rejects missing objects without buffering total output",
+    "Runtime permits one recovery from OS spawn EAGAIN within the original deadline; all other collection failures abort the CLI before upload"
   ],
   "generatedFiles": [
     {
@@ -38,7 +42,7 @@
     },
     {
       "file": "GitReader.mjs",
-      "sha256": "447ce620849adc44e3abdf153174ce485ef5391cb29a00fc2e1d9a3064197b96"
+      "sha256": "e86e2e8251f5d9e8f50f671805d8926346aeb574bd2d6a115e5cd324555858f3"
     },
     {
       "file": "OriginClassifier.mjs",

package/bin/durability/runtime.mjs

@@ -1,15 +1,17 @@
-import { execFile } from 'node:child_process'
+import { execFile, spawn } from 'node:child_process'
+import { setTimeout as delay } from 'node:timers/promises'
 
 export const MAX_GIT_BUFFER = 16 * 1024 * 1024
 export const MAX_GIT_TIME_MS = 30_000
 let deadline = 0
+let spawnRecoveryUsed = false
 const children = new Set()
 
 export class GitMeasurementError extends Error {
   constructor(reason) { super(reason); this.reason = reason; this.fatalMeasurement = true }
 }
 export const isFatalGitError = (error) => error?.fatalMeasurement === true
-export function setDeadline(value) { deadline = value }
+export function setDeadline(value) { deadline = value; spawnRecoveryUsed = false }
 export function remainingTime() {
   const remaining = deadline - Date.now()
   if (remaining <= 0) throw new GitMeasurementError('time budget exceeded')
@@ -27,29 +29,120 @@ export function localGitEnvironment() {
   return { ...env, LC_ALL: 'C', GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: process.platform === 'win32' ? 'NUL' : '/dev/null', GIT_TERMINAL_PROMPT: '0', GIT_OPTIONAL_LOCKS: '0', GIT_NO_LAZY_FETCH: '1' }
 }
 
-export async function runGit(repoPath, args, { allowMissingConfig = false } = {}) {
+function gitArgs(repoPath, args) {
+  return [
+    '--no-pager', '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=' + (process.platform === 'win32' ? 'NUL' : '/dev/null'),
+    '-c', 'credential.helper=', '-c', 'protocol.allow=never', '-c', 'protocol.file.allow=never',
+    '-C', repoPath, ...args,
+  ]
+}
+
+function gitError(error, stderr) {
+  // EAGAIN is a failed OS spawn, before Git ran or emitted any result. A child
+  // exit status, timeout or deterministic Git/resource error is not retryable.
+  if (error.code === 'EAGAIN') {
+    const failure = new GitMeasurementError('Git process could not start because system resources are temporarily unavailable')
+    failure.retryableSpawn = true
+    return failure
+  }
+  if (error.code === 'ENOENT') return new GitMeasurementError('Git is unavailable')
+  if (error.code === 'ERR_CHILD_PROCESS_STDIO_MAXBUFFER' || error.code === 'ENOBUFS') return new GitMeasurementError('Git output exceeded the 16 MiB resource limit')
+  if (error.killed || error.signal || error.code === 'ETIMEDOUT') return new GitMeasurementError('Git command exceeded the time budget')
+  if (/missing (?:blob|tree|commit|object)|bad object|unable to read|could not read|object .*corrupt|object file .*empty|invalid object|promisor|transport .*not allowed/i.test(stderr || '')) {
+    return new GitMeasurementError('Git history is incomplete or unavailable locally')
+  }
+  // Never propagate command output, author identities or credentials to logs.
+  const safeError = new GitMeasurementError('Git command failed')
+  safeError.code = error.code
+  return safeError
+}
+
+async function withSpawnRecovery(operation) {
+  try { return await operation() } catch (error) {
+    if (!error?.retryableSpawn || spawnRecoveryUsed) throw error
+    spawnRecoveryUsed = true
+    remainingTime()
+    await delay(Math.min(100, remainingTime()))
+    remainingTime()
+    return operation()
+  }
+}
+
+export async function runGit(repoPath, args, options = {}) {
+  return withSpawnRecovery(() => runGitOnce(repoPath, args, options))
+}
+
+function runGitOnce(repoPath, args, { allowMissingConfig = false } = {}) {
   const timeout = Math.min(MAX_GIT_TIME_MS, remainingTime())
   return new Promise((resolve, reject) => {
-    const child = execFile('git', [
-      '--no-pager', '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=' + (process.platform === 'win32' ? 'NUL' : '/dev/null'),
-      '-c', 'credential.helper=', '-c', 'protocol.allow=never', '-c', 'protocol.file.allow=never',
-      '-C', repoPath, ...args,
-    ], { encoding: 'utf8', maxBuffer: MAX_GIT_BUFFER, timeout, killSignal: 'SIGKILL', windowsHide: true, env: localGitEnvironment() }, (error, stdout, stderr) => {
+    const child = execFile('git', gitArgs(repoPath, args), { encoding: 'utf8', maxBuffer: MAX_GIT_BUFFER, timeout, killSignal: 'SIGKILL', windowsHide: true, env: localGitEnvironment() }, (error, stdout, stderr) => {
       children.delete(child)
       if (!error) { try { remainingTime(); resolve(stdout) } catch (e) { reject(e) }; return }
       if (allowMissingConfig && error.code === 1) { resolve(''); return }
-      if (error.code === 'ENOENT') { reject(new GitMeasurementError('Git is unavailable')); return }
-      if (error.code === 'ERR_CHILD_PROCESS_STDIO_MAXBUFFER' || error.code === 'ENOBUFS') { reject(new GitMeasurementError('Git output exceeded the 16 MiB resource limit')); return }
-      if (error.killed || error.signal || error.code === 'ETIMEDOUT') { reject(new GitMeasurementError('Git command exceeded the time budget')); return }
-      if (/missing (?:blob|tree|commit|object)|bad object|unable to read|could not read|object .*corrupt|object file .*empty|invalid object|promisor|transport .*not allowed/i.test(stderr || '')) {
-        reject(new GitMeasurementError('Git history is incomplete or unavailable locally')); return
-      }
-      // Preserve the native reader's handling of ordinary per-file blame errors.
-      // Never propagate command output, author identities or credentials to logs.
-      const safeError = new Error('Git command failed')
-      safeError.code = error.code
-      reject(safeError)
+      reject(gitError(error, stderr))
     })
     children.add(child)
   })
 }
+
+/** Consume stdout a line at a time; retain at most one bounded line, not the full output. */
+export async function runGitLines(repoPath, args, onLine) {
+  return withSpawnRecovery(() => runGitLinesOnce(repoPath, args, onLine))
+}
+
+function runGitLinesOnce(repoPath, args, onLine) {
+  const timeout = Math.min(MAX_GIT_TIME_MS, remainingTime())
+  return new Promise((resolve, reject) => {
+    let pending = ''
+    let stderr = ''
+    let failure
+    const child = spawn('git', gitArgs(repoPath, args), {
+      stdio: ['ignore', 'pipe', 'pipe'], timeout, killSignal: 'SIGKILL', windowsHide: true, env: localGitEnvironment(),
+    })
+    children.add(child)
+    const fail = error => {
+      failure ??= error
+      child.kill('SIGKILL')
+    }
+    const emit = line => {
+      if (Buffer.byteLength(line) > MAX_GIT_BUFFER) throw gitError({ code: 'ENOBUFS' })
+      onLine(line)
+    }
+    child.stdout.setEncoding('utf8')
+    child.stdout.on('data', chunk => {
+      if (failure) return
+      try {
+        remainingTime()
+        const text = pending + chunk
+        let start = 0
+        for (let end = text.indexOf('\n'); end !== -1; end = text.indexOf('\n', start)) {
+          emit(text.slice(start, end))
+          start = end + 1
+        }
+        pending = text.slice(start)
+        if (Buffer.byteLength(pending) > MAX_GIT_BUFFER) throw gitError({ code: 'ENOBUFS' })
+      } catch (error) {
+        fail(isFatalGitError(error) ? error : new GitMeasurementError('Git output could not be parsed'))
+      }
+    })
+    child.stderr.setEncoding('utf8')
+    child.stderr.on('data', chunk => {
+      if (failure) return
+      if (Buffer.byteLength(stderr) + Buffer.byteLength(chunk) > MAX_GIT_BUFFER) fail(gitError({ code: 'ENOBUFS' }))
+      else stderr += chunk
+    })
+    child.on('error', error => { failure ??= gitError(error, stderr) })
+    child.on('close', (code, signal) => {
+      children.delete(child)
+      if (failure) { reject(failure); return }
+      if (code !== 0 || signal) { reject(gitError({ code, signal, killed: child.killed }, stderr)); return }
+      try {
+        remainingTime()
+        if (pending) emit(pending)
+        resolve()
+      } catch (error) {
+        reject(isFatalGitError(error) ? error : new GitMeasurementError('Git output could not be parsed'))
+      }
+    })
+  })
+}

package/bin/durability/worker.mjs

@@ -1,7 +1,7 @@
 import { realpath } from 'node:fs/promises'
 import { analyzeDurability } from './GitDurabilityEngine.mjs'
 import { ENGINE_COMMIT, validateReport } from './validate.mjs'
-import { runGit, setDeadline, remainingTime, stopGitChildren, GitMeasurementError } from './runtime.mjs'
+import { runGit, runGitLines, setDeadline, remainingTime, stopGitChildren, GitMeasurementError } from './runtime.mjs'
 
 process.on('SIGTERM', () => { stopGitChildren(); process.exit(2) })
 let completed = false
@@ -20,8 +20,9 @@ process.once('message', async ({ root, deadline }) => {
     if (partial.split('\n').some(line => /^extensions\.partialclone\s+\S/i.test(line) || /^remote\..*\.promisor\s+(true|yes|on|1)\s*$/i.test(line))) throw new GitMeasurementError('partial Git history cannot be measured without fetching objects')
     const headSha = (await runGit(root, ['rev-parse', '--verify', 'HEAD'])).trim()
     if (!/^[0-9a-f]{40}$/.test(headSha)) throw new GitMeasurementError('a valid SHA-1 Git HEAD is required')
-    const objects = await runGit(root, ['rev-list', '--objects', '--no-object-names', '--missing=print', 'HEAD'])
-    if (objects.split('\n').some(line => /^\?[0-9a-f]+$/.test(line))) throw new GitMeasurementError('Git history is missing local objects')
+    await runGitLines(root, ['rev-list', '--objects', '--no-object-names', '--missing=print', 'HEAD'], line => {
+      if (/^\?[0-9a-f]+$/.test(line)) throw new GitMeasurementError('Git history is missing local objects')
+    })
     const measuredAt = new Date().toISOString()
     const result = await analyzeDurability(root, {
       windowDays: 90, settleDays: 14,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@scalequality/check",
-  "version": "0.5.0",
+  "version": "0.5.2",
   "description": "ScaleQuality continuous code-maturity check for CI. Re-measures your repo on every build and fails the pipeline below your maturity line.",
   "type": "module",
   "bin": {