knodin 0.12.2 → 0.13.1

package/docs/releases/0.13.1.md

@@ -0,0 +1,216 @@
+# knodin 0.13.1
+
+Compatible patch release for KNODIN-47 and KNODIN-48: correct false stale
+evidence after repair and bound managed graph storage.
+
+## Fixed
+
+- Align freshness probes with source-indexing eligibility. Non-source files and
+  collapsed untracked directories no longer make an unchanged, repaired graph
+  appear stale.
+- Preserve exact Git paths, including renames and names containing whitespace,
+  and exclude canonical pruned trees before expanding untracked files.
+- Track relevant `.gitignore` rules, including tracked policy files omitted by
+  Git's stat cache, Git's local exclude file, and its
+  effective global exclude file with bounded reads. Retain previously indexed
+  paths for deletion detection and respect scoped-index path safety.
+- Encode dirty path sets without newline collisions. Failed Git or ignore-policy
+  checks remain unknown, including Git checkouts without their first commit.
+- Reindex relevant dirty or committed-diff source before accepting a repaired
+  baseline, so restored file sizes and timestamps cannot hide source changes.
+  Repair's initial audit also checks change times against the completed source snapshot,
+  covering source edits omitted by Git's stat-cache shortcuts.
+  Verify source and Git stability before publishing the baseline; unresolved
+  historical evidence requires explicit recovery rather than a false success.
+- Preserve first-time repair of an empty graph while refusing to guess missing
+  history or refresh timestamps for an existing graph.
+- Preserve completed-file resume behavior in direct-engine repairs with content
+  proofs that are invalidated by later index writes. Public repair retains its
+  atomic candidate cancellation behavior.
+- Separate repair's content-proof epoch from ordinary reconciliation timestamps.
+  Existing non-empty Git graphs need a one-time `knodin index --clean` before
+  this versioned proof is established. Status, query freshness, and repair
+  refuse to certify missing, malformed, or future-dated provenance rather than
+  trusting a legacy timestamp.
+- Force-reindex paths that Git reports changed, even when size and timestamp
+  match the indexed tuple, so ordinary reconciliation cannot retain old symbols.
+- Establish Knodin's existing repository-local state exclusion before candidate
+  indexing and repair record their policy baseline.
+- Accept the platform null device as an empty global Git exclude policy, and
+  retain the documented bounded stat fallback for verified nested Git paths.
+- Bound new manifest eligibility reads together with ignore-policy reads. When
+  the freshness budget is exceeded, report unknown; explicitly index a large
+  new manifest before relying on automatic freshness for it.
+- Revalidate already-dirty source content with a bounded proof (at most 256
+  paths and 1 MiB per probe). Over-budget content or unreliable Git stat settings
+  remain unknown on the cheap freshness path. A complete deep audit can verify
+  the retained source snapshot; unknown is not evidence that nothing changed.
+- Validate clean-build resume provenance strictly, preserve the original HEAD,
+  recheck equal-stat source edits, and remove deleted paths before promotion.
+  Source/Git changes during clean indexing or final promotion refuse publication
+  of the candidate rather than minting a proof over a mixed snapshot.
+- Preserve verified parsed-source proofs when seeding an unchanged linked
+  worktree. Missing, malformed, or mismatched copied proofs require extraction;
+  omitted equal-stat source edits cannot retain stale sibling symbols.
+- Reuse scoped, canonically pruned Git enumeration for cold reconciliation,
+  including unborn repositories. Failed or over-budget working-tree/commit-diff
+  enumeration cannot advance the stored HEAD or freshness baseline. Missing
+  shallow/foreign commit history requires explicit clean-index recovery rather
+  than a stat-only fallback that misses equal-stat source edits.
+- Withhold a first scoped content-proof epoch after unhealthy verification or
+  an exception, including failures after the deep audit completes. Existing
+  verified epochs are not deleted by failed scope widening.
+- Preserve structural revisions and Apex consumer paging snapshots across
+  unchanged cold opens, including unborn Git repositories. Canonical discovery
+  and relationship/semantic maintenance always run; only unchanged file
+  extraction is skipped using an exact retained parsed-source proof. Missing,
+  malformed, or changed per-file evidence forces extraction. The extraction
+  fence also retains proof before a repository epoch exists, using bounded-memory
+  streaming reads before and after parsing (additional source I/O, not a measured
+  performance gain). Failed extraction cannot publish a new per-file proof;
+  aggregate completion still requires healthy verification.
+  Worker-queued inputs also retain pre/post-parse source fences; per-queued-file
+  metadata scales with the source inventory, while streaming uses a 64 KiB read
+  buffer. Successful batched writes retain their parsed proofs; rolled-back
+  file writes cannot regain a proof at the batch boundary.
+  Stable unparsed/skipped source reads also retain their consumed-source proof
+  after pre/post-read verification; extraction and database-write errors do not.
+  Batch write failures retain successful per-file work but refuse aggregate
+  completion before publishing an epoch or baseline; resolve the write error
+  and retry indexing. A failed candidate is not promoted over the active graph.
+
+## Storage stewardship (KNODIN-48)
+
+New successfully indexed stores receive a repository-local storage policy:
+one recoverable promotion backup, a 1 GiB allocated-backup budget, and a
+512 MiB free-space reserve. Count/byte budgets override the age grace period.
+Cleanup runs after durable promotion and operation-lease release; cleanup
+failures are reported without rolling back the committed graph. The newest
+snapshot and an integrity-checked rollback remain protected, even when that
+requires reporting an over-budget condition instead of deleting recovery data.
+
+Existing unmanaged stores are not silently migrated. Run
+`knodin backups policy preview /path/to/repo --json`, inspect exact candidates,
+then `knodin backups policy apply /path/to/repo --json` to consent. Existing
+explicit dry-run/disabled policies remain respected. Configure limits with
+`--max-count`, `--max-bytes`, and `--reserve-bytes`; `--mode preview` or
+`--mode disabled` records the corresponding maintenance choice.
+
+Automatic cleanup rechecks effective policy under its lease before deletion.
+New-store initialization eligibility is durably recorded in the graph before
+publication and survives later clean replacement if policy publication fails;
+successful initialization clears it, so deliberate policy removal stays unmanaged.
+Clearing this internal marker preserves an already validated graph audit and its
+original timestamp across clean close/reopen; concurrent database changes refuse
+audit carry-forward rather than being labeled as previously verified.
+Policy reads use the shared external-state resolver for registered mirrors;
+policy initialization, apply, and automatic cleanup refuse mirror source writes,
+including source aliases. Unsupported Windows directory syncing does not turn
+an already file-synced policy publication into failure; other I/O errors remain visible.
+
+Candidate creation and repair's VACUUM preflight require conservatively
+estimated scratch space plus the reserve. Unknown capacity or allocated-size
+measurements fail closed before allocating scratch. This is an estimate, not
+a disk reservation; ENOSPC and concurrent writers can still cause refusal.
+Bounded candidate sweeping removes only managed empty/audit-only residue with
+a validated marker and confirmed exited owner. Unknown legacy owners and
+DB-bearing resumable candidates are preserved. Inventory distinguishes apparent from
+allocated bytes; APFS shared extents mean allocated totals are not guaranteed
+physical reclamation. External source mirrors and development acceptance
+directories are not graph backups and are not blanket-deleted by this policy.
+Candidate allocation revalidates the managed root and new directory before any
+owner or database write. JSON-manifest eligibility uses a no-follow descriptor
+with final pathname identity checks. Shared-index restore always releases its
+lifecycle lease and invokes post-promotion storage finalization even when cache
+lease release or candidate cleanup fails.
+
+The release auto-merge dispatch retains exact-SHA, unresolved-conversation,
+local final-gate, approval, and atomic-merge protections when Copilot review is
+unavailable because its AI-credit budget is exhausted. The explicit fallback
+reason is opt-in; ordinary dispatches still require a clean exact-head review.
+The exhausted-credit path also covers a new PR for which Copilot cannot create
+any review object; an exact-head review containing findings still blocks merge.
+The dispatch runs on the organization's approved self-hosted Linux pool so IP
+allow-list enforcement does not prevent it from reading or merging its own PR.
+The tagged release verifier allows 45 minutes for the same full-suite workload,
+preventing a successful test run from being cancelled by the former 30-minute cap.
+
+## Release verification
+
+Release-candidate pushes run full coverage without repeatedly scanning Sonar. After the current
+PR head has completed review, maintainers run `npm run release:verify` locally; it runs full
+coverage and the local Sonar quality gate, refuses a dirty tracked tree or remote-head mismatch,
+and records the exact verified SHA in Git-local metadata. Any later commit invalidates that result.
+The gate also refuses branch changes and ambiguous remote references. Isolated
+subprocess regressions cover preflight refusals, verification failures, SHA drift,
+and the evidence payload without invoking real scans.
+
+Ordinary PR pushes no longer enable auto-merge. Maintainers explicitly dispatch
+the auto-merge workflow with the PR number and exact SHA only after independently
+verifying clean current-head review and successful local final-gate evidence.
+The dispatch checks the live open PR head; it does not authenticate local evidence
+or replace protected-branch checks.
+
+Regression coverage includes repeated cold status checks after repair, genuine
+source edits, ignore-policy changes, scoped paths, symlinks, JSON eligibility,
+and unusual filenames. A freshness-only development build was also verified
+against SalesforceCI's approximately 903,000-file graph: repair completed, then
+two separate-process status checks reported healthy and fresh without changing
+repository source files or existing working-tree edits.
+
+That acceptance establishes the reported repair/status defect, not a complete
+cross-repository init/repair/status certification or a general performance gain.
+Real source changes still require graph refresh. After upgrading an affected
+checkout, run `knodin repair`. If it reports that content provenance is missing,
+run the requested one-time `knodin index --clean`; then run `knodin status`.
+
+## Additional review safety fixes
+
+Already-dirty source proofs reject symlinked paths and recheck held-file identity
+and change time; an unverifiable path reports unknown rather than false freshness.
+Candidate reads and promotion fences retain no-follow descriptor evidence.
+Interrupted candidates with unknown/live owners or nonselected databases are
+preserved rather than recursively discarded. Failed promotion rollback preserves
+uncertain files and a repair-needed marker for manual inspection instead of
+automatically accepting a substituted graph. These are boundary checks, not an
+atomic filesystem reservation against concurrent pathname changes.
+
+Mirror identities use the generated `repo_<20 hexadecimal characters>` format.
+Malformed/unreadable registries fail closed; registry writes reject unsafe targets
+and publish through an exclusive temporary file. Managed staging/source guards
+refuse symlink escapes and reject an unrelated clone path before it can be cached
+as that mirror. Canonical aliases remain supported, and a missing clone remains
+registered only at its deterministic managed path. Mirror indexing keeps graph
+state out of source but does not opt read-only mirrors into managed backup-policy
+initialization or report that intentional omission as failed maintenance.
+Duplicate mirror identities or canonical clone paths are rejected before routing.
+Owner-proven audit-only
+quarantines remain discoverable and retryable after cleanup failure; unknown or
+database-bearing quarantines remain preserved.
+
+The local final gate uses the existing npm coverage command without requiring Bun.
+The privileged merge action is pinned to an immutable upstream commit.
+
+Repair snapshot proofs now use component-wise no-follow source checks and bind
+the opened descriptor to the final pathname across the complete read. Storage
+policy reads similarly reject atomic pathname replacement, and applied policy
+changes share the cleanup lifecycle lease so deletion cannot continue under
+superseded consent.
+
+Opportunistic global retention respects configured roots and depth and revalidates
+installed consent under its deletion lease. Direct candidate cleanup refuses
+read-only mirrors. Seeded worktree promotions run the same post-audit storage
+finalizer, with durable retry after policy publication failure. Read-only mirror
+inventory measures exact backups in resolved external state, not source decoys.
+
+Global policy resolution and opportunistic retention share the same bounded scope
+check; scheduled and opportunistic deletion share lease-time consent validation.
+Mirror aliases resolve through one canonical identity seam. Mirror allocation
+previews external backup quotas, while deletion remains refused. Any skipped or
+unverified rollback inventory blocks managed allocation. Failed policy writes
+clean their unpublished temporary files. A healthy applicable bounded global
+policy can manage a new store without materializing a local policy file.
+Shared-index restores and committed worktree seeds now finalize pending storage
+policy and maintenance after promotion, including a failed post-promotion status
+check. Release dispatch uses GitHub's exact-SHA merge precondition rather than
+arming unbound auto-merge; a changed head requires fresh review and verification.

package/docs/structural-only-indexing.md

@@ -0,0 +1,20 @@
+# Structural-only indexing
+
+Use `knodin index --skip-embeddings` to build native structure without running
+semantic inference. It also works with `--clean`/`--force`, explicit file paths,
+and `--under <directory>`. Scoped indexing still discovers the repository file
+inventory before filtering; it is not a constant-time directory operation.
+
+Intentional deferral is persisted in the local graph. Reopening the graph for a
+structural query, read-side reconciliation, or watcher updates does not silently
+complete the semantic backlog. Semantic readiness remains `absent` or `partial`
+until the missing vectors are built; semantic retrieval may under-return.
+
+Run an explicit `knodin index` without `--skip-embeddings` to resume inference.
+An explicit-file index without the flag also completes the missing-vector pass,
+including deferred symbols outside those files. Existing vectors are retained.
+
+A deferred clean candidate must still pass SQLite, source coverage, source
+snapshot, and orphan checks before promotion. Only the requirement for complete
+semantic coverage is waived for an intentionally deferred graph. An interrupted
+ordinary semantic build does not gain that exemption.

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "knodin",
-	"version": "0.12.2",
+	"version": "0.13.1",
 	"knodin": {
 		"compatibility": "compatible"
 	},
@@ -46,12 +46,17 @@
 		"docs/PT-ACCESS-RECOMMENDATION.md",
 		"docs/RELEASE-0.3-EVIDENCE.md",
 		"docs/REPOSITORIES-AND-WORKTREES.md",
+		"docs/SALESFORCE-BINDINGS.md",
+		"docs/SALESFORCE-DEAD-CODE.md",
+		"docs/SCOPED-INDEXING.md",
 		"docs/SCIP-IMPORT.md",
 		"docs/SHARED-INDEX-CONTRACT.md",
 		"docs/SIGNED-UPDATES.md",
 		"docs/SYSTEMS-AND-RELATIONSHIPS.md",
 		"docs/TELEMETRY.md",
 		"docs/TOKEN-OPTIMIZER-SCORECARD.md",
+		"docs/apex-receiver-resolution.md",
+		"docs/structural-only-indexing.md",
 		"docs/assets/knodin-favicon.svg",
 		"docs/prompts/declare-multi-repository-system.md",
 		"docs/releases/0.10.0.md",
@@ -67,6 +72,8 @@
 		"docs/releases/0.12.0.md",
 		"docs/releases/0.12.1.md",
 		"docs/releases/0.12.2.md",
+		"docs/releases/0.13.0.md",
+		"docs/releases/0.13.1.md",
 		"docs/releases/0.3.0.md",
 		"docs/releases/0.4.0.md",
 		"docs/releases/0.4.1.md",
@@ -115,6 +122,7 @@
 		"test:watch": "vitest --watch",
 		"test:coverage": "tsx scripts/run-vitest-shards.ts --coverage",
 		"scan:sonar": "npm run test:coverage && sh scripts/run-sonar-scan.sh",
+		"release:verify": "tsx scripts/release-final-gate.ts",
 		"mirror:sync": "sh tools/hooks/sync-ghes-mirror.sh",
 		"test:pack-install": "tsx scripts/pack-install-smoke.ts",
 		"test:package-managers": "tsx scripts/package-manager-smoke.ts",
@@ -187,17 +195,17 @@
 		"@hono/node-server": "^2.0.5",
 		"adm-zip": "^0.6.0",
 		"fast-uri": "^4.1.1",
-		"sharp": "^0.35.3"
+		"sharp": "^0.35.4"
 	},
 	"devDependencies": {
 		"@biomejs/biome": "2.4.6",
 		"@types/bun": "^1.3.14",
 		"@types/node": "^24.10.0",
-		"@vitest/coverage-v8": "^4.1.10",
+		"@vitest/coverage-v8": "^4.1.11",
 		"ajv": "8.20.0",
 		"lefthook": "^2.1.5",
 		"tsx": "^4.21.0",
 		"typescript": "^5.9.3",
-		"vitest": "^4.1.3"
+		"vitest": "^4.1.11"
 	}
 }