@titan-design/code-graph 0.2.0 → 0.4.0

package/README.md

@@ -40,11 +40,17 @@ coupling, now `src/analysis/`. See [Graph analyses](#graph-analyses).
 Ported with TP-133: the test linker and the Istanbul coverage overlay, also in `src/analysis/`,
 and test-coverage ownership. See [Test linking and coverage](#test-linking-and-coverage).
 
-Deferred, all of it still in codewatch, all of it a follow-up on this package rather than a
-change to it:
+Ported with TP-250: package partition quality (`src/analysis/partition-quality.ts`) and
+snapshot pruning (`src/prune.ts`). See [Partition quality](#partition-quality) and
+[Pruning snapshots](#pruning-snapshots).
 
-- Graph analyses over a finished snapshot: communities, partition quality, conventions,
-  patterns, prune, reuse-delta reporting.
+Ported with TP-130: community detection and the convention layer (`src/conventions/`). See
+[Conventions](#conventions). codewatch keeps the CLI command, the `claude -p` summarizer, and
+the MCP and read-API wiring.
+
+Deferred, still in codewatch, a follow-up on this package rather than a change to it:
+
+- Reuse-delta reporting over a finished snapshot.
 
 Python support is new here rather than ported. codewatch walked TypeScript only; the parser
 already had the grammar. The Python extractor is deliberately narrower than the ts-morph one:
@@ -79,6 +85,41 @@ package's source file, not to an `npm:` external, by remapping the `dist/*.d.ts`
 ts-morph resolves back onto `src/`. That remap needs the target package built, which is why
 `pnpm build` precedes both `pnpm test` and `dag:check`.
 
+## Identity across renames
+
+Added in index version 0.15.0 (TP-187). Git rename detection writes `id_alias` rows mapping a
+file's and its module's old id to the new one. A file rename also writes an alias for every
+symbol present on both sides, so `a.ts#Job.run` follows `a.ts` to `b.ts#Job.run`. A symbol
+renamed inside a file is not followed.
+
+Each snapshot records its alias base in `attrs.aliasBase`: the snapshot whose commit its
+aliases were computed against. A new index of a ref diffs against that ref's newest committed
+snapshot, falling back to the newest committed snapshot of any ref. The bases form a tree, so
+ids can be carried between any two snapshots that share an ancestor, forward or backward.
+
+```ts
+import { aliasChain, priorSnapshotForRef, resolveAlias } from "@titan-design/code-graph";
+
+resolveAlias(store, "src/job.ts", 4, { fromSnapshotId: 1 });
+// { id: 'core/worker.ts', reason: 'move',
+//   hops: [ src/job.ts -> src/task.ts, src/task.ts -> src/worker.ts, src/worker.ts -> core/worker.ts ] }
+resolveAlias(store, "src/job.ts#Job.run", 4, { fromSnapshotId: 1 }).id; // 'core/worker.ts#Job.run'
+aliasChain(store, 1, 4).resolve; // one resolver for many ids
+priorSnapshotForRef(store, "main", { before: 7, repoRoot: "." }); // the snapshot main denotes
+```
+
+Without `fromSnapshotId`, `resolveAlias` walks from the root of the target's lineage, so an id
+from any ancestor resolves. Each snapshot's aliases are applied once, in order: they come from
+one git diff, so `a.ts` to `b.ts` plus `b.ts` to `a.ts` in one snapshot is a swap, not a
+cycle. Walks stop after 10,000 snapshots. Two snapshots with no common base fall back to the
+to-snapshot's own aliases, which is what `diffSnapshots` did before 0.15.0.
+
+`priorSnapshotForRef` prefers the snapshot of the commit git resolves the ref to, and
+otherwise the newest snapshot labelled with that ref. A snapshot written before 0.15.0 records
+no base; its base is inferred as the newest earlier snapshot with a commit, which is the one
+the 0.14.0 indexer diffed against. Nothing is written on read, so a 0.14.0 store opened
+read-only diffs and checks across renames too.
+
 ## The three reuse tiers
 
 Every run writes a fingerprint per file: a content hash and a comment/whitespace-insensitive
@@ -87,10 +128,11 @@ same `INDEX_VERSION` and sorts each file into one tier. `INDEX_VERSION` is bumpe
 metric can change for the same bytes, not only when the node or edge shape changes: 0.12.0
 marks `.tsx` files moving to the tsx grammar, which changed their complexity metrics and
 symbol spans; 0.13.0 marks the dead-code and growth-risk metrics joining the carry-forward set;
-0.14.0 marks symbol ids qualified by their enclosing scopes (TP-182). A snapshot from before
-0.14.0 is never reused, so re-index. The first 0.14.0 run after an older snapshot writes
-`requalify` id aliases from each bare-name id to its qualified successor, only where exactly one
-declaration in the file carries that name.
+0.14.0 marks symbol ids qualified by their enclosing scopes (TP-182); 0.15.0 marks symbol
+aliases on a file rename and the recorded alias base (TP-187). A snapshot from an older version
+is never reused, so the first run after an upgrade is a full index. The first 0.14.0 run after
+an older snapshot writes `requalify` id aliases from each bare-name id to its qualified
+successor, only where exactly one declaration in the file carries that name.
 
 | Tier | Trigger | Work skipped |
 |---|---|---|
@@ -127,6 +169,57 @@ The symbol layer is hidden by default. `listNodes` drops `symbol` nodes and `lis
 the graph it expects and does not have one import of thirty names read as thirty
 dependencies.
 
+## Targeted reads and the metric catalogue
+
+Added for the read API (TP-183). Three store reads answer one node or one metric without
+loading a snapshot, and a catalogue describes every metric name the package writes.
+
+```ts
+import { aggregateMetrics, describeMetric, listEdgesTouching, listMetricsForNode } from "@titan-design/code-graph";
+
+listMetricsForNode(store, snapshotId, "packages/registry/src/invoke.ts"); // every metric on one node
+listEdgesTouching(store, snapshotId, "packages/registry/src/invoke.ts"); // edges in and out; references on request
+aggregateMetrics(store, snapshotId, { name: "loc" }); // [{ name: "loc", nodeKind: "file", count, sum, min, max }]
+describeMetric("churn_90d"); // { unit: "lines", rollup: "sum", direction: "neutral", absent: "zero", window: "90d", … }
+```
+
+Three more store reads answer a report's questions without loading a snapshot:
+`listMetricNames(snapshotId)` for the distinct names stored, `topByMetric({ snapshotId,
+metric, limit, kind })` for the highest-valued nodes joined to their kind and role, and
+`replaceMetricsByName(snapshotId, name, metrics)`, which swaps one metric's whole row set in
+a single transaction so a re-ingested overlay such as coverage never accumulates stale rows.
+
+`computeDeepAst({ filePath, absPath, symbolName })` reads structure too heavy to persist
+(params, return type, class members) from the working tree on demand, and returns null when
+the file is unreadable.
+
+- **Reads.** `listMetricsForNode` searches the metric primary key, `listEdgesTouching` the edge
+  primary key plus `idx_edge_dst`, and `aggregateMetrics` `idx_metric_name`. None scans a
+  table, and no index or migration was added, so any existing store serves them. A self-loop
+  comes back once. `aggregateMetrics` groups by metric name and node kind, because
+  `utilization` and the degree metrics sit on several kinds; `count` skips null values.
+- **Catalogue.** `METRIC_CATALOGUE` holds one `MetricDescriptor` per name: `unit`,
+  `appliesTo` (node kinds), `rollup` (`sum`, `max`, `mean`, or `none`), `direction`
+  (`higher-worse`, `lower-worse`, or `neutral`), `absent`, `source`, and `description`.
+  Windowed names are templates such as `churn_{w}`; `describeMetric` resolves a stored name
+  such as `churn_90d` or `test_bus_factor_lifetime` to a concrete descriptor, and returns
+  null for a name nothing writes.
+- **`rollup: "none"`** means no rollup reproduces the group's own value. Summing fan-in counts
+  a directory's internal edges, and summing per-file commit counts counts a shared commit
+  more than once. A reader must not synthesize those.
+- **`absent`** says what a missing row means. `zero`: the writer is sparse, so count the node
+  as zero (dead code, growth risk, churn, `linked_test_count`). `exclude`: the metric does not
+  apply or was not measured (a max over no functions, `coverage_pct` before ingest), so leave
+  the node out of means, percentiles, and ranks.
+- **Completeness is tested.** `catalogue-completeness.test.ts` indexes a fixture repo with
+  history and a coverage overlay, and fails, naming the metric, when a stored name has no
+  descriptor, when a descriptor's unit or node kinds disagree with the rows, or when a
+  descriptor matches nothing.
+- **Browser-safe.** The catalogue imports only `types.ts`. The `code-graph-catalogue-pure`
+  and `code-graph-catalogue-no-node` rules in `.codewatch/check.json` hold it there, so a
+  later browser subpath can re-export it. It ships from the root export today, which does
+  pull in Node.
+
 ## Checks and diffs
 
 The rules engine turns a snapshot into pass/fail against a `check.json`. Six rule types:
@@ -148,23 +241,31 @@ engine on ids, and `validateRules(json)` validates an already-parsed rules objec
 
 The baseline is a ratchet. A violation whose key (rule id, node id, and destination id for
 edge rules) also fires on the baseline snapshot is marked `isCarryover` and counts as
-carryover, so existing debt does not block a change but new debt does. Deprecated metric and
+carryover, so existing debt does not block a change but new debt does. The baseline's node ids
+are first carried through the alias chain into the checked snapshot (`rebasedViolationKey`),
+so a moved file's violations carry over instead of reading as one resolved plus one new.
+Unmoved ids key exactly as `violationKey` always did. Deprecated metric and
 role spellings in a rules file (`lines`, `tests`) heal to their canonical names with a
 warning through `onWarn` instead of failing validation.
 
 `diffSnapshots(store, { fromSnapshotId, toSnapshotId })` reports added, removed and renamed
-nodes, added and removed edges, and metric deltas on nodes present in both. The
-to-snapshot's `id_alias` rows carry a renamed file across, so a move reads as a rename rather
-than a delete plus an add, and its edges do not churn. Metric and edge-kind spellings are
+nodes, added and removed edges, and metric deltas on nodes present in both. Ids follow the
+alias chain between the two snapshots, across every rename in between, so a move reads as a
+rename rather than a delete plus an add, and its edges do not churn. Metric and edge-kind spellings are
 canonicalised before comparing.
 
 `diffCheckResults(store, { fromSnapshotId, toSnapshotId, rules })` runs the rules on both
 snapshots and buckets each violation as new, resolved, or unchanged; unchanged metric
-violations are further split into worsened and improved by value.
+violations are further split into worsened and improved by value. From-side ids follow the
+alias chain, as in the ratchet.
 
 `scripts/dag-check-self.mjs` in the repo root runs this repo's DAG check on this engine
 instead of codewatch's CLI.
 
+The glob matching the rules use is exported too, because a CLI filters its own `--include`
+and `--exclude` flags with the same semantics: `patternToRegex` (a `*` glob when the pattern
+holds one, a case-sensitive substring otherwise), `compilePatterns`, and `matchesAny`.
+
 ## Similar symbols
 
 Ported from codewatch's `embeddings.ts` (TP-129). It answers "does something like this
@@ -231,6 +332,78 @@ snapshotSymbolCoupling(store, snapshotId); // symbol pairs co-imported by 2+ fil
 The pure `computePageRank`, `computeRelevance`, `computeSymbolConsumers`, and
 `computeSymbolCoupling` take node and edge arrays instead of a store.
 
+### Partition quality
+
+`computePartitionQuality` scores a package partition of the file graph: per-package cohesion,
+Martin instability, an abstractness proxy (the share of `role: "types"` files), a layer label
+(`top`, `middle`, `foundation`), pair coupling by intensity (`edges / files(from)`), and a
+Newman-Girvan modularity Q over the whole partition.
+
+```ts
+import { computePartitionQuality, invertBuckets } from "@titan-design/code-graph";
+
+const result = computePartitionQuality({ packages, fileByPackage, nodes, edges });
+result.modularityQ; // 0.77 on titan-platform's packages/*
+invertBuckets(fileByPackage); // file id -> package id, skipping the "" unassigned bucket
+```
+
+`resolveBarrels: true` rewrites an edge landing on a `role: "barrel"` file to the files it
+re-exports, transitively. It over-attributes: one import of one name through a barrel becomes
+one edge per re-export target, which is why it is off by default.
+
+### Pruning snapshots
+
+`planPrune` keeps the most recent `keep` snapshots (default 10) plus every snapshot whose ref
+is in `keepRefs`; `runPrune` deletes the rest and reports row counts before and after.
+
+```ts
+import { planPrune, runPrune } from "@titan-design/code-graph";
+
+planPrune(store, { keep: 10, keepRefs: ["main"] }); // { keep, remove }
+runPrune(store, { keep: 10, vacuum: true }); // { plan, rowsBefore, rowsAfter, vacuumed }
+```
+
+The domain tables declare no foreign key, so `CodeGraphStore.deleteSnapshots` clears each of
+`SNAPSHOT_SCOPED_TABLES` itself rather than relying on a cascade. `blob_cache` is
+content-addressed and not snapshot-scoped, so a prune never drops a cached embedding.
+
+## Conventions
+
+"How does this repo do X, and where does code like this belong?" Ported from codewatch's
+unmerged C-88 branch (TP-130). The layer cuts the barrel-resolved file graph into a few coarse
+areas, has an injected summarizer describe each one, and ranks areas against a question by
+embedding similarity. Verified against this release on this repo's `packages/`, with a fake
+summarizer:
+
+```ts
+import { findConventions, getConventionMap, summarizeConventions } from "@titan-design/code-graph";
+
+const summarizer = { model: "claude:sonnet", summarize: (prompt: string) => callYourLlm(prompt) };
+await summarizeConventions(store, snapshotId, summarizer);
+// { coverage: { files: 713, grouped: 513, areas: 24, summarized: 24 }, newlySummarized: 24, reused: 0, … }
+// a second run: { newlySummarized: 0, reused: 24 }
+
+getConventionMap(store, snapshotId, "claude:sonnet"); // the same areas, stored summaries only
+await findConventions(store, snapshotId, "how are CLI commands registered?", embedder, "claude:sonnet");
+// { matches: [ { label, summary, files: [ …up to 5 ], size, score }, … up to 3 ] }
+```
+
+- **The cut.** `detectCommunities` is greedy modularity (Clauset-Newman-Moore), not Leiden. It
+  is deterministic without a seed: ties resolve by sorted id. `targetCount` keeps merging past
+  the natural modularity stop until that many communities remain, and a size cap of twice the
+  ideal share keeps a dense repo from collapsing into one area. The default target is one area
+  per 25 files, clamped to 6..40. Areas under `minSize` (default 3) files are left unsummarized.
+  Disconnected components never merge, so the component count is the floor.
+- **Only the coarse level is summarized.** LLM cost is one call per area, and each summary is
+  stored in `blob_cache` under `code-graph/community-summary`, keyed by `summarizer.model` and a
+  hash of the prompt. The prompt carries the member files and their key exported signatures, so
+  an area whose membership and signatures did not change is a cache hit in any snapshot. On
+  this repo a finer cut (`targetCount: 60`) still reused 8 of its 35 areas.
+- **The package ships no LLM client.** `Summarizer` is `{ model, summarize(prompt) }`; the
+  product supplies it. `getConventionMap` and `findConventions` never call it. `findConventions`
+  throws when no summary is stored for the model, and returns candidates with scores, not
+  verdicts. Summary vectors go through the same embedding cache as similar symbols.
+
 ## Git history
 
 Ported in TP-126, strictly as codewatch had it: churn over rolling windows, first-seen dates,
@@ -255,6 +428,14 @@ plus `churnWindowDays` (the primary, default 30); `churnWindows` replaces the de
 turns all of it off. Outside git, or without a git binary, the index simply has no history
 metrics.
 
+The adapter is exported from the package root, not from `./history`, because it speaks
+`GraphMetric` and the seam below does not. A product that indexes on its own terms calls
+`loadHistoryMetrics(nodes, idRoot, options)` for both the metric rows and the primary-window
+churn entries they were built from (`LoadedHistory`), with `HistoryMetricsOptions`,
+`DEFAULT_CHURN_WINDOWS` (`[30, 90, 180]`), `resolveChurnWindows`, `windowSuffix` (`30d`,
+`lifetime`) and `computeRecencyWindows` alongside it. Node ids are the history engine's
+repo-relative paths, so both must be rooted at the same `idRoot`.
+
 Change coupling is not stored. It is computed on demand from `loadChurnEntries`, as
 codewatch's `graph coupled` command did.