pi-fovea 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +77 -66
- package/cli.ts +52 -3
- package/package.json +1 -1
- package/src/core/anchors.ts +36 -4
- package/src/core/build.ts +42 -9
- package/src/core/discover.ts +188 -0
- package/src/core/join.ts +1 -1
- package/src/core/sync.ts +35 -15
- package/src/core/types.ts +1 -0
- package/src/index.ts +5 -5
package/README.md
CHANGED
|
@@ -4,10 +4,10 @@
|
|
|
4
4
|
|
|
5
5
|
**A foveated repo-mapping extension for [Pi](https://github.com/earendil-works/pi-coding-agent)**
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
_See the whole repo on every prompt, sharp where you work and cheap everywhere else._
|
|
8
8
|
|
|
9
9
|
<p>
|
|
10
|
-
<img src="https://raw.githubusercontent.com/monotykamary/pi-fovea/main/media/cover.svg" alt="pi-fovea: a code graph seen through a fovea
|
|
10
|
+
<img src="https://raw.githubusercontent.com/monotykamary/pi-fovea/main/media/cover.svg" alt="pi-fovea: a code graph seen through a fovea, hot at the center and collapsed at the rim" width="1100">
|
|
11
11
|
</p>
|
|
12
12
|
|
|
13
13
|
[](https://www.npmjs.com/package/pi-fovea)
|
|
@@ -17,35 +17,23 @@ _Survey, focus, dwell, impact — a budget-capped field of view instead of a fol
|
|
|
17
17
|
|
|
18
18
|
</div>
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
pi-fovea hands the model a map of your repo on every prompt. The repo compiles once into a code graph across languages, where symbols, files, and route anchors join into one network. Each question becomes an interest vector that diffuses over the graph as heat. The renderer converts the field into a token-capped view: full signatures near your task, one-liners a hop away, a skeleton of the rest.
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
After each assistant turn the map re-syncs incrementally. Detection reads content hashes instead of tool events, so edits made by pi's edit/write tools, a fabric_exec inner `pi.edit`, a bash heredoc, a subagent, or an editor save outside the session all land identically. A clean turn stays silent. A turn that moves route anchors or warms files you have not looked at says so.
|
|
23
23
|
|
|
24
|
-
##
|
|
24
|
+
## What the model gets
|
|
25
25
|
|
|
26
|
-
|
|
|
27
|
-
|
|
28
|
-
|
|
|
29
|
-
|
|
|
30
|
-
|
|
|
31
|
-
|
|
|
32
|
-
| 🩸 | **Turn sync** | After every edit turn the graph re-syncs for free. Anchor shifts and unwatched warmings surface as red flags; stable turns stay silent. |
|
|
33
|
-
| 🪙 | **Token truth** | Budgets are hard caps, not hopes: the renderer fits a monotonic prefix and never exceeds `maxTokens`. |
|
|
26
|
+
| Command | Ask | Answer |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| `fovea_sketch` | where is everything? | the repo as a silhouette, with feature anchors and inferred regions ranked by mass |
|
|
29
|
+
| `fovea_focus` | what is this? | centered on a symbol, route path, or env key: hot nodes as signatures, neighbors as one-liners |
|
|
30
|
+
| `fovea_dwell` | what else? | diffuses the field one step further and returns the delta |
|
|
31
|
+
| `fovea_impact` | what does this touch? | warms everything a file, symbol, or PR base reaches across languages |
|
|
34
32
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
The repo compiles to a typed graph whose edges carry **conductance**: imports are bridges, calls are solid, and shared literals — route paths, env keys — are the cross-language welds, weighted by specificity. Your interest is a heat source `s`; the map the model receives is the heat kernel run for time `t` over the graph Laplacian:
|
|
38
|
-
|
|
39
|
-
```text
|
|
40
|
-
v(t) = e^{−tL} · s
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
- **sketch** — large `t`, hub + anchor seeds: the whole repo in one blurry-but-valid silhouette.
|
|
44
|
-
- **focus** — small `t`, your query as the seed: the fovea on exactly that feature.
|
|
45
|
-
- **dwell** — `t` ×2 per call; only newly-luminous nodes are returned.
|
|
46
|
-
- **impact** — changed files as the seed; warmth = predicted blast radius.
|
|
33
|
+
Two slash commands on top:
|
|
47
34
|
|
|
48
|
-
|
|
35
|
+
- `/fovea status` for graph stats and sync state
|
|
36
|
+
- `/fovea settings` for an overlay in your TUI, styled after pi-fabric's `/fabric settings`
|
|
49
37
|
|
|
50
38
|
## Install
|
|
51
39
|
|
|
@@ -73,34 +61,28 @@ pi install /absolute/path/to/pi-fovea
|
|
|
73
61
|
|
|
74
62
|
</details>
|
|
75
63
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
- `/fovea status` — graph stats, sync on/off
|
|
79
|
-
- `/fovea settings` — an overlay built from the same SettingsList idiom as pi-fabric's `/fabric settings`
|
|
80
|
-
|
|
81
|
-
### CLI
|
|
82
|
-
|
|
83
|
-
The same ops, stateless and pipe-friendly — for agent shells, CI, and `llmc`-style uses:
|
|
64
|
+
There is also a package for any agent shell or CI:
|
|
84
65
|
|
|
85
66
|
```sh
|
|
86
67
|
fovea sketch /path/to/repo 900
|
|
87
68
|
fovea focus /path/to/repo "/v1/messages" 800
|
|
88
69
|
fovea impact /path/to/repo --base main 1200
|
|
70
|
+
fovea rules /path/to/repo
|
|
89
71
|
fovea status /path/to/repo
|
|
90
72
|
```
|
|
91
73
|
|
|
92
|
-
|
|
74
|
+
`fovea` runs `cli.ts` via `tsx`. Install `tsx` globally, or use `pnpm fovea` from a checkout.
|
|
93
75
|
|
|
94
|
-
## Turn
|
|
76
|
+
## Turn sync
|
|
95
77
|
|
|
96
|
-
After every assistant turn
|
|
78
|
+
Turn sync is on by default. After every assistant turn the graph is re-synced against your edits: unchanged files cost nothing because every parsed fact sits behind its content hash. The verdict is **green** or **red**:
|
|
97
79
|
|
|
98
|
-
- **green
|
|
99
|
-
- **red
|
|
80
|
+
- **green**: silence in the model's context. A clean-toast shows only if you enable `sync.ackClean`.
|
|
81
|
+
- **red**: a capped custom message naming route anchors that appeared or disappeared, plus files warmed by the edit cascade that the model has not focused on yet.
|
|
100
82
|
|
|
101
|
-
The first sync
|
|
83
|
+
The first sync seeds the baseline. The first drift after it calibrates the warm neighborhood, so a steady feature cone stays quiet. Sub sequent drift turns red.
|
|
102
84
|
|
|
103
|
-
|
|
85
|
+
Turn it off per repo or globally: `/fovea settings` → Turn sync, or
|
|
104
86
|
|
|
105
87
|
```sh
|
|
106
88
|
FOVEA_TURN_SYNC=off pi
|
|
@@ -108,33 +90,42 @@ FOVEA_TURN_SYNC=off pi
|
|
|
108
90
|
|
|
109
91
|
## Configuration
|
|
110
92
|
|
|
111
|
-
Global `~/.pi/agent/fovea.json
|
|
93
|
+
Global settings live in `~/.pi/agent/fovea.json`. A trusted repo-level override sits in `<repo>/.pi/fovea.json`. Two scopes, the same model pi-fabric uses with `fabric.json`.
|
|
112
94
|
|
|
113
95
|
| Key | Default | Meaning |
|
|
114
96
|
| --- | :-----: | ------- |
|
|
115
97
|
| `sync.enabled` | `true` | the turn-sync loop |
|
|
116
|
-
| `sync.budget` | `1024` |
|
|
117
|
-
| `sync.ackClean` | `false` | toast
|
|
118
|
-
| `sync.warmFileThreshold` | `2` |
|
|
119
|
-
| `tools.defaultBudget` | `2000` | fallback maxTokens for fovea_*
|
|
98
|
+
| `sync.budget` | `1024` | token cap for the red report seen by the model |
|
|
99
|
+
| `sync.ackClean` | `false` | toast after clean structural turns |
|
|
100
|
+
| `sync.warmFileThreshold` | `2` | warmed files unseen by the model that justify turning red |
|
|
101
|
+
| `tools.defaultBudget` | `2000` | fallback maxTokens for the fovea_* tools |
|
|
120
102
|
|
|
121
|
-
##
|
|
103
|
+
## How routes are found
|
|
122
104
|
|
|
123
|
-
|
|
105
|
+
Route anchors come from port shapes, and five shapes cover almost the whole ecosystem:
|
|
124
106
|
|
|
125
107
|
| Port shape | Examples |
|
|
126
108
|
|---|---|
|
|
127
|
-
| `recv.verb("path", handlers…)` | express, koa, fastify, hono, gin, echo, chi, net/http
|
|
128
|
-
|
|
|
109
|
+
| `recv.verb("path", handlers…)` | express, koa, fastify, hono, gin, echo, chi, net/http |
|
|
110
|
+
| annotation + optional class prefix | NestJS `@Controller + @Get`, Flask and FastAPI decorators, Spring `@RequestMapping + @GetMapping` |
|
|
129
111
|
| verb embedded in the path | Go 1.22 `mux.HandleFunc("GET /x", h)` |
|
|
130
112
|
| verb as first string argument | chi `r.Method("GET", path, h)`, aiohttp `router.add_route("GET", path, h)` |
|
|
131
113
|
| receiver-less DSL macros | Rails `routes.rb`, Phoenix `router.ex`, Django `path()`, Ktor `routing { get("/x") {} }` |
|
|
132
114
|
|
|
133
|
-
File-convention routers
|
|
115
|
+
File-convention routers declare paths nowhere in code. Next.js App Router, Pages Router, SvelteKit, Nuxt, and Astro anchors therefore derive from file paths, with the verb read off exported handler names or filename suffixes.
|
|
116
|
+
|
|
117
|
+
### Discovery mode
|
|
134
118
|
|
|
135
|
-
|
|
119
|
+
When a repo writes routes in a shape fovea has never seen, the literal pass harvests every call shape and promotes statistically solid ones into implicit rules. Discovered anchors carry half the conductance of declared ones and appear with a `△` sigil. Turn sync reports their churn without letting an unconfirmed hypothesis turn the verdict red. A hub upgrades to first-class the moment a known rule matches any of its sites.
|
|
120
|
+
|
|
121
|
+
```sh
|
|
122
|
+
fovea anchors <root> --discovered # the △ hypothesis hubs only
|
|
123
|
+
fovea rules <root> # promoted rules with evidence
|
|
124
|
+
fovea rules <root> --sigs # every path-touching signature, by precision
|
|
125
|
+
fovea rules <root> --adopt # persist promotions into .fovea/rules.json
|
|
126
|
+
```
|
|
136
127
|
|
|
137
|
-
|
|
128
|
+
`.fovea/rules.json` pins community or project rules in the repo:
|
|
138
129
|
|
|
139
130
|
```json
|
|
140
131
|
{
|
|
@@ -144,23 +135,43 @@ Drop `.fovea/rules.json` in a repo to extend anchor detection beyond the built-i
|
|
|
144
135
|
}
|
|
145
136
|
```
|
|
146
137
|
|
|
147
|
-
Changing the rules file invalidates
|
|
138
|
+
A rule may declare `prefixPattern` so a class-level prefix like `@Controller('api/airports')` composes with per-method paths. Changing the rules file invalidates the anchor extraction cache alone; parsed facts above it carry over.
|
|
139
|
+
|
|
140
|
+
**Blind spots**, logged in `src/core/anchors.ts`: Rust proc-macro attributes (actix `#[get("/x")]`), constructor-assigned prefixes (Flask Blueprint, FastAPI `APIRouter(prefix=…)`, chi `Mount`, Express `Router` mounts), `scope` and `namespace` nesting in Phoenix, Rails, or Django `include()`, and tRPC/GraphQL/gRPC (no path token exists to anchor on).
|
|
141
|
+
|
|
142
|
+
## How it works
|
|
143
|
+
|
|
144
|
+
The repo compiles to a typed graph. Your question is a source vector $s$ over its nodes, and the field the model receives is the heat kernel at time $t$ over the Laplacian $L$:
|
|
145
|
+
|
|
146
|
+
$$
|
|
147
|
+
v(t) = e^{-tL} \cdot s \quad \text{with} \quad L = I - D^{-1/2} W D^{-1/2}
|
|
148
|
+
$$
|
|
149
|
+
|
|
150
|
+
The four tools are the same operator at four timescales: sketch at $t=16$ with hub and anchor seeds, focus at $t=4$ with your query as seed, dwell doubling $t$ per call with a disclosed-set delta, and impact using the changed files as seed.
|
|
151
|
+
|
|
152
|
+
The kernel is evaluated with a Chebyshev expansion. Rescale $M = L - I$ so the spectrum sits in $[-1,1]$; then with $T_k$ the Chebyshev polynomials and $I_k$ the modified Bessel functions:
|
|
153
|
+
|
|
154
|
+
$$
|
|
155
|
+
e^{-tL} = e^{-t} \left[ I_0(t) T_0(M) + 2 \sum_{k\ge 1} (-1)^k I_k(t) T_k(M) \right]
|
|
156
|
+
$$
|
|
157
|
+
|
|
158
|
+
The vectors $T_k(M) s$ are cached in the session. A new timescale costs coefficient recombination, never a second graph walk.
|
|
159
|
+
|
|
160
|
+
Discovery asks how often the argument at one slot of one call shape carries a route path, and promotes the shape past a Jeffreys-smoothed posterior:
|
|
148
161
|
|
|
149
|
-
|
|
162
|
+
$$
|
|
163
|
+
\hat{p} = \frac{\mathrm{pathN} + \frac{1}{2}}{\mathrm{n} + 1} \ge 0.55 \quad \text{with} \quad \mathrm{n} \ge 4 \text{ sites and} \ge 2 \text{ files}
|
|
164
|
+
$$
|
|
150
165
|
|
|
151
|
-
|
|
166
|
+
Measured against eight cloned projects, corpus junk sits below $\hat{p} \approx 0.27$ and real route shapes above $\hat{p} \approx 0.75$. The cutoff stays mid-cliff regardless of repo size.
|
|
152
167
|
|
|
153
|
-
-
|
|
154
|
-
- **literal joins** — route paths, env keys, OpenAPI operation paths; document-frequency-gated cliques so rare literals bridge strongly and ubiquitous ones don't become gravity wells
|
|
155
|
-
- **co-change** — mined from recent git history (Jaccard-tilted, per-file capped, cached by HEAD), so files that commute together warm each other even without a static edge
|
|
156
|
-
- **feature hubs** — route declarations and every client call of the same normalized path collapse to ONE anchor node: where client, server, and spec meet
|
|
157
|
-
- **basins** — where there are no routes at all (CLIs, kernels), sketch infers implicit features as conductance-cut regions around triangle-dense seeds
|
|
168
|
+
Lineage: spectral-graph wavelets evaluated by shared Chebyshev recurrence, progressive image coding where the budget is a bitrate over significance-ordered coefficients, and foveated rendering. Aider's PageRank repo map is the fixed-timescale special case of this field. The full walkthrough of conductance tiers, specificity bridges, hub gravity, and basins lives in [docs/heat-diffusion.md](docs/heat-diffusion.md).
|
|
158
169
|
|
|
159
|
-
##
|
|
170
|
+
## Languages
|
|
160
171
|
|
|
161
|
-
Full symbol
|
|
162
|
-
Outline-based symbols
|
|
163
|
-
Config joins through literals: **YAML
|
|
172
|
+
Full symbol and call extraction: **TypeScript, TSX, JavaScript, Python, Go, and Rust**.
|
|
173
|
+
Outline-based symbols: **Elixir, Ruby, C, C++, Java, Kotlin, Lua, PHP, Swift, Scala, Haskell, and Bash**.
|
|
174
|
+
Config joins through literals: **YAML, JSON, TOML, env, Markdown, and OpenAPI**.
|
|
164
175
|
|
|
165
176
|
## Development
|
|
166
177
|
|
|
@@ -170,6 +181,6 @@ pnpm run check # typecheck + full vitest suite
|
|
|
170
181
|
pnpm run bench # rate–distortion bench against ../pi-fabric
|
|
171
182
|
```
|
|
172
183
|
|
|
173
|
-
pi loads the extension straight from `src/` via jiti
|
|
184
|
+
pi loads the extension straight from `src/` via jiti; there is no build step. Per-repo caches live in `$TMPDIR` behind per-file content sha1 hashes, and only dirty files re-run ast-grep. Bump `CACHE_VERSION` in `src/core/build.ts` whenever extractor semantics change.
|
|
174
185
|
|
|
175
186
|
[MIT](LICENSE).
|
package/cli.ts
CHANGED
|
@@ -6,12 +6,16 @@
|
|
|
6
6
|
// fovea dwell [root] [factor] [budget] (deepens the in-process focus)
|
|
7
7
|
// fovea impact [root] [--files a,b] [--symbols x,y] [--base ref] [--no-uncommitted] [budget]
|
|
8
8
|
// fovea anchors [root] [filter] (every feature anchor, sorted)
|
|
9
|
+
// fovea rules [root] (tier-3 discovered shape hypotheses)
|
|
9
10
|
//
|
|
10
11
|
// The CLI is stateless across invocations (dwell needs a prior focus in the
|
|
11
12
|
// same process — combine ops inside pi, where sessions persist); stdout is
|
|
12
13
|
// the rendered field, nothing else, so it composes with head/grep/$().
|
|
13
14
|
|
|
15
|
+
import { statSync } from "node:fs";
|
|
14
16
|
import { ensureState, sketch, focus, dwell, impact } from "./src/core/ops.js";
|
|
17
|
+
import { aggregateFiles, posterior, promote } from "./src/core/discover.js";
|
|
18
|
+
import { DEFAULT_PACK } from "./src/core/anchors.js";
|
|
15
19
|
|
|
16
20
|
const [, , cmd = "status", ...argv] = process.argv;
|
|
17
21
|
|
|
@@ -37,9 +41,15 @@ const numAt = (i: number): number | undefined => {
|
|
|
37
41
|
return Number.isFinite(n) && n > 0 ? n : undefined;
|
|
38
42
|
};
|
|
39
43
|
// Root is the first positional that names a path; everything else is arg data.
|
|
44
|
+
// Existing directories count as paths even when bare ("next", "kernel").
|
|
40
45
|
const rootAt = (i: number): string => {
|
|
41
46
|
const p = pos[i];
|
|
42
|
-
|
|
47
|
+
if (p === undefined) return ".";
|
|
48
|
+
if (p.includes("/") || p === ".") return p;
|
|
49
|
+
try {
|
|
50
|
+
if (statSync(p).isDirectory()) return p;
|
|
51
|
+
} catch { /* not a path */ }
|
|
52
|
+
return ".";
|
|
43
53
|
};
|
|
44
54
|
|
|
45
55
|
try {
|
|
@@ -76,10 +86,49 @@ try {
|
|
|
76
86
|
const root = rootAt(0);
|
|
77
87
|
const filter = pos.find((p) => p !== root);
|
|
78
88
|
const rows = ensureState(root).graph.anchors
|
|
79
|
-
.map((a) => `${a.kind}\t${a.id}\t${a.file}:${a.line}`)
|
|
80
|
-
.filter((r) => !filter || r.includes(filter))
|
|
89
|
+
.map((a) => `${a.implicit ? "△" : " "}\t${a.kind}\t${a.id}\t${a.file}:${a.line}`)
|
|
90
|
+
.filter((r) => (!filter || r.includes(filter)) && (!flags.has("discovered") || r.startsWith("△")))
|
|
81
91
|
.sort();
|
|
82
92
|
out = rows.join("\n");
|
|
93
|
+
} else if (cmd === "rules") {
|
|
94
|
+
const root = rootAt(0);
|
|
95
|
+
const st = ensureState(root);
|
|
96
|
+
const sigs = aggregateFiles(Object.fromEntries(Object.entries(st.facts).map(([k, v]) => [k, v.sigs])));
|
|
97
|
+
const promoted = promote(sigs, DEFAULT_PACK);
|
|
98
|
+
if (flags.has("adopt") && promoted.length) {
|
|
99
|
+
const { mkdirSync, writeFileSync, readFileSync } = await import("node:fs");
|
|
100
|
+
const { join } = await import("node:path");
|
|
101
|
+
mkdirSync(join(root, ".fovea"), { recursive: true });
|
|
102
|
+
const rulesFile = join(root, ".fovea", "rules.json");
|
|
103
|
+
let existing = { rules: [] as unknown[] };
|
|
104
|
+
try { existing = JSON.parse(readFileSync(rulesFile, "utf8")); } catch { /* new */ }
|
|
105
|
+
const stamp = promoted.map((r) => ({
|
|
106
|
+
id: r.id.slice("implicit:".length),
|
|
107
|
+
langs: r.langs,
|
|
108
|
+
pattern: r.patterns[0],
|
|
109
|
+
methods: r.methods,
|
|
110
|
+
kind: r.kind,
|
|
111
|
+
}));
|
|
112
|
+
existing.rules = [...existing.rules, ...stamp];
|
|
113
|
+
writeFileSync(rulesFile, JSON.stringify(existing, null, 2) + "\n");
|
|
114
|
+
out = `wrote ${stamp.length} discovered rule(s) to .fovea/rules.json`;
|
|
115
|
+
} else if (flags.has("sigs")) {
|
|
116
|
+
out = sigs.filter((s) => s.pathN > 0)
|
|
117
|
+
.sort((a, b) => posterior(b.pathN, b.n) - posterior(a.pathN, a.n))
|
|
118
|
+
.map((s) => `${posterior(s.pathN, s.n).toFixed(2)} ${s.pathN}/${s.n} across ${s.files} files ${s.key}`)
|
|
119
|
+
.join("\n");
|
|
120
|
+
} else if (!promoted.length) {
|
|
121
|
+
out = "(no unknown shape passes the promotion floor — tier-1/2 coverage is doing fine)";
|
|
122
|
+
} else {
|
|
123
|
+
out = promoted.map((r) => JSON.stringify({
|
|
124
|
+
id: r.id.slice("implicit:".length),
|
|
125
|
+
langs: r.langs,
|
|
126
|
+
pattern: r.patterns[0],
|
|
127
|
+
methods: r.methods.replace(/\(\?i\)/, ""),
|
|
128
|
+
kind: r.kind,
|
|
129
|
+
_evidence: `p̂=${r.evidence.posterior.toFixed(2)} (${r.evidence.pathN}/${r.evidence.n} sites, ${r.evidence.files} files)`,
|
|
130
|
+
})).join("\n");
|
|
131
|
+
}
|
|
83
132
|
} else {
|
|
84
133
|
console.error(`unknown command: ${cmd}`);
|
|
85
134
|
process.exit(2);
|
package/package.json
CHANGED
package/src/core/anchors.ts
CHANGED
|
@@ -26,7 +26,8 @@ import type { Anchor } from "./types.js";
|
|
|
26
26
|
export interface AnchorRule {
|
|
27
27
|
id: string;
|
|
28
28
|
langs: string[];
|
|
29
|
-
pattern
|
|
29
|
+
/** Single ast-grep pattern; tier-3 synthesized rules use `patterns` instead. */
|
|
30
|
+
pattern?: string;
|
|
30
31
|
methods: string; // regex tested against the captured method metavar
|
|
31
32
|
kind: string;
|
|
32
33
|
/**
|
|
@@ -39,6 +40,10 @@ export interface AnchorRule {
|
|
|
39
40
|
verbFrom?: string;
|
|
40
41
|
/** Idiom writes paths mount-relative (Django `path("users/")`): root them. */
|
|
41
42
|
mountRoot?: boolean;
|
|
43
|
+
/** Synthesized tier-3 rules ship as variant lists (exact arity + trailing $$$H). */
|
|
44
|
+
patterns?: string[];
|
|
45
|
+
/** Discovered rules: half hub gravity until a real join upgrades them. */
|
|
46
|
+
implicit?: boolean;
|
|
42
47
|
}
|
|
43
48
|
|
|
44
49
|
const HTTP_VERB_RE = /^(?i:get|post|put|delete|patch|head|options)$/;
|
|
@@ -59,12 +64,19 @@ const PLACEHOLDER_ONLY = /^(:[A-Za-z_]\w*|\{[A-Za-z_]\w*\}|\[[A-Za-z_]\w*\])$/;
|
|
|
59
64
|
// Method names that are mounts, not verbs: Django urls, Rails match/root,
|
|
60
65
|
// Spring's umbrella RequestMapping. They anchor as ANY so the hub exists
|
|
61
66
|
// without pretending a verb was declared.
|
|
62
|
-
|
|
67
|
+
// Method names that mount a target rather than declare a verb. fetch(url)
|
|
68
|
+
// and redirect targets both resolve to GET; Django urlconfs and Rails mounts
|
|
69
|
+
// accept any verb.
|
|
70
|
+
const METHOD_ALIASES: Record<string, string> = {
|
|
71
|
+
PATH: "ANY", RE_PATH: "ANY", URL: "ANY", MATCH: "ANY", ROOT: "ANY",
|
|
72
|
+
REQUESTMAPPING: "ANY", RESOURCES: "ANY", FORWARD: "ANY",
|
|
73
|
+
FETCH: "GET", REDIRECT: "GET", RESPONDREDIRECT: "GET", REDIRECT_TO: "GET",
|
|
74
|
+
};
|
|
63
75
|
|
|
64
76
|
const deriveVerb = (method: string): string => {
|
|
65
77
|
let up = method.toUpperCase();
|
|
66
78
|
if (up.endsWith("MAPPING")) up = up.slice(0, -"MAPPING".length); // Spring GetMapping → GET
|
|
67
|
-
return
|
|
79
|
+
return METHOD_ALIASES[up] ?? up;
|
|
68
80
|
};
|
|
69
81
|
|
|
70
82
|
export const DEFAULT_PACK: AnchorRule[] = [
|
|
@@ -199,6 +211,24 @@ export const DEFAULT_PACK: AnchorRule[] = [
|
|
|
199
211
|
methods: "^add_url_rule$",
|
|
200
212
|
kind: "route",
|
|
201
213
|
},
|
|
214
|
+
{
|
|
215
|
+
// Client fetch with no receiver: fetch("/api/x"). Precision-audited by
|
|
216
|
+
// discovery (~93% path precision in the next.js clone corpus).
|
|
217
|
+
id: "fetch-bare",
|
|
218
|
+
langs: ["TypeScript", "Tsx", "JavaScript"],
|
|
219
|
+
pattern: "$M($P, $$$H)", // trailing $$$H absorbs the options bag; zero-arg tail matches fetch("/x") too
|
|
220
|
+
methods: "^fetch$",
|
|
221
|
+
kind: "route",
|
|
222
|
+
},
|
|
223
|
+
{
|
|
224
|
+
// Response-side route linkage: ktor respondRedirect("/myfiles") references
|
|
225
|
+
// an existing route without declaring it. Discovery found it at p̂≈0.81.
|
|
226
|
+
id: "ktor-respond-redirect",
|
|
227
|
+
langs: ["Kotlin"],
|
|
228
|
+
pattern: "$R.$M($P, $$$H)",
|
|
229
|
+
methods: "^respondRedirect$",
|
|
230
|
+
kind: "route",
|
|
231
|
+
},
|
|
202
232
|
{
|
|
203
233
|
id: "rust-router-chain",
|
|
204
234
|
langs: ["Rust"],
|
|
@@ -238,7 +268,8 @@ export const extractAnchors = (
|
|
|
238
268
|
if (p !== undefined && !prefixes.has(pm.file)) prefixes.set(pm.file, unquote(p));
|
|
239
269
|
}
|
|
240
270
|
}
|
|
241
|
-
|
|
271
|
+
const matchSets = patternRunAll(rule.patterns ?? [rule.pattern!], lang, langFiles, cwd);
|
|
272
|
+
for (const m of matchSets) {
|
|
242
273
|
const method = m.single.M;
|
|
243
274
|
const pathLike = m.single.P;
|
|
244
275
|
if (!method || !pathLike || !methodRe.test(method)) continue;
|
|
@@ -273,6 +304,7 @@ export const extractAnchors = (
|
|
|
273
304
|
nodeId: enclosing ?? `file:${m.file}`,
|
|
274
305
|
file: m.file,
|
|
275
306
|
line: m.line,
|
|
307
|
+
...(rule.implicit ? { implicit: true } : {}),
|
|
276
308
|
});
|
|
277
309
|
}
|
|
278
310
|
}
|
package/src/core/build.ts
CHANGED
|
@@ -8,10 +8,11 @@ import { existsSync, readFileSync, readdirSync, writeFileSync, mkdirSync } from
|
|
|
8
8
|
import { tmpdir } from "node:os";
|
|
9
9
|
import { basename, dirname, join as joinPath, posix } from "node:path";
|
|
10
10
|
import { spawnSync } from "node:child_process";
|
|
11
|
-
import { LANG_BY_EXT, isBinaryExt, isConfigFile } from "./astgrep.js";
|
|
11
|
+
import { LANG_BY_EXT, isBinaryExt, isConfigFile, langOf } from "./astgrep.js";
|
|
12
12
|
import { extractCalls, extractImports, extractLiterals, extractSymbols, isTestFile } from "./extract.js";
|
|
13
13
|
import { buildJoinIndex } from "./join.js";
|
|
14
14
|
import { extractAnchors, extractFileRoutes, loadRepoRules } from "./anchors.js";
|
|
15
|
+
import { aggregateFiles, harvestFile, promote, type FileSigs, type SynthesizedRule } from "./discover.js";
|
|
15
16
|
import type { CallSite, Edge, Graph, ImportSite, LiteralSite, NodeRec, SymbolRec } from "./types.js";
|
|
16
17
|
import type { AnchorDraft } from "./anchors.js";
|
|
17
18
|
import { coChangePairs } from "./cochange.js";
|
|
@@ -23,9 +24,13 @@ export interface FileFacts {
|
|
|
23
24
|
calls: CallSite[];
|
|
24
25
|
literals: LiteralSite[];
|
|
25
26
|
anchors: AnchorDraft[];
|
|
27
|
+
// Tier-3 discovery: per-file histogram of call-shape signatures
|
|
28
|
+
// (sig -> [totalSites, pathSites]); aggregated repo-wide at load to promote
|
|
29
|
+
// statistically significant unknown shapes into implicit half-weight rules.
|
|
30
|
+
sigs?: FileSigs;
|
|
26
31
|
}
|
|
27
32
|
|
|
28
|
-
const CACHE_VERSION =
|
|
33
|
+
const CACHE_VERSION = 6; // bump when extractor semantics change
|
|
29
34
|
const IGNORE_DIRS = new Set([".git", "node_modules", "dist", "vendor", ".venv", "venv", "target", "coverage", ".next", "build", "__pycache__", ".pi", ".pi-fovea", "deps", "_build", ".tox", "Pods"]);
|
|
30
35
|
const MAX_FILES = 24000;
|
|
31
36
|
// Generated dependency manifests are enormous and carry no first-class routes.
|
|
@@ -103,8 +108,12 @@ export const loadFacts = (root: string, files: string[]): Record<string, FileFac
|
|
|
103
108
|
} catch {
|
|
104
109
|
cached = undefined;
|
|
105
110
|
}
|
|
106
|
-
const { pack:
|
|
107
|
-
const
|
|
111
|
+
const { pack: basePack, fileRoutes, sha: baseRulesSha } = loadRepoRules(root);
|
|
112
|
+
const anchorPack = [...basePack];
|
|
113
|
+
// Implicit rules promote AFTER the first fact pass: their evidence lives in
|
|
114
|
+
// freshly harvested sigs, so the pack can only be finalized once facts exist.
|
|
115
|
+
let implicitRules: SynthesizedRule[] = [];
|
|
116
|
+
let rulesSha = baseRulesSha;
|
|
108
117
|
if (cached && (cached.version !== CACHE_VERSION || cached.root !== root)) cached = undefined;
|
|
109
118
|
const facts: Record<string, FileFacts> = {};
|
|
110
119
|
const dirty: string[] = [];
|
|
@@ -133,12 +142,31 @@ export const loadFacts = (root: string, files: string[]): Record<string, FileFac
|
|
|
133
142
|
putByFile(extractImports(code, root), (f, v) => f.imports.push(v));
|
|
134
143
|
putByFile(extractCalls(code, root), (f, v) => f.calls.push(v));
|
|
135
144
|
putByFile(extractLiterals(dirty, root), (f, v) => f.literals.push(v));
|
|
145
|
+
// Tier-3 harvest: regex histogram of call-shape signatures per file. Cheap
|
|
146
|
+
// (line scan, no ast-grep) and cached alongside the other facts.
|
|
147
|
+
for (const f of code) {
|
|
148
|
+
const lang = langOf(f);
|
|
149
|
+
if (!lang) continue;
|
|
150
|
+
let text = "";
|
|
151
|
+
try { text = readFileSync(joinPath(root, f), "utf8"); } catch { continue; }
|
|
152
|
+
const sigs = harvestFile(lang, text);
|
|
153
|
+
if (Object.keys(sigs).length) facts[f]!.sigs = sigs;
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
// Tier-3: promote harvested signatures into implicit rules BEFORE anchors
|
|
157
|
+
// run, and fold their ids into the rules hash so a promotion change rebuilds
|
|
158
|
+
// anchors exactly like a rules.json edit would.
|
|
159
|
+
implicitRules = promote(aggregateFiles(Object.fromEntries(Object.entries(facts).map(([k, v]) => [k, v.sigs]))), anchorPack);
|
|
160
|
+
if (implicitRules.length) {
|
|
161
|
+
anchorPack.push(...implicitRules);
|
|
162
|
+
rulesSha = createHash("sha1").update(baseRulesSha).update(JSON.stringify(implicitRules.map((r) => r.id).sort())).digest("hex");
|
|
136
163
|
}
|
|
164
|
+
const rulesChangedFinal = cached !== undefined && cached.rulesSha !== rulesSha;
|
|
137
165
|
// Anchors need enclosing-symbol resolution. Symbols, imports, calls and
|
|
138
166
|
// literals are rules-independent — when only the rule pack changed, re-run
|
|
139
167
|
// anchor extraction over every code file against cached symbols and keep
|
|
140
168
|
// every other fact (green-node reuse one level up).
|
|
141
|
-
const anchorTargets =
|
|
169
|
+
const anchorTargets = rulesChangedFinal
|
|
142
170
|
? files.filter((f) => !isConfigFile(f))
|
|
143
171
|
: dirty.filter((f) => !isConfigFile(f));
|
|
144
172
|
if (anchorTargets.length) {
|
|
@@ -151,7 +179,7 @@ export const loadFacts = (root: string, files: string[]): Record<string, FileFac
|
|
|
151
179
|
const symsByFile = new Map<string, SymbolRec[]>();
|
|
152
180
|
for (const rel of anchorTargets) {
|
|
153
181
|
symsByFile.set(rel, facts[rel]?.symbols.length ? facts[rel]!.symbols : (cached?.facts[rel]?.symbols ?? []));
|
|
154
|
-
if (
|
|
182
|
+
if (rulesChangedFinal) facts[rel] && (facts[rel]!.anchors = []);
|
|
155
183
|
}
|
|
156
184
|
const enclosingId = (file: string, line: number): string | undefined => {
|
|
157
185
|
const syms = symsByFile.get(file) ?? [];
|
|
@@ -391,13 +419,18 @@ export const assembleGraph = (root: string, files: string[], facts: Record<strin
|
|
|
391
419
|
for (const [label, sites] of draftsByLabel) {
|
|
392
420
|
const first = sites[0]!;
|
|
393
421
|
const filesOf = [...new Set(sites.map((s) => s.file))];
|
|
394
|
-
|
|
422
|
+
// A hub is implicit only when EVERY site came from a discovered rule — a
|
|
423
|
+
// match by any real rule upgrades it back to first-class instantly.
|
|
424
|
+
const hubImplicit = sites.every((s) => s.implicit === true);
|
|
425
|
+
anchors.push({ id: label, kind: first.kind, label: sites.length > 1 ? `${label} · ${sites.length} sites` : label, nodeId: first.nodeId, file: first.file, line: first.line, ...(hubImplicit ? { implicit: true } : {}) });
|
|
395
426
|
const idx = addNode(nodes, seen, {
|
|
396
427
|
id: `anchor:${label}`, name: label, kind: "anchor", file: first.file, line: first.line,
|
|
397
|
-
sig: sites.length > 1 ? `${label} (${sites.length} sites)` : label
|
|
428
|
+
sig: `${hubImplicit ? "(△ discovered) " : ""}${sites.length > 1 ? `${label} (${sites.length} sites)` : label}`, lang: "anchor",
|
|
398
429
|
});
|
|
399
430
|
(byFile.get(first.file) ?? byFile.set(first.file, []).get(first.file)!).push(idx);
|
|
400
|
-
|
|
431
|
+
// Tier-3 hubs prove themselves at half conductance; a later literal join
|
|
432
|
+
// against a first-class hub can still warm them via the channel edges.
|
|
433
|
+
const w = (hubImplicit ? 0.5 : 1) / Math.sqrt(sites.length);
|
|
401
434
|
for (const s of sites) {
|
|
402
435
|
const handler = seen.get(s.nodeId) ?? fileIdx.get(s.file)!;
|
|
403
436
|
pushEdge(idx, handler, "anchors", w);
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
// Tier-3: autonomy for route extraction. The static pack catches known port
|
|
2
|
+
// shapes; this module watches the literal channel for shapes the pack does NOT
|
|
3
|
+
// declare, promotes statistically significant ones into synthesized "implicit"
|
|
4
|
+
// rules at half hub gravity, and lets confirmatory joins against real hubs
|
|
5
|
+
// upgrade them to full first-class weight.
|
|
6
|
+
//
|
|
7
|
+
// The promotion statistic is a per-argument conditional: given the string arg
|
|
8
|
+
// at position i of call-shape (lang·shape·callee), how often does it classify
|
|
9
|
+
// as a path? A Jeffreys-smoothed posterior with global floors decides — not raw
|
|
10
|
+
// frequency, under which chaff like `assertEquals(...)` dwarfs every real shape.
|
|
11
|
+
//
|
|
12
|
+
// Corpus audit (8 repos, 183 sigs n≥4): junk bands sit below p̂≈0.27, real
|
|
13
|
+
// shapes above p̂≈0.75 — the 0.55 line is a cliff, not a gradient.
|
|
14
|
+
//
|
|
15
|
+
// Harvest is line-regex over source text — no ast-grep pass needed. Per file we
|
|
16
|
+
// store a compact signature histogram (sig -> [sites, pathSites]); promotions
|
|
17
|
+
// aggregate across ALL files of the repo, so a dependency update or a style
|
|
18
|
+
// drift can only flip a marginal signature when its repo-wide evidence moves.
|
|
19
|
+
|
|
20
|
+
import { classifyLiteral } from "./join.js";
|
|
21
|
+
|
|
22
|
+
export type Shape = "recv" | "bare" | "dec";
|
|
23
|
+
|
|
24
|
+
// Compacted per-file histogram: sigKey -> [totalSites, pathSites]
|
|
25
|
+
export type FileSigs = Record<string, [number, number]>;
|
|
26
|
+
|
|
27
|
+
const QUOTED = /^[rbfuRBFU]{0,3}(["'`])([\s\S]*)\1$/;
|
|
28
|
+
const unquote = (s: string): string | undefined => {
|
|
29
|
+
const m = QUOTED.exec(s.trim());
|
|
30
|
+
return m ? m[2] : undefined;
|
|
31
|
+
};
|
|
32
|
+
|
|
33
|
+
// Line-local call scan: grabs the callee shape (recv chain, bare, decorator)
|
|
34
|
+
// and the raw arg list of every `X(...)` on the line. Multi-line calls still
|
|
35
|
+
// land because route strings overwhelmingly sit on the call's first line.
|
|
36
|
+
const CALL_LINE_RE = /(?<at>@)?(?<recv>(?:[A-Za-z_$][\w$]*\.)+)?(?<method>[A-Za-z_$][\w$]*)\s*\((?<args>[^()]*)\)/g;
|
|
37
|
+
const STRING_RE = /([rbfuRBFU]{0,3})(["'`])((?:\\.|(?!\2)[^\\])*)\2/g;
|
|
38
|
+
|
|
39
|
+
// Callees that are noise even at 100% precision (env/fs access, string ctors,
|
|
40
|
+
// module loading — dynamic import()/require() hits the path channel but the
|
|
41
|
+
// import edge already covers it, an anchor would double-count the site).
|
|
42
|
+
const CALLEE_DENY = new Set([
|
|
43
|
+
"join", "resolve", "dirname", "basename", "expand_path",
|
|
44
|
+
"readFile", "readFileSync", "existsSync", "open", "load", "loads",
|
|
45
|
+
"import", "require",
|
|
46
|
+
// String predicates: membership tests hit the path column at high rate but
|
|
47
|
+
// never refer to routes. (Distinguishing cause from noise is callee-semantic.)
|
|
48
|
+
"startsWith", "endsWith", "contains", "includes", "equals", "equalsIgnoreCase",
|
|
49
|
+
"matches", "matchesPattern", "useParams", "matchPath",
|
|
50
|
+
]);
|
|
51
|
+
|
|
52
|
+
export const harvestFile = (lang: string, text: string): FileSigs => {
|
|
53
|
+
const sigs: FileSigs = {};
|
|
54
|
+
for (const line of text.split("\n")) {
|
|
55
|
+
if (!line.includes("(") || !(line.includes('"') || line.includes("'") || line.includes("`"))) continue;
|
|
56
|
+
CALL_LINE_RE.lastIndex = 0;
|
|
57
|
+
let c: RegExpExecArray | null;
|
|
58
|
+
while ((c = CALL_LINE_RE.exec(line))) {
|
|
59
|
+
const method = c.groups?.method;
|
|
60
|
+
if (!method || CALLEE_DENY.has(method)) continue;
|
|
61
|
+
const shape: Shape = c.groups?.at ? "dec" : c.groups?.recv ? "recv" : "bare";
|
|
62
|
+
const argsText = c.groups?.args ?? "";
|
|
63
|
+
if (!argsText) continue;
|
|
64
|
+
// True argument index of each string literal (split tolerantly on commas).
|
|
65
|
+
const args = argsText.split(",");
|
|
66
|
+
let idx = 0;
|
|
67
|
+
for (const raw of args) {
|
|
68
|
+
STRING_RE.lastIndex = 0;
|
|
69
|
+
const sm = STRING_RE.exec(raw);
|
|
70
|
+
const idxNow = idx++;
|
|
71
|
+
if (!sm) continue;
|
|
72
|
+
const lit = sm[3];
|
|
73
|
+
if (lit === undefined || lit === "") continue;
|
|
74
|
+
const key = `${lang}|${shape}|${method}|${idxNow}`;
|
|
75
|
+
const rec = sigs[key] ?? [0, 0];
|
|
76
|
+
rec[0]++;
|
|
77
|
+
if (classifyLiteral(lit) === "path") rec[1]++;
|
|
78
|
+
sigs[key] = rec;
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
return sigs;
|
|
83
|
+
};
|
|
84
|
+
|
|
85
|
+
export interface SigStats {
|
|
86
|
+
key: string;
|
|
87
|
+
lang: string;
|
|
88
|
+
shape: Shape;
|
|
89
|
+
callee: string;
|
|
90
|
+
argIdx: number;
|
|
91
|
+
n: number;
|
|
92
|
+
pathN: number;
|
|
93
|
+
files: number;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
export const aggregateFiles = (perFile: Record<string, FileSigs | undefined>): SigStats[] => {
|
|
97
|
+
const agg = new Map<string, SigStats>();
|
|
98
|
+
for (const sigs of Object.values(perFile)) {
|
|
99
|
+
if (!sigs) continue;
|
|
100
|
+
for (const [key, [n, p]] of Object.entries(sigs)) {
|
|
101
|
+
const stat = agg.get(key);
|
|
102
|
+
if (stat) {
|
|
103
|
+
stat.n += n;
|
|
104
|
+
stat.pathN += p;
|
|
105
|
+
stat.files++;
|
|
106
|
+
} else {
|
|
107
|
+
const [lang, shape, callee, argIdx] = key.split("|");
|
|
108
|
+
agg.set(key, { key, lang: lang!, shape: shape as Shape, callee: callee!, argIdx: Number(argIdx), n, pathN: p, files: 1 });
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
return [...agg.values()];
|
|
113
|
+
};
|
|
114
|
+
|
|
115
|
+
/** Jeffreys-ish posterior: p̂ = (pathN + .5) / (n + 1). */
|
|
116
|
+
export const posterior = (pathN: number, n: number): number => (pathN + 0.5) / (n + 1);
|
|
117
|
+
|
|
118
|
+
export const MIN_SITES = 4;
|
|
119
|
+
export const MIN_FILES = 2;
|
|
120
|
+
export const MIN_POSTERIOR = 0.55;
|
|
121
|
+
|
|
122
|
+
const escapeRe = (s: string): string => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
123
|
+
|
|
124
|
+
// Pattern synthesis per shape and arg position: dummy metavars for the slots
|
|
125
|
+
// the corpus says are not the path, $P at the proven position. Two variants
|
|
126
|
+
// per site — exact arity and with a trailing $$$H absorb — because some
|
|
127
|
+
// dialects only match with explicit tail holes (Python `$X, $P, $$$H`).
|
|
128
|
+
export interface SynthesizedRule {
|
|
129
|
+
id: string;
|
|
130
|
+
langs: string[];
|
|
131
|
+
patterns: string[]; // patternRunAll variants
|
|
132
|
+
methods: string;
|
|
133
|
+
kind: string;
|
|
134
|
+
implicit: true;
|
|
135
|
+
evidence: { n: number; pathN: number; files: number; posterior: number };
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
export const synthesize = (s: SigStats): SynthesizedRule | undefined => {
|
|
139
|
+
const slots: string[] = [];
|
|
140
|
+
for (let i = 0; i <= s.argIdx; i++) slots.push(i === s.argIdx ? "$P" : `$X${i}`);
|
|
141
|
+
const inner = slots.join(", ");
|
|
142
|
+
let variants: string[];
|
|
143
|
+
switch (s.shape) {
|
|
144
|
+
case "recv": variants = [`$R.$M(${inner})`, `$R.$M(${inner}, $$$H)`]; break;
|
|
145
|
+
case "bare": variants = [`$M(${inner})`, `$M(${inner}, $$$H)`]; break;
|
|
146
|
+
case "dec": variants = [`@$M(${inner})`, `@$M(${inner}, $$$H)`]; break;
|
|
147
|
+
}
|
|
148
|
+
return {
|
|
149
|
+
id: `implicit:${s.lang.toLowerCase()}:${s.shape}:${s.callee}:${s.argIdx}`,
|
|
150
|
+
langs: [s.lang],
|
|
151
|
+
patterns: variants,
|
|
152
|
+
methods: `^(?i:${escapeRe(s.callee)})$`,
|
|
153
|
+
kind: "route",
|
|
154
|
+
implicit: true,
|
|
155
|
+
evidence: { n: s.n, pathN: s.pathN, files: s.files, posterior: posterior(s.pathN, s.n) },
|
|
156
|
+
};
|
|
157
|
+
};
|
|
158
|
+
|
|
159
|
+
// Shape-shape compatibility: a discovery is only NEW when no existing rule
|
|
160
|
+
// already binds the callee with a compatible pattern shape in that language.
|
|
161
|
+
// e.g. Java's @GetMapping is in the pack, so java:dec:GetMapping promotes nothing.
|
|
162
|
+
const shapeCompatPatterns: Record<Shape, RegExp> = {
|
|
163
|
+
recv: /^\$R\.\$M\(/,
|
|
164
|
+
bare: /^\$M[(\s]/,
|
|
165
|
+
dec: /^@\$(R\.)?M\(/,
|
|
166
|
+
};
|
|
167
|
+
|
|
168
|
+
const isCovered = (sig: SigStats, pack: Array<{ langs: string[]; methods: string; pattern?: string; patterns?: string[] }>): boolean => {
|
|
169
|
+
const mre = (methods: string, callee: string): boolean => new RegExp(methods.replace(/^\^\(\?i\)/, "^(?i:")).test(callee);
|
|
170
|
+
return pack.some((r) => {
|
|
171
|
+
if (!r.langs.includes(sig.lang)) return false;
|
|
172
|
+
if (!mre(r.methods, sig.callee)) return false;
|
|
173
|
+
const pats = r.patterns ?? (r.pattern ? [r.pattern] : []);
|
|
174
|
+
return pats.some((p) => shapeCompatPatterns[sig.shape].test(p));
|
|
175
|
+
});
|
|
176
|
+
};
|
|
177
|
+
|
|
178
|
+
export const promote = (sigs: SigStats[], pack: Array<{ id: string; langs: string[]; methods: string; pattern?: string; patterns?: string[] }> = []): SynthesizedRule[] => {
|
|
179
|
+
const out: SynthesizedRule[] = [];
|
|
180
|
+
for (const s of sigs) {
|
|
181
|
+
if (s.n < MIN_SITES || s.files < MIN_FILES) continue;
|
|
182
|
+
if (posterior(s.pathN, s.n) < MIN_POSTERIOR) continue;
|
|
183
|
+
if (isCovered(s, pack)) continue;
|
|
184
|
+
const r = synthesize(s);
|
|
185
|
+
if (r) out.push(r);
|
|
186
|
+
}
|
|
187
|
+
return out;
|
|
188
|
+
};
|
package/src/core/join.ts
CHANGED
|
@@ -14,7 +14,7 @@ export interface LitOccurrence { node: number; line: number; file: string; }
|
|
|
14
14
|
|
|
15
15
|
export interface JoinEdge { a: number; b: number; w: number; }
|
|
16
16
|
|
|
17
|
-
const PLACEHOLDER_SEGMENT = /^(?::[^/]+|\{[^}/]*\}|\$\{[^}/]*\}
|
|
17
|
+
const PLACEHOLDER_SEGMENT = /^(?::[^/]+|\{[^}/]*\}|\$\{[^}/]*\}|\$[A-Za-z_]\w*|<[^/>]+>|\*+)$/; // $x = Kotlin template shorthand
|
|
18
18
|
const WORD_RE = /^[A-Za-z][\w$.\-]{6,63}$/;
|
|
19
19
|
const URLISH_RE = /^(?:https?|wss?):\/\/[^/]+/;
|
|
20
20
|
|
package/src/core/sync.ts
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
// Turn-sync: the default-on feedback loop the extension was built around.
|
|
2
2
|
// After each assistant turn, if the repo's facts version drifted ANY edit
|
|
3
|
-
// re-syncs, regardless of
|
|
4
|
-
//
|
|
5
|
-
//
|
|
3
|
+
// re-syncs, regardless of the mutation path — pi's edit/write tools, a
|
|
4
|
+
// fabric_exec inner pi.edit, a bash heredoc, a subagent run, or an editor
|
|
5
|
+
// save outside the session all land the same way, because drift is measured
|
|
6
|
+
// by diffing the baseline's content hashes against the current facts instead
|
|
7
|
+
// of trusting tool events or git.
|
|
6
8
|
//
|
|
7
9
|
// red = route anchors appeared/vanished (structural feature churn)
|
|
8
10
|
// OR warm undisclosed files >= warmFileThreshold (unseen blast radius)
|
|
@@ -11,13 +13,17 @@
|
|
|
11
13
|
// The first sync of a session only establishes the baseline (never red).
|
|
12
14
|
// Baselines reset on /new and /fork alongside fovea sessions.
|
|
13
15
|
|
|
14
|
-
import { ensureState, impact
|
|
16
|
+
import { ensureState, impact } from "./ops.js";
|
|
15
17
|
import type { RepoState } from "./ops.js";
|
|
16
18
|
import { getSession } from "./session.js";
|
|
17
19
|
|
|
18
20
|
interface SyncBaseline {
|
|
19
21
|
version: string;
|
|
20
22
|
anchors: Set<string>;
|
|
23
|
+
/** file -> content sha1 at baseline. Diffing this against the current facts
|
|
24
|
+
* yields the exact changed-file set for any mutation path — no dependence
|
|
25
|
+
* on which tool executed the write, nor on git. */
|
|
26
|
+
shas: Map<string, string>;
|
|
21
27
|
/** Steady-state warmth recorded on the most recent sync. undefined = "the
|
|
22
28
|
* first drift after baseline calibrates the neighborhood instead of
|
|
23
29
|
* escalating" — a file list appears after that calibration sync. */
|
|
@@ -29,7 +35,8 @@ const baselines = new Map<string, SyncBaseline>();
|
|
|
29
35
|
export const resetSyncBaselines = (): void => baselines.clear();
|
|
30
36
|
|
|
31
37
|
export interface SyncParams {
|
|
32
|
-
/**
|
|
38
|
+
/** Optional drift hints (e.g. files touched by pi's edit/write tools this
|
|
39
|
+
* turn). Unioned into the warmth seeds; never the source of truth. */
|
|
33
40
|
files?: string[];
|
|
34
41
|
budget: number;
|
|
35
42
|
warmFileThreshold: number;
|
|
@@ -48,6 +55,7 @@ export interface SyncOutcome {
|
|
|
48
55
|
const snapshot = (state: RepoState): SyncBaseline => ({
|
|
49
56
|
version: state.version,
|
|
50
57
|
anchors: new Set(state.graph.anchors.map((a) => a.id)),
|
|
58
|
+
shas: new Map(Object.entries(state.facts).map(([f, x]) => [f, x.sha1])),
|
|
51
59
|
});
|
|
52
60
|
|
|
53
61
|
export const sync = (root: string, params: SyncParams, now?: RepoState): SyncOutcome => {
|
|
@@ -64,10 +72,14 @@ export const sync = (root: string, params: SyncParams, now?: RepoState): SyncOut
|
|
|
64
72
|
};
|
|
65
73
|
}
|
|
66
74
|
|
|
67
|
-
// Version drifted: measure what moved.
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
const
|
|
75
|
+
// Version drifted: measure what moved. Implicit (tier-3 discovered) hubs
|
|
76
|
+
// churn is reported but NEVER escalates alone — hypotheses with no first-class
|
|
77
|
+
// backing don't get to wake the model with a red verdict.
|
|
78
|
+
const current = new Map(state.graph.anchors.map((a) => [a.id, a.implicit === true]));
|
|
79
|
+
const currentIds = new Set(current.keys());
|
|
80
|
+
const added = [...currentIds].filter((id) => !prev.anchors.has(id));
|
|
81
|
+
const removed = [...prev.anchors].filter((id) => !currentIds.has(id));
|
|
82
|
+
const newlyImplicit = added.filter((id) => current.get(id));
|
|
71
83
|
|
|
72
84
|
const session = getSession(root);
|
|
73
85
|
const disclosedFiles = new Set<string>();
|
|
@@ -76,8 +88,15 @@ export const sync = (root: string, params: SyncParams, now?: RepoState): SyncOut
|
|
|
76
88
|
if (at >= 0) disclosedFiles.add(id.slice(at + 1));
|
|
77
89
|
}
|
|
78
90
|
|
|
79
|
-
|
|
80
|
-
|
|
91
|
+
// Exact change set: facts whose content hash moved since the baseline.
|
|
92
|
+
// Deleted files can't warm anything (absent from the graph) but ride along
|
|
93
|
+
// in details for observability.
|
|
94
|
+
const hinted = (params.files ?? []).filter((f) => state.graph.byFile.has(f));
|
|
95
|
+
const changed = Object.keys(state.facts).filter(
|
|
96
|
+
(f) => prev.shas.get(f) !== state.facts[f]!.sha1 && state.graph.byFile.has(f),
|
|
97
|
+
);
|
|
98
|
+
const deleted = [...prev.shas.keys()].filter((f) => !(f in state.facts));
|
|
99
|
+
const files = [...new Set([...hinted, ...changed])];
|
|
81
100
|
|
|
82
101
|
let warmNow: Set<string> = new Set();
|
|
83
102
|
let warmNew: string[] = [];
|
|
@@ -92,17 +111,18 @@ export const sync = (root: string, params: SyncParams, now?: RepoState): SyncOut
|
|
|
92
111
|
|
|
93
112
|
baselines.set(root, { ...snapshot(state), warmed: warmNow });
|
|
94
113
|
|
|
95
|
-
const red = added.length + removed.length > 0 || warmNew.length >= Math.max(1, params.warmFileThreshold);
|
|
114
|
+
const red = (added.length - newlyImplicit.length) + removed.length > 0 || warmNew.length >= Math.max(1, params.warmFileThreshold);
|
|
96
115
|
if (!red) {
|
|
97
116
|
return {
|
|
98
117
|
structural: true, red: false, tokens: 0,
|
|
99
|
-
details: { version: state.version, anchorsDelta: added.length - removed.length, warmNew: warmNew.length },
|
|
118
|
+
details: { version: state.version, anchorsDelta: added.length - removed.length, warmNew: warmNew.length, deletedFiles: deleted },
|
|
100
119
|
};
|
|
101
120
|
}
|
|
102
121
|
|
|
103
122
|
const lines: string[] = [];
|
|
104
123
|
lines.push(`fovea sync · v ${state.version} · edit cascade did not stay local`);
|
|
105
|
-
for (const id of added.slice(0, 12)) lines.push(` ⚑=new ${id}`);
|
|
124
|
+
for (const id of added.filter((a) => !newlyImplicit.includes(a)).slice(0, 12)) lines.push(` ⚑=new ${id}`);
|
|
125
|
+
for (const id of newlyImplicit.slice(0, 6)) lines.push(` △ newly discovered hub ${id}`);
|
|
106
126
|
for (const id of removed.slice(0, 12)) lines.push(` ⚑-removed ${id}`);
|
|
107
127
|
if (warmNew.length) {
|
|
108
128
|
lines.push(` newly warm undisclosed files (revisit with fovea_focus):`);
|
|
@@ -111,6 +131,6 @@ export const sync = (root: string, params: SyncParams, now?: RepoState): SyncOut
|
|
|
111
131
|
const text = lines.join("\n");
|
|
112
132
|
return {
|
|
113
133
|
structural: true, red: true, text, tokens: Math.ceil(text.length / 4),
|
|
114
|
-
details: { version: state.version, added, removed, warmNew },
|
|
134
|
+
details: { version: state.version, added, removed, warmNew, deletedFiles: deleted },
|
|
115
135
|
};
|
|
116
136
|
};
|
package/src/core/types.ts
CHANGED
package/src/index.ts
CHANGED
|
@@ -38,11 +38,11 @@ export default function fovea(pi: ExtensionAPI) {
|
|
|
38
38
|
}
|
|
39
39
|
});
|
|
40
40
|
|
|
41
|
-
// Turn-sync loop.
|
|
42
|
-
//
|
|
43
|
-
//
|
|
44
|
-
//
|
|
45
|
-
//
|
|
41
|
+
// Turn-sync loop. The tracker below is a hint accumulator only: pi's
|
|
42
|
+
// edit/write tool starts give the warmth pass a head start, but sync relies
|
|
43
|
+
// on content-hash drift, so identical detection covers fabric_exec inner
|
|
44
|
+
// pi.edit calls, bash mutations, subagents, and out-of-band editor saves.
|
|
45
|
+
// Pure conversation turns exit at zero cost through the version fast path.
|
|
46
46
|
let turnFiles: string[] = [];
|
|
47
47
|
pi.on("turn_start", () => {
|
|
48
48
|
turnFiles = [];
|