ph-code-graph 0.11.0__tar.gz → 0.13.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. ph_code_graph-0.13.0/PKG-INFO +641 -0
  2. ph_code_graph-0.13.0/README.md +612 -0
  3. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/pyproject.toml +15 -12
  4. ph_code_graph-0.13.0/src/ph_code_graph/__init__.py +32 -0
  5. ph_code_graph-0.13.0/src/ph_code_graph/__main__.py +6 -0
  6. ph_code_graph-0.13.0/src/ph_code_graph/_config.py +142 -0
  7. ph_code_graph-0.13.0/src/ph_code_graph/_extract.py +695 -0
  8. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/src/ph_code_graph/_store.py +201 -180
  9. ph_code_graph-0.13.0/src/ph_code_graph/bundle.yaml +22 -0
  10. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/src/ph_code_graph/cli.py +8 -7
  11. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/src/ph_code_graph/engine.py +306 -295
  12. ph_code_graph-0.13.0/src/ph_code_graph/row.py +195 -0
  13. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/src/ph_code_graph/skills/code-graph/SKILL.md +10 -4
  14. ph_code_graph-0.13.0/tests/code_graph_helpers.py +102 -0
  15. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/tests/test_code_graph.py +365 -272
  16. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/tests/test_code_graph_cli.py +97 -37
  17. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/tests/test_code_graph_row.py +85 -72
  18. ph_code_graph-0.11.0/PKG-INFO +0 -573
  19. ph_code_graph-0.11.0/README.md +0 -544
  20. ph_code_graph-0.11.0/src/ph_code_graph/__init__.py +0 -32
  21. ph_code_graph-0.11.0/src/ph_code_graph/__main__.py +0 -6
  22. ph_code_graph-0.11.0/src/ph_code_graph/_config.py +0 -135
  23. ph_code_graph-0.11.0/src/ph_code_graph/_extract.py +0 -602
  24. ph_code_graph-0.11.0/src/ph_code_graph/bundle.yaml +0 -21
  25. ph_code_graph-0.11.0/src/ph_code_graph/row.py +0 -184
  26. ph_code_graph-0.11.0/tests/code_graph_helpers.py +0 -97
  27. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/.gitignore +0 -0
  28. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/LICENSE +0 -0
  29. {ph_code_graph-0.11.0 → ph_code_graph-0.13.0}/src/ph_code_graph/py.typed +0 -0
@@ -0,0 +1,641 @@
1
+ Metadata-Version: 2.5
2
+ Name: ph-code-graph
3
+ Version: 0.13.0
4
+ Summary: pH plugin: `code_index` and `code_graph`, a Python-native tree-sitter code graph with line-accurate answers.
5
+ Project-URL: Homepage, https://github.com/chastabor/pH
6
+ Project-URL: Repository, https://github.com/chastabor/pH
7
+ Project-URL: Documentation, https://github.com/chastabor/pH/blob/main/docs/README.md
8
+ Project-URL: Issues, https://github.com/chastabor/pH/issues
9
+ Author: Charles Tabor
10
+ License-Expression: MIT
11
+ License-File: LICENSE
12
+ Keywords: agent,code-graph,code-search,llm,tree-sitter
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Operating System :: MacOS
16
+ Classifier: Operating System :: POSIX :: Linux
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
21
+ Classifier: Topic :: Software Development
22
+ Classifier: Typing :: Typed
23
+ Requires-Python: >=3.12
24
+ Requires-Dist: ph-core==0.13.0
25
+ Requires-Dist: ph-utcp==0.13.0
26
+ Requires-Dist: tree-sitter-language-pack>=1.16
27
+ Requires-Dist: tree-sitter>=0.25
28
+ Description-Content-Type: text/markdown
29
+
30
+ # ph-code-graph
31
+
32
+ *`code_index` and `code_graph`: ask a codebase about its own structure, and get
33
+ back a `path:start-end` that you can `read`.*
34
+
35
+ This package has two tools, a command that runs them, and a
36
+ [UTCP](https://utcp.io) manual that describes them. It needs no Node, no Rust
37
+ toolchain, and no submodule. It uses tree-sitter through a Python wheel, and
38
+ keeps the graph in stdlib `sqlite3`.
39
+
40
+ ```bash
41
+ ph-code-graph call code_index --paths packages/ph-core --format text
42
+ ph-code-graph call code_graph --mode impact --query claim_slot --format text
43
+ ```
44
+
45
+ The same two tools reach a model in three ways: inside a
46
+ [phern](https://github.com/chastabor/pH) session, typed at a shell, or registered
47
+ with any other harness that supports UTCP. Only the first way needs phern.
48
+
49
+ ## Installing
50
+
51
+ The package needs Python 3.12 or newer, on Linux or macOS.
52
+
53
+ **With phern, there is nothing to do.** `uv tool install phern` installs this
54
+ package too, and the `rlm-indexed` profile mounts it.
55
+
56
+ **Without phern**, for a shell or for another harness:
57
+
58
+ ```bash
59
+ uv tool install ph-code-graph # or: pip install ph-code-graph
60
+ ph-code-graph install # fetch the grammars now, not during a call
61
+ ph-code-graph status
62
+ ```
63
+
64
+ That installs the `ph-code-graph` command and nothing from phern. The
65
+ dependencies are `ph-core` (the registry that the tools are written against),
66
+ `ph-utcp` and the two UTCP libraries that it names, and tree-sitter with its
67
+ language pack. That is 24 packages and about 30 MB, measured in an empty
68
+ environment. There is no model and no torch. The grammars are fetched from
69
+ GitHub one time, by `ph-code-graph install` or by the first index that can reach
70
+ GitHub. After that, nothing needs the network.
71
+
72
+ **The command arrives in 0.11.** `ph-code-graph` 0.10.0 is the plugin that ran
73
+ inside a phern session. It has no command and no manual.
74
+
75
+ **From a checkout**, `uv sync` puts the command in the environment of the
76
+ workspace (`uv run ph-code-graph ...`), and
77
+ `uv tool install ./packages/ph-code-graph` puts it on `PATH`.
78
+
79
+ ## Three ways in
80
+
81
+ ### In a phern session
82
+
83
+ The `code-graph` row registers the manual of this package with `ph-utcp`, and the
84
+ two tools appear under their own names. A profile layers the `utcp` bundle and
85
+ then this bundle. `rlm-indexed` does that:
86
+
87
+ ```bash
88
+ phern --profile rlm-indexed -p "index packages/ph-core, then tell me what breaks if I change claim_slot"
89
+ ```
90
+
91
+ On another profile, add the two rows:
92
+
93
+ ```bash
94
+ phern --profile llama --provider llama --model <model> \
95
+ --patch '{insert: [{id: utcp, name: utcp}, {id: code-graph, name: code-graph}]}' \
96
+ -p "what calls claim_slot?"
97
+ ```
98
+
99
+ Or add them in a profile document:
100
+
101
+ ```yaml
102
+ - id: utcp
103
+ name: utcp
104
+ - id: code-graph
105
+ name: code-graph
106
+ config:
107
+ languages: [python]
108
+ ```
109
+
110
+ The index is built for you. Before the first step of the agent, an indexing run
111
+ starts in the background over the `index` scope of the row (see "The two
112
+ tools"). `/code-graph status` and `/code-graph install` ask the command about its
113
+ grammars, and `phern doctor` reports where the index is.
114
+
115
+ **Run `/code-graph install` one time, before the first session.** The grammars
116
+ come from GitHub, and a confined agent cannot reach GitHub. With an empty grammar
117
+ cache, the index of such an agent skips every file. The note that tells the agent
118
+ that the run ended then names each file as skipped because no parser was
119
+ available.
120
+
121
+ ### From a command line
122
+
123
+ Use the command line to try a query by hand, or to see why a query returned
124
+ nothing. Run the command **in the tree that you ask about**. The workspace is the
125
+ working directory, or the value of `--root`.
126
+
127
+ ```bash
128
+ ph-code-graph call code_index --paths packages/ph-core --format text
129
+ ph-code-graph call code_graph --mode callers --query claim_slot --format text
130
+ ph-code-graph tools # the tools and their flags
131
+ ph-code-graph status # grammars, and what the index holds
132
+ ```
133
+
134
+ | Command | What it does |
135
+ |---|---|
136
+ | `call TOOL [--field value ...]` | Runs one tool. Each field of the tool is a flag, or pass all fields as `--args-json '{...}'` |
137
+ | `tools` | Lists the tools and their flags |
138
+ | `status` | Shows which grammars are ready, where the index is, and what it holds |
139
+ | `install` | Fetches every grammar now |
140
+ | `manual` | Prints the UTCP manual. `manual --registration` prints what a harness registers |
141
+
142
+ **A call prints one JSON object** and nothing else: `{"ok": true, "value": ...,
143
+ "text": ...}`, or `{"ok": false, "error": {"code": ..., "message": ...}}`.
144
+ `value` is the result of the tool as data. `text` is the same result, written
145
+ for a model to read. `--format text` prints only `text`. The exit code is 0, or
146
+ 1 for a failure. A failure is also in the object in both cases.
147
+
148
+ **Every call is one command.** No process stays running between calls, and no
149
+ process listens on a socket. Two flags of `call` are for a run that takes a long
150
+ time:
151
+
152
+ - **`--progress`** reports the progress of the run on stderr as JSON Lines, one
153
+ JSON object on each line. While the run works, it writes
154
+ `{"state":"working","done":4,"total":8,"what":"files"}` at most every 0.5 s.
155
+ It then writes exactly one last line:
156
+ - `"state":"done"`, with `changed` and a `note`.
157
+ - `"state":"busy"`, when another run held the index and this run did nothing.
158
+ - `"state":"failed"`, with a `code` and a `failure`.
159
+
160
+ If the stream ends with no last line, the command was killed or it died.
161
+ stdout does not change: it carries one object.
162
+ - **`--wait SECONDS`** is how long the call waits for another run that holds the
163
+ index. It overrides the `wait_s` of the row. `--wait 0` answers immediately.
164
+
165
+ **Settings** are the settings of the session row, under the row id. They are read
166
+ from three places, and a later place overrides an earlier one:
167
+ `$PH_HOME/tools/ph-code-graph.yaml`, then `$PH_TOOL_CONFIG` (JSON), then
168
+ `--config FILE`:
169
+
170
+ ```yaml
171
+ # ~/.ph/tools/ph-code-graph.yaml
172
+ code-graph:
173
+ languages: [python]
174
+ index:
175
+ paths: [src]
176
+ exclude: ["**/tests/**"]
177
+ ```
178
+
179
+ **Four variables control where things go.** None of them has to be set:
180
+
181
+ | Variable | Default | What the command keeps there |
182
+ |---|---|---|
183
+ | `PH_HOME` | `~/.ph` | `tools/ph-code-graph.yaml`, the settings above |
184
+ | `PH_CACHE` | `$XDG_CACHE_HOME/ph`, which is usually `~/.cache/ph` | The grammars, the log, and the index when nothing else says where |
185
+ | `PH_TOOL_STATE` | unset | The directory for the index. `--state DIR` does the same |
186
+ | `PH_TOOL_CONFIG` | unset | The row settings as one JSON object, for a caller that writes no file |
187
+
188
+ See "Where things live" for each path.
189
+
190
+ ### From another harness, over UTCP
191
+
192
+ Install the package where the harness runs its commands. Then register what this
193
+ command prints:
194
+
195
+ ```bash
196
+ ph-code-graph manual --registration
197
+ ```
198
+
199
+ ```json
200
+ {"name": "ph_code_graph",
201
+ "call_template_type": "cli",
202
+ "commands": [{"command": "/path/to/python -P -m ph_code_graph manual"}],
203
+ "inherit_env_vars": ["PATH", "HOME", "...", "PH_HOME", "PH_CACHE", "PH_TOOL_STATE"]}
204
+ ```
205
+
206
+ The harness runs that command one time to read the manual. After that, it runs
207
+ the command that each tool names. This example uses the Python UTCP client,
208
+ which this package installs:
209
+
210
+ ```python
211
+ import asyncio, json, subprocess
212
+
213
+ from utcp.utcp_client import UtcpClient
214
+
215
+
216
+ async def main() -> None:
217
+ registration = json.loads(
218
+ subprocess.run(
219
+ ["ph-code-graph", "manual", "--registration"], check=True, capture_output=True
220
+ ).stdout
221
+ )
222
+ client = await UtcpClient.create(config={"manual_call_templates": [registration]})
223
+ await client.call_tool("ph_code_graph.code_index", {"paths": ["src"]})
224
+ found = await client.call_tool(
225
+ "ph_code_graph.code_graph", {"mode": "callers", "query": "claim_slot"}
226
+ )
227
+ print(found["text"] if found["ok"] else found["error"])
228
+
229
+
230
+ asyncio.run(main())
231
+ ```
232
+
233
+ What a harness needs to know:
234
+
235
+ - **The tools are `ph_code_graph.code_index` and `ph_code_graph.code_graph`.** A
236
+ client puts the name of the manual before each tool name.
237
+ - **The registration names this install.** Its command is the absolute path of
238
+ the interpreter in which the package is installed, so it needs no `PATH`.
239
+ Print the registration again after the package moves. `-P` keeps the working
240
+ directory off the module path, so the command imports no file from the tree.
241
+ - **The workspace is the directory in which the command starts.** Start the
242
+ harness in the tree, or make it run its commands there. A `working_dir` on the
243
+ registered template applies to reading the manual and not to the tool calls,
244
+ so it does not move the workspace.
245
+ - **Give the model `text`.** Each answer is the envelope above, parsed. `text` is
246
+ what a model in a phern session reads, and `value` is for code. A failure is
247
+ `ok: false` with a `code`. The UTCP command-line protocol ignores exit codes,
248
+ so the envelope is the only place that reports a failure.
249
+ - **Optional arguments can be left out.** The Python client logs "Missing
250
+ argument" for each one, and the call still runs.
251
+ - **Nothing watches the tree.** Make the model call `code_index` again after it
252
+ edits. A file that has not changed is not parsed again.
253
+ - **One index for each tree.** Without `PH_TOOL_STATE`, the index is kept under
254
+ `$PH_CACHE`, keyed by the workspace. A harness whose agents work in separate
255
+ trees needs nothing more. A harness that wants an index for each agent sets
256
+ `PH_TOOL_STATE` for each agent.
257
+ - **In a sandbox, the command writes three places**: the directory of the index,
258
+ `$PH_CACHE/tree-sitter`, and `$PH_CACHE/logs`. It writes nothing in the tree.
259
+ The grammars come from GitHub the first time, so run `ph-code-graph install`
260
+ outside the sandbox. After that the tools need no network.
261
+ - **One indexing run at a time.** A `code_index` call is one command, and it
262
+ finishes before it answers. A call that is made while another run holds the
263
+ index waits `wait_s` for that run. It then answers `state: indexing`: another
264
+ run is building the graph, and this call did nothing. `code_graph` never
265
+ waits. It answers from what is indexed so far, with `indexing: true` in its
266
+ value.
267
+ - **Progress is for a harness that starts the command itself.** A standard UTCP
268
+ client merges stderr into stdout, so no template of the manual carries
269
+ `--progress`. A harness that wants indexing to run in the background runs
270
+ `ph-code-graph call code_index --progress` in the tree and reads stdout and
271
+ stderr separately. Its model can call `code_graph` during the run.
272
+ - **The instructions for the model ship with the package**, as
273
+ `ph_code_graph/skills/code-graph/SKILL.md`: index first, ask the question that
274
+ you have, then read the narrow result.
275
+
276
+ The `ph-utcp` row of phern is one such harness. The next section describes what
277
+ it adds.
278
+
279
+ ### Where the tools run, and what limits them
280
+
281
+ The tools run in the process of the `ph-code-graph` command. In a pH session,
282
+ `ctx.shell` starts that command for the agent that called the tool, in the
283
+ workspace of that agent and under its sandbox. A child agent in its own worktree
284
+ therefore indexes that worktree, and the harness process never loads
285
+ tree-sitter.
286
+
287
+ **`permissions-fs` does not screen the reads of these tools.** Its rules govern
288
+ calls through the `ctx.fs` of a session, and a command is not such a call. The
289
+ sandbox limits the command, as it limits `bash`.
290
+
291
+ **A read-only session protects the repository, not the index.** These tools
292
+ change no file in the tree. They build data that is used to query the tree. The
293
+ index is therefore kept in the scratch directory of the calling agent,
294
+ `$PH_HOME/scratch/<session>/<agent>/code-graph/`. That directory is writable in
295
+ every sandbox mode, and the tree is writable only in `workspace-write`. An agent
296
+ that may not write a tree can still index it, and cannot write the tree by doing
297
+ so. Every agent shares the grammar cache and the log of the command, under
298
+ `$PH_CACHE`, and the row grants those two paths.
299
+
300
+ ## The two tools
301
+
302
+ **`code_index(paths?, glob?, exclude?, forget?)`** parses each file and records
303
+ what it defines and references.
304
+
305
+ When a call does not say what to index, the `index` block of the row decides:
306
+
307
+ ```yaml
308
+ - id: code-graph
309
+ name: code-graph
310
+ config:
311
+ index:
312
+ paths: [packages/ph-core] # directories or files; default ["."]
313
+ include: ["**/*.py"] # default ["**/*"]; the language filter is the real filter
314
+ exclude: ["**/tests/**", "**/vendor/**"]
315
+ on_mount: true # index this when a root session starts; default true
316
+ languages: [python]
317
+ wait_s: 15 # how long code_index waits before it answers "indexing"
318
+ ```
319
+
320
+ **Indexing starts without a request, and it does not delay a call.** Before the
321
+ agent of a root session takes its first step, `ph-utcp` starts one `code_index`
322
+ run over the `index` scope, in the background. The run is a job of that agent,
323
+ under its sandbox. The footer shows the count as it changes. The agent is told
324
+ when the run ends, if the run changed the graph. `on_mount: false` leaves
325
+ indexing to `code_index` calls.
326
+
327
+ A `code_index` call starts a run, or joins the same request if it is already in
328
+ progress, and waits `wait_s` for it:
329
+
330
+ - If the run ends in time, its result is the answer to the call.
331
+ - If not, the call answers `state: indexing`, with a line that gives the
332
+ progress of the run. The run continues, and the agent is told when it ends.
333
+
334
+ **One run at a time for each index.** A run holds the lock of the index while it
335
+ works. `state: indexing` means that another run holds the index and this call
336
+ did nothing. The totals beside it are what the graph holds so far. A different
337
+ request that is made during a run is not queued, so call `code_index` again
338
+ after the run ends. `code_graph` takes no lock and never waits. It answers from
339
+ what is indexed so far, and the `indexing` field of its value is `true` while a
340
+ run holds the index.
341
+
342
+ **A child agent builds its own graph only when it has a tree that it writes to
343
+ permanently.** A child that works in the checkout of its parent, or that can
344
+ only read the project, asks the graph of its parent. That includes every child
345
+ of a read-only fan-out. The results say which tree their paths belong to. A
346
+ child with its own worktree or overlay has files that diverge, so it gets its
347
+ own graph at its first call.
348
+
349
+ How the scope settings work:
350
+
351
+ - **`paths`** points the row at one package of a monorepo. A bare `code_index`
352
+ then stays under `max_files` and does not walk everything. A file that is
353
+ named directly in `paths` is indexed whatever is excluded.
354
+ - **`exclude`** is read against the workspace root, so `**/tests/**` means the
355
+ same thing under every entry of `paths`. A directory that it covers is never
356
+ entered.
357
+ - **The `glob` of a call** replaces `include` for that call.
358
+ - **The `exclude` of a call** is *added* to the `exclude` of the row. A call can
359
+ narrow the scope of a deployment, but it cannot remove one of its exclusions.
360
+
361
+ **Nothing watches the filesystem.** There is no daemon and no file watcher. The
362
+ index changes when `code_index` runs, and at no other time. After your own
363
+ edits, run it again. That is cheap by design, so it can be a habit.
364
+
365
+ **Indexing is incremental in two layers**, and the order matters:
366
+
367
+ 1. *Does this file need to be opened?* The answer comes from **git or jj**,
368
+ which already hold a content-addressed tree (`ph.seams.changes`). A file that
369
+ the version control vouches for is never read. Measured on
370
+ `packages/ph-core/src/ph`: a re-index of 137 unchanged files went from 0.10 s
371
+ to **0.03 s**, with zero reads and zero hashes. The `tree_state` call cost
372
+ 7 ms for the whole repository of 509 files.
373
+ 2. *Is the opened content different?* The answer comes from the **sha256** of
374
+ the file, exactly as before. It never comes from an mtime. A checkout or a
375
+ rebase changes timestamps without changing content, and nobody attributes the
376
+ cost correctly when files are indexed again because git touched them.
377
+
378
+ Layer 1 decides only what to open, so the guarantee of layer 2 is unchanged. A
379
+ skip is always justified by a content hash and never by a clock. A tree with no
380
+ version control loses layer 1 and nothing else.
381
+
382
+ **`code_graph(mode, query?, …)`** answers six questions:
383
+
384
+ | `mode` | Question | What comes back |
385
+ |---|---|---|
386
+ | `search` | "something about workspace tiers" | Symbols ranked by FTS5 over name and docstring |
387
+ | `define` | "where exactly is `claim_slot`" | Every definition of that name |
388
+ | `callers` | "what calls this" | The calling symbol **and the calling line** |
389
+ | `callees` | "what does this call" | The definitions that it reaches |
390
+ | `impact` | "what breaks if I change this" | Transitive callers, ring by ring |
391
+ | `entities` | "what is big here" | Definitions by span, biggest first |
392
+
393
+ Every result carries `path:start-line-end-line`. That is the purpose of the
394
+ package: the model finds what matters in one call, and then reads exactly that.
395
+
396
+ ## Using it from the RLM
397
+
398
+ There is nothing to do. In Code Mode the model gets one callable and a generated
399
+ SDK listing. *Every* registered tool is in that listing, including a tool that
400
+ arrives through a manual. These tools therefore arrive as
401
+ `await tools.code_index(...)` and `await tools.code_graph(...)`, with no Code
402
+ Mode work in this package. The `rlm-indexed` profile is `rlm-stable` plus this
403
+ bundle and the bundle of `ph-text-index`:
404
+
405
+ ```bash
406
+ phern --profile rlm-indexed --provider llama --model <model> --mode tui
407
+ ```
408
+
409
+ This package registers a `ph.bundles` entry point. That lets `phern` compose
410
+ the profile without importing this package. It also means that an install
411
+ without this distribution sees no `rlm-indexed`, and does not see one that
412
+ fails at mount.
413
+
414
+ ## It is a name-based graph, and it says so
415
+
416
+ A reference records the *name* that it used, and `callers` and `callees` join on
417
+ that name. Two `register` methods in two classes are one name to this index.
418
+
419
+ Every answer that could be ambiguous carries `definitions`, the number of places
420
+ that define the name. When that number is above one, the rendered text says so
421
+ directly. The model then sees the ambiguity, and is not given one of the
422
+ possibilities without notice. `mode=define` lists the definitions. It does not
423
+ say which definition an edge belongs to. No name-based index can say that.
424
+
425
+ A resolved graph needs an import graph, scope, and type inference for each
426
+ language. In the Rust/Node CodeGraph that is 29,708 lines of TypeScript, and
427
+ this package deliberately did not port it. Name-based matching answers most of
428
+ what an agent asks, and reports where it cannot. That is a better trade than a
429
+ resolved graph for one language.
430
+
431
+ ## Why Python-native, and not a wrapper
432
+
433
+ The obvious plan was to add the Rust/Node CodeGraph as a submodule and wrap its
434
+ kernel with maturin. The investigation rejected that plan, for three reasons in
435
+ increasing order of importance:
436
+
437
+ 1. **`codegraph-kernel` is a `#[napi]` crate**: `crate-type = ["cdylib"]`
438
+ against `napi` and `napi-derive`. maturin builds PyO3 and cffi extensions. A
439
+ wrapper would need a fork of the crate with a rewritten boundary.
440
+ 2. **It is only an extractor.** Its whole export surface is `extract_file`,
441
+ `contract_info`, `grammar_info`, and two `cfnptr` helpers. In its own words:
442
+ "tree-sitter parse+extract with one JS boundary crossing per file". It also
443
+ has a complete TS/WASM fallback, so it is an accelerator and not the core.
444
+ 3. **The intelligence is TypeScript**: `resolution/` has 29,708 lines,
445
+ `extraction/` 25,612, `mcp/` 13,940, `graph/` 6,023, and `db/` 5,475. That is
446
+ 112,612 lines against 25,242 in the kernel. A wrapper around the kernel gives
447
+ a parser and leaves the product behind.
448
+
449
+ Python already has the parser. `tree-sitter-language-pack` is a 3.7 MB wheel. It
450
+ calls 26 languages bundled (python, ts/tsx/js, rust, go, java, csharp, c, cpp,
451
+ ruby, php, swift, kotlin, scala, dart, lua, r, and more) and has 371 available.
452
+ **None of them works offline on a fresh cache.** Measured 2026-11-11 with 1.16.2
453
+ and 1.25.0: with no network and an empty cache, no parser can be obtained,
454
+ including the python parser. With the network, the bundled set arrives as one
455
+ archive from GitHub the first time a parser is requested. After that, the pack
456
+ needs only the cache.
457
+
458
+ ## Why `sqlite3` and not `pyturso`
459
+
460
+ The session log of pH runs on turso, so this package is the exception, and the
461
+ reason is measured. Against a real 40 MB CodeGraph database (11,396 nodes,
462
+ 36,830 edges), `pyturso` served indexed lookups, joins, and aggregates
463
+ correctly. It then failed in three ways:
464
+
465
+ - **FTS5 is absent.** `pyturso` does not see a `CREATE VIRTUAL TABLE … USING
466
+ fts5`, or its shadow tables. `search` needs FTS5.
467
+ - **`Recursive CTEs are not yet supported`.** `impact` needs a recursive CTE.
468
+ - **`json_extract` silently returns NULL** where SQLite returns the value:
469
+ `0.95` against `None`, with the same rows and the same expression. A wrong
470
+ answer with no error is worse than a missing feature.
471
+
472
+ Two of the six modes here are exactly the two that turso cannot serve. Stdlib
473
+ `sqlite3` (SQLite 3.45, with FTS5 compiled in) adds no dependency and serves all
474
+ six. If turso gets both features, `_store.py` changes one import.
475
+
476
+ ## Extraction: two passes, and why
477
+
478
+ - **`process()`** is the intelligence layer of the pack. It gives definitions
479
+ with kinds, the imports of the file, docstrings, and comments. It does not
480
+ give *references*: `SymbolInfo` says that a function exists, never who calls
481
+ it.
482
+ - **The tags query** is the `tags.scm` of tree-sitter for each language. GitHub
483
+ builds its code navigation on the same data. It gives `@definition.*` and,
484
+ most important, `@reference.call`.
485
+
486
+ Measured over `packages/ph-core/src/ph`: 136 files and 1.58 MiB take 219 ms
487
+ through `process()` and 108 ms through the tags query. An index of the core of
488
+ this repository takes 2.0 s end to end, including the SQLite writes, and 0.1 s
489
+ when nothing has changed. The second parse gives more value than it costs.
490
+
491
+ Four details are easy to get wrong. Each is handled one time in `_extract`, and
492
+ each has a test:
493
+
494
+ - **`ProcessConfig` spans are 0-based.** Tree-sitter points are also 0-based,
495
+ and every line that pH shows a model is 1-based. The numbers are normalized on
496
+ the way out, so nothing downstream has to remember which pass a number came
497
+ from. An error here gives a pointer that is off by one, which makes a model
498
+ stop trusting a tool.
499
+ - **`SymbolInfo.doc` is always `None`.** The pack does not set it (measured
500
+ against a file full of docstrings). Code that trusted it would have shipped an
501
+ index with an empty `doc` column and a `search` mode that matched names only.
502
+ The prose is on two other channels. `result.docstrings` carries the pack's own
503
+ `associated_item`. `result.comments` carries doc comments (Rust `///`,
504
+ TypeScript block comments, Go `//`). Those comments have no association, so
505
+ the rule is positional and narrow: the last line of the comment must be
506
+ *immediately* above the first line of the definition.
507
+ - **The `span.end_line` of a comment node can be one past the line that the
508
+ comment occupies**, because the `///` node of Rust includes its trailing
509
+ newline. With `end_line`, the rule missed the definition directly below *and*
510
+ matched one a blank line away. The line count now comes from the text of the
511
+ comment.
512
+ - **The tags query of TypeScript is only part of the query.** Its grammar
513
+ extends the javascript grammar, and upstream splits the queries in the same
514
+ way. `class`, `function`, and `call` are in the *javascript* query. A `.ts`
515
+ file that is matched against the typescript query alone yields two tags and no
516
+ calls. `INHERITS` fixes that.
517
+ - **The python query of the pack missed module-level constants.** It looks for
518
+ an assignment inside an `expression_statement`, and the grammar of the pack
519
+ puts an assignment directly under `module`. `SUPPLEMENTS` adds a pattern that
520
+ fits the tree.
521
+
522
+ ## Limits for each language
523
+
524
+ Tests cover both of these, so they are known shapes and not surprises:
525
+
526
+ - **C has definitions but no call references.** Its tags query ships no
527
+ `@reference.*` captures. `callers` and `callees` are therefore empty for C,
528
+ while `search`, `define`, and `entities` work.
529
+ - **A bare Ruby send is not seen.** `helper` without parentheses parses as an
530
+ identifier and not as a `call`, so only `helper()` becomes an edge.
531
+
532
+ Only `reference.call` is stored. The tags queries also emit `reference.type`,
533
+ `reference.class`, and more. Each is a real relationship, but with them
534
+ `callers` would return every place that only *mentions* a type.
535
+
536
+ ## Where things live
537
+
538
+ - **The index, in a session**: `graph.db` in the scratch directory of the agent
539
+ that owns the tree, `$PH_HOME/scratch/<session>/<agent>/code-graph/`. The
540
+ owner is the calling agent, or the ancestor whose tree it shares. A child in
541
+ its own worktree indexes its own tree, so two agents on different trees never
542
+ answer from the line numbers of the other. A resumed session has the same
543
+ session id and agent id, and therefore the same index. `code_index` then
544
+ parses again only what changed. The cost is that a new session builds its own
545
+ index and does not find the index of the last session.
546
+ - **The index, from a shell**: `--state DIR` puts it in `DIR`, and so does
547
+ `$PH_TOOL_STATE`. `ph-utcp` sets that variable for each call, and another
548
+ harness can set it. With neither, the index is
549
+ `$PH_CACHE/code-graph/<digest of the workspace root>/graph.db`. It is under
550
+ the cache root because it can be rebuilt from the source. It is keyed by the
551
+ root so that two checkouts do not answer the questions of each other. That
552
+ directory records its workspace. When the workspace has been gone for a week,
553
+ the next index removes the directory.
554
+ - **The grammars**: `$PH_CACHE/tree-sitter`. The engine row sets this when the
555
+ command mounts it.
556
+ - **The writer lock**: `graph.db.lock` beside the database. It allows one
557
+ indexing run at a time for each index, across every process on the workspace.
558
+ A second run waits `wait_s` for the first. It then finds most of the tree
559
+ current, or answers `state: indexing`.
560
+ - **The log of the command**: `$PH_CACHE/logs/ph-code-graph.log`. A call prints
561
+ one JSON object, and prints nothing on stderr except the progress lines that
562
+ `--progress` asks for. A traceback therefore goes to the log.
563
+
564
+ The location of the grammars is important. The pack keeps every grammar that it
565
+ uses in a writable cache. That includes its bundled grammars, which it downloads
566
+ there on first use. The pack **fails hard** when it cannot create that
567
+ directory. Its own default is `$XDG_CACHE_HOME/…`, which is correct on a laptop
568
+ and wrong in a container with a read-only `HOME` (both measured). The row
569
+ therefore gives the pack the root that pH already uses for rebuildable
570
+ artifacts. When even that root is not writable, the row refuses to mount with
571
+ one sentence, not with a `pathlib` traceback. From the command, that sentence is
572
+ the answer of the call, with the code `MOUNT_REFUSED`.
573
+
574
+ `TREE_SITTER_LANGUAGE_PACK_CACHE_DIR` still wins if an operator set it. Their
575
+ value is what keeps the variable meaningful.
576
+
577
+ Expect two consequences:
578
+
579
+ - The **first** index on a cold grammar cache pays for the grammar fetch.
580
+ Measured on `packages/ph-core/src/ph`: 9 s cold against 2 s warm, and 0.1 s
581
+ when nothing changed.
582
+ - A command that cannot reach GitHub indexes nothing until the cache is filled.
583
+ That applies to the command of a confined agent, and to a machine with no
584
+ network. Each file is skipped, with a reason that names `/code-graph install`.
585
+
586
+ Provision the grammars deliberately, not during the turn of a user:
587
+
588
+ ```
589
+ /code-graph status # how many grammars are ready, and where they are
590
+ /code-graph install # load each one now, and name any that will not load
591
+ ```
592
+
593
+ This is a **command**, not a tool. A person asks the harness to provision, and
594
+ that costs no model turn. In a session, it runs `ph-code-graph status` or
595
+ `ph-code-graph install` and shows the output. From a shell, type those commands
596
+ directly. `phern doctor` reports the path of the index, whether the index is
597
+ built, and whether the manual was discovered. It does not start the command.
598
+ `install` fetches all eighteen grammars in about three seconds. It runs outside
599
+ the sandbox, because it needs the network.
600
+
601
+ ## The skill
602
+
603
+ The package installs a `code-graph` skill. An RLM under progressive disclosure
604
+ sees one line in its catalog and can read the page when it needs it. The page
605
+ says: index first, ask the question that you have, then `read` the narrow
606
+ result. It also says that a name-based graph reports ambiguity and does not
607
+ resolve it.
608
+
609
+ The row registers the skill. A directory scan does not find it. The skill
610
+ therefore arrives exactly when the tools do, and is removed with them.
611
+ `skills-progressive` ships an empty `paths` on purpose, because a distribution
612
+ installs a skill deliberately (I7).
613
+
614
+ ## Tests
615
+
616
+ `tests/test_code_graph.py` runs the real parser and the real SQLite index. It
617
+ deliberately tests three things:
618
+
619
+ - **Line spans against the file.** Each span is checked by slicing the real
620
+ source at the span, not against an expected number. A pointer that is off by
621
+ one still looks plausible in a diff.
622
+ - **The enclosing-symbol rule.** A call inside a nested closure belongs to the
623
+ closure, not to the 40-line function around it.
624
+ - **Incrementality.** A second index pass over an unchanged tree parses nothing,
625
+ and a file that was touched but not modified stays unchanged.
626
+
627
+ `tests/test_code_graph_cli.py` starts real processes, because its subject exists
628
+ only between processes:
629
+
630
+ - The command answers with the values that the row answers with.
631
+ - Two processes that index one tree report counts that add up.
632
+ - A real UTCP client registers the manual and calls a tool, with a list argument
633
+ and with optional arguments left out.
634
+
635
+ `tests/test_code_graph_row.py` covers the row that a session mounts:
636
+
637
+ - The tools keep their names, values, rendered text, and error codes.
638
+ - An agent in its own tree indexes that tree.
639
+ - A confined agent that may not write its tree still indexes it.
640
+ - A fresh interpreter that mounts only the rows of the session never loads
641
+ tree-sitter.