contextcompass 2.15.0__py3-none-any.whl

contextcompass_cli/_package/agent_onboarding/default/engineer/skills/src_graph_generation.md

@@ -0,0 +1,341 @@
+
+# src_graph_generation
+
+Purpose
+- Define how to BUILD and MAINTAIN the assembled source graph.
+
+## The pipeline
+
+Two stages, two scripts, both stdlib-only:
+
+```
+tools/system_documents/python/extract_graph.py     source tree -> per-file descriptors
+tools/system_documents/python/assemble_graph.py    descriptors -> src_graph.md + index
+```
+
+```bash
+python context_compass/tools/system_documents/python/extract_graph.py \
+    --src src --out context_compass/system_docs/graph
+
+python context_compass/tools/system_documents/python/assemble_graph.py \
+    --descriptors context_compass/system_docs/graph \
+    --out context_compass/system_docs
+```
+
+`--check` on either script reports without writing. Run it before assuming the
+graph is current.
+
+Three more scripts exist for cases the pipeline alone cannot serve:
+
+```
+tools/system_documents/python/graph_walker.py             state of the authored tier
+tools/system_documents/python/graph_semantics_tickets.py  turn unauthored areas into work
+tools/system_documents/python/migrate_authored_graph.py   carry a retired-format graph in
+```
+
+## Keeping the authored tier honest
+
+The mechanical tier self-heals - re-run the extractor and classes, bases and line
+numbers are right again. **The authored tier does not.** It is written once, and
+without help nothing says which parts are still true. Left alone that produces a
+graph full of confident descriptions of code that no longer works that way.
+
+Four states, and the walk through them is the maintenance loop:
+
+| state | meaning | what to do |
+| --- | --- | --- |
+| `UNSEMANTIC` | no authored fields | author it, by reading the code |
+| `AUTHORED` | authored, source unchanged since | nothing |
+| `SEMANTICS_STALE` | authored, **its source moved underneath it** | re-read, then accept |
+| `RETIRED` | gone from source, prose retained | adjudicate: move it or drop it |
+
+```bash
+python context_compass/tools/system_documents/python/graph_walker.py \
+    --descriptors context_compass/system_docs/graph --src src --report
+```
+
+Read-only. It aggregates by package rather than by node, because per-node output
+on a real codebase is thousands of lines nobody reads, and a subsystem is the
+unit an agent actually works in.
+
+Staleness is tracked **per node**, not per file: a file-level hash would mark all
+forty classes in a module stale because one changed, and a census that cries wolf
+gets ignored.
+
+Once you have RE-READ a stale node and confirmed its semantics still hold:
+
+```bash
+python context_compass/tools/system_documents/python/graph_walker.py \
+    --descriptors <dir> --accept app.mod.Alpha --apply
+```
+
+That re-stamps; it changes no prose. **Accepting without reading is how a graph
+becomes confidently wrong** - the stamp is meant to record that someone checked,
+and it cannot tell the difference.
+
+### Nothing deletes authored prose automatically
+
+A node that disappears from source keeps its authored fields under
+`nodes_retired` in the same descriptor, and is reported. Earlier this was a real
+defect: the note said `ORPHANED - resolve by hand` while the same pass wrote the
+prose out of the file, so the only copy was in git and only if you noticed.
+
+The walker also reports **stranded descriptors** - ones whose source file is gone
+entirely. The extractor walks source, so it never opens those and their nodes
+count as live forever; only a pass starting from the descriptors can see it.
+Pass `--src` to enable that check.
+
+Possible cross-file moves are **suggested, never applied**. A same-label match is
+evidence, not proof, and attaching someone's authored prose to the wrong class is
+worse than leaving it retired where they can see it.
+
+### Deleting for real: `--reconcile`
+
+When a retirement is genuinely a deletion - the thing is gone and its meaning is
+not coming back - clear it:
+
+```bash
+python context_compass/tools/system_documents/python/graph_walker.py \
+    --descriptors <dir> --reconcile
+```
+
+**This is the only sanctioned path for structural deletion, and the only verb in
+the subsystem that destroys authored work.** It lists every node and its prose,
+then asks. Without a terminal it refuses rather than assuming - a prompt that
+silently self-answers in CI is not a prompt. `--yes` answers it deliberately.
+
+Before running it, check `--report` for possible moves. If a retirement is really
+a rename, copy the authored fields onto the new node first; afterwards the only
+copy is in version control.
+
+## Turning the backlog into work - ON DEMAND
+
+```bash
+python context_compass/tools/system_documents/python/graph_semantics_tickets.py \
+    --descriptors <dir> --tickets context_compass/tickets            # dry run
+    ... --create --yes                                               # write them
+```
+
+**Off by default and prompted.** Nothing generates tickets as part of extraction
+or assembly. A tool that writes into someone's board without being asked is doing
+something hostile, so this is a separate command, a dry run unless you say
+`--create`, and a question unless you say `--yes`.
+
+**Aggregated at the package**, and that is a hard constraint rather than a
+preference. Measured against a real 575-file, 1,188-node graph with 657 nodes
+unauthored:
+
+| granularity | stories | outcome |
+| --- | --- | --- |
+| per node | 657 | destroys the board |
+| per file | 575 | destroys the board |
+| per directory (default) | 146 | still too many |
+| `--depth 4` | **36** | usable |
+| `--min-nodes 5` | **46** | usable |
+
+The attention board is a routing surface carrying about a dozen active rows. A
+package is also the unit an agent actually works in - a subsystem at a time - so
+it is the natural granularity as well as the survivable one.
+
+**The default is the containing directory, and on a deep tree that is still too
+many.** `--depth N` groups at a shallower level; `--min-nodes N` skips the
+scraps. Once the count passes 50 the tool prints both levers with the counts
+they would produce, rather than emitting 146 stories and letting you discover it
+afterwards. Neither loses work - the nodes stay in the census either way, only
+the packaging changes.
+
+A design brief for this feature estimated ~33 stories from
+`_package_candidates.json`. That file counts directories whose files share a
+naming *suffix* - a different concept from the directory a file lives in - so the
+real directory-level figure was 146. Worth knowing before you trust an estimate
+from either source.
+
+You get **one epic and one story per package with work**. Tasks are deliberately
+not generated: task granularity is the working agent's judgement, and
+pre-generating it presumes an approach to work nobody has started.
+
+Re-running **updates** rather than duplicating - ticket ids derive from the
+package path, and completed lanes are searched too. A package that gets fully
+authored is reported `SATISFIED` so you can close its story. The loop closes in
+both directions or it is just a different kind of noise.
+
+Generated tickets are **drafts** and touch no board. Route them yourself: a
+generated ticket claiming an active row is a row nobody agreed to take.
+
+### What a generated story must not become
+
+Every generated story states that semantics are authored **by reading the code**.
+Take that literally. `owns_lifecycle_of`, `uses` and `borrows` are the same
+syntax - `self._x = x` in all three - and the difference is design intent that
+appears nowhere in the source text. A cleanup-contract heuristic was tested
+against a labelled corpus and discriminated at 21% versus 21%: no signal at all.
+
+Filling a generated ticket with plausible prose inferred from class names is
+worse than leaving those nodes unsemantic, because `UNSEMANTIC` is honest and
+invented semantics read as verified.
+
+The layout is `tools/system_documents/<language>/`. The descriptor contract is
+language-neutral; only the extractor is Python-specific. A sibling directory can
+implement the same contract for another language.
+
+## Why per-file descriptors
+
+The unit is the source file, because that is the unit that changes.
+
+Earlier drafts sharded a single generated blob by namespace and bin-packed it to
+a line budget. That produced unstable boundaries - adding one node reflowed the
+packer and silently moved unrelated nodes between shards - and meaningless
+names. Anchoring to the source path fixes both: a file is a file, its descriptor
+mirrors its path, and touching one file dirties exactly one descriptor.
+
+It also makes the size problem disappear. Source files are already
+human-sized, so descriptors are too: median 43 lines, p90 83, none over 400,
+with no budget parameter anywhere.
+
+## The three tiers, and who owns them
+
+This is the contract that makes regeneration safe.
+
+| tier | owner | fields |
+|---|---|---|
+| mechanical | the script | `id`, `label`, `kind`, `file`, `lineno`, `bases`, `markers`, `public_methods`, `shape`, `source_sha256`, `specializes`/`implements` edges |
+| curation | authored | `include` |
+| semantics | authored | `role`, `responsibilities`, `owns_state`, `phases`, `edges_authored` |
+
+**On re-run the script refreshes tier 1 and never touches tiers 2 and 3.** That
+is verified behaviour, not an intention - inject an authored field, re-run, and
+it survives.
+
+## The authored schema
+
+These are the fields you may populate. Nothing generates them and nothing
+validates them, so this list is the contract.
+
+### Node fields
+
+| field | shape | meaning |
+|---|---|---|
+| `include` | bool | belongs in the composed graph. Absent means "not triaged", which is not the same as `false`. |
+| `role` | prose, one line | what this object is for. |
+| `responsibilities` | list of short phrases | what it is on the hook for. |
+| `owns_state` | list of attribute names | fields whose lifecycle it controls. |
+| `phases` | list | when in its life it matters: `init`, `validation`, `runtime`, `refresh`, `cleanup`. Open set - add one if your system needs it, but reuse these first. |
+
+### Edge fields, under `edges_authored`
+
+| field | shape | rendered |
+|---|---|---|
+| `from` | node id | yes, table |
+| `to` | node id | yes, table |
+| `relation` | `owns_lifecycle_of`, `uses`, `borrows`, `holds`, `owns`, `used_by` | yes, table |
+| `cardinality` | `one_to_one`, `one_to_many`, `many_to_one` | yes, table |
+| `phase` | list, same vocabulary as node `phases` | yes, table |
+| `why` | prose, one or two sentences | yes, beneath the table |
+| `strength` | `hard`, `borrowed`, `soft` | **no - see below** |
+
+**`strength` is stored and deliberately not rendered.** Measured against a
+hand-authored graph of 997 edges it is ~97% recoverable from `relation`:
+`hard` is the ownership relations, `borrowed` is the reference relations. A
+column that restates the column beside it costs space and adds nothing. Keep
+populating it if your tooling consumes it; the document will not show it.
+
+**`why` is rendered because it is the justification for a claim the extractor
+says it cannot make.** An authored `owns_lifecycle_of` asserts ownership where
+the syntax tree shows only a reference. Recording why, and showing it, is the
+difference between evidence and assertion. It sits beneath the table so the
+table stays scannable.
+
+The prose does not cost you a whole-document read, because there is no such
+thing here: you read the index and slice one section. A few extra lines inside
+the one section you asked for is not a cost worth optimising against.
+
+## What the script cannot do, and why
+
+Do not expect the extractor to produce a finished graph. Measured against a
+hand-authored graph of 535 nodes and 997 edges:
+
+- node identity: **94%** recovered (matching on file + label)
+- `specializes` edges: **94%** recovered with fully-qualified ids on both ends
+- edge target resolution via imports: **99%**
+- **68% of all edges are not derivable at all**
+
+`owns_lifecycle_of` (205 edges), `uses` (323), and `borrows` (119) are
+syntactically identical. Every one is "A holds a reference to B". Whether A owns
+B's lifecycle, merely uses it, or borrows it without owning is a design fact
+that does not appear anywhere in the source text. The same is true of every
+`role`, `why`, `cardinality`, and `strength`.
+
+Curation is equally underivable. The best syntactic significance rule tested
+would have suppressed 419 nodes while wrongly dropping 63 that the reference
+graph deliberately includes. So the script emits inventory and records `shape`
+signals; the `include` field is authored. Emitting a node the author can reject
+is recoverable. Silently withholding one they wanted is not.
+
+## Maintenance loop
+
+When you change a source file:
+
+1. Re-run `extract_graph.py`. Only that file's descriptor changes.
+2. Read the drift report. It names nodes that are `NEW`, `ORPHANED` (gone from
+   source but still carrying authored semantics), or `UNSEMANTIC`.
+3. Author the semantics for anything new. Resolve orphans by hand - the script
+   will not delete authored prose on your behalf.
+4. Re-run `assemble_graph.py` to rebuild the document and index together.
+
+Never regenerate one without the other. They are only guaranteed consistent
+because a single pass emits both.
+
+## The index is a byproduct
+
+`assemble_graph.py` knows each section's line range because it emitted those
+lines. It does not re-parse the document to find headings.
+
+That removes a whole class of drift. An index generated by re-walking a
+document can disagree with the document, and the only defence is a hash. Here
+they cannot disagree. The hash is still recorded, but it guards a different
+failure: someone hand-editing the assembled file.
+
+Every range is verified against its own header on every run. An off-by-one here
+silently corrupts every downstream read, so it is checked rather than trusted.
+
+## What this replaced
+
+The two generated JSON graph files - a readable graph of 776 KB and a raw graph
+of 768 KB - are retired. Together they were 1,544 KB that had to be read whole
+or not at all, because a JSON object has no addressable interior.
+
+The replacement is 972 KB of Markdown plus a 62 KB index, and a typical query
+reads a 20-40 line slice.
+
+**If you arrive holding a graph in the retired format, do not run the two-command
+sequence and hope.** Only `specializes` and `implements` are mechanical, so a
+naive re-extraction marks every node `UNSEMANTIC` and drops every authored edge.
+Measured on a mature graph that was 84% of its edges. The rule generalises: the
+better your existing graph, the more the naive path destroys, because a mature
+graph is edge-richer than a young one. Carry the authored tier across by node id
+first, then by `(file, label)`, before you assemble anything.
+
+## Anti-patterns
+
+- Hand-editing `src_graph.md` or `src_graph_index.md`. Edit descriptors, reassemble.
+- Regenerating the document without the index, or the reverse.
+- Deleting a descriptor whose node still carries authored semantics.
+- Filtering descriptor files by filename prefix. An earlier version of the
+  assembler skipped anything starting with `_` to avoid manifests, and silently
+  swallowed 14 real files including `__init__.py` and every dunder module. A
+  prefix is a naming convention, not a type.
+- Promoting an edge candidate to an edge without reading the code.
+- **Writing `context_compass/...` into a descriptor's authored prose.** The
+  generated document is clean today - it cites the source tree constantly and
+  names this package nowhere - and that is only true because nothing has typed a
+  tool path into a `role`, `responsibilities`, or `why` field. Those fields flow
+  straight through the assembler into `src_graph.md`, which ships with the
+  codebase to readers who do not have Context Compass. Cite the source; if a
+  reader needs the regeneration command, they are in this skill already.
+- Absolute paths in a descriptor. Repo-relative, or the document is correct on
+  one machine.
+
+References
+- `agent_onboarding/default/engineer/skills/src_graph_usage.md`
+- `agent_onboarding/default/engineer/skills/staleness_protocol.md`
+- `agent_onboarding/default/engineer/skills/context_protocol.md`

contextcompass_cli/_package/agent_onboarding/default/engineer/skills/src_graph_usage.md

@@ -0,0 +1,161 @@
+
+# src_graph_usage
+
+Purpose
+- Define how to READ the assembled source graph without loading all of it.
+
+## What the graph is
+
+Two files under `system_docs/`:
+
+- `src_graph.md` - one section per source file, each delimited by an HTML
+  comment naming that file.
+- `src_graph_index.md` - the exact line range of every section, plus a
+  staleness proof.
+
+Both are generated. Never hand-edit either one; see
+`agent_onboarding/default/engineer/skills/src_graph_generation.md`.
+
+## The read order (non-negotiable)
+
+1. Read `src_graph_index.md`. It is small - about 3% of the document's lines,
+   6% of its bytes.
+2. **Verify it is current** (next section).
+3. Find the row for the source file you care about.
+4. Read only that line range out of `src_graph.md`.
+
+Do not read `src_graph.md` in full. On the reference codebase it is 20,261
+lines against a 602-line index; a single file's section is typically 20-40
+lines. Reading the whole document to answer a question about one file costs
+roughly 500x what the answer needs.
+
+Do not scan the document for `BEGIN FILE` markers instead of using the index.
+The index is authoritative, it was emitted by the same pass that wrote the
+document, and scanning defeats the entire point of having ranges.
+
+## Verify before you slice
+
+The index records what the document was when it was generated:
+
+| field | how to verify |
+|---|---|
+| `line_count` | recount the document's lines |
+| `content_sha256` | rehash the document's exact bytes |
+| `line_ending` | split on THAT terminator, always `lf` here |
+
+On any mismatch the document was hand-edited or the pipeline was interrupted.
+**STOP. Do not slice.** Regenerate, or read the file directly and say which you
+did. A range from an unverified index is a guess wearing a line number.
+
+```python
+import hashlib, pathlib, re
+
+doc = pathlib.Path("context_compass/system_docs/src_graph.md")
+idx = pathlib.Path("context_compass/system_docs/src_graph_index.md").read_text(encoding="utf-8")
+
+raw = doc.read_bytes()
+claimed_lines = int(re.search(r"line_count \| (\d+)", idx).group(1))
+claimed_hash = re.search(r"content_sha256 \| `([0-9a-f]{64})`", idx).group(1)
+
+lines = raw.decode("utf-8").split("\n")
+if lines and lines[-1] == "":
+    lines.pop()
+
+if len(lines) != claimed_lines or hashlib.sha256(raw).hexdigest() != claimed_hash:
+    raise SystemExit("INDEX STALE - refusing to slice; regenerate the graph")
+```
+
+## Selecting what to read
+
+Each index row is `| lines | source | nodes | edges |`. The `edges` count covers
+both derived and authored edges, matching what that section's `Edges out` table
+renders. Edge candidates are not counted - they are guesses, not edges.
+
+- **Working one file**: match `source` against its path, read that range.
+- **Working a subsystem**: match `source` against the directory prefix and read
+  each matching range. Rows are ordered by source path, so a subsystem's files
+  are contiguous.
+- **Following a relationship**: a section's `Edges out` table gives fully
+  qualified target ids. Map the id back to a path and look that row up.
+- **Finding what points AT something**: the per-file sections carry outbound
+  edges only. Inbound edges are not answerable from one section - you must
+  either scan the index-selected sections you care about, or accept that
+  reverse lookup is the one query this layout does not make cheap.
+
+## What each section contains
+
+- `source_sha256` of the source file at generation time. If you are reasoning
+  about current code, check it against the file on disk.
+- **Nodes**: id, kind (`module`, `class`, `interface`, `abstract`, `enum`,
+  `record`), definition line, bases, markers, public methods, and the authored
+  fields (`role`, `responsibilities`, `owns_state`, `phases`) when present.
+- **Edges out**: one table, six columns - `from`, `relation`, `to`,
+  `cardinality`, `phase`, `origin`. Derived edges (`specializes`, `implements`)
+  carry `-` in the two authored columns, because those are design facts the
+  extractor cannot produce. A row of dashes is not missing data; it is the graph
+  telling you nobody has authored that relationship's semantics yet.
+- **Why lines**: beneath the table, one per authored edge that carries a `why`.
+  This is the justification for a claim the syntax tree cannot support - an
+  authored `owns_lifecycle_of` asserts ownership where the AST shows only a
+  reference. Read it before relying on the edge.
+- **Edge candidates**: AST instantiation guesses. **Unconfirmed.** These
+  over-generate roughly 8x against a hand-authored graph. Treat them as leads,
+  never as evidence.
+- **Published aliases**: module-level `Alias = Class` bindings, which are the
+  names consumers actually import.
+
+## Trust boundaries
+
+Read every section with this in mind:
+
+- Node identity, `bases`, and `specializes` are **mechanical** - derived from
+  the syntax tree, measured at 94-98% agreement with a hand-authored graph.
+- `role`, `responsibilities`, `owns_state`, `phases`, and any edge marked
+  `authored` are **authored** by an agent or human who read the code.
+- A node marked **UNSEMANTIC** has scaffold only. Its structure is trustworthy;
+  its meaning has not been established. Do not infer purpose from a name.
+- Edge candidates are **guesses**.
+
+**Authored does not mean current.** The mechanical tier is rebuilt on every
+extraction and is as fresh as the last run. Authored prose is written once and
+stays exactly as written while the code underneath it moves. A node can carry a
+confident, well-written `role` describing behaviour the class no longer has.
+
+Two states exist to tell you which is which, and neither appears in the document
+itself - ask the walker:
+
+```bash
+python context_compass/tools/system_documents/python/graph_walker.py \
+    --descriptors <dir> --src src --report
+```
+
+| state | what it means for a reader |
+| --- | --- |
+| `AUTHORED` | the source has not changed since the prose was written |
+| `SEMANTICS_STALE` | **the source moved underneath it**; treat the prose as a lead, not a fact |
+| `RETIRED` | the node is gone from source; its prose is kept for adjudication |
+
+If a claim matters and the node is stale, **read the code**. Full loop:
+`agent_onboarding/default/engineer/skills/src_graph_generation.md`.
+
+Cite what you read with `path:start_line-end_line` against `src_graph.md`, the
+same evidence convention used everywhere else in this repository.
+
+## Anti-patterns
+
+- Reading `src_graph.md` in full when the index would answer the question.
+- Slicing without verifying `line_count` and `content_sha256` first.
+- Treating an edge candidate as a real relationship.
+- Treating an UNSEMANTIC node's name as a description of its purpose.
+- Quoting a `source_sha256` as proof the code is current without comparing it
+  to the file on disk.
+- Relying on authored prose for a decision without checking whether the node is
+  `SEMANTICS_STALE`. Well-written prose about code that has since changed reads
+  exactly like well-written prose about code that has not.
+- Filling in a node's semantics from its name because a generated ticket asked
+  you to. `UNSEMANTIC` is honest; invented semantics read as verified.
+
+References
+- `agent_onboarding/default/engineer/skills/src_graph_generation.md`
+- `agent_onboarding/default/engineer/skills/context_protocol.md`
+- `agent_onboarding/default/general/skills/configuration_standards.md`

contextcompass_cli/_package/agent_onboarding/default/engineer/skills/staleness_protocol.md

@@ -0,0 +1,49 @@
+
+
+# staleness_protocol
+
+Purpose
+- Define required actions for stale documentation or ticket context.
+
+Required flow
+- Resolve stale or missing documentation/tickets before feature work.
+
+States and required actions
+- missing: create the missing doc or ticket immediately.
+- stale: refresh the doc to match current code or intent.
+- needs_review: confirm architecture and update notes if needed.
+- fresh: no action.
+- blocked: record a blocker in the relevant ticket.
+
+Noise control
+- Update docs only when state or semantic content changes.
+- Avoid churn or rewording without new information.
+
+Enforcement rule
+- Do not handwave around stale docs. When boundaries, invariants, or documented
+  source wiring change, bring the canonical `system_docs/` files current.
+- **AUTHORED documents you edit directly:** `src_architecture.md`,
+  `src_components.md`, `tests_architecture.md`, `tests_components.md`. Regenerate
+  each one's `*_index.md` in the same pass.
+- **GENERATED documents you must NOT edit:** `src_graph.md` and
+  `src_graph_index.md`. Edit the per-file descriptors and reassemble; a hand-edit
+  is overwritten by the next run and, in the meantime, breaks the index hash so
+  every slice is refused. See
+  `agent_onboarding/default/engineer/skills/src_graph_generation.md`.
+- The distinction is not cosmetic. "Update `src_graph.md`" is an instruction that
+  cannot be carried out correctly, and an agent that follows it literally
+  corrupts the staleness proof the whole slicing protocol depends on.
+
+Example transitions
+- missing -> fresh after doc creation.
+- stale -> fresh after refresh.
+- fresh -> needs_review after a significant refactor.
+
+References
+- `system_docs/src_architecture.md`
+- `system_docs/src_components.md`
+- `system_docs/src_graph.md`
+- `system_docs/tests_architecture.md`
+- `system_docs/tests_components.md`
+
+