clew-trace 1.0.0__tar.gz

clew_trace-1.0.0/.doxygen-guard.yaml

@@ -0,0 +1,62 @@
+---
+# doxygen-guard configuration for clew (dogfooding — we enforce on ourselves).
+#
+# Every function/class in the shipped Python package carries:
+#   ## @brief    one-line summary
+#   ## @version  bumped on body change
+#   ## @return   on non-void functions
+#
+# @req COVERAGE IS NOT GATED. It was, by scripts/req_coverage_audit.py, which is
+# DELETED. Nothing now checks that a requirement claiming `implemented` resolves to
+# an implementing function, or that one claiming `planned` resolves to none.
+# Last measured by hand: 30 of 34 resolve; the 4 that do not are untagged
+# deliberately (two satisfied by configuration and human review, two unbuilt).
+#
+# Also not gated: whether a TEST covers a requirement. req_test_edges is sparse.
+
+output_dir: docs/generated/
+
+# Declare where our catalog lives, so the INDEX finds it without a CLI flag.
+# Not cosmetic dogfooding: `bd2d34e` taught clew to honour this field
+# precisely because a repo that declares its catalog had the declaration read by
+# the gate and ignored by the index — and we then shipped that fix without
+# applying it to ourselves. A bare `--scope from-guard` build of this repo
+# ingested ZERO catalog rows, so `req_trace` returned implementers with no title,
+# status or acceptance, and the "30 of 34" figure was reproducible only with an
+# undocumented `--requirements` argument. Through the MCP server, which passes no
+# flags, it could not be reproduced at all.
+impact:
+  requirements:
+    file: requirements.yaml
+
+validate:
+  exclude:
+    - "^\\.venv/"
+    - "^\\.claude/"
+    - "^workspace/"
+    - "^tests/"           # tests are exempt (fixtures, asserts)
+    - "^scripts/"         # dev helpers
+    - "^acceptance/"      # benchmark harness + evidence, not shipped code
+    - "^docs/generated/"
+  # NOTE: this key was DUPLICATED — an empty `tags:` sat directly above the real
+  # one. YAML takes the last, so the real block won and nothing broke, which is
+  # why it survived. The trap is that a reorder or an insertion between them
+  # would silently hand the win to the empty mapping, dropping the declared @req
+  # id pattern and disabling tag validation with no error anywhere.
+  tags:
+    # clew's own requirement id shape. Declared so @req tags validate
+    # against it — NOT hardcoded anywhere in the pipeline.
+    #
+    # A BARE ID, nothing else. This pattern used to carry an optional
+    # `( \[inferred\])?` suffix, because the project had a convention where a tag
+    # could mark its own mapping as inferred rather than asserted, and
+    # doxygen-guard validates the WHOLE remainder of the `@req` line rather than
+    # just the first token.
+    #
+    # The convention was RETIRED 2026-07-30 by the owner: "I would rather not put
+    # requirements as inferred or not — they should simply be the stated ID,
+    # inferred is noise." The marker is gone from the tags, the parser and the
+    # schema — `req_edges.confidence` no longer exists, because a column that can
+    # hold only one value states nothing.
+    req:
+      pattern: "^REQ-DDB-[A-Z]+-[0-9]{3}$"

clew_trace-1.0.0/.gitignore

@@ -0,0 +1,69 @@
+.venv/
+# BOTH spellings, because `.venv/` with the trailing slash matches a DIRECTORY only and so
+# does not match a SYMLINK named `.venv`. `git add -A` would then stage a tracked symlink
+# pointing at an absolute machine path. Same hazard as the two `doxygen_out` spellings below:
+# one spelling of an ignore silently covers less than a reader assumes.
+.venv
+__pycache__/
+*.pyc
+*.egg-info/
+.ipynb_checkpoints/
+workspace/
+
+# Tool caches. `.ruff_cache/` is the one that matters: ruff writes its own
+# `.ruff_cache/.gitignore` containing `*`, so the directory self-hides and looks handled —
+# but that is ruff's file, not this repo's guarantee. Delete the cache and the protection
+# goes with it.
+.cache/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.tox/
+htmlcov/
+.coverage
+
+# Build outputs. `python -m build` runs in CI, so anyone reproducing it locally creates these.
+dist/
+build/
+
+# Working-instructions and scratch. `.claude/CLAUDE.md` lives here deliberately: how someone
+# instructs their own agent is theirs to maintain, not a shipped asset.
+.claude/
+
+# The doxygen-guard output directory the guard's own config declares
+# (`output_dir: docs/generated/`), so any guard run recreates it.
+docs/generated/
+
+# Built indexes and their caches. DEFENSE IN DEPTH, not convenience: clew is pointed at
+# third-party repos, so a stray database in the tree is that repo's source graph — symbol
+# names, file paths, call structure — in a single file. It is the highest-value thing here to
+# leak by accident, and a multi-megabyte binary is exactly what a hurried `git add -A`
+# swallows without anyone reading the diff. `--output` accepts any path, so these patterns
+# exist to keep that true when someone builds an index somewhere new.
+*.db
+*.db-journal
+*.db-wal
+*.idxcache
+
+# Doxygen scratch from a HAND-RUN doxygen. The pipeline forces OUTPUT_DIRECTORY away from the
+# target repo, so this only catches manual invocations.
+#
+# TWO SPELLINGS, not a glob. The dot-prefixed form is the one the pipeline produces and
+# `doxygen_out/` never matched it — a leading dot makes it a non-match, which once staged 47
+# files of doxygen XML. `*doxygen_out/` would also swallow a deliberately-named directory, and
+# an over-broad ignore that hides a real file is a worse hazard than the mess it prevents.
+#
+# Deliberately NOT ignoring bare `html/` or `xml/` for the same reason: those are ordinary
+# directory names.
+doxygen_out/
+.doxygen_out/
+
+# Private-target acceptance evidence. A cell transcript quotes the target's source verbatim,
+# so committing one publishes that source. THIS IS THE ONLY PROTECTION — there is no scrub
+# gate, so a `git add -f` here is unguarded. Both spellings, per the rule at the top.
+acceptance/targets/internal
+acceptance/targets/internal/
+
+# In-flux findings. A committed workbook becomes a citable claim before it has been measured;
+# measured results go in `acceptance/targets/*/` beside the transcripts that produced them.
+acceptance/docs/

clew_trace-1.0.0/.pre-commit-config.yaml

@@ -0,0 +1,58 @@
+---
+exclude: |
+  (?x)^(
+    \.venv/.*|
+    \.claude/.*|
+    workspace/.*|
+    docs/generated/.*|
+    acceptance/targets/.*|
+    tests/data/.*|
+    .*\.egg-info/.*
+  )$
+
+repos:
+  - repo: https://github.com/brandon-arrendondo/knots
+    rev: v1.15.1
+    hooks:
+      - id: knots
+
+  - repo: https://github.com/tvanfossen/doxygen-guard
+    rev: 1.4.2
+    hooks:
+      - id: doxygen-guard
+        args: [validate]
+        files: ^(clew|acceptance|tests)/.*\.py$
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.16.1
+    hooks:
+      - id: ruff
+        args: ["--fix", "--select=E4,E7,E9,F", "--target-version=py310", "--cache-dir=.cache/ruff"]
+      - id: ruff-format
+        args: ["--line-length=100", "--target-version=py310", "--cache-dir=.cache/ruff"]
+
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-json
+      - id: check-toml
+      - id: check-added-large-files
+        args: ['--maxkb=512']
+      - id: check-merge-conflict
+      - id: check-symlinks
+      - id: mixed-line-ending
+
+  - repo: local
+    hooks:
+      - id: pytest
+        name: pytest
+        ## `cache_dir` is set once, in pyproject's `[tool.pytest.ini_options]`. It used to be
+        ## overridden here as well, which is two places for one setting and therefore one place
+        ## to forget when it changes.
+        entry: .venv/bin/python -m pytest tests/ -q
+        language: system
+        pass_filenames: false
+        always_run: true

clew_trace-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tristan Van Fossen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

clew_trace-1.0.0/PKG-INFO

@@ -0,0 +1,284 @@
+Metadata-Version: 2.5
+Name: clew-trace
+Version: 1.0.0
+Summary: Queryable, layered symbol database for C, C++ and Python codebases: doxygen + tree-sitter call graph, threads, locks and dataflow edges, served to AI agents over MCP.
+Project-URL: Homepage, https://github.com/tvanfossen/clew
+Project-URL: Repository, https://github.com/tvanfossen/clew
+Project-URL: Issues, https://github.com/tvanfossen/clew/issues
+License-Expression: MIT
+License-File: LICENSE
+Keywords: call-graph,documentation,doxygen,embedded,mcp,static-analysis,traceability,tree-sitter
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Documentation
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Requires-Dist: anyio>=4.0
+Requires-Dist: doxygen-guard<2,>=1.4.2
+Requires-Dist: kconfiglib<15,>=14
+Requires-Dist: mcp-types<3,>=2
+Requires-Dist: mcp<3,>=2
+Requires-Dist: pyyaml<8,>=6.0
+Requires-Dist: rich>=13.0
+Requires-Dist: tomli>=2; python_version < '3.11'
+Requires-Dist: tree-sitter-c<0.23.5,>=0.23
+Requires-Dist: tree-sitter-cpp<0.23.5,>=0.23
+Requires-Dist: tree-sitter-python<0.23.5,>=0.23
+Requires-Dist: tree-sitter<0.24,>=0.23
+Provides-Extra: dev
+Requires-Dist: build>=1.0; extra == 'dev'
+Requires-Dist: pre-commit>=3.6; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: twine>=5.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# clew
+
+> *A **clew** is the ball of thread Ariadne gave Theseus — the thing you follow to find your way
+> out of the labyrinth. It is the archaic spelling, and the direct ancestor, of the word "clue".*
+
+**Give an agent the thread, not a map of the whole maze.** `clew` runs doxygen and tree-sitter
+over a C, C++ or Python repository and compiles the result into a SQLite graph — symbols, call
+edges, callbacks, threads, locks, dataflow, requirement links, file inventory, prose — then
+serves it over MCP. Pull on one symbol and you get its callers, its locks, the thread it runs on
+and the requirement it satisfies.
+
+**What it is for is aggregation, not capability.** Nothing here is a question `grep` cannot
+answer. The difference is that "who calls this, transitively, across a function-pointer
+boundary, and which of those run on another thread" is one query instead of a dozen searches
+and a mental model you rebuild every session. It is a cache for work you would otherwise redo.
+
+**doxygen-guard is optional.** [It](https://github.com/tvanfossen/doxygen-guard) is a
+pre-commit gate that keeps a repo's doxygen accurate, and a repo that uses it gets a richer
+index — briefs, requirement tags, versioned comments. But `clew` needs neither the gate nor a
+Doxyfile: a repo that declares nothing gets its whole tree indexed, and three of the four repos
+in `acceptance/targets/` are measured that way. If you want the gate, it is a separate tool with
+one job; if you do not, this still works.
+
+## The four tools
+
+An agent is the user, and these are the whole of what the server exposes. A human-facing browser
+over the index is deliberately deferred: repository searchability for a person is what an IDE
+already is, so building a second one is not where the value is.
+
+| tool | for |
+|---|---|
+| **`dossier`** | Everything about one named symbol in ONE reply: body excerpt, brief, both edge directions, locks held, the thread it runs on, requirements and covering tests, liveness. There is no separate `callers` or `chain_trace` tool — this is why. |
+| **`search`** | Find the name when you do not have one, or enumerate a whole layer. `corpus=` picks: symbols, prose, files, config, locks, threads. An empty result is graded and says what it did *not* read. |
+| **`index(action=…)`** | Administration, not questions: `refresh` builds, `status` diagnoses, `targets` lists every indexed repo, `stats` grades one, `cull` deletes aged-out databases. |
+| **`propose_declaration`** | Reads the repo's evidence and proposes a `.clew.yaml`, entirely as comments, for when a causal layer looks emptier than the code warrants. |
+
+Every reply names the `target` it answered from, and carries a `staleness` block only when that
+index is actually stale — so a current index costs nothing to establish.
+
+Reads are live: query tools open the database per call. Writes are refused when the server
+process predates its own source, because a build through stale logic can silently drop whole
+layers and then report success.
+
+## What's in the database
+
+| Layer | Answers |
+|---|---|
+| Structure | "what is this symbol?" — doxygen symbol tables, briefs, extents |
+| Prose | "*why* does this subsystem exist?" — README/docs markdown, FTS5-indexed |
+| Call graph | "who calls this, and how sure are we?" — doxygen ×2 + tree-sitter, every edge carrying `source` + `confidence` |
+| Callbacks | "who reaches it through a function pointer?" — registration→dispatch resolution |
+| Shared-key dataflow | "writing this key affects whom?" — data-model / queue edges the call graph cannot see |
+| **Threads** | "which thread does this run on?" — spawn-site harvest + membership closure |
+| **Dispatch semantics** | inline vs queued vs keyed; edge-triggered; **which hops cross a thread boundary** |
+| **Terminus** | where a chain leaves the repo through an externally-registered callback |
+| Requirements | "what implements / tests REQ-X?" — `@req` traceability |
+| Liveness | "does this code even run?" — reachability over the non-fuzzy call graph |
+
+The point of the semantic layers (threads, dispatch mode, terminus) is **causal
+chains**: reconstructing "message arrives → who handles it → where it goes →
+which thread hops → where it leaves the library → what requirements it touches"
+as a single query, not a manual archaeology session.
+
+## Measured
+
+Answering from the index is compared against an agent with only `Read`/`Grep`/`Glob`/`Bash` —
+same repository, same model, same sitting, same frozen questions. Four repositories, both arms,
+two model tiers. Every transcript, metric and per-mark grade is committed beside its rubric in
+[`acceptance/targets/`](https://github.com/tvanfossen/clew/tree/main/acceptance/targets); the results are in each target's `result.md`.
+
+Run it on yours: [`acceptance/targets/TEMPLATE/`](https://github.com/tvanfossen/clew/tree/main/acceptance/targets/TEMPLATE) has the rubric
+template and the runbook.
+
+## What semver covers
+
+**Covered** — a breaking change here means a major bump:
+
+- the `clew` CLI verbs and their flags (`init`, `propose`, `export`, and the
+  build invocation)
+- the MCP **tool names and their wire keys** — a consumer reads `edge_class`,
+  `confidence`, `source` and friends by name
+- `clew.query`'s public functions and the dataclasses they return
+
+**Not covered**, and deliberately so:
+
+- **the SQLite schema.** `CLEW_BUILD_VERSION` exists precisely because the
+  schema moves; a consumer detects a stale index by comparing it, not by assuming
+  stability. Query through `clew.query`, not with your own SQL.
+
+  `CLEW_BUILD_VERSION`, `read_build_signature` and `index_unusable_reason` are
+  re-exported from `clew.query` and are therefore **covered**, so the documented
+  way to check freshness does not reach into an internal:
+
+  ```python
+  from clew.query import CLEW_BUILD_VERSION, read_build_signature
+
+  if read_build_signature("clew.db") != CLEW_BUILD_VERSION:
+      ...  # rebuild: this index predates the current pipeline
+  ```
+- `clew.*` internals — anything not re-exported from `query`
+- the `.idxcache` format
+
+## Install
+
+The package is always required — the server *is* the `clew-mcp` command it installs, and no
+plugin or registration step can supply that. Install it so the command is on PATH whatever
+virtualenv happens to be active:
+
+```bash
+pipx install clew-trace          # or: pip install --user clew-trace
+sudo apt install doxygen         # REQUIRED, and not pip-installable
+```
+
+A plain `pip install` inside a project venv also works, but only while that venv is on the PATH
+your editor launches the server with — which is the usual reason a correctly-registered server
+shows as failed to connect.
+
+Then register it, EITHER as a Claude Code plugin:
+
+```
+/plugin marketplace add tvanfossen/clew
+/plugin install clew@clew
+```
+
+or by hand, which also doctors the environment and names what is missing:
+
+```bash
+clew init                        # writes ./.mcp.json  (--scope global for user-level)
+```
+
+Pick one. Both register a server named `clew`, so doing both leaves two sources for one entry —
+run `claude mcp remove clew` first if you have already used `clew init`.
+
+`doxygen` is a C++ binary, so it cannot be a Python dependency — `clew init` checks for it and
+tells you if it is missing. Everything else, including the MCP SDK, comes with the package.
+
+### Working on clew itself
+
+```bash
+python3 -m venv .venv
+.venv/bin/pip install -e .[dev]   # pipeline + doxygen-guard + gates
+sudo apt install doxygen          # external binary (plantuml optional)
+```
+
+## Build a database
+
+```bash
+.venv/bin/python -m clew \
+    --output clew.db \
+    --repo-root /path/to/repo
+```
+
+That is the whole common case. The Doxyfile is discovered, and a repo that ships none gets
+one synthesized over its declared scope.
+
+**Six arguments, and only six:**
+
+| Flag | For |
+|---|---|
+| `--output` | where the database is written |
+| `--repo-root` | the repository to index |
+| `--declare FILE` | a declaration document stated for this build (see below) |
+| `--exclude PATH…` | paths to leave out; recorded into the index and replayed by later builds |
+| `--rebuild` | ignore every cached entry, then re-warm the cache |
+| `--verbose` | more on stderr; changes nothing that reaches the index |
+
+Everything a target repo can *declare* lives in one place — its own
+`.clew.yaml` — and `--declare` states the same document for a repository you do
+not own and must leave byte-identical:
+
+```yaml
+# a --declare document, or <repo>/.clew.yaml
+index_scope:          {roots: [src, include], excludes: [src/generated]}
+preprocessor:         {predefined: [MY_FEATURE_C], config_header: auto}
+requirements:         docs/requirements.yaml
+thread_patterns:      {spawns: [{name: osThreadNew, entry_arg_index: 0}]}
+shared_key_patterns:  {writers: [{name_prefix: Store_Set}], readers: [{name_prefix: Store_Get}]}
+locks:                {locks: [{name: MyGuard, form: raii}]}
+dispatch:             {interfaces: [...]}
+mqtt_dispatch:        {subscribe_functions: [...]}
+data_model:           tools/dm_full.toml
+kconfig:              {path: Kconfig}
+event_tags:           {emits: produce, handles: consume}
+entry_patterns:       ["%trampoline%", "on_%"]
+enrich:               docs/architecture_topics.yaml
+```
+
+Nothing about a target repo is hardcoded: every section above is a declared override of
+a built-in default. The identical mapping is the `options` argument of `build_index()` and
+of the MCP `index(action="refresh")` tool, so an agent with no shell can state any of it.
+
+The repo's `.doxygen-guard.yaml` — which supplies the declared `@req` id pattern and the
+catalog column mapping — is **discovered** from `--repo-root`: its root, then the path the
+doxygen-guard pre-commit hook names in its own `--config` argument, then
+`conf/ | config/ | .config/`.
+
+### Commit what you stated
+
+`--declare` reads a document in; `export` writes one out, so a discovery made against a
+repository you do not own can be committed to one you do:
+
+```bash
+.venv/bin/python -m clew export --index clew.db          # to stdout
+.venv/bin/python -m clew export --index clew.db --output .clew.yaml
+```
+
+It emits **only what was stated** — never the built-in defaults. That is deliberate: a file
+asserting you declared every default would freeze those defaults into your repo, so a later
+improvement to one would be shadowed by a value nobody remembers committing. An index that
+recorded no statement exports a document that says so.
+
+## Without an agent
+
+The same four capabilities are importable, returning JSON-serializable dataclasses. This is what
+the MCP server is a view over, so no query logic exists twice.
+
+```python
+from clew.query import dossier, search
+
+dossier("clew.db", "sensor_poll")    # identity, reqs, both edge directions, locks, threads, liveness
+search("clew.db", "retry backoff")   # find the name when you do not have one
+```
+
+## Layout
+
+| Path | What |
+|---|---|
+| `clew/` | The pipeline (`python -m clew`) |
+| `clew/query/` | The stable query API the MCP server is a view over |
+| `tests/` | `.venv/bin/python -m pytest tests/ -q` (add `--integration` for the tier that builds real repos) |
+| `acceptance/targets/<t>/<version>/` | Frozen rubrics and the committed results of every grid run against them |
+| `acceptance/bench/` | The harness. Acceptance-only by design — deliberately NOT in the pre-commit gate |
+
+## Dogfooding
+
+clew runs its own gate: `.venv/bin/pre-commit run --all-files` → ruff (lint +
+format), doxygen-guard (`@brief`/`@version`/`@return` presence on the shipped
+package), and the full test suite. Its own catalog lives in `requirements.yaml`.
+
+## doxygen-guard
+
+[doxygen-guard](https://github.com/tvanfossen/doxygen-guard) is a separate tool, consumed here as
+a pip dependency and pinned by `rev:` in `.pre-commit-config.yaml`. Its scope is
+validation, traceability and change impact; nothing here extends it, and `clew` does not require
+it — see the top of this file.