codegraph-brain 0.27.0__tar.gz → 0.28.0__tar.gz

codegraph_brain-0.28.0/.github/test-count-baseline

@@ -0,0 +1 @@
+2943

codegraph_brain-0.28.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.28.0"
+}

{codegraph_brain-0.27.0 → codegraph_brain-0.28.0}/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## [0.28.0](https://github.com/zaebee/codegraph-brain/compare/codegraph-brain-v0.27.0...codegraph-brain-v0.28.0) (2026-10-02)
+
+
+### Features
+
+* **bench:** agent A/B harness for cgis vs no cgis ([#547](https://github.com/zaebee/codegraph-brain/issues/547)) ([788d67f](https://github.com/zaebee/codegraph-brain/commit/788d67f230f1129379afb08596da5a4ef9e1c302))
+* **bench:** cgis-instructed and cgis-forced arms, four multi-hop impact tasks ([#549](https://github.com/zaebee/codegraph-brain/issues/549)) ([deef289](https://github.com/zaebee/codegraph-brain/commit/deef289fe9a9a302088661acbf3c6f3d33fc50fe))
+
 ## [0.27.0](https://github.com/zaebee/codegraph-brain/compare/codegraph-brain-v0.26.0...codegraph-brain-v0.27.0) (2026-10-02)
 
 

{codegraph_brain-0.27.0 → codegraph_brain-0.28.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: codegraph-brain
-Version: 0.27.0
+Version: 0.28.0
 Summary: Semantic code graph for AI agents — deterministic FQN resolution, impact analysis and architectural drift gates, exposed over MCP.
 Project-URL: Homepage, https://github.com/zaebee/codegraph-brain
 Project-URL: Repository, https://github.com/zaebee/codegraph-brain

codegraph_brain-0.28.0/benchmarks/agent_ab/.gitignore

@@ -0,0 +1,3 @@
+# Raw stream-json sessions: large, and they quote repository source verbatim
+# (owner-api is private). results.jsonl keeps the scored line for each.
+transcripts/

codegraph_brain-0.28.0/benchmarks/agent_ab/README.md

@@ -0,0 +1,92 @@
+# Agent A/B: the same questions with and without cgis (#543)
+
+Does an agent answer a code question better, or cheaper, when the cgis MCP
+server is connected? This directory holds the questions, their answer keys and
+the results. The runner is `scripts/agent_ab.py`; parsing and scoring live in
+`src/cgis/bench/` and are unit-tested on recorded transcripts, so the harness
+can change without spending anything.
+
+Design notes and cost estimates: `research/543.md` in the project files.
+
+## What is measured
+
+**Primary, fixed before the first paid run:**
+
+- `recall`: the share of the answer key the answer names (symbols, files and
+  literal facts, pooled). `precision`: the share of what it names that is in
+  the key or in its `allowed_*` lists.
+- `cost_usd`: `total_cost_usd` from the session's closing event.
+
+The hypothesis is that the `cgis` arm reaches at least the control arm's recall
+for no more cost on `impact` and `flow` questions, and that `control`-type
+questions show what cgis costs when it cannot help.
+
+**Secondary:** tool calls by name, distinct files read and bytes read, turns,
+wall time, token usage, and the context the session ends holding
+(`residual_context`).
+
+**For #220, not for ranking arms:** `sufficiency` (what the agent did right
+after each cgis answer) and `allocation` (the share of files cgis named that
+the final answer relied on). Definitions are in the docstring of
+`src/cgis/bench/transcript.py`.
+
+The answer key is written from the code and checked by hand, never taken from
+the cgis graph: a key derived from the graph would agree with every edge the
+resolver gets wrong. Each task's `notes` say how it was checked.
+
+## Arms
+
+| arm | MCP | plugin skills | graph files | cgis / uv on PATH |
+|---|---|---|---|---|
+| `control` | none (`--strict-mcp-config`, empty config) | no | all deleted | no |
+| `cgis` | this checkout's `cgis-mcp` | yes (the plugin minus its `.mcp.json`) | `graph.db` built before the clock starts | no |
+| `cgis-instructed` | as `cgis` | as `cgis` | as `cgis` | one line: query cgis first |
+| `cgis-forced` | as `cgis` | as `cgis` | as `cgis` | as `cgis-instructed`, and the guard refuses Read/Grep/Glob/Bash until one cgis call |
+
+`cgis-instructed` exists because the first pilot's `cgis` arm made no cgis call in
+12 of 12 sessions: with the server connected and the skill loaded, Sonnet still
+went straight to Grep. It stands in for #542's MCP server instructions. It barely moved the agent (2 cgis
+calls in 18 sessions), so `cgis-forced` makes the first graph query mandatory: it
+measures what the graph adds once used, apart from whether the agent picks it.
+
+All arms run under `cgis.bench.guard` as a PreToolUse hook, which refuses the
+cgis CLI, uv, sqlite3 and any read of `graph.db`/`graph.json` through Bash or
+the file tools. Without it the control arm is not a control: codegraph's own
+benchmark caught its control agent calling their CLI through Bash in 26 of 28
+runs. The same predicate marks a finished run `contaminated` if a blocked call
+ever returned output; such runs are counted in the report and left out of the
+medians.
+
+All arms allow `Read`, `Grep`, `Glob` and `Bash` (plus `mcp__cgis` in the
+treatment arms, which has no server in control) and refuse edits, web access and
+sub-agents, under `--permission-mode dontAsk`. Each session starts in a fresh
+detached worktree at the task's pinned commit, with `--no-session-persistence`
+and `--setting-sources project`, so user settings and earlier sessions do not
+reach it.
+
+## Running
+
+```bash
+uv run python scripts/agent_ab.py run --dry-run          # commands only, spends nothing
+uv run python scripts/agent_ab.py run --task cgis-impact-language-for --runs 2
+uv run python scripts/agent_ab.py run --repo owner-api=../ownima-backend
+uv run python scripts/agent_ab.py report
+```
+
+`--model` defaults to `claude-sonnet-5-5`; `--max-budget-usd` (default 2.00)
+caps each session. Every non-dry run is a paid session, so this never runs in
+CI. Transcripts are written to `transcripts/` (git-ignored); `results.jsonl`
+gets one line per (task, arm, run) and is meant to be committed with the write-up.
+
+## Tasks
+
+| id | type | repo |
+|---|---|---|
+| `cgis-impact-language-for` | impact | cgis |
+| `cgis-flow-ingest-to-sqlite` | flow | cgis |
+| `cgis-orientation-call-resolution` | orientation | cgis |
+| `cgis-control-sqlite-pragmas` | control | cgis |
+
+This repository's `CLAUDE.md` describes its architecture in some detail. It is
+the same in both arms, but it lowers what a graph can add here, which is one
+reason owner-api and httpx come next.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-control-sqlite-pragmas.yaml

@@ -0,0 +1,23 @@
+id: cgis-control-sqlite-pragmas
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: control
+question: |
+  Which SQLite journal mode and busy timeout does `SQLiteStore` configure when it
+  opens a database, and in which method?
+gold:
+  symbols:
+    - SQLiteStore.connect
+  files:
+    - src/cgis/storage/sqlite_store.py
+  facts:
+    - WAL
+    - "5000"
+  allowed_symbols:
+    - SQLiteStore
+notes: |
+  Negative control: one file, one method, no cross-file structure. A graph should
+  not help here; this measures what cgis costs when it is not needed.
+  sqlite_store.py:83-84, inside SQLiteStore.connect (line 75), not __init__:
+  the first pilot session caught a key that said __init__.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-flow-ingest-to-sqlite.yaml

@@ -0,0 +1,50 @@
+id: cgis-flow-ingest-to-sqlite
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: flow
+question: |
+  Trace what happens when `cgis ingest <path> --output graph.db` runs: starting
+  from the CLI command, list in order the functions and methods on the main path
+  that parse source files, resolve calls, and write nodes and edges to SQLite.
+gold:
+  symbols:
+    - cli.ingest
+    - IngestionPipeline.run
+    - IngestionPipeline._process_file
+    - id: extractor.parse
+      any_of: [BaseExtractor.parse, PythonExtractor.parse, TypeScriptExtractor.parse]
+    - ResolverEngine.resolve
+    - IngestionPipeline._persist_incremental
+    - SQLiteStore.save_incremental_batch
+  files:
+    - src/cgis/cli.py
+    - src/cgis/pipeline.py
+    - src/cgis/resolver/engine.py
+    - src/cgis/storage/sqlite_store.py
+  allowed_symbols:
+    - IngestionPipeline.workspace_root
+    - SQLiteStore.record_ingest
+    - SQLiteStore.record_ingest_options
+    - build_extractors
+    - IngestionPipeline._get_extractor
+    - ImportNameCollector.collect
+    - SemanticUpliftEngine.execute_uplift
+    - SymbolResolver
+    - IndexBuilder.build
+  allowed_files:
+    - src/cgis/extractors/base.py
+    - src/cgis/extractors/python_extractor.py
+    - src/cgis/extractors/typescript_extractor.py
+    - src/cgis/extractors/registry.py
+    - src/cgis/resolver/symbols.py
+    - src/cgis/resolver/indices.py
+    - src/cgis/resolver/uplift.py
+    - src/cgis/import_names.py
+notes: |
+  Read 2026-10-02: cli.ingest (cli.py:174) opens SQLiteStore and calls
+  IngestionPipeline.run(store=..., rebuild=True); run calls _process_file per file,
+  which calls the extractor's parse; then ResolverEngine.resolve (pipeline.py:213);
+  then _persist_incremental (pipeline.py:237), which calls
+  SQLiteStore.save_incremental_batch (pipeline.py:411). Order is not scored, only
+  presence.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-impact-drift-same-name.yaml

@@ -0,0 +1,56 @@
+id: cgis-impact-drift-same-name
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: impact
+question: |
+  In this repository, which code under `src/` can end up calling
+  `PatternCatalog.load_project_domains` in `src/cgis/query/drift/catalog.py`, directly
+  or transitively? Follow the chain up to the user-facing entry points (CLI commands
+  and MCP tools, and the Guardian review runner) and name every function or method
+  on the way.
+gold:
+  symbols:
+    - DriftScorer.load_project_domains
+    - drift_service.analyze_drift
+    - GraphContextCollector.collect_drift
+    - ContextCollector.collect_all
+    - cli.drift
+    - mcp_server.cgis_drift
+    - id: guardian
+      any_of: [run_guardian, run_review_routed, GuardianReviewer.run_review, run_axis_review]
+  files:
+    - src/cgis/query/drift/drift.py
+    - src/cgis/query/drift/drift_service.py
+    - src/cgis/guardian/collector.py
+    - src/cgis/cli.py
+    - src/cgis/api/mcp_server.py
+  allowed_symbols:
+    - PatternCatalog
+    - PatternCatalog.load_project_domains
+    - DriftScorer
+    - GraphContextCollector
+    - ContextCollector
+    - GuardianReviewer
+    - run_guardian
+    - run_review_routed
+    - run_axis_review
+    - run_chunked_review
+    - chunked._single_pass
+  allowed_files:
+    - src/cgis/query/drift/catalog.py
+    - src/cgis/guardian/core.py
+    - src/cgis/guardian/axes.py
+    - src/cgis/guardian/chunked.py
+    - src/cgis/guardian/runner.py
+notes: |
+  The first hop shares the target's name: DriftScorer.load_project_domains
+  (drift.py:58) wraps it, so a grep for the name finds both definitions and three
+  call sites that must be told apart. Read 2026-10-02 at the pinned sha:
+  DriftScorer.load_project_domains is called in analyze_drift (drift_service.py:162)
+  and GraphContextCollector.collect_drift (guardian/collector.py:249); analyze_drift
+  in cli.drift (cli.py:1213) and mcp_server.cgis_drift (mcp_server.py:636);
+  collect_drift in ContextCollector.collect_all (collector.py:353), which
+  GuardianReviewer.run_review (core.py:128) and run_axis_review (axes.py:68) call,
+  up through run_review_routed to run_guardian (runner.py:472). Any one Guardian
+  hop counts for the guardian item.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-impact-language-for.yaml

@@ -0,0 +1,27 @@
+id: cgis-impact-language-for
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: impact
+question: |
+  In this repository, the function `language_for` in `src/cgis/extractors/registry.py`
+  is about to change its signature. List every function or method that calls it
+  directly, so each call site can be updated.
+gold:
+  symbols:
+    - cli.structure
+    - registry.is_supported
+    - SourceCollector.collect_full_files
+    - GraphContextCollector.sections
+    - mermaid._node_slug
+  files:
+    - src/cgis/cli.py
+    - src/cgis/extractors/registry.py
+    - src/cgis/guardian/collector.py
+    - src/cgis/query/render/mermaid.py
+  allowed_symbols:
+    - registry.language_for
+notes: |
+  Checked 2026-10-02 by grep (`language_for(` outside imports) and by reading each
+  enclosing def: cli.py:843, registry.py:88, collector.py:175 and :214,
+  mermaid.py:77. The cgis graph agrees, but the key was not taken from it.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-impact-transitive-tsconfig.yaml

@@ -0,0 +1,45 @@
+id: cgis-impact-transitive-tsconfig
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: impact
+question: |
+  In this repository, which code under `src/` can end up executing the function
+  `_existing_json` in `src/cgis/tsconfig_paths.py`, directly or transitively? Follow
+  the chain all the way up to the user-facing entry points (CLI commands and MCP
+  tools) and name every function or method on the way.
+gold:
+  symbols:
+    - tsconfig_paths._locate
+    - tsconfig_paths._options
+    - TsconfigPaths.aliases
+    - ImportNameCollector.collect
+    - IngestionPipeline.run
+    - cli.ingest
+    - mcp_server.cgis_ingest
+    - id: auto_refresh
+      any_of: [auto_refresh._refresh, auto_refresh.refresh_if_stale, auto_refresh.refreshes_graph]
+  files:
+    - src/cgis/tsconfig_paths.py
+    - src/cgis/import_names.py
+    - src/cgis/pipeline.py
+    - src/cgis/cli.py
+    - src/cgis/api/mcp_server.py
+    - src/cgis/api/auto_refresh.py
+  allowed_symbols:
+    - tsconfig_paths._existing_json
+    - TsconfigPaths
+    - ImportNameCollector
+    - IngestionPipeline
+    - auto_refresh
+    - api.mcp_server
+notes: |
+  Multi-hop impact, five calls deep before the first entry point. Read 2026-10-02 at
+  the pinned sha: _existing_json is called only in _locate (tsconfig_paths.py:190,
+  :203); _locate only in _options (:153); _options in TsconfigPaths.aliases (:122)
+  and recursively in itself (:154); aliases only in ImportNameCollector.collect
+  (import_names.py:82); collect in IngestionPipeline.run (pipeline.py:193); run in
+  cli.ingest (cli.py:236, :243), mcp_server.cgis_ingest (mcp_server.py:354) and
+  auto_refresh._refresh (auto_refresh.py:82), which refresh_if_stale and the
+  refreshes_graph decorator on most MCP tools reach when CGIS_AUTO_REFRESH=1, so any
+  MCP tool name is allowed. The cgis graph agrees; the key was checked by grep.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/cgis-orientation-call-resolution.yaml

@@ -0,0 +1,36 @@
+id: cgis-orientation-call-resolution
+repo: cgis
+sha: 715cd9a2ecee47949651344757edb4c7d09f64d7
+src_root: src
+type: orientation
+question: |
+  Which files implement turning unresolved call targets (`raw_call:<name>` edges
+  emitted by the extractors) into fully qualified names, and what does each of
+  them contribute?
+gold:
+  symbols:
+    - ResolverEngine
+    - SymbolResolver
+    - id: indices
+      any_of: [IndexBuilder, SymbolIndex]
+  files:
+    - src/cgis/resolver/engine.py
+    - src/cgis/resolver/symbols.py
+    - src/cgis/resolver/indices.py
+  allowed_symbols:
+    - ResolverEngine.resolve
+    - ResolverEngine._resolved_call_edge
+    - js_builtin_target
+    - IngestionPipeline.run
+    - Edge
+  allowed_files:
+    - src/cgis/resolver/js_builtins.py
+    - src/cgis/core/models.py
+    - src/cgis/pipeline.py
+    - src/cgis/extractors/python_extractor.py
+    - src/cgis/extractors/typescript_extractor.py
+    - src/cgis/extractors/base.py
+notes: |
+  Read 2026-10-02: engine.py drives resolution, symbols.py holds the strategies
+  over a SymbolIndex, indices.py builds the indices (module docstrings say so).
+  js_builtins.py and models.py are correct but optional.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/owner-impact-credit-wallet.yaml

@@ -0,0 +1,55 @@
+id: owner-impact-credit-wallet
+repo: owner-api
+sha: 8de22452b5d91242dd742a666ef2cd7b1eb7c1ff
+src_root: ownima-backend/app
+type: impact
+question: |
+  In this repository, the module-level function `credit_wallet` in
+  `ownima-backend/app/domains/finance/service.py` is about to change. Which code under
+  `ownima-backend/app/` (not tests, not scripts) can end up calling it, directly or
+  transitively? Follow every chain up to its entry point (an HTTP route, a worker task,
+  or a function nothing else calls) and name every function or method on the way.
+gold:
+  symbols:
+    - FinanceService.credit_wallet
+    - FinanceService.top_up
+    - TopUpService._credit
+    - TopUpService.fulfill
+    - TopUpService.get_status
+    - TopUpService.handle_webhook_event
+    - finance.routes.get_top_up
+    - finance.routes.stripe_webhook
+    - VehicleBonusCreditor._pay
+    - VehicleBonusCreditor.credit
+    - VehicleBonusWorker.credit
+    - vehicle_bonus.reconcile_vehicle_bonus
+    - VehicleBonusWorker.reconcile
+  files:
+    - ownima-backend/app/domains/finance/service.py
+    - ownima-backend/app/domains/finance/top_up_service.py
+    - ownima-backend/app/domains/finance/routes.py
+    - ownima-backend/app/domains/finance/vehicle_bonus_credit.py
+    - ownima-backend/app/worker/vehicle_bonus.py
+  allowed_symbols:
+    - finance.service.credit_wallet
+    - FinanceService
+    - TopUpService
+    - VehicleBonusCreditor
+    - VehicleBonusWorker
+    - worker.base
+  allowed_files:
+    - ownima-backend/app/worker/base.py
+notes: |
+  Same-name trap plus DI: the module function is wrapped by the method
+  FinanceService.credit_wallet (service.py:332-344), and the top-up path reaches the
+  method through an injected `self._finance` (top_up_service.py:316). Read 2026-10-02
+  at the pinned sha with an AST call index, not with cgis: credit_wallet is called in
+  FinanceService.credit_wallet (service.py:344) and VehicleBonusCreditor._pay
+  (vehicle_bonus_credit.py:202); the method in FinanceService.top_up (:367, no callers
+  of its own) and TopUpService._credit (top_up_service.py:316); _credit in fulfill
+  (:266); fulfill in get_status (:294) and handle_webhook_event (:305); those in the
+  routes get_top_up (routes.py:232) and stripe_webhook (:344). _pay in
+  VehicleBonusCreditor.credit (:142); that in VehicleBonusWorker.credit
+  (worker/vehicle_bonus.py:151) and reconcile_vehicle_bonus (:106); the latter in
+  VehicleBonusWorker.reconcile (:177). Both worker methods are registered as tasks in
+  worker/base.py:198-199. The cgis graph at this sha misses both routes.

codegraph_brain-0.28.0/benchmarks/agent_ab/tasks/owner-impact-occupied-window-same-name.yaml

@@ -0,0 +1,30 @@
+id: owner-impact-occupied-window-same-name
+repo: owner-api
+sha: 8de22452b5d91242dd742a666ef2cd7b1eb7c1ff
+src_root: ownima-backend/app
+type: impact
+question: |
+  In this repository, the function `occupied_window` in
+  `ownima-backend/app/services/reservation_conflict_service.py` is about to change its
+  signature. List every function or method under `ownima-backend/app/` (not tests,
+  not scripts) that calls it directly, so each call site can be updated.
+gold:
+  symbols:
+    - ReservationConflictService._check_conflicts_internal
+    - ReservationConflictService._find_conflicting_groups
+    - ReservationConflictService._clusters
+    - ReservationConflictService.get_pending_conflicts_for_reservation
+  files:
+    - ownima-backend/app/services/reservation_conflict_service.py
+  allowed_symbols:
+    - reservation_conflict_service.occupied_window
+    - ReservationConflictService
+notes: |
+  Same-name trap: domains/pricing/handover.py:230 defines another module-level
+  `occupied_window`, and HandoverSchedule.window_for (handover.py:195),
+  ReservationCreation.compute_actual_datetime_range (creation.py:716) and Rule.build
+  (validators.py:164) call that one (creation.py:21-28 and validators.py:18-25 import it
+  from app.domains.pricing.handover). Naming them costs precision. Read 2026-10-02 at
+  the pinned sha with an AST call index: the services function (line 836) is called at
+  reservation_conflict_service.py:241, :552, :661, :678, :736 and :753, inside the four
+  methods above.

{codegraph_brain-0.27.0 → codegraph_brain-0.28.0}/pyproject.toml

@@ -7,7 +7,7 @@ packages = ["src/cgis"]
 
 [project]
 name = "codegraph-brain"
-version = "0.27.0"
+version = "0.28.0"
 description = "Semantic code graph for AI agents — deterministic FQN resolution, impact analysis and architectural drift gates, exposed over MCP."
 readme = "README.md"
 requires-python = ">=3.12"