@pmaddire/gcie 0.1.13 → 0.1.15

package/GCIE_USAGE.md

@@ -23,9 +23,14 @@ Priority order:
 
 Primary retrieval:
 ```powershell
-gcie.cmd context <path> "<query>" --intent <edit|debug|refactor|explore> --budget <auto|int> --mode <basic|adaptive>
+gcie.cmd context <path> "<query>" --intent <edit|debug|refactor|explore> --budget <auto|int> --mode <basic|adaptive> --usage-policy <hybrid|force|minimal|off>
 ```
 
+Usage policy guidance:
+1. `hybrid` (default): balanced accuracy and token cost.
+2. `force`: strict-accuracy path with stronger fallback gating.
+3. `minimal`/`off`: smallest context for known-file, low-risk checks.
+
 Sliced retrieval:
 ```powershell
 gcie.cmd context-slices <path> "<query>" --intent <edit|debug|refactor|explore> --profile <low|recall|adaptive>
@@ -39,7 +44,7 @@ gcie.cmd adaptive-profile . --clear
 
 Post-init adaptation pipeline:
 - run from the target repo root (cd <repo> first); use . as scope
-- adaptation now bootstraps per-family method defaults before accuracy rounds (plain/plain-gapfill/plain-rescue/slices)
+- adaptation now bootstraps per-family method defaults before accuracy rounds (`plain_minimal`/`plain`/`plain_force` plus chain/gapfill/rescue/slices)
 - adaptation case generation is mixed by design (single-file, same-layer pairs, cross-subtree pairs, and some 3-file chains on larger runs)
 ```powershell
 gcie.cmd adapt . --benchmark-size 10 --efficiency-iterations 5 --clear-profile

package/README.md

@@ -6,36 +6,34 @@ It is designed for coding-agent workflows where we want to retrieve the smallest
 useful set of code and operational context instead of reading whole files or
 whole directories into the model.
 
-## How It Works
-
-GCIE builds a retrieval-oriented view of a repository and then composes context
-from several signals:
-
-1. Repository scan
-   - discovers source files, frontend files, config files, and selected docs
-2. Graph and index construction
-   - structure and relationship data
-   - semantic search index
-   - architecture-oriented metadata where available
-3. Multi-channel retrieval
-   - lexical filename/path/content matching
-   - semantic vector matching
-   - query expansion for code and system terms
-   - adjacency/support-file recovery
-4. Fusion and reranking
-   - merges candidates with stable deterministic ordering
-   - boosts exact file mentions, wiring files, and intent-relevant code
-5. Context packing
-   - returns compact snippets or file-level context depending on the task
-   - preserves important support files when confidence would otherwise be weak
-6. Fallback
-   - if the optimized path looks insufficient, GCIE can recover extra files via
-     a broader fallback search instead of silently returning thin context
-
-The practical goal is simple: return the implementation file, the wiring file,
-and the nearest supporting files that explain behavior, while avoiding the token
-cost of sending full repo surfaces to the model.
-
+## How It Works
+
+GCIE is an adaptive context retrieval engine for coding agents.
+
+At a high level:
+
+1. Index + architecture snapshot
+   - `gcie index .` scans the repo and builds retrieval artifacts under `.gcie/`.
+   - GCIE tracks architecture/retrieval state so it can route future queries better.
+
+2. Query classification
+   - `gcie context` classifies each request by intent and structure (single-file, same-layer pair, cross-layer, multi-hop).
+
+3. Retrieval routing
+   - GCIE chooses retrieval strategy (`plain`, `plain_gapfill`, `plain_chain`, or slices where useful), path scope, token budget, and usage policy (`hybrid`, `force`, or `minimal/off`).
+   - `--budget auto` uses built-in heuristics; explicit budgets are available when needed.
+
+4. Gap-fill + must-have recovery
+   - If expected support files are missing, GCIE runs targeted follow-up retrieval to recover must-have files instead of over-fetching whole repo context.
+
+5. Adaptation loop (optional but recommended)
+   - `gcie adapt .` benchmarks repo-local cases, selects per-family methods, and runs efficiency trials under an accuracy gate.
+   - Results are written to `.planning/post_init_adaptation_report.json` and `.gcie/context_config.json`.
+
+6. Fast path for day-to-day use
+   - After adaptation, most tasks should run through `gcie context` with small prompt footprints and high recall.
+
+The practical goal is to keep must-have coverage while minimizing token cost.
 ## Quick Start
 
 1. Create venv: `.venv\\Scripts\\python.exe -m venv .venv`
@@ -52,8 +50,8 @@ Use this when you want a fast drop-in setup for coding agents.
 2. Copy [GCIE_USAGE.md](c:\GBCRSS\GCIE_USAGE.md) into the target repo root.
 3. Run one index pass:
    - `gcie.cmd index .`
-4. Start using adaptive retrieval immediately:
-   - `gcie.cmd context . "<task>" --intent edit --budget auto`
+4. Start using adaptive retrieval immediately:
+   - `gcie.cmd context . "<task>" --intent edit --budget auto --mode adaptive --usage-policy hybrid`
 
 No heavy upfront tuning is required. The workflow starts portable-first and only adds local overrides after repeated miss patterns.
 
@@ -233,162 +231,94 @@ Important note:
 - `--budget 1200` consistently improved recall without needing broad manual reads
 - `1500` added more noise without materially helping more than `1200`
 
-## Core Commands
-
-- `gcie index <path>`
-- `gcie query <file.py> "<question>"`
-- `gcie debug <file.py> "<question>"`
-- `gcie context <repo|file> "<task>" --budget auto --intent <edit|debug|refactor|explore> --mode basic`
-- `gcie context-slices <repo> "<task>" --intent <edit|debug|refactor|explore> [--profile recall|low] [--stage-a 400] [--stage-b 800] [--max-total 1200] [--pin frontend/src/App.jsx] [--pin-budget 300] [--include-tests]`
-
-## How To Use It
-
-### 1. Index the repo
-
-```
-gcie index .
-```
-
-Re-run indexing after major structural changes.
-
-### 2. Start with plain `context`
-
-```
-gcie context . "<task>" --budget auto --intent <edit|debug|refactor|explore>
-```
-
-Recommended intent guidance:
-
-- `edit`: making code changes
-- `debug`: tracing a bug or incorrect behavior
-- `refactor`: changing structure or interfaces
-- `explore`: understanding code without immediate edits
-
-### 3. For cross-layer or wiring-heavy tasks, prefer a file-first query
-
-This works better than abstract phrasing:
-
-```
-gcie context . "frontend/src/App.jsx selectedTheme activeJobId /api/convert/start app.py start_convert" --budget 1200 --intent edit
-```
-
-Good query ingredients:
-
-- explicit file names
-- endpoint names
-- prop names
-- function names
-- config keys
-- state variables
-
-### 4. Use `context-slices` when you want the recall-first workflow
-
-```
-gcie context-slices . "<task>" --intent <edit|debug|refactor|explore>
-```
-
-Optional flags: `--profile low`, `--include-tests`, `--pin <path>`, `--max-total 1200`.
-
-### 5. Verify before editing
-
-GCIE should be treated as context compression, not final truth. For important
-edits, verify the returned context with a targeted local search:
-
-```
-rg -n "<key symbols>" app.py main.py frontend/src/App.jsx
-```
-
-## Usage Patterns That Work Best
-
-### Simple local tasks
-
-Use:
-
-```
-gcie context . "<task>" --budget auto --intent debug
-```
-
-### Cross-layer frontend/backend tasks
-
-Use:
-
-```
-gcie context . "<file-first symbol-rich query>" --budget 1200 --intent edit
-```
-
-Why:
-
-- the extra budget improves recall for wiring files
-- file-first phrasing reduces generic entrypoint noise
-
-### High-recall workflows
-
-Use:
-
-```
-gcie context-slices . "<task>" --intent edit --pin <expected wiring file>
-```
-
-This is still the safest mode when you already know a few must-have files.
-
-## Agent Workflow
-
-For coding agents, the safest practical pattern is:
-
-1. Run GCIE first
-2. Check that the result includes:
-   - the main implementation file
-   - the wiring or entry file
-   - at least one validation or test surface when relevant
-3. If a must-have file is missing:
-   - rerun with a more file-first query
-   - increase budget to `1000` or `1200`
-   - or pin the missing file in `context-slices`
-4. Verify with `rg` before editing
-
-This usually gives a much better accuracy/token tradeoff than broad manual file
-reading.
-
-## Cache
-
-Repo-wide context is cached to speed up repeated calls.
-
-- `gcie cache-warm .`
-- `gcie cache-status .`
-- `gcie cache-clear .`
-
-Cache file: `.gcie/cache/context_cache.json` (auto-invalidated on file changes).
-
-## Frontend and Non-Python Files
-
-Repo-wide context scans common frontend and config extensions and adds file nodes so
-queries can retrieve non-Python surfaces when relevant.
-
-Default extensions include: `.js`, `.jsx`, `.ts`, `.tsx`, `.css`, `.scss`, `.html`, `.vue`,
-plus `.json`, `.yaml`, `.yml`, `.toml`, `.md`, `.txt`.
-
-## Core Capabilities
-
-- Repository scanning
-- Graph construction (structure, call, variable, execution, git, test coverage)
-- Symbolic + semantic + hybrid retrieval
-- Bug localization
-- Minimal LLM context building
-- Architecture-aware context routing and fallback
-- Agent-friendly retrieval for edit/debug/refactor workflows
-
-## Publish For NPX
-
-From this repo:
-
-```powershell
-npm login
-npm publish --access public
-```
-
-Then users can run:
-
-```powershell
-npx -y @pmaddire/gcie@latest setup .
-```
-
+## Command Reference
+
+Use `gcie` or `gcie.cmd` on Windows.
+
+### Setup / Lifecycle
+
+- `gcie setup .`
+- `gcie setup . --force`
+- `gcie setup . --no-index`
+- `gcie setup . --adapt --adapt-benchmark-size 25 --adapt-efficiency-iterations 8 --adapt-workers 6`
+- `gcie remove .`
+- `gcie remove . --remove-planning`
+- `gcie remove . --keep-usage --keep-setup-doc`
+
+### Index and Retrieval
+
+- `gcie index .`
+- `gcie context . "<task>" --intent edit --budget auto --mode adaptive --usage-policy hybrid`
+- `gcie context . "<task>" --intent debug --budget 1200 --mode adaptive --usage-policy force`
+- `gcie context . "<task>" --intent explore --budget auto --mode basic --usage-policy off`
+- `gcie context-slices . "<task>" --intent edit --profile recall`
+- `gcie context-slices . "<task>" --intent edit --profile low --pin frontend/src/App.jsx --pin-budget 300`
+
+### Usage Policy
+
+- `hybrid` is the default. It keeps the existing balance between recall and token cost.
+- `force` always takes the richer GCIE retrieval path, even for simpler prompts.
+- `minimal` or `off` keeps retrieval tiny when you already know the target files or only need a quick probe.
+
+### Adaptation and Profile State
+
+- `gcie adapt . --benchmark-size 25 --efficiency-iterations 8 --adapt-workers 6`
+- `gcie adapt . --benchmark-size 25 --efficiency-iterations 8 --adapt-workers 6 --clear-profile`
+- `gcie adaptive-profile .`
+- `gcie adaptive-profile . --clear`
+- adaptation evaluates policy-aware candidates (`plain_minimal`, `plain`, `plain_force`) plus chain/gapfill/rescue/slices and picks per-family under an accuracy gate
+
+### Utility Commands
+
+- `gcie query <path> "<question>"`
+- `gcie debug <path> "<question>"`
+- `gcie cache-status .`
+- `gcie cache-warm .`
+- `gcie cache-clear .`
+
+## Recommended Workflow
+
+### 1) Bootstrap once per repo
+
+```powershell
+gcie setup . --adapt --adapt-benchmark-size 25 --adapt-efficiency-iterations 8 --adapt-workers 6
+```
+
+### 2) Day-to-day retrieval
+
+```powershell
+gcie context . "<task>" --intent edit --budget auto --mode adaptive --usage-policy hybrid
+```
+
+For cross-layer flows, use file-first symbol-rich queries and optionally pin budget:
+
+```powershell
+gcie context . "frontend/src/App.jsx selectedTheme /api/convert/start app.py start_convert" --intent edit --budget 1200 --mode adaptive --usage-policy force
+```
+
+### 3) Verify before edits on critical changes
+
+```powershell
+rg -n "<symbol1>|<symbol2>|<symbol3>" .
+```
+
+### 4) Re-adapt only when needed
+
+Use adaptation again after large refactors, architecture shifts, or repeated recall misses:
+
+```powershell
+gcie adapt . --benchmark-size 25 --efficiency-iterations 8 --adapt-workers 6
+```
+
+If adaptation quality drifts due stale profile state, reset first:
+
+```powershell
+gcie adaptive-profile . --clear
+gcie adapt . --benchmark-size 25 --efficiency-iterations 8 --adapt-workers 6 --clear-profile
+```
+
+## Notes
+
+- `requested_benchmark_size` can be higher than `benchmark_size` used when fewer unique repo-local benchmark cases are available.
+- `status: accuracy_locked_but_cost_risky` can appear when the selected 100%-accuracy policy is compared against a cheaper but lower-accuracy baseline.
+- Primary success criteria remain must-have coverage and pass rate; optimize cost after lock.

package/cli/app.py

@@ -1,9 +1,10 @@
-"""Typer entrypoint for GCIE CLI."""
+"""Typer entrypoint for GCIE CLI."""
 
 from __future__ import annotations
 
 import json
-import re
+import re
+from typing import Literal
 
 import typer
 
@@ -16,7 +17,8 @@ from .commands.index import run_index
 from .commands.query import run_query
 from .commands.setup import run_remove, run_setup
 
-app = typer.Typer(help="GraphCode Intelligence Engine CLI")
+app = typer.Typer(help="GraphCode Intelligence Engine CLI")
+UsagePolicy = Literal["hybrid", "force", "minimal", "off"]
 
 
 def _query_tokens(query: str) -> tuple[str, ...]:
@@ -60,6 +62,20 @@ def _auto_context_budget(query: str, intent: str | None) -> int | None:
     return None
 
 
+def _resolve_context_usage(
+    *,
+    mode: str,
+    usage_policy: UsagePolicy,
+    budget: int | None,
+) -> tuple[str, int | None, bool]:
+    """Map a high-level usage policy to the existing context command parameters."""
+    if usage_policy in {"minimal", "off"}:
+        return "basic", 0, False
+    if usage_policy == "force":
+        return "adaptive", budget, True
+    return mode, budget, False
+
+
 @app.command("index")
 def index_cmd(path: str = typer.Argument(".")) -> None:
     result = run_index(path)
@@ -85,18 +101,36 @@ def context_cmd(
     budget: str = typer.Option("auto", "--budget"),
     intent: str | None = typer.Option(None, "--intent"),
     mode: str = typer.Option("basic", "--mode", help="context mode: basic or adaptive"),
+    usage_policy: UsagePolicy = typer.Option(
+        "hybrid",
+        "--usage-policy",
+        help="GCIE usage policy: hybrid, force, minimal, or off",
+    ),
 ) -> None:
     if budget == "auto":
         budget_val = _auto_context_budget(query, intent)
     else:
         budget_val = int(budget)
 
-    if mode == "basic":
-        result = run_context_basic(path, query, budget=budget_val, intent=intent)
-    elif mode == "adaptive":
-        result = run_context(path, query, budget=budget_val, intent=intent)
-    else:
+    if mode not in {"basic", "adaptive"}:
         raise typer.BadParameter("--mode must be 'basic' or 'adaptive'")
+
+    effective_mode, effective_budget, strict_accuracy = _resolve_context_usage(
+        mode=mode,
+        usage_policy=usage_policy,
+        budget=budget_val,
+    )
+
+    if effective_mode == "basic":
+        result = run_context_basic(path, query, budget=effective_budget, intent=intent)
+    else:
+        result = run_context(
+            path,
+            query,
+            budget=effective_budget,
+            intent=intent,
+            strict_accuracy=strict_accuracy,
+        )
     typer.echo(json.dumps(result, indent=2))
 
 
@@ -217,5 +251,3 @@ def cache_warm_cmd(path: str = typer.Argument(".")) -> None:
 if __name__ == "__main__":
     app()
 
-
-

package/cli/commands/adaptation.py

@@ -1,4 +1,4 @@
-"""Post-initialization adaptation pipeline (accuracy rounds first, then efficiency rounds)."""
+"""Post-initialization adaptation pipeline (accuracy rounds first, then efficiency rounds)."""
 
 from __future__ import annotations
 
@@ -10,7 +10,7 @@ import os
 import re
 from pathlib import Path
 
-from .context import run_context
+from .context import run_context, run_context_basic
 from .context_slices import _classify_query_family, run_context_slices
 from .index import run_index
 
@@ -54,7 +54,7 @@ _IGNORED_DIRS = {
     "build",
     "coverage",
 }
-_METHOD_ORDER = ["plain", "plain_chain", "plain_gapfill", "plain_rescue", "slices"]
+_METHOD_ORDER = ["plain_minimal", "plain", "plain_force", "plain_chain", "plain_gapfill", "plain_rescue", "slices"]
 
 
 def _adapt_worker_count(workers: int | None = None) -> int:
@@ -385,7 +385,57 @@ def _evaluate_plain_case(case, *, allow_gapfill: bool = True, aggressive_gapfill
         missing_expected=tuple(missing),
         context_complete=not missing,
     )
-
+
+def _evaluate_plain_minimal_case(case) -> CaseResult:
+    path, query, budget = _plan_query(case)
+    path = _safe_scope(path)
+    payload = run_context_basic(path, query, budget=budget, intent=case.intent)
+    files = {
+        _normalize_scoped_path(path, rel)
+        for rel in (_node_to_file(item.get("node_id", "")) for item in payload.get("snippets", []))
+        if rel
+    }
+    expected = tuple(case.expected_files)
+    missing = [rel for rel in expected if rel not in files]
+    tokens = int(payload.get("tokens", 0) or 0)
+    expected_hits = len(expected) - len(missing)
+    family = _classify_query_family(query)
+    return CaseResult(
+        name=case.name,
+        family=family,
+        mode="plain_context_workflow_minimal",
+        tokens=tokens,
+        expected_hits=expected_hits,
+        expected_total=len(expected),
+        missing_expected=tuple(missing),
+        context_complete=not missing,
+    )
+
+
+def _evaluate_plain_force_case(case) -> CaseResult:
+    path, query, budget = _plan_query(case)
+    path = _safe_scope(path)
+    payload = run_context(path, query, budget=budget, intent=case.intent, strict_accuracy=True)
+    files = {
+        _normalize_scoped_path(path, rel)
+        for rel in (_node_to_file(item.get("node_id", "")) for item in payload.get("snippets", []))
+        if rel
+    }
+    expected = tuple(case.expected_files)
+    missing = [rel for rel in expected if rel not in files]
+    tokens = int(payload.get("tokens", 0) or 0)
+    expected_hits = len(expected) - len(missing)
+    family = _classify_query_family(query)
+    return CaseResult(
+        name=case.name,
+        family=family,
+        mode="plain_context_workflow_force",
+        tokens=tokens,
+        expected_hits=expected_hits,
+        expected_total=len(expected),
+        missing_expected=tuple(missing),
+        context_complete=not missing,
+    )
 
 def _evaluate_slices_case(case) -> CaseResult:
     payload = run_context_slices(
@@ -457,15 +507,19 @@ def _evaluate_slices_case(case) -> CaseResult:
     )
 
 
-def _evaluate_case_with_method(case, method: str) -> CaseResult:
-    if method == "plain":
-        return _evaluate_plain_case(case, allow_gapfill=False)
-    if method == "plain_chain":
-        return _evaluate_plain_chain_case(case)
-    if method == "plain_gapfill":
-        return _evaluate_plain_case(case, allow_gapfill=True, aggressive_gapfill=False)
-    if method == "plain_rescue":
-        return _evaluate_plain_case(case, allow_gapfill=True, aggressive_gapfill=True)
+def _evaluate_case_with_method(case, method: str) -> CaseResult:
+    if method == "plain_minimal":
+        return _evaluate_plain_minimal_case(case)
+    if method == "plain":
+        return _evaluate_plain_case(case, allow_gapfill=False)
+    if method == "plain_force":
+        return _evaluate_plain_force_case(case)
+    if method == "plain_chain":
+        return _evaluate_plain_chain_case(case)
+    if method == "plain_gapfill":
+        return _evaluate_plain_case(case, allow_gapfill=True, aggressive_gapfill=False)
+    if method == "plain_rescue":
+        return _evaluate_plain_case(case, allow_gapfill=True, aggressive_gapfill=True)
     return _evaluate_slices_case(case)
 
 
@@ -890,14 +944,18 @@ def run_post_init_adaptation(
 
     # Global candidate snapshots for transparency.
     slices_rows = _evaluate_cases_with_method(cases, 'slices', workers)
+    plain_min_rows = _evaluate_cases_with_method(cases, 'plain_minimal', workers)
     plain_rows = _evaluate_cases_with_method(cases, 'plain', workers)
+    plain_force_rows = _evaluate_cases_with_method(cases, 'plain_force', workers)
     plain_gap_rows = _evaluate_cases_with_method(cases, 'plain_gapfill', workers)
     plain_rescue_rows = _evaluate_cases_with_method(cases, 'plain_rescue', workers)
     slices_summary = _summarize('slices_accuracy_stage', slices_rows)
+    plain_min_summary = _summarize('plain_minimal_accuracy_stage', plain_min_rows)
     plain_summary = _summarize('plain_accuracy_stage', plain_rows)
+    plain_force_summary = _summarize('plain_force_accuracy_stage', plain_force_rows)
     plain_gap_summary = _summarize('plain_gapfill_accuracy_stage', plain_gap_rows)
     plain_rescue_summary = _summarize('plain_rescue_accuracy_stage', plain_rescue_rows)
-    candidates = [slices_summary, plain_summary, plain_gap_summary, plain_rescue_summary]
+    candidates = [slices_summary, plain_min_summary, plain_summary, plain_force_summary, plain_gap_summary, plain_rescue_summary]
 
     active = {
         'label': 'family_policy_selected',