@pmaddire/gcie 0.1.2

package/DEBUGGING_PLAYBOOK.md

@@ -0,0 +1,160 @@
+# DEBUGGING_PLAYBOOK.md
+
+GraphCode Intelligence Engine Debugging Playbook
+
+This document describes how agents should debug problems using the GCIE graph system.
+
+Agents must follow this structured debugging workflow instead of guessing.
+
+---
+
+DEBUGGING PRINCIPLES
+
+Always prioritize structural analysis before semantic guessing.
+
+Use graph queries whenever possible.
+
+Only use LLM reasoning after symbolic analysis.
+
+---
+
+DEBUGGING WORKFLOW
+
+When debugging a problem, follow this procedure.
+
+Step 1 — Identify Target Symbols
+
+Extract relevant symbols from the query.
+
+Examples:
+
+variable names
+function names
+file names
+modules
+
+Example query:
+
+"Why is variable diff exploding?"
+
+Target symbol:
+
+diff
+
+---
+
+Step 2 — Query Knowledge Index
+
+Use the knowledge index to find:
+
+functions that modify the variable
+functions that read the variable
+
+Example query:
+
+find functions writing variable diff
+
+---
+
+Step 3 — Trace Variable Dependencies
+
+Use the variable dependency graph.
+
+Identify:
+
+where the variable is modified
+which functions depend on it
+
+---
+
+Step 4 — Trace Call Graph
+
+From functions modifying the variable:
+
+trace upstream callers
+
+trace downstream calls
+
+This reveals execution paths that influence the variable.
+
+---
+
+Step 5 — Analyze Execution Paths
+
+Use the execution trace graph when available.
+
+Identify the runtime path leading to the issue.
+
+---
+
+Step 6 — Prioritize Suspicious Code
+
+Rank candidate functions using:
+
+recent git commits
+
+low test coverage
+
+large code complexity
+
+---
+
+Step 7 — Build Minimal Debugging Context
+
+Return only the relevant functions:
+
+the function modifying the variable
+
+its callers
+
+any functions it calls
+
+---
+
+EXAMPLE DEBUGGING FLOW
+
+Query:
+
+"Why is diff exploding?"
+
+Process:
+
+extract symbol → diff
+
+query knowledge index → functions modifying diff
+
+trace variable graph → compute_diff()
+
+trace call graph → update_state() → run_simulation()
+
+retrieve minimal code context
+
+---
+
+DEBUGGING HEURISTICS
+
+Prefer code that:
+
+modifies the target variable
+
+recently changed in git
+
+has low test coverage
+
+appears frequently in execution traces
+
+---
+
+DEBUGGING OUTPUT FORMAT
+
+When returning debugging results include:
+
+relevant functions
+
+call chain
+
+variable modifications
+
+---
+
+END DEBUGGING PLAYBOOK

package/KNOWLEDGE_INDEX.md

@@ -0,0 +1,154 @@
+# KNOWLEDGE_INDEX.md
+
+Codebase Knowledge Index Specification
+
+---
+
+PURPOSE
+
+The Knowledge Index is a structured metadata representation of the repository.
+
+It enables fast code queries without requiring an LLM.
+
+The index works alongside the graph system.
+
+---
+
+INDEX CONTENT
+
+The index stores structured metadata for:
+
+files
+classes
+functions
+variables
+imports
+dependencies
+
+---
+
+FUNCTION ENTRY FORMAT
+
+Each function entry should contain:
+
+name
+file
+start_line
+end_line
+parameters
+variables_read
+variables_written
+functions_called
+docstring
+
+Example:
+
+FunctionEntry
+
+name: compute_diff
+file: slam/update.py
+start_line: 42
+end_line: 68
+parameters: state, prediction
+variables_read: state
+variables_written: diff
+functions_called: normalize, clip
+
+---
+
+CLASS ENTRY FORMAT
+
+Each class entry should contain:
+
+class name
+file
+methods
+attributes
+base classes
+
+---
+
+FILE ENTRY FORMAT
+
+Each file entry should contain:
+
+file path
+imports
+classes defined
+functions defined
+
+---
+
+INDEX STORAGE
+
+Initial implementation should store the index in memory.
+
+Later versions may persist the index using:
+
+JSON
+SQLite
+or graph database.
+
+---
+
+QUERY TYPES
+
+The Knowledge Index must support queries such as:
+
+find functions modifying variable
+
+find functions calling function
+
+find files importing module
+
+find classes inheriting from class
+
+---
+
+INDEX USAGE IN RETRIEVAL PIPELINE
+
+Query
+↓
+Extract symbol
+↓
+Search knowledge index
+↓
+Retrieve candidate nodes
+↓
+Graph traversal
+↓
+Semantic ranking
+
+---
+
+ADVANTAGES
+
+The Knowledge Index allows many queries to be answered without LLM usage.
+
+Examples:
+
+Where is variable diff modified?
+
+Which functions call compute_diff?
+
+Which modules depend on slam.update?
+
+These queries can be answered directly using the index.
+
+---
+
+FUTURE EXTENSIONS
+
+Possible improvements:
+
+cross-language support
+
+dependency metrics
+
+code complexity scoring
+
+architecture summaries
+
+---
+
+END KNOWLEDGE INDEX SPEC

package/POTENTIAL_UPDATES

@@ -0,0 +1,130 @@
+# Potential Updates
+
+## Context
+
+Current state (from latest validation):
+
+- Hybrid workflow can reach very high recall when staged.
+- One-shot/root retrieval is still brittle in mixed-layer repos.
+- The best near-term target is improving efficiency without giving up coverage.
+
+## Candidate Updates
+
+### 1. Pivot + Skeleton Packaging
+
+What it is:
+- Keep full content for pivot files (must-have files).
+- Include only compact skeletons/signatures for adjacent support files.
+
+Why it helps:
+- Preserves correctness path while reducing token load from neighboring files.
+- Avoids paying full-file cost for files used only as context bridges.
+
+Risk:
+- Low to medium. Requires careful control so skeleton files are never used where full content is required.
+
+Expected impact:
+- High efficiency gain with low recall risk.
+
+---
+
+### 2. Path-Local First Expansion (Root Queries)
+
+What it is:
+- If query names explicit files, rank same-subtree candidates first.
+- Expand globally only when must-have coverage is still incomplete.
+
+Why it helps:
+- Reduces root-scope noise from generic/support files.
+- Aligns ranking to the strongest user-provided signal (explicit paths).
+
+Risk:
+- Medium. Needs guardrails for true cross-layer tasks.
+
+Expected impact:
+- High recall gain on root queries; moderate efficiency gain.
+
+---
+
+### 3. Dynamic Threshold Relaxation
+
+What it is:
+- Start retrieval strict.
+- Relax thresholds only if hit count/coverage is too low.
+
+Why it helps:
+- Keeps compact context on easy tasks.
+- Avoids over-expanding by default.
+
+Risk:
+- Medium. Can oscillate if thresholds are not stable.
+
+Expected impact:
+- Moderate efficiency gain; moderate stability gain.
+
+---
+
+### 4. Intent Pipeline Preset (Single Orchestration Mode)
+
+What it is:
+- One orchestrated mode that chooses strategy by intent and task shape.
+
+Why it helps:
+- Better UX and fewer manual tuning steps.
+
+Risk:
+- Medium to high. More moving parts and easier to regress.
+
+Expected impact:
+- High UX gain; uncertain near-term accuracy/efficiency gain.
+
+---
+
+### 5. Memory Budget Cap
+
+What it is:
+- Hard cap on memory/history context slice (for example 10%).
+
+Why it helps:
+- Prevents historical context from displacing code context.
+
+Risk:
+- Low.
+
+Expected impact:
+- Small to moderate efficiency gain.
+
+## Recommendation: Add Now
+
+### Add #1 first: Pivot + Skeleton Packaging
+
+This is the highest confidence next step right now.
+
+Why this makes the most sense immediately:
+
+1. It directly targets the current optimization goal: better efficiency without recall loss.
+2. It is incremental and low-risk compared with a broader orchestration rewrite.
+3. It complements the current hybrid workflow (which already identifies must-have pivots).
+4. It gives measurable gains quickly with a clear A/B test:
+   - same must-have coverage
+   - lower average tokens
+
+## Suggested Implementation Order
+
+1. Add file role marker in context packaging:
+   - `pivot` vs `adjacent_support`.
+2. For `adjacent_support`, package signatures/symbol blocks first.
+3. Keep full content for:
+   - explicit targets
+n   - chain middle files
+   - files selected by must-have gate.
+4. Run benchmark comparison:
+   - hybrid current vs hybrid + skeleton packaging
+   - compare average tokens, savings, accuracy, full-hit rate.
+
+## Success Criteria
+
+- No drop in full-hit rate.
+- No drop in average accuracy.
+- Average tokens reduced versus current hybrid workflow baseline.
+

package/PROJECT.md

@@ -0,0 +1,141 @@
+# PROJECT.md
+
+## Project Name
+
+GraphCode Intelligence Engine (GCIE)
+
+## System Purpose
+
+GCIE is a graph-first code intelligence engine that minimizes LLM context size for large repositories.  
+It answers developer queries by retrieving only execution-relevant symbols and code snippets instead of full files.  
+Primary success target: produce minimal, high-signal debugging context (for example, tracing why `diff` explodes).
+
+## Architecture Overview
+
+GCIE follows a staged pipeline:
+
+1. Repository scanning discovers source files, tests, and metadata.
+2. Parsing extracts symbols and relationships from source code.
+3. Graph builders construct specialized graphs.
+4. Knowledge index stores normalized symbol metadata for fast lookup.
+5. Retrieval engine performs symbolic traversal first, then semantic ranking.
+6. Context builder packages a minimal, ordered prompt payload.
+7. CLI exposes indexing, querying, debugging, and diagnostics workflows.
+
+## Core Subsystems
+
+1. `scanner/`
+Repository scanning, language filtering, include/exclude rules.
+
+2. `parser/`
+AST and Tree-sitter parsing, symbol extraction, normalized intermediate representation.
+
+3. `graphs/`
+Specialized graph construction and unified graph merge using NetworkX.
+
+4. `knowledge_index/`
+In-memory index for files/classes/functions/variables/imports/dependencies with query APIs.
+
+5. `retrieval/`
+Symbolic retriever, semantic retriever, hybrid ranking and candidate consolidation.
+
+6. `embeddings/`
+SentenceTransformers embedding generation and FAISS vector indexing.
+
+7. `debugging/`
+Bug localization and execution-path analysis workflows.
+
+8. `llm_context/`
+Minimal snippet extraction, ordering, deduplication, and context packaging.
+
+9. `cli/`
+Typer-based commands for index/build/query/debug/report operations.
+
+10. `tests/`
+Unit, integration, coverage, and retrieval quality validation.
+
+## Graph Models
+
+GCIE maintains these graph models and composes them into a unified knowledge graph:
+
+1. Code Structure Graph
+Nodes: files, modules, classes, functions.  
+Edges: `DEFINES`, `CONTAINS`, `IMPORTS`.
+
+2. Call Graph
+Nodes: callable symbols.  
+Edges: `CALLS`.
+
+3. Variable Dependency Graph
+Nodes: variables, functions, assignments.  
+Edges: `READS`, `WRITES`, `MODIFIES`.
+
+4. Execution Trace Graph
+Nodes: runtime function frames/events.  
+Edges: `EXECUTES`, `RETURNS`, temporal path edges.
+
+5. Git History Graph
+Nodes: commits, files, symbols.  
+Edges: `CHANGED_IN`, `TOUCHES`.
+
+6. Test Coverage Graph
+Nodes: tests, functions, files.  
+Edges: `COVERED_BY`, `ASSERTS_ON`.
+
+7. Unified Knowledge Graph
+Merged, queryable graph used by retrieval and bug localization.
+
+## Retrieval Pipeline
+
+1. Query ingestion
+Normalize query text and identify query intent (`debug`, `trace`, `dependency`, `explain`).
+
+2. Symbol extraction
+Extract candidate symbols (variable/function/class/file/module names).
+
+3. Knowledge index lookup
+Resolve exact/fuzzy symbol matches and seed candidate nodes.
+
+4. Symbolic graph traversal
+Traverse variable/call/structure/trace graphs for execution-relevant neighborhood.
+
+5. Semantic retrieval
+Use embeddings + FAISS to rank semantically similar snippets among symbolic candidates.
+
+6. Hybrid ranking
+Combine symbolic distance, semantic score, git recency, and test coverage risk weighting.
+
+7. Minimal context assembly
+Return only necessary snippets: symbol definition, writes/modifies points, callers/callees, and trace path.
+
+8. Output formatting
+Provide structured debugging payload: relevant functions, call chain, variable modifications, evidence scores.
+
+## GSD Workflow Contract
+
+1. Plan before implementation.
+2. Execute in phases with atomic tasks.
+3. Verify each phase before advancing.
+4. Keep artifacts updated (`PROJECT.md`, `ROADMAP.md`, and phase plans).
+5. Do not implement large features in a single step.
+
+## Initial Constraints
+
+1. Language/runtime: Python 3.11+.
+2. Parsers: `ast` first, Tree-sitter extension path.
+3. Graph engine: NetworkX.
+4. Semantic layer: SentenceTransformers + FAISS.
+5. Git analysis: GitPython.
+6. Execution tracing: `sys.settrace`.
+7. Coverage: Coverage.py.
+8. CLI: Typer.
+9. Retrieval principle: symbolic first, semantic second.
+
+## Success Criteria
+
+1. Repository indexing works across project files.
+2. Core graph suite is built and queryable.
+3. Symbolic + semantic hybrid retrieval returns ranked candidates.
+4. Debug query returns compact execution-relevant context only.
+5. End-to-end token usage is materially reduced versus full-file prompting.
+