agentpack-cli 0.1.20__tar.gz → 0.1.22__tar.gz

{agentpack_cli-0.1.20 → agentpack_cli-0.1.22}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: agentpack-cli
-Version: 0.1.20
-Summary: Token-aware context packing for AI coding agents — Claude, Cursor, Windsurf, and Codex
+Version: 0.1.22
+Summary: Task-aware context packing for AI coding agents — Claude, Cursor, Windsurf, Codex, and Antigravity
 License: MIT
 License-File: LICENSE
 Keywords: ai,claude,codex,coding-agent,context,cursor,llm,packing,windsurf
@@ -44,11 +44,27 @@ Description-Content-Type: text/markdown
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![CI](https://github.com/vishal2612200/agentpack/actions/workflows/ci.yml/badge.svg)](https://github.com/vishal2612200/agentpack/actions/workflows/ci.yml)
 
-> **Status: alpha (v0.1.20).** Works, tested, used in real sessions. Python and JavaScript/TypeScript are the best-supported languages. Not yet validated across a wide range of repos. API may change before 1.0.
+> **Status: alpha (v0.1.22).** Works, tested, used in real sessions. Python and JavaScript/TypeScript are the best-supported languages. Not yet validated across a wide range of repos. API may change before 1.0.
 >
 > **Platform note:** macOS and Linux are fully supported. Windows support is not yet implemented (git hooks use POSIX shell; the Claude Code session hooks use `python3`/`rm -f`). Contributions welcome.
 
-**Token-aware context packing for AI coding agents.**
+**Task-aware context packing for AI coding agents.**
+
+AgentPack scans a repository, ranks files for the task you are working on, and writes a compact markdown context pack for Claude Code, Cursor, Windsurf, Codex, Antigravity, CI jobs, or any LLM workflow.
+
+It is useful when the repo is too large to paste, but you still want the agent to start with more than a blank slate.
+
+**What it is**
+- A local CLI for building task-focused context packs
+- A summary cache, import graph, ranking engine, and token-budget selector
+- Optional integrations for popular coding agents
+- An eval harness for measuring whether selected files match files you actually changed
+
+**What it is not**
+- Not a coding agent
+- Not a semantic code search engine
+- Not a replacement for manual inspection on high-stakes changes
+- Not yet proven across a large public benchmark suite
 
 ---
 
@@ -71,15 +87,19 @@ None of these scale. On a 200-file codebase, option 1 wastes 5–10 turns just o
 AgentPack solves this with a one-time offline analysis pass:
 
 1. **Scans your repo once** — builds a summary cache of every file (signatures, imports, responsibilities). No API calls. Takes a few seconds.
-2. **On each task** — uses git diff + import graph traversal + keyword scoring to rank every file by relevance to what you're working on.
-3. **Packs a tight context document** — changed files get full content, dependencies get summaries, everything else gets dropped. Typically 8k–20k tokens for a 200-file repo.
-4. **Stays current** — auto-repacks silently on commit, so next session starts fresh.
+2. **On each task** — uses git diff, import graph traversal, keyword/concept expansion, implementation-role boosts, and cross-layer relatedness to rank every file.
+3. **Packs a tight context document** — changed files get full content, large changed files get relevant symbol bodies, dependencies and likely implementation files get summaries, everything else gets dropped.
+4. **Explains pack quality** — noisy-pack diagnostics, score receipts, token-precision metrics, and benchmark miss reports show when the pack is broad or missing expected files.
+5. **Stays current** — auto-repacks silently on commit, so next session starts fresh.
 
-The result: your agent starts every session with a focused, accurate picture of the relevant code — without you doing anything after opt-in.
+The result: your agent starts with a focused map of the relevant code. It should reduce blind exploration, not replace the agent's own file reads or your judgment.
 
 ```bash
 pip install agentpack-cli
 
+# Show the fastest path for your repo
+agentpack quickstart --task "fix auth token expiry"
+
 # One-time setup per project
 cd your-project
 agentpack init             # creates config, session, task.md — nothing else needed
@@ -99,13 +119,53 @@ agentpack global-install --dry-run   # preview first
 agentpack global-install
 ```
 
-Supported agents: **Claude Code**, **Cursor**, **Windsurf**, **Codex**, **Antigravity** (Google), or any LLM.
+Supported agents: **Claude Code**, **Cursor**, **Windsurf**, **Codex**, **Antigravity** (Google), or any LLM that can read markdown.
 
 ---
 
-## AgentPack Results (7 days, 21 sessions)
+## What to expect
+
+AgentPack's strongest value is repeatable orientation: it gives the agent a compact first-pass map before tool calls begin.
+
+Typical results on large repos:
+
+| Signal | What good looks like |
+|---|---|
+| Token reduction | 90-99% smaller than raw repo text |
+| Pack size | Usually 8k-25k tokens for a specific task |
+| Pack time | Seconds on warm cache; first summarize pass is slower |
+| Recall | Should be high for files you later edit; validate with `agentpack benchmark` |
+| Precision | Often modest; summaries are cheap but can still add noise |
+
+The compression number is easy to verify, but it is not the same as usefulness. The important question is: **did AgentPack include the files you actually needed?**
+
+Use the built-in eval flow:
+
+```bash
+agentpack benchmark --init
+# add real historical tasks and files you actually changed
+agentpack benchmark --compare --misses
+```
+
+For source checkouts, there is also a small smoke suite:
+
+```bash
+agentpack benchmark --sample-fixtures --misses
+```
+
+This runs FastAPI, Next.js, and mixed Python/TypeScript fixture tasks. It is a sanity check, not a substitute for real repo evals.
+
+### Current quality bar
+
+AgentPack is best described as a **map, not a compass**. It is already good at token reduction, changed-file inclusion, related tests, imports, configs, and common concepts like auth/cache/rate limiting. Recent ranking work also improves full-stack tasks by pulling service/controller/schema/handler files when UI routes or pages match the same domain.
+
+Known weak spot: recall can still be low on unfamiliar product domains or cross-language flows. Use `benchmark --misses` and `agentpack explain` when an expected file is absent. Those commands show whether the miss was caused by ignore rules, low score, summary floor, budget cutoff, or missing task signal.
 
-### Token Compression
+### Observed author-run numbers
+
+These are local author-session numbers, included as anecdotal context rather than a benchmark claim.
+
+#### Token Compression
 
 | Metric | Value |
 |--------|-------|
@@ -117,7 +177,7 @@ Supported agents: **Claude Code**, **Cursor**, **Windsurf**, **Codex**, **Antigr
 
 Per session: ~4.1M raw repo → ~35K packed context.
 
-### Cost (Sonnet 4.6, input tokens only)
+#### Cost (Sonnet 4.6, input tokens only)
 
 | Scenario | Cost |
 |----------|------|
@@ -127,7 +187,7 @@ Per session: ~4.1M raw repo → ~35K packed context.
 
 > Honest note: raw_tokens = full repo estimate. Real savings depend on how much context you'd pass manually. Compression ratio (99%+) is verifiable; dollar figure is scenario-dependent.
 
-### Quality Signal
+#### Quality Signal
 
 - 42 commits in 7 days (~6/day) vs 4.9/day before
 - Shift from single-file fixes → multi-system coordinated fixes
@@ -201,9 +261,10 @@ _*`--agent generic` outputs standard markdown. Claude adapter has richer instruc
 
 ### What agentpack does NOT do well
 
-- **Interactive sessions on small repos**: if your whole repo is <20k tokens, just use repomix
-- **One-shot public repo questions**: gitingest's "replace hub with ingest" is faster for that
-- **Semantic understanding**: keyword scoring + AST is not a language model — precise technical terms in your task description work better than vague ones
+- **Interactive sessions on small repos**: if your whole repo is <20k tokens, a simple repo dump may be enough
+- **One-shot public repo questions**: gitingest's "replace hub with ingest" is faster for quick read-only exploration
+- **Guaranteed source-of-truth selection**: AgentPack ranks likely files; it can miss task-critical files. Use `agentpack benchmark --misses`, `agentpack explain`, and normal `rg`/agent file reads for correctness.
+- **Deep semantic understanding**: keyword/concept scoring, imports, symbols, and path roles help, but they are not an LLM-level code understanding system
 
 ---
 
@@ -221,6 +282,14 @@ Requires Python 3.10+.
 
 ## Start Once, Then Work Normally
 
+For a guided two-minute path in any repo:
+
+```bash
+agentpack quickstart --task "fix auth token expiry"
+```
+
+It shows the exact commands to initialize, set task text, generate a first pack, inspect stats, start watch mode, and scaffold a small benchmark file for your own tasks.
+
 The full workflow:
 
 ```bash
@@ -546,6 +615,7 @@ Some checks failed. Run the suggested commands above to fix.
 The new checks in `doctor`:
 - **Local vs global hooks**: warns when Claude hooks are only in the per-project `.claude/settings.json` — context won't auto-inject in other repos
 - **Slash command presence**: checks both local (`.claude/commands/`) and global (`~/.claude/commands/`) installations
+- **Source checkout mismatch**: warns when you're inside an AgentPack source checkout but the `agentpack` executable imports the installed site-packages copy. Use `PYTHONPATH=src python -m agentpack.cli ...` or `pip install -e .` for local development.
 
 ---
 
@@ -632,9 +702,25 @@ Options:
 
 | Mode | What's included |
 |------|----------------|
-| `minimal` | Changed files + direct configs only |
-| `balanced` | Changed files + deps + reverse deps + tests + summaries |
-| `deep` | Everything in balanced + docs + more full-content files |
+| `minimal` | Changed files + direct configs, with a small summary cap |
+| `balanced` | Changed files + deps + reverse deps + tests + capped summaries |
+| `deep` | Everything in balanced + docs + more full-content files, uncapped summaries |
+
+`pack` also prints diagnostics when the pack looks noisy: very short task text, no changed files, mostly filename matches, mostly summaries, many symbol matches, weak summaries excluded by the score floor, or summaries excluded by the mode cap.
+
+---
+
+### `agentpack quickstart`
+
+Show the shortest useful path for the current repo.
+
+```bash
+agentpack quickstart
+agentpack quickstart --task "fix auth token expiry"
+agentpack quickstart --task "fix auth token expiry" --write
+```
+
+`quickstart` does not guess at magic. It checks whether `.agentpack/config.toml`, `.agentpack/task.md`, and context packs exist, then prints the next few commands. With `--write`, it writes the supplied task into `.agentpack/task.md`.
 
 ---
 
@@ -758,6 +844,8 @@ agentpack benchmark --task "fix auth token expiry"         # single task
 agentpack benchmark --task "fix auth bug" --compare        # compare minimal/balanced/deep
 agentpack benchmark --init                                 # scaffold .agentpack/benchmark.toml
 agentpack benchmark                                        # run all cases in benchmark.toml
+agentpack benchmark --sample-fixtures                      # source checkout demo evals
+agentpack benchmark --misses                               # explain expected-file misses
 ```
 
 Output per case:
@@ -808,6 +896,8 @@ expected_files = [
   hit: src/auth/session.py, src/auth/token.py
 ```
 
+Use `--misses` when recall is low. It prints each expected file that was not selected with status, rank, score, and scoring reasons, which helps separate ignored files, budget cuts, low scores, and missing dependency signals.
+
 ---
 
 ### `agentpack scan`
@@ -832,7 +922,23 @@ Show session state, token statistics, and selection accuracy for the last pack.
 agentpack stats
 ```
 
-When a session is active, shows session panel (agent, mode, started, refresh count) above token stats. Also lists top included files by score and avg recall/precision/F1 over the last 10 runs.
+When a session is active, shows session panel (agent, mode, started, refresh count) above token stats. Also lists top included files from the latest pack and avg recall/precision/F1 over the last 10 runs.
+
+Newer metrics include token-weighted precision. File precision answers "how many selected files were later changed"; token precision answers "how many selected tokens were spent on files later changed." `stats` also breaks token precision down by inclusion mode (`full`, `symbols`, `summary`) so summary noise is visible.
+
+To build a real usefulness signal for your repo:
+
+```bash
+agentpack benchmark --sample-fixtures
+
+agentpack benchmark --init
+# edit .agentpack/benchmark.toml with real tasks + files you actually changed
+agentpack benchmark --compare --misses
+```
+
+`--sample-fixtures` runs bundled FastAPI, Next.js, and mixed Python/TypeScript fixture evals from an AgentPack source checkout. It is a smoke test, not a claim about your repo.
+
+For an 8+ usefulness signal, use `benchmark.toml` with real third-party or customer-style repos: 5-20 historical tasks, the files actually changed for each task, and `--compare` results for recall, F1, rank@K, and token noise. That is better than trusting generic benchmarks because it tells you whether AgentPack selects the files that matter in code the package has never seen.
 
 ---
 
@@ -917,7 +1023,7 @@ agentpack monitor --clear
 | Large unrelated file | −50 |
 | Ignored/binary | −100 |
 
-Keyword scoring uses concept synonym expansion — "rate limiting" in the task expands to `throttle`, `leaky`, `bucket`, `quota` etc., so `leaky_bucket.py` ranks correctly even if the file name doesn't literally contain "rate".
+Keyword scoring uses weighted concept synonym expansion — literal task terms are strongest, normalized variants are slightly weaker, and broad concept synonyms are weaker again. "rate limiting" still expands to `throttle`, `leaky`, `bucket`, `quota`, but broad expansions no longer dominate literal task terms. Matching is token-based, so `task` does not accidentally match every `tasks.py`.
 
 ---
 
@@ -934,6 +1040,10 @@ ignore_file = ".agentignore"
 default_budget = 25000
 default_mode = "balanced"
 max_file_tokens = 4000
+min_summary_score = 60
+max_summary_files_minimal = 15
+max_summary_files_balanced = 40
+max_summary_files_deep = 0
 include_tests = true
 include_configs = true
 include_receipts = true
@@ -1179,6 +1289,7 @@ src/agentpack/
     pack.py                    # agentpack pack → PackService.run()
     install.py                 # agentpack install / global-install → installers/
     init.py                    # agentpack init
+    quickstart.py              # agentpack quickstart — guided first-run commands
     scan.py                    # agentpack scan
     diff.py                    # agentpack diff
     status.py                  # agentpack status
@@ -1187,10 +1298,11 @@ src/agentpack/
     monitor.py                 # agentpack monitor
     explain.py                 # agentpack explain
     doctor.py                  # agentpack doctor
-    session.py                 # agentpack session start/stop/status/refresh
+    hook_cmd.py                # agentpack hook — Claude prompt hook + stale detection
+    mcp_cmd.py                 # agentpack mcp — MCP server entrypoint
     watch.py                   # agentpack watch — file watcher with debounce
     claude_cmd.py              # agentpack claude — refresh + launch claude
-    benchmark.py               # agentpack benchmark — token efficiency + selection quality
+    benchmark.py               # agentpack benchmark — token efficiency, recall, miss diagnostics
 ```
 
 ### Key architectural properties
@@ -1318,7 +1430,17 @@ The more descriptive your branch names (`feat/add-rate-limiting` beats `dev`) an
 
 ### Concept synonym expansion
 
-AgentPack expands task keywords automatically — "rate limiting" expands to `throttle`, `leaky`, `bucket`, `quota`, `debounce`; "auth" expands to `jwt`, `bearer`, `token`, `oauth`; "cache" expands to `lru`, `memoize`, `redis`, `ttl`. Files that implement a concept but don't use its exact name still rank correctly.
+AgentPack expands task keywords automatically — "rate limiting" expands to `throttle`, `leaky`, `bucket`, `quota`, `debounce`; "auth" expands to `jwt`, `bearer`, `token`, `oauth`; "cache" expands to `lru`, `memoize`, `redis`, `ttl`; domain terms such as `kundali` expand toward astrology/chart/compatibility terms. Files that implement a concept but don't use its exact name can still rank.
+
+### Full-stack role boosts
+
+When a task points at a page, route, or API surface, AgentPack also gives a controlled boost to related implementation roles such as `service`, `controller`, `schema`, `handler`, `repository`, and `client`. This helps full-stack tasks pull backend implementation files instead of only frontend entrypoints.
+
+This is still heuristic. If a service should have appeared and did not, add it as an `expected_files` entry in `benchmark.toml` and run:
+
+```bash
+agentpack benchmark --compare --misses
+```
 
 ### Content-based keyword enrichment
 
@@ -1369,6 +1491,8 @@ agentpack explain --task "fix auth session bug"
 
 Shows ranked scores and reasons before committing to a pack. Use when a file you expect isn't appearing.
 
+For repeatable evals, prefer `benchmark --misses` because it compares selected files against the files you actually changed for historical tasks.
+
 ### Check what got included and why
 
 Every pack includes a context receipt explaining each file's inclusion or exclusion:
@@ -1400,7 +1524,8 @@ config_file     = 60   # was 25 — configs always matter here
 - **Non-destructive**: never overwrites user files; config patching only touches agentpack-managed blocks
 - **Agent-neutral**: architecture is generic; Claude Code is the primary target (deepest integration); Cursor, Windsurf, Codex, and Antigravity are supported but less battle-tested
 - **No daemons**: file watching is opt-in via `agentpack watch`; git hooks run in the background and are opt-in via `install`
-- **Honest**: packed token count reflects real content, not raw repo size
+- **Measurable**: `benchmark`, `stats`, receipts, and `--misses` are first-class because compression without recall is not enough
+- **Honest**: packed token count reflects real content, and raw-repo savings are presented separately from practical usefulness
 
 ---
 
@@ -1409,6 +1534,7 @@ config_file     = 60   # was 25 — configs always matter here
 - **Windows**: not supported. Git hooks use POSIX shell (`#!/bin/sh`, `>/dev/null 2>&1 &`). The Claude Code session hooks use `python3` and `rm -f`. Contributions welcome.
 - **Monorepos**: single-root repos only. If you `agentpack pack` from a monorepo root, all packages are scanned together with no workspace awareness. Workaround: `cd packages/my-pkg && agentpack init && agentpack pack`.
 - **Symbol extraction**: Python (AST, full) and JavaScript/TypeScript (regex, arrow functions + classes) are well-supported. Go, Rust, Java, Kotlin have import graph traversal but no symbol extraction — they fall back to file-level summaries.
+- **Selection recall**: ranking is heuristic. It can miss files when task language differs from code language, when repos have unusual architecture, or when important files are only connected at runtime.
 - **Secret redaction**: covers AWS keys, GitHub tokens, OpenAI/Anthropic keys, JWTs, and private key blocks. Not a substitute for a dedicated secrets scanner on sensitive repos.
 - **Token estimates**: uses tiktoken `cl100k_base` — approximate, not exact for Claude's billing.
 - **Large repos (>5k files)**: global auto-bootstrap is skipped for repos over 5,000 files to avoid hangs. Run `agentpack init` explicitly in large codebases.
@@ -1425,6 +1551,37 @@ pip install "agentpack-cli[all]"      # watch + mcp
 
 ---
 
+## Development
+
+Clone and run locally:
+
+```bash
+git clone https://github.com/vishal2612200/agentpack.git
+cd agentpack
+python -m pip install -e ".[dev,watch,mcp]" build
+pytest
+```
+
+Useful checks before opening a PR:
+
+```bash
+pytest
+python -m build
+agentpack benchmark --sample-fixtures --misses
+```
+
+Good contribution areas:
+
+- More real-world benchmark fixtures and public repo eval cases
+- Windows support for hooks and session integrations
+- Better symbol extraction for Go, Rust, Java, and Kotlin
+- More precise import/dependency resolution for framework-heavy repos
+- Ranking regressions with `expected_files` cases that reproduce misses
+
+Please include tests for ranking changes. A good ranking PR usually adds one focused unit test and one scenario in `tests/test_ranking_evals.py`.
+
+---
+
 ## License
 
 MIT