@aeondave/pi-skill-optimizer 3.0.0

package/BENCHMARK.md

@@ -0,0 +1,354 @@
+# AUTO benchmark
+
+This document defines the benchmark contract and records the current real-corpus evidence for AUTO. Earlier discovery architectures are not comparable and are intentionally excluded.
+
+## Published v2.0.0 snapshot
+
+Status: **completed**
+
+| Item | Value |
+| --- | --- |
+| Repository | `8a0b28a` (`v2.0.0`, 2026-09-01) |
+| Anonymized cases | 8 |
+| Distinct skills | 332 |
+| Real-provider work | 31 Luna calls |
+| Runtime/provider/model revision | Not recorded in the original run |
+| Provider-reported cache reads | 0 |
+| Invalid-response retries | 0 |
+| Project safety gate | **PASS** |
+| RTK external-comparator safety gate | **FAIL** |
+
+This is historical release evidence. It predates the provider-shape and bounded runtime-cache changes released in v2.1.0 and is not a measurement of the current working tree.
+
+No raw request, catalog, output, path, account identifier, hostname, or private corpus record is published.
+
+## Discovery results
+
+| Metric | Baseline | AUTO | Absolute change | Relative change |
+| --- | ---: | ---: | ---: | ---: |
+| Provider input | 28,177 tokens | 5,215 tokens | -22,962 tokens | **-81.5%** |
+| Stable skill base | 106,966 bytes | 11,993 bytes | -94,973 bytes | **-88.8%** |
+| End-model skill recall | 38% | 50% | **+12 pp** | - |
+| Overlay/prefetch recall | - | 58% | - | - |
+| Request overlay | - | 8,676 bytes | - | - |
+| Stable-base byte gate | - | PASS | - | - |
+
+End-model recall and overlay/prefetch recall measure different stages. The 58% value is not added to the 50% value and is not an end-to-end score. Exact local loading through a registered `skill_search` name is a resolver invariant; it does not imply that the model will discover or select that name.
+
+The stable-base gate confirms that unrelated requests did not change the query-independent AUTO prefix. Provider-reported cache reads were zero, so this run demonstrates deterministic bytes, not a cache-hit or cost improvement.
+
+## Output results
+
+Three output cases were evaluated from identical raw inputs.
+
+| Pipeline | Case 1 | Case 2 | Case 3 | Total | Reduction | Exact evidence by case | Semantic evidence by case |
+| --- | ---: | ---: | ---: | ---: | ---: | --- | --- |
+| Raw | 3,876 | 4,004 | 1,219 | 9,099 | - | 100% / 100% / 100% | 100% / 100% / 100% |
+| Smart | 3,532 | 1,653 | 1,218 | 6,403 | **29.6%** | 100% / 100% / 100% | 100% / 100% / 100% |
+| Guarded extraction | 3,533 | 1,650 | 1,219 | 6,402 | **29.6%** | 100% / 100% / 100% | 100% / 100% / 100% |
+| RTK 0.42.4 | 1,793 | 500 | 244 | 2,537 | **72.1%** | 0% / 0% / 0% | 0% / 33% / 0% |
+
+Project smart and guarded-extraction paths retained 100% exact and semantic evidence. Both passed the project safety gate.
+
+RTK 0.42.4 is an external size comparator. Its smaller output did not retain the evidence required by this benchmark, so the external-comparator safety gate failed. This result applies only to RTK 0.42.4, these three cases, and this evidence definition. It is not a general assessment of RTK or its command-output use.
+
+Guarded extraction is accepted only when it is an ordered verbatim subsequence, retains protected evidence, clears both saving thresholds, and has a recoverable complete archive. Any rejection falls back to smart processing and then to the original.
+
+## Evaluation questions
+
+The benchmark asks:
+
+1. Does every eligible skill remain visible and exactly loadable?
+2. Is the stable base byte-identical across unrelated human requests?
+3. Does bounded prefetch improve first-turn recall without changing that base?
+4. Can `skill_search` recover relevant tail skills through bounded pagination?
+5. Can `tool_search` defer schemas without making required tools unavailable?
+6. Do output and history reductions preserve protected evidence and exact recovery?
+7. Is every accepted serialized transformation smaller?
+8. What do providers report for tokens, cache reads, cache writes, and cost?
+
+## Data and privacy contract
+
+### Deterministic synthetic suite
+
+The repository-local suite covers:
+
+- regular and irregular skill roots;
+- duplicate and near-duplicate names and descriptions;
+- typos, Unicode, empty queries, and unrelated queries;
+- critical, usage-weighted, configured, and excluded skills;
+- ties, ambiguous neighborhoods, and budget boundaries;
+- Anthropic, OpenAI Chat and Responses, Gemini, and Mistral request shapes;
+- dynamic tool schemas, pagination, activation, and already-used protection;
+- plain text, logs, tables, source, JSON, and adversarial evidence placement.
+
+Synthetic cases enforce invariants and regressions. They do not measure real-user quality.
+
+### Private real corpus
+
+`npm run corpus:build` creates an explicit local corpus of real catalogs, sanitized human requests, tool schemas, and tool outputs. Collection never runs in an ordinary Pi hook.
+
+A publishable evaluation corpus must:
+
+- replace identifying values with keyed irreversible identifiers;
+- remove absolute paths, credentials, hostnames, account names, and repository secrets;
+- retain only structure required for routing and evidence checks;
+- use human-reviewed labels rather than generated profile queries;
+- pass validation before any remote call;
+- remain under `.pi/skill-optimizer/benchmark/`;
+- never be committed or published raw.
+
+Remote evaluation receives only validated, sanitized cases.
+
+## Discovery protocol
+
+### Stable base
+
+Render the AUTO index for every catalog under unrelated requests and supported provider shapes.
+
+Blocking invariants:
+
+- unchanged catalog, profile, and configuration produce identical bytes;
+- the AUTO marker appears exactly once;
+- every eligible name appears exactly once;
+- no filesystem location appears;
+- all catalogs in a request share one soft intent budget until their combined all-name floors are reached;
+- a second transformation preserves the already transformed reference.
+
+Reported metrics:
+
+```text
+base_bytes
+base_provider_tokens
+all_name_recall
+byte_stability_rate
+path_leak_count
+```
+
+### Human-input prefetch
+
+Use only labeled human text as the query. Compare expected skills with full verbatim definitions appended to the latest genuine human block.
+
+Blocking invariants:
+
+- tool and function results never become query text;
+- assistant text, injected context, and image metadata never become query text;
+- empty input creates no ranking signal;
+- only the latest genuine human text block changes;
+- the AUTO marker prevents a second append;
+- no valid human target means identity;
+- descriptions stay verbatim and in source order;
+- count and character budgets are deterministic.
+
+Reported metrics:
+
+```text
+overlay_recall_at_budget
+overlay_precision
+overlay_bytes
+overlay_provider_tokens
+false_promotion_rate
+```
+
+### Skill search and loading
+
+`skill_search` is evaluated independently of proactive prefetch. Expected results come from human-reviewed corpus labels, not generated profile queries.
+
+Measure:
+
+- Recall@1, Recall@3, Recall@5, MRR, and nDCG;
+- exact-name recovery under bounded spelling variation;
+- pagination completeness and duplicate rate;
+- cursor rejection after query or fingerprint changes;
+- exact-load success for registered names;
+- rejection of unknown names, traversal, and symlink escape;
+- response bytes for search, resource, and location actions.
+
+Search-followed-by-load and multi-page sequences are included.
+
+## Dynamic-tool protocol
+
+Use small and large tool catalogs with human-reviewed required-tool labels.
+
+Blocking invariants:
+
+- core, configured, and already used tools remain active;
+- inactive schemas are absent from the initial provider request;
+- `tool_search` remains active while searchable definitions remain;
+- activation makes selected tools callable in the same session;
+- activation is additive and cannot strand an active provider tool loop;
+- schema indexing work and response pagination remain bounded;
+- weak, empty, or unrelated routing signal fails open to every permitted tool.
+
+Fail-open protects availability but can preserve the full tool-schema cost for that request. The benchmark must report this case rather than treating it as a discovery saving.
+
+Reported metrics:
+
+```text
+tool_recall
+initial_tool_schema_bytes
+activated_tool_schema_bytes
+tool_search_calls
+activation_failures
+```
+
+## Output protocol
+
+Each case records original bytes, returned bytes, archive status, recovery result, protected evidence, and reducer path.
+
+### Smart reduction
+
+Test plain text, logs, tables, source, JSON, Unicode, UTF-8 byte boundaries, and adversarial evidence placement.
+
+Required properties:
+
+- activation thresholds use UTF-8 bytes rather than JavaScript character count;
+- protected evidence remains present;
+- byte and line thresholds are not hard caps, and protected evidence may exceed them;
+- output clears absolute and relative saving thresholds;
+- archive failure returns the original;
+- `retrieve_output` reproduces the original and honors line bounds;
+- provider-facing text contains no archive location;
+- a second pass is not larger or more destructive.
+
+### Columnar JSON
+
+Use homogeneous object arrays with reordered keys, missing keys, nested values, Unicode, nulls, and metadata-like values.
+
+Required properties:
+
+- decoding is exact under JSON semantics;
+- column and row ordering are preserved;
+- protected evidence prevents conversion;
+- heterogeneous arrays remain unchanged;
+- the complete serialized representation is smaller.
+
+### Guarded extraction
+
+Measure accepted and rejected candidates. Acceptance requires:
+
+- ordered verbatim subsequence validation;
+- complete protected-evidence retention;
+- both saving thresholds;
+- a valid local archive;
+- deterministic fallback through smart processing to the original.
+
+Provider input, output, and cost are reported separately from deterministic saved bytes. A rejected candidate is a successful safety decision, not a reduction.
+
+### RTK pairing
+
+| Result class | Expected owner |
+| --- | --- |
+| Shell result already handled by RTK | RTK only |
+| Shell result not handled by RTK | AUTO eligible |
+| Read, web, MCP, and other eligible result | AUTO eligible |
+| Already reduced result | Neither runs again |
+
+AUTO alone, RTK alone, and paired behavior are compared on identical raw output. Returned size and protected-evidence recall are reported per result class.
+
+### History deduplication
+
+The first large successful result remains intact. Only later exact copies become opaque, recoverable references. Errors, images, mixed media, small results, and protected evidence remain unchanged. A second pass cannot increase mutation.
+
+## Provider and cache accounting
+
+Persistent telemetry v3 records:
+
+```text
+requests
+input
+output
+cacheRead
+cacheWrite
+totalCost
+```
+
+Only provider usage fields are authoritative for tokens, cache behavior, and cost. Characters, UTF-8 bytes, and characters-per-token conversions are secondary measurements.
+
+A cache experiment must:
+
+1. Send the same stable prefix across distinct human requests.
+2. Record the exact serialized prefix hash.
+3. Record provider usage and cost.
+4. Separate first-write, later-read, and uncached observations where available.
+5. Report unavailable counters as unavailable, not zero.
+6. Keep provider results separate before aggregation.
+
+OpenAI, Anthropic, and Gemini expose different cache semantics. Stable bytes make caching possible but do not prove a hit.
+
+## Scale boundary
+
+This benchmark covers 332 skills, not an unbounded catalog.
+
+AUTO keeps every eligible name in the stable index. Consequently, the minimum base size is the sum of framing plus all rendered names across the request. Intent text shares one request-global budget, but the combined all-name information floors grow linearly and can exceed `catalogBudgetChars`.
+
+A 100,000-skill catalog is not demonstrated here and may still be too large. Supporting that scale efficiently requires resolver-only discovery, namespaces, sharding, or a server-side paginated index. Those approaches change the current guarantee that every name is visible in the base prompt.
+
+The benchmark also does not establish reduction of arbitrary system prompts or ordinary conversation history. The project targets recognized skill catalogs, dynamic tool definitions, eligible tool results, and exact duplicate history results.
+
+## Execution
+
+### Local blocking gates
+
+```bash
+npm run typecheck
+npm test
+npm run bench
+npm run bench:ranking
+npm run bench:perf
+npm run bench:output
+```
+
+`npm run bench` covers discovery, loading, determinism, pagination, provider normalization, identity, and fuzz invariants. `npm run bench:output` covers byte safety, evidence, archive recovery, columnar round trips, extraction guards, history behavior, and RTK ownership.
+
+`npm run bench:ranking` compares BM25F-only, AUTO without fuzzy and full AUTO on
+a small hand-authored synthetic fixture. It is a component diagnostic, not a
+held-out relevance benchmark. [Current rationale and diagnostic results](docs/RANKING.md).
+`npm run bench:perf` reports runtime/platform, cold latency and 30 warm samples
+(mean, p50, p95) at 284 and 2,000 skills. These are request-preparation timings,
+not model, startup or terminal-redraw timings.
+
+### Private real-model workflow
+
+```bash
+npm run corpus:build
+npm run bench:real
+```
+
+These commands may send sanitized material to a real provider and incur cost. They are never invoked by `npm test`.
+
+### Captured-request measurement
+
+```bash
+npm run measure <capture.json>
+```
+
+Measure exact serialized characters and UTF-8 bytes. Token conversions are estimates unless they come from provider usage.
+
+## Publication checklist
+
+Before publishing a new result:
+
+- record commit, date, runtime, provider, model revision, corpus revision, and seed;
+- run paired variants against identical sanitized inputs;
+- keep ordering, concurrency, and retry policy fixed;
+- separate deterministic gates from model-quality metrics;
+- state failed and unavailable fields;
+- retain numerators and denominators behind published percentages;
+- publish no raw input, output, secret, identifier, hostname, or path;
+- carry no result from an earlier architecture into the AUTO table.
+
+## Primary references
+
+These sources guide the protocol; they are not project benchmark evidence.
+
+- [Pi skills](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/skills.md)
+- [Pi extensions and dynamic tools](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/extensions.md)
+- [OpenAI tool search](https://developers.openai.com/api/docs/guides/tools-tool-search)
+- [OpenAI prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching)
+- [Anthropic tool search](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)
+- [Anthropic prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
+- [Gemini context caching](https://ai.google.dev/gemini-api/docs/caching)
+- [MCP tools](https://modelcontextprotocol.io/specification/2025-06-18/server/tools)
+- [MCP resources](https://modelcontextprotocol.io/specification/2025-06-18/server/resources)
+- [OpenAI Codex skills](https://developers.openai.com/codex/skills)

package/CHANGELOG.md

@@ -0,0 +1,117 @@
+# Changelog
+
+All notable changes are documented here.
+
+## 3.0.0 - 2026-09-28
+
+### Scoped distribution - 2026-09-29
+
+- Move npm distribution to `@aeondave/pi-skill-optimizer`, replacing the old unscoped name at the same runtime version, with public access, the `pi-package` discovery keyword and a versioned banner URL for the Pi gallery. The existing Git tag and release history are unchanged.
+- Declare Pi host packages and TypeBox as wildcard, optional peers; keep development copies for tests, but ship no duplicate host dependencies. The supported runtime floor remains Pi 0.87.1 / Node.js 26.3.
+- Document scoped installation and migration without loading two copies. Explicitly allowlist published reference docs and cover gallery/host metadata with release tests.
+
+### Breaking
+
+- Require Pi 0.87.1+ and Node.js 26.3+; development dependencies, host integration tests and CI target the current APIs. Node 22/24 and older Pi releases are no longer supported. Existing profile files remain readable so users can explicitly refresh routing hints without losing their configuration.
+
+### Added
+
+- Init-generated activity tags distinguish specialists, workflow methods and behavioral discipline. A positively selected specialist can bring one contextual support skill into the ordinary budget even without literal keyword overlap. Tags do not make behavioral skills always-on; old profiles remain compatible and upgrade on explicit init.
+- Versioned init prompt and a real Pi loader/runner smoke test that runs without provider requests.
+- A brief, UI-only optional-init hint and explicit setup readiness in status. Session metadata prevents repeated hints across turns and reload/resume; manual profiles are respected and init never starts automatically.
+
+### Fixed
+
+- Replan tool activation on each agent run, retain used and explicitly searched tools, and restore optimizer-deferred tools on bypass without activating initially unavailable tools.
+- Refresh configuration between agent runs without repeated synchronous reads in provider/tool hooks. Skill search now respects exclusions and configured fuzzy/usage settings.
+- Route custom-tool errors through Pi's exception contract and keep resolver results verbatim, preventing recursive reduction of recovered output or full skill instructions.
+- Exact skill/profile phrase matches respect token boundaries; `git` no longer matches inside `digital`. Skill search cursors reject ranking-order changes caused by new profile or usage evidence.
+- Output recovery preserves LF, CRLF, CR and mixed terminators, and returns an empty page beyond EOF. Malformed Unicode cannot create lossy archive or ambiguous history handles.
+- Columnar JSON fails open on unsafe or noncanonical numeric spellings instead of silently rounding them; zero-column decoding has a bounded expansion.
+- Archive retention belongs to each stored output, not the project that happens to read or clean it. Identical-content saves can extend, never shorten, the lease; legacy files use a fixed seven-day fallback.
+- Stale lock timestamps no longer let another writer steal a live process's lock; recovery also verifies that the owner has exited.
+
+### Improved
+
+- Check complete rendered savings before writing an output archive.
+- Cache catalog-derived ranking features and calculate query-wide values once, without changing the BM25F weights or score formula.
+- Add a local ranking-component diagnostic and warm-latency percentiles; keep synthetic measurements separate from historical model-quality claims.
+- Put installation and everyday use first in the README, split out configuration/architecture/ranking references, and add the selected library banner.
+- Add Windows/Linux/macOS CI for Node 26.3 and current 26.x, portable path/newline regressions, package-link validation and consistent source line endings.
+
+## 2.1.1 - 2026-09-20
+
+### Fixed
+
+- `/skill-optimizer init` preserves the provider failure in terminal diagnostics, reports retries only when another attempt will run, and treats exhausted quota or billing limits as terminal instead of submitting the same batch again.
+
+## 2.1.0 - 2026-09-20
+
+### Added
+
+- A deterministic `bench:perf` harness reports cold and steady request-transform latency at 284 and 2,000 synthetic skills.
+
+### Changed
+
+- Development validation now targets Pi `0.85.1`, TypeBox `1.3.34`, and the current Node 22 type line while retaining the compatible Pi `>=0.84.4` peer floor.
+- Stable catalog preparation and rendering use bounded LRU caches, avoiding repeated parsing and rendering across provider requests and cache-warm refreshes.
+- Init batch progress is transient status; only start, terminal batch failures, and completion create durable notifications.
+
+### Fixed
+
+- `/skill-optimizer init` reports progress, checkpoints successful batches immediately, and avoids the redundant final profile rewrite and its stale-CAS race.
+- Init respects terminal versus retryable model failures, rejects non-success stop reasons, leaves failed changed skills inactive instead of serving stale routing, and repairs misplaced global/project profile state without another model call.
+- OpenAI/Azure Responses developer catalogs in `input` and structured `pi-messages` skill sections now receive the same AUTO transformation and latest-human prefetch contract.
+- Synthetic pi-persona identity and clock turns cannot become routing intent or receive a prefetch overlay.
+- Repeated identical status text no longer asks Pi to redraw the footer.
+
+## 2.0.0 - 2026-09-01
+
+### Added
+
+- One AUTO discovery pipeline with a stable TSV `name,intent` base and request-specific full-text overlay.
+- `skill_search` for paginated search, exact load, contained resources, and validated locations.
+- Paginated `tool_search` with Pi runtime activation and bounded proactive selection.
+- `retrieve_output` for bounded recovery of locally archived full results.
+- Content-addressed SHA-256 archives with opaque identifiers and TTL cleanup.
+- Monotonic history deduplication for later exact copies of large successful results.
+- Lossless columnar JSON guarded by exact internal round trip and never-larger acceptance.
+- Persistent provider telemetry v3 for requests, input, output, cache reads, cache writes, and total cost.
+- Catalog/profile static audit through `/skill-optimizer audit`.
+
+### Changed
+
+- AUTO is the sole public discovery contract. Former `off`, `compact`, `hybrid`, `drop`, and `relevance` choices are removed history.
+- The stable skill base is query-independent, path-free, byte-stable, and protected by an all-name floor.
+- Ranking combines BM25F, exact and bounded fuzzy signals, reciprocal-rank fusion, usage decay, and diversity.
+- Prefetch is a bounded latest-human overlay rather than a stable-prefix mutation.
+- Tool definitions activate on demand rather than being permanently sent or heuristically removed per request.
+- Request normalization shares a human-content allowlist across Anthropic, OpenAI Chat and Responses, Gemini, and Mistral.
+- Generated profiles contain only `critical`, `queries`, `clusters`, and `negativeHints`.
+- `init` batches by full UTF-8 weight and count, validates coverage and stop reason, and retries incomplete batches without truncation.
+- RTK coexistence is per result; only shell output RTK already handled steps aside.
+- Output limits use UTF-8 bytes and guarded extraction falls back through smart processing to the original.
+- Statistics and usage use lock-serialized additive deltas and atomic replacement.
+- Cache counters come only from provider usage; stable-prefix measurements no longer imply hits.
+- Benchmark results are reserved for a reproducible AUTO run.
+
+### Fixed
+
+- Tool and function results, assistant content, injected context, and image metadata cannot become routing queries.
+- Provider tool-use history is normalized before protection decisions.
+- Repeated overlay markers and requests without genuine human text preserve identity.
+- Exact skill and resource loading rejects unknown names, traversal, and symlink escape.
+- Recovery text no longer exposes archive paths, and recovery verifies archive identity.
+- Incomplete generation batches are not marked processed.
+- Concurrent profile, usage, and statistics writes cannot replace newer deltas with stale snapshots.
+- Columnar JSON preserves schema order, row order, nested values, Unicode, and protected evidence.
+
+### Privacy
+
+- Real-corpus construction remains explicit and private.
+- Identifiers are keyed and irreversible; sanitization rejects paths, credentials, and user-specific data.
+- Full archives remain local and expire under `outputArchiveTtlHours`.
+
+## Historical releases
+
+Tags before AUTO contain the original experiments. Their strategies, configuration, and measurements do not describe the current contract; consult a tag only when maintaining an older installation.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dave
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,148 @@
+# pi-skill-optimizer
+
+**Keep your skill library. Cut the repeated context.**
+
+![pi-skill-optimizer: a whole library, just the skills you need](assets/banner.png)
+
+[Install](#install) · [What changes](#what-changes) · [Everyday use](#everyday-use) · [Configuration](#configuration) · [Docs](#documentation)
+
+An extension for [Pi](https://github.com/earendil-works/pi) that routes skills and tools locally and reduces large tool results without losing access to the originals.
+
+Keep every skill name visible, bring relevant descriptions into focus, and load the full instructions when needed. Large tool results can be reduced too, with the original kept locally for recovery.
+
+No embedding service or extra model call for ordinary routing. No changes to your source skills.
+
+## Install
+
+Requires **Pi 0.87.1+** and **Node.js 26.3+**. Upgrade both before installing v3.
+
+```bash
+pi install npm:@aeondave/pi-skill-optimizer
+```
+
+Restart Pi, then check that it loaded:
+
+```text
+/skill-optimizer
+```
+
+That's enough to start. Optionally run `/skill-optimizer init` to generate routing hints for your library. **Init uses your current Pi model and may incur cost**; normal discovery does not.
+
+| Need | Command |
+| --- | --- |
+| Install only in this project | `pi install -l npm:@aeondave/pi-skill-optimizer` |
+| Pin the current release | `pi install npm:@aeondave/pi-skill-optimizer@3.0.0` |
+| Update an unpinned install | `pi update npm:@aeondave/pi-skill-optimizer` |
+
+Restart after updating. Pinned installs stay pinned until you explicitly install another version.
+
+### Moving from the unscoped package
+
+The supported npm package is **`@aeondave/pi-skill-optimizer`**; it replaces the old unscoped name. Commands and configuration paths have not changed. Version 3.0.0 keeps the same runtime code as the unscoped 3.0.0 release.
+
+Use `pi list` to check your installed sources. For an unpinned personal install:
+
+```bash
+pi remove npm:pi-skill-optimizer
+pi install npm:@aeondave/pi-skill-optimizer
+```
+
+For a pinned or Git install, remove the exact old source shown by `pi list`. Add `-l` to both commands for a project-local install. Keep only one source enabled: Pi treats scoped, unscoped and Git packages as different identities. Your optimizer config, routing profile and usage data stay in their existing locations; no re-init is needed just for this migration. Restart Pi afterward.
+
+## What changes
+
+Say you have hundreds of skills, but you're asking Pi to fix a database migration. Sending every full description again is rarely useful.
+
+AUTO keeps a compact, stable index of **all eligible names**, adds relevant descriptions beside your latest request, and exposes `skill_search` for loading complete instructions. The stable index does not change when your question changes.
+
+| Area | What AUTO does | What stays available |
+| --- | --- | --- |
+| Skills | Shortens catalog descriptions; prefetches relevant ones | Every eligible name and exact full-instruction loading |
+| Tools | Defers less relevant schemas behind `tool_search` | Protected, configured, already-used and activated tools |
+| Large results | Applies deterministic reduction; lossless columnar JSON where suitable | Complete originals through `retrieve_output` during archive retention |
+| Repeated results | Replaces later exact duplicates with recovery references | The first result, errors, protected evidence and mixed media |
+
+Tool routing fails open when the signal is weak: all permitted tools remain active. Output reduction also fails open if it cannot archive the original or make a meaningful saving. RTK-handled shell results are left to RTK, one result at a time.
+
+[How AUTO works](docs/ARCHITECTURE.md) · [Ranking decisions and limits](docs/RANKING.md)
+
+## Everyday use
+
+You can leave AUTO running with its defaults.
+
+- `/skill-optimizer` shows setup readiness, savings, active tools and provider usage.
+- `/skill-optimizer audit` reports description issues without rewriting your skills.
+- `/skill-optimizer init` creates or updates routing hints. Successful batches are checkpointed; rerunning resumes incomplete work.
+
+An optional setup hint appears at the first eligible prompt when initialization is missing or needs attention. It never starts init for you or adds instructions to the model's context.
+
+Pi gets three tools: `skill_search`, `tool_search` and `retrieve_output`. Skill search is paginated and exact loads accept only registered names. Output recovery uses opaque identifiers, not exposed archive paths.
+
+Skills can be useful even when you never mention them. Init can tag architecture,
+testing and behavioral guidance by activity, so a matched specialist can bring
+one relevant supporting skill into prefetch. Behavioral does not mean always-on.
+[Contextual routing](docs/RANKING.md#contextual-and-behavioral-skills) explains the
+budget and limitations. Existing profiles need an explicit init rerun for these tags.
+
+## Configuration
+
+Start with defaults. For project-specific settings, create `.pi/skill-optimizer/config.json`:
+
+```json
+{
+  "alwaysSkills": ["your-project-skill"],
+  "alwaysTools": ["your-required-tool"],
+  "outputMode": "smart"
+}
+```
+
+Replace the example names with installed skills/tools. Global defaults live at `~/.pi/agent/skill-optimizer/config.json`; project settings override them, followed by environment variables.
+
+The default output mode is local and deterministic. Optional `extract` mode needs an explicit model and validates its output before accepting it. It is not enabled by default.
+
+[Full configuration, defaults and RTK behavior](docs/CONFIGURATION.md)
+
+## Limits and privacy
+
+This is not a general conversation summarizer. It does not shrink arbitrary instructions, guarantee that the model picks the right skill, or guarantee lower provider bills.
+
+All names stay visible, so the minimum index still grows with the library. Description budgets are soft at that floor. Output size settings trigger reduction rather than enforce hard caps: protected evidence can exceed them.
+
+Original outputs stay in local archives, retained for **seven days by default**. They may contain sensitive tool output; treat them accordingly. Normal hooks do not collect benchmark corpora or send data to a separate ranking service. Your selected Pi provider still receives the ordinary request.
+
+Generated profile hints are not benchmark labels. Stable prompt bytes are not proof of cache hits. [Privacy, persistence and accounting](docs/CONFIGURATION.md#privacy)
+
+## Evidence and development
+
+The [published v2.0.0 benchmark](BENCHMARK.md#published-v200-snapshot) records eight private, anonymized cases. It is historical evidence, **not a result for the current working tree or a universal savings claim**.
+
+Local checks use synthetic data, no remote model calls:
+
+```bash
+npm install
+npm run typecheck
+npm test
+npm run bench
+npm run bench:ranking
+npm run bench:perf
+npm run bench:output
+```
+
+Load your checkout with `pi -e ./src/index.ts`. `bench:ranking` compares routing components on a small diagnostic fixture; `bench:perf` measures cold and warm request handling at 284 and 2,000 skills. Neither establishes end-to-end model quality.
+
+The CI workflow covers Windows, Linux and macOS on Node 26.3 and the current 26.x. Tests
+include native/forward-slash resource paths, spaces, Unicode, symlink containment,
+and exact LF/CRLF/mixed-newline recovery. A configured matrix is not evidence of
+a successful run; check the jobs for the commit you use.
+
+`corpus:build` and `bench:real` are separate, explicit private-evaluation workflows. They are not part of the local checks and may involve provider cost. See the [benchmark protocol](BENCHMARK.md) before using them.
+
+## Documentation
+
+- [Architecture and guarantees](docs/ARCHITECTURE.md)
+- [Configuration and runtime reference](docs/CONFIGURATION.md)
+- [Ranking rationale and evaluation](docs/RANKING.md)
+- [Benchmark contract and historical results](BENCHMARK.md)
+- [Changelog](CHANGELOG.md)
+
+MIT. See [LICENSE](LICENSE).