trl-token-reduction 0.1.0__tar.gz

trl_token_reduction-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or Derivative
+          Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Aryan Gonsalves
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

trl_token_reduction-0.1.0/NOTICE

@@ -0,0 +1,4 @@
+Token Reduction Layer for LLM Apps & Agents
+Copyright 2026 Aryan Gonsalves
+
+Licensed under the Apache License, Version 2.0 (see LICENSE).

trl_token_reduction-0.1.0/PKG-INFO

@@ -0,0 +1,391 @@
+Metadata-Version: 2.4
+Name: trl-token-reduction
+Version: 0.1.0
+Summary: Vendor-neutral token-reduction layer for LLM apps and agents: caching, compression+fact-guard, AST/text retrieval, and cascade — with published quality-neutrality benchmarks.
+Author: Aryan Gonsalves
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/AryanGonsalves/trl-token-reduction
+Project-URL: Repository, https://github.com/AryanGonsalves/trl-token-reduction
+Keywords: llm,tokens,cost,retrieval,compression,mcp,proxy,agents,claude,openai,codex
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Provides-Extra: retrieval
+Requires-Dist: tree_sitter>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_python>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_javascript>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_typescript>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_go>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_rust>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_java>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_c_sharp>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_c>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_cpp>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_ruby>=0.21; extra == "retrieval"
+Requires-Dist: tree_sitter_php>=0.22; extra == "retrieval"
+Requires-Dist: tree_sitter_kotlin>=1.0; extra == "retrieval"
+Requires-Dist: pathspec>=0.11; extra == "retrieval"
+Provides-Extra: proxy
+Requires-Dist: pyyaml>=6.0; extra == "proxy"
+Requires-Dist: tiktoken>=0.7.0; extra == "proxy"
+Provides-Extra: models
+Requires-Dist: anthropic>=0.40.0; extra == "models"
+Requires-Dist: openai>=1.40.0; extra == "models"
+Requires-Dist: tiktoken>=0.7.0; extra == "models"
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.0; extra == "mcp"
+Provides-Extra: embed
+Requires-Dist: model2vec>=0.3; extra == "embed"
+Provides-Extra: pdf
+Requires-Dist: pypdf>=4.0; extra == "pdf"
+Provides-Extra: all
+Requires-Dist: tree_sitter>=0.21; extra == "all"
+Requires-Dist: tree_sitter_python>=0.21; extra == "all"
+Requires-Dist: tree_sitter_javascript>=0.21; extra == "all"
+Requires-Dist: tree_sitter_typescript>=0.21; extra == "all"
+Requires-Dist: tree_sitter_go>=0.21; extra == "all"
+Requires-Dist: tree_sitter_rust>=0.21; extra == "all"
+Requires-Dist: tree_sitter_java>=0.21; extra == "all"
+Requires-Dist: tree_sitter_c_sharp>=0.21; extra == "all"
+Requires-Dist: tree_sitter_c>=0.21; extra == "all"
+Requires-Dist: tree_sitter_cpp>=0.21; extra == "all"
+Requires-Dist: tree_sitter_ruby>=0.21; extra == "all"
+Requires-Dist: tree_sitter_php>=0.22; extra == "all"
+Requires-Dist: tree_sitter_kotlin>=1.0; extra == "all"
+Requires-Dist: pathspec>=0.11; extra == "all"
+Requires-Dist: pyyaml>=6.0; extra == "all"
+Requires-Dist: tiktoken>=0.7.0; extra == "all"
+Requires-Dist: anthropic>=0.40.0; extra == "all"
+Requires-Dist: openai>=1.40.0; extra == "all"
+Requires-Dist: mcp>=1.0; extra == "all"
+Requires-Dist: model2vec>=0.3; extra == "all"
+Requires-Dist: pypdf>=4.0; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Dynamic: license-file
+
+# Token-Reduction Layer for LLM Apps & Agents
+
+A vendor-neutral efficiency layer that sits in front of an expensive API model
+(Claude / GPT) and cuts token spend **several-fold with a published,
+quality-neutral benchmark** — not a hand-wavy claim. Four composable levers, each
+measured against the rule: **remove what the model doesn't need; never what it does.**
+
+> Status: research build. Every lever below is implemented, benchmarked, and
+> unit-tested offline. Numbers are from the included benchmarks (see *Reproduce*).
+> The engine is real; some big-model arms use mocks/oracles where noted — the
+> *structure* and the *quality-neutrality tests* are the contribution.
+
+## Why this exists
+
+Agents are pathologically wasteful: they resend the full system prompt + tool
+schemas + entire history every step, dump huge raw tool outputs, and route every
+step — routine or hard — to the frontier model. The savings come from removing
+that redundancy, irrelevance, and grunt work, **measured** so you know quality
+didn't move.
+
+## The four levers (all built, all measured)
+
+| Lever | What it does | Result (benchmark) | Quality |
+|-------|--------------|--------------------|---------|
+| **Caching** | bill the stable prefix once at the cache rate, not full freight each step | neutral vs a *cache-enabled* baseline (the honest baseline) | — |
+| **Compression + guard** | a local model summarizes/dedupes the growing tail; a deterministic guard re-injects any number it drops | ~37% fewer tail tokens | **100%** (guard makes it exact-safe) |
+| **Retrieval** | replace whole-file code dumps with only the relevant AST slices (tree-sitter, local, zero API tokens) | **5.9× / 83%** fewer context tokens on code-QA | **100%**, verbatim slices |
+| **Cascade** | answer verifiable/easy steps on the cheap local pipeline; escalate only hard steps to the big model | **75% fewer big-model calls** | **100%**, 0 false-accepts |
+
+**Compound (all four, one 20-turn code-agent session):**
+`18.1× cheaper · 94.5% less $ · 80% fewer big-model calls · quality 100% = naive.`
+The levers *multiply*, not just add.
+
+Honest caveat, stated up front: the compound number is a *favorable* synthetic
+workload (cascade handles the verifiable majority). Real agent mixes vary — our
+long-standing story is **3–10× on favorable workloads, modest on pure novel
+reasoning.** The point of the 18× demo is that the levers stack multiplicatively,
+not the exact figure.
+
+> **Read the retrieval numbers carefully (updated after a paired billed-token test).** The ~93% / ~99%
+> figures below measure *content reduction* — how much smaller a returned slice is than reading the
+> whole file(s) it came from. That is **not** billed-token savings in an agent loop. In a paired test
+> on a real repo (provider-billed input tokens summed over every turn, answers graded, vs a plain
+> grep+read agent), retrieval was **quality-neutral** and cheaper *per retrieval*, but a current
+> **over-retrieval failure mode** can make the *total* net-neutral-to-worse (two runs swung from ~15%
+> cheaper to ~70% costlier). So read ~93% / ~99% as "vs reading whole files," not "vs a competent
+> shell agent." The paired harness, the numbers, and the honest breakdown are in the Hoard case study
+> below.
+
+## Validated on real APIs (billed tokens, not estimates)
+
+The benchmarks above run offline. But the levers were also proven **end-to-end
+against live models**, and every number here is the **provider's own reported
+billed token count** (OpenAI `usage.prompt_tokens`; Anthropic `usage.input_tokens`
+including cache) — not our estimate, not a mock.
+
+**Flagship models, heavy tasks (the headline result).** Run on the two most advanced
+models available, over heavy tasks built from *real* content (the project's own docs
+and code, ~6–10k tokens each), baseline vs token-reduced:
+
+| Flagship model | Input tokens | Quality |
+|----------------|--------------|---------|
+| **OpenAI GPT-5.5** | **16,875 → 2,232 (86.8% fewer)** | **3/3 → 3/3, non-inferior PASS** |
+| **Anthropic Claude Opus 4.8** | **26,573 → 3,343 (87.4% fewer)** | **3/3 → 3/3, non-inferior PASS** |
+
+~87% of input tokens removed on the current frontier, with quality fully preserved on
+both. (Reproduce: `validate/heavy_bench.py`, hard-capped at $1.80/provider.) The three
+tasks are a long-document QA, a bloated agent turn, and a code-context QA.
+
+**Cross-file, multi-file codebase (the "does it hold on a bigger codebase?" test).**
+Three questions that each require tracing across 2–3 files, answered from the *whole*
+codebase (~30k tokens) vs the retrieval-reduced context, same non-inferiority test:
+
+| Cross-file codebase QA (~30k-token repo) | Input tokens | Quality |
+|------------------------------------------|--------------|---------|
+| **OpenAI GPT-5.5** | **89,845 → 3,891 (95.7% fewer)** | **3/3 → 3/3, non-inferior PASS** |
+| **Anthropic Claude Opus 4.8** | **141,628 → 6,102 (95.7% fewer)** | **3/3 → 3/3, non-inferior PASS** |
+
+On a real multi-file codebase, the retrieval-reduced context answered the cross-file
+tracing tasks as well as the full context. Honest scope: this is a ~30k-token repo —
+*medium* scale, not a 500k-line monorepo, and n=3 tasks. (Reproduce:
+`validate/bigcode_bench.py`, hard-capped at $1.80/provider.)
+
+**Independent out-of-distribution check.** Pointed the retriever at a *separate
+~52k-token production Python codebase it had never seen* (35 files, 359 symbols) and ran
+8 natural-language audit queries. Retrieval returned ~1k tokens of slices per query vs the
+52k whole-dump — **~98% fewer** — with the answer-bearing module/symbol as the top hit on
+descriptively-named queries. (Informal relevance check, not a labeled precision eval; point
+it at your own repo: `python -m validate.measure_repo_reduction --repo <path>`.)
+
+**Real-world longitudinal usage — an entire game built with the plugin (live, logged).**
+Beyond the offline/API benchmarks, the plugin was dogfooded across a *full project*: a Roblox
+game ("Hoard") built with Claude Code across the full project build (many feature, integration, and hardening
+sessions), from an empty scaffold to a 52-file / ~164k-token codebase. Two independent things were
+measured at every phase — the **direct** retrieval reduction on the repo, and the plugin's **own
+live savings log** (`TRL_SAVINGS_LOG`): exactly what the agent called and what each returned slice
+replaced.
+
+| Hoard (Luau/Roblox), tracked as the repo grew | Result |
+|---|---|
+| **Direct** reduction vs whole-repo dump | **52% -> 99%** as the repo scaled ~2.2k -> ~164k tokens (10/10 relevant top-hits on descriptive queries) |
+| **Live** agent usage (the plugin's savings log) | **188 real calls, ~2.12M tokens saved, 92.9% fewer** vs the file(s) those slices would otherwise have been read from |
+
+The live curve climbed with the codebase (85% -> 93% cumulative across phases) toward the ~98%
+per-call direct ceiling — i.e. the bigger the project got, the more a ~few-hundred-token slice beat
+reading whole files. Honest caveats, stated plainly: (1) the live "counterfactual" is the whole
+*file(s)* each slice came from — what the agent would otherwise read — not the whole repo, so it's a
+realistic per-call baseline, not a favorable one; (2) savings only accrue on **adoption** —
+feature-*creation* phases call retrieval little, integration/refactor/hardening phases call it heavily
+(where it pays most), so the cumulative number reflects real mixed usage; (3) scope is one project,
+one developer, one language, and two early phases' calls were **lost to a logging bug before it was
+fixed** (disclosed and excluded) — so the 188 calls are the reliably-logged subset and the true total
+was higher. Reproduce on your own project: `python -m validate.measure_repo_reduction --repo <path>`
+(direct) and set `TRL_SAVINGS_LOG`, then `python -m validate.savings_report <log>` (live). The exact game repo (WIP, published as a verifiable benchmark) is at **github.com/AryanGonsalves/hoard**.
+
+**Paired billed-token test (added after fair criticism).** The figures above are *content* reduction
+(slice vs whole-file), not billed tokens, and "whole file" is a generous baseline. So we ran the
+harder test: the same tasks through a real agent loop twice — once with the `retrieve_code` /
+`explain_symbol` MCP, once with a plain grep+read agent (`code_search` + `read_file`) — summing
+**provider-billed input tokens over every turn** (tool schemas + growing history + results) on
+`claude-sonnet-5`, with both answers graded 0–3 against the code. Honestly:
+
+- **Quality: neutral** (grader ~1.5/1.5, tied).
+- **Cheaper per retrieval** than grep+read on well-behaved tasks (~2–6×, since one retrieve replaces
+  several search-then-read round-trips).
+- **But an over-retrieval failure mode**: on ~1/3 of tasks the agent looped to 25+ calls and the
+  accumulated slice history ballooned to 280–380k tokens, flipping the total. Across two runs the
+  billed total swung from ~15% cheaper to ~70% costlier vs the shell agent.
+
+So the honest scope: retrieval clearly wins **vs whole-file reads** (a common default) and is
+quality-neutral, but it is **not** a blanket "90%+ billed savings" vs a disciplined shell agent, and
+the over-retrieval blowup is a real defect being fixed (cap calls / leaner slices / stop-and-answer).
+Reproduce: `python -m validate.paired_billed_bench --repo <path> --model claude-sonnet-5`.
+
+| Earlier single-lever / mid-tier runs | Reduction | Quality / correctness |
+|----------------|-----------|-----------------------|
+| Realistic verifier suite — **OpenAI**, safe mode | **2.59× cheaper**, 43.1% fewer input tokens, 62.7% fewer sub-units | 100% → 100%, non-inferiority **PASS** |
+| Live proxy, one bloated turn — **OpenAI gpt-4o-mini** | **1,672 → 730** billed input tokens (**56.3% fewer**) | output correct, fact-guard held |
+| Compression lever — **OpenAI gpt-4o-mini** | **966 → 305** (**68% fewer**) | correct |
+| Document retrieval — **OpenAI gpt-4o-mini** | **1,374 → 318** (**77% fewer**) | correct |
+| Cascade — **OpenAI gpt-4o-mini** | **89%** of steps answered locally ($0 to the API) | **100%** accuracy, 0 false-accepts |
+| Live proxy, compression + cache — **Anthropic claude-haiku-4.5** | **1,198 → 477** billed input tokens (**60% fewer**) | correct |
+
+Honest caveats, stated plainly: the suites are small, the multi-lever suite is a
+*favorable* workload, it's one model per provider, and the live-proxy rows each
+isolate a single lever. These are directional — but they are **real billed tokens
+on real models through the actual proxy**, which is the bar that matters. Reproduce
+them with your own key via `RUN.md` (`validate/live_openai.py`, `validate/live_anthropic.py`).
+
+## What we measured and learned (the honest findings)
+
+- A **cache-enabled baseline** neutralizes the caching lever — the real
+  differential comes from the tail (compression + retrieval) and from cascade.
+- **A real 3B local model is a lossy compressor**: llama-3.2-3b dropped a
+  load-bearing fact ~1 in 3 times on a hard workload. The **fact-preserving
+  guard** (re-inject any dropped number, deterministically) restores quality to
+  100% while keeping most savings. Compression is only shippable *with* the guard.
+- **Text & PDFs get the same treatment**: a separate chunk-based retriever
+  slices prose/PDF documents to the passages that matter (numbers preserved for
+  disambiguation), so non-code requests cost fewer tokens too — without touching
+  the code path.
+- **Semantic ranking, tested honestly**: on deliberately vague queries, static
+  embeddings gave *no* lift over keyword (dropped); an LLM-reranker is a small,
+  real, regression-free gain but costs one cheap model call, so it ships opt-in.
+- **For code, retrieval beats compression on both axes** — bigger token cut *and*
+  quality-safe by construction (exact slices can't paraphrase a fact away).
+- **Subscription limits (Claude Pro/Max) are server-side** — you can't turn a
+  subscription into cheap bulk API calls; the OAuth token authenticates but is
+  hard rate-limited. The clean, attackable target is metered API $, plus
+  *local-side* reduction inside coding-agent clients.
+
+## Architecture
+
+```
+CODEBASE-AWARE surface (library / MCP plugin / agent integration — can see your repo):
+  request ──► retrieve (AST slices, local)     # lever 3: don't stuff whole files
+          ──► cascade  (easy → local, $0)       # lever 4: needs a local model
+          ──► Engine.process:
+                ├─ cache-mark stable prefix      # lever 1
+                └─ compress tail + fact guard    # lever 2
+          ──► big model (only when needed)
+
+BLIND proxy (base_url swap — CANNOT see your repo):
+  request ──► Engine.process: cache-mark prefix + compress tail + fact guard   # levers 1 & 2
+          ──► retrieve ONLY over context you pass in a `documents` field       # lever 3, opt-in
+          ──► big model
+  (cascade and codebase-wide retrieval are NOT available on the blind proxy)
+```
+
+### Which lever lives on which surface
+
+The headline retrieval numbers (95.7% cross-file, and the ~87% frontier runs) come
+from the RETRIEVAL-heavy code-agent regime, which needs a surface that can **index
+your codebase**. The blind proxy can't see your repo, so on the proxy only caching +
+compression fire automatically. **Never read the 87%/95.7% as a blind-proxy number.**
+
+| Surface | caching (1) | compression + guard (2) | retrieval (3) | cascade (4) |
+|---|---|---|---|---|
+| **Blind proxy** (base_url swap) | ✓ auto | ✓ auto | only over a `documents` field you pass | ✗ |
+| **Library / Engine** (in your agent) | ✓ | ✓ | ✓ indexes your repo | ✓ if local model |
+| **MCP plugin** (Claude Code / Codex) | ✓ | ✓ | ✓ | ✓ if local model |
+| **Browser extension** (`/compress`) | — | ✓ | ✓ (text) | ✗ |
+
+The full-stack ~99% offline reduction over this repo (`validate/integrated_loop.py`)
+is a *codebase-aware* run — retrieval doing most of the work, compression + caching
+stacking on top. The blind proxy delivers the compression + caching portion only
+(the single-lever live-proxy rows above: ~56–68% on a bloated turn).
+
+- `trl/` — the shared engine (caching, compression+guard, cascade router) and
+  `trl/retrieval/` (tree-sitter extractor + retriever, **13 languages / 20 file
+  types**: Python, JS/JSX, TS/TSX, Go, Rust, Java, C#, C, C++, Ruby, PHP, Kotlin, Luau/Roblox),
+  plus a general **text + PDF** retriever (`text_index.py`, `pdf.py`) so non-code
+  documents get the same slice-don't-dump treatment.
+- `bench/` — the harness: baseline (cache-enabled) vs treatment, real token/$
+  accounting, **non-inferiority** quality gates, and one benchmark per lever plus
+  the compound `pipeline_bench.py`.
+- `tests/` — unit tests for every lever and provider parser.
+
+## Install
+
+```bash
+pip install trl-token-reduction            # core (stdlib only)
+pip install "trl-token-reduction[all]"     # + retrieval grammars, proxy, model SDKs, MCP, PDF
+```
+Or from source: `git clone … && pip install -e ".[all]"`. Installing exposes three
+commands — `trl-proxy`, `trl-retrieve`, `trl-cli` — used below. Extras are modular:
+`[retrieval]`, `[proxy]`, `[models]`, `[mcp]`, `[embed]`, `[pdf]`.
+
+**Who benefits (be honest):** anyone paying **metered API** per token (pay-as-you-go),
+and anyone on a **token-metered subscription** — Claude Code (a flat monthly fee, but
+usage is *metered in tokens* against your weekly caps) and Codex (which OpenAI moved to
+token-based pricing in Apr 2026). On a subscription this doesn't cut a bill — it makes
+you burn your quota slower, so you hit the weekly/5-hour caps later (more work per plan).
+It does **not** help the ChatGPT/Claude *chat websites* (closed endpoints; ChatGPT's cap
+is per-message, not per-token).
+
+## How you use it
+
+Four ways to put it in front of your model — pick by how much you want to change.
+
+### 1. Drop-in proxy (any app, zero code changes)
+The proxy speaks both the OpenAI **and** Anthropic wire formats. Point your
+client's `base_url` at it; keep your own key (the proxy never stores it). Every
+request gets **caching + compression (+ fact guard)** applied on the way through,
+and comes back with an `X-TRL-Tokens-Saved` header so you can see the cut per call.
+(The blind proxy can't see your codebase, so the retrieval and cascade levers live
+on the codebase-aware surfaces below — see the surface/lever table above.)
+
+```bash
+trl-proxy                              # listens on :8899, forwards upstream
+# (or: python -m proxy.server)
+```
+```python
+# OpenAI SDK — the ONLY change is base_url
+client = OpenAI(base_url="http://localhost:8899/v1", api_key=YOUR_KEY)
+# Anthropic SDK works the same via /v1/messages
+```
+Want a document shrunk before it hits the model? Add a non-standard
+`documents: [...]` field to the request — the proxy retrieves only the relevant
+passages and injects them, instead of you pasting the whole file.
+
+### 2. Coding-agent plugin (Claude Code, Codex, or any MCP client)
+Installs an MCP server `trl-retrieve` exposing `retrieve_code(query)` and
+`explain_symbol(name)`. Your agent asks for *code that answers a question* and
+gets AST slices (file:line + source) instead of stuffing whole files. Drop
+`plugin/claude-code/.mcp.json` into a repo and point `TRL_REPO` at it. **Codex**
+uses the same STDIO server — drop the `[mcp_servers.trl-retrieve]` block from
+`plugin/codex/config.toml` into `~/.codex/config.toml`. See `plugin/INSTALL.md`.
+No agent? Same thing from the shell:
+
+```bash
+trl-cli "how does auth refresh work?" --repo /path/to/repo
+# (or: python -m plugin.cli ...)
+```
+
+### 3. Library (embed the engine directly)
+Call the pieces from your own code when you control the request path:
+
+```python
+from trl import Engine, Message, load_config
+
+engine = Engine(load_config("config.yaml"))
+result = engine.process([
+    Message(role="system", content=BIG_SYSTEM_PROMPT),
+    Message(role="user", content="your request"),
+])
+# result.messages           -> cache-marked prefix + compressed tail, ready to send upstream
+# result.cache_prefix_tokens -> stable-prefix tokens billed at the cache rate
+# result.meta               -> which levers fired
+```
+
+## Claude Code plugin (retrieval tier)
+
+The repo doubles as a Claude Code plugin **and** its own marketplace — no store, no
+review, just files. Add the marketplace (`AryanGonsalves/trl-token-reduction`) and
+install the `trl` plugin. It ships:
+
+- the `trl-retrieve` **MCP server** (`retrieve_code`, `explain_symbol`) — the agent
+  pulls exact AST slices instead of reading whole files;
+- a **skill** that tells the agent to actually use those tools;
+- `/trl-index` (build/refresh the index) and `/trl-status` (show savings).
+
+It auto-targets YOUR project at runtime (`$TRL_REPO` / `$CLAUDE_PROJECT_DIR` / nearest `.git` / cwd) and caches the index at `<project>/.trl/index.json`; if it can't resolve the project it asks you to run `/trl-index` or set `TRL_REPO`.
+
+**Honest scope:** the plugin delivers the **retrieval tier** — cutting context tokens by
+fetching slices, not whole files. It does **not** deliver the full proxy-path headline
+(~87%); prefix caching + tail compression require running the local proxy and pointing
+your agent's base URL at it (two-tier note above).
+
+### Optional hosted rerank (precision, off by default)
+
+For vague natural-language questions where keyword retrieval misses the right symbol,
+`retrieve(rerank="hosted")` asks a hosted model (your own `ANTHROPIC_API_KEY`,
+`claude-haiku-4-5-20251001` by default) to reorder the shortlist. Validated on a
+15-query NL set (2 runs): free `doc_boost` **6/15** → Haiku **11/15**, Sonnet
+**8.5/15** (oracle ceiling **12/15**), with the 6/6 code loop non-inferior. It is a
+**precision option, not a token saver** — ~1¢/query via Haiku on your own key — and is
+**off by default**. The remaining 11→12 gap is retrieval-recall/shortlist coverage, not
+the reranker.