@tokenade/cli 0.7.2 → 0.7.3

package/README.md

@@ -160,6 +160,8 @@ When reporting, please include your OS, `tokenade --version`, your coding agent,
 | **System & network** | `table` · `disk` · `netstat` · `dns` · `vmstat` · `filelist` · `grep` |
 | **Web & docs** | `web` · `serp` (aliases `google`/`bing`/`ddg`) · `snapshot` · `stealth` · `curl` · `docs` · `pdf` · `notebook` |
 
+**Your own compactors.** `add-compactor --file <rules.toml>` teaches Tokenade to fold output from a command it doesn't cover yet (alias `add-filter`); `add-compactor --list` shows the ones you've added, and `add-compactor --help` prints the file format.
+
 ### Savings, receipts & reporting
 
 | Command | What it does |
@@ -348,132 +350,20 @@ $ tokenade execute --lang python --script 'print(sum(range(101)))'
 $ eval "$(tokenade shell-init)"   # add to ~/.zshrc — git/cargo/kubectl/… auto-compact
 ```
 
-### Format compactors
-
-`tokenade auto <file|->` auto-detects and dispatches — **the one to reach for**. Or call a format directly. All read a file or `-` (stdin).
-
-**`json`** — folds arrays of objects into a compact TOON table.
-```text
-$ tokenade json api.json
-{ "page": 1, "total": 20, "users": "<see [fold @ users] block below>" }
-
-[fold @ users] [×20 rows · toon] active,id,name,role
-  true,1,"u1","user"
-  …
-```
-
-**`logs`** — collapses runs of templated-equal lines (timestamps/IDs normalised).
-```text
-$ tokenade logs app.log         # 200 lines →
-<TS> INFO worker job=<N> status=done ms=<N>  [×200 lines · e.g. job=1, ms=11]
-```
-
-**`diff`** — caps context, folds multi-file patches.
-```text
-$ git diff | tokenade diff -
-diff --git a/redact.rs b/redact.rs
-@@ -175,17 +175,24 @@ fn patterns() …
-…2 context line(s) elided…
-+            r#"…(?:'[^']*'|"[^"]*"|[^\s]+)"#,
-```
-
-**`csv` / `tsv`** — drop empty columns, cap rows, truncate cells.
-```text
-$ tokenade csv users.csv
-id,name,email,signup,plan
-1,Ada,ada@x.io,2024-01-02,pro
-# 1 empty column(s) dropped
-```
-
-**`yaml`** — strips `managedFields`, `lastTransitionTime`, and other k8s noise.
-```text
-$ kubectl get deploy web -o yaml | tokenade yaml -
-metadata:
-  name: web
-  managedFields: <dropped>
-status:
-  conditions:
-  - type: Available
-    lastTransitionTime: <dropped>
-```
+### Format & output compactors
 
-**`env`** — compact an env dump **and redact secrets**.
-```text
-$ env | tokenade env -
-AWS_SECRET_ACCESS_KEY=<redacted 17 chars>
-DATABASE_URL=<redacted 21 chars>
-# [tokenade:env] 2 secret(s) redacted
-```
+Tokenade includes a family of format-aware compactors — the full list is in
+the [CLI reference](#-cli-reference--every-command) above. Each reads a file or
+`-` (stdin); when unsure, let Tokenade pick:
 
-**`table`** — any columnar CLI output (`docker ps`, `kubectl get`, `ps aux`).
-```text
-$ docker ps | tokenade table -
-IMAGE         STATUS        PORTS               NAMES
-nginx:latest  Up 2 hours    0.0.0.0:80->80/tcp  web
-```
-
-**`stack`** — fold framework frames in a stack trace, keep your code + the error.
-```text
-$ tokenade stack trace.txt
-/app/main.py:42 in <module>   run()
-/app/db.py:17 in query   raise ValueError("bad row")
-ValueError: bad row
-```
-
-**`dns`** — keep the ANSWER section of `dig`/`host`, drop the pseudo-sections.
-```text
-$ dig example.com | tokenade dns -
-example.com A NOERROR: 2 answers
-  example.com.  IN  A  93.184.216.34
-```
+```sh
+tokenade auto <file>      # auto-detect and compact
+some-noisy-cmd | tokenade auto -   # …or pipe stdin
 
-**`access`** — group an nginx/Apache log by method + path + status.
-```text
-$ tokenade access access.log
-3 request(s) · status counts: 200=2, 500=1
-  2× GET  /api/users [200]
-  1× POST /api/pay   [500]
+tokenade add-compactor --file my-rules.toml   # teach Tokenade a new compactor
+tokenade add-compactor --list                 # list the ones you've added
 ```
 
-The full family — same shape (`tokenade <name> file|-`), each tuned to one format:
-
-| Command | Common use → output |
-|---|---|
-| `md` | Markdown → drop front-matter, fold huge code blocks, strip ref-links |
-| `ndjson` | JSONL cloud logs → severity histogram + key-union schema |
-| `lock` | `Cargo.lock`/`yarn.lock` → flat `name@version` list |
-| `sql` | psql/mysql result table → capped, borderless rows |
-| `openapi` / `graphql` | API spec → endpoint / type + field list, no bodies |
-| `sourcemap` | `.map` → sources list, drop mappings blob |
-| `gitlog` | `git log` → `hash author subject` per commit |
-| `archive` | `tar -tvf`/`unzip -l` → names only, perms/size dropped |
-| `terraform` | `terraform plan` → fold "Refreshing state", drop unchanged attrs |
-| `k8s-manifest` | `helm template` → per-kind summary, drop spec bodies |
-| `describe` | `kubectl describe` → identity + status + warnings |
-| `systemd` | `systemctl status` → Active + Mem/CPU + ERR/WARN tail |
-| `journal` | `journalctl`/`dmesg` → group by service, surface ERR/WARN |
-| `container-logs` | `kubectl logs --prefix` → per-stream fold |
-| `grep` | `grep -r`/`rg` → group by file, top hits |
-| `filelist` | `find`/`tree`/`ls -laR` → per-dir counts + ext histogram |
-| `disk` | `du -h` → top entries by size |
-| `netstat` | `ss`/`netstat -an` → per-state counts, top LISTEN |
-| `vmstat` | `vmstat 1 N` → per-column min/mean/max + peaks |
-| `strace` | `strace`/`ltrace` → syscall counts + error summary |
-| `tcpdump` | `tcpdump` text → per-flow aggregation |
-| `metrics` | Prometheus `/metrics` → grouped, capped label combos |
-| `curl` | `curl -v` → fold TLS handshake + cert chain |
-| `cargo-bench` | criterion output → Δ-vs-baseline + regressions |
-| `jvmbuild` (`mvn`/`gradle`) | Maven/Gradle build → per-phase outcome |
-| `pkginstall` | pip/npm/cargo install → drop progress, keep installed list |
-| `ghactions` | GH Actions log → strip timestamps, keep `##[error]` |
-| `ansible` | `ansible-playbook` → per-task outcome, failures verbatim |
-| `web` | HTML page → clean markdown |
-| `serp` (`google`/`bing`/`ddg`) | raw search-results HTML → title/url/snippet |
-| `snapshot` | Playwright a11y-tree → compact tree |
-| `stealth` | page → detect Cloudflare/hCaptcha + remediation |
-| `docs` / `pdf` | `.docx`/`.xlsx`/`.pptx`/`.pdf` → plain text |
-| `notebook` | Jupyter `.ipynb` → code + outputs, strip base64 images |
-
 ### Savings, receipts & reporting
 
 **`gain`** — how many tokens you've saved.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tokenade/cli",
-  "version": "0.7.2",
+  "version": "0.7.3",
   "description": "Tokenade — cut your AI coding agent's token bill. Installs the Tokenade CLI (a local, paid token-reduction tool; activate via your browser).",
   "homepage": "https://tokenade.net",
   "license": "UNLICENSED",