smartpipe-cli 1.4.0rc2__tar.gz → 1.4.0rc4__tar.gz

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/CHANGELOG.md

@@ -5,11 +5,20 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) · Versioning:
 
 ## [Unreleased]
 
+
 ## [1.4.0] — 2026-07-08
 
 The identity release.
 
 ### Knowledge graphs (the highlight)
+- **The showcase**: a full verb page with the modality matrix and a
+  live-rendering Mermaid sample, a cookbook chapter written from real
+  runs against the downloadable playground corpus, the README highlight
+  with a screenshot of the case graph, agent SKILL recipes, and `graph`
+  on the welcome screen. Found en route: the playground download in the
+  docs pointed at the source tarball (no media inside) - all three
+  sites now fetch the actual 26 MB release asset.
+
 - **`smartpipe graph --fast` turns a corpus into a knowledge graph for
   exactly $0.** A local zero-shot NER model (GLiNER-small over ONNX -
   no new dependencies, no torch, a one-time ~190 MB download like the
@@ -44,6 +53,15 @@ The identity release.
   power path.
 
 ### The item — one law for everything in a pipe
+- **Nested field paths, everywhere you read.** `where 'user.plan ==
+  "pro"'`, `sort --by metrics.score`, `chart items[0].category`,
+  `summarize 'avg(metrics.score) by user.plan'`, and `"Is {user.plan}
+  worth it?"` in prompts - one grammar (`a.b.c`, `items[0]`,
+  `a.b['weird key']`) across all five surfaces. A column literally
+  named `user.plan` always wins over the traversal, so CSV headers
+  with dots keep working. Extraction names stay flat - paths are for
+  reading, and the error says so.
+
 - **Dates are a type, and they come back ISO.** `{due date}` and
   `{ts datetime}` join the brace grammar. However the model phrases it
   ("Jan 15, 2026", "15/01/2026"), the field canonicalizes to
@@ -75,9 +93,20 @@ The identity release.
   double-underscore namespace that rides every record: `__source` (path plus
   HOW the item was cut - file, lines, jsonl, pages, minutes, segment index),
   `__media` (one transport object for bytes crossing a pipe), `__score`
-  (join), `__invalid`/`__error`/`__raw` (kept failures). Known fields
-  round-trip through saved files; unknown `__` fields warn once and carry.
-  User data owns everything up to one leading underscore.
+  (top_k and join similarity), `__rank` (top_k), `__snapshot` (top_k
+  --stream markers), `__distance` (outliers), `__invalid`/`__error`/`__raw`
+  (kept failures). Known fields round-trip through saved files; unknown `__`
+  fields warn once and carry. User data owns everything up to one leading
+  underscore - which is why the last single-underscore stragglers moved:
+  pre-1.4 `top_k`/`outliers` wrote `_score`/`_rank`/`_snapshot`/`_distance`,
+  and those spellings stay readable for one release (CSV/TSV still sort them
+  into the trailing columns) but are never written again. In the same sweep,
+  `cluster --explode` overwriting an existing `cluster` field became a
+  disclosed overwrite (one stderr warning) instead of a silent one.
+- **top_k ranks can no longer collide.** Ranking keyed on each item's
+  source index, so two page-1 records from different PDFs fought over one
+  slot and a result silently vanished. Ranks are now run-scoped ordinals in
+  arrival order, in both the embedded and precomputed-vector paths.
 - **Records in, records out.** A plain-prompt `map` over records now returns
   a record (`{"result": …}` plus the spine) instead of bare prose - structure
   and provenance survive transformation. Text lines still leave as text.
@@ -105,6 +134,22 @@ The identity release.
   `where`/`summarize` report rows that had no fields; `--strict-rows` turns
   the notes into errors. Multi-line plain output into a pipe warns that
   line framing is ambiguous (use `--output json`).
+- **`.sem` files are strict about their rows now.** A saved pipeline is a
+  program, and a stray plain line in a record stream is a bug, not a
+  shrug: `.sem` runs default to `--strict-rows` and fail at ingestion
+  naming the exact line, before any spend. Opt out with
+  `strict-rows = false` (top-level or per-stage); ad-hoc terminal runs
+  keep the permissive census. Eleven verbs that accepted `--strict-rows`
+  without actually enforcing it now do, and a text-only `where` predicate
+  over a log no longer trips strict mode over "missing fields". A new
+  docs page explains the granularity ladder (file → split → lines/rows)
+  and when mixing happens.
+- **Synthesized records tell you what they summarize.** join pairs carry
+  `__sources` (both parents' refs); reduce windows stamp
+  `{"as": "window", "span": [1, 100], "count": 100}`, groups
+  `{"as": "group", "group": …, "count": N}`; cluster and diff summary
+  rows carry `{"as": "cluster"|"diff", "count": N}`. An audit trail for
+  every row that speaks for other rows - and `--bare` strips all of it.
 
 ### Judgment you can gate, retry, and keep
 - **`--keep-invalid`** (map/extend): after the repair retry fails, keep the
@@ -114,6 +159,13 @@ The identity release.
   `--check FILE` validates rows (exit 0/1), `--example` prints a validating
   instance, bare stdin mode is a REPL. **`--dry-run`** (map/extend) prints
   the composed first request and exits before any model resolution.
+- **`schema --check` stopped failing your own pipelines.** The check is
+  open-world now: rows from the documented extend workflow (original
+  fields kept, spine aboard) pass instead of tripping "additional
+  properties are not allowed" on every row, and `?`-marked fields may be
+  absent, not just null. Extras earn one dim hint; `--strict` restores
+  the closed world for contract enforcement. The schema sent to models
+  at extraction time is unchanged.
 - **Nullability is declared.** `?` on any type (`string?`, `number[]?`, bare
   `field?`) compiles to a null union; bare fields mean scalar-or-scalar-list
   and never admit null (D48, shipped earlier in this cycle).
@@ -162,6 +214,19 @@ The identity release.
   over alphabetical luck (cloud tags compete equally), and speaks in the
   same color voice as the other commands with a menu of detected local
   tags and provider-prefixed cloud examples.
+- **Exit 141 is now deliberate, and the -13 flake is dead.** smartpipe used
+  to arm the default SIGPIPE handler, so a downstream `| head` killed it by
+  raw signal - and, one run in a few hundred, a stray EPIPE on asyncio's
+  own self-pipe killed a healthy run mid-teardown, skipping the receipt
+  (the CI flake that delayed rc3). SIGPIPE now stays ignored; a closed
+  stdout surfaces as BrokenPipeError and exits a silent, deliberate 141
+  with every finally honored. (click 8.4's own EPIPE-to-exit-1 trap is
+  defused on the way.) One visible delta: `kill -PIPE` no longer
+  terminates smartpipe; pipeline behavior is unchanged.
+- **A files-then-stdin wait has a name.** Chaining positional files into a
+  stdin read used to end in a silent hang when nothing was piped. All three
+  chain paths now say once, on stderr:
+  `note: files done - now reading stdin (pipe data or close it; files-only: add < /dev/null)`.
 - **Docs bug, live-caught: `cluster | chart` drew flat bars.** The summary
   rows are one-per-cluster, so charting them counts every theme once. The
   README quick-tour step, the `cluster` help epilog, and the `.sem`
@@ -230,15 +295,24 @@ The identity release.
 
 ### Documents parse where they enter, vectors stay honest
 - **The `ocr-model` role: a document parser at ingestion.** Set
-  `smartpipe config ocr-model mistral/mistral-ocr-latest` (or
-  `SMARTPIPE_OCR_MODEL`, or `--ocr-model` per run) and ingested PDFs and
-  images parse through it - one item per page on the `pages` cut,
-  markdown as the text, disclosed per row and metered as a paid
-  conversion. Mistral refs ride the dedicated `/v1/ocr` wire
-  (live-verified, whole PDFs in one call); any other model ref works as
-  chat-vision with extract-the-text framing, so `ollama/llava` is a free
-  local OCR. Parse failures fall back to the local extraction ladder,
-  disclosed, never fatal. Unset = exactly the old behavior.
+  `ocr-model = "mistral/mistral-ocr-latest"` in config.toml (the setup
+  flow's OCR stage stamps it; `SMARTPIPE_OCR_MODEL` and `--ocr-model`
+  override per run) and ingested PDFs and images parse through it - one
+  item per page on the `pages` cut, markdown as the text, disclosed per
+  row and metered as a paid conversion. Mistral refs ride the dedicated
+  `/v1/ocr` wire (live-verified, whole PDFs in one call); any other
+  model ref works as chat-vision with extract-the-text framing, so
+  `ollama/llava` is a free local OCR. Parse failures fall back to the
+  local extraction ladder, disclosed, never fatal. Unset = exactly the
+  old behavior.
+- **Reader mode does what you configured.** `smartpipe scan.pdf` used to
+  be pinned "zero model calls" even when you had set an ocr-model - the
+  reader now honors the role with the same per-row disclosure, local
+  fallback, and `--max-calls` belt as the verbs (plus a preflight note
+  when more than 20 files will parse). No ocr-model configured = the
+  same free path as always, pinned by a zero-HTTP-calls test. Found en
+  route: the mistral OCR wire was never counted by `--max-calls`
+  anywhere - it is belted now.
 - **The `media-embed-model` role: one joint space for pixels and prose.**
   Text keeps `embed-model`; media routes to the joint-space model
   (jina-clip-v2 and friends). If a run would mix two vector spaces, the
@@ -256,6 +330,42 @@ The identity release.
   media-embedding capability.
 
 ### Setup that reads the room
+- **Rich now backs `config show` and `doctor`.** It is a deliberate core
+  dependency under the no-extras policy, imported only while those human-facing
+  screens render so `--help`, pipes, and `NO_COLOR` keep their existing contracts.
+- **The interactive config wizard can go backward.** Model, embedding, OCR, and
+  final review screens expose explicit Back rows; returning to a stage restores
+  its pre-stage checkpoint, and typed model prompts accept `back`, `b`, `no`, or
+  `n` instead of accidentally saving `ollama/n`.
+
+
+- **Discarding config is terminal and interruption is ordinary.** Choosing
+  discard no longer opens the shell-completion prompt after saying "Not saved";
+  Ctrl-C or EOF at any Click prompt exits 130 instead of showing the internal-bug
+  screen.
+
+- **`auth login` now connects every provider, not just ChatGPT.** Pick
+  from the full list - OpenAI appears twice because it really is two
+  wires (API key vs ChatGPT login, and the login wire has no
+  embeddings) - paste your key at a masked prompt after the create-a-key
+  URL, and it validates live before storing (with an honest
+  "store anyway - the provider may be having a bad minute" escape).
+  Keys live at `~/.local/share/smartpipe/auth.json`, owner-only
+  permissions from the first byte, masked everywhere they surface, and
+  the environment variable always wins. `auth list` shows what's
+  connected and which source is live; `auth logout` removes.
+- **Setup walks three questions in order: text model, embeddings,
+  document OCR.** Every stage shows every provider - connected ones
+  badged, unconnected ones connectable inline without leaving the flow.
+  The embedding stage only lists wires that embed; OCR is optional and
+  skippable in one keypress. At the end, one consented verification
+  pass (~5 tiny requests) probes what your models can actually do -
+  and if the basic text check fails, smartpipe reports a setup fault
+  instead of pretending your model lacks modalities. Results feed the
+  capability chips you see in every picker from then on, alongside a
+  public model registry and your own declarations for self-hosted
+  models.
+
 - **The schema workshop.** `smartpipe schema` with no arguments at a
   terminal opens a small interactive draft loop - your schema pinned up
   top as colored braces, commands scrolling below: `/add name type`,
@@ -276,7 +386,26 @@ The identity release.
   overwrites one you chose yourself), and one y/N question adds a backup
   `fallback-model` for provider outages. After a `doctor --probe`,
   catalog entries carry capability chips (`sees, hears - probed 2d ago`)
-  - probed facts, never claims. All the setter subcommands still work.
+  - probed facts, never claims.
+- **`smartpipe use` - setup is one word now.** `use gemini` (or
+  `use claude-opus-4-8`, `use ollama/qwen3:8b`) stamps a coherent bundle
+  in one shot: the model, its paired embedder, and the captions posture,
+  each line disclosed with a ✓ and a reason. No credential for the named
+  provider = a clean refusal that names the exact `auth login` command,
+  nothing stamped. Bare `use` opens the interactive flow. `smartpipe
+  using` answers "what am I running?" with every setting, its value, and
+  WHERE it came from (flag, env, config file, default) in one aligned
+  grid. The config.toml you get is signed by its door -
+  `# stamped by: smartpipe use (2026-07-10T06:38Z)` - so a mystery
+  config explains itself. The old setter subcommands (`config model`,
+  `config embed-model`, …) and profiles are gone; an old config with
+  profile keys loads with one warning and cleans itself on the next
+  save.
+- **The no-model dead end offers a hand.** At a real terminal, the
+  no-model error now ends with `run setup now? [Y/n]` - accept, walk the
+  same setup flow, and it closes with "saved - rerun your command".
+  Scripts and CI see byte-identical output to before (the offer is
+  TTY-gated), and the exit code stays honest.
 
 ### Getting it and keeping it current
 - **One-line install on every platform.**

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/CITATION.cff

@@ -7,5 +7,5 @@ authors:
     given-names: Prabal
 repository-code: "https://github.com/prabal-rje/smartpipe"
 license: Apache-2.0
-version: 1.4.0rc2
+version: 1.4.0rc4
 date-released: '2026-07-08'

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/CONTRIBUTING.md

@@ -9,7 +9,7 @@ Requires [uv](https://docs.astral.sh/uv/) and Python ≥ 3.11 (uv can fetch one
 
 ```console
 $ git clone https://github.com/prabal-rje/smartpipe && cd smartpipe
-$ uv sync --all-extras          # creates .venv with dev + optional deps
+$ uv sync --all-extras          # creates .venv with the project and dev tools
 $ uv run smartpipe                # welcome screen = working install
 ```
 
@@ -64,12 +64,13 @@ Reverting to built-in search is emptying the three values again.
 
 ## New dependencies
 
-Core install weight is a feature: the runtime dependency list is `click`, `httpx`,
-`jsonschema`, `tomli-w` — a snapshot test guards it. Anything heavier goes behind an
-optional extra, and heavy imports stay function-local (startup time is budgeted).
+Core install weight is a feature, but D46 deliberately ships the complete
+multimodal surface in core: there are no optional extras. The frozen dependency
+snapshot makes every direct or transitive addition an explicit review decision.
+Heavy imports stay function-local so startup remains budgeted.
 
 **New heavy import? Function-local or justify.** `--help` must never import
-`httpx`, `jsonschema`, `anthropic`, or `markitdown` —
+`httpx`, `jsonschema`, `anthropic`, `markitdown`, or `rich` —
 `tests/test_startup_imports.py` is the enforcement (it runs `-X importtime` and
 fails on any banned module). `make startup` gives an advisory wall-clock number;
 the import test is the deterministic gate.

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: smartpipe-cli
-Version: 1.4.0rc2
+Version: 1.4.0rc4
 Summary: Semantic pipes and queries for your terminal - PDFs, images, audio, video, and text through Unix verbs that understand.
 Project-URL: Homepage, https://github.com/prabal-rje/smartpipe
 Project-URL: Changelog, https://github.com/prabal-rje/smartpipe/blob/main/CHANGELOG.md
@@ -32,6 +32,7 @@ Requires-Dist: matplotlib>=3.8
 Requires-Dist: pdfminer-six>=20231228
 Requires-Dist: plotext>=5
 Requires-Dist: pypdf>=5.0
+Requires-Dist: rich>=13.9
 Requires-Dist: tomli-w>=1.0
 Description-Content-Type: text/markdown
 
@@ -48,9 +49,9 @@ Description-Content-Type: text/markdown
 Run PDFs, images, audio, video, and text through Unix verbs that understand
 their input.
 
-[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v3.gif)](https://prabal-rje.github.io/smartpipe/demo/)
+[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v5.gif)](https://prabal-rje.github.io/smartpipe/demo/)
 
-*[▶ Watch the full 70-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
+*[▶ Watch the full 86-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
 
 ## Install
 
@@ -92,7 +93,7 @@ and runs that tool's own upgrade.
 smartpipe auth login
 ```
 
-No ChatGPT plan? Use local [Ollama][ollama] or a cloud API key - see
+No ChatGPT plan? Bring a cloud API key, or run local [Ollama][ollama] - see
 [Models & providers][models].
 
 ## Examples
@@ -139,6 +140,7 @@ into ordinary Unix tools. Every verb is documented at
 | [`distinct`][distinct] | fold near-duplicates; `--exact` is free | `sort -u`, by meaning |
 | [`diff`][diff] | what distinguishes two sets | the post-incident answer |
 | [`outliers`][outliers] | the items least like the rest | novelty, surfaced |
+| [`graph`][graph] | corpus → entity/relationship graph; `--fast` is free | the case wall, with citations |
 
 **Free verbs** never call a model. Run them first to cut the corpus before any
 paid stage:
@@ -154,15 +156,16 @@ paid stage:
 | [`chart`][cli] | terminal bars, SVG, facets, time series | quick plots |
 
 Some semantic verbs have a **conditionally free mode**: `join --on` (key
-equality, no prompt), `distinct --exact` (hash-only folding), `map`/`extend`
-`--dry-run` (compose without sending), and `smartpipe schema` with a
-braces/DSL expression. Each stays at zero model calls by construction.
+equality, no prompt), `distinct --exact` (hash-only folding), `graph --fast`
+(local NER, on-device), `map`/`extend` `--dry-run` (compose without sending),
+and `smartpipe schema` with a braces/DSL expression. Each stays at zero model
+calls by construction.
 
 ## A one-minute tour
 
 ```bash
 # 1. point smartpipe at a model (ChatGPT login, a cloud key, or local Ollama)
-smartpipe config
+smartpipe use
 
 # 2. ask one question across a folder of mixed documents
 smartpipe map "What does this say about pricing?" 'docs/*.pdf'
@@ -192,6 +195,9 @@ smartpipe map "Extract {vendor string, invoice_number string, total number}" 'in
 # 8. video RAG, no vector database: index a folder of recordings once, ask any day
 smartpipe embed 'sessions/**/*.mp4' > sessions.embeddings
 smartpipe top_k 3 --near "user gives up after the coupon fails" < sessions.embeddings
+
+# 9. the corpus as a knowledge graph - $0: local NER, zero model calls
+smartpipe graph --fast 'case-files/*.md' --save case.html
 ```
 
 A `.txt` on a pipe arrives one item per line; `--as file` treats the whole
@@ -206,6 +212,18 @@ Numbers 7 and 8 are full recipes -
 New to this? The [Learn track][quickstart] starts at zero and assumes nothing, including
 what a "model" is.
 
+### Highlight: the knowledge graph
+
+Number 9 is new: `graph --fast` turns a folder of mixed files into an
+interactive, **cited** knowledge graph without a single model call - a local
+NER model finds the entities you name, co-occurrence weights the edges, and
+hovering any edge shows the files behind it. A focus prompt upgrades the
+strongest edges to model-read relations. Full story:
+[the `graph` page](https://prabal-rje.github.io/smartpipe/verbs/graph/) and
+[the cookbook recipe](docs/cookbook/knowledge-graph.md).
+
+![smartpipe graph --save case.html: an interactive knowledge graph with per-edge source citations](docs/assets/graph-hero.png)
+
 ## Try it on real files
 
 No corpus handy? [smartpipe-playground][playground] ships 26 MB of CC0 /
@@ -213,8 +231,8 @@ public-domain practice files - invoices, reports, photos, recordings, screen
 sessions, and JSONL data:
 
 ```bash
-curl -L https://github.com/prabal-rje/smartpipe-playground/archive/refs/tags/v1.tar.gz | tar xz
-cd smartpipe-playground-1
+curl -L https://github.com/prabal-rje/smartpipe-playground/releases/download/v1/smartpipe-playground-v1.tar.gz | tar xz
+cd smartpipe-playground
 
 smartpipe map "Extract {vendor, invoice_number, total number}" 'invoices/*.pdf'
 smartpipe filter "the customer sounds frustrated" 'recordings/*.mp3'
@@ -236,9 +254,11 @@ For chat, [Ollama][ollama] runs models on your machine. Any cloud model sends
 that run's data to its provider - `gpt-5.4-mini`, `claude-opus-4-8`,
 `gemini-3.1-flash-lite`, `mistral-large-latest`, `openrouter/…`.
 
-API keys come from environment variables and are never stored. ChatGPT
-subscribers can run `smartpipe auth login` instead. `smartpipe usage` keeps
-local run and token totals; see [Privacy & security][privacy] for the details.
+API keys come from environment variables, or `smartpipe auth login` stores
+one securely for you. ChatGPT subscribers can log in instead of using a key.
+`smartpipe usage` keeps local run and token totals; see
+[Privacy & security][privacy] for where credentials live and how to remove
+them.
 
 ## Unix behavior
 
@@ -257,7 +277,7 @@ Full docs: **[prabal-rje.github.io/smartpipe][docs]**.
 - [Install][install] - packages and platforms
 - [Working with files & media][files] - PDFs, scans, images, audio, video
 - [CLI reference][cli] - every flag, format, and exit code
-- [Models & providers][models] - local Ollama, cloud keys, ChatGPT login
+- [Models & providers][models] - ChatGPT login, cloud keys, local Ollama
 - [Privacy & security][privacy]
 
 ## How to cite
@@ -303,6 +323,7 @@ lives in [`qa/`](qa/README.md). The CLI surface is a SemVer contract.
 [distinct]: https://prabal-rje.github.io/smartpipe/verbs/distinct/
 [diff]: https://prabal-rje.github.io/smartpipe/verbs/diff/
 [outliers]: https://prabal-rje.github.io/smartpipe/verbs/outliers/
+[graph]: https://prabal-rje.github.io/smartpipe/verbs/graph/
 [where]: https://prabal-rje.github.io/smartpipe/verbs/where/
 [summarize]: https://prabal-rje.github.io/smartpipe/verbs/summarize/
 [sort]: https://prabal-rje.github.io/smartpipe/verbs/sort/

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/README.md

@@ -11,9 +11,9 @@
 Run PDFs, images, audio, video, and text through Unix verbs that understand
 their input.
 
-[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v3.gif)](https://prabal-rje.github.io/smartpipe/demo/)
+[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v5.gif)](https://prabal-rje.github.io/smartpipe/demo/)
 
-*[▶ Watch the full 70-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
+*[▶ Watch the full 86-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
 
 ## Install
 
@@ -55,7 +55,7 @@ and runs that tool's own upgrade.
 smartpipe auth login
 ```
 
-No ChatGPT plan? Use local [Ollama][ollama] or a cloud API key - see
+No ChatGPT plan? Bring a cloud API key, or run local [Ollama][ollama] - see
 [Models & providers][models].
 
 ## Examples
@@ -102,6 +102,7 @@ into ordinary Unix tools. Every verb is documented at
 | [`distinct`][distinct] | fold near-duplicates; `--exact` is free | `sort -u`, by meaning |
 | [`diff`][diff] | what distinguishes two sets | the post-incident answer |
 | [`outliers`][outliers] | the items least like the rest | novelty, surfaced |
+| [`graph`][graph] | corpus → entity/relationship graph; `--fast` is free | the case wall, with citations |
 
 **Free verbs** never call a model. Run them first to cut the corpus before any
 paid stage:
@@ -117,15 +118,16 @@ paid stage:
 | [`chart`][cli] | terminal bars, SVG, facets, time series | quick plots |
 
 Some semantic verbs have a **conditionally free mode**: `join --on` (key
-equality, no prompt), `distinct --exact` (hash-only folding), `map`/`extend`
-`--dry-run` (compose without sending), and `smartpipe schema` with a
-braces/DSL expression. Each stays at zero model calls by construction.
+equality, no prompt), `distinct --exact` (hash-only folding), `graph --fast`
+(local NER, on-device), `map`/`extend` `--dry-run` (compose without sending),
+and `smartpipe schema` with a braces/DSL expression. Each stays at zero model
+calls by construction.
 
 ## A one-minute tour
 
 ```bash
 # 1. point smartpipe at a model (ChatGPT login, a cloud key, or local Ollama)
-smartpipe config
+smartpipe use
 
 # 2. ask one question across a folder of mixed documents
 smartpipe map "What does this say about pricing?" 'docs/*.pdf'
@@ -155,6 +157,9 @@ smartpipe map "Extract {vendor string, invoice_number string, total number}" 'in
 # 8. video RAG, no vector database: index a folder of recordings once, ask any day
 smartpipe embed 'sessions/**/*.mp4' > sessions.embeddings
 smartpipe top_k 3 --near "user gives up after the coupon fails" < sessions.embeddings
+
+# 9. the corpus as a knowledge graph - $0: local NER, zero model calls
+smartpipe graph --fast 'case-files/*.md' --save case.html
 ```
 
 A `.txt` on a pipe arrives one item per line; `--as file` treats the whole
@@ -169,6 +174,18 @@ Numbers 7 and 8 are full recipes -
 New to this? The [Learn track][quickstart] starts at zero and assumes nothing, including
 what a "model" is.
 
+### Highlight: the knowledge graph
+
+Number 9 is new: `graph --fast` turns a folder of mixed files into an
+interactive, **cited** knowledge graph without a single model call - a local
+NER model finds the entities you name, co-occurrence weights the edges, and
+hovering any edge shows the files behind it. A focus prompt upgrades the
+strongest edges to model-read relations. Full story:
+[the `graph` page](https://prabal-rje.github.io/smartpipe/verbs/graph/) and
+[the cookbook recipe](docs/cookbook/knowledge-graph.md).
+
+![smartpipe graph --save case.html: an interactive knowledge graph with per-edge source citations](docs/assets/graph-hero.png)
+
 ## Try it on real files
 
 No corpus handy? [smartpipe-playground][playground] ships 26 MB of CC0 /
@@ -176,8 +193,8 @@ public-domain practice files - invoices, reports, photos, recordings, screen
 sessions, and JSONL data:
 
 ```bash
-curl -L https://github.com/prabal-rje/smartpipe-playground/archive/refs/tags/v1.tar.gz | tar xz
-cd smartpipe-playground-1
+curl -L https://github.com/prabal-rje/smartpipe-playground/releases/download/v1/smartpipe-playground-v1.tar.gz | tar xz
+cd smartpipe-playground
 
 smartpipe map "Extract {vendor, invoice_number, total number}" 'invoices/*.pdf'
 smartpipe filter "the customer sounds frustrated" 'recordings/*.mp3'
@@ -199,9 +216,11 @@ For chat, [Ollama][ollama] runs models on your machine. Any cloud model sends
 that run's data to its provider - `gpt-5.4-mini`, `claude-opus-4-8`,
 `gemini-3.1-flash-lite`, `mistral-large-latest`, `openrouter/…`.
 
-API keys come from environment variables and are never stored. ChatGPT
-subscribers can run `smartpipe auth login` instead. `smartpipe usage` keeps
-local run and token totals; see [Privacy & security][privacy] for the details.
+API keys come from environment variables, or `smartpipe auth login` stores
+one securely for you. ChatGPT subscribers can log in instead of using a key.
+`smartpipe usage` keeps local run and token totals; see
+[Privacy & security][privacy] for where credentials live and how to remove
+them.
 
 ## Unix behavior
 
@@ -220,7 +239,7 @@ Full docs: **[prabal-rje.github.io/smartpipe][docs]**.
 - [Install][install] - packages and platforms
 - [Working with files & media][files] - PDFs, scans, images, audio, video
 - [CLI reference][cli] - every flag, format, and exit code
-- [Models & providers][models] - local Ollama, cloud keys, ChatGPT login
+- [Models & providers][models] - ChatGPT login, cloud keys, local Ollama
 - [Privacy & security][privacy]
 
 ## How to cite
@@ -266,6 +285,7 @@ lives in [`qa/`](qa/README.md). The CLI surface is a SemVer contract.
 [distinct]: https://prabal-rje.github.io/smartpipe/verbs/distinct/
 [diff]: https://prabal-rje.github.io/smartpipe/verbs/diff/
 [outliers]: https://prabal-rje.github.io/smartpipe/verbs/outliers/
+[graph]: https://prabal-rje.github.io/smartpipe/verbs/graph/
 [where]: https://prabal-rje.github.io/smartpipe/verbs/where/
 [summarize]: https://prabal-rje.github.io/smartpipe/verbs/summarize/
 [sort]: https://prabal-rje.github.io/smartpipe/verbs/sort/

smartpipe_cli-1.4.0rc4/demo/video/README.md

@@ -0,0 +1,23 @@
+# smartpipe demo video
+
+A self-contained [Remotion](https://remotion.dev) project that renders the
+86-second smartpipe demo (1920x1080, 30 fps): eight scenes - cold open, the
+hook, multimodal, cost honesty, index, search, graph, close - all driven by
+one reusable `<Terminal>` component and the frame-math constants in
+`src/config.ts` (tune scene lengths there). Run `npm install` once, then
+`npm run studio` to preview in the browser or `npm run render` to produce
+`out/smartpipe-demo.mp4`; each scene is also registered as its own
+composition (`ColdOpen`, `Hook`, `Multimodal`, `CostHonesty`, `ScaleA`,
+`ScaleB`, `Graph`, `Close`), so `npx remotion render Hook out/hook.mp4`
+re-renders one scene independently. The narrated cut is the `MainNarrated`
+composition: the script text lives in `src/narration/lines.json`, and
+`RIME_API_KEY=... node scripts/fetch-narration.mjs` synthesizes the
+voiceover wavs (Rime.AI, speaker cupola) into the gitignored
+`public/narration/` and refreshes `src/narration/durations.json` - fetch
+before rendering `MainNarrated`. The music bed is
+`public/silicon-prism-waltz.m4a` (60 s, looped over the 86 s cut with a
+fade-out, held far below the voice); if you ever need a
+no-strings-attached replacement, `node scripts/make-music.mjs` (ffmpeg
+required) synthesizes an original ambient track from scratch into
+`public/music.m4a` - royalty-free by construction - and you can point the
+`<Audio>` element in `src/Main.tsx` at it.

{smartpipe_cli-1.4.0rc2 → smartpipe_cli-1.4.0rc4}/demo/video/src/Main.tsx

@@ -5,22 +5,34 @@ import { Background } from "./components/Background";
 import { Close } from "./scenes/Close";
 import { ColdOpen } from "./scenes/ColdOpen";
 import { CostHonesty } from "./scenes/CostHonesty";
+import { Graph } from "./scenes/Graph";
 import { Hook } from "./scenes/Hook";
 import { Multimodal } from "./scenes/Multimodal";
-import { Scale } from "./scenes/Scale";
+import { ScaleA } from "./scenes/ScaleA";
+import { ScaleB } from "./scenes/ScaleB";
+import { duckFactor, Narration } from "./Narration";
 
-/** The full ~70s cut: one persistent background, six scenes in series. */
-export const Main: React.FC = () => {
+/** Music bed base gain. Owner order (2026-07-09): "lower BG volume much
+ *  more" — 0.55 → 0.14, about −12 dB. Texture, never presence. */
+const MUSIC_BASE = 0.14;
+
+export type MainProps = { narrated?: boolean };
+
+/** The full 86s cut: one persistent background, eight scenes in series.
+ *  With `narrated`, the Rime voiceover plays on top, a subtitle band shows
+ *  the spoken words, and the music ducks a further few dB under speech. */
+export const Main: React.FC<MainProps> = ({ narrated = false }) => {
   return (
     <AbsoluteFill>
-      {/* Music bed: Silicon Prism Waltz (60s, looped to cover the 70s cut).
+      {/* Music bed: Silicon Prism Waltz (60s, looped to cover the 86s cut).
           Volume envelope: quick fade-in, steady bed, fade-out over the close.
           A fully synthesized fallback lives at scripts/make-music.mjs. */}
       <Audio
         loop
         src={staticFile("silicon-prism-waltz.m4a")}
         volume={(f) =>
-          0.55 *
+          MUSIC_BASE *
+          (narrated ? duckFactor(f) : 1) *
           interpolate(
             f,
             [0, 36, TOTAL_FRAMES - 96, TOTAL_FRAMES - 8],
@@ -43,13 +55,20 @@ export const Main: React.FC = () => {
         <Series.Sequence durationInFrames={SCENE.cost}>
           <CostHonesty />
         </Series.Sequence>
-        <Series.Sequence durationInFrames={SCENE.scale}>
-          <Scale />
+        <Series.Sequence durationInFrames={SCENE.scaleA}>
+          <ScaleA />
+        </Series.Sequence>
+        <Series.Sequence durationInFrames={SCENE.scaleB}>
+          <ScaleB />
+        </Series.Sequence>
+        <Series.Sequence durationInFrames={SCENE.graph}>
+          <Graph />
         </Series.Sequence>
         <Series.Sequence durationInFrames={SCENE.close}>
           <Close />
         </Series.Sequence>
       </Series>
+      {narrated ? <Narration /> : null}
     </AbsoluteFill>
   );
 };